Foundations and Trends® in Programming Languages > Vol 7 > Issue 3

Neurosymbolic Programming

By Swarat Chaudhuri, The University of Texas at Austin, USA, swarat@cs.utexas.edu | Kevin Ellis, Cornell University, USA, kellis@cornell.edu | Oleksandr Polozov, Google, USA, polozov@google.com | Rishabh Singh, Google, USA, rising@google.com | Armando Solar-Lezama, Massachusetts Institute of Technology, USA, asolar@csail.mit.edu | Yisong Yue, The California Institute of Technology, USA, yyue@caltech.edu

 
Suggested Citation
Swarat Chaudhuri, Kevin Ellis, Oleksandr Polozov, Rishabh Singh, Armando Solar-Lezama and Yisong Yue (2021), "Neurosymbolic Programming", Foundations and TrendsĀ® in Programming Languages: Vol. 7: No. 3, pp 158-243. http://dx.doi.org/10.1561/2500000049

Publication Date: 09 Dec 2021
© 2021 S. Chaudhuri et al.
 
Subjects
Program synthesis,  Deep learning
 

Free Preview:

Download extract

Share

Download article
In this article:
1. Introduction
2. The Landscape of Neurosymbolic Programming
3. Motivating Goals
4. Learning Algorithms
5. Conclusion
References

Abstract

We survey recent work on neurosymbolic programming, an emerging area that bridges the areas of deep learning and program synthesis. Like in classic machine learning, the goal here is to learn functions from data. However, these functions are represented as programs that can use neural modules in addition to symbolic primitives and are induced using a combination of symbolic search and gradient-based optimization.

Neurosymbolic programming can offer multiple advantages over end-to-end deep learning. Programs can sometimes naturally represent long-horizon, procedural tasks that are difficult to perform using deep networks. Neurosymbolic representations are also, commonly, easier to interpret and formally verify than neural networks. The restrictions of a programming language can serve as a form of regularization and lead to more generalizable and data-efficient learning. Compositional programming abstractions can also be a natural way of reusing learned modules across learning tasks.

In this monograph, we illustrate these potential benefits with concrete examples from recent work on neurosymbolic programming. We also categorize the main ways in which symbolic and neural learning techniques come together in this area.We conclude with a discussion of the open technical challenges in the field.

DOI:10.1561/2500000049
ISBN: 978-1-68083-934-0
98 pp. $70.00
Buy book (pb)
 
ISBN: 978-1-68083-935-7
98 pp. $140.00
Buy E-book (.pdf)
Table of contents:
1. Introduction
2. The Landscape of Neurosymbolic Programming
3. Motivating Goals
4. Learning Algorithms
5. Conclusion
References

Neurosymbolic Programming

Neurosymbolic programming is an emerging area that bridges the areas of deep learning and program synthesis. As in classical machine learning, the goal is to learn functions from data. However, these functions are represented as programs that can use neural modules in addition to symbolic primitives and are induced using a combination of symbolic search and gradient-based optimization.

Neurosymbolic programming can offer multiple advantages over end-to-end deep learning. Programs can sometimes naturally represent long-horizon, procedural tasks that are difficult to perform using deep networks. Neurosymbolic representations are also, commonly, easier to interpret and formally verify than neural networks. The restrictions of a programming language can serve as a form of regularization and lead to more generalizable and data-efficient learning. Compositional programming abstractions can also be a natural way of reusing learned modules across learning tasks.

In this monograph, the authors illustrate these potential benefits with concrete examples from recent work on neurosymbolic programming. They also categorize the main ways in which symbolic and neural learning techniques come together in this area and conclude with a discussion of the open technical challenges in the field.

The comprehensive review of neurosymbolic programming introduces the reader to the topic and provides an insightful treatise on an increasingly important topic at the intersection of programming languages and machine learning.

 
PGL-049