A research group at the Massachusetts Institute of Technology has set out to bring probabilistic programming to practitioners, presenting their programming system Gen to the AI community.
A number of the tasks in statistics or machine learning can be mapped to probabilistic models, which are comparatively easy to analyse and solve. Probabilistic programming is a technique developed to give those without detailed knowledge of the implementation of said algorithms a shot at writing probabilistic models. Probabilistic programming languages or systems are meant to help with that – roughly by taking care of the analysis part.
In the description of Marco Cusumano-Towner et. al’s approach, the authors claim that – though useful – probabilistic programming is at this stage restricted to some classes of models. “Existing systems lack the flexibility and efficiency needed for practical use with more challenging models arising in fields like computer vision and robotics.”
Their answer to those challenges is Gen, a general-purpose probabilistic programming system which aims to combine automation, flexibility and speed. It is meant to offer users from a variety of fields a way of writing high-performing models and algorithms without “having to deal with equations or manually write high-performance code”.
To help with the modeling part, the system contains a number of embedded modeling languages, while high-level inference programs can be written in the host language.
For a first implementation of Gen, its designers used Julia as a host language – another project started at MIT about ten years ago. It was introduced to the general public in a blog post in 2012, with its creators citing greed (mainly for an easier, faster, open source, interactive, compiled multi-purpose language) as a driving force behind their creation.
The embedded modeling languages mentioned for Gen are realised as Julia embeddings. They comprise of a Turing-complete Dynamic Modeling Language, a Static Modeling Language to support static analysis, and a TensorFlow Modeling Language – all to create the generative functions that define generative models. Other novel constructs included to make devs’ lives easier are a “generative function interface for encapsulating probabilistic models, combinators that exploit common patterns of conditional independence, and an inference library that empowers users to implement efficient inference algorithms at a high level of abstraction”.
According to a paper describing the system, its creators were able to get good results on problems in the object tracking, and pose estimation realm, as well as with “inferring the structure of a time series”. An MIT press release also mentions the system already finding its way into other projects, such as collaborations with Intel, IBM Watson and the U.S. Department of Defense’s DARPA project for Machine Common Sense.
More details can be found in the proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation.