### Scaling Exact Inference for Discrete Probabilistic Programs

*Proceedings of the ACM on Programming Languages* (OOPSLA 2020), 4, OOPSLA, Article 140 (November 2020).

**Distinguished Paper Award**

Steven Holtzen, Guy Van den Broeck, Todd Millstein

Probabilistic programming languages (PPLs) are an expressive
means of representing and reasoning about probabilistic models. The computational
challenge of *probabilistic inference* remains the primary roadblock for
applying PPLs in practice. Inference is fundamentally hard, so there is no
one-size-fits all solution. In this work, we target scalable inference for an
important class of probabilistic programs: those whose probability distributions are
*discrete*. Discrete distributions are common in many fields, including text
analysis, network verification, artificial intelligence, and graph analysis, but they
prove to be challenging for existing PPLs.
We develop a domain-specific probabilistic programming language called Dice that
features a new approach to exact discrete probabilistic program inference. Dice exploits
program structure in order to *factorize* inference, enabling us to perform exact
inference on probabilistic programs with hundreds of thousands of random variables. Our
key technical contribution is a new reduction from discrete probabilistic programs to
*weighted model counting* (WMC). This reduction separates the structure of the
distribution from its parameters, enabling logical reasoning tools to exploit that
structure for probabilistic inference. We (1) show how to compositionally reduce Dice
inference to WMC, (2) prove this compilation correct with respect to a denotational
semantics, (3) empirically demonstrate the performance benefits over prior approaches,
and (4) analyze the types of structure that allow Dice to scale to large probabilistic
programs.

[PDF | Implementation | Project Page]