Share this page:

Semantic Probabilistic Layers for Neuro-Symbolic Learning

Kareem Ahmed, Stefano Teso, Kai-Wei Chang, Guy Van den Broeck, and Antonio Vergari, in NeurIPS, 2022.

Download the full text

Abstract

We design a predictive layer for structured-output prediction (SOP) that can be plugged into any neural network guaranteeing its predictions are consistent with a set of predefined symbolic constraints. Our Semantic Probabilistic Layer (SPL) can model intricate correlations, and hard constraints, over a structured output space all while being amenable to end-to-end learning via maximum likelihood. SPLs combine exact probabilistic inference with logical reasoning in a clean and modular way, learning complex distributions and restricting their support to solutions of the constraint. As such, they can faithfully, and efficiently, model complex SOP tasks beyond the reach of alternative neuro-symbolic approaches. We empirically demonstrate that SPLs outperform these competitors in terms of accuracy on challenging SOP tasks including hierarchical multi-label classification, pathfinding and preference learning, while retaining perfect constraint satisfaction.

Bib Entry

@inproceedings{ahmed2022semantic,
  title = {Semantic Probabilistic Layers for Neuro-Symbolic Learning},
  author = {Ahmed, Kareem and Teso, Stefano and Chang, Kai-Wei and den Broeck, Guy Van and Vergari, Antonio},
  booktitle = {NeurIPS},
  year = {2022}
}

Related Publications

  1. On the Paradox of Learning to Reason from Data, IJCAI, 2023
  2. Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models, NeurIPS, 2023
  3. AVIS: Autonomous Visual Information Seeking with Large Language Models, NeurIPS, 2023
  4. A Survey of Deep Learning for Mathematical Reasoning, ACL, 2023
  5. Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step, ACL, 2023
  6. Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning, ICLR, 2023
  7. Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering, NeurIPS, 2022
  8. Neuro-Symbolic Entropy Regularization, UAI, 2022