CMSA New Technologies in Mathematics Seminar: Large Language Models as Noisy Enumerators of Commonsense Knowledge


View Calendar
April 5, 2023 2:00 pm - 3:00 pm

Sean Welleck - University of Washington and Allen Institute for AI

Large language models capture massive amounts of commonsense knowledge. However, this knowledge can be noisy and yield logically inconsistent inferences. How can we effectively use these "noisy enumerators"? First, we discuss maieutic inference, which resolves noisy LLM reasoning chains into a consistent prediction using a maximum satisfiability solver. Next, we explore the shift from traditional human-written knowledge graphs to knowledge distilled from LLMs. We introduce Rainier, a system that refines LLM-generated knowledge through reinforcement learning, resulting in improved question answering capabilities. These pave the way for future learning and inference-time techniques that ensure reliable reasoning.

This seminar will be held in person and on Zoom: