Can Transformers Do Enumerative Geometry?

How can Transformers model and learn enumerative geometry? What is a systematic procedure for using Transformers in abductive knowledge discovery within a mathematician-machine collaboration? In this work, we introduce a Neural Enumerative Reasoning model for computation of ψ-class intersection numbers on the moduli space of curves. By reformulating the problem as a continuous optimization task, we compute intersection numbers across a wide value range from 10e-45 to 10e45. To capture the recursive nature inherent in these intersection numbers, we propose the Dynamic Range Activator (DRA), a new activation function that enhances the Transformer’s ability to model recursive patterns and handle severe heteroscedasticity. Given precision requirements for computing the intersections, we quantify the uncertainty of the predictions using Conformal Prediction with a dynamic sliding window adaptive to the partitions of equivalent number of marked points. Beyond simply computing intersection numbers, we explore the enumerative “world-model” of Transformers. Our interpretability analysis reveals that the network is implicitly modeling the Virasoro constraints in a purely data-driven manner. Moreover, through abductive hypothesis testing, probing, and causal inference, we uncover evidence of an emergent internal representation of the the large-genus asymptotic of ψ-class intersection numbers. This opens up new possibilities in inferring asymptotic closed-form expressions directly from limited amount of data.
This talk is based on https://openreview.net/pdf?id=4X9RpKH4Ls.

In person or on Zoom:

https://harvard.zoom.us/j/92220006185?pwd=V3mrb4cNSbgRXtNJtRJkTvWFVhmbI5.1
Password: cmsa