Top Mathematicians Grade AI’s Ability to Solve Toughest Problems

Less than a year ago, a Harvard math professor joined a handful of world-leading scholars on an ambitious experiment designed to better define the capabilities of AI systems and delineate the boundaries with human expertise.

They repeatedly challenged AI systems to crack 10 tough research math problems, which scholars had recently solved but not yet published. In the consortium’s latest round of test results, unveiled last week, AI systems collectively solved all 10 problems. But the models still lagged in the ability to clearly communicate and document their results.

“In the last months, AI has changed the ecosystem for research mathematics,” said Lauren K. Williams, Dwight Parker Robinson Professor of Mathematics and co-organizer of the consortium behind the testing. “Essentially all the media around AI and mathematics focuses on success stories. That makes it hard to get a more nuanced, objective view of the current state of public models. Can they just do everything? If not, where do they fall short?”

Read the full story in The FAS Current.