For years, the development of artificial intelligence has been fueled by a paradox: while models are becoming increasingly adept at passing standardized tests, they often lack the depth of reasoning required to tackle truly creative, novel problems. A new initiative from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) aims to bridge this gap with the launch of MathNet. By curating over 30,000 high-quality, proof-based math problems from International Mathematical Olympiad (IMO) archives across 47 countries, researchers have created an expansive new benchmark for evaluating AI intelligence.

Unlike existing datasets that rely heavily on informal, community-sourced forums, MathNet draws its strength from official national competition booklets. These documents contain expert-authored solutions that offer multiple, detailed approaches to single problems. This depth provides models with a far richer signal for learning complex reasoning. Perhaps most crucially, the dataset integrates non-English and image-based problems, challenging the industry-wide trend of training models primarily on English-language data. This geographic and linguistic diversity is essential for ensuring that models develop a universal, rather than culture-specific, understanding of mathematical concepts.

The initial benchmarks from this dataset offer a sobering reality check for the current state of generative AI. Even top-tier models like GPT-5 failed to achieve perfect scores, struggling with roughly one-third of the problems provided. The results were particularly revealing regarding visual reasoning: models that perform well on text often falter significantly when faced with problems that include diagrams or figures. Furthermore, the reliance on English-centric training data became glaringly apparent, with many open-source models failing entirely to solve problems written in languages such as Mongolian.

Beyond simple accuracy, the project introduces a sophisticated retrieval benchmark. This task asks models to identify when two problems share the same underlying mathematical structure, even if they use different notations or languages. This capability is critical for moving beyond simple pattern matching toward genuine understanding. As the researchers emphasize, exposing AI to a global tapestry of mathematical traditions—from Romanian combinatorics to Brazilian number theory—is a necessary step toward building more robust and adaptable reasoning systems. By making this resource open to the public, the team hopes to provide a standard that encourages the development of models that can think as globally as they compute.

For years, the development of artificial intelligence has been fueled by a paradox: while models are becoming increasingly adept at passing standardized tests, they often lack the depth of reasoning required to tackle truly creative, novel problems. A new initiative from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) aims to bridge this gap with the launch of MathNet. By curating over 30,000 high-quality, proof-based math problems from International Mathematical Olympiad (IMO) archives across 47 countries, researchers have created an expansive new benchmark for evaluating AI intelligence.

Unlike existing datasets that rely heavily on informal, community-sourced forums, MathNet draws its strength from official national competition booklets. These documents contain expert-authored solutions that offer multiple, detailed approaches to single problems. This depth provides models with a far richer signal for learning complex reasoning. Perhaps most crucially, the dataset integrates non-English and image-based problems, challenging the industry-wide trend of training models primarily on English-language data. This geographic and linguistic diversity is essential for ensuring that models develop a universal, rather than culture-specific, understanding of mathematical concepts.

The initial benchmarks from this dataset offer a sobering reality check for the current state of generative AI. Even top-tier models like GPT-5 failed to achieve perfect scores, struggling with roughly one-third of the problems provided. The results were particularly revealing regarding visual reasoning: models that perform well on text often falter significantly when faced with problems that include diagrams or figures. Furthermore, the reliance on English-centric training data became glaringly apparent, with many open-source models failing entirely to solve problems written in languages such as Mongolian.

Beyond simple accuracy, the project introduces a sophisticated retrieval benchmark. This task asks models to identify when two problems share the same underlying mathematical structure, even if they use different notations or languages. This capability is critical for moving beyond simple pattern matching toward genuine understanding. As the researchers emphasize, exposing AI to a global tapestry of mathematical traditions—from Romanian combinatorics to Brazilian number theory—is a necessary step toward building more robust and adaptable reasoning systems. By making this resource open to the public, the team hopes to provide a standard that encourages the development of models that can think as globally as they compute.