This article examines the evolution of scientific discovery through the lens of history, the impact of artificial intelligence on research, and the future of mathematical inquiry, based on a conversation with mathematician Terence Tao.
Kepler and the "High-Temperature" LLM
The story of Johannes Kepler’s discovery of the laws of planetary motion serves as a compelling parallel for modern AI. Kepler spent years testing various aesthetic and geometric theories—such as inscribing Platonic solids between planetary spheres—to explain the universe. While his initial theories were incorrect, he had access to a high-quality dataset: the observations of Tycho Brahe.
Kepler’s process involved iterating through random, often astrologically influenced ideas until he found empirical regularities that fit Brahe’s precise data. This mirrors how Large Language Models (LLMs) operate today. When equipped with a verifiable data source, these models can "try" millions of hypotheses. Even if many are nonsensical or "slop," the model can eventually identify empirical regularities that lead to genuine scientific progress.
The Shift in Scientific Paradigms
Traditionally, science followed a linear path: identify a problem, form a hypothesis, and collect data to test it. Today, the paradigm has shifted toward "big data first." Researchers now analyze massive datasets to deduce patterns, effectively reversing the classical scientific method.
While hypothesis generation was once the prestige aspect of science, AI has driven the cost of this process toward zero. The new bottleneck is no longer generating ideas, but rather the verification, validation, and curation of those ideas to distinguish between true progress and digital noise.
The Bottleneck of Verification and Persuasion
As AI enables the generation of thousands of scientific theories daily, the existing peer-review system faces an existential challenge. Historically, we relied on social filters—like academic gatekeeping and long-term scientific consensus—to isolate signal from noise. Scaling these filters to keep pace with AI output is a significant, unsolved problem.
Furthermore, math and science remain inherently social endeavors. As demonstrated by the historical gap between Newton’s Principia and Darwin’s Origin of Species, having the correct answer is insufficient if it cannot be communicated effectively. Success often depends on:
- Persuasion: The ability to craft a narrative that convinces the community to invest time in a new theory.
- Narrative Utility: The power to synthesize disparate facts into a compelling vision, even when some mechanistic details are missing.
- Ablation: The ability of future scientists to "ablate" or refactor complex AI-generated proofs into understandable, elegant, and standard human mathematical concepts.
The Complementarity of AI and Math
Current AI tools excel at breadth—exploring many options and standard techniques rapidly—while human experts excel at depth.
The Plateau of "One-Shot" Solutions
Recent excitement regarding AI solving dozens of Erdős problems has plateaued. AI models are highly effective at solving problems with little existing literature by applying known techniques in novel combinations. However, they struggle to make "partial progress" or identify intermediate stages that require persistent, incremental human-like effort.
Human-AI Hybrids
Mathematics is unlikely to be fully replaced by AI in the near term. Instead, it is moving toward a hybrid model:
- Auxiliary Tasks: AI currently handles boilerplate code, formatting, and literature searches, allowing mathematicians to produce richer, broader papers.
- Experimental Math: AI is revolutionizing the "experimental" side of math, allowing for large-scale data gathering about what techniques work, a practice historically absent from the field.
- Refactoring: In the future, entire professions may emerge focused on taking AI-generated proofs and refactoring them to make the logic transparent and aesthetically sound.
The Future of Scientific Discovery
The fear that AI might solve the Riemann Hypothesis with "incomprehensible gobbledygook" is tempered by the nature of formal proof assistants like Lean. By formalizing proofs, researchers can study components in isolation, effectively decomposing gnarly problems into manageable chunks.
If we are to navigate this era of rapid change, we may need to develop new "semi-formal" languages for mathematical strategy—frameworks that allow us to communicate the plausibility and narrative of an idea before a full proof exists.
Ultimately, the goal is not to automate away the scientist, but to leverage AI to handle the "broad" exploration of the mountain range of knowledge, identifying islands of difficulty where human experts can then apply their depth. Despite the uncertainty, the most resilient approach for aspiring mathematicians is to maintain an adaptable mindset, embracing both traditional foundational training and the experimental opportunities created by modern AI.