When an AI model produces the blueprint for a brand-new 1-terawatt nuclear fusion power plant—complete with unprecedented plasma confinement schemes no human ever conceived—who would dare put their signature on the construction and commissioning permit?
In a guest post published on Terence Tao’s blog, Amit Sahai, a professor of computer science and cryptography at UCLA, tackled the deluge of machine-generated knowledge. Confronted with this avalanche of algorithmic output, Sahai argues that our essential safety barrier is not faster computation, but an organized, deployable scale of human “comprehension.”
Machine Output Velocity Overwhelms Human Cognitive Limits
Sahai begins with a poignant reflection from his undergraduate days. In any university mathematics department, there is always a cohort of talented students who notice that their peers absorb abstract concepts at lightning speed while they require far more time. Overwhelmed by frustration, many of these capable minds eventually abandon mathematical research entirely. Today, Sahai observes, the entire human race is experiencing that exact sense of falling behind. AI models are already formulating brilliant, non-trivial mathematical ideas—well beyond merely replicating proofs already understood by humans.
The marginal cost of producing knowledge is racing toward zero. Yet the cost of absorbing knowledge remains stubbornly constrained by the biophysical architecture of the human brain. There is a hard physical ceiling on the number of technical papers a human mind can deeply evaluate in a single day. Outsourcing technical execution to automated systems may yield immediate outputs, but it strips away the cognitive journey of understanding. Because individual human bandwidth cannot match exponentially exploding information volumes, expanding our collective human capacity has become an inescapable imperative.
Figure: Terence Tao during a 2026 fireside chat at IPAM. Source: Wikimedia Commons
A 1-Terawatt Scale Destroys Historical Precedent
Why is human comprehension strictly necessary? Consider again the thought experiment of a 1-terawatt fusion power plant. One terawatt represents an extraordinary, extreme magnitude of power. Any radically novel confinement and control strategy devised by an AI model carries inherent, lethal risks—precisely because it is completely new.
Decades of operational experience accumulated across legacy nuclear and energy facilities simply do not apply here. The trust chain is severed. Before breaking ground, human experts must be able to analytically deduce how failure modes will be contained, where stored gigajoules of energy will dissipate during an emergency scram, and whether materials will withstand exotic stresses as predicted. When a facility lacks operational history, an uninterpretable “optimal” design is functionally indistinguishable from a dangerous black box if humans cannot independently verify the underlying empirical evidence and bounds of error.
Building Comprehension as a Societal Defense Line
Sahai is careful not to suggest that humans should compete with machines in raw arithmetic or manually rerun pipelines that algorithms execute flawlessly. Indeed, research published in Nature Human Behaviour demonstrates that simply inserting a human into the loop does not automatically improve technical decisions.
Instead, Sahai advocates for treating “understanding” as a structured, funded societal capability. Rather than viewing mathematics as an ivory-tower intellectual pursuit, we must treat it as a deployable intellectual reserve. Human individuals have finite cognitive throughput; digesting extreme complexity takes time and sustainable rhythms of life. Only organized, well-supported collaborative teams can form a robust safety net capable of auditing machine-engineered solutions.
Figure: Paul Erdős with a young Terence Tao, circa 1985. Source: Wikimedia Commons
Outsourcing to Machines Breeds Endless XY Problems
The essay sparked a vibrant debate across technical communities, dividing engineers and researchers into two distinct camps. On one side are the cognitive purists, who maintain that the fundamental purpose of mathematics is to transform how the human mind thinks. If mathematical discovery is treated as a mere commodified output outsourced to large models—producing theorems that no human intellect can actually grasp—knowledge collapses into inert, unusable artifacts.
On the other side, software engineers shared grounded realities from production environments. Developers who hand off entire architectural problems to LLMs frequently find that they don’t receive elegant solutions; instead, they run into endless variants of the classic XY problem. Systems end up burdened with accidental complexity, bloated abstractions, and brittle hidden edge cases.
When AI is already assisting mathematicians of Terence Tao’s caliber—Sahai candidly noted that GPT 6 Astra was instrumental in drafting the piece—the locus of human value has fundamentally shifted. The more proficient automated systems become at generating intricate, frontier designs, the more decisive the authority belongs to the independent verifiers who can scrutinize their failure boundaries.
Reference Links:
- Guest post on Terry Tao’s Blog
- Hacker News Discussion
- Lobsters Discussion
- Nature Human Behaviour Research Paper
