In September 2026, solo developer Mithil Vakde scored 44% on the public ARC-AGI-1 evaluation after running for just 1.5 hours on a single RTX 5090 GPU. The entire training run cost merely 67 cents. With a single consumer-grade graphics card and a bill of under one dollar, his solution directly matched the scores of top specialized solvers like TRM and HRM, while leaving a host of multi-million-dollar general LLMs behind.
Created by François Chollet in 2019, ARC-AGI has long been considered the benchmark for measuring human-like abstract reasoning, with the community offering a $1 million prize bounty. A 6-year-old benchmark benchmarked by the industry was tackled by a single individual using a fundamental Transformer architecture.
67 Cents for a 44% Score
Mithil Vakde’s 44% score showcases the incredible efficiency of the open-source community at smaller parameter scales. When an earlier version of this project was released, it sparked public discussions among top researchers, including Lucas Beyer from Google DeepMind, Jeremy Howard from fast.ai, and Rohan Anil. A model developed by an individual in a hardware-constrained environment performed on par with elite teams from major tech labs.
The brute-force approach of burning money to stack data for large language models shows clear fatigue in few-shot reasoning. Top specialized solvers demand complex architectures and expensive compute investments, whereas this open-source approach reproduced equivalent capabilities for under a dollar. Stacking compute cannot hide algorithmic inefficiency on specific tasks; precise architectural pruning proves far more effective than brute-force scaling.
Figure: ARC-1 Public Benchmark Performance Comparison. Source: Mithil Vakde’s blog
Dropping Key Components Halves the Score
Rather than departing from the Transformer framework, this approach precisely incorporates modern architectural components such as SwiGlu, RMSNorm, 3D RoPE positional encodings, and the NorMuon optimizer. In the ablation studies published by the author, removing either 3D RoPE or per-task embedding instantly causes the model’s score to plunge from 44% down to around 24%.
| Configuration | Score |
|---|---|
| Full Configuration (3D RoPE + per-task embedding) | 44% |
| Without 3D RoPE | ~24% |
| Without per-task embedding | ~24% |
| Baseline Configuration Only | 18% |
| Train Output Tokens Only (vs. previous version) | 40% → 44% |
The essence of the ARC task lies in processing two-dimensional grids and color channels, and 3D RoPE maps onto this multi-dimensional structure perfectly. Customized spatial positional encodings and embedding mechanisms tailored to specific tasks precisely address the model’s weakness in local feature extraction.
These ablation data highlight a crucial point: precise modeling of task structure is far more valuable than stacking parameters. The brute-force training across all parameters common in general LLMs hits a wall in highly abstract reasoning tasks. Deep architecture tuning aligned with task domain logic serves as the true leverage for high performance.
Figure: Ablation Study: Sharp score drop without 3D RoPE and per-task embedding. Source: Mithil Vakde’s blog
High Research Costs Limit Exploration Space
Mithil Vakde offers a bold assessment: large language models hold no inherent advantage in sample efficiency. The reason no one unlocked this path on ARC over the past 6 years was not an insurmountable technical barrier, but rather that prohibitive experiment costs locked down the exploration space for lab researchers. Single training runs costing thousands or tens of thousands of dollars prevent major lab teams from running dozens or hundreds of ablation experiments, making it nearly impossible to discover which fine architecture details actually matter for ARC tasks.
Trial-and-error costs of a few cents empower an individual developer to rapidly iterate through various component combinations. Every seemingly minor parameter adjustment can be verified within 1.5 hours. General capabilities stacked by tech giants with millions of dollars in training fees look clumsy when facing single-point tasks requiring dense trial-and-error and precision strikes.
Figure: Optimal scores across other ablations. Source: Mithil Vakde’s blog
Sample Efficiency Takes Center Stage
When the union of solved tasks across multi-run executions reaches 55%, the author estimates that Transformers alone could reach a threshold of 65% on ARC. This fully open-sourced solution, complete with model weights and training code, garnered 546 points and 146 comments on Hacker News, igniting community excitement over the potential of small models. Beyond its stellar performance on ARC-AGI-1, the model also achieved 7% on the significantly harder ARC-2 benchmark.
Replicating the results of top research teams for 67 cents highlights the inefficiency of multi-million-dollar corporate spending. Compute dominance does not directly translate into benchmark scores; the sample efficiency demonstrated by the open-source community at fraction-of-a-dollar costs is becoming the defining factor in the next phase of AI competition.
Reference Links:
- Mithil Vakde’s Blog
- HN Discussion
- Lobsters Discussion
