DeepSeek Tops the Hardest AI Benchmark: Costs Under $5 a Day

DeepSeek Tops the Hardest AI Benchmark: Costs Under $5 a Day

AIDeepSeekCost

Sources:HN + web research · HN

In August 2026, a Chinese AI model topped the efficiency leaderboard in the ARC-AGI challenge, widely regarded as the hardest benchmark for human-like reasoning in AI. The model, named DeepSeek V4 Flash 0731, is a lightweight, high-speed variant of the open-weights foundation model DeepSeek V4. Official ARC Prize results show that on the more challenging ARC-AGI-2 benchmark, it scored 61.4% at a cost of 4 cents per task; on ARC-AGI-1, it achieved 89.0% at just 2 cents per task.

On the same day, the news surged to 754 points and 451 comments on Hacker News, becoming the top tech discussion on the platform. Just a day prior, the post had accumulated 365 points, more than doubling its engagement in 24 hours.

What makes this milestone intriguing is the stark contrast: while developers are accustomed to believing that cheaper tools mean inferior quality, a budget-friendly open-weights model from China just claimed the top efficiency spot on the most grueling reasoning benchmark in the industry. Community members testing it firsthand reported running 5 to 6 parallel sessions with 12 concurrent streams throughout an entire day of heavy usage, spending under $5 in total.

What the “Hardest AI Benchmark” Actually Tests

The tasks in ARC-AGI appear deceptive in their simplicity: visual grid puzzles featuring colored squares. An AI system must infer underlying transformation rules from a few visual examples and generate the correct answer. These problems are intuitive for humans—most people can solve them given a bit of time. For AI models, however, ARC-AGI is notoriously difficult because it evaluates abstract reasoning where memorizing training data is useless. According to the ARC Prize creators, the benchmark was specifically designed to target core AI weaknesses.

Over the past two years, progress by frontier models on this benchmark has been relatively slow, with many systems burning through several dollars—or more—per problem attempt. In 2025, the ARC Prize updated its rules: all submitted scores must be accompanied by an efficiency metric. Scores indicate capability, while cost reveals how expensive that capability actually is.

ARC Prize Official Efficiency Chart: Capabilities vs. Cost, Titled "Scale is Not Enough" Figure: Official ARC Prize efficiency chart (titled “Scale is Not Enough”). All benchmark results after ARC-AGI-2 must report efficiency metrics. Source: arcprize.org

Based on public disclosures, DeepSeek V4 Flash 0731 stands out as one of the most prominent models on the efficiency leaderboard to combine high accuracy with exceptionally low cost. The ARC Prize team noted that it “sets a new frontier for cost versus performance.”

ARC-AGI-2 Public Benchmark Sample Task: Pattern Recognition from Grid Puzzles Figure: Sample task from the public ARC-AGI-2 benchmark (symbolic reasoning). Simple for humans, notorious for tripping up AI models. Source: arcprize.org

Engineering Takeaway: At 2 cents per question, testing costs effectively vanish from the equation. Where labs previously had to budget evaluations carefully, developers can now run thousands of test iterations without hesitation. That shift matters even more than the benchmark score itself.

How DeepSeek Made It So Cheap

Some might assume that low cost implies cut corners. However, public architectural details show that DeepSeek V4 Flash achieves its cost reduction through three structural design layers.

The first layer is a sparse Mixture-of-Experts (MoE) architecture. Although the model contains approximately 284 billion total parameters, it activates only about 13 billion parameters per task. Think of it like a library housing nearly 300 billion volumes where a reader only opens the few books required for a specific inquiry, drastically saving energy. Activating only a small subset of experts per query is currently one of the primary industry strategies for reducing inference costs.

The second layer is prompt caching optimization. Many computations can be reused when an AI answers queries, and the official API achieves a cache hit rate of roughly 99%. Independent evaluation firm Artificial Analysis estimated that DeepSeek V4 Flash averages roughly 3 cents per benchmark task. Tech media reports suggest this is less than one percent of the cost incurred by certain top-tier proprietary models in the US.

The third layer stems from open-source ecosystem competition. Because DeepSeek V4 Flash weights are open, anyone can download and self-host the model, allowing third-party platforms and community providers to set competitive pricing. Prices are dictated by market competition rather than a single vendor.

DeepSeek V4 Flash 0731 Benchmark Result Cover Image: Scores of 89.0% and 61.4% Figure: Cover image produced by a third-party blog for the benchmark result, highlighting scores and per-task costs. Source: explainx.ai

Engineering Takeaway: When architecture, caching, and open-source ecosystem dynamics stack together, the cost collapse is structural. As long as this design paradigm persists, low pricing is here to stay.

Expensive AI Subscriptions Face an Awkward Reality

Real-world community usage highlights this shift clearly. Hacker News user LaurensBER shared that even when running 5 to 6 sessions with 12 concurrent streams all day, spending exceeded under $5 for an entire day of intense work. Despite subscribing to Claude Max, the user found themselves rarely opening it, as returning to subscription tiers meant constantly worrying about rate limits. The user remarked that while expensive models are indeed more powerful, using them feels like a step backward due to quota anxiety.

Another user, abixb, was even more blunt: “If this is true, American AI labs are in big trouble. Unless Chinese models are banned via national security orders, no one is going to pay a premium for frontier models.”

While sharp, the comment hits a default assumption in the industry: frontier AI models ought to be expensive. Subscribing to AI services traditionally meant buying quotas—paying tens or hundreds of dollars a month only to face queuing and rate limits under heavy use. In contrast, DeepSeek V4 Flash operates on pay-as-you-go pricing so cheap that tasks can be offloaded liberally: letting AI automatically fix failing tests, inspect anomalous log files, or generate missing test coverage. Developers previously hesitated to adopt these workflows due to prohibitive costs.

Engineering Takeaway: When AI becomes so inexpensive that intermediate output is disposable, usage behavior shifts from conservative conservation to unrestricted execution. As HN users pointed out, a $10 budget grants $140 worth of equivalent usage—making it hard to burn through even if you try. That transformation impacts daily workflows far more than any leaderboard rank.

What This Means for Everyday Users

For non-technical users who don’t write code, the benefits of this efficiency race are immediate: AI services as a whole will become cheaper. As open-weights models drive costs down, proprietary vendors will be forced to lower prices or risk losing users. The expanding free tiers and falling monthly subscription fees seen across various AI tools over the past year are directly tied to this ongoing cost race.

Looking ahead, AI will expand into scenarios where it was previously cost-prohibitive. Small businesses unable to afford dedicated support staff can deploy AI assistants; parents helping children with homework can ask endless questions without worrying about token bills; and everyday tasks like organizing bills, drafting documents, or refining resumes can be initial-drafted by AI before human review.

A caveat is warranted: cheap does not mean omnipotent. Top efficiency rankings highlight cost-effectiveness, but DeepSeek V4 Flash still trails top-tier closed models on certain highly complex tasks. HN commenters also pointed out that in long, multi-step workflows, minor errors from smaller models can compound into larger failures. A practical recommendation is to use affordable models as the primary driver and reserve premium models for fallback tasks.

Final Thoughts

Public data indicates that this cost collapse is far from over. The rapid catch-up speed of open-source models is forcing the entire industry to recalculate unit economics.

This milestone is best viewed as a clear signal: the AI competition is expanding from “who is smarter” to “who is cheaper.” While we wait for the next breakthroughs in intelligence, the era of accessible affordability has already arrived. At under $5 a day—cheaper than a boba tea for many—this model has claimed the top spot on the efficiency leaderboard of humanity’s hardest AI benchmark.

References:

  • ARC Prize: DeepSeek V4 Flash 0731 Evaluation Results
  • HN Discussion (item?id=49214008)