ARC-AGI-3 Shows Self-Upgrading AI Barely Improves

AI that tries to upgrade itself barely got any smarter on a tough puzzle test called ARC-AGI-3, gaining just 2.8 points after rewriting its own code twice, researchers found.

Key Takeaways

AI systems that rewrote their own control code twice improved by only 2.8 points on average on puzzle games they had never seen
Researchers tested 27 versions and tracked both their scores and the dollar cost of running them
Each AI started from the same basic code, practiced on five games, rewrote its code twice and was then tested on 20 new games
Scores rose more on the practice games than on the new ones, a sign the AI was learning the practice set rather than getting smarter

The study, shared on X by AI researcher Elvis Saravia, put leading AI models to work on ARC-AGI-3, a test where an AI has to explore games it has never seen, work out the goal on its own and act through code instead of answering a simple question.

AI Tried To Upgrade Itself And Mostly Failed

Every AI model comes wrapped in control software that decides when it should use tools, save notes, retry a failed step or think harder about a tricky move. Researchers often call this wrapper a harness, and it can matter as much as the model inside it.

The study asked a simple question: can today’s top AI models build that control software themselves, then make it better?

Each model started from the same basic code, built its own setup, practiced on five games and then rewrote its code twice. Every version was then scored on 20 fresh games it had never seen.

Across the 27 attempts, results ranged from zero to almost perfect. That spread shows much better setups really exist.

The catch is that letting the AI improve itself did not get much closer to them. On the 20 unseen games, scores rose by just 2.8 points on average, and the researchers’ margin of error ran from a 1.4-point drop to a 7.4-point gain.

Scores on the five practice games jumped more than on the new ones. That is a classic sign the AI was memorizing the practice set rather than getting genuinely smarter.

Some versions also cut the cost of running the AI, but those savings sometimes came with weaker results.

Also Read: Meta’s Muse Wonder Full Scores 79, Trails Anthropic’s Claude By Two Points

What Is ARC-AGI-3 And Why It Is So Hard

ARC-AGI-3 is not a chatbot quiz. Instead of answering one question, the AI has to move through new game worlds, figure out what it is supposed to do and keep getting better as it goes.

That makes the control software around the model unusually important. Every step depends on what the AI remembers, when it retries and which tools it picks, so a weak setup can waste even a strong model.

It also reflects a bigger shift in how AI gets judged. For years, labs raced to train ever-bigger models. Now the race is moving to the software that turns a model into an agent that can actually get work done.

Also Read: OpenAI Locks Down AI Training As Stolen Models Cost Near Zero To Copy

Why Self-Fixing AI Is Not A Shortcut Yet

The study does not say AI can never improve itself. It says the gains are smaller and shakier than practice runs suggest.

That is a blow to teams betting that AI will soon upgrade itself with little human help.

It also hits the budget. If an AI burns more computing power rewriting its own code than it saves on the job, the math stops working fast. Businesses pay for the whole system, not just a better test score.

For companies putting AI agents into customer support, research or office work, the lesson is simple: better testing and human checks may pay off more than letting the AI rewrite itself unchecked.

What to watch next is whether labs run longer ARC-AGI-3 experiments with outside feedback, such as bigger practice sets or human reviews between rewrites. If those widen the gains on unseen games, self-improving AI could become a real shortcut. Until then, it works, but only a little.