Baidu's ERNIE Evolve Agent System Sets NanoGPT and NanoChat Records by Rewriting AI Code

The system takes a code repository, an objective and evaluation criteria, then chooses research directions, changes the code and runs experiments. A change is kept only if it passes a hidden evaluator that none of the agents can see and a separate reviewer that checks whether the improvement is genuine or exploits a loophole in the test, according to a detailed account of the release by MarkTechPost. Baidu described the work on X as a practical step toward recursive self-improvement, in which AI systems improve the software used to build and serve AI.

The release matters because automated research has become a competitive front among Chinese labs, with Tencent's Hunyuan team and smaller groups publishing rival systems, and because Baidu says it has already applied the method to production engineering on its own chips rather than only to benchmarks.

How the System Works

ERNIE Evolve runs two nested loops. In the outer loop, a Director agent picks the next research direction and a starting version of the code, drawing on a bank of past experimental results and a knowledge base of papers and code. In the inner loop, a Selector picks a candidate and the next hypothesis, and an Executor implements it in an isolated environment. Several research lanes run in parallel, each with its own tree of code versions, and failed experiments are logged so later rounds avoid repeating them. Gains must exceed a noise threshold to count. Baidu has not said which large language models power the agents.

On NanoChat AutoResearch, the system reached a validation score of 0.8763 bits per byte within a five-minute, single-GPU budget on an Nvidia B200, by adding an n-gram branch and a document-retrieval branch alongside the network. Rival systems listed in the comparison scored between 0.8878 and 0.9109, with lower being better. On NanoGPT Speedrun Track 1, it reached the target loss in 38.9 seconds on eight H100s, against a previous record of 39.9 seconds, and on Track 3 it needed 2,590 steps against 2,690. It fell short on SOL-ExecBench, a set of 235 GPU kernel problems, scoring 0.819 against a leaderboard best of 0.827.

Gains on Kunlunxin Chips

Baidu then pointed the system at three engineering tasks on ERNIE-5.5-Flash, a model still in development, where engineers had already tuned the baselines. On Baidu's Kunlunxin P800 chips, generation speed rose to 623.6% of the baseline over four research rounds, helped by speculative decoding through the model's multi-token prediction layer and reduced host-side overhead. Throughput in deterministic inference mode, used for reinforcement learning, rose 33.7% in about three days, and single-request decoding on GPUs became 14.9% faster with accuracy unchanged. Only three of seven research directions on that last task produced gains.

The results are self-reported and the margins on public benchmarks are small. Baidu has not released the system itself; it has published the final code and logs from the runs on GitHub, which lets others check the outputs but not reproduce the process that found them.