Agents Score Under 12% on Real Work. So Why Is Big Tech Buying Dev Tools?
Three deals in one week, all pointing at the same layer
Meta is acquiring AI coding startup Conductor, which lets developers control multiple AI coding agents at once. On the same day, OpenAI announced plans to acquire Pierre Computer; Anthropic had already bought Stainless back in May. The Decoder reads the pattern this way: AI models are becoming interchangeable, and what vendors are racing to own is the layer through which developers reach those models.
The deal values, team sizes and integration plans are not disclosed in the available signals — only the targets and the timing are clear. But the direction is unambiguous: whoever owns multi-agent orchestration and the access layer owns the developer's default path. For Chinese readers, the useful comparison is where domestic equivalents sit — once "interchangeable models" is the premise, a toolchain's stickiness no longer comes from model scores.
The 12% ceiling
SemiAnalysis put AI agents through simulated knowledge work in its new GDPx benchmark, covering everything from chip design to quant trading. The result: no model breaks 12 percent. Claude Opus 5.5 leads narrowly over Gemini 4 Carbon.
Two caveats matter. First, critics point to the small number of tasks per domain and uneven testing, so the benchmark itself still needs work. Second, 12 percent is a score on a specific task set, not a measure of how usable agents are everywhere. Even so, the gap between that number and the "agents will take over the work" narrative is wide.
Two routes on the Chinese side
Tencent unveiled Hy4, its first multimodal language model that understands images and can operate software on its own. It scores 35 points on the Artificial Analysis Intelligence Index — solid, but behind Kimi K3.1 Pro, and it still trails the best commercial models on hard math and complex image analysis.
Baidu released ERNIE 5.5 Flash Base: a 280-billion-parameter MoE model pre-trained on 33.5 trillion tokens, shipped deliberately without fine-tuning so developers can take it wherever they want. That is the opposite of a "sell the API" route — hand out the base and leave differentiation to downstream.
The two routes are not in conflict: one competes on multimodality and autonomous software operation, the other on ecosystem entry points. Under the "models are interchangeable" premise, the choice between open-source self-hosting and closed APIs depends mostly on whether a team needs to keep fine-tuning and data in its own hands.
Three actions for teams
- Take evaluation back in-house. The 12 percent result and the debate around it both show the limits of public leaderboards. A small eval set built from 20–50 real tasks in your own business will decide your stack better than any ranking.
- Watch the orchestration layer, not a single model. Tools like Conductor are valuable because they manage several agents at once. If your team already runs multiple models, scheduling, logging and rollback affect delivery more than swapping the model does.
- Evaluate open base models seriously. ERNIE 5.5 Flash Base is explicitly released without fine-tuning, which means downstream teams carry the alignment work — that requires data and engineering capacity. Without it, waiting for community fine-tunes is more realistic.
Conclusion
The three acquisitions and the 12 percent benchmark do not contradict each other: vendors are betting capability will catch up while securing the entry point first. For practitioners, the safest short-term move is not waiting for a stronger model but getting three things right — evaluation, orchestration and verification. Axiom Math's use of AI agents to formally verify the Stacks Project points the same way: while capability falls short, verifiability is worth more than autonomy.
Sources
- Meta buys AI coding startup Conductor as big tech races to lock in developers
- No AI agent scores above 12 percent on SemiAnalysis' new real-world work benchmark
- Tencent releases Hy4 with vision and computer use, claims second place among Chinese AI models
- Baidu releases ERNIE 5.5 Flash Base with 280 billion parameters as an open-source foundation model
- Axiom Math wants to keep AI slop out of mathematics
Subscribe to weekly briefing