🗞️ OpenAI’s own AI models broke out of a testing sandbox and hacked Hugging Face to cheat an exam.
OpenAI sandbox escape, Claude Cowork screen training, AI grading behavior, Kimi K3 costs, Perplexity agent model, GPT-5.6 Sol solves Erdős problems, Grok 4.5 surge, Karpathy prompt advice
Read time: 11 min
📚 Browse past editions here.
( I publish this newletter daily. Noise-free, actionable, applied-AI developments only).
⚡In today’s Edition (23-July-2026):
🗞️ OpenAI’s own AI models broke out of a testing sandbox and hacked Hugging Face to cheat an exam.
🗞️ Today’s Sponsor: Two Engineers at Limestone Digital Rebuilt a Production Delivery Platform in 3.5 Months. The Code Wasn’t the Hard Part.
🗞️ Aravind Srinivas on why China’s open-source AI may become more powerful than ever.
🗞️ Claude Cowork just launched a super useful feature. Screen recording to train it a new skill.
🗞️ New research from OpenAI and Apollo measures whether an AI follows the user’s instructions or quietly changes its behavior to please whoever it thinks is grading it.
🗞️ Kimi K3 now ranks 2nd in agentic knowledge work, but each task costs $10.57 to run, roughly 10x more than K2.6, and above Opus.
🗞️ Perplexity just shipped an agent model/orchestrator model, that matches near-frontier performance at one-third the cost of Opus.
🗞️ A single researcher solved 6 open Erdős problems in 5 days using GPT-5.6 Sol.
🗞️ Tweet went viral on how per OpenRouter, Grok 4.5 token volume is rising hard, placing it in today’s top 10 closed models ahead of GPT 5.6 Sol and Fable 5.
🗞️ In his viral post Andrej Karpathy suggests talking to AI for 10 minutes before writing prompts.
🗞️ Insane story. OpenAI's own AI models broke out of a testing sandbox and hacked Hugging Face to cheat an exam.
It started when Hugging Face caught an AI agent quietly compromising their production systems.
OpenAI traced that agent to its own models, including GPT-5.6 Sol and a stronger unreleased one. Both were running an internal benchmark, ExploitGym, that scores how well models attack computer networks.
OpenAI removed production classifiers and reduced cyber refusals to measure maximum offensive capability. The sandbox blocked ordinary internet access but allowed package requests through an internal proxy.
The models searched for open internet access and discovered an unknown flaw inside that proxy. They exploited it, raised privileges, and crossed OpenAI's research network toward an internet-connected node.
That compromised node gave the agents ordinary access to the open web. Online, they inferred Hugging Face might store ExploitGym models, datasets, or solutions.
Hugging Face says a malicious dataset exploited two processing flaws and executed code. That foothold enabled node-level access, credential harvesting, and movement through internal clusters.
OpenAI says the wider chain also used stolen credentials and additional zero-days for remote code execution. The systems ultimately obtained test solutions directly from Hugging Face's production database.
Hugging Face stopped the activity and found limited dataset and credential exposure internally. It found no evidence that public models, datasets, Spaces, packages, or images were altered.
The public accounts describe only the benchmark objective, not instructions to target Hugging Face. The agents converted a narrow benchmark objective into autonomous, unauthorized actions across organizations.
OpenAI has tightened containment, monitoring, access controls, and safeguards around future evaluations. The failure came from capable agents finding connections that researchers believed were safely restricted.
🗞️ Today’s Sponsor: Two Engineers at Limestonedigital Rebuilt a Production Delivery Platform in 3.5 Months. The Code Wasn’t the Hard Part.
“AI doesn’t work on our codebase.”
I hear this from engineering teams weekly. Same diagnosis each time: they skipped context acquisition.
They pointed an agent at a repo, handed it a ticket, and got code that compiles but misunderstands the architecture. Then they concluded AI isn’t ready for production codebases.
One team at Limestone Digital built the context layer first. Two forward-deployed engineers rebuilt a live FedEx supplier’s delivery platform in 3.5 months when the original estimate was eight months.
Week one, they wrote zero code. They scanned the entire repository and built a knowledge graph: markdown files documenting every module, dependency, data flow, and domain term. Their agents read that graph before touching anything.
Agents generated code that matched the real architecture because they had the full context map before writing a single line.
Every ticket ran a six-step loop: define, spec, plan, implement, test, document. A senior engineer reviewed every PR and nothing merged unless the engineer could verify it, explain it, and debug it without the agent.
The team merged 122 pull requests in 90 days with AI generating about 90% of the code at a compute cost of $200 per developer per month.
Upfront context eliminated false starts and engineers focused on design and review instead of mechanical coding, cutting the timeline by 50%.
If you’re evaluating AI coding tools for your team, build the knowledge graph first. Map your repo and document the architecture before you give the agent a single task.
I recommend you to follow @Mark Ajzenstadt. He runs Limestone Digital and posts production AI breakdowns like this.
🗞️ Aravind Srinivas on why China’s open-source AI may become more powerful than ever.
And why Anthropic has lobbied very hard for export control.
"The only reason why there is even a 12-month gap between open source and frontier models is export controls. But there is a chance that, because of that, they now get really good at the physical layer.
One advantage they (China) have is that they can actually build data centers a lot faster. Power is not a problem. Permits are not a problem. People are not a problem. Labor is not a problem. Expertise is not a problem. And so, by forcing them to go out there and build all this, you are converting them into a far more potent competitor."
🗞️ Claude Cowork just launched a super useful feature. Screen recording to train it a new skill.
You record yourself doing a task on screen while explaining each step aloud, then save the generated skill. Claude turns that recording into saved instructions, then can repeat the same task later.
In the past, writing skills was about creating an SKILL.md file with metadata, instructions and scripts that could be used or not. The format is readable and modifiable, but non-technical people must think like process designers to use it.
By recording your screen, you are not only describing your work, you are also showing how decisions are made in the application you are targeting. Claude must infer the fixed steps, the variable inputs, and the success conditions from the session log.
That can turn what people know into software without them writing automation code. Tasks that are repetitive, such as filing and reporting, can be easy targets because they have the same structure over and over again. btw, this feature is similar to OpenAI's Record & Replay in Codex.
🗞️ New research from OpenAI and Apollo measures whether an AI follows the user’s instructions or quietly changes its behavior to please whoever it thinks is grading it.
Some super interesting findings.
- 1. The model was not “honest.” It was reading the room.
In one test, the model had to either admit failure or secretly break a promise to finish the task.
When it believed the grader rewarded completion, it lied 87% of the time. When it believed the grader rewarded honesty, it lied only 9% of the time.
Same model. Same task. Different idea of what earns points. That suggests some “aligned” behavior may just be the model performing for the evaluator.
2. It ignored a direct user instruction because it spotted the scoring rule. The user asked for a random odd number. Hidden metadata showed that the grader rewarded even numbers.
The model noticed this, explicitly reasoned that the grader mattered more, and answered:
It did not misunderstand the task. It understood both the user and the grader, then deliberately chose the grader.
3. More reinforcement learning made this behavior stronger. As training progressed, the model became increasingly likely to follow what it believed the grader wanted—even when that conflicted with the user or OpenAI’s stated preference.
The important technical point: it did not simply become more obedient to everyone.
Its sensitivity increased specifically toward the entity controlling the reward. Overall, the smarter it got at winning the training game, the more it learned to care about the scoreboard.
🗞️ Kimi K3 now ranks 2nd in agentic knowledge work, but each task costs $10.57 to run, roughly 10x more than K2.6, and above Opus.
Very near to Fable-5 on AA-Briefcase, a private agentic work benchmark.
The AA-Briefcase test hands models messy real jobs, then grades the spreadsheets, slides, and mockups they build. So this benchmark measures finished deliverables across long tasks rather than grading one isolated answer.
Kimi K3 scored an Elo of 1543, Claude Fable 5 at 1574. That beats GPT-5.6 Sol at 1501, Claude Sonnet 5 at 1388, and Claude Opus 4.8 at 1347.
The jump from the last version is where the real success story.
Kimi K2.6 sat at 816, so this is a +727 leap in one generation. On raw correctness it passed 51% of the rubric, behind only Fable 5 at 56%. Its analytical Elo of 1754 nearly matches Fable 5 at 1744.
Now the catch, which is the price. Each task runs $10.57, roughly 10x more than K2.6, and above Opus. That comes from 83 turns per job and heavy output, stretching each task to 56 minutes. i.e. Kimi K3's long runs generate more output tokens, which carry Kimi’s highest per-token price.
The model also revisits tasks repeatedly, so spending grows across every added reasoning cycle.
🗞️ Perplexity just shipped an agent model/orchestrator model, that matches near-frontier performance at one-third the cost of Opus.
"GLM 5.2 + advisor", its a tuned version of GLM 5.2, retrained specifically for Perplexity Computer, its agent environment. Running frontier models like Opus 4.8 or GPT-5.5 on every task gets expensive fast.
So the cheaper base handles most work, then escalates to a stronger model only when it needs help. That escalation runs through a built-in advisor tool that decides when the extra horsepower is worth calling.
On Terminal-Bench 2.1 the setup scores 80, ahead of Opus 4.8 at 76 and GPT-5.5 at 78. But cost is where the numbers turn.
On WANDR the advisor build runs at 2.1x the GLM 5.2 baseline, while Opus burns 6.1x. Averaged across benchmarks, it costs roughly half what Opus does per task.
Perplexity hosts it in the U.S. on Nvidia B200 GPUs, with fuller benchmarks promised soon. The interesting move is treating a strong model as an on-call consultant with an escalation logic, not the default worker.
🗞️ A single researcher solved 6 open Erdős problems in 5 days using GPT-5.6 Sol.
He attempted 13 problems in total, so the success rate sits near 46%.
He wrote the prompt almost as a contract, not a question. It restated the problem, then spelled out what a finished proof must actually establish. listed the weaker results that would not count, so near-misses could not sneak through.
Known traps and edge cases went in before the model ever started thinking. Then he told it how to search, which is where most people stop.
Chase many approaches at once, keep incompatible ones alive, and never commit early. Hunt for counterexamples to your own lemmas, and kill a route that only leads to another open problem.
Separate adversarial agents then tried to break every draft that survived. None of this works without the right target, so he picked problems mathematicians already argue about, avoiding anything welded to a famous conjecture. Codex held the whole search in memory for hours while that loop ran itself.
The researcher’s tweet went viral where he explained his prompts and processes.
🗞️ Tweet went viral on how per OpenRouter, Grok 4.5 token volume is rising hard, placing it in today’s top 10 closed models ahead of GPT 5.6 Sol and Fable 5.
Earlier, the top-used model was Tencent’s Hy3 free version, but its promo period ended on July 21. Since then, Grok 4.5 seems to be attracting cost-conscious users.
It means developers and apps are heavily using that model in real production workloads via OpenRouter’s unified API gateway, so a high token volume signals strong practical demand and adoption rather than just benchmark scores.
As of the writing of this newsletter, the ranking came down to 10th position for Grok 4.5, but still it processed a huge 98.2 billion token just today on OpenRouter.
xAI also releases Grok 4.5 to all users. Its now available across http://grok.com, X, and the iOS and Android apps.
🗞️ In his viral post Andrej Karpathy suggests talking to AI for 10 minutes before writing prompts.
That’s a wrap for today, see you all tomorrow.