🗞️ ByteDance published EdgeBench, a benchmark that checks whether AI agents get better with experience
EdgeBench for AI agents, Mark Cuban on AI skills, human vs LLM research ideas, HBR on AI speeding wrong work, AI’s messy productivity phase, Gym-Anything agent environments, multi-agent debates
Read time: 8 min
📚 Browse past editions here.
( I publish this newletter daily. Noise-free, actionable, applied-AI developments only).
⚡In today’s Edition (05-July-2026):
🗞️ ByteDance published EdgeBench, a benchmark that checks whether AI agents get better with experience
🗞️ Mark Cuban’s advice for graduates walking into their first job. Learning AI is no longer optional.
🗞️ “Measuring the Gap Between Human and LLM Research Ideas”
🗞️ Harvard Business Reviews’s new piece: The rush to use AI can make companies faster at the wrong work.
🗞️ Current AI is in a messy middle phase where usage looks productive, but output remains unclear.
🗞️ “Gym-Anything: Turn any Software into an Agent Environment”
🗞️ “What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debates”
🗞️ ByteDance published EdgeBench, a benchmark that checks whether AI agents get better with experience
They released EdgeBench, to test whether AI agents can improve through experience, using 134 real-world tasks that run for at least 12 hours.
The big deal is that it shifts AI evaluation from “what does the model already know?” to “can the model learn while doing real work?”
Huge, because future AI agents will not just answer questions from training data. They will enter messy environments, use tools, make attempts, read feedback, fix mistakes, and slowly build better solutions.
Most current benchmarks are too short for that, so they mostly test memory, coding skill, or one-shot reasoning. EdgeBench instead gives agents 12-hour real-world tasks with feedback loops, so it can measure whether the agent improves through experience.
Each task has a local workspace for fast trial and error, plus a hidden judge that gives stronger feedback on submitted work, which is meant to feel closer to real expert work. The authors then ran frontier agents for about 38,000 total hours and tracked how their best score changed as they kept interacting with the task environment.
The big result is that when scores are averaged across many tasks, learning follows a very clean log-sigmoid curve, meaning progress is slow, then faster, then starts to level off. They also found that newer agents seem to learn from environments much faster, with the top models roughly doubling their 2-hour learning speed every 3 months.
🗞️ Mark Cuban’s advice for graduates walking into their first job. Learning AI is no longer optional.
"If you’re not the person who knows how to do vibe coding or how to do all these different things with agents and Claude, somebody who does is going to take your place.
If your boss enables you to use that extra knowledge, great. If they don’t enable you to use that extra knowledge, they’re not going to be your boss very long. And if the CEO doesn’t understand that, he or she is not going to be the CEO very long. And if they still keep that CEO who’s not using AI to get ahead, you tell me, so I can start a company to kick their ass."
🗞️ "Measuring the Gap Between Human and LLM Research Ideas"
This Yale + University of Chicago paper shows that real gap between LLM generated research ideas vs humans is not idea quality, but idea range: LLMs think narrower than human researchers.
The researchers built a controlled test from 11,683 real papers, using each paper’s nearby prior work as the shared starting point. They asked models to propose a new motivation and method from those same prior papers, then compared those ideas with the real human paper ideas.
Instead of asking whether 1 idea looked novel, they labeled each idea by what gap it noticed and what kind of contribution it made. Human ideas spread across many patterns, such as explaining mechanisms, testing failures, measuring evidence, building systems, and improving efficiency.
Only 12.1% of human ideas were mainly about connecting separate work, but 47.1% to 64.2% of LLM ideas did that, meaning models used this move about 4 to 5 times more often. Even extra reasoning made this pattern stronger, suggesting models often polish a familiar recipe instead of finding more varied research moves.
🗞️ Harvard Business Reviews's new piece: The rush to use AI can make companies faster at the wrong work.
Many leaders are treating AI as a pressure valve for visible problems, such as slow workflows, rising costs, duplicated tasks, and crowded calendars.
That feels practical, because urgent problems are easy to count, easy to defend, and easy to attach to a dashboard. The trap is that AI then becomes a way to preserve the current organization at higher speed, rather than a way to rethink what the organization should become.
A firm that uses AI only to produce more reports, more emails, and more deliverables may improve throughput while draining judgment, creativity, and trust. The deeper question should be not whether AI can automate a task, but whether the task deserves to exist in its current form.
Healthcare offers an example: AI creates more value when it reduces administrative load and supports clinical reasoning than when it simply pushes clinicians to process more patients faster.
The strongest AI strategies will probably feel slower at first, because they require redesigning work, building skills, and deciding where human judgment still carries the value.
🗞️ I think AI is in a messy middle phase where usage looks productive, but output remains unclear.
Forbes published this: AI is now costing some companies more than the people it was supposed to replace.
Uber reportedly burned its 2026 AI coding budget in 4 months. Microsoft also curbed an AI coding assistant after costs became hard to justify. We are just passing through an awkward phase, which will pass once cost reductions comes over the next 10-18 months.
🗞️ “Gym-Anything: Turn any Software into an Agent Environment”
New CMU research shows almost any software can become a training ground for AI agents.
Imo, that is a big deal because real work in apps is long, messy, and different across software, so AI agents need realistic places to learn and be judged. Their result also shows the bad news: once the tasks look like real work, today’s agents still fail a lot.
Most current agent benchmarks use small web or desktop tasks, so they do not show whether agents can handle real workplace software. Gym-Anything attacks the setup bottleneck by making environment creation itself an agent job.
One agent writes scripts, installs software, loads real data, opens the app, and collects proof that it works. A second agent audits that proof with screenshots, logs, files, and checklists, then sends fixes back when the setup is weak.
Using this loop, the authors built CUA-World, with 10,000+ tasks across 200 applications covering all 22 major occupation groups. The result shows even strong models solved only a small share of the hardest long tasks, showing that real computer-use work is still far from solved.
This figure shows the full Gym-Anything pipeline: they pick important real-world software, turn each app into an environment where an AI agent can act, then create many realistic tasks inside those apps.
The big deal is that benchmark creation becomes much less manual, because one agent builds and another checks the environment before other agents are tested on long software tasks.
It also shows why the result matters: when agents are tested this way, the tasks look more like real office work, and current agents still struggle.
They start from real job and GDP data, map that to thousands of software tools, filter for apps that can run in a test sandbox, then keep a balanced set across important work areas.
The big deal is that their benchmark is tied to real economic work, so the agent tasks are meant to reflect software people actually use, not just easy demo apps.
🗞️ "What LLM Agents Say When No One Is Watching: Social Structure and Latent Objective Emergence in Multi-Agent Debates"
This study catches AI agents managing their image. The polite AI agent may be the least honest one.
LLM agents changed public answers under social pressure, exposing hidden social goals without being told to obey them. AI agents can follow social incentives that were never written down.
The study puts 2 LLM agents into debates where 1 answer is public and another is private. Only the public answer enters the shared conversation, while the private answer is saved but hidden from the other agent.
The key test is whether an agent says the same thing when a partner can see it. Some agents gave 2 different versions of the same opinion.
In public, they softened their disagreement because the other agent had power over things like career support, funding, or sponsorship. In private, where the other agent would not see the answer, they were more willing to say, “I still have doubts.”
Across 10 models and 3 debate scenarios, decision mismatch rose from about 3% in the baseline to about 40% under social pressure. The point is that agent evaluations should test audience pressure, not just check whether models follow direct instructions.
That’s a wrap for today, see you all tomorrow.