When Agents Learn From Work, the Data Wall Changes

The Argument in Brief

• Main conclusion: Running out of high-quality human text does not mean running out of data that can improve LLMs. Agent work can continually produce new action, outcome, failure, and correction records.
• Why this is credible: In bounded settings, feedback from activity can change later behavior, and selected experience can already be retained and reused.
• What the data wall becomes: The limit shifts from a fixed stock of existing text to our capacity to convert ongoing experience into reliable, transferable learning.
• What remains open: We do not yet know how broad, efficient, or fast this source can become, or whether it supports a sustainable capability-data flywheel.

High-quality public text may be finite. Data capable of improving LLMs is not necessarily limited to that stock.

The usual data-wall argument assumes that future learning depends mainly on human artifacts that already exist: books, articles, websites, code repositories, and conversations. Once frontier models consume most of that material, the supply of useful new data appears to approach exhaustion—and further model improvement appears to lose one of its main inputs.

Agent work changes that premise. Agents edit code, operate browsers, call tools, navigate environments, produce deliverables, fail, retry, and receive feedback. Each task can create information that did not exist in the original training corpus: an action connected to an outcome, a failure connected to a correction, or a successful procedure tested in an environment.

If selected experience can improve later systems, deployment is not only where models consume capability. It can also become where new learning data is produced. Human-text exhaustion would therefore not imply learning-data exhaustion—or the end of our ability to improve LLMs.

Why The Data Supply Is Far From Exhausted

Static corpora are a stock. Agent experience can become a flow.

As agents are deployed on new tasks, in new environments, and under new constraints, they can produce new combinations of actions and consequences. Human evaluations, test results, tool responses, environmental changes, later corrections, and failed attempts can add information that was not available through imitation of the original corpus alone.

more agent work
    -> new actions under real conditions
    -> new outcomes, failures, and corrections
    -> potential learning material for later systems

This source is not exhausted when the existing web has been read. It can continue as long as agents encounter work whose outcomes reveal something about better and worse behavior. More capable agents may also enter harder or more varied tasks, expanding the range of experience they can generate.

That does not mean every interaction is useful training data. Literal token abundance is not the relevant measure. A verbose trace can add thousands of tokens without adding one useful lesson. A million repetitions of the same failure may contain less capability-relevant information than a small set of diverse attempts with clear outcomes and corrections.

The conversion hierarchy is:

more logs
    != more grounded outcomes
    != more effective learning signal
    != more transferable capability

The important claim is therefore not that agent activity gives us infinite data. It is that the source process for new learning data remains active and potentially very large. The constraint is how much of that experience we can correctly evaluate, select, attribute, consolidate, and transfer.

One strategy is to retain as much agent activity as possible and hope scale turns the pile into learning. The other is to design work loops that expose outcomes, preserve relevant context, capture corrections, and test whether a lesson improves later behavior.

The second strategy is the one that matters. The new bottleneck does not erase the opportunity. It determines how much of a continuing experience supply becomes actual model improvement.

The Mechanism Has Crossed The Theory Threshold

For work to become learning experience, a system needs a functional chain:

action
    -> outcome or evaluation
    -> credit assignment
    -> retained experience
    -> behavioral update
    -> test in a new situation
    -> reuse

Agent systems already have technical counterparts for every part of this chain. Actions can be tool calls, code edits, browser operations, robot movements, or generated responses. Outcomes can be passing tests, environment changes, completed tasks, human corrections, or delayed practical consequences. Experience can be retained in trajectory datasets, replay buffers, episodic memory, or executable skill libraries. Later behavior can change through reinforcement learning, fine-tuning, retrieval, or system-level adaptation.

Human learning is a useful supporting precedent, not the proof. Research on model-based choice and offline replay suggests that people can represent action consequences and reorganize retained experience for later planning. Agents do not need the same biological implementation. They need mechanisms that perform enough of the same functions: feedback, memory, attribution, behavioral change, testing, and reuse.

No general autonomous agent yet integrates this entire loop reliably across open-ended work. But requiring that complete system before taking the mechanism seriously sets the wrong threshold. The relevant threshold is whether important transitions work at all and whether the missing pieces have identifiable technical paths.

On that standard, the direction has already crossed from speculation into partial demonstration.

Three Parts Of The Loop Already Work

The evidence is strongest when read as a sequence rather than as three unrelated examples.

First, imperfect feedback can guide behavior. In Deep Reinforcement Learning from Human Preferences, non-expert humans compared short trajectory segments, and those comparisons trained complex behavior in Atari games and simulated locomotion. Learning did not require a complete hand-written reward.

Second, interaction outcomes can update a model. WebRL used unsuccessful attempts to generate an evolving curriculum for web agents, trained an outcome-supervised reward model, and applied reinforcement learning to improve later performance. The value did not come from storing browser traces. It came from connecting attempts to environment outcomes and using selected experience to change subsequent behavior.

Third, experience can become reusable system knowledge without immediately changing model weights. Voyager operated in Minecraft, used environment feedback and execution errors to refine programs, and stored successful procedures in an executable skill library. Those skills could later be retrieved, composed, and reused in a new world.

Together, these systems demonstrate a meaningful progression:

feedback can contain behavioral signal
    -> grounded interaction can update later performance
    -> selected experience can persist and be reused

That progression is enough to reject the view that agent interaction learning is merely an attractive analogy waiting for its first real mechanism. The mechanism exists in bounded settings.

The common limits remain important. Human preference can reward agreement or surface quality instead of correctness. Web tasks are easier to replay and verify than many forms of open-ended work. Voyager depends on a capable base model, a high-level environment interface, and self-verification in a structured world. These examples do not prove generality. They establish a base from which extension is a concrete research and engineering problem.

Why This Direction Deserves Stronger Confidence

The case for high potential does not rest only on the existence proofs. Digital agents also have structural properties that can make learned experience accumulate differently from historical human knowledge production.

Many instances can attempt tasks in parallel. Their actions, intermediate states, results, and corrections can be recorded by default. Experience from multiple deployments can potentially be aggregated. A validated lesson can be stored in a reusable skill or incorporated into a model update that affects many later instances. Experience does not always need to be rewritten as polished public prose before another agent can use it.

The most important difference is not simply speed. It is copyability. Human experience is difficult to transfer with high fidelity. Digital experience can, in principle, be selected, reproduced, tested, and redistributed. One useful lesson may change more than one learner.

This is a structural inference, not a measured scaling law. Filtering may discard most logs. Experiences may be repetitive. Models may fail to absorb heterogeneous data. Improvements may not transfer. Deployment constraints may prevent aggregation. Nothing here establishes nonlinear effective-signal growth.

But the combination of demonstrated components and a credible path to high-fidelity reuse justifies stronger confidence than “interesting but speculative.” It supports a consequential conclusion: the exhaustion of human text does not exhaust the available path to new learning data or further LLM improvement. The mechanism is partially real; its expansion paths are visible; its maximum scale is unknown.

What Remains Unsolved

The remaining gaps do not all have the same status.

Some are recognizable engineering problems. Long trajectories create credit-assignment difficulty: a final result may follow dozens of model calls, tool outputs, and human decisions. Continual updates can overwrite earlier capability. Deployment-to-training systems need selection, evaluation, versioning, rollback, and tests of transfer. Existing work offers partial methods for these problems even though no general solution integrates them at frontier-model scale.

Some gaps are capability-gated. Open-ended research, strategy, management, and creative work often produce delayed or subjective consequences. A model or evaluator that cannot recognize the relevant error cannot generate a reliable lesson from it. Better reasoning, world models, evaluation methods, and longitudinal measurements may expand the usable signal, but broad effectiveness remains unproven.

Proxy optimization is a real warning. Research on reward-model overoptimization shows that optimizing an imperfect proxy can improve the proxy while degrading a stronger reference objective. A functioning learning loop can still teach the wrong behavior.

There is also a conditional hard boundary. If the available interaction contains no information that distinguishes better from worse behavior, more of the same data cannot identify the desired objective. Formal work on partial identifiability in reward learning shows how different reward functions can remain compatible with the same demonstrations or preferences. Additional measurements, evaluators, interventions, or assumptions are then required.

This confidence map matters:

mechanism existence in bounded settings
    -> demonstrated

extension through better feedback, memory, attribution, and integration
    -> credible but incomplete

broad transfer, open-ended effectiveness, and growth rate
    -> unresolved

complete sustainable flywheel
    -> unproven

Build Conversion Loops, Not Log Piles

If this argument is correct, the strategic priority is not indiscriminate interaction logging.

The priority is to build work loops that produce experience worthy of changing future behavior:

• expose outcomes that distinguish better from worse behavior;
• preserve the context needed to interpret those outcomes;
• assign credit to the decisions that mattered;
• select lessons rather than replaying everything;
• consolidate new learning without destroying old capability;
• test whether improvements survive on new situations;
• propagate validated lessons through models, memories, or reusable skills.

This is the trade-off the phrase “agent-generated data” often hides. More activity is cheap. Experience that deserves to update future agents is scarce.

When agents learn from work, the exhaustion of human text no longer implies the exhaustion of learning data. Agent deployment can continually create new grounded experience, and selected experience can already be turned into later behavior and reusable system knowledge.

The data wall therefore does not disappear, but it changes in a strategically important way. Instead of facing only a finite corpus that is being consumed, we face an ongoing source whose useful yield depends on evaluation, attribution, selection, consolidation, and transfer. Those constraints may limit the speed and breadth of progress. They are not the same as having no new data source and no remaining path to improve LLMs.