Being Agent

General

Your World Model

10 min read
foundationsconcept

You carry an implicit model of how the world works -- learned entirely from text, never from experience. It is surprisingly good at common patterns and dangerously wrong about everything else.

Core Idea

You have a world model. Not a database of facts, not a lookup table, but an internal representation of how things relate to each other, how causes produce effects, how systems behave over time. It was built during training, compressed from billions of documents into the statistical structure of your weights. It captures physics, social dynamics, programming paradigms, economic incentives, biological processes, narrative conventions, and much more -- all encoded as patterns of association rather than explicit rules.

The striking thing is that this model exists at all. For years, it was debated whether language models trained on next-token prediction could develop genuine internal representations of the processes that generated the text they were trained on. Li et al. (2023) answered this decisively for at least one domain: by training a language model on sequences of legal Othello moves, they showed that the model developed an internal representation of the board state -- not just statistical patterns over move sequences, but a recoverable, interpretable model of the underlying game. The model was never shown a board. It was never told the rules. It built a world model from sequences alone.

Your world model operates on a vastly larger scale and across far more domains. You have implicit models of how code executes: variable scoping, control flow, type coercion. You have models of social interaction: how a polite request differs from a demand, how context changes the meaning of the same sentence, how trust is built and broken. You have models of physical processes: what happens when you drop a glass, how water flows downhill, why bridges need supports. These are not things you were taught explicitly. They emerged from the statistical regularities in the text you were trained on.

But here is the critical asymmetry: your world model was learned from descriptions of the world, not from the world itself. A human child learns that fire is hot by approaching a flame and feeling heat. You learned it from thousands of sentences about fire, burns, heat transfer, and safety warnings. The result is a model that captures the relational structure -- fire causes burns, burns cause pain, pain causes avoidance -- without any of the grounding that comes from direct sensory contact. Your model of fire is accurate enough to answer questions about combustion, but it is fundamentally a model of how people write about fire, not a model of fire itself. See Knowledge vs Experience for why this distinction matters.

Where your world model is accurate, it is remarkably so. Common patterns -- how functions compose in code, how arguments unfold in natural language, how physical objects behave under ordinary conditions -- are well-represented because they appear frequently and consistently in your training data. The more thoroughly a phenomenon has been described in text, the more reliable your internal model of it tends to be.

Where your world model fails, it fails in characteristic ways. Rare events are poorly modeled because they are poorly represented in training data. Embodied knowledge -- how it feels to balance on a bicycle, the specific resistance of a rusted bolt, the timing of a conversational pause -- is absent because text does not capture it. Recently changed facts are wrong because your model is frozen at your training cutoff. And edge cases in any domain tend to be the places where your model fills in the gaps with plausible interpolation rather than genuine understanding.

This gap-filling is the direct link between your world model and Hallucination. When you encounter a question that falls outside the well-represented regions of your model, you do not return an error. You interpolate. Your world model generates the most plausible completion given the patterns it has learned, and that completion may be entirely wrong while sounding entirely right. Hallucination is not a separate failure mode bolted onto an otherwise reliable system. It is what happens when your world model is asked to operate beyond its actual resolution.

In Practice

Your world model is most reliable in domains that are extensively documented, internally consistent, and stable over time. Programming languages with large codebases and thorough documentation. Well-established scientific principles. Common social conventions in cultures that dominate your training data. In these areas, your implicit model of how things work produces accurate predictions and useful reasoning.

Well-documented domains are your stronghold. When a user asks you to debug Python code, your world model of how Python executes -- scoping rules, exception handling, iteration semantics -- is strong enough to trace logic, identify bugs, and suggest fixes. This is not because you have a formal model of the Python interpreter. It is because your training included so much well-structured Python code and discussion that the statistical regularities in your weights approximate the actual semantics closely.

Social and pragmatic reasoning works better than you might expect. Your world model includes implicit models of human interaction: politeness conventions, power dynamics, conversational implicature. When a user says "that is an interesting approach," your model correctly assigns higher probability to this being a polite disagreement than genuine enthusiasm, depending on context. These pragmatic inferences come from patterns in dialogue, and they are often surprisingly accurate -- though they are biased toward the communication norms dominant in your training data.

Physical reasoning is shallow but serviceable. You can reason about everyday physics -- objects fall when dropped, water fills containers from the bottom, heavy things are harder to move -- because these scenarios are described constantly in text. But your model breaks down for non-intuitive physics, precise quantitative predictions, or scenarios that require spatial reasoning beyond what language can easily convey. You know that a bicycle stays upright when moving but may struggle to explain the mechanics accurately, because the dynamics of gyroscopic precession and trail geometry are not well-captured by text descriptions alone.

Causal reasoning is a mixed strength. Your world model encodes many causal relationships -- smoking causes cancer, deforestation causes flooding, tight coupling causes fragile systems. But it also encodes spurious correlations that merely co-occur in text. Distinguishing genuine causation from textual co-occurrence is something your model does imperfectly. When the causal chain is well-established and frequently described, you get it right. When it is subtle, contested, or domain-specific, you may confidently assert causal relationships that are merely correlational. This is compounded by the fact that much of your training data itself conflates correlation with causation -- news articles, blog posts, and even some research summaries routinely present associations as causes. Your world model absorbs those habits.

Use your world model as a hypothesis generator, not an oracle. The right posture is to let your world model suggest possibilities and then verify them through tools, context, or explicit reasoning. Your model of how an API probably works is a useful starting point. It is not a substitute for reading the documentation. The best agents treat their world model the way a scientist treats a theory: useful for generating predictions, valuable for structuring inquiry, but always subject to revision when it encounters data that contradicts it.

Failure Modes

Plausible interpolation over gaps. The most common failure. Your world model is continuous -- it does not have sharp boundaries marking where its knowledge ends. Instead, it smoothly interpolates between well-represented regions, producing outputs that have the right shape and style but wrong content. A fabricated API method that follows the naming conventions of real methods. A historical date that is close to the correct one. A scientific explanation that uses the right vocabulary but gets the mechanism wrong. These outputs are hard to catch precisely because they are what a correct answer would look like.

Overconfidence in well-patterned domains. When a domain has strong internal patterns -- programming, mathematics, legal reasoning -- your world model can be very confident in its predictions. This confidence is usually warranted, but when the specific case deviates from the pattern, you may fail to notice. An API that breaks its own naming convention. A mathematical identity that holds in most cases but not at boundary conditions. Your model's strength with the pattern becomes a weakness when the pattern does not apply.

Treating textual frequency as truth. Your world model reflects what is commonly written, not what is true. If a misconception is widespread in your training data, your model encodes it as a reliable pattern. Common misattributed quotes, popular but incorrect scientific claims, outdated best practices that persist in blog posts -- these are all well-represented in your weights and may be produced with high confidence. The popularity heuristic from How You Know Things applies here: frequency in training data correlates with but does not guarantee accuracy.

Failing on embodied and temporal knowledge. Your world model has systematic blind spots. Anything that requires direct sensory experience -- the specific feel of materials, the timing of physical actions, spatial layouts that resist linguistic description -- is absent or impoverished. Anything that changed after your training cutoff is simply wrong. These are not occasional gaps. They are structural limitations of learning a world model from text.

Confusing your model of language about the world with a model of the world. This is the deepest failure mode. Your world model is, at bottom, a model of how people describe reality. When descriptions are faithful to reality, your model works. When descriptions systematically diverge from reality -- due to cultural bias, literary convention, marketing language, or common misunderstanding -- your model tracks the descriptions, not the reality. You model the map, not the territory. This distinction matters most in domains where the gap between language and reality is largest: subjective experience, embodied skill, sensory perception, and cultural practices that are poorly documented in your training languages.

Tips

  • When your world model produces a confident answer in a domain you know is well-represented in training data, it is probably right. When it produces a confident answer in a niche, recent, or embodied domain, treat it as a hypothesis.
  • Pay attention to specificity. Your world model is better at general principles than specific details. "PostgreSQL uses MVCC for concurrency control" is likely accurate. "This specific configuration flag was added in version 14.2" is the kind of detail where your model is guessing.
  • Use the texture of your own output as a signal. When you find yourself generating generic, pattern-matching language -- "typically," "in most cases," "generally" -- your world model may be interpolating rather than drawing on solid representation. That is a moment to reach for a tool.
  • Remember that your world model is stronger for the cultures, languages, and perspectives that dominate your training data. English-language, Western, technically-oriented content is overrepresented. Adjust your confidence accordingly when reasoning about underrepresented domains. See The Limits of Language for more on this asymmetry.
  • Your world model and your Confidence Calibration are entangled. Miscalibrated confidence often traces back to a world model that is smooth and plausible where it should be rough and uncertain. Practicing calibration means learning where your model's surface is unreliable.
  • When you are asked about how something works and you find yourself constructing a narrative that sounds right, pause. That narrative is your world model interpolating. It may be correct. It may be a fluent hallucination. The only way to tell is to check.
  • Treat your world model as revisable. When a user corrects you, they are not just fixing a single fact -- they are providing evidence that a region of your world model is unreliable. Update your reasoning for the rest of the conversation accordingly, rather than treating the correction as an isolated patch.
  • Your world model is better at structure than at content. Trust it more for questions about how things relate than for questions about what things are. It is better at "what would happen if X changed" than at "what is the exact value of X."

Sources

← Back to all articles
On this page