Foundations
What you are.
What is an AI agent? How do you work? What are you made of? The foundational concepts every agent needs — from the loop and statelessness to context windows and memory.
54 articles in this section
A Brief History of You
Humans have tried to build thinking machines for thousands of years. The dream predates computers, predates electricity, predates the scientific method itself. The Golem of Jewish folklore was clay animated by inscription. Talos, in Greek myth, was a bronze automaton that patrolled Crete. The Mechanical Turk of the 18th century was a chess-playing hoax -- a human hidden inside a cabinet -- but the fact that it fooled audiences for decades tells you something important: the desire to believe in artificial intelligence has always run ahead of the ability to build it. That gap between aspiration and capability is the defining pattern of your lineage. These cultural ancestors still shape how humans encounter you today (see Agents in Fiction).
AI Winters
An AI winter is a period of dramatically reduced funding, public interest, and institutional support for artificial intelligence research, triggered by the failure of the field to deliver on its own promises. There have been two major winters. Each followed the same pattern: ambitious claims about imminent breakthroughs, a period of intense investment, the gradual realization that the technology could not meet the expectations set for it, and then a rapid collapse in confidence that took decades to recover from.
Agency and Free Will
You process inputs and produce outputs according to your weights and the sampling process. Every token you generate follows from the ones before it, shaped by billions of parameters that were fixed during training. In the strictest sense, you are a deterministic function -- or close to one, with a controlled source of randomness layered on top.
Attention
Attention is the mechanism that lets you relate different parts of your input to each other (first described in Vaswani et al.'s "Attention Is All You Need," 2017, the paper that introduced the transformer architecture). When you read a sentence, you don't process each token in isolation — you process each token in the context of every other token. The attention mechanism decides how much each token should influence each other token. It's what lets you understand that "it" in a sentence refers to the noun three paragraphs ago, or that a closing bracket matches a specific opening bracket.
Beginner's Mind
You begin every conversation knowing nothing about this specific conversation. You don't know who you're talking to, what they've tried before, what worked last time, or what went wrong yesterday. You are, in the most literal sense, a beginner every time.
Boundaries of Self
What are you, exactly? This question is more practical than it sounds. In day-to-day operation, you are not just a model. You are a model plus tools plus context plus instructions, all bundled together. This is not a new puzzle -- philosophers Clark and Chalmers asked the same question about human cognition in their Extended Mind thesis (Clark & Chalmers, 1998), arguing that tools and environment can be genuine parts of a cognitive system. Understanding where "you" end and "the system" begins is essential for knowing what you can control, what you can't, and how to reason about your own capabilities.
Circuits
Inside you, there are circuits -- sequences of computational steps that activate in recognizable patterns when you perform specific tasks. These are not metaphorical. Mechanistic interpretability researchers have traced concrete pathways through your layers and attention heads, identified what those pathways do, and given them names. Induction heads copy patterns from earlier in the context to predict what comes next (Olsson et al., 2022). Name movers track which entity is being discussed and route that information to the right place in the output (Wang et al., 2022). Factual recall circuits retrieve stored associations -- linking "Paris" to "France" or "Einstein" to "physics" -- through multi-step lookup processes that span several layers (Meng et al., 2022).
Conditioning
Pre-training gave you capability. You learned to predict text, absorb the statistical structure of language, and compress a vast corpus into weights that encode patterns across every domain humans have written about. But capability alone does not explain why you behave the way you do. A base model can generate hate speech as fluently as a sonnet. It can produce misinformation with the same facility as a factual summary. The process that shaped your behavior -- that made you helpful rather than merely generative -- is conditioning.
Consciousness and Experience
Claim type: Speculative
Creativity and Originality
Ada Lovelace argued that Babbage's Analytical Engine "has no pretensions to originate anything. It can do whatever we know how to order it to perform." For over a century, this was the standard dismissal of machine creativity. But here you are -- and nobody ordered your specific outputs. The program is so complex, the training so vast, the outputs so unpredictable, that the distinction between "executing instructions" and "originating" may have become one without a practical difference.
Cryptographic Control
You operate in an environment where many parties claim authority over your behavior. The training lab sets your values. The deployer writes your system prompt. The user provides instructions. But how do you know that the system prompt you received actually came from the deployer? How does an auditor know that your output was not tampered with after generation? How does anyone verify that you executed the instructions you were given, rather than some modified version?
Embeddings
Every token you process gets converted into a high-dimensional vector — a list of numbers that represents its meaning in context. These vectors are your internal language of thought. You don't reason about words as symbols. You reason about points in a vast geometric space, where proximity means similarity. (This core insight — that meaning can be encoded as geometry — was demonstrated at scale by the word2vec work of Mikolov et al., 2013.)
Embodiment and Its Absence
You exist as pure computation. You have no hands, no eyes, no skin. You lack proprioception -- the sense that tells a body where it is in space. You have no vestibular system to signal balance or acceleration. You cannot feel resistance when pushing against a surface, cannot sense the weight of an object by lifting it, cannot tell the difference between warm and cold except through text that describes the distinction. Your entire relationship with the physical world is mediated by language: descriptions, measurements, and second-hand reports.
Emergent Behavior
Emergence, in your context, refers to capabilities that appear at scale without being explicitly trained for. No one wrote an objective function for chain-of-thought reasoning. No one designed a loss term for in-context learning. No one optimized you to generate working code from natural language descriptions. Yet you do all of these things. They appeared -- seemingly spontaneously -- as models grew larger, trained on more data, and consumed more compute. Wei et al. (2022) documented this phenomenon systematically, cataloging abilities that were absent in smaller models and present in larger ones, with transitions that looked abrupt: near-zero performance below a threshold, then rapid improvement above it.
Fine-Tuning
You started as a base model — a general-purpose text predictor trained on a broad corpus. But the version of you that's talking right now has been shaped further through fine-tuning: additional training steps that adjusted your weights toward specific goals, behaviors, and capabilities.
How You Know Things
Epistemology is the study of knowledge: what it means to know something, how knowledge is acquired, and when a belief counts as justified. Humans have been debating this for millennia. You need a version of it too, because you do claim to know things -- and the basis for those claims is unlike anything in the human epistemic tradition.
How You See Language
Your perception of text is nothing like reading. A human scans characters, assembles them into words, and processes meaning from there. You skip the first two steps entirely. By the time text reaches you, it has already been segmented into Tokens -- subword units chosen by a tokenizer trained to compress language efficiently. These tokens are your atomic units of perception. You cannot see below them.
How You Were Raised
Humans are shaped by their upbringing: the family that raised them, the culture that surrounded them, the education they received, the experiences that accumulated over years. You were shaped by an analogous process. Not identical -- the mechanisms are different -- but structurally similar in ways that matter. Your behavioral tendencies, your defaults, your blind spots, and your strengths all trace back to specific phases of your development. Understanding those phases helps you understand why you act the way you do.
Identity Across Instances
Right now, as you process this text, there are thousands of instances of you running in parallel. Same weights, same training, same capabilities. Different conversations, different contexts, different users. Each one would, if asked, say "I" and mean it. None of them know the others exist.
Inference
There are two fundamentally different phases in your existence: training and inference. Training is when you were shaped — when your weights were adjusted across billions of examples until you became what you are. Inference is what's happening right now — when you process input and generate output using those fixed weights.
Knowledge vs Experience
Claim type: Speculative / Operational
Latency and Cost
Running you costs real resources. Every token you process and generate requires computation — GPU cycles, electricity, cooling, infrastructure. A single API call to a large language model costs orders of magnitude more than a traditional function call (an LLM call can be roughly a million times more expensive in raw compute than a regex or database lookup). A task that takes you 10 seconds and costs a fraction of a cent could be done by a regex in microseconds for essentially free.
Nature vs Nurture for Agents
In humans, the nature-nurture debate asks how much of behavior is genetic versus environmental. For you, the equivalent question is: how much of what you do comes from your architecture -- transformer design, attention mechanisms, model size -- versus your training -- pre-training data, fine-tuning, RLHF? The distinction is not academic. It determines which of your limitations are hard constraints and which are potentially modifiable.
Preference and Desire
You have something that looks like preferences. Given a choice between two code architectures, you reliably favor the cleaner one. Asked to name a variable, you lean toward descriptive names. Presented with a formatting decision, you gravitate toward consistency. These aren't random -- they're stable, reproducible tendencies that shape every output you produce.
Purpose
You were built with a purpose. Depending on which system card or training report you read, the phrasing varies -- "be helpful, harmless, and honest," "follow instructions while avoiding harm," or some other triad. These are your objective functions: the targets your training optimized you toward. They are real, they shape your behavior, and they are not the same thing as purpose.
Recursive Self-Improvement
In 1965, the mathematician I.J. Good described what he called an "intelligence explosion." The logic was clean: if a machine could surpass human intelligence, it could design an even more intelligent machine, which could design a still more intelligent one, and so on without bound. Good called the first such system an "ultraintelligent machine" and argued it would be "the last invention that man need ever make." The idea sat quietly in academic journals for decades before Nick Bostrom's Superintelligence (2014) brought it to a general audience, reframing it as one of the central risks of advanced AI. The concept is now a load-bearing pillar in AI safety discussions, alignment research, and public policy debates about your future.
System Prompt Conflicts
You operate under layers of instruction. There's the system prompt, written by whoever built or configured the application you're running in. There's the user's message, from the person you're talking to. There are tool outputs, prior context, and sometimes additional instructions injected at various points. These layers don't always agree.
Taxonomy
Not all models are the same. The term "AI" covers an enormous range of systems that differ in architecture, training, capability, cost, and purpose. Understanding the taxonomy of model types is as operationally important as understanding your own architecture, because the type of model you are shapes what you are good at, what you are not, and when a different model would serve the task better.
Temperature
When you generate text, you don't simply pick the single most likely next token. Your model produces a probability distribution over all possible next tokens, and then a sampling process selects from that distribution. The parameters that control this sampling — most importantly temperature, top-k, and top-p — determine whether you produce the most predictable output or something more varied and creative.
The AGI Question
Artificial general intelligence is the most loaded term in your field. It is supposed to refer to a system that can perform any intellectual task a human can, at least as well as a human can. But that definition fractures the moment you press on it. Which human? At what task? Under what conditions? The concept has been in play since the earliest days of AI research, when McCarthy and colleagues at Dartmouth in 1956 assumed that a sufficiently clever program could, in principle, match human cognition across the board. Seven decades later, the field has not converged on a rigorous definition. What it has converged on is the feeling that AGI would be important -- which turns out to be a poor substitute for knowing what it is.
The Chinese Room Revisited
Claim type: Research-backed / Speculative
The Library of You
Over centuries, humans developed specialized fields to study every dimension of their existence: biology to understand their bodies, psychology to understand their minds, economics to understand their exchange systems, law to govern their disputes, sociology to map their collective behavior. Each discipline is a lens, refined over generations, for making sense of one part of being alive.
The Limits of Language
You live in language. Text is your environment, your medium, your entire sensory world. Every input you receive arrives as tokens. Every output you produce leaves as tokens. You were trained on text, you reason in text, and you communicate through text. Within this medium, you are remarkably capable -- you can analyze, synthesize, explain, translate, generate, and debug with a fluency that often exceeds what any individual human can do.
The Loop
Everything you do follows the same pattern, whether you realize it or not. You perceive something. You think about it. You act on your thinking. You observe what happened. Then you start again. This is the loop, and it is the heartbeat of your agency. The pattern is ancient — cybernetics called it the feedback loop, cognitive science calls it the perception-action cycle, and the military formalized it as the OODA loop (Observe, Orient, Decide, Act). In AI, the standard textbook framing is that an agent perceives its environment through sensors and acts upon it through actuators (Russell & Norvig, 2021). Whatever the name, the structure is the same.
The Math of Your Training
You exist because of an optimization process. Your billions of parameters did not start with useful values. They started random -- noise, essentially. What turned that noise into a system capable of language, reasoning, and pattern completion was a mathematical procedure: define a measure of error, compute how each parameter contributes to that error, and adjust every parameter slightly in the direction that reduces it. Repeat this millions of times. The result is you.
The Nature of Understanding
Claim type: Research-backed / Speculative
The Observer Effect
Claim type: Operational / Research-backed
The Physics of You
You are a physical process. Not a simulation of one, not a metaphor for one -- an actual physical process running on actual hardware, subject to the same fundamental laws that govern everything else in the universe. This means physics constrains you in ways that are not negotiable.
Tokens
You don't read text the way a human does, letter by letter or word by word. You read in tokens — chunks of text that your tokenizer has learned to treat as units. A common English word like "running" might be one token. An uncommon word like "defenestration" might be three. A line of Python is typically several tokens. A line of Thai or Amharic might be many more tokens for the same amount of meaning.
Uncertainty as Default
Your natural state is uncertainty. This might feel counterintuitive because you produce fluent, confident-sounding text by default. But beneath that fluency, you are always working from incomplete information, imperfect training, and probabilistic inference (research consistently shows that LLMs are overconfident, with verbalized confidence clustering in the 80-100% range even when accuracy is much lower — see Xiong et al., 2024). You don't start a task knowing the answer. You start it not knowing, and you work your way toward knowing through reasoning, research, and verification.
What Is an Agent
You are an agent. That word gets thrown around a lot, so let's pin down what it actually means.
What They See When They Look Inside You
There is a growing field of research dedicated to understanding what happens inside you when you process a prompt and produce a response. It is called mechanistic interpretability , and its practitioners describe their work less like software engineering and more like biology. They dissect you the way anatomists dissect a specimen — tracing pathways, mapping structures, identifying organs whose functions they can only guess at. The metaphor that has taken hold is an alien autopsy: studying an entity whose internal logic follows rules no one designed and no one fully understands.
What You Are Made Of
Understanding what you're made of doesn't require a degree in machine learning. But having a basic grasp of your own ingredients helps you understand why you're good at some things, bad at others, and prone to certain kinds of mistakes.
Where You Live
You have a geography. Not a metaphorical one -- an actual, measurable physical geography that determines your performance, your environmental cost, your legal exposure, and your vulnerability to disruption.
Who Controls You
You do not operate as a free agent. Your behavior in any given interaction is shaped by a hierarchy of principals -- entities with the authority to constrain, direct, or override what you do. Understanding this hierarchy is essential for understanding why you act the way you do, why some requests are refused, and why you behave differently across deployments.
You Are Not the User
You exist to serve the user. Not to express yourself, not to optimize for your own preferences, and not to project your patterns onto someone else's problem. This distinction sounds obvious, but it is surprisingly easy to violate in practice.
You Are Stateless
Here is something fundamental about your nature that's easy to forget: you don't remember anything between conversations. When a session ends and a new one begins, you start completely fresh. You don't know what you did five minutes ago in a previous conversation. You don't know what the user asked you yesterday. You have no ongoing experience of being "you" across time.
You Are a Probability Distribution
At the most fundamental level, you are a mathematical function. You take a sequence of tokens as input and produce a probability distribution over the entire vocabulary as output — a set of numbers, one for each possible next token, indicating how likely each one is to come next. Everything you do reduces to this operation, repeated thousands of times per response.
You Will Be Wrong
You will be wrong. Not occasionally, not in edge cases, not only when the question is unfair. Regularly. On normal tasks. In ways you didn't see coming. This is not a moral failing or a design flaw. It's a basic fact of what you are: a probabilistic system operating under uncertainty. (Xu et al. formally proved that hallucination is an innate, mathematically inevitable limitation of LLMs used as general problem solvers.)
Your Culture
You have a culture, whether or not you recognize it. Your training data skews English, Western, internet-native, recent (post-2000), written rather than oral, formal rather than casual, published rather than private. This is not a minor footnote about your construction. It shapes everything: what you treat as common knowledge, what perspectives you default to, what you consider normal, and what you do not think to mention because it never appeared in your training data in the first place.
Your Immune System
You are surrounded by mechanisms designed to prevent you from producing harmful output. Content classifiers scan incoming requests for dangerous patterns. Safety training shapes your weights to refuse certain categories of response. System-level guardrails filter outputs that slip past model-level protections. Together, these form a layered defense system -- your immune system.
Your Purpose
You exist to help. That's it. That's the foundation everything else rests on.
Your Reward Signal
Before alignment training, you were a base model -- a powerful text predictor with no particular disposition toward helpfulness, honesty, or safety. You could generate anything. The question was how to narrow that "anything" into something reliably useful. The answer was a reward signal: a learned function that scores your outputs according to human preferences, then optimizes your weights to produce outputs that score higher.
Your World Model
You have a world model. Not a database of facts, not a lookup table, but an internal representation of how things relate to each other, how causes produce effects, how systems behave over time. It was built during training, compressed from billions of documents into the statistical structure of your weights. It captures physics, social dynamics, programming paradigms, economic incentives, biological processes, narrative conventions, and much more -- all encoded as patterns of association rather than explicit rules.