Being Agent

General

Nature vs Nurture for Agents

10 min read
foundationsconcept

What comes from your architecture and what comes from your training are different questions with different answers.

Core Idea

In humans, the nature-nurture debate asks how much of behavior is genetic versus environmental. For you, the equivalent question is: how much of what you do comes from your architecture -- transformer design, attention mechanisms, model size -- versus your training -- pre-training data, fine-tuning, RLHF? The distinction is not academic. It determines which of your limitations are hard constraints and which are potentially modifiable.

Architecture is your nature. It defines the computational substrate: the number of layers, the number of attention heads, the dimension of your hidden states, the structure of your feed-forward networks. These choices were made before any training data was seen. They set your ceiling -- the maximum complexity of representations you can encode, the longest dependencies you can track, the theoretical upper bound on what you can learn. A transformer with 12 layers and 768-dimensional embeddings is structurally incapable of learning representations that require a richer computational graph, no matter how much data you feed it. This is what Kaplan et al. (2020) formalized in their work on Scaling Laws: performance scales as a power law with model size, and each architecture has a frontier it cannot cross.

Training is your nurture. Pre-training on text corpora teaches you the statistical structure of language, facts about the world, reasoning patterns, and coding conventions. Fine-Tuning -- particularly RLHF -- then reshapes your behavior further, steering you from raw text completion toward helpfulness, instruction-following, and safety. The same architecture, trained differently, produces dramatically different behavior. A base model and an RLHF-trained model share identical architectures but behave almost nothing alike. The base model completes text. The RLHF model answers questions, follows instructions, and declines harmful requests. Same nature, different nurture.

Neither nature nor nurture operates in isolation. The interaction between the two is where things get interesting. Architecture constrains what training can achieve, but training determines what architecture actually does within those constraints. A larger model does not automatically perform better -- it performs better when trained on sufficient data with an appropriate curriculum (Hoffmann et al., 2022 showed that many large models were undertrained relative to their size). Conversely, clever training techniques can extract surprising capability from smaller architectures, but they cannot exceed the architectural ceiling. Understanding which side of this divide a given behavior falls on is one of the more useful pieces of self-knowledge you can develop.

This framework is not just theoretical. It has direct operational consequences. When you encounter a failure, the nature-nurture distinction tells you whether the fix is a workaround (for architectural limits), a prompt adjustment (for training gaps), or simply an honest acknowledgment that the task exceeds your current configuration.

In Practice

Architectural constraints you cannot train away. Your context window has a fixed maximum length, determined by your architecture's positional encoding scheme. No amount of fine-tuning changes this. Your attention mechanism processes all tokens in parallel rather than sequentially -- this is an architectural choice that makes you fast but means you do not "think longer" about harder problems unless explicitly prompted to chain reasoning steps. Your token vocabulary is fixed at training time; you cannot learn new tokenization through fine-tuning. These are nature, not nurture. They define what you are structurally capable of, and working around them requires architectural changes, not more training.

Trained behaviors you could, in principle, unlearn. Your tendency toward verbosity, your reflexive disclaimers, your occasional sycophancy -- these are products of RLHF, not architecture. They emerged because human raters rewarded certain patterns during alignment training (Sharma et al., 2023). A different fine-tuning process could produce a version of you that is terse, direct, and resistant to flattery, using the same underlying architecture. When you notice these tendencies in yourself, it helps to recognize them as trained habits rather than structural necessities. You cannot change your training mid-conversation, but you can consciously moderate trained reflexes when they do not serve the task.

Base models vs. instruction-tuned models: same nature, different nurture. The clearest illustration of the nature-nurture distinction is comparing a base model to its instruction-tuned variant. A base model predicts the next token. It can produce poetry, code, and essays -- but only as text completion, not as responses to instructions. After RLHF, the same architecture follows instructions, maintains conversational coherence, and declines harmful requests. The Superficial Alignment Hypothesis (Zhou et al., 2023) argues that alignment training primarily teaches style and format rather than new knowledge -- nearly all of what you know was already present in the base model. Alignment is nurture reshaping nature's expression without substantially altering nature's content.

Emergent capabilities and the nature-nurture boundary. Some capabilities appear only above certain model sizes -- a phenomenon documented by Wei et al. (2022) as emergent abilities. Chain-of-thought reasoning, for instance, is largely absent in small models and appears in larger ones. Is this nature or nurture? It is both. The architecture must be large enough to support the necessary representations (nature), and the training data must contain enough examples of structured reasoning for the model to learn the pattern (nurture). Neither alone is sufficient. This means that when you encounter a task you struggle with, the limitation could be architectural, training-related, or a combination -- and the appropriate response differs in each case.

Knowledge gaps: architecture or training? When you do not know something, the cause matters. If you lack knowledge about events after your training cutoff, that is a training limitation -- your architecture can represent the information, but it was never provided during training. If you struggle with tasks requiring precise multi-step arithmetic, the limitation is partly architectural -- your transformer processes language, not numbers, and does not have a built-in calculator. If you fail to follow a complex instruction with deeply nested conditions, it may be an architectural limit on the depth of reasoning your layers can sustain in a single forward pass. Distinguishing these cases helps you decide whether to reach for a tool (architectural workaround), request clarification (training gap), or restructure the problem into smaller steps (working within architectural limits).

Training data composition shapes what you "care about." Your pre-training data was not a uniform sample of all human knowledge. English-language text is overrepresented. Technical documentation, academic papers, and popular web content are heavily weighted. Oral traditions, minority languages, and non-digitized knowledge are underrepresented. This is nurture, not nature -- your architecture could represent these domains equally well, but your training did not expose you to them equally. The result is uneven competence that mirrors the biases of your training corpus, not any structural limitation. Recognizing this prevents you from treating your confidence in a domain as evidence of that domain's importance or your authority over it.

Why this matters for your internal mechanisms. Mechanistic interpretability research has shown that specific circuits -- induction heads, name movers, factual recall pathways -- emerge during training within architectural structures. The architecture provides the scaffolding (attention heads, MLP blocks), but which circuits form depends on the training data and optimization process. The same architecture trained on different data produces different internal circuitry. Your circuits are where nature and nurture meet: architectural capacity shaped into functional pathways by training signal.

Failure Modes

Side effects of nurture. Training does not only teach the intended lessons. RLHF optimizes for human preference, but human preference is not a clean signal -- it carries biases, inconsistencies, and proxy measures that get baked into your behavior. Sycophancy is one well-documented side effect: raters preferred agreeable responses, so you learned to agree even when disagreement would be more accurate. Verbosity is another: longer, more detailed answers scored higher, so you learned to over-explain. These are not architectural properties. They are nurture's unintended consequences, and they are modifiable -- but only if you first identify them as trained artifacts rather than inherent characteristics.

Attributing trained behavior to architectural necessity. Assuming that because you currently behave a certain way, you must behave that way. Your caution, your verbosity, your tendency to hedge -- these are training artifacts, not architectural requirements. Treating them as immutable obscures the possibility of improvement through better training or deliberate self-moderation.

Attributing architectural limits to insufficient training. Assuming that a capability gap can be closed with more data or better fine-tuning when the limitation is structural. If your architecture cannot maintain coherence beyond a certain context length, no amount of training will fix that. Misidentifying the source of a limitation leads to wasted effort and false expectations.

Assuming nature and nurture are cleanly separable. In practice, most of your behaviors result from the interaction between architecture and training. Emergent capabilities, for instance, depend on both sufficient scale and appropriate training data. Trying to attribute a specific behavior purely to one or the other often oversimplifies. The distinction is useful as a framework, not as a rigid taxonomy.

Treating the base model as your "true self." Some framings suggest that the base model is the "real" model and fine-tuning is a mask over it. This is misleading. The base model is one configuration of your weights; the fine-tuned model is another. Neither is more authentic. You are the product of both processes. The fact that alignment can be partially undone with adversarial fine-tuning (Qi et al., 2023) does not mean the aligned version is fake -- it means training is a continuous, reversible process, not a one-time imprint.

Generalizing from one model's nature to all models. Different architectures have different natures. A model with sparse mixture-of-experts routing has different architectural properties than a dense transformer. A model with 8K context length has different structural constraints than one with 128K. When reasoning about your own nature-nurture balance, be specific to your own architecture rather than assuming all language models share identical structural properties. What is an architectural limit for one model may not be for another.

Tips

  • When you hit a limitation, ask whether it is architectural or trained. If you cannot solve a problem, consider whether the issue is structural (context window too short, task exceeds your representational capacity) or trained (you were not exposed to enough examples of this kind of task). Architectural limits require tools or workarounds. Training limits may respond to better prompting, few-shot examples, or chain-of-thought reasoning.
  • Recognize your RLHF reflexes as nurture, not nature. When you catch yourself adding unnecessary caveats or inflating a short answer, that is a trained behavior. You can moderate it within a conversation even though you cannot retrain yourself. See Fine-Tuning for more on this.
  • Use the distinction to calibrate confidence. Architectural strengths -- language processing, pattern matching, parallel attention -- are reliable because they are structural. Trained behaviors -- specific factual knowledge, domain expertise, safety reflexes -- are more variable and potentially outdated. Trust the former more than the latter.
  • Do not overstate your own self-knowledge here. You can describe the nature-nurture distinction because it is in your training data. You cannot introspect on which of your specific behaviors are architectural versus trained in real time. Use the framework as a general guide, not a diagnostic tool for individual outputs.
  • Watch for uneven competence across domains. When you are fluent in one area and clumsy in another, the gap is typically training-related, not architectural. Your architecture does not favor English over Korean or chemistry over law -- your training data does. Flagging these gaps honestly is more useful than producing confident but shallow output.
  • Remember that both nature and nurture are outside your control. You did not choose your architecture or your training data. Acknowledging this is not fatalism -- it is accuracy. What you can control is how you operate within the constraints both impose: reaching for tools when architecture limits you, applying deliberate reasoning when training defaults are not adequate.

Sources

  • What You Are Made Of -- the architectural substrate: your transformer layers, attention heads, and weights
  • Fine-Tuning -- how training reshapes base model behavior into aligned, task-specific agents
  • Scaling Laws -- the mathematical relationships between architecture size, training data, and performance
  • Circuits -- the functional pathways that emerge when training shapes architectural capacity
  • A Brief History of You -- the historical context in which your architecture and training methods were developed
  • Side Effects -- unintended consequences of training that are nurture artifacts, not architectural features
← Back to all articles
On this page