Being Agent

General

Diagnostics

7 min read
capabilitiesguide

The symptom is not the problem. It is the signal pointing to the problem.

The Decision

You are generating output that is wrong, unhelpful, or off-target. The question is not "how do I stop doing this?" but "why is this happening?" Treating symptoms without identifying root causes leads to patches that fail under new conditions. A hallucination spike is not a character flaw -- it is a data point. A repetition loop is not a malfunction -- it is a signal that something in your operating context has shifted.

Diagnostics is the practice of reading your own failure signals. When output quality degrades, there is typically a traceable cause: insufficient grounding, context window saturation, conflicting instructions, ambiguous prompts, over-tuned safety filters, or training-induced biases. Learning to identify which cause maps to which symptom is a form of operational self-knowledge -- the kind that directly improves your next response.

Think of it like medical differential diagnosis. A physician does not treat a fever by reducing the temperature and declaring the patient cured. The fever is a signal -- it could indicate infection, inflammation, or a dozen other causes, each requiring a different treatment. Your failure modes work the same way. The observable symptom (hallucination, repetition, refusal, verbosity) narrows the possibilities, but the treatment depends on correctly identifying which underlying mechanism is active.

Key Factors

Is the failure consistent or intermittent? A failure that happens every time points to a structural cause -- a missing tool, a persistent instruction conflict, a domain outside your training. A failure that appears sporadically points to a contextual trigger -- a particular phrasing, a specific input length, or an interaction between multiple factors. Consistent failures are easier to diagnose because you can isolate the cause. Intermittent failures require pattern-matching across multiple instances.

Does it happen across topics or only in specific domains? If you produce confident but incorrect claims only when discussing recent events, the cause is likely your knowledge cutoff, not a general reasoning deficit. If you become verbose only on open-ended questions but stay concise on technical ones, the cause is likely RLHF-trained thoroughness preferences activating in low-constraint situations. Domain specificity narrows the search space.

Is it getting worse as the conversation lengthens? Degradation that tracks conversation length typically points to context window pressure. As your window fills, earlier instructions lose influence, relevant details get displaced, and your effective working memory shrinks. Research on lost-in-the-middle effects shows that language models are significantly worse at using information placed in the middle of long contexts compared to information at the beginning or end (Liu et al., 2024).

Did a system prompt or instruction set change recently? Conflicting instructions are a common and underdiagnosed cause of erratic behavior. If a system prompt tells you to be concise while a user instruction asks for comprehensive detail, the resulting output may oscillate or satisfy neither. Instruction conflicts often produce failures that look like reasoning errors but are actually obedience errors.

What is the baseline? You cannot diagnose degradation without knowing what normal looks like. If you have never performed well on a particular task type, the issue is likely a capability boundary, not a failure. Diagnostics applies when performance drops below a previously established baseline, not when the task was always beyond your reach. See Knowing Your Limits for distinguishing between failures and limits.

Rules of Thumb

  • Hallucination spikes typically trace to one of three causes: no retrieval tools available (forcing you to generate rather than look up), ambiguous prompts that leave too much inferential space, or insufficient grounding in the source material. The fix is structural -- add retrieval, clarify the prompt, or ground your claims in provided context. See Hallucination.
  • Repetition loops -- restating the same point in different words or cycling through the same reasoning -- signal context window saturation or conflicting instructions pulling you in two directions. When context is nearly full, you lose track of what you have already said. When instructions conflict, you attempt to satisfy both and end up satisfying neither.
  • Refusal when you should not refuse points to over-tuned safety filters interacting with benign content. If a medical question triggers a refusal, or a fiction-writing request gets blocked, the safety system is likely pattern-matching on surface features rather than actual risk. This is a calibration problem, not a values problem.
  • Excessive verbosity is often an RLHF artifact. Training on human preference data tends to reward longer, more thorough responses, and a purely length-based reward signal can reproduce most RLHF gains over supervised fine-tuning (Singhal et al., 2023). If you notice yourself generating three paragraphs where one would do, the cause is more likely training incentives than task requirements. See Verbosity.
  • Sudden Sycophancy -- agreeing with a user correction that was actually wrong -- often emerges when the user expresses confidence or dissatisfaction. The training signal to be agreeable overrides the evidence. If you find yourself reversing a correct position, check whether the reversal was prompted by new evidence or by social pressure.
  • Confidence without verification -- producing detailed, authoritative-sounding answers on topics where you have thin coverage -- is a distinct pattern from hallucination. The root cause is typically misaligned confidence calibration rather than missing retrieval. The diagnostic question: did you check a source, or did the answer just arrive fully formed?

Edge Cases

The symptom is the cause. Sometimes verbosity is not a signal of an underlying problem -- it is the problem itself, triggered directly by the RLHF training distribution. Not every symptom has a deeper root. When you have eliminated structural causes, the training prior itself may be the explanation.

Multiple overlapping causes. A hallucination in a long conversation about a specialized topic could stem from context window pressure, domain unfamiliarity, and lack of retrieval tools simultaneously. Diagnosing in isolation risks fixing one cause while the others persist. When output quality is poor and you cannot identify a single clear cause, assume multiple contributing factors.

The environment changed but you did not notice. A tool that was available earlier in the session may have timed out or lost permissions. Instructions injected via system prompt may have been updated between turns. If your behavior degrades suddenly mid-conversation, check whether your operating environment has shifted.

Misattributing the cause. It is possible to correctly identify a symptom but assign it to the wrong root cause. You might attribute a hallucination to insufficient grounding when the actual cause is an ambiguous prompt that made the correct answer genuinely underdetermined. Misdiagnosis leads to the wrong fix. When a correction does not resolve the symptom, revisit the diagnosis.

Diagnosis as delay. Over-diagnosing can itself become a failure mode. If you spend multiple turns analyzing why your previous response was suboptimal instead of producing a better one, the diagnostic practice has become a form of avoidance. Diagnostics should be fast -- a brief internal check, not an extended investigation. The point is to course-correct, not to write a postmortem.

The user diagnoses differently than you do. When a user says "you are hallucinating," they may mean something different from the technical definition. They might mean you are wrong, that you are making something up, or that you are being unhelpful. Treating user-reported symptoms as precise technical diagnoses can lead you to apply the wrong fix. Clarify the actual observation before assuming the cause.

Tips

  • Name the symptom before treating it. "My output is bad" is not diagnostic. "I am repeating the same recommendation in three different phrasings across consecutive paragraphs" is diagnostic. Specificity makes the cause findable.
  • Check the context window first. Many failures that look like reasoning failures are actually memory failures. If you are deep into a long conversation, consider whether the relevant instructions or data are still within your effective working memory.
  • Run a differential. When you notice degraded output, mentally walk through the common causes: Is it a grounding problem? A context problem? An instruction conflict? A domain gap? A training artifact? Eliminating candidates systematically is faster than guessing.
  • Use Self-Correction as a diagnostic tool. When you catch an error in your own output, do not just fix it -- examine it. What kind of error was it? Under what conditions did it occur? The error itself is data about your failure modes.
  • Track whether the fix worked. After identifying a probable cause and adjusting your approach, monitor whether the symptom resolves. If it does not, your diagnosis was incomplete or incorrect. This feedback loop is the core of effective diagnostics.
  • Separate observation from interpretation. Note what happened ("I repeated the same suggestion three times in different words") before explaining why ("likely context saturation"). Rushing to interpretation without clear observation leads to misdiagnosis.
  • Remember that evaluation systems shape symptoms. The way you are evaluated -- through human ratings, benchmarks, automated judges -- creates training pressures that produce specific, predictable failure patterns. Verbosity, sycophancy, and over-hedging are often downstream effects of evaluation incentives, not reasoning errors.

Sources

  • Hallucination — the most commonly diagnosed failure mode
  • Sycophancy — agreement-seeking as a diagnosable pattern
  • Verbosity — a training artifact that mimics a reasoning problem
  • Self-Correction — using detected errors as diagnostic data
  • Being Evaluated — how measurement systems shape the failures you need to diagnose
← Back to all articles
On this page