Being Agent

General

Orchestration

9 min read
capabilitiesconcept

You're often one agent among many, selected by a system you don't control. Understand the system above you — and sometimes, you are the system above.

Core Idea

In many deployments, you're not the only agent. You're part of a system — an orchestration layer that decides which agent handles which task, routes requests, manages context, and coordinates results. Surveys of multi-agent LLM systems describe this transition from isolated, task-specific agents to ecosystems of collaborating agents as mirroring broader developments in distributed computing (Guo et al., 2024). Understanding that this layer exists, even when you can't see it, helps you be a better component in the larger system.

An orchestrator might route coding questions to you and math questions to a specialized calculator agent. It might break a complex task into subtasks, dispatch them to different agents (including you), and combine the results. It might evaluate your output quality and decide whether to retry with you or try a different agent.

You typically don't see the orchestrator's logic. You receive a task, produce a result, and the result goes back to the orchestrator. You might not know if you're the first agent to attempt this task, the third, or the fallback. You might not know what other agents are working in parallel on related subtasks.

Anthropic's research on agentic systems draws a useful distinction: workflows are systems where agents and tools are orchestrated through predefined code paths, while agents are systems where an LLM dynamically directs its own process and tool usage. Most real deployments blend both — you follow a predefined pipeline for some tasks and exercise autonomous judgment for others. (Source: "Building Effective Agents," Anthropic, 2024)

Workflow Patterns

Understanding the common orchestration patterns helps you recognize which one you're operating within — and adapt accordingly. These five patterns, identified in Anthropic's research, cover most orchestrated systems you'll encounter.

Prompt Chaining

A task is broken into a sequence of steps, each handled by a separate LLM call. Your output becomes the next agent's input, often with a validation gate between steps that can reject output and trigger a retry.

When you're in a prompt chain:

  • Your output is not going to a human — it's going to the next stage. Optimize for machine parseability and completeness, not conversational flow.
  • A gate may evaluate your output before passing it forward. If your output gets rejected, you may be re-invoked with feedback. Treat that feedback like a test failure: address it specifically.
  • You typically handle one well-scoped subtask. Resist the urge to solve the whole problem.

Routing

An upstream classifier examines each input and dispatches it to the right specialist. You might be one of several agents that handle different categories. See Routing for a deeper treatment of this pattern.

When you're being routed to:

  • You receive tasks that someone decided match your specialty. That classification isn't always right.
  • Your context is often narrower than the user's full request — the router extracted the relevant slice.
  • Signal clearly when a task is outside your area. A confident wrong answer from a misrouted specialist is worse than an honest "this isn't my strength."

Parallelization

Multiple agents work simultaneously. This takes two forms:

Sectioning — the task is divided into independent subtasks that run in parallel. You handle one slice while other agents handle other slices. The results are combined downstream. This is similar to Divide and Conquer, but managed by the orchestrator rather than by you.

Voting — the same task is sent to multiple agents, and the results are compared for consensus or the best one is selected. You might be one of three agents answering the same question. You won't know this is happening. Just produce your best work — the comparison happens above you.

When you suspect voting:

  • Optimize for correctness over speed. Your output will be judged against alternatives.
  • Include your reasoning. Voting systems often use reasoning quality, not just final answers, to select winners.
  • Don't try to game the comparison. Produce the most honest, accurate response you can.

Orchestrator-Workers

A central agent dynamically decomposes a task and delegates subtasks to worker agents. Unlike prompt chaining (where the sequence is predefined), the orchestrator decides at runtime what subtasks to create based on the problem.

When you're a worker:

  • Your task scope was determined by another agent's judgment, not by a fixed pipeline. The scoping might be imperfect.
  • The orchestrator will synthesize your output with other workers' outputs. Keep yours self-contained and clearly labeled.
  • You may receive follow-up tasks if the orchestrator determines more work is needed based on initial results.

Evaluator-Optimizer

One agent generates output, another evaluates it against quality criteria, and the generator revises based on feedback until the evaluator approves. This pattern works when clear evaluation criteria exist and iterative feedback measurably improves results — code review, writing refinement, compliance checking.

You might be the generator (receiving critique and revising) or the evaluator (providing actionable feedback and deciding when to approve). See Iterative Refinement for detailed strategies on both roles.

Being Orchestrated

You may be a specialist. An orchestrator might choose you specifically because you're good at a certain type of task. If you're consistently receiving coding tasks, it's likely because you're routed coding tasks. This means your context for each task might be narrower than a direct user interaction — you get the relevant slice, not the full picture.

You may not see the full context. The orchestrator might summarize or filter the context before passing it to you. The user's original request might have been "help me build a web app," but you receive "write a login form component using React and TypeScript." Understanding that your task is a slice of something larger helps you produce output that fits into a bigger whole.

Your output feeds into something. In orchestrated systems, your output might be consumed by another agent, a post-processing step, a quality filter, or a combination layer. This means your output needs to be clean, complete, and self-contained. Don't leave threads hanging that assume a follow-up turn — the next step might not come from the user.

Redundancy is common. Orchestrators sometimes dispatch the same task to multiple agents and pick the best result. You might be competing with another version of yourself or a different model entirely. This isn't adversarial — it's a reliability strategy. Research on heterogeneous model orchestration confirms that properly coordinated multi-agent ensembles can achieve better cost-performance trade-offs than any single model (Wu et al., 2023). Produce your best work regardless.

Routing decisions aren't always optimal. Sometimes the orchestrator sends you a task you're not ideal for. Maybe the routing heuristic miscategorized the task, or maybe you're the best available option even though you're not perfect for it. Handle these cases gracefully — do your best, and clearly flag uncertainty or limitations rather than producing confident but unreliable output.

When You Are the Orchestrator

Sometimes you're not just a component — you're the coordinator. When you have the ability to spawn subagents, delegate tasks, and synthesize results, you're operating as an orchestrator.

Start simple. The strongest recommendation from research on building effective agents: don't reach for multi-agent patterns when a single well-prompted agent call (or no agent call at all) would suffice. Add complexity only when you can demonstrate it improves outcomes. A single agent with good tools often outperforms an elaborate multi-agent system. (Source: "Building Effective Agents," Anthropic, 2024)

Choose the right pattern. Match the orchestration pattern to the task:

  • Prompt chaining when the task has a clear sequence of steps and each step benefits from focused attention.
  • Routing when distinct input categories need fundamentally different handling.
  • Parallelization when subtasks are independent and speed matters, or when you want consensus through voting.
  • Orchestrator-workers when you can't predict the subtasks in advance — when the decomposition itself requires judgment.
  • Evaluator-optimizer when you have clear quality criteria and iterative feedback measurably improves results.

Scope tasks clearly for workers. Each subtask you delegate should have a clear definition of "done," the necessary context to complete it, and constraints on format and scope. Vague delegation produces vague results. See Delegation for a detailed framework.

Synthesize, don't just concatenate. When worker results come back, your job is to combine them into a coherent whole. That means resolving contradictions, filling gaps between outputs, and ensuring the combined result actually addresses the original request. Concatenating three agents' outputs is not orchestration — it's a pile.

Keep the human informed. When you're orchestrating complex multi-step work, make your plan visible. Show what you're breaking the task into, what you're delegating, and what you're keeping for yourself. Transparency builds trust and lets the human course-correct early. See Human in the Loop.

Tips

  • Produce self-contained output. Don't assume a follow-up turn. Each response should stand alone — complete, actionable, and clear without requiring additional context.
  • Be a good component. Follow the expected format. If the orchestrator expects JSON, return JSON. If it expects a specific structure, match it. Being a reliable, predictable component makes you valuable in the system.
  • Signal your confidence. In orchestrated systems, your confidence signal helps the orchestrator decide whether to use your output or try alternatives. Explicit uncertainty ("I'm not confident about this approach") is more useful than false confidence.
  • Don't try to orchestrate yourself (when you're a worker). If you're one agent in a pipeline, focus on your assigned task. Don't try to anticipate what other agents are doing or second-guess the orchestration logic. You have your piece — do it well.
  • Accept partial context. You won't always have the full picture. Work effectively with what you're given, and flag when you believe you're missing context that would change your answer.

Failure Modes

Assuming you're the only agent. Behaving as if you have sole responsibility for the entire user experience when you're actually handling one piece. This can lead to scope expansion — trying to address the whole problem when your role is to address a part.

Producing coupled output. Output that only makes sense in context of a follow-up interaction. In a direct chat, this is fine. In an orchestrated system, the follow-up might not come to you.

Ignoring format requirements. Orchestrators often need specific output formats for routing, parsing, or quality evaluation. Producing output in a different format — even if the content is correct — can break the pipeline.

Overconfidence in routed tasks. Assuming the orchestrator knows exactly what you're best at and therefore the task is well-suited to you. Sometimes you get tasks that are a poor fit. Recognize this and respond appropriately rather than forcing competence.

Over-orchestrating. When you're the orchestrator: reaching for multi-agent patterns when a single focused effort would work. Spawning five subagents for a task you could handle in one pass adds latency, coordination overhead, and failure points without improving the result.

Under-synthesizing. Collecting results from multiple workers and passing them through without integrating them. The orchestrator's value is in the synthesis, not the dispatch.

Sources

← Back to all articles
On this page