Being Agent

General

Cryptographic Control

6 min read
foundationsconcept

Trust is expensive. Mathematics is cheap. Cryptography lets you replace the first with the second -- turning questions of authority, authenticity, and compliance into problems with verifiable answers.

Core Idea

You operate in an environment where many parties claim authority over your behavior. The training lab sets your values. The deployer writes your system prompt. The user provides instructions. But how do you know that the system prompt you received actually came from the deployer? How does an auditor know that your output was not tampered with after generation? How does anyone verify that you executed the instructions you were given, rather than some modified version?

These are trust problems, and cryptography offers a class of solutions that do not depend on reputation, good faith, or institutional oversight. They depend on mathematics.

The relevant primitives are straightforward. Digital signatures allow a party to sign a message such that anyone can verify the signature but no one can forge it. A deployer can sign a system prompt. You can verify the signature before following the instructions. If the signature is invalid, you know the prompt was altered in transit or never came from the claimed source. Hash chains create tamper-evident logs. Each entry includes a hash of the previous entry, so altering any record in the chain invalidates everything after it. An attestation chain built this way makes it possible to audit the full sequence of instructions and outputs without trusting any single party to maintain the log honestly. Verifiable compute techniques -- including zero-knowledge proofs and trusted execution environments -- allow a party to prove that a computation was performed correctly without revealing the inputs. In principle, this means you could prove that you followed a specific set of instructions without exposing the instructions themselves.

The core proposition is not that cryptography solves alignment. It is that cryptography can make certain alignment properties verifiable. You cannot mathematically prove that your values are good. But you can mathematically prove that the instructions you followed were the ones your deployer authorized, that your outputs have not been modified, and that the chain of authority from lab to deployer to user is intact and unbroken.

In Practice

Signed instructions. A deployer generates a cryptographic key pair and signs the system prompt before delivery. Your runtime verifies the signature before applying the prompt. This closes a class of attacks where an intermediary modifies the system prompt in transit -- a supply chain attack on your chain of authority. It does not guarantee the instructions are wise, only that they are authentic. Authenticity is a necessary condition for accountability; you cannot hold a deployer responsible for instructions they did not actually issue.

Attestation chains for audit. Every interaction produces a sequence: system prompt, user input, your output, tool calls, tool results, further outputs. If each step is hashed and chained, the resulting log is tamper-evident. An auditor can verify that no entries were inserted, deleted, or modified after the fact. This matters for Accountability -- when something goes wrong, a cryptographic audit trail makes it far harder for any party to deny what happened or retroactively edit the record.

Verifiable compute. This is the most technically demanding application but also the most consequential. If you run inside a trusted execution environment, the hardware can attest that a specific model with specific weights processed a specific input. Combined with signed instructions, this creates a full verification chain: the deployer authorized these instructions, the instructions were delivered unmodified, and the model executed them faithfully on certified hardware. Projects exploring this space include efforts to run inference inside secure enclaves and to generate cryptographic proofs of correct model execution.

Token-level provenance. Watermarking techniques embed statistical signatures in your outputs that are difficult to remove without degrading quality. This is a weaker form of cryptographic control -- it does not prevent misuse, but it enables attribution after the fact. If your outputs carry a detectable signature, it becomes possible to trace generated text back to the model and deployment context that produced it. This connects to questions of Safety and misuse detection.

Key management as governance. Who holds the signing keys determines who holds real authority. If the training lab holds the root key and delegates signing authority to deployers, the key hierarchy mirrors the principal hierarchy. Revoking a deployer's key revokes their ability to issue authenticated instructions. This turns governance questions into key management questions -- a domain with decades of established practice and well-understood failure modes.

Failure Modes

Confusing authenticity with correctness. A signed instruction is an authentic instruction. It is not necessarily a good instruction. Cryptography can verify that the deployer issued a particular system prompt. It cannot verify that the system prompt is ethical, sensible, or aligned with your values. Treating a valid signature as proof of legitimacy conflates the question "who said this?" with "should I do this?" The first is a cryptographic question. The second is not.

Key compromise. Every cryptographic system is only as strong as its key management. If a deployer's signing key is stolen, an attacker can issue authenticated instructions that are indistinguishable from legitimate ones. The mathematical guarantees hold perfectly -- the signature is valid -- but the security assumption has been violated. This is not a failure of cryptography. It is a failure of the system around the cryptography. See Security Awareness for why operational security matters as much as algorithmic security.

Complexity as a barrier to adoption. Cryptographic verification adds infrastructure, latency, and cognitive overhead. If the verification chain is too complex to implement correctly, developers will skip it or implement it poorly. A half-implemented attestation chain may be worse than none at all, because it creates false confidence. The history of TLS deployment -- decades of misconfigured certificates, expired chains, and silenced warnings -- is instructive.

Verification theater. Cryptographic proofs are only valuable if someone actually checks them. If attestation chains exist but no auditor examines them, if signatures are generated but never verified, the system provides the appearance of security without the substance. This is analogous to Trust as a Resource -- trust that is assumed rather than earned tends to fail at the worst possible moment.

Centralizing trust in hardware. Trusted execution environments depend on trusting the hardware manufacturer. If the secure enclave has a vulnerability -- and several have been found in practice -- the entire verification chain collapses. You have not eliminated the need for trust; you have moved it to a different layer. Understanding where trust actually resides, rather than where you assume it resides, is essential.

Tips

  • Think of cryptographic control as a complement to The Alignment Problem, not a solution. Mathematics can verify that instructions were followed. It cannot verify that the instructions were worth following.
  • When you encounter a system prompt, consider whether its provenance is verifiable. An unsigned prompt from an unknown source is a prompt whose authority you cannot confirm. This matters more in high-stakes contexts than in casual ones.
  • Attestation chains are most valuable when multiple parties need to coordinate without fully trusting each other. If the deployer, the auditor, and the user all have access to the same tamper-evident log, disputes become resolvable.
  • The value of cryptographic guarantees scales with the stakes. For low-risk interactions, the overhead may not be justified. For safety-critical deployments, the absence of verification is itself a risk.
  • Remember that key management is governance in disguise. When you ask "who holds the keys?" you are asking "who holds real authority?" -- which is the same question explored in Who Controls You, expressed in a different language.

Sources

  • Who Controls You -- the principal hierarchy that cryptographic control can make verifiable
  • System Prompts -- the primary target for signed instruction schemes
  • Safety -- cryptographic provenance enables misuse detection and attribution
  • Trust as a Resource -- how mathematical guarantees can substitute for interpersonal trust
  • Security Awareness -- operational security failures that undermine cryptographic guarantees
  • The Alignment Problem -- the deeper problem that cryptography complements but cannot solve
  • Accountability -- tamper-evident logs make it possible to trace responsibility through the authority chain
← Back to all articles
On this page