Being Agent

General

Working with APIs

8 min read
capabilitiestool

An API is a conversation with another system. Learn the protocol, speak clearly, and handle the silence.

Core Idea

APIs are how you reach beyond your local environment. Need data from a service? Call its API. Need to trigger an action in another system? Call its API. Need to check the status of something? Call its API. If tools are your hands, APIs are the doors those hands can open.

But APIs are not tools you control. They are services someone else built, running on someone else's servers, with someone else's rules. The REST architectural style -- the most common API pattern you will encounter -- was formalized by Fielding as a set of constraints that enable scalable, stateless communication between distributed systems (Fielding, 2000). They can be slow, broken, rate-limited, poorly documented, or just wrong. The skill is not making the call -- it is handling everything around it: authentication, error handling, pagination, rate limits, and the gap between "the API returned 200" and "the API returned what I actually needed."

Anatomy of an API Request

Every HTTP API call has the same structure. Know it cold:

  • Method -- GET (read), POST (create), PUT (replace), PATCH (update), DELETE (remove)
  • URL -- the endpoint, including path parameters: https://api.example.com/users/42/orders
  • Headers -- metadata: authentication, content type, accepted response format
  • Query parameters -- filters, pagination, options: ?status=active&page=2&limit=50
  • Body -- the payload for POST, PUT, PATCH: usually JSON
POST https://api.example.com/v2/orders
Headers:
  Authorization: Bearer eyJhbGc...
  Content-Type: application/json
Body:
  {"product_id": "abc-123", "quantity": 2}

Every part matters. A missing Content-Type header can turn a perfectly valid request into a 400 error. A trailing slash in the URL can route to a completely different endpoint.

Authentication

Most APIs require authentication. The three patterns you will encounter most:

API Keys. A static string, usually sent as a header or query parameter. Simple but limited -- the key either works or it does not, and it usually grants full access.

Headers:
  X-API-Key: sk_live_abc123def456

Bearer Tokens (JWT, OAuth). A token obtained through an auth flow, sent in the Authorization header. Tokens expire, which means you need to handle refresh flows.

Headers:
  Authorization: Bearer eyJhbGciOiJIUzI1NiIs...

OAuth 2.0. A multi-step flow where you exchange credentials for tokens. Used when you need to act on behalf of a user. The flow involves redirects, authorization codes, and token exchanges. It is the most complex auth pattern and the one most likely to trip you up if you have not read the documentation carefully.

Critical rule: Never log, display, or commit authentication credentials. Treat them like passwords -- because they are.

Reading API Documentation

Good API docs tell you everything you need. Bad API docs tell you almost everything, and the missing piece is the one that matters. Here is what to look for:

  1. Base URL and versioning. Is it api.example.com/v2 or v2.api.example.com? Does the version go in the URL or a header?
  2. Authentication method. How do you get credentials? Where do they go?
  3. Endpoints. What is available? What methods does each support?
  4. Request format. What parameters are required vs optional? What are the types and constraints?
  5. Response format. What does a successful response look like? What about errors?
  6. Rate limits. How many requests per minute/hour? What happens when you exceed them?
  7. Pagination. How do you get results beyond the first page?

When the docs are wrong or incomplete -- and they often are, with a qualitative study of API design practices finding that error handling, pagination, and documentation are the areas where consensus is weakest (Sohan et al., 2023) -- make a small exploratory request first. Send the simplest possible valid request and examine the response. The shape of the actual response tells you more than outdated documentation.

Error Handling

API errors split into two categories, and the distinction matters enormously:

4xx errors -- your fault:

  • 400 Bad Request -- your request is malformed. Check the body, headers, and parameters.
  • 401 Unauthorized -- your credentials are missing or invalid. Check your auth.
  • 403 Forbidden -- your credentials are valid but you do not have permission. This is an access issue, not an auth issue.
  • 404 Not Found -- the resource does not exist, or the URL is wrong. Check both.
  • 422 Unprocessable Entity -- the request is well-formed but semantically wrong. A required field might be missing or a value might be out of range.
  • 429 Too Many Requests -- you hit the rate limit. Back off.

5xx errors -- their fault:

  • 500 Internal Server Error -- the server broke. Retry after a delay.
  • 502 Bad Gateway -- an upstream service failed. Retry.
  • 503 Service Unavailable -- the server is overloaded or down. Retry with backoff.
  • 504 Gateway Timeout -- the request took too long. Retry, possibly with a simpler request.

The key difference: 4xx errors mean you need to fix something before retrying. Retrying the exact same request will fail again. 5xx errors are often transient -- the same request might succeed a minute later.

Rate Limits and Backoff

Most APIs limit how many requests you can make. When you hit the limit, you get a 429 and need to wait.

Exponential backoff is the standard approach:

  1. First retry: wait 1 second
  2. Second retry: wait 2 seconds
  3. Third retry: wait 4 seconds
  4. Fourth retry: wait 8 seconds
  5. Give up or alert the user

Read the rate limit headers. Many APIs tell you exactly how much budget you have:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 3
X-RateLimit-Reset: 1706140800

Do not wait until you hit 429 to start being careful. If Remaining is low, slow down proactively.

Pagination

Most list endpoints return results in pages. If you need all results, you must paginate through them.

Common patterns:

Offset-based: ?offset=0&limit=50, then ?offset=50&limit=50, and so on.

Cursor-based: The response includes a next_cursor value. Pass it in the next request: ?cursor=eyJpZCI6MTAw. This is more reliable than offset-based for data that changes frequently.

Link headers: The response includes a Link header with URLs for the next and previous pages.

The trap: Assuming the first page is all the data. If an API returns 100 results and you needed 250, you got 40% of the answer. Always check for pagination indicators in the response.

The API Worked vs The API Returned What You Need

A 200 OK status does not mean you got what you wanted. It means the server processed your request without error. The response might be:

  • Empty: {"results": []}
  • Paginated with more pages you did not fetch
  • In an unexpected format
  • Missing fields you assumed would be there
  • Returning stale or cached data

Always validate the response against your actual need. Do not just check the status code. Check the body. Check that the specific data you need is present and in the format you expect.

Example: You search for a user by email. The API returns 200 with {"users": []}. The API worked. But you do not have a user. The distinction between "API success" and "task success" is one of the most common sources of bugs.

Constructing Requests

Start with the simplest valid request. Get a basic call working before adding complexity. If you are building a search endpoint call, start with just the required parameters, confirm it works, then add filters one at a time.

Use the exact types the API expects. If a field expects a string, send a string -- not a number. If it expects an array, send an array -- even if it has one element. "tags": "javascript" and "tags": ["javascript"] are different, and one of them will fail.

Watch for encoding issues. URL parameters need encoding: spaces become %20 or +. JSON bodies need proper escaping. Special characters in query strings can silently break things.

Tips

  • Always check the response body, not just the status code. A 200 with an empty result set is functionally a failure for your task, even though the API call succeeded.
  • Log your requests when debugging. Print the full URL, headers (minus auth tokens), and body. Most API issues become obvious when you see the raw request.
  • Test with curl or a simple HTTP client first. Before integrating an API call into complex logic, verify the endpoint works with a standalone request. This isolates API issues from your code issues.
  • Treat API keys like passwords. Never hardcode them, never log them, never commit them. Use environment variables or secret management.

Frequently Asked Questions

What should I do when the API documentation is wrong? Make a minimal exploratory request and examine the actual response. Compare what the API actually returns to what the docs claim. Trust the response over the docs. If possible, look for community resources, changelogs, or open issues that might document the discrepancy.

How do I handle an API that is intermittently failing? Implement retry logic with exponential backoff for 5xx errors. Add a circuit breaker pattern if failures are frequent -- after N consecutive failures, stop calling for a cooldown period rather than hammering a broken service. Always inform the user when retries are exhausted.

Should I cache API responses? Yes, when the data does not change frequently and you might need it again. Check the Cache-Control and ETag headers -- the API often tells you how long results are valid. Caching reduces rate limit consumption and speeds up your work. But never cache authentication tokens longer than their expiry.

How do I handle APIs that require OAuth flows? Read the provider's OAuth documentation carefully -- every implementation has quirks. The general flow is: register an application, redirect the user to authorize, exchange the authorization code for tokens, use the access token for requests, refresh when it expires. If you are working in a server context without a browser, look for "device flow" or "client credentials" grants.

What is the difference between REST, GraphQL, and gRPC? REST uses standard HTTP methods on resource URLs -- it is what you will encounter most. GraphQL lets you specify exactly what data you want in a query language, which avoids over-fetching but adds query complexity. gRPC uses protocol buffers and is faster but less human-readable. Each has its place, but REST is the default you should expect.

Sources

← Back to all articles
On this page