Calling External APIs Reliably
Why every outbound HTTP call needs a timeout, how to retry transient failures with backoff and jitter, and when not to retry at all.
Calling External APIs Reliably
The player loads from youtube-nocookie.com when you press play. Nothing reaches YouTube before that.
Not started
What you'll learn
- Without an explicit timeout, a hung connection to a third-party API can hold a thread or connection open indefinitely, and enough of those piling up can take down your entire service, not just the one slow request.
- A transient failure — a timeout, a 5xx, a dropped connection — is worth retrying; a 4xx client error almost never is, because retrying a malformed request just produces the same error again.
- Exponential backoff spaces retries out with growing delays so a struggling dependency isn't hit harder the more it's already failing; jitter adds randomness so a fleet of clients retrying after the same outage don't all slam the dependency back at the exact same instant.
- Retrying is only safe when the call is idempotent — retrying a 'create this charge' request that actually succeeded but whose response got lost can charge a customer twice unless the call carries an idempotency key.
Your code is a client too
Most backend lessons on reliability focus on your own service staying up. This one is about the other direction: your code is constantly acting as a client to someone else’s service — a payments processor, a mapping API, an email provider, an internal microservice you don’t control the uptime of. Every one of those calls can be slow, down, or flaky, and none of that is something you can fix. What you control is how your code behaves when it is.
Rule one: always set a timeout
Here’s the failure mode that catches people who’ve never been bitten by it: you call a third-party API with whatever HTTP client your language ships, using the default settings. Most defaults either have no timeout at all, or one measured in minutes. One day that API gets slow — not down, just slow, maybe because of a problem on their end — and every request your service sends to it sits there, waiting, holding a thread or a connection open. Do that enough times, across enough concurrent requests, and you run out of threads or connections to serve anything else. A dependency that was supposed to be a small feature just took down your checkout flow, your homepage, everything sharing that resource pool — not because it returned an error, but because it never returned at all.
The fix is mechanical and non-negotiable: every outbound call gets an explicit timeout, set deliberately rather than inherited from a library default. What the right number is depends on the call — a few seconds for something a user is waiting on, longer for a background job that can afford to be patient — but “no timeout” is never the right answer. A timeout turns an indefinite hang into a fast, specific failure you can actually handle.
Rule two: retry the failures that are worth retrying
Once a call times out or comes back with an error, the next question is whether trying again makes sense. Not every failure is the same kind of failure. A 503 Service Unavailable, a connection reset, or your own timeout from rule one are usually transient — the dependency is overloaded, restarting, or briefly unreachable, and the exact same request might well succeed a moment later. A 400 Bad Request or 422 Unprocessable Entity means the request itself was wrong; sending it again, unmodified, produces the same rejection every time. Retrying those just wastes a round trip and, if you’re retrying aggressively, adds needless load to a service that already told you your request was broken.
The practical rule: retry on timeouts, connection errors, and 5xx responses (and often 429 rate-limit responses, since those are explicitly “try again later”). Don’t retry on 4xx responses except 429 — those need a code fix, not a retry.
Rule three: back off, and add jitter
A naive retry loop — fail, wait one second, try again, fail, wait one second, try again — doesn’t actually help a struggling dependency; it just adds a steady drumbeat of load on top of whatever’s already overwhelming it. Exponential backoff fixes the spacing: wait 1 second before the first retry, 2 before the second, 4 before the third, and so on, so your own request rate against a failing dependency drops off instead of staying constant.
But backoff alone has its own failure mode at scale. If a dependency goes down for a moment and a thousand of your clients were all mid-request, they all compute the same backoff schedule and all retry at the same synchronized instants — 1 second later, then 2 seconds later, in lockstep. That’s a thundering herd, and it can hit the dependency with a spike right as it’s trying to recover. Jitter — adding a small random amount to each computed delay — spreads those retries out over a window instead of a single instant. Production-grade retry libraries (Python’s tenacity, for instance, which the embedded video walks through building toward) implement backoff with jitter as the default, specifically because getting this by hand, consistently, across every outbound call in a codebase, is easy to get subtly wrong.
Rule four: retrying isn’t free when the call has a side effect
All of the above assumes retrying is safe to do at all. It isn’t, automatically. If a call creates a resource or moves money, and the original request actually succeeded server-side but the response back to you got lost — the connection dropped a moment after the charge went through, say — a naive retry repeats the side effect: a second charge, a second row. The fix is an idempotency key: a unique identifier your client generates once per logical operation and sends with every attempt, including retries, which a well-built API uses to recognize “I’ve already done this one” and return the original result instead of doing it again. Retrying without one is how a reliability fix turns into a duplicate-charges bug.
Where this stops being your HTTP client’s problem
Everything above is about a single outbound call: what does your actual client code do on this one request. Once you’re coordinating this behavior across many services calling many other services — tripping a breaker to stop calling a dependency at all once it’s clearly down, isolating one slow dependency so it can’t exhaust resources shared with others — you’re in architecture territory, covered in Circuit Breakers and Retries. That lesson is the service-mesh-level version of the same problem; this one is what you actually type into your HTTP client call.
Check yourself
What actually goes wrong if you call a third-party API with no timeout set?
Most HTTP client libraries default to no timeout, or an impractically long one, meaning a connection that never gets a response can hang the calling thread or connection indefinitely. If enough requests to a slow or dead dependency pile up this way, they can exhaust your service's thread pool or connection limit, degrading or taking down requests that have nothing to do with that dependency.
Why retry a 503 but not a 400?
A 503 (service unavailable) or a timeout usually signals a transient condition — the dependency is overloaded or briefly unreachable — that may well succeed on a second attempt moments later. A 400 (bad request) means the request itself was malformed or invalid; sending the exact same malformed request again produces the exact same 400. Retrying only helps when the failure is about timing or load, not about the content of the request.
What problem does jitter solve that exponential backoff alone doesn't?
If every client computes its backoff delay the same deterministic way (wait 1s, then 2s, then 4s...), and a dependency has an outage affecting many clients at once, all of them retry at the same moments — a thundering herd that can overwhelm the dependency right as it's trying to recover. Jitter adds randomness to each client's wait time so retries spread out over a window instead of landing in synchronized bursts.
Why is it unsafe to blindly retry a request that creates or charges something?
A retry assumes the original request failed outright, but it might have succeeded on the server and only the response back to you got lost — a dropped connection after the charge went through, for instance. Retrying blindly in that case repeats the side effect: a second charge, a second row inserted. The call needs to be idempotent, typically via a client-generated idempotency key the server can recognize as 'I've already done this one,' before a retry is safe for anything that isn't a pure read.
Go deeper
Notes
Notes are saved to your account.
Everything else on this page works without one — progress and bookmarks keep themselves in this browser and come with you when you sign in. Notes are the one thing worth not trusting to a browser you might clear.
Your notes could not be loaded.