Idempotency Keys: Making Retries Boring Since Forever

Here's a fun thought experiment I run in architecture reviews: your client POSTs a payment, the request times out, and the client has no idea whether the charge happened. What does it do? Retry and risk double-charging? Give up and risk an angry unpaid order? Ask the user, who will absolutely click the button again either way? Every answer is wrong — because the API forced an unanswerable question. Idempotency keys make the question disappear, and they're cheap enough that every serious API should have them. Let's build the mechanism properly, because the naive version has traps.

The contract

The client generates a unique key per logical operation (not per HTTP attempt!) and sends it as a header. The server guarantees: however many times this key arrives, the operation executes once, and every request with that key receives the same response.

POST /v1/payments
Idempotency-Key: pay_order4211_a3f8c...   # client-generated, stable across retries

# timeout... client retries with the SAME key
# → server replays the original response; no second charge exists anywhere

Retries become safe by contract, which unlocks everything downstream: aggressive client timeouts, queue-driven retry policies, chaos-tolerant integrations. Stripe popularized the pattern; there's no reason your API can't offer the same courtesy.

The implementation, with its three traps disarmed

public function store(PaymentRequest $request)
{
    $key = $request->header('Idempotency-Key') ?? abort(400, 'Idempotency-Key required');

    // TRAP 1 disarmed: atomic claim via unique index — not check-then-insert.
    try {
        $record = IdempotencyRecord::create([
            'key' => $key,
            'request_hash' => hash('sha256', $request->getContent()),
            'status' => 'processing',
        ]);
    } catch (UniqueConstraintViolationException) {
        $record = IdempotencyRecord::where('key', $key)->first();

        // TRAP 2 disarmed: same key + DIFFERENT body = client bug. Refuse loudly.
        if ($record->request_hash !== hash('sha256', $request->getContent())) {
            abort(422, 'Idempotency key reused with different payload.');
        }

        // TRAP 3 disarmed: the concurrent-processing window.
        if ($record->status === 'processing') {
            return response()->json(['status' => 'processing'], 409,
                ['Retry-After' => 2]);          // first attempt still running — wait
        }

        return response()->json($record->response_body, $record->response_status);
    }

    $response = $this->payments->charge($request->validated());   // executes ONCE

    $record->update([
        'status' => 'completed',
        'response_status' => $response->status(),
        'response_body' => $response->body(),
    ]);

    return $response;
}

Why each trap matters: (1) check-then-insert has the same race as every check-then-act bug — two concurrent retries both pass the check; the unique index makes claiming atomic. (2) Without the payload hash, a buggy client reusing keys gets silently served someone else's cached response — an incredibly confusing bug to debug from the outside. (3) The 409-while-processing answer handles the genuinely hard case: retry #2 arriving while attempt #1 is still mid-flight. Blocking until #1 finishes is also acceptable; executing again is not.

Operational details that finish the job: store failed responses too, but only deterministic failures (a 422 should replay; a 503 from your PSP should not — clear the record on transient failure so a retry can genuinely retry), expire records after ~24–72h with a cleanup job, and scope keys per API consumer so two integrators can't collide.

The part people miss: it composes all the way down

Your endpoint is idempotent — but is the work behind it? If charging calls a PSP, pass an idempotency key to the PSP (derive it from yours). If the endpoint enqueues jobs, the jobs need their own idempotency. If it emits events, consumers dedupe by event ID. The property must hold at every layer that has a side effect, because a timeout can strike between any two of them. The mental model: idempotency isn't a feature of an endpoint; it's a discipline of a call chain.

Cheaper alternatives, when the full mechanism is too much

• Natural keys. "One invoice per order per month" is a unique constraint — the request's own semantics dedupe it, no header required. Always prefer this when the domain offers it.
• State-machine guards. UPDATE orders SET status='paid' WHERE id=? AND status='pending' — affected-rows zero means someone beat you; read and return the current state. Idempotent by construction.
• PUT semantics. Full-resource replacement is naturally idempotent; where PUT fits, the problem dissolves.

The full key mechanism earns its keep where the operation is a creation with consequences — payments, orders, legal documents, outbound messages — exactly where a duplicate costs money or trust. There, it's not optional polish; it's the difference between "the network hiccuped" being a non-event and being a refund workflow.

This post completes a reliability trilogy with webhook signatures and queue design — the three disciplines that make distributed failure boring. If double-charges or duplicate orders have ever reached your support inbox, let's make them structurally impossible.