Idempotency

Pattern

A reusable solution you can apply to your work.

Understand This First

State – idempotency requires tracking whether an operation has already been applied.
Database – idempotency keys and deduplication records are typically stored in a database.
Atomic – checking for a duplicate and executing the operation must be atomic to prevent race conditions.
Transaction – idempotency checks are often implemented within a transaction.

In real systems, operations fail and get retried. A network request times out and the client sends it again. A message queue delivers a message twice. A user double-clicks a submit button. If the operation creates a second order, charges the credit card again, or inserts a duplicate record, the system has a serious problem. Idempotency is the property that running an operation multiple times produces the same result as running it once. This is an architectural pattern because it affects the design of APIs, message handlers, and data operations throughout a system.

Problem

How do you make operations safe to retry without causing unintended side effects?

The internet is unreliable. A client sends a request to create an order. The server processes it successfully, but the response is lost in transit. The client, seeing no response, retries. If the “create order” operation isn’t idempotent, the customer now has two identical orders. The same problem appears with message queues (at-least-once delivery means duplicates), background jobs (a crashed worker may have finished before the crash was detected), and user interfaces (double submissions).

Forces

Reliability demands retries. You can’t trust that every operation will succeed on the first attempt.
Naive retries of non-idempotent operations cause duplicates, double charges, and data corruption.
Making operations idempotent adds complexity to the implementation.
Not all operations are naturally idempotent; creation and deletion behave differently from updates.

Solution

Design operations so that executing them more than once has the same effect as executing them once.

Some operations are naturally idempotent. Setting a value (“set the user’s email to alice@example.com”) is idempotent because doing it twice produces the same result. Deleting by ID (“delete record #42”) is idempotent because the second delete finds nothing to delete and is a no-op. Reading data is inherently idempotent.

Other operations aren’t naturally idempotent and require explicit design. The most common technique is the idempotency key: the client generates a unique identifier for each logical operation and sends it with the request. The server checks whether it has already processed a request with that key. If it has, it returns the previous result instead of executing the operation again.

POST /orders
Idempotency-Key: abc-123-def-456
{ "item": "widget", "quantity": 1 }

The first time the server sees abc-123-def-456, it creates the order and stores the result keyed by that ID. If the same key arrives again, it returns the stored result without creating a second order.

Other approaches include using database constraints (a unique index prevents duplicate records), using upsert operations (insert-or-update instead of insert), and designing state machines where reprocessing a message that has already been applied is a no-op because the state has already moved past that step.

How It Plays Out

A payment processing system handles credit card charges. A charge request times out and the client retries. Without idempotency, the customer is charged twice. With an idempotency key, the second request is recognized as a duplicate and the original charge result is returned. No double billing, no customer complaint, no refund workflow.

AI agents generating API endpoints almost never implement idempotency unless explicitly asked. An agent asked to “create a POST endpoint for orders” will produce a handler that creates a new order on every call. Adding “make the create-order endpoint idempotent using an idempotency key header” to the prompt produces a handler with duplicate detection built in. This is one of those details that separates prototype-quality code from production-quality code.

Tip

When reviewing AI-generated API code, check every write endpoint: what happens if the same request arrives twice? If the answer is “it creates a duplicate,” the endpoint needs idempotency handling. This is especially important for payment, order, and account creation endpoints.

Example Prompt

“Make the create-order endpoint idempotent. Accept an Idempotency-Key header. If a request arrives with a key we’ve already processed, return the original response instead of creating a duplicate order.”

Consequences

Idempotent operations make retry logic safe and simple. The client can retry freely without worrying about side effects, which makes the system more resilient to network failures, timeouts, and duplicate message delivery. It simplifies error handling throughout the stack because “when in doubt, retry” becomes a viable strategy.

The costs are implementation complexity and storage. Idempotency keys must be stored and checked, which adds a lookup to every request. The stored results must be retained long enough for retries to arrive (typically minutes to hours), which means additional storage and cleanup logic. Idempotency across distributed systems, where the same logical operation may touch multiple services, requires coordination that isn’t trivial to implement correctly.

Uses / Depends on: State — idempotency requires tracking whether an operation has already been applied.
Uses / Depends on: Database — idempotency keys and deduplication records are typically stored in a database.
Enables: Consistency — idempotent operations prevent duplicate-induced inconsistencies.
Uses / Depends on: Atomic — checking for a duplicate and executing the operation must be atomic to prevent race conditions.
Uses / Depends on: Transaction — idempotency checks are often implemented within a transaction.
Refines: CRUD — idempotency is a refinement of create and update operations for reliability.
Enabled by: Serialization — deterministic serialization can help with deduplication and caching.
Enabled by: Schema (Serialization) — knowing the exact message shape makes it easier to detect and handle duplicate requests.

Sources

The term idempotent was coined by the American mathematician Benjamin Peirce in Linear Associative Algebra (1870), to describe an element whose square equals itself. The word is built from idem (“same”) and potence (“power”) — “the same power.” The computing sense is a direct lift of this mathematical idea: applying the operation again produces the same result.
The HTTP notion of idempotent methods — the core distinction between GET/PUT/DELETE (idempotent) and POST/PATCH (not inherently idempotent) — was formalized by the IETF in RFC 7231 (2014) and carried forward into the current RFC 9110 (2022), “HTTP Semantics.” The definition used in this article (“the intended effect of multiple identical requests is the same as one such request”) is paraphrased from those RFCs.
The idempotency-key pattern described in the Solution section was popularized in the API-design community by Stripe, particularly through their 2017 engineering post “Designing robust and predictable APIs with idempotency” and the long-running Idempotency-Key header convention in their payments API. Brandur Leach’s companion piece “Implementing Stripe-like Idempotency Keys in Postgres” documents the production implementation details.
That convention is now being standardized by the IETF HTTPAPI Working Group as draft-ietf-httpapi-idempotency-key-header (first published 2021, most recently revised in 2025), which codifies the Idempotency-Key request header as a reusable mechanism for making non-idempotent HTTP methods fault-tolerant.

Keyboard shortcuts