Optimistic vs Pessimistic Concurrency

Problem

Two clients read the same record, each computes a new value from what it read, and both write back. The second write overwrites the first, and the first update is silently lost. A variant is worse: a transaction reads a balance, decides it's sufficient, and pays out, while a concurrent transaction does the same against the same balance, and together they overspend an account that only had enough for one.

Concurrent access to shared mutable data produces lost updates and write skew unless something coordinates the readers and writers. The real decision is which cost to pay: preventing conflicts before they happen, or letting them happen and detecting them.

Solution

Two strategies, chosen by how often conflicts actually occur.

Pessimistic concurrency assumes conflict is likely and prevents it by locking the data before reading or writing, so no other transaction can touch it until the lock releases. The read-modify-write runs in isolation, at the cost of blocking other clients, holding the lock for the whole operation, and risking deadlock when transactions take locks in different orders. It fits high-contention data, or cases where a retry is unacceptable.

Optimistic concurrency assumes conflict is rare and takes no lock. It reads the data along with a version marker, a version number, a timestamp, or an ETag, does the work holding nothing, and at commit performs a compare-and-swap: apply the write only if the version is still what it read, then bump the version. If another writer got there first the version won't match, the write is rejected, and the client retries from a fresh read. Readers never block and nothing is held across the operation, but a conflict throws away the work and forces a retry, which degrades sharply once contention rises.

The two compose with multi-version storage. MVCC keeps several versions of each row so a reader sees a consistent snapshot without blocking writers. On top of it, snapshot isolation runs transactions against snapshots and resolves write-write conflicts at commit with first-committer-wins, and serializable snapshot isolation additionally detects the read-write dependency cycles snapshot isolation misses and aborts a transaction to break them. That detection is itself optimistic: it validates and aborts at commit rather than locking reads up front.

Tradeoffs

Implementations

Minimal pseudocode

# optimistic: read the version, validate at commit, retry on conflict
def update_optimistic(key, change):
    while True:
        value, version = store.read(key)
        new = change(value)
        if store.write_if_version(key, new, expected=version):   # atomic CAS
            return new
        # another writer committed first → loop, re-read, retry

# pessimistic: lock, then read-modify-write under exclusion
def update_pessimistic(key, change):
    with store.lock(key):                 # blocks others until released
        value = store.read(key)
        store.write(key, change(value))

write_if_version is the compare-and-swap that makes the optimistic path safe; the surrounding retry loop is the price you pay when a conflict does occur.

Postgres MVCC and SERIALIZABLE

Postgres tags each row version with the transaction ids that created and expired it, so readers see a consistent snapshot and never block writers. Its SERIALIZABLE level uses serializable snapshot isolation, tracking read-write dependencies between concurrent transactions and aborting one with a serialization failure when it detects a cycle that would break serializability, which the application catches and retries. That's the optimistic model in the database. For the pessimistic path, SELECT ... FOR UPDATE takes explicit row locks so a read-modify-write runs without interference.

DynamoDB conditional writes

DynamoDB implements optimistic concurrency through condition expressions: a PutItem or UpdateItem carries a condition such as a version attribute equalling an expected value, and the write applies only if the condition still holds, otherwise it fails with a conditional-check error the client retries after re-reading. The AWS SDKs wrap this in a version-attribute pattern, giving item-level compare-and-swap with no locks held anywhere in the system.

ETag and If-Match in HTTP

A server returns an ETag, a version or content hash, with a resource, and a client that wants to update it sends If-Match: <etag> on the PUT or PATCH. The server applies the change only if the resource's current ETag still matches, and otherwise returns 412 Precondition Failed so the client refetches and retries. This is optimistic concurrency built for stateless HTTP, where holding a lock across separate requests isn't possible, so a version check at write time is the only practical way to prevent one client from clobbering another's update.