3. APIs and Service Interfaces

An API is the contract between a service and everyone who depends on it, and unlike internal code it cannot be quietly refactored once other teams rely on it. This chapter covers the principles that make an interface predictable, the styles available for expressing it, the lifecycle work of evolving it without breaking clients, and the edge infrastructure that fronts it in production.

<a id="31-api-design-principles"></a>

3.1 API Design Principles

These principles apply regardless of style — REST, GraphQL, or gRPC — because they concern how an interface behaves under the realities of networks, retries, and growth.

Resource Modeling

The hardest part of API design is deciding what the API is about. A poorly modelled API grows one bespoke endpoint per feature until nobody can predict what exists; a well-modelled one lets a developer guess the URL correctly on the first try. The difference is whether the design is organized around nouns that exist in the domain or around the verbs a particular screen happened to need.

A resource is a thing worth naming and addressing: an order, a user, a subscription. Once you have identified resources, the operations mostly fall out of the protocol — HTTP already supplies create, read, update, and delete, so you do not invent createOrder, getOrder, and cancelOrderV2.

The analogy is a filing cabinet versus a to-do list. A filing cabinet is organized by what things are, so anyone can find a document without being told where it went. A to-do list is organized by what someone wanted to do that week, and it makes sense only to its author.

VERB-ORIENTED (grows without bound)     RESOURCE-ORIENTED (predictable)

 POST /createOrder                       POST   /orders
 POST /getOrderById                      GET    /orders/{id}
 POST /cancelOrder                       DELETE /orders/{id}
 POST /getOrdersForUser                  GET    /users/{id}/orders
 POST /addItemToOrder                    POST   /orders/{id}/items
 POST /updateOrderShippingAddress        PATCH  /orders/{id}
 POST /getOrderInvoicePdf                GET    /orders/{id}/invoice

 7 endpoints, 7 conventions to learn     4 nouns, 1 convention to learn

Naming conventions that carry their weight:

• Plural nouns for collections (/orders, not /order), so a collection and a member read differently: /orders versus /orders/42.
• Nesting expresses containment, not navigation. /orders/42/items is fine because items belong to the order. /users/7/orders/42/items/9/product is not — stop nesting once the child has its own identity.
• No verbs in paths. The method is the verb.
• Lowercase, hyphenated for multi-word names: /shipping-addresses.

The awkward cases are actions that are not really CRUD: "cancel", "publish", "retry", "send". Two legitimate answers exist, and both are better than inventing a verb endpoint.

### Option A: model the state change as a field update
PATCH /orders/42
Content-Type: application/json

{"status": "cancelled"}
# Good when the action is genuinely just a state transition and the client
# is allowed to set that state directly.

### Option B: model the action itself as a subordinate resource
POST /orders/42/cancellations
Content-Type: application/json

{"reason": "customer_request"}
# Good when the action has its own data, can happen more than once, needs an
# audit trail, or has side effects beyond changing one field. You now have
# something addressable: GET /orders/42/cancellations/1

# Resource modelling shows up directly in the routing table. If the table is
# readable, the API is discoverable.
routes = [
    ("POST",   "/orders",                  create_order),
    ("GET",    "/orders",                  list_orders),        # filtered/paged
    ("GET",    "/orders/{order_id}",       get_order),
    ("PATCH",  "/orders/{order_id}",       update_order),       # partial update
    ("DELETE", "/orders/{order_id}",       cancel_order),
    ("GET",    "/orders/{order_id}/items", list_order_items),   # containment
    ("POST",   "/orders/{order_id}/items", add_order_item),
]
# A new engineer can predict every one of these without reading the docs.
# That predictability is the entire return on resource modelling.

Symptom	Underlying modelling problem
Endpoint count grows with every UI change	Modelled around screens, not domain nouns
Verbs in the path (/getUser)	Ignoring HTTP methods
Deeply nested URLs (4+ levels)	Nesting used for navigation, not containment
Same data returned by five endpoints	No canonical resource identified
Clients call 6 endpoints to render a page	Resources too granular for the use case

Key Takeaways

• Model the API around domain nouns; let HTTP methods supply the verbs.
• Plural collection names, containment-only nesting, no verbs in paths.
• Non-CRUD actions become either a state field update or a subordinate resource.
• Predictability is the goal: a developer should guess the URL correctly without the docs.

🧪 Practice

1. Redesign these as resource-oriented endpoints: POST /searchUsers, POST /markNotificationRead, GET /getUserSubscriptionStatus.
2. Model an API for a library system with books, copies, members, and loans. Decide which relationships justify nesting.
3. Interview: How would you model "send a password reset email" as a resource? (Hint: the action produces something with its own lifetime and identity — name that thing.)

Idempotency

Networks fail in an especially nasty way: the request succeeded but the response was lost. From the client's side that is indistinguishable from the request never arriving, so the client retries — and if the operation is not idempotent, the customer is charged twice. Idempotency is the property that makes retrying safe, and retrying is unavoidable in any distributed system.

An operation is idempotent if performing it once and performing it many times leave the system in the same state. Note the definition is about state, not about the response: returning 409 Conflict on a duplicate is still idempotent behaviour if the state changed only once.

The analogy is a light switch versus a doorbell. Pressing "set switch to on" five times leaves one light on. Pressing a doorbell five times rings it five times.

THE AMBIGUOUS FAILURE

  Client                         Server
    |--- POST /payments -------->|
    |                            |  charge succeeds, state written
    |         X  response lost   |
    |                            |
    |  ??? Did it work ???       |
    |                            |
    |--- POST /payments (retry)->|  <- without idempotency: charged twice
                                    with idempotency: recognized, replayed

HTTP defines which methods are expected to be idempotent, and clients, proxies, and libraries rely on it — an HTTP client will happily auto-retry a GET or PUT but should never auto-retry a POST.

Method	Safe (no change)	Idempotent	Notes
GET	Yes	Yes	Never use it to mutate state
HEAD	Yes	Yes	Headers only
PUT	No	Yes	Replaces the whole resource
DELETE	No	Yes	Deleting twice leaves it deleted
POST	No	No	The reason idempotency keys exist
PATCH	No	Not inherently	{"count": 5} is; {"count": "+1"} is not

For POST, the standard technique is an idempotency key: the client generates a unique value per logical operation and resends the same key on every retry. The server records the key with the outcome and replays that outcome instead of acting again.

def create_payment(request, db, gateway):
    key = request.headers.get("Idempotency-Key")
    if not key:
        return error(400, "Idempotency-Key header is required")

    # Atomic claim: INSERT fails if the key already exists. Checking first and
    # then inserting is a race — two concurrent retries would both pass.
    try:
        db.execute(
            "INSERT INTO idempotency (key, request_hash, status) VALUES (%s,%s,'IN_PROGRESS')",
            (key, hash_body(request.body)),
        )
    except UniqueViolation:
        row = db.fetch_one("SELECT * FROM idempotency WHERE key = %s", (key,))
        if row.request_hash != hash_body(request.body):
            # Same key, different payload: the client has a bug. Refuse loudly.
            return error(422, "Idempotency-Key reused with a different body")
        if row.status == "IN_PROGRESS":
            return error(409, "Request already in progress; retry shortly")
        return replay(row.response)          # identical response, no second charge

    result = gateway.charge(request.body)    # the real side effect, exactly once
    db.execute("UPDATE idempotency SET status='DONE', response=%s WHERE key=%s",
               (serialize(result), key))
    return result

Two design notes. Keys need a retention window — 24 hours is typical, long enough to cover client retries without growing the table forever. And the key must be generated by the client per logical operation, not per HTTP attempt: a key regenerated on each retry provides no protection at all.

Key Takeaways

• Retries are inevitable because a lost response is indistinguishable from a lost request.
• Idempotency is about resulting state, not about returning an identical response.
• GET, PUT, DELETE are idempotent by contract; POST needs an idempotency key.
• Claim the key atomically, store the outcome, replay it, and expire keys after a bounded window.

🧪 Practice

1. Classify as idempotent or not: PUT /users/7 {"name":"Ada"}, POST /orders/7/items, DELETE /sessions/abc, "increment view count".
2. Design the idempotency table: columns, primary key, TTL strategy, and what happens on a concurrent duplicate.
3. Interview: A client retries with the same idempotency key but a different request body. What should the API do and why? (Hint: consider which is more dangerous — silently honouring the first body, or the second.)

Statelessness

If a server remembers something about a client between requests, that client is now tied to that server. Losing the instance loses the state, a load balancer must keep routing them together, and you cannot scale in or deploy without disruption. Statelessness is the discipline of making every request carry everything needed to process it, which is what makes horizontal scaling and zero-downtime deploys straightforward.

Stateless does not mean the system has no state — it means the server process holds none between requests. The state moves somewhere designed to hold it: a database, a shared cache, or the request itself in the form of a signed token.

The analogy is a supermarket checkout versus a personal tailor. Any cashier can serve any customer because everything needed is on the conveyor belt. A tailor remembers your measurements, so you must return to the same one.

 STATEFUL (sticky)                       STATELESS

  [client A] --> [srv 1]  memory: A       [client A] --\
  [client B] --> [srv 2]  memory: B       [client B] ---> [LB] --> [any srv]
                                                                      |
  srv 1 dies -> A is logged out                                       v
  scale in    -> sessions lost                            [Redis / DB / token]
  deploy      -> everyone disrupted        any server serves any request

# Stateful: correctness depends on which instance answers.
UPLOAD_PROGRESS = {}                       # process memory

def upload_chunk_stateful(session_id, chunk):
    UPLOAD_PROGRESS.setdefault(session_id, []).append(chunk)   # lost on restart
    # A retry routed to another instance sees an empty list and corrupts the file.

# Stateless: the request identifies everything; state lives in shared storage.
def upload_chunk_stateless(upload_id, chunk_number, chunk, store, db):
    store.put(f"uploads/{upload_id}/{chunk_number}", chunk)    # shared object store
    db.execute("INSERT INTO upload_chunks (upload_id, n) VALUES (%s,%s) "
               "ON CONFLICT DO NOTHING", (upload_id, chunk_number))  # idempotent
    return {"received": chunk_number}
    # Any instance can serve any chunk, in any order, with retries.

Where to put the state is itself a trade-off:

Approach	Pros	Cons
Shared cache (Redis) session	Revocable instantly; small cookies	An extra network hop and a dependency
Signed token (JWT) in the client	No server lookup; scales trivially	Cannot revoke before expiry; size limits; leaks if stored badly
Database	Durable, queryable	Slowest; adds load to the primary store
Sticky sessions	No code changes	Uneven load, lost sessions, painful deploys

Statelessness has limits worth naming. WebSocket connections are inherently pinned to an instance (Chapter 2), and caching per-instance is a deliberate, useful form of state. The rule is not "never hold state" but "never hold state whose loss breaks correctness".

Key Takeaways

• Stateless servers let any instance serve any request, enabling scaling and safe deploys.
• State moves to a shared store or into a signed token carried by the client.
• Sticky sessions are a workaround with real costs: uneven load and disruptive deploys.
• The rule is not zero state — it is no state whose loss breaks correctness.

🧪 Practice

1. Identify the state in a multi-step checkout wizard and decide where each piece should live.
2. Compare Redis-backed sessions with JWTs for an app that must support immediate logout across devices.
3. Interview: Your API server caches user permissions in memory for 5 minutes. Is it still stateless? (Hint: ask what breaks if that cache vanishes versus what breaks if a session store vanishes.)

Error Handling and Status Codes

Errors are part of the API contract, not an afterthought. Clients need to answer three questions from a failure: was it my fault or yours?, should I retry?, and what exactly do I tell the user? An API that returns 200 OK with {"error": "..."} in the body answers none of them and defeats every proxy, monitor, and HTTP client in the path.

HTTP already encodes the first two answers in the status code, and the classes matter more than the individual numbers.

Class	Meaning	Client should retry?
2xx	Success	n/a
3xx	Redirection	Follow it
4xx	Client error	No — the same request will fail again
5xx	Server error	Yes, with backoff

CHOOSING A 4xx

  400 Bad Request          malformed syntax, unparseable body
  401 Unauthorized         not authenticated (you are nobody)  <- misnamed
  403 Forbidden            authenticated but not permitted (you are somebody,
                           just not the right somebody)
  404 Not Found            no such resource — or hiding one you may not see
  405 Method Not Allowed   right path, wrong verb
  409 Conflict             state conflict: duplicate, version mismatch
  410 Gone                 it existed and is permanently removed
  422 Unprocessable        syntactically valid, semantically wrong
  429 Too Many Requests    rate limited; pair with Retry-After

CHOOSING A 5xx

  500 Internal Error       unhandled — a bug; never leak the stack trace
  502 Bad Gateway          an upstream returned garbage
  503 Service Unavailable  overloaded or in maintenance; pair with Retry-After
  504 Gateway Timeout      an upstream did not answer in time

The status code is the machine-readable half. The body carries the rest, and standardizing its shape (RFC 9457, "Problem Details") saves every client from writing a bespoke parser.

HTTP/1.1 422 Unprocessable Content
Content-Type: application/problem+json

{
  "type":     "https://api.example.com/problems/validation-error",
  "title":    "Validation failed",
  "status":   422,
  "detail":   "The order could not be created because 2 fields are invalid.",
  "instance": "/orders",
  "trace_id": "01HQ8X4M2N",
  "errors": [
    {"field": "quantity", "code": "min_value",     "message": "must be at least 1"},
    {"field": "currency", "code": "unsupported",   "message": "'XBT' is not supported"}
  ]
}

# A stable error contract: machine-readable code, human-readable message,
# and a trace id that ties the client's report to your logs.
class ApiError(Exception):
    def __init__(self, status, code, message, retryable=False, **extra):
        self.status, self.code = status, code
        self.message, self.retryable = message, retryable
        self.extra = extra

def handle(exc, trace_id):
    if isinstance(exc, ApiError):
        body = {"code": exc.code, "message": exc.message,
                "retryable": exc.retryable, "trace_id": trace_id, **exc.extra}
        return response(exc.status, body)
    # Never let an unexpected exception leak internals to the caller: the
    # message may contain SQL, file paths, or credentials.
    log.exception("unhandled", trace_id=trace_id)
    return response(500, {"code": "internal_error",
                          "message": "An unexpected error occurred.",
                          "retryable": True, "trace_id": trace_id})

Three rules that prevent most client-side pain: stable machine-readable codes (insufficient_funds never changes, even when the sentence does), never leak internals in messages, and say when to retry — Retry-After on 429 and 503 turns guesswork into cooperation.

Key Takeaways

• The status class tells the client whose fault it is and whether retrying can help; 4xx no, 5xx yes.
• Never return 200 with an error body — it breaks proxies, monitoring, and client libraries.
• Provide a stable machine-readable code plus a human message and a trace id.
• Send Retry-After with 429 and 503, and never leak stack traces.

🧪 Practice

1. Pick status codes for: expired auth token, deleting an already-deleted order, quantity of -5, downstream payment provider timeout, unsupported Accept header.
2. Design the error body for a bulk endpoint where 3 of 100 items failed. Which status code applies?
3. Interview: When would you return 404 instead of 403 for a resource the caller may not access? (Hint: consider what the difference between the two responses tells an attacker.)

Pagination Strategies

Any collection that can grow must be paginated. Returning 2 million rows kills the server's memory, the network, and the client, and adding LIMIT later is a breaking change — so pagination is designed in from the first version, even when the table has ten rows.

There are two mainstream approaches, and the difference between them is one of the most commonly missed points in API design.

Offset pagination (?page=3&limit=20) is the familiar one: skip N rows, take M. It is simple, allows jumping to an arbitrary page, and shows total counts. It has two serious flaws at scale. OFFSET 100000 forces the database to scan and discard 100,000 rows, so deep pages get progressively slower. And because it addresses positions rather than items, concurrent inserts and deletes shift the window — items get skipped or repeated across pages.

Cursor (keyset) pagination addresses the last item seen instead: "give me 20 items after this one". The query becomes an indexed range scan with constant cost regardless of depth, and it is stable under concurrent writes.

 THE OFFSET DRIFT BUG

  t=0  page 1: LIMIT 3 OFFSET 0  -> [ I10, I9, I8 ]   (newest first)
  t=1  a new item I11 is inserted
  t=2  page 2: LIMIT 3 OFFSET 3  -> [ I8, I7, I6 ]
                                      ^^^ I8 seen twice, and nothing was
                                          skipped only by luck; a deletion
                                          would have skipped an item instead

 CURSOR PAGINATION IS STABLE

  page 1: WHERE id < MAX  ORDER BY id DESC LIMIT 3 -> [I10, I9, I8], cursor=I8
  page 2: WHERE id < I8   ORDER BY id DESC LIMIT 3 -> [I7, I6, I5]
          inserts at the top do not disturb the window

import base64, json

# OFFSET: simple, but O(offset) work and unstable under writes.
def list_offset(db, page, limit):
    rows = db.query("SELECT * FROM posts ORDER BY id DESC LIMIT %s OFFSET %s",
                    (limit, (page - 1) * limit))
    total = db.scalar("SELECT count(*) FROM posts")     # itself expensive at scale
    return {"items": rows, "page": page, "total": total}

# CURSOR: constant cost, stable ordering. The cursor is opaque on purpose so
# you can change its internals without breaking clients.
def encode_cursor(row):
    return base64.urlsafe_b64encode(
        json.dumps({"id": row.id, "created_at": row.created_at.isoformat()}).encode()
    ).decode()

def list_cursor(db, cursor=None, limit=20):
    if cursor:
        c = json.loads(base64.urlsafe_b64decode(cursor))
        # Compare the full sort key, not just the timestamp: ties on created_at
        # would otherwise skip or repeat rows.
        rows = db.query(
            "SELECT * FROM posts WHERE (created_at, id) < (%s, %s) "
            "ORDER BY created_at DESC, id DESC LIMIT %s",
            (c["created_at"], c["id"], limit + 1))     # fetch one extra
    else:
        rows = db.query("SELECT * FROM posts ORDER BY created_at DESC, id DESC "
                        "LIMIT %s", (limit + 1,))
    has_more = len(rows) > limit                       # the extra row tells us
    rows = rows[:limit]
    return {"items": rows,
            "next_cursor": encode_cursor(rows[-1]) if has_more and rows else None}

Aspect	Offset	Cursor
Deep-page performance	Degrades linearly	Constant
Stability under writes	Items skipped/duplicated	Stable
Jump to arbitrary page	Yes	No
Total count available	Yes (at a cost)	Usually not
Sort flexibility	Any column	Sort key must be in the cursor and indexed
Best for	Small, admin-style tables	Feeds, timelines, large collections

Rule of thumb: cursor pagination for anything user-facing or unbounded; offset only where page numbers are a genuine requirement and the table is small. Always enforce a maximum page size server-side — otherwise ?limit=1000000 becomes a denial-of-service vector.

Key Takeaways

• Design pagination in from v1; adding limits later is a breaking change.
• Offset pagination degrades with depth and drifts when the data changes.
• Cursor pagination is constant-cost and stable; the cursor must encode the full sort key.
• Make cursors opaque and always cap page size on the server.

🧪 Practice

1. Explain concretely how a user could miss an item while scrolling an offset-paginated feed during active posting.
2. Write the cursor payload and SQL predicate for sorting by (score DESC, id DESC).
3. Interview: A client needs "jump to page 500" on a billion-row table. What do you propose? (Hint: question the requirement first, then consider what an approximate answer would cost versus an exact one.)

Filtering and Sorting Conventions

Once a collection endpoint exists, clients immediately want a subset of it, ordered their way. If you do not define a convention, you will end up with /orders/pending, /orders/recent, and /orders/by-customer — the endpoint explosion resource modelling was meant to prevent. Filtering and sorting belong in the query string, because they describe which representation of the collection you want, not a different resource.

The design tension is between expressiveness and safety. A rich filter language is convenient for clients and dangerous for you: unbounded query flexibility means unbounded query cost, and a filter on an unindexed column is a full table scan triggered by a stranger.

### A conventional, boring, predictable collection query
GET /orders?status=pending&created_after=2026-01-01&sort=-created_at&limit=20

#   status=pending          exact match filter
#   created_after=...       range filter, explicit and readable
#   sort=-created_at        leading '-' means descending; comma-separate for
#                           multiple keys: sort=-created_at,id
#   limit=20                page size, capped server-side

### Multiple values: repeat the key or comma-separate — pick ONE and document it
GET /orders?status=pending&status=paid
GET /orders?status=pending,paid

# Allowlist everything. This is the difference between a filter API and an
# accidental SQL injection surface with a performance bug attached.
FILTERABLE = {                       # column -> (sql_op, parser)
    "status":         ("=",  str),
    "customer_id":    ("=",  int),
    "created_after":  (">=", parse_date),
    "created_before": ("<",  parse_date),
    "total_min":      (">=", Decimal),
}
SORTABLE = {"created_at", "total", "id"}      # all indexed; nothing else allowed
MAX_LIMIT = 100

def build_query(params):
    clauses, args = [], []
    for key, raw in params.items():
        if key in ("sort", "limit", "cursor"):
            continue
        if key not in FILTERABLE:
            raise ApiError(400, "unknown_filter", f"Unknown filter '{key}'")
        op, parse = FILTERABLE[key]
        clauses.append(f"{key.replace('_after','').replace('_before','')} {op} %s")
        args.append(parse(raw))               # parsed, then bound — never interpolated

    sort = params.get("sort", "-created_at")
    order = []
    for term in sort.split(","):
        desc = term.startswith("-")
        col = term.lstrip("-")
        if col not in SORTABLE:               # unindexed sort = table scan
            raise ApiError(400, "unsortable_field", f"Cannot sort by '{col}'")
        order.append(f"{col} {'DESC' if desc else 'ASC'}")
    order.append("id DESC")                   # tiebreaker: makes paging deterministic

    limit = min(int(params.get("limit", 20)), MAX_LIMIT)
    return clauses, order, limit, args

Concern	Convention
Exact match	?status=paid
Multiple values	?status=paid&status=pending (documented consistently)
Ranges	?created_after=, ?total_min= (explicit suffixes)
Sorting	?sort=-created_at,id (- prefix = descending)
Sparse fields	?fields=id,total,status (reduces payload size)
Free-text search	?q=... — a different mechanism; often a search index
Unknown parameter	Fail with 400, do not ignore silently

That last row matters more than it seems. Silently ignoring an unrecognized filter means a client typo (?statuss=paid) returns every order and the bug surfaces as a data leak or a runaway query rather than an error.

Always add a deterministic tiebreaker to the sort. Without it, rows with equal sort values come back in arbitrary order and pagination silently skips or duplicates them.

Key Takeaways

• Filtering and sorting go in the query string, not in new endpoints.
• Allowlist filterable and sortable fields; every sortable field must be indexed.
• Reject unknown parameters instead of ignoring them.
• Always append a unique tiebreaker to the sort so pagination is deterministic.

🧪 Practice

1. Design the query parameters for a product search with category, price range, in-stock flag, and sorting by price or rating.
2. Explain what goes wrong when sorting by a non-unique column without a tiebreaker while paginating.
3. Interview: A client wants arbitrary boolean filter expressions ((a AND b) OR c). How do you respond? (Hint: think about what you can guarantee about query cost, and what a purpose-built search layer offers that a database table does not.)

<a id="32-api-styles"></a>

3.2 API Styles

The same domain can be exposed through several interface styles, each optimizing for a different consumer and a different set of constraints.

REST

REST is the default style for public and general-purpose APIs, and its popularity comes from a simple bargain: instead of inventing a protocol, use the one the web already has. HTTP supplies methods, status codes, caching, content negotiation, and authentication, and every proxy, CDN, browser, and client library in existence already understands them.

REST — Representational State Transfer — describes an architectural style with a few constraints: a client-server split, statelessness, cacheability, a uniform interface, and layering. In practice "RESTful" usually means resource-oriented HTTP with sensible methods and status codes, which is what the previous subchapter described.

The constraint that gives the most value and is most often skipped is cacheability. Because a GET is safe and addressable by URL, any layer in the path can cache it, and that is why a well-designed REST API gets a CDN for free while an RPC-over-POST API does not.

### Cache-friendliness is a design decision, not an accident
GET /products/42
Accept: application/json

HTTP/1.1 200 OK
Cache-Control: public, max-age=60, stale-while-revalidate=300
ETag: "v7-9f2c"
Content-Type: application/json

{"id": 42, "name": "Desk lamp", "price": "39.00", "currency": "EUR"}

### A conditional request costs almost nothing when unchanged
GET /products/42
If-None-Match: "v7-9f2c"

HTTP/1.1 304 Not Modified
# No body transferred. The same ETag also powers optimistic concurrency:
# PUT /products/42 with If-Match: "v7-9f2c" fails with 412 if someone else
# changed it first — lost-update prevention for free.

# REST's weakness: the response shape is fixed by the server, so clients
# either over-fetch or make several calls.

# Over-fetching: the mobile list view needs 3 fields, gets 40.
GET /users/7            # -> full profile: bio, preferences, addresses, ...

# Under-fetching (the N+1 problem): rendering one screen takes many round trips.
GET /orders/42          # -> {"customer_id": 7, "item_ids": [1, 2, 3]}
GET /users/7            # -> customer details
GET /items/1            # -> item 1
GET /items/2            # -> item 2
GET /items/3            # -> item 3
# 5 sequential round trips. At 60 ms each, that is 300 ms of pure latency.
#
# Standard mitigations, in order of preference:
#   ?fields=id,name,price      sparse fieldsets, fixes over-fetching
#   ?expand=customer,items     server-side expansion, fixes under-fetching
#   a purpose-built endpoint   e.g. GET /orders/42/summary for one screen

Richardson's maturity model is a useful vocabulary for how far an API commits to the style:

Level	Description	Typical reality
0	One endpoint, everything is POST	RPC wearing an HTTP costume
1	Resources have distinct URLs	A common starting point
2	Proper methods and status codes	What most people mean by REST
3	Hypermedia links drive navigation (HATEOAS)	Rare outside specific domains

Level 2 is where the practical value sits. Level 3 (responses embedding links to available next actions) delivers real decoupling for long-lived, independently evolving clients, but most teams find the added complexity is not repaid.

Key Takeaways

• REST reuses HTTP's methods, status codes, and caching instead of inventing a protocol.
• Cacheable GETs are the biggest practical win: CDNs and proxies work without extra effort.
• ETag powers both cheap revalidation (304) and optimistic concurrency (If-Match, 412).
• Its weakness is fixed response shapes: over-fetching and N+1 round trips.

🧪 Practice

1. Design REST endpoints and cache headers for a product catalogue where prices change hourly and descriptions change rarely.
2. Show how ETag plus If-Match prevents two editors overwriting each other, including the status code on conflict.
3. Interview: A mobile team says your REST API is too chatty. Give three solutions and their trade-offs. (Hint: one changes the response, one adds an endpoint, one changes the style entirely.)

GraphQL

REST's fixed response shapes create a structural tension: the server decides what a resource looks like, but the client knows what it needs. Every new screen either over-fetches, makes extra round trips, or demands a new endpoint — and the endpoint count grows with the UI. GraphQL resolves that tension by letting the client specify the shape of the response.

A GraphQL API exposes a single endpoint and a strongly typed schema describing every field and relationship. The client sends a query naming exactly the fields it wants, including nested relationships, and receives exactly that structure in one round trip.

The analogy: REST is a set menu — the kitchen decides what comes on the plate. GraphQL is à la carte, where the diner composes the plate from a published menu.

# The schema is the contract: typed, introspectable, and self-documenting.
type Query {
  order(id: ID!): Order
}

type Order {
  id:        ID!
  total:     Money!
  status:    OrderStatus!
  customer:  User!          # a relationship the client may or may not traverse
  items:     [OrderItem!]!
}

type User { id: ID!, name: String!, email: String! }
type OrderItem { id: ID!, quantity: Int!, product: Product! }

# One request replaces the five REST round trips from the previous topic.
query OrderScreen {
  order(id: "42") {
    total
    status
    customer { name }           # only the field the screen renders
    items {
      quantity
      product { name, price }   # nested traversal, still one round trip
    }
  }
}

// The response mirrors the query exactly — no guessing, no unused fields.
{
  "data": {
    "order": {
      "total": "58.00",
      "status": "PAID",
      "customer": { "name": "Ada" },
      "items": [{ "quantity": 2, "product": { "name": "Lamp", "price": "29.00" } }]
    }
  }
  // Note: errors arrive in a top-level "errors" array, usually with HTTP 200.
  // That is the single biggest operational surprise when adopting GraphQL.
}

What you gain is real; what you give up is often underestimated.

Aspect	Benefit	Cost
Response shape	Client-specified; no over-fetching	Query cost is unpredictable for the server
Round trips	One request for a whole screen	Resolver fan-out can cause N+1 database queries
Schema	Typed, introspectable, self-documenting	Schema becomes a coordination point across teams
Endpoints	One	HTTP caching, status codes, and CDNs largely stop applying
Versioning	Additive evolution, deprecate fields	Old fields linger; nothing forces cleanup
Monitoring	Per-field usage analytics	"Slow endpoint" dashboards are meaningless — every query differs

# The N+1 trap is GraphQL's signature failure mode. A query asking for 100
# orders and each order's customer naively triggers 1 + 100 database queries.
#
# The fix is batching within a single request tick (the DataLoader pattern):
async def batch_load_users(user_ids):
    rows = await db.query("SELECT * FROM users WHERE id = ANY(%s)", (user_ids,))
    by_id = {r.id: r for r in rows}
    return [by_id.get(uid) for uid in user_ids]      # order must match input

user_loader = DataLoader(batch_load_users)
# Each resolver calls user_loader.load(id); the loader collects all ids raised
# during one tick and issues ONE query. 101 queries become 2.

Production GraphQL also needs guards a REST API gets for free: query depth limits and complexity scoring (a single deeply nested query can be a denial-of-service vector), and persisted queries (clients register queries by hash, restoring cacheability and preventing arbitrary queries from strangers).

Key Takeaways

• GraphQL lets clients specify the response shape, eliminating over-fetching and multi-round-trip screens.
• The schema is a typed, introspectable contract, and evolution is additive with field deprecation.
• You lose HTTP caching, meaningful status codes, and predictable server-side query cost.
• Production GraphQL requires DataLoader batching, depth/complexity limits, and persisted queries.

🧪 Practice

1. Write a GraphQL query fetching a user's last 5 orders with item names and total, and count the REST calls it replaces.
2. Explain how a malicious client could build an expensive query from a friendly-looking schema, and name two defences.
3. Interview: How do you cache a GraphQL API? (Hint: the unit that made REST cacheable was the URL — ask what plays that role here, and what has to happen at the field level instead.)

gRPC and Protocol Buffers

Internal service-to-service traffic has different priorities from public APIs. Nobody is exploring it in a browser, the clients are your own services, calls happen millions of times a day, and every millisecond and byte is multiplied across the fleet. In that setting, JSON over HTTP/1.1 is wasteful: text parsing burns CPU, field names are repeated in every message, and there is no enforced schema.

gRPC targets exactly this. You define services and messages in a .proto file, generate strongly typed client and server code in a dozen languages, and the calls travel as compact binary frames over HTTP/2 — which brings multiplexing over one long-lived connection and header compression (Chapter 2).

Protocol Buffers is the serialization format. Instead of transmitting field names, it transmits small integer field numbers, which is why the encoding is compact and why field numbers are permanent once assigned.

syntax = "proto3";
package orders.v1;

service OrderService {
  rpc GetOrder    (GetOrderRequest)    returns (Order);              // unary
  rpc ListOrders  (ListOrdersRequest)  returns (stream Order);       // server stream
  rpc RecordEvents(stream OrderEvent)  returns (RecordSummary);      // client stream
  rpc Watch       (stream WatchRequest) returns (stream OrderUpdate); // bidirectional
}

message Order {
  string id        = 1;   // the NUMBER is the wire identity, not the name.
  string status    = 2;   // Renaming a field is safe; reusing a number is not.
  int64  total_cents = 3;
  reserved 4;             // 4 was deleted; reserving stops accidental reuse
  repeated OrderItem items = 5;
}

# Generated code turns a network call into an ordinary typed method call.
import grpc
from orders.v1 import orders_pb2, orders_pb2_grpc

channel = grpc.insecure_channel("orders:50051")     # one long-lived HTTP/2 connection
stub = orders_pb2_grpc.OrderServiceStub(channel)

# Unary call with a deadline — gRPC propagates deadlines across hops, so a
# downstream service knows how much time is left rather than working on a
# request whose caller already gave up.
try:
    order = stub.GetOrder(orders_pb2.GetOrderRequest(id="42"), timeout=0.250)
    print(order.status)                              # typed field, no dict lookup
except grpc.RpcError as e:
    if e.code() == grpc.StatusCode.DEADLINE_EXCEEDED:
        ...                                          # typed status codes, not 500s

# Server streaming: results arrive incrementally over the same connection.
for order in stub.ListOrders(orders_pb2.ListOrdersRequest(customer_id="7")):
    process(order)                                   # no pagination machinery needed

PAYLOAD COMPARISON (same order record)

  JSON       {"id":"42","status":"PAID","total_cents":5800}      46 bytes
  Protobuf   0a 02 34 32 12 04 50 41 49 44 18 a8 2d              13 bytes

  ~3-4x smaller on the wire, and parsing is a byte-level decode instead of
  string scanning. Multiplied across millions of internal calls per day, this
  is real CPU and real bandwidth.

Dimension	gRPC	REST/JSON
Payload	Binary, compact	Text, human-readable
Schema	Required, code-generated	Optional (OpenAPI)
Transport	HTTP/2 only	Any HTTP version
Streaming	First-class, all four modes	SSE/WebSockets bolted on
Browser support	Needs a proxy (gRPC-Web)	Native
Debuggability	Needs tooling (grpcurl)	curl and a browser
Best for	Internal service-to-service	Public and browser-facing APIs

The compatibility rules are strict and worth memorizing: never change a field number, never change a field's type, and reserved any number you delete. Adding new optional fields is always safe, which is what makes proto schemas evolvable.

Key Takeaways

• gRPC is built for internal, high-volume service-to-service calls: binary payloads over multiplexed HTTP/2.
• Protobuf identifies fields by number, so numbers are permanent and deleted ones must be reserved.
• Code generation gives typed clients, typed status codes, and deadline propagation across hops.
• Browsers cannot speak gRPC directly, and debugging requires dedicated tooling.

🧪 Practice

1. Write a .proto service for a user profile API with one unary and one server-streaming method.
2. Explain exactly what breaks if a team renames field 3 versus reusing number 3 for a different field.
3. Interview: Why does gRPC deadline propagation matter more than a simple per-call timeout? (Hint: think about a chain of five services and what each is doing after the user has already seen an error.)

SOAP and Legacy RPC

You will meet SOAP. It runs payroll, banking, insurance, healthcare, telecom provisioning, and government integrations, and those systems are not going anywhere. Understanding why it looks the way it does turns "legacy nonsense" into a set of deliberate — if heavyweight — engineering decisions.

SOAP (Simple Object Access Protocol) is an XML messaging protocol from an era when the priority was enterprise interoperability with formal guarantees across mutually distrustful organizations. It brings a machine-readable contract (WSDL), and a family of WS-* standards for security, transactions, and reliable messaging — features REST leaves to the transport or the application.

Older still is plain RPC (CORBA, XML-RPC, Java RMI): call a remote function as if it were local. The idea is seductive and the flaw is famous — hiding the network makes developers forget it can be slow, partition, or fail entirely.

<!-- A SOAP request: the envelope carries headers (security, transactions)
     alongside the body (the actual call). -->
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope">
  <soap:Header>
    <wsse:Security>                      <!-- message-level, not transport-level -->
      <wsse:UsernameToken>
        <wsse:Username>svc_billing</wsse:Username>
        <wsse:Password Type="...#PasswordDigest">QZk...</wsse:Password>
      </wsse:UsernameToken>
    </wsse:Security>
  </soap:Header>
  <soap:Body>
    <GetOrderRequest xmlns="http://example.com/orders">
      <OrderId>42</OrderId>
    </GetOrderRequest>
  </soap:Body>
</soap:Envelope>

<!-- Errors are Faults inside a 500 response, not HTTP status codes. -->
<soap:Fault>
  <soap:Code><soap:Value>soap:Sender</soap:Value></soap:Code>
  <soap:Reason><soap:Text>Order 42 not found</soap:Text></soap:Reason>
</soap:Fault>

The crucial difference from REST is that security and reliability live in the message, not the transport. A WS-Security signature survives being forwarded through a queue, stored, and audited later; TLS protects only the hop it is on. For a payment instruction passing through four organizations, that distinction is the entire point.

Aspect	SOAP	REST
Format	XML only	Usually JSON
Contract	WSDL (mandatory, machine-readable)	OpenAPI (optional)
Transport	HTTP, SMTP, JMS, MQ	HTTP
Errors	SOAP Faults	HTTP status codes
Security	WS-Security (message-level)	TLS + OAuth (transport-level)
Transactions	WS-AtomicTransaction	Sagas, application-level
Payload size	Large (verbose XML)	Compact
Where you find it	Finance, healthcare, government, telecom	Everywhere else

# Integrating with SOAP in a modern stack: wrap it, do not spread it.
from zeep import Client                     # generates a client from the WSDL

class LegacyBillingGateway:
    """Anti-corruption layer: SOAP stays behind this boundary."""
    def __init__(self, wsdl_url):
        self._client = Client(wsdl_url)     # WSDL describes types and operations

    def get_invoice(self, invoice_id: str) -> Invoice:
        try:
            raw = self._client.service.GetInvoice(InvoiceId=invoice_id)
        except Fault as f:                  # SOAP Fault, not an HTTP status
            raise InvoiceNotFound(invoice_id) from f
        return Invoice(                     # translate into YOUR domain model
            id=raw.InvoiceId,
            total=Decimal(raw.TotalAmount),
            issued_at=raw.IssuedDate,
        )
# The rest of the codebase never imports zeep and never sees XML. When the
# legacy system is eventually replaced, one class changes.

Key Takeaways

• SOAP is XML messaging with a mandatory WSDL contract and WS-* standards for security and transactions.
• Its distinguishing feature is message-level (not transport-level) security and reliability, which survives forwarding and storage.
• Errors are SOAP Faults, not HTTP status codes; the transport need not even be HTTP.
• Isolate legacy integrations behind an anti-corruption layer that translates into your own domain model.

🧪 Practice

1. List three capabilities SOAP provides out of the box that a REST API must build itself.
2. Explain why message-level signing matters when a request passes through three intermediaries.
3. Interview: You must integrate a 15-year-old SOAP billing system into a new microservice platform. Describe your approach. (Hint: think about where the translation lives and what you would want to be true on the day the legacy system is finally switched off.)

Webhooks

Every API discussed so far requires the client to ask. But some events originate on the server's side — a payment settles, a video finishes encoding, a shipment is scanned — and the client has no way to know when. The naive answer is polling: ask every 30 seconds, waste 99% of the requests, and still learn about it up to 30 seconds late.

Webhooks invert the direction. The consumer registers a URL, and the provider sends an HTTP POST to it whenever a relevant event occurs. It is a "reverse API" — your API calls their API — and it turns a polling loop into a push notification.

 POLLING                              WEBHOOK

  You --> them  "anything?"  no        Them --> you  POST /hooks/payments
  You --> them  "anything?"  no                      {"type":"payment.succeeded"}
  You --> them  "anything?"  YES
                                       Zero wasted requests.
  2,880 requests/day per resource      Latency: near zero.
  Latency: up to the poll interval     Cost: you must run a public endpoint
                                             that is always available.

Receiving webhooks correctly is where most integrations go wrong, because the sender is an untrusted party on the public internet and delivery guarantees are weak. Four things are non-negotiable.

import hmac, hashlib, time

MAX_SKEW_SECONDS = 300

def receive_webhook(request, secret, db, queue):
    # 1. VERIFY THE SIGNATURE. Anyone can POST to a public URL; the signature
    #    is the only proof the payload came from the provider.
    ts  = request.headers["X-Timestamp"]
    sig = request.headers["X-Signature"]
    expected = hmac.new(secret.encode(),
                        f"{ts}.".encode() + request.raw_body,   # sign raw bytes
                        hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, sig):     # constant-time: avoids timing attacks
        return response(401)

    # 2. REJECT REPLAYS. A signature stays valid forever unless it is time-bound.
    if abs(time.time() - int(ts)) > MAX_SKEW_SECONDS:
        return response(401)

    event = json.loads(request.raw_body)

    # 3. DEDUPLICATE. Delivery is at-least-once: retries mean the same event
    #    arrives more than once, and events can arrive OUT OF ORDER.
    if not db.try_insert_event(event["id"]):       # unique constraint on event id
        return response(200, {"status": "duplicate_ignored"})

    # 4. ACKNOWLEDGE FAST, PROCESS LATER. Providers time out in seconds and
    #    disable endpoints that are slow or failing.
    queue.publish("webhook.received", event)
    return response(200)                            # ack within milliseconds

The provider side has its own obligations:

Provider responsibility	Why
Sign every payload	The consumer cannot otherwise trust the sender
Include a unique event id and timestamp	Enables deduplication and replay rejection
Retry with exponential backoff	Consumers have outages too
Cap retries, then dead-letter	Do not retry forever into a black hole
Provide a replay/history API	Lets consumers recover from a lost window
Document ordering guarantees	Almost always "none" — say so explicitly

Because webhooks require a publicly reachable, always-available endpoint, many providers now offer a pull alternative (an events endpoint the consumer polls with a cursor) for consumers who cannot host one. Offering both is increasingly the norm.

Key Takeaways

• Webhooks invert the direction: the provider POSTs to a consumer-registered URL when an event occurs.
• Always verify an HMAC signature over the raw body and reject stale timestamps.
• Delivery is at-least-once and unordered — deduplicate by event id and never assume sequence.
• Acknowledge immediately and process asynchronously; providers disable slow endpoints.

🧪 Practice

1. Implement signature verification and explain why signing the parsed JSON instead of the raw body is a vulnerability.
2. Design a retry schedule with backoff and a dead-letter policy for failed deliveries.
3. Interview: A webhook consumer was down for 6 hours. How do they recover without losing events? (Hint: consider what the provider must have kept, and what the consumer must be able to ask for.)

Choosing Between API Styles

Style debates are usually arguments about defaults dressed up as arguments about technology. The productive question is never "which is best?" but "who is the consumer, what do they need, and what constraints do they operate under?" — and the answer routinely differs for different parts of one system.

Three questions settle most cases:

1. Who calls it? A browser, a third-party developer, a mobile app, or another one of your services.
2. What is the interaction shape? Fetching data, invoking an operation, streaming, or reacting to events.
3. What constrains you? Public reach and debuggability, per-request cost at volume, or an existing enterprise contract.

DECISION SKETCH

  Is the caller a browser or a third-party developer?
    |-- yes: is the client's data need highly variable per screen?
    |          |-- yes -> GraphQL (add persisted queries + complexity limits)
    |          `-- no  -> REST (cacheable, debuggable, universally understood)
    `-- no (internal service-to-service):
               |-- high volume / low latency / streaming -> gRPC
               `-- low volume, simple                    -> REST is fine

  Does the OTHER side need to notify YOU?      -> Webhooks (or an event stream)
  Is there an existing enterprise contract?    -> SOAP, behind an adapter

Criterion	REST	GraphQL	gRPC	Webhooks	SOAP
Public/third-party API	Excellent	Good	Poor	Complement	Legacy
Browser support	Native	Native	Needs proxy	n/a	Awkward
Internal high-volume	Adequate	Adequate	Excellent	n/a	Poor
HTTP caching	Excellent	Poor	None	n/a	None
Payload efficiency	Medium	Medium	High	Medium	Low
Streaming	SSE only	Subscriptions	Native	n/a	No
Debuggability (curl)	Excellent	Good	Poor	Good	Poor
Schema enforcement	Optional	Built in	Built in	Provider-defined	Built in
Learning curve	Low	Medium	Medium	Low	High

A realistic large system uses several at once, and that is not inconsistency — it is matching the tool to the consumer:

 A TYPICAL COMPOSITE ARCHITECTURE

  [Browser / mobile] --GraphQL or REST--> [API gateway / BFF]
                                                 |
                                        gRPC     |     gRPC
                              +------------------+------------------+
                              v                  v                  v
                        [Orders svc]      [Inventory svc]     [Pricing svc]
                              |
                     webhooks | (outbound to partners)
                              v
                        [Partner endpoints]

  Public edge: REST/GraphQL — debuggable, cacheable, browser-friendly.
  Internal:    gRPC — fast, typed, streaming.
  Partners:    webhooks — they do not want to poll you.

Two closing cautions. Do not adopt GraphQL to fix a data-modelling problem — a badly modelled domain is just as badly modelled behind a schema. And do not use gRPC for a public API unless every consumer is technical and tooled for it; the debuggability cost lands on your support team.

Key Takeaways

• Choose by consumer and interaction shape, not by preference; one system legitimately uses several styles.
• REST for public and cacheable; GraphQL for variable client data needs; gRPC for internal high-volume and streaming.
• Webhooks complement any style — they solve provider-to-consumer notification, not data access.
• Style never fixes a modelling problem, and public APIs pay for undebuggable protocols in support cost.

🧪 Practice

1. Pick a style for each and justify: a public payments API, an internal fraud-scoring service called 50,000 times/second, a mobile app dashboard, notifying merchants of refunds.
2. Design the composite architecture for an e-commerce platform, naming the style at each boundary.
3. Interview: When is adopting GraphQL the wrong answer to "our API is too chatty"? (Hint: consider what else could be causing chattiness and what new operational burden GraphQL adds.)

<a id="33-api-lifecycle-management"></a>

3.3 API Lifecycle Management

An API is a promise to people you cannot deploy alongside; this subchapter is about keeping that promise while still being able to change the system.

API Versioning Strategies

Once someone else's code depends on your API, you have lost the ability to change it freely. But requirements keep moving, so you need a way to introduce change without breaking the clients you cannot upgrade — and versioning is that mechanism.

The first thing to internalize is that versioning is a last resort, not a routine step. Every live version is code you maintain, test, monitor, and debug. Two supported versions roughly double that surface; five make it unmanageable. The best change is one that needs no new version at all, which is what the next two topics are about.

When a breaking change is genuinely unavoidable, four placements are available.

### 1. URI path — most common, most visible, easiest to route and cache
GET /v2/orders/42

### 2. Query parameter — easy to add, easy to forget, messy to cache
GET /orders/42?version=2

### 3. Custom header — keeps URLs clean, invisible in a browser address bar
GET /orders/42
X-API-Version: 2

### 4. Content negotiation — the most "correct" by HTTP semantics, least used
GET /orders/42
Accept: application/vnd.example.order.v2+json

Strategy	Visible in logs/browser	Cache-friendly	Routing effort	Common in practice
URI path (/v2/)	Yes	Yes	Trivial	Very common
Query parameter	Yes	Poor (varies by param)	Easy	Uncommon
Custom header	Only in logs	Needs Vary	Easy	Common internally
Accept media type	Only in logs	Needs Vary	Moderate	Rare

URI path versioning wins on pragmatics: a developer can paste the URL into a browser, a CDN can cache each version separately, a gateway can route /v1/ and /v2/ to different deployments, and support conversations are unambiguous.

The second decision is granularity. Versioning the whole API means one breaking change forces every client onto a new version of everything. Versioning per resource limits the blast radius but multiplies bookkeeping.

# A practical middle ground: one global major version, with additive changes
# handled by feature negotiation rather than by a version bump.

@route("GET", "/v1/orders/{id}")
def get_order(request, order_id):
    order = repo.get(order_id)
    body = {
        "id":     order.id,
        "status": order.status,
        "total":  str(order.total),
    }
    # New fields are additive and therefore NOT a breaking change; they can
    # ship inside v1 without asking anyone to migrate.
    body["currency"] = order.currency

    # Behaviour changes that WOULD break someone go behind an explicit opt-in,
    # so existing clients keep the old semantics until they choose otherwise.
    if "itemized-totals" in request.headers.get("X-API-Features", ""):
        body["totals"] = {"net": str(order.net), "tax": str(order.tax)}
    return body

Two rules keep versioning sane. Never version for additive changes — new optional fields and new endpoints break nobody. And never let a version live forever without a stated end date; a version with no sunset is a permanent maintenance obligation you agreed to by accident.

Key Takeaways

• Versioning is a last resort; the cheapest change is one requiring no new version.
• URI path versioning is the pragmatic default: visible, cacheable, trivially routable.
• Version granularity trades blast radius against bookkeeping — a single major version plus additive change works for most APIs.
• Every version needs a stated sunset date from the day it ships.

🧪 Practice

1. Classify as breaking or non-breaking: adding an optional response field, renaming a field, making an optional request field required, adding a new enum value, tightening a validation rule.
2. Design the routing and deployment layout for running /v1 and /v2 side by side with shared business logic.
3. Interview: A client depends on a field you must remove for legal reasons. Walk through your plan. (Hint: separate "stop returning it" from "stop supporting the version", and think about who you must talk to first.)

Backward and Forward Compatibility

Versioning is expensive, so the real skill is avoiding the need for it. That means understanding precisely which changes break clients and which do not — and the answer is not symmetric, because there are two different directions of compatibility.

Backward compatibility: new server code works with old clients. This is the one everyone thinks of, and it is what lets you deploy without coordinating with consumers.

Forward compatibility: old code works with data produced by newer versions. This is the one people forget, and it depends mostly on how clients are written — specifically, whether they tolerate fields they do not recognize.

 WHY BOTH MATTER DURING A ROLLING DEPLOY

  t=0   all instances v1                     clients v1
  t=1   half v1, half v2  <-- BOTH versions serve traffic simultaneously
  t=2   all instances v2                     clients still v1 for weeks

  Backward compatibility keeps old clients working against v2 servers.
  Forward compatibility keeps v1 servers (and old clients) working when they
  receive v2-shaped data — from a peer instance, a queue, or a cache.
  A rolling deploy REQUIRES both, even if no client ever changes.

The rule that makes forward compatibility possible is tolerant reading (Postel's law applied to APIs): ignore unknown fields rather than rejecting them. A client that validates strictly against a closed schema breaks the moment the server adds a field — which means the server can never add a field.

// Fragile: strict validation makes every additive server change a breakage.
function parseOrderStrict(json) {
  const allowed = ["id", "status", "total"];
  for (const key of Object.keys(json)) {
    if (!allowed.includes(key)) {
      throw new Error(`Unexpected field: ${key}`);   // server adds "currency" -> boom
    }
  }
  return json;
}

// Tolerant: read what you need, ignore the rest, default what is missing.
function parseOrder(json) {
  return {
    id:       json.id,
    status:   json.status,
    total:    json.total,
    currency: json.currency ?? "EUR",   // tolerate absence: works against old servers
                                        // and against new ones
  };
  // Unknown fields are simply not read. The server is now free to add fields.
}

Change	Backward compatible?	Notes
Add an optional response field	Yes	Requires tolerant clients
Add an optional request field	Yes	Must have a sensible default
Add a new endpoint	Yes	Nobody is calling it yet
Add a new enum value	No, usually	Old clients may not handle it — plan an "unknown" branch
Remove or rename a field	No	Deprecate first, remove later
Make an optional field required	No	Existing requests start failing
Narrow a type (string -> int)	No	Parsing breaks
Widen a type (int -> string)	No	Also breaks: clients parse the old type
Tighten validation	No	Previously accepted requests now fail
Loosen validation	Yes	Nothing previously valid becomes invalid
Change an error status code	No	Clients branch on status
Change default sort order or paging	No	Silent behaviour change — the worst kind

Two entries deserve emphasis. New enum values are a breaking change in disguise: an old client with a switch over three statuses will hit its default branch — or crash — when a fourth appears. Document from v1 that clients must handle unknown enum values gracefully. And silent behaviour changes (a different default page size, a changed sort order, a newly applied filter) are worse than outright errors, because they corrupt results without anyone noticing.

Key Takeaways

• Backward compatibility: new servers serve old clients. Forward compatibility: old code tolerates new data.
• Rolling deploys require both, even when no client changes.
• Tolerant reading — ignore unknown fields, default missing ones — is what makes additive change possible.
• New enum values and silent behaviour changes are breaking changes that look harmless.

🧪 Practice

1. Take an API response you know and list five additive changes and three breaking ones.
2. Write a tolerant client parser that handles an unknown enum value without crashing, and say what it should do with the record.
3. Interview: Why is widening a field's type also a breaking change? (Hint: think about the client's deserializer, not the server's serializer.)

Schema Evolution

Compatibility rules are easy to state and hard to enforce by discipline alone. Schema evolution is the practice of making those rules mechanical — encoding the contract in a schema file that tooling can check, so an incompatible change fails a build rather than a customer's integration.

This matters most where the producer and consumer are deployed independently and data outlives the code that wrote it: message queues, event streams, and stored records. A JSON blob written today may be read in two years by code that does not exist yet.

Each serialization format handles evolution differently, and the differences are practical rather than academic.

Format	Schema	Evolution mechanism	Reader needs writer's schema?
JSON	Optional (JSON Schema)	Convention and discipline	No
Protobuf	Required	Field numbers; unknown fields preserved	No
Avro	Required	Reader and writer schemas resolved at read time	Yes
Thrift	Required	Field ids, like Protobuf	No

// Protobuf's evolution rules, made explicit.
message Order {
  string id           = 1;
  string status       = 2;
  int64  total_cents  = 3;

  reserved 4;                  // was `legacy_discount`; number retired forever
  reserved "legacy_discount";  // and the NAME is retired too, so nobody revives it

  string currency     = 5;     // ADDING a new optional field: always safe
  repeated string tags = 6;    // repeated fields default to empty, never null
}

// SAFE:      add a field with a fresh number; rename a field (name is cosmetic)
// UNSAFE:    change a number; change a type; reuse a retired number
// Old readers keep the unknown fields intact when re-serializing, which means a
// v1 service can round-trip a v2 message without destroying data.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Order",
  "type": "object",
  "required": ["id", "status", "total"],
  "properties": {
    "id":     {"type": "string"},
    "status": {"type": "string", "enum": ["pending", "paid", "cancelled"]},
    "total":  {"type": "string", "pattern": "^[0-9]+\\.[0-9]{2}$"}
  },
  "additionalProperties": true
}

That last line is the whole game for JSON. additionalProperties: true makes consumers tolerant, so the producer can add fields. Setting it to false locks the schema and turns every future addition into a breaking change.

# Make compatibility a build-time check, not a code-review hope.
# Run in CI on every pull request that touches a schema:
#
#   buf breaking --against '.git#branch=main'          # protobuf
#   sq compatibility check --level BACKWARD            # schema registry
#
def check_backward_compatible(old_schema, new_schema):
    errors = []
    for name, field in old_schema.fields.items():
        new = new_schema.fields.get(name)
        if new is None:
            errors.append(f"{name}: removed (breaks existing readers)")
        elif new.type != field.type:
            errors.append(f"{name}: type {field.type} -> {new.type}")
        elif new.required and not field.required:
            errors.append(f"{name}: became required")
    for name, field in new_schema.fields.items():
        if name not in old_schema.fields and field.required:
            errors.append(f"{name}: new required field breaks old writers")
    return errors        # non-empty -> fail the build

A schema registry operationalizes this for event streams: producers register schemas, the registry rejects incompatible versions, and consumers fetch the schema they need by id. Compatibility modes are configurable — BACKWARD (new consumers read old data), FORWARD (old consumers read new data), or FULL (both), with FULL being the right default when producers and consumers deploy on independent schedules.

Key Takeaways

• Schema evolution turns compatibility rules into automated checks instead of review discipline.
• Protobuf field numbers are permanent; reserve retired numbers and names.
• For JSON, additionalProperties: true is what keeps consumers tolerant and producers free to add fields.
• Use a schema registry with FULL compatibility when producers and consumers deploy independently.

🧪 Practice

1. Evolve a User protobuf message to split name into first_name and last_name without breaking anyone. Show both stages.
2. Explain why Avro requires the writer's schema at read time and what infrastructure that implies.
3. Interview: An event schema in production has a field nobody uses but everyone is afraid to remove. How do you remove it safely? (Hint: you need evidence before action — where does that evidence come from?)

Deprecation Policies

Removing something from an API is not a technical act, it is a coordination problem. The code change takes minutes; getting dozens of teams or thousands of external developers to stop calling an endpoint takes months. A deprecation policy is the published process that makes that coordination predictable instead of adversarial.

Without one, two failure modes appear. Either nothing is ever removed — and you maintain endpoints nobody has used since 2019 — or things are removed abruptly and you break paying customers. A policy replaces both with a schedule everyone can plan around.

A STANDARD DEPRECATION TIMELINE

  T-0        Announce.  Changelog, email, docs banner. Ship the replacement
             FIRST — never deprecate before the alternative exists.
  T-0..      Signal in-band. Deprecation and Sunset headers on every response;
             log every call with the caller's identity.
  T+1 month  Contact the top callers directly. Offer migration help.
  T+3 months Brownouts: return errors for short, announced windows (e.g. 30
             minutes) so silent integrations discover the problem while
             someone is watching.
  T+6 months Sunset. Return 410 Gone with a link to the migration guide.
  T+9 months Delete the code.

  Timeline scales with audience: internal services can move in weeks; a public
  API with enterprise contracts may need 12-24 months.

### Deprecation is announced in-band, on every response, per RFC 8594 / 9745
GET /v1/orders/42

HTTP/1.1 200 OK
Deprecation: true
Sunset: Sat, 01 Aug 2026 00:00:00 GMT
Link: <https://docs.example.com/migrate/v2>; rel="deprecation"; type="text/html"
Warning: 299 - "Deprecated API: /v1/orders is removed on 2026-08-01. Use /v2/orders."

### After sunset — 410 Gone, not 404: the difference is "it is gone" versus
### "it never existed", and only one of those tells the caller what to do.
HTTP/1.1 410 Gone
Content-Type: application/problem+json

{"code": "endpoint_sunset",
 "message": "GET /v1/orders was removed on 2026-08-01.",
 "migration_guide": "https://docs.example.com/migrate/v2"}

# You cannot deprecate what you cannot measure. Usage telemetry per caller is
# the difference between a confident removal and a hopeful one.
def deprecation_middleware(handler, endpoint, sunset_date):
    def wrapped(request):
        metrics.increment("api.deprecated_call", tags={
            "endpoint":  endpoint,
            "client_id": request.auth.client_id,     # WHO is still calling
            "version":   request.headers.get("User-Agent", "unknown"),
        })
        response = handler(request)
        response.headers["Deprecation"] = "true"
        response.headers["Sunset"] = sunset_date.strftime("%a, %d %b %Y %H:%M:%S GMT")
        return response
    return wrapped

# The removal decision then becomes evidence-based:
#   "3 clients, 40 calls/day, all internal"        -> remove next month
#   "180 clients, 2M calls/day, 12 enterprise"     -> the timeline is longer,
#                                                     and it starts with phone calls

The single most important rule: ship the replacement before announcing the deprecation. Telling people to stop using something that has no alternative generates resentment, not migrations.

Key Takeaways

• Deprecation is a coordination problem with a published schedule, not a code change.
• Signal in-band with Deprecation, Sunset, and Link headers on every response.
• Instrument usage per caller — removal decisions must be evidence-based.
• Ship the replacement first, use brownouts to flush out silent callers, and return 410 Gone after sunset.

🧪 Practice

1. Write a deprecation notice for an endpoint being replaced, including headers, changelog entry, and timeline.
2. Design the telemetry needed to answer "can we remove this endpoint?" in one query.
3. Interview: Your largest customer refuses to migrate before sunset. What do you do? (Hint: separate what is technically possible from what is commercially wise, and consider options between "break them" and "support it forever".)

API Documentation and Contracts

An undocumented API is effectively private: consumers cannot use it without reading your source or guessing, and every question becomes a support ticket. But hand-written documentation has a fatal flaw — it drifts. Within months, the docs describe an API that no longer exists, which is worse than no docs at all because it is confidently wrong.

The fix is to make the documentation a machine-readable contract that lives with the code, is validated in CI, and generates both the human-readable docs and the client libraries. The contract stops being a description of the API and becomes part of it.

OpenAPI is the standard for HTTP APIs (the equivalents being .proto files for gRPC and the SDL for GraphQL).

openapi: 3.1.0
info:
  title: Orders API
  version: 1.4.0
paths:
  /orders/{orderId}:
    get:
      operationId: getOrder            # becomes the generated client's method name
      summary: Retrieve a single order
      parameters:
        - name: orderId
          in: path
          required: true
          schema: {type: string, pattern: '^[0-9]+$'}
      responses:
        '200':
          description: The order
          content:
            application/json:
              schema: {$ref: '#/components/schemas/Order'}
        '404':
          description: No such order
          content:
            application/problem+json:
              schema: {$ref: '#/components/schemas/Problem'}
components:
  schemas:
    Order:
      type: object
      required: [id, status, total]
      properties:
        id:     {type: string, example: "42"}
        status: {type: string, enum: [pending, paid, cancelled]}
        total:  {type: string, example: "58.00"}
        currency: {type: string, default: "EUR"}   # optional: additive, safe

One artifact drives many outputs, which is what makes the contract worth maintaining:

                       +--> interactive docs (Swagger UI, Redoc)
                       |
                       +--> client SDKs in N languages (generated)
   openapi.yaml -------+
                       +--> server request/response validation (middleware)
                       |
                       +--> mock server for consumer teams before you build
                       |
                       +--> CI diff check: does this PR break the contract?

There are two ways to keep the contract honest, and the choice matters:

Approach	How it works	Trade-off
Design-first	Write the spec, review it, generate stubs, then implement	Better design, parallel client work; needs discipline to keep in sync
Code-first	Annotate handlers; generate the spec from code	Never drifts from implementation; design decisions happen implicitly, one endpoint at a time

# Code-first with runtime validation: the framework derives the schema from
# type annotations, so the docs cannot drift from the handler signature.
from pydantic import BaseModel, Field

class OrderResponse(BaseModel):
    id: str
    status: Literal["pending", "paid", "cancelled"]
    total: str = Field(pattern=r"^\d+\.\d{2}$")
    currency: str = "EUR"

@app.get("/orders/{order_id}", response_model=OrderResponse, responses={404: {...}})
def get_order(order_id: str) -> OrderResponse:
    ...
# The OpenAPI document, the docs page, and the response validation all come
# from this one declaration. Changing the model changes all three at once.

Whatever the approach, documentation needs more than a schema dump. The parts consumers actually need — and that generators cannot produce — are authentication setup, a copy-pasteable first request, error semantics, rate limits, pagination conventions, idempotency rules, and a changelog.

Key Takeaways

• Hand-written docs drift; a machine-readable contract validated in CI does not.
• One OpenAPI (or .proto, or SDL) artifact generates docs, SDKs, validation, mocks, and breaking-change checks.
• Design-first gives better design; code-first guarantees accuracy — pick deliberately.
• Generated reference is not enough: add auth setup, a working example, error semantics, limits, and a changelog.

🧪 Practice

1. Write an OpenAPI fragment for a paginated list endpoint with filters and a cursor.
2. List everything a new developer needs to make their first successful call to an API you know, and check whether its docs cover all of it.
3. Interview: How do you keep documentation in sync with implementation across 40 services? (Hint: think about what has to happen automatically, and at which point in the pipeline a mismatch should fail.)

Contract Testing

Integration testing across services does not scale. Spinning up twelve services to test one is slow, flaky, and expensive, so teams mock their dependencies instead — and mocks lie. A mock encodes what the consumer believes the provider does, and the day the provider changes, every test still passes and production breaks.

Contract testing closes that gap. The consumer's expectations are recorded as a machine-readable contract, and the provider verifies against it in its own CI. Nobody has to run both services together, but both are checked against the same shared truth.

 INTEGRATION TESTS              MOCKS                  CONTRACT TESTS

 [consumer]                     [consumer]             [consumer]
     |                              |                       |
  real call                    fake response          records expectation
     |                              |                       |
 [provider]                    (nothing)              [ contract file ]
                                                            |
 accurate but slow,            fast but can              verified in
 flaky, hard to run            silently diverge      provider's own CI
                                                     -> fast AND accurate

The flow is consumer-driven: the consumer writes tests against a stub whose interactions are captured into a pact; the provider replays those interactions against its real implementation.

// CONSUMER side: this test both verifies consumer code and emits the contract.
const { PactV3, MatchersV3: M } = require("@pact-foundation/pact");

const provider = new PactV3({ consumer: "web-checkout", provider: "orders-api" });

describe("fetching an order", () => {
  it("returns the fields checkout needs", async () => {
    provider
      .given("order 42 exists and is paid")      // provider state, set up on its side
      .uponReceiving("a request for order 42")
      .withRequest({ method: "GET", path: "/orders/42" })
      .willRespondWith({
        status: 200,
        headers: { "Content-Type": "application/json" },
        // Match on TYPE and shape, not exact values — otherwise the contract
        // breaks whenever real data changes, which teaches people to ignore it.
        body: {
          id:     M.string("42"),
          status: M.regex("pending|paid|cancelled", "paid"),
          total:  M.decimal(58.0),
        },
      });

    await provider.executeTest(async (mock) => {
      const order = await fetchOrder(mock.url, "42");   // real consumer code
      expect(order.total).toBe(58.0);
    });
  });
});
// Output: a pact file listing exactly the fields this consumer depends on.

# PROVIDER side: replay every consumer's pact against the real service in CI.
#
#   pact-verifier --provider-base-url=http://localhost:8080 \
#                 --pact-broker-url=https://pacts.internal \
#                 --provider=orders-api --publish-verification-results
#
# Provider states are the setup hooks the contract refers to:
@provider_state("order 42 exists and is paid")
def setup_paid_order():
    db.insert_order(id="42", status="paid", total="58.00")

# The provider's build now fails if it removes `status`, renames `total`, or
# changes the response code — BEFORE the change reaches the consumer.

The property that makes this valuable: the contract records only the fields a consumer actually uses. The provider learns exactly what it may change freely and what is load-bearing, which is precisely the information missing from the deprecation discussion earlier.

Test type	Speed	Accuracy	Catches
Unit	Fast	n/a	Logic errors in one component
Contract	Fast	High	Interface mismatches between services
Integration	Slow	High	Wiring, config, real infrastructure
End-to-end	Slowest	Medium	Whole-journey failures; flaky at scale

Contract testing does not replace the others; it removes the need for most cross-service integration tests, which are the slow and flaky ones.

Key Takeaways

• Mocks drift silently; contract tests verify the provider against the consumer's recorded expectations.
• Consumer-driven contracts capture only the fields consumers actually depend on.
• Providers verify all consumer pacts in their own CI, so breakage is caught before deployment.
• Match on types and shapes, not exact values, or the contract becomes noise people learn to ignore.

🧪 Practice

1. Write a consumer contract for an endpoint returning a user profile where the consumer uses only id and display_name.
2. Explain how contract tests would have prevented a specific outage you have seen or read about.
3. Interview: A provider wants to remove a field. How do contract tests tell them whether it is safe? (Hint: think about what the set of published pacts collectively represents.)

<a id="34-api-gateways-and-edge-services"></a>

3.4 API Gateways and Edge Services

The gateway is the single front door to a fleet of services; what it takes on determines how much every service behind it can stop worrying about.

Gateway Responsibilities

As soon as a system has more than a handful of services, a set of concerns appears in every one of them: authentication, TLS, rate limiting, logging, CORS, request size limits, retries. Implementing those twenty times produces twenty subtly different behaviours and twenty places to fix a bug. An API gateway is a specialized reverse proxy (Chapter 2) that handles these cross-cutting concerns once, at the edge.

The analogy is a building's reception desk. Visitors are checked in, badged, directed to the right floor, and logged — once, at the entrance — rather than each department running its own security check.

 WITHOUT A GATEWAY                    WITH A GATEWAY

  [client] --> [orders svc]  auth?     [client]
  [client] --> [users svc]   auth?         |
  [client] --> [search svc]  auth?     [API gateway]  <- TLS, authn, rate limit,
                                            |            routing, logging, CORS
  Every service reimplements:               |
   TLS, authn, rate limits, CORS,     +-----+-----+-----+
   logging, request limits            v           v     v
                                  [orders]   [users]  [search]
  Each service is publicly              (private network, trust the gateway's
  exposed and independently             assertions, focus on business logic)
  attackable

Responsibility	What the gateway does
TLS termination	One certificate store, one renewal process
Authentication	Validate tokens, reject anonymous traffic early
Authorization (coarse)	Scope and role checks; fine-grained stays in services
Rate limiting and quotas	Per client, per plan, per endpoint
Routing	Path, header, or weight-based to the right service
Request/response transform	Header injection, protocol translation (REST to gRPC)
Observability	One correlation id, uniform access logs and metrics
Resilience	Timeouts, retries, circuit breaking, load shedding
Security hygiene	Body size limits, CORS, WAF rules, IP allow/deny

# A gateway route declaration: policy as configuration, not as code in
# every service.
routes:
  - name: orders
    match:
      path: /v1/orders/**
      methods: [GET, POST, PATCH, DELETE]
    upstream: http://orders.internal:8080
    policies:
      - jwt_auth:      {issuer: "https://auth.example.com", audience: "api"}
      - rate_limit:    {key: "$jwt.sub", limit: 1000, window: 60s}
      - timeout:       {request: 3s}          # bound every upstream call
      - retry:         {attempts: 2, on: [502, 503, 504], budget: 10%}
        # ^ retry budget caps retries at 10% of traffic: without it, a partial
        #   outage turns into a retry storm that finishes the service off
      - cors:          {origins: ["https://app.example.com"], max_age: 600}
      - request_limit: {max_body: 1MB}
      - strip_headers: [X-Internal-Trace, X-Debug]   # never leak internals out

The gateway's great danger is becoming a distributed monolith's control plane: business logic creeps into gateway configuration, every team needs a change to it to ship anything, and the config becomes an undocumented, untestable program owned by nobody. The discipline is a hard line — the gateway handles concerns that are identical for all traffic; anything requiring domain knowledge belongs in a service.

It is also a single point of failure by construction. Run it redundantly across AZs, keep it stateless, load-test it as its own service, and make sure its own failure modes (config push, certificate expiry) are as well-drilled as any backend's.

Key Takeaways

• A gateway centralizes cross-cutting concerns so services do not each reimplement them.
• Typical duties: TLS, authn, rate limiting, routing, observability, timeouts, and security hygiene.
• Keep business logic out of gateway config, or it becomes an untestable shared program.
• The gateway is a single point of failure — make it redundant, stateless, and load-tested.

🧪 Practice

1. List which of these belong in the gateway and which in the service: JWT signature validation, "can this user edit this order?", request size limits, currency conversion, CORS.
2. Design gateway routes for three services with different auth requirements (public, authenticated, internal-only).
3. Interview: What happens to your system when the gateway is down, and how do you reduce that risk? (Hint: consider both redundancy and whether every class of traffic must pass through it.)

Authentication at the Edge

Authentication is expensive and easy to get wrong, so doing it in every service is both wasteful and risky: twenty implementations mean twenty chances to skip a signature check. Terminating authentication at the edge means unauthenticated traffic never reaches your services at all, and the validation logic exists once.

The pattern has three steps: the gateway validates the credential, it translates it into a trusted internal representation, and services trust that representation because they are only reachable through the gateway.

 EDGE AUTHENTICATION FLOW

  [client] --Authorization: Bearer eyJ...--> [gateway]
                                                 |
                        1. verify signature against the issuer's public key
                           (cached JWKS — no network call per request)
                        2. check exp, iss, aud, and required scopes
                        3. reject early: 401 if invalid, 403 if scope missing
                                                 |
                        4. strip the client token, inject verified identity
                                                 v
                              X-User-Id: 7
                              X-Scopes: orders:read orders:write
                              X-Trace-Id: 01HQ8X4M2N
                                                 |
                                                 v
                                          [orders service]
                    trusts these headers ONLY because the network guarantees
                    no other path exists — plus mTLS between gateway and service

# Gateway-side validation. The two costly operations are both cached.
import jwt
from cachetools import TTLCache

JWKS = TTLCache(maxsize=8, ttl=3600)     # public keys change rarely

def authenticate(request):
    header = request.headers.get("Authorization", "")
    if not header.startswith("Bearer "):
        raise Unauthorized("missing bearer token")     # 401
    token = header[7:]

    try:
        unverified = jwt.get_unverified_header(token)
        key = jwks_for(unverified["kid"])              # key id -> public key, cached
        claims = jwt.decode(
            token, key,
            algorithms=["RS256"],       # PIN the algorithm. Accepting the token's
                                        # own "alg" enables the classic 'alg: none'
                                        # and HMAC-with-public-key attacks.
            audience="api.example.com", # who the token was minted for
            issuer="https://auth.example.com",
        )                               # exp and nbf are checked by the library
    except jwt.InvalidTokenError as e:
        raise Unauthorized(str(e))                     # 401

    if "orders:read" not in claims.get("scope", "").split():
        raise Forbidden("scope orders:read required")  # 403, not 401

    return claims

The critical design question is what the gateway can and cannot decide.

Decision	Where it belongs	Why
Is this token authentic and unexpired?	Gateway	Identical for all traffic
Does the caller have the orders:write scope?	Gateway	Coarse, encoded in the token
Is the caller allowed to edit order 42?	Service	Needs domain data the gateway lacks
Is this field visible to this role?	Service	Needs the response, not just the request

Coarse authorization at the edge, fine-grained authorization in the service. A gateway that starts querying the orders database to make decisions has absorbed domain logic and become a bottleneck.

Two failure modes to design around. Revocation: a signed JWT stays valid until it expires, so immediate logout needs either short lifetimes with refresh tokens, or a revocation list checked at the edge. And the confused deputy: if a service can be reached without passing through the gateway, those injected identity headers become a trivial impersonation vector — enforce it with network policy and mTLS, never by convention.

Key Takeaways

• Validate credentials once at the edge; unauthenticated traffic never reaches services.
• Strip the client token and inject a verified internal identity that services can trust.
• Coarse checks (authentic, scoped) at the gateway; resource-level authorization in the service.
• Pin the signing algorithm, cache JWKS, and make the gateway the only network path to services.

🧪 Practice

1. Explain the difference between 401 and 403 and give a request that should receive each.
2. Design immediate logout for JWT-based auth without checking a database on every request.
3. Interview: Your gateway injects X-User-Id and services trust it. What could go wrong? (Hint: enumerate every network path that reaches a service and ask what stops a caller from setting that header themselves.)

Rate Limiting and Quotas

An API without limits is an API whose availability is controlled by its least careful client. One buggy retry loop, one aggressive scraper, or one enthusiastic customer can consume the capacity everyone else depends on. Rate limiting converts that shared risk into a bounded, per-client contract — and it is as much a fairness and business mechanism as a technical one.

Two related concepts are worth separating. A rate limit bounds short-term speed ("100 requests per minute") and protects capacity. A quota bounds longer-term consumption ("1 million requests per month") and usually encodes a commercial plan. Most APIs need both.

The algorithms themselves — token bucket, leaky bucket, sliding window — are covered in Chapter 6.3. What matters here is the interface design around them.

CHOOSING THE LIMIT KEY (this decision matters more than the algorithm)

  by API key / client_id   fair per customer; the default for authenticated APIs
  by user id               prevents one user of a shared client hogging it
  by IP address            the only option for unauthenticated traffic, but
                           NAT and mobile carriers put thousands behind one IP
  by endpoint              expensive endpoints deserve their own, tighter limit
  by tenant                multi-tenant fairness in B2B products

  Best practice: layer them. A per-IP limit on the login endpoint, a per-client
  limit overall, and a tighter per-endpoint limit on expensive operations.

# Distributed limiting: the counter must be shared, or N gateway instances
# each allow the full limit and the effective limit is N times too high.
LUA_TOKEN_BUCKET = """
local key, rate, burst, now = KEYS[1], tonumber(ARGV[1]), tonumber(ARGV[2]), tonumber(ARGV[3])
local b = redis.call('HMGET', key, 'tokens', 'ts')
local tokens = tonumber(b[1]) or burst
local ts     = tonumber(b[2]) or now
tokens = math.min(burst, tokens + (now - ts) * rate)   -- refill by elapsed time
local allowed = tokens >= 1
if allowed then tokens = tokens - 1 end
redis.call('HMSET', key, 'tokens', tokens, 'ts', now)
redis.call('EXPIRE', key, 120)                          -- self-cleaning keys
return {allowed and 1 or 0, math.floor(tokens)}
"""
# Executed as one Lua script so check-and-decrement is atomic; doing it as
# separate GET and SET calls is a race that leaks capacity under concurrency.

def check_limit(redis, client_id, rate_per_sec, burst):
    allowed, remaining = redis.eval(LUA_TOKEN_BUCKET, 1,
                                    f"rl:{client_id}", rate_per_sec, burst, time.time())
    return bool(allowed), remaining

Communicating the limit is as important as enforcing it. A client that cannot see its budget can only discover it by hitting the wall.

### Every response carries the client's current standing
HTTP/1.1 200 OK
RateLimit-Limit: 1000
RateLimit-Remaining: 847
RateLimit-Reset: 34                 # seconds until the window refills

### And a rejection says exactly how long to wait — never make clients guess
HTTP/1.1 429 Too Many Requests
Retry-After: 34
Content-Type: application/problem+json

{"code": "rate_limit_exceeded",
 "message": "Rate limit of 1000 requests/minute exceeded.",
 "retry_after_seconds": 34,
 "docs": "https://docs.example.com/rate-limits"}

Design decision	Guidance
Limit key	Prefer client/user identity; IP only for anonymous traffic
Burst allowance	Allow short bursts — real traffic is not smooth
Response headers	Always expose limit, remaining, and reset
Rejection status	429 with Retry-After; 503 only for server overload
Cost weighting	Charge expensive endpoints more than one "unit"
Failure mode	Decide deliberately: fail-open (availability) or fail-closed (protection) when Redis is down
Internal traffic	Exempt or limit separately; do not throttle your own health checks

That failure-mode row is the one teams discover during an incident. If the rate limiter's backing store goes down, does the gateway let everything through or reject everything? For most public APIs, fail-open is correct — a rate limiter outage should not become an API outage — but the choice must be made in advance, not by whichever exception handler happens to run.

Key Takeaways

• Rate limits protect short-term capacity; quotas encode longer-term commercial plans.
• The limit key (client, user, IP, endpoint) matters more than the algorithm; layer several.
• Distributed limiting needs a shared, atomically updated counter — otherwise the real limit is N times the configured one.
• Always return 429 with Retry-After and expose limit/remaining/reset headers, and decide fail-open vs. fail-closed up front.

🧪 Practice

1. Design limits for a three-tier API product (free, pro, enterprise), including burst behaviour and a monthly quota.
2. Explain why per-IP limiting punishes legitimate users on mobile networks and corporate VPNs.
3. Interview: Your rate limiter's Redis cluster fails. Should the gateway fail open or closed? (Hint: the answer differs for a public read API and for a login endpoint — say why.)

Request Routing and Aggregation

A gateway's most basic job is deciding where a request goes, and its most debated job is deciding whether it should combine several backend calls into one response. The first is unambiguously the gateway's business; the second is powerful and easy to abuse.

Routing rules resolve a request to an upstream by path, method, header, or weight. Because the rules are configuration rather than code, they also become the mechanism for progressive delivery — canaries, blue-green cutovers, and traffic mirroring all fall out of weighted routing.

routes:
  # 1. Path-based: the common case, one prefix per service
  - match: {path: /v1/orders/**}
    upstream: orders.internal

  # 2. Header-based: route internal testers to a new build
  - match: {path: /v1/search/**, headers: {X-Beta: "true"}}
    upstream: search-v2.internal

  # 3. Weighted: a canary release — 5% of traffic to the new version
  - match: {path: /v1/search/**}
    upstreams:
      - {target: search-v1.internal, weight: 95}
      - {target: search-v2.internal, weight: 5}

  # 4. Mirroring: send a copy to the new service, discard its response.
  #    Real production traffic, zero user risk. Beware side effects — never
  #    mirror writes to a service that will actually perform them.
  - match: {path: /v1/pricing/**}
    upstream: pricing-v1.internal
    mirror:   {target: pricing-v2.internal, percent: 10}

Aggregation addresses the N+1 round-trip problem from a different angle than GraphQL: instead of the client making five calls, the gateway makes them — on the fast internal network, in parallel — and returns one response.

// Gateway-side aggregation: 5 sequential client round trips (5 x 60 ms over
// the internet) become 1 client round trip plus parallel internal calls (~15 ms).
async function orderScreen(req, res) {
  const orderId = req.params.id;
  const order = await svc.orders.get(orderId);        // must come first

  // Everything independent runs concurrently, not sequentially.
  const [customer, items, shipping] = await Promise.all([
    svc.users.get(order.customerId),
    svc.catalog.getMany(order.itemIds),
    svc.shipping.quote(orderId).catch(() => null),    // degrade, do not fail:
  ]);                                                 // a missing quote should not
                                                      // break the whole screen
  res.json({
    id: order.id,
    total: order.total,
    customer: { name: customer.name },
    items,
    shippingEstimate: shipping?.estimate ?? null,     // explicit partial response
  });
}

The trade-offs are real and worth naming before adopting aggregation:

Benefit	Cost
One client round trip instead of many	The gateway now knows about response shapes
Parallel fan-out on a fast network	Its latency is the slowest call, not the average
Partial failure handled centrally	Failure semantics get complicated (what is fatal?)
Smaller payloads for mobile clients	Domain knowledge creeps into shared infrastructure

The guardrail: aggregate mechanically, do not compute. Fetching several resources and assembling them into one JSON document is composition. Calculating a discount, deciding an order's status, or validating a business rule is domain logic and belongs in a service. When aggregation genuinely needs client-specific shaping, the answer is usually not a fatter shared gateway but the pattern in the next topic.

Key Takeaways

• Routing by path, header, and weight turns canaries, blue-green, and mirroring into configuration.
• Aggregation collapses client round trips by fanning out in parallel on the internal network.
• An aggregated response is only as fast as its slowest call — set per-call timeouts and degrade gracefully.
• Compose data at the gateway; never compute business logic there.

🧪 Practice

1. Write routing rules for a canary that sends 1% of traffic to v2 while pinning internal testers to v2 by header.
2. Implement an aggregation where one of three calls is optional and its failure must not fail the request.
3. Interview: When does gateway aggregation become an anti-pattern? (Hint: watch for the moment a gateway change is required to ship a feature, and ask who owns that config.)

Backend-for-Frontend Pattern

A single API serving a web app, an iOS app, and a partner integration ends up serving none of them well. The web dashboard wants rich, denormalized payloads; the mobile app wants tiny responses because of battery and bandwidth; the partner wants a stable contract that never changes. Every change becomes a negotiation, and the API drifts toward a lowest common denominator that is verbose for one client and insufficient for another.

The Backend-for-Frontend (BFF) pattern resolves this by giving each client type its own thin API layer, owned by the team that builds that client. The BFF talks to the shared downstream services and shapes their responses for exactly one consumer.

 ONE SHARED API                        BACKEND-FOR-FRONTEND

  [web] ----\                           [web]     --> [web BFF]     --\
  [iOS] -----> [ one public API ]       [iOS]     --> [mobile BFF]  ---> [services]
  [partner] -/         |                [partner] --> [partner API] --/
                       v
                  [services]            Each BFF is owned by its client team,
                                        ships on that client's schedule, and
  Every change is a three-way           optimizes for exactly one consumer.
  negotiation; payloads suit nobody

// MOBILE BFF: one endpoint per screen, minimal payload, aggressive shaping.
app.get("/mobile/order-screen/:id", async (req, res) => {
  const order = await services.orders.get(req.params.id);
  const [customer, items] = await Promise.all([
    services.users.get(order.customerId),
    services.catalog.getMany(order.itemIds),
  ]);

  res.json({
    // Pre-formatted for display: the phone does no work and ships no
    // formatting library. This IS client code — it just runs on a server.
    title: `Order #${order.reference}`,
    statusLabel: STATUS_LABELS[order.status],
    totalFormatted: formatMoney(order.total, order.currency),
    customerName: customer.name,
    // Only what this screen renders: 3 fields per item, not 40.
    items: items.map(i => ({ name: i.name, qty: i.quantity, thumb: i.thumbnailUrl })),
  });
});

// WEB BFF: the same underlying services, a different shape entirely —
// full item records, edit permissions, and audit history for the desktop view.

Aspect	Shared API	BFF per client
Payload fit	Compromise for everyone	Optimal per client
Release cadence	Coupled across all clients	Independent per client team
Ownership	A central API team (a bottleneck)	The client team owns its BFF
Code duplication	Low	Some logic repeats across BFFs
Services to run	Fewer	One more deployable per client type
Best for	Few clients, similar needs	Divergent clients, independent teams

The failure modes are predictable. Too many BFFs — one per screen or per minor platform variant — multiplies operational cost for little gain; the useful granularity is one per client type with a distinct team and distinct needs. And business logic leaking into the BFF recreates the distributed monolith: a BFF should orchestrate and shape, while pricing, validation, and state transitions stay in the services that own them.

A BFF is not a replacement for the gateway. The gateway still handles TLS, authentication, and rate limiting for everyone; the BFF sits behind it and concerns itself only with shaping.

Key Takeaways

• A BFF gives each client type its own API layer, owned by that client's team.
• It removes the lowest-common-denominator compromise and decouples release schedules.
• Costs are duplicated logic and extra deployables — create one per client type, not per screen.
• BFFs orchestrate and shape; business logic stays in the owning services, and the gateway still fronts everything.

🧪 Practice

1. Design the response shape for the same "order details" screen on a smartwatch, a phone, and a desktop dashboard.
2. Identify which logic in a BFF you have seen (or can imagine) should have been in a downstream service instead.
3. Interview: When is a BFF overkill? (Hint: count the client types, the teams, and how much their data needs actually differ.)

Chapter Summary

Concept	The one-line version
Resource modeling	Organize around domain nouns; HTTP methods supply the verbs
Idempotency	Retries are inevitable — POST needs a client-generated idempotency key
Statelessness	No server state whose loss breaks correctness; any instance serves any request
Errors	4xx do not retry, 5xx do; stable codes, trace ids, never leak internals
Pagination	Cursor for anything unbounded; offset drifts and degrades with depth
Filtering	Query string with allowlisted, indexed fields and a deterministic tiebreaker
REST	Reuses HTTP semantics and caching; pays with fixed response shapes
GraphQL	Client-specified shapes; pays with query cost, N+1 resolvers, lost caching
gRPC	Binary over HTTP/2 for internal high-volume calls; field numbers are forever
SOAP	XML with message-level security and formal contracts; isolate behind an adapter
Webhooks	Provider pushes to a consumer URL: sign it, deduplicate it, ack fast
Style choice	Decided by consumer and interaction shape; one system uses several
Versioning	A last resort; URI path is the pragmatic default; every version needs a sunset
Compatibility	Tolerant reading enables additive change; new enum values secretly break clients
Schema evolution	Make compatibility a CI check, not review discipline
Deprecation	A schedule plus per-caller telemetry; ship the replacement first
Contracts	One machine-readable spec generates docs, SDKs, validation, and mocks
Contract testing	Consumer expectations verified in the provider's CI — fast and accurate
Gateway	Cross-cutting concerns once at the edge; keep domain logic out
Edge auth	Validate once, inject verified identity; coarse at the edge, fine in services
Rate limiting	The key matters more than the algorithm; shared atomic counters; 429 + Retry-After
Aggregation	Compose data at the gateway, never compute
BFF	One API layer per client type, owned by that client's team

---

<a id="4-data-storage-and-database-design"></a>