10. Architectural Patterns and Styles

Everything in the previous chapters — storage, caching, messaging, consistency, resilience — has to be arranged into a shape, and that shape is the architecture. This chapter covers the major styles a system can take: the monolith and its internal structuring patterns, the service-oriented decomposition that often replaces it, the event-driven models that let services cooperate without calling each other, and the cloud-native and serverless platforms that changed what a deployable unit even is. The recurring lesson is that these styles are not a ladder from primitive to advanced but a set of trades between coupling, operational cost, and the number of independent teams a system must support.

<a id="101-monolithic-and-layered-architectures"></a>

10.1 Monolithic and Layered Architectures

A monolith is a single deployable unit containing all of an application's functionality. This subchapter looks at what that actually buys and costs, the internal structuring patterns that keep a large codebase from turning to mud, and the disciplined ways out when a monolith genuinely stops fitting.

Monolith Structure and Trade-offs

A monolithic application is one where all the code ships together: one build, one artifact, one process type, usually one database. Everything talks to everything else through in-process function calls. The style has a poor reputation, which is largely undeserved — the reputation belongs to badly structured monoliths, and a badly structured distributed system is considerably worse.

The reason to start here is that the monolith is the default that every other style has to justify beating. Consider what a function call gives you for free. It takes nanoseconds, it cannot be partially delivered, it either returns or throws, it never needs a retry policy, it joins the caller's database transaction, and a compiler checks both sides of it. Turning that call into a network hop trades away every one of those properties in exchange for the ability to deploy and scale the two halves separately. That is sometimes an excellent trade. It is never a free one.

 MONOLITH                              DISTRIBUTED SERVICES

 +---------------------------+         +--------+  +--------+  +---------+
 |  one process              |         | orders |->| pricing|->|inventory|
 |    orders  -> pricing     |         +--------+  +--------+  +---------+
 |      |          |         |             |           |            |
 |      v          v         |             v           v            v
 |    inventory -> shipping  |          +------+    +------+    +------+
 +---------------------------+          | db_o |    | db_p |    | db_i |
              |                         +------+    +------+    +------+
        +-----------+
        |  one DB   |        call:    nanoseconds  ->  milliseconds
        +-----------+        failure: exception    ->  timeout, partial state
                             txn:     ACID         ->  saga + compensation
                             deploy:  one unit     ->  N independent units

The trade-offs split cleanly into what a monolith is good at and what it is bad at, and both lists sharpen as the organization grows.

Dimension	Monolith	Distributed services
Local development	Clone, run, done	Run N services or mock them
Cross-cutting change	Compiler-verified rename	Coordinated deploys, versioned contracts
Transactions	One ACID transaction	Sagas and compensation (see 8.5)
Debugging	One stack trace	Distributed tracing across hops
Deploy blast radius	The whole application	One service
Scaling granularity	Everything scales together	Scale only the hot component
Team autonomy	Shared codebase, shared release	Independent release per team
Failure isolation	A memory leak takes down all of it	A sick service can be shed
Technology choice	One stack	Per-service choice (a cost as well as a perk)

The scaling row deserves a second look, because it is the argument most often made and most often misapplied. A monolith scales horizontally perfectly well — run more copies behind a load balancer. What it cannot do is scale one part independently, and that only matters when the resource profiles genuinely differ.

# When does per-component scaling actually pay?
# Peak load and per-request CPU cost differ wildly across features.
COMPONENTS = {
    # name            rps,   cpu_ms_per_request
    "checkout":      (200,     40),
    "catalog":      (9000,      5),
    "image_resize":   (50,   2000),   # rare, but brutally expensive
}

CPU_MS_PER_CORE = 1000

def cores(rps, cpu_ms):
    return rps * cpu_ms / CPU_MS_PER_CORE

per_component = {n: cores(r, c) for n, (r, c) in COMPONENTS.items()}
print(per_component)
# {'checkout': 8.0, 'catalog': 45.0, 'image_resize': 100.0}

# A monolith must run enough identical copies to serve the WHOLE profile, and
# every copy carries every component's memory footprint and dependencies.
print(sum(per_component.values()))          # 153.0 cores of demand
print(per_component["image_resize"])        # 100.0 of them, from 0.5% of traffic
#
# THAT is a scaling argument: one outlier dominates compute and can be pulled
# out on its own. "Catalog gets more traffic than checkout" is not an argument
# -- both are cheap, and identical copies of the monolith serve both fine.

The real limit of a monolith is usually organizational rather than technical. One codebase with one release train means every team's changes queue behind every other team's. With five engineers that is a non-issue; with two hundred it is the dominant cost, and the coordination overhead starts to exceed the operational overhead of running separate services. That crossover — not requests per second — is the honest trigger for decomposition.

Key Takeaways

• A monolith is one deployable unit whose components communicate by in-process calls, which gives speed, ACID transactions, and compile-time contracts free.
• Splitting into services trades those guarantees for independent deployment, scaling, and failure isolation: a real trade, never a free upgrade.
• Monoliths scale horizontally fine; what they cannot do is scale one component independently, which only matters when resource profiles genuinely differ.
• The usual breaking point is organizational — one release train shared by many teams — not load.
• Most "monolith problems" are structure problems, fixable without distributing anything.

🧪 Practice

1. List five guarantees a function call provides that a network call does not, and name the mechanism each one requires once the call crosses a process.
2. Take a feature in a system you know, estimate its peak RPS and per-request CPU, and decide whether extracting it would meaningfully change capacity.
3. Interview: A team wants microservices because "the monolith doesn't scale". What do you ask before agreeing? (Hint: separate the compute argument from the deployment-coordination argument and find out which one they feel.)

Layered Architecture

Layered architecture is the oldest and most widely used way to impose structure inside a single application. Code is grouped by technical responsibility, and dependencies are allowed in only one direction: presentation depends on application logic, which depends on domain logic, which depends on data access. Nothing depends upward.

The reason for the one-way rule is that dependency direction determines what you can change safely and what you can test in isolation. If the domain never imports the web framework, business rules can be tested without an HTTP server; if data access never imports the domain, an ORM can be swapped without touching business rules. Layers buy that independence with a rule simple enough that a linter can enforce it.

   +--------------------------------------------+
   |  Presentation   HTTP handlers, CLI, gRPC   |  serialization, status codes
   +--------------------------------------------+
                     | depends on
                     v
   +--------------------------------------------+
   |  Application    use cases, orchestration   |  transactions, workflow
   +--------------------------------------------+
                     | depends on
                     v
   +--------------------------------------------+
   |  Domain         entities, business rules   |  no framework imports
   +--------------------------------------------+
                     | depends on
                     v
   +--------------------------------------------+
   |  Persistence    repositories, SQL, clients |  storage details
   +--------------------------------------------+

   Rule: arrows point DOWN only. An upward import is an architecture bug.

# A layered slice of "place an order". Note what each layer is allowed to know.

# ---- domain: pure rules, no framework, no I/O ------------------------------
class Order:
    def __init__(self, items, customer_tier):
        self.items = items
        self.customer_tier = customer_tier

    def total(self):
        gross = sum(i["price"] * i["qty"] for i in self.items)
        discount = 0.10 if self.customer_tier == "gold" else 0.0
        return round(gross * (1 - discount), 2)    # a rule, testable alone

# ---- persistence: knows storage, knows nothing about HTTP ------------------
class OrderRepository:
    def __init__(self, conn):
        self.conn = conn

    def save(self, order_id, order):
        self.conn.execute("INSERT INTO orders (id, total) VALUES (?, ?)",
                          (order_id, order.total()))

# ---- application: orchestrates the use case and owns the transaction -------
class PlaceOrder:
    def __init__(self, repo, payments):
        self.repo, self.payments = repo, payments

    def execute(self, cmd):
        order = Order(cmd["items"], cmd["tier"])
        self.payments.charge(cmd["customer_id"], order.total())
        self.repo.save(cmd["order_id"], order)
        return {"order_id": cmd["order_id"], "total": order.total()}

# ---- presentation: protocol only; no business rules live here --------------
def http_post_orders(request, place_order: PlaceOrder):
    try:
        return 201, place_order.execute(request.json)   # delegate, do not decide
    except ValueError as e:
        return 400, {"error": str(e)}                   # translate to protocol

Two failure modes are worth naming. The first is anaemic layering, where each layer is a pass-through: the handler calls a service that calls a repository, each adding a file and nothing else. That is ceremony, not structure. The second is the sinkhole, where domain objects are just data bags and the rules end up in the application layer or, worse, in HTTP handlers — at which point the layer names document nothing.

Layering also has a structural limitation: it slices by technology, so one feature is smeared across all of it. Changing "how orders work" touches four directories, while changing "how we persist things" touches one. That is the right trade when technical concerns churn and the wrong one when features churn, which is exactly what motivates the modular approach two topics down.

Key Takeaways

• Layers group code by technical responsibility and allow dependencies in one direction only; the one-way rule is what makes isolation and testing possible.
• "The domain imports no framework" is the highest-value constraint — enforce it with a linter or an architecture test, not with a convention.
• Pass-through layers that only forward calls add ceremony without structure.
• Layering slices by technology, so a feature change touches every layer; that is its main weakness.

🧪 Practice

1. Take an HTTP handler that contains business rules and split it into presentation, application, and domain pieces without changing behaviour.
2. Write a test that fails if any file in the domain package imports a web or database library; a simple import scan is enough.
3. Interview: When is strict layering the wrong choice? (Hint: think about what a typical change request touches, and whether that maps to one layer or all of them.)

Modular Monoliths

A modular monolith keeps the single deployable unit but slices the code by business capability rather than by technical layer, and enforces boundaries between those slices as strictly as if they were separate services. It exists because the industry noticed that most monolith pain came from missing internal boundaries and most microservice pain came from the network — which raises the obvious question of whether you can have the boundaries without the network.

The mental model is a set of modules, each owning its data and exposing a narrow published interface. Inside a module, layers can be as elaborate or as thin as that module needs. Across modules, only the published interface is legal: no reaching into another module's internals, and critically, no reaching into another module's tables.

 LAYERED (slice by technology)          MODULAR (slice by capability)

 +--------------------------------+     +---------+ +---------+ +---------+
 | web: orders, catalog, billing  |     | Orders  | | Catalog | | Billing |
 +--------------------------------+     |  web    | |  web    | |  web    |
 | svc: orders, catalog, billing  |     |  logic  | |  logic  | |  logic  |
 +--------------------------------+     |  data   | |  data   | |  data   |
 | dao: orders, catalog, billing  |     +----+----+ +----+----+ +----+----+
 +--------------------------------+          |          |           |
              |                               +-- published API ----+
       one schema, shared                     (in-process calls or an
       freely by everything                    internal event bus)

 A feature change touches               A feature change touches
 every layer.                           one module.

// A published interface plus a package-private implementation is the cheapest
// boundary a language will enforce for you.

// ---- orders module: the ONLY type other modules may see --------------------
package com.shop.orders;

public interface OrdersApi {                      // public: this is the contract
    OrderSummary place(PlaceOrderCommand cmd);
    OrderSummary find(String orderId);
}

// ---- implementation is package-private: unreachable from other packages ----
package com.shop.orders;

class OrderService implements OrdersApi {         // note: no 'public' modifier
    private final OrderRepository repo;           // internal, invisible outside
    public OrderSummary place(PlaceOrderCommand cmd) { /* ... */ }
    public OrderSummary find(String orderId)        { /* ... */ }
}

// ---- billing module may depend on the interface, never the internals -------
package com.shop.billing;

import com.shop.orders.OrdersApi;                 // legal
// import com.shop.orders.OrderRepository;        // does not compile: not public

class InvoiceService {
    private final OrdersApi orders;               // talks through the contract
}

The hard part is data. Modules sharing one physical database drift toward sharing tables, and a shared table is the most binding coupling there is — invisible to the compiler, discovered only when someone's migration breaks someone else's query. The discipline that makes modular monoliths work is schema-per-module: each module owns its tables, and cross-module reads go through the published API even though a JOIN would technically work.

-- Enforce data ownership with schemas and grants, not with good intentions.
CREATE SCHEMA orders;
CREATE SCHEMA billing;

CREATE TABLE orders.orders    (id uuid PRIMARY KEY, total numeric);
CREATE TABLE billing.invoices (id uuid PRIMARY KEY, order_id uuid, amount numeric);

-- Deliberately NO foreign key from billing.invoices to orders.orders: a
-- cross-module FK is exactly the coupling that makes later extraction hard.

CREATE ROLE billing_module;
GRANT USAGE ON SCHEMA billing TO billing_module;
GRANT SELECT, INSERT, UPDATE ON ALL TABLES IN SCHEMA billing TO billing_module;
-- and deliberately not:  GRANT USAGE ON SCHEMA orders TO billing_module;
-- The database now rejects the shortcut a code review might miss.

The payoff is optionality. A modular monolith that respects its boundaries can have any module lifted out into a service later, because the call sites already go through an interface and the data is already separated — extraction becomes a transport change rather than a rewrite. This makes it the best default for most new systems: the deployment simplicity of a monolith, with the option to distribute the parts that eventually earn it.

Key Takeaways

• A modular monolith slices by business capability, enforces module boundaries, and still ships as one deployable unit.
• The published-interface rule needs language-level enforcement — package visibility, a module system, or architecture tests — to survive deadlines.
• Data ownership matters more than code structure: schema-per-module with no cross-module foreign keys is what keeps future extraction cheap.
• It is the best default for new systems: most of the boundary benefit, none of the network cost, and a short path to services later.

🧪 Practice

1. Re-group a layered codebase's directories by capability and list every import that would become illegal under a published-interface rule.
2. Design the schema split for a system with users, orders, and payments, stating what replaces each cross-module foreign key you had to remove.
3. Interview: If a modular monolith gives most of the benefits, why does anyone run microservices? (Hint: think about what one deployable unit still forces teams to share, and what independent scaling of one module would require.)

Hexagonal and Clean Architecture

Hexagonal architecture — also called ports and adapters, and closely related to Clean and Onion architecture — is a refinement of layering that fixes its awkwardest part. In classic layering the domain sits above persistence and therefore depends on it, so business rules end up importing database concepts. Hexagonal architecture inverts that: the domain declares interfaces (ports) describing what it needs, and infrastructure supplies implementations (adapters) that plug into them. All dependencies point inward.

The analogy is a household appliance. The appliance defines a plug shape; mains power, a generator, or a battery pack can all fill it, and the appliance never imports the power station. In the same way, an order use case declares "I need something that can save an order", and Postgres, DynamoDB, or an in-memory fake each satisfy it — with identical domain code in all three cases, which is why tests become both fast and honest.

                     driving (primary) adapters
            HTTP        CLI         gRPC        test harness
              \          |           /              /
               v         v          v              v
             +--------------------------------------+
             |            PORTS (in)                |
             |   +------------------------------+   |
             |   |   DOMAIN + USE CASES         |   |
             |   |   depends on NOTHING outside |   |
             |   +------------------------------+   |
             |            PORTS (out)               |
             +--------------------------------------+
               ^         ^          ^              ^
              /          |           \              \
         Postgres     Kafka        Stripe        in-memory fake
                     driven (secondary) adapters

   Every arrow points INWARD. Infrastructure knows the domain;
   the domain does not know infrastructure.

from abc import ABC, abstractmethod

# ---- PORT (out): defined BY the domain, in the domain's vocabulary ---------
class OrderStore(ABC):
    @abstractmethod
    def save(self, order: dict) -> None: ...
    @abstractmethod
    def by_customer(self, customer_id: str) -> list: ...
    # Vocabulary check: no "row", no "query", no "session". If the port leaks
    # storage concepts, the inversion has failed.

class PaymentGateway(ABC):
    @abstractmethod
    def charge(self, customer_id: str, amount: float) -> str: ...

# ---- USE CASE: pure logic that depends only on ports -----------------------
class PlaceOrder:
    def __init__(self, store: OrderStore, payments: PaymentGateway):
        self.store, self.payments = store, payments     # injected, not imported

    def execute(self, customer_id, items):
        total = sum(i["price"] * i["qty"] for i in items)
        if total <= 0:
            raise ValueError("empty order")
        receipt = self.payments.charge(customer_id, total)
        self.store.save({"customer_id": customer_id, "total": total,
                         "receipt": receipt})
        return receipt

# ---- ADAPTERS (driven): infrastructure implements the domain's interface ---
class PostgresOrderStore(OrderStore):
    def __init__(self, conn): self.conn = conn
    def save(self, order):
        self.conn.execute(
            "INSERT INTO orders (customer_id, total, receipt) VALUES (%s,%s,%s)",
            (order["customer_id"], order["total"], order["receipt"]))
    def by_customer(self, customer_id):
        return self.conn.query("SELECT * FROM orders WHERE customer_id = %s",
                               (customer_id,))

class InMemoryOrderStore(OrderStore):               # a first-class adapter
    def __init__(self): self.rows = []
    def save(self, order): self.rows.append(order)
    def by_customer(self, cid):
        return [r for r in self.rows if r["customer_id"] == cid]

class FakePayments(PaymentGateway):
    def charge(self, customer_id, amount): return "rcpt_test"

# ---- The test needs no database, no network, no container ------------------
def test_place_order_records_total():
    store = InMemoryOrderStore()
    PlaceOrder(store, FakePayments()).execute("c1", [{"price": 10.0, "qty": 2}])
    assert store.rows[0]["total"] == 20.0

Clean Architecture is the same principle with a prescribed set of rings (entities, use cases, interface adapters, frameworks) and one explicit law: the dependency rule — source dependencies may point only inward, and nothing in an inner ring may know the name of anything in an outer ring. Data crossing a boundary crosses as a simple structure owned by the inner ring, never as an ORM entity or a framework request object.

The cost is indirection: more interfaces, more wiring, and a composition root that assembles the real adapters at startup. For a CRUD service with no interesting rules that is overhead with no return. The pattern pays when domain logic is genuinely complex, when it must outlive its infrastructure, or when fast, dependency-free tests are worth real money.

Concern	Classic layered	Hexagonal / Clean
Dependency arrow	Domain -> persistence	Persistence -> domain (inverted)
Interface owner	Infrastructure defines it	The domain defines the port
Swapping a store	Touches domain code	Write a new adapter
Unit tests	Need a DB or heavy mocks	In-memory adapters, no I/O
Cost	Low	Extra interfaces and wiring

Key Takeaways

• Hexagonal architecture inverts the layered dependency so infrastructure depends on the domain, never the reverse.
• Ports are interfaces the domain defines in its own vocabulary; adapters are the infrastructure implementations that plug into them.
• The dependency rule — inner rings never name outer rings — is the whole idea; the rest is naming.
• The practical payoff is tests that run with no I/O and infrastructure that can be replaced without touching business rules.
• It is overhead for thin CRUD services; use it where domain logic is real.

🧪 Practice

1. Find a class that imports both business rules and a database client, and split it into a port, an adapter, and a use case.
2. Write an in-memory adapter for one repository interface and convert one integration test into a unit test with it.
3. Interview: What is the difference between a port and a repository interface that happens to live in the data layer? (Hint: ask who defines the interface and whose vocabulary it is written in — that is the whole distinction.)

Monolith-to-Microservices Migration

Migration is where architecture meets project management, and it fails far more often from sequencing mistakes than from technical ones. The governing principle is that a migration must deliver value continuously, because a rewrite that delivers nothing for eighteen months gets cancelled at month fourteen with the old system still running and a half-built new one to maintain.

The first question is not "how" but "whether, and which". Extraction is justified for a component when at least one of these is concretely true: it needs to scale differently, it changes at a very different rate, it has a different reliability or compliance requirement, or a distinct team wants to own its release cadence. "It feels like a separate thing" is not on the list.

# Score candidates before touching code. The point is to rank, not to be exact.
CANDIDATES = {
    # name           coupling  churn  scale_diff  team_boundary  risk
    "reporting":     (2,       3,     4,          4,             1),
    "image_resize":  (1,       1,     5,          2,             1),
    "user_profile":  (4,       2,     1,          2,             3),
    "checkout":      (5,       5,     2,          3,             5),
}
# coupling : how many other components read its data   (LOWER is easier)
# churn / scale_diff / team_boundary                   (HIGHER is more benefit)
# risk     : blast radius if the extraction goes wrong (LOWER is safer)

def score(c):
    coupling, churn, scale, team, risk = c
    return (churn + scale + team) - (coupling + risk)

for name, c in sorted(CANDIDATES.items(), key=lambda kv: -score(kv[1])):
    print(f"{name:14} {score(c):+d}")
# reporting      +8    read-mostly and isolated: an ideal first cut
# image_resize   +6    low coupling, huge scale difference: extract early
# user_profile   -2
# checkout       -3    highest value AND highest coupling: extract LAST
#
# The common mistake is starting with checkout because it matters most. Start
# where coupling is lowest, so the team learns the mechanics cheaply.

The sequence that repeatedly works looks like this:

 1. Modularize IN PLACE     draw the boundary inside the monolith first; fix
                            the imports and the shared tables there, where a
                            mistake costs a compile error, not an outage

 2. Split the data          give the module its own schema; remove cross-boundary
                            joins and foreign keys; it now reads only its own data

 3. Introduce the seam      route every caller through one interface or facade

 4. Extract the service     move the code behind the seam out of process; the
                            facade becomes an HTTP or gRPC client

 5. Migrate traffic         shadow, compare, shift a percentage at a time, and
                            keep the old path switchable for at least a release

 6. Delete the old path     the step teams skip, which is why they end up
                            maintaining both forever

Step 2 is where migrations actually die. Code is easy to move; data is not. Splitting a table that four features join against means finding every join and deciding, for each, whether the data should be duplicated, referenced by ID, or fetched through an API — and accepting that some queries become two calls plus an in-memory merge. Doing that before a network exists means every mistake shows up as a failing test rather than a production incident.

Two safety nets are worth the effort. Shadowing sends the same request to both implementations, returns the old result, and logs divergences, so you learn whether the new service is correct under real traffic while risking nothing. A flagged switch at the seam then lets you shift 1%, 10%, 100%, and back again in seconds if error rates move.

import random

def place_order(cmd, old_impl, new_client, pct_to_new, log):
    old_result = old_impl(cmd)
    new_result = None
    try:
        new_result = new_client(cmd)                 # shadow: always exercised
        if new_result != old_result:
            log.warn("divergence", id=cmd["id"], old=old_result, new=new_result)
    except Exception as e:
        log.warn("new_path_error", id=cmd["id"], error=str(e))

    # Cutover: only a controlled share of traffic is SERVED the new answer.
    if new_result is not None and random.random() * 100 < pct_to_new:
        return new_result
    return old_result
# Shadowing doubles the work and, for writes, doubles the side effects -- so
# shadow read paths and idempotent operations directly, and for side-effecting
# writes shadow against a staging store or compare derived state instead.

Key Takeaways

• Migrate incrementally and keep shipping; big-bang rewrites get cancelled with two systems running.
• Justify each extraction with a concrete driver — scaling, churn, compliance, team ownership — not with a feeling that something is separate.
• Extract low-coupling components first to learn the mechanics cheaply; the most business-critical component goes last.
• Modularize and split the data inside the monolith before moving anything over a network; the data split is what kills migrations.
• Shadow and compare, shift traffic by percentage, and actually delete the old path.

🧪 Practice

1. Score three components of a system you know on coupling, churn, scaling difference, and team boundary, and justify which you would extract first.
2. Pick a table joined by two candidate services and write out the options for removing that join, with the consistency cost of each.
3. Interview: How do you validate that an extracted service behaves identically to the code it replaced? (Hint: think about running both and comparing, and what has to change when the operation has side effects.)

Strangler Fig Pattern

The strangler fig pattern is the standard technique for replacing a system incrementally. It is named after the tropical fig that germinates in a host tree's canopy, sends roots down around the trunk, and gradually takes over until the host rots away and the fig stands on its own. Applied to software: put a routing layer in front of the old system, build new functionality in the new one, redirect one route at a time, and eventually the old system serves nothing and can be deleted.

The reason this beats a rewrite is that it keeps risk proportional to progress. At every moment most of the system is the old, well-understood one, and the newly migrated slice is small enough to reason about and to reverse. There is never a day on which everything changes at once.

 PHASE 1  facade in, nothing moved       PHASE 2  some routes moved

   clients                                 clients
      |                                       |
   [ facade / proxy ]                      [ facade / proxy ]
      |                                     /            \
   [   monolith    ]              /orders  v              v  everything else
                                 [ orders svc ]      [   monolith   ]

 PHASE 3  most routes moved               PHASE 4  host removed

   clients                                 clients
      |                                       |
   [ facade / proxy ]                      [ facade / API gateway ]
     /    |     \        \                   /      |       \
  orders users cart   [ monolith ]        orders  users    cart
                       (one route left)

The facade is usually existing infrastructure rather than something new: an API gateway, an ingress controller, or an nginx config. What matters is that clients address the facade rather than the systems behind it, so moving a route is invisible to them.

# The strangler routing table, expressed as proxy configuration.
upstream monolith    { server monolith.internal:8080; }
upstream orders_svc  { server orders.internal:8080;   }
upstream catalog_svc { server catalog.internal:8080;  }

server {
    listen 443 ssl;
    server_name api.example.com;

    # Migrated: these paths no longer reach the monolith at all.
    location /api/orders/ { proxy_pass http://orders_svc; }

    # Partial migration: a share of traffic goes to the new service while the
    # old path stays available. split_clients hashes a STABLE key, so a given
    # user gets a consistent backend instead of flapping between systems.
    split_clients "${http_x_user_id}" $catalog_backend {
        10%     catalog_svc;          # ramp: 10% -> 50% -> 100%
        *       monolith;
    }
    location /api/catalog/ { proxy_pass http://$catalog_backend; }

    # Everything else: still the host tree.
    location / { proxy_pass http://monolith; }
}

Three habits separate strangler migrations that finish from ones that stall. Route at the smallest useful granularity — a path, or even one endpoint, rather than "all of billing" — so each step is reversible. Keep the seam thin: once the facade holds business logic or elaborate payload transformation, you have built a third system to maintain. And define done per route as "the old code is deleted", not "the old code is unreachable"; unreached code still gets compiled, patched, and audited.

The pattern's hardest case is shared state. If the new service and the old monolith both need to read and write the same data during the transition, you must choose: keep a single writer and have the other call it, dual-write with a reconciliation job, or replicate one direction with change data capture and accept lag. Whichever you pick, pick it explicitly and time-box it — "both systems write the same table" is not a state to still be in a year later.

Transition strategy	How it works	Main risk
Single writer	New service owns writes; the old calls it	Extra hop; old code must change
Dual write	Both write; a job reconciles differences	Divergence, partial failures
CDC replication	Stream one system's changes to the other	Replication lag; one-way only
Shared table	Both read and write directly	No boundary at all — avoid

Key Takeaways

• Put a facade in front, move one route at a time, and delete the old path behind it; risk stays proportional to progress.
• Use existing routing infrastructure as the facade and keep it thin — logic in the seam creates a third system.
• Percentage routing on a stable hash key lets a route ramp safely and roll back instantly.
• Shared mutable state during the transition is the hard part: choose single writer, dual write plus reconciliation, or CDC deliberately, and time-box it.
• A route is done when the old implementation is deleted, not when it is unused.

🧪 Practice

1. Write the proxy configuration that sends a single read endpoint to a new service for 5% of users, keyed on a stable identifier.
2. For a feature both systems must write during migration, compare a single-writer plan with a dual-write plan and state the failure each allows.
3. Interview: How do you tell whether a strangler migration is actually progressing? (Hint: think about which metric separates real progress from routes added to the new system while the old code still ships.)

<a id="102-service-oriented-architectures"></a>

10.2 Service-Oriented Architectures

Once a system is split across processes, the questions that matter are where the lines go and what crosses them. This subchapter covers the principles behind microservices, the domain-driven techniques for finding boundaries that hold, the communication choices between services, and the anti-pattern that results when the lines are drawn in the wrong place.

Microservices Principles

Microservices is an architectural style in which an application is built as a suite of small, independently deployable services, each owning its data and communicating over the network. "Small" is the least important word in that sentence and the one everyone fixates on. The load-bearing word is independently deployable: if two services must be released together, you have paid the full cost of distribution and received none of the benefit.

The principles that actually define the style are these:

• Independent deployability. A service can be released without coordinating with any other team. This implies backward-compatible contracts and no shared release train.
• Business capability alignment. A service maps to something the business does ("pricing", "fulfilment"), not to a technical layer ("the DAO service") or an entity table ("the user-table service").
• Data ownership. Each service owns its storage exclusively; nobody else reads or writes it directly. This is the boundary that matters most.
• Decentralized governance. Teams choose their own internals within a small set of platform-wide rules (observability, auth, deployment).
• Design for failure. Every call can time out, so every caller has a timeout, a retry policy, a fallback, and a circuit breaker (chapter 9).
• Smart endpoints, dumb pipes. Logic lives in services; the transport does routing and delivery, not business rules.

The prerequisites are as important as the principles, because microservices move complexity from inside the code to between the services, and the platform has to absorb it. A team that cannot deploy on demand, cannot trace a request across hops, and has no automated testing will not be rescued by splitting the codebase; they will simply get the same defects with network partitions added.

 WHAT MOVES WHERE

   MONOLITH                              MICROSERVICES
   +---------------------------+         +------+ +------+ +------+
   | complexity is INSIDE:     |         | each service is simple |
   |  - module tangle          |         +------+ +------+ +------+
   |  - big build              |                 |
   |  - shared release         |         complexity is BETWEEN:
   +---------------------------+          - contracts and versioning
                                          - partial failure, retries
   needs: discipline                      - distributed tracing
                                          - data consistency (sagas)
                                          - N pipelines, N dashboards

                                        needs: a platform team's worth of
                                               automation, or it eats you

# The honest readiness checklist. Every "no" is work you will do anyway --
# the only question is whether you do it before or during the incident.
prerequisites:
  deploy_on_demand:        # can any team ship to prod without a release meeting?
    required: true
  automated_tests:         # can you refactor a service without manual QA?
    required: true
  distributed_tracing:     # can you answer "which hop was slow?" in one place?
    required: true
  centralized_logging:     # correlated by request id across services
    required: true
  service_discovery:       # services find each other without hardcoded IPs
    required: true
  contract_testing:        # a provider change that breaks a consumer fails CI
    required: true
  on_call_per_service:     # the team that ships it carries the pager
    required: true

Sizing deserves one direct answer, because it is the most common question. A service should be as small as it can be while still owning a complete business capability and its data — the useful test is whether a typical feature request can be delivered by changing one service. If most changes touch three services, they are too small or split along the wrong seam. The team-size heuristic ("a service a small team can own") is a consequence of that rule, not a replacement for it.

Key Takeaways

• The defining property is independent deployability; services that must be released together deliver cost without benefit.
• Align services with business capabilities, not with technical layers or individual tables.
• Exclusive data ownership is the boundary that matters most; a shared database erases every other boundary you drew.
• Microservices move complexity from inside the code to between the services, so the platform (CI/CD, tracing, discovery, contract tests) is a prerequisite, not a follow-up.
• Right size = a complete capability whose typical change request touches one service.

🧪 Practice

1. Take the readiness checklist above and score a team you know; identify the single missing item that would hurt most on day one.
2. Sketch a five-service decomposition of an e-commerce system and mark, for each service, the business capability and the data it exclusively owns.
3. Interview: What is the first thing you would put in place before splitting a monolith? (Hint: think about what you lose the instant a call becomes a network hop, and what would let you debug it.)

Service Boundaries and Domain-Driven Design

The hardest question in service architecture is where to cut. Cut in the wrong place and every feature requires changes in three services and a coordinated release; cut well and most changes are local. Domain-driven design (DDD) is the body of technique for finding those lines, and its central claim is that good boundaries follow the business's own structure rather than the data model.

The most useful heuristic is cohesion of change: things that change together belong together. Its inverse is equally useful — if two components have never been modified in the same pull request, they are probably separable. Because most teams have years of version history, this is measurable rather than a matter of taste.

# Mining git history for boundaries: which files keep changing together?
from itertools import combinations
from collections import Counter

# In practice: `git log --name-only --pretty=format:---` parsed into commits.
COMMITS = [
    {"order.py", "order_repo.py", "order_api.py"},
    {"order.py", "pricing.py"},
    {"order.py", "order_repo.py"},
    {"catalog.py", "catalog_index.py"},
    {"catalog.py", "catalog_index.py", "search.py"},
    {"pricing.py", "promo.py"},
    {"order.py", "order_api.py"},
]

pairs = Counter()
for files in COMMITS:
    for a, b in combinations(sorted(files), 2):
        pairs[(a, b)] += 1

for (a, b), n in pairs.most_common(5):
    print(f"{n}x  {a} <-> {b}")
# 3x  order.py <-> order_repo.py       ) these move as one unit:
# 2x  order.py <-> order_api.py        ) one service
# 2x  catalog.py <-> catalog_index.py  ) another unit
# 1x  order.py <-> pricing.py          ) weak link: a good boundary candidate
#
# High co-change ACROSS a proposed boundary predicts coordinated releases.
# Run this before drawing the diagram, not after the migration hurts.

DDD adds vocabulary for the structural decisions. A domain is the business area; subdomains partition it into core (your competitive advantage), supporting (necessary but not differentiating), and generic (solved elsewhere — buy it). That classification is directly actionable: invest engineering in the core, keep supporting subdomains simple, and never build a generic one yourself.

Subdomain type	Example (a retailer)	Strategy
Core	Pricing and promotions engine	Build; best engineers; invest
Supporting	Order fulfilment workflow	Build simply; keep boring
Generic	Authentication, email, invoicing	Buy or use a managed service

The other DDD tool worth knowing at the boundary level is the aggregate: a cluster of objects with one root, treated as a single consistency unit. The rule is that an aggregate is updated in one transaction and references other aggregates only by identity. That rule matters enormously for services, because an aggregate is the smallest thing that can be transactionally consistent — so a boundary that cuts through an aggregate creates a distributed transaction, and a boundary that follows aggregate edges does not.

 GOOD BOUNDARY: follows the aggregate edge

   Orders service                    Inventory service
   +--------------------+            +---------------------+
   | Order (root)       |            | StockItem (root)    |
   |   OrderLine        |  --sku-->  |   reserved, on_hand |
   |   ShippingAddress  |  by ID     +---------------------+
   +--------------------+
   one transaction per order          one transaction per item
   cross-aggregate work becomes an event or a saga -- deliberately

 BAD BOUNDARY: cuts through one consistency unit

   Order service        Order-line service     every order write is now
   +-----------+        +-----------------+    a two-phase problem across
   | Order     |<------>| OrderLine       |    two databases, for no benefit
   +-----------+        +-----------------+

Three sanity checks catch most bad boundaries before they are built. Would a typical feature request touch one service or several? Does any operation need a transaction spanning two of them? And does the boundary match a team boundary — because Conway's law says the architecture will drift toward the communication structure of the organization whether you plan for it or not.

Key Takeaways

• Boundaries follow the business structure, not the data model; things that change together belong together.
• Co-change analysis over git history turns boundary-drawing into evidence rather than opinion.
• Classify subdomains as core, supporting, or generic, and spend engineering effort accordingly.
• An aggregate is the unit of transactional consistency; boundaries that cut through one manufacture distributed transactions.
• Check every proposed boundary against feature locality, transaction scope, and team structure (Conway's law).

🧪 Practice

1. Run a co-change analysis on a repository you have access to and propose two boundaries the data supports.
2. Classify five subdomains of a system you know as core, supporting, or generic, and say what you would stop building.
3. Interview: A proposed split puts "orders" and "order lines" in different services. What is your objection? (Hint: ask what a single transaction must cover, and what happens when one of the two writes fails.)

Bounded Contexts

A bounded context is the boundary within which a model and its language are consistent. Outside it, the same word may mean something different, and that is allowed. It is the single most useful idea in DDD for service design, because it resolves the argument that otherwise consumes every data-modelling meeting: the attempt to build one canonical definition of "customer" or "product" for the entire company.

That attempt always fails, and the reason is worth internalizing. To sales, a customer is a lead with a pipeline stage and an account owner. To billing, a customer is a legal entity with a tax ID and a payment method. To support, a customer is a person with a ticket history and a timezone. A unified model must satisfy all three, so it accumulates every field, every optional flag, and a change requested by any one team risks breaking the other two. The bounded context alternative is to let each context define "customer" its own way and to translate at the boundary.

 ONE CANONICAL MODEL                 BOUNDED CONTEXTS

    +-------------------+            SALES ctx        BILLING ctx
    |    Customer       |            +----------+     +-------------+
    | id, name, email,  |            | Lead     |     | Payer       |
    | tax_id, pipeline, |            | stage    |     | tax_id      |
    | ticket_count,     |            | owner    |     | payment_mtd |
    | payment_method,   |            | score    |     | currency    |
    | timezone, tier,   |            +----+-----+     +------+------+
    | 40 more fields... |                 |                  |
    +-------------------+                 +-- customer_id ---+
      every team blocked                        (shared identity only)
      on every change                    SUPPORT ctx
                                         +-------------+
                                         | Contact     |
                                         | timezone    |
                                         | tickets     |
                                         +-------------+
       Same person. Three models. One shared ID. No coordination cost.

What crosses a context boundary is an explicit contract, and the translation lives in an anti-corruption layer (ACL) — a piece of code whose only job is to map the other context's language into yours, so a foreign model never leaks into your domain.

# Anti-corruption layer: the ONLY place that knows the billing context's shape.
class BillingClient:
    """Support context's view of billing. Translates, never leaks."""

    def __init__(self, http): self.http = http

    def account_standing(self, customer_id: str) -> dict:
        raw = self.http.get(f"/billing/payers/{customer_id}")   # foreign model
        # Map billing's vocabulary into the support context's vocabulary.
        return {
            "is_delinquent": raw["dunning_state"] in ("late", "collections"),
            "plan_name":     raw["subscription"]["tier_label"],
            # Deliberately NOT returned: tax_id, payment_method, currency.
            # Support has no use for them, and passing them through would make
            # every billing field change a support-context change.
        }

# In the support domain, nothing knows the word "dunning_state" exists.
def can_open_priority_ticket(customer, billing: BillingClient):
    standing = billing.account_standing(customer.id)
    return not standing["is_delinquent"]

Contexts relate to each other in a few recognizable ways, and naming the relationship tells you who absorbs change.

Relationship	Meaning	Who absorbs change
Shared kernel	Two contexts share a small model, changed jointly	Both (costly)
Customer / supplier	Downstream's needs influence upstream's roadmap	Upstream, partly
Conformist	Downstream simply adopts upstream's model	Downstream fully
Anti-corruption layer	Downstream translates at the boundary	The ACL only
Open host service	Upstream publishes a stable, general-purpose API	Upstream
Published language	A shared, versioned interchange schema (events, DTOs)	Schema governance

In practice, a bounded context is the natural unit for a service, and a context map — a diagram of contexts plus the relationship on each edge — is the most valuable architecture artefact a service-based system can have. It records not just who calls whom, but who is obliged to adapt when a model changes.

Key Takeaways

• A bounded context is the scope in which a model and its terms are consistent; the same word may legitimately mean different things in different contexts.
• One canonical enterprise model fails because it must satisfy every consumer, which couples every team to every change.
• Translate at boundaries with an anti-corruption layer so foreign models never leak into your domain.
• Name the relationship on each boundary — conformist, ACL, open host — because it determines who absorbs change.
• A bounded context is the natural unit for a service, and a context map is the artefact that keeps the boundaries honest.

🧪 Practice

1. Take one entity in a system you know and write three different definitions of it, one per team; list which fields each actually needs.
2. Implement an anti-corruption layer for an external API you consume, keeping its field names out of your domain entirely.
3. Interview: Two teams argue about the definition of "order status". How do you resolve it? (Hint: consider that both may be right inside their own boundary, and what has to exist for that to work.)

Inter-Service Communication

Once services are separate, every interaction becomes a design decision with consequences for coupling, latency, and failure behaviour. The first and largest choice is synchronous request/response versus asynchronous messaging, and it is better framed as a question about temporal coupling: must both services be healthy at the same moment for the interaction to succeed?

Synchronous calls are simple to reason about — you get an answer, or an error, right now — but they chain availability. If service A calls B calls C synchronously, A's availability is bounded by the product of all three, and a slowdown anywhere shows up as latency everywhere upstream. Asynchronous messaging decouples in time: the sender publishes and moves on, the receiver processes when it can, and a down receiver produces a backlog rather than an outage. The price is that answers are not immediate and the system becomes eventually consistent.

 SYNCHRONOUS CHAIN                     ASYNCHRONOUS FLOW

  client                                client
    | 200ms                                | 20ms  (accepted)
    v                                      v
  [ order ] --80ms--> [ pricing ]        [ order ] --> ( broker ) --> [ pricing ]
    |                                                        |
    +---60ms--> [ inventory ]                                +-----> [ inventory ]

  availability = 0.99 * 0.99 * 0.99      order stays up if pricing is down;
               = 0.970                   work queues and drains later
  latency      = sum of the chain        latency = time to enqueue
  failure      = user sees an error      failure = backlog, then catch-up

# The same business step, two shapes -- and two very different failure stories.

# --- synchronous: the caller owns the outcome and the waiting ---------------
def place_order_sync(cmd, pricing, inventory, orders_db):
    price = pricing.quote(cmd["items"])            # network hop; may time out
    if not inventory.reserve(cmd["items"]):        # network hop; may time out
        raise OutOfStock()
    return orders_db.insert(cmd, price)
    # If inventory.reserve() succeeds but the insert fails, stock is reserved
    # for an order that does not exist -- a compensating action is now required.

# --- asynchronous: the caller records intent and publishes ------------------
def place_order_async(cmd, orders_db, outbox):
    order_id = orders_db.insert_pending(cmd)       # local transaction ...
    outbox.append({                                # ... same transaction
        "type": "OrderPlaced", "order_id": order_id, "items": cmd["items"],
    })
    return {"order_id": order_id, "status": "pending"}   # answer immediately
    # A relay publishes the outbox rows after commit (transactional outbox,
    # see 7.3): the DB write and the message can no longer disagree.

Within synchronous communication, the protocol choice is secondary but real.

Style	Best for	Watch out for
REST/HTTP	Public APIs, wide client support	Chatty round trips, weak schema by default
gRPC	Internal service-to-service, low latency	Browser support, schema/deploy coupling
GraphQL	Aggregating many sources for varied UIs	N+1 resolvers, caching, query cost control
Messaging	Decoupling, buffering, fan-out	Eventual consistency, ordering, duplicates

Three rules apply regardless of the choice. Every synchronous call needs a timeout shorter than the caller's own deadline, or a slow dependency turns into a thread pool exhaustion (see 9.2). Contracts must be versioned and backward-compatible, because you cannot deploy both sides at once — additive changes only, and never repurpose a field. And avoid deep synchronous chains: each hop multiplies failure probability and adds tail latency, so two hops is normal, five is a design smell, and a chain that loops back is a distributed deadlock waiting for load.

// Contract evolution rules made concrete in a schema.
syntax = "proto3";

message OrderRequest {
  string order_id  = 1;
  string sku       = 2;
  int32  quantity  = 3;
  // Adding an OPTIONAL field with a NEW tag number is backward compatible:
  // old servers ignore it, old clients never send it.
  string coupon_code = 4;

  // Never do either of these to a deployed contract:
  //   - reuse tag 2 for a different meaning  (silent data corruption)
  //   - make a new field required            (old clients break instantly)
  reserved 5, 6;              // tags retired by a past change: never recycled
  reserved "legacy_discount"; // and neither is the old name
}

Key Takeaways

• The first choice is temporal coupling: synchronous means both parties must be healthy now; asynchronous trades immediacy for availability.
• Synchronous chains multiply failure probability and add tail latency; keep them shallow and always set timeouts shorter than the caller's deadline.
• Asynchronous messaging turns a dependency outage into a backlog, at the cost of eventual consistency.
• Pair a local write with a message publish using a transactional outbox so the database and the broker cannot disagree.
• Contracts must evolve additively — new optional fields, retired tags reserved forever — because both sides never deploy at once.

🧪 Practice

1. Take a three-hop synchronous chain and compute its end-to-end availability and p99 latency from per-hop numbers; then redo it with one hop made async.
2. Convert one synchronous side-effect call into an outbox-published event and describe what the caller now returns to the user.
3. Interview: When would you deliberately choose a synchronous call over an event? (Hint: think about who needs the answer before responding to the user, and about operations that must be rejected rather than queued.)

Distributed Monolith Anti-Pattern

A distributed monolith is a system split into services that still must be deployed, tested, and reasoned about as a unit. It is the worst of both worlds: the operational cost of a distributed system with the coupling of a monolith, and it is by far the most common failure mode of a microservices migration.

The reason it happens is that decomposition was done along the wrong lines — usually technical layers or database tables rather than business capabilities — so every real feature cuts across many services. The symptoms are recognizable long before anyone names the pattern.

 SYMPTOM CHECKLIST -- any two of these means you have one

 [ ] Releases are coordinated: "deploy A, then B, then C, in that order"
 [ ] One feature routinely requires changes to 3+ services
 [ ] Services share a database or read each other's tables
 [ ] Local development requires running the entire system
 [ ] A single service's rollback forces other rollbacks
 [ ] Integration tests are the only tests anyone trusts
 [ ] Services are versioned together (all at v4.2.1)
 [ ] One service being down means the whole system is down

 WRONG CUT (technical layers)            RIGHT CUT (capabilities)

   [ web service ]                        [ orders svc  ]  own web+logic+data
         |                                [ catalog svc ]  own web+logic+data
   [ business-logic service ]             [ billing svc ]  own web+logic+data
         |
   [ data-access service ]                 One feature -> one service.
         |
   [   shared database   ]

   One feature -> all three services, in a fixed deploy order.
   Three network hops added; zero independence gained.

The most damaging variant is the shared database. If two services read the same tables, the schema is a public API with no version, no owner, and no contract test. A column rename becomes a cross-team outage, and nothing about "service boundaries" in the code changes that, because the coupling is in the storage layer where no compiler will find it.

# A CI check that detects coordinated releases -- the clearest symptom.
DEPLOY_LOG = [   # (timestamp_minutes, service, version)
    (0,   "orders",  "v41"), (2,  "pricing", "v41"), (3, "inventory", "v41"),
    (240, "orders",  "v42"), (241,"pricing", "v42"), (243,"inventory", "v42"),
    (700, "catalog", "v13"),
]

def lockstep_groups(log, window=15):
    groups, current = [], []
    for ts, svc, ver in sorted(log):
        if current and ts - current[-1][0] <= window:
            current.append((ts, svc, ver))
        else:
            if len(current) > 1: groups.append(current)
            current = [(ts, svc, ver)]
    if len(current) > 1: groups.append(current)
    return groups

for g in lockstep_groups(DEPLOY_LOG):
    print([s for _, s, _ in g])
# ['orders', 'pricing', 'inventory']
# ['orders', 'pricing', 'inventory']
#
# The same set shipping together twice is not a coincidence -- it is one
# component wearing three service names. Independent deployability is the
# claim; the deploy log is the evidence.

Getting out is unglamorous and specific. Merge services that always ship together back into one — a smaller number of correct boundaries beats a large number of wrong ones, and re-merging is a legitimate architectural move rather than an admission of defeat. Break the shared database by assigning each table exactly one owner and replacing foreign reads with an API call or a replicated read model. Replace synchronous chains that exist purely to fetch data with events that push the data ahead of time. And make the invariant explicit: any service that cannot be deployed alone is not a service yet.

Key Takeaways

• A distributed monolith has service-level operational cost with monolith-level coupling; it is the default outcome of splitting along the wrong lines.
• The diagnostic is the deploy log: services that always ship together are one component with several names.
• A shared database is the strongest form of the anti-pattern — the schema becomes an unversioned public API.
• The fix includes merging services back together; fewer correct boundaries beat many wrong ones.
• Make "deployable alone" a hard invariant, and test it.

🧪 Practice

1. Take a deploy history and identify every set of services that ships in lockstep; propose a merge for the strongest group.
2. Pick one table read by two services, assign an owner, and design the replacement access path for the non-owner.
3. Interview: How would you tell, from the outside, whether a company's "microservices" are really a distributed monolith? (Hint: ask what happens when one team wants to ship on a Friday afternoon.)

Data Ownership per Service

Exclusive data ownership is the boundary that makes all the others real. The rule is simple: each piece of data has exactly one service that may write it and one schema that only that service may touch. Everyone else goes through its API or consumes its events. Without this rule, code-level boundaries are decoration.

The reason it is non-negotiable is that a database schema shared by several services is a contract with none of the properties of a contract. It has no version, no deprecation path, no consumer tests, and no owner who can approve a change. Worse, it lets any service violate another's invariants: a service that writes directly to orders can create an order in a state the orders service's own code would reject.

 SHARED DATABASE                        DATABASE PER SERVICE

  [orders] [billing] [reporting]        [orders]      [billing]   [reporting]
     \        |         /                  |             |            |
      \       |        /                [orders_db]  [billing_db]  [warehouse]
     +-------------------+                  |             |            ^
     |   one schema      |                  +---- events -+------------+
     +-------------------+
                                          reads of foreign data go through
  any service can break                   an API call or a replicated
  any other's invariants                  read model -- never a JOIN

The obvious objection is that queries spanning services become hard, and that is true — it is the actual cost of the pattern. There are four standard answers, and choosing the right one per case is the real work.

Need	Technique	Cost
A few fields, fresh	Synchronous API call	Latency, availability coupling
Many reads, tolerant of lag	Replicate a read model locally	Storage, staleness, sync code
Cross-service reporting	Stream to a warehouse/lake	Pipeline, minutes-to-hours lag
Composing several sources per UI	Aggregator / BFF service	Another service; fan-out latency

Replicating a read model is the most useful and least understood of these. The consuming service subscribes to the owner's events and keeps a local copy of just the fields it needs — denormalized, read-optimized, and explicitly stale. It turns a synchronous dependency into an asynchronous one, which is usually the whole point.

# Orders keeps a LOCAL copy of the few customer fields it needs, fed by events
# from the customers service. It never calls customers on the request path.

def handle_customer_event(event, local_db):
    if event["type"] in ("CustomerCreated", "CustomerUpdated"):
        local_db.upsert("customer_read_model", {
            "customer_id": event["customer_id"],
            "display_name": event["name"],       # only what orders actually uses
            "country":      event["country"],    # for tax rules
            "tier":         event["tier"],       # for discounts
            "updated_at":   event["occurred_at"],
        })
        # Deliberately NOT copied: email, phone, address history. Copy the
        # fields you use; every extra field is a change you must now absorb.

def place_order(cmd, local_db):
    # No network hop, no availability coupling: the customers service can be
    # down for an hour and orders keeps working with data that is minutes old.
    customer = local_db.get("customer_read_model", cmd["customer_id"])
    discount = 0.10 if customer["tier"] == "gold" else 0.0
    ...

Two disciplines keep this from becoming a mess. Be explicit about which service is the source of truth for each field — copies are copies, and a copy must never be edited locally. And decide, per use, whether stale data is acceptable: a stale display name is fine, a stale account balance used to authorize a withdrawal is not. Where staleness is unacceptable, either call the owner synchronously or move the decision into the owning service, which is often the better answer — put the logic where the data lives rather than shipping the data to the logic.

Key Takeaways

• Each piece of data has exactly one owning service that writes it; everyone else uses an API or events.
• A shared schema is a contract with no version, no owner, and no consumer tests, and it lets other services violate the owner's invariants.
• Cross-service queries are the real cost; answer them with API calls, replicated read models, a warehouse, or an aggregator — chosen per case.
• Replicated read models copy only the fields you use, are never edited locally, and convert a synchronous dependency into an asynchronous one.
• Decide explicitly whether staleness is acceptable; if it is not, move the decision to the service that owns the data.

🧪 Practice

1. For a report that joins three services' data today, design the pipeline that produces it without any service reading another's tables.
2. Define a read model for one consumer: list exactly the fields it needs, the events that maintain it, and the maximum staleness it tolerates.
3. Interview: A service needs another service's data on every request. What do you propose? (Hint: consider whether the data or the decision should move, and what the freshness requirement really is.)

<a id="103-event-driven-architectures"></a>

10.3 Event-Driven Architectures

Event-driven architecture inverts the direction of dependency: instead of a service calling the services it needs, it announces what happened and interested parties react. This subchapter covers what an event should carry, the patterns built on treating events as the primary record, the two ways to coordinate a multi-service workflow, and how to live with the eventual consistency that follows.

Event Notification vs. Event-Carried State Transfer

An event is a statement that something happened, in the past tense, published by the service that owns the fact. The decision nobody makes deliberately enough is how much data the event carries, and it has large consequences for coupling, payload size, and how much the consumer needs to know.

Event notification publishes a minimal fact — an identifier and a type — and consumers call back for details if they need them. Event-carried state transfer (ECST) publishes the relevant state inside the event, so a consumer can act without calling anyone. Notification keeps payloads tiny and the schema stable, but recreates the synchronous dependency it was meant to remove. ECST removes that dependency entirely, at the cost of bigger events, a richer schema to maintain, and data duplicated across consumers.

 EVENT NOTIFICATION                     EVENT-CARRIED STATE TRANSFER

 [orders] --"OrderPlaced{id}"-->        [orders] --"OrderPlaced{id, customer,
              (broker)                              items, total, address}"-->
                 |                                        (broker)
                 v                                           |
            [shipping]                                       v
                 |  GET /orders/{id}                    [shipping]
                 v                                       acts immediately;
            [orders]  <-- still a runtime dependency     orders may be DOWN
                          on the publisher

 small payloads, stable schema          no callback, no coupling at read time
 publisher must be up to consume        larger payloads, versioned schema,
 N consumers = N callbacks              duplicated state in each consumer

// Notification: "something happened, go look".
{
  "type": "OrderPlaced",
  "order_id": "ord_8891",
  "occurred_at": "2026-08-26T10:15:04Z",
  "version": 1
}

// Event-carried state transfer: "something happened, and here is what you need".
{
  "type": "OrderPlaced",
  "order_id": "ord_8891",
  "occurred_at": "2026-08-26T10:15:04Z",
  "version": 3,
  "customer": { "id": "cus_12", "tier": "gold", "country": "IT" },
  "items": [ { "sku": "SKU-1", "qty": 2, "unit_price": 19.99 } ],
  "total": 35.98,
  "ship_to": { "city": "Milano", "postcode": "20121", "country": "IT" }
}

# The same consumer, written both ways. Note what each one depends on.

def on_order_placed_notification(event, orders_api, shipments):
    # Callback on the publisher: correct, but shipping is now DOWN whenever
    # orders is down, and N consumers produce N calls per event.
    order = orders_api.get(event["order_id"])       # synchronous dependency
    shipments.create(order["ship_to"], order["items"])

def on_order_placed_ecst(event, shipments):
    # Self-contained: no call, no coupling, and it can be replayed later from
    # the log and still produce the same result -- the event IS the input.
    shipments.create(event["ship_to"], event["items"])

A useful middle path is to carry the state a consumer needs to decide, plus an ID for anything it might need to display. A third variant, the thin event with a versioned snapshot link, publishes an ID plus a pointer to an immutable snapshot — useful when payloads are large (documents, images) but consumers still need a stable view.

Property	Notification	Event-carried state
Payload size	Tiny	Proportional to the state
Runtime coupling	Consumer calls back	None
Schema surface	Minimal, stable	Large, needs versioning
Consumer replay	Sees today's data	Sees data as it was then
Publisher load	Grows with consumers	Constant
Data duplication	None	In every consumer

The replay row is subtle and often decides the matter. If a consumer re-processes last month's events after a bug fix, a notification-based consumer calls the API and gets current state — reconstructing history with today's values, which is usually wrong. An ECST consumer replays exactly what was true when the event was emitted.

Key Takeaways

• Events are past-tense facts published by the owner of that fact.
• Notification carries an ID and forces a callback, which reintroduces runtime coupling to the publisher.
• Event-carried state transfer makes consumers self-sufficient and replayable, at the cost of larger payloads, schema versioning, and duplicated data.
• Carry the state needed to decide; carry IDs for what is merely displayed.
• Replay behaviour often settles the choice: notifications reconstruct history using today's data, which is usually incorrect.

🧪 Practice

1. Design both event shapes for "user changed email" and list which consumers each shape would force to call back.
2. Take an ECST event and mark each field as decision-relevant or display-only; propose a trimmed payload.
3. Interview: A consumer must re-process a month of events after fixing a bug. Which event style makes that safe? (Hint: think about what "the customer's tier" meant then versus what an API returns now.)

Event Sourcing

Event sourcing stores state as an append-only sequence of events rather than as a mutable current row. Instead of UPDATE accounts SET balance = 120, you append Deposited{amount: 20} and compute the balance by folding the events. Current state becomes a derived value; the log becomes the source of truth.

The motivation is that ordinary state storage destroys information. A row tells you the balance is 120; it cannot tell you that it was 500 yesterday, that a refund was reversed, or who authorized what. Any business that needs an audit trail, temporal queries ("what did this look like on the 3rd?"), or the ability to re-derive state after fixing a bug has already been rebuilding a partial event log in audit_log tables. Event sourcing makes the log primary and derives everything else from it.

 STATE-ORIENTED                          EVENT-SOURCED

  accounts                                events (append only)
  +----+---------+                        +---+----------+----------------+
  | id | balance |                        | 1 | Opened   | {}             |
  | 7  |   120   |   <- overwritten       | 2 | Deposited| {amount: 100}  |
  +----+---------+      every write       | 3 | Withdrew | {amount: 30}   |
                                          | 4 | Deposited| {amount: 50}   |
  history: gone                           +---+----------+----------------+

                                          balance = fold(events) = 120
                                          balance on the 3rd = fold(events<=3)
                                          why 120? read events 1..4

from functools import reduce

# ---- events are immutable facts; the aggregate replays them ----------------
def apply(state, event):
    kind, data = event["type"], event["data"]
    if kind == "Opened":       return {"balance": 0, "status": "open"}
    if kind == "Deposited":    return {**state, "balance": state["balance"] + data["amount"]}
    if kind == "Withdrew":     return {**state, "balance": state["balance"] - data["amount"]}
    if kind == "Frozen":       return {**state, "status": "frozen"}
    return state

def replay(events, upto=None):
    chosen = events if upto is None else [e for e in events if e["seq"] <= upto]
    return reduce(apply, chosen, {})

EVENTS = [
    {"seq": 1, "type": "Opened",    "data": {}},
    {"seq": 2, "type": "Deposited", "data": {"amount": 100}},
    {"seq": 3, "type": "Withdrew",  "data": {"amount": 30}},
    {"seq": 4, "type": "Deposited", "data": {"amount": 50}},
]

print(replay(EVENTS))              # {'balance': 120, 'status': 'open'}
print(replay(EVENTS, upto=3))      # {'balance': 70,  'status': 'open'}
#                                    ^ time travel, for free

# ---- writing: validate against replayed state, then APPEND -----------------
def withdraw(store, account_id, amount, expected_version):
    state = replay(store.load(account_id))
    if state["status"] == "frozen":       raise Rejected("account frozen")
    if state["balance"] < amount:         raise Rejected("insufficient funds")
    store.append(account_id,
                 {"type": "Withdrew", "data": {"amount": amount}},
                 expected_version)        # optimistic concurrency: the append
                                          # fails if another writer moved the
                                          # stream forward since we replayed

Replaying an entire stream on every read is impractical for long-lived aggregates, so two mechanisms make it workable. Snapshots persist the folded state every N events, so replay starts from the snapshot rather than the beginning. And read models (next topic) maintain query-shaped projections continuously, so ordinary reads never touch the event log at all.

def load_fast(store, account_id, snapshot_every=100):
    snap = store.latest_snapshot(account_id)             # {'seq': 400, 'state': ...}
    start = snap["seq"] if snap else 0
    state = snap["state"] if snap else {}
    tail = store.load_since(account_id, start)           # only the recent events
    state = reduce(apply, tail, state)
    if len(tail) >= snapshot_every:
        store.save_snapshot(account_id, state, seq=start + len(tail))
    return state
    # A snapshot is a cache, never the truth. If it is wrong, delete it and
    # replay -- which is only possible because the log is authoritative.

The costs are real and should be stated plainly. Event schemas are forever: a 2019 event must still be readable in 2029, so you need upcasting (transforming old event shapes on read) and a strict no-rewrite policy. Deleting personal data conflicts with an immutable log, which is normally resolved by crypto-shredding — encrypting personal fields with a per-subject key and destroying the key on erasure. And the model is unfamiliar enough that a team adopting it everywhere will slow down; it belongs in the parts of the domain where history genuinely matters — ledgers, orders, compliance-heavy workflows — not in the CRUD around them.

Key Takeaways

• Event sourcing stores an append-only log of facts and derives current state by folding it; the log is the source of truth.
• It gives a complete audit trail, time-travel queries, and the ability to re-derive state after fixing a bug — things ordinary rows destroy.
• Snapshots make replay cheap and are always a cache that can be deleted and rebuilt.
• Event schemas are permanent: plan for upcasting old shapes and never rewrite history.
• Personal-data erasure needs crypto-shredding; apply event sourcing to the parts of the domain where history matters, not everywhere.

🧪 Practice

1. Model a shopping cart as events and write the fold that produces the current cart plus a function that answers "what was in it an hour ago?".
2. Add snapshotting to that model and show that deleting a snapshot changes no answer, only the time taken.
3. Interview: How do you satisfy a data-deletion request in an event-sourced system? (Hint: think about what must stay immutable and what could be made unreadable instead.)

CQRS

CQRS — Command Query Responsibility Segregation — separates the model used to change state from the model used to read it. Commands go through one path that enforces invariants; queries are served from separate, purpose-built read models. The two sides can use different schemas, different stores, and scale independently.

The motivation is that reads and writes want opposite things. Writes want a normalized model that enforces invariants: one place per fact, constraints, transactions. Reads want denormalized, pre-joined shapes matching each screen, with no joins at request time. A single model serving both compromises on each — which shows up as a "dashboard query" with eleven joins running against the tables the checkout path also needs.

                 command side                    query side

  client --> [ command handler ]           client --> [ query handler ]
                    |                                       |
              validate invariants                     no rules, no joins
                    |                                       |
             +--------------+                       +----------------+
             | write model  |  --events-->          |  read model(s) |
             | normalized   |   projector           |  denormalized  |
             +--------------+                       +----------------+
             Postgres, strict                       Elasticsearch / Redis /
             transactions                           a wide "screen-shaped" table

   Writes: correctness.   Reads: shape and speed.   Bridge: events + lag.

# ---- COMMAND side: rules live here, and only here --------------------------
class PlaceOrderHandler:
    def __init__(self, write_db, events): self.db, self.events = write_db, events

    def handle(self, cmd):
        if not cmd["items"]:
            raise ValueError("empty order")
        with self.db.transaction():                    # normalized + ACID
            order_id = self.db.insert("orders", {"customer_id": cmd["customer_id"],
                                                 "status": "placed"})
            for it in cmd["items"]:
                self.db.insert("order_items", {"order_id": order_id, **it})
            self.events.publish({"type": "OrderPlaced", "order_id": order_id,
                                 "customer_id": cmd["customer_id"],
                                 "items": cmd["items"]})   # same transaction
        return order_id                                # returns an ID, not a view

# ---- PROJECTOR: turns events into whatever shape a screen wants ------------
def project_order_placed(event, read_db, customers_read_model):
    customer = customers_read_model.get(event["customer_id"])
    read_db.upsert("order_summaries", {             # one row per SCREEN, not
        "order_id":      event["order_id"],         # per normalized entity
        "customer_name": customer["display_name"],  # pre-joined at write time
        "item_count":    len(event["items"]),
        "total":         sum(i["price"] * i["qty"] for i in event["items"]),
        "status":        "placed",
    })

# ---- QUERY side: a single-row lookup, no joins, no business rules ----------
def get_order_summary(order_id, read_db):
    return read_db.get("order_summaries", order_id)

CQRS is frequently paired with event sourcing but is entirely independent of it: you can run CQRS with two SQL schemas and a projector fed by change data capture, and you can event-source without separate read models. The pairing is popular because an event log is a natural projector input.

The cost is the lag between the two sides and the user-visible weirdness it creates: a user submits a form, is redirected to a list, and does not see their own change. This must be handled deliberately, not discovered.

Technique	How it works	Trade-off
Return the result	Command returns the created entity's view	Only fixes the next screen
Optimistic UI	Client renders the expected result immediately	Client must handle rejection
Read-your-writes routing	Route that user's reads to the write model	Extra load on the write side
Version token	Client passes the version it must see	Query may block or retry

Use CQRS where read and write loads or shapes genuinely diverge — a product catalogue with heavy faceted search and rare updates, an order system with complex dashboards — and not for a CRUD screen, where it doubles the model count for no return.

Key Takeaways

• CQRS separates the write model (normalized, invariant-enforcing) from read models (denormalized, shaped per screen).
• The two sides scale, store, and evolve independently; the bridge is events or change data capture, and the bridge has lag.
• CQRS and event sourcing are independent; they are combined often, not necessarily.
• Plan explicitly for read-your-writes, or users will not see the change they just made.
• Apply it where read and write requirements genuinely diverge, not to CRUD.

🧪 Practice

1. Design one read model for an "orders dashboard" screen and list the events that keep it current.
2. Implement read-your-writes for a form submission using two of the four techniques in the table and compare their cost.
3. Interview: What breaks if a projector is offline for an hour? (Hint: think about what is still correct, what is stale, and what happens when the projector restarts and processes the backlog.)

Choreography vs. Orchestration

A workflow spanning several services can be coordinated in two ways. Choreography: each service reacts to events and emits its own, with no central controller — the flow is an emergent property of the subscriptions. Orchestration: a coordinator explicitly tells each service what to do and tracks the workflow's state.

The trade is between coupling and visibility. Choreography adds services without changing existing ones — a new consumer just subscribes — which is why it scales so well organizationally. But nobody owns the workflow: no code describes it, and answering "why is this order stuck?" means correlating logs across five services. Orchestration puts the process in one readable place with explicit state, at the cost of a component that knows about everyone and can become a bottleneck.

 CHOREOGRAPHY                            ORCHESTRATION

 [order] --OrderPlaced--> (bus)          [ order saga orchestrator ]
      /        |        \                  |  1. reserve stock  ->[inventory]
     v         v         v                 |  2. charge card    ->[payments]
 [payment] [inventory] [email]             |  3. book shipment  ->[shipping]
     |         |                           |  4. send email     ->[email]
     v         v                           |
 PaymentTaken StockReserved                +-- state: step 2, retrying
     |                                         compensation is explicit
     v
 [shipping] (waits for BOTH)             one place to read the process;
                                         one place to change it
 no central knowledge;
 the flow exists only in the
 subscription graph

# ORCHESTRATION: the workflow is code you can read, test, and query.
class PlaceOrderSaga:
    STEPS = [
        # (do,                 undo)
        ("reserve_stock",      "release_stock"),
        ("charge_payment",     "refund_payment"),
        ("book_shipment",      "cancel_shipment"),
    ]

    def __init__(self, services, store): self.svc, self.store = services, store

    def run(self, order_id):
        done = []
        for do, undo in self.STEPS:
            self.store.set_state(order_id, step=do, status="running")  # queryable
            try:
                getattr(self.svc, do)(order_id)
                done.append(undo)
            except Exception as e:
                # Compensate in REVERSE order: undo only what actually happened.
                for compensation in reversed(done):
                    getattr(self.svc, compensation)(order_id)
                self.store.set_state(order_id, step=do, status="failed",
                                     error=str(e))
                raise
        self.store.set_state(order_id, step="complete", status="ok")

# CHOREOGRAPHY: the same flow, but no single place describes it.
def on_order_placed(event, inventory, bus):
    inventory.reserve(event["order_id"])
    bus.publish({"type": "StockReserved", "order_id": event["order_id"]})

def on_stock_reserved(event, payments, bus):
    payments.charge(event["order_id"])
    bus.publish({"type": "PaymentTaken", "order_id": event["order_id"]})

def on_payment_failed(event, inventory, bus):
    inventory.release(event["order_id"])      # compensation lives with each
    bus.publish({"type": "OrderCancelled", "order_id": event["order_id"]})
    # To understand the whole flow you must read every handler in every repo,
    # and nothing tells you a step is missing until an order sits half-done.

Dimension	Choreography	Orchestration
Coupling	Low; add consumers freely	Coordinator knows all steps
Workflow visibility	Emergent; no single description	Explicit and queryable
Debugging a stuck flow	Correlate logs across services	Read the saga's state row
Changing the process	Edit several services	Edit one component
Failure handling	Compensation scattered	Compensation centralized
Bottleneck risk	None	The coordinator
Best for	Notifications, fan-out reactions	Transactional multi-step business processes

The practical rule is to choose by how much the flow matters as a thing. If the sequence has business meaning, needs status reporting, has ordering constraints, or requires compensation on failure, orchestrate it — order fulfilment, loan approval, subscription provisioning. If services merely need to know something happened and their reactions are independent, choreograph — send the email, update the search index, warm a cache. Most real systems use both, and the mistake is choreographing a business-critical transaction because events were already available.

Key Takeaways

• Choreography is event reactions with no central controller; orchestration is an explicit coordinator that drives and tracks steps.
• Choreography minimizes coupling and maximizes extensibility but leaves the workflow undocumented and hard to debug.
• Orchestration gives one readable, queryable definition of the process and centralizes compensation, at the cost of a component that knows everyone.
• Orchestrate flows with business meaning, ordering, or compensation; choreograph independent reactions to a fact.
• The orchestrator must persist workflow state — an in-memory saga loses the process on restart.

🧪 Practice

1. Model a four-step checkout as an orchestrated saga with a compensation for each step, and state what happens if the orchestrator crashes at step 3.
2. Rewrite the same flow as choreography and list every place a developer must look to answer "where is order 8891 stuck?".
3. Interview: Your orchestrator has become a bottleneck and knows about twelve services. What do you do? (Hint: consider whether it is one workflow or several, and which steps really need central sequencing.)

Eventual Consistency in Practice

Eventual consistency means that if updates stop, all replicas and derived views will converge to the same state — but at any given moment they may disagree. Chapter 8 covered the theory; this topic is about the design work it forces, most of which is product design rather than infrastructure.

The first move is to stop asking "is eventual consistency acceptable?" as a system-wide question. It is a per-operation question, and the answers differ sharply within one product. A like counter can lag ten seconds. An account balance shown before a withdrawal cannot. Writing this down as a table is the most valuable half hour in an event-driven design.

Operation	Tolerable lag	Why
Follower count on a profile	Minutes	Nobody can tell, nobody is harmed
Search index after an edit	Seconds	Users retry; a stale hit is survivable
Order status after checkout	Immediate	The user just acted and is watching
Balance check for a payout	Immediate	Overdraft is a financial loss
Inventory count on a listing	Seconds	Oversell must be handled at reserve time

The second move is to design the user experience of inconsistency rather than letting it leak. A user who submits a change and does not see it assumes failure and retries, which creates duplicates. The standard remedies are read-your-writes routing for that user, optimistic UI that renders the expected outcome immediately, and explicit "pending" states that tell the truth: the system has accepted the request and has not finished.

# Read-your-writes without making the whole system strongly consistent.
# After a write, pin THAT user's reads to the authoritative source briefly.

PIN_WINDOW_SECONDS = 5

def after_write(session, now):
    session["read_pin_until"] = now + PIN_WINDOW_SECONDS   # only this user

def get_orders(user, session, now, write_db, read_replica):
    if session.get("read_pin_until", 0) > now:
        return write_db.query_orders(user.id)     # authoritative, costlier
    return read_replica.query_orders(user.id)     # cheap, may lag
    # 99.9% of reads still come from the replica; only the few seconds after a
    # user's own write are routed to the primary. This is the cheapest fix for
    # the single most common eventual-consistency complaint.

The third move is to make consumers idempotent and order-tolerant, because at-least-once delivery and reordering are facts of life in every broker. Idempotency comes from deduplicating on an event ID or from making the operation naturally repeatable. Ordering is handled by carrying a version or timestamp and ignoring anything older than what you already applied — last-write-wins on a monotonic field is crude but correct for most projections.

def apply_profile_update(event, read_db, seen):
    # 1. Idempotence: the same event delivered twice must change nothing.
    if event["event_id"] in seen:
        return
    seen.add(event["event_id"])

    # 2. Order tolerance: an older event arriving late must not overwrite newer
    #    state. Compare a monotonic version carried BY the event.
    current = read_db.get("profiles", event["user_id"]) or {"version": -1}
    if event["version"] <= current["version"]:
        return                                   # stale replay: drop it

    read_db.upsert("profiles", {"user_id": event["user_id"],
                                "name": event["name"],
                                "version": event["version"]})

Finally, converge on purpose. Systems that rely only on incremental events drift: messages are lost, projectors have bugs, and consumers are down during a schema change. Mature event-driven systems run a periodic reconciliation job that compares derived state against the source of truth and repairs differences, and they monitor consumer lag as a first-class SLI — because lag is the metric that tells you whether "eventually" is currently five seconds or five hours.

Key Takeaways

• Consistency requirements are per-operation, not per-system; write the table of tolerable lag before designing the flows.
• Design the UX of inconsistency: read-your-writes pinning, optimistic UI, and honest "pending" states prevent user-driven duplicate submissions.
• Every consumer must be idempotent (dedupe by event ID) and order-tolerant (ignore versions older than what it applied).
• Run reconciliation jobs against the source of truth; incremental updates drift.
• Monitor consumer lag as a first-class signal — it is the measurable form of "eventually".

🧪 Practice

1. Build the tolerable-lag table for a system you know and mark the operations that must not be served from a derived view.
2. Make an existing event handler idempotent and order-tolerant, then prove it by replaying its events shuffled and duplicated.
3. Interview: A user updates their profile and the old name still appears. Walk through your options. (Hint: distinguish fixing the lag from fixing what this particular user sees, and consider which is cheaper.)

Materialized Views

A materialized view is a query result stored as data and kept up to date, rather than computed on each request. In an event-driven system it is the natural consumer of events: subscribe to the facts, maintain a table shaped exactly like the screen or the API response, and serve reads from it with a single lookup.

The reason to precompute is that read and write frequencies rarely match. A seller dashboard showing revenue per product per day might be requested a hundred times an hour and change a few times an hour. Recomputing a large aggregation on every request wastes work proportional to the read rate; maintaining it on write does work proportional to the (much lower) write rate. The trade is storage plus staleness in exchange for a predictable, cheap read.

 COMPUTE ON READ                        MATERIALIZED VIEW

 GET /dashboard                          GET /dashboard
     |                                       |
     v                                       v
 SELECT p.name, SUM(oi.qty*oi.price)     SELECT * FROM seller_dashboard
 FROM orders o                           WHERE seller_id = ?
 JOIN order_items oi ON ...                  |
 JOIN products p ON ...                      v
 WHERE o.seller_id = ? AND o.day = ?     one indexed row read: ~1 ms
 GROUP BY p.name                              ^
     |                                        |
     v                                   [ projector ] <-- OrderPlaced events
 3-table join, 400 ms, every request      updates on WRITE, not on read

-- Some databases maintain views for you. Postgres materialized views are
-- explicit snapshots -- fast to read, but stale until refreshed.
CREATE MATERIALIZED VIEW seller_daily_revenue AS
SELECT o.seller_id,
       date_trunc('day', o.placed_at) AS day,
       COUNT(*)                        AS order_count,
       SUM(oi.qty * oi.unit_price)     AS revenue
FROM orders o
JOIN order_items oi ON oi.order_id = o.id
GROUP BY 1, 2;

CREATE UNIQUE INDEX ON seller_daily_revenue (seller_id, day);

-- CONCURRENTLY avoids locking readers during the rebuild; it requires the
-- unique index above and is slower than a plain REFRESH.
REFRESH MATERIALIZED VIEW CONCURRENTLY seller_daily_revenue;
-- Note the model: full recompute on a schedule. Simple and self-correcting,
-- but the cost grows with total data, not with what changed.

# Event-driven maintenance: incremental, so cost tracks CHANGE, not data size.
def on_order_placed(event, view_db):
    day = event["occurred_at"][:10]
    view_db.execute("""
        INSERT INTO seller_daily_revenue (seller_id, day, order_count, revenue)
        VALUES (%s, %s, 1, %s)
        ON CONFLICT (seller_id, day) DO UPDATE
          SET order_count = seller_daily_revenue.order_count + 1,
              revenue     = seller_daily_revenue.revenue + EXCLUDED.revenue
    """, (event["seller_id"], day, event["total"]))
    # Incremental updates are cheap but accumulate error: a missed or
    # double-applied event silently skews the number forever. Which is why
    # incremental views need periodic full recomputation to reconcile.

The two maintenance strategies are worth comparing directly, because most systems end up using both — incremental for freshness, periodic full rebuild for correctness.

Strategy	Cost scales with	Freshness	Failure behaviour
Full refresh on timer	Total data size	Up to the interval	Self-correcting each run
Incremental on events	Number of changes	Seconds	Errors accumulate; needs repair
Rebuild from log	Log length	On demand	The recovery mechanism itself

Three design rules make materialized views durable. Keep them disposable: a view must be rebuildable from the source of truth, which means the projector is deterministic and the events are retained long enough to replay. Version the view, so a schema change builds orders_summary_v2 alongside v1 and switches reads once it has caught up — far safer than mutating a live view. And monitor staleness explicitly, exposing the age of the newest applied event so a stalled projector is an alert rather than a support ticket about wrong numbers.

Key Takeaways

• A materialized view precomputes a query result and serves reads with a single lookup; it pays off whenever reads outnumber writes.
• Database-maintained views refresh in full on a schedule — simple and self-correcting, but the cost scales with total data.
• Event-driven projections update incrementally, so cost tracks change rate, at the risk of accumulated drift; reconcile with periodic full rebuilds.
• Views must be disposable and rebuildable from the source of truth; that is what makes bug fixes and schema changes safe.
• Build a new version alongside the old and switch reads after catch-up, and alert on view staleness.

🧪 Practice

1. Take a slow multi-join dashboard query and design the view table plus the events that maintain it, including the primary key.
2. Write the incremental projector for that view, then write the full rebuild that reconciles it, and describe how you would detect divergence.
3. Interview: A materialized view has been showing wrong totals for a week. What is your recovery plan? (Hint: think about what makes a rebuild possible at all, and how you would switch traffic without a gap.)

<a id="104-cloud-native-and-serverless"></a>

10.4 Cloud-Native and Serverless

Cloud-native describes applications built to be run by a platform that schedules, scales, and restarts them without human intervention. This subchapter covers the application-level contract that makes that possible, the packaging and orchestration layer beneath it, the serverless extreme where the platform owns the process lifecycle entirely, and the architectural choices — buy versus build, and how to isolate tenants — that surround them.

Twelve-Factor App Principles

The twelve-factor methodology is a set of application-level rules that make a process safely disposable, replicable, and portable. The rules predate Kubernetes and serverless but describe exactly the contract those platforms assume: the platform reserves the right to kill your process, start ten more, move them to another machine, and inject configuration. An application that violates the contract will appear to work and then fail in ways that look like platform bugs.

The factors that carry the most architectural weight are these:

• Config in the environment. Anything that differs between deployments — credentials, endpoints, feature flags — comes from environment variables, not from files baked into the build. The same artifact must run in staging and production.
• Backing services are attached resources. A database, a cache, or a queue is referenced by a URL in config, so swapping a local Postgres for a managed one is a config change, not a code change.
• Processes are stateless. Nothing persistent lives in local memory or on local disk; session state goes to a shared store. This is what makes any instance able to serve any request.
• Disposability. Start fast and shut down gracefully on SIGTERM: stop accepting new work, finish what is in flight, then exit. Slow startup delays scaling; ungraceful shutdown drops requests on every deploy.
• Dev/prod parity. Keep environments and backing services similar, so a bug cannot hide behind SQLite-in-dev, Postgres-in-prod.
• Logs are event streams. Write to stdout and let the platform route; an app that manages its own log files fights the platform's rotation and collection.

import os, signal, sys, time

# --- Factor 3: config from the environment, with no production defaults -----
DB_URL   = os.environ["DATABASE_URL"]                # required: fail fast
PORT     = int(os.environ.get("PORT", "8080"))       # platform-assigned
LOG_LEVEL= os.environ.get("LOG_LEVEL", "info")
# Note there is no `if env == "production"` branch anywhere: the SAME image
# runs everywhere, and only the injected values differ.

# --- Factor 6: stateless. Session data goes to a shared store, never here. --
# sessions = {}          # WRONG: instance 2 cannot see instance 1's sessions
sessions = RedisStore(os.environ["REDIS_URL"])       # RIGHT: shared, external

# --- Factor 9: disposability. Handle SIGTERM or lose in-flight requests. ----
shutting_down = False

def handle_sigterm(signum, frame):
    global shutting_down
    shutting_down = True                 # 1. fail readiness -> LB stops sending
    time.sleep(5)                        # 2. let the LB notice before we close
    server.stop(grace_seconds=30)        # 3. finish in-flight work
    sys.exit(0)

signal.signal(signal.SIGTERM, handle_sigterm)

def readiness():
    # The platform asks "should I send traffic?" -- answer honestly.
    return 503 if shutting_down else 200

# --- Factor 11: logs to stdout as a stream; the platform collects them ------
def log(event, **fields):
    print({"event": event, "level": LOG_LEVEL, **fields}, flush=True)

The most consequential factor is statelessness, because it is the one that quietly breaks. An in-memory cache is state; so is an uploaded file written to local disk, a background timer that only one instance should run, and a "warm-up" step whose result lives in a global. Each works with one instance and produces confusing, load-dependent bugs with several. The test is blunt: if killing any instance at any moment and starting a fresh one elsewhere loses nothing a user cares about, the application is stateless.

Factor	Violation that looks fine locally	Failure at scale
Config in env	config/production.yaml in the image	Rebuild per environment; leaked secrets
Stateless processes	In-memory session map	Random logouts across instances
Disposability	No SIGTERM handler	Dropped requests on every deploy
Logs as streams	Writing to /var/log/app.log	Logs lost when the container dies
Backing services	Hardcoded localhost:5432	Cannot move or fail over the DB

Key Takeaways

• Twelve-factor is the contract that lets a platform kill, restart, replicate, and relocate your process safely.
• Config comes from the environment so one artifact runs everywhere; no environment branches in code.
• Statelessness is the factor that breaks most subtly — in-memory caches, local files, and singleton timers are all state.
• Handle SIGTERM by failing readiness first, then draining, or every deploy drops requests.
• Log to stdout and treat backing services as attached, swappable resources.

🧪 Practice

1. Audit a service against the five factors in the table and write the smallest change that fixes the worst violation.
2. Add a graceful-shutdown path that fails readiness, drains for 30 seconds, and exits; verify no request is dropped during a rolling restart.
3. Interview: An app works with one instance but users are randomly logged out with three. What is wrong? (Hint: ask where session state lives and which instance can see it.)

Containers and Orchestration

A container packages an application with its dependencies into an image that runs identically anywhere the same kernel interface exists. Chapter 2 covered the mechanics — namespaces for isolation, cgroups for limits, a shared kernel rather than a guest OS. What matters architecturally is what containers make possible: a uniform deployable unit, so the platform can treat every workload the same way.

That uniformity is the precondition for orchestration. Once every service is "an image plus a config plus a resource request", a scheduler can place it on any machine with capacity, restart it when it dies, scale it by count, and roll a new version out gradually — without knowing anything about the language inside.

 WITHOUT ORCHESTRATION                  WITH ORCHESTRATION

 host-1: svc-a, svc-a, svc-b            you declare:  svc-a: 6 replicas, 500m CPU
 host-2: svc-b, svc-c                   the scheduler decides:
 host-3: (idle, wrong config)             - which hosts have room
                                          - spread across failure domains
 placement: a human and a wiki             - restart what dies
 a host dies: a human notices              - drain a host being replaced
 scaling up: a ticket
                                       you describe the DESIRED state;
                                       a control loop makes reality match

The core idea underneath every orchestrator is the reconciliation loop: compare desired state with observed state, act to close the gap, repeat forever. It is worth understanding as a pattern, because it explains why declarative platforms behave the way they do — including why deleting a pod by hand immediately gets you a new one.

# The control loop that every orchestrator is a large, careful version of.
def reconcile(desired, observe, create, destroy, interval=5):
    while True:
        actual = observe()                       # what is really running now
        for name, spec in desired.items():
            running = actual.get(name, 0)
            if running < spec["replicas"]:
                create(name, spec, count=spec["replicas"] - running)
            elif running > spec["replicas"]:
                destroy(name, count=running - spec["replicas"])
        sleep(interval)
# Consequences worth internalizing:
#  - Manual changes are UNDONE: the loop only respects the declared state.
#  - Convergence is eventual: "kubectl apply" returns before reality matches.
#  - The loop is level-triggered, not edge-triggered, so a missed event is
#    harmless -- the next pass observes reality again and corrects.

Image construction has real architectural consequences. Large images slow every scale-up and deploy, because the node must pull them; images running as root widen the blast radius of any compromise; images that bake in configuration break dev/prod parity. A multi-stage build addresses the first two directly.

# ---- build stage: compilers and dev dependencies live only here ------------
FROM golang:1.22 AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download                       # cached layer: deps change rarely
COPY . .
RUN CGO_ENABLED=0 go build -o /out/app ./cmd/server

# ---- runtime stage: only the binary ships --------------------------------
FROM gcr.io/distroless/static:nonroot     # no shell, no package manager
COPY --from=build /out/app /app
USER nonroot:nonroot                      # never root in the runtime image
EXPOSE 8080
ENTRYPOINT ["/app"]
# Result: ~15 MB instead of ~900 MB. That difference is felt on every cold
# node pull -- which is exactly when you are already scaling under load.

Two boundaries are worth stating plainly. Containers isolate by namespace on a shared kernel, so they are a weaker security boundary than a VM — for untrusted, multi-tenant code, use stronger isolation (microVMs, separate node pools). And containers are ephemeral by design: anything written to the container filesystem disappears on restart, so persistence belongs in an attached volume or, better, an external service.

Key Takeaways

• Containers make every workload a uniform deployable unit, which is what makes orchestration possible.
• Orchestrators run a reconciliation loop: declared desired state, observed actual state, actions to close the gap — so manual changes get reverted.
• Convergence is eventual and level-triggered; a missed event self-heals on the next pass.
• Multi-stage builds and minimal base images cut pull time and attack surface; image size is felt exactly when you are scaling under load.
• Containers share a kernel (weaker isolation than VMs) and have ephemeral filesystems; state belongs outside.

🧪 Practice

1. Convert a single-stage Dockerfile to multi-stage with a non-root runtime and measure the image size difference.
2. Write the pseudocode for a reconciliation loop that also handles a version change, not just a replica-count change.
3. Interview: Why does deleting a running container by hand not remove the workload? (Hint: think about what the platform is comparing against and how often.)

Kubernetes Concepts for Architects

Kubernetes is the dominant orchestrator, and an architect needs its object model rather than its full command surface. The model is small: a handful of objects that compose, all governed by the same declarative reconciliation loop.

                    +----------------------------------+
   kubectl apply -> |  API server  (desired state)      |
                    +----------------------------------+
                        |            ^
              controllers watch    kubelets report
                        v            |
   Deployment  -> ReplicaSet -> Pod (1+ containers, shared network + volumes)
        ^                            |
        |                       scheduled onto a Node
   Service (stable virtual IP + DNS) --> selects Pods by LABEL, load balances
        ^
   Ingress / Gateway (HTTP routing, TLS termination) --> Service

   ConfigMap / Secret --> injected into Pods as env vars or mounted files
   HorizontalPodAutoscaler --> adjusts the Deployment's replica count
   PersistentVolumeClaim --> durable storage attached to a Pod

The objects that matter for design decisions:

Object	What it is	Architectural meaning
Pod	Smallest unit; 1+ containers sharing network and volumes	The scheduling and failure unit
Deployment	Declares replicas of a stateless pod template	Rolling updates and rollbacks
StatefulSet	Pods with stable identity and storage	For databases and quorum systems
Service	Stable virtual IP + DNS over pods	Decouples callers from pod churn
Ingress/Gateway	HTTP(S) routing into the cluster	Where the strangler facade often lives
ConfigMap/Secret	Injected configuration	Twelve-factor config, externalized
HPA	Scales replicas on a metric	The autoscaling policy, as data

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders
spec:
  replicas: 4
  selector:
    matchLabels: { app: orders }         # must match the pod template's labels
  template:
    metadata:
      labels: { app: orders }
    spec:
      containers:
      - name: app
        image: registry.example.com/orders:1.14.2   # pin a digest in production
        ports: [ { containerPort: 8080 } ]
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef: { name: orders-db, key: url }   # config from env
        resources:
          requests: { cpu: "250m", memory: "256Mi" }  # used for SCHEDULING
          limits:   { cpu: "1000m", memory: "512Mi" } # enforced at RUNTIME
        # Two probes with two different jobs -- conflating them causes outages.
        readinessProbe:                  # "send me traffic?"  -> removed from
          httpGet: { path: /readyz, port: 8080 }   #    the Service if failing
          periodSeconds: 5
        livenessProbe:                   # "am I wedged?"      -> RESTARTED
          httpGet: { path: /healthz, port: 8080 }  #    if failing
          periodSeconds: 10
          failureThreshold: 3

Three details cause most production surprises. Requests versus limits: requests drive scheduling and limits are enforced at runtime — exceeding a memory limit is an OOM kill, while exceeding a CPU limit is throttling, which shows up as mysterious tail latency. Liveness probes are dangerous when they check dependencies: if /healthz returns 503 because the database is slow, Kubernetes restarts every pod simultaneously and turns a degraded dependency into a full outage; liveness should test only whether this process is wedged, while readiness may consider dependencies. And PodDisruptionBudget plus anti-affinity are what keep a rolling node upgrade from taking all replicas of a service at once.

Architecturally, the important consequence is that a great deal of what used to be application code becomes platform configuration: retries and traffic shifting at the ingress or service mesh, autoscaling policy in an HPA, secrets injection, rolling deploys and rollback. That is a real reduction in bespoke code, paid for with a platform that must itself be operated, upgraded, and understood — which is the honest argument for a managed control plane over a self-run one.

Key Takeaways

• The object model is small and composes: Pod, Deployment/StatefulSet, Service, Ingress, ConfigMap/Secret, HPA — all reconciled from declared state.
• Requests drive scheduling; limits are enforced at runtime — memory over limit is an OOM kill, CPU over limit is throttling that looks like latency.
• Readiness controls traffic, liveness controls restarts; a liveness probe that checks dependencies converts a degraded dependency into a cluster-wide restart.
• StatefulSets exist for workloads needing stable identity and storage; most application services should be Deployments.
• Kubernetes moves retries, scaling, secrets, and rollout policy from code into configuration — real leverage, at the cost of operating the platform.

🧪 Practice

1. Write a Deployment plus Service for a stateless API with sensible requests, limits, and both probes, and explain each probe's endpoint.
2. Explain what happens to a pod that exceeds its memory limit versus its CPU limit, and how each looks in monitoring.
3. Interview: Every pod of a service restarted at once during a database slowdown. What is the most likely cause? (Hint: look at what the liveness probe was checking.)

Function-as-a-Service

Function-as-a-Service (FaaS) takes the platform's ownership of the process to its conclusion: you deploy a function, the platform runs it in response to an event, scales the number of concurrent executions from zero to thousands, and bills for execution time. There are no instances to size, no idle capacity to pay for, and no servers to patch.

The architectural shift is that the unit of deployment becomes a single handler, and the unit of scaling becomes a single invocation. That produces an extremely good fit for spiky, event-driven, embarrassingly parallel work — and an extremely poor fit for anything that wants to hold state, keep a long-lived connection, or respond with consistently low latency.

# A FaaS handler: no server, no port, no lifecycle. Just the work.
import json, os

# Module scope runs ONCE per execution environment, not once per request.
# Put expensive setup here so warm invocations skip it entirely.
db = connect(os.environ["DATABASE_URL"])          # reused across invocations

def handler(event, context):
    # The handler must be idempotent: platforms retry on failure, and an
    # at-least-once event source can deliver the same record twice.
    record = json.loads(event["body"])
    key = record["idempotency_key"]

    if db.exists("processed", key):               # dedupe on a durable key
        return {"statusCode": 200, "body": '{"status":"already_done"}'}

    db.transaction(lambda tx: (
        tx.insert("orders", record),
        tx.insert("processed", {"key": key}),     # same transaction as the work
    ))
    # context.get_remaining_time_in_millis() exists because there IS a hard
    # timeout -- long work must be split or moved to a different compute model.
    return {"statusCode": 201, "body": '{"status":"created"}'}

The constraints are the design content, and they are not negotiable:

Constraint	Typical shape	Design consequence
Execution timeout	Seconds to ~15 minutes	Long jobs must be chunked or moved
No local state	Filesystem is ephemeral and scratch	State lives in a database or object store
Cold starts	Tens of ms to seconds	Latency-sensitive paths need warming or containers
Connection pooling	One pool per environment, N environments	Databases need a proxy/pooler
Per-invocation billing	Cost tracks usage exactly	Great at low/spiky volume, poor at sustained high volume

The database connection issue deserves emphasis because it is the most common way a FaaS design fails in production. Each concurrent execution environment holds its own connections, so 500 concurrent invocations can attempt 500 connections against a database sized for 100 — the function scales elastically and the database does not. The fixes are a connection proxy that multiplexes, an HTTP data API, or deliberately capping concurrency.

# Cost crossover: FaaS versus a container running continuously.
def faas_monthly(invocations, avg_ms, gb, per_req=0.20e-6, per_gb_s=0.0000166667):
    gb_seconds = invocations * (avg_ms / 1000) * gb
    return invocations * per_req + gb_seconds * per_gb_s

def container_monthly(instances, hourly=0.05):
    return instances * hourly * 730

for rps in (1, 10, 100, 500):
    inv = rps * 60 * 60 * 730
    # A rough sizing rule: ~100 rps per instance at 200 ms per request.
    instances = max(2, round(rps / 100 * 2))          # x2 for redundancy
    print(f"{rps:4} rps  faas ${faas_monthly(inv, 200, 0.5):8.2f}"
          f"   containers ${container_monthly(instances):8.2f}")
#    1 rps  faas $    5.24   containers $   73.00
#   10 rps  faas $   52.42   containers $   73.00
#  100 rps  faas $  524.16   containers $   73.00
#  500 rps  faas $ 2620.80   containers $  365.00
#
# Serverless wins decisively at low and spiky volume and loses badly at
# sustained high volume. Both numbers ignore operational effort, which is
# precisely the term that favours serverless for small teams.

The right architectural use is therefore targeted rather than total: glue between managed services, event processing, scheduled jobs, webhook receivers, and bursty workloads, alongside long-running containers for the steady, latency- sensitive core. "All serverless" and "no serverless" are both usually wrong.

Key Takeaways

• FaaS makes a single handler the unit of deployment and a single invocation the unit of scaling, with no idle cost.
• Handlers must be stateless and idempotent: platforms retry, and event sources deliver at least once.
• Hard timeouts, ephemeral storage, and cold starts are the binding constraints, not implementation details.
• Elastic concurrency against a fixed-size database needs a connection proxy or a concurrency cap.
• Cost favours FaaS at low and spiky volume and containers at sustained high volume; mix the two deliberately.

🧪 Practice

1. Convert a scheduled batch job into a function, identifying what must move out of local state and how you would chunk the work under a timeout.
2. Compute the monthly cost crossover between FaaS and containers for your own traffic profile, and state which assumption the answer is most sensitive to.
3. Interview: A function works in testing and exhausts database connections in production. Why? (Hint: think about how many execution environments exist at peak concurrency and what each one opens.)

Cold Starts and Concurrency Limits

A cold start is the latency added when a request must wait for a new execution environment: allocate a sandbox, load the runtime, initialize the code, then run the handler. A warm invocation skips all but the last step. This is the defining latency characteristic of serverless platforms and the most common reason teams abandon them for user-facing paths.

 COLD INVOCATION                          WARM INVOCATION

 |--sandbox--|--runtime--|--init--|--handler--|          |--handler--|
     ~50ms      ~100ms     ~200ms    ~20ms                   ~20ms
 total: ~370 ms                                          total: ~20 ms

 What lands in each bucket:
   sandbox  platform: image/snapshot size, VPC attachment
   runtime  language: JVM and .NET slowest, Go/Rust/JS fastest
   init     YOURS: imports, config loads, client construction, DB connect
   handler  the actual request

The init phase is the part you control, and it is usually the largest. Every import, every SDK client constructed at module scope, and every configuration fetch happens before the first request is served. Reducing dependencies, lazily constructing clients that only some code paths need, and avoiding network-dependent initialization all cut cold-start time directly.

import os

# ---- module scope: paid on EVERY cold start --------------------------------
# Constructing a client is cheap; its first network call is not. Construct
# eagerly (so warm calls reuse it), but never make network calls at import.
s3 = make_s3_client()                     # no I/O yet

_analytics = None                         # only some requests need this
def analytics():
    global _analytics
    if _analytics is None:                # lazily built: cold path unaffected
        _analytics = make_analytics_client()
    return _analytics

# Anti-pattern: fetching secrets at import time adds a full round trip to the
# cold start of every environment, on every scale-up.
#   SECRET = secrets_manager.get("db-password")     # DON'T
# Prefer an injected environment variable, or fetch on first use and cache.

def handler(event, context):
    ...

Concurrency, not requests per second, is the quantity that determines how many environments exist — and Little's Law converts between them. A function averaging 200 ms at 500 rps needs about 100 concurrent environments; if traffic doubles instantly, 100 more must be cold-started, and every one of those requests pays the cold-start penalty.

def concurrency(rps, avg_seconds):
    return rps * avg_seconds                 # Little's Law: L = lambda * W

print(concurrency(500, 0.200))               # 100.0 environments in steady state
print(concurrency(500, 1.500))               # 750.0  <- slow handlers cost
#                                               concurrency, and concurrency
#                                               is what hits the account limit

def cold_start_share(rps, avg_s, spike_factor, cold_ms, warm_ms):
    steady = concurrency(rps, avg_s)
    new_envs = steady * (spike_factor - 1)              # environments to create
    reqs_in_spike = rps * spike_factor * avg_s
    cold_frac = min(1.0, new_envs / reqs_in_spike)
    p_avg = cold_frac * cold_ms + (1 - cold_frac) * warm_ms
    return round(cold_frac * 100, 1), round(p_avg, 1)

print(cold_start_share(500, 0.2, 2.0, 370, 20))   # (50.0, 195.0)
# During an instant doubling, half the requests are cold and the average
# latency is 10x normal. THAT is what users experience during a flash sale --
# and why provisioned/warm concurrency exists.

The mitigations, in order of preference: reduce init work (cheapest and helps everywhere), choose a fast runtime for latency-sensitive functions, provisioned concurrency to keep N environments permanently warm (removes cold starts and removes the scale-to-zero cost benefit for those N), and snapshot-based restore where the platform offers it.

Concurrency limits also serve as a blast-radius control. Account-level limits are shared across functions, so one runaway function can starve the others — reserving concurrency per function both guarantees capacity to the important ones and caps the damage a buggy loop can do, including to the databases downstream.

Key Takeaways

• Cold start = sandbox + runtime + your init; the init phase is the part you control and usually the largest.
• Never do network I/O at module scope; construct clients eagerly but fetch lazily and cache.
• Concurrency, not RPS, determines environment count: L = rps x duration, so slow handlers multiply concurrency and hit limits sooner.
• Traffic spikes force mass cold starts, so average latency degrades exactly when load is highest; provisioned concurrency is the direct fix.
• Per-function concurrency reservations both guarantee capacity and cap the blast radius of a runaway function.

🧪 Practice

1. Compute the steady-state concurrency for 2,000 rps at 150 ms, then at 600 ms, and say what breaks first as duration grows.
2. Take a handler with heavy module-scope initialization and restructure it into eager-construct/lazy-connect, listing what you moved and why.
3. Interview: Your p99 latency is fine at steady load and terrible during spikes. What is happening and what are your options? (Hint: think about how many new execution environments a spike requires and what each one costs.)

Managed Services vs. Self-Hosted

Every architecture makes a series of buy-versus-run decisions: managed Postgres or your own, hosted Kafka or a self-run cluster, a search service or an Elasticsearch cluster you operate. The decision is usually framed as cost, which is why it is usually made badly — the sticker price of a managed service is visible while the cost of running the alternative is spread across salaries, on-call, and the features you did not build.

The honest comparison is total cost of ownership over the expected lifetime, including the parts nobody itemizes: initial setup, upgrades and patching, backup and verified restore, capacity planning, security review, the on-call burden, and the opportunity cost of engineers doing this instead of product work.

# TCO comparison over a year -- with the invisible costs made visible.
ENG_HOUR = 100          # loaded cost of an engineer-hour

def self_hosted(instances, hourly, setup_h, monthly_ops_h, incidents, hours_each):
    infra = instances * hourly * 24 * 365
    labour = (setup_h + monthly_ops_h * 12 + incidents * hours_each) * ENG_HOUR
    return infra + labour

def managed(monthly_fee, monthly_ops_h, incidents, hours_each):
    labour = (monthly_ops_h * 12 + incidents * hours_each) * ENG_HOUR
    return monthly_fee * 12 + labour

# A 3-node Kafka cluster, sized identically both ways.
diy = self_hosted(instances=3, hourly=0.34, setup_h=120,
                  monthly_ops_h=16, incidents=4, hours_each=8)
svc = managed(monthly_fee=1800, monthly_ops_h=2, incidents=1, hours_each=4)

print(f"self-hosted ${diy:,.0f}   managed ${svc:,.0f}")
# self-hosted $47,133   managed $24,400
#
# Infrastructure is $8,900 of the self-hosted total; the other $38,000 is
# people. Any comparison that comes out strongly in favour of self-hosting has
# usually priced the servers and forgotten the engineers.

The calculation flips in specific, recognizable situations, and those are the ones worth naming rather than arguing from principle.

Prefer managed when	Prefer self-hosted when
The component is generic (see subdomains, 10.2)	It is your core differentiator
The team is small or has no relevant expertise	You have deep in-house expertise already
Time to market matters more than unit cost	Scale makes the managed premium enormous
The workload is spiky or unpredictable	Load is steady and predictable
Compliance is satisfied by the provider	Regulation forbids the provider or the region
You need it working next week	You need control the service does not expose

The real risk of managed services is lock-in, and the useful response is to size it rather than to fear it. Ask what a migration would actually cost: a managed Postgres is standard Postgres, so the exit is a dump and restore. A proprietary serverless database with a bespoke query API may have no equivalent anywhere, so the exit is a rewrite. Depend freely on the first kind; depend on the second only where the payoff is large and deliberate, and keep the coupling behind a port (10.1) so the blast radius of a change is one adapter.

A final decision rule that resolves most arguments: run it yourself only if operating it well is something your business should be good at. Nobody wins customers by patching Kafka brokers.

Key Takeaways

• Compare total cost of ownership, not sticker price; labour usually dominates the self-hosted side.
• Managed services suit generic subdomains, small teams, spiky load, and speed to market; self-hosting suits core differentiators, deep expertise, steady scale, and hard regulatory constraints.
• Price lock-in concretely by asking what an exit would cost — standard-protocol services are cheap to leave, proprietary APIs are not.
• Hide proprietary dependencies behind a port so a change touches one adapter.
• Run it yourself only if operating it well is a thing your business should be good at.

🧪 Practice

1. Build a TCO model for one component in your stack with explicit setup, monthly ops, and incident-hour assumptions, then find the break-even scale.
2. For one managed dependency, write the exit plan and estimate its cost in engineer-weeks.
3. Interview: When is it right to run your own database? (Hint: think about expertise, scale economics, and whether the constraint is regulatory or merely a preference for control.)

Multi-Tenancy Models

Multi-tenancy is serving many customers (tenants) from shared infrastructure. The architectural decision is where the isolation boundary sits, and it ranges from a shared row-level model to a full stack per tenant. The choice affects cost per tenant, blast radius, compliance posture, and how hard it is to give one large customer special treatment — so it is genuinely difficult to change later.

 SHARED EVERYTHING          SCHEMA PER TENANT         STACK PER TENANT

 +---------------------+    +---------------------+   +------+  +------+
 |   one app cluster   |    |   one app cluster   |   | app  |  | app  |
 +---------------------+    +---------------------+   +------+  +------+
 |     one database    |    |  db: schema_a       |   | db_a |  | db_b |
 |  rows tagged with   |    |      schema_b       |   +------+  +------+
 |     tenant_id       |    |      schema_c       |    tenant A  tenant B
 +---------------------+    +---------------------+

 cheapest per tenant        middle ground             most expensive
 one bad query -> all       per-tenant backup/        strongest isolation
 hardest isolation          restore possible          N deploys to manage

Model	Cost/tenant	Isolation	Per-tenant restore	Scale limit
Shared schema (row)	Lowest	Weakest	Hard	Thousands+ of tenants
Schema per tenant	Low-medium	Medium	Straightforward	Hundreds to low thousands
Database per tenant	Medium-high	Strong	Trivial	Hundreds
Stack per tenant	Highest	Strongest	Trivial	Tens

In the shared model, everything depends on the tenant filter being present on every query, and "every developer remembers" is not a control. The enforcement must be structural — row-level security in the database, or a repository layer that makes an unfiltered query impossible to express.

-- Structural isolation: the DATABASE enforces the filter, not the application.
ALTER TABLE documents ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON documents
  USING (tenant_id = current_setting('app.tenant_id')::uuid);

-- The application sets the tenant once per request/transaction:
SET LOCAL app.tenant_id = '3f2b...';

-- Now a forgotten WHERE clause returns this tenant's rows only:
SELECT * FROM documents;          -- implicitly filtered by the policy
-- A cross-tenant leak now requires bypassing RLS deliberately, rather than
-- forgetting a clause during a rushed Friday change.

# The "noisy neighbour" problem: shared infrastructure means one tenant's
# workload can consume capacity others need. Fairness must be explicit.
def admit(tenant_id, quotas, usage):
    q = quotas.get(tenant_id, quotas["default"])
    if usage.concurrent(tenant_id) >= q["max_concurrent"]:
        raise TooManyRequests(tenant_id)         # cap concurrency per tenant
    if usage.rate(tenant_id) >= q["rps"]:
        raise RateLimited(tenant_id)             # cap throughput per tenant
    return True

QUOTAS = {
    "default":      {"rps": 50,   "max_concurrent": 10},
    "enterprise_7": {"rps": 2000, "max_concurrent": 200},   # paid tier
}
# Without per-tenant quotas, one customer's batch import degrades everyone
# else's latency, and your SLO is effectively set by your worst-behaved user.

Three cross-cutting concerns follow from any shared model. Every log line, metric, and trace should carry the tenant ID, because otherwise you cannot answer "is this slow for everyone or for one customer?". Migrations must be planned per model — one shared schema means one migration for everyone and no staged rollout, while schema-per-tenant means a migration runner that handles partial failure across hundreds of schemas. And tiering is legitimate: the common mature pattern is a shared pool for most tenants plus dedicated stacks for the few large or regulated ones, which keeps the cost curve sane while satisfying the customers who will pay for isolation.

Key Takeaways

• The multi-tenancy decision is where the isolation boundary sits, and it trades cost per tenant against blast radius and compliance posture.
• Shared-schema isolation must be enforced structurally — row-level security or a repository that cannot express an unfiltered query.
• Noisy neighbours are guaranteed without explicit per-tenant quotas on rate and concurrency.
• Tag all telemetry with tenant ID; without it you cannot separate a customer problem from a system problem.
• Migration strategy differs sharply per model, and a hybrid — shared pool plus dedicated stacks for large or regulated tenants — is the common mature answer.

🧪 Practice

1. Choose a tenancy model for a B2B SaaS with 5,000 small customers and 10 enterprise customers, and justify the split.
2. Implement row-level security for one table and write a test proving a missing WHERE clause cannot leak another tenant's rows.
3. Interview: One tenant's bulk import is degrading latency for everyone. What do you change? (Hint: think about what limits exist per tenant today, and where you would enforce a new one without deploying to every service.)

---

<a id="11-security-privacy-and-compliance"></a>