1. Foundations of System Design

This chapter builds the vocabulary and mental habits that every later chapter assumes: what "system design" actually means, the quality attributes that designs are judged by, how to turn a vague prompt into numbers you can defend, and how to reason about — and record — trade-offs. Get these foundations right and the rest of system design becomes applied engineering rather than guesswork.

<a id="11-what-system-design-is"></a>

1.1 What System Design Is

Before designing anything, you need to know what kind of decisions belong to system design, at what level of detail they are made, and what inputs (requirements, constraints, stakeholders) drive them.

System Design vs. Software Design

A single program running on one machine has a simple failure model: it either runs or it does not. The moment your product spans several machines, a network, and a database that outlives any single process, new questions appear that no amount of clean class design answers. What happens when the network drops a message? Where does data live, and who owns it? Which component becomes the bottleneck at ten times the traffic? Those questions are the subject of system design.

Software design is the craft of structuring code inside a component: classes, modules, interfaces, algorithms, and dependency direction. System design is the craft of structuring components into a working whole: services, data stores, queues, network boundaries, and the operational behaviour that emerges from them.

An analogy: software design is designing the interior of a house — where the rooms go, how you move between them, whether the kitchen layout works. System design is city planning — where the water mains, power grid, roads, and hospitals go, how the city keeps working when a substation fails, and how it absorbs a doubling of the population. Beautiful houses in a city with no water supply are still uninhabitable.

SOFTWARE DESIGN (inside one component)      SYSTEM DESIGN (between components)

  +----------------------------+             [Client]
  |  UrlShortenerService       |                 |
  |   - encoder: Base62        |                 v
  |   - repo: UrlRepository    |            [Load Balancer]
  |   + shorten(url): String   |              /         \
  |   + resolve(id): String    |        [API srv]     [API srv]
  +----------------------------+             |             |
            |                                +------+------+
            v                                       |
  +----------------------------+           [Cache]  |  [ID generator]
  |  UrlRepository (interface) |                \   |   /
  +----------------------------+                 [Database]
                                                     |
                                             [Replica / backups]

// Software design question: what is the right abstraction boundary here?
public interface UrlRepository {
    void save(String id, String longUrl);
    Optional<String> find(String id);   // returns empty instead of null
}

// System design question: what does `find` actually do at 50k reads/second?
//   - hit an in-process cache first?           (latency vs. memory cost)
//   - hit a shared cache cluster next?         (extra hop, eviction policy)
//   - fall through to a replica, not primary?  (stale reads become possible)
//   - what if the cache cluster is down?       (thundering herd on the DB)

Dimension	Software design	System design
Unit of thought	Class, module, function	Service, data store, network hop
Main risks	Coupling, unclear abstractions	Bottlenecks, partial failure, data loss
Typical artifact	Class diagram, interface, tests	Architecture diagram, capacity model, ADR
Failure model	Exception, wrong output	Timeout, partition, degraded mode, cascading failure
Feedback loop	Seconds (compile and test)	Days to months (load, growth, incidents)

The two are not rivals: a system made of well-designed components is easier to split, replace, and scale, and a system with sane boundaries makes each component's code simpler.

Key Takeaways

• Software design structures code inside a component; system design structures components into a whole.
• System design exists because networks, machines, and data outlive single processes and fail partially.
• System-level artifacts are diagrams, capacity models, and decision records — not class hierarchies.
• Good system boundaries make component-level code simpler, and vice versa.

🧪 Practice

1. Take a personal project and list every design decision you made. Sort each into "software design" or "system design", and note which category you spent the least time on.
2. Draw the two-level picture (component internals and component topology) for a blogging platform with comments and image uploads.
3. Interview: A candidate answers "design a URL shortener" by describing class names and methods. What is missing from that answer, and what would you ask next? (Hint: think about what changes when the service receives one billion redirects a day instead of ten.)

High-Level vs. Low-Level Design

Design happens at two zoom levels, and mixing them is one of the most common ways an interview or a design review goes off the rails. If you start naming database columns before agreeing on whether reads or writes dominate, you are polishing the details of a shape that may not survive.

High-level design (HLD) answers: what are the major pieces, how do they talk, where does state live, and how does data flow through the system? Its output is boxes and arrows with protocols and data stores labelled. Low-level design (LLD) answers: inside one box, what are the exact schemas, API signatures, algorithms, class structures, and edge-case rules?

Think of an architect's site plan versus the construction drawings for a single room. Both are needed. The site plan is what you argue about first, because moving a wall is cheap and moving a building is not.

HIGH-LEVEL DESIGN — "Notification service"

 [Producer svc] --publish--> [Kafka: notif.events] --consume--> [Dispatcher]
                                                                    |
                              +-------------------+----------------+
                              |                   |                |
                          [Email wkr]        [SMS wkr]        [Push wkr]
                              |                   |                |
                          [Email API]         [SMS API]        [APNs/FCM]

 State:    [Postgres: user_prefs]     [Redis: dedup keys, 24h TTL]
 Contract: at-least-once delivery, therefore consumers must be idempotent

-- LOW-LEVEL DESIGN: one box, fully specified
CREATE TABLE user_prefs (
    user_id      BIGINT       PRIMARY KEY,
    email_opt_in BOOLEAN      NOT NULL DEFAULT TRUE,
    sms_opt_in   BOOLEAN      NOT NULL DEFAULT FALSE,
    quiet_from   TIME         NULL,         -- local time; NULL = no quiet hours
    quiet_to     TIME         NULL,
    timezone     TEXT         NOT NULL,     -- IANA name, e.g. 'Europe/Rome'
    updated_at   TIMESTAMPTZ  NOT NULL DEFAULT now()
);

-- The read path is by user_id only, so the primary key index suffices;
-- no secondary index is added until a real query needs one.

# LOW-LEVEL DESIGN: the idempotency rule the HLD contract demands
def handle(event, redis, sender):
    key = "notif:" + event["event_id"]         # producer-supplied unique id
    # SET NX returns False if the key already exists -> duplicate delivery
    if not redis.set(key, "1", nx=True, ex=86_400):
        return "duplicate-skipped"             # at-least-once made safe
    sender.send(event)                         # side effect happens once
    return "sent"

Question	HLD	LLD
Do we need a message queue at all?	yes	no
Which columns are indexed?	no	yes
Is the API REST or gRPC?	yes	no
What is the exact error payload shape?	no	yes
Which service owns user data?	yes	no
What is the retry backoff formula?	no	yes

A practical rule for a 45-minute design interview: spend roughly the first two thirds at HLD, then let the interviewer pull you into one or two LLD areas (usually the data model or the hottest code path).

Key Takeaways

• HLD is components, contracts, and data flow; LLD is schemas, signatures, and algorithms.
• Settle HLD first — reversing a high-level decision is far more expensive.
• Every HLD arrow implies an LLD contract (format, retries, idempotency).
• In interviews, drive HLD yourself and drill into LLD where prompted.

🧪 Practice

1. Classify these as HLD or LLD: choosing Cassandra over Postgres; choosing a composite partition key; putting a CDN in front of images; picking the exact JWT claim names.
2. Draw an HLD for a food-delivery order flow, then produce the LLD schema for the orders table only.
3. Interview: You presented an HLD and the interviewer says "zoom into the write path". What three artifacts should you be ready to produce? (Hint: one describes structure, one describes an interface, one describes behaviour under failure.)

Functional Requirements

A functional requirement (FR) states what the system must do — an observable behaviour a user or another system can trigger and verify. FRs are the reason the system exists; everything else in design is in service of delivering them at acceptable quality.

The reason to write them down explicitly, even for a "simple" prompt, is that FRs determine the shape of the system. "Users can follow other users" implies a graph and a fan-out problem. "Users can search their own message history" implies an index. One sentence of FR can add an entire subsystem, so a design that starts before the FR list is agreed on is a design built on sand.

Good FRs read as actor + action + object + observable outcome, and are testable. Keep quality adjectives ("quickly", "reliably") out of them — those belong to non-functional requirements.

Prompt: "Design a ride-hailing app."

Core FRs (must have; these drive the architecture)
  FR1  A rider can request a ride from a pickup to a dropoff location.
  FR2  The system matches a request to exactly one nearby available driver.
  FR3  Rider and driver can see each other's live location during a trip.
  FR4  The system computes a fare and charges the rider when the trip ends.
  FR5  A rider can view past trips and receipts.

Out of scope for v1 (state this explicitly!)
  Ride pooling, scheduled rides, driver payout ledger, ratings and reviews.

# FRs map almost mechanically onto an API surface — a useful sanity check.
# FR1 -> POST /rides              {pickup, dropoff}  -> 202 + ride_id
# FR2 -> (internal) matching service assigns exactly one driver_id
# FR3 -> GET  /rides/{id}/location  (or a WebSocket stream)
# FR4 -> POST /rides/{id}/complete  -> triggers fare calculation + payment
# FR5 -> GET  /rides?rider_id=...&cursor=...
#
# If an FR maps to no endpoint, event, or background job, it is underspecified.

A useful discipline is to rank FRs before designing:

Rank	FR	Architectural consequence
P0	Request a ride	Write path, request store
P0	Match to one driver	Geospatial index, uniqueness/locking
P0	Live location	Streaming transport, very high write rate
P1	Fare and payment	Transactions, idempotency, external PSP
P2	Trip history	Read-optimized store, pagination

Key Takeaways

• FRs describe observable behaviour: actor, action, object, outcome.
• Each FR tends to imply concrete architecture (index, queue, transport).
• Rank FRs; design for P0 first and explicitly defer the rest.
• Naming what is out of scope is as valuable as naming what is in.

🧪 Practice

1. Write five functional requirements for a collaborative document editor, and mark which one is hardest to satisfy.
2. For each FR you wrote, name the API endpoint or event it implies.
3. Interview: Given the prompt "design Twitter", list the three FRs you would commit to in the first two minutes and justify the ranking. (Hint: pick the ones whose read/write pattern most shapes the storage design.)

Non-Functional Requirements

Two systems can satisfy identical functional requirements and still be wildly different: one serves 100 users from a laptop, the other serves 100 million across three continents under a 99.99% availability contract. Non-functional requirements (NFRs) capture how well the system must do what it does, and they — not the FRs — are what force load balancers, caches, replicas, queues, and multi-region topologies into your diagram.

The single most important property of an NFR is that it must be measurable. "The system should be fast" cannot be designed for, tested, or violated. "p99 read latency under 200 ms at 5,000 requests per second" can be.

A useful template: [metric] [operator] [target] [under what conditions] [measured where].

Vague NFR	Measurable NFR
"Must be fast"	p99 latency < 200 ms for GET /timeline at 5k RPS, measured at the load balancer
"Must be available"	99.95% monthly availability of the write API, excluding scheduled maintenance
"Must scale"	Sustains 10x current peak (50k RPS) with proportional cost and no code changes
"Must not lose data"	RPO <= 5 minutes and RTO <= 30 minutes for the loss of a full region
"Must be secure"	TLS 1.2+ in transit, AES-256 at rest, every PII read audited
"Must be cheap"	Infrastructure cost <= $0.60 per 1,000 API requests at steady state

Categories to sweep through so you do not miss a whole class of requirement

  Performance   latency percentiles, throughput, time to first byte
  Scalability   growth headroom, elasticity, per-tenant limits
  Availability  uptime target, degradation modes, maintenance windows
  Reliability   correctness, RPO/RTO, retry and duplicate behaviour
  Security      authn/authz, encryption, audit trails, secret handling
  Compliance    GDPR/HIPAA/PCI, data residency, retention and deletion
  Operability   deploy frequency, rollback time, observability, on-call load
  Cost          unit economics, budget ceiling, cost per tenant

# An NFR you cannot turn into an assertion is a wish, not a requirement.
def assert_nfrs(results):
    assert results.p99_ms < 200,       f"p99 {results.p99_ms}ms exceeds 200ms"
    assert results.error_rate < 0.001, f"error rate {results.error_rate}"
    assert results.throughput_rps > 5000
    # These same assertions live in the load-test suite and in the alert rules,
    # which is how an NFR stays true after launch instead of only at design time.

NFRs also conflict with one another, which is why the next subchapter treats them as quality attributes to be traded off rather than boxes to be ticked: stronger consistency costs latency, higher availability costs money, tighter security costs developer velocity.

Key Takeaways

• NFRs specify quality: performance, availability, security, cost, operability, compliance.
• An NFR is only real if it is measurable and can fail a test.
• NFRs, far more than FRs, dictate topology (caches, replicas, regions).
• NFRs conflict; expect to negotiate targets, not to maximize all of them.

🧪 Practice

1. Rewrite these as measurable NFRs: "search should feel instant", "the system should handle Black Friday", "we shouldn't lose orders".
2. For a photo-sharing app, write one NFR in each of the eight categories listed above.
3. Interview: You are told the target is 99.999% availability. What follow-up questions do you ask before accepting it? (Hint: think about what it costs, what exactly it is measured against, and whether every endpoint needs the same number.)

Constraints and Assumptions

No design happens in a vacuum. Constraints are conditions you do not get to choose — they narrow the solution space from outside. Assumptions are gaps in your knowledge that you fill in deliberately so you can keep moving, and label so they can be challenged later.

The discipline matters because unstated assumptions are the most common cause of designs that are technically excellent and practically wrong. If you silently assume a 100:1 read/write ratio and the real ratio is 1:1, your cache-heavy design collapses. State it out loud and someone corrects you in thirty seconds instead of three months.

Constraints come in recognisable families:

Type	Examples
Business	Launch in 3 months; budget $20k/month; must support a free tier
Regulatory	EU user data stays in the EU; 7-year financial record retention
Technical	Must integrate a legacy SOAP billing system; on-premise only
Organizational	Four engineers, none with Kubernetes experience
Physical	Speed of light: roughly 150 ms round trip Europe to Australia
Contractual	A third-party API allows 100 requests/second, hard cap

Physical constraints deserve special respect: no architecture beats the speed of light, so a globally synchronous, strongly consistent write will always pay cross-region round trips.

ASSUMPTION LOG — URL shortener (write this before designing)

 A1  100M new URLs per month                    [source: PM estimate]
 A2  Read:write ratio about 100:1               [assumed, VERIFY]
 A3  Average long URL length about 100 bytes    [assumed]
 A4  Links never expire unless the user deletes [confirmed with PM]
 A5  Traffic peaks at 3x the average            [assumed, VERIFY]
 A6  Custom aliases are a v2 feature            [scope decision]

 The VERIFY flags make risk visible: if A2 is wrong by 10x, the sizing and
 cost of the entire caching layer changes.

# Make assumptions executable so their impact is testable, not rhetorical.
ASSUMPTIONS = {
    "new_urls_per_month": 100_000_000,
    "read_write_ratio":   100,
    "peak_factor":        3,
}

def write_qps(a=ASSUMPTIONS):
    # ~2.6M seconds in a month; round to 2.5M for easy mental arithmetic
    return a["new_urls_per_month"] / 2_500_000

def peak_read_qps(a=ASSUMPTIONS):
    return write_qps(a) * a["read_write_ratio"] * a["peak_factor"]

print(round(write_qps()), round(peak_read_qps()))   # 40 12000
# Change one assumption, re-run, and watch which parts of the design move.

Key Takeaways

• Constraints are imposed on you; assumptions are chosen by you and must be labelled as such.
• Unstated assumptions are the top cause of confidently wrong designs.
• Keep an assumption log with a source and a "verify" flag per entry.
• Physical limits (speed of light, disk seek time) are non-negotiable.

🧪 Practice

1. List five constraints that would apply to a hospital records system but not to a photo-sharing app.
2. Write an assumption log for "design a food delivery app" with at least six entries, each tagged confirmed or assumed.
3. Interview: You assume a 100:1 read/write ratio and the interviewer says "actually it's 1:1". Which parts of your design change first? (Hint: consider what caching buys you when almost every request mutates state.)

Stakeholders and Design Goals

A system has more customers than its users. Product managers want features shipped; end users want speed; the on-call engineer wants the thing to page rarely and be debuggable at 3 a.m.; finance wants predictable spend; security and legal want data handled correctly. Each group judges the same design by a different metric, and those metrics conflict.

Identifying stakeholders early does two things. It surfaces requirements you would otherwise miss (nobody writes "must be debuggable" on a feature ticket until after the first outage), and it gives you a defensible way to break ties: when two stakeholders' goals collide, you appeal to the priority order, not to the technical detail.

Stakeholder	Cares about	Metric they judge you by
End user	Speed, correctness, uptime	p95 latency, error rate
Product / business	Time to market, velocity	Lead time for change, roadmap slip
On-call engineer	Debuggability, blast radius	MTTR, pages per week
Finance	Predictable, proportional spend	Cost per active user or request
Security	Least privilege, auditability	Vulnerabilities, audit findings
Legal / compliance	Residency, retention, deletion	Audit result, data-request latency
Partner / API user	Stable contracts, clear errors	Breaking changes per year

Turn these into an explicit, ordered set of design goals. The ordering is the whole point — an unordered list of goals gives no guidance when two of them pull apart.

DESIGN GOALS — internal payments service (ordered, v1)

 G1  Correctness: never double-charge, never lose a settled payment
 G2  Availability: 99.99% on the charge endpoint
 G3  Auditability: every state transition reconstructable for 7 years
 G4  Latency: p99 < 800 ms end to end
 G5  Cost: < $0.002 per transaction
 G6  Developer velocity: a new payment method integrated in < 2 weeks

 Tie-break rule: the lower-numbered goal wins.
 -> Faced with a faster path (G4) versus a guaranteed-once path (G1), take the
    slower, safer path — and record why in the design doc.

# The same trade-off expressed as code the team can point at during review.
def on_charge_request(req, ledger, gateway):
    # G1 outranks G4: record intent durably BEFORE calling the gateway, even
    # though it adds a synchronous durable write to the hot path.
    intent = ledger.record_intent(req.idempotency_key, req.amount)
    if intent.already_settled:
        return intent.result                   # replay-safe: no second charge
    result = gateway.charge(req)               # external call; may time out
    ledger.record_result(intent.id, result)    # crash here is reconcilable
    return result

Key Takeaways

• Users are only one stakeholder; ops, finance, security, and legal also define real requirements.
• Each stakeholder judges the design by a different metric — collect them.
• Convert stakeholder concerns into an ordered list of design goals.
• The ordering is the tie-break rule you will lean on in every trade-off.

🧪 Practice

1. List the stakeholders for an internal HR system and the one metric each would use to call it a success.
2. Write an ordered set of five design goals for a public API product, then describe a concrete decision the ordering would settle.
3. Interview: Product wants a feature that would drop availability from 99.99% to 99.9%. How do you frame the decision? (Hint: convert both sides into the same unit — minutes of downtime, revenue, or affected users.)

<a id="12-core-quality-attributes"></a>

1.2 Core Quality Attributes

Quality attributes are the dimensions along which every design is judged and traded off. Each has a definition, a way to measure it, and a characteristic cost — knowing all three is what lets you argue about designs with numbers instead of adjectives.

Scalability

Scalability is not "being fast". A system is scalable if it can absorb more work by adding more resources, at a cost that grows no worse than proportionally. A fast system that saturates at 1,000 requests per second and cannot be improved by adding hardware is not scalable; a slower system that handles 100x the load by adding 100x the machines is.

Start with the "why": load grows, and there are only two responses. Vertical scaling buys a bigger machine — simple, but bounded by the largest machine that exists and usually a single point of failure. Horizontal scaling adds more machines — effectively unbounded, but only works if the work can be split, which is the hard part.

Two laws govern how well splitting works. Amdahl's law says the speedup from parallelism is capped by the fraction of work that is inherently serial. The Universal Scalability Law adds a second, nastier term: coordination between nodes (locks, consensus, cross-shard queries) makes throughput decrease past some node count. This is why "just add servers" eventually stops helping and starts hurting.

Throughput vs. nodes

 ideal (linear)      .-'
                 .-'
              .-'          Amdahl (serial fraction caps it)
           .-'        ______________________
        .-'      ____/
     .-'    ____/                 USL (coordination cost bends it down)
  .-'  ____/        ................
  |___/........''''                 ''''....
  +----------------------------------------------> nodes
                                          knee

def amdahl_speedup(serial_fraction, n):
    """Max speedup with n workers when `serial_fraction` cannot be parallelized."""
    return 1 / (serial_fraction + (1 - serial_fraction) / n)

for n in (1, 10, 100, 1000):
    print(n, round(amdahl_speedup(0.05, n), 1))
# 1 1.0 | 10 6.9 | 100 16.8 | 1000 19.6
# 5% serial work caps you at 20x no matter how many machines you buy.
# Design implication: hunt down the serial part (a global lock, a single
# sequence generator, one shared table) before buying more capacity.

Aspect	Vertical scaling	Horizontal scaling
Ceiling	Largest available machine	Practically unbounded
Complexity	Low (no code changes)	High (sharding, coordination)
Failure impact	Single point of failure	Degrades gracefully
Cost curve	Superlinear at the high end	Roughly linear
Downtime to scale	Usually requires a restart	Add nodes live
Best for	Databases early on, simplicity	Stateless services, large scale

Stateless components scale horizontally almost for free; state is what makes scaling hard, which is why so much of this curriculum is about data.

Key Takeaways

• Scalability is the ability to absorb load by adding resources at proportional cost — not raw speed.
• Vertical scaling is simple but capped; horizontal scaling is unbounded but demands splittable work.
• Amdahl's law caps speedup by the serial fraction; the USL shows coordination can make more nodes worse.
• Stateless services scale trivially; state is the real scaling problem.

🧪 Practice

1. A service spends 10% of each request in a globally locked section. What is the maximum speedup from parallelism? Compute it with the code above.
2. Name three things in a typical web application that are inherently serial and block horizontal scaling.
3. Interview: Your API scales linearly to 20 servers, then throughput plateaus and p99 latency rises. What do you investigate first? (Hint: what do all 20 servers share?)

Availability

Availability is the fraction of time a system is able to serve requests successfully. It is stated as a percentage — the famous "nines" — but the percentage is meaningless until you say of what, measured how, and over what window. "99.9% availability" of a health-check endpoint is a very different promise from 99.9% of successful checkout requests measured at the client.

The formula is straightforward:

Availability = uptime / (uptime + downtime)
             = MTBF / (MTBF + MTTR)

  MTBF = mean time between failures   (make failures rarer)
  MTTR = mean time to recovery        (make recovery faster)

MTTR is usually the cheaper lever. Halving recovery time improves availability exactly as much as doubling time between failures, and fast rollback, good runbooks, and automated failover are far more achievable than never failing.

Availability	Downtime/year	Downtime/month	Downtime/week	Typical for
99%	3.65 days	7.2 hours	1.68 hours	Internal tools
99.9%	8.77 hours	43.8 minutes	10.1 minutes	Standard SaaS
99.95%	4.38 hours	21.9 minutes	5.04 minutes	Business-critical SaaS
99.99%	52.6 minutes	4.38 minutes	1.01 minutes	Payment, infrastructure
99.999%	5.26 minutes	26.3 seconds	6.05 seconds	Telco core, very rare

Availability composes, and the arithmetic is unforgiving. Dependencies in series (every request needs all of them) multiply; redundant components in parallel multiply their failure probabilities, which is why redundancy is the primary tool for high availability.

from functools import reduce

def in_series(*avails):
    """All must be up: availabilities multiply -> the total is always lower."""
    return reduce(lambda a, b: a * b, avails)

def in_parallel(*avails):
    """Any one suffices: failure probabilities multiply -> total is higher."""
    return 1 - reduce(lambda a, b: a * b, [1 - x for x in avails])

# A request touching LB -> API -> DB -> cache, each 99.9%
print(round(in_series(0.999, 0.999, 0.999, 0.999) * 100, 3))   # 99.601
# Four "three nines" dependencies give you well under three nines overall.

# Three independent API replicas, each 99%
print(round(in_parallel(0.99, 0.99, 0.99) * 100, 4))           # 99.9999
# Redundancy is enormously effective — IF failures are truly independent.

That last caveat is the whole game: shared power, a shared config push, a shared dependency, or correlated overload breaks independence, and the parallel formula silently overstates your real availability.

Key Takeaways

• Availability = MTBF / (MTBF + MTTR); reducing MTTR is usually cheaper than increasing MTBF.
• Always state what is measured, from where, and over what window.
• Serial dependencies multiply availabilities down; parallel redundancy multiplies failure probabilities down.
• Redundancy only helps to the extent failures are independent.

🧪 Practice

1. A request path has five services at 99.95% each. What is the end-to-end availability, and how many minutes per month is that?
2. Your SLO is 99.9% monthly. You have used 30 minutes of downtime this month. How much error budget remains?
3. Interview: How would you raise a service from 99.9% to 99.99% without rewriting it? (Hint: look at the MTTR side of the formula and at what is currently in series.)

Reliability

Availability asks "was it up?"; reliability asks "did it do the right thing?". A system can be perfectly available and deeply unreliable — returning stale balances, dropping one order in a thousand, or double-charging on retry. Users forgive a brief outage far more readily than a wrong number.

Formally, reliability is the probability that a system performs its intended function correctly for a given period under stated conditions. It covers correctness of results, absence of data loss or corruption, and correct behaviour under retries and partial failure.

The distinction matters because the two are improved by different means. Availability is improved with redundancy and fast failover. Reliability is improved with idempotency, checksums, transactions, validation, replication of data, backups, and testing under fault injection.

                 Available          Not available
              +------------------+------------------+
   Reliable   | Ideal            | Down but honest: |
              |                  | returns 503, no  |
              |                  | corrupted state  |
              +------------------+------------------+
   Unreliable | The dangerous    | Down and losing  |
              | quadrant: up and | data             |
              | quietly wrong    |                  |
              +------------------+------------------+

 The top-right is recoverable. The bottom-left destroys trust silently.

# Unreliable: a retry after a timeout can charge the customer twice.
def transfer(account, amount, gateway):
    return gateway.charge(account, amount)     # no dedup, no record

# Reliable: idempotency key + durable record makes retries safe.
def transfer_safe(account, amount, key, db, gateway):
    existing = db.get_transfer(key)            # has this key been seen?
    if existing:
        return existing.result                 # replay the recorded outcome
    db.insert_transfer(key, account, amount, status="PENDING")  # durable intent
    try:
        result = gateway.charge(account, amount, idempotency_key=key)
        db.update_transfer(key, status="DONE", result=result)
        return result
    except TimeoutError:
        # Unknown outcome: leave PENDING for the reconciler to settle rather
        # than guessing. Never silently retry a non-idempotent side effect.
        raise

Metric	Meaning	Improves by
MTBF	Mean time between failures	Better testing, redundancy, limits
MTTF	Mean time to failure (non-repairable)	Better components
MTTR	Mean time to repair/recover	Automation, runbooks, rollback
Error rate	Fraction of requests wrong or failed	Validation, idempotency, retries with backoff

Key Takeaways

• Availability is "up"; reliability is "correct". They are improved by different mechanisms.
• Silent wrongness is worse than visible downtime — fail loudly.
• Idempotency, durable intent, and reconciliation are the core reliability patterns for side effects.
• Track MTBF and MTTR explicitly; they are the levers behind both attributes.

🧪 Practice

1. Give an example of a system that is highly available but unreliable, and one that is reliable but poorly available.
2. Make this operation idempotent: "append a comment to a post" called over a flaky network with client retries.
3. Interview: A payment API times out after the charge succeeded downstream. What must the design guarantee, and how? (Hint: the client cannot distinguish "not done" from "done but the response was lost".)

Maintainability

Most of the money spent on a system is spent after it launches. Maintainability is the quality attribute that governs that cost: how easily people can operate, understand, and change the system over its lifetime. It is the attribute most often sacrificed under deadline pressure and the one whose absence is most expensive later.

It splits into three concerns:

• Operability — can the people running it keep it healthy? (Good telemetry, predictable behaviour, easy rollback, documented runbooks.)
• Simplicity — can a new engineer form a correct mental model quickly? (Managed complexity, clear abstractions, few surprises.)
• Evolvability — can it be changed safely? (Tests, loose coupling, stable contracts, small deployable units.)

The useful mental model is accidental versus essential complexity. Essential complexity comes from the problem (payments really do have chargebacks). Accidental complexity comes from your solution (three config systems, two RPC frameworks, a bespoke retry layer). Maintainability work is mostly the removal of accidental complexity.

Maintainability shows up in delivery metrics (DORA)

  Deployment frequency   how often you ship          -> higher is better
  Lead time for change   commit to production        -> lower is better
  Change failure rate    deploys causing incidents   -> lower is better
  Time to restore        incident to recovery        -> lower is better

  These are outcome measures. If they are bad, the cause is usually
  architectural, not individual.

# Low maintainability: policy, transport, and business rules entangled.
def process_order(order):
    if order["type"] == "digital":
        requests.post("https://fulfil.internal/d", json=order, timeout=3)
    elif order["type"] == "physical":
        requests.post("https://fulfil.internal/p", json=order, timeout=3)
    # Adding a third type edits this function; testing it needs the network.

# Higher maintainability: one seam for policy, one for transport.
class OrderProcessor:
    def __init__(self, handlers):        # injected -> testable without network
        self._handlers = handlers        # {"digital": fn, "physical": fn}

    def process(self, order):
        handler = self._handlers.get(order["type"])
        if handler is None:
            raise UnsupportedOrderType(order["type"])   # explicit, not silent
        return handler(order)
# A new order type is now a registration, not a modification.

Key Takeaways

• Maintainability = operability + simplicity + evolvability, and it dominates lifetime cost.
• Distinguish essential complexity (inherent) from accidental complexity (self-inflicted, removable).
• The DORA metrics make maintainability measurable rather than aesthetic.
• Seams — interfaces, injection points, stable contracts — are what make change cheap.

🧪 Practice

1. Identify one piece of accidental complexity in a codebase you know and describe what removing it would cost and save.
2. Write a runbook entry for "the queue consumer is lagging": symptoms, checks, actions, escalation.
3. Interview: How would you measure whether an architecture change improved maintainability? (Hint: pick metrics that a stakeholder outside the team would also accept.)

Latency and Throughput

Latency is how long one operation takes. Throughput is how many operations complete per unit of time. They are related but independent: a system can have low latency and low throughput (a fast single-threaded service), or high latency and high throughput (a batch pipeline moving terabytes with a one-hour delay).

The analogy is a highway. Latency is how long your car takes to drive from A to B. Throughput is how many cars per hour the road carries. Adding lanes raises throughput without making any single trip faster; raising the speed limit cuts latency. And past a certain traffic density, adding cars makes everyone slower — the queueing effect that makes latency explode near saturation.

Little's Law ties the two together and is the single most useful formula in capacity work:

  L = λ × W

  L = average number of requests in the system (concurrency)
  λ = arrival rate (throughput, requests/second)
  W = average time in the system (latency, seconds)

  Example: 2,000 RPS with 50 ms average latency
           L = 2000 × 0.05 = 100 concurrent requests in flight
  -> you need at least ~100 threads/connections, plus headroom.

Never describe latency with an average. Averages hide the tail, and the tail is what users feel — especially because a single page view often fans out to many backend calls, so the slowest of them sets the perceived latency.

def percentile(samples, p):
    s = sorted(samples)
    idx = min(int(len(s) * p / 100), len(s) - 1)   # nearest-rank, clamped
    return s[idx]

latencies = [10] * 990 + [2000] * 10               # 99% fast, 1% terrible
print(sum(latencies) / len(latencies))             # 29.9  <- mean looks fine
print(percentile(latencies, 50))                   # 10
print(percentile(latencies, 99))                   # 2000  <- the truth

# Tail amplification: if one page issues 20 parallel backend calls, the chance
# that at least one hits the p99 path is 1 - 0.99**20 = 18%.
print(round((1 - 0.99 ** 20) * 100, 1))            # 18.2

Term	Definition	Typical target
p50 (median)	Half of requests are faster	Feels like "normal" speed
p95	1 in 20 requests is slower	Common SLO anchor
p99	1 in 100 requests is slower	Where tail problems surface
p99.9	1 in 1,000 requests is slower	Matters at high fan-out
Throughput	Completed operations per second	Sized from peak traffic

Key Takeaways

• Latency is per-operation time; throughput is operations per unit time.
• Little's Law (L = λ × W) converts between concurrency, rate, and latency.
• Report percentiles, never averages; the tail is the user experience.
• Fan-out amplifies tails: many parallel calls make p99 the common case.

🧪 Practice

1. A service handles 500 RPS with 200 ms average latency. How many requests are in flight on average? How many worker threads would you provision?
2. Given the latency list [5, 7, 9, 11, 900], compute mean, p50, and p99, then explain which one you would put in an SLO and why.
3. Interview: Your p50 is 20 ms and your p99 is 3 s. Name four plausible causes. (Hint: think about garbage collection, cold caches, lock contention, and one unlucky shard.)

Consistency

Consistency answers a deceptively simple question: after a write, what do subsequent reads see? On a single machine the answer is "the write". The moment data is replicated across machines, replicas diverge for some window, and you have to decide what to promise.

Why not always promise the strongest thing? Because strong consistency requires coordination, coordination requires round trips, and round trips cost latency and availability — a replica that cannot reach its peers must either block or serve possibly-stale data. That is the essence of the CAP and PACELC results covered in Chapter 8; here we only need the vocabulary.

Write W (value = 5) to the primary at t=0. Replica lag = 100 ms.

 t=0     t=50ms      t=100ms     t=150ms
  |         |            |           |
  W --> primary: 5      ...          ...
        replica: 4  ->  replica: 5   replica: 5
            ^
            |
    A read routed here at t=50ms returns the OLD value 4.

 Strong consistency:   that read is not allowed to return 4
                       (route to primary, or block until replicated)
 Eventual consistency: returning 4 is allowed; convergence is guaranteed
                       only "eventually"

Model	Guarantee	Cost
Strong (linearizable)	Every read sees the latest committed write	Highest latency, lowest availability under partition
Sequential	All nodes see operations in the same order	Cheaper than linearizable
Causal	Causally related operations are seen in order	Good middle ground
Read-your-writes	A client always sees its own writes	Session-scoped, cheap
Monotonic reads	A client never sees time go backwards	Session-scoped, cheap
Eventual	Replicas converge if writes stop	Lowest latency, highest availability

# A very common, very cheap compromise: read-your-writes via read routing.
def get_profile(user_id, session, primary, replica):
    # If this session wrote recently, its own read must not go to a lagging
    # replica; everyone else can enjoy the cheap replica read.
    if session.wrote_within_ms(user_id, 500):
        return primary.read(user_id)
    return replica.read(user_id)

The practical skill is choosing per use case, not per system: a bank balance needs strong consistency, a "likes" counter is fine eventually consistent, and a user's own profile edit usually needs only read-your-writes.

Key Takeaways

• Consistency defines what a read may return after a write, once data is replicated.
• Stronger consistency costs latency and availability; there is no free version.
• Session guarantees (read-your-writes, monotonic reads) solve most perceived staleness cheaply.
• Choose a consistency level per operation, not once for the whole system.

🧪 Practice

1. Classify each as needing strong or eventual consistency: account balance, view count, seat reservation, username availability check, follower count.
2. Describe a user-visible bug caused by a lack of read-your-writes.
3. Interview: Your feed shows a comment the user just posted as missing on refresh. Diagnose and fix without making all reads strongly consistent. (Hint: the guarantee needed is scoped to one session.)

Durability

Durability is the promise that once the system has acknowledged a write, that data will not be lost — not to a process crash, not to a disk failure, not to a data centre fire. It is distinct from availability (the data may be temporarily unreachable and still perfectly durable) and from consistency (a stale read is not a lost write).

The mechanics come down to two questions: how many independent copies exist, and when do you acknowledge the write relative to those copies being made? Acknowledging before the data is safely persisted trades durability for latency, and it is a trade some systems make deliberately.

Write acknowledgement points (fastest to safest)

 1. In memory on one node        -> lost on process crash
 2. OS page cache on one node    -> lost on machine power loss
 3. fsync'd to disk on one node  -> lost on disk/node loss
 4. fsync'd + replicated to 1 peer in the same AZ -> lost on AZ failure
 5. Replicated across 3 AZs      -> survives a data centre failure
 6. Replicated across regions    -> survives a regional disaster

 Each step down adds latency and cost. Pick the level per data class:
 payment records at 5-6, analytics events at 2-3.

Two acronyms make the target concrete:

• RPO (Recovery Point Objective) — how much data you can afford to lose, measured in time. RPO = 5 minutes means backups/replication must be at most five minutes behind.
• RTO (Recovery Time Objective) — how long recovery may take.

# Durability is a write-path configuration decision, not a vague hope.
# Quorum settings on a replicated store with replication factor N = 3:
N, W, R = 3, 2, 2
print("strongly consistent reads:", W + R > N)   # True: 2+2 > 3
# W=2 -> the write is acknowledged only after 2 replicas have it.
#        Losing one node cannot lose an acknowledged write.
# W=1 -> lower latency, but an acknowledged write can vanish with one node.

# Estimating annual data-loss risk with independent failures (illustrative):
p_node_loss_per_year = 0.02                        # 2% chance per node
print(round(p_node_loss_per_year ** 3 * 100, 6))   # 0.0008% for all three
# The number is only as good as the independence assumption: same rack, same
# power, same bad deploy, and the copies fail together.

Replication is not backup. Replication faithfully copies your mistakes — a DELETE without a WHERE clause propagates to every replica in milliseconds. Point-in-time backups, immutability, and delayed replicas protect against logical corruption; replicas protect against hardware loss.

Key Takeaways

• Durability means an acknowledged write survives failures; it is set by copy count and acknowledgement point.
• RPO (data loss tolerance) and RTO (recovery time) turn durability into testable numbers.
• Quorum settings (W + R > N) tie durability to consistency and latency.
• Replication is not backup — it replicates logical corruption too.

🧪 Practice

1. For each data class, propose an RPO and RTO: payment ledger, user avatars, clickstream analytics, session tokens.
2. With N = 5 replicas, list the (W, R) pairs that give strongly consistent reads, and rank them by write latency.
3. Interview: A team says "we have three replicas, so we don't need backups". What do you say? (Hint: think about what a replica does with a bad migration script.)

Cost Efficiency

Cost is a first-class quality attribute, not an afterthought. Every design choice in this book — an extra replica, a cache tier, a second region, stronger consistency — has a price, and a design that meets every technical target while costing more than the product earns is a failed design.

The key move is to stop thinking in monthly totals and start thinking in unit economics: cost per request, per active user, per gigabyte stored, per tenant. Unit costs are what tell you whether growth improves or destroys your margins, and they make architectural options directly comparable.

# Unit economics for a hypothetical API tier
monthly_requests   = 500_000_000
compute_cost       = 4_200      # autoscaled application servers
database_cost      = 3_100      # primary + 2 replicas + storage
cache_cost         = 700
egress_cost        = 2_400      # data transfer out is often the surprise
observability_cost = 900        # logs and metrics scale with traffic too

total = compute_cost + database_cost + cache_cost + egress_cost + observability_cost
cost_per_million = total / (monthly_requests / 1_000_000)
print(total, round(cost_per_million, 2))   # 11300 22.6  -> $22.60 per 1M requests

# Now the design question becomes concrete: a cache tier that costs $700/month
# and removes 60% of database reads is worth it only if it defers more than
# $700/month of database scaling — and you can compute that.

Cost driver	Typical surprise	Common lever
Compute	Idle over-provisioned capacity	Autoscaling, right-sizing, spot
Storage	Old data never deleted	Tiering to cold storage, retention
Network egress	Cross-AZ and internet transfer dominates the bill	CDN, compression, AZ-local routing
Managed services	Per-request pricing at scale	Batch, cache, or self-host at scale
Observability	Log volume grows with traffic and with debugging habits	Sampling, cardinality limits
Engineering time	The largest and least tracked cost of all	Simplicity, managed services early

Note the last row. Engineering salaries usually dwarf infrastructure bills for small teams, which is why "self-host it to save $400/month" is often a bad trade, and why simplicity is a cost optimization.

Cost also trades directly against the other attributes: going from 99.9% to 99.99% availability typically means multi-AZ or multi-region redundancy and can double infrastructure cost for 47 fewer minutes of downtime per year. Whether that is worth it is a business question with a numeric answer.

Key Takeaways

• Measure unit costs (per request, per user, per GB), not just monthly totals.
• Network egress, storage growth, and observability are the usual hidden drivers.
• Engineering time is a real cost — simplicity is a cost optimization.
• Every extra nine of availability has a price; state it and let the business decide.

🧪 Practice

1. Compute the cost per 1,000 requests for a service costing $8,000/month at 200M requests/month. What if traffic halves but the fleet does not shrink?
2. List three architectural changes that reduce egress cost, and the trade-off each brings.
3. Interview: The CFO asks why the bill doubled after adding a second region. How do you justify it? (Hint: convert availability into downtime minutes, then into revenue or contractual penalties.)

<a id="13-requirement-gathering-and-estimation"></a>

1.3 Requirement Gathering and Estimation

Every real design starts from an underspecified prompt. This subchapter is the bridge from "design Instagram" to a set of numbers — QPS, bytes, machines — that make architectural choices decidable rather than opinionated.

Clarifying Ambiguous Requirements

"Design a chat app" is not a specification; it is a topic. The gap between the prompt and a designable problem is filled by questions, and asking them well is a skill that is graded explicitly in interviews and rewarded implicitly at work. Designing before clarifying is how teams build the wrong system efficiently.

The trap to avoid is a scattershot list of trivia ("does it support dark mode?"). Good clarifying questions are the ones whose answers change the architecture. Ask in a funnel: scope first, then scale, then quality, then constraints.

THE CLARIFICATION FUNNEL

 1. Scope     Who are the users? What are the top 3 things they do?
              What is explicitly out of scope for v1?
 2. Scale     How many users (total / daily active)? How much data?
              Read-heavy or write-heavy? What is the peak-to-average ratio?
 3. Quality   Latency target? Availability target? Consistency needs?
              Is stale data acceptable, and for how long?
 4. Context   Greenfield or integrating with existing systems?
              Team size, deadline, budget, regulatory constraints?
              Global or single-region? Mobile clients on poor networks?

A test for whether a question is worth asking: can you name two different architectures that the two possible answers would lead to? If not, assume a reasonable default, state the assumption, and move on.

Question	Why it changes the design
"Read-heavy or write-heavy?"	Decides caching, replication, and index strategy
"Must messages be ordered per conversation?"	Decides partitioning key and sequencing mechanism
"Is 30-second staleness acceptable?"	Decides strong vs. eventual consistency and cost
"Global users or one region?"	Decides replication topology and latency budget
"Do we need message search?"	Adds or removes an entire indexing subsystem
"Does it support dark mode?"	Changes nothing — do not ask this

# A compact intake structure worth reusing at the start of any design.
REQUIREMENTS = {
    "scope":   {"in": ["1:1 chat", "group chat <=100", "online presence"],
                "out": ["voice/video", "file sharing", "message search"]},
    "scale":   {"dau": 50_000_000, "msgs_per_user_per_day": 40,
                "peak_factor": 3, "read_write_ratio": 5},
    "quality": {"p99_send_ms": 300, "availability": 0.9995,
                "ordering": "per-conversation", "staleness_ok_s": 0},
    "context": {"regions": ["us", "eu"], "mobile_first": True,
                "team": 6, "deadline_months": 4},
}
# Everything downstream — QPS, storage, topology — derives from this dict.

Key Takeaways

• Clarify before designing; the prompt is never the specification.
• Ask questions in a funnel: scope, then scale, then quality, then context.
• A question is worth asking only if different answers lead to different architectures.
• When a question is not worth the time, state a default assumption aloud and proceed.

🧪 Practice

1. Write eight clarifying questions for "design a file storage service like Dropbox", grouped by the four funnel stages.
2. For three of them, describe the two different architectures the two possible answers would produce.
3. Interview: The interviewer answers every clarifying question with "you decide". What do you do? (Hint: they are testing whether you can pick defensible defaults and say them out loud.)

Defining System Scope

Scope is the boundary around what you are designing. Drawing it explicitly does two things: it prevents the design from sprawling into every adjacent problem, and it makes the interfaces to the outside world visible — which is where most integration risk lives.

The classic tool is a context diagram: your system as one box in the middle, every external actor and system around it, and a labelled arrow for each interaction. If you cannot draw it, you do not yet know what you are building.

CONTEXT DIAGRAM — "Ride matching service" (the box we are designing)

     [Rider app]                                  [Driver app]
          |  request ride, poll status                 |  location updates,
          v                                            v  accept/decline
   +--------------------------------------------------------------+
   |                    RIDE MATCHING SERVICE                     |
   |   (in scope: request intake, driver index, matching, state)  |
   +--------------------------------------------------------------+
      |                |                  |                 |
      v                v                  v                 v
 [Payments svc]   [Maps/ETA API]    [Notification svc]  [Analytics sink]
  (external)       (3rd party)        (internal)         (internal)

 Out of scope: pricing model, driver onboarding, fraud detection, payouts.

An explicit in/out table is worth more than paragraphs of prose:

In scope	Out of scope (v1)
Accepting and storing ride requests	Surge pricing computation
Maintaining an index of nearby drivers	Driver background checks and onboarding
Matching a request to exactly one driver	Payment capture and driver payouts
Publishing trip state changes	Fraud detection
Exposing trip status to clients	Long-term analytics and reporting

Two habits keep scope honest. First, write the out of scope column first — it is easier and it surfaces disagreement immediately. Second, for every external box, name the contract: protocol, expected latency, failure behaviour, and what your system does when that dependency is down. Half of all incident postmortems trace to an unexamined arrow crossing the system boundary.

Key Takeaways

• Scope defines what you build and, just as importantly, what you do not.
• A context diagram makes every external dependency and its contract visible.
• Write the out-of-scope list explicitly — it prevents sprawl and surfaces disagreement early.
• For each boundary-crossing arrow, define the failure behaviour, not just the happy path.

🧪 Practice

1. Draw a context diagram for an e-commerce checkout service, including at least four external systems.
2. For each external system, write one sentence describing what your service does when it is unavailable.
3. Interview: Mid-design, the interviewer adds "also support group ordering". How do you respond without discarding your design? (Hint: decide whether it is a new component, a change to an existing one, or a v2 item — and say which.)

Back-of-the-Envelope Calculations

Back-of-the-envelope (BOTE) estimation is the practice of getting to an answer that is right within an order of magnitude, in under a minute, using round numbers. Its purpose is not precision — it is decision-making. Whether a data set is 50 GB or 50 TB determines whether it fits in memory on one machine or needs a sharded cluster, and you can tell the difference without a spreadsheet.

The technique rests on three habits: round aggressively to one significant figure, keep units explicit at every step, and memorize a handful of anchor numbers so you never have to look them up mid-conversation.

ANCHORS WORTH MEMORIZING

 Time                                Powers of two
  1 day     = 86,400 s ~ 10^5 s       2^10 ~ 1 thousand   (KB)
  1 month   ~ 2.6M s   ~ 2.5 x 10^6   2^20 ~ 1 million    (MB)
  1 year    ~ 31.5M s  ~ 3 x 10^7     2^30 ~ 1 billion    (GB)
                                      2^40 ~ 1 trillion   (TB)
 Sizes (typical)
  UUID 16 B | timestamp 8 B | short text field ~50 B
  Tweet-sized record ~ 300 B | thumbnail ~ 20 KB | photo ~ 2 MB
  1 minute of 1080p video ~ 50 MB

Operation	Order of magnitude	Notes
L1 cache reference	~1 ns	Baseline
Main memory reference	~100 ns	100x slower than L1
Read 1 MB sequentially from memory	~3 us
SSD random read	~15-100 us	NVMe at the fast end
Round trip inside one data centre	~0.5 ms	Same-region service call
Read 1 MB sequentially from SSD	~0.5 ms
HDD seek	~10 ms	Why random HDD I/O is fatal
Round trip across a continent	~50-80 ms	Physics, not software
Round trip Europe to Australia	~150-300 ms	Physics, not software

The most valuable lesson in that table is the gap: memory is roughly 100,000 times faster than a cross-continent round trip. Almost every performance design decision is about moving work up that table.

# BOTE worked example: "will this dataset fit in RAM?"
users              = 200_000_000
bytes_per_user     = 500                    # rounded generously
raw = users * bytes_per_user                # 100_000_000_000 bytes
print(raw / 1e9, "GB")                      # 100.0 GB

# Overhead is never zero: indexes, pointers, fragmentation. Assume 2x.
print(raw * 2 / 1e9, "GB")                  # 200.0 GB
# Verdict: too big for a comfortable single cache node (typical 64-256 GB),
# fine for a small sharded cluster. That is the decision the estimate exists
# to make — no further precision is useful.

Sanity-check every result against something you know. If a calculation says a photo service needs 4 PB per day, ask whether that is plausible for the user count you assumed; an order-of-magnitude slip in an intermediate step is the usual culprit.

Key Takeaways

• BOTE aims for the right order of magnitude, fast — precision is not the goal.
• Round to one significant figure, carry units explicitly, and use anchors (10^5 s/day, 2^30 ~ 1 billion).
• Memory-to-network latency spans about five orders of magnitude; most optimization is moving work up that ladder.
• Always sanity-check the final number against something you already know.

🧪 Practice

1. Estimate the storage needed for one year of chat messages: 50M DAU, 40 messages/day, ~200 bytes each, replication factor 3.
2. Estimate how long it takes to transfer 10 TB over a 1 Gbps link, then over a 10 Gbps link. Compare with shipping physical disks.
3. Interview: Estimate the number of servers needed to serve 1M concurrent WebSocket connections. (Hint: start from connections per server and memory per connection, and remember file descriptor limits.)

Traffic and QPS Estimation

QPS (queries per second) is the number that sizes almost everything downstream: server count, connection pools, cache capacity, database throughput. Getting it approximately right early prevents both under-provisioning (an outage) and over-provisioning (a wasted budget).

The chain is always the same: from users, to actions per user per day, to requests per day, to average QPS, to peak QPS. The last step matters most — systems must be sized for the peak, not the average, and real traffic is spiky: daily cycles, lunchtime spikes, marketing pushes, and time zones stack up. A peak-to-average factor of 2x to 5x is typical; flash-sale or live-event systems can see 20x or more.

THE ESTIMATION CHAIN

  DAU  x  actions/user/day  =  actions/day
  actions/day / 86,400      =  average QPS
  average QPS x peak factor =  peak QPS        <- size for this
  peak QPS split by read/write ratio -> read QPS and write QPS

DAU              = 100_000_000
actions_per_user = 10            # posts, likes, reads: everything hitting the API
peak_factor      = 3             # measured if possible; assumed otherwise
read_write_ratio = 100           # 100 reads per write

requests_per_day = DAU * actions_per_user            # 1_000_000_000
avg_qps          = requests_per_day / 86_400         # ~11,574
peak_qps         = avg_qps * peak_factor             # ~34,722

write_qps = peak_qps / (read_write_ratio + 1)        # ~344
read_qps  = peak_qps - write_qps                     # ~34,378

for name, value in [("avg", avg_qps), ("peak", peak_qps),
                    ("peak read", read_qps), ("peak write", write_qps)]:
    print(f"{name:11} {value:,.0f} QPS")
# avg          11,574 QPS
# peak         34,722 QPS
# peak read    34,378 QPS
# peak write      344 QPS

Read those results the way a designer would. ~344 writes/second is comfortable for a single well-tuned relational primary. ~34,000 reads/second is not — that number is what justifies a cache tier and read replicas. The estimate did not just produce a figure; it produced an architecture.

Trap	Correction
Sizing for average QPS	Size for peak; averages hide the outage
Ignoring fan-out	One user action may cause 10 internal service calls
Assuming a uniform day	Model time zones and daily peaks explicitly
Forgetting background load	Cron jobs, re-indexing, and backfills share the fleet
Counting only human traffic	Bots, scrapers, retries, and health checks add up

Fan-out deserves special attention: if one page view triggers 12 internal calls, your internal QPS is 12x the external QPS, and internal services must be sized on the larger number.

Key Takeaways

• Derive QPS as DAU x actions/day / 86,400, then multiply by a peak factor.
• Always size for peak QPS, and split it by the read/write ratio.
• Internal QPS is external QPS times fan-out — size internal services on it.
• The QPS split is what justifies (or rules out) caches, replicas, and sharding.

🧪 Practice

1. Compute average and peak QPS for 5M DAU, 20 actions/day, peak factor 4.
2. Recompute assuming each user action fans out to 8 internal service calls, and state which component you would worry about first.
3. Interview: Your service sees 2x traffic every day at 20:00 local time across three continents. How do you provision? (Hint: consider whether the peaks overlap, and what autoscaling can and cannot react to in time.)

Storage and Bandwidth Estimation

Storage estimation answers "how much data will exist, and for how long?". Bandwidth estimation answers "how many bytes per second cross which boundary?". Both matter because they set costs and because they frequently break the design before CPU ever does — network egress is often the largest line on a cloud bill, and unbounded storage growth is the most common source of nasty surprises in year two.

The method mirrors QPS estimation: bytes per record, times records per day, times retention, times replication.

STORAGE CHAIN                           BANDWIDTH CHAIN

 bytes/record                            requests/second (peak)
   x records/day                           x bytes/response
   = bytes/day                             = bytes/second
   x retention (days)                      x 8
   = logical bytes                         = bits/second  -> compare to NIC/link
   x replication factor
   x (1 + index/overhead)
   = provisioned bytes

# Storage: one year of a Twitter-like write load
writes_per_day   = 1_000_000_000        # 1B new records/day
bytes_per_record = 300                  # text + ids + timestamps, rounded up
replication      = 3
overhead         = 1.3                  # indexes, metadata, fragmentation

bytes_per_day = writes_per_day * bytes_per_record             # 3.0e11
per_year_raw  = bytes_per_day * 365                           # 1.095e14
provisioned   = per_year_raw * replication * overhead

print(round(bytes_per_day / 1e12, 2), "TB/day")               # 0.3 TB/day
print(round(per_year_raw / 1e12), "TB/year logical")          # 110 TB/year
print(round(provisioned / 1e12), "TB/year provisioned")       # 427 TB/year

# Bandwidth: serving the read path
peak_read_qps   = 34_000
bytes_per_resp  = 2_000                 # 2 KB JSON response
egress_bytes_s  = peak_read_qps * bytes_per_resp               # 68 MB/s
print(round(egress_bytes_s / 1e6, 1), "MB/s",
      round(egress_bytes_s * 8 / 1e9, 2), "Gbps")             # 68.0 MB/s 0.54 Gbps

Half a gigabit per second of egress is a real cost line and a strong argument for a CDN and for compression on large responses. And 427 TB/year of provisioned storage says plainly that a retention policy or a tiering strategy is not optional.

Content type	Rough size	Design implication
Text record/row	100 B - 1 KB	Cheap; billions fit in a sharded database
Thumbnail	~20 KB	Cache aggressively at the edge
Photo (compressed)	1 - 5 MB	Object storage + CDN, never in the database
1 min 1080p video	~50 MB	Object storage, chunked, adaptive bitrate
Log line	200 B - 2 KB	Volume explodes with traffic; sample it

Two rules of thumb worth internalizing: store large binary objects in object storage and keep only a pointer in your database; and set a retention policy for every data class on day one, because deleting data later is politically and technically harder than never keeping it.

Key Takeaways

• Storage = bytes/record x records/day x retention x replication x overhead.
• Bandwidth = peak QPS x bytes/response; convert to bits when comparing with link capacity.
• Egress and unbounded growth are the two costs teams most often miss.
• Blobs go to object storage behind a CDN; databases hold pointers.

🧪 Practice

1. Estimate one year of provisioned storage for a photo service: 10M photos/day, 2 MB each plus three thumbnails, replication factor 3.
2. Estimate peak egress bandwidth for a video service streaming to 100,000 concurrent viewers at 5 Mbps, and say what that implies about a CDN.
3. Interview: Your database grows 1 TB per week and queries are slowing down. What are your options? (Hint: some options reduce data, some redistribute it, some change how it is accessed — name at least one of each.)

Capacity Planning Basics

Capacity planning converts estimates into a concrete fleet: how many servers, threads, connections, and shards, with enough headroom to survive both a spike and a failure. Under-provision and you take an outage; over-provision and you burn money — and the target is not "enough for peak" but "enough for peak, minus a failed component, plus room to react".

Three ideas carry most of the weight.

Utilization targets. Never plan to run at 100%. Queueing theory says latency rises roughly as 1 / (1 - utilization), so a resource at 90% is not "a bit slower than 80%" — it is dramatically worse. Target 50-70% at peak for latency-sensitive services.

Latency multiplier vs. utilization (M/M/1 intuition)

 utilization  0.5    0.7    0.8    0.9    0.95   0.99
 multiplier   2x     3.3x   5x     10x    20x    100x

  |                                                  *
  |                                              *
  |                                        *
  |                            *
  |            *      *
  +---------------------------------------------------> utilization
   The knee is real. Capacity plans that ignore it produce latency incidents,
   not cost savings.

N+1 and N+2 redundancy. Provision enough capacity that losing one instance (N+1) or one availability zone (N+2, or 3 AZs at 2/3 capacity each) does not push the survivors past their utilization target.

Little's Law for concurrency. Thread pools and connection pools are sized from L = λ × W, not guessed.

peak_qps            = 34_000
rps_per_server      = 1_000       # measured under load test, not assumed
target_utilization  = 0.6         # keep off the latency knee
azs                 = 3           # must survive losing one AZ

usable_per_server = rps_per_server * target_utilization        # 600
servers_needed    = peak_qps / usable_per_server               # ~56.7 -> 57

# Survive one AZ loss: the remaining (azs - 1) must carry the full peak.
servers_total = -(-servers_needed // (azs - 1)) * azs          # ceil then x AZs
print(int(servers_needed), int(servers_total))                 # 56 87

# Concurrency per server via Little's Law
avg_latency_s = 0.050
in_flight     = (peak_qps / servers_needed) * avg_latency_s
print(round(in_flight, 1), "concurrent requests per server")   # 30.0
# -> thread/connection pool of ~30, plus headroom; a pool of 8 would queue,
#    and a pool of 500 would thrash memory and context switches.

Input to a capacity plan	Where it comes from
Peak QPS	Traffic estimation plus measured history
Per-server capacity	Load testing a single instance to its knee
Utilization target	Latency SLO and the queueing knee
Redundancy factor	Availability target and failure domains
Growth rate	Product forecast; plan 6-12 months ahead
Scaling lead time	How fast you can actually add capacity

That last row is the one people forget. Autoscaling that takes four minutes to boot an instance cannot absorb a spike that arrives in thirty seconds — so either pre-warm capacity ahead of known events, or shed load gracefully with queues and rate limits (Chapter 6).

Key Takeaways

• Size for peak, at a utilization target well below saturation — latency explodes near 100%.
• Add redundancy so losing an instance or an AZ does not exceed the target.
• Use Little's Law to size thread and connection pools instead of guessing.
• Measure per-server capacity with a load test; account for scaling lead time.

🧪 Practice

1. Plan the fleet for 12,000 peak QPS, 800 RPS/server measured, 60% utilization target, across two AZs surviving the loss of one.
2. A service handles 3,000 RPS with 120 ms latency. Size its connection pool, and explain what happens if you set it to one tenth of that.
3. Interview: Traffic triples during a 90-second flash sale. Autoscaling needs 4 minutes. What is your plan? (Hint: three families of answers — pre-provision, queue the work, or reject some of it deliberately.)

<a id="14-design-process-and-trade-offs"></a>

1.4 Design Process and Trade-offs

Requirements and numbers in hand, the remaining skill is process: a repeatable method for producing a design, finding its weak point, choosing between options with reasons, and recording those reasons so the system can keep evolving after you have moved on.

Structured Design Methodology

Design under pressure — an interview, a deadline, a room full of opinions — goes badly without a method. A structured approach guarantees that requirements precede architecture, that scale drives storage choice rather than fashion, and that you never run out of time before the interesting parts.

The method below is deliberately linear, but the loop back from step 6 is the real point: designs are refined, not produced in one pass.

THE SIX-STEP METHOD                              45-min interview budget

 1. Clarify requirements                                  ~5 min
    FRs, NFRs, out of scope, constraints, assumptions
 2. Estimate scale                                        ~5 min
    DAU -> QPS -> storage -> bandwidth; note what it implies
 3. Define the API and data model                         ~5-10 min
    Endpoints/events; entities, keys, and access patterns
 4. Draw the high-level design                            ~10-15 min
    Client -> edge -> services -> stores; label every arrow
 5. Deep dive on 1-2 components                           ~10 min
    Usually the bottleneck from step 2 or the hardest FR
 6. Identify bottlenecks, failures, and next steps        ~5 min
    Scale it 10x, kill a component, name the trade-offs

    -> then loop: the deep dive usually invalidates part of step 4

Two rules make the method work in practice. Drive the data model from the access patterns, not from an entity-relationship diagram drawn in a vacuum: the queries you must serve determine the keys, and the keys determine whether the store can be partitioned. And narrate the "why" at each step — a design that is only a diagram is unreviewable, because reviewers cannot tell which lines are load-bearing.

STEP 3 EXAMPLE — driving the model from access patterns (URL shortener)

 Access patterns          Frequency      Implied design
 ----------------------------------------------------------------------
 resolve(short_id)        ~34,000 QPS    Point lookup by primary key ->
                                         key-value store + cache
 create(long_url)         ~350 QPS       Write path; needs unique id gen
 list by owner            ~5 QPS         Secondary index or separate table
 analytics by short_id    batch          Separate store; do not burden the
                                         hot path with counters

 Verdict: the dominant pattern is a single-key point read at high volume.
 That single line rules out anything that requires a scan or a join.

Step	Common mistake	Fix
1	Jumping to architecture in the first minute	Write FRs and NFRs down first
2	Skipping estimation	One minute of arithmetic decides the store
3	Modelling entities without access patterns	Enumerate queries and their frequencies
4	Unlabelled arrows	Label protocol, sync/async, and failure mode
5	Deep-diving a trivial component	Dive where scale or difficulty concentrates
6	Presenting the design as if it has no weaknesses	Name the bottleneck yourself, with numbers

Key Takeaways

• Follow a fixed order: requirements, estimates, API and data model, HLD, deep dive, bottlenecks.
• Estimation belongs before architecture — the numbers pick the components.
• Derive the data model from access patterns and their frequencies.
• Narrate the reasoning; an unexplained diagram cannot be reviewed.

🧪 Practice

1. Apply steps 1-3 to "design a pastebin service" and write the output for each step.
2. Take an existing service you know and reverse-engineer its access patterns with rough frequencies. Does its storage choice match them?
3. Interview: You have ten minutes left and have only finished step 4. What do you prioritize? (Hint: which remaining step most demonstrates senior judgment?)

Identifying Bottlenecks

A system's capacity is set by its most constrained resource, and everything else is slack. Optimizing anything other than the bottleneck produces no improvement at all — this is the core insight of the theory of constraints, and it is why "make the code faster" is so often wasted effort when the real limit is a single database primary.

Bottlenecks announce themselves through saturation: a resource near its limit, a queue that grows, latency climbing while throughput stays flat. The practical method is to go resource by resource. The USE method (Utilization, Saturation, Errors) applied to CPU, memory, disk, and network on every component finds most of them quickly.

FINDING THE BOTTLENECK BY SIGNATURE

  Throughput flat, latency rising, CPU high        -> CPU bound
  Throughput flat, CPU low, queue depth growing    -> downstream/lock bound
  Latency spikes with GC pauses or swap activity   -> memory bound
  High iowait, disk queue length > 1               -> disk bound
  Retransmits, bandwidth at link capacity          -> network bound
  All servers idle, one database at 100% CPU       -> the classic: shared state

  +------+     +------+     +---------+     +--------------+
  |Client| --> | LB   | --> | API x20 | --> |  DB primary  |  <-- 98% CPU
  +------+     +------+     +---------+     +--------------+
                              5% CPU              ^
                                                  |
                          Adding API servers here changes nothing.

# Amdahl-style reasoning applied to a request's time budget.
budget_ms = {
    "lb":        2,
    "app_cpu":   15,
    "db_query": 120,     # 78% of the total
    "cache":      3,
    "serialize":  5,
}
total = sum(budget_ms.values())                    # 145
print(total)

# Halving app CPU time saves 7.5 ms  -> 5% better
# Halving the DB query saves 60 ms   -> 41% better
for name, ms in budget_ms.items():
    saved = ms / 2
    print(f"{name:10} halving saves {saved:5.1f} ms ({saved/total:5.1%})")
# Optimize the biggest term first. Everything else is rounding error.

Bottlenecks also move. Fix the database and the next constraint appears — perhaps the connection pool, then the network, then serialization CPU. Expect to repeat the exercise rather than to finish it, and keep the measurement in place so you can tell where the constraint has migrated.

Key Takeaways

• System capacity equals the capacity of the most constrained resource.
• Optimizing a non-bottleneck yields nothing; find it before tuning anything.
• Look for saturation signatures: flat throughput, growing queues, rising latency.
• Bottlenecks move after each fix — measure continuously, not once.

🧪 Practice

1. Given the time budget above, propose three concrete changes to the dominant term and estimate the gain from each.
2. A service's CPU is at 30%, memory at 40%, and latency has tripled. List four candidate bottlenecks and the metric that would confirm each.
3. Interview: You add servers and throughput does not improve. Walk through your diagnosis. (Hint: enumerate everything the servers share, from the database to the load balancer to a rate-limited third-party API.)

Trade-off Analysis

There are no free wins in system design. Every choice buys something and pays for it somewhere else: a cache buys read latency and pays in staleness and memory; a queue buys resilience and pays in complexity and eventual consistency; a second region buys availability and pays in cost and write latency. Senior judgment is not knowing "the right answer" — it is naming the currency each option is paid in.

Make trade-offs explicit with a small decision matrix. Weight the criteria by your ordered design goals from section 1.1, score the options, and let the result inform (not dictate) the decision. The number is less valuable than the conversation it forces.

Criterion (weight)	Option A: Postgres	Option B: DynamoDB	Option C: Cassandra
Write scale (0.30)	2	5	5
Query flexibility (0.25)	5	2	2
Operational burden (0.20)	4	5	2
Team familiarity (0.15)	5	3	2
Cost at scale (0.10)	4	2	4
Weighted total	3.80	3.65	3.10

weights = {"write_scale": .30, "query_flex": .25, "ops": .20,
           "familiarity": .15, "cost": .10}
options = {
    "postgres":  {"write_scale": 2, "query_flex": 5, "ops": 4, "familiarity": 5, "cost": 4},
    "dynamodb":  {"write_scale": 5, "query_flex": 2, "ops": 5, "familiarity": 3, "cost": 2},
    "cassandra": {"write_scale": 5, "query_flex": 2, "ops": 2, "familiarity": 2, "cost": 4},
}
for name, scores in options.items():
    total = sum(scores[k] * w for k, w in weights.items())
    print(f"{name:10} {total:.2f}")
# postgres 3.80 | dynamodb 3.65 | cassandra 3.10
#
# Postgres and DynamoDB are within noise of each other: the matrix says the
# decision hinges on whether write scale or query flexibility is the real
# constraint — which is a requirements question, not a database question.

Recurring trade-off axes worth having at your fingertips:

Axis	You gain	You pay
Consistency vs. availability	Correct reads under partition	Rejected requests, latency
Latency vs. durability	Fast acknowledgements	Risk of losing recent writes
Normalization vs. denormalization	Write simplicity, no anomalies	Slower reads, more joins
Caching vs. freshness	Read latency and cost	Staleness, invalidation bugs
Synchronous vs. asynchronous	Simple reasoning, immediate result	Coupling, cascading failures
Monolith vs. microservices	Independent scaling and deploys	Network failures, ops burden
Build vs. buy	Control and fit	Engineering time and upkeep

Two guardrails. First, prefer the simplest design that meets the stated NFRs — complexity added "for scale we might need" is a real cost paid today for a speculative benefit. Second, distinguish reversible from irreversible choices: spend your analysis budget on the irreversible ones.

Key Takeaways

• Every design choice has a currency it is paid in; name it explicitly.
• Use a weighted matrix tied to your ordered design goals to structure the comparison.
• Close scores are a signal that the decision depends on an unresolved requirement.
• Favour the simplest option that meets the NFRs; reserve deep analysis for irreversible decisions.

🧪 Practice

1. Build a weighted matrix comparing REST and gRPC for an internal service-to-service API, with weights you can justify.
2. For each axis in the table above, give a real system where the trade-off was resolved in each direction, and why.
3. Interview: "Would you use SQL or NoSQL here?" How do you answer well? (Hint: the strong answer starts with an access pattern, not a preference.)

Documenting Design Decisions

A design that lives only in a diagram and in one engineer's head decays fast. Six months later nobody remembers why the queue is there, so someone removes it and rediscovers the reason during an incident. Documentation is not bureaucracy; it is how a system stays intelligible longer than the tenure of the people who built it.

The two artifacts that carry the load are the design document (a proposal, written before building, to align people and surface objections while changes are cheap) and the architecture decision record (a short, immutable note recording one decision and its reasoning — covered in the next topic).

A design doc worth writing follows a predictable shape:

DESIGN DOC TEMPLATE

 1. Title, author, date, status        draft | in review | approved | superseded
 2. Context and problem                what is broken or needed, with evidence
 3. Goals / Non-goals                  ordered; non-goals prevent scope creep
 4. Requirements                       FRs and measurable NFRs
 5. Proposed design                    diagram + data model + key flows
 6. Alternatives considered            each with why it was rejected
 7. Trade-offs and risks               what this design is bad at
 8. Migration / rollout plan           including how to roll back
 9. Operational impact                 monitoring, alerts, on-call, cost
10. Open questions                     the honest list

Sections 6 and 7 are where the value concentrates. A document that presents one option with no alternatives and no acknowledged weaknesses is a sales pitch, and reviewers will treat it as one.

<!-- Excerpt: the two sections most often written badly -->
## Alternatives considered

**A. Synchronous write to both stores.** Rejected: couples checkout
availability to the analytics store (0.9995 x 0.999 = 0.9985 -> misses the
99.95% NFR) and adds ~40 ms to p99 checkout latency.

**B. Nightly batch export.** Rejected: 24-hour freshness violates the
"dashboard within 5 minutes" requirement (NFR-4).

**C. CDC from the transaction log (chosen).** Meets freshness with ~30 s lag,
keeps checkout decoupled; costs an extra pipeline to operate.

## Trade-offs and risks

- Analytics data is eventually consistent; the dashboard may briefly disagree
  with the ledger. Accepted: the ledger remains the source of truth.
- New operational surface: a CDC pipeline whose lag must be alerted on.
- If the pipeline is down > 6 h, replay requires log retention >= 6 h.

Write the doc before implementation, keep it short enough that people actually read it (two to five pages), and mark it superseded rather than deleting it when it is replaced — the trail of superseded documents is the system's intellectual history.

Key Takeaways

• Documentation preserves reasoning after the authors and the context are gone.
• A design doc states context, goals, requirements, design, alternatives, trade-offs, and rollout.
• Alternatives-considered and known-weaknesses are the highest-value sections.
• Write it before building; supersede rather than delete.

🧪 Practice

1. Write a one-page design doc for adding full-text search to an existing application, including at least two rejected alternatives.
2. Take a system you built and write its "trade-offs and risks" section honestly.
3. Interview: How do you convince a team that resists writing design docs? (Hint: tie it to a cost they already feel, such as review time or repeated incidents.)

Architecture Decision Records (ADRs)

A design doc describes a whole proposal; an ADR records exactly one decision, in one page, immutably. The distinction matters because decisions have a much longer half-life than proposals: teams need to answer "why is it like this?" far more often than "what did we plan to build?".

The defining property of an ADR is immutability. You never edit an approved ADR to reflect a new decision — you write a new ADR that supersedes it. The result is an append-only log of the architecture's reasoning, in chronological order, that a new engineer can read front to back.

# ADR-014: Use Kafka for order event distribution

- **Status:** Accepted        <!-- Proposed | Accepted | Deprecated | Superseded by ADR-021 -->
- **Date:** 2026-03-11
- **Deciders:** Platform team
- **Related:** ADR-009 (service boundaries), supersedes ADR-004

## Context

Six services need order state changes. Today the order service calls each of
them synchronously, so its availability is the product of seven services
(0.999^7 = 99.3%, below the 99.9% NFR), and adding a consumer requires
changing the order service.

## Decision

Publish order state changes to a Kafka topic partitioned by `order_id`.
Consumers subscribe independently and must be idempotent. Retention: 7 days.

## Consequences

**Positive**
- Order service availability no longer depends on consumers.
- New consumers are added with no change to the producer.
- Replay is possible within the retention window.

**Negative**
- Consumers see eventual consistency (typical lag < 1 s).
- Kafka becomes a critical dependency requiring operational expertise.
- At-least-once delivery: every consumer must implement deduplication.

**Neutral**
- Ordering is guaranteed per `order_id` only, which matches our requirement.

Field	Purpose
Status	Lifecycle marker; superseded ADRs stay in the log
Context	The forces at play — constraints, numbers, and pressures
Decision	One sentence in the active voice: "We will ..."
Consequences	Both the wins and the costs, stated plainly

Practical conventions: store ADRs in the repository next to the code (docs/adr/0014-kafka-for-order-events.md) so they are versioned with it; number them sequentially; keep them to one page; and write one only for decisions that are costly to reverse or that people will question later — not for every library choice.

Key Takeaways

• An ADR records a single decision: context, decision, consequences.
• ADRs are immutable — supersede with a new record rather than editing.
• Consequences must include the negatives; that is what future readers need.
• Keep ADRs in version control beside the code, numbered and short.

🧪 Practice

1. Write an ADR for "we will use UUIDv7 instead of auto-increment integers for primary keys", including at least two negative consequences.
2. Find a decision in a codebase you know whose reasoning is undocumented and write the ADR that should have existed.
3. Interview: When would you not write an ADR? (Hint: think about the cost of reversal and how many people the decision constrains.)

Evolutionary Architecture

No system is designed once and finished. Requirements change, traffic grows by orders of magnitude, and the assumptions in your first design log go stale. Evolutionary architecture is the practice of designing so that change stays cheap — optimizing for the ability to change rather than for a predicted end-state that will not arrive as predicted.

This starts with classifying decisions by reversibility. A two-way door can be walked back cheaply: a library choice, a cache eviction policy, an internal API shape. A one-way door is expensive or impossible to undo: a public API contract, a data model that has accumulated a petabyte, a choice of cloud primitive with no equivalent elsewhere. Decide two-way doors fast and one-way doors slowly.

DECISION SPEED BY REVERSIBILITY

  high  |  Deliberate, prototype,     |  Deliberate hard;
  cost  |  write an ADR               |  this is where senior
  to    |  (e.g. datastore choice)    |  time belongs
  undo  |                             |  (e.g. public API contract)
        |-----------------------------+---------------------------
   low  |  Just decide                |  Decide fast, revisit later
        |  (e.g. logging library)     |  (e.g. internal service split)
        +-----------------------------+---------------------------
              low blast radius              high blast radius

The complementary tool is the fitness function: an automated check that asserts an architectural property continuously, so erosion is caught the way a failing test is caught rather than discovered during an incident.

# Fitness functions run in CI and fail the build when the architecture drifts.
def test_p99_latency_within_slo(load_test):
    assert load_test.p99_ms < 200            # performance is a testable property

def test_no_layering_violation(imports):
    # The domain layer must not depend on the web layer, ever.
    assert not any(i.startswith("app.web") for i in imports.of("app.domain"))

def test_service_dependency_graph_is_acyclic(graph):
    assert graph.is_acyclic()                # cycles make deploys undeployable

def test_public_api_is_backward_compatible(old_schema, new_schema):
    assert new_schema.is_compatible_with(old_schema)   # one-way door, guarded

Three habits keep evolution possible in practice:

• Defer irreversible commitments until the requirement is real. YAGNI is not laziness; it is refusing to pay today for a guess about tomorrow.
• Migrate incrementally. The strangler fig pattern routes a slice of traffic to a new implementation behind a facade, grows the slice, then removes the old path — no big-bang rewrite, and a rollback at every step.
• Keep contracts stable and internals fluid. Versioned APIs and events let you rewrite everything behind them without coordinating a global change.

STRANGLER FIG MIGRATION

  t0   [Client] -> [Facade] -> [Legacy]                     100% legacy
  t1   [Client] -> [Facade] -> [Legacy]  90%
                          \-> [New]      10%   (shadow or real traffic)
  t2   [Client] -> [Facade] -> [Legacy]  10%
                          \-> [New]      90%
  t3   [Client] -> [Facade] -> [New]                        legacy removed

  Every step is individually reversible, which is the entire point.

Finally, revisit the assumption log from section 1.1 on a schedule. Most architectural decay is not a wrong decision — it is a right decision whose assumptions quietly stopped being true.

Key Takeaways

• Design for changeability, not for a predicted end-state.
• Separate two-way doors (decide fast) from one-way doors (decide carefully).
• Fitness functions turn architectural properties into automated tests.
• Migrate incrementally behind stable contracts; revisit assumptions on a schedule.

🧪 Practice

1. Classify as one-way or two-way doors: choosing a programming language, choosing an event schema format, exposing a public webhook, picking a logging library.
2. Write three fitness functions for a system you know — one for performance, one for structure, one for compatibility.
3. Interview: You must replace a 10-year-old monolith that still serves all production traffic. Describe your approach. (Hint: the answer is a sequence of reversible steps, and the first one is not writing code.)

Chapter Summary

Concept	The one-line version
System design	Structuring components — services, stores, networks — not just code
HLD vs. LLD	Boxes and contracts first; schemas and algorithms second
FRs vs. NFRs	What it does vs. how well; NFRs dictate the topology
Constraints/assumptions	Constraints are imposed, assumptions are chosen — label and verify them
Quality attributes	Scalability, availability, reliability, maintainability, latency, consistency, durability, cost
Availability math	Series multiplies down; parallel redundancy multiplies failure down
Little's Law	L = lambda x W ties concurrency, throughput, and latency together
Estimation	DAU -> QPS -> storage -> bandwidth -> fleet size, to one significant figure
Capacity planning	Size for peak, below the queueing knee, minus one failure domain
Bottlenecks	Capacity is set by the most constrained resource; it moves after each fix
Trade-offs	Name the currency each option is paid in; weight by ordered design goals
ADRs	One decision per record, immutable, consequences included
Evolutionary design	Decide two-way doors fast, one-way doors slowly; migrate incrementally

---

<a id="2-computing-and-networking-fundamentals"></a>