1. Foundations of Computer Networking

This chapter builds the vocabulary and mental models that every later chapter depends on: what a network actually is, how networks are classified by scale and shape, the two fundamentally different ways data can be moved across a shared infrastructure, and the numbers used to measure whether a network is doing its job well. Everything that follows — layering, addressing, routing, congestion control — is an elaboration of the ideas introduced here. Get these foundations right and the rest of networking becomes engineering detail rather than memorisation.

<a id="11-what-is-a-network"></a>

1.1 What Is a Network

This subchapter defines the basic objects of study — the machines, the wires between them, the roles they play, and the historical accidents that made today's Internet look the way it does.

Definition and Purpose of Networks

Theory

Before networks, a computer was an island. Data moved between machines by physically carrying tapes or disks — the "sneakernet". This is slow, error-prone, and impossible to automate. The moment two organisations or two people need to cooperate in real time, physical transport of media stops being viable.

A computer network is a collection of autonomous computing devices interconnected by a communication system that allows them to exchange data. Two words in that definition carry weight:

• Autonomous — no device controls another's execution. A CPU and its RAM are connected, but RAM is not autonomous, so that is not a network. Two laptops exchanging files are.
• Exchange data — the purpose is moving information, not sharing a clock or a power supply.

Why build one? Four enduring motivations:

Motivation	What it means	Everyday example
Resource sharing	Expensive or scarce resources serve many users	One office printer, one GPU cluster
Communication	People and programs exchange messages	Email, chat, video calls
Remote access	Use a machine you are not sitting at	SSH into a server, cloud computing
Reliability	Replicate data and paths so failure is survivable	RAID across hosts, multiple ISP links

A useful analogy is the road system. Individual buildings (computers) are useless in isolation for commerce; roads (links) plus junctions (routers) plus traffic rules (protocols) turn a set of buildings into an economy. Notice that the road network does not care what is in the truck — it only cares about getting the truck from A to B. Networks are the same: they move bits and are deliberately ignorant of their meaning.

Examples

Without a network: three isolated islands
  [PC A]        [PC B]        [PC C]
   (files)       (files)       (printer)
   -> to print from A, walk a USB stick to C

With a network: one shared system
  [PC A]---+
           |
  [PC B]---+---[ switch ]---[ printer ]
           |
  [PC C]---+
   -> any host uses any resource, programmatically

# The simplest possible demonstration that a network exists:
# send a probe to another host and see whether it answers.
ping -c 3 example.com
# 64 bytes from 93.184.216.34: icmp_seq=1 ttl=56 time=11.3 ms
#        ^ the other machine exists, is reachable, and answered in 11.3 ms

Key takeaways

• A network is autonomous devices plus a communication system used to exchange data.
• The four classic drivers are resource sharing, communication, remote access, and reliability.
• Networks are content-agnostic: they move bits, not meaning.
• The value of a network grows superlinearly with the number of connected parties, which is why interconnection (the Internet) beat isolated private networks.

🧪 Practice

1. List five networked interactions you personally performed in the last hour, and classify each under one of the four motivations in the table above.
2. Argue whether a USB flash drive carried between two buildings counts as a network. Define precisely which part of the definition it satisfies or violates. (Hint: consider latency and bandwidth — a station wagon full of tapes has enormous bandwidth.)
3. Interview-style: Your company has 50 employees who each need occasional access to a very expensive rendering machine. Describe two designs — one networked, one not — and state the dominant cost in each. (Hint: think about idle time of the expensive resource, not just purchase price.)

Hosts, Links, and Nodes

Theory

To reason about networks precisely, we abstract them as a graph. The vertices are devices; the edges are communication channels.

• A host (also end system) is a device that originates or ultimately consumes data: a laptop, a phone, a web server, a sensor. Hosts run the applications people care about.
• A link is a communication channel connecting two or more devices. It may be a copper cable, a fibre strand, or a slice of radio spectrum. Links are characterised by their capacity (bits per second) and their propagation delay.
• A node is any device attached to the network. Every host is a node, but so is every intermediate device — switches, routers, access points — whose job is to relay data it did not create and will not consume.

The critical distinction is between end systems and intermediate systems. Hosts sit at the edges and care about what is being sent. Intermediate nodes sit in the middle and care only about where it must go next. This separation is the reason the Internet scales: a router in Frankfurt forwarding your video stream neither knows nor cares that it is video.

Links come in two connectivity flavours:

• Point-to-point — exactly two endpoints share the link (a fibre between two routers). No contention.
• Broadcast / multi-access — many devices share one medium (classic Ethernet coax, Wi-Fi). Simultaneous transmissions interfere, so an access-control rule is needed (covered in Chapter 4).

Examples

Graph abstraction of a small network

   [Host A]                         [Host D]
       \                               /
        \                             /
     [Switch S1]---[Router R1]---[Switch S2]
        /                             \
       /                               \
   [Host B]                         [Host E]

 Hosts:  A, B, D, E          (create and consume data)
 Nodes:  A, B, D, E, S1, S2, R1   (everything attached)
 Links:  A-S1, B-S1, S1-R1, R1-S2, S2-D, S2-E   (7 edges)

# traceroute reveals the intermediate nodes between you and a host.
traceroute -n example.com
#  1  192.168.1.1      1.2 ms    <- your home router (intermediate node)
#  2  10.20.0.1        9.8 ms    <- ISP access router
#  3  62.115.120.1    11.4 ms    <- ISP backbone router
#  4  93.184.216.34   11.9 ms    <- the destination host (end system)
# Each line is one node; each hop between lines traverses one or more links.

Key takeaways

• Model a network as a graph: nodes are devices, links are edges.
• Hosts are end systems that produce or consume data; intermediate nodes only relay it.
• The host/intermediate split lets the core stay application-agnostic, which is what makes the Internet scalable.
• Links are either point-to-point (no contention) or broadcast (contention, needs a MAC protocol).

🧪 Practice

1. Draw the graph of your home network, labelling every node as host or intermediate and every link as point-to-point or broadcast.
2. Run traceroute (or tracert on Windows) to three websites in different countries. How many intermediate nodes does each path contain, and where do the paths diverge?
3. A network has 6 hosts all connected to a single switch. How many links exist? Now replace the switch with a shared Wi-Fi access point — how many physical links exist, and how does contention change?

Clients, Servers, and Peers

Theory

"Client" and "server" describe roles in a conversation, not categories of hardware. The role is defined by who initiates:

• A client initiates a request and waits for a response. It typically has an intermittent connection, a dynamic address, and does not need to be reachable from outside.
• A server waits passively for incoming requests and responds. It must be always-on and reachable at a stable, well-known address, because clients must be able to find it.

A single machine is routinely both. Your laptop is a client to a web server, and a server to the phone streaming from it. A web server is a server to browsers and a client to its database.

The peer-to-peer (P2P) model removes the asymmetry: every participant both requests and serves. Its defining property is self-scaling — each new peer adds demand but also adds capacity. In a pure client-server system, every new user adds load with no added capacity, so the server must be scaled by its operator.

Property	Client-Server	Peer-to-Peer
Who initiates	Client only	Any peer
Capacity as users grow	Fixed (operator must scale)	Grows with the user base
Address stability needed	On the server	Ideally on all peers (hard behind NAT)
Failure of one node	Server failure kills service	Degrades gracefully
Management, billing, control	Centralised and simple	Distributed and hard
Typical examples	HTTP, SMTP, SQL databases	BitTorrent, blockchain gossip, WebRTC

Most real systems are hybrid: a central server handles discovery, identity, and coordination, while bulk data flows peer-to-peer. A video-call service uses servers for signalling ("who is in this room?") and direct peer connections for the media itself.

Examples

# The role difference expressed in code — same library, opposite posture.

# SERVER: binds a well-known address and waits.
import socket
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
srv.bind(("0.0.0.0", 8080))   # claim a stable, known address:port
srv.listen(5)                  # become passive: wait for others
conn, addr = srv.accept()      # blocks until a client initiates
conn.sendall(b"hello\n")

# CLIENT: knows where the server is and initiates.
import socket
cli = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
cli.connect(("example.com", 8080))  # the active, initiating side
print(cli.recv(1024))

Scaling behaviour with N downloaders of one 1 GB file

Client-server:            Peer-to-peer:
  server upload = 100 Mb/s  each peer uploads what it has
  N=10  -> 10 Mb/s each     N=10  -> peers feed each other
  N=100 -> 1 Mb/s each      N=100 -> aggregate capacity also x10
  capacity is CONSTANT      capacity GROWS with N

Key takeaways

• Client and server are roles per conversation, not machine types; one host can be both simultaneously.
• The client initiates; the server waits at a stable, well-known address.
• P2P self-scales because every new participant contributes capacity as well as demand.
• Hybrid designs dominate in practice: centralised coordination, decentralised data transfer.

🧪 Practice

1. For each of these, name the client and the server: loading a web page, sending an email, a DNS lookup, a printer receiving a job. Identify any case where one machine plays both roles.
2. Modify the Python server above to echo back whatever the client sends, then connect to it with telnet localhost 8080 or nc localhost 8080.
3. Interview-style: Why is pure peer-to-peer file distribution rare for commercial video streaming despite its scaling advantage? (Hint: think about upload bandwidth asymmetry in consumer connections, NAT, and who is accountable for content.)

Network Edge vs Network Core

Theory

It is useful to split the Internet into two regions with completely different jobs.

The network edge is where hosts attach: your devices, plus the access network that connects them to their provider — DSL, cable, fibre-to-the-home, Wi-Fi, 4G/5G. The edge is where applications live, where policy about users is enforced, and where bandwidth is usually scarcest and most variable.

The network core is the mesh of high-capacity routers and long-haul links that interconnect access networks. The core has one job: forward packets toward their destination as fast as possible. It deliberately holds no per-application state.

This split reflects a deep design decision known as the end-to-end principle: put functionality (reliability, encryption, ordering) at the edges in the end systems, and keep the core simple and dumb. A simple core is fast, cheap, and easy to scale; a smart core would need to be upgraded for every new application. That the Internet could accommodate the Web, video streaming, and video calling without core routers being redesigned each time is the payoff of this choice.

Access networks vary along a few axes worth remembering:

Access type	Typical downstream	Shared or dedicated	Note
DSL	10–100 Mb/s	Dedicated to the exchange	Rate falls with distance
Cable (HFC)	100 Mb/s–1 Gb/s	Shared among neighbours	Peak-hour contention
Fibre (FTTH)	1–10 Gb/s	Often shared (PON)	Lowest latency and jitter
4G/5G cellular	10 Mb/s–1 Gb/s	Shared per cell	Mobility, variable quality
Enterprise Ethernet	1–100 Gb/s	Dedicated per port	Switched, full-duplex

Examples

                     NETWORK CORE
        (high-capacity routers, no per-app state)

              [R]========[R]========[R]
             //  \\      //         \\
          [R]     [R]==[R]           [R]
           |                          |
      -----+--------------------------+-----
           |         NETWORK EDGE     |
      [ISP access net]          [Data centre]
       /     |     \                  |
  [phone] [laptop] [TV]          [web servers]

Key takeaways

• The edge holds hosts and access networks; the core holds routers whose only job is forwarding.
• The end-to-end principle puts intelligence at the edge and keeps the core simple, which is why the Internet absorbed new applications without redesign.
• Access networks differ in capacity, in whether the medium is shared, and in latency stability.
• Performance problems are far more often at the edge (the "last mile") than in the core.

🧪 Practice

1. Identify your own access network type and find its advertised downstream and upstream rates. Is the medium shared with neighbours?
2. Run a speed test at 3 a.m. and at 9 p.m. Explain any difference using the shared/dedicated distinction.
3. Interview-style: Give one concrete function that the end-to-end principle says belongs in the hosts rather than in routers, and one exception where putting it in the network is justified anyway. (Hint: reliable delivery versus congestion signalling.)

Brief History of the Internet

Theory

The Internet's design is not the product of a single clean specification; it is the accumulated result of specific problems solved in a specific order. Knowing the sequence explains many otherwise puzzling details.

1961–1968 — Packet switching is invented. Leonard Kleinrock analyses queueing for store-and-forward networks; Paul Baran (RAND) and Donald Davies (NPL) independently propose chopping messages into blocks to survive link failure. The telephone network's dedicated-circuit model is rejected as too fragile and too wasteful for bursty computer traffic.

1969 — ARPANET. Four nodes (UCLA, SRI, UCSB, Utah) are connected using Interface Message Processors — the first routers. The first message crashes the receiving host after two characters.

1973–1974 — TCP/IP is designed. Vint Cerf and Bob Kahn address internetworking: joining heterogeneous networks that each have their own technology. Their answer is a universal packet format and a gateway that translates nothing but forwards everything. The original single TCP is later split into TCP (reliability, end systems) and IP (addressing and forwarding, everywhere).

1983 — Flag day. ARPANET switches from NCP to TCP/IP on 1 January. This is the birth of the Internet as we know it, and the last time a coordinated protocol switch of that kind was possible.

1983–1985 — DNS. Hosts had been found via a manually distributed HOSTS.TXT file, which stopped scaling. Paul Mockapetris designs a hierarchical, delegated, cached naming system.

1986–1995 — NSFNET and commercialisation. The US National Science Foundation builds a backbone connecting supercomputing centres, and the acceptable-use policy barring commercial traffic is lifted in the early 1990s. NSFNET is decommissioned in 1995; the backbone becomes a market of competing commercial providers.

1989–1993 — The Web. Tim Berners-Lee at CERN specifies URLs, HTTP, and HTML. The Mosaic browser (1993) makes it visual, and traffic explodes.

1998 onward — Scaling pains and repairs. IPv6 is specified in response to address exhaustion; CIDR and NAT buy time; CDNs move content closer to users; TLS becomes default; QUIC (2021) rebuilds transport on UDP because middleboxes had ossified TCP.

Year	Event	Why it still matters
1969	ARPANET's first four nodes	Proof that packet switching works
1974	TCP/IP design published	The "internetworking" idea: one format over many networks
1983	NCP to TCP/IP cutover	IP becomes the narrow waist
1984	DNS deployed	Names decouple from addresses
1991	World Wide Web released	Drives mass adoption
1993	Mosaic browser	Makes the Web usable by non-specialists
1995	NSFNET retired	Internet becomes commercial and multi-provider
1998	IPv6 standardised (RFC 2460)	Response to IPv4 exhaustion
2021	HTTP/3 and QUIC standardised	Transport innovation routed around ossification

Key takeaways

• Packet switching was chosen over circuit switching for resilience and for efficiency with bursty traffic.
• "Internetworking" means connecting heterogeneous networks with a common packet format — that is the actual invention behind TCP/IP.
• The 1983 flag day was the last coordinated upgrade; ever since, changes must be incrementally deployable, which explains the slowness of IPv6.
• Commercialisation in 1995 turned a single backbone into a competitive market of interconnected providers, making routing policy an economic question.

🧪 Practice

1. Build a timeline of ten events from 1960 to today, and for each write one sentence on what present-day design detail it explains.
2. Explain why a coordinated "flag day" upgrade to IPv6 is impossible now but was possible for TCP/IP in 1983. Quantify your reasoning with rough host counts.
3. Interview-style: What problem was NAT introduced to solve, and what property of the original Internet architecture did it break? (Hint: think about address exhaustion and about which host can initiate a connection.)

<a id="12-network-types-and-scale"></a>

1.2 Network Types and Scale

Networks are classified first by geographic reach, because scale dictates the technology, the ownership model, and the economics.

PAN, LAN, MAN, and WAN

Theory

The physics of distance drives everything. A signal over one metre of copper and a signal over 5,000 km of fibre face completely different problems: the long link cannot avoid propagation delay (about 5 microseconds per kilometre in fibre), costs orders of magnitude more per metre, and usually crosses property boundaries it does not own. So networks are built in distinct classes.

• PAN (Personal Area Network) — a few metres, around one person. Bluetooth earbuds, a smartwatch, a wireless keyboard. Optimised for very low power, not speed.
• LAN (Local Area Network) — a room, floor, or building, typically under a few hundred metres. Ethernet and Wi-Fi. Owned entirely by one organisation, so it can be fast and cheap: high bandwidth, microsecond latency, no leasing fees.
• MAN (Metropolitan Area Network) — a city or campus cluster, tens of kilometres. Often provider-operated metro Ethernet or fibre rings connecting an organisation's several sites.
• WAN (Wide Area Network) — countries and continents. Almost always built on circuits leased from carriers, because laying your own transoceanic fibre is a different business. High latency is unavoidable, and cost per bit is highest.

The key inversion to remember: as distance grows, bandwidth gets more expensive and latency becomes irreducible. A LAN can afford to be wasteful; a WAN cannot. This is why techniques like caching, compression, and chattiness-reduction matter enormously over a WAN and barely at all over a LAN.

Class	Typical span	Typical technology	Typical rate	Latency	Ownership
PAN	1–10 m	Bluetooth, BLE, NFC, Zigbee	1–50 Mb/s	ms	Personal
LAN	10 m – 1 km	Ethernet, Wi-Fi	1–100 Gb/s	< 1 ms	Single organisation
MAN	1–50 km	Metro Ethernet, fibre ring, DOCSIS	1–100 Gb/s	1–5 ms	Provider or city
WAN	50 km+	MPLS, leased lines, SD-WAN, submarine fibre	1 Mb/s – 400 Gb/s	10–300 ms	Carrier, leased

Two boundary cases are worth noting. A SAN (Storage Area Network) is a LAN-scale network specialised for block storage traffic, and a WLAN is simply a LAN whose medium is radio — the "W" changes the medium, not the scale.

Examples

Scale ladder, with a representative round-trip time

 PAN   [watch]~[phone]                          RTT ~ 10 ms   (2 m)
 LAN   [laptop]--[switch]--[NAS]                RTT ~ 0.3 ms  (30 m)
 MAN   [office A]==[metro ring]==[office B]     RTT ~ 2 ms    (20 km)
 WAN   [London]=====[submarine fibre]=====[NY]  RTT ~ 70 ms   (5,600 km)

 Note: the PAN has the SHORTEST distance but NOT the lowest RTT.
 Radio protocol overhead, not distance, dominates at small scale.

# Measure the ladder yourself: each should land in a different order of magnitude.
ping -c 4 192.168.1.10        # LAN host      -> ~0.3 ms
ping -c 4 your-isp-gateway    # access/MAN    -> ~5-15 ms
ping -c 4 www.example.com     # WAN           -> ~80-250 ms

Key takeaways

• Classification is by geographic span, and span determines cost, achievable latency, and ownership model.
• LANs are owned outright, so bandwidth is cheap; WANs are leased, so bandwidth is expensive and latency is bounded by physics.
• Propagation in fibre is roughly 200,000 km/s — about 5 µs per km — and no protocol can beat it.
• Application design that ignores the LAN/WAN distinction (chatty request-response over a WAN) produces the classic "works in dev, crawls in production" failure.

🧪 Practice

1. Classify each of these: Bluetooth headphones, an office Wi-Fi network, a university connecting three city campuses, a bank linking London and Singapore.
2. Compute the minimum possible round-trip time between London and Sydney (about 17,000 km of fibre) at 200,000 km/s. Compare it with a real ping result and explain the gap.
3. Interview-style: An application makes 200 sequential database queries per page load. It is fast in the same data centre and unusable when the database is moved to another continent. Explain why, in numbers. (Hint: multiply the query count by the RTT.)

Enterprise, Campus, and Data Center Networks

Theory

Beyond raw distance, networks are also classified by what they are for, because purpose determines traffic pattern, and traffic pattern determines topology.

An enterprise network is the whole of an organisation's networking: offices, remote workers, branch sites, WAN links, and the security perimeter. Its defining characteristic is heterogeneity and policy — who may reach what matters more than raw throughput.

A campus network covers several buildings in one geographic area under one owner (a university, a hospital, a corporate headquarters). Classically built in three tiers:

• Access layer — switches that end-user devices plug into. High port count, low cost.
• Distribution layer — aggregates access switches, enforces policy, does inter-VLAN routing.
• Core layer — high-speed backbone that moves traffic between distribution blocks and does as little as possible.

A data centre network connects servers to each other and to the outside. Its traffic profile is radically different from a campus. In a campus, most traffic is north-south: from a user, up through the tiers, and out to the Internet. In a data centre, most traffic is east-west: server to server, as one user request fans out across microservices, caches, and databases. Three-tier trees handle north-south well but create bottlenecks and unused links for east-west traffic, so data centres moved to leaf-spine designs where every leaf connects to every spine, giving equal-cost paths and uniform any-to-any bandwidth.

Aspect	Campus	Data centre
Dominant traffic	North-south (user to Internet)	East-west (server to server)
Typical topology	Three-tier tree	Leaf-spine (Clos)
Endpoints	Laptops, phones, printers	Servers, VMs, containers
Endpoint mobility	High (people move)	Low physically, high virtually
Key metric	Coverage, policy, cost per port	Bisection bandwidth, latency, oversubscription
Typical port speed	1 Gb/s	25–400 Gb/s

Examples

CAMPUS: three-tier                DATA CENTRE: leaf-spine
                                  every leaf reaches every spine
        [ Core ]                     [Spine1]   [Spine2]
        /      \                      |  \       /  |
   [Dist A]   [Dist B]                |   \     /   |
    /    \     /    \                 |    \   /    |
[Acc][Acc] [Acc][Acc]              [Leaf1] X X  [Leaf2]
  |    |     |    |                   |           |
users users users users           servers      servers

Tree: one path up, blocked links   Clos: many equal-cost paths,
under spanning tree.               all links carry traffic (ECMP).

Key takeaways

• Purpose determines traffic direction, and traffic direction determines topology.
• Campus networks are three-tier (access, distribution, core) and optimised for north-south traffic.
• Data centres are east-west dominated and use leaf-spine so any server can reach any other at predictable bandwidth.
• Enterprise networking is mostly about policy and heterogeneity; data centre networking is mostly about uniform bandwidth and low latency.

🧪 Practice

1. Draw a three-tier campus for a school with four buildings of 200 users each, and label which layer enforces VLAN policy.
2. In a leaf-spine fabric with 4 spines and 16 leaves, how many spine-facing links does each leaf need, and how many total leaf-to-spine links exist?
3. Explain why a microservices architecture increases east-west traffic. Estimate the internal request count for one user page load that touches 8 services, each calling 2 others.

Internet, Intranet, and Extranet

Theory

These three terms are distinguished by who is allowed in, not by technology. All three use the same protocols — IP, TCP, HTTP, DNS. The difference is the access boundary.

• The Internet (capital I) is the single global network of interconnected autonomous networks, publicly reachable, using the public IP address space and the global DNS root. There is exactly one.
• An intranet is a private network using Internet technologies, reachable only by an organisation's own members. Company wikis, HR portals, internal APIs. It typically uses private address space (RFC 1918) and an internal DNS view.
• An extranet is an intranet with controlled access extended to specific outside parties — suppliers, partners, key customers. It is the intersection: private, but deliberately porous at named points, usually via VPN, federated identity, or a mutually authenticated API.

Think of it as a building. The Internet is the public street. The intranet is the interior of your offices, badge-controlled. The extranet is the meeting room a courier is escorted into: inside the building, but only that room, and only for named visitors.

	Internet	Intranet	Extranet
Who can access	Anyone	Employees only	Employees plus named partners
Address space	Public IP	Usually private (RFC 1918)	Private, reached via VPN or gateway
DNS	Global root	Internal zones, split-horizon	Internal zones exposed selectively
Trust model	Zero	Historically high (now questioned)	Explicit, per-partner
Example	example.com	wiki.internal	Supplier ordering portal

A modern caveat: the assumption that "inside the intranet means trusted" is now considered a design flaw. Zero trust networking treats every request as untrusted regardless of where it originates, authenticating and authorising each one. The perimeter still exists, but it is no longer the security model.

Examples

                    [ THE INTERNET ]
                           |
                    [ firewall / VPN gateway ]
                     /                    \
            [ INTRANET ]              [ EXTRANET zone ]
      wiki, HR, internal APIs      partner order portal,
      employees only               supplier file exchange
                                   named partners only

# Split-horizon DNS: the same name resolves differently
# depending on which side of the boundary you ask from.
dig +short portal.example.com @8.8.8.8          # public resolver
# 203.0.113.50      <- the extranet gateway, publicly reachable

dig +short portal.example.com @10.0.0.53        # internal resolver
# 10.4.2.17         <- the real internal server, private address

Key takeaways

• Internet, intranet, and extranet differ by access policy, not by protocol — all three run TCP/IP.
• There is exactly one Internet; there are countless intranets.
• An extranet is a deliberately controlled opening in an intranet for named external parties.
• "Inside equals trusted" is obsolete; zero trust authenticates every request regardless of origin.

🧪 Practice

1. For your own organisation or school, name one service on each of the three: Internet-facing, intranet-only, extranet.
2. Explain how split-horizon DNS works and one concrete problem it solves. What breaks if a laptop caches the internal answer and then goes home?
3. Interview-style: An attacker phishes one employee laptop and gains a foothold on the intranet. Explain why a perimeter-only security model fails here and what zero trust would change. (Hint: consider lateral movement.)

Internet Service Providers and Tiers

Theory

No single organisation owns the Internet. It is roughly 75,000 independently operated networks that agree to carry each other's traffic. Providers are informally ranked by how much of the Internet they can reach without paying anyone.

• Tier 1 — can reach the entire Internet purely through settlement-free peering with other Tier 1 networks. They buy transit from nobody. There are only a handful (examples: Lumen, Arelion, GTT, NTT, Tata, Telia-class carriers). Membership is defined by this property, not by size alone.
• Tier 2 — has substantial peering but must buy transit from at least one upstream provider to reach the whole Internet. Most large national and regional carriers are Tier 2.
• Tier 3 — buys transit only; a local access provider selling connectivity to homes and businesses.

Two commercial relationships explain nearly all Internet routing policy:

• Transit — a customer pays a provider to carry its traffic to and from everywhere. Money flows one way; the provider advertises the customer's routes to the whole Internet.
• Peering — two networks exchange traffic between their own customers only, usually without payment, because both benefit by avoiding transit fees. Crucially, a peer will not carry your traffic onward to its other peers or providers. That restriction is what keeps the hierarchy from collapsing.

The consequence: a route's selection is often about money, not distance. A path with fewer hops may be rejected because it would cost transit fees, while a longer peering path is free. This is why Internet paths are frequently not the geometrically shortest ones.

Examples

The money flows downward; traffic flows both ways.

        [Tier 1 A]<--- settlement-free peering --->[Tier 1 B]
             |                                         |
        $ transit                                 $ transit
             |                                         |
        [Tier 2 C]<--- peering (own customers) --->[Tier 2 D]
             |                                         |
        $ transit                                 $ transit
             |                                         |
        [Tier 3 E]                                [Tier 3 F]
             |                                         |
          homes                                     homes

Rule: C will carry E's traffic to A (E is a paying customer).
      C will NOT carry D's traffic to A (D is a free peer, not a customer).

# Inspect which networks a path crosses and who operates them.
traceroute -A example.com
#  3  10.x.x.x       [AS64500]   <- your Tier 3 ISP
#  5  62.x.x.x       [AS1299]    <- a Tier 1 transit provider
#  8  93.184.216.34  [AS15133]   <- the destination's network
# The [ASnnnn] tags name the independently operated networks crossed.

Key takeaways

• Tier 1 is defined by needing no transit — reaching everything via settlement-free peering.
• Transit is a paid, full-reachability service; peering is usually free and covers only each party's own customers.
• Peering does not transit: a peer will not carry your traffic to its own upstream.
• Internet routes follow commercial policy at least as much as geography, so paths are often not the shortest ones.

🧪 Practice

1. Look up your own ISP's AS number (for example with whois or a public BGP looking glass) and find out whether it buys transit.
2. Using the diagram above, explain why traffic from E to F might traverse a Tier 1 even though C and D peer directly.
3. Interview-style: Two large networks of similar size refuse to peer, forcing their mutual customers onto slower paths. What is each side's incentive, and what leverage ends such disputes? (Hint: consider traffic ratios and who feels customer complaints first.)

Autonomous Systems and Peering

Theory

An Autonomous System (AS) is the unit of independent routing policy: a collection of IP prefixes under a single administrative domain that presents one coherent routing policy to the outside world. Each AS gets a globally unique ASN (Autonomous System Number), 16-bit originally (0–65535) and now 32-bit, assigned by IANA through the Regional Internet Registries.

Why does this abstraction exist? Because global routing cannot scale to individual routers. If every router had to know the path to every other router, the routing table would be unmanageably large and unstable. Instead the problem is split in two:

• Intra-AS routing (interior gateway protocols: OSPF, IS-IS) — inside one AS, where the operator controls everything and can optimise for shortest path or lowest delay.
• Inter-AS routing (BGP) — between ASes, where the goal is not the shortest path but the policy-compliant path. BGP exchanges reachability as AS paths, not as internal details, so each AS keeps its interior private.

Peering is where ASes physically and logically connect. Two forms:

• Private peering — a dedicated link (often a cross-connect in a colocation facility) between exactly two networks. Predictable capacity, used for large traffic volumes.
• Public peering at an IXP — an Internet Exchange Point operates a shared switching fabric; each member connects once and can then peer with many others over that single port. Far cheaper per peer. Major IXPs (AMS-IX, DE-CIX, LINX) carry multiple terabits per second.

Peering policies are usually published as one of: open (will peer with anyone), selective (criteria such as traffic ratio, geographic presence, minimum volume), or restrictive (effectively closed — typical of Tier 1 networks protecting transit revenue).

Examples

Public peering at an IXP: one port, many peers

    AS 64500 ---+
                |
    AS 64501 ---+---[ IXP switching fabric ]---+--- AS 64510
                |                              |
    AS 64502 ---+                              +--- AS 64511

 Without the IXP: each pair needs its own cross-connect (N*(N-1)/2 links).
 With the IXP:    each member needs exactly 1 link (N links).
 For 6 members: 15 links becomes 6 links.

How BGP describes a route — an AS path, not a hop list

 Prefix 93.184.216.0/24  reachable via  AS_PATH: 64500 1299 15133
                                                  |     |     |
                                                  |     |     +-- origin AS
                                                  |     +-------- a transit provider
                                                  +-------------- your ISP

 Shorter AS_PATH is usually preferred, but LOCAL_PREF (policy)
 outranks path length in BGP's decision process.

Key takeaways

• An AS is one administrative domain with one routing policy, identified by a globally unique ASN.
• Routing is split into interior (optimise cost inside your own network) and exterior (obey policy between networks).
• IXPs reduce the cost of interconnection from O(N²) private links to O(N) ports.
• BGP advertises AS-level paths, hiding interior structure and keeping the global routing table AS-granular rather than router-granular.

🧪 Practice

1. Find the ASN of three organisations you use daily. Which of them operate their own AS rather than relying on a provider's?
2. Calculate the number of cross-connects needed for 40 networks to fully interconnect privately, versus the number of IXP ports for the same result.
3. Interview-style: Why does BGP prefer policy over shortest path, and what would break if it simply always picked the shortest AS path? (Hint: consider a network being forced to carry free traffic between two of its paying competitors.)

<a id="13-network-topologies"></a>

1.3 Network Topologies

A topology is the shape of a network — which nodes connect to which. Shape determines cost, fault tolerance, and how traffic contends, so it is a design decision with permanent consequences.

Bus and Ring Topologies

Theory

A bus topology attaches every node to one shared cable. Any transmission propagates along the whole cable and reaches everyone; each node inspects the destination address and ignores what is not addressed to it. This was original Ethernet (10BASE5, 10BASE2), where a thick coaxial cable ran through the building and machines tapped into it.

The bus is maximally cheap in cable: N nodes need one cable. But it has three fatal properties. First, only one node can transmit at a time — the medium is shared, so total capacity is divided among all users and collisions must be detected and resolved (CSMA/CD). Second, a single break in the cable splits the network in two and typically kills it entirely. Third, troubleshooting is miserable: a fault anywhere affects everyone, with no way to isolate a segment.

A ring topology connects each node to exactly two neighbours, forming a closed loop. Data travels in one direction, node to node, each regenerating the signal as it passes. Access is usually mediated by a token: a special frame circulates, and only the holder may transmit. This eliminates collisions entirely and makes worst-case delay deterministic — important for industrial and real-time systems. Token Ring (IEEE 802.5) and FDDI used this.

The ring's weaknesses mirror the bus. A single node or link failure breaks the loop. Latency grows with node count because every hop adds a regeneration delay. Dual-ring designs (FDDI) add a second counter-rotating ring so the network can "wrap" around a break — the ancestor of today's resilient metro fibre rings.

	Bus	Ring
Cable cost	Lowest	Low (N links)
Media access	Contention (CSMA/CD)	Token, collision-free
Delay determinism	Poor, probabilistic	Good, bounded
Single failure	Kills the network	Kills the ring (unless dual)
Scalability	Poor, degrades with load	Degrades with node count
Modern use	Obsolete in LANs; survives in CAN bus, I2C	Metro fibre rings, SONET/SDH, industrial

Examples

BUS                                  RING
                                          .--[B]--.
[A]--+--[B]--+--[C]--+--[D]              /         \
     |       |       |                 [A]         [C]
  ===========================           \         /
  one shared coaxial cable               '--[D]--'
                                      token circulates clockwise;
  A transmits -> B, C, D all hear it   only the holder may transmit
  B transmits at the same time
       -> COLLISION, both back off

Why a dual ring survives a cut

  Normal:            After a cut between B and C:
   A -> B -> C -> D    A -> B  ][  C -> D
   ^              |    ^    |      ^    |
   '--------------'    '----+------+----'
                       traffic "wraps" onto the
                       counter-rotating secondary ring

Key takeaways

• A bus shares one medium: cheap in cable, but capacity is divided, collisions occur, and one cable fault kills everything.
• A ring passes data node to node, usually with a token, giving collision-free access and bounded worst-case delay.
• Both have a single point of failure; dual counter-rotating rings fix this by wrapping around a break.
• Neither survives in modern LANs, but ring resilience lives on in metro and carrier fibre.

🧪 Practice

1. In a 10 Mb/s bus with 20 equally active nodes, what is the average throughput per node ignoring collision overhead? How does the real figure compare, and why?
2. A token ring has 12 nodes, each adding 1 bit-time of latency, on a 4 Mb/s link. Roughly how long does the token take to circulate once if the ring's total propagation delay is 20 µs?
3. Interview-style: Modern Ethernet is logically descended from a bus but no longer suffers collisions. What changed? (Hint: think about what a switch does to the collision domain, and about full-duplex.)

Star and Extended Star

Theory

In a star, every node connects by its own dedicated link to a central device. Nothing connects directly to anything else. This is how essentially every modern wired LAN is built: hosts plug into a switch.

The star fixes the bus's problems by construction:

• Fault isolation — a cut cable affects exactly one node. The rest of the network does not notice.
• Dedicated bandwidth — with a switch (rather than a hub) at the centre, each link is its own collision domain running full-duplex. A 1 Gb/s port gives that host 1 Gb/s in each direction simultaneously, not a share of a common pool.
• Simple diagnosis — every link terminates in one place, so link status per port is directly observable.
• Easy growth — adding a node is plugging in a cable, with no disruption.

The cost is more cable (N links for N nodes, all radiating from one point) and a single point of failure at the centre. If the switch dies, everything attached dies with it. This is mitigated with redundant switches, stacking, or dual-homed hosts.

An extended star (hierarchical star, star-of-stars) scales the idea: central devices connect to a higher-level central device. This is exactly the campus three-tier design — access switches form stars for users, and those switches are themselves points in a star centred on a distribution switch. It keeps the star's fault isolation while avoiding the impossible cable runs a single flat star would need across a building.

Examples

STAR                              EXTENDED STAR

   [A]   [B]                               [Core switch]
     \   /                                  /         \
      \ /                          [Dist switch]   [Dist switch]
   [switch]                          /      \          /     \
      / \                        [Acc]    [Acc]    [Acc]   [Acc]
     /   \                        /|\      /|\      /|\     /|\
   [C]   [D]                    users    users    users   users

 Each link is private:          Fault isolation at every level;
 A's cable break affects        one access switch failing affects
 only A.                        only its own users.

# On a switch, per-port state makes fault isolation observable.
# (Cisco-style CLI)
show interfaces status
# Port    Name    Status       Vlan   Duplex  Speed
# Gi1/0/1 alice   connected    10     a-full  a-1000
# Gi1/0/2 bob     notconnect   10       auto    auto   <- only bob's cable is down
# Gi1/0/3 carol   connected    20     a-full  a-1000

Key takeaways

• A star gives every node a dedicated link to a central device — the dominant modern LAN topology.
• Fault isolation, per-port full-duplex bandwidth, and trivial diagnosis are the reasons it won.
• The centre is a single point of failure; redundancy must be added deliberately.
• Extended (hierarchical) stars scale the pattern across buildings while preserving isolation at each level.

🧪 Practice

1. Compare total cable length for 24 users in a bus versus a star, assuming users are spread along a 100 m corridor and the switch sits at the midpoint.
2. Design an extended star for three floors of 48 users each. How many access switches, how many uplinks, and where would you add redundancy?
3. Interview-style: A star's centre is a single point of failure. Name two concrete mitigations and the trade-off each introduces. (Hint: consider redundant uplinks and what protocol must then prevent loops.)

Mesh and Partial Mesh

Theory

In a full mesh, every node has a direct link to every other node. It is the maximally redundant topology: any node can reach any other in one hop, and it takes N−1 simultaneous link failures to isolate a node.

The cost is combinatorial. The number of links is:

L = N * (N - 1) / 2

and each node needs N−1 interfaces. For 5 nodes that is 10 links; for 50 nodes it is 1,225 links and 49 ports per device. This is why full mesh is essentially never used physically beyond a handful of nodes.

A partial mesh connects only the pairs that matter — typically every node has two or three links, chosen so that at least two disjoint paths exist between any pair. This buys most of the resilience for a small fraction of the cost, and is how the Internet core and most WAN backbones are actually built.

The distinction between physical and logical mesh matters here. VPN overlays and MPLS often create a full logical mesh (every site can reach every site directly at the tunnel level) on top of a sparse physical partial mesh. The classic scaling problem this creates — N² tunnels to configure — is what hub-and-spoke and dynamic multipoint VPN designs exist to solve.

Nodes (N)	Full mesh links	Ports per node
4	6	3
5	10	4
10	45	9
20	190	19
50	1,225	49

Examples

FULL MESH (N=5, 10 links)        PARTIAL MESH (N=5, 6 links)

      [A]-------[B]                   [A]-------[B]
       |\       /|                     |         |
       | \     / |                     |         |
       |  \   /  |                     |         |
       |   [E]   |                    [E]       [C]
       |  /   \  |                      \       /
       | /     \ |                       \     /
      [D]-------[C]                       '-[D]-'
   any pair: 1 hop               A to C: 2 disjoint paths exist
   fails only if isolated        (A-B-C and A-E-D-C)

def mesh_links(n: int) -> int:
    """Links required for a full mesh of n nodes."""
    return n * (n - 1) // 2

def redundancy_cost(n: int, partial_degree: int = 3) -> None:
    full = mesh_links(n)
    # Partial mesh: each node holds `partial_degree` links,
    # each link shared by 2 nodes -> divide by 2.
    partial = n * partial_degree // 2
    print(f"N={n}: full={full} links, partial(deg {partial_degree})={partial} links")
    print(f"       saving: {100 * (1 - partial / full):.1f}%")

redundancy_cost(50)
# N=50: full=1225 links, partial(deg 3)=75 links
#        saving: 93.9%

Key takeaways

• Full mesh gives one-hop reachability and maximum redundancy at a cost of N(N−1)/2 links.
• Link count grows quadratically, so full physical mesh is impractical past a handful of nodes.
• Partial mesh with two or three links per node delivers path redundancy at near-linear cost — the real-world choice for WAN cores.
• A logical full mesh (tunnels, MPLS) over a sparse physical partial mesh is common, but reintroduces the N² problem in configuration.

🧪 Practice

1. Compute the link count for a full mesh of 8, 16, and 64 nodes. At what point does the cost become obviously prohibitive?
2. Design a partial mesh for 6 branch offices such that any single link failure leaves everything reachable. What is the minimum number of links?
3. Interview-style: A company has 30 sites and wants any-to-any VPN connectivity. Compare full-mesh VPN with hub-and-spoke, in tunnel count and in latency. (Hint: hub-and-spoke has N−1 tunnels but doubles the path for spoke-to-spoke traffic.)

Tree and Hybrid Topologies

Theory

A tree (hierarchical) topology arranges nodes in parent-child levels with exactly one path between any two nodes. It is the natural shape for aggregation: many cheap low-speed ports at the leaves, fewer expensive high-speed links as you move up.

Its defining property is that it has no loops. Exactly one path exists between any pair. This is a virtue for Ethernet, where frames have no TTL and a loop would let broadcast traffic circulate forever, amplifying until the network collapses — a broadcast storm. Spanning Tree Protocol exists precisely to reduce any physical Ethernet topology to a logical tree.

But it is also the tree's weakness: no alternative path means no redundancy, and all traffic between subtrees must cross the root, making upper links a bottleneck. Real designs therefore add redundant physical links (making the physical topology a partial mesh) and let STP, or better, link aggregation and routed designs, manage them.

A hybrid topology combines forms deliberately — which is what every real network is. A campus is a tree of stars. A data centre is a Clos fabric (a structured partial mesh) whose leaves are stars of servers. A carrier network is a ring of partial meshes. Purity is an exam concept; hybrids are practice.

Oversubscription is the key tree metric: the ratio of aggregate downstream capacity to upstream capacity. An access switch with 48 × 1 Gb/s ports and one 10 Gb/s uplink is 48:10, or 4.8:1 oversubscribed. That is fine for office users who are idle most of the time, and disastrous for backup traffic that saturates every port at once.

Examples

TREE with oversubscription annotated

                 [Core]
              2 x 40 Gb/s up = 80 Gb/s
                /        \
        [Dist A]          [Dist B]
   4 x 10 Gb/s down        ...
   = 40 Gb/s               |
      /   |   \
 [Acc] [Acc] [Acc]
  48 x 1 Gb/s each = 144 Gb/s of edge ports
  into 40 Gb/s of uplink  ->  3.6 : 1 oversubscribed

HYBRID: a tree of stars, with a redundant ring at the core
   [Core1]===[Core2]
      |  \   /  |        <- redundant core (mesh)
      |   \ /   |
   [Dist]  X  [Dist]     <- dual-homed distribution
      |         |
   [Acc]     [Acc]       <- stars of users

Key takeaways

• A tree has exactly one path between any two nodes: no loops, natural aggregation, but no redundancy.
• Ethernet needs loop-freedom because frames have no TTL; STP enforces a logical tree over a looped physical topology.
• Oversubscription ratio quantifies the bottleneck at each tier and must match the real traffic profile.
• Every production network is a hybrid; the named topologies are building blocks, not choices.

🧪 Practice

1. Calculate the oversubscription ratio for an access switch with 24 × 1 Gb/s ports and two 10 Gb/s uplinks in a LAG. Is that acceptable for office users? For a backup network?
2. Explain why an Ethernet loop causes a broadcast storm but an IP routing loop does not cause an equivalent meltdown. Name the specific header field responsible.
3. Sketch a hybrid topology for a hospital with an emergency department that cannot tolerate any single failure. Mark every redundant element.

Physical vs Logical Topology

Theory

The physical topology is how the cables actually run. The logical topology is how data actually flows. They are frequently different, and confusing them is a classic source of misdiagnosis.

The canonical example is 10BASE-T Ethernet with a hub. Physically it is a star: every machine has its own cable to a central box. Logically it is a bus: the hub repeats every incoming signal out of every other port, so all machines share one collision domain and contend exactly as they would on a coax cable. Replace the hub with a switch and the physical topology is unchanged, but the logical topology becomes a set of independent point-to-point links — collisions vanish.

Other important mismatches:

• Token Ring was physically a star (all cables to a central MAU) but logically a ring — the MAU wired the ports into a loop internally.
• VLANs create several logical LANs on one physical switch. Two machines in the same rack, in the same switch, on different VLANs cannot reach each other without a router. Physically adjacent, logically distant.
• Wireless is physically a star (all clients to an access point) but logically a shared bus — everyone contends for the same radio channel.
• VPNs and overlays (VXLAN, GRE, WireGuard) build a logical topology of tunnels across an arbitrary physical Internet path. Two hosts one logical hop apart may be 15 physical hops apart.

Technology	Physical topology	Logical topology
10BASE-T with hub	Star	Bus (one collision domain)
10BASE-T with switch	Star	Point-to-point per port
Token Ring with MAU	Star	Ring
Wi-Fi	Star (to the AP)	Bus (shared channel)
VLANs on one switch	Star	Multiple isolated broadcast domains
VXLAN overlay	Whatever the underlay is	Flat L2 across data centres

The practical rule: cable faults are physical; reachability faults are logical. When a link light is on but traffic does not flow, stop looking at cables and start looking at VLANs, routing, and firewall policy.

Examples

IDENTICAL physical star, OPPOSITE logical behaviour

  With a HUB (logical bus):        With a SWITCH (logical p2p):
     [A]  [B]                         [A]  [B]
       \  /                             \  /
      [HUB]                          [SWITCH]
       /  \                             /  \
     [C]  [D]                         [C]  [D]

  A sends to B:                    A sends to B:
   -> signal repeated to B,C,D      -> frame forwarded ONLY to B
   -> C and D must ignore it        -> C and D see nothing
   -> C transmitting = collision    -> C and D can transmit
   -> 1 collision domain               simultaneously
   -> 100 Mb/s shared by all        -> 4 collision domains
                                    -> 100 Mb/s per port, full-duplex

# Physically adjacent, logically separated: same switch, different VLANs.
show vlan brief
# VLAN Name      Status  Ports
# 10   staff     active  Gi1/0/1, Gi1/0/2
# 20   guest     active  Gi1/0/3, Gi1/0/4
# Gi1/0/1 and Gi1/0/3 are 2 cm apart in the same chassis,
# but a packet between them must leave the switch and be routed.

Key takeaways

• Physical topology is cabling; logical topology is data flow. They routinely differ.
• A hub makes a physical star behave as a logical bus; a switch makes it behave as point-to-point links.
• VLANs, wireless, and overlays are all cases where logical distance has nothing to do with physical distance.
• Diagnose accordingly: link lights and cable testers address physical faults; VLAN, routing, and ACL inspection address logical ones.

🧪 Practice

1. For each of hub, switch, wireless access point, and router, state the physical and logical topology of a four-host network built around it.
2. Two servers in the same rack cannot ping each other, but both reach the Internet. List three logical causes and the command you would run to check each.
3. Interview-style: A VXLAN overlay presents two hosts in different cities as being on the same subnet. What does this hide from the application, and what can go wrong because of it? (Hint: consider assumptions the application makes about latency and broadcast behaviour on a "local" subnet.)

<a id="14-switching-paradigms"></a>

1.4 Switching Paradigms

Once several nodes share an infrastructure, a fundamental question arises: how is capacity allocated to a conversation? The answers — reserve it in advance, or compete for it moment by moment — define two different worlds.

Circuit Switching

Theory

Circuit switching reserves an end-to-end path, with dedicated capacity, for the entire duration of a conversation. It is the model of the traditional telephone network.

Three phases:

1. Setup — a signalling exchange walks the path from source to destination, and each switch along the way reserves resources for this call. If any switch has no free capacity, the call is blocked and fails immediately.
2. Data transfer — the reserved capacity is exclusively the caller's. It is available whether used or not. Delay is constant and low (essentially just propagation), and there is no queueing, no jitter, and no loss.
3. Teardown — resources are explicitly released.

Capacity is divided among circuits by FDM (each circuit gets its own frequency band) or TDM (each circuit gets a fixed time slot in a repeating frame).

The guarantee is the strength: once established, quality is predictable, which is why circuit switching was correct for voice, where a steady 64 kb/s stream must arrive on time and a busy signal is preferable to a degraded call.

The waste is the weakness. Computer traffic is bursty: a terminal session or a web browse is idle most of the time, punctuated by bursts. A reserved circuit sitting idle is capacity nobody else can use. For a source that is active 10 percent of the time, circuit switching wastes 90 percent of what it reserved. It also fails badly: if a switch on the reserved path dies, the call drops — there is no rerouting mid-conversation.

Examples

Circuit switching: capacity reserved along the whole path

 [A] ===== [S1] ===== [S2] ===== [S3] ===== [B]
      slot 2     slot 2     slot 2
      RESERVED for A-B, whether or not A is speaking

 TDM frame on each link (4 slots per frame):
  | 1 | 2 | 3 | 4 | 1 | 2 | 3 | 4 | 1 | 2 | 3 | 4 |
        ^           ^           ^
        A-B's slot recurs every frame, guaranteed.
        If A is silent, the slot is transmitted empty and wasted.

# How many concurrent circuits fit on a link, and what it costs when idle.
link_capacity = 1_000_000        # 1 Mb/s
circuit_rate  = 100_000          # each circuit reserves 100 kb/s

max_circuits = link_capacity // circuit_rate
print(max_circuits)              # 10 -> the 11th call is BLOCKED

# Now the efficiency problem: each user is active only 10% of the time.
duty_cycle = 0.10
useful = max_circuits * circuit_rate * duty_cycle
print(f"utilised: {useful / link_capacity:.0%}")   # utilised: 10%
# 90% of a fully "full" link carries nothing at all.

Key takeaways

• Circuit switching reserves end-to-end capacity for the whole call: setup, transfer, teardown.
• It guarantees constant rate, constant delay, no jitter, and no loss once established.
• A call is either admitted with full quality or blocked outright — there is no graceful degradation.
• It is badly matched to bursty data traffic, where reserved-but-idle capacity is pure waste.

🧪 Practice

1. A 10 Mb/s link uses TDM with 64 kb/s circuits. How many simultaneous calls can it carry? What happens to call 157?
2. A user sends data 5 percent of the time. Compute the fraction of a reserved circuit that carries useful data, and state how many such users could share the same capacity if bursts never overlapped.
3. Interview-style: Why did the telephone network choose circuit switching, and why would you not build a video-conferencing service that way today? (Hint: think about what "blocked" means to a caller versus what "degraded" means to a viewer.)

Packet Switching

Theory

Packet switching takes the opposite approach: reserve nothing. Data is split into packets, each carrying a header with the destination address. Each packet is sent independently and competes for link capacity with every other packet in the network, on demand.

The enabling idea is statistical multiplexing. Because bursty sources are rarely all active at once, a link can be shared by far more users than its capacity would support under reservation. If 35 users each need 100 kb/s while active but are active only 10 percent of the time, a 1 Mb/s link serves them all with an extremely low probability of more than 10 being active simultaneously. Circuit switching could have served only 10.

The price is that there are no guarantees. When more traffic arrives at an output link than it can send, the excess is queued, adding variable delay. If the queue fills, packets are dropped. Delay varies (jitter) and loss happens — the network offers best effort, and reliability, if the application needs it, is built at the endpoints.

The benefits beyond efficiency are substantial:

• Resilience — packets are routed hop by hop, so a failed link can be routed around mid-conversation without the conversation knowing.
• No setup cost — the first packet can be sent immediately, which matters enormously for short transactions like a DNS query.
• Simplicity in the core — routers hold no per-conversation state, so they scale and recover from reboots without losing anyone's session.

Property	Circuit switching	Packet switching
Resource allocation	Reserved in advance	On demand, competitive
Setup delay	Yes, before any data	None
Delay characteristics	Constant, predictable	Variable (queueing)
Packet loss	None once established	Possible (buffer overflow)
Efficiency with bursty traffic	Poor	High
Behaviour at overload	New calls blocked	Everyone degrades
Per-conversation state in the core	Yes	No
Failure recovery	Call drops	Reroute around failure

Examples

Statistical multiplexing on one 1 Mb/s link

 Users A..E each burst at 100 kb/s, active 10% of the time.

 time -->
 A  ###.......###...........###......
 B  ...###.......###....###.........
 C  ......###.........###.......###.
 D  ###......###...........###......
 E  .......###....###..........###..
    ^^^      ^^^
    overlap: queue briefly, then drains
    Most of the time the link carries 1-2 bursts, well under capacity.

# Probability that too many users are simultaneously active.
from math import comb

n, p, capacity_users = 35, 0.10, 10   # 35 users, 10% duty, link fits 10 at once

# P(more than 10 of 35 users active at the same instant)
p_overload = sum(comb(n, k) * p**k * (1 - p)**(n - k)
                 for k in range(capacity_users + 1, n + 1))
print(f"P(overload) = {p_overload:.6f}")   # P(overload) = 0.000424
# 35 users share a link that circuit switching would give to only 10,
# and congestion occurs about 0.04% of the time.

Key takeaways

• Packet switching allocates nothing in advance; packets carry their own destination and compete for capacity.
• Statistical multiplexing lets many more bursty users share a link than reservation would allow.
• The cost is variable delay and possible loss — best-effort service, with reliability pushed to the endpoints.
• No per-conversation state in the core gives resilience, instant start-up, and scalability.

🧪 Practice

1. Recompute the overload probability above for 50 users and for a 20 percent duty cycle. At what point does the link stop being viable?
2. List three applications that tolerate loss but not delay, and three that tolerate delay but not loss. Which paradigm suits each?
3. Interview-style: Packet switching has no admission control, so at overload everyone suffers rather than some being refused. Is that better or worse than blocking, and for which applications? (Hint: contrast an emergency phone call with a file download.)

Message Switching

Theory

Message switching is the historical middle ground, and understanding why it lost clarifies why packets are small.

An entire message — a whole telegram, a complete file — is sent as one unit from node to node. Each intermediate node receives the complete message, stores it (on disk if necessary), and forwards it when the next link is free. There is no end-to-end reservation, so links are shared efficiently, but the unit of transfer is unbounded.

This has three disabling consequences:

1. Delay accumulates catastrophically. Each hop must receive the entire message before sending any of it. Total delay is roughly (number of hops) × (message size / link rate). A 100 MB message over 4 hops at 10 Mb/s takes 4 × 80 s = 320 s, even though the first bit could have arrived in 80 s with pipelining.
2. Storage requirements are unbounded. Every intermediate node must buffer a whole message of arbitrary size, so nodes need disks, not just memory.
3. Head-of-line blocking is severe. One enormous message monopolises a link, and every small message queued behind it waits. There is no fairness.

Packet switching is message switching with one crucial addition: bound the unit size. Once messages are chopped into packets of at most a few kilobytes, transmission pipelines — a packet can be forwarded on hop 2 while the next packet is still arriving on hop 1 — and the delay collapses from (hops × message time) to roughly (message time + hops × packet time).

Message switching is not entirely dead. It survives wherever store-and-forward of complete units is the point: email relaying through SMTP servers, message queues (Kafka, RabbitMQ), and delay-tolerant networking for deep space, where end-to-end paths never exist simultaneously.

Examples

Delay comparison: one 3 Mb message over 3 hops at 1 Mb/s
(whole message takes 3 s to transmit on one link)

MESSAGE SWITCHING (no pipelining: each node waits for all 3 Mb)
 hop1 [===3s===]
 hop2           [===3s===]
 hop3                     [===3s===]
 total = 3 x 3 s = 9 s

PACKET SWITCHING (3 packets of 1 Mb, 1 s each, pipelined)
 hop1 [p1][p2][p3]
 hop2     [p1][p2][p3]
 hop3         [p1][p2][p3]
 total = 3 x 1 s + 2 x 1 s = 5 s

 More, smaller packets -> closer to the 3 s lower bound.

def message_switch_delay(msg_bits, rate, hops):
    """Whole message must land at each node before moving on."""
    return hops * (msg_bits / rate)

def packet_switch_delay(msg_bits, rate, hops, packet_bits):
    """Pipelined: first packet walks all hops, the rest follow one behind."""
    n_packets = msg_bits / packet_bits
    t_packet = packet_bits / rate
    return hops * t_packet + (n_packets - 1) * t_packet

M, R, H = 24_000_000, 1_000_000, 3          # 24 Mb, 1 Mb/s, 3 hops
print(message_switch_delay(M, R, H))                    # 72.0 s
print(packet_switch_delay(M, R, H, 8_000_000))          # 40.0 s  (3 packets)
print(packet_switch_delay(M, R, H, 80_000))             # 24.16 s (300 packets)
# Smaller packets -> better pipelining -> total approaches the 24 s floor.

Key takeaways

• Message switching forwards entire messages hop by hop with no reservation and no size limit.
• Delay is multiplied by hop count because no pipelining is possible, and nodes need unbounded storage.
• Packet switching is message switching with a bounded unit size, which enables pipelining and fairness.
• The store-and-forward-whole-messages model survives in email, message queues, and delay-tolerant networking.

🧪 Practice

1. Compute end-to-end delay for a 10 MB message over 5 hops at 100 Mb/s, under message switching and under packet switching with 1500-byte packets.
2. Using the code above, plot delay against packet size. Why does the curve flatten, and what real-world cost stops you choosing a 1-byte packet?
3. Interview-style: SMTP is store-and-forward at the application layer but runs over packet-switched TCP/IP. Why both? (Hint: consider what happens when the destination mail server is offline for an hour.)

Datagram vs Virtual Circuit Service

Theory

Within packet switching, there are still two ways to organise forwarding.

In datagram service, each packet is entirely self-contained and independent. It carries the full destination address, and every router makes a fresh forwarding decision for it based only on that address and the current routing table. Packets of the same conversation may take different paths, arrive out of order, or be duplicated. There is no connection setup and no per-flow state in routers. IP works this way.

In virtual circuit (VC) service, a connection is established before data flows. During setup, a path is chosen and each router along it installs an entry in a table mapping an incoming VC identifier to an outgoing interface and a new VC identifier. Packets then carry only that short label, not a full address. All packets of a conversation follow the same path and arrive in order. Routers hold per-connection state. X.25, Frame Relay, and ATM worked this way; MPLS is the modern descendant.

The trade-off is the recurring one in networking: state versus resilience.

• Per-flow state enables order guarantees, resource reservation, quality-of-service commitments, and small headers.
• No state means a router can crash and reboot without destroying anyone's connection, and traffic reroutes instantly around a failure.

The Internet chose datagrams, and the choice was deliberate: the design goal was survivability of communication in the face of node loss. A stateless core cannot lose state it does not have.

	Datagram	Virtual Circuit
Setup phase	None	Required before data
Router state per flow	None	One table entry per VC
Addressing in packets	Full destination address	Short VC label
Path	May vary per packet	Fixed for the connection
Ordering	Not guaranteed	Guaranteed
Router failure	Reroute; flows survive	All VCs through it are destroyed
QoS reservation	Hard	Natural
Examples	IP, UDP over IP	ATM, Frame Relay, X.25, MPLS

Note carefully: TCP provides a connection but does not make IP a virtual circuit. TCP's state lives only in the two end hosts; routers in between remain oblivious. This is the end-to-end principle in action.

Examples

DATAGRAM: every packet decides for itself

  pkt[dst=B]--> [R1] --route lookup--> [R2] --> [B]
  pkt[dst=B]--> [R1] --route lookup--> [R3] --> [B]   (different path, fine)
  Router table: destination prefix -> next hop   (no per-flow entry)

VIRTUAL CIRCUIT: path fixed at setup, labels swapped per hop

  SETUP: A --> R1 --> R2 --> B  installs:
    R1: in(port1, VC=7)  -> out(port3, VC=22)
    R2: in(port1, VC=22) -> out(port2, VC=4)

  DATA:  pkt[VC=7] -> R1 -> pkt[VC=22] -> R2 -> pkt[VC=4] -> B
         ^ tiny label, no address; lookup is an exact match on a small table

# Datagram forwarding: stateless longest-prefix lookup, per packet.
import ipaddress
routing_table = [
    (ipaddress.ip_network("10.0.0.0/8"),     "eth0"),
    (ipaddress.ip_network("10.1.0.0/16"),    "eth1"),   # more specific
    (ipaddress.ip_network("0.0.0.0/0"),      "eth2"),   # default route
]

def forward(dst: str) -> str:
    addr = ipaddress.ip_address(dst)
    # Longest prefix match: the most specific matching route wins.
    best = max((n for n, _ in routing_table if addr in n),
               key=lambda n: n.prefixlen)
    return dict(routing_table)[best]

print(forward("10.1.5.9"))   # eth1  -> matched /16, not /8
print(forward("10.9.0.1"))   # eth0  -> matched /8
print(forward("8.8.8.8"))    # eth2  -> fell through to the default
# Note: no memory of previous packets. Each call is independent.

Key takeaways

• Datagram forwarding is stateless and per-packet; virtual circuits install per-flow state at setup.
• VCs give ordering, small headers, and natural QoS; datagrams give resilience and instant start-up.
• IP is a datagram service by deliberate design, chosen for survivability under node failure.
• TCP's connection state exists only in the endpoints — it does not turn IP into a virtual-circuit network.

🧪 Practice

1. A router carrying 2 million concurrent flows fails and reboots. Describe the impact under datagram forwarding and under virtual circuits.
2. Why are VC labels shorter than addresses, and what does that buy in a high-speed router? Estimate the header saving for a 40-byte packet.
3. Interview-style: MPLS reintroduces virtual circuits into IP networks. What does an operator gain that plain IP routing cannot give them? (Hint: think about traffic engineering and about forcing traffic onto a path that is not the shortest.)

Store-and-Forward Transmission

Theory

Store-and-forward is the rule that a packet switch must receive a packet in its entirety before it begins transmitting it onward. Every router in the Internet works this way by default.

Why? Two reasons. First, the checksum or CRC is at the end of the frame — a switch cannot know whether a packet is corrupt until the last bit arrives, and forwarding corrupt packets wastes downstream capacity. Second, output links may be busy or of different speeds, so a packet arriving at 10 Gb/s destined for a 1 Gb/s port must be buffered regardless.

The cost is one full transmission delay per hop. Transmission delay is the time to push all bits of a packet onto a link:

d_trans = L / R        L = packet length in bits, R = link rate in bits/s

For a path with N links carrying one packet, ignoring propagation, queueing, and processing:

total = N * (L / R)

For a file of P packets sent back to back over N links, pipelining means only the first packet pays the full N-hop cost:

total = N * (L / R) + (P - 1) * (L / R)

Some switches implement cut-through forwarding: start transmitting as soon as the destination address has been read (typically the first 6 bytes for Ethernet), without waiting for the rest. This drops per-hop latency to microseconds and is used in high-frequency trading and low-latency data centre fabrics. The trade-off is that corrupt frames are propagated rather than discarded, and it only works when input and output rates match. Fragment-free is a middle setting that waits for the first 64 bytes, which catches the collision fragments that account for most corruption.

Mode	Waits for	Latency	Error handling
Store-and-forward	Entire frame	Highest (L/R per hop)	Drops corrupt frames
Fragment-free	First 64 bytes	Medium	Catches collision fragments
Cut-through	First 6–14 bytes	Lowest	Forwards corrupt frames

Examples

Store-and-forward: one packet, three links

 A ---L1--- R1 ---L2--- R2 ---L3--- B

 t=0    A starts sending          [========]
 t=L/R  A done; R1 has it all              [========]
 t=2L/R R1 done; R2 has it all                      [========]
 t=3L/R B has it all
 Total = 3 * (L/R)

Pipelined, three packets:
 A  [p1][p2][p3]
 R1     [p1][p2][p3]
 R2         [p1][p2][p3]
 Total = 3*(L/R) + 2*(L/R) = 5*(L/R), not 9*(L/R)

def store_and_forward(packets: int, packet_bits: int, rate_bps: int,
                      hops: int) -> float:
    """End-to-end time ignoring propagation, queueing and processing."""
    t = packet_bits / rate_bps               # transmission delay for one packet
    # First packet traverses every hop; each further packet adds one t (pipelining).
    return hops * t + (packets - 1) * t

# 1 MB file, 1500-byte packets, 100 Mb/s links, 4 hops
L = 1500 * 8                                  # 12,000 bits per packet
P = (1024 * 1024) // 1500 + 1                 # 700 packets
print(f"{store_and_forward(P, L, 100_000_000, 4):.4f} s")   # 0.0844 s

# Same file, 1 packet per hop would be:
print(f"{store_and_forward(1, 1024*1024*8, 100_000_000, 4):.4f} s")  # 0.3355 s
# 4x worse -> this is exactly why packets are bounded in size.

Key takeaways

• Store-and-forward means each switch receives the whole packet before forwarding, costing L/R per hop.
• The reason is error checking (the CRC is at the end) and rate mismatch between ports.
• Pipelining makes the per-hop cost paid once rather than per packet, which is the core argument for small packets.
• Cut-through trades error detection for latency and is used in low-latency fabrics where rates match.

🧪 Practice

1. Compute the total delay for one 1500-byte packet across 5 links at 1 Gb/s, ignoring propagation. Then add 10 ms propagation per link and recompute.
2. Using the function above, find the file-transfer time for 10 MB over 6 hops at 1 Gb/s with 1500-byte versus 9000-byte (jumbo) packets. Explain the difference.
3. Interview-style: A trading firm insists on cut-through switching. What are they buying, what are they giving up, and when does cut-through silently fail to help? (Hint: consider what happens when the ingress port is 10 Gb/s and the egress port is 1 Gb/s.)

<a id="15-performance-metrics"></a>

1.5 Performance Metrics

"The network is slow" is not a diagnosis. This subchapter provides the vocabulary and the arithmetic to say precisely which quantity is bad, why, and whether it can be improved at all.

Bandwidth and Throughput

Theory

These two words are used interchangeably in conversation and mean different things in engineering.

Bandwidth (more precisely capacity) is the maximum rate a link can carry, measured in bits per second. It is a property of the link: a 1 Gb/s Ethernet port has 1 Gb/s of capacity whether or not anything is using it. It is a ceiling, not a promise.

Throughput is the rate actually achieved by a transfer, measured over some interval. It is always less than or equal to bandwidth and depends on the whole path, the protocol, the endpoints, and competing traffic.

The governing rule is the bottleneck link: end-to-end throughput is bounded by the slowest link on the path. A path of 1 Gb/s, 1 Gb/s, 10 Mb/s, 1 Gb/s delivers at most 10 Mb/s. Upgrading any other link changes nothing — a point worth remembering before approving an expensive upgrade.

When R links are shared, throughput is further divided. If a bottleneck link of capacity R carries N simultaneous flows sharing fairly, each gets roughly R/N.

Two units traps cause endless confusion:

• Bits versus bytes. Network rates are in bits per second (lowercase b: Mb/s, Gb/s); file sizes and download managers use bytes (uppercase B: MB/s). A 100 Mb/s link downloads at most about 12.5 MB/s. Dividing by 8 resolves most "my connection is slower than advertised" complaints.
• Decimal versus binary prefixes. Network vendors use decimal: 1 Mb/s = 1,000,000 b/s. Memory uses binary: 1 MiB = 1,048,576 bytes. Mixing them introduces a 7 percent error at the mega scale and 10 percent at giga.

Examples

The bottleneck determines everything

 [client]--1 Gb/s--[R1]--1 Gb/s--[R2]--10 Mb/s--[R3]--1 Gb/s--[server]
                                     ^^^^^^^^^
                               end-to-end throughput <= 10 Mb/s

 Upgrading the 1 Gb/s links to 100 Gb/s: no change whatsoever.
 Upgrading the 10 Mb/s link to 100 Mb/s: 10x improvement.

def effective_throughput(link_rates_bps, concurrent_flows=1):
    """End-to-end rate is set by the slowest hop, shared among flows."""
    bottleneck = min(link_rates_bps)
    return bottleneck / concurrent_flows

path = [1_000_000_000, 1_000_000_000, 10_000_000, 1_000_000_000]
bps = effective_throughput(path)
print(f"{bps/1e6:.0f} Mb/s = {bps/8/1e6:.2f} MB/s")   # 10 Mb/s = 1.25 MB/s

# 40 users sharing the same bottleneck:
bps = effective_throughput(path, concurrent_flows=40)
print(f"{bps/1e3:.0f} kb/s each")                      # 250 kb/s each

# Download time for a 700 MB file at the full bottleneck rate:
print(f"{700 * 8e6 / 10_000_000 / 60:.1f} minutes")    # 9.3 minutes

Key takeaways

• Bandwidth is link capacity (a ceiling); throughput is the rate actually achieved.
• End-to-end throughput is capped by the slowest link on the path — upgrading anything else is wasted money.
• Shared bottlenecks divide among flows, roughly R/N under fair sharing.
• Divide rates in bits by 8 to get bytes; network prefixes are decimal, memory prefixes are binary.

🧪 Practice

1. A path has links of 100 Mb/s, 1 Gb/s, 50 Mb/s, and 1 Gb/s. What is the maximum throughput? Which single upgrade helps, and what is the new ceiling after it?
2. Your ISP sells "300 Mb/s" and your download manager shows 35 MB/s. Is the ISP delivering what it promised? Show the arithmetic.
3. Interview-style: A user reports that a file transfer is slow, yet iperf between the same two hosts reaches full line rate. Where would you look next? (Hint: iperf tests the network; the transfer also involves disks, protocol behaviour, and the application.)

Latency Components: Transmission, Propagation, Queuing, Processing

Theory

Total one-way delay for a packet across a single link is the sum of four distinct components. Distinguishing them is the single most useful diagnostic skill in networking, because each has a different cause and a different fix.

1. Transmission delay (d_trans = L / R) — the time to push all L bits of the packet onto the wire at rate R. This is a serialisation cost and depends on packet size and link speed, not distance. A 1500-byte packet on a 1 Mb/s link takes 12 ms; on 1 Gb/s it takes 12 µs. Fix: a faster link, or smaller packets.

2. Propagation delay (d_prop = d / s) — the time for a bit to travel distance d at signal speed s, roughly 200,000 km/s in fibre (about two-thirds of c) and near c in air. This depends only on distance. London to New York is about 5,600 km of fibre, so roughly 28 ms one way. No technology fixes this. Fix: move the data closer (a CDN), or send less often.

3. Queuing delay (d_queue) — time spent waiting in the output buffer behind other packets. This is the only variable component and the source of jitter. It depends on the traffic intensity La/R, where L is packet size, a is the arrival rate, and R is the link rate. As La/R approaches 1, queueing delay grows without bound; above 1 the queue overflows and packets are dropped. Fix: less load, more capacity, or smarter queue management.

4. Processing delay (d_proc) — time for the router to examine the header, verify the checksum, and look up the next hop. Modern hardware forwarding makes this microseconds, usually negligible. Fix: better hardware, simpler forwarding logic.

d_node = d_proc + d_queue + d_trans + d_prop
d_end_to_end = sum of d_node over every hop

The essential intuition: transmission delay is about size, propagation delay is about distance, queueing is about load. Doubling your bandwidth halves transmission delay and does nothing at all to propagation delay — which is why a 10 Gb/s transatlantic link still has a 56 ms round trip and why "just buy more bandwidth" fails to fix latency-bound applications.

Component	Depends on	Typical LAN	Typical WAN	Fixable by more bandwidth?
Transmission	Packet size / link rate	~12 µs	~12 µs–1 ms	Yes
Propagation	Distance	< 1 µs	5–150 ms	No
Queuing	Load (La/R)	0–ms	0–100s of ms	Partly
Processing	Router hardware	µs	µs	No

Examples

Where the time goes: 1500-byte packet, London to New York, 100 Mb/s

 d_trans = 12000 bits / 100e6     = 0.12 ms   <- 0.4% of the total
 d_prop  = 5600 km / 200000 km/s  = 28.0 ms   <- 99% of the total
 d_queue = (lightly loaded)       = 0.2 ms
 d_proc  = 10 hops x 20 us        = 0.2 ms
                                    --------
 one-way total                    = 28.5 ms

 Upgrade the link to 10 Gb/s: d_trans falls to 0.0012 ms.
 New total: 28.4 ms. A 100x bandwidth increase bought 0.4%.

SPEED_IN_FIBRE = 200_000_000          # metres per second (~2/3 c)

def link_delay(packet_bytes, rate_bps, distance_m,
               queue_s=0.0, proc_s=0.0):
    d_trans = (packet_bytes * 8) / rate_bps      # size / speed
    d_prop  = distance_m / SPEED_IN_FIBRE        # distance / signal speed
    return {
        "transmission_ms": d_trans * 1000,
        "propagation_ms":  d_prop * 1000,
        "queuing_ms":      queue_s * 1000,
        "processing_ms":   proc_s * 1000,
        "total_ms":       (d_trans + d_prop + queue_s + proc_s) * 1000,
    }

# Same packet, same distance, two very different link rates.
for rate in (100_000_000, 10_000_000_000):
    d = link_delay(1500, rate, 5_600_000, queue_s=0.0002, proc_s=0.0002)
    print(f"{rate/1e9:>5.2f} Gb/s -> trans {d['transmission_ms']:.4f} ms, "
          f"total {d['total_ms']:.2f} ms")
#  0.10 Gb/s -> trans 0.1200 ms, total 28.52 ms
# 10.00 Gb/s -> trans 0.0012 ms, total 28.40 ms

Key takeaways

• Four components: transmission (size/rate), propagation (distance), queueing (load), processing (hardware).
• Only propagation delay is irreducible; it is set by physics at roughly 5 µs per km of fibre.
• Queueing is the only variable component, so it is the source of jitter and the target of congestion control.
• More bandwidth reduces transmission delay only; on long paths that is a rounding error.

🧪 Practice

1. Compute all four components for a 64-byte packet on a 1 Gb/s LAN link of 100 m with negligible queueing. Which dominates, and by how much?
2. Repeat for a 1500-byte packet on a 10 Mb/s satellite link with 36,000 km to a geostationary satellite (signal in vacuum, 300,000 km/s). Which dominates now?
3. Interview-style: An application takes 4 seconds to load over a 1 Gb/s link with a 100 ms RTT. The team wants to upgrade to 10 Gb/s. Advise them. (Hint: work out how many sequential round trips 4 seconds implies before considering bandwidth at all.)

Round-Trip Time

Theory

Round-trip time (RTT) is the time for a packet to travel from source to destination and for a response to return. It is the single most consequential number for interactive and request-response applications, because most protocols are conversational: they cannot send the next thing until the previous answer arrives.

RTT is not always twice the one-way delay. Three reasons:

1. Routing is often asymmetric. The path out and the path back may differ in length, capacity, and congestion, because each direction's route is chosen independently by different networks according to their own policy.
2. Queueing differs per direction. The return path may be congested while the forward path is idle — common on asymmetric consumer links where upstream is much thinner than downstream.
3. The remote host takes time to respond. A ping reply is generated in the kernel and is nearly free, but an HTTP response involves application processing that is included in any measurement taken at the application layer.

RTT sets a hard floor on protocol performance because handshakes cost round trips:

Operation	Round trips before the first byte of data
DNS lookup (uncached)	1 (often more, if recursing)
TCP three-way handshake	1
TLS 1.2 handshake	2
TLS 1.3 handshake	1
TLS 1.3 with 0-RTT resumption	0
QUIC first connection	1 (transport and crypto combined)

A fresh HTTPS request with TLS 1.2 therefore costs DNS + TCP + TLS = 4 RTTs before any content moves. At a 200 ms RTT that is 800 ms of pure waiting, independent of bandwidth. This arithmetic is the entire motivation for TLS 1.3, QUIC, connection reuse, and CDNs.

Examples

# ping measures RTT directly. Note min/avg/max/mdev:
ping -c 5 example.com
# 64 bytes from 93.184.216.34: icmp_seq=1 ttl=56 time=88.2 ms
# 64 bytes from 93.184.216.34: icmp_seq=2 ttl=56 time=87.9 ms
# rtt min/avg/max/mdev = 87.9/88.4/89.6/0.61 ms
#                        ^min = the path with empty queues (pure prop + trans)
#                                    ^max - min = queueing variation (jitter)

# curl breaks an HTTPS request into its round-trip-shaped phases:
curl -w "dns:%{time_namelookup}s tcp:%{time_connect}s tls:%{time_appconnect}s \
first_byte:%{time_starttransfer}s total:%{time_total}s\n" \
     -o /dev/null -s https://example.com
# dns:0.021s tcp:0.110s tls:0.298s first_byte:0.401s total:0.402s
#     ^DNS      ^+1 RTT    ^+2 RTT       ^+1 RTT
# Setup consumed 0.298 s; transferring the content took 0.104 s.

def page_load_floor(rtt_ms, tls_version="1.3", dns_cached=False):
    """Minimum time before the first content byte, counting round trips only."""
    trips = 0
    trips += 0 if dns_cached else 1        # DNS resolution
    trips += 1                              # TCP three-way handshake
    trips += 2 if tls_version == "1.2" else 1   # TLS negotiation
    trips += 1                              # HTTP request and response
    return trips * rtt_ms

print(page_load_floor(200, "1.2"))              # 1000 ms -> 5 RTTs
print(page_load_floor(200, "1.3"))              #  800 ms -> 4 RTTs
print(page_load_floor(20,  "1.3", True))        #   60 ms -> CDN nearby, warm DNS
# The last line is why CDNs exist: cut the RTT, not the bandwidth.

Key takeaways

• RTT is the round-trip latency and is the limiting factor for conversational protocols.
• It is not necessarily twice the one-way delay: routes are asymmetric and congestion is directional.
• Every handshake costs whole RTTs before any data flows, so setup can dominate short transfers.
• Reducing RTT means reducing distance (CDNs, edge compute) or reducing round trips (TLS 1.3, QUIC, connection reuse).

🧪 Practice

1. Ping five hosts on different continents and tabulate min, avg, and max. Estimate the fibre distance implied by each minimum RTT.
2. Use the curl -w command above against three sites and identify which phase dominates for each.
3. Interview-style: A mobile app makes 12 sequential API calls on launch. On a 4G connection with a 90 ms RTT, launch takes 3 seconds. What single change gives the biggest improvement? (Hint: count the round trips, then consider what batching or parallelism does to that count.)

Jitter and Packet Loss

Theory

Jitter is the variation in packet delay — not how long packets take, but how inconsistently. A stream where every packet takes exactly 100 ms is perfectly usable for a phone call; a stream averaging 50 ms but ranging from 10 ms to 200 ms is not, because the receiver cannot reconstruct a steady output from an unsteady input.

Jitter comes almost entirely from variable queueing delay: packets arriving when a queue is deep wait longer than packets arriving when it is shallow. Secondary causes include route changes mid-stream and, on wireless, retransmission at the link layer.

Real-time applications defeat jitter with a jitter buffer (playout buffer): the receiver deliberately delays playback by some amount, accumulating packets so that late ones still arrive before they are needed. The trade-off is explicit — a bigger buffer absorbs more jitter but adds that much latency to the conversation. Voice systems typically use 20–60 ms, and adapt the size dynamically.

Packet loss is packets that never arrive. Causes, in rough order of frequency:

• Buffer overflow at a congested router — by far the most common, and it is not a malfunction. It is how a best-effort network signals congestion, and it is what TCP interprets as "slow down".
• Bit errors detected by CRC, causing the frame to be discarded — common on wireless, rare on fibre.
• TTL expiry from a routing loop.
• Policy drops at a firewall or rate limiter.

Loss affects applications very differently. TCP retransmits, so loss becomes reduced throughput and increased latency rather than missing data — and because TCP also halves its sending rate on loss, even 1 percent loss can devastate throughput on a high-RTT path. UDP-based real-time media does not retransmit (a late packet is useless anyway) and instead conceals loss, so 1–2 percent is tolerable and only becomes audible above that.

Application	Loss tolerance	Jitter tolerance	Latency tolerance
File transfer	Zero (TCP repairs it)	High	High
Web browsing	Zero (repaired)	High	Moderate (< 1 s)
Voice call	1–3%	Low (< 30 ms)	Low (< 150 ms one way)
Video streaming (buffered)	Zero (repaired)	High (buffer absorbs it)	High (seconds)
Live gaming	1–2%	Very low	Very low (< 50 ms)

Examples

Jitter and the playout buffer

 Sent   (even spacing, every 20 ms):
   |----|----|----|----|----|----|

 Received (queueing varies per packet):
   |--|-------|-|---------|--|----|
     ^ gaps and bunching = jitter

 Played out after a 60 ms buffer:
   |----|----|----|----|----|----|
   Smooth again, at the cost of 60 ms of extra delay.
   A packet arriving > 60 ms late is discarded: effective loss.

def jitter_stats(arrival_times_ms):
    """RFC 3550-style view: spacing variation between consecutive packets."""
    gaps = [b - a for a, b in zip(arrival_times_ms, arrival_times_ms[1:])]
    mean = sum(gaps) / len(gaps)
    # Mean absolute deviation of inter-arrival gaps = practical jitter measure.
    jitter = sum(abs(g - mean) for g in gaps) / len(gaps)
    return {"mean_gap_ms": mean, "jitter_ms": jitter,
            "max_gap_ms": max(gaps), "min_gap_ms": min(gaps)}

steady  = [0, 20, 40, 60, 80, 100]           # perfect 20 ms spacing
jittery = [0, 12, 47, 55, 94, 100]           # same average, erratic

print(jitter_stats(steady))
# {'mean_gap_ms': 20.0, 'jitter_ms': 0.0,  'max_gap_ms': 20, 'min_gap_ms': 20}
print(jitter_stats(jittery))
# {'mean_gap_ms': 20.0, 'jitter_ms': 13.6, 'max_gap_ms': 39, 'min_gap_ms': 6}
# Identical average rate; only the second one sounds broken.

Key takeaways

• Jitter is delay variation, caused mostly by variable queueing; it matters more than average delay for real-time media.
• Jitter buffers convert variation into fixed extra latency — a direct, unavoidable trade.
• Most packet loss is deliberate buffer-overflow drop signalling congestion, not equipment failure.
• TCP converts loss into lost throughput; real-time UDP media conceals it, so the same loss rate has wildly different consequences.

🧪 Practice

1. Run ping -c 50 to a distant host and compute max − min from the output. Is that spread acceptable for a voice call?
2. Given arrivals at 0, 25, 30, 70, 75, 120 ms for a stream sent every 20 ms, compute the jitter and the minimum buffer needed to play smoothly.
3. Interview-style: Users report choppy video calls but file downloads at full speed on the same link. What does this tell you, and what would you measure? (Hint: the two applications are sensitive to different metrics — one to throughput, one to delay variation.)

Bandwidth-Delay Product

Theory

The bandwidth-delay product (BDP) is the amount of data "in flight" on a path at any instant — data that has been sent but not yet acknowledged:

BDP = bandwidth (bits/s) x round-trip time (s)      [bits]

The physical picture is a pipe. Bandwidth is the pipe's diameter; delay is its length. The product is its volume. A high-bandwidth, high-latency path — a "long fat network" or LFN — holds an enormous amount of data in transit.

This matters because of a hard constraint on any sliding-window protocol: a sender that must wait for acknowledgement can have at most one window of unacknowledged data outstanding. Therefore:

max throughput = window size / RTT

If the window is smaller than the BDP, the sender fills the pipe, stops, and waits for an ACK — leaving the link idle for most of each round trip. The link's capacity is irrelevant; the window is the limit.

The consequences are concrete and historically important. TCP's original window field is 16 bits, capping the window at 65,535 bytes. On a 1 Gb/s link with 100 ms RTT, the BDP is 12.5 MB, but without the window scaling option (RFC 7323) TCP can only keep 64 KB in flight — a maximum of about 5.2 Mb/s, or 0.5 percent of the link. This is the classic symptom: a hugely expensive intercontinental link delivering single-digit megabits per connection.

Examples

The pipe analogy

 Low BDP (LAN):   1 Gb/s, 1 ms RTT
   [sender]|##|                     |[receiver]
   BDP = 125 KB    -> a small window fills it easily

 High BDP (LFN):  1 Gb/s, 200 ms RTT
   [sender]|####################### |[receiver]
   BDP = 25 MB     -> a 64 KB window leaves the pipe 99.7% empty

def bdp_bytes(bandwidth_bps: float, rtt_ms: float) -> float:
    """Bytes in flight needed to keep the pipe completely full."""
    return bandwidth_bps * (rtt_ms / 1000) / 8

def throughput_with_window(window_bytes: float, rtt_ms: float) -> float:
    """A window-limited sender can send at most one window per RTT."""
    return window_bytes * 8 / (rtt_ms / 1000)

paths = [("LAN",          1_000_000_000,   1),
         ("National",     1_000_000_000,  30),
         ("Transatlantic",1_000_000_000,  80),
         ("Satellite",      100_000_000, 600)]

for name, bw, rtt in paths:
    need = bdp_bytes(bw, rtt)
    got  = throughput_with_window(65_535, rtt)     # unscaled TCP window
    print(f"{name:14} BDP={need/1e6:8.2f} MB   "
          f"64KB window gives {got/1e6:7.2f} Mb/s of {bw/1e6:.0f} Mb/s")

# LAN            BDP=    0.12 MB   64KB window gives  524.28 Mb/s of 1000 Mb/s
# National       BDP=    3.75 MB   64KB window gives   17.48 Mb/s of 1000 Mb/s
# Transatlantic  BDP=   10.00 MB   64KB window gives    6.55 Mb/s of 1000 Mb/s
# Satellite      BDP=    7.50 MB   64KB window gives    0.87 Mb/s of 100 Mb/s

Key takeaways

• BDP = bandwidth × RTT: the data in flight needed to keep a path fully utilised.
• A window-limited sender achieves at most window / RTT, regardless of link capacity.
• On long fat networks, an undersized window is the usual cause of disappointing throughput.
• TCP window scaling exists specifically to lift the 64 KB ceiling; buffer sizing on high-BDP paths must be deliberate.

🧪 Practice

1. Compute the BDP for a 10 Gb/s link with a 150 ms RTT. What window size in megabytes is required to saturate it?
2. A transfer over a 500 Mb/s link with a 120 ms RTT achieves only 4.4 Mb/s. Work backwards to find the effective window size and name the likely cause.
3. Interview-style: Why does a single TCP connection often fail to saturate a high-BDP link even with window scaling enabled, and what do tools like parallel transfers actually exploit? (Hint: consider how congestion control grows the window after loss, and how long that takes at high RTT.)

Goodput and Protocol Overhead

Theory

Goodput is the rate of useful application data delivered, excluding headers, retransmissions, and protocol chatter. Throughput counts every bit on the wire; goodput counts only the bits the application asked for. Goodput is always lower, and the gap is where performance mysteries hide.

Three things consume the difference:

1. Header overhead. Every layer adds bytes. A TCP segment over IPv4 over Ethernet carries:

Ethernet framing:  preamble 8 + header 14 + FCS 4 + inter-frame gap 12 = 38 bytes
IPv4 header:       20 bytes (no options)
TCP header:        20 bytes (no options; commonly 32 with timestamps)
                   ---------
                   78 bytes of overhead per frame

With a 1500-byte MTU, the payload is 1500 − 20 − 20 = 1460 bytes, transmitted inside 1538 bytes of wire time. Efficiency is 1460 / 1538 = 94.9 percent. Small packets are far worse: a 1-byte payload still costs 78 bytes of overhead — about 1.3 percent efficiency. This is why interactive protocols like SSH are enormously less efficient per byte than bulk transfer, and why jumbo frames (9000-byte MTU) reach roughly 99.1 percent.

2. Retransmissions. Every retransmitted byte occupies the link twice but counts once toward goodput. At loss rate p, roughly a fraction p of transmission is repeated work.

3. Protocol chatter. Handshakes, ACKs, TLS records, DNS queries, HTTP headers. On a short transfer these can exceed the payload: fetching a 100-byte API response may move several kilobytes of TCP, TLS, and HTTP header material.

MTU / payload	Wire bytes per frame	Efficiency
1 byte payload	79	1.3%
536-byte payload (min MSS)	614	87.3%
1460-byte payload (standard MSS)	1538	94.9%
8960-byte payload (jumbo)	9038	99.1%

Examples

ETH_OVERHEAD = 8 + 14 + 4 + 12     # preamble + header + FCS + interframe gap
IP_HDR, TCP_HDR = 20, 20

def goodput(link_bps, payload_bytes, loss_rate=0.0):
    frame_on_wire = ETH_OVERHEAD + IP_HDR + TCP_HDR + payload_bytes
    header_eff = payload_bytes / frame_on_wire
    # Retransmitted bytes use the link but deliver nothing new.
    loss_eff = 1 - loss_rate
    return {
        "efficiency_pct": header_eff * loss_eff * 100,
        "goodput_mbps":   link_bps * header_eff * loss_eff / 1e6,
    }

for payload in (1, 536, 1460, 8960):
    g = goodput(1_000_000_000, payload)
    print(f"payload {payload:>5} B -> {g['efficiency_pct']:5.1f}% "
          f"= {g['goodput_mbps']:6.1f} Mb/s of 1000 Mb/s")
# payload     1 B ->   1.3% =   12.7 Mb/s of 1000 Mb/s
# payload   536 B ->  87.3% =  873.0 Mb/s of 1000 Mb/s
# payload  1460 B ->  94.9% =  949.3 Mb/s of 1000 Mb/s
# payload  8960 B ->  99.1% =  991.4 Mb/s of 1000 Mb/s

# Now add 2% packet loss to the standard case:
print(goodput(1_000_000_000, 1460, loss_rate=0.02))
# {'efficiency_pct': 93.0, 'goodput_mbps': 930.3}

Where a 1538-byte wire frame actually goes

 |<--------------------- 1538 bytes on the wire --------------------->|
 | pre 8 | eth 14 | ip 20 | tcp 20 | payload 1460          | fcs 4 |gap 12|
 |<-- 78 bytes of overhead (5.1%) -->|<-- 94.9% useful -->|

Key takeaways

• Goodput is useful application data per second; throughput counts headers and retransmissions too.
• Fixed per-frame overhead (about 78 bytes for TCP/IPv4/Ethernet) makes small packets dramatically inefficient.
• Standard 1500-byte MTU yields about 94.9 percent efficiency; jumbo frames reach about 99.1 percent.
• Retransmissions and handshake chatter widen the throughput-to-goodput gap, especially for short transfers.

🧪 Practice

1. Compute wire efficiency for a VoIP packet carrying a 20-byte audio payload over UDP (8-byte header) and IPv4 over Ethernet. Why is voice so expensive per useful byte?
2. Using the function above, find goodput for a 10 Gb/s link at 1460-byte payload with 0.5 percent loss. How much capacity does the loss actually cost?
3. Interview-style: Enabling jumbo frames raises efficiency from 94.9 to 99.1 percent. Why is this not enabled everywhere? (Hint: consider what happens when one device on the path has a 1500-byte MTU, and how path MTU discovery can fail.)

---

<a id="2-network-architecture-and-layered-models"></a>