2. Network Architecture and Layered Models
Chapter 1 described what networks are made of and how fast they go; this chapter explains how the enormous complexity of moving bits between arbitrary machines is decomposed into manageable pieces. Layering is the single most important organising idea in networking — it is what lets a web browser written in 2026 talk over a fibre link designed decades later without either knowing the other exists. We build the principle from first principles, examine the two reference models that dominate the field, and end with the institutions that decide what everyone agrees on.
<a id="21-principles-of-layering"></a>
2.1 Principles of Layering
This subchapter develops layering as an engineering technique: what a layer promises, how layers talk to each other, what happens to data as it descends and ascends the stack, and what layering costs in return for the clarity it buys.
Protocols, Services, and Interfaces
Theory
Imagine writing a program that sends a photo to a friend across the world. Without any structure you would have to handle, in one body of code: which voltage represents a 1 on the cable, how to find your friend's machine among billions, what to do when a router discards your data, how to reassemble fragments in order, and what a "photo" even is. Nobody can write that program correctly, and nobody could reuse it when the cable is replaced by radio.
The solution is the same one used for large software everywhere: decompose into layers, where each layer offers a service to the layer above and is implemented using the service of the layer below. Three terms must be kept strictly separate, because most confusion about networking comes from blurring them.
| Term | Question it answers | Direction | Example |
|---|---|---|---|
| Service | What does this layer do for its user? | Vertical (layer N to layer N+1) | "Deliver a byte stream reliably and in order" |
| Protocol | How do two peers at the same layer cooperate to provide it? | Horizontal (layer N here to layer N there) | TCP's sequence numbers, ACKs, retransmission timers |
| Interface | How is the service invoked in this implementation? | Vertical, local | The Berkeley socket API: send(), recv() |
A postal analogy makes the split concrete. The service the post office offers you is "put an address on this envelope and it will arrive". The protocol is what postal workers do among themselves — sorting codes, mail-van schedules, customs forms — which you never see and which they can change freely. The interface is the physical letterbox on your street corner: the local mechanism by which you hand work over.
Two crucial consequences follow:
- A layer's service is a contract; its protocol is an implementation detail. As long as the contract holds, the protocol can be replaced. TCP's congestion control has been rewritten many times (Reno, CUBIC, BBR) without a single application changing.
- Peers communicate logically, not physically. Your TCP and the server's TCP behave as if they exchange segments directly. In reality every segment goes down your stack, across links, and up the other stack. The horizontal arrow is a useful fiction; the vertical path is the truth.
A layer is called transparent when its user cannot tell it is there. IP is nearly transparent to TCP — TCP knows only that packets may be lost, duplicated, or reordered, not which routers did it.
Examples
Peer protocols (horizontal, logical) vs services (vertical, real)
Host A Host B
+---------------+ application protocol (HTTP) +---------------+
| Application |<- - - - - - - - - - - - - - - ->| Application |
+---------------+ +---------------+
| service: "send this byte stream reliably" ^
v (interface: socket API) |
+---------------+ transport protocol (TCP) +---------------+
| Transport |<- - - - - - - - - - - - - - - ->| Transport |
+---------------+ +---------------+
| service: "get this packet to an IP address" ^
v |
+---------------+ network protocol (IP) +---------------+
| Network |<- - - - - - - - - - - - - - - ->| Network |
+---------------+ +---------------+
| ^
v |
======================= physical path ===================
dashed = logical peer conversation (protocol)
solid = real data path through services and interfaces
# The interface is the API; the service is what it promises;
# the protocol is what you never see.
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # SOCK_STREAM selects the SERVICE:
# reliable, ordered, byte-stream
s.connect(("example.com", 80)) # INTERFACE call; TCP's 3-way
# handshake PROTOCOL runs unseen
s.sendall(b"GET / HTTP/1.1\r\nHost: example.com\r\n\r\n")
# We never wrote a sequence number, an ACK, or a retransmission timer.
# TCP's peer protocol handled all of it beneath the interface.
print(s.recv(128).split(b"\r\n")[0]) # b'HTTP/1.1 200 OK'
s.close()
# Switching the service, not the code structure:
u = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # SERVICE: unreliable datagrams
# Same interface style, different contract: messages may vanish or arrive out of order.
Key takeaways
- A service is what a layer promises upward; a protocol is how same-layer peers cooperate; an interface is the local API used to request the service.
- Services are vertical and real; peer protocols are horizontal and logical — data physically travels down, across, and up.
- Because the service is a contract, the protocol implementing it can be replaced without touching users of that layer.
- Confusing "protocol" with "service" is the root of most layering misunderstandings; always ask "what is promised" before "how is it done".
🧪 Practice
- For each of DNS, Ethernet, and TLS, write one sentence naming the service it provides and one naming a mechanism belonging to its protocol.
- The socket call
setsockopt(s, IPPROTO_TCP, TCP_NODELAY, 1)disables Nagle's algorithm. IsTCP_NODELAYpart of the service, the protocol, or the interface? Justify your answer.- Interview-style: Your team swaps the server's congestion control from CUBIC to BBR and clients need no changes. Explain precisely why, in layering terms. (Hint: identify which of service, protocol, and interface changed and which stayed fixed.)
Service Primitives and Service Access Points
Theory
Saying "a layer offers a service" is still vague. Formally, a service is defined by a small set of primitives — the operations its users may invoke and the events they may receive. The OSI work standardised four, and although few real APIs use these names, the vocabulary is universal in specifications and exam questions.
| Primitive | Who issues it | Meaning |
|---|---|---|
| REQUEST | Service user (layer N+1) | Ask the layer to do something: "connect", "send this data" |
| INDICATION | Service provider, to the remote user | Tell the peer's user that something happened: "a connection is being requested" |
| RESPONSE | Remote service user | Answer an indication: "I accept" |
| CONFIRM | Service provider, back to the original user | Report the outcome: "your connection is established" |
A service using all four is confirmed — the requester learns the result. A service using only REQUEST and INDICATION is unconfirmed: fire-and-forget, like UDP.DATA.request producing UDP.DATA.indication with nothing coming back. Connection setup is the classic confirmed service; sending a datagram is the classic unconfirmed one.
The second concept answers a question that layering raises immediately: if layer N serves several users above it, how does an arriving message reach the right one? The answer is the Service Access Point (SAP) — a labelled attachment point at the boundary between layers, identified by a SAP address (or, in TCP/IP terminology, a demultiplexing key). Think of a SAP as an apartment letterbox: the building has one street address (the layer), but each resident has their own slot, and the label on the slot is what the mail carrier reads.
Every layer in a real stack carries a SAP identifier in its header for exactly this purpose:
| Layer | SAP identifier in the header | Field width | Example values |
|---|---|---|---|
| Data link (Ethernet II) | EtherType | 16 bits | 0x0800 = IPv4, 0x86DD = IPv6, 0x0806 = ARP |
| Data link (802.2 LLC) | DSAP / SSAP | 8 bits each | 0x06 = IP, 0xAA = SNAP escape |
| Network (IPv4) | Protocol | 8 bits | 1 = ICMP, 6 = TCP, 17 = UDP |
| Network (IPv6) | Next Header | 8 bits | same registry as IPv4 Protocol |
| Transport (TCP/UDP) | Destination port | 16 bits | 80 = HTTP, 443 = HTTPS, 53 = DNS |
This is demultiplexing: one incoming stream of frames fans out into many independent conversations, and at each layer boundary the SAP field selects the next destination. Multiplexing is the same mechanism run downward — many users collapse onto one lower-layer service.
Examples
A confirmed service (connection establishment), in primitives
User A (L5) Provider (L4) User B (L5)
| CONNECT.request | |
|------------------->| |
| | CONNECT.indication |
| |---------------------->|
| | CONNECT.response |
| |<----------------------|
| CONNECT.confirm | |
|<-------------------| |
| ... connection established both sides ...|
An unconfirmed service has only the first two arrows.
SAPs turn one wire into many conversations (demultiplexing chain)
arriving Ethernet frame
|
EtherType = 0x0800 ------> IPv6 (0x86DD)? ARP (0x0806)?
v (other SAPs, unused here)
IPv4
|
Protocol = 6 --------------> ICMP (1)? UDP (17)?
v
TCP
|
dst port = 443 -------------> port 22 -> sshd
v port 53 -> resolver
browser socket
(203.0.113.9:51514 <-> 93.184.216.34:443)
# Reading SAP fields out of a captured frame by hand.
frame = bytes.fromhex(
"aabbccddeeff112233445566" "0800" # dst MAC, src MAC, EtherType
"4500003c1c4640004006" # IPv4: ver/IHL ... TTL=64, Protocol=6
"b1e6c0a80001c0a80002" # checksum, src IP, dst IP
"c9860050" # TCP: src port 51590, dst port 80
)
ethertype = int.from_bytes(frame[12:14], "big") # link-layer SAP
ihl = (frame[14] & 0x0F) * 4 # IPv4 header length in bytes
proto = frame[14 + 9] # network-layer SAP
dst_port = int.from_bytes(frame[14 + ihl + 2 : 14 + ihl + 4], "big") # transport SAP
print(f"EtherType=0x{ethertype:04x} Protocol={proto} dst_port={dst_port}")
# EtherType=0x0800 Protocol=6 dst_port=80
# Three SAP lookups, three layer boundaries crossed.
Key takeaways
- Services are specified by primitives: REQUEST, INDICATION, RESPONSE, CONFIRM; confirmed services use all four, unconfirmed only the first two.
- A SAP is the addressable attachment point between adjacent layers; its identifier appears as a field in the lower layer's header.
- EtherType, IP Protocol number, and TCP/UDP port are the same idea at three different boundaries.
- Demultiplexing is a chain of SAP lookups performed on the way up; multiplexing is the mirror image on the way down.
🧪 Practice
- Classify as confirmed or unconfirmed: a UDP
sendto(), a TCPconnect(), an ARP request, an ICMP echo request. Justify each.- IPv4's Protocol field is 8 bits and TCP's port field is 16 bits. Compute the maximum number of distinct upper-layer protocols and of simultaneous server services each can demultiplex, and explain why the two widths differ so much.
- Interview-style: Two browser tabs are both connected to
93.184.216.34:443. The arriving packets have identical destination IP and port — how does the kernel deliver each to the right socket? (Hint: a SAP need not be a single field; think about what uniquely identifies a TCP connection.)
Encapsulation and Decapsulation
Theory
If each layer adds its own control information, where does that information physically go? The answer is the mechanism that makes layering work on real wires: encapsulation. Layer N treats everything handed down from layer N+1 as an opaque blob — the payload or SDU (Service Data Unit) — and wraps it with its own header (and sometimes a trailer), producing its own PDU (Protocol Data Unit). That PDU becomes the SDU of layer N−1.
The nesting-envelope analogy is exact. You write a letter (application data). You put it in an envelope addressed to a person in a company (transport: which service inside the building). That envelope goes into a courier pouch addressed to the company's street address (network: which building on the planet). The pouch goes into a mail sack addressed to the next sorting depot (link: which machine on this hop). Each handler reads only its own layer's wrapper and never opens the inner ones.
Two properties of encapsulation deserve emphasis:
- Opacity. A layer must not inspect its payload. When it does anyway — as NAT devices and "deep packet inspection" middleboxes do — the layering contract is violated, and this is precisely why such devices break new protocols and force innovation to route around them.
- Symmetry. Decapsulation on the receiving side strips headers in exactly the reverse order, and the SAP field in each header tells the receiver which layer-above handler to pass the remainder to.
One subtlety separates the network layer from the rest: link-layer headers are rewritten at every hop, while the IP header travels end to end (its TTL and checksum change, but the addresses do not). A router decapsulates the frame, examines the packet, and re-encapsulates it in a brand-new frame for the next link — which may be a completely different link technology.
Here is what the wrapping costs for an ordinary HTTPS request over Ethernet:
| Layer | Adds | Cumulative bytes on a 1460-byte payload |
|---|---|---|
| Application (HTTP/1.1 headers) | ~200–800 B | varies |
| TLS record | 5 B header + 16 B auth tag | +21 |
| Transport (TCP) | 20 B (up to 60 with options) | +20 |
| Network (IPv4) | 20 B (up to 60 with options) | +20 |
| Link (Ethernet II) | 14 B header + 4 B FCS | +18 |
| Physical framing | 8 B preamble + 12 B interframe gap | +20 |
Examples
Encapsulation on the way down; decapsulation on the way up
SENDER RECEIVER
App [ HTTP request ] [ HTTP request ] ^
v add TCP header ^ strip TCP hdr |
TCP [TCPh| HTTP request ] [TCPh| HTTP request ] |
v add IP header ^ strip IP hdr |
IP [IPh |TCPh| HTTP request ] [IPh |TCPh| HTTP ] |
v add Ethernet header + trailer ^ strip frame |
Eth [ETHh|IPh |TCPh| HTTP request |FCS ] [ETHh|IPh |TCPh|..|FCS] |
v |
Phy 101101000111010...bits on the wire...------------------------------>+
Each layer sees only its own header; everything inside is opaque payload.
Re-encapsulation at a router: the frame changes, the packet does not
Laptop --Wi-Fi--> Router --Fibre--> ISP --Ethernet--> Server
hop 1: [802.11 hdr | IP src=10.0.0.5 dst=93.184.216.34 | TCP | data ]
hop 2: [ PPPoE hdr | IP src=10.0.0.5 dst=93.184.216.34 | TCP | data ]
hop 3: [ Eth hdr | IP src=10.0.0.5 dst=93.184.216.34 | TCP | data ]
^^^^^^^^^^^ rewritten every hop ^^^^^^^^^^^ unchanged end to end
(TTL decrements: 64 -> 63 -> 62)
# Build a frame by encapsulation, then take it apart by decapsulation.
def encapsulate(payload: bytes) -> bytes:
tcp = b"TCP[sport=51590,dport=80]" # stand-ins for real binary headers
ip = b"IP[src=10.0.0.5,dst=93.184.216.34]"
eth = b"ETH[src=aa:bb,dst=cc:dd,type=0x0800]"
seg = tcp + b"|" + payload # payload is opaque to TCP
pkt = ip + b"|" + seg # segment is opaque to IP
return eth + b"|" + pkt # packet is opaque to Ethernet
def decapsulate(frame: bytes) -> bytes:
for _ in range(3): # strip exactly as many headers as added
_header, frame = frame.split(b"|", 1) # each layer removes only its own
return frame
wire = encapsulate(b"GET / HTTP/1.1")
print(wire)
# b'ETH[...]|IP[...]|TCP[...]|GET / HTTP/1.1'
print(decapsulate(wire))
# b'GET / HTTP/1.1' <- byte-identical to what the application sent
Key takeaways
- Encapsulation wraps the layer above's PDU as an opaque payload plus this layer's header (and possibly trailer); decapsulation reverses it in mirror order.
- Layer N's PDU becomes layer N−1's SDU — the recursive relationship that makes the stack composable.
- Link-layer headers are stripped and rebuilt at every hop; the IP header survives end to end apart from TTL and checksum.
- Header overhead is real and cumulative: roughly 78 bytes per TCP/IPv4/Ethernet frame before any application bytes.
- Middleboxes that peek past their own layer break encapsulation and are the main cause of protocol ossification.
🧪 Practice
- Draw the encapsulation stack for a DNS query sent over UDP/IPv6 on a Wi-Fi link, naming the PDU at each layer.
- A VPN encapsulates an entire IP packet inside another IP packet (IP-in-IP or WireGuard over UDP). Compute the total overhead and explain why VPN links commonly need an MTU below 1500.
- Interview-style: Why can a router forward a packet from a Wi-Fi link to a fibre link without the sending host knowing anything about fibre? (Hint: state which header is rebuilt at the hop and which is carried unchanged.)
Protocol Data Units per Layer
Theory
The word "packet" gets used loosely for anything travelling on a network, but specifications are precise: the unit of data at each layer has its own name, and using the right one signals that you know which layer you are discussing. The generic term is PDU (Protocol Data Unit) = header + SDU (+ trailer).
| Layer (TCP/IP) | OSI layer | PDU name | Addressing it carries | Typical size limit |
|---|---|---|---|---|
| Application | 5–7 | Message / Data (formally APDU, PPDU, SPDU) | Names and URIs | Unbounded |
| Transport | 4 | Segment (TCP) / Datagram (UDP) | Port numbers | MSS, typically 1460 B |
| Network | 3 | Packet (IPv4/IPv6 datagram) | IP addresses | 65,535 B theoretical; MTU in practice |
| Data link | 2 | Frame | MAC addresses | MTU, typically 1500 B |
| Physical | 1 | Bit / Symbol | None | N/A |
Three refinements matter in practice.
A "datagram" means two different things. At the transport layer it means a UDP PDU; at the network layer it means an IP PDU. Context disambiguates, and the phrase "IP datagram" versus "UDP datagram" is worth saying in full.
The MTU governs everything above it. The link's Maximum Transmission Unit — 1500 bytes on standard Ethernet — is the largest payload a frame can carry. The IP packet must fit inside it, so TCP advertises an MSS (Maximum Segment Size) of MTU − 20 (IP) − 20 (TCP) = 1460 bytes, ensuring its segments never force fragmentation. This is a clean example of a lower layer's limit propagating upward through the stack by negotiation rather than by violation of layering.
PDU boundaries are not preserved by every layer. UDP and IP preserve message boundaries: one sendto() produces exactly one datagram. TCP does not — it is a byte stream, so two send() calls may arrive as one recv() or be split across three. Applications that need records over TCP must frame them themselves (length prefixes, delimiters), which is a constant source of bugs.
Examples
One application message, four PDU names
"GET /index.html HTTP/1.1..." <- MESSAGE (application)
|
[TCP hdr 20B | 1460 bytes of it ] <- SEGMENT (transport)
|
[IP hdr 20B | segment 1480B ] <- PACKET (network)
|
[Eth 14B | packet 1500B | FCS 4B] <- FRAME (link)
|
10110100011101000101110... <- BITS/SYMBOLS (physical)
A 4000-byte message becomes 3 segments -> 3 packets -> 3 frames.
# The MTU at the bottom decides the MSS at the top.
def mss_from_mtu(mtu, ip_version=4, tcp_options=12):
ip_hdr = 20 if ip_version == 4 else 40 # IPv6's fixed header is 40 bytes
tcp_hdr = 20 + tcp_options # options: timestamps (10) + padding (2)
return mtu - ip_hdr - tcp_hdr
for label, mtu, ver in [("Ethernet",1500,4), ("Ethernet v6",1500,6),
("PPPoE DSL",1492,4), ("WireGuard",1420,4), ("Jumbo",9000,4)]:
print(f"{label:<12} MTU={mtu:<5} -> usable MSS = {mss_from_mtu(mtu, ver)} bytes")
# Ethernet MTU=1500 -> usable MSS = 1448 bytes
# Ethernet v6 MTU=1500 -> usable MSS = 1428 bytes
# PPPoE DSL MTU=1492 -> usable MSS = 1440 bytes
# WireGuard MTU=1420 -> usable MSS = 1368 bytes
# Jumbo MTU=9000 -> usable MSS = 8948 bytes
# How many PDUs does a file become?
import math
file_bytes, mss = 1_000_000, 1448
print(math.ceil(file_bytes / mss), "segments -> the same number of packets and frames")
# 691 segments -> the same number of packets and frames
# Read the real MTU and the MSS the kernel will advertise.
$ ip link show eth0
# 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP
# ^^^^^^^^ link-layer PDU limit
$ ss -ti dst 93.184.216.34
# ESTAB 0 0 10.0.0.5:51590 93.184.216.34:443
# cubic wscale:7,7 rtt:88.2/0.6 mss:1448 cwnd:10 bytes_sent:2914
# ^^^^^^^^ transport PDU limit derived from the MTU
Key takeaways
- PDU names by layer: message (application), segment or UDP datagram (transport), packet/IP datagram (network), frame (link), bit or symbol (physical).
- A PDU is header + payload; the payload is the SDU handed down from the layer above.
- The link MTU constrains the IP packet, which constrains the TCP MSS — a lower-layer limit propagating upward by negotiation.
- TCP does not preserve message boundaries; UDP and IP do. Application-level framing is mandatory over TCP.
🧪 Practice
- Name the PDU at every layer for a 3-byte MQTT publish sent over TLS/TCP/IPv4/Wi-Fi, and compute the total bytes actually placed on the air.
- An application calls
send()three times with 10, 20, and 30 bytes on a TCP socket. List the possible sequences ofrecv()results the peer might observe, and explain what invariant TCP does guarantee.- Interview-style: A VPN drops the path MTU to 1420 but the sender keeps emitting 1500-byte frames with DF set. What breaks, and what symptom does the user report? (Hint: think about which ICMP message should appear and what happens when a firewall discards it.)
Benefits and Costs of Layering
Theory
Layering is not free, and treating it as an unmitigated good produces poor engineering judgement. It is a deliberate trade: modularity and evolvability in exchange for overhead and lost information.
What layering buys.
- Decomposition. Each layer is a tractable problem with a written contract. TCP's authors did not need to understand optical amplifiers.
- Independent evolution. IPv4 to IPv6, copper to fibre, HTTP/1.1 to HTTP/2 — each replaced without rewriting the others. The Internet survived a 10⁶-fold increase in link speed because link technology sits behind a stable interface.
- Interoperability. Vendors implement to the contract, so a Linux host talks to a Cisco router talks to an Apple laptop.
- Reuse. One TCP implementation serves every application; one Ethernet driver serves every protocol above it.
- Fault isolation. A troubleshooting methodology falls out of the model: test layer by layer (see Chapter 14).
What layering costs.
- Header overhead. Roughly 78 bytes per frame, as computed in Chapter 1 — 5 percent of a full-size frame, but 98 percent of a one-byte keystroke.
- Duplicated functionality. Error detection exists in Ethernet's FCS, IP's header checksum, TCP's checksum, and TLS's authentication tag. Retransmission exists in both 802.11 and TCP.
- Information hiding harms optimisation. TCP interprets loss as congestion because IP cannot tell it that the loss was radio interference — the classic wireless throughput collapse.
- Copying and context switches. A naive implementation copies data at every boundary; this is why real stacks use zero-copy techniques, scatter-gather I/O, and offload engines that deliberately blur the boundaries.
The practical resolution is layer violation where it pays, done explicitly. Real systems are full of sanctioned violations: TCP's checksum covers a "pseudo-header" of IP addresses (layer 4 reading layer 3); NAT rewrites ports (layer 3 device editing layer 4); TCP segmentation offload has the NIC build segments (layer 2 hardware doing layer 4 work); QUIC moves transport into user space over UDP to escape kernel and middlebox ossification. Cross-layer design is legitimate engineering — the sin is doing it accidentally.
| Dimension | Strict layering | Cross-layer optimisation |
|---|---|---|
| Development cost | Low — clear contracts | High — coupled components |
| Performance | Leaves headroom on the table | Can be dramatically better |
| Evolvability | Excellent | Poor — changes ripple |
| Typical use | Protocol standards, general-purpose stacks | Wireless, data centres, HPC, kernel bypass |
Examples
Duplicated error detection across four layers
TLS [ ... record ... | 16-byte AEAD auth tag ] <- integrity + authenticity
TCP [ ... segment ... ] 16-bit checksum <- end-to-end, weak
IP [ header ] 16-bit header checksum (IPv4) <- header only; dropped in IPv6
Eth [ ... frame ... ] 32-bit CRC (FCS) <- per hop, strong, discarded next hop
Four mechanisms, four purposes, and only TLS's survives a malicious router.
# Quantifying the cost of layering for different traffic profiles.
HDR = 78 # Ethernet + IPv4 + TCP framing overhead, bytes
profiles = [
("SSH keystroke", 1),
("VoIP frame (G.711)", 160),
("HTTP API response", 800),
("Bulk file transfer", 1448),
]
for name, payload in profiles:
wire = payload + HDR
print(f"{name:<20} payload={payload:>5}B wire={wire:>5}B "
f"overhead={HDR/wire*100:5.1f}%")
# SSH keystroke payload= 1B wire= 79B overhead= 98.7%
# VoIP frame (G.711) payload= 160B wire= 238B overhead= 32.8%
# HTTP API response payload= 800B wire= 878B overhead= 8.9%
# Bulk file transfer payload= 1448B wire= 1526B overhead= 5.1%
# The same layering is nearly free for bulk data and catastrophic for tiny messages.
# A sanctioned layer violation you can observe: the NIC doing TCP's job.
$ ethtool -k eth0 | grep -E 'segmentation|checksum'
# rx-checksumming: on <- layer 2 hardware verifying a layer 4 checksum
# tx-checksumming: on
# tcp-segmentation-offload: on <- kernel hands down a 64 KB "super-segment";
# generic-segmentation-offload: on the NIC splits it into MSS-sized segments
# Layering is preserved in the specification and violated in the silicon, on purpose.
Key takeaways
- Layering trades performance and information for modularity, interoperability, independent evolution, and debuggability — usually a good trade.
- Costs are concrete: cumulative header overhead, duplicated error control, hidden information that prevents optimisation, and per-boundary data movement.
- Overhead is negligible for bulk transfer and dominant for tiny messages; judge layering against the traffic profile, not in the abstract.
- Deliberate cross-layer design (offload, NAT, QUIC, wireless-aware transports) is normal engineering; accidental layer violation is a bug.
🧪 Practice
- List every mechanism in the stack that can retransmit lost data, and explain one scenario where two of them fighting each other degrades performance.
- Using the script above, compute the overhead percentage for a 40-byte IoT sensor reading over 6LoWPAN with a 127-byte MTU. What does this imply about header compression for IoT?
- Interview-style: TCP on a Wi-Fi link performs poorly during interference. Explain the layering reason and name two fixes — one that respects layering and one that violates it. (Hint: what does TCP assume packet loss means, and who actually knows the truth?)
<a id="22-the-osi-reference-model"></a>
2.2 The OSI Reference Model
The OSI model is the shared vocabulary of the networking profession even though almost nothing implements it literally; this subchapter walks its seven layers bottom-up, stating what each is responsible for and which real protocols occupy it.
Physical Layer Responsibilities
Theory
The Open Systems Interconnection reference model was published by ISO as ISO/IEC 7498-1 in 1984, aiming to let equipment from any vendor interoperate. Its protocol suite lost to TCP/IP, but its seven-layer decomposition won as a teaching and diagnostic framework — when an engineer says "that's a layer 2 problem", everyone knows what is meant.
Layer 1, the physical layer, moves individual bits across a physical medium. It is the only layer that touches reality: voltages, light pulses, radio waves. Its defining property is that it has no concept of a message, an address, or an error — it is a bit pipe, and a bit pipe that delivers a corrupted bit has done its job as far as it knows.
The physical layer specifies four categories of thing:
| Category | What it fixes | Example |
|---|---|---|
| Mechanical | Connectors, pinouts, cable dimensions | RJ45 8P8C plug, LC fibre connector, pin 1 = TX+ |
| Electrical / optical | Voltage or power levels, timing, impedance | ±1 V differential, 100 Ω twisted pair, 1310 nm laser |
| Functional | Meaning of each circuit | Which pins carry data, clock, or ground |
| Procedural | Sequence of events to establish the link | Ethernet autonegotiation of speed and duplex |
Concepts that belong here and nowhere else: bit rate (bits per second), baud rate (symbols per second), encoding (how a bit becomes a signal — NRZ, Manchester, 8B/10B), modulation (ASK, FSK, PSK, QAM), transmission mode (simplex, half duplex, full duplex), and multiplexing of the medium itself (TDM, FDM, WDM). Chapter 3 covers all of these in depth.
Devices operating purely at layer 1 — repeaters, hubs, media converters, optical amplifiers — regenerate or convert signals without understanding them. A hub receiving a frame on one port copies the electrical signal to every other port; it cannot read the destination address, which is why hubs create one giant collision domain and have been obsolete for two decades.
The analogy: layer 1 is the road surface and the wheels. It does not know where the truck is going or what is inside; it only guarantees that a wheel turning here produces motion there.
Examples
Layer 1 is the only layer that exists physically
Host A Host B
[ L2 frame: 1011000101110100... ] [ 1011000101110100... ]
| encode to signal ^ decode
v |
+-----------+ +-----------+
| PHY / NIC |==== copper, fibre, or radio ====>| PHY / NIC |
+-----------+ (repeaters may sit here) +-----------+
Manchester encoding of the bits 1 0 1 1 0:
bit: 1 0 1 1 0
signal: _|‾ ‾|_ _|‾ _|‾ ‾|_
^ every bit has a mid-bit transition -> the clock is recoverable
from the data itself, with no separate clock wire
# Everything in this output is layer 1: no addresses, no packets.
$ ethtool eth0
# Settings for eth0:
# Supported ports: [ TP ] <- mechanical: twisted pair
# Supported link modes: 10baseT/Half 100baseT/Full 1000baseT/Full
# Advertised auto-negotiation: Yes <- procedural: speed/duplex negotiation
# Speed: 1000Mb/s <- electrical: signalling rate
# Duplex: Full <- functional: both directions at once
# Link detected: yes <- is there a signal on the wire at all?
# A layer 1 fault has a layer 1 symptom: no link, or errors with no frames.
$ ethtool -S eth0 | grep -iE 'crc|symbol|carrier'
# rx_crc_errors: 41821 <- bits arriving corrupted: cable, connector, or noise
# rx_symbol_errors: 3902 <- the PHY could not even decode a valid symbol
# No amount of IP configuration fixes this.
Key takeaways
- Layer 1 transports raw bits over a medium and specifies mechanical, electrical, functional, and procedural characteristics.
- It has no addresses, no frames, and no error recovery — a corrupted bit is delivered as faithfully as a correct one.
- Bit rate, baud rate, encoding, modulation, duplex mode, and physical multiplexing all live here.
- Repeaters and hubs are layer 1 devices: they regenerate signals without interpreting them.
- Layer 1 faults show as no link, CRC errors, or symbol errors — never as routing problems.
🧪 Practice
- Classify each as layer 1 or not: RJ45 pinout, MAC address, autonegotiation, VLAN tag, 1310 nm wavelength, CRC check.
- A 1000BASE-T link autonegotiates to 100 Mb/s half duplex. Which of the four physical-layer categories is involved, and what physical cause would you suspect first?
- Interview-style: A user reports a "slow network".
ethtool -Sshows 40,000 CRC errors and climbing. Why is investigating TCP settings a waste of time? (Hint: which layer's failure is invisible to, but destructive for, every layer above?)
Data Link Layer Responsibilities
Theory
Layer 1 gives you an unreliable bit pipe. Layer 2, the data link layer, turns that bit pipe into a usable link between two directly connected devices. The key phrase is directly connected: the data link layer's horizon is one hop. It has no idea the Internet exists.
It solves four problems that layer 1 leaves open.
- Framing. The bit stream must be divided into discrete units. Where does one frame end and the next begin? Techniques include byte/bit stuffing, length fields, and — in Ethernet — a preamble with a start-of-frame delimiter plus an enforced interframe gap.
- Physical addressing. On a shared medium, which attached device should take this frame? MAC addresses are 48-bit identifiers burned into the NIC: the first 24 bits are the OUI (Organisationally Unique Identifier) assigned by the IEEE to a manufacturer, the last 24 are vendor-assigned.
FF:FF:FF:FF:FF:FFis broadcast. MAC addresses are flat — they encode no location, which is exactly why they cannot be used for global routing. - Error detection. A CRC in the frame's trailer (the FCS, Frame Check Sequence) detects corruption. Note detection, not correction: Ethernet silently discards a bad frame and leaves recovery to someone above.
- Medium access control. When several devices share a medium, who transmits when? CSMA/CD on legacy Ethernet, CSMA/CA on Wi-Fi, token passing on older rings.
The IEEE splits layer 2 into two sublayers, a distinction that appears constantly in 802 documents:
| Sublayer | Standard | Responsibility |
|---|---|---|
| LLC (Logical Link Control) | IEEE 802.2 | Multiplexing upper-layer protocols (DSAP/SSAP), optional flow and error control; medium-independent |
| MAC (Media Access Control) | 802.3, 802.11, … | Addressing and arbitration of the specific medium |
Layer 2 devices are switches (also called bridges): they read destination MAC addresses, learn which port each address lives behind, and forward frames only where they need to go — creating one collision domain per port while leaving a single broadcast domain (unless VLANs split it). Chapter 4 covers all of this.
Examples
Ethernet II frame: framing, addressing, demultiplexing, error detection
|pre 7B|SFD 1B| dst MAC 6B | src MAC 6B |type 2B| payload 46-1500B |FCS 4B|
^framing ^who takes it ^who sent it ^SAP ^opaque to layer 2 ^error
detect
A MAC address decoded: 00:1A:2B : 3C:4D:5E
^^^^^^^^ ^^^^^^^^
OUI (vendor) NIC-specific
Second-least-significant bit of the first byte = 1 -> locally administered
Least-significant bit of the first byte = 1 -> multicast/broadcast
# Decoding what a MAC address tells you (and what it does not).
def describe_mac(mac: str) -> dict:
first = int(mac.split(":")[0], 16)
return {
"oui": mac[:8].upper(), # vendor prefix, IEEE-assigned
"multicast": bool(first & 0b00000001), # I/G bit: 1 = group address
"local": bool(first & 0b00000010), # U/L bit: 1 = not globally unique
"location": None, # flat address: encodes NO topology
}
for m in ["00:1a:2b:3c:4d:5e", "ff:ff:ff:ff:ff:ff", "02:42:ac:11:00:02"]:
print(m, describe_mac(m))
# 00:1a:2b:3c:4d:5e {'oui': '00:1A:2B', 'multicast': False, 'local': False, ...}
# ff:ff:ff:ff:ff:ff {'oui': 'FF:FF:FF', 'multicast': True, 'local': True, ...}
# 02:42:ac:11:00:02 {'oui': '02:42:AC', 'multicast': False, 'local': True, ...}
# ^ a Docker container: locally administered, not from a vendor
# A switch's MAC table is layer 2 learning made visible.
$ show mac address-table
# Vlan Mac Address Type Ports
# 10 001a.2b3c.4d5e DYNAMIC Gi1/0/1 <- learned from this frame's SOURCE
# 10 00e0.4c68.01aa DYNAMIC Gi1/0/2
# 20 0242.ac11.0002 DYNAMIC Gi1/0/7
# Frames for an unknown destination are flooded to all ports in the VLAN.
Key takeaways
- Layer 2 provides node-to-node delivery over a single link: framing, MAC addressing, error detection, and medium access control.
- MAC addresses are 48-bit, flat, and globally unique by construction (OUI + vendor part) — they carry no location information.
- The FCS detects errors but does not correct them; corrupted frames are discarded silently.
- IEEE splits layer 2 into LLC (802.2, medium-independent multiplexing) and MAC (medium-specific arbitration).
- Switches are layer 2 devices: they learn source MACs per port and forward by destination MAC.
🧪 Practice
- Explain why a network built only from switches cannot scale to Internet size, referring specifically to the flatness of MAC addresses.
- Given the frame layout above, compute the minimum and maximum number of bytes on the wire for one Ethernet II frame including preamble and interframe gap.
- Interview-style: Two hosts on the same switch can ping each other; one of them cannot reach the Internet. Which layer would you suspect and why? (Hint: what does working same-subnet connectivity already prove about layers 1 and 2?)
Network Layer Responsibilities
Theory
Layer 2 can deliver a frame to a neighbour. But the machine you want is eleven hops away, on a network built from Wi-Fi, fibre, and submarine cable, run by seven different companies. Layer 3, the network layer, provides host-to-host delivery across multiple, heterogeneous networks. This is the layer that creates "the Internet" out of a collection of unrelated networks — the job is called internetworking.
Three responsibilities define it.
Logical addressing. Layer 3 introduces addresses that are hierarchical rather than flat: an IPv4 address like 93.184.216.34 with a prefix length /24 splits into a network part and a host part. Hierarchy is what makes routing tractable — a core router does not store 4 billion host entries, it stores a few hundred thousand prefixes and aggregates everything below them. This is precisely what MAC addresses cannot do.
Routing and forwarding. These are distinct and frequently confused:
| Routing | Forwarding | |
|---|---|---|
| Question | Which path should traffic take? | Which port does this packet leave by? |
| Timescale | Seconds to minutes (control plane) | Nanoseconds per packet (data plane) |
| Mechanism | OSPF, BGP, static configuration | Longest-prefix-match lookup in the FIB |
| Output | The routing table (RIB) | A forwarding decision |
Fragmentation and reassembly. When a packet is larger than the next link's MTU, IPv4 routers may split it; the destination reassembles. IPv6 removed this from routers entirely, requiring the source to discover the path MTU instead — a deliberate simplification of the fast path.
The service IP offers upward is deliberately minimal: best-effort, connectionless delivery. Packets may be lost, duplicated, delayed arbitrarily, or delivered out of order, and IP will not tell you. This looks like a weakness and is in fact the design's greatest strength — a minimal contract is one that every conceivable link technology can honour, which is what allowed IP to run over everything from carrier pigeons (RFC 1149, only half a joke) to 800 Gb/s optics.
Supporting protocols live here too: ICMP (error and diagnostic messages — this is what ping and traceroute use), ARP (arguably straddling layers 2 and 3), and the routing protocols themselves. Chapters 5 and 6 cover the layer in full.
Examples
Hierarchical addressing enables aggregation; flat addressing does not
Core router's view with aggregation:
203.0.113.0/24 -> next hop ISP-B (one entry)
covers 256 hosts, each of which would otherwise need its own entry.
Forwarding = longest prefix match:
packet dst = 203.0.113.77
0.0.0.0/0 via 198.51.100.1 (default, match length 0)
203.0.0.0/16 via 198.51.100.9 (match length 16)
203.0.113.0/24 via 198.51.100.20 (match length 24) <-- WINS
203.0.113.64/26 via 198.51.100.33 (match length 26) <-- actually WINS
The most specific matching prefix is chosen, always.
import ipaddress
# A miniature forwarding table and a longest-prefix-match lookup.
table = [
(ipaddress.ip_network("0.0.0.0/0"), "198.51.100.1"), # default route
(ipaddress.ip_network("203.0.0.0/16"), "198.51.100.9"),
(ipaddress.ip_network("203.0.113.0/24"), "198.51.100.20"),
(ipaddress.ip_network("203.0.113.64/26"), "198.51.100.33"),
]
def forward(dst: str) -> str:
addr = ipaddress.ip_address(dst)
# Every matching prefix is a candidate; the longest one wins.
matches = [(net, hop) for net, hop in table if addr in net]
net, hop = max(matches, key=lambda pair: pair[0].prefixlen)
return f"{dst:<15} -> {hop:<13} (matched {net})"
for d in ["203.0.113.77", "203.0.113.200", "203.0.55.1", "8.8.8.8"]:
print(forward(d))
# 203.0.113.77 -> 198.51.100.33 (matched 203.0.113.64/26)
# 203.0.113.200 -> 198.51.100.20 (matched 203.0.113.0/24)
# 203.0.55.1 -> 198.51.100.9 (matched 203.0.0.0/16)
# 8.8.8.8 -> 198.51.100.1 (matched 0.0.0.0/0)
# Layer 3 in action: the packet survives hops; the frame does not.
$ traceroute -n example.com
# 1 192.168.1.1 1.1 ms <- each line is one layer 3 hop (a router)
# 2 10.20.0.1 9.8 ms <- between lines, the layer 2 frame was rebuilt
# 3 62.115.120.1 11.4 ms possibly on an entirely different medium
# 4 93.184.216.34 11.9 ms
# traceroute works by abusing a layer 3 field: TTL=1,2,3... to elicit ICMP
# "time exceeded" from each successive router.
Key takeaways
- Layer 3 delivers packets host-to-host across multiple heterogeneous networks — the internetworking layer.
- Logical addresses are hierarchical (network part + host part), which permits route aggregation; MAC addresses cannot do this.
- Routing (building the table, control plane, slow) and forwarding (per-packet lookup, data plane, fast) are separate activities.
- Forwarding uses longest-prefix match: the most specific matching route always wins.
- IP's service is best-effort and connectionless — packets may be lost, reordered, or duplicated, and this minimal contract is what makes universal connectivity possible.
🧪 Practice
- Given routes
10.0.0.0/8,10.1.0.0/16, and10.1.2.0/24, state the chosen route for10.1.2.9,10.1.9.9, and10.9.9.9.- Explain why removing router fragmentation in IPv6 speeds up the data plane, and what the source host must do instead.
- Interview-style: Why is IP's "best-effort" service considered a feature rather than a deficiency? (Hint: think about what guarantees a satellite link, a lossy radio link, and a fibre link can all simultaneously promise.)
Transport Layer Responsibilities
Theory
Layer 3 delivers packets to a host. But a host runs dozens of programs at once, and the packet is meant for exactly one of them — and it may have been lost or reordered on the way. Layer 4, the transport layer, provides process-to-process delivery and, optionally, the reliability that IP declines to offer.
This is the lowest layer that is implemented end to end only. Routers along the path do not process transport headers (NAT devices do, which is a layer violation with far-reaching consequences). If your TCP thinks a segment is lost, no router in the middle knows or cares.
Its responsibilities:
- Multiplexing and demultiplexing via port numbers, the transport-layer SAP. A socket — the tuple (protocol, local IP, local port, remote IP, remote port) — uniquely identifies a connection, which is how a thousand clients can all talk to port 443 on one server without confusion.
- Reliability (optional): sequence numbers detect loss and reordering, acknowledgements confirm receipt, timers trigger retransmission.
- Ordering (optional): buffering out-of-order arrivals until the gap is filled.
- Flow control: stopping a fast sender from overwhelming a slow receiver, via a sliding window.
- Congestion control: stopping senders collectively from overwhelming the network — a different problem with a different solution, and the reason the Internet did not collapse in 1986.
- Connection management: establishment and teardown, for connection-oriented service.
The two classical choices present a stark trade:
| TCP | UDP | |
|---|---|---|
| Connection | Connection-oriented (3-way handshake) | Connectionless |
| Reliability | Guaranteed delivery and ordering | None |
| Flow / congestion control | Yes | No |
| Boundaries | Byte stream — no message boundaries | Preserves datagram boundaries |
| Header | 20–60 bytes | 8 bytes |
| Overhead on setup | ≥ 1 RTT before data | Zero |
| Typical use | Web, email, file transfer, SSH | DNS, VoIP, gaming, QUIC, video |
Port ranges are worth memorising: 0–1023 well-known (assigned by IANA, traditionally requiring privilege to bind on Unix), 1024–49151 registered, 49152–65535 dynamic/ephemeral (what your client gets automatically).
The usual analogy: layer 3 gets the letter to the right building; layer 4 gets it to the right desk, and if it is registered mail, chases it until it is signed for. Chapter 7 develops all of this.
Examples
The 4-tuple is what makes one server port serve thousands of clients
Server 93.184.216.34:443 — all four connections share a dst IP and port
proto local addr:port remote addr:port -> distinct socket?
TCP 93.184.216.34:443 198.51.100.7:51514 yes
TCP 93.184.216.34:443 198.51.100.7:51515 yes (different src port)
TCP 93.184.216.34:443 203.0.113.9:51514 yes (different src IP)
UDP 93.184.216.34:443 198.51.100.7:51514 yes (different protocol)
Demultiplexing key = (protocol, src IP, src port, dst IP, dst port)
import socket, time
# Same destination, two transports: the service contract differs, not the address.
def try_tcp(host, port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(3)
t0 = time.time()
s.connect((host, port)) # blocks for a full RTT: SYN -> SYN/ACK -> ACK
print(f"TCP connected in {(time.time()-t0)*1000:.1f} ms — a handshake happened")
print(" local socket end:", s.getsockname()) # the ephemeral port the OS chose
s.close()
def try_udp(host, port):
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
t0 = time.time()
s.sendto(b"probe", (host, port)) # returns immediately: nothing was negotiated,
print(f"UDP sent in {(time.time()-t0)*1000:.3f} ms — no handshake, no guarantee")
s.close() # and no way to know if it arrived
try_tcp("example.com", 80)
# TCP connected in 88.4 ms — a handshake happened
# local socket end: ('10.0.0.5', 51590) <- ephemeral port from the 49152+ range
try_udp("example.com", 9999)
# UDP sent in 0.042 ms — no handshake, no guarantee
# Every live transport-layer conversation on the machine, with its 4-tuple.
$ ss -tunap
# Netid State Local Address:Port Peer Address:Port Process
# tcp LISTEN 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=812))
# tcp ESTAB 10.0.0.5:51590 93.184.216.34:443 users:(("firefox",pid=4410))
# tcp ESTAB 10.0.0.5:51591 93.184.216.34:443 users:(("firefox",pid=4410))
# udp UNCONN 10.0.0.5:68 0.0.0.0:* users:(("dhclient",pid=655))
# ^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^
# two ESTAB rows differ only in the local port — that is the demux key
Key takeaways
- Layer 4 provides process-to-process delivery; port numbers are its SAP and the 4-tuple (plus protocol) uniquely identifies a connection.
- It is the first strictly end-to-end layer: routers do not participate in transport semantics.
- Optional services include reliability, ordering, flow control (protecting the receiver), and congestion control (protecting the network).
- TCP trades latency and overhead for a reliable ordered byte stream; UDP trades all guarantees for an 8-byte header and zero setup.
- Port ranges: 0–1023 well-known, 1024–49151 registered, 49152–65535 ephemeral.
🧪 Practice
- Explain why flow control and congestion control cannot be replaced by a single mechanism, giving a scenario where each acts alone.
- A server has 10,000 established connections on port 443. How many distinct destination ports is it using, and what is actually distinguishing the connections?
- Interview-style: DNS traditionally uses UDP but falls back to TCP for large responses. Explain both choices in transport-layer terms. (Hint: consider request size, RTT cost of a handshake, and what happens when a response exceeds what one datagram can safely carry.)
Session, Presentation, and Application Layers
Theory
The top three OSI layers are grouped here for an honest reason: in the TCP/IP world they are not separate layers at all. TCP/IP collapses them into a single application layer, and real protocols scatter their functions freely. Knowing what OSI intended for each is still valuable, because the functions exist even where the layers do not.
Layer 5 — Session. Manages dialogues between applications: establishing, maintaining, and tearing down sessions; dialogue control (deciding whose turn it is to send in half-duplex exchanges); synchronisation via checkpoints, so a two-hour transfer that fails at minute 119 resumes from the last checkpoint rather than from zero.
Where the function actually lives today: HTTP cookies and session tokens, TLS session resumption (tickets and PSKs), SIP for call setup, RPC session state, WebSocket connection lifecycle, SMB/NFS session establishment.
Layer 6 — Presentation. Concerned with the syntax and semantics of transmitted data rather than its delivery. Three classic functions: translation between machine representations (character encodings, byte order), compression, and encryption.
Where it lives today: TLS (encryption is presentation work sitting between application and transport), Content-Encoding: gzip, JSON/XML/Protocol Buffers serialisation, UTF-8, JPEG and H.264 codecs, ASN.1/BER in SNMP and X.509 certificates.
Byte order deserves a note: network protocols use big-endian ("network byte order") because the early standards did, while x86 and ARM are little-endian — so every host must convert. That is a presentation-layer concern handled by htons()/ntohs() in practice.
Layer 7 — Application. The layer that provides network services to user programs. Note the precision: your web browser is not layer 7; HTTP is layer 7, and the browser is a user of it.
| OSI layer | Core functions | Where it really lives in TCP/IP |
|---|---|---|
| 7 Application | Service interfaces to programs | HTTP, SMTP, DNS, FTP, SSH, DHCP |
| 6 Presentation | Encoding, compression, encryption | TLS, gzip, UTF-8, JSON, Protobuf, JPEG |
| 5 Session | Dialogue control, checkpoints, resumption | TLS resumption, HTTP cookies, SIP, RPC |
A caution for interviews and exams: real engineers say "layer 7" to mean everything above layer 4. A "layer 7 load balancer" makes decisions using HTTP paths and headers. Nobody says "layer 6 firewall".
Examples
One HTTPS request, with OSI 5-7 functions labelled where they really occur
+-------------------------------------------------------------+
| GET /api/users HTTP/1.1 | L7 application
| Host: api.example.com | protocol
| Cookie: session=8f3a... <---- L5 session state |
| Accept-Encoding: gzip <---- L6 negotiation |
| Content-Type: application/json <---- L6 syntax declaration |
+-------------------------------------------------------------+
| serialise to UTF-8 JSON L6 translation
| compress with gzip L6 compression
| wrap in a TLS record L6 encryption
| L5 resumption via ticket
v
+-------------------------------------------------------------+
| TCP segment -> IP packet -> Ethernet frame -> bits | L4..L1
+-------------------------------------------------------------+
import json, gzip, struct
payload = {"user": "ada", "role": "admin", "note": "byte order matters"}
# --- Layer 6 work, done explicitly ---
serialised = json.dumps(payload).encode("utf-8") # translation: object -> UTF-8 bytes
compressed = gzip.compress(serialised) # compression
print(f"serialised={len(serialised)}B compressed={len(compressed)}B")
# serialised=63B compressed=72B <- gzip LOSES on tiny payloads; overhead is ~20B
# Byte order is presentation-layer translation too:
port = 443
print(struct.pack(">H", port).hex(), "= big-endian (network byte order)") # 01bb
print(struct.pack("<H", port).hex(), "= little-endian (x86 host order)") # bb01
# Sending the wrong one puts your traffic on port 47873 instead of 443.
# --- Layer 5 work: a session identifier carried above a stateless transport ---
session = {"id": "8f3a2c...", "expires": 1789000000}
# HTTP itself is stateless; the SESSION is reconstructed from this token on
# every request, possibly over a brand-new TCP connection.
Key takeaways
- OSI layers 5–7 are collapsed into one application layer by TCP/IP; the functions survive even though the layer boundaries do not.
- Session: dialogue control, checkpointing, and resumption — seen today in TLS session tickets, cookies, and SIP.
- Presentation: translation (character sets, byte order), compression, and encryption — seen today in TLS, gzip, UTF-8, and serialisation formats.
- Application: the protocol that serves user programs (HTTP, SMTP, DNS) — not the program itself.
- Network byte order is big-endian; hosts are usually little-endian, so conversion is mandatory.
- In industry usage, "layer 7" loosely means everything above the transport layer.
🧪 Practice
- Assign each to an OSI layer 5, 6, or 7 as originally intended:
Content-Encoding: br, a TLS session ticket, an SMTPMAIL FROMcommand, UTF-16 to UTF-8 conversion, an RPC call ID.- Explain why gzip made the payload in the example larger, and state a rule of thumb for when compression is worth applying.
- Interview-style: HTTP is described as stateless, yet websites remember you are logged in. Which OSI layer's function is being reimplemented, and where does the state actually live? (Hint: think about what the server stores versus what the client presents on each request.)
<a id="23-the-tcpip-model"></a>
2.3 The TCP/IP Model
OSI is the model everyone teaches; TCP/IP is the model everything actually runs. This subchapter covers its four layers, how they map onto OSI, and the two architectural principles — the narrow waist and the end-to-end argument — that explain why this stack and not another one took over the world.
Link, Internet, Transport, and Application Layers
Theory
The TCP/IP model was not designed and then implemented; it was extracted from working code. It describes the protocols of the ARPANET and its successors, which is why it is sometimes said that OSI has a model with no protocols while TCP/IP has protocols with no model. RFC 1122 (1989) is the canonical statement of it and defines four layers.
| Layer | Also called | Job | Address used | Key protocols |
|---|---|---|---|---|
| Application | Process layer | Everything user-facing, including encoding and sessions | Names, URIs | HTTP, DNS, SMTP, SSH, DHCP, QUIC's application binding |
| Transport | Host-to-host | Process-to-process delivery, optional reliability | Port numbers | TCP, UDP, SCTP, DCCP |
| Internet | Network layer | Host-to-host delivery across networks | IP addresses | IPv4, IPv6, ICMP, IGMP, IPsec |
| Link | Network access / network interface | Delivery over one physical link | MAC addresses | Ethernet, Wi-Fi, PPP, ARP, DSL, LTE |
Two differences from OSI are structural, not cosmetic.
The link layer is one layer, not two. RFC 1122 deliberately declines to separate physical from data link, because from IP's perspective they are a single opaque service: "hand me a way to get this packet to the next hop over whatever this is". This is why TCP/IP accommodates new media so easily — there is nothing to redesign, only a new link layer to write.
The application layer is one layer, not three. Session and presentation concerns are left to the application protocol and its libraries. This is more flexible (HTTP chooses its own compression negotiation) and less tidy (every protocol reinvents session handling).
The model's philosophy differs too. OSI was specified by committee before implementation and was strict about layer boundaries. TCP/IP grew from running code, was standardised by rough consensus, and is pragmatic — it tolerates violations that work, such as the TCP pseudo-header.
Some texts show a five-layer hybrid (application, transport, network, link, physical). This is a teaching convenience that keeps OSI's useful physical/link split while dropping OSI's unused session and presentation layers. It is a perfectly respectable model and matches how practitioners actually reason.
Examples
The four-layer stack with real protocol names in each box
+----------------------------------------------------------+
| APPLICATION HTTP DNS SMTP SSH DHCP NTP BGP gRPC |
+----------------------------------------------------------+
| TRANSPORT TCP UDP SCTP |
+----------------------------------------------------------+
| INTERNET IPv4 IPv6 ICMP ICMPv6 IGMP IPsec |
+----------------------------------------------------------+
| LINK Ethernet Wi-Fi PPP LTE/5G DOCSIS ARP |
+----------------------------------------------------------+
Note BGP sits at the APPLICATION layer (it runs over TCP port 179)
even though its purpose is to control the INTERNET layer. Function and
layer are different questions.
# One command, all four layers visible at once.
$ curl -sv https://example.com -o /dev/null 2>&1 | head -12
# * Host example.com:443 was resolved. <- APPLICATION (DNS)
# * IPv4: 93.184.216.34 <- INTERNET (address obtained)
# * Trying 93.184.216.34:443...
# * Connected to example.com (93.184.216.34) port 443 <- TRANSPORT (TCP handshake)
# * TLSv1.3 (OUT), TLS handshake, Client hello (1): <- APPLICATION (TLS/presentation)
# > GET / HTTP/1.1 <- APPLICATION (HTTP)
# > Host: example.com
$ ip neigh # <- LINK (MAC of next hop)
# 192.168.1.1 dev wlan0 lladdr 4c:ed:fb:11:22:33 REACHABLE
$ ip route get 93.184.216.34 # <- INTERNET (which way out)
# 93.184.216.34 via 192.168.1.1 dev wlan0 src 192.168.1.42
# Each layer's identifier for the exact same conversation.
conversation = {
"application": "https://example.com/index.html", # a NAME and a resource
"transport": ("TCP", 51590, 443), # PORTS
"internet": ("192.168.1.42", "93.184.216.34"), # IP ADDRESSES (end to end)
"link": ("a4:83:e7:aa:bb:cc", "4c:ed:fb:11:22:33"), # MACs (this hop only)
}
for layer, ident in conversation.items():
print(f"{layer:<12} {ident}")
# Note the link-layer destination is the ROUTER's MAC, not the server's —
# layer 2 only ever addresses the next hop.
Key takeaways
- TCP/IP has four layers — application, transport, internet, link — defined in RFC 1122 and derived from working implementations.
- It merges OSI's physical and data link into one link layer, and OSI's session, presentation, and application into one application layer.
- Each layer has its own address type: names/URIs, ports, IP addresses, MAC addresses.
- The link-layer destination is always the next hop; only the IP addresses are end to end.
- A protocol's layer is determined by what it runs over, not by what it controls — BGP is an application-layer protocol that manages routing.
🧪 Practice
- Place DHCP, ARP, ICMP, TLS, and QUIC in the four-layer model, and note which of them are genuinely awkward to place and why.
- Run
ip route getandip neighfor an external address on your machine. Explain why the MAC address returned does not belong to the destination host.- Interview-style: Why did TCP/IP's refusal to standardise the link layer turn out to be an advantage? (Hint: count how many link technologies have appeared and disappeared since 1983 without IP changing.)
Mapping TCP/IP to OSI
Theory
You will be asked to map the two models constantly — in exams, in interviews, and whenever two engineers from different backgrounds discuss the same problem. The mapping is mostly clean and has exactly two ragged edges.
| OSI | TCP/IP (RFC 1122) | 5-layer hybrid | Notes |
|---|---|---|---|
| 7 Application | Application | Application | |
| 6 Presentation | Application | Application | TLS lands here functionally |
| 5 Session | Application | Application | |
| 4 Transport | Transport | Transport | Direct correspondence |
| 3 Network | Internet | Network | Direct correspondence |
| 2 Data link | Link | Data link | OSI splits further into LLC + MAC |
| 1 Physical | Link | Physical | TCP/IP declines to separate these |
Ragged edge one: protocols that refuse to sit still.
- ARP maps IP addresses to MAC addresses. It is carried directly in Ethernet frames with its own EtherType (
0x0806), so it is not inside IP — yet it serves IP and understands IP addresses. Different textbooks call it layer 2, layer 2.5, or layer 3. RFC 1122 places it in the link layer; the honest answer in an interview is to state the ambiguity and explain both views. - ICMP is encapsulated inside IP packets (Protocol 1), which by the encapsulation rule would make it layer 4 — but it carries control messages for IP, so it is universally called layer 3.
- TLS sits between the application and TCP: OSI layer 6 by function, application layer by TCP/IP's reckoning.
- MPLS sits between layer 2 and layer 3 and is routinely called "layer 2.5".
- QUIC implements transport functions (reliability, congestion control, multiplexing) in user space above UDP: layer 4 by function, layer 7 by encapsulation.
Ragged edge two: the models were built for different worlds. OSI's session and presentation layers presuppose connection-oriented terminal sessions and heterogeneous mainframe encodings — the world of 1980. TCP/IP presupposes that applications are smart enough to handle their own encoding. Neither is wrong; they solve different problems.
The practical rule: use OSI's numbers for diagnosis and conversation, use TCP/IP's structure for understanding implementation. "Layer 3 issue" is clearer than "internet layer issue", but when you read a kernel source tree you will find four layers, not seven.
Examples
Side-by-side mapping with the awkward protocols marked
OSI TCP/IP Protocols
+-----------------+ +------------+ +---------------------------+
| 7 Application | | | | HTTP SMTP DNS SSH BGP |
+-----------------+ | | +---------------------------+
| 6 Presentation |-->| Application|<--| TLS gzip JSON ASN.1 | *
+-----------------+ | | +---------------------------+
| 5 Session | | | | TLS tickets cookies SIP |
+-----------------+ +------------+ +---------------------------+
| 4 Transport |-->| Transport |<--| TCP UDP SCTP [QUIC] | *
+-----------------+ +------------+ +---------------------------+
| 3 Network |-->| Internet |<--| IPv4 IPv6 [ICMP] IPsec | *
+-----------------+ +------------+ +---------------------------+
| 2 Data link | | | | Ethernet Wi-Fi [ARP] | *
+-----------------+ | Link | +---------------------------+
| 1 Physical | | | | copper fibre radio |
+-----------------+ +------------+ +---------------------------+
[*] = contested placement; MPLS would sit between 2 and 3
# A lookup table for the mapping, plus the honest "it depends" cases.
osi_to_tcpip = {
7: "Application", 6: "Application", 5: "Application",
4: "Transport", 3: "Internet",
2: "Link", 1: "Link",
}
contested = {
"ARP": "link layer per RFC 1122; often taught as 2.5 because it serves IP",
"ICMP": "inside IP by encapsulation, but called layer 3 by function",
"TLS": "OSI 6 by function, TCP/IP application layer by structure",
"QUIC": "transport function, carried over UDP, implemented in user space",
"MPLS": "layer 2.5: label switching below IP, above the link",
}
print({k: osi_to_tcpip[k] for k in sorted(osi_to_tcpip, reverse=True)})
for name, note in contested.items():
print(f"{name:<5} {note}")
Key takeaways
- OSI 5–7 collapse into TCP/IP's application layer; OSI 1–2 collapse into its link layer; layers 3 and 4 correspond directly.
- The five-layer hybrid model (application, transport, network, link, physical) is the practical compromise most practitioners reason with.
- ARP, ICMP, TLS, MPLS, and QUIC all sit awkwardly; naming the ambiguity is a better answer than picking a number confidently.
- A protocol's layer by encapsulation and by function can differ — ICMP and QUIC are the standard examples.
- Use OSI numbering for troubleshooting vocabulary and TCP/IP structure for reasoning about implementations.
🧪 Practice
- Build the full mapping table from memory, then mark every row where the correspondence is one-to-many.
- Argue both sides of "ARP is a layer 2 protocol" in three sentences each, citing encapsulation for one side and function for the other.
- Interview-style: Which OSI layer does TLS belong to? (Hint: the strong answer explains why the question has two defensible answers and names the criterion each uses.)
The Hourglass Model and IP Narrow Waist
Theory
Why did this particular architecture take over, when OSI had governments and standards bodies behind it? A large part of the answer is one diagram: the hourglass.
Draw the protocol universe by counting how many distinct protocols exist at each layer. Applications are innumerable — every web service, game, and messaging app. Link technologies are innumerable — Ethernet, Wi-Fi, LTE, 5G, DOCSIS, satellite, Bluetooth, fibre channel. But in the middle, there is exactly one protocol that everything must speak: IP. This is the narrow waist.
many applications \/
>< IP <- the waist: exactly one protocol
many link layers /\
The design rule that produces this shape is "IP over everything, and everything over IP". The consequences are profound:
- Innovation cost is O(1), not O(n·m). A new application needs to work with IP, not with 40 link technologies. A new radio technology needs to carry IP, not to interoperate with 10,000 applications. Without the waist, connecting n applications to m networks requires n × m adaptations; with it, n + m.
- Permissionless innovation. Neither end of the hourglass needs anyone's approval. The web was deployed without asking a single network operator, because it only needed TCP/IP, which was already everywhere.
- The waist must stay thin to stay universal. Every feature added to IP is a feature every link technology must support and every router must implement. This is why IP's service is minimal, and why proposals to enrich it (IP multicast, IntServ, IP QoS) have largely failed to deploy globally while proposals above it succeed.
The dark side is ossification. Precisely because everything depends on the waist, the waist cannot change. IPv6 was specified in 1998 and took over two decades to reach meaningful deployment. Worse, middleboxes — NATs, firewalls, and load balancers that inspect TCP and UDP headers — have effectively widened the waist to include TCP and UDP: a genuinely new transport protocol cannot be deployed because middleboxes drop what they do not recognise. SCTP is the casualty; QUIC is the workaround, delivering a new transport by disguising it as UDP, with everything above the UDP header encrypted so no middlebox can inspect or ossify it.
Examples
The hourglass, with rough protocol counts
HTTP SMTP DNS SSH RTP gRPC MQTT SIP FTP NTP BGP ... (thousands)
\ \ | / / /
\ \ | / / /
+-------- TCP, UDP (+SCTP, DCCP) --------+ (a handful)
|
IP (ONE)
|
+-------- ARP, NDP, and link adaptation --+
/ / | \ \ \
Ethernet Wi-Fi LTE 5G DOCSIS satellite fibre PPP ... (dozens)
Connecting a new app to every network: 1 adaptation, not 40.
# The combinatorial argument for a narrow waist, made numeric.
def adaptations(n_apps, n_links, narrow_waist: bool):
# Without a common layer, every application must be ported to every network.
return (n_apps + n_links) if narrow_waist else (n_apps * n_links)
for apps, links in [(10, 5), (100, 20), (5000, 40)]:
with_waist = adaptations(apps, links, True)
without_waist = adaptations(apps, links, False)
print(f"{apps:>5} apps x {links:>3} links -> "
f"with IP: {with_waist:>6} without: {without_waist:>7} "
f"ratio {without_waist/with_waist:.0f}x")
# 10 apps x 5 links -> with IP: 15 without: 50 ratio 3x
# 100 apps x 20 links -> with IP: 120 without: 2000 ratio 17x
# 5000 apps x 40 links -> with IP: 5040 without: 200000 ratio 40x
# The saving grows without bound — this is the whole architectural argument.
Ossification: how the waist got fatter than intended
Designed waist: Effective waist today:
+--------+ +--------------------+
| IP | | IP + TCP/UDP |
+--------+ | headers that |
| middleboxes parse |
+--------------------+
Consequence: SCTP (a good transport, standardised 2000) is undeployable on
the public Internet — NATs and firewalls drop IP protocol 132.
QUIC's answer: look exactly like UDP, encrypt everything above the UDP
header, and leave middleboxes nothing to ossify.
Key takeaways
- The hourglass model: many applications on top, many link technologies below, and exactly one protocol — IP — at the narrow waist.
- The waist turns an O(n × m) integration problem into O(n + m), which is the core reason the Internet scaled.
- The waist must remain minimal: every feature added to IP must be supported everywhere, so IP's service stays deliberately poor.
- Universality creates ossification — the waist is the hardest part of the architecture to change, as IPv6's deployment timeline shows.
- Middleboxes widened the effective waist to include TCP and UDP headers; QUIC evades this by running over UDP with everything above it encrypted.
🧪 Practice
- Explain in your own words why adding a mandatory QoS field to IP would be architecturally expensive, referring to the waist.
- Using the function above, compute the adaptation count for 50,000 applications and 60 link technologies. What does the ratio imply for a proposal to have two competing waist protocols?
- Interview-style: Why is QUIC built on UDP rather than directly on IP, given that UDP adds 8 bytes of useless header? (Hint: the answer has nothing to do with efficiency and everything to do with what deployed middleboxes will forward.)
End-to-End Argument
Theory
The end-to-end argument, stated by Saltzer, Reed, and Clark in 1984, is the design principle that decides which layer a function belongs in. It is the most quoted piece of reasoning in network architecture, and it is routinely misquoted, so the precise statement matters:
A function should be implemented at a lower layer only if it can be completely and correctly implemented there. If correctness requires knowledge available only at the endpoints, the function must be implemented at the endpoints — and a lower-layer implementation, being incomplete, is at best a performance optimisation, never a substitute.
The canonical example is careful file transfer. Suppose you want to move a file from disk A to disk B with certainty it arrived intact. Make every link perfectly reliable with link-layer checksums and retransmission. Is the file now guaranteed correct? No — the data can still be corrupted by a faulty router's memory while being copied between interfaces, by a buggy driver, by a bad DMA transfer, or by a disk write error at either end. None of these are on any link, so no link-layer mechanism can catch them. The only way to be sure is an end-to-end check: the receiving application computes a checksum over the file it wrote and compares it to one computed by the sender.
And once you have that end-to-end check, the link-layer checks are no longer needed for correctness. They may still be worth having for performance: a lossy wireless link that retransmits locally avoids paying a whole end-to-end round trip for each loss. That is the correct role for lower-layer reliability — an optimisation for the common case, not a guarantee.
This principle explains a great deal of the Internet's shape:
| Function | Where it belongs | Why |
|---|---|---|
| Reliability | Endpoints (TCP) | Only the endpoints know whether the application received the data |
| Encryption | Endpoints (TLS) | Link encryption leaves data in the clear at every intermediate node |
| Duplicate suppression | Endpoints | Only the endpoint knows what it has already processed |
| Congestion control | Endpoints, with network hints | Only the sender can slow down; ECN is the network's advisory input |
| Fast retransmit on Wi-Fi | Link layer, as optimisation | Correctness still rests with TCP above |
| Forwarding | Network | Endpoints do not know the topology |
Its second consequence is political as well as technical: keep the network simple and put intelligence at the edges. A simple network is cheap, fast, and — crucially — does not need to be upgraded when applications change. This is the technical foundation of the network-neutrality argument and the reason innovation on the Internet has historically not required operator permission.
The argument has real limits. Performance optimisations at lower layers are often essential rather than optional (Wi-Fi without link-layer ARQ is unusable). Some functions genuinely need network support: multicast, DDoS mitigation, and latency guarantees cannot be delivered from the edges alone. The argument tells you where correctness must live, not where every mechanism must live.
Examples
Why perfect links do not make a correct transfer
[ Disk A ] -> [ App A ] -> [ TCP ] -> R1 =====> R2 =====> [ TCP ] -> [ App B ] -> [ Disk B ]
^ ^
each link is checksummed and retransmitted
-- but corruption HERE, inside a router's
memory while copying between interfaces,
is invisible to every link-layer check
and regenerates a valid CRC on the way out.
Only a check computed by App A and verified by App B covers the whole path,
including the disks, the drivers, and the routers' internals.
import hashlib, random
def link_layer_crc_ok(chunk: bytes) -> bool:
return True # every link verified the chunk it carried: all good
def transfer(data: bytes, corrupt_in_router=False) -> bytes:
received = bytearray(data)
if corrupt_in_router:
i = random.randrange(len(received))
received[i] ^= 0x01 # a bit flip in router memory, between two links:
# after this, the NEXT link computes a fresh, VALID
# CRC over the already-corrupted bytes
return bytes(received)
original = b"the contents of an important file" * 100
delivered = transfer(original, corrupt_in_router=True)
print("every link-layer check passed:", link_layer_crc_ok(delivered))
# every link-layer check passed: True <- and yet:
print("end-to-end digests match:",
hashlib.sha256(original).hexdigest() == hashlib.sha256(delivered).hexdigest())
# end-to-end digests match: False <- only the endpoint check catches it
# The end-to-end argument as everyday practice: verify at the endpoints.
$ sha256sum ubuntu.iso # computed by the RECEIVER, over the final file
# a1f2...9c ubuntu.iso
$ cat SHA256SUMS | grep ubuntu.iso # published by the SENDER
# a1f2...9c ubuntu.iso
# TCP checksums, Ethernet CRCs, and TLS tags all passed on the way — and this
# check is still the only one that covers the disk write at both ends.
Key takeaways
- A function belongs at a lower layer only if it can be implemented completely and correctly there; otherwise it must be at the endpoints.
- Lower-layer versions of end-to-end functions are legitimate performance optimisations, never correctness guarantees.
- Careful file transfer is the canonical example: perfect links do not protect against corruption inside routers, drivers, or disks.
- The principle produces a simple network core with intelligence at the edges — cheap, fast, and evolvable without operator permission.
- It has limits: multicast, DDoS defence, and hard latency bounds genuinely require network participation.
🧪 Practice
- Wi-Fi retransmits lost frames at the link layer even though TCP already retransmits. Explain why this does not contradict the end-to-end argument.
- A company proposes putting virus scanning in its routers. Analyse the proposal using the end-to-end argument, naming at least two things the router cannot correctly determine.
- Interview-style: TCP already checksums every segment. Why do download pages still publish SHA-256 hashes? (Hint: enumerate everything in the path that TCP's checksum does not cover, at both ends.)
Protocol Stack Traversal Walkthrough
Theory
Everything in this chapter converges here. We trace a single HTTPS request from a laptop on a home Wi-Fi network to a web server, naming every layer action in order. If you can narrate this walkthrough from memory, you understand layering.
Scenario. Laptop 192.168.1.42 (MAC a4:83:e7:aa:bb:cc) on Wi-Fi. Home router 192.168.1.1 (MAC 4c:ed:fb:11:22:33), public address 198.51.100.7. Server example.com = 93.184.216.34:443.
Phase 1 — Name resolution (before any of it can start). The application has a name, not an address. The resolver sends a DNS query, itself a full stack traversal in miniature: application PDU → UDP datagram to port 53 → IP packet → frame to the router's MAC. The answer returns 93.184.216.34.
Phase 2 — Down the sending stack.
- Application. The browser builds
GET / HTTP/1.1\r\nHost: example.com\r\n\r\nand writes it to a socket. TLS encrypts it into a record first (presentation work). - Transport. TCP had already opened a connection (SYN, SYN-ACK, ACK — one RTT). It assigns a sequence number, prepends a 20-byte header with source port 51590 and destination port 443, and computes a checksum that — by sanctioned layer violation — covers a pseudo-header of the IP addresses. PDU: segment.
- Internet. IP prepends a 20-byte header: source
192.168.1.42, destination93.184.216.34, protocol 6 (the SAP for TCP), TTL 64. It consults the routing table: the destination is not on-link, so the next hop is the default gateway192.168.1.1. PDU: packet. - Link — address resolution. The link layer must address the next hop, not the destination. It needs the gateway's MAC. If not cached, ARP broadcasts "who has 192.168.1.1?" and the router answers. Note that ARP asks about the gateway's IP, never the server's.
- Link — framing. An 802.11 header is built with destination MAC = the router's, EtherType
0x0800, and an FCS trailer. PDU: frame. - Physical. Bits are modulated onto 5 GHz radio.
Phase 3 — Through the middle. The router receives the frame, verifies the FCS, strips the 802.11 header (decapsulation to layer 3), decrements TTL to 63, recomputes the IP header checksum, performs NAT — rewriting the source to 198.51.100.7:51590 and editing the TCP header, a deliberate layer violation — then looks up the destination, re-encapsulates in a new frame appropriate to the WAN link, and transmits. Every subsequent router repeats this: layer 2 is rebuilt at every hop; layer 3 survives; layer 4 is not touched (except by NAT).
Phase 4 — Up the receiving stack. The server's NIC validates the FCS, reads EtherType 0x0800 → IP. IP verifies the checksum, confirms the destination address is its own, reads Protocol 6 → TCP. TCP verifies its checksum, finds the socket matching the 4-tuple, places bytes in order in the receive buffer, and sends an ACK. The application reads the bytes, TLS decrypts them, and the HTTP server parses the request.
Examples
Full traversal with the header stack at each point
LAPTOP HOME ROUTER SERVER
------ ----------- ------
L7 HTTP request
L6 TLS record
L4 +TCP(51590->443)
L3 +IP(192.168.1.42 -> 93.184.216.34) TTL=64
L2 +802.11(dst=4c:ed:fb:11:22:33) <- the ROUTER's MAC, not the server's
L1 radio ================================>
L2 strip 802.11
L3 TTL 64->63, NAT src -> 198.51.100.7
L2 +Ethernet(dst = ISP router MAC) <- NEW frame
L1 fibre =========================>
... more hops, each rebuilding L2 ...
L2 strip, L3 verify,
L4 demux to :443 socket,
L6 TLS decrypt,
L7 HTTP parse
Unchanged end to end: the TCP payload.
Changed every hop: the entire layer 2 frame, plus TTL and IP checksum.
# Watch the same traversal on a real machine.
$ sudo tcpdump -i wlan0 -n -e -c 4 'host 93.184.216.34'
# 10:14:02.118 a4:83:e7:aa:bb:cc > 4c:ed:fb:11:22:33, ethertype IPv4 (0x0800), \
# length 74: 192.168.1.42.51590 > 93.184.216.34.443: Flags [S], seq 2847193, \
# win 64240, options [mss 1460,sackOK,TS,nop,wscale 7], length 0
# ^L2 src/dst MAC ^L2 SAP ^L3 addresses ^L4 ports ^L4 flags
#
# 10:14:02.206 4c:ed:fb:11:22:33 > a4:83:e7:aa:bb:cc, ethertype IPv4 (0x0800), \
# length 74: 93.184.216.34.443 > 192.168.1.42.51590: Flags [S.], seq 918273, \
# ack 2847194, win 65535, length 0
# ^ the reply's L2 SOURCE is the router again — the server's MAC never appears here
# Only the layer 3 addresses and layer 4 ports identify the real endpoints.
$ ip route get 93.184.216.34
# 93.184.216.34 via 192.168.1.1 dev wlan0 src 192.168.1.42
# ^^^ next hop (L2 target) ^^^ our L3 identity
# The demultiplexing chain on the receiving side, as explicit code.
def receive(frame: dict):
if frame["fcs_valid"] is False:
return "L2: FCS failed -> frame discarded silently" # no recovery here
if frame["ethertype"] != 0x0800:
return f"L2: unknown SAP 0x{frame['ethertype']:04x} -> not for IP"
pkt = frame["payload"]
if pkt["dst_ip"] != MY_IP:
return "L3: not addressed to me -> forward (router) or drop (host)"
if pkt["protocol"] != 6:
return f"L3: protocol {pkt['protocol']} -> not TCP"
seg = pkt["payload"]
key = (pkt["src_ip"], seg["src_port"], pkt["dst_ip"], seg["dst_port"])
return f"L4: delivered to socket {key}" # 4-tuple demux
MY_IP = "93.184.216.34"
print(receive({"fcs_valid": True, "ethertype": 0x0800,
"payload": {"dst_ip": "93.184.216.34", "src_ip": "198.51.100.7",
"protocol": 6,
"payload": {"src_port": 51590, "dst_port": 443}}}))
# L4: delivered to socket ('198.51.100.7', 51590, '93.184.216.34', 443)
Key takeaways
- A request descends the stack gaining a header per layer, crosses hops where only layer 2 is rebuilt, and ascends the peer's stack shedding headers in reverse.
- Layer 2 addresses the next hop only; layer 3 addresses the true endpoints; the server's MAC never appears on the client's link.
- Each upward step is a SAP lookup: EtherType → protocol number → port → socket 4-tuple.
- Routers touch layers 1–3 (decrement TTL, recompute the IP checksum, re-frame); NAT additionally edits layer 4, violating layering.
- DNS resolution is a complete stack traversal that must finish before the main one can begin.
🧪 Practice
- Redo the walkthrough for a DNS query over UDP/IPv6, naming the SAP value used at each boundary.
- At hop 5 of the path, list exactly which header fields have changed since the packet left the laptop and which have not.
- Interview-style: You type a URL and press Enter. Describe what happens, layer by layer. (Hint: the strongest answers begin before any packet exists — with name resolution and the routing-table decision — and explicitly separate what changes per hop from what is end to end.)
<a id="24-standards-and-governance"></a>
2.4 Standards and Governance
Protocols only work because independent organisations agree on them; this subchapter covers who writes the rules, how a specification becomes a standard, and who hands out the numbers and names that make global addressing possible.
IETF, RFCs, and the Standards Track
Theory
Nobody owns the Internet, which raises an obvious question: how does anything ever get agreed? The answer is the IETF (Internet Engineering Task Force) — an open, volunteer, individual-membership body with no formal membership requirement at all. You do not join the IETF; you participate by joining a mailing list. Decisions are made by rough consensus and running code, a phrase from David Clark's 1992 talk that remains the organisation's credo: no formal voting, and a working implementation outweighs an elegant argument.
Work happens in working groups, each with a narrow charter, organised into areas (routing, security, transport, applications, and so on) overseen by the IESG (Internet Engineering Steering Group). The IAB (Internet Architecture Board) provides architectural oversight and appeals; the IRTF does longer-horizon research; the RFC Editor publishes the documents.
The output is the RFC ("Request for Comments") — a name that is a historical accident from 1969, when Steve Crocker chose a deliberately humble title to avoid seeming presumptuous. RFCs are numbered sequentially, are never revised once published, and are never withdrawn; a newer RFC obsoletes or updates an older one, and the old text remains available forever.
Not every RFC is a standard. The category matters:
| Category | Meaning | Example |
|---|---|---|
| Standards Track | Intended to become an Internet Standard | RFC 9293 (TCP), RFC 9110 (HTTP Semantics) |
| Informational | Useful information, no standing | RFC 1149 (IP over avian carriers) |
| Experimental | Unproven; deploy with care | Many early congestion-control proposals |
| Best Current Practice (BCP) | Operational or process guidance | RFC 2119 (MUST/SHOULD keywords), BCP 38 |
| Historic | Superseded or abandoned | RFC 821 (original SMTP) |
The standards track itself has, since RFC 6410 (2011), only two maturity levels rather than three — the old Draft Standard step was removed because almost nothing ever advanced past it:
Internet-Draft -> Proposed Standard -> Internet Standard
(expires in 6 (IESG-approved, (demonstrated interoperability
months; not a stable, implementable) and wide deployment; given an
published RFC) additional STD number)
In practice Proposed Standard is where deployment happens — HTTP/2 and TLS 1.3 are used universally while sitting at that level. Treating "only a Proposed Standard" as immature misreads the process.
One RFC you must know by number: RFC 2119 defines the requirement keywords. When a specification says MUST, MUST NOT, SHOULD, SHOULD NOT, or MAY in capitals, these have exact meanings — SHOULD means "there may be valid reasons to ignore this, but understand the consequences first", which is very different from MAY.
Examples
The life of a specification
idea on a mailing list
|
draft-ietf-quic-transport-00 <- Internet-Draft, expires in 6 months
| (working group iterations, interop test events, revisions -34)
IETF Last Call + IESG review
|
RFC 9000 Proposed Standard <- permanent number, immutable text
|
widespread deployment + interop
|
possible Internet Standard (STD nn)
Relationships between RFCs:
RFC 9293 (TCP, 2022) OBSOLETES RFC 793 (1981) and 8 others
RFC 793 is now Historic
The text of RFC 793 still exists and is still citable.
# RFCs are plain text and permanently addressable — fetch and inspect one.
$ curl -s https://www.rfc-editor.org/rfc/rfc2119.txt | head -14
# Network Working Group S. Bradner
# Request for Comments: 2119 Harvard University
# BCP: 14 <- also a Best Current Practice
# Category: Best Current Practice March 1997
#
# Key words for use in RFCs to Indicate Requirement Levels
# Search the index for every standards-track RFC about QUIC:
$ curl -s https://www.rfc-editor.org/rfc-index.txt | grep -i 'quic' | head -3
# 9000 QUIC: A UDP-Based Multiplexed and Secure Transport. ... Standards Track
# 9001 Using TLS to Secure QUIC. ... Standards Track
# 9002 QUIC Loss Detection and Congestion Control. ... Standards Track
# RFC 2119 keywords are a formal conformance vocabulary, not English emphasis.
REQUIREMENT = {
"MUST": "absolute requirement; an implementation ignoring it is non-conformant",
"MUST NOT": "absolute prohibition",
"SHOULD": "ignore only with full understanding of the consequences",
"SHOULD NOT": "discouraged; valid exceptions exist",
"MAY": "truly optional; peers must interoperate whether or not it is done",
}
spec = "A sender MUST NOT send a segment larger than the MSS, and SHOULD use PMTUD."
for kw in ("MUST NOT", "SHOULD"):
if kw in spec:
print(f"{kw:<10} -> {REQUIREMENT[kw]}")
# MUST NOT -> absolute prohibition
# SHOULD -> ignore only with full understanding of the consequences
Key takeaways
- The IETF is open and volunteer-based, operating by rough consensus and running code, with no formal membership or voting.
- RFCs are permanent and immutable; newer RFCs obsolete or update older ones rather than replacing their text.
- RFC categories: Standards Track, Informational, Experimental, Best Current Practice, and Historic.
- The standards track has two levels since RFC 6410: Proposed Standard and Internet Standard; most deployed protocols never leave Proposed Standard.
- RFC 2119 keywords (MUST, SHOULD, MAY) carry precise conformance meanings when capitalised.
🧪 Practice
- Look up RFC 9293 and list which RFCs it obsoletes. What does this tell you about how long the original TCP specification remained authoritative?
- Rewrite this sentence using RFC 2119 keywords correctly: "Implementations really ought to validate the certificate, and it would be nice if they logged failures."
- Interview-style: TLS 1.3 is only a "Proposed Standard" yet secures most of the web. Is that a problem? (Hint: think about what advancement to Internet Standard actually requires and why maturity levels stopped tracking real-world use.)
IEEE 802 Standards Family
Theory
The IETF standardises from the internet layer upward. Below that — the physical and data link layers of local and metropolitan networks — the authority is the IEEE 802 LAN/MAN Standards Committee, formed in February 1980 (hence "802"). The division of labour is clean and worth internalising: IEEE builds the links, IETF builds the internetwork.
IEEE 802 split layer 2 into the LLC and MAC sublayers described earlier, which is why 802.2 (LLC) is medium-independent while 802.3 and 802.11 each define their own MAC plus physical layer.
The working groups you will actually meet:
| Standard | Subject | Notes |
|---|---|---|
| 802.1 | Bridging, architecture | 802.1Q VLAN tagging, 802.1X port authentication, 802.1D/w/s spanning tree, 802.1AX link aggregation |
| 802.2 | Logical Link Control | Largely historical; Ethernet II framing won instead |
| 802.3 | Ethernet | Wired LAN: 10BASE-T through 800GBASE-R, plus Power over Ethernet (802.3af/at/bt) |
| 802.11 | Wireless LAN (Wi-Fi) | a/b/g/n/ac/ax/be — marketed as Wi-Fi 4/5/6/7 |
| 802.15 | Wireless PAN | 802.15.1 Bluetooth origins, 802.15.4 the basis of Zigbee and Thread |
| 802.16 | Broadband wireless (WiMAX) | Effectively superseded by LTE |
| 802.5 | Token Ring | Historic; disbanded |
Two naming conventions repay learning.
Ethernet physical-layer names encode their own specification. 1000BASE-T decomposes as speed (1000 Mb/s) + signalling (BASEband, as opposed to broadband) + medium (T = twisted pair, SX/LX = short/long-wavelength fibre, CX = copper twinax). So 10GBASE-LR is 10 Gb/s baseband over long-reach single-mode fibre.
Wi-Fi generation numbers are marketing, not IEEE. The Wi-Fi Alliance — a separate trade association that runs certification and owns the "Wi-Fi" trademark — introduced "Wi-Fi 6" for 802.11ax because consumers could not rank letter suffixes. Both names refer to the same IEEE document.
A recurring exam point: 802.11 is not "wireless Ethernet". It shares the 48-bit MAC address space and presents the same service to IP, which is exactly why they interoperate through a bridge — but its MAC is fundamentally different (CSMA/CA with acknowledgements rather than CSMA/CD, a four-address frame format, association and authentication state). The service matches; the protocol does not.
Examples
Where IEEE stops and IETF starts
+-----------------------------+
| Application (IETF: RFCs) |
| Transport (IETF: RFCs) |
| Internet (IETF: RFCs) | <- IP, TCP, HTTP, DNS ...
+=============================+ <- the handover point
| LLC 802.2 |
| MAC 802.3 / 802.11 / ... | <- IEEE 802
| PHY 802.3 / 802.11 / ... |
+-----------------------------+
Decoding an Ethernet PHY name:
10G BASE - L R
| | | |
speed baseband long reach/encoding
(Mb/s wavelength (single-mode fibre)
or Gb/s)
# Parse IEEE Ethernet media names and map Wi-Fi standards to marketing names.
MEDIUM = {"T": "twisted pair copper", "SR": "short-reach multimode fibre",
"LR": "long-reach single-mode fibre", "CR": "twinaxial copper"}
def decode_ethernet(name: str) -> str:
speed, rest = name.split("BASE-")
return f"{name:<12} {speed:>5} baseband over {MEDIUM.get(rest, 'other medium')}"
for n in ["1000BASE-T", "10GBASE-SR", "100GBASE-LR", "25GBASE-CR"]:
print(decode_ethernet(n))
# 1000BASE-T 1000 baseband over twisted pair copper
# 10GBASE-SR 10G baseband over short-reach multimode fibre
# 100GBASE-LR 100G baseband over long-reach single-mode fibre
# 25GBASE-CR 25G baseband over twinaxial copper
WIFI = {"802.11n": "Wi-Fi 4", "802.11ac": "Wi-Fi 5",
"802.11ax": "Wi-Fi 6", "802.11be": "Wi-Fi 7"}
# IEEE assigns the left column; the Wi-Fi Alliance markets the right one.
# Two IEEE 802 standards visible side by side on one machine.
$ iw dev wlan0 info
# Interface wlan0
# addr a4:83:e7:aa:bb:cc <- 48-bit MAC: shared IEEE address space
# type managed
# channel 36 (5180 MHz), width: 80 MHz <- 802.11 PHY parameters
$ bridge vlan show
# port vlan-id
# eth0 10 PVID Egress Untagged <- 802.1Q VLAN tagging
# eth1 20
Key takeaways
- IEEE 802 standardises physical and data link layers for LANs and MANs; the IETF standardises from the internet layer up.
- Key groups: 802.1 (bridging, VLANs, 802.1X), 802.3 (Ethernet), 802.11 (Wi-Fi), 802.15 (WPAN, Bluetooth and Zigbee lineage).
- Ethernet media names encode speed, signalling type, and medium:
10GBASE-LR= 10 Gb/s baseband over long-reach single-mode fibre. - Wi-Fi generation numbers come from the Wi-Fi Alliance, a trade group, not from IEEE; 802.11ax and Wi-Fi 6 are the same thing.
- 802.11 shares MAC addressing with Ethernet and offers the same service to IP, but its MAC protocol is entirely different.
🧪 Practice
- Decode
40GBASE-SR4and100BASE-TXinto speed, signalling, and medium.- State which body — IEEE or IETF — owns each: VLAN tags, IP fragmentation, Power over Ethernet, TCP window scaling, Wi-Fi channel widths.
- Interview-style: A frame arrives on Wi-Fi and leaves on Ethernet through a bridge, unchanged above layer 2. What makes this possible? (Hint: identify what the two standards deliberately share and what they do not.)
ITU-T, ISO, and W3C Roles
Theory
The IETF and IEEE are not the only bodies with authority over things you use daily. Three others shape large parts of the landscape, and each has a distinct character worth understanding — because the kind of organisation predicts the kind of standard it produces.
ITU-T (International Telecommunication Union, Telecommunication Standardization Sector). The ITU is a United Nations agency, founded in 1865 as the International Telegraph Union — older than the telephone. Its members are nation states plus sector members, which makes it the most formal and the most political of the bodies here. Its outputs are called Recommendations, organised into lettered series:
| Series | Subject | Examples |
|---|---|---|
| G | Transmission systems, digital networks | G.711 (voice codec), G.992 (ADSL), G.9701 (G.fast) |
| H | Audiovisual and multimedia | H.264/AVC, H.265/HEVC (jointly with ISO MPEG), H.323 |
| V | Data communication over telephone networks | V.90, V.92 (dial-up modems) |
| X | Data networks and open system communication | X.25, X.500, X.509 (the certificate format in every TLS handshake) |
| Y | Next-generation networks, IoT | Y.2060 (IoT overview) |
The ITU also allocates radio spectrum globally through ITU-R and the World Radiocommunication Conference — which is why Wi-Fi operates in the bands it does.
ISO (International Organization for Standardization), with IEC for electrotechnical work. A network of national standards bodies (ANSI, BSI, DIN, and so on), not governments and not individuals. ISO gave networking the OSI model itself (ISO/IEC 7498-1), and also adopted IEEE 802 as the ISO/IEC 8802 series. Its wider relevance today is in management-system standards such as ISO/IEC 27001 for information security. Note the name is not an acronym — "ISO" is derived from the Greek isos, "equal", so it is the same in every language.
W3C (World Wide Web Consortium). Founded by Tim Berners-Lee in 1994 at MIT; members are organisations. It standardises the web above HTTP: HTML, CSS, DOM, SVG, XML, WebAssembly, and the WCAG accessibility guidelines. Its outputs progress through Working Draft → Candidate Recommendation → Proposed Recommendation → W3C Recommendation.
One complication students routinely get wrong: HTML is no longer standardised by the W3C alone. After years of divergence, a 2019 memorandum of understanding made WHATWG's HTML Living Standard the single authoritative version, with W3C contributing rather than publishing a competing snapshot. "Living standard" means continuously updated with no version numbers — the opposite of the RFC model. Meanwhile JavaScript is ECMA International (ECMA-262), not W3C at all.
| Body | Membership | Output name | Style | Domain |
|---|---|---|---|---|
| IETF | Individuals, open | RFC | Rough consensus, running code | Internet protocols |
| IEEE 802 | Individuals, paid | Standard | Formal ballot | LAN/MAN links |
| ITU-T | Nation states | Recommendation | Treaty-like, formal | Telecom, codecs, spectrum |
| ISO/IEC | National bodies | International Standard | Formal ballot | Models, management systems |
| W3C | Organisations | Recommendation | Consensus + patent policy | Web technologies |
| WHATWG | Browser vendors | Living Standard | Continuous revision | HTML, DOM, Fetch |
Examples
One video call, five standards bodies
H.264 video encoding ............... ITU-T (with ISO/IEC MPEG)
G.711 / Opus audio ................. ITU-T / IETF
WebRTC APIs in the browser ......... W3C
SRTP, ICE, STUN, TURN transport .... IETF (RFCs)
Wi-Fi carrying it all .............. IEEE 802.11
Spectrum the Wi-Fi uses ............ ITU-R + national regulators
No single body could have produced this; the layering is what lets
them each own a piece without coordinating with the others.
# An ITU-T Recommendation you can read out of any TLS connection.
$ openssl s_client -connect example.com:443 </dev/null 2>/dev/null \
| openssl x509 -noout -subject -issuer -dates
# subject=CN = example.com
# issuer=C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
# notBefore=Jan 30 00:00:00 2026 GMT
# notAfter=Mar 1 23:59:59 2027 GMT
# The certificate's structure is ITU-T X.509, encoded in ISO/IEC ASN.1 DER,
# carried by an IETF protocol (TLS), to display an W3C-standardised web page.
# Which body owns which piece of a web request?
ownership = {
"HTML parsing": "WHATWG (Living Standard), with W3C participation",
"CSS": "W3C",
"JavaScript language": "ECMA International (ECMA-262)",
"HTTP/2, HTTP/3": "IETF (RFC 9113, RFC 9114)",
"TLS 1.3": "IETF (RFC 8446)",
"X.509 certificates": "ITU-T (profiled for the Internet by IETF RFC 5280)",
"Ethernet frame": "IEEE 802.3",
"OSI reference model": "ISO/IEC 7498-1",
}
for item, body in ownership.items():
print(f"{item:<22} {body}")
# Note X.509: defined by ITU-T, then PROFILED by the IETF — bodies build on
# each other rather than duplicating work.
Key takeaways
- ITU-T is a UN agency with state members producing Recommendations (G, H, V, X, Y series) covering telecom, codecs, X.509, and — via ITU-R — spectrum.
- ISO/IEC is a federation of national standards bodies; it published the OSI model (ISO/IEC 7498-1) and adopted IEEE 802 as ISO/IEC 8802.
- W3C standardises web technologies above HTTP; its output is a Recommendation.
- HTML is now the WHATWG Living Standard, not a versioned W3C specification, and JavaScript belongs to ECMA International.
- Bodies build on each other's work: the IETF profiles ITU-T's X.509 for Internet use rather than reinventing it.
🧪 Practice
- Assign the correct body to each:
ISO/IEC 27001,RFC 8446,802.1Q,H.265,WCAG 2.2,ECMA-262.- Explain why the ITU's membership model makes its standards process slower and more political than the IETF's, and name one domain where that formality is an advantage.
- Interview-style: Why does an X.509 certificate carry an ITU-T name when it is used almost exclusively by an IETF protocol? (Hint: consider the era X.500 directories were designed for and what "profiling" an existing standard means.)
IANA and ICANN Responsibilities
Theory
Protocols need agreed numbers, and names need to be unique. Port 443 must mean HTTPS everywhere or nothing works; example.com must resolve to the same place for everyone or the web fractures. Somebody has to be the single authoritative registry — and because uniqueness is the entire point, that function cannot be distributed away.
IANA (Internet Assigned Numbers Authority) is that registry. It began as essentially one person: Jon Postel, who maintained the numbers by hand from the 1970s until his death in 1998. Today it is operated by PTI (Public Technical Identifiers), an affiliate of ICANN. IANA has exactly three functions:
| Function | What it covers |
|---|---|
| Protocol parameters | Every registry an RFC creates: port numbers, IP protocol numbers, EtherTypes, URI schemes, media types, DNS record types, HTTP status codes and headers, TLS cipher suites |
| Number resources | Allocating large IP address blocks and AS numbers to the five RIRs (never to end users) |
| Root zone management | The DNS root zone file — which nameservers are authoritative for .com, .uk, .org, and every other TLD |
ICANN (Internet Corporation for Assigned Names and Numbers), founded in 1998, is the policy organisation around IANA. It is a California non-profit operating a multistakeholder model: governments, businesses, technical bodies, and civil society participate through supporting organisations (GNSO for generic names, ccNSO for country codes, ASO for addresses) and advisory committees (GAC for governments, SSAC for security, RSSAC for root servers). ICANN accredits registrars, contracts with registries, runs the gTLD expansion programme, and administers the UDRP dispute process for trademark conflicts in domain names.
The relationship in one sentence: ICANN makes the policy; IANA performs the function. Until October 2016 IANA operated under a contract with the US Department of Commerce's NTIA; the IANA stewardship transition ended that contract and moved oversight to the global multistakeholder community — the most significant governance change in the Internet's history, and a standard exam question.
What neither of them does is equally important: ICANN does not control content, does not run the Internet, and cannot take a website down. It coordinates unique identifiers. Confusing identifier coordination with governance of the network is the most common misconception in this area.
Examples
Who assigns what
+---------------------+
| ICANN | policy, multistakeholder
+----------+----------+
|
+----------v----------+
| IANA (via PTI) | performs the function
+----+---------+------+
| | \
protocol | | \ root zone
parameters | | \ (delegations for
(ports, proto | | .com, .uk, .org ...)
numbers, ...) | |
| number resources
| (large blocks -> RIRs only)
v
RIRs: ARIN | RIPE NCC | APNIC | LACNIC | AFRINIC
|
ISPs / LIRs
|
end users
# IANA registries are the reason these files exist on every Unix machine.
$ grep -E '^(https|ssh|domain)\b' /etc/services
# ssh 22/tcp # IANA well-known port
# domain 53/tcp
# domain 53/udp
# https 443/tcp
$ grep -E '^(icmp|tcp|udp)\b' /etc/protocols
# icmp 1 ICMP # IANA IP protocol numbers -> the IPv4 Protocol field
# tcp 6 TCP
# udp 17 UDP
# The root zone IANA maintains, queried directly:
$ dig +short NS . | head -3
# a.root-servers.net.
# b.root-servers.net.
# c.root-servers.net.
$ dig +short NS com. | head -2
# a.gtld-servers.net. <- this delegation is an IANA root zone entry
# b.gtld-servers.net.
# Port ranges are an IANA policy, visible in any socket program.
RANGES = [(0, 1023, "Well-known: IANA assigned, privileged to bind on Unix"),
(1024, 49151, "Registered: IANA registered on request"),
(49152, 65535, "Dynamic/ephemeral: never assigned, OS picks freely")]
def classify(port: int) -> str:
for lo, hi, desc in RANGES:
if lo <= port <= hi:
return f"{port:>5} {desc}"
for p in (22, 443, 3306, 8080, 51590):
print(classify(p))
# 22 Well-known: IANA assigned, privileged to bind on Unix
# 443 Well-known: IANA assigned, privileged to bind on Unix
# 3306 Registered: IANA registered on request
# 8080 Registered: IANA registered on request
# 51590 Dynamic/ephemeral: never assigned, OS picks freely
Key takeaways
- IANA maintains three registries: protocol parameters, number resources allocated to RIRs, and the DNS root zone.
- ICANN sets policy through a multistakeholder model; IANA, operated by PTI, performs the technical function.
- The IANA stewardship transition in October 2016 ended US government contractual oversight in favour of community stewardship.
- IANA allocates IP blocks only to the five RIRs, never directly to end users.
- ICANN coordinates unique identifiers; it does not govern content, run networks, or remove websites.
- Port ranges — well-known, registered, and ephemeral — are IANA policy visible in
/etc/services.
🧪 Practice
- Find three IANA registries you rely on without thinking about them and name the protocol field each one populates.
- Explain why the root zone cannot be maintained by a distributed consensus of operators, referring to what uniqueness requires.
- Interview-style: A government asks ICANN to take a website offline. What can ICANN actually do? (Hint: distinguish the domain delegation from the hosting, and identify which party ICANN has a contract with.)
Regional Internet Registries
Theory
IANA allocates address space in enormous blocks, but it does not deal with ISPs or companies. In between sits a tier of five Regional Internet Registries (RIRs), non-profit membership organisations that manage number resources for a defined geography. The hierarchy is strictly three-level: IANA → RIR → LIR/ISP → end user.
| RIR | Region | Established |
|---|---|---|
| RIPE NCC | Europe, Middle East, Central Asia | 1992 |
| APNIC | Asia-Pacific | 1993 |
| ARIN | United States, Canada, parts of the Caribbean | 1997 |
| LACNIC | Latin America and the Caribbean | 2002 |
| AFRINIC | Africa | 2004 |
They coordinate globally through the NRO (Number Resource Organization) and collectively form ICANN's Address Supporting Organization. Each develops its own allocation policy through open, bottom-up Policy Development Processes — anyone may propose a policy, and regional policies genuinely differ.
RIR responsibilities:
- Allocating IPv4 and IPv6 address blocks to members (LIRs — Local Internet Registries — typically ISPs and large enterprises).
- Assigning Autonomous System Numbers used by BGP, introduced in Chapter 1.
- Maintaining the public WHOIS/RDAP databases that record who holds which resource — the basis for abuse reporting and network troubleshooting.
- Operating RPKI certificate authorities, issuing ROAs (Route Origin Authorizations) that cryptographically state which AS may originate a prefix — the foundation of modern BGP hijack defence.
- Running reverse DNS delegation for their address space.
The defining event in RIR history is IPv4 exhaustion. The IANA free pool ran out on 3 February 2011. The RIRs then depleted their own pools in turn: APNIC in April 2011, RIPE NCC in September 2012, LACNIC in June 2014, ARIN in September 2015, and AFRINIC in 2017. Each adopted austerity policies — RIPE NCC's "last /8" policy limited each member to a single /22 (1,024 addresses), and even that ran out on 25 November 2019, after which RIPE moved to a waiting list fed only by returned space.
Two consequences shape today's Internet. First, an IPv4 transfer market emerged: addresses are now bought and sold, with RIR-recorded transfers and prices that have climbed into the tens of dollars per address. Second, exhaustion is the direct economic reason for carrier-grade NAT and the continuing push for IPv6, which has 2¹²⁸ addresses and needs no scarcity policy at all.
Examples
The delegation chain for one address, top to bottom
IANA allocates 93.0.0.0/8 ------------------> RIPE NCC
RIPE NCC allocates 93.184.0.0/16 ---------------> an LIR (ISP)
LIR assigns 93.184.216.0/24 ------------> the customer
Customer uses 93.184.216.34 --------------> one host
Each step is recorded in WHOIS and, for routing, in an RPKI ROA:
prefix 93.184.216.0/24, origin AS15133, max length /24
-> any BGP announcement of this prefix from another AS is INVALID
# Trace an address back up the delegation chain yourself.
$ whois 93.184.216.34 | grep -iE '^(inetnum|netname|org|country|source)'
# inetnum: 93.184.216.0 - 93.184.216.255
# netname: EDGECAST-NETBLK-03
# country: EU
# source: RIPE <- which RIR holds the delegation
$ whois -h whois.radb.net ' -i origin AS15133' | head -4 # which prefixes an AS holds
# route: 93.184.216.0/24
# origin: AS15133
# Modern replacement for WHOIS: RDAP returns structured JSON.
$ curl -s https://rdap.db.ripe.net/ip/93.184.216.34 | head -6
# {"handle":"93.184.216.0 - 93.184.216.255","startAddress":"93.184.216.0",
# "endAddress":"93.184.216.255","ipVersion":"v4","name":"EDGECAST-NETBLK-03",
# "type":"ASSIGNED PA", ...
# Why exhaustion was arithmetic, not mismanagement.
IPV4_TOTAL = 2**32
IPV4_USABLE = IPV4_TOTAL - 2**24*16 - 2**20*16 - 2**16*256 # minus 10/8, 172.16/12,
# 192.168/16 and similar
IPV6_TOTAL = 2**128
print(f"IPv4 total : {IPV4_TOTAL:,}")
print(f"IPv4 usable : ~{IPV4_USABLE:,}")
print(f"World popul. : ~8,100,000,000")
print(f"addresses per person: {IPV4_USABLE/8.1e9:.2f}")
# IPv4 total : 4,294,967,296
# IPv4 usable : ~4,043,309,056
# addresses per person: 0.50
# Half an address each — before counting phones, servers, and IoT devices.
print(f"IPv6 per person: {IPV6_TOTAL/8.1e9:.2e}")
# IPv6 per person: 4.20e+28 <- scarcity policy becomes unnecessary
Key takeaways
- Five RIRs — ARIN, RIPE NCC, APNIC, LACNIC, AFRINIC — allocate number resources within defined regions, coordinated by the NRO.
- The delegation hierarchy is IANA → RIR → LIR/ISP → end user, with each step recorded in WHOIS or RDAP.
- RIRs also assign AS numbers, run reverse DNS delegation, and operate the RPKI CAs that issue ROAs for BGP origin validation.
- The IANA IPv4 free pool was exhausted on 3 February 2011; individual RIRs followed between 2011 and 2017.
- Exhaustion produced austerity policies, waiting lists, a commercial IPv4 transfer market, carrier-grade NAT, and the economic case for IPv6.
🧪 Practice
- Run
whoison your own public IP address. Identify the RIR, the LIR that holds the allocation, and the AS number announcing it.- Explain how an RPKI ROA would have prevented a BGP hijack in which an unrelated AS announced someone else's
/24.- Interview-style: IPv6 was ready in 1998 but IPv4 exhaustion still caused a crisis. Why was adoption so slow? (Hint: think about the narrow waist, who pays the cost of upgrading, and who receives the benefit.)
<a id="3-the-physical-layer"></a>