2. Processes

The process is the operating system's central abstraction: the unit of execution, of resource ownership, and of protection, and the thing that every other mechanism in this book ultimately serves or constrains. This chapter defines what a process is as opposed to a program, examines the data structures and memory layout that give it substance, follows it through creation and termination, and ends with the mechanisms by which processes — deliberately isolated from one another — communicate anyway. Understanding processes precisely makes scheduling, memory management, and security follow naturally.

<a id="21-process-concept"></a>

2.1 Process Concept

This section builds the mental model: what a process actually is, how its memory is arranged, what states it moves through, what the kernel records about it, and what happens when the CPU switches from one to another.

Program vs Process

Theory

The distinction seems pedantic until it becomes the source of confusion, so it is worth nailing down first.

A program is a passive entity: a file on disk containing instructions, initialised data, and metadata describing how to load it. It has no state, consumes no CPU, and can sit unchanged for years. /bin/ls is a program.

A process is an active entity: a program in execution, together with everything that execution needs — the current values of the registers, a program counter marking the next instruction, a stack of pending function calls, a heap of dynamically allocated memory, a set of open files, an identity (which user is running it), and the accounting the kernel keeps about it.

The relationship is one-to-many, in both directions:

• One program, many processes. Ten users running ls simultaneously produce ten processes from one program file. They share the read-only instruction pages in physical memory but have entirely separate stacks, heaps, and register sets. Nothing one does affects another.
• One process, many programs over its lifetime. A process can replace its program image (via execve) while keeping its identity, open files, and relationships — the basis of how shells work.

Analogy: a program is a recipe; a process is a cook actively making the dish. The recipe is one document in the drawer; three cooks can work from copies of it simultaneously, each at a different step, each with their own pans and half-chopped onions. Burning one dish does not affect the others, and a cook could put down one recipe and pick up another without ceasing to be that cook.

The practical payoff: when reasoning about a running system, never ask "what does the program do" but "what state is this process in". Bugs, resource usage, and scheduling all attach to the process, not to the file.

Example

   ON DISK (passive)                  IN MEMORY (active)

   /bin/ls                            process 4412 (user alice)
   +------------------+                 PC=0x4011a8  registers, stack, heap
   | ELF header       |    execve       open files: 0,1,2  cwd=/home/alice
   | .text  (code)    |  ----------->
   | .data  (init'd)  |               process 4419 (user bob)
   | .rodata          |                 PC=0x401340  its own stack and heap
   | symbol table     |                 open files: 0,1,2,7  cwd=/tmp
   +------------------+
     ONE file                          TWO independent processes
                                       (the .text pages are shared read-only)

#include <stdio.h>
#include <unistd.h>

/* This same program, run three times, yields three processes with
   three different PIDs and three independent copies of `counter`. */
int counter = 0;                 /* .data section in the FILE;
                                    a private, writable page in each PROCESS */
int main(void) {
    printf("program image: same for everyone\n");
    printf("process identity: pid=%d, parent=%d\n", getpid(), getppid());
    counter++;                   /* each process increments ITS OWN copy */
    printf("counter=%d\n", counter);   /* always 1, no matter how many run */
    return 0;
}

./demo & ./demo & ./demo &   # one program file, three processes
ps -o pid,ppid,cmd -C demo   # three distinct PIDs, one command

Key takeaways

• A program is a passive file on disk; a process is an active execution with state.
• One program can back many concurrent processes, each with private data, stack, and registers.
• One process can execute several programs over its lifetime while keeping its identity and open files.
• Resource usage, scheduling, and faults are properties of processes, never of program files.

🧪 Practice

1. Start three copies of the same long-running program and use ps -o pid,ppid,rss,cmd to show that they are distinct processes. Which columns differ, and which do not?
2. Explain why two processes running /bin/bash can have different current directories despite identical program images.
3. Interview-style: "If ten processes run the same 4 MB program, how much physical memory does the code occupy?" Hint: distinguish read-only text pages from writable data pages, and think about page sharing.

Process Memory Layout

Theory

Every process gets its own virtual address space — a private, apparently contiguous range of addresses. That space is not undifferentiated; it is divided into regions with different contents, permissions, and growth behaviour, and knowing the layout explains a large fraction of real-world bugs.

From low addresses upward:

Region	Contents	Permissions	Size behaviour
Text (code)	Machine instructions	Read + execute, not writable	Fixed at load time
Data	Initialised globals and statics	Read + write	Fixed
BSS	Uninitialised globals (zero-filled)	Read + write	Fixed; occupies no space in the file
Heap	Dynamic allocations (malloc, new)	Read + write	Grows upward on demand
Memory maps	Shared libraries, mapped files, large allocations	Varies	Placed between heap and stack
Stack	Call frames: locals, return addresses, saved registers	Read + write	Grows downward automatically

Three design decisions in this layout are worth understanding:

• Heap and stack grow toward each other from opposite ends of the free space. This lets a program be heap-heavy or stack-heavy without the OS having to guess the split in advance.
• Text is read-only and executable; stack and heap are writable and (on modern systems) non-executable. The W^X ("write xor execute") rule is what stops classic code injection: an attacker who writes shellcode onto the stack finds that page is not executable.
• BSS exists to keep executables small. A 100 MB zeroed array costs zero bytes in the file — the loader just maps zero-filled pages.

The classic failures map directly onto the layout: a stack overflow is deep recursion growing the stack into a guard page; a memory leak is a heap that grows without bound; a segmentation fault is touching an address in no mapped region; and writing through a pointer to a string literal faults because literals live in read-only memory.

Example

   Virtual address space of one process (typical x86-64 Linux)

   0x7fff_ffff_ffff  +------------------------+  high addresses
                     |  kernel space (not     |  mapped but inaccessible
                     |  reachable in ring 3)  |  from user mode
   0x7fff_ffff_0000  +------------------------+
                     |  environment, argv     |
                     +------------------------+
                     |  STACK                 |  locals, return addresses
                     |          |             |  grows DOWNWARD
                     |          v             |
                     +------------------------+
                     |  (guard page)          |  faults on overflow
                     |                        |
                     |  ~~~ free space ~~~    |
                     |                        |
                     +------------------------+
                     |  memory maps           |  libc.so, mmap'd files,
                     |  (shared libraries)    |  big malloc allocations
                     +------------------------+
                     |          ^             |
                     |          |             |  grows UPWARD
                     |  HEAP                  |  malloc / new
                     +------------------------+  <- program break (brk)
                     |  BSS (zero-filled)     |  uninitialised globals
                     +------------------------+
                     |  DATA                  |  initialised globals
                     +------------------------+
                     |  TEXT (code)           |  read + execute only
   0x0000_0000_4000  +------------------------+
   0x0000_0000_0000  |  unmapped              |  null-pointer traps land here
                     +------------------------+

#include <stdio.h>
#include <stdlib.h>

int   initialised   = 42;      /* DATA: value stored in the executable file  */
int   uninitialised;           /* BSS:  no file space, zeroed at load time   */
const char *literal = "hi";    /* the STRING lives in read-only text/rodata  */

void recurse(int depth) {
    char frame[1024];          /* 1 KB of STACK per call: recursion is finite */
    frame[0] = (char)depth;
    if (depth < 100000) recurse(depth + 1);   /* eventually: stack overflow   */
}

int main(void) {
    int local = 1;                     /* STACK */
    int *dynamic = malloc(sizeof(int)); /* pointer on the stack, data on HEAP */

    printf("text     ~ %p\n", (void *)main);
    printf("data     ~ %p\n", (void *)&initialised);
    printf("bss      ~ %p\n", (void *)&uninitialised);
    printf("heap     ~ %p\n", (void *)dynamic);
    printf("stack    ~ %p\n", (void *)&local);
    /* Addresses print in increasing order: text < data < bss < heap < stack */

    /* literal[0] = 'H';  <- would segfault: rodata is not writable */
    free(dynamic);
    return 0;
}

cat /proc/self/maps      # the kernel's own view of a process's regions
                         # columns: range, perms (r-xp, rw-p), offset, path

Key takeaways

• A process's address space is divided into text, data, BSS, heap, memory maps, and stack.
• Heap grows upward and stack downward, letting either dominate the free space between them.
• Text is read-execute and stack/heap are write-no-execute, which blocks classic code injection.
• Most memory bugs are layout violations: stack overflow, leaks, null dereference, writing to read-only literals.

🧪 Practice

1. Run the program above and confirm the address ordering. Then run it twice — do the addresses match? Explain (consider ASLR).
2. Inspect cat /proc/self/maps. Identify the text, heap, stack, and at least one shared library, using the permission column as evidence.
3. Interview-style: "Why does declaring int big[10000000]; as a global produce a small executable, but as a local variable it crashes?" Hint: compare where each lives and which region has a hard size limit.

Process States

Theory

A process is not always running — in fact, on a busy system it is running a small fraction of the time. The kernel tracks what each process is doing with a state, and the set of legal states and transitions is the skeleton of the scheduler.

The classic five-state model:

State	Meaning	What it is waiting for
New	Being created; PCB allocated, not yet admitted	Admission by the kernel
Ready	Runnable, waiting only for a CPU	A free core (scheduler's decision)
Running	Executing instructions on a CPU right now	Nothing — it has the CPU
Blocked (Waiting)	Cannot proceed until an event occurs	I/O completion, a lock, a signal, a child
Terminated (Zombie)	Finished executing; exit status not yet collected	Its parent to call wait

The transitions matter more than the names:

• Ready → Running — dispatch: the scheduler chose this process.
• Running → Ready — preemption: the time slice expired or a higher-priority process became ready. The process is still perfectly able to run.
• Running → Blocked — the process requested something unavailable (a disk read, an empty pipe). It is voluntarily giving up the CPU because it cannot use it.
• Blocked → Ready — the event occurred. Note it goes to Ready, not Running: being unblocked does not mean being scheduled.
• Running → Terminated — exit() or fatal signal.

The critical insight for reasoning about performance: Ready and Blocked are completely different problems. A process stuck in Ready is starved of CPU — the machine is overloaded, and adding cores or reducing load helps. A process stuck in Blocked is waiting for a device or a lock — more CPU would change nothing. Confusing them leads to the wrong fix.

Real kernels refine the model. Linux distinguishes interruptible sleep (S, wakeable by signals) from uninterruptible sleep (D, usually mid-I/O, and the cause of processes that cannot be killed), plus stopped (T) and zombie (Z).

Example

   Classic five-state model

              admit                dispatch
   [NEW] ------------> [READY] ---------------> [RUNNING] ----> [TERMINATED]
                          ^                        |    exit
                          |     preemption         |
                          +------------------------+
                          |                        |
                          |                        | request I/O / wait
                          |    event occurs        v
                          +------------------ [BLOCKED]

   Note: BLOCKED never goes directly to RUNNING.
   Being woken up only makes you eligible; the scheduler still decides.

   Linux state letters, as shown by ps:
     R  running or runnable (running + ready share this letter)
     S  interruptible sleep   (waiting, can be woken by a signal)
     D  uninterruptible sleep (usually mid-I/O; kill -9 will not work)
     T  stopped (SIGSTOP, or under a debugger)
     Z  zombie (exited, awaiting its parent's wait())

# A simulation making the transitions explicit.
class Process:
    def __init__(self, pid):
        self.pid, self.state = pid, "NEW"

    def admit(self):        self._to("READY",   from_={"NEW"})
    def dispatch(self):     self._to("RUNNING", from_={"READY"})
    def preempt(self):      self._to("READY",   from_={"RUNNING"})   # still able to run
    def block(self, why):   self._to("BLOCKED", from_={"RUNNING"})   # cannot proceed
    def wake(self):         self._to("READY",   from_={"BLOCKED"})   # NOT to RUNNING
    def exit(self):         self._to("TERMINATED", from_={"RUNNING"})

    def _to(self, new, from_):
        if self.state not in from_:                  # illegal transitions are bugs
            raise RuntimeError(f"{self.state} -> {new} is not permitted")
        self.state = new

p = Process(1)
p.admit(); p.dispatch(); p.block("disk read"); p.wake(); p.dispatch(); p.exit()

ps -eo pid,stat,wchan:20,cmd | head   # STAT column plus what a sleeper waits on

Key takeaways

• The five classic states are New, Ready, Running, Blocked, and Terminated.
• Preemption moves Running to Ready; blocking moves Running to Blocked — the causes are entirely different.
• A woken process re-enters Ready, never Running directly; the scheduler retains control.
• Ready-starvation means too little CPU; Blocked-starvation means slow I/O or lock contention. The fixes are unrelated.

🧪 Practice

1. Run ps -eo pid,stat,cmd and count processes in each state. Explain why S dominates on an idle desktop.
2. Write a program that sleeps, one that spins, and one that reads a large file. Observe each in ps and match the state letter to the cause.
3. Interview-style: "Why can't kill -9 terminate a process in state D?" Hint: ask where the process is executing at that moment and what would happen to an in-flight DMA transfer.

Process Control Block

Theory

If a process is to be stopped and later resumed as if nothing happened, the kernel must store everything needed to reconstruct it. That store is the Process Control Block (PCB) — in Linux, struct task_struct. It is the process, as far as the kernel is concerned: the process's memory and registers may be swapped, saved, or scattered, but the PCB is the authoritative record tying them together.

A PCB holds roughly five families of information:

Category	Contents	Used by
Identity	PID, PPID, user/group IDs, process group, session	Security, signals, job control
CPU state	Saved registers, program counter, stack pointer, flags	Context switching
Scheduling	State, priority, time slice remaining, CPU time used, queue links	The scheduler
Memory	Page table base, region descriptors, limits	The MMU and memory manager
Resources	Open file table, working directory, signal handlers, IPC objects	File and IPC subsystems

Three properties follow from this:

• The PCB lives in kernel memory, never in the process's own address space. A process cannot read or corrupt its own PCB — otherwise it could grant itself privileges or priority.
• PCBs are linked into many structures at once. The same PCB sits in the run queue (or a wait queue), in the parent's child list, in a global process table, and in a hash by PID. Kernel data structures overlap heavily.
• PCB size bounds the process count. Linux's task_struct is several kilobytes; with kernel stacks, each process costs on the order of 8-16 KB before any user memory. This is the concrete reason processes are "heavier" than threads.

Analogy: a hospital patient chart. The patient (the running program) may be in surgery, in a ward, or in the waiting room; the chart records identity, current status, medications, and who is responsible. Staff coordinate entirely through charts, and the chart is kept by the hospital, not carried by the patient.

Example

   Process Control Block

   +--------------------------------------+
   | pid = 4412        ppid = 4400        |  identity
   | uid = 1000        gid = 1000         |
   +--------------------------------------+
   | state = BLOCKED   priority = 20      |  scheduling
   | time_slice = 3ms  cpu_time = 1.42s   |
   | *next_in_queue ------------------->  |
   +--------------------------------------+
   | saved registers:                     |  CPU state
   |   PC  = 0x4011a8   SP = 0x7ffd2a10   |  (valid only when NOT running)
   |   rax = 0x0  rbx = 0x5  ...          |
   +--------------------------------------+
   | page table base = 0x1a2b3000         |  memory
   | text/data/heap/stack region limits   |
   +--------------------------------------+
   | open files: [0]->tty [1]->pipe ...   |  resources
   | cwd = /home/alice                    |
   | signal handlers, pending signals     |
   +--------------------------------------+

   One PCB, many lists at once:

   run queue:   PCB_7 -> PCB_3 -> PCB_9
   disk wait:   PCB_4412 -> PCB_88
   child list of 4400: PCB_4412 -> PCB_4413

/* A teaching-sized PCB: every field answers "what must survive a switch?" */
struct pcb {
    /* identity */
    pid_t  pid, ppid;
    uid_t  uid;

    /* CPU state: meaningful only while the process is NOT on a CPU */
    struct context { uint64_t rip, rsp, rbp, rax, rbx, rflags; } ctx;

    /* scheduling */
    enum { NEW, READY, RUNNING, BLOCKED, ZOMBIE } state;
    int    priority, time_slice_left;
    uint64_t cpu_time_used;

    /* memory */
    void  *page_table_base;          /* loaded into CR3 on switch */
    struct mm_region *regions;

    /* resources */
    struct file *open_files[MAX_FD]; /* index = file descriptor number */
    char   cwd[PATH_MAX];
    sighandler_t handlers[NSIG];

    /* linkage: one PCB belongs to several lists simultaneously */
    struct pcb *next_in_queue;       /* run queue OR a wait queue */
    struct pcb *parent, *children, *next_sibling;
};

ls /proc/self/            # the kernel exporting PCB contents as files
cat /proc/self/status     # state, PIDs, UIDs, memory, signal masks
ls -l /proc/self/fd/      # the open file table from the PCB

Key takeaways

• The PCB is the kernel's complete record of a process and the reason a process can be suspended and resumed.
• It holds identity, saved CPU state, scheduling data, memory descriptors, and resource tables.
• It resides in kernel memory, unreachable by the process itself, which is what makes protection enforceable.
• Its size is a real cost: a few kilobytes per process before any user memory, which is why threads are cheaper.

🧪 Practice

1. Read /proc/self/status and match ten fields to the PCB categories in the table above.
2. Explain what would go wrong if the saved register set were stored in the process's own stack rather than in kernel memory.
3. Interview-style: "Why is the saved program counter meaningless while a process is in the Running state?" Hint: ask where the authoritative copy of the PC lives at that moment.

Context Switching

Theory

A context switch is the act of saving one process's CPU state and restoring another's, so that a single core can be shared. It is the mechanism that makes multiprogramming and time-sharing possible, and it is pure overhead — no user work happens during it.

The sequence, triggered by a timer interrupt, a blocking system call, or a higher-priority wakeup:

1. Trap into the kernel — hardware saves a minimal frame (PC, flags) and switches to kernel mode.
2. Save the outgoing context — the remaining general-purpose registers, stack pointer, and (lazily, if used) floating-point and vector state are written into the outgoing PCB.
3. Update state and queues — mark the process Ready or Blocked and place its PCB on the appropriate queue.
4. Choose a successor — the scheduler picks the next process (Chapter 4 covers how).
5. Switch address space — load the new page-table base register. On x86 this writes CR3.
6. Restore the incoming context — load registers from the incoming PCB.
7. Return from trap — resume in user mode at the new process's saved PC.

The cost has two components, and the second is larger and less visible:

• Direct cost: the register save/restore and scheduling logic, roughly 1-5 microseconds.
• Indirect cost: cache and TLB pollution. The new process finds caches full of the old process's data and must refill them from memory. The TLB may be flushed entirely on address-space change (mitigated by tagged TLBs / PCIDs). This "cache-warming" penalty often exceeds the direct cost by an order of magnitude.

This is precisely why threads of the same process are cheaper to switch between: step 5 is skipped, and the caches and TLB stay largely valid.

Analogy: switching a workbench between two craftsmen. Putting away one set of tools and laying out the other is the direct cost. The larger cost is that the incoming craftsman's materials, references, and half-finished parts are all in storage and must be fetched again.

Example

   Context switch on a timer interrupt

   Process A (user)      | Kernel                        | Process B (user)
   ----------------------+-------------------------------+------------------
   executing ...         |                               |
       TIMER INTERRUPT --> hardware: save PC/flags,       |
                         | switch to kernel mode+stack    |
                         | save A's registers into PCB_A  |
                         | PCB_A.state = READY            |
                         | enqueue(run_queue, PCB_A)      |
                         | next = schedule()  -> PCB_B    |
                         | load CR3 = PCB_B.page_table    | <- address space change
                         |   (TLB flush / PCID switch)    |
                         | restore registers from PCB_B   |
                         | PCB_B.state = RUNNING          |
                         | IRET ------------------------->| resumes exactly
                         |                                | where it left off
   (frozen; caches now   |                                |
    hold B's data)       |                                |

   Cost profile:
     direct   : ~1-5 us   (register save/restore, scheduler)
     indirect : ~10-100 us of degraded performance while caches/TLB refill

/* The core of a context switch, in shape if not in detail. */
void context_switch(struct pcb *out, struct pcb *in) {
    save_registers(&out->ctx);          /* freeze the outgoing process       */

    if (out->page_table_base != in->page_table_base)
        load_page_table(in->page_table_base);  /* skipped for threads of the
                                                  SAME process -- this is why
                                                  thread switches are cheap  */
    in->state = RUNNING;
    out->state = (out->state == RUNNING) ? READY : out->state;

    restore_registers(&in->ctx);        /* execution continues as `in` on the
                                          next instruction after ITS save    */
}

vmstat 1        # the "cs" column: context switches per second
pidstat -w 1    # voluntary (blocked) vs involuntary (preempted) switches per process
perf stat -e context-switches,cache-misses ./program

Key takeaways

• A context switch saves the outgoing PCB's CPU state, switches address space, and restores the incoming one.
• It performs no useful work: it is overhead paid for the ability to share a CPU.
• Indirect cost from cache and TLB pollution usually dominates the direct register-save cost.
• Thread switches within a process skip the address-space change and preserve cache state, making them far cheaper.

🧪 Practice

1. Use vmstat 1 while idle and while running several CPU-bound loops. Explain the change in the cs column.
2. Use pidstat -w to distinguish voluntary from involuntary switches for an I/O-bound and a CPU-bound program. Which is which, and why?
3. Interview-style: "A server handling 100,000 requests per second spends 30% of CPU in context switches. What would you change?" Hint: consider thread pools, event loops, batching, and CPU affinity rather than faster switching.

<a id="22-process-lifecycle"></a>

2.2 Process Lifecycle

Processes are created by other processes, run, and end — this section follows that arc, including the failure modes that arise when a parent neglects its responsibilities.

Process Creation

Theory

Processes do not appear spontaneously. Every process except the first is created by another process, which makes the set of all processes a tree. At the root is the process the kernel starts by hand at boot (PID 1: init, systemd, or launchd); everything else descends from it.

Creating a process requires answering four questions, and different operating systems answer them differently:

1. What is the new process's address space? Either a copy of the parent's, or a fresh one loaded from a program file.
2. What resources does it get? All of the parent's open files, some subset, or none.
3. Do parent and child run concurrently? Usually yes; some systems make the parent wait.
4. What is the parent's continuing relationship? It typically retains the ability to wait for the child and receives notification when it exits.

Two design philosophies dominate:

	UNIX: fork + exec	Windows: CreateProcess
Model	Split: duplicate, then optionally replace image	Combined: create and load in one call
Parameters	Almost none — the child inherits everything	Large structure specifying every detail
Customisation	Arbitrary code runs in the child before exec	Only what the parameter block expresses
Cost	Duplication (mitigated by copy-on-write)	No duplication
Composability	Very high — pipes, redirection, privilege dropping	Lower; requires explicit flags and handle lists

The UNIX split looks wasteful and is in fact its greatest strength: the interval between fork and exec is a window in which the child is an ordinary process that can be configured arbitrarily — redirect file descriptors, change directory, set resource limits, drop privileges — using the same system calls available anywhere else. No special "process creation options" API is needed because ordinary code does the job.

Modern kernels make the copy cheap with copy-on-write: parent and child initially share all physical pages marked read-only, and a page is duplicated only when one of them writes to it. A fork followed immediately by exec therefore copies almost nothing.

Example

   The process tree

   systemd (PID 1)
     |-- sshd (900)
     |     |-- sshd: alice (4400)
     |           |-- bash (4401)
     |                 |-- vim (4412)
     |                 |-- make (4420)
     |                       |-- cc1 (4421)
     |                       |-- as  (4422)
     |-- cron (905)
     +-- nginx (910)
           |-- nginx worker (911)
           +-- nginx worker (912)

   fork() with copy-on-write

   before fork:            after fork (no copying yet):
   parent -> [page A rw]   parent -> [page A ro] <- child     shared, read-only
                           on first write by either:
                           parent -> [page A' rw]
                           child  -> [page A  rw]             now duplicated

#include <stdio.h>
#include <unistd.h>
#include <sys/wait.h>

int main(void) {
    printf("parent starts, pid=%d\n", getpid());

    pid_t pid = fork();          /* ONE call, TWO returns: the defining oddity */

    if (pid < 0) {
        perror("fork");          /* out of PIDs or memory */
        return 1;
    }
    if (pid == 0) {
        /* CHILD: fork returned 0 here. Separate address space (copy-on-write),
           same open files, same cwd, new PID. */
        printf("child:  pid=%d parent=%d\n", getpid(), getppid());
        _exit(0);
    }
    /* PARENT: fork returned the child's PID here. */
    printf("parent: created child %d\n", pid);
    wait(NULL);                  /* collect the child, avoiding a zombie */
    return 0;
}

pstree -p          # the live process tree, with PIDs
ps -eo pid,ppid,cmd --forest | head -30

Key takeaways

• Every process is created by another, forming a tree rooted at PID 1.
• UNIX splits creation (fork) from program loading (exec); Windows combines them in CreateProcess.
• The gap between fork and exec is where redirection, privilege dropping, and resource limits are applied.
• Copy-on-write makes fork cheap by sharing pages read-only until one side writes.

🧪 Practice

1. Draw the process tree produced by a shell running ls | wc -l. How many fork calls occur, and who is the parent of each process?
2. Write a program that forks twice unconditionally (no exec, no exit between them). How many processes exist at the end? Draw the tree and explain.
3. Interview-style: "Why does fork() return twice, and how does each process know which one it is?" Hint: focus on the single differing value and why the kernel chose that convention rather than a flag.

fork and exec

Theory

These two calls deserve close attention because their interaction confuses nearly everyone at first, and because almost every process on a UNIX system is created by their combination.

fork() duplicates the calling process. The child receives a copy of the address space (copy-on-write), copies of the file descriptor table, the same current directory, the same signal handler dispositions, and the same register values — including the program counter, which is why it resumes at exactly the same instruction. What differs: the PID, the parent PID, the return value of fork itself, pending signals (cleared), and resource usage counters (reset).

execve() does not create anything. It replaces the calling process's program image: the text, data, heap, and stack are discarded and rebuilt from the named executable, and execution begins at the new program's entry point. Everything that identifies the process survives: PID, PPID, open file descriptors, current directory, and (importantly) file descriptor redirections set up beforehand.

The consequences of that survival rule are the whole point:

• Open file descriptors persist across exec unless marked close-on-exec (FD_CLOEXEC). This is how a shell redirects a child's output before deciding what program to run.
• execve does not return on success — there is no code left to return to. Any statement after it executes only if the call failed.
• Because signal handlers are function pointers into a program image that no longer exists, they reset to default across exec, while signal masks are inherited.

Three inheritance rules are worth memorising:

Property	Across fork	Across exec
PID	New	Unchanged
Address space	Copied (COW)	Replaced entirely
Open file descriptors	Copied (shared offsets)	Preserved unless FD_CLOEXEC
Signal handlers	Inherited	Reset to default
Current directory, umask	Inherited	Preserved

Example

   fork + exec: what changes, what survives

   parent (pid 100)
     text: shell   fds: 0=tty 1=tty 2=tty
        |
        | fork()
        v
   child  (pid 101)                        <- new PID, everything else copied
     text: shell   fds: 0=tty 1=tty 2=tty
        |
        | open("out.txt") -> fd 3
        | dup2(3, 1)                       <- configure BEFORE exec
        | close(3)
        v
   child  (pid 101)
     text: shell   fds: 0=tty 1=out.txt 2=tty
        |
        | execve("/bin/ls", ...)
        v
   child  (pid 101)                        <- SAME pid, NEW program
     text: ls      fds: 0=tty 1=out.txt 2=tty   <- redirection survived

#include <stdio.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/wait.h>

int main(void) {
    pid_t pid = fork();

    if (pid == 0) {
        /* --- the customisation window: ordinary code, ordinary syscalls --- */
        int fd = open("out.txt", O_WRONLY | O_CREAT | O_TRUNC, 0644);
        dup2(fd, STDOUT_FILENO);   /* make fd 1 refer to out.txt */
        close(fd);                 /* the duplicate is no longer needed */
        chdir("/tmp");             /* run the child somewhere else */
        /* ------------------------------------------------------------------ */

        execlp("ls", "ls", "-l", (char *)NULL);
        /* Reached ONLY if exec failed: the image was never replaced. */
        perror("execlp");
        _exit(127);
    }

    waitpid(pid, NULL, 0);
    printf("child finished; its output is in /tmp/out.txt\n");
    return 0;
}

# Higher-level languages wrap the same pair. subprocess does fork+exec for you,
# and exposes the customisation window as keyword arguments.
import subprocess
with open("out.txt", "w") as f:
    subprocess.run(["ls", "-l"], stdout=f, cwd="/tmp")  # dup2 + chdir, hidden

Key takeaways

• fork duplicates a process; exec replaces a process's program image without creating anything.
• After a successful exec there is no return — following code runs only on failure.
• File descriptors survive both calls (unless FD_CLOEXEC), which is what makes redirection possible.
• The window between the two is where all child customisation happens, using ordinary system calls.

🧪 Practice

1. Write a program that execs /bin/echo and prints a message afterwards. Explain why the message never appears.
2. Modify the redirection example to send stderr to the same file as stdout. Which order of dup2 calls works, and why does the other fail?
3. Interview-style: "What is FD_CLOEXEC for, and what leaks without it?" Hint: imagine a server with a private socket or key file that spawns an untrusted helper.

Process Termination

Theory

A process ends in one of two ways: it decides to, or something decides for it.

Voluntary termination happens when the process calls exit() (or returns from main, which calls exit implicitly). The process supplies an exit status — a small integer, conventionally 0 for success and non-zero for failure — which is its final message to its parent.

Involuntary termination happens when the process receives a fatal signal: SIGSEGV from an invalid memory access, SIGKILL from an administrator, SIGTERM from a shutdown sequence, or SIGABRT from a failed assertion. The kernel records which signal killed it, which the parent can distinguish from a normal exit.

On termination, the kernel performs a cleanup that is worth knowing precisely because it is more thorough than people expect:

• All open file descriptors are closed (which may flush and release locks, and signal EOF to pipe readers).
• The address space is torn down and its physical pages freed.
• Memory-mapped regions are unmapped; shared memory attachments are detached.
• Children are re-parented to init/systemd, never killed.
• Most of the PCB is freed — but not all: a minimal record survives so the exit status can be delivered. This is the zombie state.

Two points that surprise people:

• Killing a parent does not kill its children. They are orphaned and adopted, not terminated. Killing a whole job requires signalling the process group (kill -TERM -PGID) or using a container/cgroup.
• SIGKILL cannot be caught, blocked, or ignored — it is handled entirely by the kernel — which is exactly why it works when SIGTERM does not, and why a process in uninterruptible sleep can still resist it until the I/O completes.

The exit family also distinguishes exit() (runs atexit handlers, flushes stdio buffers) from _exit() (immediate, no cleanup). After a fork, a child that will not exec should generally use _exit() to avoid flushing the parent's buffered data a second time.

Example

   Termination paths and what the parent learns

   exit(3)          -----> WIFEXITED(status)   true,  WEXITSTATUS = 3
   return 3 from main ---> identical to exit(3)
   SIGSEGV          -----> WIFSIGNALED(status) true,  WTERMSIG = 11
   kill -9          -----> WIFSIGNALED(status) true,  WTERMSIG = 9
   SIGSTOP          -----> WIFSTOPPED(status)  true (not terminated)

   Cleanup at exit

   +-- close all file descriptors  (pipe readers see EOF; locks released)
   +-- unmap address space, free physical pages
   +-- detach shared memory, release semaphores held
   +-- re-parent any children to PID 1
   +-- keep: pid + exit status  ---> ZOMBIE until the parent waits

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>

int main(void) {
    pid_t pid = fork();
    if (pid == 0) {
        exit(42);                     /* voluntary, with a status */
        /* try replacing with: abort();  -> terminated by SIGABRT (6) */
    }

    int status;
    waitpid(pid, &status, 0);

    if (WIFEXITED(status))            /* normal exit? */
        printf("exited normally, status %d\n", WEXITSTATUS(status));
    else if (WIFSIGNALED(status))     /* killed by a signal? */
        printf("killed by signal %d\n", WTERMSIG(status));
    return 0;
}

./demo; echo "shell sees: $?"    # the shell reads the same exit status
# a signal death is reported by the shell as 128 + signal number:
sh -c 'kill -9 $$'; echo $?      # prints 137  (128 + 9)

Key takeaways

• Processes end voluntarily via exit with a status, or involuntarily via a fatal signal.
• The kernel closes descriptors, frees memory, and re-parents children — it does not kill them.
• A minimal record survives until the parent collects the status; that is the zombie state.
• SIGKILL cannot be caught or ignored, which is why it succeeds where SIGTERM is swallowed.

🧪 Practice

1. Write a program returning various statuses and confirm each with echo $?. What happens with exit(256), and why?
2. Start a background process, kill its parent, and observe the child's new PPID with ps. Explain what happened.
3. Interview-style: "Why should a SIGTERM handler exist in a server, when SIGKILL will work anyway?" Hint: think about in-flight requests, buffered data, and lock files.

Orphan and Zombie Processes

Theory

These two are frequently confused and are in fact opposites: an orphan has lost its parent, a zombie has lost nothing but is waiting for its parent to act.

Zombie (defunct). When a process terminates, the kernel cannot discard it completely, because the exit status is data the parent may still want. So it frees the address space, the file descriptors, and nearly everything else, but retains the PCB entry holding the PID and exit status. The process is dead — it holds no memory and consumes no CPU — but its slot in the process table remains. The parent clears it by calling wait/waitpid, an act called reaping.

A zombie is harmless individually and dangerous in bulk. A long-running server that forks children and never waits accumulates entries until the process table or PID space is exhausted, at which point no process on the system can be created. This is a classic production outage.

Orphan. When a parent terminates before its child, the child is not killed — it is re-parented to PID 1. init/systemd exists partly to be a reliable adoptive parent: it calls wait in a loop forever, so any orphan that later exits is reaped immediately and never becomes a lasting zombie.

This yields the neat rule: orphans cannot become permanent zombies; only processes with a living, negligent parent can.

Three ways to avoid zombies:

1. Call wait/waitpid — blocking, or in a SIGCHLD handler, or with WNOHANG in an event loop.
2. Set SIGCHLD to SIG_IGN (or use SA_NOCLDWAIT), which tells the kernel to reap automatically. You lose the exit status.
3. Double-fork: fork a child that immediately forks a grandchild and exits. The grandchild is orphaned instantly and adopted by init, which reaps it. This is the traditional daemonisation trick.

Analogy: a zombie is a completed hospital discharge form sitting in a tray waiting for a signature — the patient has gone home, but the paperwork occupies a slot in the filing cabinet. An orphan is a patient whose assigned doctor left; the hospital reassigns them automatically.

Example

   ZOMBIE                                ORPHAN

   parent (alive, not waiting)           parent EXITS
      |                                     |
   child EXITS                           child still RUNNING
      |                                     |
   PCB retained: pid + status            re-parented to PID 1
   state = Z (defunct)                   state = R/S (normal)
   consumes: a process table slot        consumes: nothing unusual
   fixed by: parent calls wait()         handled by: init reaps it later

   ps output:
     PID  PPID STAT CMD
    4413  4412 Z    [demo] <defunct>     <- zombie: parent 4412 is negligent
    4499     1 S    ./worker             <- orphan: PPID became 1

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/wait.h>
#include <signal.h>

/* Reap children asynchronously: the correct pattern for a forking server. */
void reap(int sig) {
    (void)sig;
    /* Loop: several children may have exited while we were not looking,
       and signals do not queue. WNOHANG makes waitpid non-blocking. */
    while (waitpid(-1, NULL, WNOHANG) > 0)
        ;
}

int main(void) {
    struct sigaction sa = { .sa_handler = reap, .sa_flags = SA_RESTART };
    sigemptyset(&sa.sa_mask);
    sigaction(SIGCHLD, &sa, NULL);   /* without this, children pile up as Z */

    for (int i = 0; i < 5; i++) {
        if (fork() == 0) {
            sleep(1);
            _exit(i);                /* _exit: skip stdio flush in the child */
        }
    }
    sleep(3);
    printf("no zombies: check with ps during the run\n");
    return 0;
}

ps -eo pid,ppid,stat,cmd | awk '$3 ~ /^Z/'   # list zombies, if any
cat /proc/sys/kernel/pid_max                 # the table a zombie leak exhausts

Key takeaways

• A zombie is a terminated process whose exit status has not yet been collected; it holds a process table slot only.
• An orphan is a live process whose parent died; it is re-parented to PID 1 and is not a problem.
• Zombies accumulate only under a living, negligent parent, and can exhaust the PID space.
• Avoid them by calling wait/waitpid (often via a SIGCHLD handler), ignoring SIGCHLD, or double-forking.

🧪 Practice

1. Write a program that forks a child which exits immediately while the parent sleeps 30 seconds without waiting. Find the zombie in ps and explain each column.
2. Reverse it: make the parent exit first and the child sleep. Show the child's PPID becoming 1.
3. Interview-style: "A production server slowly stops being able to fork. ps shows thousands of <defunct> entries. Diagnose and fix." Hint: which resource is exhausted, and which single missing call causes it?

Process Hierarchies

Theory

The parent-child relationship is not just bookkeeping — the kernel builds several grouping mechanisms on top of it, and they are what make job control, terminal handling, and modern container isolation work.

The hierarchy has four levels, each nesting inside the last:

Level	Identifier	Purpose
Process	PID	The unit of execution and protection
Process group	PGID	A set of related processes signalled together — one shell "job"
Session	SID	A set of process groups attached to one controlling terminal — one login
Container/cgroup	cgroup path, namespaces	A resource- and visibility-isolated subtree

Process groups exist so that signals can address a whole pipeline. When you type Ctrl-C, the terminal driver sends SIGINT to the foreground process group, not to one process — which is why ps aux | grep x | sort all die together. A shell places every pipeline in its own group precisely to make this work.

Sessions exist to associate process groups with a controlling terminal, and to define what happens when that terminal disappears. When an SSH connection drops, the kernel sends SIGHUP to the session leader, which is why long jobs die on logout — and why nohup, setsid, screen, and tmux exist: they detach the process from the session so the hangup never reaches it.

Daemonisation is the classic recipe built from all of this: fork (so the parent can exit and the shell moves on), setsid() (become a session leader with no controlling terminal), fork again (so the daemon can never acquire a terminal, since only a session leader can), then chdir("/"), reset umask, and redirect the standard descriptors to /dev/null.

Modern systems add namespaces and cgroups as a stronger hierarchy: a PID namespace gives a subtree its own PID numbering (its init is PID 1 inside, though it has a different PID outside), and a cgroup imposes collective CPU and memory limits on a subtree. Containers are exactly this — a process subtree with restricted visibility and bounded resources — and systemd uses the same mechanism to reliably kill an entire service, which signalling by PGID alone cannot guarantee.

Example

   Session, groups, and processes for one login

   session 4400 (controlling terminal /dev/pts/2)
     |
     +-- process group 4401  (the shell itself, session leader's group)
     |     +-- bash (4401)
     |
     +-- process group 4450  FOREGROUND  <- receives Ctrl-C (SIGINT)
     |     +-- ps   (4450)   |
     |     +-- grep (4451)   | one pipeline = one group = one "job"
     |     +-- sort (4452)   |
     |
     +-- process group 4460  background (job %2)
           +-- make (4460)

   Terminal closes -> SIGHUP to the session -> jobs die
   unless detached with nohup / setsid / tmux.

   Container view (PID namespace)

   host:      systemd(1) --- containerd(800) --- app(9021)
   inside:                                       app(1)   <- same process,
                                                             different PID

#include <unistd.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/stat.h>

/* Classic daemonisation: detach from terminal and session entirely. */
void daemonise(void) {
    if (fork() > 0) exit(0);      /* 1. parent exits: shell regains its prompt
                                        and the child is not a group leader   */
    setsid();                     /* 2. new session + group, no controlling
                                        terminal; SIGHUP can no longer arrive */
    if (fork() > 0) exit(0);      /* 3. second fork: we are no longer a session
                                        leader, so we can NEVER acquire a tty */
    chdir("/");                   /* 4. do not pin a filesystem we do not own */
    umask(0);                     /* 5. predictable permissions               */

    int null = open("/dev/null", O_RDWR);
    dup2(null, 0); dup2(null, 1); dup2(null, 2);  /* 6. detach std streams    */
    if (null > 2) close(null);
}

ps -eo pid,ppid,pgid,sid,tty,cmd | head    # the four identifiers side by side
kill -TERM -4450                           # signal the whole GROUP (note the -)
systemd-cgls                               # the cgroup hierarchy systemd manages

Key takeaways

• Processes nest into process groups (jobs), sessions (logins), and cgroups/namespaces (containers).
• Terminal signals such as Ctrl-C go to the foreground process group, which is why pipelines die together.
• Losing a terminal sends SIGHUP to the session; nohup, setsid, and tmux exist to escape it.
• Containers are a process subtree with its own PID namespace and resource limits, giving reliable group control.

🧪 Practice

1. Run sleep 100 | cat & and inspect PID, PGID, and SID with ps -o pid,pgid,sid,cmd. Explain the grouping.
2. Demonstrate the difference between kill 1234 and kill -- -1234 on a pipeline. Which terminates everything?
3. Interview-style: "Why does the daemonisation recipe fork twice?" Hint: only a session leader can acquire a controlling terminal — so make sure you are not one.

<a id="23-interprocess-communication"></a>

2.3 Interprocess Communication

Processes are deliberately isolated, so cooperating processes need explicit, kernel-mediated channels; this section covers the two fundamental models and the four mechanisms built on them.

Shared Memory Model

Theory

Isolation is the point of a process — and it is also an obstacle whenever two processes must cooperate. Shared memory is the most direct way around it: the kernel maps the same physical pages into two or more address spaces, and from then on the processes communicate simply by reading and writing ordinary variables.

The performance argument is decisive for bulk data. Once the mapping is established, communication involves no system calls at all — a write by one process is immediately visible to the other, at memory speed. Every other IPC mechanism copies data through the kernel, typically twice (user → kernel → user). For a video frame or a large data set, that difference is enormous.

The cost is that the kernel, having set up the mapping, steps out of the way entirely — and with it goes all synchronisation. Shared memory provides no notification that data has arrived, no mutual exclusion, and no ordering guarantees. Two processes incrementing a shared counter will lose updates exactly as two threads would. Shared memory is therefore never used alone: it is paired with semaphores, mutexes in shared memory, or atomic operations (Chapter 5).

There are two common ways to obtain it on UNIX: POSIX shared memory (shm_open + mmap, file-descriptor based, the modern choice) and System V shared memory (shmget + shmat, key-based, older). mmap with MAP_SHARED on a regular file gives the same effect with persistence.

Analogy: shared memory is a whiteboard in a shared room. Anyone can read or write instantly, with no messenger involved — which is why it is fast, and why two people writing at once produce nonsense unless they agree on turn-taking.

Example

   Two processes, one physical region

   Process A virtual space        physical memory        Process B virtual space
   +--------------------+                                +--------------------+
   | ...                |         +-------------+        | ...                |
   | 0x7f00_0000  ------+-------> |  shared     | <------+---- 0x7f88_0000    |
   | ...                |         |  pages      |        | ...                |
   +--------------------+         +-------------+        +--------------------+

   Different virtual addresses, SAME physical frames.
   A write by A is visible to B with no system call and no copy.

   Cost comparison for 1 MB of data:
     pipe / socket : user -> kernel buffer -> user   (2 copies + syscalls)
     shared memory : write it once, in place         (0 copies, 0 syscalls)

#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
#include <semaphore.h>

struct shared {
    sem_t  lock;          /* the synchronisation shared memory does NOT provide */
    int    counter;
    char   message[256];
};

int main(void) {
    int fd = shm_open("/demo", O_CREAT | O_RDWR, 0600);  /* a name, not a file */
    ftruncate(fd, sizeof(struct shared));                /* size the region    */

    struct shared *s = mmap(NULL, sizeof *s, PROT_READ | PROT_WRITE,
                            MAP_SHARED,      /* MAP_SHARED: visible to others;
                                                MAP_PRIVATE would be copy-on-write
                                                and therefore useless here     */
                            fd, 0);
    close(fd);                                /* the mapping outlives the fd   */

    sem_init(&s->lock, 1 /* pshared: usable across processes */, 1);

    if (fork() == 0) {
        sem_wait(&s->lock);                   /* without this, updates are lost */
        s->counter++;
        snprintf(s->message, sizeof s->message, "child wrote %d", s->counter);
        sem_post(&s->lock);
        _exit(0);
    }
    wait(NULL);

    sem_wait(&s->lock);
    printf("parent reads: %s (counter=%d)\n", s->message, s->counter);
    sem_post(&s->lock);

    munmap(s, sizeof *s);
    shm_unlink("/demo");                      /* the name persists until removed */
    return 0;
}

ls /dev/shm/          # POSIX shared memory objects appear here
ipcs -m               # System V shared memory segments, with attach counts

Key takeaways

• Shared memory maps the same physical pages into several address spaces, giving zero-copy communication.
• After setup it requires no system calls, making it the fastest IPC for bulk data.
• It provides no synchronisation or notification whatsoever; it must be paired with semaphores or atomics.
• Use MAP_SHARED (not MAP_PRIVATE) and remember that the name persists until explicitly unlinked.

🧪 Practice

1. Remove the semaphore calls from the example, have two children increment the counter one million times each, and observe lost updates. Explain the mechanism.
2. Benchmark transferring 100 MB between processes via a pipe versus shared memory. Explain the ratio in terms of copies and system calls.
3. Interview-style: "Two processes map the same region and one stores a pointer in it for the other to follow. What goes wrong?" Hint: compare the two virtual addresses in the diagram above.

Message Passing Model

Theory

The alternative to sharing memory is to exchange messages: discrete units of data copied from one process to another by the kernel, through send and receive operations. The processes never share an address space; the kernel is in the middle of every exchange.

This trades performance for safety and generality:

• Synchronisation comes free. A message arriving is the notification. There is no race over "is the data ready" because the message is not deliverable until it is complete.
• It works across machines. Since the model already assumes copying through an intermediary, replacing that intermediary with a network changes the cost but not the semantics — which is why sockets and RPC are message passing.
• It is safer. No shared writable state means no data races over the transferred data; ownership transfers with the message.
• It is slower for bulk data, because every byte is copied at least twice and every exchange costs system calls.

Three design axes define any message-passing system:

Axis	Options	Implication
Naming	Direct (name the process) vs indirect (name a mailbox/port/queue)	Indirect allows many-to-many and decoupled lifetimes
Synchronisation	Blocking (synchronous) vs non-blocking (asynchronous)	Blocking send is a rendezvous; non-blocking needs buffering
Buffering	Zero capacity, bounded, or unbounded	Zero forces rendezvous; bounded means send can block; unbounded can exhaust memory

The blocking choice deserves care. A blocking send with zero buffering is a rendezvous: sender and receiver meet, which gives strong ordering but couples their speeds. A non-blocking send into a bounded queue is what most real systems use, and it forces you to decide what happens when the queue is full — block, drop, or fail — a decision that reappears in every message broker and network stack.

Analogy: shared memory is a whiteboard; message passing is the postal system. Slower, involves an intermediary, and requires copying — but the recipient knows exactly when something arrived, and it works just as well to the next building as to the next desk.

Example

   Message passing

   Process A                 kernel                    Process B
   ---------                 ------                    ---------
   send(B, msg) ---------> [ copy in ]
                           [ queue:  ]
                           [ [m1][m2]] ------------> receive(&msg)
                                                     [ copy out ]

   Two copies, two system calls -- and no shared state at all.

   Naming options

   DIRECT                              INDIRECT (mailbox / port / queue)
   send(P2, m)                         send(mbox_A, m)
   receive(P1, &m)                     receive(mbox_A, &m)
   both must know each other           senders and receivers decoupled;
   one-to-one                          many-to-many possible

   Buffering and blocking

   capacity 0   : send blocks until receive happens      (rendezvous)
   capacity N   : send blocks only when full             (usual case)
   unbounded    : send never blocks -- until memory runs out

# Message passing between processes, with an explicit bounded queue.
from multiprocessing import Process, Queue

def producer(q):
    for i in range(5):
        q.put({"seq": i, "payload": f"item {i}"})   # serialised + copied
    q.put(None)                                     # sentinel: end of stream

def consumer(q):
    while True:
        msg = q.get()          # BLOCKS until a message exists -- the arrival
        if msg is None:        # of the message IS the synchronisation
            break
        print("received", msg["seq"], msg["payload"])

q = Queue(maxsize=10)          # bounded: put() blocks when full -> backpressure
p1 = Process(target=producer, args=(q,))
p2 = Process(target=consumer, args=(q,))
p1.start(); p2.start(); p1.join(); p2.join()

Key takeaways

• Message passing copies discrete messages through the kernel; processes share no memory.
• Delivery is itself the synchronisation, eliminating the races inherent in shared memory.
• The model extends unchanged across machines, which is why sockets and RPC use it.
• Naming (direct vs mailbox), blocking behaviour, and buffer capacity are the three defining design choices.

🧪 Practice

1. Set maxsize=1 in the example and add a slow consumer. Describe the producer's behaviour and name the effect.
2. Compare shared memory and message passing across five criteria: speed, synchronisation, safety, network transparency, and complexity. Present it as a table.
3. Interview-style: "When a bounded queue fills, should send block, drop, or fail?" Hint: argue from the consequences for a logging system, a video stream, and a payments pipeline separately.

Pipes and Named Pipes

Theory

A pipe is the simplest concrete message-passing channel in UNIX: a kernel-managed byte buffer with a write end and a read end. It is a one-directional stream with no message boundaries — write 100 bytes then 50, and the reader may receive 150 in one read, or 3 bytes at a time.

An anonymous pipe (pipe()) has no name in the file system. It is created by one process and passed to another the only way it can be: by inheritance across fork. That restriction is why anonymous pipes connect related processes — parent to child, or siblings created by the same shell.

A named pipe or FIFO (mkfifo) has a path in the file system, so unrelated processes can open it by name. Despite appearing as a file, it stores nothing; the path is a rendezvous point, and data still lives only in the kernel buffer.

Two behaviours make pipes work as well as they do:

• Flow control is automatic. The buffer is finite (64 KB by default on Linux). A writer that outruns the reader blocks when the buffer is full; a reader that outruns the writer blocks when it is empty. Producers and consumers are throttled to each other's speed with no code.
• EOF is meaningful. When all write ends are closed, the reader's read returns 0, signalling end of stream. Conversely, writing to a pipe with no readers raises SIGPIPE, which by default kills the writer — this is why head can terminate a long pipeline early.

The most common bug is failing to close unused ends. If a parent keeps the write end open after handing it to a child, the reader never sees EOF and hangs forever — because some process still holds a write end.

Analogy: a pipe is a length of hose between two people. Water flows one way; if the receiver stops drinking, the hose fills and the pourer must wait; if the pourer walks away, the receiver knows there is no more water coming.

Example

   Anonymous pipe created before fork

   parent: pipe(fd) -> fd[0] read end, fd[1] write end
       |
       | fork()  -- the child inherits BOTH descriptors
       v
   parent               kernel buffer (64 KB)               child
   close(fd[0])  --->  [ ...bytes... ]  --->  close(fd[1])
   write(fd[1])                               read(fd[0])

   Each side MUST close the end it does not use, or:
     - reader never sees EOF (a write end is still open somewhere)
     - the pipeline hangs

   How the shell builds `ls | wc -l`

   pipe(p)
   fork -> child1: dup2(p[1], 1); close both; exec ls    (stdout -> pipe)
   fork -> child2: dup2(p[0], 0); close both; exec wc    (stdin  <- pipe)
   parent: close(p[0]); close(p[1]); wait for both       (critical!)

#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <sys/wait.h>

int main(void) {
    int fd[2];
    pipe(fd);                        /* fd[0] = read end, fd[1] = write end */

    if (fork() == 0) {               /* ---- child: the reader ---- */
        close(fd[1]);                /* MUST close the write end, or no EOF */
        char buf[128];
        ssize_t n;
        while ((n = read(fd[0], buf, sizeof buf)) > 0)   /* 0 = EOF */
            fwrite(buf, 1, n, stdout);
        close(fd[0]);
        _exit(0);
    }

    close(fd[0]);                    /* ---- parent: the writer ---- */
    const char *msg = "streamed through the kernel\n";
    write(fd[1], msg, strlen(msg));
    close(fd[1]);                    /* closing the last write end sends EOF */
    wait(NULL);
    return 0;
}

mkfifo /tmp/chan                  # a named pipe: a rendezvous point, not storage
cat /tmp/chan &                   # reader blocks until a writer opens it
echo "hello unrelated process" > /tmp/chan
ls -l /tmp/chan                   # note the leading 'p' in the mode column

Key takeaways

• A pipe is a unidirectional kernel byte buffer with no message boundaries.
• Anonymous pipes are inherited across fork, so they connect related processes; FIFOs have a path and connect any processes.
• Blocking on a full or empty buffer provides automatic flow control between producer and consumer.
• Closing unused ends is mandatory: EOF depends on all write ends being closed, and writing with no reader raises SIGPIPE.

🧪 Practice

1. Remove close(fd[1]) from the parent in the example. Explain precisely why the program hangs.
2. Implement ls | wc -l in C using pipe, fork, dup2, and execlp, closing every unused descriptor.
3. Interview-style: "Why does yes | head -1 terminate instead of running forever?" Hint: what does yes receive when it writes to a pipe whose reader has gone?

Signals

Theory

A signal is not a data channel — it is an asynchronous notification, a software interrupt delivered to a process. Signals carry no payload beyond their number (except the rarely used sigqueue variant); the meaning is entirely in which signal was sent.

Signals arise from three sources: the kernel reporting a fault (SIGSEGV, SIGFPE), a user or program requesting something (SIGINT from Ctrl-C, SIGTERM from kill), or an event the process asked to hear about (SIGCHLD on child exit, SIGALRM on timer expiry).

A process may respond in three ways: run a handler, ignore the signal, or accept the default action (usually terminate, sometimes stop or ignore). SIGKILL and SIGSTOP are exceptions to all of this — they cannot be caught, blocked, or ignored, precisely so an administrator always retains control.

The subtlety that dominates practice is that a handler runs asynchronously, interrupting the process at an arbitrary instruction. That leads to strict rules:

• Only async-signal-safe functions may be called from a handler. printf and malloc are not: if the signal arrives while the process is already inside malloc, calling it again corrupts the heap. write is safe.
• Shared state must be volatile sig_atomic_t, since the compiler cannot see that a handler modifies it.
• Signals do not queue. Standard signals are a single pending bit — ten SIGCHLDs while blocked deliver one. This is why SIGCHLD handlers must loop with waitpid(..., WNOHANG).
• Slow system calls may be interrupted, returning EINTR, unless restarted with SA_RESTART.

The modern preference is to keep handlers trivial: set a flag, or write a byte to a self-pipe / signalfd so the main event loop handles the event synchronously where normal code is safe.

Analogy: a signal is a fire alarm, not a phone call. It tells you that something happened, not what or why, it can interrupt you mid-sentence, and two alarms ringing at once sound like one.

Example

   Common signals

   Signal    Default action        Typical source
   -------   -------------------   -----------------------------------------
   SIGINT    terminate             Ctrl-C, to the foreground process group
   SIGTERM   terminate             kill (polite request; catchable)
   SIGKILL   terminate (forced)    kill -9  -- CANNOT be caught or ignored
   SIGSEGV   terminate + core      invalid memory access
   SIGCHLD   ignore                a child exited or stopped
   SIGHUP    terminate             controlling terminal closed
   SIGSTOP   stop                  Ctrl-Z / job control -- also uncatchable
   SIGPIPE   terminate             wrote to a pipe with no reader

   Asynchronous delivery

   main flow:  ... a = b + c;  [SIGNAL ARRIVES] ...
                                     |
                                     v  handler runs on the same thread
                               handler() { flag = 1; }   <- keep it trivial
                                     |
               ... resumes here <-----+

#include <stdio.h>
#include <signal.h>
#include <unistd.h>
#include <string.h>

/* volatile sig_atomic_t: the compiler must not cache it, and reads/writes
   must be indivisible -- the handler can fire between any two instructions. */
static volatile sig_atomic_t shutdown_requested = 0;

void on_term(int sig) {
    shutdown_requested = 1;                 /* safe: just set a flag */
    const char m[] = "signal received\n";
    write(STDERR_FILENO, m, sizeof m - 1);  /* write() is async-signal-safe;
                                               printf() is NOT */
}

int main(void) {
    struct sigaction sa;
    memset(&sa, 0, sizeof sa);
    sa.sa_handler = on_term;
    sa.sa_flags   = SA_RESTART;             /* restart interrupted syscalls
                                               instead of failing with EINTR */
    sigemptyset(&sa.sa_mask);
    sigaction(SIGTERM, &sa, NULL);          /* prefer sigaction over signal() */
    sigaction(SIGINT,  &sa, NULL);

    printf("pid %d: try  kill -TERM %d  or Ctrl-C\n", getpid(), getpid());
    while (!shutdown_requested)             /* handle it HERE, synchronously,
                                               where normal code is safe      */
        pause();                            /* sleep until any signal arrives */

    printf("clean shutdown: flushing, closing, releasing locks\n");
    return 0;
}

kill -l                 # every signal name and number
kill -TERM 4412         # polite: the process may clean up
kill -KILL 4412         # forced: the kernel does it, no handler runs
trap 'echo caught' INT  # shells catch signals too

Key takeaways

• Signals are asynchronous notifications carrying only an identity, not data.
• A process can catch, ignore, or take the default action — except for SIGKILL and SIGSTOP.
• Handlers interrupt arbitrary code, so only async-signal-safe functions and volatile sig_atomic_t state are permissible.
• Standard signals do not queue; handlers should set a flag or use a self-pipe and let the main loop do the work.

🧪 Practice

1. Write a program that catches SIGINT, prints a message on the third one, and exits. Explain why a counter must be volatile sig_atomic_t.
2. Deliberately call printf inside a handler and send signals rapidly during heavy printf use in the main loop. Describe what can go wrong and why.
3. Interview-style: "Why must a SIGCHLD handler call waitpid in a loop?" Hint: signals of the same type do not queue while one is pending.

Sockets

Theory

Sockets generalise IPC to the case where the two processes may be on different machines. A socket is an endpoint for communication, identified locally by a file descriptor and (for network sockets) globally by an IP address and port number.

The abstraction's great virtue is that the same API serves both cases. A UNIX domain socket (AF_UNIX) connects processes on one machine through a file-system path, bypassing the network stack entirely and running roughly as fast as a pipe — while adding bidirectionality, message boundaries if desired, and the ability to pass file descriptors and verify peer credentials. An internet socket (AF_INET) uses the same calls to reach another machine.

The second choice is the transport type:

	Stream (SOCK_STREAM, TCP)	Datagram (SOCK_DGRAM, UDP)
Connection	Established before use	Connectionless
Reliability	Ordered, retransmitted, no duplicates	Best effort: may drop, reorder, duplicate
Boundaries	None — a byte stream	Preserved — one send is one receive
Cost	Handshake, state, acknowledgements	Minimal
Use for	HTTP, SSH, databases	DNS, video, telemetry, games

The server-side sequence is worth memorising because it is identical everywhere: socket() to create the endpoint, bind() to claim an address, listen() to mark it passive with a backlog queue, and accept() to remove one completed connection from that queue — returning a new descriptor for that client while the original keeps listening. Clients need only socket() and connect().

Two practical points: because streams have no message boundaries, application protocols must supply their own framing (length prefixes, delimiters, or fixed sizes) — assuming one send equals one recv is the most common networking bug. And SO_REUSEADDR exists because a closed listening address stays in TIME_WAIT, which otherwise prevents an immediate restart.

Example

   Socket lifecycle

   SERVER                                CLIENT
   socket()   create endpoint            socket()
   bind()     claim address:port
   listen()   mark passive, backlog N
   accept()   <=== blocks ==========     connect()   (TCP handshake)
     returns a NEW fd for this client
   read()/write() on the new fd  <---->  read()/write()
   close(client_fd)                      close()
   (listening fd stays open for more)

   Address families, one API

   AF_UNIX  : /tmp/app.sock     same machine, no network stack, fastest,
                                can pass file descriptors and peer credentials
   AF_INET  : 93.184.216.34:80  any machine, routed over IP

import socket

# --- Server: the four-call sequence ---
srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)     # TCP endpoint
srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)   # allow fast restart
srv.bind(("127.0.0.1", 9000))                               # claim the address
srv.listen(16)                                              # backlog of 16

conn, addr = srv.accept()      # blocks; returns a NEW socket for this client
with conn:
    # A stream has NO message boundaries: recv may return a partial message.
    # Real protocols frame explicitly -- here, a 4-byte length prefix.
    header = conn.recv(4)
    length = int.from_bytes(header, "big")
    body = b""
    while len(body) < length:                 # loop until the full message
        chunk = conn.recv(length - len(body)) # arrives; one recv is NOT enough
        if not chunk:
            raise ConnectionError("peer closed early")
        body += chunk
    conn.sendall(b"ok")

# --- Same API, local-only transport: a UNIX domain socket ---
import socket, os
path = "/tmp/app.sock"
if os.path.exists(path):
    os.unlink(path)                                   # stale socket files persist
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) # only the family changed
s.bind(path)
s.listen(1)

Key takeaways

• A socket is a communication endpoint; the same API serves local (AF_UNIX) and networked (AF_INET) processes.
• Stream sockets are reliable, ordered, and boundary-free; datagram sockets are best-effort with preserved boundaries.
• The server sequence is always socket, bind, listen, accept, and accept returns a new per-client descriptor.
• Streams require application-level framing; assuming one send maps to one recv is a persistent source of bugs.

🧪 Practice

1. Write an echo server and client over TCP, then convert both to AF_UNIX by changing only the family and address. Benchmark both.
2. Send two 10-byte messages back to back over TCP and show that the receiver may read 20 bytes at once. Fix it with length-prefix framing.
3. Interview-style: "Why does accept() return a new socket instead of reusing the listening one?" Hint: consider a server with 10,000 simultaneous clients and one listening address.

Remote Procedure Calls

Theory

Message passing works, but writing distributed code as explicit send/receive pairs is tedious and error-prone: every call needs manual serialisation, correlation of replies with requests, and error handling. Remote Procedure Call hides that behind the most familiar abstraction in programming — the function call. You write result = service.get_user(42); the runtime turns it into a message, sends it, waits, and returns the reply as a normal return value.

The machinery is a matched pair of generated stubs:

1. The client stub implements the function locally. It marshals the arguments into a serialised message, sends it, and blocks awaiting the reply.
2. The server skeleton receives the message, unmarshals the arguments, calls the real function, marshals the return value, and sends it back.
3. The client stub unmarshals the result and returns it, so the caller sees an ordinary function return.

Marshalling must handle representation differences across machines — byte order, word size, floating-point format, and pointer meaninglessness — which is why RPC systems define an interface in a neutral IDL (Protocol Buffers, Thrift, OpenAPI) and generate both stubs from it.

The essential caution — and the standard interview topic — is that RPC is not a local call, however much it looks like one. The differences are semantic, not cosmetic:

	Local call	Remote call
Latency	~1 ns	0.1-100+ ms
Failure modes	The function throws	Timeout, partial failure, lost reply, slow peer
Argument passing	By reference possible	By value only; pointers are meaningless
Partial failure	Impossible	The call may have executed even though you saw an error

That last row is the crux. If a reply is lost, the client cannot distinguish "not executed" from "executed, reply lost". This forces an explicit choice of delivery semantics: at-most-once (never retry; safe but may not happen), at-least-once (retry; requires idempotent operations), or exactly-once (only achievable with deduplication and sequence numbers, and never for free). Modern systems (gRPC, Thrift) therefore expose timeouts, retries, and deadlines as first-class concerns rather than hiding them.

Example

   Anatomy of an RPC

   CLIENT                                          SERVER
   ------                                          ------
   result = get_user(42)
        |
   client stub:
     marshal {method:"get_user", args:[42], id:7}
     send ------------------------------------->  receive
                                                  server skeleton:
                                                    unmarshal
                                                    call real get_user(42)
                                                    marshal {id:7, result:{...}}
     receive <--------------------------------- send
     unmarshal, match id 7 to the waiting call
        |
   returns User(...)   <- looks exactly like a local return

   What can go wrong between the two arrows:
     request lost      -> server never ran it
     reply lost        -> server RAN it; client sees a timeout
     server slow       -> client times out; server still executing
   The client cannot tell these apart. Hence idempotency and request IDs.

// The IDL: one neutral definition, both stubs generated from it.
syntax = "proto3";

service UserService {
  rpc GetUser (GetUserRequest) returns (User);
}

message GetUserRequest { int32 id = 1; }
message User {
  int32  id    = 1;
  string name  = 2;
  string email = 3;
}

import grpc, user_pb2, user_pb2_grpc

channel = grpc.insecure_channel("localhost:50051")
stub = user_pb2_grpc.UserServiceStub(channel)

try:
    # Looks like a method call; is a network round trip.
    user = stub.GetUser(user_pb2.GetUserRequest(id=42), timeout=2.0)
    #                                                   ^^^^^^^^^^^ never omit:
    # without a deadline a dead peer blocks this caller indefinitely.
    print(user.name)
except grpc.RpcError as e:
    # Failure modes with no local equivalent -- and note that DEADLINE_EXCEEDED
    # does NOT mean the server did nothing.
    if e.code() == grpc.StatusCode.DEADLINE_EXCEEDED:
        print("timed out; the call may or may not have executed")
    elif e.code() == grpc.StatusCode.UNAVAILABLE:
        print("peer unreachable; safe to retry only if idempotent")

Key takeaways

• RPC makes a remote request look like a function call by generating client stubs and server skeletons from an IDL.
• Marshalling handles serialisation and machine representation differences; pointers cannot be passed.
• RPC is not a local call: latency is millions of times higher and partial failure is possible.
• A lost reply is indistinguishable from a lost request, so choose at-most-once or at-least-once semantics deliberately and make retried operations idempotent.

🧪 Practice

1. Define a two-method service in an IDL of your choice, generate the stubs, and implement both sides locally.
2. Add an artificial 5-second delay in the server and a 1-second client deadline. Show that the server still completes the work after the client gives up, and explain the implication.
3. Interview-style: "How would you make a transfer_money RPC safe to retry?" Hint: think about a client-supplied identifier and what the server must remember.

---

<a id="3-threads-and-concurrency"></a>