In this lecture we explore what changes when communication leaves the machine. We treat the NIC as an I/O device, explain why packets exist as discrete metadata-rich transfer units, introduce addressing and forwarding, and trace the send and receive paths through the kernel's network stack.
Lecture Date
📅 April 20, 2026
Standard
I/O and Networking
Topics Covered
Network Interface CardsPackets and FramesMAC and IP AddressingProtocol LayeringSend and Receive Paths
Last lecture we explored the design space of inter-process communication:
Isolation creates the need — Processes cannot simply reach into one another's memory; the kernel must provide sanctioned channels
Different mechanisms, different tradeoffs — Pipes, FIFOs, signals, message queues, shared memory, and Unix domain sockets each occupy a niche defined by data movement, structure, direction, naming, and coordination burden
The kernel is not a process — It is privileged code that executes using borrowed process context, mediating every channel
Sockets dissolve boundaries — Unix domain sockets use the same API as network sockets, turning the difference between local and remote communication into a deployment detail rather than an architectural one
Three luxuries — All IPC enjoyed a shared kernel, optional shared memory, and a shared scheduler
That left us with a clear next question: what happens when those three luxuries disappear? What changes when the other endpoint lives on a different machine — with its own kernel, its own memory, and its own scheduler?
Today we follow a message out of the machine and see what the operating system must do differently when communication can no longer rely on shared local state.
Today's Agenda
When Communication Leaves the Machine — The three luxuries IPC enjoyed and why they vanish
The NIC Is Still Just I/O — Keeping networking anchored in the driver, DMA, and interrupt story
Why Packets Exist — Discrete transfer units with enough metadata to survive unshared, unreliable links
Addresses and Hops — Local-link delivery versus routed delivery across multiple machines
Layering as Division of Labor — The standard models, and building the packet layer by layer
The Send Path — From application intent to wire transmission
The Full Journey — End-to-end animation across machines and routers
The Receive Path — From wire to application delivery
The IPC Mirror — What the kernel collapses when communication stays local
Looking Forward — Packets move, but the OS still owes applications usable transport abstractions
When Communication Leaves the Machine
In LN20, IPC worked because both endpoints shared everything that matters. One kernel saw both processes. One scheduler knew when each was ready, blocked, or running. Shared memory was physically possible because both processes mapped the same RAM. The kernel could enforce ordering, manage buffers, and guarantee delivery — because it was the sole authority over both sides of the conversation.
Networking begins where those luxuries end.
What Is Lost
IPC Luxury
What Happened Locally
What Happens Remotely
Shared kernel
One OS mediates both endpoints, enforcing rules and managing buffers
Each machine has its own OS; no entity sees both sides
Shared memory
Processes can map the same physical region for zero-copy transfer
Physically impossible — the machines have separate RAM
Shared scheduler
The kernel knows when both endpoints are ready
Neither scheduler knows about the other machine's processes
Local IPC vs Remote Communication
IPC enjoys a shared kernel. Networking begins where those luxuries vanish.
With no single authority, new problems appear:
No guaranteed delivery — data can be lost on the wire, corrupted in transit, or arrive out of order
No shared clock — the sender and receiver cannot agree on timing without explicit coordination
No shared naming — file paths and PIDs are meaningless across machines; a new naming system is needed
No instant notification — the receiver cannot be woken by the sender's kernel; the notification must travel physically
🤔 The Fundamental Shift: IPC is communication under centralized control. Networking is communication under no centralized control. Every mechanism we build from here forward exists to compensate for what was lost when the conversation left the machine.
But the Pattern Is the Same
Despite everything that changes, the U-bend pattern from LN19 still holds. A process makes a request. The kernel mediates. Data crosses a boundary. The result comes back. What is different is the width of the bend — it now stretches across a wire, through hardware on both sides, and across two separate kernels.
💡 Connection to LN19: The fractal U-bend observation applies directly. The network is a U-bend nested inside the application's U-bend: the local kernel descends into its NIC driver, the data crosses the wire, and the remote kernel ascends from its NIC driver to deliver the data. Each side runs its own half of the horseshoe.
The NIC Is Still Just I/O
Before networking starts to feel like a separate discipline, let's anchor it back where it belongs: the I/O story.
A Network Interface Card (NIC) is a device. It has a controller, it connects to the host through a bus (typically PCIe), it uses DMA to move data, and it raises interrupts when work completes. From the kernel's perspective, it is managed by a driver, just like the temperature sensor in LN19.
What Makes the NIC Special
The NIC follows every I/O pattern we have already studied:
I/O Concept
How It Appears in the NIC
Controller / functional hardware
Controller: PCIe interface, MAC engine, descriptor rings. Functional: PHY transceiver, signal encoding
DMA
Packets are DMA'd between NIC buffers and host memory — the CPU does not copy each byte
Interrupts
The NIC raises an interrupt when a packet arrives or a transmission completes
Driver
A kernel module that programs the NIC's descriptor rings, manages buffers, and handles interrupts
Buffering
Ring buffers hold packets waiting for the kernel to process them
💡 Connection to LN17: The NIC appears in our system interconnect diagram as a PCIe device with a controller interface and a PHY transceiver. It is the same device anatomy — controller plus functional hardware — that every I/O device exhibits.
📌 Callback to LN18: Network devices were the exception to "everything is a file." They do not appear in /dev/ — they use the socket API instead. Now we can see why: networking does not fit the sequential open/read/write/close model because connections are stateful, bidirectional, multiplexed, and addressed to remote endpoints. The socket API exists because the file abstraction broke down here.
NIC Device Anatomy — I/O for External Connections
Hover Send or Receive to trace data flow through the NIC.
The NIC's Job
The NIC performs a narrow but critical function:
On send: accept a packet from host memory (via DMA), encode it for the physical medium (Ethernet, WiFi), and transmit it onto the link
On receive: capture a signal from the physical medium, decode it into a packet, DMA it into host memory, and interrupt the kernel
Everything above that — routing, addressing, protocol handling, delivery to the right process — is the kernel's responsibility, not the NIC's. The NIC moves bits onto and off of a wire. The kernel gives those bits meaning.
Why Packets Exist
In IPC, the kernel could move data as a continuous byte stream (pipes) or as discrete messages (queues). Both worked because the kernel managed the buffer and both endpoints were local.
Once data must survive outside the machine, a byte stream is no longer viable. There is no shared buffer to stream through. Data must be packaged into self-contained units that can be handled independently by every machine and link along the path.
Why Not Just Stream Bytes?
In local IPC, the kernel provided context that byte streams could rely on:
What the Kernel Provided Locally
Why Packets Must Carry It Instead
The kernel knew who the sender was
The packet header must identify the source
The kernel knew who the receiver was
The packet header must identify the destination
The kernel enforced ordering
The packet header must include sequence information
The kernel detected errors via memory protection
The packet header must include error-detection codes
The kernel guaranteed delivery
The protocol must handle retransmission if needed
Every header field exists because some piece of information that was free in local communication must now be explicit in each transfer unit.
🤔 Key Framing: Packets are the tax you pay for leaving the machine. Every header byte compensates for shared state that no longer exists.
Stage 0 — Raw Payload
What IPC moved: just raw bytes, no metadata needed.
Frames vs Packets
The terminology reflects the layered nature of networking:
A packet is placed inside a frame for each physical hop, and the frame is stripped and rebuilt at every router. The packet survives the entire journey; the frame is local to each link.
💡 Key Insight: Frames are ephemeral — they live and die on each link. Packets are durable — they survive the entire journey from source to destination. The frame is the envelope for one leg of the trip; the packet is the letter inside.
Addresses and Hops
A packet knows where it wants to go. But how does it get there? The answer depends on whether the destination is on the same physical link or across many links.
Two Kinds of Addresses
Property
MAC Address
IP Address
Scope
One physical link
Global (end-to-end)
Assigned by
Hardware manufacturer
Network configuration (DHCP, static)
Changes with location?
No — burned into the NIC
Yes — depends on which network you join
Used by
Link layer (Ethernet, WiFi)
Network layer (IP)
Analogous to
A person's fingerprint
A person's mailing address
📌 Key Point: MAC addresses answer "who is next on this link?" IP addresses answer "who is the final destination?" Both are needed because most communication crosses multiple links to reach its destination.
Routing: One Hop at a Time
When the destination is on the same local link, the sender can deliver the frame directly using the destination's MAC address. When the destination is on a different network, the packet must be routed — forwarded hop by hop through one or more routers until it reaches the destination's local link.
Packet Created
Source wraps payload in IP + TCP headers. IP src/dst stay constant for the entire journey.
Step 1 / 7
At each hop:
The router receives a frame on one link
It strips the link-layer frame to reveal the packet inside
It reads the destination IP address from the packet header
It consults its routing table to determine the next hop
It wraps the packet in a new frame addressed to the next hop's MAC address
It sends the new frame onto the outgoing link
The packet survives the entire journey unchanged. The frames are rebuilt at every hop. This is why networking needs two kinds of addresses — one for each link segment, and one for the overall destination.
💡 Key Insight: Routing is postal delivery. The letter inside (the packet) always says the same destination address. The envelope (the frame) gets replaced at every post office along the way, addressed to the next office in the chain. No single post office needs to know the full route — it only needs to know the next hop.
Layering as Division of Labor
By now we have seen several distinct concerns in network communication:
moving bits across a physical link
addressing and forwarding packets across multiple links
ensuring reliability, ordering, and flow control for applications
Trying to solve all of these in one mechanism would produce an unmanageable tangle. Networking therefore splits the work into layers, each responsible for one concern.
This division is not an invention of this course. It is an industry-standard framework with two common models:
The Standard Models
TCP/IP Layer
OSI Equivalent
What We Call It
Job
Application
Application + Presentation + Session
(covered in LN22 and beyond)
Application-specific protocols
Transport
Transport
Transport
Ports, reliability, ordering, flow control
Internet
Network
Network
IP addressing, routing across many links
Link
Data Link
Link
Frames, MAC addresses, one-hop delivery
(Physical)
Physical
(folded into Link)
Electrical signals, encoding — handled by the NIC
The OSI model uses 7 layers; the TCP/IP model compresses them to 5 (or sometimes 4). For an OS course, the TCP/IP model is more practical because it matches how real network stacks are implemented. The OSI model's upper three layers (Application, Presentation, Session) collapse into one, and the Physical layer is folded into Link because the OS rarely deals with signal encoding directly — the NIC handles that.
📚 Historical Note: The OSI model was designed by committee in the 1970s-80s as an international standard. The TCP/IP model was designed by practitioners (Cerf, Kahn, Clark) building the actual Internet. The practitioner model won in implementation; the committee model survived in textbooks. Both describe the same separation of concerns.
Building the Packet, Layer by Layer
Each layer adds its own header when sending. Let's watch the packet grow as we walk through each layer.
Transport layer — The kernel adds a transport header: source port, destination port, and (for TCP) a sequence number and checksum. The application data is now a segment (TCP) or datagram (UDP).
Stage 1 — Transport
The kernel adds ports, sequence numbers, and a checksum.
Network layer — The kernel adds an IP header: source IP address, destination IP address, TTL (time to live), and a protocol field identifying the transport layer above. The data is now a packet.
Stage 2 — Network
The kernel adds IP addresses, TTL, and a protocol identifier.
Link layer — The kernel determines the next hop's MAC address, adds a frame header (source MAC, destination MAC, EtherType) and appends a CRC for error detection. The data is now a frame — ready for the wire.
Stage 3 — Link (Complete Frame)
MAC addresses and a CRC complete the frame for the wire.
The result is an envelope-in-envelope structure. Each layer wraps the output of the layer above it:
[ Link Header [ Network Header [ Transport Header [ Application Data ] ] ] CRC ]
Why Layering Matters
Layering is not just an organizational convenience. It is a separation of concerns that lets each layer evolve independently:
The link layer can change from Ethernet to WiFi without affecting the IP layer
The network layer can route packets through any number of hops without knowing whether the transport provides reliability
The transport layer can offer different guarantees (reliable streams, unreliable datagrams) without knowing what physical medium carries the data
💡 Connection to LN18: This is the same principle as device independence in I/O software. Just as the OS provides a uniform interface to applications regardless of whether the underlying device is an SSD or a USB drive, the network stack provides a uniform transport interface regardless of whether the link is Ethernet, WiFi, or cellular.
📌 Key Point: Each layer has one job. The link layer asks "how do I move this frame to the next device?" The network layer asks "how do I get this packet to the right machine?" The transport layer asks "how do I give the application the communication behavior it needs?" No layer tries to answer the others' questions.
The Full Journey — Send, Transit, and Receive
Let's follow our toy example end-to-end: a process wants to send "hello" to a process on another machine. This interactive walkthrough traces every layer of the send path, through two routers, and back up the receive path — combining everything we've built so far into one continuous animation.
Send Path
Application: write()
The process calls write() on a socket — a syscall, the same left-descent into the kernel we traced in LN19. Raw payload crosses the user/kernel boundary.
1 / 20
📌 Key Point: The process never touches the wire. Every step after the syscall is kernel code and hardware doing privileged work — the same pattern we traced through LN19's U-bend. The application expressed intent. The kernel and NIC made it physical.
🤔 Key Observation: Watch how the packet's interior never changes during transit. Every router only touches the outermost layer (the frame). The packet core — your application data wrapped in transport and network headers — travels the entire distance untouched. This is encapsulation in action.
💡 Connection to LN19: The receive path is the driver U-bend, extended through multiple protocol layers. The NIC is the device. The interrupt handler is the top half. The protocol processing is deferred work. The sleeping process wakes when the data reaches its socket buffer. Every concept from the driver lecture reappears here — just with more layers of interpretation between the hardware event and the application.
🤔 Notice the symmetry: The send path wraps headers layer by layer going down. The receive path strips headers layer by layer going up. Encapsulation on send, decapsulation on receive. The structure is perfectly mirrored.
The IPC Mirror — What the Kernel Collapses
Now that we have seen the full network journey, let's revisit local IPC from LN20 through this new lens. When two processes communicate via a Unix domain socket on the same machine, the code path mimics a network exchange — the process calls write() on a socket, the kernel receives the data, and the receiving process calls read() on its socket. The API is identical.
But look at what vanishes:
Network Journey Step
What Happens in Local IPC
Transport header added
The kernel can skip transport encoding — both endpoints share the same kernel memory
IP header added
No IP address needed — the kernel knows both endpoints directly
Frame header + CRC added
No frame needed — there is no physical link to cross
NIC DMA, wire transmission
No NIC involved — data stays in kernel buffers
Router frame strip/rebuild
No routers — there is no routing
Receive-side NIC, interrupt
No interrupt — the kernel is already holding the data
Transport/Network header strip
Nothing to strip — headers were never added
The kernel collapses the entire network stack because it is the single authority over both endpoints. It doesn't need packets, headers, or physical transmission — it can simply copy from one socket buffer to another.
The IPC Mirror — What the Kernel Collapses
Same API. The kernel replaces the entire network.
💡 This is the LN20 insight, visualized: Unix domain sockets use the network API without the network cost. The socket abstraction gives applications a uniform interface; the kernel decides whether the data needs to leave the machine. When it doesn't, the kernel short-circuits the entire stack.
Looking Forward
This lecture explained how packets move: from application through the kernel stack, through the NIC, across the wire, and back up through the remote kernel stack to the remote application. That is the physical story of networking.
But packets alone are too raw for most software. Consider what the application must deal with if it works directly at the packet level:
packets can arrive out of order
packets can be lost entirely
packets can be duplicated
packets can arrive faster than the receiver can process them
multiple applications on the same machine share one IP address
None of these problems existed in local IPC. The shared kernel handled ordering, guaranteed delivery, prevented duplication, managed flow control, and multiplexed by file descriptor.
The next lecture asks: how does the OS rebuild those guarantees on top of unreliable packet delivery? The answer is transport protocols — TCP and UDP — and the socket abstraction that lets applications program against them.
💡 Preview: TCP rebuilds the reliable, ordered byte stream that pipes provided locally — but over unreliable packets. UDP provides the lightweight, boundary-preserving message delivery that signals and datagrams provided locally — but without guarantees. The socket API unifies both behind one interface, just as file_operations unified block and character devices in LN18.
Summary
Networking begins where IPC's three luxuries — shared kernel, shared memory, shared scheduler — disappear
The NIC is still just an I/O device: it has a controller, uses DMA, raises interrupts, and is managed by a driver
Packets exist because data leaving the machine must carry its own context — every header field replaces shared state that was free in local communication
Frames are link-local transfer units that are rebuilt at every hop; packets survive the entire journey end-to-end
MAC addresses identify devices on one physical link; IP addresses identify endpoints across the entire network
Routers forward packets hop by hop, replacing frames at each link while preserving the packet inside
Layering is standardized (TCP/IP and OSI models) and separates concerns: link (one hop), network (many hops), transport (application behavior)
The packet builds up progressively: raw payload → transport header wraps it → network header wraps that → frame header and CRC complete it for the wire
The send path wraps headers layer by layer as data descends from application to wire
The receive path strips headers layer by layer as data ascends from wire to application — the perfect mirror of the send path
Both paths are extensions of the driver U-bend from LN19 — the same I/O patterns with additional protocol interpretation
Local IPC collapses the entire network stack — when both endpoints share a kernel, all headers, routing, and physical transmission vanish; the kernel short-circuits to a buffer copy
Packets alone are too raw; the next lecture explains how transport protocols rebuild usable communication abstractions on top of them
📝 Lecture Notes
Key Definitions:
Term
Definition
Network Interface Card (NIC)
An I/O device specialized for sending and receiving packets, using DMA and interrupts like any other device
Packet
A discrete unit of network data containing a payload and headers with metadata (source, destination, protocol, error detection)
Frame
A link-layer transfer unit — a packet wrapped in metadata for one physical hop (MAC addresses, CRC)
MAC Address
A hardware identifier for local-link delivery — which device on this link should receive the frame
IP Address
A logical identifier for end-to-end delivery — which machine across the network is the final destination
Router
A device that forwards packets between links by examining destination IP addresses and consulting routing tables
Frame vs Packet:
Property
Frame
Packet
Layer
Link
Network
Addressing
MAC addresses (local link)
IP addresses (end-to-end)
Lifespan
One physical hop
Entire source-to-destination journey
Rebuilt at routers?
Yes — new frame for each link
No — packet contents persist
What IPC Had vs What Networking Must Build:
IPC Luxury
Networking Replacement
Shared kernel mediates delivery
Each side has its own kernel; protocols coordinate
Shared memory for zero-copy
Data must be serialized into packets and physically transmitted
Shared scheduler for timing
No shared clock; protocols must handle timing and ordering