← networks book ⊞ All topics

How the Internet Works End-to-End — A Packet's Journey

Typing https://example.com and seeing a page appear takes roughly a tenth of a second — and in that tenth of a second a name is looked up in a global directory, a hardware address is shouted for on the local wire, the request is wrapped in five nested layers of packaging, two separate negotiations happen before a single byte of content moves, and the data hops across dozens of independent networks and back. Every other networking topic is a zoom-in on one leg of this trip. Walk it once end-to-end and the rest of the field has somewhere to hang.

Key Components

IP address
The end-to-end identifier of a machine on the internet (e.g. 93.184.215.14). It names the final recipient of the journey and — this is the load-bearing part — it does not change as the data crosses the network.
MAC address
The hop-to-hop hardware address burned into a network chip. It identifies "the device physically next to me on this one link," and it is rewritten at every hop. IP and MAC are two kinds of address with two different scopes; conflating them is the single most common beginner error.
DNS (Domain Name System)
The internet's distributed phone book: name in, IP out. A resolver answers from cache when it can, otherwise it walks a hierarchy (root → .com top-level-domain server → the authoritative server for the domain). Answers carry a TTL (time-to-live) that bounds how long they may be cached.
Default gateway & ARP
A machine can only physically reach devices on its own local network (LAN). Anything else is handed to the default gateway — the router — to forward onward. To address a frame to that router, the machine needs the router's MAC, so it uses ARP (Address Resolution Protocol): "who has 192.168.1.1? Tell me your MAC."
Encapsulation
Wrapping data in nested layers, each adding its own header for its own job: HTTP inside TLS inside TCP inside IP inside Ethernet. Every layer reads only its own header and treats everything inside as opaque cargo. Decapsulation is the same process in reverse at the receiver.
TCP and UDP (the transport layer)
The two protocols that sit directly on top of IP and use port numbers to identify which application on the machine a message belongs to. TCP adds reliability — sequence numbers, acknowledgments, retransmission, ordering — behind a three-way handshake (SYN / SYN-ACK / ACK). UDP adds essentially nothing but ports.
Router, routing table, and best-effort delivery
A router joins networks. For each arriving packet it reads the destination IP, consults its routing table for the best next neighbour, rewrites the link-layer header, decrements the TTL hop counter, and forwards. No router knows the whole path. The network as a whole promises nothing — it may drop, duplicate, or reorder packets. That is what best-effort means.

Concrete Example

Take one laptop on a home network requesting https://example.com. Six things happen, in order.

Act 1 — name resolution. Machines route by IP address, not by name, so the very first step is a DNS lookup. It happens before any connection to the web server exists, and it usually rides on UDP:

laptop  → resolver : "A record for example.com?"        (UDP, port 53)
resolver→ root     : "who handles .com?"                 ┐
resolver→ .com TLD : "who is authoritative for example.com?"  ├ only on a cache miss
resolver→ auth NS  : "what is example.com?"              ┘
resolver→ laptop   : "93.184.215.14, TTL 3600"           (now cached for an hour)

Act 2 — leaving the local network. The laptop compares the destination IP against its own subnet. Not local, so the packet must go to the default gateway. It knows the gateway's IP (192.168.1.1) but needs its MAC to build a frame, so it ARPs for it:

laptop → (broadcast) : ARP "who has 192.168.1.1? tell 192.168.1.5"
router → laptop      : ARP "192.168.1.1 is at a4:2b:8c:11:9f:03"

Note carefully what was not asked for: the laptop never ARPs for the server's MAC. The server is not on this link, so its MAC is meaningless here — and unknowable. The laptop asks only "who is next?"

Act 3 — building the packet. Each layer wraps the one above it, adding a header for its own job. The real message ends up buried deepest, inside progressively more shipping-and-handling metadata:

        ┌─────────────────────────────────────────────┐
Frame → │ Eth │ IP │ TCP │ TLS │ HTTP: "GET /"         │ ← the actual request
        └─────────────────────────────────────────────┘
          ▲     ▲     ▲     ▲
          │     │     │     └─ encryption (the padlock)
          │     │     └─ "port 443, reliable, in order"
          │     └─ "from my IP → to 93.184.215.14"     (end-to-end, never changes)
          └─ "from my MAC → to router's MAC"           (this one hop only)

Act 4 — two handshakes before any content. TCP first agrees that both sides are talking, then TLS proves the server's identity with a certificate signed by a trusted authority and derives a shared secret:

laptop → server : SYN      "let's talk? here is my starting sequence number"
server → laptop : SYN-ACK  "sure — got yours, here is mine"
laptop → server : ACK      "confirmed, we are connected"
        ⇄        : TLS      certificate, key agreement
laptop → server : GET /    (encrypted)

This is why latency dominates page load: several full round-trips complete before one byte of HTML is requested.

Act 5 — crossing the internet. The internet is not one network. It is on the order of 75,000 independently operated networks — Autonomous Systems: ISPs, backbone carriers, data centres — that agree to pass each other's traffic. At every router the same tiny decision repeats: read the destination IP, look up the best next neighbour in the routing table, rewrite the link-layer header for that next hop, decrement TTL, forward. traceroute makes the chain visible:

$ traceroute example.com
 1  192.168.1.1        1.2 ms     ← home router (the gateway ARP found)
 2  10.24.0.1          9.8 ms     ← ISP access network
 3  ae-11.core1.isp.net  11.4 ms  ← ISP core
 4  peer.ix.example    24.7 ms    ← exchange point between two Autonomous Systems
 5  edge.example.com   26.1 ms    ← destination network

No single router on that list knows the whole route. Each knows only the next best step — like driving across a continent using nothing but road signs. Full path knowledge is distributed across thousands of routers and kept roughly in sync by routing protocols.

Act 6 — arrival, response, and the return trip. At the server (in practice often a load balancer or edge cache first) everything unwraps in reverse: Ethernet header stripped, IP checked, TCP matched to the right connection, TLS decrypted, and only then is the HTTP request read. The response runs the whole machine backwards — HTTP → TLS → TCP → IP with source and destination swapped → Ethernet — back through the routers, possibly along a different path, up the laptop's stack, and into the renderer. The returned HTML usually references more resources, triggering dozens more journeys, most of which reuse the already-open connection.

Visual Model

Think of posting a letter. The address on the envelope is the IP address: it names the final recipient and never changes, no matter where the letter is. But the letter does not teleport there. It goes mailbox → local post office → sorting hub → destination hub → carrier, and every one of those handoffs is a short-range, purely local relationship between two parties who can physically see each other. That handoff is the MAC address. The envelope is the plan; the handoffs are the execution. Once you hold those two apart, routing, switching, and ARP stop being mysterious.

The second model to burn in is smart edges, dumb middle. The routers in the core do one thing: forward packets toward a destination, best-effort, promising nothing. All the intelligence — reliability, ordering, retransmission, encryption — lives in the two endpoint machines. This is the exact inverse of the old telephone system (smart switching network, dumb handsets), and it is why the internet scaled: there is no central brain that has to be upgraded, and none that can fail.

Step through the full round trip below. Five participants, six acts, and — worth watching for — the destination IP stays constant the entire way while the link-layer addressing is rebuilt at every hop.

Step 1 of N
Laptop 192.168.1.5 DNS resolver port 53, UDP Home router gateway 192.168.1.1 Internet core ~75k networks example.com 93.184.215.14 "what is the IP for example.com?" "93.184.215.14" — cached until TTL expires ARP broadcast: "who has 192.168.1.1?" "…is at a4:2b:8c:11:9f:03" — the gateway's MAC encapsulation on the laptop — outermost header is the most local Eth IP TCP TLS HTTP: GET / this hop end-to-end port 443 encrypts SYN — "let's talk? here is my starting sequence number" SYN-ACK — "sure, got yours, here is mine" ACK — connection established (1 round trip spent) TLS — server proves identity with a signed certificate, both derive a shared key encrypted GET / at every router: read destination IP → routing table → next hop rewrite MAC header, TTL − 1, forward. IP unchanged. decapsulate up the stack: Eth → IP → TCP → TLS → HTTP 200 OK + HTML — re-encapsulated, possibly a different route home laptop decapsulates, TCP reorders, TLS decrypts, browser renders

Loading…

Misconception check: "if TCP runs only on the endpoints, is it even a protocol?"

Yes. A protocol is an agreed set of rules for a conversation — "if I send X, you reply Y." But rules have to be executed by something, and TCP's rules are implemented in software, specifically in the operating system kernel on each machine. So "TCP runs only on the endpoints" is a statement about where the code lives, not about whether TCP counts as a protocol. The routers in between do not run TCP at all: they read the IP header, forward, and never look inside.

   LAPTOP                 routers in the middle              SERVER
 ┌────────┐          ┌─────┐   ┌─────┐   ┌─────┐          ┌────────┐
 │  TCP   │──────────│ IP  │───│ IP  │───│ IP  │──────────│  TCP   │
 │(kernel)│          │only │   │only │   │only │          │(kernel)│
 └────────┘          └─────┘   └─────┘   └─────┘          └────────┘
     ▲                                                        ▲
     └────────── the TCP conversation is only between these two ──────┘

Routers are postal trucks: they carry the sealed envelope and never open it.

That framing also settles the usual TCP-vs-UDP confusion. Both are transport-layer protocols, both run as kernel software on the endpoints, and both use port numbers to say which application a message is for. The shared structure is why they feel similar. Their jobs, however, are opposites — read the row for UDP as a list of things it deliberately does not do:

Transport Detects loss Retransmits Restores order Handshake first Guarantees delivery Setup cost
UDP no no no no — just fire no none — send immediately
TCP yes yes yes yes — SYN / SYN-ACK / ACK yes, or reports failure a round trip, plus waiting on ACKs

The insight that makes the table unnecessary: raw IP gives only unreliable, best-effort delivery. UDP ≈ raw IP with almost nothing added — the thinnest possible wrapper, essentially just port numbers — so it exposes the network's native "no promises" nature directly to the application. TCP = raw IP plus a reliability machine (sequence numbers, acknowledgments, retransmission, reordering, flow control) that hides that unreliability behind a clean, ordered pipe.

So the instinct "isn't the underlying network sort of UDP-like already?" is exactly right. The network is unreliable. UDP leaves that showing; TCP papers over it.

One line to keep: the network is dumb and unreliable — UDP hands you that raw and fast, TCP does the extra work to hide it.

Which raises the obvious question: why would anyone choose UDP? Because reliability costs time. Handshakes, waiting for acknowledgments, and retransmission delays all add latency. For a video call or a game, a slightly-damaged frame delivered now beats a perfectly-recovered frame delivered 200 ms late. Speed over guarantees — that is the whole trade.

Deeper — Edge Cases & Gotchas

Packets are independent, and that is the point

Two packets belonging to the same request can take different routes through the internet and arrive out of order — or not at all. The network makes no promises, and that is a deliberate design choice, not a defect: packet switching rather than circuit switching. Nothing in the middle holds per-conversation state, so nothing in the middle is a single point of failure and nothing in the middle has to be upgraded when a new application protocol appears. Reliability is reconstructed at the edges by TCP, which notices gaps in sequence numbers and asks for the missing pieces again.

This has a practical consequence worth internalising: when a connection "feels slow," the middle of the network is almost never doing anything clever about it. What you are usually observing is TCP at the endpoints detecting loss and backing off.

The TTL field is a loop insurance policy

Because no router knows the full path, misconfigured or transiently inconsistent routing tables can send a packet in a circle. The TTL (time-to-live) counter in the IP header is decremented at every hop, and the packet is discarded when it reaches zero. Without it, a routing loop would accumulate packets forever. traceroute is a clever abuse of this mechanism: send packets with TTL 1, 2, 3… and each router in turn is forced to discard one and report back, revealing itself.

Why the whole path is not knowable from one place

Route knowledge is distributed and only roughly in sync, maintained by routing protocols — BGP between Autonomous Systems, OSPF inside one. During reconvergence after a link failure, different routers can briefly hold contradictory views of the best path. The design tolerates this precisely because packets are independent and TTL bounds the damage.

Anti-pattern: assuming a device should ARP for the destination it actually wants to reach.
laptop wants 93.184.215.14
  ✗  ARP: "who has 93.184.215.14? tell 192.168.1.5"
     → broadcast reaches only the local link
     → nobody on this LAN owns that IP
     → no reply, ever; the packet is never sent

  ✓  destination is outside my subnet
     → ARP for the DEFAULT GATEWAY's IP (192.168.1.1)
     → frame: dst MAC = router, dst IP = 93.184.215.14

Why it breaks: ARP is a broadcast on the local link, and the local link is the only place it can go. A machine on the far side of the internet cannot hear it and has no MAC address that means anything here. The fix is not a better ARP query — it is recognising that MAC addressing is always and only local. The IP header keeps naming the true destination; the Ethernet header names whoever is next. Every router then repeats this same "who's next?" question for its own next hop.

Anti-pattern: treating "UDP is unreliable" as a warning label rather than a specification, and reaching for TCP by default in latency-sensitive systems. TCP's guarantees are not free — a lost packet stalls everything behind it in the stream (the receiver cannot deliver later data until the gap is filled) while the sender waits out a retransmission timeout. For a live video call that means the picture freezes to recover a frame nobody will care about by the time it arrives. UDP's "unreliability" is the correct behaviour there: drop it and move on.
Anti-pattern: counting only the HTTP request when reasoning about page-load latency. Before the first byte of GET / leaves the machine, a cold load has already paid for a DNS lookup, a TCP three-way handshake, and a TLS handshake — several full round trips, each one bounded by the speed of light and the distance to the server. This is why connection reuse (keep-alive), DNS caching, and putting content physically closer to users are such large wins: they remove round trips, not bytes.

Test Yourself

The laptop wants to reach a server at 93.184.215.14. Why does it ARP for the router's MAC address rather than the server's?

What guarantees does the network in the middle make about delivering your packets, and who actually provides reliability?

A colleague argues that UDP must be "a different kind of thing" from TCP because TCP does so much more. Using the raw-IP framing, explain what UDP and TCP each add to IP — and what would change for a video call if it switched from UDP to TCP.