← Interview Prep

Ping & Traceroute

The two tools every network engineer reaches for first — and every field, every character, and every quirk an interviewer can ask you to explain.

Ping and traceroute look trivial from the CLI, but they are dense with detail: two ICMP message types, a fistful of header fields, and a set of output characters that each encode a precise failure mode. This page walks both tools field by field, decodes the Cisco IOS output alphabet, and explains the patterns interviewers love to probe — starting with the famous .!!!!.

Ping answers “is it reachable, and how fast?” Traceroute answers “which path does it take, and where does it break?” Both are built on the same primitive — ICMP running directly over IP — but they exploit different fields: ping uses Echo, traceroute weaponizes TTL.

How ping works

Ping sends an ICMP Echo Request and waits for the matching ICMP Echo Reply. ICMP is protocol number 1, carried directly inside an IP packet (there is no TCP or UDP layer — EtherType 0x0800 → IP proto 1).

The two message types

Direction	Type	Code	Meaning
Request (sent by pinger)	`8`	`0`	Echo Request
Reply (sent by target)	`0`	`0`	Echo Reply

A common gotcha: the Echo Reply is type 0, not type 8. The responder copies the payload back verbatim and flips the type field from 8 to 0.

The ICMP Echo header, field by field

Field	Size	Purpose
Type	1 byte	`8` request / `0` reply
Code	1 byte	`0` for echo
Checksum	2 bytes	Covers the ICMP header + payload
Identifier	2 bytes	Demultiplexes replies to the right process/socket. On Unix it is often the sending process's PID; behind NAT it doubles as the session key (ICMP has no ports).
Sequence Number	2 bytes	Increments per probe (1, 2, 3…). Lets the sender match each reply to its request and detect loss / reordering / duplicates.
Payload	variable	Arbitrary data, echoed back unchanged. Used to measure RTT and to test specific sizes/patterns.

Identifier + Sequence: how replies are matched

Because ICMP has no port numbers, the kernel needs another way to know which reply belongs to which request. The Identifier distinguishes one ping instance from another (and one host from another behind NAT), while the Sequence Number distinguishes the individual probes within that instance. When a reply arrives, the sender looks up the (Identifier, Sequence) tuple to find the timestamp it recorded at send time.

Payload and pattern

The payload is arbitrary but meaningful in practice. Linux ping stuffs a timestamp into the first bytes so RTT can be computed even if no local table is kept; the rest is a fixed incrementing pattern (0x10 0x11 0x12 …). Cisco IOS defaults to a 100-byte packet with a repeating 0xABCD pattern. You can change size and pattern to hunt for MTU issues or pattern-sensitive faults (e.g. ping ip 10.0.0.1 size 1500 df-bit).

TTL and RTT

TTL is set by the sender (commonly 64 on Linux/macOS, 128 on Windows, 255 on Cisco-originated traffic) and decremented by each router. The TTL in the reply hints at how many hops the return path took — a quick sanity check on path length and OS fingerprint.
RTT is measured entirely by the sender: it records a timestamp when the Echo Request leaves and subtracts it from the arrival time of the matching Echo Reply. The target does no timing at all — it just reflects the packet as fast as it can. That is why RTT includes both forward and return path latency, plus the responder's (usually tiny) turnaround.

A successful exchange, end to end

Sender builds an Echo Request: type 8, code 0, a chosen Identifier, Sequence 1, a payload with a timestamp.
Records the send time keyed by (Identifier, 1).
Packet is routed to the target (ARP for next-hop, TTL−1 at each router, as usual).
Target receives it, swaps type 8 → 0, keeps Identifier/Sequence/payload, recomputes the ICMP checksum, and sends the Echo Reply back to the source.
Reply routes home; the sender matches (Identifier, 1), computes RTT = now − send time, prints one ! (or one line on Unix).
Repeat with Sequence 2, 3, … until the count is exhausted.

$ ping -c 4 10.0.30.22
64 bytes from 10.0.30.22: icmp_seq=1 ttl=61 time=2.11 ms
64 bytes from 10.0.30.22: icmp_seq=2 ttl=61 time=1.83 ms
64 bytes from 10.0.30.22: icmp_seq=3 ttl=61 time=1.90 ms
64 bytes from 10.0.30.22: icmp_seq=4 ttl=61 time=1.88 ms

--- 10.0.30.22 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss
rtt min/avg/max/mdev = 1.83/1.93/2.11/0.10 ms

Cisco IOS ping output characters

Cisco IOS prints one character per probe. Memorize this table — it is a classic quick-fire interview question, and each symbol maps to a specific ICMP event.

Char	Meaning	What it really indicates
`!`	Reply received	An Echo Reply came back in time — success.
`.`	Timed out	No reply within the timeout window (default 2 s). Loss, filtering, or slow resolution — see below.
`U`	Destination unreachable	A router returned ICMP Destination Unreachable (type 3) — no route, host down, or admin filter.
`Q`	Source quench	Received an ICMP Source Quench (type 4) — a legacy congestion signal, now deprecated.
`M`	Could not fragment	ICMP type 3 code 4: packet too big and the DF bit was set, so a router had to drop it. The MTU black-hole symptom.
`?`	Unknown packet type	An ICMP message IOS didn't recognize / couldn't classify.
`&`	Packet lifetime exceeded	TTL hit 0 in transit — a router returned ICMP Time Exceeded (type 11). Usually a routing loop or a too-low initial TTL.
`I`	Interrupted	The operator aborted the ping (Ctrl-Shift-6 / break sequence).

The classic `.!!!!` pattern

You run a five-probe ping from a Cisco router and see:

R1# ping 10.0.30.22

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.30.22, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 1/2/4 ms

The first probe timed out (.), the next four succeeded (!!!!), giving 80 percent (4/5). This is normal, not a fault — and being able to explain why is the whole point of the question.

Why the first packet (or two) is dropped

When the router originates the very first Echo Request, it frequently does not yet have the L2 rewrite information it needs to forward the packet:

ARP / adjacency is incomplete. The router knows the next-hop IP but not its MAC. It must send an ARP Request and wait for the reply before it can build the frame.
CEF has no adjacency yet — a “glean” happens. In CEF, the FIB entry for a directly connected host points to a glean adjacency meaning “I know the prefix, but I still need to resolve this specific host's MAC.” The first packet is punted to the control plane (process-switched) to trigger that ARP resolution.
The first packet often dies waiting. While ARP resolves, that first Echo Request is either dropped or delayed past the 2-second timeout, so the sender prints ..

By the time probe 2 is sent, the ARP entry and CEF adjacency are fully populated, the packet is hardware/CEF-switched in the data plane, and every subsequent probe returns !. On a multi-hop path you may see ..!!! instead: the first probe can trigger resolution on more than one segment (or hit a control-plane punt at more than one device), so occasionally the first two probes are lost.

The one-liner for the interview: the first ping seeds the ARP/CEF adjacency; it is punted/process-switched while resolution completes and times out, then the populated entry lets the rest fly. An 80 percent success rate on the first ping to a new destination is expected behavior, not a problem.

How to prove it isn't a real fault

Ping again immediately — the second run is usually !!!!! (100 percent), because the adjacency is now cached.
show ip arp 10.0.30.22 / show adjacency — the entry now exists.
Pre-populating ARP (or a prior data flow) makes even the first ping succeed fully.

Why `.` appears intermittently later

A dropped probe in the middle of a run is a different story — the adjacency already exists, so it points to a genuine issue:

Real packet loss — congestion, a flapping link, a bad cable/optic, buffer drops on a busy interface.
ICMP rate-limiting — routers cap how many ICMP replies/errors they generate per interval (IOS ip icmp rate-limit). Under a burst, some Echo Replies simply aren't generated.
CoPP / control-plane policing — ICMP destined to or generated by the router is control-plane traffic. A CoPP policy can police it, dropping probes under load while data-plane forwarding is unaffected. This is a frequent cause of “ping to the router is lossy but traffic through it is fine.”
MTU / DF issues — if you bumped the size and set df-bit, an undersized link yields M (or silent . when ICMP errors are filtered), exposing a PMTUD black hole.
Asymmetric or unstable routing — a reconvergence event or a per-flow ECMP change can drop the odd packet.

Position matters. Loss at the start of a ping = resolution/punt, benign. Loss in the middle/steady state = real drops, rate-limiting, or CoPP. Always say which one you're looking at.

How traceroute works

Traceroute maps the path hop by hop by abusing the TTL field. It sends probes with deliberately small, incrementing TTLs and collects the ICMP errors each router returns.

The TTL trick

Send a probe with TTL = 1. The first router decrements it to 0, drops the packet, and returns ICMP Time Exceeded (type 11, code 0) — sourced from its own address. Hop 1 revealed.
Send TTL = 2. It survives hop 1 (TTL→1), dies at hop 2 (TTL→0), which returns Time Exceeded. Hop 2 revealed.
Increment TTL (3, 4, 5…) until a probe finally reaches the destination.
Typically three probes per TTL, so you see three latency samples per hop.

How the destination is detected

The Time Exceeded trick reveals intermediate hops, but the final hop is signalled differently depending on probe type:

UDP probes (classic Unix): aimed at a high, presumably-unused port. The destination has nothing listening, so it replies with ICMP Port Unreachable (type 3, code 3) — that's how traceroute knows it arrived.
ICMP Echo probes (Windows tracert): the destination returns a normal Echo Reply (type 0).
TCP SYN probes: the destination answers with SYN-ACK (open port) or RST (closed) — either way, proof of arrival.

Probe types compared

Probe	Default on	Intermediate hop signal	Destination signal	Firewall friendliness
UDP high ports	Linux/macOS `traceroute`	Time Exceeded (11/0)	Port Unreachable (3/3)	Often blocked; ports vary per probe
ICMP Echo	Windows `tracert`, `traceroute -I`	Time Exceeded (11/0)	Echo Reply (0/0)	Blocked wherever ICMP is filtered
TCP SYN	`traceroute -T`, `tcptraceroute`	Time Exceeded (11/0)	SYN-ACK / RST	Best — targets a real open port (e.g. 80/443) to slip through firewalls

Windows `tracert` vs Unix `traceroute` — the port question

A common interview framing: "can you point traceroute at a specific port?" — and the answer follows from which layer the probe lives at:

Windows tracert defaults to ICMP Echo. ICMP is an L3 protocol with no port field, so there is no port to specify — you're probing the host, not a service.
Unix traceroute defaults to UDP (an L4 protocol), so you can aim it at a port — and with -T (TCP) you target a real service port like 443. This is what lets TCP/UDP traceroute slip through firewalls that drop ICMP.

All three probe types still use the same TTL trick for intermediate hops; only the transport (and thus port-addressability + how the destination is detected) differs. Note Unix traceroute can also send ICMP with -I, so "traceroute = UDP" is only the default, not a hard rule.

A sample traceroute

$ traceroute 93.184.216.34
traceroute to 93.184.216.34, 30 hops max, 60 byte packets
 1  10.0.10.1      0.412 ms  0.388 ms  0.377 ms
 2  10.0.255.1     1.02 ms   0.98 ms   1.10 ms
 3  100.64.0.1     3.4 ms    3.2 ms    3.6 ms
 4  * * *
 5  203.0.113.9    12.1 ms   11.8 ms   12.4 ms
 6  198.51.100.6   14.9 ms   40.2 ms   14.7 ms
 7  93.184.216.34  15.1 ms   14.9 ms   15.0 ms

Reading the output like an engineer

* * * (all three probes lost at a hop). It does not necessarily mean the path is broken. Common causes: the hop rate-limits or deprioritizes its ICMP Time-Exceeded generation (CoPP), it is configured not to send ICMP errors at all, or a firewall filters the probe/response. If later hops still answer, the traffic clearly passed through the silent hop — so * * * in the middle is usually cosmetic. Only * * * all the way to the destination signals a real reachability problem.
Non-monotonic latency (hop 6 showing 40 ms while hops 5 and 7 are ~15 ms). This is expected and not a fault. The number is the RTT to that router's control plane generating the ICMP error — a task that is deprioritized relative to data-plane forwarding. A hop can be slow to reply while forwarding transit traffic at line rate. Judge a hop by whether later hops' latencies are sane, not by one hop's ICMP turnaround.
Asymmetric return paths. Each hop's Time Exceeded travels back over whatever path routing chooses from that router to you — which may differ from the forward path and from the neighboring hops' return paths. So per-hop RTTs blend forward + (varying) return latency; that alone can make numbers jump around.
ECMP load-balancing. With equal-cost paths, routers hash the packet header to pick an egress. Classic traceroute varies the UDP destination port (and thus the 5-tuple) on every probe, so different probes can hash onto different paths — producing phantom hops, inconsistent addresses at one TTL, or impossible-looking latencies. Paris-traceroute fixes this by holding the flow-identifying fields constant across probes so every packet follows one ECMP path, giving a stable, accurate trace.
MPLS core hidden by TTL no-propagate. In an MPLS core, if the ingress LSR is configured with no-propagate (no mpls ip propagate-ttl), the IP TTL is copied into the label only at ingress and back only at egress — transit LSRs decrement the label TTL, not the IP TTL. The result: the entire MPLS core appears as a single hop, hiding the P routers. (With TTL propagation enabled, or via RFC 4950 MPLS extensions, the core hops reappear, sometimes annotated with their MPLS labels.)

Going deeper / follow-up questions

Why is the ICMP Echo Reply type 0 and not type 8? (The responder flips the type field and reflects the payload.)
How does ICMP survive NAT with no ports? (The Identifier field becomes the session key, and NAT must rewrite addresses inside the embedded packet of ICMP errors.)
You ping a router and get 40 percent loss, but traffic through it is fine. Why? (CoPP / ICMP rate-limiting on the control plane vs. clean data-plane forwarding.)
Ping works but a large-payload transfer stalls. What's the failure? (PMTUD black hole — DF-set packets dropped with ICMP type 3 code 4 filtered; the M character.)
Traceroute shows * * * at hop 4 but the destination answers at hop 7. Is the path broken? (No — the silent hop simply doesn't return ICMP; traffic clearly passed it.)
Two consecutive traceroutes show different hop 5 addresses. Why, and how do you get a stable trace? (ECMP hashing per-probe; use Paris-traceroute.)
A provider's traceroute crosses a continent in “one hop.” How? (MPLS with TTL no-propagate hiding the P routers.)
Why can a middle hop show higher latency than the hops after it? (Deprioritized control-plane ICMP generation, not data-plane latency.)
Why does TCP traceroute sometimes succeed where UDP/ICMP is fully filtered? (It targets a real open service port like 443 that firewalls must permit.)

Interview Prep · Ping & Traceroute. Pair this with Life of a Packet for the underlying hop-by-hop forwarding model that both tools depend on.