udpscope/docs/superpowers/specs/2026-10-09-udp-packet-capture-design.md

UDP packet capture: design

todo.md item 4: optional udp frame storage for debugging of strange behaviours (store last <N>, default = 1000, udp packets binary content on disk for forensic analysis).

The strange behaviours worth forensics on are all wire behaviours — a counter that skips, a datagram that arrives twice, a fragment that never completes, a CONFIG whose header does not parse, a burst that outruns the reorder ring. Every one of them is invisible by the time a udps_frame_t reaches Acquisition::on_data: the client has already validated the magic, reassembled the fragments, resequenced what it could and dropped what it could not. A capture that is to answer what did the wire actually send must therefore sit at recv(), and must store the bytes without interpreting them.

This is a debugging aid, not a feature of the instrument. It is off by default, it adds nothing to the glance tier of the status bar, and it must not be able to cost a single packet when it is on.

The decisions taken, and what each one rules out

Decision	Alternative rejected
Always-on rolling ring while enabled; the last N are on disk at all times	Keeping the ring in RAM and flushing on an anomaly or on demand — a crash or a SIGKILL then leaves nothing, which is the case a forensic capture exists for
A fixed-size file of N slots, plus a converter to pcap in tools/	Writing pcap directly (grows, so "last N" needs two rotating files and the newest N is split across both) or one file per datagram (1000 files, an open/write/close per packet in the receive path)
mmap once, memcpy per datagram	A pwrite per datagram (a syscall, and on a busy disk a blocking one, between two recv() calls) or a third thread with a queue (a second copy of every datagram, and this application's two-thread property is load-bearing)
One path per source, previous capture kept one deep	A timestamped file per enable (needs the retention policy todo.md item 9 defines and this does not build) or a single path overwritten (re-enabling destroys the capture you enabled for)
--capture-packets[=N] as well as the popover button	Button only — by the time the window is up and clickable, the first connect and its CONFIG are already history, and those are exactly the "strange behaviours" hardest to reproduce

Architecture

Three parts, each with one responsibility and a seam that does not leak the other two.

1. The client hands over raw datagrams

deps/udpsclient gains one additive entry point:

/** Every datagram as recv() returned it, before the magic check, reassembly or
 *  resequencing: the bytes a capture has to store, and the only place malformed
 *  or duplicate datagrams are still visible. Runs on the polling thread, inside
 *  udps_client_poll(), like every other callback. `bytes` is valid until the
 *  callback returns. */
typedef void (*udps_datagram_cb)(const uint8_t *bytes, size_t len,
                                 double recv_time, void *user);

void udps_client_set_datagram_cb(udps_client_t *client,
                                 udps_datagram_cb cb, void *user);

A separate setter, not a fourth parameter on udps_client_set_callbacks (udps_client.h:283): that signature has call sites, and a capture hook is no reason to change them.

The call site is udps_client.c:1458, immediately after

ssize_t n = recv(c->udp_fd, c->rxbuf, UDPS_RX_BUF_BYTES, 0);

returns a positive n, and before anything is read out of rxbuf. recv_time is the same clock the client already stamps arrivals with, so a capture's times and the stream statistics agree.

Boundary, stated rather than discovered later: in multicast mode CONFIG arrives over TCP (udps_client.c:1142) and this hook does not see it. The capture is of the UDP socket. A malformed multicast CONFIG is therefore out of reach, and README.md will say so.

2. PacketCapture owns the file

New unit, src/capture.hpp / src/capture.cpp. One class, one responsibility — own the ring file and store a datagram into it. It includes no GTK, no ScopeModel, and no client types beyond <cstdint>; its whole interface is:

class PacketCapture {
 public:
  /// Renames an existing capture to `capture-prev.bin`, creates and reserves the new file,
  /// maps it. Returns false and leaves the object disabled on any failure, with `error()` set.
  bool open(const std::string& path, std::uint32_t slot_count);
  void close();                                   ///< unmap, close; idempotent
  bool enabled() const noexcept;

  /// One datagram. Called only from the producer thread. Never allocates, never blocks,
  /// never fails: an over-long datagram is truncated to the slot and counted.
  void store(const std::uint8_t* bytes, std::size_t len, double recv_time) noexcept;

  std::uint64_t stored() const noexcept;          ///< datagrams since open()
  std::uint64_t truncated() const noexcept;
  std::uint32_t slot_count() const noexcept;
  const std::string& path() const noexcept;
  /// Copies the last open() failure out, or writes an empty string. Not a
  /// `const std::string&`: see "What crosses between the threads" below.
  void error(char* out, std::size_t cap) const noexcept;
};

What crosses between the threads

The producer thread writes this state and the GTK thread reads it every frame for the popover, so each field has to be publishable without a lock:

Field	Type	Why it is safe
enabled_, stored_, truncated_	std::atomic<…>, relaxed	counters and a flag, not a consistent set — the same treatment StreamStats already gets
path_	std::string	written once by the GTK thread before it ever sets capture_want_, and only read afterwards
the open failure	char error_[128] plus std::atomic<bool> error_set_	the producer fills the buffer and then sets the flag with a release store; the GTK thread copies the buffer only when the flag reads true

The error is deliberately not a std::string accessor. A std::string written by the producer inside open() and read by the GTK thread on the next frame is a data race with a reallocating buffer — the kind that reads correctly for months. The fixed buffer with a flag published after it is the same release-store trick the slot's seq field uses, for the same reason.

The file

offset 0, 64 bytes — header
  magic        u8[8]  "UDPSCAP\0"
  version      u32    1
  slot_count   u32    N
  slot_bytes   u32    sizeof(record header) + UDPS_RX_BUF_BYTES, 8-byte aligned
  cursor       u32    next slot index to write (a hint; see below)
  t0_realtime  f64    CLOCK_REALTIME at open(), the epoch recv_time is relative to
  peer_addr    u32    the source's IPv4, network order
  peer_port    u16
  local_port   u16
  reserved             to 64 bytes, zeroed

offset 64 + i * slot_bytes, for i in [0, N) — slot i
  t_arrival    f64    the client's recv_time
  len          u32    bytes stored (after truncation, if any)
  orig_len     u32    bytes the wire delivered, so a truncation is visible
  bytes        u8[len]
  seq          u64    1-based ordinal since open(); written LAST

seq is written last, after the bytes, with a release store. Three things follow, and they are the reason the format is this and not simpler:

• A slot carrying a non-zero seq is complete. No fsync is needed for the file to be readable after a crash: the kernel's dirty pages are the file, and a half-copied slot still has seq == 0.
• seq, not cursor, is the order. After the ring wraps, the newest slot is the one with the largest seq, and the reader needs no trust in a header field that is updated out of band.
• A torn write is detectable, not silent. The reader reports the count of empty and torn slots rather than averaging over them.

cursor is maintained anyway, because a human running xxd on the file wants to know where to look.

Why the file is reserved, not sparse

posix_fallocate reserves the whole file at open(). The obvious alternative — ftruncate and let the file stay sparse, so it costs only the bytes actually written — is wrong with mmap: a store into a page the filesystem cannot back raises SIGBUS, which kills udpscope outright. A capture must not be able to take down the instrument it is observing, and an out-of-space condition must be an error at the moment the user clicks, not a crash ten minutes later.

At the default N = 1000 and UDPS_RX_BUF_BYTES = 65552, the file is ~62.6 MiB. That is stated in the help text, in the tooltip and in the popover row, because it is the one cost of switching the feature on.

3. Acquisition drives it, and the GTK thread only ever sets an atomic

Acquisition owns the PacketCapture, installs the datagram callback next to the three it already installs (src/acquisition.cpp:62), and does every file operation on the producer thread.

GTK thread (the button)              producer thread (the datagram callback)
---------------------------          --------------------------------------------
capture_want_.store(true)    ──►     reads capture_want_ at the top of each datagram
capture_path_ written once           rising edge  → open() : rename, create, fallocate, mmap
  before the first store               falling edge → close(): munmap, close
                                     then, if enabled, store(bytes, len, recv_time)

No mutex, no queue, no second copy of any datagram, and the application still has exactly two threads.

The one-off cost of a toggle is the open(): a rename, a create, a posix_fallocate of 62.6 MiB and an mmap, on the producer thread. It is a user-initiated act that happens once per toggle, and README.md will say that it can cost the packets in flight during it. Moving it off the producer thread would need the lock or the thread this design exists to avoid, for an event that happens when a human clicks a button.

The duration is measured, not asserted: open() records its own wall time and the probe prints it as open_ms. posix_fallocate on a local extent-based filesystem is a metadata operation and should be a handful of milliseconds, but that is a claim about the filesystem, not about this code, and a reading above ~50 ms is a finding worth putting in README.md rather than a number to predict here.

Error handling, and only at this boundary

This is a syscall boundary, so it is validated; nothing else here is.

Condition	Behaviour
open, posix_fallocate or mmap fails (ENOSPC, EACCES, a read-only cache directory)	capture stays off, error() carries strerror, one line on stderr, the toggle button returns to off and the popover row shows the reason
the cache directory does not exist	created with g_mkdir_with_parents, 0700; a failure there is the row above
len > UDPS_RX_BUF_BYTES	impossible — the slot is sized to the client's own receive buffer — but store() truncates to the slot, records orig_len, and counts it in truncated() rather than trusting the impossibility
the previous capture-prev.bin cannot be replaced	the capture still opens; the rename failure is reported, the old previous is kept
capture enabled with no connection yet	the button is insensitive until connected, because the path is keyed on the source; --capture-packets defers open() to the first datagram, which is what arms it before the first CONFIG

Surfaces

The popover button and row

The status bar's detail popover (src/ui.cpp:7619-7672) is a two-column grid: four bold headings at rows 0, 3, 5, 6 and seven value labels at rows 0..6, in app->popover_label[11]. It gains a fifth heading, Capture, and an eighth value label; the array becomes [13] and the kHeadings / kHeadingRow tables gain their entry. update_status_popover (src/ui.cpp:6101) writes the new label like the others — gtk_label_set_text only, no rebuild.

The row reads one of:

off
on · 412/1000 slots · 26.1 MiB · ~/.cache/udpscope/192.168.1.40-5000/capture.bin
failed: No space left on device

Below the grid, a GtkToggleButton labelled Capture packets with the media-record-symbolic icon and a tooltip naming the full path and the file's size. toggled sets the atomic and nothing else. Insensitive until connected.

Nothing is added to format_detail()'s note string: src/status_text.cpp:86 is a char note[160] already close to its worst case, and the capture state is not something the exit dump's one line needs.

The command line

  --capture-packets N  store the last N raw datagrams on disk for forensics, in a
                       ring under the cache directory (1000 slots is ~62.6 MiB);
                       off unless given, and toggleable from the status bar

Parsed in src/config.cpp's existing style — both --capture-packets N and --capture-packets=N, N in 1..100000 — and the value is required, not optional. --capture-packets with an optional value cannot be told apart from --capture-packets followed by a value in this parser, and every other option in it takes the space-separated form, so a bare flag would turn --capture-packets 500 into unknown argument: 500. The 1000 default is the slot count the popover button uses when the flag was not given; with the flag, the button uses the N the flag named.

Config::capture_packets is a std::uint32_t: 0 means the flag was absent, and the button then uses 1000.

The flag arms the ring before the first datagram; it is not persisted in the session file. That persistence belongs with todo.md item 3's work, alongside the relative-time toggle deferred to the same place.

tools/capture_to_pcap.py

$ tools/capture_to_pcap.py ~/.cache/udpscope/192.168.1.40-5000/capture.bin out.pcap
1000 slots, 412 stored, 0 torn, 0 truncated → 412 packets, 26.1 MiB
$ wireshark out.pcap

Reads the header, validates the magic and version, walks the slots, orders by seq, skips seq == 0, and emits a classic pcap (LINKTYPE_ETHERNET, microsecond timestamps from t0_realtime + t_arrival) with a synthesised Ethernet/IPv4/UDP header per record built from the header's peer_addr, peer_port and local_port. Pure standard library — the mock already forces a hard numpy dependency on tools/, and this script has no reason to add to that.

It also prints the counts above to stderr, which is what makes it a diagnostic in its own right: torn or truncated above zero is a finding about the capture, before anything about the stream.

Verification

udpscope has no unit test suite; the capture is verified by the application's own headless self-check, and the converter by running it.

UDPSCOPE_PROBE_CAPTURE_MS toggles capture through the same handler the button uses, waits a scheduled interval, then reads the file back off disk and prints one line:

[capture-probe] enabled=1 path=… slots=1000 stored=412 cursor=412 torn=0 truncated=0
                header_ok=1 first_seq=1 last_seq=412 payload=1648000 file=65576064
                open_ms=3.1 t0=1760000000.123

payload is the sum of every stored slot's len — the bytes the wire actually delivered, which is what tells you whether the ring is holding what you think. file is the reserved size, 64 + slot_count * slot_bytes, and is a constant for a given N: it is in the line so that a mismatch against the file on disk is a reading rather than something to go and check.

What each field has to read, and the defect a wrong reading names, goes in docs/claude/probe-readings.md beside the other probes. Two cases in docs/claude/verification.md:

1. enable at runtime — the probe toggles on mid-run; stored must equal the datagrams the exit dump counts from that instant, torn=0, and first_seq=1.
2. wrap — --capture-packets=16 against the standard mock, run long enough to wrap several times; stored exceeds slot_count, last_seq - first_seq == 15, torn=0. A wrap that loses ordering, or a cursor that disagrees with seq, shows here and nowhere else.

Then the converter, run over the wrap case's file: exit 0, a record count equal to slot_count, and the pcap's own first and last record timestamps inside the run's wall-clock interval. The check is the converter's printed counts plus a struct.unpack of the pcap's first and last record headers — no Wireshark, no capinfos, nothing outside the standard library, so it runs wherever the self-check runs.

The standing baselines apply unchanged: a clean build at exactly four warnings, an incremental at two, and zero Gtk-CRITICAL in any headless run.

Out of scope

• Replaying a capture back into udpscope. That is todo.md item 5's territory (opening stored waveforms) and a different feature with a different shape.
• Retention beyond one deep. capture.bin and capture-prev.bin, nothing more, until item 9 defines the cache-size preferences.
• Session persistence of the enable state. Deferred to item 3 with the relative-time toggle, so one piece of work adds the session fields rather than two.
• Capturing the multicast TCP CONFIG channel. Stated as a limitation in README.md.
• Compression, encryption or rotation of the ring. A fixed file of raw bytes is what a forensic reader wants, and gzip exists.

Documentation to update

• README.md — the feature, the 62.6 MiB cost, the producer stall on toggle, the multicast CONFIG limitation, the converter's usage.
• SPECS.md — nothing: this is a todo.md item, not a spec requirement, and not a deviation.
• todo.md — item 4 struck, on one line, with the flag's name (GFM strikethrough is inline and cannot span list items).
• docs/claude/build-and-run.md — the new option; verification.md and probe-readings.md — the probe and its two cases; docs/claude/invariants-data-path.md — one paragraph on why the hook is at recv() and why the file is reserved rather than sparse.