udpscope/docs/superpowers/specs/2026-10-09-udp-packet-capture-design.md

<!-- SPDX-License-Identifier: MIT -->
<!-- SPDX-FileCopyrightText: 2026 Martino Ferrari -->

# UDP packet capture: design

`todo.md` item 4: *optional udp frame storage for debugging of strange behaviours (store last
`<N>`, default = 1000, udp packets binary content on disk for forensic analysis).*

The strange behaviours worth forensics on are all *wire* behaviours — a counter that skips, a
datagram that arrives twice, a fragment that never completes, a CONFIG whose header does not parse,
a burst that outruns the reorder ring. Every one of them is invisible by the time a
`udps_frame_t` reaches `Acquisition::on_data`: the client has already validated the magic,
reassembled the fragments, resequenced what it could and dropped what it could not. A capture that
is to answer *what did the wire actually send* must therefore sit at `recv()`, and must store the
bytes without interpreting them.

This is a debugging aid, not a feature of the instrument. It is off by default, it adds nothing to
the glance tier of the status bar, and it must not be able to cost a single packet when it is on.

## The decisions taken, and what each one rules out

| Decision | Alternative rejected |
| --- | --- |
| Always-on rolling ring while enabled; the last N are on disk at all times | Keeping the ring in RAM and flushing on an anomaly or on demand — a crash or a `SIGKILL` then leaves nothing, which is the case a forensic capture exists for |
| A fixed-size file of N slots, plus a converter to pcap in `tools/` | Writing pcap directly (grows, so "last N" needs two rotating files and the newest N is split across both) or one file per datagram (1000 files, an open/write/close per packet in the receive path) |
| `mmap` once, `memcpy` per datagram | A `pwrite` per datagram (a syscall, and on a busy disk a blocking one, between two `recv()` calls) or a third thread with a queue (a second copy of every datagram, and this application's two-thread property is load-bearing) |
| One path per source, previous capture kept one deep | A timestamped file per enable (needs the retention policy `todo.md` item 9 defines and this does not build) or a single path overwritten (re-enabling destroys the capture you enabled for) |
| `--capture-packets[=N]` as well as the popover button | Button only — by the time the window is up and clickable, the first connect and its CONFIG are already history, and those are exactly the "strange behaviours" hardest to reproduce |

## Architecture

Three parts, each with one responsibility and a seam that does not leak the other two.

### 1. The client hands over raw datagrams

`deps/udpsclient` gains **one additive entry point**:

```c
/** Every datagram as recv() returned it, before the magic check, reassembly or
 *  resequencing: the bytes a capture has to store, and the only place malformed
 *  or duplicate datagrams are still visible. Runs on the polling thread, inside
 *  udps_client_poll(), like every other callback. `bytes` is valid until the
 *  callback returns. */
typedef void (*udps_datagram_cb)(const uint8_t *bytes, size_t len,
                                 double recv_time, void *user);

void udps_client_set_datagram_cb(udps_client_t *client,
                                 udps_datagram_cb cb, void *user);
```

A separate setter, **not** a fourth parameter on `udps_client_set_callbacks`
(`udps_client.h:283`): that signature has call sites, and a capture hook is no reason to change
them.

The call site is `udps_client.c:1458`, immediately after

```c
ssize_t n = recv(c->udp_fd, c->rxbuf, UDPS_RX_BUF_BYTES, 0);
```

returns a positive `n`, and before anything is read out of `rxbuf`. `recv_time` is the same clock
the client already stamps arrivals with, so a capture's times and the stream statistics agree.

**Boundary, stated rather than discovered later:** in multicast mode CONFIG arrives over TCP
(`udps_client.c:1142`) and this hook does not see it. The capture is of the UDP socket. A malformed
*multicast* CONFIG is therefore out of reach, and `README.md` will say so.

### 2. `PacketCapture` owns the file

New unit, `src/capture.hpp` / `src/capture.cpp`. One class, one responsibility — own the ring file
and store a datagram into it. It includes no GTK, no `ScopeModel`, and no client types beyond
`<cstdint>`; its whole interface is:

```cpp
class PacketCapture {
 public:
  /// Renames an existing capture to `capture-prev.bin`, creates and reserves the new file,
  /// maps it. Returns false and leaves the object disabled on any failure, with `error()` set.
  bool open(const std::string& path, std::uint32_t slot_count);
  void close();                                   ///< unmap, close; idempotent
  bool enabled() const noexcept;

  /// One datagram. Called only from the producer thread. Never allocates, never blocks,
  /// never fails: an over-long datagram is truncated to the slot and counted.
  void store(const std::uint8_t* bytes, std::size_t len, double recv_time) noexcept;

  std::uint64_t stored() const noexcept;          ///< datagrams since open()
  std::uint64_t truncated() const noexcept;
  std::uint32_t slot_count() const noexcept;
  const std::string& path() const noexcept;
  /// Copies the last open() failure out, or writes an empty string. Not a
  /// `const std::string&`: see "What crosses between the threads" below.
  void error(char* out, std::size_t cap) const noexcept;
};
```

#### What crosses between the threads

The producer thread writes this state and the GTK thread reads it every frame for the popover, so
each field has to be publishable without a lock:

| Field | Type | Why it is safe |
| --- | --- | --- |
| `enabled_`, `stored_`, `truncated_` | `std::atomic<…>`, relaxed | counters and a flag, not a consistent set — the same treatment `StreamStats` already gets |
| `path_` | `std::string` | written **once** by the GTK thread before it ever sets `capture_want_`, and only read afterwards |
| the open failure | `char error_[128]` plus `std::atomic<bool> error_set_` | the producer fills the buffer and *then* sets the flag with a release store; the GTK thread copies the buffer only when the flag reads true |

The error is deliberately **not** a `std::string` accessor. A `std::string` written by the producer
inside `open()` and read by the GTK thread on the next frame is a data race with a reallocating
buffer — the kind that reads correctly for months. The fixed buffer with a flag published after it
is the same release-store trick the slot's `seq` field uses, for the same reason.

#### The file

```
offset 0, 64 bytes — header
  magic        u8[8]  "UDPSCAP\0"
  version      u32    1
  slot_count   u32    N
  slot_bytes   u32    sizeof(record header) + UDPS_RX_BUF_BYTES, 8-byte aligned
  cursor       u32    next slot index to write (a hint; see below)
  t0_realtime  f64    CLOCK_REALTIME at open(), the epoch recv_time is relative to
  peer_addr    u32    the source's IPv4, network order
  peer_port    u16
  local_port   u16
  reserved             to 64 bytes, zeroed

offset 64 + i * slot_bytes, for i in [0, N) — slot i
  t_arrival    f64    the client's recv_time
  len          u32    bytes stored (after truncation, if any)
  orig_len     u32    bytes the wire delivered, so a truncation is visible
  bytes        u8[len]
  seq          u64    1-based ordinal since open(); written LAST
```

`seq` is written last, after the bytes, with a release store. Three things follow, and they are the
reason the format is this and not simpler:

- **A slot carrying a non-zero `seq` is complete.** No `fsync` is needed for the file to be
  readable after a crash: the kernel's dirty pages are the file, and a half-copied slot still has
  `seq == 0`.
- **`seq`, not `cursor`, is the order.** After the ring wraps, the newest slot is the one with the
  largest `seq`, and the reader needs no trust in a header field that is updated out of band.
- **A torn write is detectable, not silent.** The reader reports the count of empty and torn slots
  rather than averaging over them.

`cursor` is maintained anyway, because a human running `xxd` on the file wants to know where to
look.

#### Why the file is reserved, not sparse

`posix_fallocate` reserves the whole file at `open()`. The obvious alternative — `ftruncate` and
let the file stay sparse, so it costs only the bytes actually written — is wrong with `mmap`: a
store into a page the filesystem cannot back raises **`SIGBUS`**, which kills udpscope outright.
A capture must not be able to take down the instrument it is observing, and an out-of-space
condition must be an error at the moment the user clicks, not a crash ten minutes later.

At the default `N = 1000` and `UDPS_RX_BUF_BYTES = 65552`, the file is **~62.6 MiB**. That is
stated in the help text, in the tooltip and in the popover row, because it is the one cost of
switching the feature on.

### 3. `Acquisition` drives it, and the GTK thread only ever sets an atomic

`Acquisition` owns the `PacketCapture`, installs the datagram callback next to the three it already
installs (`src/acquisition.cpp:62`), and does **every** file operation on the producer thread.

```
GTK thread (the button)              producer thread (the datagram callback)
---------------------------          --------------------------------------------
capture_want_.store(true)    ──►     reads capture_want_ at the top of each datagram
capture_path_ written once           rising edge  → open() : rename, create, fallocate, mmap
  before the first store               falling edge → close(): munmap, close
                                     then, if enabled, store(bytes, len, recv_time)
```

No mutex, no queue, no second copy of any datagram, and the application still has exactly two
threads.

The one-off cost of a toggle is the `open()`: a rename, a create, a `posix_fallocate` of 62.6 MiB
and an `mmap`, on the producer thread. It is a user-initiated act that happens once per toggle, and
`README.md` will say that it can cost the packets in flight during it. Moving it off the producer
thread would need the lock or the thread this design exists to avoid, for an event that happens
when a human clicks a button.

The duration is **measured, not asserted**: `open()` records its own wall time and the probe prints
it as `open_ms`. `posix_fallocate` on a local extent-based filesystem is a metadata operation and
should be a handful of milliseconds, but that is a claim about the filesystem, not about this code,
and a reading above ~50 ms is a finding worth putting in `README.md` rather than a number to
predict here.

### Error handling, and only at this boundary

This is a syscall boundary, so it is validated; nothing else here is.

| Condition | Behaviour |
| --- | --- |
| `open`, `posix_fallocate` or `mmap` fails (ENOSPC, EACCES, a read-only cache directory) | capture stays off, `error()` carries `strerror`, one line on stderr, the toggle button returns to off and the popover row shows the reason |
| the cache directory does not exist | created with `g_mkdir_with_parents`, 0700; a failure there is the row above |
| `len > UDPS_RX_BUF_BYTES` | impossible — the slot is sized to the client's own receive buffer — but `store()` truncates to the slot, records `orig_len`, and counts it in `truncated()` rather than trusting the impossibility |
| the previous `capture-prev.bin` cannot be replaced | the capture still opens; the rename failure is reported, the old previous is kept |
| capture enabled with no connection yet | the button is insensitive until connected, because the path is keyed on the source; `--capture-packets` defers `open()` to the first datagram, which is what arms it before the first CONFIG |

## Surfaces

### The popover button and row

The status bar's detail popover (`src/ui.cpp:7619-7672`) is a two-column grid: four bold headings
at rows 0, 3, 5, 6 and seven value labels at rows 0..6, in `app->popover_label[11]`. It gains a
fifth heading, **Capture**, and an eighth value label; the array becomes `[13]` and the
`kHeadings` / `kHeadingRow` tables gain their entry. `update_status_popover`
(`src/ui.cpp:6101`) writes the new label like the others — `gtk_label_set_text` only, no rebuild.

The row reads one of:

```
off
on · 412/1000 slots · 26.1 MiB · ~/.cache/udpscope/192.168.1.40-5000/capture.bin
failed: No space left on device
```

Below the grid, a `GtkToggleButton` labelled **Capture packets** with the
`media-record-symbolic` icon and a tooltip naming the full path and the file's size. `toggled`
sets the atomic and nothing else. Insensitive until connected.

Nothing is added to `format_detail()`'s note string: `src/status_text.cpp:86` is a `char
note[160]` already close to its worst case, and the capture state is not something the exit dump's
one line needs.

### The command line

```
  --capture-packets N  store the last N raw datagrams on disk for forensics, in a
                       ring under the cache directory (1000 slots is ~62.6 MiB);
                       off unless given, and toggleable from the status bar
```

Parsed in `src/config.cpp`'s existing style — both `--capture-packets N` and
`--capture-packets=N`, `N` in 1..100000 — and the value is **required**, not optional.
`--capture-packets` with an optional value cannot be told apart from `--capture-packets` followed
by a value in this parser, and every other option in it takes the space-separated form, so a bare
flag would turn `--capture-packets 500` into `unknown argument: 500`. The 1000 default is the
slot count the **popover button** uses when the flag was not given; with the flag, the button uses
the N the flag named.

`Config::capture_packets` is a `std::uint32_t`: 0 means the flag was absent, and the button then
uses 1000.

The flag arms the ring before the first datagram; it is not persisted in the session file. That
persistence belongs with `todo.md` item 3's work, alongside the relative-time toggle deferred to
the same place.

### `tools/capture_to_pcap.py`

```
$ tools/capture_to_pcap.py ~/.cache/udpscope/192.168.1.40-5000/capture.bin out.pcap
1000 slots, 412 stored, 0 torn, 0 truncated → 412 packets, 26.1 MiB
$ wireshark out.pcap
```

Reads the header, validates the magic and version, walks the slots, orders by `seq`, skips
`seq == 0`, and emits a classic pcap (`LINKTYPE_ETHERNET`, microsecond timestamps from
`t0_realtime + t_arrival`) with a synthesised Ethernet/IPv4/UDP header per record built from the
header's `peer_addr`, `peer_port` and `local_port`. Pure standard library — the mock already
forces a hard `numpy` dependency on `tools/`, and this script has no reason to add to that.

It also prints the counts above to stderr, which is what makes it a diagnostic in its own right:
`torn` or `truncated` above zero is a finding about the capture, before anything about the stream.

## Verification

udpscope has no unit test suite; the capture is verified by the application's own headless
self-check, and the converter by running it.

`UDPSCOPE_PROBE_CAPTURE_MS` toggles capture **through the same handler the button uses**, waits a
scheduled interval, then reads the file back off disk and prints one line:

```
[capture-probe] enabled=1 path=… slots=1000 stored=412 cursor=412 torn=0 truncated=0
                header_ok=1 first_seq=1 last_seq=412 payload=1648000 file=65576064
                open_ms=3.1 t0=1760000000.123
```

`payload` is the sum of every stored slot's `len` — the bytes the wire actually delivered, which is
what tells you whether the ring is holding what you think. `file` is the reserved size,
`64 + slot_count * slot_bytes`, and is a constant for a given N: it is in the line so that a
mismatch against the file on disk is a reading rather than something to go and check.

What each field has to read, and the defect a wrong reading names, goes in
`docs/claude/probe-readings.md` beside the other probes. Two cases in
`docs/claude/verification.md`:

1. **enable at runtime** — the probe toggles on mid-run; `stored` must equal the datagrams the
   exit dump counts from that instant, `torn=0`, and `first_seq=1`.
2. **wrap** — `--capture-packets=16` against the standard mock, run long enough to wrap several
   times; `stored` exceeds `slot_count`, `last_seq - first_seq == 15`, `torn=0`. A wrap that loses
   ordering, or a `cursor` that disagrees with `seq`, shows here and nowhere else.

Then the converter, run over the wrap case's file: exit 0, a record count equal to `slot_count`,
and the pcap's own first and last record timestamps inside the run's wall-clock interval. The
check is the converter's printed counts plus a `struct.unpack` of the pcap's first and last record
headers — no Wireshark, no `capinfos`, nothing outside the standard library, so it runs wherever
the self-check runs.

The standing baselines apply unchanged: a clean build at exactly four warnings, an incremental at
two, and zero `Gtk-CRITICAL` in any headless run.

## Out of scope

- **Replaying a capture back into udpscope.** That is `todo.md` item 5's territory (opening stored
  waveforms) and a different feature with a different shape.
- **Retention beyond one deep.** `capture.bin` and `capture-prev.bin`, nothing more, until item 9
  defines the cache-size preferences.
- **Session persistence of the enable state.** Deferred to item 3 with the relative-time toggle,
  so one piece of work adds the session fields rather than two.
- **Capturing the multicast TCP CONFIG channel.** Stated as a limitation in `README.md`.
- **Compression, encryption or rotation of the ring.** A fixed file of raw bytes is what a
  forensic reader wants, and `gzip` exists.

## Documentation to update

- `README.md` — the feature, the 62.6 MiB cost, the producer stall on toggle, the multicast CONFIG
  limitation, the converter's usage.
- `SPECS.md` — nothing: this is a `todo.md` item, not a spec requirement, and not a deviation.
- `todo.md` — item 4 struck, on one line, with the flag's name (GFM strikethrough is inline and
  cannot span list items).
- `docs/claude/build-and-run.md` — the new option; `verification.md` and `probe-readings.md` — the
  probe and its two cases; `docs/claude/invariants-data-path.md` — one paragraph on why the hook is
  at `recv()` and why the file is reserved rather than sparse.