working on todo.md features

README.md

@@ -26,7 +26,8 @@ Options: `--server`, `--port`, `--multicast`, `--interface`, `--buffer-mb`
 (size of the sample buffer, default 200), `--max-window` (largest time
 window the toolbar offers, default 600), `--holdoff` (default trigger hold-off),
 `--recv-buffer`, `--plots` (the number of stacked plots to start with, 1..8; rows are
-added and removed from the **Layout editor** afterwards), `--relative-time` (see below).
+added and removed from the **Layout editor** afterwards), `--relative-time` (see below),
+`--capture-packets N` (see "Packet capture" below).
 
 Every CONFIG is dumped to stderr once -- publish mode and, per signal, its name,
 element count, rate, time mode and time-signal index -- because a signal that
@@ -407,11 +408,201 @@ floored to a **power of two** (`src/model.cpp:83`, `:110`), so a third more weig
 third of the span; it costs a whole halving. The measured figures above are the correct ones; a
 per-byte computation gives the wrong answer for this reason.
 
+## Packet capture
+
+**`--capture-packets N`** stores the last N raw datagrams on disk for forensic
+analysis — the bytes as `recv()` returned them, before the client validates the
+magic, reassembles fragments or resequences counters. It is a debugging aid for
+*wire* behaviours (a counter that skips, a datagram that arrives twice, a fragment
+that never completes, a CONFIG whose header does not parse), not a feature of the
+instrument: it is off unless the flag is given, and it adds nothing to the status
+bar's glance tier.
+
+The ring lives at `~/.cache/udpscope/<host>-<port>/capture.bin`, the previous
+capture is kept one deep as `capture-prev.bin`, and it can be toggled at any time
+from the status bar's detail popover (the **Capture packets** button, insensitive
+until connected). At the default 1000 slots the file is **~62.6 MiB**, reserved up
+front, which is why the size is stated in the tooltip and in the popover row.
+
+Three things it deliberately is not, and the reasons are in
+`docs/superpowers/specs/2026-09-29`'s sibling design note:
+
+- **It cannot cost a packet.** The capture is an `mmap` and a `memcpy` on the
+  producer thread between two `recv()` calls; there is no queue, no lock and no
+  second copy. The one exception is the toggle itself: `open()` — a rename, a
+  create, a `posix_fallocate` and an `mmap` — runs on the producer thread when you
+  click, and measured ~1–3 ms here. The popover row shows the measured `open` cost
+  in the probe line; a reading far above that is a filesystem finding.
+- **It is not interpreted.** `tools/capture_to_pcap.py` reads the file back and
+  writes a classic pcap, so wireshark does the interpreting:
+
+  ```sh
+  tools/capture_to_pcap.py ~/.cache/udpscope/127.0.0.1-8000/capture.bin out.pcap
+  # 1000 slots, 16 stored, 0 torn, 0 truncated, 0 empty -> 16 packets, 1.0 MiB, ...
+  ```
+
+  The counts it prints are a diagnostic in their own right: `torn` above zero means
+  the capture died mid-copy (a crash is exactly what the on-disk ring exists to
+  survive), and `truncated` means a datagram longer than the client's receive
+  buffer, which should be impossible.
+- **It does not see the multicast CONFIG channel.** In multicast mode CONFIG
+  arrives over TCP, and the capture is of the UDP socket only. A malformed
+  *multicast* CONFIG is out of reach.
+
+## Captured events
+
+Trigger mode keeps the last **32 triggered acquisitions** in a cache, so an event you did not have
+time to look at is still there:
+
+```
+~/.cache/udpscope/192.168.1.40-8000/events/
+  20261009-123250-0001.npy   # the event, in the export format
+  20261009-123250-0001.txt   # when, how long, how many signals, the rate, the edge
+  20261009-123250-0001.bmp   # a picture of it, drawn from the samples
+```
+
+The **left pane** is opened on demand from the toolbar's event-history button, is collapsed by
+default, and its **divider can be dragged**: it starts at 240 pixels and remembers the width you last
+chose, and while it is closed it takes no room at all. Both the button and the pane exist while **recording is on**, or while an event is on
+screen: with the cache disabled and nothing being examined there is nothing to show, and a visible
+empty column beside the plot is what *off* must not look like. Recording is switched in the **trigger
+panel**, with the trigger's other options rather than in the pane whose contents it decides, and it is
+**off by default** -- writing every latch to disk is the user's decision about their disk.
+
+Each row carries **its own picture** beside the numbers -- time, duration, signal count, rate -- in a
+fixed 126x42 cell (the preview's own 3:1), so the picture is the same size in every row whatever the
+pane's width. Clicking a row **replays** it: the same loader `--open` uses, on the event's file. **The layout does
+not change** -- the rows, which signal sits in which slot, the colours, the row count, the time window,
+the cursors and the measurements all stay exactly where they are, and now read the stored event instead
+of the stream; that is what "show me that event" asks for, and rebinding the rows to the file would
+rearrange the user's work under a click. What changes is the data, and the trigger is left, because a
+file has no edge to wait for. Cursors, measurements, math channels and another export all work on a
+stored event. **Live** takes you back to the stream.
+
+Two buttons under the pane's header read the **selection**:
+
+- *Compare* puts the two selected events in **two plots side by side**, each with **the layout you
+  already have** -- the same rows, the same signals in the same slots, the same colours -- one showing
+  one event and one the other. Their axes are **synchronised**: zoom or pan a row in either plot and the
+  same row of the other follows.
+- *Accumulate* overlays up to four selected events in the **same traces**, translucent, so where they
+  agree the trace is brighter. The events' copies take the free slots of the row that already holds
+  their signal (a row holds eight traces), so your own traces never move.
+
+Both need events with the same signals and the same length, which is what makes a shared axis honest;
+the build checks it and says so in the status bar when it does not hold.
+
+A group is a file view like a replay: the acquisition is stopped (a live producer would push into a
+model whose channel count no longer matches the stream), the trigger is left, and *Live* returns.
+
+A replay leaves trigger mode, because a file has no edge to wait for -- an armed trigger keeps a
+*frozen capture* on screen (`display_mode()` stays `Held` while the model holds a snapshot), and a
+held display draws that snapshot rather than the file that was just loaded. The events pane stays
+open through it: it is available while a stored event is on screen.
+
+Three things worth knowing:
+
+- **The samples are the export format**, so an event is readable by everything that reads an
+  export: `--open`, the chooser's Waveforms page, and `numpy`/`pandas`.
+- **The picture is drawn from the data**, one strip per signal with min/max per pixel column and the
+  trigger edge as a brighter line -- not captured from the window: the plot is a `GtkGLArea`, and a
+  GL area's ink is not in the widget's paint tree, so a snapshot of it is empty. A data-drawn
+  preview is also the honest one: it shows the samples, not the zoom the window happened to be at.
+- **The picture is decoded synchronously** (`gdk_texture_new_from_filename()` into
+  `gtk_picture_set_paintable()`), not through `gtk_picture_set_filename()`: that path uses the image
+  loader's sandboxed, asynchronous decoder, which here fails silently -- the picture keeps a
+  paint-able and an allocation, so a probe reports success while the pane shows an empty rectangle.
+  It is then **scaled once to the row's size** and cached by file: a `GtkPicture`'s natural size is
+  its content, so the stored 480-pixel preview given straight to a row claims the row's whole width
+  and squeezes the text out of it -- a size request on the picture is a floor, not a cap.
+- **A replay stops the acquisition** (a live producer would overwrite the loaded samples within a
+  frame), so the status bar names the file until *Live* starts a fresh client and thread. The
+  replayed samples age out of the ring as the stream refills it.
+- The switch in the pane's header stops the recording; the cache is only useful if it was already
+  on when the event you wanted happened, which is why it defaults to on.
+
+## Opening an exported waveform
+
+An export can be opened again — to look at a run you no longer have the source of, and to analyse it
+with everything the scope offers:
+
+```sh
+udpscope --open ~/udpscope-20261009-113200.npy
+```
+
+or start udpscope with no `--server` and use the chooser's **Waveforms** page, which lists the files
+opened before (newest first, with the ones that have since been deleted marked *missing* rather than
+left to fail on the click) and has an **Open…** button for a new one.
+
+A waveform is a *source that never arrives*, and the design follows from that: there is no socket, no
+acquisition thread, and no starvation notice — the status bar's rate is the file's own stored rate and
+its Stream row names the file. Everything else is the same machinery a live run uses: one signal and
+channel per column, **one row per column** (so a four-signal file opens as four stacked traces, with
+three free slots on every row to drag more onto), the samples pushed into the same ring, the display
+epoch pinned to the file's last stamp so the axis reads `[-window, 0]` exactly as the export did — which means cursors, measurements,
+math channels, the trigger's search and another export all work on it unchanged. The signal names come
+from the file: a `.npy` written by `write_npy()` is a structured array whose *dtype* names its columns
+(`time`, `abs_seconds`, then one field per signal), so there is no sidecar to lose.
+
+Only `.npy` is read: it is the format the item asked for and the one that carries names. A `.csv`
+export is for other tools.
+
+## Named configurations
+
+What you arranged is remembered per data source, and a source can hold **several named
+configurations** — different experiments on the same streamer want different plots, different
+measurements, and sometimes a different buffer. Each is one file under the source's own
+directory:
+
+```
+~/.config/udpscope/192.168.1.40_8000/
+  active          # the configuration this source is on, one line
+  bench.toml      # a configuration, plus its note
+  emi-test.toml
+```
+
+A configuration is the whole session: the window, the rows and their signals, the calibration,
+the trace colours, which signals were enabled, the math channels with their graphs, the
+measurements — **and the settings that can only be decided at startup**, which is why they live
+here rather than only on the command line:
+
+```toml
+comment = "5 kHz sine, trigger on Ch1, 8 MB is plenty"
+buffer_mb = 8
+relative_time = false
+capture_packets = 0
+```
+
+The **buffer size** is the one that matters most: two sources on one machine rarely want the same
+ring, and the value is applied before the model exists, which is the only moment it can be — by
+whichever of the chooser's rows opened the session, **and by `apply_stored_settings()` when the
+source was named on the command line**. A saved host's settings are read either way: opening
+`udpscope --server 10.0.0.5 --port 8000` gets that source's stored buffer size, clock and capture
+ring, and a flag the user actually typed wins over the file (`--buffer-mb`, `--relative-time`,
+`--capture-packets`) because it is the more recent statement of intent. Switching configurations *inside* a running
+app applies everything that can change live (layout, plots, math, measurements, enables) and says
+so in the status line about the rest (buffer, clock, packet capture): they take effect on the next
+start.
+
+The **chooser** (start with no `--server`) lists one row per configuration, with its note and its
+settings, so two configurations of one source are told apart before they are opened. Its **New
+host…** form takes the address *and* those settings — the configuration's name and note, the buffer
+size, and relative time — so a host can be set up before it has ever streamed; the same form opens
+from a row's pencil button to **edit** an existing configuration, pre-filled from its file, and
+editing the name or the address is a *move* (the old file is deleted, so the list never shows the
+same session twice). Connector settings are stored even for a configuration with no signals yet,
+which is why those rows exist at all. The app's
+menu has **Configurations…**: the same list for the running source, with *Save this session* —
+a name and a note — and a delete button per row. Opening one saves the current session first, so
+switching is never the gesture that loses what it switches from.
+
+A session written before configurations existed is imported on first use as the configuration
+named `default`: copied into the directory, remembered as active, and the old file renamed to
+`<host>_<port>.toml.imported` rather than deleted, so the import can be undone by hand.
+
 ## The session layout file
 
-What you arranged is remembered per data source, in
-`~/.config/udpscope/<host>_<port>.toml` (`$XDG_CONFIG_HOME` is honoured; the
-host's unsafe characters are replaced with `_`). The file holds the display
+A configuration's file holds the display
 window, the row layout and which signal each of a row's slots plots, and the
 per-signal gain and offset:
 
@@ -861,6 +1052,15 @@ another math channel can be its source. The operations are `channel` (a source),
 stored rate), `sum`, `difference`, `product` and `ratio`; the arithmetic ops take two inputs, which
 is what makes branching — the sum of a raw and a filtered channel — expressible at all.
 
+**A gap is a gap in a filtered trace, never the end of it.** A hole the resequencer reserved is
+NaN in the source, and the filters deliberately do not integrate it: the hole shows as a hole in
+the filtered trace, the filter's history stays the last real samples it saw, and the trace picks
+up again after it. (Integrating it poisoned the biquad's state for ever — every later sample
+NaN — which is what a dropped packet used to do to every low pass, systematically in relative
+time, where a counter gap reserves NaN slots by construction.) The exit dump's
+`math '<name>': finite=N/M` line is where that reads: `finite` must be about `M` minus the hole
+share, and `finite=0` beside a non-zero `hole_slots` is the defect back.
+
 The editor is a compact node graph: each node is a box, its inputs are the squares on its left edge
 and its output the square on its right. Click a box to select it and its parameters appear in the
 form beside the canvas; **drag from an output square to an input square to wire them** (the old
@@ -1389,10 +1589,17 @@ signal if needed, binds it to that row's first free trace slot, and rebuilds the
 figure so the legend name is right (there is one widget, so the row a drop lands
 on is `PlotArea::row_at()`). The status line reports the outcome of a drop
 (`plotting 'sine' on plot 1`, `plot 1 already holds four signals`), so an action
-that changes nothing still says why. Each plot's y axis has a right-click menu
-with auto scale, a fixed range, **Fit X axis**, **Fit all**, and one
-**Remove '<signal>' from plot N** entry per occupied trace slot (a plot with nothing on
-it says so instead). The two fits are the same pair the overflow menu and the `f` key
+that changes nothing still says why. Each plot has a right-click menu in the same
+compact row style as the app's own menu — the label at one left column, a shortcut flush right,
+18 px rows — holding **Y axis: auto scale**, **Y axis: fixed range...**, **Fit X axis**, **Fit
+all** (the `f` key's action, so it shows `F`), and **Remove signal**, which swaps the menu to a
+second view — one entry per trace on that plot, plus **All traces on this plot** — with a **Back**
+row to return (a plot with nothing on it says so instead). The removals are a view of their own
+rather than a flat list because as a flat list they were the same long sentence repeated once per
+trace, which is most of the menu spent on one verb; and they *replace* the menu instead of floating
+beside it as a submenu, because a second popover is a second thing to dismiss and a `GtkMenuButton`
+that owns one toggles itself out of sync as soon as anything else closes it — its row stopped
+opening at all on the second click. The two fits are the same pair the overflow menu and the `f` key
 offer, split by what they touch: **Fit X axis** puts the x view back on the whole time
 window and leaves every y scale exactly as the user set it, **Fit all** does that and
 auto-scales the y axes too. Neither changes the time window itself — see below.
@@ -1985,7 +2192,7 @@ now — they are listed so they are not mistaken for anything else:
   progress. What the spec was protecting still holds — a *held* event is bound to its
   snapshot alone, and the axis is the event's own window from the edge onwards rather
   than a sliding live view.
-- **A plot holds at most four signals** (`RowSet::kSlotsPerRow`), because a `Plot`
+- **A plot holds at most eight signals** (`RowSet::kSlotsPerRow`), because a `Plot`
   cannot give up a trace and the slots are therefore created up front.
 - **One toggle drives both cursor orientations.** The vertical markers and the
   horizontal ruler exist separately but share one show/hide and one reset.
@@ -2036,6 +2243,15 @@ stall test. Calling that one "connected, but no data is arriving" blamed a secon
 that does not exist; the usual cause is a streamer that is not running, or one serving a
 different port.
 
+**A link loss does not cost the acquisition.** When the stream comes back and its CONFIG
+names the same signals as before — the common case, because whatever knocked the link out
+did not reconfigure the producer — the reconnect keeps the buffer, its history, the
+trigger's state and any held event, and the outage appears as the gap in the data that it
+was. Only a CONFIG that actually *changes* the stream re-plans the buffer, which is the
+one case where dropping history is honest. The mock can reproduce the loss itself:
+`tools/mock_udpstreamer.py --blackout-at 5 --blackout 3.5` goes silent past the client's
+silence timeout and then resumes, which is the EMI case end to end.
+
 ## Limitations
 
 - **Multi-element (vector/matrix) signals are acquired with a zero-order hold.**