udpscope/README.md

# udpscope

A GTK4 oscilloscope-style viewer for a UDPStreamer source: it receives one stream
with [`deps/udpsclient`](deps/udpsclient), buffers it in two identical large
circular buffers, and draws it with [`deps/plotter`](deps/plotter).

`SPECS.md` is the requirement document; this file describes what was built.

## Build and run

```sh
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo
cmake --build build

./build/udpscope --server 127.0.0.1 --port 8000            # unicast
./build/udpscope --server 10.0.0.5 --port 8000 \
                 --multicast 239.1.2.3 --interface 10.0.0.9  # multicast
```

Options: `--server`, `--port`, `--multicast`, `--interface`, `--buffer-mb`
(size of the sample buffer, default 200), `--max-window` (largest time
window the toolbar offers, default 600), `--holdoff` (default trigger hold-off),
`--recv-buffer`, `--plots` (initial number of stacked plots, 1..4).

Every CONFIG is dumped to stderr once -- publish mode and, per signal, its name,
element count, rate, time mode and time-signal index -- because a signal that
refuses to plot is almost always a signal the stream described in an unexpected
way. The status line carries the same story live: `plot 1: 2 trace(s), 1400
samples`, and `time x3` when the shared clock is decimated.

Headless self-check (used by the smoke tests): `UDPSCOPE_EXIT_AFTER_MS=<ms>`
switches trigger mode on, waits for an event, then leaves trigger mode the way the
"Go live" button does, closes the window, and prints on stdout the status line in
both states, the frame and per-channel sample counts, each plot's painted-frame
count, which channel each trace slot holds, and the allocated size and reveal
state of both side panels.

## Layout

```
src/config.*       command line
src/model.*        the sample buffer, the partition plan, the signal/channel lists
src/channel.*      ChannelBuffer (a ring pair as a plot::data::Series) and SlotSeries
src/acquisition.*  the UDPStreamer client, the producer thread, frame decoding
src/trigger.*      the trigger configuration and the producer-side state machine
src/plot_host.*    the one plot::Figure, its rows' traces and cursors, the x link group
src/export.*       the visible range as a table, and the .npy / .csv writers
src/ui.*           the GTK4 window: toolbar, trigger panel, signal panel, plot box
```

Threading: one producer thread owns the `udps_client_t` and calls
`udps_client_poll()`; the GTK main thread owns every widget and the GL contexts.
They share `ScopeModel` under one mutex, which the producer takes once per DATA
frame and the GUI thread takes once per frame around binding the traces.
Samples cross the boundary through `plot::data::RingBuffer` (single producer,
single consumer), never through the lock.

## How the spec maps onto the code

**One buffer of `N+1` slices, plus frozen event snapshots.** `ScopeModel`
allocates one `BufferStore` per CONFIG: one shared time-base ring plus one value
ring per channel (N enabled signals -- see the limitation on multi-element signals
below). The spec's second buffer is the latched `EventSnapshot`, which is already
an immutable copy of the captured window (see "Trigger" below); a second live ring
bought nothing and cost half the memory budget, so `--buffer-mb 200` really is
200 MB.
`plan_partition()` sizes the rings to cover `--max-window` seconds and then grows
them with the rest of the budget. When a channel's declared sample rate is lower
than the fastest channel's, it is decimated by the integer ratio
(`Partition::stride`); when even the target window does not fit the budget, the
shared clock is decimated too (`Partition::time_stride`). Because the split is by
*time span* rather than by bytes, every channel and the time base retain the same
history -- which is what makes one shared time slice meaningful. With no
decimation the two rules coincide and every slice is the same size.

The ring is always filling, whatever the trigger is doing. The pre-trigger history
has to already be there when an edge arrives; while acquisition followed the
trigger state, the first latch after arming opened on however little had
accumulated since.

**Circular, time from the streamed data.** Each ring overwrites its oldest
sample. Timestamps come from the stream's time signal when the CONFIG names one
(`UDPS_TIME_FULL_ARRAY` / `FIRST_SAMPLE` / `LAST_SAMPLE`), and otherwise from an
arrival-anchored estimate that accounts for `udps_frame_t::lost` by dividing the
elapsed wall time over the cycles the gap covers. `recv_time` is used only as the
fallback when the stream provides no usable time signal (it is then the sample's x,
via the arrival estimate); it is never used when the stream's own time signal
carries a span.

**Trigger: what freezes is the view, not acquisition.** `TriggerEngine` runs the
edge/hold-off/normal-single state machine on the selected channel, and once the
requested post-trigger share of the *event span* (the event-window control, not the
whole ring) has been collected, `ScopeModel::publish_event()` copies that window out
of the live ring into an immutable `EventSnapshot` and publishes it atomically.
Pre-trigger history is what the ring already holds, so the percentage is a share of
the event window and not of the buffer.

In trigger mode a trace is bound to the snapshot and to nothing else: the last
event latched, or nothing at all before the first one. Never the live ring, not
even while the post window is filling — a picture that changes only on an event is
the whole point of trigger mode (SPECS: "data visualisation should not be refreshed
till next event"), and binding the live ring mid-capture made the plot scroll for
most of every event cycle.

**The whole trigger path measures the window in stream seconds, and never in
samples.** `TriggerEngine` is fed each sample's timestamp and latches once that clock
has advanced past the post window; `publish_event()` then finds the slice covering
`[edge - pre, edge + post]` by binary search over the time ring. The two are the same
clock the x axis reads, so a capture covers the span that was asked for by
construction.

It did not used to. The engine took a post-trigger window in *samples*, converted
from seconds by a measured estimate of the trigger channel's cadence, and that
estimate is what made captures wrong — most visibly as the reported defect *"only the
pre window is filled, the post window is completely empty"*. Any stream whose time
signal advances in large steps (one sample per stream second, say) drove the estimate
to its 1 S/s floor; the sample count then came out as 1, which the engine read as "no
post window requested" and latched on the edge sample itself. The event held a single
sample, its span was zero, and the UI fell back to pinning a round `[-1, +1]` axis
that nothing filled past x = 0 — an axis that looked right, which is why the defect
read as a drawing bug rather than a capture bug. Removing the conversion removes the
whole failure class: there is no longer a rate estimate to be wrong.

An edge is refused until the buffer reaches back a full pre-trigger window behind it,
so a capture cannot open on a half-filled pre window. The gate compares the oldest
buffered stamp against the edge rather than timing how long the trigger has been
armed: the ring fills whether or not the trigger is on, so the history is normally
already there and waiting would delay the first event for nothing. It only bites just
after a rebuild — a CONFIG, enabling the signal, or a calibration change all drop the
history — which is exactly when the first event used to latch on an almost empty pre
window and draw half a plot.

The event is published on the sample that latched it, but only after every channel of
that sample has reached its ring: the cut is clamped to the shortest value ring, so
publishing from the middle of the channel loop dropped the newest sample — the very one
the post window had been waiting for. That cost one sample of twenty thousand on a
10 kS/s stream, and the whole post side on a one-sample-per-second one.

The axis is still pinned to what the snapshot actually captured, read off its own
stamps rather than off the request: the two can differ by a sample at each end, and
pinning the request would claim coverage the data does not have. The window's bounds
are rounded to the time ring's own `f32` before being compared against it, so a sample
sitting exactly on a bound is kept whatever the stream's absolute time happens to be —
in f64 that decision fell to how the stamp rounded, which cost a sample at 10 kS/s and
a third of the capture at 1 S/s.

**Where the capture is drawn was a separate bug from what it contained**, and it
outlived the fixes above. The plotting library's `gather()` emits x *relative to*
`x0()` and the renderer adds `x0()` back; the event series emitted absolute stamps, so
the renderer added `x0()` twice and drew the whole capture one pre-trigger window to the
left of the axis it had just been pinned to. Nothing about the numbers looked wrong —
the snapshot held the right samples over the right span, and the crosshair (which reads
`x_at()`, never `gather()`) reported correct values — but with a 50/50 split the right
half of the plot was empty, zooming out showed the trace ending at the x = 0 marker, and
zooming in past the shift showed nothing at all. The self-check now prints `data=` and
`drawn=` side by side for the bound trace so the contract is checked rather than
assumed.

**The live view had two clipping bugs of its own**, both wearing the same face as that
one: the older part of the trace blank, the newest part fine, and the crosshair still
reading values in the blank part. Neither is a data bug — the samples were always there.

The first is the requested time window being quietly redefined. `clamp_to_data()` stops a
zoom-out from running off the end of the ring, but it runs every frame and cannot tell
that apart from a ring that has only just started filling: the first frames after a
signal is enabled hold a couple of hundred samples, a 2 s window was clamped to the
~0.02 s they covered, and nothing there ever grows a span back. The view then stayed
pinned to the newest 20 ms of a 6.5 s buffer for the rest of the session. It now clamps
against the larger of the retained span and the requested window, so a partly filled
window simply has an empty left-hand side.

The second is subtler and needs a saturated ring. A `ChannelBuffer` sample is a (value,
stamp) pair split across two rings of equal capacity, and the published window is pinned
by `acquire()` while both rings keep sliding under it — so the oldest samples of the
window point at time slots the producer has since overwritten. Those used to read back as
`0`, a perfectly plausible coordinate, which landed in `x_bounds()` and from there in
`dx()` and `x0()`. That is the renderer's entire index mapping: it converts the visible x
range into indices with `(x - x0()) / dx()`, so *one* unreadable slot inflated `dx()` by
the ratio of elapsed stream time to the retained window, scaled every index up with it,
and the gather began well inside the visible range. The stamp is now clamped to the oldest
readable slot, which costs at most a short plateau of equal stamps at the very left edge
— the same bargain the library's own `StreamSeries` documents, where an overwritten slot
costs a y value and never the geometry. The self-check prints `dx_ratio` and the
renderer's index window beside the binary-searched one, so both are checked rather than
assumed.

**The panels and the toolbar.** The header bar carries only what you reach for
*while watching a signal*: the time window, the plot layout (1..4 rows),
pause/resume, the trigger toggle, the side-panel toggle and an overflow menu. Those
four buttons are icons with tooltips; nine text buttons was a wall of words to read
past every time you wanted the one you needed. The overflow menu holds **Fit to
data**, **Cursors**, **Reset cursors**, **Export visible range…** and the
**Signal list** toggle — the things you set up once. The side column and the two panel toggles stay hidden until a
CONFIG arrives — there is nothing to configure before the stream describes itself —
and the first CONFIG reveals the signal panel (the trigger panel stays behind its
own toggle).

The trigger panel selects the **signal** (every signal the CONFIG described, not
only the acquired ones, so the list is never empty while a stream is known), the
edge, level, pre-trigger percentage, hold-off and mode, and arms the trigger.

**The event window is its own control**, separate from the display window: it is
how much of the stream one captured event covers, and the pre-trigger percentage
is a share of *that*. Tying it to the display window was a mistake — with the
600 s default a latch took 300 s (half of it) to fill, so the view sat frozen for
minutes and the trigger looked dead. With the default 50 ms event window a latch
completes 25 ms after the edge. When an event latches and the plot is zoomed out
wider than the event window, the display zooms to the event so the captured
waveform fills the plot; the window dropdown follows.

**Hold-off is the quiet time after an event has been captured** — measured from
the end of the capture, not from the edge, because the capture itself already
blocks re-arming.
Choosing a signal, switching trigger mode on, or arming the trigger all enable
that signal's acquisition — a trigger on a signal nobody acquires could never
fire.

**Trigger mode is the switch between streaming and triggering**, and the status
line always names the state it is in:

| status line | meaning |
|---|---|
| `[live]` | trigger mode off: the view follows the stream continuously |
| `[trigger: armed, waiting for an event]` | watching for an edge; the view shows the previous state |
| `[trigger: EVENT HELD …]` | an event was latched; the view is frozen on it |
| `[trigger: disarmed (view frozen)]` | armed and waiting are not the same thing — nothing will arrive until it is armed again |

While an event is on screen a **red vertical line marks the trigger instant** on
every row, at the edge's own stream time — the latch may complete a post-window
later, but the line marks the edge, so the pre/post split can be read straight off
the plot. It is not one of the user's cursors: it has its own colour, ignores the
cursor show/hide toggle, and is parked (invisible) when no event is held.

**The status line.** It starts with an activity badge and then the mode, the
connection, the measured rate, the packet/frame/gap counters, what plot 1 holds and
the buffer's size and span. Three of those are worth calling out:

- **The badge blinks green at 1 Hz while the pipeline is actually running** — the
  stream is connected, acquisition is not paused, and in trigger mode the trigger is
  armed. Anything else leaves it a steady grey, with the reason on its tooltip
  (`paused`, `trigger disarmed`, `waiting for the stream`). *Blinking* is the signal,
  not the colour: a steady badge is a stopped pipeline, and that distinction survives
  a screenshot. It is one glyph whose colour is all that changes, so blinking it
  cannot change its width — a badge that grew and shrank twice a second would move the
  label beside it and re-lay out the plot with it.
- **`rate 9.99 kS/s` is the rate the buffer actually stored over the last second**, not
  the rate the CONFIG declared. The app does **no** decimation (see `plan_partition()`),
  so the two normally agree and a gap between them is the datasource's real wire rate.
  One figure when every channel agrees within a percent, `lo..hi` when they do not;
  the per-channel breakdown, stored against declared, is on the line's tooltip. A
  per-channel list in the line itself would make its width depend on the stream.
- **`last event 12 s ago`** (trigger mode only) ages the picture on screen, in seconds,
  then minutes, then hours. A frozen view looks the same whether the last event was a
  second or an hour ago, so without it there is no way to tell a working trigger from
  a dead one. Measured on the wall clock, not the stream's: it is a question about how
  long *you* have been waiting. Before the first latch it reads `no event yet`.

Switching the header bar's **trigger** button off returns to `[live]` and resumes
drawing the stream; the trigger panel carries the same action as a
**Go live (trigger off)** button, so the way out is inside the panel that froze
the view. Switching trigger mode on arms it immediately (arming is what "wait for
an event" means), and in single-shot mode the trigger disarms itself after a
latch, which the status line reports.

The signal panel is one line per signal: a colour chip naming the trace it is drawn
as, a checkbox that acquires it, its name, its unit, and a `⋯` menu holding the
information popup and the gain/offset calibration popup.

**The chip's colour belongs to the signal, not to the slot it landed in.** The
palette is indexed by position in the signal list (`signal_color_index()`), so a
signal dropped on two rows is the same colour in both and the chip names a trace
unambiguously; indexing by trace slot made the same signal blue on row 1 and orange
on row 2, and then the chip named neither. A signal that is on no plot — acquired or
not — gets a **grey** chip rather than no chip, so every row of the panel is the same
shape and the column of chips reads as a legend. Because `plot::Plot` fixes a trace's
colour at `add_trace()`, the index is stored on the row (`Row::color_index`) and a
colour change goes through a figure rebuild, which a drop already forces.

A `GtkSwitch` and two
always-visible buttons made a row about twice as tall (46 px against 26 px now),
which is what turned a four-signal stream into a panel you have to scroll; Adwaita's
button and list-row minimums, not the widgets themselves, set that height, so the
row carries a `.udpscope-compact` style class that relaxes them.

**Signals start disabled**, because enabling one is what starts acquiring it.
Enable a signal with its checkbox, or drag it onto a plot: the drop enables the
signal if needed, binds it to that row's first free trace slot, and rebuilds the
figure so the legend name is right (there is one widget, so the row a drop lands
on is `PlotArea::row_at()`). The status line reports the outcome of a drop
(`plotting 'sine' on plot 1`, `plot 1 already holds four signals`), so an action
that changes nothing still says why. Each plot's y axis has a right-click menu
with auto scale, a fixed range, and one **Remove '<signal>' from plot N** entry per
occupied trace slot (a plot with nothing on it says so instead).

**The plot does not twitch.** Two independent causes were fixed, both of which
showed up as the x axis moving a few pixels every frame — including while paused,
which is how the second one was found:

1. *The gutter followed the y tick labels.* `TickFormat::Si` changes their width
   as the auto-fitted range moves (`1.5` → `15` → `150` → `1.5k`), and
   `set_min_y_gutter()` is only a floor — the library still takes the larger of
   measurement and floor. `RowSet` therefore reserves the width of the widest
   label the formatter can produce, measured once with the plot's own monospaced
   font, and hands it to the *figure* (`Figure::set_min_y_gutter()`, added for
   this): `Figure::layout()` clears every row's own floor each frame to measure
   what its labels need, so a floor written onto the rows would survive exactly
   until the next layout. The figure seeds its common gutter — the maximum over
   the rows, which is also what aligns their axes — with that reservation.
2. *The status line drove the pane width.* A `GtkLabel`'s minimum width is its
   text, and the status line is rewritten every frame; as its counters and
   messages changed width they moved the paned divider, which resized the plot,
   which re-laid out the axes and their ticks. The label is now ellipsized, so its
   minimum is “…”; the plot keeps the width it was given.

The self-check reports both. `viewport.w` must collapse to a single value, and
`axes.x`, `axes.w` and `y-gutter` must settle: the one-shot label reservation
lands a few frames after the first measured frame, and because the figure serves
its floor out of the label strip the reservation converges over two layouts, so
their ranges show that settle and nothing after it. The companion `axes.w
changes: N (last at frame F)` line is what separates a settle from the twitch — a
handful of changes near the start is settled, a count near the frame count is the
bug. `x-ticks` is content, and varies by one as a live window slides over a round
number:

```
layout over 952 frames: axes.x 67..80, axes.w 546..559, viewport.w 627..627,
                        y-gutter 44..74, x-ticks 20..21, x-desync 0
axes.w changes: 2 (last at frame 35, side panel up=1)
```

Set `UDPSCOPE_PROBE_PAUSE_MS=<ms>` to pause the acquisition part-way through a
headless run, so the numbers are measured on a still picture.

The app also logs, once per second, one line per acquired channel with the number
of samples stored and the rate that implies, next to the rate the CONFIG declared:

```
[udpscope] ch 1 'ch2_2kHz' stored=8500 (+4603 kS/s, declared 0 S/s) stride=1 clock=1 y=[-0.98, 0.98]
```

The app no longer decimates (`stride=1 clock=1`), so a stored rate far below the
declared one is the datasource's own wire rate, not something the viewer did.

**Cursors start hidden** and are switched on from the overflow menu; **Reset cursors**
switches them on as well, because a reset nobody can see reads as a dead button.
The two panels collapse vertically (`SLIDE_DOWN`), so hiding one gives its space
back instead of leaving an invisible gap in the column.

The window dropdown offers the standard ladder up to `--max-window`, capped to
what one buffer actually holds (the cap is recomputed when a CONFIG changes the
partition, and never on a mere enable or calibration change). A span that is not
on the ladder — `Fit` frames whatever the ring holds, which is rarely a round
number — gets its own entry rather than being rounded to a neighbour, so the
dropdown reads what is on screen instead of approximating it.

**Fit to data** (overflow menu) does what the plot's own `f` key does: auto-scales every row's y
axis and re-frames the x window on the data the ring holds, propagated to every
row. The status line acknowledges it (`fitted to the buffered data (1.25 s)`).

**Shared x axis.** The rows are the rows of **one `plot::Figure`**, drawn by one
`plot::gtk4::PlotArea` in figure mode, and they all join a single x link group.
A gesture changes the row it landed on; `Figure::layout()` adopts that change
into the group and pushes it to the others, so panning or zooming any row moves
them all with nothing to propagate by hand. The live follow is the *group's*
(`set_x_link_follow`), not each row's — `Figure::link_x()` refuses a member whose
own axis still follows, because a following axis would reinterpret the pushed
range as its own span and offset behind its own edge, which is per-row drift by
construction. The figure also owns the left gutter and the crosshair, so a
vertical feature lands in the same pixel column in every row.

The self-check's layout line carries `x-desync N`, the number of frames in which
any row's x range differed from row 0's by more than a pixel of data. **It must
read 0.**

`Figure`'s row count is fixed at construction, so there is one structural path:
a changed row count, or a changed legend name (the one thing a `Plot` cannot
express), builds a new figure and the area is handed it. That costs no GL state —
the widget keeps its context. A displayed event survives the rebuild because the
pinned `[-pre, +post]` window is restated rather than re-derived from the live
edge.

**Export visible range…** (overflow menu) writes every signal bound to a visible row,
over the shared x window, at the **stored** sample rate — never the LOD pyramid's
buckets, which are the envelope the renderer drew rather than the acquisition. A
signal on several rows is one column; the columns come out in channel order, so the
file reads in the same order as the signal panel. The table has two time columns:
`time` is the axis as displayed (stream seconds live, `[-pre, +post]` while an event
is held) and `stream_time` is always the stream's own clock, so two exports taken in
different modes line up.

Two formats, picked by the extension the save dialog ends up with:

| file | what it is |
|---|---|
| `.npy` | one NumPy structured array, `time`/`stream_time` as `<f8` and every signal as `<f4` — the width the ring stores, so the file invents no precision and is about half an all-`f8` one. `pd.DataFrame(np.load(path))` is the whole reader. |
| `.csv` | the same table as text, `%.17g` for the times and `%.9g` for the values: the shortest widths that round-trip each type exactly. |

Parquet (the original ask) would have meant linking Arrow — tens of megabytes of
dependency for a feature whose output a three-line reader already handles.

The table is collected when the menu item is clicked, not when the dialog is
answered: a live view keeps scrolling while the file chooser is up. The live cut goes
through `ScopeModel::live_window_locked()`, the same index range `publish_event()`
uses, so an export and a capture of the same window hold the same samples.

## Review findings not (yet) implemented

These are known deviations, found by a spec/code review, that are deliberate for
now — they are listed so they are not mistaken for anything else:

- **There is one live buffer, not two.** The spec's second buffer is served by the
  latched `EventSnapshot` — an immutable copy of the captured window, which is
  exactly what "the buffer the view is frozen on" needs to be. Two live rings would
  also mean acquisition had to follow the trigger state, which is what left the first
  latch after arming with no pre-trigger history.
- **The buffer is allocated at the first CONFIG, not at initialisation.**
  The spec's "N+1 split" needs N, which only a CONFIG gives; `--buffer-mb` is the
  *budget*, and the power-of-two ring rounding means the final
  allocation lands in `(budget/2, budget]` rather than exactly on it. Every CONFIG,
  enable/disable and calibration re-partitions (and drops the history) because the
  buffer layout *is* the channel list.
- **Channels are not decimated at ingest.** The spec's `window * rate > buffer`
  rule was implemented as `stride = round(rate_fastest / rate_channel)`, which for
  a 2 kHz sine on a 1 MHz clock is exactly one sample per period: every kept
  sample sat at the same phase and the channel plotted as a flat line. Channels
  are now stored at the shared clock's rate and the LOD min/max pyramid preserves
  the extremes at draw time. When the requested window does not fit the budget the
  *shared clock* is decimated instead (the status line's `time xN`).
- **Pre-trigger % is a share of the *event* window**, not of the retained history —
  see the trigger section above for why (the spec's sentence predates the event
  window being a separate control).
- **A plot holds at most four signals** (`RowSet::kSlotsPerRow`), because a `Plot`
  cannot give up a trace and the slots are therefore created up front.
- **One toggle drives both cursor orientations.** The vertical markers and the
  horizontal ruler exist separately but share one show/hide and one reset.
- **The time reference is the first *enabled* channel's signal.** A CONFIG whose
  time signal belongs to a signal that is disabled contributes no stream time, and
  the app falls back to arrival-based timing (with a warning on stderr).

## Limitations

- **Multi-element (vector/matrix) signals are acquired with a zero-order hold.**
  `udps_client` decodes exactly one *sample* of `num_rows x num_cols` values per
  DATA packet for such a signal, regardless of the publish mode, so it cannot be
  decimated on its declared rate. Each element becomes one channel sampled on the
  shared clock's own cadence with the packet's value held until the next packet —
  exactly faithful in `strict` mode, and the only reading of the wire that does
  not invent samples in `accumulate` mode. The info popup and the CONFIG dump on
  stderr report the element counts.
- **The event window can be shorter than the buffer.** The trigger latches a window
  of the event-span selection; the ring holds far more, so with trigger mode off the
  rest is scrollable history.
- **Calibration is applied at ingest**, so changing gain or offset rebuilds the
  buffers (and drops the history). Applying it at read time would need the LOD
  pyramid to be gain-aware.
- **Cursors are per row** and not synchronised with each other; show/hide parks
  them outside the axis range rather than removing them, because `Plot` has no
  `remove_cursor`. The trigger marker is the one exception: it has its own cursor
  and is independent of the cursor toggle.
- **Verified headless only.** Acquisition, partitioning, the trigger state
  machine and its integration with the buffers, and rendering under Xvfb
  (llvmpipe) were exercised; the interactive paths (drag and drop, the dialogs,
  the right-click menu) were not driven by an automated test.