diff --git a/docs/superpowers/specs/2026-08-27-udpscope-direct-udps-imgui-scope-design.md b/docs/superpowers/specs/2026-08-27-udpscope-direct-udps-imgui-scope-design.md new file mode 100644 index 0000000..a56ec97 --- /dev/null +++ b/docs/superpowers/specs/2026-08-27-udpscope-direct-udps-imgui-scope-design.md @@ -0,0 +1,535 @@ +# UDPScope — direct-to-streamer ImGui oscilloscope + +Date: 2026-08-27 +Status: design approved + +## 1. Purpose + +A bench oscilloscope that attaches straight to a single `UDPStreamer` and needs +nothing else: no StreamHub, no Go toolchain, no browser, no web server. One +binary plus the fonts it ships with, dropped on a machine that may have none of +the project's other infrastructure installed. + +The data path is `UDPStreamer → Common/Client/c → UDPScope`. Everything the Go +hub and the C++ StreamHub do — ring buffering, decimation, the trigger — the +scope does locally, for one source. + +Success criterion: deployment simplicity. The feature bar is "what a bench scope +needs", not parity with the StreamHub client. + +## 2. Scope + +In: + +- Single UDPS source, unicast or multicast, with the library's reconnect + behaviour. +- Free-grid multi-plot view built by splitting panes. +- Normal and Single trigger modes, plus an always-on live view. +- Cursors and window measurements. +- Layout and settings persistence. +- CSV export of the displayed capture. + +Out: + +- Auto and Roll trigger modes. +- X/Y (Lissajous) plotting. +- Multiple simultaneous sources. +- History-on-disk / zooming beyond the buffered window. + +## 3. Architecture + +Two threads and one shared object. + +### 3.1 Receiver thread + +Owns the `udps_client_t` and does nothing but + +```c +while (!stop) { udps_client_poll(cli, 20); } +``` + +The library creates no threads and performs all work inside `poll()`, so one +thread owning one client is its intended use. All three callbacks fire on this +thread. + +- `on_config` — copies the `udps_signal_t` table into `SignalStore`, resizes + rings, resets the time base. +- `on_data` — reconstructs per-element timestamps (§5), appends samples to the + rings, and runs the trigger detector (§6) over the trigger signal's samples. +- `on_event` — records connection state and the last error for the status bar. + +Frame memory belongs to the library and is invalid once the callback returns, so +the receiver copies on the spot. It does that anyway when it appends to a ring. + +The alternative — polling inside the render loop, as `udps_dump` does — was +rejected. `SDL_GL_SwapWindow` blocks on vsync, so for up to ~16 ms per frame +nothing drains the socket, and a slow frame (window resize, CSV export) stretches +that to hundreds of milliseconds. At 1 MSps a 200 ms stall is ~3 MB of datagrams +that must survive in `SO_RCVBUF` or be lost, and loss shows up as silent holes in +a scope trace. + +A lock-free SPSC staging queue between the two threads was also considered and +rejected as speculative: at 60 Hz the GUI holds the store lock for microseconds +per read, so there is no measured contention to remove. Revisit only if +profiling shows otherwise. + +### 3.2 GUI thread + +`main.cpp`'s SDL2 + OpenGL3 + ImGui + ImPlot loop, same shape as +`Client/streamhub/main.cpp`: same context setup, same Catppuccin Mocha styling, +same executable-relative font resolution (installed → build tree → source tree). + +Each frame it locks the store briefly, reads the windows it needs, unlocks, then +decimates and draws from its own copies. No drawing happens under the lock. + +### 3.3 Shared state + +`SignalStore` is the entire shared surface: the signal metadata table, one +`SignalBuffer` per signal, the measured per-signal sample rate, the trigger +state, and the connection status. A single `std::mutex` guards all of it. + +Keeping the shared surface to one named class is deliberate — there is exactly +one place to look for a race. Note that `SignalBuffer.h`'s doc comment claims +the buffer is thread-safe; it is not, it has no locks. `SignalStore` provides +all the locking. + +### 3.4 Module layout + +``` +Client/udpscope/ + CMakeLists.txt SDL2 + OpenGL + FetchContent(ImGui, ImPlot, GoogleTest) + main.cpp SDL/ImGui/ImPlot bootstrap, style, fonts, event loop + App.{h,cpp} owns Receiver/SignalStore/PaneTree/UI state; update() + Receiver.{h,cpp} the thread, the udps_client_t, the three C callbacks + SignalStore.{h,cpp} signal table + rings + trigger state; the one mutex + TimeBase.{h,cpp} producer-clock to wall-clock reconstruction + Trigger.{h,cpp} edge detector and FSM + PaneTree.{h,cpp} BSP split tree: split/close/layout/hit-test + PaneView.cpp ImPlot rendering of one pane + SignalList.cpp side panel, drag source + TriggerBar.cpp trigger controls and status badge + Measure.{h,cpp} cursor deltas and window statistics + Decimate.{h,cpp} min/max envelope decimation + Settings.{h,cpp} layout and config persistence + Export.{h,cpp} CSV writer + tests/ unit tests for the framework-free modules +``` + +`PaneTree`, `Trigger`, `TimeBase`, `Measure`, `Decimate`, `Settings` and +`Export` are framework-free C++17 with no ImGui and no UDPS dependency. Each is +unit-testable without a window or a socket. That is where the fiddly logic lives +and none of it should need a GUI to exercise. + +### 3.5 Reuse + +- `../streamhub/SignalBuffer.h` is used verbatim. It is header-only and + framework-free, and the Qt client already reuses it this way. +- `../streamhub/resources/` supplies the fonts and `Icons.h`. +- `../../Common/Client/c/udps_client.c` is compiled directly into a CMake + `udpsclient` static target, so building the scope does not require running the + C library's own Makefile first. + +Nothing under `Client/streamhub/` is modified. + +### 3.6 Decimation + +`SignalBuffer.h` ships `LTTBDecimate` and the scope does not use it. LTTB +selects representative points and will silently drop a one-sample glitch; on a +scope that glitch is usually the thing being looked for. `Decimate.{h,cpp}` +implements a min/max envelope instead — for each screen column, emit the column's +minimum and maximum in time order — which preserves extremes exactly. This is +the same conclusion the Go hub reached when it replaced LTTB with +`minMaxDecimate`. + +The emitted pair must stay in time order, not value order, or downstream +binary-searching by time breaks. + +## 4. Signal model + +### 4.1 Identity + +Signals are identified by **name** everywhere in the UI and in persisted +settings, never by index. A reconnect or a streamer reconfiguration may reorder +or renumber signals; pane assignments must survive that. + +On a CONFIG update: + +- Signals present before and after keep their ring and their pane assignments. +- New signals appear in the signal list, unassigned. +- Signals that vanished keep their pane assignment, drawn greyed and labelled + missing. A transient reconnect must not destroy a layout the user built. + +### 4.2 Array handling + +A UDPS signal may be a scalar or an `num_rows × num_cols` array. An array is +either a **packed time burst** (N consecutive samples, unrolled onto the time +axis) or a **true vector** (a profile, plotted against element index for the most +recent frame). + +`time_mode` decides, and only one case is genuinely ambiguous: + +| `time_mode` | elements | treated as | +|---|---|---| +| `FULL_ARRAY`, `FIRST_SAMPLE`, `LAST_SAMPLE` | > 1 | burst, always | +| `PACKET` | > 1 | burst by default, user-switchable to vector | +| any | 1 | scalar | + +`PACKET` means the producer stamped the whole datagram with one time, which is +what a genuine vector looks like *and* what a burst carrying no time metadata +looks like. The default is burst, matching `UDPSourceSession`'s handling so the +scope and StreamHub agree on the same stream; the signal list offers a per-signal +"profile" toggle for the case where it really is a vector. `sampling_rate` is not +used as the discriminator — a `PACKET` burst frequently declares no rate, and §5 +reconstructs its spacing from inter-packet timing instead. + +A vector-mode signal is excluded from trigger capture and from the shared X axis. +A pane holds either time signals or vector signals, not both; dropping across the +two is rejected with a tooltip. + +## 5. Time base + +The C library's `udps_frame_element_time()` is explicitly an arrival-anchored +*estimate*. It is not sufficient: the kernel frequently delivers several queued +datagrams in one burst, so two packets are processed microseconds apart even +though each represents ~10 ms of signal, and arrival-time interpolation then +crams a packet's samples into that tiny gap. The trace renders as a sawtooth. +`Source/Applications/StreamHub/UDPSourceSession.cpp` documents this failure and +solves it; the scope reproduces the same rules on top of the C API, all of whose +inputs are present in `udps_frame_t`. + +Per signal, in priority order: + +1. **`FULL_ARRAY` with a declared time signal** — per-element timestamps read + from the referenced time signal's array, offset onto wall clock by the + calibration below. +2. **`FIRST_SAMPLE` / `LAST_SAMPLE` with a declared time signal** — anchor from + element 0 of the time signal, then `± k / sampling_rate` per element, + forwards for `FIRST_SAMPLE` and backwards for `LAST_SAMPLE`. +3. **Accumulated scalar** (`num_samples > 1`, one element per sample) — base + from the packet's embedded producer `hrt`, which is sampled at acquisition + and so immune to the burst-delivery problem, then `base + k × dt`. `dt` is + `1 / sampling_rate` when declared; otherwise the `hrt` gap to the previous + packet divided by *that* packet's sample count, which is exactly one cycle + period because the flushes carry contiguous RT cycles; otherwise 1 ms until + the first gap is known. +4. **`PACKET` burst with no time signal** — elements span + `(lastPacketTime, thisPacketTime]`, i.e. dated backwards from arrival rather + than forwards from it. The samples were acquired *before* the packet arrived, + and spanning backwards keeps ring time strictly monotonic under jitter, where + forward extrapolation would let one packet overlap the next. The first packet + after connect is dropped rather than stored with wrongly spaced timestamps. +5. **Plain scalar, or no usable time reference at all** — packet arrival wall + time (`recv_time`). + +Time-signal values convert with the protocol's documented units: `1e-9` s per +count for `UDPS_T_UINT64`, `1e-6` otherwise. + +**Calibration.** A time signal or `hrt` gives a producer clock, not wall clock. +`TimeBase` maintains an offset per producer clock, established once from the +first packet (`offset = recv_time − producerSeconds`) and thereafter only +corrected if the residual drifts beyond a threshold, so the trace does not jitter +with network delay. + +**`hrt` tick rate.** StreamHub converts `hrt` using the local MARTe +`HighResolutionTimer` frequency, which is only valid when the client runs on the +producer's host. The scope cannot assume that, so `TimeBase` estimates +ticks-per-second by least-squares fit of `hrt` against `recv_time` over a rolling +window of packets, yielding both rate and offset. Until the fit has enough +samples (default 32 packets) rule 5 is used. + +**Display epoch.** The X axis shows seconds relative to the first sample of the +session, so it reads `0.000 … N s` rather than `1.7e9`. Absolute wall-clock time +is shown in the readout and written to CSV exports. + +## 6. Trigger + +Runs in the receiver thread, over every sample as it is appended. An edge +detector that only ran at 60 Hz would see the signal through the frame rate, +quantise the trigger time to the frame, and miss edges whenever the GUI hitched. + +### 6.1 Configuration + +| Field | Meaning | +|---|---| +| `signalName` | which signal is watched | +| `edge` | rising / falling / both | +| `threshold` | crossing level, in signal units | +| `hysteresis` | signal must first leave the band `threshold ± hysteresis`; 0 disables | +| `windowSec` | total capture span | +| `prePercent` | share of the window before the trigger instant | +| `mode` | normal / single | + +There is no holdoff setting. The capture window is its own holdoff: the FSM +cannot re-arm until the current capture has been harvested. + +### 6.2 State machine + +``` +Idle ──arm──> Armed ──edge──> Collecting ──post filled──> Held + ^ ^ │ + │ └────────────── mode == normal ────────────┤ + └──────────────────── disarm ─────────────────────────────┘ +``` + +- **Armed** additionally requires the ring to already hold `preSec` of samples — + otherwise the capture would come back short. Until it does, the status badge + shows a fill percentage. This is a real state users see, not an internal + detail. +- **Collecting** ends when the trigger signal's newest sample time reaches + `tTrig + postSec + kHarvestMarginSec`. +- **Held** freezes the display on the capture. `normal` re-arms immediately; + `single` waits for a manual re-arm. + +### 6.3 Edge detection + +Consecutive samples `(t0,v0) → (t1,v1)` cross rising when `v0 < threshold` and +`v1 >= threshold`. The trigger instant is linearly interpolated: +`tTrig = t0 + (threshold − v0) / (v1 − v0) × (t1 − t0)`, giving sub-sample +resolution so the capture is stable on screen rather than jittering by one +sample period. + +With `hysteresis > 0`, a rising cross is only accepted if the signal has been +below `threshold − hysteresis` since the previous trigger; symmetrically for +falling. `both` alternates direction. + +### 6.4 Harvest + +Once `Collecting` completes, the GUI thread extracts +`[tTrig − preSec, tTrig + postSec]` from every displayed burst signal's ring into +a **capture snapshot** — a plain, non-circular copy owned by the GUI thread. +Panes render the snapshot, frozen, until the next capture replaces it. The +snapshot is what CSV export writes and what measurements are computed over. + +`kHarvestMarginSec` covers late or out-of-order packets only; unlike the Go hub +there is no push-tick latency, because the receiver knows precisely when samples +land. Default `max(0.05 s, 2 × observed frame span)`. + +## 7. Buffering + +One `SignalBuffer` per signal, sized by + +``` +capacityPoints = ceil(measuredRate × windowSec × kRingMargin) // kRingMargin = 4 +``` + +clamped to `[4096, maxPointsPerSignal]` (`--max-mpts`, default 8 Mpts ≈ 128 MB). + +`measuredRate` is observed (samples appended ÷ elapsed), not the declared +`sampling_rate`, which may be 0. + +**Why the margin is explicit.** A capture is not read out when its last sample +arrives but `kHarvestMarginSec` later. A ring holding exactly the window has +already overwritten the front of its own capture by then, and every shot comes +back missing its head. That is precisely the defect just fixed in +`Common/Client/go/wshub` (`captureLagSec`), and the constant carries this comment +so it cannot recur silently. A factor of 4 covers the window itself, the harvest +margin, and rate-estimation error, with headroom for panning the live view. + +**Resizing** is triggered by a CONFIG change, a trigger-window change, or the +measured rate drifting more than 2× from the sizing assumption. `SignalBuffer:: +setCapacity` clears the buffer, which would blank the display; `SignalStore` +instead reads out the newest `min(count, newCapacity)` points and re-pushes them +after the resize, so the live view survives. Resizing never happens while a +capture is being harvested. + +## 8. User interface + +### 8.1 Screen layout + +``` +┌────────────────────────────────────────────────────────────┐ +│ menu bar: File View Help [connection badge] │ +├────────────────────────────────────────────────────────────┤ +│ trigger bar: signal | edge | thr | hyst | win | pre% | │ +│ Norm/1x | [ARMED 62%] | Arm Disarm │ +├──────────────┬─────────────────────────────────────────────┤ +│ signal list │ │ +│ (drag │ pane tree │ +│ source) │ │ +│ │ │ +├──────────────┴─────────────────────────────────────────────┤ +│ status: packets, MB, frames, gaps, dropped, reconnects │ +└────────────────────────────────────────────────────────────┘ +``` + +The status bar reports `udps_stats_t` directly; `counter_gaps` and +`fragments_dropped` are the honest indication that the scope is not seeing +everything, and are highlighted when non-zero. + +### 8.2 Pane tree + +```cpp +struct PaneNode { + bool leaf; + // leaf: + std::vector signals; // name, colour, line width, VScale + // split: + Orientation orient; // Columns (side by side) | Rows (stacked) + double ratio; // first child's share, 0..1 + std::unique_ptr a, b; +}; +``` + +Operations, all in `PaneTree` and all testable without a GUI: +`splitLeaf(leaf, orient)`, `closeLeaf(leaf)` (the parent is replaced by the +surviving sibling), `layout(rect) → [(leaf*, rect)]`, `hitTestSplitter(pos)`, +`hitTestHandle(pos)`. + +**Splitting.** Hovering a pane reveals four inset split handles at the midpoints +of its edges plus an ✕ in the top-right corner. Clicking the left or right handle +splits into columns; top or bottom splits into rows. The new pane is empty and +takes half the space. + +Handles are used rather than "click anywhere on the pane border" because a pane +in the middle of a 3×3 layout touches no window edge, and a rule keyed on window +edges would leave such panes unsplittable. Insetting the handles also keeps them +clear of the splitter drag zone. + +**Resizing.** The shared border between two panes is a drag target that adjusts +the parent split's `ratio`. Minimum pane size 80 px; a drag that would go below +it clamps. + +**Assignment.** The signal list on the left is an ImGui drag source; dropping a +signal on a pane appends it to that pane's list, overlaid on the existing traces. +Right-clicking a trace's legend entry removes it or opens its colour, line width +and vertical scale. + +### 8.3 Axes + +All time panes share one X axis: pan or zoom in any pane moves all of them, which +is what makes comparing signals across panes meaningful. In live mode the X range +follows the newest sample; any manual pan or zoom detaches it, and a "Live" +button re-attaches. + +Vertical scale is per assignment, with three modes: + +- **Auto** — fit to the data visible in the current X range. +- **Range** — use the `range_min`/`range_max` the CONFIG packet already carries, + which is free and is the physically meaningful full scale. +- **Manual** — volts-per-division and offset, as on a bench scope. + +### 8.4 Cursors and measurements + +Two global vertical cursors A and B, toggled from the View menu. Readout shows +`tA`, `tB`, `Δt`, `1/Δt`, and per displayed signal the value at each cursor and +`ΔV`. + +Per pane, a collapsible readout gives each signal's min, max, peak-to-peak, mean +and RMS over the current X range. All statistics are computed from the +undecimated ring or capture data, never from the decimated screen points, so a +narrow spike is counted even when it is not individually drawn. + +## 9. Persistence + +`Settings` writes a line-based recursive format, chosen over JSON because the +pane tree is nested (so INI is awkward) and vendoring a JSON header for one file +works against the single-binary goal. Roughly 100 lines of writer plus recursive +descent parser. + +``` +udpscope 1 +source host=127.0.0.1 port=44500 multicast= iface= dataport=0 +trigger signal=Voltage edge=rising thr=0.5 hyst=0.01 win=0.1 pre=20 mode=normal +cursors on 0.0123 0.0456 +tree + split cols 0.5 + leaf + sig Voltage color=#89b4fa width=1.5 vs=auto + sig Current color=#fab387 width=1.5 vs=manual div=0.2 off=0.0 + leaf + sig Temp color=#a6e3a1 width=1.5 vs=range +``` + +Location: `$XDG_CONFIG_HOME/udpscope/session.conf`, falling back to +`~/.config/udpscope/session.conf`, overridable with `--config PATH`. Saved on +clean exit and from File → Save Layout. A malformed or version-mismatched file +is reported in the status bar and ignored, never partially applied. + +## 10. CSV export + +File → Export writes the current capture snapshot (or, when not triggered, the +visible live window) in long format: + +``` +signal,time_s,wallclock_s,value +Voltage,0.000000,1756291200.123456,0.4981 +Voltage,0.000001,1756291200.123457,0.4993 +``` + +Long format rather than one column per signal because signals carry independent +timestamps, so a wide format would require resampling and would silently +misrepresent the data. Export covers all signals assigned to any pane, or a +single pane on request. + +## 11. Command line + +Mirrors `example/udps_dump.c` so the two are interchangeable: + +``` +udpscope [--host ADDR] [--port N] [--multicast GROUP] [--iface ADDR] + [--data-port N] [--silence SEC] [--config PATH] [--max-mpts N] +``` + +Long `--` options only. Defaults `127.0.0.1:44500`, matching `udps_dump`. + +Precedence: an explicitly given command-line option wins over the same field in +the settings file; anything not given on the command line comes from the settings +file; anything in neither takes the built-in default. Connecting somewhere on the +command line therefore does not silently rewrite the saved session, but saving +the layout afterwards does record the new source. + +## 12. Build + +`Client/udpscope/CMakeLists.txt`, following `Client/streamhub/CMakeLists.txt`: + +- `find_package(OpenGL REQUIRED)`; SDL2 via `CONFIG` then a pkg-config fallback. +- `FetchContent` for ImGui v1.91.8 and ImPlot v0.17, built into an `imgui_lib` + static target compiled with `-w`. +- `add_library(udpsclient STATIC ../../Common/Client/c/udps_client.c)` compiled + as C99 with the library's own warning flags. +- App compiled with `-Wall -Wextra -Wno-unused-parameter`. +- Fonts copied next to the binary; `APP_RESOURCE_DIR` points at the source tree + as the final fallback. +- Install rules for the binary, fonts, `.desktop` entry and icon. + +```bash +cd Client/udpscope && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build +``` + +## 13. Testing + +`UDPSCOPE_BUILD_TESTS` (default ON) adds GoogleTest via `FetchContent` and a +single `udpscope_tests` binary covering the framework-free modules: + +- **PaneTree** — split produces the expected geometry; close promotes the + sibling and reclaims its space; ratios survive a round trip through + `layout()`; hit-testing distinguishes handle from splitter; minimum size + clamps. +- **Trigger** — rising, falling and both edges; interpolated trigger time on a + known ramp; hysteresis suppresses a noisy re-cross; the fill gate refuses to + arm without a pre-window; normal re-arms and single does not. +- **TimeBase** — time-signal conversion at both unit scales; offset calibration; + the `hrt` rate fit converges to a known tick rate; the burst-delivery case + produces monotonic evenly spaced timestamps rather than a sawtooth; a `PACKET` + burst stream stays strictly monotonic under jittered arrivals, with no overlap + between consecutive packets. +- **Decimate** — extremes are preserved; output stays in time order; input + shorter than the budget passes through unchanged. +- **Settings** — write/parse round trip reproduces the tree exactly; malformed + input is rejected without partial application. +- **Export** — long-format output for a known snapshot. + +`Receiver` and `SignalStore` are exercised indirectly: a test feeds synthetic +`udps_frame_t` structures to the receiver's data-callback logic, which is +factored out of the C callback into a plain method so it can be called without a +socket. + +Manual verification: run `Test/Configurations/streamhub_demo.cfg`'s streamer and +point the scope at it, alongside `udps_dump` for cross-checking. + +## 14. Documentation + +- `Docs/UDPScope.md` — user guide: CLI, pane interaction, trigger, measurements, + export, settings file format. +- A row in the README component table and in the documentation index. +- A line in `CLAUDE.md`'s build section, next to the ImGui and Qt clients.