diff --git a/docs/superpowers/specs/2026-08-16-udpstreamer-signal-calibration-and-config-design.md b/docs/superpowers/specs/2026-08-16-udpstreamer-signal-calibration-and-config-design.md new file mode 100644 index 0000000..6f29526 --- /dev/null +++ b/docs/superpowers/specs/2026-08-16-udpstreamer-signal-calibration-and-config-design.md @@ -0,0 +1,220 @@ +# Per-signal calibration and persistent hub config + +Date: 2026-08-16 +Scope: `Client/udpstreamer/static/`, `Common/Client/go/wshub/`, `Source/Applications/StreamHub/` + +## Problem + +The web oscilloscope plots whatever the streamer sends. A signal carrying raw ADC +counts cannot be read in volts, and the vertical-scale toolbar's V/div and Offset +are display-only: they move the trace on screen but do not change what the cursor, +hover readout or CSV export report. + +Separately, the only persistent state is the source list, saved server-side by the +`saveSources` WebSocket command. That command is fire-and-forget — the browser +never learns whether the write succeeded — and there is no way to re-read the file +without restarting the hub. + +## Goals + +1. A per-signal affine calibration `y = raw * scale + offset`, with an optional + unit override, applied consistently everywhere a value is shown. +2. Calibration stored in the hub's config file alongside the sources, so it + survives a browser reload and is shared between browsers. +3. A left-sidebar section to save and reload that config, with success/error + feedback. +4. Identical behaviour from the Go hub (`Common/Client/go/wshub`) and the C++ + `StreamHub`, per the protocol-parity rule in CLAUDE.md. + +## Non-goals + +- Calibrating on the data path. Raw samples stay raw in the rings, in recorded + history, and in the trigger comparator. +- Per-array-element calibration. One entry covers a whole array signal. +- Named profiles. One config file, the existing `-sources-file` / `SourcesFile`. +- Persisting display state (layout, trace colours, V/div, window, trigger config). + +## Data model + +A calibration entry is keyed by `(source label, signal base name)`: + +| Field | Type | Default | Validation | +|---|---|---|---| +| `source` | string | — | must be non-empty | +| `signal` | string | — | base signal name, no `[i]` suffix | +| `scale` | float64 | `1` | finite, non-zero | +| `offset` | float64 | `0` | finite | +| `unit` | string | `""` | trimmed, max 16 chars; empty means "use the streamer's unit" | + +The key uses the source **label**, not the runtime id (`s1`, `s2`). Ids are +assigned in add-order at startup, so a saved calibration keyed by id would rebind +to a different source whenever the source list order changed. Labels default to +the address when the user leaves the label blank, which keeps them unique in +practice; two sources sharing a label share a calibration, which is a documented +consequence rather than an error. + +Array signals get one entry covering every element. The V-Scale toolbar can be +opened on a single element (`Adc[3]`), so its calibration row states which base +signal and how many elements the edit affects. + +## Config file format + +The file remains a flat JSON array of **flat** objects. Sources keep their current +shape; calibration entries are appended as additional elements: + +```json +[ + {"label": "wave", "addr": "127.0.0.1:44500"}, + {"label": "mc", "addr": "127.0.0.1:44501", "multicastGroup": "239.0.0.1", "dataPort": 44502}, + {"source": "wave", "signal": "Adc", "scale": 0.00030518, "offset": -1.25, "unit": "V"} +] +``` + +A block containing `addr` is a source; a block containing `signal` is a +calibration; anything else is skipped with a warning. + +The flatness is a hard constraint, not a preference. `StreamHub::LoadSourcesFile` +(`StreamHub.cpp:854`) is a hand-rolled scanner that takes each `{` up to the next +`}` as one object. A nested `"calibration": { ... }` inside a source entry would +truncate at the inner brace and corrupt the parse. Keeping every element flat lets +that scanner gain a single discriminator branch, and every config file written by +the current binaries still loads unchanged in both hubs. + +## WebSocket protocol + +New frames, implemented identically in both hubs: + +| Direction | Frame | +|---|---| +| hub → client | `{"type":"calibration","cal":[{"source","signal","scale","offset","unit"}, …]}` | +| client → hub | `{"type":"setCalibration","source","signal","scale","offset","unit"}` | +| hub → client | `{"type":"configSaved","ok":bool,"path":string,"error":string}` | +| client → hub | `{"type":"reloadConfig"}` | +| hub → client | `{"type":"configReloaded","ok":bool,"error":string}` | + +`calibration` is broadcast when a client connects and after every accepted +`setCalibration` or successful `reloadConfig`. It is a separate message rather +than a field on the existing `sources` broadcast because the C++ +`BroadcastSources` serialises into a fixed 4096-byte buffer that a calibration +table would overflow. + +`setCalibration` is validated hub-side against the table above. An invalid entry +is rejected and no broadcast is emitted, so the offending client reverts to the +last broadcast value. + +`saveSources` keeps its name and now writes both sources and calibration entries. +It gains the `configSaved` acknowledgement it currently lacks. + +## Reload semantics + +`reloadConfig` re-reads the config file, then: + +- **replaces** the calibration table wholesale with the file's contents; +- **adds** any source present in the file that is not currently active; +- **never** removes, restarts, or reconnects a live source. + +Reload must not interrupt streaming, so an unsaved source the user added stays +running. The asymmetry (calibration replaced, sources merged) is deliberate: +calibration is cheap to reapply, a source is a live UDP session. + +## Hub implementation + +**Go (`Common/Client/go/wshub/sources.go`, `hub.go`).** A +`map[string]calEntry` keyed by `source + "\x00" + signal`, owned by the +`SourceManager` behind its existing `sync.RWMutex`. `setCalibration` and +`reloadConfig` are dispatched from the `readPump` command switch in `hub.go` +alongside `addSource`/`removeSource`/`saveSources`/`zoom`. `SourceConfig` gains a +sibling `CalConfig` type; `Save` writes both slices into one array; +`Load` decodes into `[]map[string]json.RawMessage` and discriminates per element. + +**C++ (`Source/Applications/StreamHub/StreamHub.{h,cpp}`).** A fixed +`kMaxCalibration = 256` array of +`{StreamString source, signal, unit; float64 scale, offset;}` — no STL, per the +`Source/Components` and StreamHub style rules. `HandleSetCalibration`, +`HandleReloadConfig` and `BroadcastCalibration` mirror the existing +`HandleAddSource` / `BroadcastSources` shape, with `BroadcastCalibration` using +its own 16 KiB buffer like `BroadcastConfig`. `LoadSourcesFile` gains the +discriminator branch; `HandleSaveSources` appends the calibration entries and +sends `configSaved`. + +## SPA implementation + +Calibration composes into the existing vertical-scale transform: + +``` +y_cal = raw * scale + offset +y_norm = (y_cal - vsOffset) / divValue + screenPos +``` + +`applyVScaleNorm` (`app.js:186`) and its `applyDigitalNorm` / `applyMixedNorm` +siblings are the only places raw values enter the display path. Applying the +calibration there leaves the norm↔value inverse arithmetic untouched, so the +hover readout, cursors, rulers, Y-axis tick values and the V-Scale toolbar's own +V/div and Offset fields all report calibrated units without further change. V/div +and Offset are redefined as "calibrated units per division" and "calibrated value +at screen centre"; they remain a display concern, distinct from calibration. + +Four sites need explicit handling because they bypass that path: + +1. `resolveVScale` (`app.js:109`) `range` mode reads `meta.rangeMin` / + `meta.rangeMax` from the streamer CONFIG. Both are calibrated and then + re-ordered, since a negative `scale` swaps them. +2. `exportAllCSV` (`app.js:2871`) fetches full-resolution data from the ring or + history and formats it directly. It calibrates each column and writes the + effective unit into the header. +3. Trigger threshold. The hub compares against raw samples, so the SPA sends + `(threshold - offset) / scale` and displays the inverse. V2 capture frames + arrive raw and flow through the normal display path, so they need nothing. +4. Unit display: the sidebar `sig-unit` badge, the hover readout and the V-Scale + header show the override when set, otherwise the streamer's `sig.unit`. + +**Calibration UI.** A new row in `#vscale-menu` (`index.html:187`), the toolbar +already opened by clicking a signal in a plot: + +``` +Cal Scale [ 1 ] Offset [ 0 ] Unit [ V ] [Reset] +``` + +with a header line naming the base signal and element count it will affect. Edits +apply locally, send `setCalibration`, and are confirmed by the hub's broadcast. +`Reset` restores `scale=1, offset=0, unit=""` and sends that. + +**Config UI.** The collapsible "Add Source" section (`app.js:3345`) is renamed +"Sources & Config" and keeps its address/label/multicast inputs and Connect +button. The existing fire-and-forget "Save list" button is replaced by **Save** +and **Reload**, plus a one-line status area rendering the `configSaved` / +`configReloaded` ack: the written path on success, the error text on failure. + +**Fallback.** The SPA seeds its calibration table from +`localStorage['udpscope.calibration']` at page load, and overwrites it wholesale +the first time a `calibration` message arrives. The mirror is rewritten on every +change, including changes received from the hub. A hub binary that predates this +change never sends `calibration`, so the mirror simply remains authoritative; +the same holds for a hub started without a config file. + +**Validation.** The same rules as the hub are enforced in the input handlers: a +non-finite or zero `scale` reverts the field to its last accepted value. + +## Testing + +**Go.** Table tests for the heterogeneous-array parse (current-format file, +new-format file, unrecognised block, malformed entry), `setCalibration` +validation including `scale = 0` and non-finite values, and a save→load +round-trip asserting sources and calibration both survive. + +**C++.** Extend an E2E `chain` scenario's config file with a calibration entry and +assert the hub re-serialises it unchanged after a `saveSources`, which exercises +the discriminator branch in `LoadSourcesFile` and the writer in +`HandleSaveSources`. + +**Browser.** `node --check static/app.js`, plus a manual pass: set a scale and +offset on a live signal and confirm the plot, hover readout, cursor readout, CSV +export and trigger threshold all agree; reload the page and confirm the +calibration returns; press Reload and confirm an unsaved edit is discarded while +the live source keeps streaming. + +## Documentation + +`Docs/StreamHub-API.md` gains the four new frames; `Docs/WebUI.md` gains the +calibration row and the Sources & Config section; `ARCHITECTURE.md` §6 gains the +config file format.