MARTe-Integrated-Components/docs/superpowers/specs/2026-08-16-udpstreamer-signal-calibration-and-config-design.md

# Per-signal calibration and persistent hub config

Date: 2026-08-16
Scope: `Client/udpstreamer/static/`, `Common/Client/go/wshub/`, `Source/Applications/StreamHub/`

## Problem

The web oscilloscope plots whatever the streamer sends. A signal carrying raw ADC
counts cannot be read in volts, and the vertical-scale toolbar's V/div and Offset
are display-only: they move the trace on screen but do not change what the cursor,
hover readout or CSV export report.

Separately, the only persistent state is the source list, saved server-side by the
`saveSources` WebSocket command. That command is fire-and-forget — the browser
never learns whether the write succeeded — and there is no way to re-read the file
without restarting the hub.

## Goals

1. A per-signal affine calibration `y = raw * scale + offset`, with an optional
   unit override, applied consistently everywhere a value is shown.
2. Calibration stored in the hub's config file alongside the sources, so it
   survives a browser reload and is shared between browsers.
3. A left-sidebar section to save and reload that config, with success/error
   feedback.
4. Identical behaviour from the Go hub (`Common/Client/go/wshub`) and the C++
   `StreamHub`, per the protocol-parity rule in CLAUDE.md.

## Non-goals

- Calibrating on the data path. Raw samples stay raw in the rings, in recorded
  history, and in the trigger comparator.
- Per-array-element calibration. One entry covers a whole array signal.
- Named profiles. One config file, the existing `-sources-file` / `SourcesFile`.
- Persisting display state (layout, trace colours, V/div, window, trigger config).

## Data model

A calibration entry is keyed by `(source label, signal base name)`:

| Field | Type | Default | Validation |
|---|---|---|---|
| `source` | string | — | must be non-empty |
| `signal` | string | — | base signal name, no `[i]` suffix |
| `scale` | float64 | `1` | finite, non-zero |
| `offset` | float64 | `0` | finite |
| `unit` | string | `""` | trimmed, max 16 UTF-8 bytes; empty means "use the streamer's unit" |

The key uses the source **label**, not the runtime id (`s1`, `s2`). Ids are
assigned in add-order at startup, so a saved calibration keyed by id would rebind
to a different source whenever the source list order changed. Labels default to
the address when the user leaves the label blank, which keeps them unique in
practice; two sources sharing a label share a calibration, which is a documented
consequence rather than an error.

Array signals get one entry covering every element. The V-Scale toolbar can be
opened on a single element (`Adc[3]`), so its calibration row states which base
signal and how many elements the edit affects.

## Config file format

The file remains a flat JSON array of **flat** objects. Sources keep their current
shape; calibration entries are appended as additional elements:

```json
[
  {"label": "wave", "addr": "127.0.0.1:44500"},
  {"label": "mc",   "addr": "127.0.0.1:44501", "multicastGroup": "239.0.0.1", "dataPort": 44502},
  {"source": "wave", "signal": "Adc", "scale": 0.00030518, "offset": -1.25, "unit": "V"}
]
```

A block containing `addr` is a source; a block containing `signal` is a
calibration; anything else is skipped with a warning.

The flatness is a hard constraint, not a preference. `StreamHub::LoadSourcesFile`
(`StreamHub.cpp:854`) is a hand-rolled scanner that takes each `{` up to the next
`}` as one object. A nested `"calibration": { ... }` inside a source entry would
truncate at the inner brace and corrupt the parse. Keeping every element flat lets
that scanner gain a single discriminator branch, and every config file written by
the current binaries still loads unchanged in both hubs.

## WebSocket protocol

New frames, implemented identically in both hubs:

| Direction | Frame |
|---|---|
| hub → client | `{"type":"calibration","cal":[{"source","signal","scale","offset","unit"}, …]}` |
| client → hub | `{"type":"setCalibration","source","signal","scale","offset","unit"}` |
| hub → client | `{"type":"configSaved","ok":bool,"path":string,"error":string}` |
| client → hub | `{"type":"reloadConfig"}` |
| hub → client | `{"type":"configReloaded","ok":bool,"path":string,"error":string}` |

`calibration` is broadcast when a client connects and after every accepted
`setCalibration` or successful `reloadConfig`. It is a separate message rather
than a field on the existing `sources` broadcast because the C++
`BroadcastSources` serialises into a fixed 4096-byte buffer that a calibration
table would overflow.

`setCalibration` is validated hub-side against the table above. An invalid entry
is rejected and no broadcast is emitted, so the offending client reverts to the
last broadcast value.

`saveSources` keeps its name and now writes both sources and calibration entries.
It gains the `configSaved` acknowledgement it currently lacks.

## Reload semantics

`reloadConfig` re-reads the config file, then:

- **replaces** the calibration table wholesale with the file's contents;
- **adds** any source present in the file that is not currently active;
- **never** removes, restarts, or reconnects a live source.

Reload must not interrupt streaming, so an unsaved source the user added stays
running. The asymmetry (calibration replaced, sources merged) is deliberate:
calibration is cheap to reapply, a source is a live UDP session.

## Hub implementation

**Go (`Common/Client/go/wshub/sources.go`, `hub.go`).** A
`map[string]calEntry` keyed by `source + "\x00" + signal`, owned by the
`SourceManager` behind its existing `sync.RWMutex`. `setCalibration` and
`reloadConfig` are dispatched from the `readPump` command switch in `hub.go`
alongside `addSource`/`removeSource`/`saveSources`/`zoom`. `SourceConfig` gains a
sibling `CalConfig` type; `Save` writes both slices into one array;
`Load` decodes into `[]map[string]json.RawMessage` and discriminates per element.

**C++ (`Source/Applications/StreamHub/StreamHub.{h,cpp}`).** A fixed
`kMaxCalibration = 256` array of `CalibrationEntry` structs with fixed
`char[]` fields (`source[128]`, `signal[128]`, `unit[17]`) and `float64` scale
and offset — no STL and no per-entry heap allocation, per the `Source/Components`
and StreamHub style rules. `HandleSetCalibration`,
`HandleReloadConfig` and `BroadcastCalibration` mirror the existing
`HandleAddSource` / `BroadcastSources` shape, with `BroadcastCalibration` using
its own 16 KiB buffer like `BroadcastConfig`. `LoadSourcesFile` gains the
discriminator branch; `HandleSaveSources` appends the calibration entries and
sends `configSaved`.

## SPA implementation

Calibration composes into the existing vertical-scale transform:

```
y_cal  = raw * scale + offset
y_norm = (y_cal - vsOffset) / divValue + screenPos
```

`applyVScaleNorm` (`app.js:186`) and its `applyDigitalNorm` / `applyMixedNorm`
siblings are the only places raw values enter the display path. Applying the
calibration there leaves the norm↔value inverse arithmetic untouched, so the
hover readout, cursors, rulers, Y-axis tick values and the V-Scale toolbar's own
V/div and Offset fields all report calibrated units without further change. V/div
and Offset are redefined as "calibrated units per division" and "calibrated value
at screen centre"; they remain a display concern, distinct from calibration.

Four sites need explicit handling because they bypass that path:

1. `resolveVScale` (`app.js:109`) `range` mode reads `meta.rangeMin` /
   `meta.rangeMax` from the streamer CONFIG. Both are calibrated and then
   re-ordered, since a negative `scale` swaps them.
2. `exportAllCSV` (`app.js:2871`) fetches full-resolution data from the ring or
   history and formats it directly. It calibrates each column and writes the
   effective unit into the header.
3. Trigger threshold. The hub compares against raw samples, so the SPA sends
   `(threshold - offset) / scale` and displays the inverse. V2 capture frames
   arrive raw and flow through the normal display path, so they need nothing.
4. Unit display: the sidebar `sig-unit` badge, the hover readout and the V-Scale
   header show the override when set, otherwise the streamer's `sig.unit`.

**Calibration UI.** A new row in `#vscale-menu` (`index.html:187`), the toolbar
already opened by clicking a signal in a plot:

```
Cal   Scale [ 1 ]   Offset [ 0 ]   Unit [ V ]   [Reset]
```

with a header line naming the base signal and element count it will affect. Edits
apply locally, send `setCalibration`, and are confirmed by the hub's broadcast.
`Reset` restores `scale=1, offset=0, unit=""` and sends that.

**Config UI.** The collapsible "Add Source" section (`app.js:3345`) is renamed
"Sources & Config" and keeps its address/label/multicast inputs and Connect
button. The existing fire-and-forget "Save list" button is replaced by **Save**
and **Reload**, plus a one-line status area rendering the `configSaved` /
`configReloaded` ack: the written path on success, the error text on failure.

**Fallback.** The SPA seeds its calibration table from
`localStorage['udpscope.calibration']` at page load, and overwrites it wholesale
the first time a `calibration` message arrives. The mirror is rewritten on every
change, including changes received from the hub. A hub binary that predates this
change never sends `calibration`, so the mirror simply remains authoritative;
the same holds for a hub started without a config file.

**Validation.** The same rules as the hub are enforced in the input handlers: a
non-finite or zero `scale` keeps the rejected text in the field and marks it
with a red `cal-invalid` border, so the user can see what was wrong.

## Testing

**Go.** Table tests for the heterogeneous-array parse (current-format file,
new-format file, unrecognised block, malformed entry), `setCalibration`
validation including `scale = 0` and non-finite values, and a save→load
round-trip asserting sources and calibration both survive.

**C++.** A standalone Go program at `Test/E2E/suite/client/configcheck/` connects
to either hub (Go or C++) via WebSocket and asserts identical calibration
behaviour: it exercises `setCalibration`, `saveSources → configSaved`,
`reloadConfig → configReloaded`, and verifies that the saved config file and all
broadcast frames are sorted and complete.  The program exits non-zero on any
deviation from the protocol, making it runnable against both hubs in CI.

**Browser.** `node --check static/app.js`, plus a manual pass: set a scale and
offset on a live signal and confirm the plot, hover readout, cursor readout, CSV
export and trigger threshold all agree; reload the page and confirm the
calibration returns; press Reload and confirm an unsaved edit is discarded while
the live source keeps streaming.

## Documentation

`Docs/StreamHub-API.md` gains the five new frames (`calibration`, `setCalibration`,
`configSaved`, `reloadConfig`, `configReloaded`); `Docs/WebUI.md` gains the
calibration row and the Sources & Config section; `ARCHITECTURE.md` §6 gains the
config file format.