MARTe-Integrated-Components/docs/superpowers/specs/2026-08-16-udpstreamer-signal-calibration-and-config-design.md

Per-signal calibration and persistent hub config

Date: 2026-08-16 Scope: Client/udpstreamer/static/, Common/Client/go/wshub/, Source/Applications/StreamHub/

Problem

The web oscilloscope plots whatever the streamer sends. A signal carrying raw ADC counts cannot be read in volts, and the vertical-scale toolbar's V/div and Offset are display-only: they move the trace on screen but do not change what the cursor, hover readout or CSV export report.

Separately, the only persistent state is the source list, saved server-side by the saveSources WebSocket command. That command is fire-and-forget — the browser never learns whether the write succeeded — and there is no way to re-read the file without restarting the hub.

Goals

1. A per-signal affine calibration y = raw * scale + offset, with an optional unit override, applied consistently everywhere a value is shown.
2. Calibration stored in the hub's config file alongside the sources, so it survives a browser reload and is shared between browsers.
3. A left-sidebar section to save and reload that config, with success/error feedback.
4. Identical behaviour from the Go hub (Common/Client/go/wshub) and the C++ StreamHub, per the protocol-parity rule in CLAUDE.md.

Non-goals

• Calibrating on the data path. Raw samples stay raw in the rings, in recorded history, and in the trigger comparator.
• Per-array-element calibration. One entry covers a whole array signal.
• Named profiles. One config file, the existing -sources-file / SourcesFile.
• Persisting display state (layout, trace colours, V/div, window, trigger config).

Data model

A calibration entry is keyed by (source label, signal base name):

Field	Type	Default	Validation
source	string	—	must be non-empty
signal	string	—	base signal name, no [i] suffix
scale	float64	1	finite, non-zero
offset	float64	0	finite
unit	string	""	trimmed, max 16 chars; empty means "use the streamer's unit"

The key uses the source label, not the runtime id (s1, s2). Ids are assigned in add-order at startup, so a saved calibration keyed by id would rebind to a different source whenever the source list order changed. Labels default to the address when the user leaves the label blank, which keeps them unique in practice; two sources sharing a label share a calibration, which is a documented consequence rather than an error.

Array signals get one entry covering every element. The V-Scale toolbar can be opened on a single element (Adc[3]), so its calibration row states which base signal and how many elements the edit affects.

Config file format

The file remains a flat JSON array of flat objects. Sources keep their current shape; calibration entries are appended as additional elements:

[
  {"label": "wave", "addr": "127.0.0.1:44500"},
  {"label": "mc",   "addr": "127.0.0.1:44501", "multicastGroup": "239.0.0.1", "dataPort": 44502},
  {"source": "wave", "signal": "Adc", "scale": 0.00030518, "offset": -1.25, "unit": "V"}
]

A block containing addr is a source; a block containing signal is a calibration; anything else is skipped with a warning.

The flatness is a hard constraint, not a preference. StreamHub::LoadSourcesFile (StreamHub.cpp:854) is a hand-rolled scanner that takes each { up to the next } as one object. A nested "calibration": { ... } inside a source entry would truncate at the inner brace and corrupt the parse. Keeping every element flat lets that scanner gain a single discriminator branch, and every config file written by the current binaries still loads unchanged in both hubs.

WebSocket protocol

New frames, implemented identically in both hubs:

Direction	Frame
hub → client	{"type":"calibration","cal":[{"source","signal","scale","offset","unit"}, …]}
client → hub	{"type":"setCalibration","source","signal","scale","offset","unit"}
hub → client	{"type":"configSaved","ok":bool,"path":string,"error":string}
client → hub	{"type":"reloadConfig"}
hub → client	{"type":"configReloaded","ok":bool,"error":string}

calibration is broadcast when a client connects and after every accepted setCalibration or successful reloadConfig. It is a separate message rather than a field on the existing sources broadcast because the C++ BroadcastSources serialises into a fixed 4096-byte buffer that a calibration table would overflow.

setCalibration is validated hub-side against the table above. An invalid entry is rejected and no broadcast is emitted, so the offending client reverts to the last broadcast value.

saveSources keeps its name and now writes both sources and calibration entries. It gains the configSaved acknowledgement it currently lacks.

Reload semantics

reloadConfig re-reads the config file, then:

• replaces the calibration table wholesale with the file's contents;
• adds any source present in the file that is not currently active;
• never removes, restarts, or reconnects a live source.

Reload must not interrupt streaming, so an unsaved source the user added stays running. The asymmetry (calibration replaced, sources merged) is deliberate: calibration is cheap to reapply, a source is a live UDP session.

Hub implementation

Go (Common/Client/go/wshub/sources.go, hub.go). A map[string]calEntry keyed by source + "\x00" + signal, owned by the SourceManager behind its existing sync.RWMutex. setCalibration and reloadConfig are dispatched from the readPump command switch in hub.go alongside addSource/removeSource/saveSources/zoom. SourceConfig gains a sibling CalConfig type; Save writes both slices into one array; Load decodes into []map[string]json.RawMessage and discriminates per element.

C++ (Source/Applications/StreamHub/StreamHub.{h,cpp}). A fixed kMaxCalibration = 256 array of {StreamString source, signal, unit; float64 scale, offset;} — no STL, per the Source/Components and StreamHub style rules. HandleSetCalibration, HandleReloadConfig and BroadcastCalibration mirror the existing HandleAddSource / BroadcastSources shape, with BroadcastCalibration using its own 16 KiB buffer like BroadcastConfig. LoadSourcesFile gains the discriminator branch; HandleSaveSources appends the calibration entries and sends configSaved.

SPA implementation

Calibration composes into the existing vertical-scale transform:

y_cal  = raw * scale + offset
y_norm = (y_cal - vsOffset) / divValue + screenPos

applyVScaleNorm (app.js:186) and its applyDigitalNorm / applyMixedNorm siblings are the only places raw values enter the display path. Applying the calibration there leaves the norm↔value inverse arithmetic untouched, so the hover readout, cursors, rulers, Y-axis tick values and the V-Scale toolbar's own V/div and Offset fields all report calibrated units without further change. V/div and Offset are redefined as "calibrated units per division" and "calibrated value at screen centre"; they remain a display concern, distinct from calibration.

Four sites need explicit handling because they bypass that path:

1. resolveVScale (app.js:109) range mode reads meta.rangeMin / meta.rangeMax from the streamer CONFIG. Both are calibrated and then re-ordered, since a negative scale swaps them.
2. exportAllCSV (app.js:2871) fetches full-resolution data from the ring or history and formats it directly. It calibrates each column and writes the effective unit into the header.
3. Trigger threshold. The hub compares against raw samples, so the SPA sends (threshold - offset) / scale and displays the inverse. V2 capture frames arrive raw and flow through the normal display path, so they need nothing.
4. Unit display: the sidebar sig-unit badge, the hover readout and the V-Scale header show the override when set, otherwise the streamer's sig.unit.

Calibration UI. A new row in #vscale-menu (index.html:187), the toolbar already opened by clicking a signal in a plot:

Cal   Scale [ 1 ]   Offset [ 0 ]   Unit [ V ]   [Reset]

with a header line naming the base signal and element count it will affect. Edits apply locally, send setCalibration, and are confirmed by the hub's broadcast. Reset restores scale=1, offset=0, unit="" and sends that.

Config UI. The collapsible "Add Source" section (app.js:3345) is renamed "Sources & Config" and keeps its address/label/multicast inputs and Connect button. The existing fire-and-forget "Save list" button is replaced by Save and Reload, plus a one-line status area rendering the configSaved / configReloaded ack: the written path on success, the error text on failure.

Fallback. The SPA seeds its calibration table from localStorage['udpscope.calibration'] at page load, and overwrites it wholesale the first time a calibration message arrives. The mirror is rewritten on every change, including changes received from the hub. A hub binary that predates this change never sends calibration, so the mirror simply remains authoritative; the same holds for a hub started without a config file.

Validation. The same rules as the hub are enforced in the input handlers: a non-finite or zero scale keeps the rejected text in the field and marks it with a red cal-invalid border, so the user can see what was wrong.

Testing

Go. Table tests for the heterogeneous-array parse (current-format file, new-format file, unrecognised block, malformed entry), setCalibration validation including scale = 0 and non-finite values, and a save→load round-trip asserting sources and calibration both survive.

C++. Extend an E2E chain scenario's config file with a calibration entry and assert the hub re-serialises it unchanged after a saveSources, which exercises the discriminator branch in LoadSourcesFile and the writer in HandleSaveSources.

Browser. node --check static/app.js, plus a manual pass: set a scale and offset on a live signal and confirm the plot, hover readout, cursor readout, CSV export and trigger threshold all agree; reload the page and confirm the calibration returns; press Reload and confirm an unsaved edit is discarded while the live source keeps streaming.

Documentation

Docs/StreamHub-API.md gains the four new frames; Docs/WebUI.md gains the calibration row and the Sources & Config section; ARCHITECTURE.md §6 gains the config file format.