# StreamHub — Developer Guide `Source/Applications/StreamHub/` is a headless C++ application (MARTe2-linked, MARTe2 coding style, no STL) that aggregates one or more UDPS sources (`UDPStreamer` DataSources) and serves them to oscilloscope clients over WebSocket. The Go hub (`Client/udpstreamer`) is the feature-complete reference implementation of the same protocol and is kept untouched. Wire protocols: [Protocol.md](Protocol.md) (UDPS, source → hub) and [StreamHub-API.md](StreamHub-API.md) (WebSocket, hub → clients). --- ## 1. Source layout | File | Role | |------|------| | `main.cpp` | CLI entry (`-cfg file.cfg -port N -maxPoints N`), signal handling | | `StreamHub.{h,cpp}` | Top-level object: config, push loop, WS command dispatch, broadcasts | | `UDPSourceSession.{h,cpp}` | One UDPS source: `UDPSClient` listener, payload decode, wall-clock calibration, ring buffers, stats | | `SignalRingBuffer.h` | Per-signal (t,v) ring: monotonic write counter, `ReadSince` cursor reads, binary-search `ReadRange` | | `TriggerEngine.{h,cpp}` | Trigger FSM (IDLE/ARMED/COLLECTING/TRIGGERED), edge detection per decoded sample | | `UDPSourceStats.h` | 512-entry cycle/frag/byte rings → avg/std/min/max, rate, 20-bin histogram | | `HistoryWriter.{h,cpp}` | Disk-backed circular history: per-signal `.shist` files, `WriteTick`, `ReadRange` (binary search via `pread`), disk space monitoring | | `WSServer.{h,cpp}` | RFC 6455 server: handshake, framing, per-client write mutex, broadcast/unicast | | `WSFrame.h`, `SHA1.h`, `Base64.h` | Header-only WS plumbing, shared with the ImGui client | | `LTTB.h` | Largest-Triangle-Three-Buckets decimation | The UDPS client itself lives in the shared library `Source/Components/Interfaces/UDPStream/` (`UDPSClient`, also used by `DebugService`). Note: in multicast mode the server delivers CONFIG over the **TCP control connection**, so `UDPSClient` selects on both the multicast UDP socket and the TCP socket and frames TCP reads. ## 2. Thread model | Thread | Created by | Work | |--------|-----------|------| | main / push loop | `StreamHub::Run()` | At `PushRate` Hz: `PushData()` (serialise v1 frames), trigger servicing (capture finalisation, auto-rearm), `PushStats()` at `StatsRate` Hz | | WS accept | `WSServer::Start()` | `accept()` + handshake, spawns client readers | | WS client reader ×16 | `WSServer` | Reads frames, may contain several coalesced frames per TCP read; dispatches JSON to `StreamHub::OnWSCommand(json, len, slotIdx)` | | UDPS receive ×32 | `UDPSClient::Start()` (one per session) | select() on UDP (+TCP in multicast), reassembles fragments, calls `UDPSourceSession` listener callbacks | Synchronisation: - Each `SignalRingBuffer` has its own `FastPollingMutexSem`; writers are the UDPS receive threads, readers are the push loop and zoom handlers. - `WSServer` has a per-client write mutex (push loop and command replies can write concurrently). - WS commands run on reader threads, but mutating operations (ring resize via `setMaxPoints`) are deferred to the push loop through pending atomics. - Session slots use an `active` flag; removal never compacts the array, so indices stay stable. ## 3. Time base (wall-clock calibration) All timestamps exposed to clients are **Unix wall-clock seconds** (float64). Each session calibrates per time-source: - First DATA packet anchors `pktCalibOffset = wallNow − hrt/hrtFreq`; thereafter `packetT = pktCalibOffset + hrt/hrtFreq`. - Each referenced time signal gets its own offset on first value; `timerToSec = 1e-9` for `uint64` time signals, `1e-6` otherwise. - The time-signal offset is **snapped** only on a genuine discontinuity in the source: reconnect, CONFIG change, or the source clock jumping backward (a looping/rewinding producer such as a rewinding `FileReader`). - Plain *drift* — a source free-running on its own clock, or remote-vs-local HRT frequency error — is **slewed**, not snapped. Past a 2 s threshold the offset is nudged toward wall clock by at most 10 % of the packet's own duration. Snapping instead would shift the whole published timeline in one step and so tear a hole of exactly the drift into a stream that is in fact continuous; a source drifting past the threshold repeatedly used to produce a train of 2 s holes. Drift is the honest reading, and the trade-off `TimeArrayGAM`'s `Anchor = Continuous` explicitly asks for: a producer that cannot sustain its nominal sample rate will fall progressively behind wall clock, and the hub reports that rather than hiding it. The Go hub anchors once and never re-anchors, so it never had the hole. Per `timeMode`: | timeMode | Timestamping | |----------|-------------| | FIRST/LAST_SAMPLE | anchor = calibrated time-signal value (fallback `packetT`); `t = anchor ± k·dt` from `samplingRate` | | FULL_ARRAY | `t[k]` = calibrated time array element | | PACKET, n=1 | `packetT` | | PACKET, n>1 | elements span `(lastPktWall, wallNow]` — backward anchoring, deliberately different from the Go hub (which extrapolates forward and overlaps the next packet under jitter); keeps ring time strictly monotonic | Multi-element PACKET signals are exposed per element as `name[i]`. ## 4. Push path (only-new samples) `SignalRingBuffer` keeps a monotonic `totalWritten`; the hub keeps a per-(session, signal) cursor and uses `ReadSince(cursor, …)` (clamped to the oldest sample on overrun). Each tick serialises only new samples, LTTB-capped to `MaxPushPoints` (default 50) per signal — LTTB is applied only to temporal signals (multi-element, timeMode ≠ PACKET). Cursors advance even with zero WS clients so a connecting client never receives a backlog burst. Cursors are reset on (re)CONFIG and on ring resize. This replaces the original "re-send the last N points each tick" design, which caused visible trace corruption (LTTB picked different points per overlapping window). ## 5. Trigger engine Hub-side, web-client semantics (`setTrigger` fields in [StreamHub-API.md](StreamHub-API.md)): ``` IDLE --arm--> ARMED --edge crossing--> COLLECTING --every source past trigTime+postSec+0.15s--> TRIGGERED TRIGGERED --rearm (single) / auto ~200ms (normal, unless stopped)--> ARMED any --disarm--> IDLE ``` `UDPSourceSession` calls `TriggerEngine::CheckSample` for every decoded sample of the configured signal (signal index cached per config epoch). Each source is read `[trigTime−preSec, trigTime+postSec]`, LTTB-capped to 20 000 pts/signal and appended to a binary **version 2** capture frame; every FSM transition broadcasts a `triggerState` event. Once fired, that event carries `trigTime` **and** the window latched at fire time (`preSec`/`postSec`). Clients draw the still-filling capture from their own buffers on that axis long before the v2 frame arrives — for a long window at a high rate the hub stays silent for seconds — and the trigger bar's window and pre-% are editable, so without the latched values a client would place the filling trace on whatever window the operator happened to be typing. Older hubs omit both fields; clients fall back to their local config. The COLLECTING deadline is on the **data's** clock, via `UDPSourceSession::ProducerNewestTime()` — `trigTime` comes from sample timestamps, and a source free-running on its own clock sits seconds away from `clock_gettime()`, so a wall-clock deadline chops exactly that offset off every capture's tail. Only signals actually timestamped from a time signal count toward that reading: PACKET-timed ones (including the time array itself) are stamped on arrival and would just report "now". Sources are harvested independently — `BeginTriggerCapture`, `HarvestTriggerCapture` per source as *it* becomes ready, `FinishTriggerCapture` once all are in — with the frame accumulating in `capBuf_` across push ticks. Waiting for the slowest source before reading any of them lets the leaders' rings roll past the pre-trigger region first, losing the head of their traces. A 2 s wall-clock watchdog bounds the wait for a source that stopped advancing: it is harvested short, with a warning naming the source and how far it got. `setTrigger` also records the requested window, and each stats tick the push loop runs `GrowRingsForTrigger()`. A ring whose measured rate (`Count() / TimeSpan()`, since UDPS sources usually advertise `samplingRate = 0`) cannot hold `window + 0.5 s` is grown in place to `rate × (window + 0.5) × 1.2` points, clamped to `RingMaxMB` per signal. `SignalRingBuffer::Grow()` copies oldest→newest and leaves `count` / `totalWritten` untouched so the per-client push cursors survive the resize. Rings never shrink; a hub left with a 5 s window on a 5 MSps source will sit at the ceiling. ## 6. Configuration ``` WSPort = 8090 MaxPoints = 20000 // legacy global cap (overridable with -maxPoints) PushRate = 30 // Hz MaxPushPoints = 50 // per signal per push StatsRate = 1 // Hz RingTemporal = 1000000 // initial ring capacity, temporal signals (pts) RingScalar = 100000 // ring capacity, scalar/PACKET signals (pts) RingMaxMB = 128 // per-signal growth ceiling (MiB) for trigger windows SourcesFile = "streamhub_sources.json" // saveSources persistence AllowedOrigins = "http://127.0.0.1:8099,http://localhost:8099" // comma/space-separated WebSocket Origin allowlist (max 8 × 128 chars). // Without it the handshake only accepts an Origin whose host matches // the request Host, so a browser serving the SPA from another port // (run_streamhub.sh: SPA 8099, hub 8090) gets 403. Non-browser // clients send no Origin and are unaffected. Sources = { Src1 = { Label = "PSU" Addr = "127.0.0.1" Port = 44500 MulticastGroup = "239.0.0.1" DataPort = 44503 } // multicast optional } ``` Sources from `SourcesFile` are loaded after the static `Sources` block; sources added at runtime via WS `addSource` get ids `s1, s2, …`. ### History configuration An optional `+History` block enables disk-backed circular storage (`HistoryWriter`). The `+` prefix is MARTe2 `StandardParser` syntax for a child node; the hub looks for both `+History` and `History` as the node name. ``` +History = { Directory = "/data/streamhub_history" // required DurationHours = 1 // hours of data to retain per signal (default 1) Decimation = 10 // keep every Nth sample (default 1) FlushIntervalSec = 5 // header flush period in seconds (default 5) MinDiskFreeMB = 500 // pause writes below this threshold (default 500) } ``` Per-signal file capacity is computed at source CONFIG time: `capacity = ceil(DurationHours × 3600 × samplingRate / Decimation)`, minimum 1000 pairs. The Go hub (`Client/udpstreamer`) carries the same archive and the same file format, configured with flags instead of a config node: `-history-dir` (defaults to `/udpstreamer-history`; empty disables), `-history-window-sec`, `-history-decimation`, `-history-flush-sec`, `-history-min-free-mb` (negative disables the check; 0 means the 500 MB default, where the C++ `MinDiskFreeMB = 0` disables it) and `-history-max-mpts`, a per-signal budget in millions of stored points, defaulting to 16 MPts (256 MB). The budget exists because the timespan alone cannot bound the file: 600 s of a 1 MSps signal is 9.6 GB. **The Go hub sizes its files from the window, not from a retention period.** The archive exists to answer a zoom or a trigger capture after the in-memory rings have rolled past it, and neither ever asks for more than the live or trigger window — so a file holds `windowSec × rate` samples (plus 25 % headroom, since a capture is read back a window after its first sample was written), and never hours of them. Retaining an hour instead meant a 1 s live window was archived at a thousandth of the resolution the same budget could have bought. The budget is therefore spent on resolution, not on span. A signal too fast to archive sample-for-sample within it is stored as a **min/max envelope**: `bucket` source samples collapse to their two extremes, with `bucket` the narrowest that makes the window fit. The `.shist` header's `decimation` field carries `bucket × Decimation`, so a reader knows the stored resolution, and a file is only reopened when it matches. `Hub.retuneRings` re-sizes the files once a second alongside the rings, from the same `activeWindowSec()`. A file's capacity and bucket are fixed at creation, so a re-size discards what it held; two rules keep that rare. A file is only grown when it no longer covers the window, and only shrunk when it is enveloped (`bucket > 1`), covers more than twice the window, and a narrower bucket is actually available — a file already at full resolution is left alone however short the window becomes, so arming a 1 s trigger does not throw away the seconds the capture is about to ask for. `historyInfo` is re-broadcast whenever a re-size happens. The budget is also settable at runtime from the web UI (the history badge in the status bar) via the `setHistoryBudget` WS command; `historyInfo` reports it as `maxMPtsPerSignal` and reports each signal's `bucket`. Changing it re-creates the files, so the archived samples are lost — a file's capacity and bucket width are fixed at creation and an existing envelope cannot be re-bucketed into a different one. History is on by default in the Go hub because it is what holds a trigger capture at full resolution — see *Trigger captures* below. Signals whose producer declares `samplingRate = 0` — every UDPS source — are not sized from a guess: the file is opened only once the hub has measured the rate off the live stream, which it retries once a second. ### `.shist` binary file format Each signal gets one file: `//.shist`. The file size is fixed at creation (`64 + capacity × 16` bytes) and never grows. ``` Offset Size Field 0 4 Magic: "SHR1" 4 4 uint32 version (1) 8 4 uint32 capacity (max pairs) 12 4 uint32 head (next write position, 0-based, wraps at capacity) 16 4 uint32 count (valid entries, ≤ capacity) 20 4 uint32 decimation 24 8 float64 tOldest (Unix seconds) 32 8 float64 tNewest (Unix seconds) 40 24 reserved (zero-padded to 64 bytes) 64 … data: capacity × 16 bytes (float64 time + float64 value per pair) ``` Data is written at the `head` position and wraps circularly. The oldest valid entry is at logical index `(head + capacity − count) % capacity`. All I/O uses `pwrite`/`pread` (no `mmap`), so concurrent reads from WS threads are safe without locking. Headers are flushed to disk every `FlushIntervalSec` seconds and on shutdown. On restart, if an existing file has matching magic, version and capacity, it is reopened — head/count/time bounds are restored from the on-disk header. ### History query path `historyZoom` requests (see [StreamHub-API.md](StreamHub-API.md)) call `HistoryWriter::ReadRange` which performs binary search over the circular file using `pread` to locate the `[t0, t1]` window, then copies matching pairs. If the result exceeds the requested `n`, decimation is applied (same decimator as in-memory zoom: `LTTBDecimate` in the C++ hub, `minMaxDecimate` in the Go one). Both the web SPA and ImGui client issue `historyZoom` in parallel with regular `zoom` and merge the results: history covers the older part of the visible window, the in-memory ring covers the recent part. A range wider than the read budget is thinned across its whole width with a stride, not truncated at the front: answering a 10 s query with its first few milliseconds reads as an empty plot to a client and sends it back to its own coarse copy of the data. ### In-memory buffer policy (Go hub) Each temporal signal gets one ring holding a fixed **budget** of `(t, v)` pairs: 10 M points, 160 MB, settable with `-ring-mpts`. Scalar signals keep a flat 100 000-packet ring, where a megasample budget would be waste. Rings start at 250 k points and are grown to the budget on demand, so a source that is configured but never sends costs nothing. Like the disk archive, the budget buys **resolution, not span**. Once a second `retuneRings` compares the measured source rate against the window being displayed and picks each ring's min/max `bucket`: | condition | bucket | effect | |---|---|---| | `Sps × window ≤ budget` | 1 | stored verbatim; the ring reaches further back than the window, which is free zoom headroom | | `Sps × window > budget` | `⌈2 × Sps × window × 1.25 ÷ capacity⌉` | `bucket` samples collapse to their two extremes, so the whole window fits | A bucket costs two points (its minimum and its maximum), hence the factor 2 — and why a bucket of 2 covers no more ground than a bucket of 1. The window is the **trigger's** while a trigger is armed: its pre-window has to already be in the ring when the trigger fires, or the capture has nothing to back-fill from. Otherwise it is the widest window any connected client has reported with the `setWindow` command, defaulting to 10 s for clients that never send one. Sizing for the live window matters as much as for a capture: a fixed sample-count ring covers ~6 s at 1 MSps, so a zoom on a 60 s timescale used to come back with only its tail. Retuning is hysteretic — a bucket is held while it covers the window without covering more than twice it. Sharing one threshold for up and down makes a rate jittering across a bucket boundary halve and double the stored resolution every second. Live pushes, the disk archive and the trigger comparator all see every sample: `ingest` hands the raw batch to each, and only the ring's own copy is reduced. ### Trigger captures (Go hub) A trigger capture is delivered as a decimated snapshot (20 000 points), so a zoom into it has to come from full-resolution storage. The rings are tuned to ~1.25× the trigger window, so they roll past a captured window shortly after the capture — and the trigger rearms and starts refilling them immediately. **In-memory double buffer.** The rings are the write half; `captureHold` (`capturehold.go`) is the read half. As `buildTriggerCapture` lifts each signal's window out of its ring it publishes the *undecimated* slice into the hold, and `zoomSlice` answers from the hold rather than the ring for any range the held window fully contains. The swap happens only once the next capture is complete — which is also the moment the client stops displaying the previous one — so the shot being explored is never overwritten by the acquisition running behind it. A capture that came back empty does not swap, so it cannot blank the window on screen. **Waiting for the buffer.** An armed trigger ignores edges until its buffers reach back far enough for a capture taken now to come back whole (`fillLocked` in `trigger.go`, fed by `refreshTriggerFill` from the trigger signal's own ring — once per tick, and again on every trigger command so that an `arm` cannot fire on a stale measurement). Firing earlier can only produce a capture whose front was never recorded, which is what made the first shot after a widened window come back short. What must hold is that the buffer spans the whole window *at harvest time* — its newest sample is then `trigTime + post`, so anything less has lost the front of the capture. It keeps filling while the post-window is collected, so the shortfall it may start with is what it will make up in that time, measured rather than assumed: ``` need = windowSec − growth × postSec (floored at the pre-trigger window) ``` `growth` is the ring's span growth in seconds per second, sampled over at least `bufGrowthIntervalSec` and smoothed. The three regimes fall out of the one formula: | ring | growth | needs | |---|---|---| | still filling | 1 | the pre-trigger window — everything after the trigger is yet to be recorded anyway | | full, re-bucketing for a longer window | 0…1 | in between: it drops dense old samples to take sparse new ones, so it fills slower than real time and the front of the capture recedes while the post-window elapses | | full, settled | 0 | the whole window — which a ring tuned for that window already exceeds, so nothing actually waits | Measured at 1 MSps, widening 10 s → 30 s with 50 % pre: growth settles at ~0.65, so `need` converges on ~20.4 s of the 30 s and the trigger fires ~12 s after arming with a capture that is 100 % complete. Requiring the whole window instead would have waited 26 s for the same result. The gate measures the trigger signal's ring, not the narrowest of all of them: a signal that never reaches back that far would otherwise stop the trigger from ever firing. It is disabled outright when there is no ring to measure or nothing is needed, and `forceTrigger` overrides it. While it holds off, `triggerState` carries `bufferFill`/`bufferNeedSec` and is re-broadcast as the fraction climbs, so the UI shows `ARMED 42%` rather than a trigger that looks stuck. **Back-filling a short capture.** A ring only spans the window once it has rolled over completely at its current min/max bucket, which takes as long as the window itself; widen the window, or arm right after setting it, and the first captures start late and the client draws a blank front half. `backfillCaptureHead` (`trigger.go`) therefore prepends whatever of `[t0, ring's first sample)` the archive still holds, budgeting the read by the share of the window being filled and trimming the overlap so the frame's timestamps stay ascending. It needs history enabled; without it the capture is simply short, and the hub logs by how much. The hold declines any range its own samples do not actually cover, so a stretch neither source could supply falls through to the archive instead of being redrawn as the same hole on every zoom and every *fit*. The hold declines ranges reaching outside its window: those are live zooms, and only the rings still track the stream. Inside the window it needs no trigger-state gating, because retuning never rewrites stored samples — a ring that still covers the range holds the very same points. It is cleared when `updateConfig` rebuilds the rings, since a restarted producer can replay the same timestamps. Budget: the hold costs one window per signal on top of the ring budget, up to a further ~0.8 × `-ring-mpts`. Nothing is held until the first capture fires. **On disk.** The archive covers what the hold cannot: ranges wider than the capture window, and sessions where the hub restarted. It is circular and sized from that same window, so it too wraps over a captured shot within a window of delivering it. When a capture is delivered, the hub therefore copies `[trigTime − pre, trigTime + post]` out of each `.shist` into a `.cap` file, laid out as a full non-wrapping `.shist` (`capacity == count`, `head == 0`) so the same `readRange` reads it. A `historyZoom` whose range the capture file fully contains is answered from it; anything wider is answered from the archive. The copy is replaced by the next trigger and by nothing else — rearming keeps it, because the client is still showing that capture. Budget: a capture costs one window's worth of disk per signal on top of `-history-max-mpts`. Protecting the window in place instead — pinning the region and refusing to wrap onto it — does not work, and was tried: a capture held for longer than the archive covers stops the archive dead, and the resulting hole lands exactly where the *next* capture's pre-trigger window belongs. ## 7. Build & test ```bash source env.sh # always, for build and run make -f Makefile.gcc apps # builds StreamHub.ex # or: make -C Source/Applications/StreamHub -f Makefile.gcc ./Build/x86-linux/StreamHub/StreamHub.ex -cfg hub.cfg make -f Makefile.gcc test ./Build/x86-linux/GTest/MainGTest.ex # 71 unit tests, incl. # SignalRingBuffer (ReadSince / binary-search ReadRange / wrap), # TriggerEngine FSM, LTTB — sources in Test/Applications/StreamHub/ cd Test/E2E/suite && ./run_e2e.sh # full-stack E2E (see below) ./run_streamhub.sh -w -g # interactive demo stack ``` ### End-to-end test `Test/E2E/suite/run_e2e.sh` is the unified E2E suite covering the whole streaming + debug chain (`chain`/`direct`/`recorder`/`debug`/`tcplogger` scenario kinds, see `Test/E2E/suite/scenarios.py`), including StreamHub live push, zoom, window and trigger checks via the Go `chain-client`. It builds everything, runs the scenario matrix plus the stress matrix, and produces a consolidated `report_data.json` + Typst PDF report (`Test/E2E/suite/E2E_Report.typ`). See the script's `--help` for options. When changing the WS protocol, update **in lockstep**: this hub, the Go hub (`Common/Client/go/wshub`), the browser SPA (`Client/udpstreamer/static`), the ImGui client (`Client/streamhub/Protocol.cpp`), the E2E `chain-client` (`Test/E2E/suite/client`), and [StreamHub-API.md](StreamHub-API.md). ## 8. Gotchas - **No STL** in this directory (MARTe2 style); use `StreamString`, `FastPollingMutexSem`, fixed arrays. - Link against UDPStream **dynamically** (`LIBRARIES += -lUDPStream`), never `LIBRARIES_STATIC` — the GTest binary links every `.a` it finds and would get duplicate symbols. - WS text frames may arrive coalesced; never NUL-terminate a payload in place without restoring the byte (it is the first header byte of the next frame). - Zoom replies must print `t` with `%.17g`: at Unix-epoch magnitudes `%.6f` destroys µs resolution.