Implemented new C++ logic

This commit is contained in:
Martino Ferrari
2026-06-12 15:25:13 +02:00
parent 617b5bd712
commit f25bd7f08e
220 changed files with 39185 additions and 850 deletions
+231
View File
@@ -0,0 +1,231 @@
# StreamHub WebSocket API
StreamHub exposes a single WebSocket server (default port **8090**, any URL path).
Text frames carry JSON commands/events; binary frames carry data pushes and
trigger captures. The Go reference hub (`Client/udpstreamer`) implements the
identical protocol; both the browser SPA and the ImGui client work against either.
For the upstream UDPS wire format (source → hub) see [Protocol.md](Protocol.md).
All numbers in binary frames are **little-endian**. All timestamps are
**Unix wall-clock seconds** (float64) — see *Time base* in
[StreamHub-Developer.md](StreamHub-Developer.md).
---
## 1. Commands (client → hub, JSON text frames)
Every command is a JSON object with a `type` field.
### `ping`
```json
{"type":"ping"}
```
Reply (unicast): `{"type":"pong"}`.
### `addSource`
```json
{"type":"addSource","label":"PSU","addr":"192.168.0.10:44500",
"multicastGroup":"239.0.0.1","dataPort":44503}
```
- `addr``host:port` of the UDPStreamer control port.
- `multicastGroup` / `dataPort` — optional; if present the hub joins the
multicast group for DATA and keeps a TCP control connection for CONNECT/CONFIG.
- The hub assigns ids `s1`, `s2`, … and broadcasts an updated `sources` event.
### `removeSource`
```json
{"type":"removeSource","id":"s1"}
```
### `saveSources`
```json
{"type":"saveSources"}
```
Persists the current dynamically-added source list to the hub's `SourcesFile`
(JSON array of `{label,addr,multicastGroup,dataPort}`); it is reloaded at startup.
### `getSources` / `getConfig` / `getStats`
```json
{"type":"getSources"}
{"type":"getConfig","sourceId":"s1"}
{"type":"getStats"}
```
Force a broadcast of the corresponding event.
### `setTrigger`
```json
{"type":"setTrigger","signal":"s1:Sine","edge":"rising","threshold":0.0,
"windowSec":0.1,"prePercent":20,"mode":"normal"}
```
- `signal` — full key `src:sig`, or `src:sig[i]` to trigger on element *i* of a
multi-element PACKET signal.
- `edge``"rising"`, `"falling"` or `"both"`.
- `windowSec` — total capture window (clamped to 1e-4 … 10 s).
`preSec = windowSec * prePercent / 100`, `postSec = windowSec preSec`.
- `mode``"normal"` (auto-rearm ~200 ms after capture) or `"single"`
(stays TRIGGERED until `rearm`).
### `arm` / `disarm` / `rearm` / `trigStop`
```json
{"type":"arm"}
{"type":"disarm"}
{"type":"rearm"}
{"type":"trigStop","stopped":true}
```
- `arm` — IDLE → ARMED.
- `disarm` — any state → IDLE.
- `rearm` — TRIGGERED → ARMED (single mode).
- `trigStop` — sets the *stopped* flag suppressing auto-rearm in normal mode;
omit `stopped` to toggle.
Every transition is broadcast as a `triggerState` event.
### `zoom`
```json
{"type":"zoom","reqId":17,"t0":1765370000.123,"t1":1765370000.223,
"n":2400,"signals":"s1:Sine,s2:Wave[0]"}
```
- `reqId` — echoed in the reply; lets the client match request/response.
- `t0`/`t1` — Unix seconds window.
- `n` — target points per signal. Absent or `<10` → 2400; `n ≤ 0` → **no
decimation** (raw ring contents).
- `signals` — comma-separated full keys; absent → all signals of all sources.
- The reply is **unicast** to the requesting client only.
### `setMaxPoints`
```json
{"type":"setMaxPoints","maxPoints":50000}
```
Resizes all ring buffers (applied safely inside the push loop; push cursors are
reset). Broadcasts `maxPointsUpdated`.
---
## 2. Events (hub → client, JSON text frames)
### `sources`
```json
{"type":"sources","sources":[
{"id":"s1","label":"PSU","addr":"192.168.0.10:44500","state":"streaming"}]}
```
Sent on client connect, after add/remove/getSources, and when a source first
delivers its CONFIG.
### `config`
```json
{"type":"config","sourceId":"s1","publishMode":0,"signals":[
{"name":"Sine","typeCode":10,"quantType":0,"numDimensions":0,
"numRows":1,"numCols":1,"rangeMin":-1.0,"rangeMax":1.0,
"timeMode":0,"samplingRate":5000000.0,"timeSignalIdx":-1,"unit":"V"}]}
```
Field semantics follow the UDPS signal descriptor ([Protocol.md](Protocol.md)).
`numElements = numRows × numCols`.
### `stats`
Sent at `StatsRate` Hz (default 1 Hz):
```json
{"type":"stats","sources":{"s1":{
"state":"streaming","totalReceived":1234,"totalLost":0,
"rateHz":100.1,"rateStdHz":0.3,
"fragsPerCycle":3.0,"bytesPerCycle":4200.0,
"cycleAvgMs":10.0,"cycleStdMs":0.1,"cycleMinMs":9.8,"cycleMaxMs":10.4,
"cycleHistMin":9.8,"cycleHistMax":10.4,"cycleHist":[0,1,5,"…(20 bins)"]}}}
```
### `triggerState`
```json
{"type":"triggerState","state":"triggered","mode":"normal",
"stopped":false,"trigTime":1765370000.1234567}
```
`state``idle | armed | collecting | triggered`; `trigTime` present once a
trigger has fired.
### `zoom` (reply)
```json
{"type":"zoom","reqId":17,"signals":{
"s1:Sine":{"t":[1765370000.1230000,"…"],"v":[0.123456789,"…"]}}}
```
`t` is serialised with `%.17g` (full float64 precision — required for
µs windows at Unix-epoch magnitudes), `v` with `%.9g`.
### `maxPointsUpdated`
```json
{"type":"maxPointsUpdated","maxPoints":50000}
```
---
## 3. Binary frames (hub → client)
The first byte of every binary WS frame is a **version** discriminator.
### Version 1 — data push
Sent at `PushRate` Hz per source. Contains **only the samples that are new
since the previous push** (per-signal cursors hub-side), LTTB-decimated to at
most `MaxPushPoints` (default 50) per signal.
```
[1] version = 1
[1] sourceIdLen (L)
[L] sourceId (UTF-8, no NUL)
[4] numSignals (uint32)
per signal:
[2] keyLen (uint16) (K)
[K] key = signal name; "name[i]" per element for multi-element PACKET signals
[4] pairCount (uint32) (N)
[N×8] t (float64, Unix seconds)
[N×8] v (float64, physical units)
```
Clients must append samples verbatim — there is no overlap between pushes.
### Version 2 — trigger capture
Broadcast once per capture (FSM COLLECTING → TRIGGERED). Contains *all*
signals over `[trigTime preSec, trigTime + postSec]`, each LTTB-decimated
to ≤ 20 000 points.
```
[1] version = 2
[8] trigTime (float64, Unix seconds)
[8] preSec (float64)
[8] postSec (float64)
[4] numSignals (uint32)
per signal:
[2] keyLen (uint16) (K)
[K] fullKey = "src:sig" (UTF-8)
[4] pairCount (uint32) (N)
[N×8] t (float64, Unix seconds)
[N×8] v (float64)
```
---
## 4. Limits
| Limit | Value |
|-------|-------|
| Concurrent WS clients | 16 |
| UDPS source sessions | 32 |
| Max received WS payload | 64 KiB |
| Max sent WS payload | 4 MiB |
+172
View File
@@ -0,0 +1,172 @@
# StreamHub — Developer Guide
`Source/Applications/StreamHub/` is a headless C++ application (MARTe2-linked,
MARTe2 coding style, no STL) that aggregates one or more UDPS sources
(`UDPStreamer` DataSources) and serves them to oscilloscope clients over
WebSocket. The Go hub (`Client/udpstreamer`) is the feature-complete reference
implementation of the same protocol and is kept untouched.
Wire protocols: [Protocol.md](Protocol.md) (UDPS, source → hub) and
[StreamHub-API.md](StreamHub-API.md) (WebSocket, hub → clients).
---
## 1. Source layout
| File | Role |
|------|------|
| `main.cpp` | CLI entry (`-cfg file.cfg -port N -maxPoints N`), signal handling |
| `StreamHub.{h,cpp}` | Top-level object: config, push loop, WS command dispatch, broadcasts |
| `UDPSourceSession.{h,cpp}` | One UDPS source: `UDPSClient` listener, payload decode, wall-clock calibration, ring buffers, stats |
| `SignalRingBuffer.h` | Per-signal (t,v) ring: monotonic write counter, `ReadSince` cursor reads, binary-search `ReadRange` |
| `TriggerEngine.{h,cpp}` | Trigger FSM (IDLE/ARMED/COLLECTING/TRIGGERED), edge detection per decoded sample |
| `UDPSourceStats.h` | 512-entry cycle/frag/byte rings → avg/std/min/max, rate, 20-bin histogram |
| `WSServer.{h,cpp}` | RFC 6455 server: handshake, framing, per-client write mutex, broadcast/unicast |
| `WSFrame.h`, `SHA1.h`, `Base64.h` | Header-only WS plumbing, shared with the ImGui client |
| `LTTB.h` | Largest-Triangle-Three-Buckets decimation |
The UDPS client itself lives in the shared library
`Source/Components/Interfaces/UDPStream/` (`UDPSClient`, also used by
`DebugService`). Note: in multicast mode the server delivers CONFIG over the
**TCP control connection**, so `UDPSClient` selects on both the multicast UDP
socket and the TCP socket and frames TCP reads.
## 2. Thread model
| Thread | Created by | Work |
|--------|-----------|------|
| main / push loop | `StreamHub::Run()` | At `PushRate` Hz: `PushData()` (serialise v1 frames), trigger servicing (capture finalisation, auto-rearm), `PushStats()` at `StatsRate` Hz |
| WS accept | `WSServer::Start()` | `accept()` + handshake, spawns client readers |
| WS client reader ×16 | `WSServer` | Reads frames, may contain several coalesced frames per TCP read; dispatches JSON to `StreamHub::OnWSCommand(json, len, slotIdx)` |
| UDPS receive ×32 | `UDPSClient::Start()` (one per session) | select() on UDP (+TCP in multicast), reassembles fragments, calls `UDPSourceSession` listener callbacks |
Synchronisation:
- Each `SignalRingBuffer` has its own `FastPollingMutexSem`; writers are the
UDPS receive threads, readers are the push loop and zoom handlers.
- `WSServer` has a per-client write mutex (push loop and command replies can
write concurrently).
- WS commands run on reader threads, but mutating operations (ring resize via
`setMaxPoints`) are deferred to the push loop through pending atomics.
- Session slots use an `active` flag; removal never compacts the array, so
indices stay stable.
## 3. Time base (wall-clock calibration)
All timestamps exposed to clients are **Unix wall-clock seconds** (float64).
Each session calibrates per time-source:
- First DATA packet anchors `pktCalibOffset = wallNow hrt/hrtFreq`; thereafter
`packetT = pktCalibOffset + hrt/hrtFreq`.
- Each referenced time signal gets its own offset on first value;
`timerToSec = 1e-9` for `uint64` time signals, `1e-6` otherwise.
- Re-anchoring on reconnect, CONFIG change, or if computed time drifts > 2 s
from wall clock (source restart / remote-vs-local HRT frequency drift).
Per `timeMode`:
| timeMode | Timestamping |
|----------|-------------|
| FIRST/LAST_SAMPLE | anchor = calibrated time-signal value (fallback `packetT`); `t = anchor ± k·dt` from `samplingRate` |
| FULL_ARRAY | `t[k]` = calibrated time array element |
| PACKET, n=1 | `packetT` |
| PACKET, n>1 | elements span `(lastPktWall, wallNow]` — backward anchoring, deliberately different from the Go hub (which extrapolates forward and overlaps the next packet under jitter); keeps ring time strictly monotonic |
Multi-element PACKET signals are exposed per element as `name[i]`.
## 4. Push path (only-new samples)
`SignalRingBuffer` keeps a monotonic `totalWritten`; the hub keeps a
per-(session, signal) cursor and uses `ReadSince(cursor, …)` (clamped to the
oldest sample on overrun). Each tick serialises only new samples, LTTB-capped
to `MaxPushPoints` (default 50) per signal — LTTB is applied only to temporal
signals (multi-element, timeMode ≠ PACKET). Cursors advance even with zero WS
clients so a connecting client never receives a backlog burst. Cursors are
reset on (re)CONFIG and on ring resize.
This replaces the original "re-send the last N points each tick" design, which
caused visible trace corruption (LTTB picked different points per overlapping
window).
## 5. Trigger engine
Hub-side, web-client semantics (`setTrigger` fields in
[StreamHub-API.md](StreamHub-API.md)):
```
IDLE --arm--> ARMED --edge crossing--> COLLECTING --wallNow ≥ trigTime+postSec+0.15s--> TRIGGERED
TRIGGERED --rearm (single) / auto ~200ms (normal, unless stopped)--> ARMED
any --disarm--> IDLE
```
`UDPSourceSession` calls `TriggerEngine::CheckSample` for every decoded sample
of the configured signal (signal index cached per config epoch). On
finalisation the push loop reads `[trigTimepreSec, trigTime+postSec]` from all
rings, LTTB-caps to 20 000 pts/signal and broadcasts a binary **version 2**
capture frame; every FSM transition broadcasts a `triggerState` event.
## 6. Configuration
```
WSPort = 8090
MaxPoints = 20000 // legacy global cap (overridable with -maxPoints)
PushRate = 30 // Hz
MaxPushPoints = 50 // per signal per push
StatsRate = 1 // Hz
RingTemporal = 1000000 // ring capacity, temporal signals (pts)
RingScalar = 100000 // ring capacity, scalar/PACKET signals (pts)
SourcesFile = "streamhub_sources.json" // saveSources persistence
Sources = {
Src1 = { Label = "PSU" Addr = "127.0.0.1" Port = 44500
MulticastGroup = "239.0.0.1" DataPort = 44503 } // multicast optional
}
```
Sources from `SourcesFile` are loaded after the static `Sources` block;
sources added at runtime via WS `addSource` get ids `s1, s2, …`.
## 7. Build & test
```bash
source env.sh # always, for build and run
make -f Makefile.gcc apps # builds StreamHub.ex
# or: make -C Source/Applications/StreamHub -f Makefile.gcc
./Build/x86-linux/StreamHub/StreamHub.ex -cfg hub.cfg
make -f Makefile.gcc test
./Build/x86-linux/GTest/MainGTest.ex # 71 unit tests, incl.
# SignalRingBuffer (ReadSince / binary-search ReadRange / wrap),
# TriggerEngine FSM, LTTB — sources in Test/Applications/StreamHub/
./run_e2e_test.sh # full-stack E2E (see below)
./run_streamhub.sh -w -g # interactive demo stack
```
### End-to-end test
`./run_e2e_test.sh` builds everything, launches the demo MARTe2 application
(`Test/Configurations/streamhub_demo.cfg`: 3 UDPStreamers — multicast scalars,
FirstSample/LastSample arrays, FullArray + uint64 ns time array) plus a
StreamHub on port 8095, then runs the Go WS client `Test/E2E/streamhub` which
verifies: `sources`/`config` events, ≥10 binary v1 pushes with wall-clock and
strictly monotonic time on all streams, `stats` shape, a `zoom` round-trip
(reqId echo, unicast), and a complete trigger cycle (setTrigger → arm →
binary v2 capture → triggered → disarm). Logs land in
`/tmp/streamhub_e2e_{marte,hub}.log`. Exit 0 iff every check passes.
When changing the WS protocol, update **in lockstep**: this hub, the Go hub
(`Common/Client/go/wshub`), the browser SPA (`Client/udpstreamer/static`), the
ImGui client (`Client/streamhub/Protocol.cpp`), the E2E client
(`Test/E2E/streamhub`), and [StreamHub-API.md](StreamHub-API.md).
## 8. Gotchas
- **No STL** in this directory (MARTe2 style); use `StreamString`,
`FastPollingMutexSem`, fixed arrays.
- Link against UDPStream **dynamically** (`LIBRARIES += -lUDPStream`), never
`LIBRARIES_STATIC` — the GTest binary links every `.a` it finds and would get
duplicate symbols.
- WS text frames may arrive coalesced; never NUL-terminate a payload in place
without restoring the byte (it is the first header byte of the next frame).
- Zoom replies must print `t` with `%.17g`: at Unix-epoch magnitudes `%.6f`
destroys µs resolution.
+119
View File
@@ -0,0 +1,119 @@
# StreamHub — User Guide
StreamHub turns signals streamed by MARTe2 `UDPStreamer` DataSources into a
multi-user oscilloscope. One hub process aggregates any number of sources and
serves up to 16 simultaneous clients: a browser SPA and/or the native ImGui
desktop client.
```
MARTe2 app (UDPStreamer) ──UDP 44500──▶ StreamHub ──WebSocket 8090──▶ browser / ImGui
```
## 1. Quick start (demo stack)
```bash
source env.sh
./run_streamhub.sh -w -g
```
This builds everything, starts a demo MARTe2 application with three streamers
(scalars over multicast, two array streamers), a StreamHub on port 8090, the
web UI server on port 8080 (`-w`) and the ImGui client (`-g`). Then open:
```
http://localhost:8080/?hub=localhost:8090
```
Useful flags: `-p <port>` WS port, `-n <pts>` ring size, `-s` skip rebuild,
`-h` full help (including the list of ports used).
## 2. Running against your own application
1. Add a `UDPStreamer` DataSource to your MARTe2 configuration
(see [UDPStreamer.md](UDPStreamer.md)).
2. Write a hub configuration, e.g. `hub.cfg`:
```
WSPort = 8090
Sources = {
MyApp = { Label = "MyApp" Addr = "192.168.0.10" Port = 44500 }
}
```
For multicast streamers add `MulticastGroup = "239.0.0.1"` and
`DataPort = 44503`. All keys and defaults:
[StreamHub-Developer.md](StreamHub-Developer.md) §6.
3. Run the hub and a client:
```bash
source env.sh
./Build/x86-linux/StreamHub/StreamHub.ex -cfg hub.cfg
# web UI
cd Client/webui && go build -o streamhub-webui . && ./streamhub-webui
# then browse http://localhost:8080/?hub=<hub-host>:8090
# or native client
./Client/streamhub/build/StreamHubClient -host <hub-host> -port 8090
```
Sources can also be added at runtime from either client ("Add source": label +
`host:port`, optional multicast group/data port). "Save sources" persists the
runtime list to `streamhub_sources.json` so the hub reloads it on restart.
## 3. Using the oscilloscope (web UI and ImGui)
Both clients offer the same workflow:
- **Plotting** — drag signals from the source sidebar onto a plot. Choose a
grid layout (1×1 … 4×1 / 2×2 / 1×4). Live plots scroll with wall-clock time;
pick the time window (1/5/10/30/60 s).
- **Zoom** — scroll/drag to zoom. When zoomed in, the client automatically
fetches high-resolution data from the hub for the visible window. *Back*
steps through the zoom history, *Fit* frames the data, *Reset/Live* returns
to the scrolling live view.
- **Cursors** — enable cursors A/B and drag them; the readout shows ΔT and the
value of each trace at both cursors.
- **V-scale modes** — *normal*, *digital* (booleans stacked as logic traces),
*mixed*.
- **Signal styling** — per-trace colour, line width, markers.
- **Stats** — per source: packet rate ± std, lost packets, fragments and bytes
per cycle, cycle-time min/avg/max/std and a 20-bin cycle-time histogram.
## 4. Trigger
The trigger runs **in the hub**, so a capture is identical in every connected
client and uses full-rate ring data (not the decimated live stream).
1. Pick the trigger **signal** (for array signals: a specific element), the
**edge** (rising/falling/both) and the **threshold**.
2. Pick the capture **window** (100 µs … 10 s) and the **pre-trigger**
percentage (how much of the window precedes the trigger instant).
3. Mode **normal** re-arms automatically ~200 ms after each capture
(*Stop* pauses re-arming); mode **single** captures once and waits for
*Rearm*.
4. Press **Arm**. Badge states: `IDLE → ARMED → COLLECTING → TRIGGERED`.
On capture, the trigger view shows all plotted signals relative to the trigger
instant (marker at t = 0) over `[-pre, +post]`.
## 5. Troubleshooting
| Symptom | Check |
|---------|-------|
| Source stuck in *connecting* | Streamer host/port reachable? For multicast: group/data port match the streamer, and routing allows IGMP |
| No data but source *streaming* | `getStats` rate ≈ RT cycle rate? Firewall on UDP data port? |
| Web UI loads but no signals | Did you pass `?hub=host:8090` (the SPA talks WS directly to the hub, not the web server)? |
| Trigger never fires | Threshold inside the signal's actual range? Edge direction correct? |
| Time axis jumps once at start | Normal: first-packet wall-clock calibration; it also re-anchors after a source restart |
Hub-side logs go to stdout; the demo scripts write
`/tmp/streamhub_e2e_{marte,hub}.log`.
## 6. Further reading
- [StreamHub-API.md](StreamHub-API.md) — WebSocket protocol (write your own client)
- [StreamHub-Developer.md](StreamHub-Developer.md) — internals, build, tests
- [UDPStreamer.md](UDPStreamer.md) — configuring the MARTe2 DataSource
- [Protocol.md](Protocol.md) — UDPS wire format