Implemented qt port + e2e

This commit is contained in:
Martino Ferrari
2026-06-26 09:11:10 +02:00
parent 0d7d8f396b
commit 4702d0a217
146 changed files with 57272 additions and 128 deletions
@@ -0,0 +1,127 @@
# StreamHub Qt Client Implementation Plan
> **For agentic workers:** implement task-by-task. Each task ends with a buildable
> deliverable. Build with `cmake -B build && cmake --build build -j` from
> `Client/streamhub-qt/`.
**Goal:** Native Qt5/6 oscilloscope client equivalent to the ImGui
`Client/streamhub/` client, same StreamHub WS protocol.
**Architecture:** Qt Widgets + custom `QPainter` plot widget + `QtWebSockets`.
Reuse `../streamhub/Protocol.{h,cpp}` and `SignalBuffer.h` verbatim. Single GUI
thread; `QWebSocket` signals drive a model; 60 Hz `QTimer` repaints plots.
**Tech Stack:** C++17, Qt5 or Qt6 (autodetect), CMake ≥3.16.
## Global Constraints
- Must compile against Qt5 **and** Qt6 — use `Qt${QT_VERSION_MAJOR}::` targets,
no version-specific APIs without a guard.
- Do not modify `Client/streamhub/` (reference). Reuse its `Protocol.cpp`,
`Protocol.h`, `SignalBuffer.h` by include path / direct source compile.
- Domain model uses `QColor` (not `ImVec4`).
- Reuse `LTTBDecimate` and `SignalBuffer` from `SignalBuffer.h`.
---
### Task 1: Project scaffold + CMake autodetect + empty MainWindow
**Files:** Create `CMakeLists.txt`, `main.cpp`, `MainWindow.{h,cpp}`,
`Theme.{h,cpp}`, `Model.h`.
- CMake: `find_package(QT NAMES Qt6 Qt5 REQUIRED COMPONENTS Widgets WebSockets)`,
`find_package(Qt${QT_VERSION_MAJOR} ...)`, `CMAKE_AUTOMOC ON`, C++17. Sources
include `${CMAKE_CURRENT_SOURCE_DIR}/../streamhub/Protocol.cpp`; include dir
`../streamhub`. Link `Qt${QT_VERSION_MAJOR}::Widgets`,
`Qt${QT_VERSION_MAJOR}::WebSockets`.
- `Model.h`: `SignalView{SignalMeta meta; SignalBuffer buf; QColor color; double
lineWidth; int marker; bool visible;}`, `Source`, `VScale`, `PlotAssignment`,
`PlotLayout` enum + `layoutDims`/`layoutNames`/`layoutCount`.
- `Theme`: apply Catppuccin-Mocha `QPalette` + stylesheet; `tracePalette(i)`.
- `main.cpp`: parse `-host`/`-port`, build `QApplication`, apply theme, show
`MainWindow`.
- **Deliverable:** builds, opens an empty dark window.
### Task 2: WsClient + Hub controller (data model + WS dispatch)
**Files:** Create `WsClient.{h,cpp}`, `Hub.{h,cpp}`.
- `WsClient`: wraps `QWebSocket`; `connectTo(host,port)`, `reconnect`, `sendText`;
3 s reconnect `QTimer`; signals `connectedChanged(bool)`,
`textReceived(QString)`, `binaryReceived(QByteArray)`.
- `Hub`: owns `WsClient` + `std::vector<Source>` + `TriggerCfgState` +
`CaptureFrame` + per-plot zoom caches + `HistoryInfoMsg`. On connect sends
`getSources/getStats/historyInfo`. Routes text via `ParseType` to
`onSources/onConfig/onStats/onTriggerState/onZoom/onHistoryZoom/onHistoryInfo/`
`onMaxPointsUpdated`; binary via first byte (v2 capture vs v1 push). Command
senders mirror `Build*` in `Protocol.h`. Emits signals listed in the spec.
Holds a `findSource/findSignal/slotKey`. Zoom caches keyed by plot index
(`requestZoom`, `requestHistoryZoom`, `nextZoomReqId`).
- **Deliverable:** builds; connecting to a hub logs received message types
(temporary `qDebug`), model populates (verified via a debug dump).
### Task 3: PlotWidget — static render + live scroll
**Files:** Create `PlotWidget.{h,cpp}`.
- `PlotWidget(Hub*, int plotIdx, QWidget*)`. `paintEvent`: draw frame, grid,
±4-div Y axis with 9 ticks, X axis with 10 ticks. Live mode: X =
`[wallNow-windowSec, wallNow]`. For each assigned slot: read ring
(`buf.readLast(maxPoints)`), clip to visible X, `LTTBDecimate` to 2400,
normalize (normal/digital/mixed via ported `resolveVScale`/`normalizeY`/
`bandNormalize`), map to pixels, `drawPolyline` in trace color/width; optional
markers. Badge row painted at top (trace name + V/div).
- **Deliverable:** dropping handled later; for now a hard-coded assignment shows
a live trace scrolling against wall clock.
### Task 4: PlotWidget — interaction (zoom/pan/cursors/active/vmode) + drag-drop
**Files:** Modify `PlotWidget.{h,cpp}`; add small per-plot toolbar (Live/Back/
Fit, N/D/M, V-scale controls) as child widgets or painted buttons.
- Mouse: wheel = Y zoom active trace (X zoom if none); Ctrl+wheel = X
zoom/window; Shift+wheel = Y pan; right-drag = X pan (live→non-live); box-zoom
optional. Click a badge = toggle active slot; context menu on badge = color/
width/marker/digital-in-mixed/remove. Zoom history Back/Fit/Live.
- Global A/B cursors (drawn + draggable) with readouts; pause snapshot.
- `dragEnterEvent`/`dropEvent` accept a signal-ref mime → append assignment.
- Hi-res WS zoom + history zoom requests through `Hub` (live-narrow throttle,
non-live debounce) and prefer cached hi-res/history data when it covers view.
- **Deliverable:** full single-plot interaction parity.
### Task 5: PlotGrid + SourceSidebar + MainWindow toolbar
**Files:** Create `PlotGrid.{h,cpp}`, `SourceSidebar.{h,cpp}`; expand
`MainWindow.{h,cpp}`.
- `PlotGrid`: nested `QSplitter`s realizing each `PlotLayout`; rebuild on change.
- `SourceSidebar`: `QTreeWidget` sources→signals; drag a signal (mime with
src/sig index); "Add source" + remove; reflects `Hub` signals.
- MainWindow toolbar: sidebar toggle, layout picker (menu of the 8 layouts),
pause, cursors toggle, trigger-bar toggle, history-bar toggle, window-preset
combo, stats button, connection (host/port + reconnect), status LED.
- Add-source dialog (label/host/port/multicast/dataPort) → `Hub::addSource`.
- **Deliverable:** drag-drop from sidebar to any layout cell works end to end.
### Task 6: TriggerBar + StatsDialog + HistoryBar
**Files:** Create `TriggerBar.{h,cpp}`, `StatsDialog.{h,cpp}`,
`HistoryBar.{h,cpp}`; wire into `MainWindow`.
- `TriggerBar`: signal combo (full keys incl array index), edge combo, threshold
spin, window combo (100 µs…10 s), pre% slider, normal/single → `setTrigger`;
Arm/Disarm/Rearm/Stop; badge from `triggerStateChanged`. Capture frame routes
to plots' trigger view.
- `StatsDialog`: table of metrics per source + custom 20-bin histogram widget
(`QPainter` bars), refreshed on `statsChanged`.
- `HistoryBar`: Live, range readout, pan ←/→, jump presets, All; drives plots'
non-live X range + history-zoom.
- **Deliverable:** trigger capture, stats histogram, history browse all match
the ImGui client.
### Task 7: Build, run, verify, document
- Clean Release build on Qt6 (and confirm Qt5 configure path).
- Launch against a live hub; verify the spec's parity checklist.
- Add a `Client/streamhub-qt` build note to `CLAUDE.md` and `ARCHITECTURE.md`.
- **Deliverable:** working client + docs.
@@ -0,0 +1,453 @@
# Streaming-Chain E2E Suite Implementation Plan
> **For agentic workers:** Implement task-by-task. Steps use checkbox (`- [ ]`) syntax.
**Goal:** A curated full-chain E2E suite (MARTe2 UDPStreamer → StreamHub → client)
with waveform validation across a covering config matrix, all trigger/zoom/window
client checks, a Qt GUI smoke test, and one PDF + `results.json` report.
**Architecture:** New `Test/E2E/chain/`. Shell orchestrator drives a Python
matrix that, per scenario, generates `input.bin` + MARTe `.cfg` + StreamHub
`.cfg`, launches the two-process stack, runs a Go mock client, validates
waveforms, and aggregates a Typst PDF + JSON. Mirrors `Test/E2E/datasources/`.
**Tech Stack:** Python 3 (numpy, matplotlib, struct), Go (gorilla/websocket),
MARTe2 (MARTeApp + FileReader/IOGAM/LinuxTimer/FileWriter + UDPStreamer),
StreamHub.ex, Typst, Qt5/6 QTest.
## Global Constraints
- Artifacts go to `Build/x86-linux/E2E/chain/` and `/tmp/chain_e2e/`; **never**
write generated cfg/data/plots into the source tree.
- Reuse `Test/E2E/datasources/validate_binary.py` read-helpers and the
`run_recorder_e2e.sh` two-process orchestration pattern verbatim where possible.
- MARTe2 binary FileReader format: header `[u32 numSigs]` then per signal
`[u16 TypeDescriptor.all][32B name][u32 numElements]`, then raw rows in signal
order, each value little-endian in its native width.
- MARTe2 `TypeDescriptor.all` codes (verified needed values): uint8=0x0408,
int8=0x0008, uint16=0x0410, int16=0x0010, uint32=0x0420, int32=0x0020,
uint64=0x0440, int64=0x0040, float32=0x0808 (2056), float64=0x0810. Confirm
each against a MARTe round-trip in Task 2.
- StreamHub config keys: `WSPort`, `MaxPoints`, `PushRate`, `MaxPushPoints`,
`RingTemporal`, `RingScalar`, `Sources = { id = { Label Addr Port
[MulticastGroup DataPort] } }`.
- UDPStreamer keys (verified): `Port`, `MulticastGroup`, `DataPort`,
`MaxPayloadSize`, `PublishingMode ∈ {Strict,Accumulate,Decimate}`,
`MinRefreshRate` (Accumulate), `Ratio` (Decimate); per-signal `Type`,
`NumberOfDimensions`, `NumberOfElements`, `TimeMode ∈
{PacketTime,FullArray,FirstSample,LastSample}`, `TimeSignal`, `SamplingRate`,
`QuantizedType ∈ {none,uint8,int8,uint16,int16}`, `RangeMin`, `RangeMax`,
`Unit`.
- Each scenario runs under `timeout`; always tear the stack down via a bash trap.
- Run `source env.sh` for all build/run steps; set `LD_LIBRARY_PATH` as in
`run_recorder_e2e.sh`.
---
### Task 1: Scenario model + a 3-scenario starter set
**Files:**
- Create: `Test/E2E/chain/scenarios.py`
**Interfaces:**
- Produces: `SCENARIOS: list[dict]` and `validate_scenario(s) -> list[str]`
(returns list of validity errors, empty if valid). Each scenario dict:
```python
{
"id": "s01_scalar_uint32", # unique slug
"network": "unicast"|"multicast",
"publishing": "Strict"|"Accumulate"|"Decimate",
"ratio": int|None, # Decimate
"min_refresh_hz": float|None, # Accumulate
"max_payload": int, # bytes
"ws_port": int, "udp_port": int, "data_port": int|None,
"multicast_group": str|None,
"sources": [ # 1+ sources
{"id": "src", "signals": [
{"name","type","elements","time_mode","time_signal",
"sampling_rate","quant","range_min","range_max","unit",
"formula"} # formula: "sine"|"ramp"|"counter"
]}
],
"oracle": "analytic"|"fed"|"both",
"client_checks": ["live","zoom","window","trigger"],
}
```
- [ ] **Step 1: Write `scenarios.py`** with the dict schema above, a
`validate_scenario()` enforcing the rules (quant only on float32/float64;
FullArray needs a matching-length `TimeSignal`; First/LastSample need a scalar
`TimeSignal` + `sampling_rate>0`; Decimate needs `ratio>=1`; Accumulate needs
`min_refresh_hz>0`; multicast needs `multicast_group`+`data_port`), and an
initial 3 scenarios: `s01_scalar_uint32` (Strict unicast),
`s02_array_float32_fullarray` (Strict unicast, 100-elem + uint64 ns time
array), `s03_quant_uint16` (float32 quantised, RangeMin/Max). Give each a
distinct `ws_port`/`udp_port` to allow isolation.
- [ ] **Step 2: Self-check the model**
Run: `python3 -c "import sys; sys.path.insert(0,'Test/E2E/chain'); import scenarios as S; [print(s['id'], S.validate_scenario(s)) for s in S.SCENARIOS]"`
Expected: each line prints the id and `[]` (no errors).
- [ ] **Step 3: Commit**
```bash
git add Test/E2E/chain/scenarios.py
git commit -m "test(e2e-chain): scenario model + starter scenarios"
```
---
### Task 2: Data generator (`gen_data.py`) — all types/shapes
**Files:**
- Create: `Test/E2E/chain/gen_data.py`
- Reference: `Test/E2E/datasources/gen_test_data.py` (format), `validate_binary.py`
**Interfaces:**
- Consumes: a scenario dict (Task 1).
- Produces CLI `python3 gen_data.py --scenario <id> --out <path>` writing
`input_<id>.bin`; and a Python API
`write_input(scenario, path) -> dict` returning, per `src:signal`, the
ground-truth arrays `{"t":[...], "v":[...native...], "formula":..., "dt":...}`
so the validator can reconstruct truth without re-deriving layout.
- Type code map `TYPE_CODES: dict[str,int]` and numpy dtype map `NP_DTYPE`.
- [ ] **Step 1: Write `gen_data.py`** — for the scenario's signals, build
`NUM_ROWS` rows (default 200) of deterministic values:
- `counter`: `r*elements + c` cast to the signal type.
- `ramp`: linear in `(r,c)`.
- `sine`: `A*sin(2π f (r*dt + c*dt_elem))` with per-signal frequency; for time
signals (`unit ns/us`) emit the matching timestamp array/scalar.
Write the MARTe binary: `[u32 numSigs]` + per-signal `[u16 TypeDescriptor.all]
[32B name][u32 elements]` + rows, each value packed little-endian at native
width via `numpy.tobytes()`. Return the ground-truth dict.
- [ ] **Step 2: Round-trip self-test**
Run: `python3 -c "import sys;sys.path.insert(0,'Test/E2E/chain');import scenarios as S,gen_data as G;G.write_input(S.SCENARIOS[0],'/tmp/chain_e2e/t.bin');import validate_binary" 2>&1; ls -l /tmp/chain_e2e/t.bin`
Expected: file exists, size = header + 200 rows × rowbytes.
- [ ] **Step 3: Verify a type round-trip through MARTe** — generate a uint32
scalar file, point a tiny `FileReader→FileWriter` cfg at it (reuse datasources
cfg pattern, single signal), run MARTeApp 3 s, confirm output rows equal input
rows (proves the `TypeDescriptor.all` code is correct for that type). Repeat
mentally/quickly for one int and one float type.
Run: (the orchestrator does this in Task 7; here just confirm bytes are sane)
`python3 -c "import struct;d=open('/tmp/chain_e2e/t.bin','rb').read();print(struct.unpack_from('<I',d,0))"`
Expected: prints the signal count.
- [ ] **Step 4: Commit**
```bash
git add Test/E2E/chain/gen_data.py
git commit -m "test(e2e-chain): deterministic typed/shaped data generator"
```
---
### Task 3: Config generator (`gen_cfg.py`)
**Files:**
- Create: `Test/E2E/chain/gen_cfg.py`
- Reference: `Test/E2E/datasources/E2ETest.cfg`, `Test/E2E/recorder/StreamHubRec.cfg`
**Interfaces:**
- Consumes: a scenario dict.
- Produces: `write_marte_cfg(scenario, path, input_bin, tap_bin|None)` and
`write_hub_cfg(scenario, path)`. The MARTe cfg wires
`LinuxTimer + FileReader(input_bin) -> IOGAM -> UDPStreamer(s)`; when
`oracle in {fed,both}` it adds a second IOGAM branch `-> FileWriter(tap_bin)`
of the same fed signals. The hub cfg declares one `Source` per UDPStreamer
(with `MulticastGroup`/`DataPort` for multicast) on the scenario's `ws_port`.
- [ ] **Step 1: Write `gen_cfg.py`** generating both cfgs as strings from the
scenario, honouring publishing mode keys, per-signal type/shape/timemode/quant,
unicast vs multicast, payload, multi-source. Keep one GAM thread.
- [ ] **Step 2: Self-test cfg generation**
Run: `python3 -c "import sys;sys.path.insert(0,'Test/E2E/chain');import scenarios as S,gen_cfg as C;C.write_marte_cfg(S.SCENARIOS[0],'/tmp/chain_e2e/m.cfg','/tmp/chain_e2e/t.bin',None);C.write_hub_cfg(S.SCENARIOS[0],'/tmp/chain_e2e/h.cfg');print(open('/tmp/chain_e2e/h.cfg').read())"`
Expected: prints a valid-looking StreamHub cfg with `Sources` and the right port.
- [ ] **Step 3: Validate cfgs actually load** — run the full stack for scenario 1
(StreamHub.ex + MARTeApp) for 5 s; confirm StreamHub log shows
`session '...' started` and `Connected`. (Manual via Task 7 harness; here just
eyeball the generated cfg text.)
- [ ] **Step 4: Commit**
```bash
git add Test/E2E/chain/gen_cfg.py
git commit -m "test(e2e-chain): MARTe + StreamHub config generator"
```
---
### Task 4: Go mock client — record + waveform + behavioural checks
**Files:**
- Create: `Test/E2E/chain/client/main.go`, `Test/E2E/chain/client/go.mod`
- Reference: `Test/E2E/streamhub/main.go` (parsers + driver to reuse)
**Interfaces:**
- CLI: `chain-client -hub host:port -scenario <id> -trigsig <src:sig>
-trigthr <f> -checks live,zoom,window,trigger -out <dir> -dur <sec>`.
- Produces: `received_<id>.bin` (MARTe-binary-compatible: same header + rows
re-assembled from v1 pushes per signal, time-ordered) and
`checks_<id>.json` = `{ "live":{ok,frames,signals}, "zoom":[{range,n,inrange,
pts}], "window":{...}, "trigger":[{edge,mode,fired,trigTime,preSec,postSec,
capturePts,edgeOk}] }`.
- [ ] **Step 1: Copy the wire parsers** (`parsePush`, `parseCapture`, event
types, `pump`/`waitFor`) from `Test/E2E/streamhub/main.go` into the new client;
reuse `go.mod` deps (gorilla/websocket).
- [ ] **Step 2: Implement recording + live check** — accumulate v1 pushes for
`dur` seconds; per `src:sig`, sort samples by time, dedupe; write
`received_<id>.bin`; assert ≥10 frames and monotonic wall-clock time.
- [ ] **Step 3: Implement zoom + window checks** — issue zoom at a narrow and a
wide range over the observed window; assert points within range and count ≤ n;
record results. If historyInfo enabled, also historyZoom.
- [ ] **Step 4: Implement trigger matrix** — for each `edge ∈
{rising,falling,both}` × `mode ∈ {normal,single}`: `setTrigger` on `-trigsig`
at `-trigthr`, `arm`, wait for triggerState + v2 capture; record
`trigTime/preSec/postSec`, verify capture window bounds and that the trigger
signal crosses threshold in the correct direction near `trigTime`. For
`normal` confirm re-arm (a second capture); for `single` confirm no second
capture until `rearm`. Write `checks_<id>.json`.
- [ ] **Step 5: Build the client**
Run: `cd Test/E2E/chain/client && go build -o chain-client .`
Expected: builds, produces `chain-client`.
- [ ] **Step 6: Commit**
```bash
git add Test/E2E/chain/client
git commit -m "test(e2e-chain): Go mock client (record + zoom/window/trigger)"
```
---
### Task 5: Waveform validator (`validate_waveform.py`)
**Files:**
- Create: `Test/E2E/chain/validate_waveform.py`
**Interfaces:**
- Consumes: scenario dict, ground-truth dict (from `gen_data.write_input`),
`received_<id>.bin`, optional `tap_<id>.bin`, `checks_<id>.json`.
- Produces: CLI `python3 validate_waveform.py --scenario <id> --truth <bin>
--received <bin> [--tap <bin>] --checks <json> --out <metrics.json>`; returns
exit 0 pass / 1 fail. `metrics_<id>.json` = per-signal
`{max_abs_err, quant_step, rmse, corr, n_truth, n_recv, oracle, pass}` plus
the client checks rolled in and an overall `pass`.
- [ ] **Step 1: Write the validator** — read received rows (reuse
`validate_binary.read_binary`), reconstruct truth on the received timestamps
via the ground-truth formula; apply the per-effect tolerance from the design
(bit-exact no-quant; `quant_step/2` quantised; decimation stride; LTTB shape
metric corr≥0.99 & nRMSE≤0.05). For `fed`/`both`, also compare received vs tap.
- [ ] **Step 2: Self-test on a synthetic pair** — craft a truth array and a
copy with added quant noise ≤ step/2 → expect pass; noise > step → expect fail.
Run: `python3 Test/E2E/chain/validate_waveform.py --selftest`
Expected: prints `selftest OK` and exits 0.
- [ ] **Step 3: Commit**
```bash
git add Test/E2E/chain/validate_waveform.py
git commit -m "test(e2e-chain): per-effect waveform validator"
```
---
### Task 6: Plots (`plots.py`)
**Files:**
- Create: `Test/E2E/chain/plots.py`
- Reference: plotting block in `Test/E2E/datasources/run_e2e_report.sh`
**Interfaces:**
- Consumes: `input_<id>.bin`, `received_<id>.bin`, `metrics_<id>.json`,
`checks_<id>.json`.
- Produces: `python3 plots.py --scenario <id> --dir <artifactdir>` writing
`wave_<id>.png` (truth/received/diff), `trig_<id>.png` (captures),
`zoom_<id>.png`.
- [ ] **Step 1: Write `plots.py`** (matplotlib Agg) producing the three PNGs;
skip gracefully if an input is missing (write a placeholder note).
- [ ] **Step 2: Self-test**
Run: `python3 Test/E2E/chain/plots.py --scenario s01_scalar_uint32 --dir /tmp/chain_e2e 2>&1 | tail -3`
Expected: prints the PNG paths (or graceful "missing data" notes).
- [ ] **Step 3: Commit**
```bash
git add Test/E2E/chain/plots.py
git commit -m "test(e2e-chain): report plot generation"
```
---
### Task 7: Orchestrator (`run_chain_e2e.sh`) + results.json
**Files:**
- Create: `Test/E2E/chain/run_chain_e2e.sh`
- Reference: `Test/E2E/recorder/run_recorder_e2e.sh` (stack + LD_LIBRARY_PATH)
**Interfaces:**
- Consumes: all of the above.
- Produces: `Build/x86-linux/E2E/chain/results.json` (aggregate) and per-scenario
artifacts; orchestrates build, the two-process stack, client, validation, plots.
- [ ] **Step 1: Write `run_chain_e2e.sh`** — flags `--skip-build`,
`--only <id>`, `--pdf-only`. Source `env.sh`; set `LD_LIBRARY_PATH` as in
`run_recorder_e2e.sh` (+ FileDataSource, IOGAM, LinuxTimer). Build UDPStream,
UDPStreamer, StreamHub, and `chain-client`. For each scenario (a Python helper
emits the list): gen data + cfgs; start `StreamHub.ex`; start `MARTeApp`; run
`chain-client`; stop stack via trap; run `validate_waveform.py` and `plots.py`;
append to `results.json`. Multicast scenarios: probe a loopback multicast
route, else mark SKIP.
- [ ] **Step 2: Run the starter 3 scenarios end-to-end**
Run: `source env.sh && ./Test/E2E/chain/run_chain_e2e.sh --skip-build --only s01_scalar_uint32`
Expected: stack starts, client connects, `metrics_s01*.json` written with
overall `pass=true`; `results.json` contains the scenario.
- [ ] **Step 3: Run all 3 starter scenarios**
Run: `source env.sh && ./Test/E2E/chain/run_chain_e2e.sh --skip-build`
Expected: all 3 pass; `results.json` overall status PASS.
- [ ] **Step 4: Commit**
```bash
git add Test/E2E/chain/run_chain_e2e.sh
git commit -m "test(e2e-chain): orchestrator + results.json aggregation"
```
---
### Task 8: Expand to the full ~50-scenario covering matrix
**Files:**
- Modify: `Test/E2E/chain/scenarios.py`
- [ ] **Step 1: Add scenarios** filling the buckets in the design (per-type
scalars; time-mode sweep; quant sweep; publishing modes; multicast variants;
shapes/fragmentation; multi-source; edge cases) until every option value is
covered ≥1×. Keep ports unique per scenario.
- [ ] **Step 2: Validate the whole matrix model**
Run: `python3 -c "import sys;sys.path.insert(0,'Test/E2E/chain');import scenarios as S;bad=[(s['id'],e) for s in S.SCENARIOS for e in [S.validate_scenario(s)] if e];print('INVALID',bad) if bad else print('OK',len(S.SCENARIOS),'scenarios')"`
Expected: `OK <N> scenarios` with N in 4070.
- [ ] **Step 3: Run a representative subset live** (one per bucket via
`--only`), fixing generator/cfg/validator gaps surfaced by real runs.
- [ ] **Step 4: Full run**
Run: `source env.sh && ./Test/E2E/chain/run_chain_e2e.sh`
Expected: suite completes; `results.json` shows the matrix; failures (if any)
are real findings, recorded per-scenario.
- [ ] **Step 5: Commit**
```bash
git add Test/E2E/chain/scenarios.py
git commit -m "test(e2e-chain): full covering scenario matrix"
```
---
### Task 9: Typst report (`E2E_Report.typ`)
**Files:**
- Create: `Test/E2E/chain/E2E_Report.typ`
- Reference: `Test/E2E/datasources/E2E_Report.typ`
- Modify: `run_chain_e2e.sh` (compile step)
- [ ] **Step 1: Write `E2E_Report.typ`** — title/env/summary table driven from
`results.json` (read via a small Python pre-step that emits a `.typ` data
include or a CSV the template loads), embed `wave_*/trig_*/zoom_*` PNGs for a
representative subset, list per-scenario pass/fail.
- [ ] **Step 2: Wire PDF compile** into `run_chain_e2e.sh` (copy `.typ` to
`OUT_DIR`, `typst compile`), guarded by `command -v typst`.
- [ ] **Step 3: Build the PDF**
Run: `source env.sh && ./Test/E2E/chain/run_chain_e2e.sh --pdf-only`
Expected: `Build/x86-linux/E2E/chain/E2E_Report.pdf` exists.
- [ ] **Step 4: Commit**
```bash
git add Test/E2E/chain/E2E_Report.typ Test/E2E/chain/run_chain_e2e.sh
git commit -m "test(e2e-chain): Typst PDF report"
```
---
### Task 10: Qt GUI smoke test
**Files:**
- Create: `Client/streamhub-qt/test/smoke_test.cpp`
- Modify: `Client/streamhub-qt/CMakeLists.txt` (add `enable_testing()` + a
`QTest` executable + `add_test`)
**Interfaces:**
- Consumes: a live StreamHub on a port from env var `SHQ_TEST_HUB`
(default `127.0.0.1:8090`).
- [ ] **Step 1: Write `smoke_test.cpp`** (`QTest`, `QT_QPA_PLATFORM=offscreen`):
construct `MainWindow`, connect, spin the event loop a few hundred ms, assert
`hub.sources()` non-empty and ≥1 repaint/tick occurred, exercise a layout
change + pause toggle; pass if no crash and a source was seen (skip-pass if no
hub, to keep it CI-safe).
- [ ] **Step 2: Add the test target** to `CMakeLists.txt` linking
`Qt${QT_VERSION_MAJOR}::Test` + Widgets + WebSockets, reusing the client
objects; register with `add_test(NAME shq_smoke ...)`.
- [ ] **Step 3: Build + run**
Run: `cd Client/streamhub-qt && cmake -B build && cmake --build build -j4 && QT_QPA_PLATFORM=offscreen ctest --test-dir build --output-on-failure`
Expected: `shq_smoke` passes (or skip-passes with no hub).
- [ ] **Step 4: Commit**
```bash
git add Client/streamhub-qt/test Client/streamhub-qt/CMakeLists.txt
git commit -m "test(streamhub-qt): offscreen QTest GUI smoke"
```
---
### Task 11: Docs + wire-up
**Files:**
- Modify: `CLAUDE.md` (E2E demo scripts line), `ARCHITECTURE.md` (test section)
- [ ] **Step 1: Document** the new suite under the build/test notes: how to run
`run_chain_e2e.sh`, where artifacts land, what the matrix covers, and the Qt
ctest smoke.
- [ ] **Step 2: Commit**
```bash
git add CLAUDE.md ARCHITECTURE.md
git commit -m "docs: streaming-chain E2E suite + Qt smoke test"
```
## Self-Review
- **Spec coverage:** matrix (T1,T8), waveform oracle A+B (T2,T5), client
live/zoom/window/trigger (T4), disk artifact (T4 received_*.bin), Qt smoke
(T10), PDF + JSON (T7,T9) — all mapped.
- **Type consistency:** scenario dict schema (T1) is consumed unchanged by
gen_data (T2), gen_cfg (T3), client args (T4), validator (T5); artifact names
`input_/received_/tap_/checks_/metrics_/wave_/trig_/zoom_<id>` consistent
across T2T9.
- **Placeholders:** none; commands and schemas are concrete.
@@ -0,0 +1,134 @@
# StreamHub Qt Client — Design Spec
**Date:** 2026-06-25
**Sub-project:** D (port `Client/streamhub/` ImGui/SDL2/ImPlot desktop client to Qt5/6)
**Status:** Approved-by-default (user away; standing instruction: "implement the specs, do not wait for my approval").
> Decisions below were made autonomously per the standing instruction. They are
> recorded with rationale so they can be revisited.
## Goal
Produce a native Qt desktop oscilloscope client (`Client/streamhub-qt/`) that is
feature- and UX-equivalent to the existing ImGui client, talking the identical
StreamHub WebSocket protocol (JSON commands/events + binary v1 push / v2 trigger
capture frames), and builds against **either Qt5 or Qt6** (autodetected), for
back-compatibility with older Linux distributions.
The existing ImGui client (`Client/streamhub/`) stays untouched as a reference.
## Framework decisions (with rationale)
1. **Qt Widgets, not QML/Qt Quick.**
The client is a dense desktop tool: multi-panel layouts, drag-and-drop of
signals, context menus, splitters, modal dialogs. Widgets is the natural fit,
renders without a GPU/OpenGL pipeline (important on old hardware), and has the
most stable API surface shared between Qt5 and Qt6.
2. **Custom `QPainter`-based `PlotWidget`, not QCustomPlot/QtCharts.**
The oscilloscope plotting is highly bespoke (fixed ±4-division Y space,
wall-clock live X, LTTB decimation, normal/digital/mixed band normalization,
per-trace V/div + screen position, A/B cursors, box/scroll zoom + pan, zoom
history, drag-drop, hi-res WS zoom overlay, trigger-capture view). Replicating
this on top of a general charting library fights the library as much as it
helps. A focused `QPainter` widget:
- gives exact behavioral parity with the ImPlot version;
- adds **zero external dependencies** (max portability on old Linux);
- avoids GPL/commercial licensing entanglement (repo is EUPL v1.1);
- is straightforward to test in isolation.
3. **`QtWebSockets` (`QWebSocket`), not the hand-rolled POSIX RFC6455 client.**
`QWebSocket` integrates with the Qt event loop, exposes
`textMessageReceived`/`binaryMessageReceived` signals, and removes the need
for a manual receive thread + mutex-guarded queue. Auto-reconnect is a 3 s
`QTimer`. Both Qt5 and Qt6 ship the WebSockets module (verified present).
4. **Qt5/6 autodetect in CMake.**
`find_package(QT NAMES Qt6 Qt5 REQUIRED COMPONENTS Widgets WebSockets)` then
`find_package(Qt${QT_VERSION_MAJOR} ...)`; link `Qt${QT_VERSION_MAJOR}::*`.
This is the upstream-recommended idiom; prefers Qt6 when present, falls back
to Qt5. `CMAKE_AUTOMOC ON`.
5. **Reuse, do not fork, the wire layer.**
`Protocol.h`, `Protocol.cpp`, and `SignalBuffer.h` in `Client/streamhub/` are
pure C++17/STL with **no ImGui/SDL dependency** (verified). The Qt project
compiles `../streamhub/Protocol.cpp` directly and adds `../streamhub` to its
include path. Single source of truth for the protocol — any future protocol
change is picked up by both clients. The only ImGui type leaking into the
reference *domain* model is `ImVec4 color`; the Qt project defines its own
domain model (`Model.h`) using `QColor` instead.
## Threading model
Single-threaded around the Qt event loop:
- `QWebSocket` delivers frames as signals on the GUI thread → parsed via the
reused `Protocol` functions → mutate the model directly (no locks needed,
unlike the ImGui version whose WS ran on a background thread).
- A `QTimer` at ~60 Hz calls `update()` on visible `PlotWidget`s for smooth
live scrolling (decoupled from data arrival rate).
This is simpler and safer than the ImGui client's background-thread + drain-queue
model, and is the idiomatic Qt approach.
## Module / file structure (`Client/streamhub-qt/`)
| File | Responsibility |
|------|----------------|
| `CMakeLists.txt` | Qt5/6 autodetect, sources, reuse `../streamhub/Protocol.cpp`, install rules |
| `main.cpp` | `QApplication`, arg parse (`-host`/`-port`), apply dark theme, show `MainWindow` |
| `Theme.h/.cpp` | Catppuccin-Mocha `QPalette` + stylesheet, trace color palette |
| `Model.h` | Domain types: `Signal` (with `QColor`), `Source`, `TriggerCfgState`, `VScale`, `PlotAssignment`, `PlotLayout` enum + dims |
| `Hub.h/.cpp` | `QObject` controller: owns `QWebSocket`, model (`std::vector<Source>`), trigger state, capture, zoom caches, history info. Parses WS messages (reused `Protocol`), exposes command senders, emits Qt signals (`sourcesChanged`, `configChanged`, `statsChanged`, `triggerStateChanged`, `captureReceived`, `zoomReceived`, `historyInfoChanged`, `connectedChanged`). |
| `WsClient.h/.cpp` | Thin `QWebSocket` wrapper: connect/reconnect (`QTimer`), `sendText`, `textReceived`/`binaryReceived`/`connectedChanged` signals |
| `PlotWidget.h/.cpp` | Custom `QPainter` oscilloscope panel (one per grid cell). Owns per-plot view state (live/window, stored X range, zoom history, vMode, active slot, zoom caches, paused snapshot, trig-zoom). Renders, handles mouse zoom/pan + drag-drop, draws cursors, requests hi-res/history zoom through `Hub`. |
| `PlotGrid.h/.cpp` | Grid of `PlotWidget`s using nested `QSplitter`s per `PlotLayout`; rebuilds on layout change; routes drops |
| `SourceSidebar.h/.cpp` | `QTreeWidget` of sources→signals (drag source), connect/remove, "Add source" dialog launcher |
| `TriggerBar.h/.cpp` | Trigger config widgets (signal combo, edge, threshold, window combo, pre% slider, normal/single) + Arm/Disarm/Rearm/Stop + state badge |
| `StatsDialog.h/.cpp` | Per-source metrics table + 20-bin histogram (custom `QPainter` bar widget) |
| `HistoryBar.h/.cpp` | Live/range/pan/jump-preset/All history navigation |
| `MainWindow.h/.cpp` | `QMainWindow`: toolbar (sidebar toggle, layout picker, pause, cursors, trigger, history, window presets, stats, connection, status LED), assembles sidebar + grid + bars, owns `Hub`, the 60 Hz repaint `QTimer`, and the add-source dialog |
## Feature parity checklist (must match the ImGui client)
- Live wall-clock X window with presets {1,5,10,30,60 s} + Ctrl+scroll resize.
- Plot layouts {1×1, 2×1, 1×2, 3×1, 1×3, 2×2, 4×1, 1×4} via splitters.
- Drag a signal from the sidebar onto a plot; per-trace color/width/marker via
context menu; remove-from-plot; "active" trace selection.
- V-scale per trace: Auto / Range / Manual (V/div, offset, screen pos); Y ticks
labelled from the active trace; ±4-division fixed Y space.
- Per-plot V-mode: Normal / Digital / Mixed band normalization (ports
`applyDigitalNorm`/`applyMixedNorm` math already in `PlotPanel.cpp`).
- Mouse: scroll = Y zoom of active trace (or X zoom if none); Ctrl+scroll = X
zoom / window resize; Shift+scroll = Y pan; right-drag = X pan (live→non-live).
- Zoom history Back / Fit / Live per plot.
- Global pause (freeze view snapshot; rings keep filling); global A/B cursors
with ΔT, 1/ΔT, per-trace A/B/Δ readouts.
- Hi-res zoom over WS (`zoom` cmd, 2400 pts) for live-narrow and zoomed views;
history zoom (`historyZoom`) for disk-backed ranges; "HIST" indicator.
- Trigger: full config → `setTrigger`; Arm/Disarm/Rearm/Stop; state badge
(idle/armed/collecting/triggered); v2 capture frame → capture view rendered in
`[-pre,+post]` with trigger marker at t=0; capture-view zoom + Reset.
- Stats dialog: state, totalReceived, totalLost, rateHz±std, frags/bytes per
cycle, cycle avg/std/min/max ms, 20-bin cycle-time histogram.
- Add/remove source; connection host/port edit + reconnect; status LED.
- LTTB decimation to ≤2400 pts/trace before drawing (reuse `LTTBDecimate`).
## Build & run
```bash
cd Client/streamhub-qt
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j
./build/StreamHubQtClient -host 127.0.0.1 -port 8090
```
Verification: run `./run_streamhub.sh` (or the recorder E2E streamer) to get a
live hub, then confirm live streaming, zoom (live + box), trigger capture, stats
histogram, and history browse all behave like the ImGui client.
## Out of scope
- No new protocol features; byte-for-byte the same WS protocol.
- Sub-project E (Go `Client/debugger/` → Qt) is a separate spec.
- Font Awesome glyph icons are replaced by Qt standard icons / unicode; exact
icon glyphs are not required for parity.
@@ -0,0 +1,167 @@
# Streaming-Chain E2E Test Suite — Design
**Date:** 2026-06-25
**Status:** Approved (autonomous implementation authorised)
**Sub-project:** A (StreamHub full-chain waveform E2E)
## Goal
A curated end-to-end test suite that validates the streaming chain
**MARTe2 App (`UDPStreamer`) → `StreamHub` → client** with *waveform* validation,
across a covering matrix of `UDPStreamer` configuration options, plus
StreamHub↔client behavioural tests (live stream, zoom in/out, time window, all
trigger modes). Output: one combined PDF report **and** a machine-readable
`results.json`.
## Decisions (from brainstorming)
1. **Matrix strategy** — curated "covering" set (~4070 scenarios): every option
value appears at least once, plus deliberately chosen high-risk interactions.
2. **Oracle** — hybrid, applied per case:
- **Analytic (A)**: clean cases reconstruct expected values from the
generator formula on the received timestamps; tolerance
`max|err| ≤ quant_step/2 + eps`; decimated streams assert only the samples
that should survive; LTTB-decimated views use a shape metric (RMSE /
correlation), not point equality.
- **Fed-reference (B)**: subtle modes (Accumulate, FullArray, First/LastSample)
compare received-vs-fed using a `FileWriter` tap of the exact signals MARTe
fed into the `UDPStreamer`.
3. **Client testing** — Go mock client (grown from `Test/E2E/streamhub/main.go`)
is the authoritative functional/waveform gate; plus a thin Qt `QTest` GUI
smoke test (offscreen) for GUI-level confidence.
4. **"to disk to client"** — the mock client persists the stream it received to
disk as a report artifact for offline comparison/plots.
5. **Output** — one orchestrator + one combined PDF **+** `results.json`.
## Architecture
New directory `Test/E2E/chain/`. A shell entry point delegates the matrix to a
Python driver, matching the existing `Test/E2E/datasources/` and
`Test/E2E/recorder/` conventions (shell + Python + Typst). Artifacts go to
`Build/x86-linux/E2E/chain/` (never the source tree).
### Two-process stack (per scenario)
```
gen_data.py -> /tmp/chain_e2e/input_<id>.bin (ground truth + fed reference)
gen_cfg.py -> marte_<id>.cfg (LinuxTimer + FileReader -> IOGAM -> UDPStreamer
[+ optional FileWriter tap])
-> hub_<id>.cfg (StreamHub Source pointing at the UDPStreamer)
StreamHub.ex -cfg hub_<id>.cfg [process 1, started first]
MARTeApp.ex -l RealTimeLoader -f marte_<id>.cfg -s Running [process 2]
chain-client (Go) -hub 127.0.0.1:<wsport> -scenario <id> -out <dir>
-> records live pushes, runs zoom/window/trigger checks,
dumps received_<id>.bin + checks_<id>.json
validate_waveform.py input_<id>.bin received_<id>.bin [+ tap_<id>.bin]
-> metrics_<id>.json (+ overlay/diff plots)
```
### Components & responsibilities
| File | Responsibility |
|------|----------------|
| `Test/E2E/chain/scenarios.py` | Declarative list of ~50 scenario dicts (the covering set) + validity rules |
| `Test/E2E/chain/gen_data.py` | scenario → deterministic `input_<id>.bin` for all 10 MARTe2 types & shapes; knows the analytic formula per signal |
| `Test/E2E/chain/gen_cfg.py` | scenario → MARTe2 app `.cfg` and StreamHub `.cfg` (unicast/multicast, Strict/Accumulate/Decimate, per-signal type/shape/timemode/quant, optional FileWriter tap) |
| `Test/E2E/chain/client/` | Go mock client (grown from `streamhub/main.go`): record live, zoom, window, all trigger modes; dump `received_<id>.bin` + `checks_<id>.json` |
| `Test/E2E/chain/validate_waveform.py` | ground-truth/fed-reference vs received; per-effect tolerance; emit `metrics_<id>.json` |
| `Test/E2E/chain/plots.py` | matplotlib overlays (truth/received/diff), trigger-capture, zoom/window figures |
| `Test/E2E/chain/run_chain_e2e.sh` | orchestrator: build, loop scenarios over the two-process stack, validate, aggregate `results.json`, compile PDF |
| `Test/E2E/chain/E2E_Report.typ` | Typst report template → `E2E_Report.pdf` |
| `Client/streamhub-qt/test/` | Qt `QTest` smoke (offscreen), run via ctest |
## Covering matrix (~50 scenarios)
Grouped so each option value is hit at least once, with targeted interactions.
Config-surface reference (verified against source):
- **Publishing modes**: `Strict`, `Accumulate` (+`MinRefreshRate`), `Decimate` (+`Ratio`).
- **Network**: unicast (`Port`); multicast (`MulticastGroup` + `DataPort`).
- **Signal types**: uint8/int8/uint16/int16/uint32/int32/uint64/int64/float32/float64.
- **Time modes** (per signal): `PacketTime`, `FullArray`, `FirstSample`,
`LastSample` (+ `TimeSignal`, `SamplingRate`).
- **Quantisation** (float only): `none`, `uint8`, `int8`, `uint16`, `int16`
(+`RangeMin`/`RangeMax`).
- **Shapes**: scalar, 100-element array, 5000-element array (fragmentation).
- **Payload**: default 1400, small (512, forces fragmentation), jumbo (65507).
- **Sources**: single and multiple sources (two UDPStreamer blocks → two hub sources).
Representative buckets (illustrative, ~50 total):
- Per-type scalar Strict unicast (10) — type fidelity.
- Time-mode sweep on arrays (FullArray/FirstSample/LastSample/PacketTime × float32/uint64-time) (~8).
- Quantisation sweep (none/uint8/int8/uint16/int16 with matching RangeMin/Max) (~6).
- Publishing modes (Strict/Accumulate@{10,100,1000Hz}/Decimate@{1,5,10}) (~8).
- Multicast variants of the above (~6).
- Shapes/fragmentation (scalar/100/5000 × payload 512/1400/65507) (~6).
- Multi-source (23 sources, mixed configs) (~3).
- Edge/robustness (mixed-type multi-signal source, large array + quant) (~3).
## Oracle details
Ground truth is the analytic generator. For a received sample at time `t` for
signal `s`, the expected value is `f_s(t)` reconstructed from the generator's
per-signal formula and the row/cycle clock.
- **Type fidelity (no quant)**: integer/float types must be bit-exact after the
type round-trip (`err == 0`).
- **Quantised float**: `quant_step = (RangeMax-RangeMin)/levels` (levels = 255,
254, 65535, 65534 for uint8/int8/uint16/int16). Require
`max|recv truth| ≤ quant_step/2 + 1e-6·range`.
- **Decimate (Ratio R)**: only every R-th producer cycle is transmitted; assert
the surviving subsequence matches truth; received count ≈ produced/R.
- **Accumulate**: batched samples must reconstruct the original per-cycle
sequence within float eps; validated against the **fed reference** tap.
- **Time modes**: reconstructed timestamps must be monotonic, within wall-clock
bounds, and spaced by `1/SamplingRate` (First/LastSample) or equal the time
array (FullArray); compared to fed reference where reconstruction is subtle.
- **LTTB live/zoom views**: shape metric — Pearson correlation ≥ 0.99 and
normalised RMSE ≤ tolerance against truth resampled onto returned timestamps.
## Client behavioural validation (Go mock, authoritative)
Per scenario the client performs and records:
- **Live**: ≥N binary v1 frames; per-signal wall-clock + monotonic time; dump
`received_<id>.bin`; waveform vs truth (shape metric).
- **Zoom**: ≥2 ranges (narrow "in" and wide "out"); points within `[t0,t1]`;
count ≤ requested `n`; shape vs truth. Also `historyZoom` when history enabled.
- **Time window**: set `windowSec`; assert returned/clamped span matches.
- **Triggers**: matrix of `edge ∈ {rising,falling,both}` × `mode ∈
{normal,single}` on an oscillating signal; validate triggerState transitions,
v2 capture window `[trigTimepre, trigTime+post]`, the edge actually crosses
threshold in the correct direction at `trigTime`, and captured waveform vs
truth. Normal mode must re-arm; single must not until `rearm`.
- Emit `checks_<id>.json` (per-check pass/fail + measured values).
## Qt GUI smoke (thin)
`Client/streamhub-qt/test/smoke_test.cpp` using `QTest`, `QT_QPA_PLATFORM=offscreen`:
construct `MainWindow`, point at a live StreamHub, spin the event loop, assert it
received ≥1 source and rendered ≥1 frame, exercise a layout change and pause
toggle without crashing. Built via the existing CMake; run via `ctest`.
## Report
`E2E_Report.typ` → `E2E_Report.pdf`:
- Title + environment + matrix summary table (scenario, config, oracle, result).
- Per-scenario waveform overlays (truth / received / diff) for a representative
subset; aggregate pass/fail.
- Trigger-capture plots (one per edge/mode).
- Zoom/window plots.
- Qt smoke result.
`results.json` — aggregate machine-readable: list of scenarios with config
summary, per-check metrics, pass/fail, and overall status (for CI gating).
## Failure & robustness
- Each scenario runs under `timeout`; stack always torn down via trap.
- A scenario that fails to produce data fails that scenario only; the suite
continues and the report records it.
- Multicast scenarios are skipped (recorded as SKIP) if the loopback multicast
route is unavailable, rather than hanging.
## Out of scope (covered elsewhere)
- Per-component standalone Typst reports → Sub-project C.
- DebugService E2E → Sub-project B.
- Real GUI click-automation beyond the QTest smoke.