Implemented hearthbit client side + tests

This commit is contained in:
Martino Ferrari
2026-08-09 18:51:51 +02:00
parent 915a192b16
commit 1ddb4fe356
12 changed files with 994 additions and 285 deletions
+199 -271
View File
@@ -1,313 +1,241 @@
# AGENTS.md
# Repository Guidelines
Guide for agents working in the MARTe2 Integrated Components repository. Read
`ARCHITECTURE.md` and `CLAUDE.md` for deeper detail; this file focuses on
non-obvious knowledge, commands, and conventions that are not self-evident from
a single file read.
Guide for AI assistants working in the MARTe2 Integrated Components repository.
Focuses on non-obvious facts: commands, conventions, cross-module contracts, and
gotchas that are not self-evident from a single file read.
---
## Project Overview
## Repository at a glance
MARTe2 component library with **two independent real-time data paths** sharing
one binary wire protocol (`Common/UDP/UDPSProtocol.h`):
MARTe2 component library with two independent real-time data paths:
1. **Streaming path**`UDPStreamer` DataSource (serialises signals to UDPS
binary packets on UDP 44500) → `StreamHub` (headless C++ hub: ring buffers,
LTTB decimation, trigger FSM, history writer) → WebSocket 8090 → clients
(browser SPA, native ImGui, native Qt).
2. **Debug path**`DebugService` Interface patches `ClassRegistryDatabase` at
1. **Streaming path**`UDPStreamer` DataSource serialises DDB signals into UDPS
binary packets on UDP → `StreamHub` (headless C++ hub: ring buffers, LTTB
decimation, trigger FSM, history writer, binary recorder) → WebSocket 8090 →
clients (browser SPA, native ImGui, native Qt).
2. **Debug path**`DebugService` patches `ClassRegistryDatabase` at
`Initialise()` so `ConfigureApplication()` wraps all `MemoryMap*Broker` types
with `DebugBrokerWrapper<T>`. Zero application code changes. Exposes TCP 8080
(text commands), UDP 8081 (trace telemetry), TCP 8082 (`TcpLogger`).
with `DebugBrokerWrapper<T>`**zero application code changes**. Exposes
TCP 8080 (text commands), UDP 8081 (UDPS trace telemetry), TCP 8082
(`TcpLogger` log forward).
Key shared artefact: `Common/UDP/UDPSProtocol.h` — the UDPS binary wire format
(17-byte packed header, 136-byte signal descriptors, little-endian). It is
deliberately MARTe2-free and is mirrored across **four** codebases that must stay
in sync: C++ producers (`UDPStreamer`, `DebugService`), C++ consumer
(`Source/Components/Interfaces/UDPStream/UDPSClient`), Go decoder
(`Common/Client/go/udpsprotocol`), and the JS client parsers. Any protocol
change must be mirrored in all of them.
## Architecture & Data Flow
StreamHub WebSocket protocol: JSON text frames for commands/events, binary
frames for data pushes (spec in `ARCHITECTURE.md` §6). The Go hub
(`Client/udpstreamer`) and the C++ StreamHub implement the identical protocol;
browser JS, ImGui, and Qt clients must stay compatible with both.
```
[SineArrayGAM/TimeArrayGAM] → DDB → UDPStreamer (UDPS over UDP)
├─→ UDPStreamerClient (input DS back into a MARTe2 RT app, round-trip)
└─→ StreamHub: UDPSourceSession (receive thread → SignalRingBuffer)
→ push loop @30Hz: LTTB decimate temporal sigs → WS binary frames → clients
DebugService: patches broker builders at Initialise(); TCP 8080 commands,
UDP 8081 telemetry, TcpLogger 8082 (REPORT_ERROR → "LOG <LEVEL> <desc>" lines)
```
---
- **Wire protocol**: `Common/UDP/UDPSProtocol.h` is the canonical spec (17-byte
packed header, magic `0x53504455` 'UDPS', 136-byte signal descriptors,
CONFIG/DATA/ACK/CONNECT/DISCONNECT packet types, quant/time/publish modes).
Deliberately MARTe2-free so Go clients reuse it. **Mirrored across four
codebases that must stay in sync**: C++ producers (UDPStreamer, DebugService),
C++ consumer (`Source/Components/Interfaces/UDPStream/UDPSClient`), Go decoder
(`Common/Client/go/udpsprotocol/protocol.go`), and JS parsers
(`Client/udpstreamer/static/`, `Client/debugger/static/`). Any protocol change
must be mirrored in all of them.
- **WS protocol** has two implementations — Go hub (`Common/Client/go/wshub`) and
C++ StreamHub — that must behave identically; every client (SPA, ImGui, Qt)
must satisfy both. JSON text frames for commands/events (`addSource`,
`removeSource`, `setTrigger`, `arm`, `zoom`, `historyZoom`, `recStart`…), binary
frames for data pushes (live v1 + trigger capture v2).
- **Threading model**: RT threads only spinlock+memcpy (`FastPollingMutexSem`);
all socket I/O, fragmentation, and reassembly lives on background
`SingleThreadService` threads. StreamHub: per-session UDPSClient receive
threads + WS accept/read threads + one push loop.
- **DebugService patching**: `PatchRegistry()` replaces the ObjectBuilder for 11
`MemoryMap*Broker` classes; runs only when `ControlPort > 0`; static guard
against double-patching; wrappers persist for process lifetime.
## Environment setup (do this first, always)
## Key Directories
`source env.sh` is **required** before building *or* running anything. It sets:
| Path | Purpose |
|---|---|
| `Source/Components/DataSources/UDPStreamer/` | Output DataSource; UDP I/O on bg thread, RT thread only spinlock+memcpy in `Synchronise()` |
| `Source/Components/DataSources/UDPStreamerClient/` | Input DataSource (shared `UDPSClient`), double-buffered ready/scratch |
| `Source/Components/GAMs/` | `SineArrayGAM` (float32 sine, continuous phase), `TimeArrayGAM` (us-timer → per-sample timestamp array) |
| `Source/Components/Interfaces/DebugService/` | Registry patching, `DebugBrokerWrapper.h`, TCP/UDP services |
| `Source/Components/Interfaces/TCPLogger/` | `LoggerConsumerI` forwarding `REPORT_ERROR` to ≤8 TCP clients |
| `Source/Components/Interfaces/UDPStream/` | Plain-C++ helpers (not MARTe2 Objects): `UDPSClient` (auto-reconnect + fragment reassembly), `UDPSServer` (not thread-safe — owner's Execute thread only) |
| `Source/Applications/StreamHub/` | Standalone app (links MARTe2 core): `StreamHub`, `UDPSourceSession`, `WSServer`, `TriggerEngine`, `HistoryWriter`, `BinaryRecorder`, `LTTB`, `SignalRingBuffer` |
| `Common/UDP/` | Canonical wire protocol (header-only, MARTe2-free) |
| `Common/Client/go/` | Go mirror: `udpsprotocol` (decoder), `wshub` (WS hub client) |
| `Client/udpstreamer/` | Go legacy direct-UDP oscilloscope web UI (connects straight to UDPStreamer, no StreamHub) |
| `Client/webui/` | Go thin static server; SPA talks WS directly to C++ StreamHub (discovers via `GET /hub`) |
| `Client/debugger/` | Go debug web UI for DebugService |
| `Client/streamhub/` | Native ImGui+SDL2+OpenGL oscilloscope (C++17, no MARTe2) |
| `Client/streamhub-qt/` | Native Qt Widgets oscilloscope (Qt6 preferred, Qt5 fallback) |
| `Test/` | GTest, legacy Integration tests, Configurations (.cfg), E2E suite |
| `Docs/` | Per-component reference: `Protocol.md`, `UDPStreamer.md`, `StreamHub-{API,UserGuide,Developer}.md`, `DebugService.md`, `WebUI.md`, `Tutorial.md`, `E2E-Suite.md` |
- `MARTe2_DIR` (default `~/workspace/MARTe2`) and `MARTe2_Components_DIR`
(default `~/workspace/MARTe2-components`) — external dependencies, not in
this repo. Edit `env.sh` if your MARTe2 install lives elsewhere.
- `TARGET=x86-linux`
- `LD_LIBRARY_PATH` — prepends MARTe2 Core, MARTe2-components (LinuxTimer,
FileDataSource), and this repo's built `.so`s (UDPStreamer, SineArrayGAM,
TimeArrayGAM, DebugService, TCPLogger).
## Development Commands
Without sourcing `env.sh`, builds fail (can't find `MakeDefaults`) and binaries
fail to load shared libs. The E2E scripts source it themselves, but a bare
`make` or `./MainGTest.ex` from a fresh shell will not.
---
## Build commands
All C++ builds use the MARTe2 `Makefile.gcc` wrapper system. There is one
top-level `Makefile.gcc` and one per component directory. `TARGET` defaults to
`x86-linux`; pass `TARGET=x86-linux` explicitly when invoking `make -C` from
elsewhere.
`source env.sh` is **mandatory** before any MARTe2 build or run (sets
`MARTe2_DIR`, `MARTe2_Components_DIR`, `TARGET=x86-linux`, `LD_LIBRARY_PATH`).
The E2E scripts source it themselves; a bare `make` from a fresh shell will not
work. `run_streamhub.sh` hard-errors if `MARTe2_DIR` is unset.
```bash
source env.sh
make -f Makefile.gcc core # all C++ MARTe2 components (UDPStreamer, GAMs, DebugService, TCPLogger, UDPStream)
make -f Makefile.gcc apps # StreamHub standalone app
make -f Makefile.gcc test # build GTest + Integration test binaries
make -f Makefile.gcc core # 7 components (UDPStream interface FIRST, then UDPStreamer, UDPStreamerClient, GAMs, TCPLogger, DebugService)
make -f Makefile.gcc apps # StreamHub standalone app → Build/x86-linux/StreamHub/StreamHub.ex
make -f Makefile.gcc test # GTest + Integration test binaries + component test libs
make -f Makefile.gcc all # core + apps + test
make -f Makefile.gcc clean
make -f Makefile.gcc # = all: core apps test
# Build a single component (each component dir has its own Makefile.gcc):
make -C Source/Components/DataSources/UDPStreamer -f Makefile.gcc
make -C Source/Applications/StreamHub -f Makefile.gcc
# Single component:
make -C Source/Components/GAMs/SineArrayGAM -f Makefile.gcc
```
Build output goes to `Build/x86-linux/` — shared libs under
`Build/x86-linux/Components/<...>/`, the StreamHub executable at
`Build/x86-linux/StreamHub/StreamHub.ex`, test binaries under
`Build/x86-linux/GTest/` and `Build/x86-linux/Test/Integration/`.
Build output `Build/x86-linux/` mirroring `PACKAGE` paths (both `libX.so` and
`X.so` are produced). `compile_commands.json` (repo root, gitignored) feeds
LSP/clangd; CMake clients export their own into `Client/*/build/`.
`compile_commands.json` is generated at the repo root (gitignored) for LSP /
clangd. The CMake-based clients (ImGui, Qt) also export it into their `build/`.
### Non-MARTe2 clients (separate build systems, no env.sh needed)
### Non-MARTe2 clients (no env.sh needed)
```bash
# Go clients (UDPS web client, debug client, E2E chain-client)
cd Common/Client/go && go build ./...
cd Client/debugger && go build ./...
cd Client/udpstreamer && go build ./... # or: cd Client/webui && go build
cd Test/E2E/suite/client && go build ./... # chain-client (E2E driver) + its tests
# ImGui desktop client (needs SDL2; fetches Dear ImGui + ImPlot via FetchContent)
cd Client/debugger && go build ./...
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
# Qt desktop client (Qt5 or Qt6 Widgets + WebSockets; autodetects, prefers Qt6)
cd Client/streamhub-qt && cmake -B build && cmake --build build
```
---
### Key scripts
## Run / test commands
### Unit & integration tests (C++)
```bash
./Build/x86-linux/GTest/MainGTest.ex # UDPStreamer unit tests
./Build/x86-linux/GTest/MainGTest.ex --gtest_filter='Name*' # single test
./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex # DebugService integration tests
```
### Go tests
```bash
cd Test/E2E/suite/client && go test ./...
```
### Python framework tests (E2E framework logic, no live stack needed)
```bash
cd Test/E2E/suite && python3 -m unittest tests_py
```
### E2E / demo scripts (build + launch the full stack)
| Script | What it does |
| Script | Purpose |
|---|---|
| `./run_streamhub.sh` | Launch MARTe2 app (UDPStreamer) + StreamHub, optionally web UI (`-w`) and ImGui client (`-g`). Ports documented in its header. |
| `./Test/E2E/suite/run_e2e.sh` | **Unified E2E suite**: per-scenario generates data + cfgs, runs MARTe2+StreamHub, drives the Go `chain-client` (live/zoom/window/trigger) for `chain` scenarios plus `direct`/`recorder`/`debug`/`tcplogger` scenario kinds, runs a stress matrix, unit suites + coverage, builds a Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--pdf-only`, `--cpp-coverage`, `--skip-coverage`, `--skip-stress`, `--skip-datasources`, `--skip-recorder`, `--skip-debug`, `--skip-tcplogger`. |
| `./Test/E2E/suite/run_stress.sh` | **Capacity/stress harness**: sweeps one load axis at a time (signal size/count, subscriber fan-out, source count, WS-client count, zoom rate), gates on survival+liveness (hard) and RSS+zoom-p95 latency (soft). Flags: `--skip-build`, `--only <id>`, `--axis <axis>`. |
| `./run_streamhub.sh` | Demo stack: build + launch MARTe2 app + StreamHub, optional web (`-w`) / ImGui (`-g`) clients. Flags `-m/-c` MARTe2 dirs, `-b TARGET`, `-p WS_PORT`, `-n MAX_POINTS` (actual default 1000000, header says 10000), `-s` skip build. Generates temp hub cfg with `+History`/`+Recorder` blocks in `/tmp`. Ctrl-C kills all. |
| `./Test/E2E/suite/run_e2e.sh` | Full E2E: 57-scenario matrix + stress + unit suites + gcov coverage + Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--pdf-only`, `--skip-coverage`, `--skip-stress`, `--skip-datasources`, `--skip-recorder`, `--skip-debug`, `--skip-tcplogger` |
| `./Test/E2E/suite/run_stress.sh` | Capacity harness: sweeps one load axis at a time (`--axis`), hard gates survival+liveness, soft gates RSS+zoom-p95 |
The E2E suite produces `Build/x86-linux/E2E/chain/` artifacts: `report_data.json`,
`history.jsonl` (per-run headline metrics for trend/regression tracking),
`trend_*.png` plots, and `E2E_Report.pdf` (compiled from `E2E_Report.typ` via
`typst`). `--cpp-coverage` triggers an instrumented `--coverage` rebuild, captures
with `lcov` (restricted to `Source/*` + `Test/*`), then restores a clean build.
## Code Conventions & Common Patterns
---
- **No STL in `Source/Components/**` (and StreamHub)**: use `StreamString` (not
`std::string`), `FastPollingMutexSem`/`EventSem` (not `std::mutex`/threads),
fixed arrays / MARTe2 `Vector<T>` (not `std::vector`), `REPORT_ERROR` /
`REPORT_ERROR_STATIC` macros (no exceptions). C stdlib is fine. Heap
`new`/`delete[]` is normal. STL/C++17 is fine in `Client/streamhub/` and
`Client/streamhub-qt/`.
- **RT hot-path rule**: `FastPollingMutexSem` on real-time hot paths, never OS
mutexes; RT cycle must not block on the scheduler.
- **Class registration**: `CLASS_REGISTER_DECLARATION()` in the class `public:`
section of the header; `CLASS_REGISTER(Name, "1.0")` at the end of the `.cpp`
inside `namespace MARTe`. Every component `.cpp` ends with it.
- **EUPL v1.1 license headers** on all C++ sources and `Makefile.inc` — preserve
on new files.
- **Per-component build**: each dir has one-line `Makefile.gcc` wrapper
(`include Makefile.inc`) + `Makefile.inc` declaring `OBJSX`, `PACKAGE`,
`ROOT_DIR`, `INCLUDES` (re-declared per file, ~12 MARTe2 layer dirs),
`LIBRARIES`, including `MakeStdLibDefs.$(TARGET)` then
`MakeStdLibRules.$(TARGET)`. Generated `depends.x86-linux` (gcc -MM) is
committed but **never hand-edited** — delete to regenerate.
- **Qt client**: `QT_NO_KEYWORDS` is required (reused `Protocol.h` structs have
members named `signals`); Qt classes use `Q_SIGNALS`/`Q_SLOTS`/`Q_EMIT`. Run
with long options: `--host HOST --port 8090` (single-dash misparsed). Single
GUI thread, 60 Hz QTimer repaint.
- **StreamHub config** is *not* a MARTe2 `RealTimeApplication`: `Hub = { WSPort
MaxPoints PushRate MaxPushPoints RingTemporal RingScalar +Recorder{...}
Sources={id={Label Addr Port}} }`. `+History` keys: `Directory` (required),
`DurationHours` (1), `Decimation` (1), `FlushIntervalSec` (5),
`MinDiskFreeMB` (500). `.shist` files: 64-byte header ('SHR1') + circular
(t,v) float64 pairs.
- **UDPStreamer config**: `Port` (44500; multicast data = `DataPort`, default
`Port+1`), `MaxPayloadSize` (1400), `PublishingMode` `Strict`/`Accumulate`,
per-signal `Signals={Name={Type,Unit,NumberOfDimensions,NumberOfElements,
TimeMode}}` with `TimeMode` `PacketTime`/`FirstSample`/`LastSample`/`FullArray`;
multicast needs `MulticastGroup` + `Interface`.
## Code organization
## Important Files
```
Common/
UDP/UDPSProtocol.h # shared UDPS binary protocol (header-only, MARTe2-free)
Client/go/ # Go packages: udpsprotocol (decoder), wshub (WS hub client)
Source/
Components/ # MARTe2 components — NO STL allowed here
DataSources/UDPStreamer/ # real-time UDP signal streaming DataSource
DataSources/UDPStreamerClient/ # (MARTe2-side UDPS client DataSource)
GAMs/SineArrayGAM/ # sine-wave array generator GAM
GAMs/TimeArrayGAM/ # time-reference array GAM
Interfaces/DebugService/ # tracing/forcing/breakpoint Interface (registry-patched brokers)
Interfaces/TCPLogger/ # REPORT_ERROR → TCP forwarder (LoggerConsumerI)
Interfaces/UDPStream/ # UDPSClient/UDPSServer (C++ consumer/producer of UDPS)
Applications/StreamHub/ # headless C++ hub app (links MARTe2 core but follows MARTe2 style)
Client/
debugger/ # Go web client for DebugService (browser UI in static/)
udpstreamer/ # Go web server + static SPA for UDPStreamer (legacy direct-UDP path)
webui/ # Go StreamHub WebSocket web UI (newer, hub-based)
streamhub/ # native ImGui desktop client (SDL2 + OpenGL + ImPlot, C++17, no MARTe2)
streamhub-qt/ # native Qt Widgets desktop client (C++17, Qt5/Qt6, no MARTe2)
Test/
GTest/ # GTest harness (MainGTest.cpp) for UDPStreamer unit tests
Integration/ # DebugService integration tests (link against DebugService .so)
Components/DataSources/UDPStreamer/ # UDPStreamer unit test sources
Configurations/ # MARTe2 .cfg files for tests and demos
E2E/
chain/ # streaming-chain E2E + stress suite (Python orchestrator + Go client)
datasources/ recorder/ streamhub/ # older per-component E2E scripts
Docs/ # per-component reference docs (Protocol, UDPStreamer, StreamHub-*, DebugService, ...)
docs/superpowers/{specs,plans}/ # design specs + implementation plans (dated)
ARCHITECTURE.md # full architecture (data flow diagrams, protocol tables, WS protocol)
- `env.sh` — environment; source first, always.
- `Makefile.gcc` / `Makefile.inc` (root) — build orchestration.
- `Common/UDP/UDPSProtocol.h` — canonical wire format; changing it triggers the
4-way mirror checklist above.
- `Source/Applications/StreamHub/main.cpp` — hub entry (`[-cfg file.cfg]
[-port N] [-maxPoints N]`); hub **must be heap-allocated** (~128 MB, exceeds
the 8 MB stack).
- `Test/Configurations/*.cfg` — MARTe2 app configs (`$App = { Class =
RealTimeApplication }` with `+Functions`, `+DataSources`, `+States`, `+Timings`
blocks); `streamhub_demo.cfg` and `TestApp.cfg` are good templates.
- `Test/E2E/suite/{scenarios,gen_data,gen_cfg,validate_waveform,stress}.py` —
declarative scenario matrix and generators consumed identically by the Go
chain-client and validators.
- `Client/debugger/main.go` — `-addr :7777` default, `-enable-dangerous-commands`
safety gate (CR-4) for FORCE/PAUSE/RESUME/STEP/BREAK/MSG.
## Runtime/Tooling Preferences
- **OS**: Linux x86_64 (`TARGET=x86-linux`). External deps live outside this
repo: `MARTe2_DIR` (default `~/workspace/MARTe2`) and
`MARTe2_Components_DIR` (default `~/workspace/MARTe2-components`) — edit
`env.sh` if they differ. `env.sh`'s `LD_LIBRARY_PATH` does **not** cover
UDPStreamerClient/UDPStream lib dirs.
- **C++**: MARTe2 `Makefile.gcc` wrapper system, gtest-1.7.0 for tests.
- **Go**: `go 1.21`; modules use `replace marte2/common => ../../Common/Client/go`
(`gorilla/websocket` v1.5.1). Go binaries are gitignored.
- **ImGui client**: needs SDL2; CMake FetchContent pins Dear ImGui **v1.91.8** +
ImPlot **v0.17** (`implot_items.cpp` is a slow -O3 TU, ~2 min rebuild).
- **Qt client**: Qt6 preferred, Qt5 fallback, Widgets + WebSockets, custom
QPainter plotting (no QtCharts).
- **E2E report**: `typst compile E2E_Report.typ`; Python 3 + numpy for the suite.
- Remove `vgore.*` core dumps when you see them; they are not gitignored.
## Testing & QA
Four test layers; `env.sh` + built stack required for all but the standalone
ones. Only `tests_py.py`, Go tests, and the built C++ test binaries run
standalone.
```bash
./Build/x86-linux/GTest/MainGTest.ex --gtest_filter='Name*' # C++ GTest
./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex # legacy DebugService runtime tests
cd Test/E2E/suite/client && go test ./... # Go chain-client unit tests
cd Test/E2E/suite && python3 -m unittest tests_py # framework logic, standalone
```
### Component layout convention
- **GTest**: `MainGTest.ex` currently holds only `DebugServiceGTest`
(TraceRingBuffer SPSC, DebugSignalInfo, BreakOp). Component GTests
(`UDPStreamerGTest.cpp` ~46 cases, `StreamHubTest.a`, `UDPStreamerClientTest.a`)
compile **as libraries only — no standalone executable**.
- **Legacy IntegrationTests.ex**: 9 printf-narrated DebugService runtime tests,
always returns 0; `collect.py` parses stdout blocks.
- **E2E suite** (`run_e2e.sh`): 57 curated scenarios (s01s57) across kinds
`chain`/`direct`/`recorder`/`debug`/`debug_pause_resume`/`tcplogger`, driven
against live MARTeApp.ex + StreamHub.ex + Go chain-client. `scenarios.py` is a
curated covering set: **every configurable UDPStreamer option value appears in
≥1 scenario** — add a scenario when adding an option.
- **Oracle gates** (`validate_waveform.py`): **fidelity** (every received value
within `tol` of ground truth; 0 for un-quantised ints, float epsilon for
un-quantised floats, `quant_step/2 + 1e-6·range` for quantised) is the
**correctness gate**. **Shape** is a *gross* sanity gate + tracked metric
(`corr >= 0.5`, `nRMSE <= 0.30` relaxed by quant step, frequency searched
±5% band); a correct sinusoid yields corr ~0.820.98, wrong frequency
collapses to ~0.00. Do **not** tighten shape into a correctness gate —
timestamp calibration (Phase-A) is pending.
- **Stress** (`run_stress.sh`): 7 axes (signal size/count/fan-out/sources/WS
clients/zoom rate), hard gates survival+liveness, soft gates RSS+zoom-p95.
- **Coverage**: `--cpp-coverage` rebuilds with gcov, captures via `lcov`
restricted to `Source/*` + `Test/*`, then restores a clean build.
- Artifacts → `Build/x86-linux/E2E/chain/`: `results.json` (XFAIL/XPASS for
`known_issue` markers), `report_data.json`, `history.jsonl`, `trend_*.png`,
`E2E_Report.pdf`; stress → `stress/stress_results.json`.
Every MARTe2 component directory contains:
- `Makefile.gcc` — thin wrapper (`TARGET = x86-linux` then `include Makefile.inc`).
- `Makefile.inc` — real build definition: `OBJSX`, `PACKAGE`, `INCLUDES`,
`LIBRARIES`, includes `$(MARTe2_DIR)/MakeDefaults/MakeStdLibDefs.$(TARGET)`
and `MakeStdLibRules.$(TARGET)`, and `depends.$(TARGET)`.
- `depends.x86-linux` / `dependsRaw.x86-linux` — generated header dependency
files (committed; regenerated by the build). Do not hand-edit.
- `<ClassName>.{h,cpp}` — the implementation, with `CLASS_REGISTER(ClassName,
"1.0")` at the bottom of the `.cpp` (MARTe2 class-registration macro).
To add a new component, copy an existing one and adapt `Makefile.inc`'s
`OBJSX` / `PACKAGE` / `INCLUDES` / `LIBRARIES`.
---
## Conventions and gotchas
### No STL in MARTe2 components
`Source/Components/**` must not use the C++ standard library. Use MARTe2 types:
`StreamString` (not `std::string`), `FastPollingMutexSem` (not `std::mutex`),
fixed arrays / `StaticList` (not `std::vector`), MARTe2 error macros
(`REPORT_ERROR`, `REPORT_ERROR_STATIC`) instead of exceptions. Includes come
from `$(MARTe2_DIR)/Source/Core/...` (BareMetal, Scheduler, FileSystem).
`Source/Applications/StreamHub/` links MARTe2 core and follows the same
no-STL-on-RT-paths style, but is a standalone app (not a loadable component).
STL / C++17 is **fine** in `Client/streamhub/` and `Client/streamhub-qt/`
(framework-free C++17, no MARTe2 dependency).
### RT hot-path mutex rule
Use `FastPollingMutexSem` on real-time hot paths — never OS mutexes
(`pthread_mutex`, `std::mutex`). The RT cycle must not block on the scheduler.
### Protocol change checklist
Changing `Common/UDP/UDPSProtocol.h` (or the Go mirror) requires mirroring
across all four consumers listed in §1, plus the JS client parsers in
`Client/udpstreamer/static/`. The Go decoder lives in
`Common/Client/go/udpsprotocol/protocol.go` and re-declares the constants
(`HeaderSize`, `SigDescSize`, packet types, quant types, time modes) — it does
**not** include the C header.
### Qt client: `QT_NO_KEYWORDS`
`Client/streamhub-qt/` reuses `../streamhub/Protocol.{h,cpp}` and
`SignalBuffer.h` verbatim. Those structs have members named `signals`, which
collide with Qt's `signals`/`slots`/`emit` macros. The Qt build sets
`QT_NO_KEYWORDS` and all Qt classes use `Q_SIGNALS:`/`Q_SLOTS:`/`Q_EMIT`. Do
not remove that define, and do not use the lowercase Qt keywords in Qt-client
code. Single GUI thread — `QWebSocket` signals arrive on the GUI thread, no
locks; a 60 Hz `QTimer` drives repaint. Run with long options:
`./build/StreamHubQtClient --host HOST --port 8090` (single-dash `-host` is
misparsed as clustered short flags).
### StreamHub history
`HistoryWriter` (`Source/Applications/StreamHub/HistoryWriter.{h,cpp}`) does
disk-backed circular storage: per-signal `.shist` files with a 64-byte header
and a pre-allocated circular region of `(float64 time, float64 value)` pairs.
Configured via a `+History` block (`Directory` required; `DurationHours`
default 1, `Decimation` default 1, `FlushIntervalSec` default 5,
`MinDiskFreeMB` default 500). WS commands: `historyInfo`, `historyZoom`.
### E2E scenario / stress matrix
`Test/E2E/suite/scenarios.py` is a *curated covering set*: every configurable
UDPStreamer option value appears in at least one scenario, plus high-risk
interactions. `stress.py` is the capacity sibling — it sweeps one load axis at
a time and records survival/liveness (hard gates) and RSS/zoom-p95 latency
(soft gates). Both feed the same `gen_data.py` / `gen_cfg.py` generators. When
adding a new UDPStreamer option, add a scenario that exercises it and update
`validate_scenario` / the stress matrix accordingly.
`validate_waveform.py` has two gates: **fidelity** (every received value within
`tol` of some ground-truth value; `tol` is 0 for un-quantised ints, a float
epsilon for un-quantised floats, `quant_step/2 + 1e-6·range` for quantised
floats) — this is the **correctness** gate. **Shape** (least-squares sine fit,
correlation ≥ 0.99) is a gross-sanity gate + tracked metric, pending Phase-A
timestamp calibration. Do not tighten the shape gate to a correctness gate.
### Gitignored build artefacts
`Build/`, `compile_commands.json`, `*.gcov`/`*.gcda`/`*.gcno` (coverage), Go
binaries (`Client/*/marte2debugger`, `udpstreamer-webui`, `streamhub-e2e`,
`Client/*/bin/`), and the `udp_standalone_webui/` scratch dir are all
gitignored. `vgore.*` core dumps (e.g. `vgcore.26968`) are not — remove them.
---
## Documentation map
- `ARCHITECTURE.md` — full architecture: data-flow diagrams, UDPS packet/header
tables, CONFIG payload, StreamHub WebSocket protocol (commands, events,
binary frames), threading model.
- `Docs/` — per-component reference docs (`Protocol.md`, `UDPStreamer.md`,
`SineArrayGAM.md`, `DebugService.md`, `StreamHub-{UserGuide,API,Developer}.md`,
`Tutorial.md`, `WebUI.md`).
- `docs/superpowers/specs/` — design specs (dated, e.g.
`2026-06-26-stress-suite-report-integration-design.md`).
- `docs/superpowers/plans/` — implementation plans paired with specs.
- `.superpowers/sdd/` — task briefs, reports, and a `progress.md` ledger for
in-flight feature work (not always present; per-branch).
- `README.md` — high-level overview + quick-start config snippets.
- `CLAUDE.md` — condensed agent guidance (overlaps with this file).
---
## Ports reference (defaults)
## Ports Reference (defaults)
| Port | Protocol | Component | Purpose |
|------|----------|-----------|---------|
| 44500 | UDP | UDPStreamer | scalar signals (unicast control) |
| 44501 | UDP | UDPStreamer | packed arrays (FirstSample/LastSample) |
| 44502 | UDP | UDPStreamer | packed arrays (FullArray) |
|---|---|---|---|
| 44500 | UDP | UDPStreamer | scalar signals (unicast control + data) |
| 44501/44502 | UDP | UDPStreamer | packed arrays (FirstSample/LastSample, FullArray) |
| 44503 | UDP | UDPStreamer | multicast data (group 239.0.0.1) |
| 8080 | TCP | DebugService | text command channel |
| 8081 | UDP | DebugService | trace telemetry stream |
| 8080 | TCP | DebugService | text command channel (one client at a time, newline-terminated) |
| 8081 | UDP | DebugService | trace telemetry (UDPS format) |
| 8082 | TCP | TcpLogger | REPORT_ERROR log forward |
| 8090 | TCP/WS | StreamHub | WebSocket (commands + binary data) |
| 8080 | TCP | udpstreamer-webui | web UI listen (legacy direct-UDP path) |
| 9090 | TCP | debugger | debug web UI listen |
Note the 8080 collision: DebugService control vs. the legacy web UI listen port.
In combined demos that run both, the scripts adjust one of them — check the
script header before assuming.
---
## License
EUPL v1.1 — license headers are present on all C++ sources. Preserve them on
new files; do not relicense.
| 7777 | TCP | Client/debugger | debug web UI (older docs say 9090; current flag is `-addr`) |
| 8080 | TCP | Client/udpstreamer, Client/webui | web UI listen (collides with DebugService in combined demos — scripts adjust) |