Implemented hearthbit client side + tests
This commit is contained in:
@@ -1,313 +1,241 @@
|
||||
# AGENTS.md
|
||||
# Repository Guidelines
|
||||
|
||||
Guide for agents working in the MARTe2 Integrated Components repository. Read
|
||||
`ARCHITECTURE.md` and `CLAUDE.md` for deeper detail; this file focuses on
|
||||
non-obvious knowledge, commands, and conventions that are not self-evident from
|
||||
a single file read.
|
||||
Guide for AI assistants working in the MARTe2 Integrated Components repository.
|
||||
Focuses on non-obvious facts: commands, conventions, cross-module contracts, and
|
||||
gotchas that are not self-evident from a single file read.
|
||||
|
||||
---
|
||||
## Project Overview
|
||||
|
||||
## Repository at a glance
|
||||
MARTe2 component library with **two independent real-time data paths** sharing
|
||||
one binary wire protocol (`Common/UDP/UDPSProtocol.h`):
|
||||
|
||||
MARTe2 component library with two independent real-time data paths:
|
||||
|
||||
1. **Streaming path** — `UDPStreamer` DataSource (serialises signals to UDPS
|
||||
binary packets on UDP 44500) → `StreamHub` (headless C++ hub: ring buffers,
|
||||
LTTB decimation, trigger FSM, history writer) → WebSocket 8090 → clients
|
||||
(browser SPA, native ImGui, native Qt).
|
||||
2. **Debug path** — `DebugService` Interface patches `ClassRegistryDatabase` at
|
||||
1. **Streaming path** — `UDPStreamer` DataSource serialises DDB signals into UDPS
|
||||
binary packets on UDP → `StreamHub` (headless C++ hub: ring buffers, LTTB
|
||||
decimation, trigger FSM, history writer, binary recorder) → WebSocket 8090 →
|
||||
clients (browser SPA, native ImGui, native Qt).
|
||||
2. **Debug path** — `DebugService` patches `ClassRegistryDatabase` at
|
||||
`Initialise()` so `ConfigureApplication()` wraps all `MemoryMap*Broker` types
|
||||
with `DebugBrokerWrapper<T>`. Zero application code changes. Exposes TCP 8080
|
||||
(text commands), UDP 8081 (trace telemetry), TCP 8082 (`TcpLogger`).
|
||||
with `DebugBrokerWrapper<T>` — **zero application code changes**. Exposes
|
||||
TCP 8080 (text commands), UDP 8081 (UDPS trace telemetry), TCP 8082
|
||||
(`TcpLogger` log forward).
|
||||
|
||||
Key shared artefact: `Common/UDP/UDPSProtocol.h` — the UDPS binary wire format
|
||||
(17-byte packed header, 136-byte signal descriptors, little-endian). It is
|
||||
deliberately MARTe2-free and is mirrored across **four** codebases that must stay
|
||||
in sync: C++ producers (`UDPStreamer`, `DebugService`), C++ consumer
|
||||
(`Source/Components/Interfaces/UDPStream/UDPSClient`), Go decoder
|
||||
(`Common/Client/go/udpsprotocol`), and the JS client parsers. Any protocol
|
||||
change must be mirrored in all of them.
|
||||
## Architecture & Data Flow
|
||||
|
||||
StreamHub WebSocket protocol: JSON text frames for commands/events, binary
|
||||
frames for data pushes (spec in `ARCHITECTURE.md` §6). The Go hub
|
||||
(`Client/udpstreamer`) and the C++ StreamHub implement the identical protocol;
|
||||
browser JS, ImGui, and Qt clients must stay compatible with both.
|
||||
```
|
||||
[SineArrayGAM/TimeArrayGAM] → DDB → UDPStreamer (UDPS over UDP)
|
||||
├─→ UDPStreamerClient (input DS back into a MARTe2 RT app, round-trip)
|
||||
└─→ StreamHub: UDPSourceSession (receive thread → SignalRingBuffer)
|
||||
→ push loop @30Hz: LTTB decimate temporal sigs → WS binary frames → clients
|
||||
DebugService: patches broker builders at Initialise(); TCP 8080 commands,
|
||||
UDP 8081 telemetry, TcpLogger 8082 (REPORT_ERROR → "LOG <LEVEL> <desc>" lines)
|
||||
```
|
||||
|
||||
---
|
||||
- **Wire protocol**: `Common/UDP/UDPSProtocol.h` is the canonical spec (17-byte
|
||||
packed header, magic `0x53504455` 'UDPS', 136-byte signal descriptors,
|
||||
CONFIG/DATA/ACK/CONNECT/DISCONNECT packet types, quant/time/publish modes).
|
||||
Deliberately MARTe2-free so Go clients reuse it. **Mirrored across four
|
||||
codebases that must stay in sync**: C++ producers (UDPStreamer, DebugService),
|
||||
C++ consumer (`Source/Components/Interfaces/UDPStream/UDPSClient`), Go decoder
|
||||
(`Common/Client/go/udpsprotocol/protocol.go`), and JS parsers
|
||||
(`Client/udpstreamer/static/`, `Client/debugger/static/`). Any protocol change
|
||||
must be mirrored in all of them.
|
||||
- **WS protocol** has two implementations — Go hub (`Common/Client/go/wshub`) and
|
||||
C++ StreamHub — that must behave identically; every client (SPA, ImGui, Qt)
|
||||
must satisfy both. JSON text frames for commands/events (`addSource`,
|
||||
`removeSource`, `setTrigger`, `arm`, `zoom`, `historyZoom`, `recStart`…), binary
|
||||
frames for data pushes (live v1 + trigger capture v2).
|
||||
- **Threading model**: RT threads only spinlock+memcpy (`FastPollingMutexSem`);
|
||||
all socket I/O, fragmentation, and reassembly lives on background
|
||||
`SingleThreadService` threads. StreamHub: per-session UDPSClient receive
|
||||
threads + WS accept/read threads + one push loop.
|
||||
- **DebugService patching**: `PatchRegistry()` replaces the ObjectBuilder for 11
|
||||
`MemoryMap*Broker` classes; runs only when `ControlPort > 0`; static guard
|
||||
against double-patching; wrappers persist for process lifetime.
|
||||
|
||||
## Environment setup (do this first, always)
|
||||
## Key Directories
|
||||
|
||||
`source env.sh` is **required** before building *or* running anything. It sets:
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `Source/Components/DataSources/UDPStreamer/` | Output DataSource; UDP I/O on bg thread, RT thread only spinlock+memcpy in `Synchronise()` |
|
||||
| `Source/Components/DataSources/UDPStreamerClient/` | Input DataSource (shared `UDPSClient`), double-buffered ready/scratch |
|
||||
| `Source/Components/GAMs/` | `SineArrayGAM` (float32 sine, continuous phase), `TimeArrayGAM` (us-timer → per-sample timestamp array) |
|
||||
| `Source/Components/Interfaces/DebugService/` | Registry patching, `DebugBrokerWrapper.h`, TCP/UDP services |
|
||||
| `Source/Components/Interfaces/TCPLogger/` | `LoggerConsumerI` forwarding `REPORT_ERROR` to ≤8 TCP clients |
|
||||
| `Source/Components/Interfaces/UDPStream/` | Plain-C++ helpers (not MARTe2 Objects): `UDPSClient` (auto-reconnect + fragment reassembly), `UDPSServer` (not thread-safe — owner's Execute thread only) |
|
||||
| `Source/Applications/StreamHub/` | Standalone app (links MARTe2 core): `StreamHub`, `UDPSourceSession`, `WSServer`, `TriggerEngine`, `HistoryWriter`, `BinaryRecorder`, `LTTB`, `SignalRingBuffer` |
|
||||
| `Common/UDP/` | Canonical wire protocol (header-only, MARTe2-free) |
|
||||
| `Common/Client/go/` | Go mirror: `udpsprotocol` (decoder), `wshub` (WS hub client) |
|
||||
| `Client/udpstreamer/` | Go legacy direct-UDP oscilloscope web UI (connects straight to UDPStreamer, no StreamHub) |
|
||||
| `Client/webui/` | Go thin static server; SPA talks WS directly to C++ StreamHub (discovers via `GET /hub`) |
|
||||
| `Client/debugger/` | Go debug web UI for DebugService |
|
||||
| `Client/streamhub/` | Native ImGui+SDL2+OpenGL oscilloscope (C++17, no MARTe2) |
|
||||
| `Client/streamhub-qt/` | Native Qt Widgets oscilloscope (Qt6 preferred, Qt5 fallback) |
|
||||
| `Test/` | GTest, legacy Integration tests, Configurations (.cfg), E2E suite |
|
||||
| `Docs/` | Per-component reference: `Protocol.md`, `UDPStreamer.md`, `StreamHub-{API,UserGuide,Developer}.md`, `DebugService.md`, `WebUI.md`, `Tutorial.md`, `E2E-Suite.md` |
|
||||
|
||||
- `MARTe2_DIR` (default `~/workspace/MARTe2`) and `MARTe2_Components_DIR`
|
||||
(default `~/workspace/MARTe2-components`) — external dependencies, not in
|
||||
this repo. Edit `env.sh` if your MARTe2 install lives elsewhere.
|
||||
- `TARGET=x86-linux`
|
||||
- `LD_LIBRARY_PATH` — prepends MARTe2 Core, MARTe2-components (LinuxTimer,
|
||||
FileDataSource), and this repo's built `.so`s (UDPStreamer, SineArrayGAM,
|
||||
TimeArrayGAM, DebugService, TCPLogger).
|
||||
## Development Commands
|
||||
|
||||
Without sourcing `env.sh`, builds fail (can't find `MakeDefaults`) and binaries
|
||||
fail to load shared libs. The E2E scripts source it themselves, but a bare
|
||||
`make` or `./MainGTest.ex` from a fresh shell will not.
|
||||
|
||||
---
|
||||
|
||||
## Build commands
|
||||
|
||||
All C++ builds use the MARTe2 `Makefile.gcc` wrapper system. There is one
|
||||
top-level `Makefile.gcc` and one per component directory. `TARGET` defaults to
|
||||
`x86-linux`; pass `TARGET=x86-linux` explicitly when invoking `make -C` from
|
||||
elsewhere.
|
||||
`source env.sh` is **mandatory** before any MARTe2 build or run (sets
|
||||
`MARTe2_DIR`, `MARTe2_Components_DIR`, `TARGET=x86-linux`, `LD_LIBRARY_PATH`).
|
||||
The E2E scripts source it themselves; a bare `make` from a fresh shell will not
|
||||
work. `run_streamhub.sh` hard-errors if `MARTe2_DIR` is unset.
|
||||
|
||||
```bash
|
||||
source env.sh
|
||||
|
||||
make -f Makefile.gcc core # all C++ MARTe2 components (UDPStreamer, GAMs, DebugService, TCPLogger, UDPStream)
|
||||
make -f Makefile.gcc apps # StreamHub standalone app
|
||||
make -f Makefile.gcc test # build GTest + Integration test binaries
|
||||
make -f Makefile.gcc core # 7 components (UDPStream interface FIRST, then UDPStreamer, UDPStreamerClient, GAMs, TCPLogger, DebugService)
|
||||
make -f Makefile.gcc apps # StreamHub standalone app → Build/x86-linux/StreamHub/StreamHub.ex
|
||||
make -f Makefile.gcc test # GTest + Integration test binaries + component test libs
|
||||
make -f Makefile.gcc all # core + apps + test
|
||||
make -f Makefile.gcc clean
|
||||
make -f Makefile.gcc # = all: core apps test
|
||||
|
||||
# Build a single component (each component dir has its own Makefile.gcc):
|
||||
make -C Source/Components/DataSources/UDPStreamer -f Makefile.gcc
|
||||
make -C Source/Applications/StreamHub -f Makefile.gcc
|
||||
# Single component:
|
||||
make -C Source/Components/GAMs/SineArrayGAM -f Makefile.gcc
|
||||
```
|
||||
|
||||
Build output goes to `Build/x86-linux/` — shared libs under
|
||||
`Build/x86-linux/Components/<...>/`, the StreamHub executable at
|
||||
`Build/x86-linux/StreamHub/StreamHub.ex`, test binaries under
|
||||
`Build/x86-linux/GTest/` and `Build/x86-linux/Test/Integration/`.
|
||||
Build output → `Build/x86-linux/` mirroring `PACKAGE` paths (both `libX.so` and
|
||||
`X.so` are produced). `compile_commands.json` (repo root, gitignored) feeds
|
||||
LSP/clangd; CMake clients export their own into `Client/*/build/`.
|
||||
|
||||
`compile_commands.json` is generated at the repo root (gitignored) for LSP /
|
||||
clangd. The CMake-based clients (ImGui, Qt) also export it into their `build/`.
|
||||
|
||||
### Non-MARTe2 clients (separate build systems, no env.sh needed)
|
||||
### Non-MARTe2 clients (no env.sh needed)
|
||||
|
||||
```bash
|
||||
# Go clients (UDPS web client, debug client, E2E chain-client)
|
||||
cd Common/Client/go && go build ./...
|
||||
cd Client/debugger && go build ./...
|
||||
cd Client/udpstreamer && go build ./... # or: cd Client/webui && go build
|
||||
cd Test/E2E/suite/client && go build ./... # chain-client (E2E driver) + its tests
|
||||
|
||||
# ImGui desktop client (needs SDL2; fetches Dear ImGui + ImPlot via FetchContent)
|
||||
cd Client/debugger && go build ./...
|
||||
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
|
||||
|
||||
# Qt desktop client (Qt5 or Qt6 Widgets + WebSockets; autodetects, prefers Qt6)
|
||||
cd Client/streamhub-qt && cmake -B build && cmake --build build
|
||||
```
|
||||
|
||||
---
|
||||
### Key scripts
|
||||
|
||||
## Run / test commands
|
||||
|
||||
### Unit & integration tests (C++)
|
||||
|
||||
```bash
|
||||
./Build/x86-linux/GTest/MainGTest.ex # UDPStreamer unit tests
|
||||
./Build/x86-linux/GTest/MainGTest.ex --gtest_filter='Name*' # single test
|
||||
./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex # DebugService integration tests
|
||||
```
|
||||
|
||||
### Go tests
|
||||
|
||||
```bash
|
||||
cd Test/E2E/suite/client && go test ./...
|
||||
```
|
||||
|
||||
### Python framework tests (E2E framework logic, no live stack needed)
|
||||
|
||||
```bash
|
||||
cd Test/E2E/suite && python3 -m unittest tests_py
|
||||
```
|
||||
|
||||
### E2E / demo scripts (build + launch the full stack)
|
||||
|
||||
| Script | What it does |
|
||||
| Script | Purpose |
|
||||
|---|---|
|
||||
| `./run_streamhub.sh` | Launch MARTe2 app (UDPStreamer) + StreamHub, optionally web UI (`-w`) and ImGui client (`-g`). Ports documented in its header. |
|
||||
| `./Test/E2E/suite/run_e2e.sh` | **Unified E2E suite**: per-scenario generates data + cfgs, runs MARTe2+StreamHub, drives the Go `chain-client` (live/zoom/window/trigger) for `chain` scenarios plus `direct`/`recorder`/`debug`/`tcplogger` scenario kinds, runs a stress matrix, unit suites + coverage, builds a Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--pdf-only`, `--cpp-coverage`, `--skip-coverage`, `--skip-stress`, `--skip-datasources`, `--skip-recorder`, `--skip-debug`, `--skip-tcplogger`. |
|
||||
| `./Test/E2E/suite/run_stress.sh` | **Capacity/stress harness**: sweeps one load axis at a time (signal size/count, subscriber fan-out, source count, WS-client count, zoom rate), gates on survival+liveness (hard) and RSS+zoom-p95 latency (soft). Flags: `--skip-build`, `--only <id>`, `--axis <axis>`. |
|
||||
| `./run_streamhub.sh` | Demo stack: build + launch MARTe2 app + StreamHub, optional web (`-w`) / ImGui (`-g`) clients. Flags `-m/-c` MARTe2 dirs, `-b TARGET`, `-p WS_PORT`, `-n MAX_POINTS` (actual default 1000000, header says 10000), `-s` skip build. Generates temp hub cfg with `+History`/`+Recorder` blocks in `/tmp`. Ctrl-C kills all. |
|
||||
| `./Test/E2E/suite/run_e2e.sh` | Full E2E: 57-scenario matrix + stress + unit suites + gcov coverage + Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--pdf-only`, `--skip-coverage`, `--skip-stress`, `--skip-datasources`, `--skip-recorder`, `--skip-debug`, `--skip-tcplogger` |
|
||||
| `./Test/E2E/suite/run_stress.sh` | Capacity harness: sweeps one load axis at a time (`--axis`), hard gates survival+liveness, soft gates RSS+zoom-p95 |
|
||||
|
||||
The E2E suite produces `Build/x86-linux/E2E/chain/` artifacts: `report_data.json`,
|
||||
`history.jsonl` (per-run headline metrics for trend/regression tracking),
|
||||
`trend_*.png` plots, and `E2E_Report.pdf` (compiled from `E2E_Report.typ` via
|
||||
`typst`). `--cpp-coverage` triggers an instrumented `--coverage` rebuild, captures
|
||||
with `lcov` (restricted to `Source/*` + `Test/*`), then restores a clean build.
|
||||
## Code Conventions & Common Patterns
|
||||
|
||||
---
|
||||
- **No STL in `Source/Components/**` (and StreamHub)**: use `StreamString` (not
|
||||
`std::string`), `FastPollingMutexSem`/`EventSem` (not `std::mutex`/threads),
|
||||
fixed arrays / MARTe2 `Vector<T>` (not `std::vector`), `REPORT_ERROR` /
|
||||
`REPORT_ERROR_STATIC` macros (no exceptions). C stdlib is fine. Heap
|
||||
`new`/`delete[]` is normal. STL/C++17 is fine in `Client/streamhub/` and
|
||||
`Client/streamhub-qt/`.
|
||||
- **RT hot-path rule**: `FastPollingMutexSem` on real-time hot paths, never OS
|
||||
mutexes; RT cycle must not block on the scheduler.
|
||||
- **Class registration**: `CLASS_REGISTER_DECLARATION()` in the class `public:`
|
||||
section of the header; `CLASS_REGISTER(Name, "1.0")` at the end of the `.cpp`
|
||||
inside `namespace MARTe`. Every component `.cpp` ends with it.
|
||||
- **EUPL v1.1 license headers** on all C++ sources and `Makefile.inc` — preserve
|
||||
on new files.
|
||||
- **Per-component build**: each dir has one-line `Makefile.gcc` wrapper
|
||||
(`include Makefile.inc`) + `Makefile.inc` declaring `OBJSX`, `PACKAGE`,
|
||||
`ROOT_DIR`, `INCLUDES` (re-declared per file, ~12 MARTe2 layer dirs),
|
||||
`LIBRARIES`, including `MakeStdLibDefs.$(TARGET)` then
|
||||
`MakeStdLibRules.$(TARGET)`. Generated `depends.x86-linux` (gcc -MM) is
|
||||
committed but **never hand-edited** — delete to regenerate.
|
||||
- **Qt client**: `QT_NO_KEYWORDS` is required (reused `Protocol.h` structs have
|
||||
members named `signals`); Qt classes use `Q_SIGNALS`/`Q_SLOTS`/`Q_EMIT`. Run
|
||||
with long options: `--host HOST --port 8090` (single-dash misparsed). Single
|
||||
GUI thread, 60 Hz QTimer repaint.
|
||||
- **StreamHub config** is *not* a MARTe2 `RealTimeApplication`: `Hub = { WSPort
|
||||
MaxPoints PushRate MaxPushPoints RingTemporal RingScalar +Recorder{...}
|
||||
Sources={id={Label Addr Port}} }`. `+History` keys: `Directory` (required),
|
||||
`DurationHours` (1), `Decimation` (1), `FlushIntervalSec` (5),
|
||||
`MinDiskFreeMB` (500). `.shist` files: 64-byte header ('SHR1') + circular
|
||||
(t,v) float64 pairs.
|
||||
- **UDPStreamer config**: `Port` (44500; multicast data = `DataPort`, default
|
||||
`Port+1`), `MaxPayloadSize` (1400), `PublishingMode` `Strict`/`Accumulate`,
|
||||
per-signal `Signals={Name={Type,Unit,NumberOfDimensions,NumberOfElements,
|
||||
TimeMode}}` with `TimeMode` `PacketTime`/`FirstSample`/`LastSample`/`FullArray`;
|
||||
multicast needs `MulticastGroup` + `Interface`.
|
||||
|
||||
## Code organization
|
||||
## Important Files
|
||||
|
||||
```
|
||||
Common/
|
||||
UDP/UDPSProtocol.h # shared UDPS binary protocol (header-only, MARTe2-free)
|
||||
Client/go/ # Go packages: udpsprotocol (decoder), wshub (WS hub client)
|
||||
Source/
|
||||
Components/ # MARTe2 components — NO STL allowed here
|
||||
DataSources/UDPStreamer/ # real-time UDP signal streaming DataSource
|
||||
DataSources/UDPStreamerClient/ # (MARTe2-side UDPS client DataSource)
|
||||
GAMs/SineArrayGAM/ # sine-wave array generator GAM
|
||||
GAMs/TimeArrayGAM/ # time-reference array GAM
|
||||
Interfaces/DebugService/ # tracing/forcing/breakpoint Interface (registry-patched brokers)
|
||||
Interfaces/TCPLogger/ # REPORT_ERROR → TCP forwarder (LoggerConsumerI)
|
||||
Interfaces/UDPStream/ # UDPSClient/UDPSServer (C++ consumer/producer of UDPS)
|
||||
Applications/StreamHub/ # headless C++ hub app (links MARTe2 core but follows MARTe2 style)
|
||||
Client/
|
||||
debugger/ # Go web client for DebugService (browser UI in static/)
|
||||
udpstreamer/ # Go web server + static SPA for UDPStreamer (legacy direct-UDP path)
|
||||
webui/ # Go StreamHub WebSocket web UI (newer, hub-based)
|
||||
streamhub/ # native ImGui desktop client (SDL2 + OpenGL + ImPlot, C++17, no MARTe2)
|
||||
streamhub-qt/ # native Qt Widgets desktop client (C++17, Qt5/Qt6, no MARTe2)
|
||||
Test/
|
||||
GTest/ # GTest harness (MainGTest.cpp) for UDPStreamer unit tests
|
||||
Integration/ # DebugService integration tests (link against DebugService .so)
|
||||
Components/DataSources/UDPStreamer/ # UDPStreamer unit test sources
|
||||
Configurations/ # MARTe2 .cfg files for tests and demos
|
||||
E2E/
|
||||
chain/ # streaming-chain E2E + stress suite (Python orchestrator + Go client)
|
||||
datasources/ recorder/ streamhub/ # older per-component E2E scripts
|
||||
Docs/ # per-component reference docs (Protocol, UDPStreamer, StreamHub-*, DebugService, ...)
|
||||
docs/superpowers/{specs,plans}/ # design specs + implementation plans (dated)
|
||||
ARCHITECTURE.md # full architecture (data flow diagrams, protocol tables, WS protocol)
|
||||
- `env.sh` — environment; source first, always.
|
||||
- `Makefile.gcc` / `Makefile.inc` (root) — build orchestration.
|
||||
- `Common/UDP/UDPSProtocol.h` — canonical wire format; changing it triggers the
|
||||
4-way mirror checklist above.
|
||||
- `Source/Applications/StreamHub/main.cpp` — hub entry (`[-cfg file.cfg]
|
||||
[-port N] [-maxPoints N]`); hub **must be heap-allocated** (~128 MB, exceeds
|
||||
the 8 MB stack).
|
||||
- `Test/Configurations/*.cfg` — MARTe2 app configs (`$App = { Class =
|
||||
RealTimeApplication }` with `+Functions`, `+DataSources`, `+States`, `+Timings`
|
||||
blocks); `streamhub_demo.cfg` and `TestApp.cfg` are good templates.
|
||||
- `Test/E2E/suite/{scenarios,gen_data,gen_cfg,validate_waveform,stress}.py` —
|
||||
declarative scenario matrix and generators consumed identically by the Go
|
||||
chain-client and validators.
|
||||
- `Client/debugger/main.go` — `-addr :7777` default, `-enable-dangerous-commands`
|
||||
safety gate (CR-4) for FORCE/PAUSE/RESUME/STEP/BREAK/MSG.
|
||||
|
||||
## Runtime/Tooling Preferences
|
||||
|
||||
- **OS**: Linux x86_64 (`TARGET=x86-linux`). External deps live outside this
|
||||
repo: `MARTe2_DIR` (default `~/workspace/MARTe2`) and
|
||||
`MARTe2_Components_DIR` (default `~/workspace/MARTe2-components`) — edit
|
||||
`env.sh` if they differ. `env.sh`'s `LD_LIBRARY_PATH` does **not** cover
|
||||
UDPStreamerClient/UDPStream lib dirs.
|
||||
- **C++**: MARTe2 `Makefile.gcc` wrapper system, gtest-1.7.0 for tests.
|
||||
- **Go**: `go 1.21`; modules use `replace marte2/common => ../../Common/Client/go`
|
||||
(`gorilla/websocket` v1.5.1). Go binaries are gitignored.
|
||||
- **ImGui client**: needs SDL2; CMake FetchContent pins Dear ImGui **v1.91.8** +
|
||||
ImPlot **v0.17** (`implot_items.cpp` is a slow -O3 TU, ~2 min rebuild).
|
||||
- **Qt client**: Qt6 preferred, Qt5 fallback, Widgets + WebSockets, custom
|
||||
QPainter plotting (no QtCharts).
|
||||
- **E2E report**: `typst compile E2E_Report.typ`; Python 3 + numpy for the suite.
|
||||
- Remove `vgore.*` core dumps when you see them; they are not gitignored.
|
||||
|
||||
## Testing & QA
|
||||
|
||||
Four test layers; `env.sh` + built stack required for all but the standalone
|
||||
ones. Only `tests_py.py`, Go tests, and the built C++ test binaries run
|
||||
standalone.
|
||||
|
||||
```bash
|
||||
./Build/x86-linux/GTest/MainGTest.ex --gtest_filter='Name*' # C++ GTest
|
||||
./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex # legacy DebugService runtime tests
|
||||
cd Test/E2E/suite/client && go test ./... # Go chain-client unit tests
|
||||
cd Test/E2E/suite && python3 -m unittest tests_py # framework logic, standalone
|
||||
```
|
||||
|
||||
### Component layout convention
|
||||
- **GTest**: `MainGTest.ex` currently holds only `DebugServiceGTest`
|
||||
(TraceRingBuffer SPSC, DebugSignalInfo, BreakOp). Component GTests
|
||||
(`UDPStreamerGTest.cpp` ~46 cases, `StreamHubTest.a`, `UDPStreamerClientTest.a`)
|
||||
compile **as libraries only — no standalone executable**.
|
||||
- **Legacy IntegrationTests.ex**: 9 printf-narrated DebugService runtime tests,
|
||||
always returns 0; `collect.py` parses stdout blocks.
|
||||
- **E2E suite** (`run_e2e.sh`): 57 curated scenarios (s01–s57) across kinds
|
||||
`chain`/`direct`/`recorder`/`debug`/`debug_pause_resume`/`tcplogger`, driven
|
||||
against live MARTeApp.ex + StreamHub.ex + Go chain-client. `scenarios.py` is a
|
||||
curated covering set: **every configurable UDPStreamer option value appears in
|
||||
≥1 scenario** — add a scenario when adding an option.
|
||||
- **Oracle gates** (`validate_waveform.py`): **fidelity** (every received value
|
||||
within `tol` of ground truth; 0 for un-quantised ints, float epsilon for
|
||||
un-quantised floats, `quant_step/2 + 1e-6·range` for quantised) is the
|
||||
**correctness gate**. **Shape** is a *gross* sanity gate + tracked metric
|
||||
(`corr >= 0.5`, `nRMSE <= 0.30` relaxed by quant step, frequency searched
|
||||
±5% band); a correct sinusoid yields corr ~0.82–0.98, wrong frequency
|
||||
collapses to ~0.00. Do **not** tighten shape into a correctness gate —
|
||||
timestamp calibration (Phase-A) is pending.
|
||||
- **Stress** (`run_stress.sh`): 7 axes (signal size/count/fan-out/sources/WS
|
||||
clients/zoom rate), hard gates survival+liveness, soft gates RSS+zoom-p95.
|
||||
- **Coverage**: `--cpp-coverage` rebuilds with gcov, captures via `lcov`
|
||||
restricted to `Source/*` + `Test/*`, then restores a clean build.
|
||||
- Artifacts → `Build/x86-linux/E2E/chain/`: `results.json` (XFAIL/XPASS for
|
||||
`known_issue` markers), `report_data.json`, `history.jsonl`, `trend_*.png`,
|
||||
`E2E_Report.pdf`; stress → `stress/stress_results.json`.
|
||||
|
||||
Every MARTe2 component directory contains:
|
||||
|
||||
- `Makefile.gcc` — thin wrapper (`TARGET = x86-linux` then `include Makefile.inc`).
|
||||
- `Makefile.inc` — real build definition: `OBJSX`, `PACKAGE`, `INCLUDES`,
|
||||
`LIBRARIES`, includes `$(MARTe2_DIR)/MakeDefaults/MakeStdLibDefs.$(TARGET)`
|
||||
and `MakeStdLibRules.$(TARGET)`, and `depends.$(TARGET)`.
|
||||
- `depends.x86-linux` / `dependsRaw.x86-linux` — generated header dependency
|
||||
files (committed; regenerated by the build). Do not hand-edit.
|
||||
- `<ClassName>.{h,cpp}` — the implementation, with `CLASS_REGISTER(ClassName,
|
||||
"1.0")` at the bottom of the `.cpp` (MARTe2 class-registration macro).
|
||||
|
||||
To add a new component, copy an existing one and adapt `Makefile.inc`'s
|
||||
`OBJSX` / `PACKAGE` / `INCLUDES` / `LIBRARIES`.
|
||||
|
||||
---
|
||||
|
||||
## Conventions and gotchas
|
||||
|
||||
### No STL in MARTe2 components
|
||||
|
||||
`Source/Components/**` must not use the C++ standard library. Use MARTe2 types:
|
||||
`StreamString` (not `std::string`), `FastPollingMutexSem` (not `std::mutex`),
|
||||
fixed arrays / `StaticList` (not `std::vector`), MARTe2 error macros
|
||||
(`REPORT_ERROR`, `REPORT_ERROR_STATIC`) instead of exceptions. Includes come
|
||||
from `$(MARTe2_DIR)/Source/Core/...` (BareMetal, Scheduler, FileSystem).
|
||||
|
||||
`Source/Applications/StreamHub/` links MARTe2 core and follows the same
|
||||
no-STL-on-RT-paths style, but is a standalone app (not a loadable component).
|
||||
|
||||
STL / C++17 is **fine** in `Client/streamhub/` and `Client/streamhub-qt/`
|
||||
(framework-free C++17, no MARTe2 dependency).
|
||||
|
||||
### RT hot-path mutex rule
|
||||
|
||||
Use `FastPollingMutexSem` on real-time hot paths — never OS mutexes
|
||||
(`pthread_mutex`, `std::mutex`). The RT cycle must not block on the scheduler.
|
||||
|
||||
### Protocol change checklist
|
||||
|
||||
Changing `Common/UDP/UDPSProtocol.h` (or the Go mirror) requires mirroring
|
||||
across all four consumers listed in §1, plus the JS client parsers in
|
||||
`Client/udpstreamer/static/`. The Go decoder lives in
|
||||
`Common/Client/go/udpsprotocol/protocol.go` and re-declares the constants
|
||||
(`HeaderSize`, `SigDescSize`, packet types, quant types, time modes) — it does
|
||||
**not** include the C header.
|
||||
|
||||
### Qt client: `QT_NO_KEYWORDS`
|
||||
|
||||
`Client/streamhub-qt/` reuses `../streamhub/Protocol.{h,cpp}` and
|
||||
`SignalBuffer.h` verbatim. Those structs have members named `signals`, which
|
||||
collide with Qt's `signals`/`slots`/`emit` macros. The Qt build sets
|
||||
`QT_NO_KEYWORDS` and all Qt classes use `Q_SIGNALS:`/`Q_SLOTS:`/`Q_EMIT`. Do
|
||||
not remove that define, and do not use the lowercase Qt keywords in Qt-client
|
||||
code. Single GUI thread — `QWebSocket` signals arrive on the GUI thread, no
|
||||
locks; a 60 Hz `QTimer` drives repaint. Run with long options:
|
||||
`./build/StreamHubQtClient --host HOST --port 8090` (single-dash `-host` is
|
||||
misparsed as clustered short flags).
|
||||
|
||||
### StreamHub history
|
||||
|
||||
`HistoryWriter` (`Source/Applications/StreamHub/HistoryWriter.{h,cpp}`) does
|
||||
disk-backed circular storage: per-signal `.shist` files with a 64-byte header
|
||||
and a pre-allocated circular region of `(float64 time, float64 value)` pairs.
|
||||
Configured via a `+History` block (`Directory` required; `DurationHours`
|
||||
default 1, `Decimation` default 1, `FlushIntervalSec` default 5,
|
||||
`MinDiskFreeMB` default 500). WS commands: `historyInfo`, `historyZoom`.
|
||||
|
||||
### E2E scenario / stress matrix
|
||||
|
||||
`Test/E2E/suite/scenarios.py` is a *curated covering set*: every configurable
|
||||
UDPStreamer option value appears in at least one scenario, plus high-risk
|
||||
interactions. `stress.py` is the capacity sibling — it sweeps one load axis at
|
||||
a time and records survival/liveness (hard gates) and RSS/zoom-p95 latency
|
||||
(soft gates). Both feed the same `gen_data.py` / `gen_cfg.py` generators. When
|
||||
adding a new UDPStreamer option, add a scenario that exercises it and update
|
||||
`validate_scenario` / the stress matrix accordingly.
|
||||
|
||||
`validate_waveform.py` has two gates: **fidelity** (every received value within
|
||||
`tol` of some ground-truth value; `tol` is 0 for un-quantised ints, a float
|
||||
epsilon for un-quantised floats, `quant_step/2 + 1e-6·range` for quantised
|
||||
floats) — this is the **correctness** gate. **Shape** (least-squares sine fit,
|
||||
correlation ≥ 0.99) is a gross-sanity gate + tracked metric, pending Phase-A
|
||||
timestamp calibration. Do not tighten the shape gate to a correctness gate.
|
||||
|
||||
### Gitignored build artefacts
|
||||
|
||||
`Build/`, `compile_commands.json`, `*.gcov`/`*.gcda`/`*.gcno` (coverage), Go
|
||||
binaries (`Client/*/marte2debugger`, `udpstreamer-webui`, `streamhub-e2e`,
|
||||
`Client/*/bin/`), and the `udp_standalone_webui/` scratch dir are all
|
||||
gitignored. `vgore.*` core dumps (e.g. `vgcore.26968`) are not — remove them.
|
||||
|
||||
---
|
||||
|
||||
## Documentation map
|
||||
|
||||
- `ARCHITECTURE.md` — full architecture: data-flow diagrams, UDPS packet/header
|
||||
tables, CONFIG payload, StreamHub WebSocket protocol (commands, events,
|
||||
binary frames), threading model.
|
||||
- `Docs/` — per-component reference docs (`Protocol.md`, `UDPStreamer.md`,
|
||||
`SineArrayGAM.md`, `DebugService.md`, `StreamHub-{UserGuide,API,Developer}.md`,
|
||||
`Tutorial.md`, `WebUI.md`).
|
||||
- `docs/superpowers/specs/` — design specs (dated, e.g.
|
||||
`2026-06-26-stress-suite-report-integration-design.md`).
|
||||
- `docs/superpowers/plans/` — implementation plans paired with specs.
|
||||
- `.superpowers/sdd/` — task briefs, reports, and a `progress.md` ledger for
|
||||
in-flight feature work (not always present; per-branch).
|
||||
- `README.md` — high-level overview + quick-start config snippets.
|
||||
- `CLAUDE.md` — condensed agent guidance (overlaps with this file).
|
||||
|
||||
---
|
||||
|
||||
## Ports reference (defaults)
|
||||
## Ports Reference (defaults)
|
||||
|
||||
| Port | Protocol | Component | Purpose |
|
||||
|------|----------|-----------|---------|
|
||||
| 44500 | UDP | UDPStreamer | scalar signals (unicast control) |
|
||||
| 44501 | UDP | UDPStreamer | packed arrays (FirstSample/LastSample) |
|
||||
| 44502 | UDP | UDPStreamer | packed arrays (FullArray) |
|
||||
|---|---|---|---|
|
||||
| 44500 | UDP | UDPStreamer | scalar signals (unicast control + data) |
|
||||
| 44501/44502 | UDP | UDPStreamer | packed arrays (FirstSample/LastSample, FullArray) |
|
||||
| 44503 | UDP | UDPStreamer | multicast data (group 239.0.0.1) |
|
||||
| 8080 | TCP | DebugService | text command channel |
|
||||
| 8081 | UDP | DebugService | trace telemetry stream |
|
||||
| 8080 | TCP | DebugService | text command channel (one client at a time, newline-terminated) |
|
||||
| 8081 | UDP | DebugService | trace telemetry (UDPS format) |
|
||||
| 8082 | TCP | TcpLogger | REPORT_ERROR log forward |
|
||||
| 8090 | TCP/WS | StreamHub | WebSocket (commands + binary data) |
|
||||
| 8080 | TCP | udpstreamer-webui | web UI listen (legacy direct-UDP path) |
|
||||
| 9090 | TCP | debugger | debug web UI listen |
|
||||
|
||||
Note the 8080 collision: DebugService control vs. the legacy web UI listen port.
|
||||
In combined demos that run both, the scripts adjust one of them — check the
|
||||
script header before assuming.
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
EUPL v1.1 — license headers are present on all C++ sources. Preserve them on
|
||||
new files; do not relicense.
|
||||
| 7777 | TCP | Client/debugger | debug web UI (older docs say 9090; current flag is `-addr`) |
|
||||
| 8080 | TCP | Client/udpstreamer, Client/webui | web UI listen (collides with DebugService in combined demos — scripts adjust) |
|
||||
|
||||
Reference in New Issue
Block a user