# Repository Guidelines

Guide for AI assistants working in the MARTe2 Integrated Components repository.
Focuses on non-obvious facts: commands, conventions, cross-module contracts, and
gotchas that are not self-evident from a single file read.

## Project Overview

MARTe2 component library with **two independent real-time data paths** sharing
one binary wire protocol (`Common/UDP/UDPSProtocol.h`):

1. **Streaming path** — `UDPStreamer` DataSource serialises DDB signals into UDPS
   binary packets on UDP → `StreamHub` (headless C++ hub: ring buffers, LTTB
   decimation, trigger FSM, history writer, binary recorder) → WebSocket 8090 →
   clients (browser SPA, native ImGui, native Qt).
2. **Debug path** — `DebugService` patches `ClassRegistryDatabase` at
   `Initialise()` so `ConfigureApplication()` wraps all `MemoryMap*Broker` types
   with `DebugBrokerWrapper<T>` — **zero application code changes**. Exposes
   TCP 8080 (text commands), UDP 8081 (UDPS trace telemetry), TCP 8082
   (`TcpLogger` log forward).

## Architecture & Data Flow

```
[SineArrayGAM/TimeArrayGAM] → DDB → UDPStreamer (UDPS over UDP)
   ├─→ UDPStreamerClient (input DS back into a MARTe2 RT app, round-trip)
   └─→ StreamHub: UDPSourceSession (receive thread → SignalRingBuffer)
        → push loop @30Hz: LTTB decimate temporal sigs → WS binary frames → clients
DebugService: patches broker builders at Initialise(); TCP 8080 commands,
UDP 8081 telemetry, TcpLogger 8082 (REPORT_ERROR → "LOG <LEVEL> <desc>" lines)
```

- **Wire protocol**: `Common/UDP/UDPSProtocol.h` is the canonical spec (17-byte
  packed header, magic `0x53504455` 'UDPS', 136-byte signal descriptors,
  CONFIG/DATA/ACK/CONNECT/DISCONNECT packet types, quant/time/publish modes).
  Deliberately MARTe2-free so Go clients reuse it. **Mirrored across four
  codebases that must stay in sync**: C++ producers (UDPStreamer, DebugService),
  C++ consumer (`Source/Components/Interfaces/UDPStream/UDPSClient`), Go decoder
  (`Common/Client/go/udpsprotocol/protocol.go`), and JS parsers
  (`Client/udpstreamer/static/`, `Client/debugger/static/`). Any protocol change
  must be mirrored in all of them.
- **WS protocol** has two implementations — Go hub (`Common/Client/go/wshub`) and
  C++ StreamHub — that must behave identically; every client (SPA, ImGui, Qt)
  must satisfy both. JSON text frames for commands/events (`addSource`,
  `removeSource`, `setTrigger`, `arm`, `zoom`, `historyZoom`, `recStart`…), binary
  frames for data pushes (live v1 + trigger capture v2).
- **Threading model**: RT threads only spinlock+memcpy (`FastPollingMutexSem`);
  all socket I/O, fragmentation, and reassembly lives on background
  `SingleThreadService` threads. StreamHub: per-session UDPSClient receive
  threads + WS accept/read threads + one push loop.
- **DebugService patching**: `PatchRegistry()` replaces the ObjectBuilder for 11
  `MemoryMap*Broker` classes; runs only when `ControlPort > 0`; static guard
  against double-patching; wrappers persist for process lifetime.

## Key Directories

| Path | Purpose |
|---|---|
| `Source/Components/DataSources/UDPStreamer/` | Output DataSource; UDP I/O on bg thread, RT thread only spinlock+memcpy in `Synchronise()` |
| `Source/Components/DataSources/UDPStreamerClient/` | Input DataSource (shared `UDPSClient`), double-buffered ready/scratch |
| `Source/Components/GAMs/` | `SineArrayGAM` (float32 sine, continuous phase), `TimeArrayGAM` (us-timer → per-sample timestamp array; `Anchor = FirstSample|LastSample|Continuous`, use `Continuous` for contiguous sources so a lost RT cycle cannot hole the time base) |
| `Source/Components/Interfaces/DebugService/` | Registry patching, `DebugBrokerWrapper.h`, TCP/UDP services |
| `Source/Components/Interfaces/TCPLogger/` | `LoggerConsumerI` forwarding `REPORT_ERROR` to ≤8 TCP clients |
| `Source/Components/Interfaces/UDPStream/` | Plain-C++ helpers (not MARTe2 Objects): `UDPSClient` (auto-reconnect + fragment reassembly), `UDPSServer` (not thread-safe — owner's Execute thread only) |
| `Source/Applications/StreamHub/` | Standalone app (links MARTe2 core): `StreamHub`, `UDPSourceSession`, `WSServer`, `TriggerEngine`, `HistoryWriter`, `BinaryRecorder`, `LTTB`, `SignalRingBuffer` |
| `Common/UDP/` | Canonical wire protocol (header-only, MARTe2-free) |
| `Common/Client/go/` | Go mirror: `udpsprotocol` (decoder), `wshub` (WS hub client) |
| `Client/udpstreamer/` | Go legacy direct-UDP oscilloscope web UI (connects straight to UDPStreamer, no StreamHub) |
| `Client/webui/` | Go thin static server; SPA talks WS directly to C++ StreamHub (discovers via `GET /hub`) |
| `Client/debugger/` | Go debug web UI for DebugService |
| `Client/streamhub/` | Native ImGui+SDL2+OpenGL oscilloscope (C++17, no MARTe2) |
| `Client/streamhub-qt/` | Native Qt Widgets oscilloscope (Qt6 preferred, Qt5 fallback) |
| `Test/` | GTest, legacy Integration tests, Configurations (.cfg), E2E suite |
| `Docs/` | Per-component reference: `Protocol.md`, `UDPStreamer.md`, `StreamHub-{API,UserGuide,Developer}.md`, `DebugService.md`, `WebUI.md`, `Tutorial.md`, `E2E-Suite.md` |

## Development Commands

`source env.sh` is **mandatory** before any MARTe2 build or run (sets
`MARTe2_DIR`, `MARTe2_Components_DIR`, `TARGET=x86-linux`, `LD_LIBRARY_PATH`).
The E2E scripts source it themselves; a bare `make` from a fresh shell will not
work. `run_streamhub.sh` hard-errors if `MARTe2_DIR` is unset.

```bash
source env.sh

make -f Makefile.gcc core    # 7 components (UDPStream interface FIRST, then UDPStreamer, UDPStreamerClient, GAMs, TCPLogger, DebugService)
make -f Makefile.gcc apps    # StreamHub standalone app → Build/x86-linux/StreamHub/StreamHub.ex
make -f Makefile.gcc test    # GTest + Integration test binaries + component test libs
make -f Makefile.gcc all     # core + apps + test
make -f Makefile.gcc clean

# Single component:
make -C Source/Components/GAMs/SineArrayGAM -f Makefile.gcc
```

Build output → `Build/x86-linux/` mirroring `PACKAGE` paths (both `libX.so` and
`X.so` are produced). `compile_commands.json` (repo root, gitignored) feeds
LSP/clangd; CMake clients export their own into `Client/*/build/`.

### Non-MARTe2 clients (no env.sh needed)

```bash
cd Common/Client/go && go build ./...
cd Client/debugger && go build ./...
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
cd Client/streamhub-qt && cmake -B build && cmake --build build
```

### Key scripts

| Script | Purpose |
|---|---|
| `./run_streamhub.sh` | Demo stack: build + launch MARTe2 app + StreamHub, optional web (`-w`) / ImGui (`-g`) clients. Flags `-m/-c` MARTe2 dirs, `-b TARGET`, `-p WS_PORT`, `-n MAX_POINTS` (actual default 1000000, header says 10000), `-s` skip build. Generates temp hub cfg with `+History`/`+Recorder` blocks in `/tmp`. Ctrl-C kills all. |
| `./Test/E2E/suite/run_e2e.sh` | Full E2E: 57-scenario matrix + stress + unit suites + gcov coverage + Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--pdf-only`, `--skip-coverage`, `--skip-stress`, `--skip-datasources`, `--skip-recorder`, `--skip-debug`, `--skip-tcplogger` |
| `./Test/E2E/suite/run_stress.sh` | Capacity harness: sweeps one load axis at a time (`--axis`), hard gates survival+liveness, soft gates RSS+zoom-p95 |

## Code Conventions & Common Patterns

- **No STL in `Source/Components/**` (and StreamHub)**: use `StreamString` (not
  `std::string`), `FastPollingMutexSem`/`EventSem` (not `std::mutex`/threads),
  fixed arrays / MARTe2 `Vector<T>` (not `std::vector`), `REPORT_ERROR` /
  `REPORT_ERROR_STATIC` macros (no exceptions). C stdlib is fine. Heap
  `new`/`delete[]` is normal. STL/C++17 is fine in `Client/streamhub/` and
  `Client/streamhub-qt/`.
- **RT hot-path rule**: `FastPollingMutexSem` on real-time hot paths, never OS
  mutexes; RT cycle must not block on the scheduler.
- **Class registration**: `CLASS_REGISTER_DECLARATION()` in the class `public:`
  section of the header; `CLASS_REGISTER(Name, "1.0")` at the end of the `.cpp`
  inside `namespace MARTe`. Every component `.cpp` ends with it.
- **EUPL v1.1 license headers** on all C++ sources and `Makefile.inc` — preserve
  on new files.
- **Per-component build**: each dir has one-line `Makefile.gcc` wrapper
  (`include Makefile.inc`) + `Makefile.inc` declaring `OBJSX`, `PACKAGE`,
  `ROOT_DIR`, `INCLUDES` (re-declared per file, ~12 MARTe2 layer dirs),
  `LIBRARIES`, including `MakeStdLibDefs.$(TARGET)` then
  `MakeStdLibRules.$(TARGET)`. Generated `depends.x86-linux` (gcc -MM) is
  committed but **never hand-edited** — delete to regenerate.
- **Qt client**: `QT_NO_KEYWORDS` is required (reused `Protocol.h` structs have
  members named `signals`); Qt classes use `Q_SIGNALS`/`Q_SLOTS`/`Q_EMIT`. Run
  with long options: `--host HOST --port 8090` (single-dash misparsed). Single
  GUI thread, 60 Hz QTimer repaint.
- **StreamHub config** is *not* a MARTe2 `RealTimeApplication`: `Hub = { WSPort
  MaxPoints PushRate MaxPushPoints RingTemporal RingScalar RingMaxMB AllowedOrigins
  +Recorder{...} Sources={id={Label Addr Port}} }`. `AllowedOrigins` is the
  WebSocket Origin allowlist — without it a browser serving the SPA from a
  different port than the hub is rejected 403.
  `+History` keys: `Directory` (required),
  `DurationHours` (1), `Decimation` (1), `FlushIntervalSec` (5),
  `MinDiskFreeMB` (500). `.shist` files: 64-byte header ('SHR1') + circular
  (t,v) float64 pairs.
- **UDPStreamer config**: `Port` (44500; multicast data = `DataPort`, default
  `Port+1`), `MaxPayloadSize` (1400), `PublishingMode` `Strict`/`Accumulate`,
  per-signal `Signals={Name={Type,Unit,NumberOfDimensions,NumberOfElements,
  TimeMode}}` with `TimeMode` `PacketTime`/`FirstSample`/`LastSample`/`FullArray`;
  multicast needs `MulticastGroup` + `Interface`.

## Important Files

- `env.sh` — environment; source first, always.
- `Makefile.gcc` / `Makefile.inc` (root) — build orchestration.
- `Common/UDP/UDPSProtocol.h` — canonical wire format; changing it triggers the
  4-way mirror checklist above.
- `Source/Applications/StreamHub/main.cpp` — hub entry (`[-cfg file.cfg]
  [-port N] [-maxPoints N]`); hub **must be heap-allocated** (~128 MB, exceeds
  the 8 MB stack).
- `Test/Configurations/*.cfg` — MARTe2 app configs (`$App = { Class =
  RealTimeApplication }` with `+Functions`, `+DataSources`, `+States`, `+Timings`
  blocks); `streamhub_demo.cfg` and `TestApp.cfg` are good templates.
- `Test/E2E/suite/{scenarios,gen_data,gen_cfg,validate_waveform,stress}.py` —
  declarative scenario matrix and generators consumed identically by the Go
  chain-client and validators.
- `Client/debugger/main.go` — `-addr :7777` default, `-enable-dangerous-commands`
  safety gate (CR-4) for FORCE/PAUSE/RESUME/STEP/BREAK/MSG.

## Runtime/Tooling Preferences

- **OS**: Linux x86_64 (`TARGET=x86-linux`). External deps live outside this
  repo: `MARTe2_DIR` (default `~/workspace/MARTe2`) and
  `MARTe2_Components_DIR` (default `~/workspace/MARTe2-components`) — edit
  `env.sh` if they differ. `env.sh`'s `LD_LIBRARY_PATH` does **not** cover
  UDPStreamerClient/UDPStream lib dirs.
- **C++**: MARTe2 `Makefile.gcc` wrapper system, gtest-1.7.0 for tests.
- **Go**: `go 1.21`; modules use `replace marte2/common => ../../Common/Client/go`
  (`gorilla/websocket` v1.5.1). Go binaries are gitignored.
- **ImGui client**: needs SDL2; CMake FetchContent pins Dear ImGui **v1.91.8** +
  ImPlot **v0.17** (`implot_items.cpp` is a slow -O3 TU, ~2 min rebuild).
- **Qt client**: Qt6 preferred, Qt5 fallback, Widgets + WebSockets, custom
  QPainter plotting (no QtCharts).
- **E2E report**: `typst compile E2E_Report.typ`; Python 3 + numpy for the suite.
- Remove `vgore.*` core dumps when you see them; they are not gitignored.

## Testing & QA

Four test layers; `env.sh` + built stack required for all but the standalone
ones. Only `tests_py.py`, Go tests, and the built C++ test binaries run
standalone.

```bash
./Build/x86-linux/GTest/MainGTest.ex --gtest_filter='Name*'     # C++ GTest
./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex  # legacy DebugService runtime tests
cd Test/E2E/suite/client && go test ./...                        # Go chain-client unit tests
cd Test/E2E/suite && python3 -m unittest tests_py                # framework logic, standalone
```

- **GTest**: `MainGTest.ex` currently holds only `DebugServiceGTest`
  (TraceRingBuffer SPSC, DebugSignalInfo, BreakOp). Component GTests
  (`UDPStreamerGTest.cpp` ~46 cases, `StreamHubTest.a`, `UDPStreamerClientTest.a`)
  compile **as libraries only — no standalone executable**.
- **Legacy IntegrationTests.ex**: 9 printf-narrated DebugService runtime tests,
  always returns 0; `collect.py` parses stdout blocks.
- **E2E suite** (`run_e2e.sh`): 57 curated scenarios (s01–s57) across kinds
  `chain`/`direct`/`recorder`/`debug`/`debug_pause_resume`/`tcplogger`, driven
  against live MARTeApp.ex + StreamHub.ex + Go chain-client. `scenarios.py` is a
  curated covering set: **every configurable UDPStreamer option value appears in
  ≥1 scenario** — add a scenario when adding an option.
- **Oracle gates** (`validate_waveform.py`): **fidelity** (every received value
  within `tol` of ground truth; 0 for un-quantised ints, float epsilon for
  un-quantised floats, `quant_step/2 + 1e-6·range` for quantised) is the
  **correctness gate**. **Shape** is a *gross* sanity gate + tracked metric
  (`corr >= 0.5`, `nRMSE <= 0.30` relaxed by quant step, frequency searched
  ±5% band); a correct sinusoid yields corr ~0.82–0.98, wrong frequency
  collapses to ~0.00. Do **not** tighten shape into a correctness gate —
  timestamp calibration (Phase-A) is pending.
- **Stress** (`run_stress.sh`): 7 axes (signal size/count/fan-out/sources/WS
  clients/zoom rate), hard gates survival+liveness, soft gates RSS+zoom-p95.
- **Coverage**: `--cpp-coverage` rebuilds with gcov, captures via `lcov`
  restricted to `Source/*` + `Test/*`, then restores a clean build.
- Artifacts → `Build/x86-linux/E2E/chain/`: `results.json` (XFAIL/XPASS for
  `known_issue` markers), `report_data.json`, `history.jsonl`, `trend_*.png`,
  `E2E_Report.pdf`; stress → `stress/stress_results.json`.

## Ports Reference (defaults)

| Port | Protocol | Component | Purpose |
|---|---|---|---|
| 44500 | UDP | UDPStreamer | scalar signals (unicast control + data) |
| 44501/44502 | UDP | UDPStreamer | packed arrays (FirstSample/LastSample, FullArray) |
| 44503 | UDP | UDPStreamer | multicast data (group 239.0.0.1) |
| 8080 | TCP | DebugService | text command channel (one client at a time, newline-terminated) |
| 8081 | UDP | DebugService | trace telemetry (UDPS format) |
| 8082 | TCP | TcpLogger | REPORT_ERROR log forward |
| 8090 | TCP/WS | StreamHub | WebSocket (commands + binary data) |
| 7777 | TCP | Client/debugger | debug web UI (older docs say 9090; current flag is `-addr`) |
| 8080 | TCP | Client/udpstreamer, Client/webui | web UI listen (collides with DebugService in combined demos — scripts adjust) |