Repository Guidelines

Guide for AI assistants working in the MARTe2 Integrated Components repository. Focuses on non-obvious facts: commands, conventions, cross-module contracts, and gotchas that are not self-evident from a single file read.

Project Overview

MARTe2 component library with two independent real-time data paths sharing one binary wire protocol (Common/UDP/UDPSProtocol.h):

Streaming path — UDPStreamer DataSource serialises DDB signals into UDPS binary packets on UDP → StreamHub (headless C++ hub: ring buffers, LTTB decimation, trigger FSM, history writer, binary recorder) → WebSocket 8090 → clients (browser SPA, native ImGui, native Qt).
Debug path — DebugService patches ClassRegistryDatabase at Initialise() so ConfigureApplication() wraps all MemoryMap*Broker types with DebugBrokerWrapper<T> — zero application code changes. Exposes TCP 8080 (text commands), UDP 8081 (UDPS trace telemetry), TCP 8082 (TcpLogger log forward).

Architecture & Data Flow

[SineArrayGAM/TimeArrayGAM] → DDB → UDPStreamer (UDPS over UDP)
   ├─→ UDPStreamerClient (input DS back into a MARTe2 RT app, round-trip)
   └─→ StreamHub: UDPSourceSession (receive thread → SignalRingBuffer)
        → push loop @30Hz: LTTB decimate temporal sigs → WS binary frames → clients
DebugService: patches broker builders at Initialise(); TCP 8080 commands,
UDP 8081 telemetry, TcpLogger 8082 (REPORT_ERROR → "LOG <LEVEL> <desc>" lines)

Wire protocol: Common/UDP/UDPSProtocol.h is the canonical spec (17-byte packed header, magic 0x53504455 'UDPS', 136-byte signal descriptors, CONFIG/DATA/ACK/CONNECT/DISCONNECT packet types, quant/time/publish modes). Deliberately MARTe2-free so Go clients reuse it. Mirrored across four codebases that must stay in sync: C++ producers (UDPStreamer, DebugService), C++ consumer (Source/Components/Interfaces/UDPStream/UDPSClient), Go decoder (Common/Client/go/udpsprotocol/protocol.go), and JS parsers (Client/udpstreamer/static/, Client/debugger/static/). Any protocol change must be mirrored in all of them.
WS protocol has two implementations — Go hub (Common/Client/go/wshub) and C++ StreamHub — that must behave identically; every client (SPA, ImGui, Qt) must satisfy both. JSON text frames for commands/events (addSource, removeSource, setTrigger, arm, zoom, historyZoom, recStart…), binary frames for data pushes (live v1 + trigger capture v2).
Threading model: RT threads only spinlock+memcpy (FastPollingMutexSem); all socket I/O, fragmentation, and reassembly lives on background SingleThreadService threads. StreamHub: per-session UDPSClient receive threads + WS accept/read threads + one push loop.
DebugService patching: PatchRegistry() replaces the ObjectBuilder for 11 MemoryMap*Broker classes; runs only when ControlPort > 0; static guard against double-patching; wrappers persist for process lifetime.

Key Directories

Path	Purpose
`Source/Components/DataSources/UDPStreamer/`	Output DataSource; UDP I/O on bg thread, RT thread only spinlock+memcpy in `Synchronise()`
`Source/Components/DataSources/UDPStreamerClient/`	Input DataSource (shared `UDPSClient`), double-buffered ready/scratch
`Source/Components/GAMs/`	`SineArrayGAM` (float32 sine, continuous phase), `TimeArrayGAM` (us-timer → per-sample timestamp array)
`Source/Components/Interfaces/DebugService/`	Registry patching, `DebugBrokerWrapper.h`, TCP/UDP services
`Source/Components/Interfaces/TCPLogger/`	`LoggerConsumerI` forwarding `REPORT_ERROR` to ≤8 TCP clients
`Source/Components/Interfaces/UDPStream/`	Plain-C++ helpers (not MARTe2 Objects): `UDPSClient` (auto-reconnect + fragment reassembly), `UDPSServer` (not thread-safe — owner's Execute thread only)
`Source/Applications/StreamHub/`	Standalone app (links MARTe2 core): `StreamHub`, `UDPSourceSession`, `WSServer`, `TriggerEngine`, `HistoryWriter`, `BinaryRecorder`, `LTTB`, `SignalRingBuffer`
`Common/UDP/`	Canonical wire protocol (header-only, MARTe2-free)
`Common/Client/go/`	Go mirror: `udpsprotocol` (decoder), `wshub` (WS hub client)
`Client/udpstreamer/`	Go legacy direct-UDP oscilloscope web UI (connects straight to UDPStreamer, no StreamHub)
`Client/webui/`	Go thin static server; SPA talks WS directly to C++ StreamHub (discovers via `GET /hub`)
`Client/debugger/`	Go debug web UI for DebugService
`Client/streamhub/`	Native ImGui+SDL2+OpenGL oscilloscope (C++17, no MARTe2)
`Client/streamhub-qt/`	Native Qt Widgets oscilloscope (Qt6 preferred, Qt5 fallback)
`Test/`	GTest, legacy Integration tests, Configurations (.cfg), E2E suite
`Docs/`	Per-component reference: `Protocol.md`, `UDPStreamer.md`, `StreamHub-{API,UserGuide,Developer}.md`, `DebugService.md`, `WebUI.md`, `Tutorial.md`, `E2E-Suite.md`

Development Commands

source env.sh is mandatory before any MARTe2 build or run (sets MARTe2_DIR, MARTe2_Components_DIR, TARGET=x86-linux, LD_LIBRARY_PATH). The E2E scripts source it themselves; a bare make from a fresh shell will not work. run_streamhub.sh hard-errors if MARTe2_DIR is unset.

source env.sh

make -f Makefile.gcc core    # 7 components (UDPStream interface FIRST, then UDPStreamer, UDPStreamerClient, GAMs, TCPLogger, DebugService)
make -f Makefile.gcc apps    # StreamHub standalone app → Build/x86-linux/StreamHub/StreamHub.ex
make -f Makefile.gcc test    # GTest + Integration test binaries + component test libs
make -f Makefile.gcc all     # core + apps + test
make -f Makefile.gcc clean

# Single component:
make -C Source/Components/GAMs/SineArrayGAM -f Makefile.gcc

Build output → Build/x86-linux/ mirroring PACKAGE paths (both libX.so and X.so are produced). compile_commands.json (repo root, gitignored) feeds LSP/clangd; CMake clients export their own into Client/*/build/.

Non-MARTe2 clients (no env.sh needed)

cd Common/Client/go && go build ./...
cd Client/debugger && go build ./...
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
cd Client/streamhub-qt && cmake -B build && cmake --build build

Key scripts

Script	Purpose
`./run_streamhub.sh`	Demo stack: build + launch MARTe2 app + StreamHub, optional web (`-w`) / ImGui (`-g`) clients. Flags `-m/-c` MARTe2 dirs, `-b TARGET`, `-p WS_PORT`, `-n MAX_POINTS` (actual default 1000000, header says 10000), `-s` skip build. Generates temp hub cfg with `+History`/`+Recorder` blocks in `/tmp`. Ctrl-C kills all.
`./Test/E2E/suite/run_e2e.sh`	Full E2E: 57-scenario matrix + stress + unit suites + gcov coverage + Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--pdf-only`, `--skip-coverage`, `--skip-stress`, `--skip-datasources`, `--skip-recorder`, `--skip-debug`, `--skip-tcplogger`
`./Test/E2E/suite/run_stress.sh`	Capacity harness: sweeps one load axis at a time (`--axis`), hard gates survival+liveness, soft gates RSS+zoom-p95

Code Conventions & Common Patterns

No STL in Source/Components/** (and StreamHub): use StreamString (not std::string), FastPollingMutexSem/EventSem (not std::mutex/threads), fixed arrays / MARTe2 Vector<T> (not std::vector), REPORT_ERROR / REPORT_ERROR_STATIC macros (no exceptions). C stdlib is fine. Heap new/delete[] is normal. STL/C++17 is fine in Client/streamhub/ and Client/streamhub-qt/.
RT hot-path rule: FastPollingMutexSem on real-time hot paths, never OS mutexes; RT cycle must not block on the scheduler.
Class registration: CLASS_REGISTER_DECLARATION() in the class public: section of the header; CLASS_REGISTER(Name, "1.0") at the end of the .cpp inside namespace MARTe. Every component .cpp ends with it.
EUPL v1.1 license headers on all C++ sources and Makefile.inc — preserve on new files.
Per-component build: each dir has one-line Makefile.gcc wrapper (include Makefile.inc) + Makefile.inc declaring OBJSX, PACKAGE, ROOT_DIR, INCLUDES (re-declared per file, ~12 MARTe2 layer dirs), LIBRARIES, including MakeStdLibDefs.$(TARGET) then MakeStdLibRules.$(TARGET). Generated depends.x86-linux (gcc -MM) is committed but never hand-edited — delete to regenerate.
Qt client: QT_NO_KEYWORDS is required (reused Protocol.h structs have members named signals); Qt classes use Q_SIGNALS/Q_SLOTS/Q_EMIT. Run with long options: --host HOST --port 8090 (single-dash misparsed). Single GUI thread, 60 Hz QTimer repaint.
StreamHub config is not a MARTe2 RealTimeApplication: Hub = { WSPort MaxPoints PushRate MaxPushPoints RingTemporal RingScalar +Recorder{...} Sources={id={Label Addr Port}} }. +History keys: Directory (required), DurationHours (1), Decimation (1), FlushIntervalSec (5), MinDiskFreeMB (500). .shist files: 64-byte header ('SHR1') + circular (t,v) float64 pairs.
UDPStreamer config: Port (44500; multicast data = DataPort, default Port+1), MaxPayloadSize (1400), PublishingMode Strict/Accumulate, per-signal Signals={Name={Type,Unit,NumberOfDimensions,NumberOfElements, TimeMode}} with TimeMode PacketTime/FirstSample/LastSample/FullArray; multicast needs MulticastGroup + Interface.

Important Files

env.sh — environment; source first, always.
Makefile.gcc / Makefile.inc (root) — build orchestration.
Common/UDP/UDPSProtocol.h — canonical wire format; changing it triggers the 4-way mirror checklist above.
Source/Applications/StreamHub/main.cpp — hub entry ([-cfg file.cfg] [-port N] [-maxPoints N]); hub must be heap-allocated (~128 MB, exceeds the 8 MB stack).
Test/Configurations/*.cfg — MARTe2 app configs ($App = { Class = RealTimeApplication } with +Functions, +DataSources, +States, +Timings blocks); streamhub_demo.cfg and TestApp.cfg are good templates.
Test/E2E/suite/{scenarios,gen_data,gen_cfg,validate_waveform,stress}.py — declarative scenario matrix and generators consumed identically by the Go chain-client and validators.
Client/debugger/main.go — -addr :7777 default, -enable-dangerous-commands safety gate (CR-4) for FORCE/PAUSE/RESUME/STEP/BREAK/MSG.

Runtime/Tooling Preferences

OS: Linux x86_64 (TARGET=x86-linux). External deps live outside this repo: MARTe2_DIR (default ~/workspace/MARTe2) and MARTe2_Components_DIR (default ~/workspace/MARTe2-components) — edit env.sh if they differ. env.sh's LD_LIBRARY_PATH does not cover UDPStreamerClient/UDPStream lib dirs.
C++: MARTe2 Makefile.gcc wrapper system, gtest-1.7.0 for tests.
Go: go 1.21; modules use replace marte2/common => ../../Common/Client/go (gorilla/websocket v1.5.1). Go binaries are gitignored.
ImGui client: needs SDL2; CMake FetchContent pins Dear ImGui v1.91.8 + ImPlot v0.17 (implot_items.cpp is a slow -O3 TU, ~2 min rebuild).
Qt client: Qt6 preferred, Qt5 fallback, Widgets + WebSockets, custom QPainter plotting (no QtCharts).
E2E report: typst compile E2E_Report.typ; Python 3 + numpy for the suite.
Remove vgore.* core dumps when you see them; they are not gitignored.

Testing & QA

Four test layers; env.sh + built stack required for all but the standalone ones. Only tests_py.py, Go tests, and the built C++ test binaries run standalone.

./Build/x86-linux/GTest/MainGTest.ex --gtest_filter='Name*'     # C++ GTest
./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex  # legacy DebugService runtime tests
cd Test/E2E/suite/client && go test ./...                        # Go chain-client unit tests
cd Test/E2E/suite && python3 -m unittest tests_py                # framework logic, standalone

GTest: MainGTest.ex currently holds only DebugServiceGTest (TraceRingBuffer SPSC, DebugSignalInfo, BreakOp). Component GTests (UDPStreamerGTest.cpp ~46 cases, StreamHubTest.a, UDPStreamerClientTest.a) compile as libraries only — no standalone executable.
Legacy IntegrationTests.ex: 9 printf-narrated DebugService runtime tests, always returns 0; collect.py parses stdout blocks.
E2E suite (run_e2e.sh): 57 curated scenarios (s01–s57) across kinds chain/direct/recorder/debug/debug_pause_resume/tcplogger, driven against live MARTeApp.ex + StreamHub.ex + Go chain-client. scenarios.py is a curated covering set: every configurable UDPStreamer option value appears in ≥1 scenario — add a scenario when adding an option.
Oracle gates (validate_waveform.py): fidelity (every received value within tol of ground truth; 0 for un-quantised ints, float epsilon for un-quantised floats, quant_step/2 + 1e-6·range for quantised) is the correctness gate. Shape is a gross sanity gate + tracked metric (corr >= 0.5, nRMSE <= 0.30 relaxed by quant step, frequency searched ±5% band); a correct sinusoid yields corr ~0.82–0.98, wrong frequency collapses to ~0.00. Do not tighten shape into a correctness gate — timestamp calibration (Phase-A) is pending.
Stress (run_stress.sh): 7 axes (signal size/count/fan-out/sources/WS clients/zoom rate), hard gates survival+liveness, soft gates RSS+zoom-p95.
Coverage: --cpp-coverage rebuilds with gcov, captures via lcov restricted to Source/* + Test/*, then restores a clean build.
Artifacts → Build/x86-linux/E2E/chain/: results.json (XFAIL/XPASS for known_issue markers), report_data.json, history.jsonl, trend_*.png, E2E_Report.pdf; stress → stress/stress_results.json.

Ports Reference (defaults)

Port	Protocol	Component	Purpose
44500	UDP	UDPStreamer	scalar signals (unicast control + data)
44501/44502	UDP	UDPStreamer	packed arrays (FirstSample/LastSample, FullArray)
44503	UDP	UDPStreamer	multicast data (group 239.0.0.1)
8080	TCP	DebugService	text command channel (one client at a time, newline-terminated)
8081	UDP	DebugService	trace telemetry (UDPS format)
8082	TCP	TcpLogger	REPORT_ERROR log forward
8090	TCP/WS	StreamHub	WebSocket (commands + binary data)
7777	TCP	Client/debugger	debug web UI (older docs say 9090; current flag is `-addr`)
8080	TCP	Client/udpstreamer, Client/webui	web UI listen (collides with DebugService in combined demos — scripts adjust)

14 KiB Raw Blame History Unescape Escape