martino/MARTe-Integrated-Components
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
45dcb9a71f991ce465d3a710ee7bfece91666e17
MARTe-Integrated-Components/AGENTS.md
T

15 KiB
Raw Blame History

AGENTS.md

Guide for agents working in the MARTe2 Integrated Components repository. Read ARCHITECTURE.md and CLAUDE.md for deeper detail; this file focuses on non-obvious knowledge, commands, and conventions that are not self-evident from a single file read.

Repository at a glance

MARTe2 component library with two independent real-time data paths:

  1. Streaming path — UDPStreamer DataSource (serialises signals to UDPS binary packets on UDP 44500) → StreamHub (headless C++ hub: ring buffers, LTTB decimation, trigger FSM, history writer) → WebSocket 8090 → clients (browser SPA, native ImGui, native Qt).
  2. Debug path — DebugService Interface patches ClassRegistryDatabase at Initialise() so ConfigureApplication() wraps all MemoryMap*Broker types with DebugBrokerWrapper<T>. Zero application code changes. Exposes TCP 8080 (text commands), UDP 8081 (trace telemetry), TCP 8082 (TcpLogger).

Key shared artefact: Common/UDP/UDPSProtocol.h — the UDPS binary wire format (17-byte packed header, 136-byte signal descriptors, little-endian). It is deliberately MARTe2-free and is mirrored across four codebases that must stay in sync: C++ producers (UDPStreamer, DebugService), C++ consumer (Source/Components/Interfaces/UDPStream/UDPSClient), Go decoder (Common/Client/go/udpsprotocol), and the JS client parsers. Any protocol change must be mirrored in all of them.

StreamHub WebSocket protocol: JSON text frames for commands/events, binary frames for data pushes (spec in ARCHITECTURE.md §6). The Go hub (Client/udpstreamer) and the C++ StreamHub implement the identical protocol; browser JS, ImGui, and Qt clients must stay compatible with both.

Environment setup (do this first, always)

source env.sh is required before building or running anything. It sets:

  • MARTe2_DIR (default ~/workspace/MARTe2) and MARTe2_Components_DIR (default ~/workspace/MARTe2-components) — external dependencies, not in this repo. Edit env.sh if your MARTe2 install lives elsewhere.
  • TARGET=x86-linux
  • LD_LIBRARY_PATH — prepends MARTe2 Core, MARTe2-components (LinuxTimer, FileDataSource), and this repo's built .sos (UDPStreamer, SineArrayGAM, TimeArrayGAM, DebugService, TCPLogger).

Without sourcing env.sh, builds fail (can't find MakeDefaults) and binaries fail to load shared libs. The E2E scripts source it themselves, but a bare make or ./MainGTest.ex from a fresh shell will not.

Build commands

All C++ builds use the MARTe2 Makefile.gcc wrapper system. There is one top-level Makefile.gcc and one per component directory. TARGET defaults to x86-linux; pass TARGET=x86-linux explicitly when invoking make -C from elsewhere.

source env.sh

make -f Makefile.gcc core    # all C++ MARTe2 components (UDPStreamer, GAMs, DebugService, TCPLogger, UDPStream)
make -f Makefile.gcc apps    # StreamHub standalone app
make -f Makefile.gcc test    # build GTest + Integration test binaries
make -f Makefile.gcc clean
make -f Makefile.gcc         # = all: core apps test

# Build a single component (each component dir has its own Makefile.gcc):
make -C Source/Components/DataSources/UDPStreamer -f Makefile.gcc
make -C Source/Applications/StreamHub -f Makefile.gcc

Build output goes to Build/x86-linux/ — shared libs under Build/x86-linux/Components/<...>/, the StreamHub executable at Build/x86-linux/StreamHub/StreamHub.ex, test binaries under Build/x86-linux/GTest/ and Build/x86-linux/Test/Integration/.

compile_commands.json is generated at the repo root (gitignored) for LSP / clangd. The CMake-based clients (ImGui, Qt) also export it into their build/.

Non-MARTe2 clients (separate build systems, no env.sh needed)

# Go clients (UDPS web client, debug client, E2E chain-client)
cd Common/Client/go && go build ./...
cd Client/debugger   && go build ./...
cd Client/udpstreamer && go build ./...        # or: cd Client/webui && go build
cd Test/E2E/suite/client && go build ./...     # chain-client (E2E driver) + its tests

# ImGui desktop client (needs SDL2; fetches Dear ImGui + ImPlot via FetchContent)
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build

# Qt desktop client (Qt5 or Qt6 Widgets + WebSockets; autodetects, prefers Qt6)
cd Client/streamhub-qt && cmake -B build && cmake --build build

Run / test commands

Unit & integration tests (C++)

./Build/x86-linux/GTest/MainGTest.ex                              # UDPStreamer unit tests
./Build/x86-linux/GTest/MainGTest.ex --gtest_filter='Name*'       # single test
./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex   # DebugService integration tests

Go tests

cd Test/E2E/suite/client && go test ./...

Python framework tests (E2E framework logic, no live stack needed)

cd Test/E2E/suite && python3 -m unittest tests_py

E2E / demo scripts (build + launch the full stack)

Script What it does
./run_streamhub.sh Launch MARTe2 app (UDPStreamer) + StreamHub, optionally web UI (-w) and ImGui client (-g). Ports documented in its header.
./Test/E2E/suite/run_e2e.sh Streaming-chain E2E suite: per-scenario generates data + cfgs, runs MARTe2+StreamHub, drives the Go chain-client (live/zoom/window/trigger), validates the recorded waveform against an analytic/fed oracle, runs unit suites + coverage, builds a Typst PDF report. Flags: --skip-build, --only <id>, --cpp-coverage, --pdf-only, --stress.
./Test/E2E/suite/run_stress.sh Capacity/stress harness: sweeps one load axis at a time (signal size/count, subscriber fan-out, source count, WS-client count, zoom rate), gates on survival+liveness (hard) and RSS+zoom-p95 latency (soft). Flags: --skip-build, --only <id>, --axis <axis>.

The E2E suite produces Build/x86-linux/E2E/chain/ artifacts: report_data.json, history.jsonl (per-run headline metrics for trend/regression tracking), trend_*.png plots, and E2E_Report.pdf (compiled from E2E_Report.typ via typst). --cpp-coverage triggers an instrumented --coverage rebuild, captures with lcov (restricted to Source/* + Test/*), then restores a clean build.

Code organization

Common/
  UDP/UDPSProtocol.h            # shared UDPS binary protocol (header-only, MARTe2-free)
  Client/go/                    # Go packages: udpsprotocol (decoder), wshub (WS hub client)
Source/
  Components/                   # MARTe2 components — NO STL allowed here
    DataSources/UDPStreamer/    # real-time UDP signal streaming DataSource
    DataSources/UDPStreamerClient/   # (MARTe2-side UDPS client DataSource)
    GAMs/SineArrayGAM/          # sine-wave array generator GAM
    GAMs/TimeArrayGAM/          # time-reference array GAM
    Interfaces/DebugService/    # tracing/forcing/breakpoint Interface (registry-patched brokers)
    Interfaces/TCPLogger/       # REPORT_ERROR → TCP forwarder (LoggerConsumerI)
    Interfaces/UDPStream/       # UDPSClient/UDPSServer (C++ consumer/producer of UDPS)
  Applications/StreamHub/       # headless C++ hub app (links MARTe2 core but follows MARTe2 style)
Client/
  debugger/                     # Go web client for DebugService (browser UI in static/)
  udpstreamer/                  # Go web server + static SPA for UDPStreamer (legacy direct-UDP path)
  webui/                        # Go StreamHub WebSocket web UI (newer, hub-based)
  streamhub/                    # native ImGui desktop client (SDL2 + OpenGL + ImPlot, C++17, no MARTe2)
  streamhub-qt/                 # native Qt Widgets desktop client (C++17, Qt5/Qt6, no MARTe2)
Test/
  GTest/                        # GTest harness (MainGTest.cpp) for UDPStreamer unit tests
  Integration/                  # DebugService integration tests (link against DebugService .so)
  Components/DataSources/UDPStreamer/   # UDPStreamer unit test sources
  Configurations/               # MARTe2 .cfg files for tests and demos
  E2E/
    chain/                      # streaming-chain E2E + stress suite (Python orchestrator + Go client)
    datasources/ recorder/ streamhub/   # older per-component E2E scripts
Docs/                           # per-component reference docs (Protocol, UDPStreamer, StreamHub-*, DebugService, ...)
docs/superpowers/{specs,plans}/ # design specs + implementation plans (dated)
ARCHITECTURE.md                 # full architecture (data flow diagrams, protocol tables, WS protocol)

Component layout convention

Every MARTe2 component directory contains:

  • Makefile.gcc — thin wrapper (TARGET = x86-linux then include Makefile.inc).
  • Makefile.inc — real build definition: OBJSX, PACKAGE, INCLUDES, LIBRARIES, includes $(MARTe2_DIR)/MakeDefaults/MakeStdLibDefs.$(TARGET) and MakeStdLibRules.$(TARGET), and depends.$(TARGET).
  • depends.x86-linux / dependsRaw.x86-linux — generated header dependency files (committed; regenerated by the build). Do not hand-edit.
  • <ClassName>.{h,cpp} — the implementation, with CLASS_REGISTER(ClassName, "1.0") at the bottom of the .cpp (MARTe2 class-registration macro).

To add a new component, copy an existing one and adapt Makefile.inc's OBJSX / PACKAGE / INCLUDES / LIBRARIES.

Conventions and gotchas

No STL in MARTe2 components

Source/Components/** must not use the C++ standard library. Use MARTe2 types: StreamString (not std::string), FastPollingMutexSem (not std::mutex), fixed arrays / StaticList (not std::vector), MARTe2 error macros (REPORT_ERROR, REPORT_ERROR_STATIC) instead of exceptions. Includes come from $(MARTe2_DIR)/Source/Core/... (BareMetal, Scheduler, FileSystem).

Source/Applications/StreamHub/ links MARTe2 core and follows the same no-STL-on-RT-paths style, but is a standalone app (not a loadable component).

STL / C++17 is fine in Client/streamhub/ and Client/streamhub-qt/ (framework-free C++17, no MARTe2 dependency).

RT hot-path mutex rule

Use FastPollingMutexSem on real-time hot paths — never OS mutexes (pthread_mutex, std::mutex). The RT cycle must not block on the scheduler.

Protocol change checklist

Changing Common/UDP/UDPSProtocol.h (or the Go mirror) requires mirroring across all four consumers listed in §1, plus the JS client parsers in Client/udpstreamer/static/. The Go decoder lives in Common/Client/go/udpsprotocol/protocol.go and re-declares the constants (HeaderSize, SigDescSize, packet types, quant types, time modes) — it does not include the C header.

Qt client: QT_NO_KEYWORDS

Client/streamhub-qt/ reuses ../streamhub/Protocol.{h,cpp} and SignalBuffer.h verbatim. Those structs have members named signals, which collide with Qt's signals/slots/emit macros. The Qt build sets QT_NO_KEYWORDS and all Qt classes use Q_SIGNALS:/Q_SLOTS:/Q_EMIT. Do not remove that define, and do not use the lowercase Qt keywords in Qt-client code. Single GUI thread — QWebSocket signals arrive on the GUI thread, no locks; a 60 Hz QTimer drives repaint. Run with long options: ./build/StreamHubQtClient --host HOST --port 8090 (single-dash -host is misparsed as clustered short flags).

StreamHub history

HistoryWriter (Source/Applications/StreamHub/HistoryWriter.{h,cpp}) does disk-backed circular storage: per-signal .shist files with a 64-byte header and a pre-allocated circular region of (float64 time, float64 value) pairs. Configured via a +History block (Directory required; DurationHours default 1, Decimation default 1, FlushIntervalSec default 5, MinDiskFreeMB default 500). WS commands: historyInfo, historyZoom.

E2E scenario / stress matrix

Test/E2E/suite/scenarios.py is a curated covering set: every configurable UDPStreamer option value appears in at least one scenario, plus high-risk interactions. stress.py is the capacity sibling — it sweeps one load axis at a time and records survival/liveness (hard gates) and RSS/zoom-p95 latency (soft gates). Both feed the same gen_data.py / gen_cfg.py generators. When adding a new UDPStreamer option, add a scenario that exercises it and update validate_scenario / the stress matrix accordingly.

validate_waveform.py has two gates: fidelity (every received value within tol of some ground-truth value; tol is 0 for un-quantised ints, a float epsilon for un-quantised floats, quant_step/2 + 1e-6·range for quantised floats) — this is the correctness gate. Shape (least-squares sine fit, correlation ≥ 0.99) is a gross-sanity gate + tracked metric, pending Phase-A timestamp calibration. Do not tighten the shape gate to a correctness gate.

Gitignored build artefacts

Build/, compile_commands.json, *.gcov/*.gcda/*.gcno (coverage), Go binaries (Client/*/marte2debugger, udpstreamer-webui, streamhub-e2e, Client/*/bin/), and the udp_standalone_webui/ scratch dir are all gitignored. vgore.* core dumps (e.g. vgcore.26968) are not — remove them.

Documentation map

  • ARCHITECTURE.md — full architecture: data-flow diagrams, UDPS packet/header tables, CONFIG payload, StreamHub WebSocket protocol (commands, events, binary frames), threading model.
  • Docs/ — per-component reference docs (Protocol.md, UDPStreamer.md, SineArrayGAM.md, DebugService.md, StreamHub-{UserGuide,API,Developer}.md, Tutorial.md, WebUI.md).
  • docs/superpowers/specs/ — design specs (dated, e.g. 2026-06-26-stress-suite-report-integration-design.md).
  • docs/superpowers/plans/ — implementation plans paired with specs.
  • .superpowers/sdd/ — task briefs, reports, and a progress.md ledger for in-flight feature work (not always present; per-branch).
  • README.md — high-level overview + quick-start config snippets.
  • CLAUDE.md — condensed agent guidance (overlaps with this file).

Ports reference (defaults)

Port Protocol Component Purpose
44500 UDP UDPStreamer scalar signals (unicast control)
44501 UDP UDPStreamer packed arrays (FirstSample/LastSample)
44502 UDP UDPStreamer packed arrays (FullArray)
44503 UDP UDPStreamer multicast data (group 239.0.0.1)
8080 TCP DebugService text command channel
8081 UDP DebugService trace telemetry stream
8082 TCP TcpLogger REPORT_ERROR log forward
8090 TCP/WS StreamHub WebSocket (commands + binary data)
8080 TCP udpstreamer-webui web UI listen (legacy direct-UDP path)
9090 TCP debugger debug web UI listen

Note the 8080 collision: DebugService control vs. the legacy web UI listen port. In combined demos that run both, the scripts adjust one of them — check the script header before assuming.

License

EUPL v1.1 — license headers are present on all C++ sources. Preserve them on new files; do not relicense.

Reference in New Issue View Git Blame Copy Permalink