Martino Ferrari
13 KiB
System Architecture
1. Overview
The MARTe2 Debug Suite is a C++ server library that instruments running MARTe2 applications
without modifying their source code. It exposes an abstract DebugServiceI interface that
decouples the broker injection layer from the communication transport, allowing multiple
transport backends to coexist (currently TCP/UDP and HTTP/SSE).
┌────────────────────────────────────────────────────────────────────┐
│ MARTe2 Application │
│ ┌──────────┐ ┌───────────────────────┐ ┌──────────────────┐ │
│ │ GAM(s) │ │ DebugBrokerWrapper │ │ FastScheduler │ │
│ │ │◄──│ (Registry-patched) │ │ (unmodified) │ │
│ └──────────┘ └──────────┬────────────┘ └──────────────────┘ │
│ │ RT-path API │
└────────────────────────────┼───────────────────────────────────────┘
│
┌────────▼────────┐
│ DebugServiceI │ ← Abstract interface (singleton)
└────────┬────────┘
┌─────────────┴──────────────┐
│ │
┌────────▼────────┐ ┌─────────▼────────┐
│ DebugService │ │ WebDebugService │
│ TCP/UDP │ │ HTTP/SSE │
└────────┬────────┘ └─────────┬─────────┘
│ │
┌──────────┼──────────┐ ┌──────────┼──────────┐
│TCP 8080 │UDP 8081 │ │HTTP 8090 │SSE stream │
│(commands)│(telemetry│ │(commands)│(telemetry)│
└────┬─────┴────┬─────┘ └─────┬────┴────┬──────┘
│ │ │ │
┌────▼──────────▼─────┐ ┌─────────▼─────────▼──────┐
│ Rust/egui GUI │ │ Web Browser (built-in UI)│
│ (Tools/gui_client) │ │ http://localhost:8090 │
└─────────────────────┘ └───────────────────────────┘
2. Core Mechanism: Registry Patching
The "Zero-Code-Change" requirement is met by intercepting the MARTe2 ClassRegistryDatabase.
When DebugService or WebDebugService initialises, PatchRegistry() replaces the
ObjectBuilder for every standard MemoryMap*Broker class in the registry. When
RealTimeApplication::ConfigureApplication() runs afterward it instantiates wrapped broker
objects instead of the originals — the application sees no difference.
Patched broker types:
| Class | Wrapper |
|---|---|
MemoryMapInputBroker |
DebugBrokerWrapper<MemoryMapInputBroker> |
MemoryMapOutputBroker |
DebugBrokerWrapper<MemoryMapOutputBroker> |
MemoryMapSynchronisedInputBroker |
DebugBrokerWrapper<MemoryMapSynchronisedInputBroker> |
MemoryMapSynchronisedOutputBroker |
DebugBrokerWrapper<MemoryMapSynchronisedOutputBroker> |
MemoryMapMultiBufferBroker |
DebugBrokerWrapper<MemoryMapMultiBufferBroker> |
MemoryMapMultiBufferOutputBroker |
DebugBrokerWrapper<MemoryMapMultiBufferOutputBroker> |
MemoryMapAsynchronousInputBroker |
DebugBrokerWrapperAsync<MemoryMapAsynchronousInputBroker> |
MemoryMapAsynchronousOutputBroker |
DebugBrokerWrapperAsync<MemoryMapAsynchronousOutputBroker> |
MemoryMapInterpolatedInputBroker |
DebugBrokerWrapper<MemoryMapInterpolatedInputBroker> |
MemoryMapStatefulOutputBroker |
DebugBrokerWrapper<MemoryMapStatefulOutputBroker> |
MemoryMapStatefulInputBroker |
DebugBrokerWrapper<MemoryMapStatefulInputBroker> |
3. DebugServiceI Abstraction Layer
Source/Components/Interfaces/DebugService/DebugServiceI.h defines the abstract singleton
interface. It has two logical groups:
3.1 RT-path API
Called from broker threads on every RT cycle. Must be as cheap as possible; the only
permissible synchronisation primitive is FastPollingMutexSem.
| Method | Purpose |
|---|---|
RegisterSignal(addr, type, name, …) |
Called once per signal during broker Init() |
ProcessSignal(info, size, timestamp) |
Apply forcing, check break, push trace sample |
RegisterBroker(…) |
Cache broker metadata for active/break index updates |
IsPaused() / SetPaused(bool) |
Query / set global pause state |
ConsumeStepIfNeeded(gamName, thread) |
Decrement step counter on output-broker Execute |
3.2 Control-path API
Called from server/command-handler threads; latency acceptable.
| Method | Purpose |
|---|---|
ForceSignal / UnforceSignal |
Persistent value injection |
TraceSignal(name, enable, decimation) |
Enable/disable ring-buffer tracing |
SetBreak / ClearBreak |
Conditional breakpoints |
IsInstrumented(path, …) |
Query whether a signal is reachable |
RegisterMonitorSignal / UnmonitorSignal |
Slow-rate signal polling |
3.3 Static singleton pattern
// Concrete implementation registers itself:
DebugServiceI::SetInstance(this); // in Initialise()
DebugServiceI::SetInstance(NULL_PTR(…)); // in destructor
// Broker wrappers retrieve it (no ORD search on the hot path):
DebugServiceI *svc = DebugServiceI::GetInstance();
if (svc != NULL_PTR(DebugServiceI*)) { ... }
Shared data structures (SignalAlias, BrokerInfo) are also defined in DebugServiceI.h
so both transport implementations can use them without a circular dependency.
4. Transport Implementations
4.1 DebugService (TCP/UDP)
Source/Components/Interfaces/DebugService/DebugService.h/.cpp
Runs two SingleThreadService threads:
| Thread | Role |
|---|---|
Server() |
Accepts one TCP client at a time; reads text commands; writes JSON/text responses |
Streamer() |
Drains TraceRingBuffer; assembles and sends UDP datagrams; polls monitored signals |
Communication ports (all configurable):
| Port | Protocol | Purpose |
|---|---|---|
ControlPort (default 8080) |
TCP text | Commands and responses |
UdpPort (default 8081) |
UDP binary | High-speed telemetry (TraceHeader + samples) |
LogPort (default 8082) |
TCP stream | Log forwarding via TcpLogger sidecar |
4.2 WebDebugService (HTTP/SSE)
Source/Components/Interfaces/WebDebugService/WebDebugService.h/.cpp
Serves a single HTTP port (default 8090) with an embedded browser UI. Runs two
SingleThreadService threads:
| Thread | Role |
|---|---|
HttpServer() |
Accepts TCP connections; routes GET/POST/OPTIONS; upgrades SSE clients |
Streamer() |
Drains TraceRingBuffer → SSE events; polls monitored signals; heartbeat |
HTTP endpoints:
| Endpoint | Method | Purpose |
|---|---|---|
/ |
GET | Serves embedded single-page application (HTML/CSS/JS) |
/api/command |
POST | Execute a debug command; returns text response |
/api/events |
GET | Server-Sent Events stream for real-time telemetry (up to 8 clients) |
The embedded web UI (Source/Components/Interfaces/WebDebugService/WebUI.h) is a
self-contained single-page application using Chart.js for real-time signal plots and a
Catppuccin Mocha dark theme. No install or server-side tooling required.
5. Signal Flow
5.1 Registration (startup, single-threaded)
ConfigureApplication()
└─► DebugBrokerWrapper::Init()
└─► DebugBrokerHelper::InitSignals()
├─► DebugServiceI::RegisterSignal() (canonical name + GAM alias)
└─► DebugServiceI::RegisterBroker()
5.2 RT loop (hot path, multi-threaded)
RealTimeThread::Execute()
└─► DebugBrokerWrapper::Execute()
├─► Base::Execute() (data moved in/out of GAM memory)
└─► DebugBrokerHelper::Process()
├─► For each active signal:
│ └─► DebugServiceI::ProcessSignal()
│ ├─► if isForcing: memcpy forcedValue → signal memory
│ ├─► if isTracing & decimation fires:
│ │ └─► tracePushMutex → TraceRingBuffer::Push()
│ └─► if breakOp set: evaluate condition → SetPaused(true)
└─► (output brokers only) DebugServiceI::ConsumeStepIfNeeded()
5.3 Telemetry delivery
DebugService (UDP)
Streamer thread
└─► TraceRingBuffer::Pop()
└─► assemble into streamerPacketBuffer (MTU=1400)
└─► BasicUDPSocket::Write() → client
WebDebugService (SSE)
Streamer thread
└─► TraceRingBuffer::Pop()
└─► BroadcastSse("message", {"type":"trace","name":…,"value":…})
└─► write "event: message\ndata: {…}\n\n" to each SSE socket
6. TraceRingBuffer
Single-producer / single-consumer circular byte buffer defined in DebugCore.h.
Buffer layout per entry: [ID:4][Timestamp:8][Size:4][Data:N]
Push()is serialised bytracePushMutex(multiple RT threads write; one Streamer reads).Pop()is called exclusively by the Streamer thread.- Ring is initialised to 4 MB in both transport implementations.
- On corrupt
sizefield (size >= bufferSize): discards all entries to prevent cascading bad reads (Fix #5).
7. Performance Hardening
The following fixes were applied after initial implementation to improve correctness and real-time safety.
| Fix | Area | Change |
|---|---|---|
| #1 | UDP streamer | Named constants STREAMER_MTU = 1400 and STREAMER_BUFFER_SIZE = 4096; removed magic numbers |
| #2 | Multi-producer ring | tracePushMutex serialises TraceRingBuffer::Push() from concurrent RT threads |
| #3 | Break evaluation | Break indices copied to stack-local array while holding activeMutex; condition evaluated outside the lock |
| #4 | Decimation counter | Atomic::Add used for lock-free per-signal decimation in multi-producer ProcessSignal |
| #5 | Ring corruption | Pop() skips only the current corrupt entry; falls back to full discard only when size >= bufferSize |
| #6 | TCP rate limit | Server disconnects clients sending > 100 commands/second (CMD_RATE_LIMIT = 100) |
| #7 | Broker index swap | Vec::Swap inside spinlock followed by free outside; avoids holding the lock during deallocation |
| #8 | Idle timeout | Active TCP client disconnected after 30 s of silence (CLIENT_IDLE_TIMEOUT_MS = 30000) |
| #9 | VALUE output size | GetSignalValue caps output at 256 elements (GET_VALUE_MAX_ELEMENTS); includes "Truncated" flag |
| #10 | TCP framing | inputBuffer accumulates partial commands across Read() calls; bounded at 8 KiB (INPUT_BUFFER_MAX) |
| #11 | CDB cursor | Initialise() never calls MoveToRoot() on the CDB passed by the framework; avoids corrupting ReferenceContainer traversal |
8. Repository Layout
Source/
Core/Types/
Vec/ — RT-safe dynamic array (no STL)
Result/ — Error-carrying return type
Components/Interfaces/
DebugService/
DebugCore.h — DebugSignalInfo, TraceRingBuffer, BreakOp
DebugServiceI.h — Abstract interface + SignalAlias, BrokerInfo
DebugBrokerWrapper.h — DebugBrokerHelper, wrapper templates, registry builder
DebugService.h/.cpp — TCP/UDP transport implementation
WebDebugService/
WebUI.h — Embedded single-page HTML/CSS/JS application
WebDebugService.h/.cpp — HTTP/SSE transport implementation
TCPLogger/
TcpLogger.h/.cpp — LoggerConsumerI forwarding logs to TCP clients
Test/
Integration/ — Integration test suite (180 s SIGALRM guard)
UnitTests/ — Unit test suite
Tools/
gui_client/ — Rust/egui native GUI (DebugService TCP/UDP transport)