Martino FerrariandClaude Opus 4.6 fbae7d712c fix(udps): stop packets being dated from an earlier time base
Reported as samples sporadically carrying a previous packet's timestamp:
holes on one side of the stream and collisions on the other, in both the
Go and the MARTe2 receiver. That it appeared in both is what located it
-- the shared cause is upstream of either client. Four independent
defects, all of which end in a packet's values being placed at a time
that is not theirs.

Reassembly slot exhaustion (the "Reassembly slots full; evicting oldest"
flood). Chunk size was learnt only from fragment 0, so an out-of-order
burst destroyed a packet whose bytes had all arrived and left the slot
occupied until the 2 s GC. Slots were keyed on the counter alone, but
DATA and CONFIG number independently, so equal counters merged the two
streams. The 32-byte received-mask covered 256 of the 512 fragments the
client accepts, so a duplicate above 255 was counted as new and the
packet was delivered with a hole of stale bytes in it. And one datagram
was read per Execute(), which cannot drain a fast producer. Fixed with a
pendingTail deferral, (counter, type) keying, a 64-byte mask, a
256-datagram drain, counter-age slot reclamation, and a 1 Hz aggregated
warning in place of the per-eviction flood.

UDPStreamer dropping whole Accumulate batches. EventSem::ResetWait is
Reset-then-Wait, so a Post() landing while the sender thread was inside
ServiceClients()/SendData() was destroyed by the next Reset. The batch
was then skipped with dataReady false, readyFill was never cleared, and
the following flush overwrote it: an entire run of RT cycles never
reached the wire. The record of pending work now lives in the buffers
rather than in the semaphore edge, which also removes up to
UDPS_DATA_WAIT_MS of latency; genuine backpressure overwrites are
counted and reported. Against the unfixed code the new test sees
2999/3000 batches never consumed.

Period inflation after loss. Accumulated scalars carry no SamplingRate,
so the receiver derives dt from the sender-clock gap -- but dividing it
by the previous packet's sample count is only right while nothing is
lost. One loss doubles the reported period, which spreads a batch a full
batch past its own end and into the range the next packet claims. That
is the hole and the collision, exactly. Inferring the cycle count from
the estimate's own period is not a way out: it has a stable fixed point
wherever gap/dt is an integer, so a real rate change locks it at the old
one for good (AccumDtGTest.FollowsSustainedRateChange).

The packet counter removes the ambiguity, so all three receivers now
order on it: a DATA packet that does not advance the counter is dropped
rather than delivered, because its values are older than data already
handed over. Ordering is on the signed difference so it survives the
uint32 wrap, and the sequence resets on reconnect, where the producer's
counter restarts independently of ours. The loss count that falls out of
the same delta feeds the period estimate as cycles = prevN * (1 + lost),
which reduces exactly to gap/prevN when nothing is lost and therefore
still tracks a genuine rate change. UDPSClient::AcceptDataCounter (C++),
udpsprotocol.SequenceGate (Go), decode_data (C).

The C client's existing gap counter was wrap-unsafe and let a stale
packet rewind last_counter, which made every subsequent gap wrong; it
uses the same code now. Docs/Protocol.md gains an Ordering DATA section
stating the requirement for any receiver, including ones outside this
repository.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-09-02 01:18:49 +02:00
2026-05-29 13:29:59 +02:00
2026-08-21 23:24:48 +02:00
2026-05-29 13:29:59 +02:00
2026-06-12 15:25:13 +02:00
2026-05-29 13:29:59 +02:00

MARTe2 Integrated Components

A unified MARTe2 library providing real-time signal streaming and on-the-fly debugging for control applications built with MARTe2.


Overview

This repository integrates two complementary capabilities:

Capability Component Purpose
Signal streaming UDPStreamer DataSource Continuously stream selected signals to a browser-based oscilloscope over UDP
Signal debugging DebugService Interface On-demand signal tracing, value forcing, and conditional breakpoints — zero application code changes required
Sine generation SineArrayGAM Generate continuous sine-wave arrays for testing and simulation
Time stamping TimeArrayGAM Provide time-reference arrays aligned to an RT cycle
Log forwarding TCPLogger Interface Forward REPORT_ERROR log events to TCP clients in real time
Integrated client Common/Client/go Go packages for UDPS protocol and WebSocket hub
Debug web client Client/debugger Browser-based debug UI communicating with DebugService

Repository Structure

MARTe_Integrated_components/
├── Common/
│   ├── UDP/                        UDPS binary protocol header (shared by all components)
│   └── Client/go/                  Go packages: udpsprotocol, wshub
├── Source/Components/
│   ├── DataSources/UDPStreamer/     Real-time UDP signal streaming DataSource
│   ├── GAMs/SineArrayGAM/          Sine-wave array generator GAM
│   ├── GAMs/TimeArrayGAM/          Time-reference array GAM
│   ├── Interfaces/DebugService/    Signal tracing/forcing/breakpoint Interface
│   └── Interfaces/TCPLogger/       TCP log-forwarding LoggerConsumerI
├── Test/
│   ├── GTest/                      GTest harness for UDPStreamer unit tests
│   ├── Integration/                Integration tests for DebugService
│   ├── Components/DataSources/UDPStreamer/  UDPStreamer unit tests
│   └── Configurations/             MARTe2 config files for tests and demos
├── Client/debugger/                Go web client for DebugService
└── Docs/                           Documentation

Components

UDPStreamer DataSource

Streams MARTe2 signals over UDP using the UDPS binary protocol. Clients register by sending a CONNECT packet; the server then sends CONFIG (signal metadata) and continuous DATA packets. Features:

  • Optional 16-bit quantization (configurable per signal: QuantizedType)
  • Packed high-frequency bursts (NumberOfElements > 1 with SamplingRate)
  • Automatic packet fragmentation for payloads exceeding MaxPayloadSize

See Docs/UDPStreamer.md and Docs/Protocol.md.

SineArrayGAM

Generates a continuous float32 sine-wave array every RT cycle. Used as a signal source for testing and demo applications. Configurable: Frequency, Amplitude, Phase, SamplingRate, NumberOfElements.

See Docs/SineArrayGAM.md.

TimeArrayGAM

Generates a time-reference uint64 array. Each element holds the timestamp of the corresponding sample in a packed burst, computed from the RT cycle timestamp and the configured SamplingRate.

Anchor selects how the burst is placed in time:

Anchor out[k]
FirstSample input + k · period
LastSample input (N1k) · period
Continuous input(first cycle) + (n + k) · period

FirstSample/LastSample re-read the timer each cycle, so a lost RT cycle (LinuxTimer re-phases with counter += nCycles) punches a whole-period hole into the time base even though only one array of samples was produced. Use Continuous when the data signal is itself contiguous (SineArrayGAM never skips phase): it latches the timer once and then advances an internal sample counter by N per cycle, like an acquisition card running off its own clock.

DebugService Interface

Instruments a running MARTe2 application without modifying its source code. On Initialise() it patches the ClassRegistryDatabase to wrap all standard MemoryMap*Broker types. When RealTimeApplication::ConfigureApplication() runs afterward the application transparently uses the wrapped brokers.

Capabilities accessible over TCP (port 8080 by default):

  • DISCOVER — enumerate all signals with type and alias metadata
  • TRACE — enable/disable high-speed UDP telemetry per signal (with decimation)
  • FORCE / UNFORCE — inject persistent values into signals on the RT path
  • BREAK — set conditional breakpoints (>, <, ==, etc.)
  • PAUSE / RESUME / STEP — execution stepping
  • TREE / INFO / LS — live ORD navigation
  • VALUE — read current signal value on demand
  • MSG — send MARTe2 Message to any ORD object

High-speed signal telemetry is streamed as UDP binary datagrams (port 8081). Logs are forwarded via TcpLogger (port 8082).

See Docs/DebugService.md.

TCPLogger Interface

A LoggerConsumerI that forwards every MARTe2 REPORT_ERROR call to up to 8 TCP clients on a configurable port. Works as a sidecar to DebugService.

StreamHub Application

Headless C++ hub (Source/Applications/StreamHub/) that aggregates multiple UDPStreamer sources and serves them to oscilloscope clients over WebSocket (port 8090): ring buffers with wall-clock time calibration, LTTB decimation, hub-side trigger engine, per-window zoom. Clients: browser SPA (Client/webui + Client/udpstreamer/static) and native ImGui desktop client (Client/streamhub). Demo: ./run_streamhub.sh -w -g; E2E test: Test/E2E/suite/run_e2e.sh.

See Docs/StreamHub-UserGuide.md, Docs/StreamHub-API.md and Docs/StreamHub-Developer.md.

UDPS Protocol

The Common/UDP/UDPSProtocol.h header defines the shared binary wire format used by both UDPStreamer and DebugService. It is intentionally free of MARTe2-specific dependencies so it can also be used by Go clients (via Common/Client/go/udpsprotocol).

See Docs/Protocol.md.


Build

Prerequisites

  • MARTe2 built and installed (set MARTe2_DIR)
  • MARTe2-components built and installed (set MARTe2_Components_DIR)
  • GCC toolchain, make
  • Go 1.21+ (for clients)

Setup

# Edit env.sh to set MARTe2_DIR and MARTe2_Components_DIR, then:
source env.sh

Build all C++ components

make -f Makefile.gcc core

Build and run tests

make -f Makefile.gcc test
source env.sh
./Build/x86-linux/Test/Integration/IntegrationTests

Build Go clients

# Integrated UDPS web client
cd Common/Client/go && go build ./...

# Debug web client
cd Client/debugger && go build ./...

Clean

make -f Makefile.gcc clean

Quick Start

1. Stream signals with UDPStreamer

Add to your MARTe2 config:

+Streamer = {
    Class          = UDPStreamer
    Port           = 44500
    MaxPayloadSize = 1400
    Signals = {
        Voltage = { Type = float32; Unit = "V" }
    }
}

Launch the web client:

cd Common/Client/go
./udpstreamer-webui --streamer 127.0.0.1:44500 --listen :8080

Open http://localhost:8080 and drag signals onto plots.

2. Debug a running application with DebugService

Add to your MARTe2 config (as a sibling of +App):

+DebugService = {
    Class       = DebugService
    ControlPort = 8080
    UdpPort     = 8081
    LogPort     = 8082
}
+Logger = { Class = TcpLogger; Port = 8082 }

Launch the debug client:

cd Client/debugger
./debugger --listen :9090

Open http://localhost:9090, explore the object tree, trace signals, force values.


Documentation

Document Contents
Docs/Protocol.md UDPS binary wire protocol specification
Docs/UDPStreamer.md UDPStreamer DataSource configuration reference
Docs/UDPS-C-Client.md Standalone C/C++ UDPS receiver library (Common/Client/c)
Docs/SineArrayGAM.md SineArrayGAM configuration reference
Docs/DebugService.md DebugService TCP API and architecture
Docs/Tutorial.md Step-by-step tutorial covering both components
Docs/WebUI.md Web client user guide
Docs/StreamHub-UserGuide.md StreamHub oscilloscope user guide (web + ImGui clients)
Docs/StreamHub-API.md StreamHub WebSocket protocol (commands, events, binary frames)
Docs/StreamHub-Developer.md StreamHub internals, threading, time base, build & E2E tests
ARCHITECTURE.md System architecture overview

License

Licensed under the EUPL v1.1 (see individual source files for copyright notices).

S
Description
No description provided
Readme
43 MiB
Languages
C++ 35.5%
Makefile 23.3%
JavaScript 11.3%
Go 10.2%
C 7.6%
Other 12.1%