martino/MARTe-Integrated-Components
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: martino/MARTe-Integrated-Components:feature/stress-report-integration
Branches Tags
martino/MARTe-Integrated-Components:main martino/MARTe-Integrated-Components:feature/udpscope martino/MARTe-Integrated-Components:feature/stress-report-integration
...
compare: martino/MARTe-Integrated-Components:7c5eb31a52ac4b34c7edce54067f109d97201fb4
Branches Tags
martino/MARTe-Integrated-Components:main martino/MARTe-Integrated-Components:feature/udpscope martino/MARTe-Integrated-Components:feature/stress-report-integration
62 Commits
feature/stress-report-integration ... 7c5eb31a52
Author SHA1 Message Date
Martino FerrariandClaude Opus 4.6 7c5eb31a52 feat: standalone C/C++ UDPS client library 
Consuming a UDPStreamer feed so far meant either linking MARTe2 (UDPSClient)
or writing Go (Common/Client/go/udpsprotocol). Common/Client/c fills the gap
for plain C/C++ integrators: two files depending on nothing but libc and BSD
sockets, covering the whole receive path — CONNECT, fragment reassembly,
CONFIG/DATA decoding with dequantisation, keepalives and silence-triggered
reconnect. No threads are spawned; udps_client_poll() does all the work and
runs every callback, so it drops into an existing event loop unsynchronised.

Verified against run_udp_producer.sh at 1 Msps: unicast (120 MiB, no loss) and
multicast with 12-fragment cycles (116k datagrams, no loss).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-22 16:51:40 +02:00
Martino Ferrari e03c60db25 Implemented and fixed many issues 2026-08-21 23:24:48 +02:00
Martino FerrariandClaude Sonnet 4.6 14d5351a81 fix: UDPSClient uses two-arg Join so multicast receiver lands on the right interface 
UDPSClient::ConnectMulticast was calling the single-arg BasicUDPSocket::Join,
which forwards NULL as the local interface and lets the kernel bind to
INADDR_ANY.  On a multi-homed host (or when the server sends on loopback via
Interface = "127.0.0.1") the client joins the wrong interface and silently
receives nothing.

Fix: read the optional Interface key inside the useMulticast block in
UDPSClient::Initialise; in ConnectMulticast call the two-arg
Join(group, interface) when Interface is set, and fall back to the one-arg
call otherwise to preserve the existing INADDR_ANY behaviour for configs that
omit it.

Forward the new optional Interface key through UDPStreamerClient (read from
DataSource config, written into the UDPSClient ConfigurationDatabase only
when non-empty).  Extend the "Joined multicast group" log to report the
interface name or "default".

Regression test TestExecute_MulticastReceivesDataOnInterface: mock TCP
control listener + multicast UDP DATA socket with IP_MULTICAST_IF set to
127.0.0.1, verifying a uint32 value of 424242 reaches DataSource signal
memory.  Confirmed FAILED without the Join fix and PASSED with it.

Suite: 133/133 (was 132/132 before this commit).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 08:23:41 +02:00
Martino FerrariandClaude Opus 4.6 61a2aa3988 docs: Interface takes an interface IP, not an interface name 
UDPSServer parses Interface with inet_addr() to set IP_MULTICAST_IF, so
a name like "eth0" yields INADDR_NONE and Initialise fails. Every
example in UDPStreamer.md used "eth0", so anyone following the docs hit
that failure; the .cfg files in Test/ already used a dotted-quad.

Replace the examples with 192.168.1.10, state the dotted-quad
requirement in the parameter table and the Multicast section, and note
that receivers must join on the matching interface or they silently
receive nothing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 08:09:21 +02:00
Martino FerrariandClaude Opus 4.6 c827ecefff test: fix UDPStreamer multicast tests broken by mandatory Interface 
Commit 3e0a481 made Interface mandatory for multicast in both
UDPStreamer::Initialise and UDPSServer::Initialise and updated
Docs/UDPStreamer.md, but left the GTest configs untouched. All four
multicast tests have failed since.

Add Interface to the five multicast configs. The field is parsed with
inet_addr(), so it takes a dotted-quad, not an interface name; 127.0.0.1
keeps the tests self-contained and off the LAN.

TestInitialise_MulticastMode_InvalidDataPort was passing for the wrong
reason: UDPStreamer.cpp rejected it on the missing Interface before the
DataPort == Port check could run. It now proves what its name claims.

TestExecute_MulticastConnectDataDisconnect additionally needed the
two-argument Join(): the single-argument form passes INADDR_ANY, so the
reader joined the default-route interface while the server sent on
loopback via IP_MULTICAST_IF, and the DATA datagram never arrived.

GTest: 132/132 (was 128/132), multicast subset stable over 3 runs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 08:06:35 +02:00
Martino FerrariandClaude Opus 4.6 72c286db33 chore: untrack SDD scratch reports 
These subagent handoff artifacts were committed before
.superpowers/sdd/.gitignore took effect. They are ephemeral
scratch, not project content.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 08:01:09 +02:00
Martino Ferrari 7aba1260be Merge branch 'feature/signal-calibration-config' 
Per-signal data calibration (value = raw * scale + offset) with an optional
unit override, plus sources+calibration persistence in a single config file,
implemented identically in the Go hub and the C++ StreamHub.
 2026-08-17 08:00:09 +02:00
Martino FerrariandClaude Sonnet 4.6 1f8592f854 fix: address all 9 findings from final calibration code review 
- calibration.js: fix baseSignalName('[0]') parity with Go/C++ (>= 0 not > 0)
- calibration.test.js: add assertions for '[0]' edge case in two existing tests
- app.js: remove stale typeof guard around refreshVScaleMenu (always defined)
- app.js: call refreshTrigThresholdField on trig-signal change (both assignment sites)
- index.html: drop maxlength='16' on unit input; normaliseCal is the sole enforcer
- configcheck/main.go: delete dead nextOneOf function (no callers)
- hub_calibration_test.go: delete orphaned waitBroadcast comment (function never existed)
- calibration.go: correct arrayIndexSuffix comment to document known Go/C++ difference
- Docs/StreamHub-API.md: add calibration entry count and unit byte limits to §5 table
- spec: fix configReloaded missing path field, '16 chars'→'16 UTF-8 bytes', StreamString→char[], chain scenario→configcheck program, four→five new frames

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:57:28 +02:00
Martino FerrariandClaude Sonnet 4.6 42f5a726af fix: three StreamHub C++ defects from final code review 
Finding 1 (HandleReloadConfig data loss): move ClearCalibration inside
LoadSourcesFile so the table is wiped only after a successful fread.
Adds a clearCalibration bool parameter (default false); the reload path
passes true, the startup path passes false.

Finding 2 (JSON injection via unit/source/signal): add JsonEscape()
static helper (escapes \", \\, \n \r \t, and \u00XX for other control
chars). Applied at all three emission sites: BroadcastCalibration,
HandleSaveSources, and BroadcastSources (label). Teach JsonGetString to
unescape the same set on read, so values round-trip correctly.

Finding 3 (%.17g verbosity): add ShortFloat() static helper that tries
%.15g then %.16g then %.17g, stopping at the first precision whose
strtod() output compares equal to the original. Applied at both float
emission sites. 0.1 now prints as "0.1", not "0.10000000000000001".

Minor: fix two inaccurate comments in StreamHub.h — the CalibrationEntry
rationale (not a 133 MB / address-limit issue; the real reason is no
per-entry heap churn, STL-free, trivially copyable) and "chars" to "bytes"
for the unit cap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:50:01 +02:00
Martino FerrariandClaude Sonnet 4.6 686fc2ce7d docs: fix three review findings in calibration documentation 
- Finding 1 (Critical): replace "16 chars" with "16 UTF-8 bytes" in the
  setCalibration unit field description (StreamHub-API.md) and the Cal·Unit
  toolbar row (WebUI.md); note that multi-byte characters consume more than
  one byte and that truncation never splits a character.

- Finding 2 (Important): correct the claim that a sources broadcast after
  reloadConfig is conditional on new sources being added — that is true only
  of the Go hub. The C++ hub calls BroadcastSources() unconditionally on
  success. Both the reloadConfig command description and the configReloaded
  event description in StreamHub-API.md are updated; the Reload bullet in
  WebUI.md is updated with a brief note. Clients must tolerate an unsolicited
  sources frame after any reload.

- Finding 3 (Minor): the configSaved failure example used "no SourcesFile
  configured", which matches neither hub. Corrected to the C++ form
  "no sources file configured" and added a note that the exact error text
  is not part of the protocol contract (Go uses "no sources-file configured").

Source evidence: calibration.go (maxUnitLen, len(), rune-repair loop),
StreamHub.cpp (kMaxUnitLen, byte strncpy, HandleReloadConfig unconditional
BroadcastSources, HandleSaveSources error string).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:34:48 +02:00
Martino FerrariandClaude Sonnet 4.6 d26b78b7f6 docs: document per-signal calibration and config save/reload 
Update Docs/StreamHub-API.md (new setCalibration/reloadConfig commands,
calibration/configSaved/configReloaded events, §4 config file format),
ARCHITECTURE.md §6 (updated command/event tables and Config File Format
subsection), Docs/WebUI.md (Cal row in V-Scale Toolbar, Sources & Config
sidebar section). Also corrects the spec's Validation sentence to match
the shipped cal-invalid border behaviour instead of silent field revert.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:27:25 +02:00
Martino FerrariandClaude Sonnet 4.6 e37eec0276 webui: add the Sources & Config sidebar section 
Renames the "Add Source" section to "Sources & Config" and replaces the
fire-and-forget "Save list" button with Save and Reload, plus a one-line
status area that renders the hub's configSaved/configReloaded ack.

onConfigAck replaces the no-op stub Task 7 left behind. It branches on the
frame type because configSaved carries a path and configReloaded does not,
and surfaces the hub's error text on failure rather than failing silently.
The status is kept outside the DOM because buildSidebar() recreates this
section on every sources broadcast.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:23:08 +02:00
Martino FerrariandClaude Sonnet 4.6 5917ab1bf2 Restore calibration parity: revert unit-stripping from normaliseCal, fix CSV quoting at export 
Finding 1: revert the comma/quote strip added in 3cb998c from
normaliseCal() in calibration.js. The strip broke byte-identical parity
with Go CalConfig.Normalise and C++ StreamHub::SetCalibrationEntry, both
of which only trim whitespace and cap at 16 UTF-8 bytes. Delete the
companion test that asserted the now-removed behaviour (suite returns to
18 tests).

Finding 2: fix the actual CSV-safety problem at the point of use in
exportAllCSV() in app.js. Header cells (time column and signal columns)
are now RFC 4180-quoted: wrapped in double quotes with any embedded
double quote doubled. This safely handles units or signal names that
contain commas or quotes without touching normaliseCal.

Finding 3: update two stale comments in app.js that called the trigger
threshold or rawFromNorm result 'raw' — Task 9 moved trig.threshold into
calibrated units throughout, so the comments now say 'calibrated'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 01:15:13 +02:00
Martino FerrariandClaude Sonnet 4.6 3cb998c5da webui: calibrate CSV export, trigger threshold and unit display 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:55:07 +02:00
Martino FerrariandClaude Sonnet 4.6 4908c039a5 fix(webui): keep cal-invalid styling on focused field during hub broadcast 
refreshVScaleMenu() was unconditionally removing the cal-invalid class
from all three calibration inputs, which silently cleared the red-border
error indicator on a field the user was editing whenever a hub
calibration broadcast arrived. Apply the same focused-element guard
already used for .value writes so a rejected value's error styling
persists until the user corrects it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:52:04 +02:00
Martino FerrariandClaude Sonnet 4.6 4e3a90b2c6 webui: add calibration editor to the V-Scale toolbar 
Adds the Cal row (Scale / Offset / Unit / Reset) to #vscale-menu, its
CSS, and the refreshVScaleMenu() / commitCal() logic that wires it to
calTable and the hub via setCalibrationWS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:48:39 +02:00
Martino Ferrari 0e9ec61226 webui: apply per-signal calibration to the whole display path 2026-08-17 00:43:18 +02:00
Martino FerrariandClaude Opus 4.6 7312dd0ca0 docs(calibration.js): explain why one UTF-8 repair pass suffices in JS 
TextEncoder always emits well-formed UTF-8, so truncation can strand at
most a lead byte plus three continuations — one repair pass covers it.
The C++ hub needs a loop because its input is raw bytes off the wire.
Also drops a dead variable from the byte-boundary test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 00:41:00 +02:00
Martino FerrariandClaude Opus 4.6 48d62c1f80 fix(calibration.js): cap unit at 16 UTF-8 bytes, matching both hubs 
normaliseCal was using String.length/.slice() (UTF-16 code units), so
multi-byte characters like °, Ω, µ could slip through oversized. Now uses
TextEncoder to slice at 16 bytes, then repairs any incomplete trailing
UTF-8 sequence by walking back over continuation bytes to find the lead
byte and dropping the incomplete rune — exactly mirroring Go's
utf8.DecodeLastRuneInString loop and the C++ walk-back in
StreamHub::SetCalibrationEntry. Adds 4 new test cases covering the
non-ASCII/boundary scenarios, and corrects the canonical test command in
the task-6 report to 'cd Client/udpstreamer && node --test'.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 00:36:01 +02:00
Martino FerrariandClaude Sonnet 4.6 21d084d2ea webui: add pure calibration module with unit tests 
Implements Calib JS module (calibration.js) with affine transform primitives,
CalTable, and normaliseCal matching Go CalConfig.Normalise semantics; 14 node
--test cases all pass. Loads before app.js in index.html.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:29:53 +02:00
Martino FerrariandClaude Sonnet 4.6 b1c2a34eea test(configcheck): harden frame-strictness, sort-order, and document BroadcastSources difference 
Fix round 1 review findings:

1. Frame-matching strategy: add protocolFrameTypes set; next() now fails immediately
   on an out-of-order protocol frame instead of silently discarding it. Ambient
   frames (data, stats, triggerState, monotonicState) are logged and skipped.
   nextSkipping() accepts explicitly-listed protocol frames that legitimately differ
   in order across hubs (connect-time "sources" before "calibration" in C++).

2. BroadcastSources documented exception: the extra "sources" frame C++ emits after
   reload is accepted and logged as [KNOWN DIFFERENCE] with full rationale; the
   report recommendation to remove it has been retracted (it is required for correct
   client-side source-list updates after reload-with-new-sources).

3. Sort-order constraint exercised: three calibration entries submitted in
   reverse-sort order (src2/Beta, src1/Zeta, src1/Alpha); each broadcast and the
   saved config file are asserted to be sorted by source then signal.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:26:12 +02:00
Martino FerrariandClaude Sonnet 4.6 73ba725d8f test: add cross-hub calibration and config-persistence parity checker 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:14:15 +02:00
Martino FerrariandClaude Sonnet 4.6 bdc74f5fd2 StreamHub: loop UTF-8 tail repair to match Go CalConfig.Normalise() 
The single-pass repair left invalid bytes when the candidate lead byte
had class 0 (illegal 0xF8-0xFF bytes, or a bare continuation byte
reached after the 3-byte backward-scan cap). Convert to a loop with a
`cut` flag mirroring Go's loop: each iteration either makes no cut
(exits) or strictly reduces ulen by >= 1 byte (terminates in <= 16
iterations). Also treat expected==0 as a cut target, matching Go's
behaviour of stripping any byte that decodes as an invalid one-byte
sequence.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:06:45 +02:00
Martino FerrariandClaude Sonnet 4.6 93e00d0c21 fix(StreamHub): correct UTF-8 tail repair to run only after truncation 
The previous walk-back in SetCalibrationEntry was unconditional, corrupting
short valid units ending in multi-byte characters (e.g. Omega, mu, degree).
Also failed to drop an orphaned lead byte left after stripping continuation
bytes. Restructured to use a 256-byte staging buffer so truncation can be
detected, then repair runs only in the truncation branch. Algorithm now
matches Go CalConfig.Normalise() exactly: scan back over continuation bytes
(up to 3), find the lead byte, derive expected sequence length, cut if
incomplete. Covers all cases: orphaned continuation, orphaned lead, cut on
lead byte.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:02:33 +02:00
Martino FerrariandClaude Sonnet 4.6 2f9b135c62 task-4-report: append Fix round 1 section 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 23:55:49 +02:00
Martino FerrariandClaude Sonnet 4.6 957be793ae StreamHub: fix calibration trim, UTF-8 unit truncation, and sort order 
Finding 1: Add TrimInPlace helper; apply trim→strip-[i]→empty-check order
in SetCalibrationEntry (matching Go Normalise()), so whitespace-padded
source/signal from WS clients normalise identically to config-file loads.

Finding 2: Trim unit before truncating to kMaxUnitLen, then walk back
continuation bytes (0x80-0xBF) to avoid leaving a partial UTF-8 rune,
matching Go's utf8.DecodeLastRuneInString loop.

Finding 3: BroadcastCalibration and HandleSaveSources now emit entries
sorted by source then signal (insertion sort over an index array, no STL),
producing byte-identical calibration frames and config files to the Go hub.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 23:55:46 +02:00
Martino FerrariandClaude Sonnet 4.6 cdafb877a3 StreamHub: per-signal calibration, config reload, whitespace-tolerant JSON 
- Add CalibrationEntry (heap-allocated array[256], char[] fields to stay within
  the 133 MB struct's canonical address limit) plus calibrationMutex_ and
  numCalibration_.
- Implement SetCalibrationEntry/ClearCalibration, BroadcastCalibration,
  BroadcastConfigAck, HandleSetCalibration, HandleReloadConfig.
- Wire setCalibration and reloadConfig into OnWSCommand dispatch.
- Broadcast calibration to each newly connected client after triggerState.
- Fix JSON round-trip bug: replace JsonGetString/JsonGetBool helpers with a
  shared whitespace-tolerant JsonFindValue (tolerates "key" : "value" as
  written by HandleSaveSources); add JsonIsFinite (no <cmath>).
- Extend LoadSourcesFile(bool skipActive) to also parse calibration blocks;
  SourceIsActive checks live sessions before starting a duplicate.
- Extend HandleSaveSources to persist calibration blocks; emit configSaved ack.
- Reload semantics: calibration replaced wholesale, sources added only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 19:56:44 +02:00
Martino FerrariandClaude Sonnet 4.6 ffe7cb1cc5 wshub: add setCalibration/reloadConfig frames and config acks 
Wire five new WebSocket frames into hub.go: setCalibration (client→hub),
calibration (hub→client broadcast), reloadConfig (client→hub), configSaved and
configReloaded (hub→client acks with ok/path/error). Extend hubCmd with cal
field, add buildCalibrationMsg and buildConfigAckMsg builders, send calibration
on client connect, and run Save/Reload on separate goroutines to avoid blocking
the Run() select loop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 19:36:09 +02:00
Martino FerrariandClaude Sonnet 4.6 dfd257cfd9 wshub: persist calibration alongside sources; add config reload 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 19:27:23 +02:00
Martino FerrariandClaude Sonnet 4.6 66efd74dd5 wshub: fix CalConfig.Normalise parity gaps vs C++ twin and JS client 
Finding 1: strip trailing [digits] array-element suffix from Signal so one
calibration entry covers an entire array signal, matching the C++ strchr
truncation and the JS equivalent.

Finding 2: after the 16-byte Unit truncation, drop any trailing partial UTF-8
rune so json.Marshal never emits replacement characters; keeps byte limit in
sync with C++ strncpy(u, unit, kMaxUnitLen).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 19:21:41 +02:00
Martino Ferrari 47f1567a26 wshub: add per-signal calibration store and flat config-file codec 2026-08-16 19:16:22 +02:00
Martino Ferrari 6d26e8191c docs: implementation plan for per-signal calibration and persistent hub config 2026-08-16 19:14:34 +02:00
Martino FerrariandClaude Opus 4.6 5ab1721df5 docs: spec per-signal calibration and persistent hub config 
Design for a per-signal affine calibration (scale/offset/unit) applied
client-side and stored server-side alongside the source list, so signals in
raw units can be read in engineering units and the setup survives a reload.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-16 18:17:07 +02:00
Martino Ferrari 2370848994 added trigger 2026-08-13 10:28:56 +02:00
Martino Ferrari ff5ad22447 included jitter correction on client 2026-08-13 10:28:43 +02:00
Martino Ferrari a49ab5ba25 Added silence timeout as floating point 2026-08-10 17:18:50 +02:00
Martino Ferrari 1ddb4fe356 Implemented hearthbit client side + tests 2026-08-09 18:51:51 +02:00
Martino Ferrari 915a192b16 fixed multicast updated tests 2026-07-25 16:46:25 +02:00
Martino Ferrari 3e0a481c13 added interface and added join to multicast 2026-07-25 12:26:51 +02:00
Martino Ferrari 2d5ca20ae4 minor changes and addeed debug tests 2026-07-02 16:27:40 +02:00
Martino Ferrari f2042d624b Implemented full e2e testing 2026-07-02 10:10:57 +02:00
Martino FerrariandClaude Opus 4.6 f8c79131c9 docs: refresh run_e2e.sh flag list in AGENTS.md 
Task 7 added --skip-coverage/--skip-stress/--skip-datasources/
--skip-recorder/--skip-debug/--skip-tcplogger but the AGENTS.md table row
was never updated to mention them; it also still listed a --stress flag
that has never existed. Sync with the script's actual --help output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-02 00:04:22 +02:00
Martino FerrariandClaude Opus 4.6 28d149f536 fix(e2e): assert real FORCE/TRACE/BREAK acks in the debug scenario 
The final whole-branch review found runDebugScript only checked
mc.IsConnected() after sending FORCE/TRACE/BREAK/UNFORCE -- a liveness
check that would PASS even if DebugService silently no-op'd every command
(e.g. a signal-name/wire-format bug), undercutting the design's stated
rationale for this scenario ("catching wire-format/serialization bugs the
in-process suite cannot"). This mirrors the tautology already fixed for
the tcplogger scenario in an earlier task, but had not been applied here.

Added waitForAck(), which polls the sink's recorded "text_line" events for
DebugServiceBase::HandleCommand's real "OK <TOKEN> <count>\n" reply and
requires count > 0 -- HandleCommand prints this for every one of
FORCE/UNFORCE/TRACE/BREAK regardless of enable/disable direction, and
count is always "number of signals actually matched", so count==0 means
the signal path was never resolved. Verified with a real negative control
(temporarily pointing all commands at a nonexistent signal name): the
scenario now correctly FAILs, then reverted and reconfirmed PASS against
the real signal.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-02 00:03:47 +02:00
Martino FerrariandClaude Opus 4.6 9a39cf923a fix(e2e): isolate coverage-pass WORK dir; filter chain-only e2e report section 
The final whole-branch review (post Task 10) found two real cross-task
integration bugs:

- run_e2e.sh's coverage-instrumented scenario re-run only rebound OUT_DIR
  in its subshell, not WORK. proc_perf.py/plots.py write perf_*.json and
  wave_*.png into WORK, so every --cpp-coverage run (the default) silently
  clobbered the primary pass's perf/waveform data with the instrumented
  re-run's numbers before report_build.py read them -- defeating the
  "uncontaminated performance metrics" goal of the coverage-double-run
  design. Fixed by rebinding WORK the same way OUT_DIR already was.

- report_build.py's build_e2e() iterated all results["scenarios"] with no
  kind filter, so the direct/recorder/debug/tcplogger scenarios (already
  covered by their own dedicated report sections since Task 8) also leaked
  into the chain-only Scenarios/Performance sections and headline e2e
  pass/fail count as degenerate rows. Fixed by filtering to kind=="chain".

Verified end-to-end with a full ./run_e2e.sh run: coverage pass now writes
to /tmp/chain_e2e/coverage_pass/ (confirmed via log), and report_data.json's
e2e section now reports 51 (chain-only) scenarios instead of all 56.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-01 23:59:54 +02:00
Martino FerrariandClaude Opus 4.6 07b6b4898a fix(e2e): rebuild test binaries when restoring non-instrumented build 
The post-coverage restore step ran `make clean` (which also wipes
Test/GTest, Test/Integration and Test/Components/*) but only rebuilt
`core apps`, leaving MainGTest.ex/IntegrationTests.ex deleted after
every --cpp-coverage run instead of restored to their plain (non-gcov)
form.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-01 22:55:48 +02:00
Martino Ferrari 03c7a95e9b docs: fix stale run_combined_test.sh comment in combined_test.cfg 
Minor follow-up from Task 9's retirement review.
 2026-07-01 22:08:21 +02:00
Martino Ferrari 45dcb9a71f docs: fix remaining dangling references to retired run_e2e_test.sh 
README.md and Docs/StreamHub-Developer.md still pointed at the deleted
run_e2e_test.sh / Test/E2E/streamhub Go client after their removal in the
previous commit; repoint at Test/E2E/suite/run_e2e.sh.
 2026-07-01 22:05:29 +02:00
Martino Ferrari 4286ea4539 chore(e2e): retire streamhub/datasources/recorder standalone scripts superseded by run_e2e.sh 2026-07-01 22:04:08 +02:00
Martino FerrariandClaude Opus 4.6 8337d678be feat(e2e): render direct/recorder/debug/tcplogger/stress sections in the unified report 
Extends report_build.py with build_by_kind() (per-scenario-kind pass/fail
rollup) and a ported build_stress()/stress_headline()/stress_plots() (scaling
curves per stress axis), wires both into the headline KPIs, regression
tracking, and report_data.json. E2E_Report.typ renders the four new
per-kind tables plus a Stress Tests section (per-axis case tables + scaling
plots, gracefully degrading to placeholders when a kind/stress data is
absent). run_e2e.sh now passes --stress-results so the report actually
receives real stress data instead of silently omitting it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-01 21:57:11 +02:00
Martino Ferrari b65ac06ce2 feat(e2e): instrument-first coverage flow with double-run + new skip flags; port stress multi-fragment sizing 2026-07-01 21:07:03 +02:00
Martino FerrariandClaude Opus 4.6 83d0a060fe feat(e2e): add debug/tcplogger E2E scenario kinds using debugclient 
Adds a trimmed debug_e2e.cfg (DebugService on 8080/8081, TcpLogger on
9090) and two new scenarios (s55_debug_force_trace_break, kind=debug;
s56_tcplogger_delivery, kind=tcplogger) reusing it, with matching
run_e2e.sh scenario-list/dispatch wiring and debugclient build steps.

Also fixes a real bug found while wiring s56: debugclient's tcplogger
check was tautological (it matched MarteController's own local
"CMD"-level echo of the outgoing command, which contains the same text
as the triggered event, instead of a line actually delivered over the
real TCPLogger TCP socket) and its trigger command (an invalid FORCE)
never reaches DebugServiceBase's REPORT_ERROR at all. Switched the
trigger to a MSG-to-missing-destination command (which does call
REPORT_ERROR) and the match to require the real log text
("not found in ORD"), recorded only after a connect-settle baseline —
verified with a positive run (real Warning-level TCPLogger line
received) and a negative control (LogPort=0 disables TcpLogger and the
scenario correctly FAILs).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-01 19:36:47 +02:00
Martino Ferrari efb4ea48fb feat(e2e): add debugclient Go tool for DebugService/TCPLogger E2E scenarios 
Standalone headless client that drives a running MARTeApp.ex's
DebugService (TCP 8080 commands, UDP 8081 trace) and TCPLogger (TCP
9090) via marte2debugger/controller's NewHeadlessMarteController, for
the upcoming Test/E2E/suite "debug"/"tcplogger" scenario kinds. Scripts
FORCE/TRACE/BREAK for -mode debug, and triggers+waits for a TCPLogger
log event for -mode tcplogger; reports PASS/FAIL as
result_<scenario>.json/status_<scenario>.txt in -out.
 2026-07-01 19:18:34 +02:00
Martino Ferrari f0f83110a4 fix(debugger): extract MarteController into an importable controller package 
Client/debugger was entirely package main, which Go forbids importing from
another module ("is a program, not an importable package") -- discovered
while wiring the new debugclient E2E tool against NewHeadlessMarteController.
Move martecontrol.go and its test into a new marte2debugger/controller
subpackage (package controller) and update Client/debugger/main.go to call
controller.NewMarteController/controller.DangerousCommandsEnabled. No
behavioral change to the browser-facing server.
 2026-07-01 19:18:08 +02:00
Martino Ferrari 269b2c4d97 refactor(debugger): extract MarteController sink so it can run headless 
Add a sink func(v any) field so MarteController's event stream can be
routed somewhere other than the browser WebSocket hub. NewMarteController
now sets sink to broadcast through the hub as before; a new
NewHeadlessMarteController(sink) constructor builds an instance with
hub == nil for the upcoming debugclient E2E tool. Direct m.hub.* calls
(SetSourceState/UpdateConfigForSource/PushDataForSource) are now guarded
with nil checks so a headless controller doesn't panic.
 2026-07-01 19:12:41 +02:00
Martino FerrariandClaude Opus 4.6 1d78b45963 feat(e2e): dispatch direct/recorder scenario kinds in run_e2e.sh 
Extends the scenario-list builder and main loop to actually execute
s52_direct_unicast/s53_direct_multicast (self-contained single-MARTeApp
FileReader->UDPStreamer->UDPStreamerClient->FileWriter round trip) and
s54_recorder (StreamHub BinaryRecorder round trip, ported from
run_recorder_e2e.sh) alongside the existing chain scenarios, and tags
each results.json record with its scenario kind.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-01 19:06:30 +02:00
Martino Ferrari ef58553c63 feat(e2e): add kind discriminator + direct/recorder scenario definitions 2026-07-01 18:51:18 +02:00
Martino Ferrari 69e52af20b refactor(e2e): rename Test/E2E/chain -> Test/E2E/suite, run_chain_e2e.sh -> run_e2e.sh 2026-07-01 18:42:33 +02:00
Martino Ferrari 1fcc4e4e6d fix(streamhub-test): unblock make test by fixing BoundsCheckTest C++98 build and registering both orphaned GTest files 2026-07-01 18:34:16 +02:00
Martino FerrariandClaude Opus 4.6 462b05b71a docs(testing): implementation plan for unified test/E2E/reporting/coverage pipeline 
Plan derived from the approved 2026-07-01 design spec; covers scenario
kind unification (chain/direct/recorder/debug/tcplogger), instrument-first
double-run coverage, and DebugService/TCPLogger E2E via debugclient.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-01 18:29:29 +02:00
Martino FerrariandClaude Opus 4.6 dcaa466736 docs(testing): design for unified test/E2E/reporting/coverage pipeline 
Consolidates the fragmented chain/stress/streamhub/datasources/recorder
E2E suites, unit-test collection, and coverage into one entry point and
one report, adds new DebugService/TCPLogger E2E coverage, and fixes the
Test/Applications/StreamHub build break blocking `make test`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-07-01 18:14:09 +02:00
Martino Ferrari 0bea41f866 Implemented better testing and fixed skipepd frames 2026-07-01 16:39:34 +02:00
Martino FerrariandClaude Sonnet 4.6 7a326c5d78 test(gtest): wire DebugServiceGTest into the shared MainGTest binary 
Add DebugServiceGTest.cpp (TraceRingBuffer, DebugSignalInfo, BreakOp
regression coverage for the HI-4/HI-9 fixes) to Test/GTest's OBJSX so it
builds and runs as part of ./Build/x86-linux/GTest/MainGTest.ex alongside
the rest of the unit suite. 17/17 tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-07-01 09:34:20 +02:00
178 changed files with 30474 additions and 4983 deletions
Expand all files Collapse all files
AGENTS.md
+244
View File
@@ -0,0 +1,244 @@
# Repository Guidelines
Guide for AI assistants working in the MARTe2 Integrated Components repository.
Focuses on non-obvious facts: commands, conventions, cross-module contracts, and
gotchas that are not self-evident from a single file read.
## Project Overview
MARTe2 component library with **two independent real-time data paths** sharing
one binary wire protocol (`Common/UDP/UDPSProtocol.h`):
1. **Streaming path** — `UDPStreamer` DataSource serialises DDB signals into UDPS
binary packets on UDP → `StreamHub` (headless C++ hub: ring buffers, LTTB
decimation, trigger FSM, history writer, binary recorder) → WebSocket 8090 →
clients (browser SPA, native ImGui, native Qt).
2. **Debug path** — `DebugService` patches `ClassRegistryDatabase` at
`Initialise()` so `ConfigureApplication()` wraps all `MemoryMap*Broker` types
with `DebugBrokerWrapper<T>` — **zero application code changes**. Exposes
TCP 8080 (text commands), UDP 8081 (UDPS trace telemetry), TCP 8082
(`TcpLogger` log forward).
## Architecture & Data Flow
```
[SineArrayGAM/TimeArrayGAM] → DDB → UDPStreamer (UDPS over UDP)
├─→ UDPStreamerClient (input DS back into a MARTe2 RT app, round-trip)
└─→ StreamHub: UDPSourceSession (receive thread → SignalRingBuffer)
→ push loop @30Hz: LTTB decimate temporal sigs → WS binary frames → clients
DebugService: patches broker builders at Initialise(); TCP 8080 commands,
UDP 8081 telemetry, TcpLogger 8082 (REPORT_ERROR → "LOG <LEVEL> <desc>" lines)
```
- **Wire protocol**: `Common/UDP/UDPSProtocol.h` is the canonical spec (17-byte
packed header, magic `0x53504455` 'UDPS', 136-byte signal descriptors,
CONFIG/DATA/ACK/CONNECT/DISCONNECT packet types, quant/time/publish modes).
Deliberately MARTe2-free so Go clients reuse it. **Mirrored across four
codebases that must stay in sync**: C++ producers (UDPStreamer, DebugService),
C++ consumer (`Source/Components/Interfaces/UDPStream/UDPSClient`), Go decoder
(`Common/Client/go/udpsprotocol/protocol.go`), and JS parsers
(`Client/udpstreamer/static/`, `Client/debugger/static/`). Any protocol change
must be mirrored in all of them.
- **WS protocol** has two implementations — Go hub (`Common/Client/go/wshub`) and
C++ StreamHub — that must behave identically; every client (SPA, ImGui, Qt)
must satisfy both. JSON text frames for commands/events (`addSource`,
`removeSource`, `setTrigger`, `arm`, `zoom`, `historyZoom`, `recStart`…), binary
frames for data pushes (live v1 + trigger capture v2).
- **Threading model**: RT threads only spinlock+memcpy (`FastPollingMutexSem`);
all socket I/O, fragmentation, and reassembly lives on background
`SingleThreadService` threads. StreamHub: per-session UDPSClient receive
threads + WS accept/read threads + one push loop.
- **DebugService patching**: `PatchRegistry()` replaces the ObjectBuilder for 11
`MemoryMap*Broker` classes; runs only when `ControlPort > 0`; static guard
against double-patching; wrappers persist for process lifetime.
## Key Directories
| Path | Purpose |
|---|---|
| `Source/Components/DataSources/UDPStreamer/` | Output DataSource; UDP I/O on bg thread, RT thread only spinlock+memcpy in `Synchronise()` |
| `Source/Components/DataSources/UDPStreamerClient/` | Input DataSource (shared `UDPSClient`), double-buffered ready/scratch |
| `Source/Components/GAMs/` | `SineArrayGAM` (float32 sine, continuous phase), `TimeArrayGAM` (us-timer → per-sample timestamp array; `Anchor = FirstSample|LastSample|Continuous`, use `Continuous` for contiguous sources so a lost RT cycle cannot hole the time base) |
| `Source/Components/Interfaces/DebugService/` | Registry patching, `DebugBrokerWrapper.h`, TCP/UDP services |
| `Source/Components/Interfaces/TCPLogger/` | `LoggerConsumerI` forwarding `REPORT_ERROR` to ≤8 TCP clients |
| `Source/Components/Interfaces/UDPStream/` | Plain-C++ helpers (not MARTe2 Objects): `UDPSClient` (auto-reconnect + fragment reassembly), `UDPSServer` (not thread-safe — owner's Execute thread only) |
| `Source/Applications/StreamHub/` | Standalone app (links MARTe2 core): `StreamHub`, `UDPSourceSession`, `WSServer`, `TriggerEngine`, `HistoryWriter`, `BinaryRecorder`, `LTTB`, `SignalRingBuffer` |
| `Common/UDP/` | Canonical wire protocol (header-only, MARTe2-free) |
| `Common/Client/go/` | Go mirror: `udpsprotocol` (decoder), `wshub` (WS hub client) |
| `Client/udpstreamer/` | Go legacy direct-UDP oscilloscope web UI (connects straight to UDPStreamer, no StreamHub) |
| `Client/webui/` | Go thin static server; SPA talks WS directly to C++ StreamHub (discovers via `GET /hub`) |
| `Client/debugger/` | Go debug web UI for DebugService |
| `Client/streamhub/` | Native ImGui+SDL2+OpenGL oscilloscope (C++17, no MARTe2) |
| `Client/streamhub-qt/` | Native Qt Widgets oscilloscope (Qt6 preferred, Qt5 fallback) |
| `Test/` | GTest, legacy Integration tests, Configurations (.cfg), E2E suite |
| `Docs/` | Per-component reference: `Protocol.md`, `UDPStreamer.md`, `StreamHub-{API,UserGuide,Developer}.md`, `DebugService.md`, `WebUI.md`, `Tutorial.md`, `E2E-Suite.md` |
## Development Commands
`source env.sh` is **mandatory** before any MARTe2 build or run (sets
`MARTe2_DIR`, `MARTe2_Components_DIR`, `TARGET=x86-linux`, `LD_LIBRARY_PATH`).
The E2E scripts source it themselves; a bare `make` from a fresh shell will not
work. `run_streamhub.sh` hard-errors if `MARTe2_DIR` is unset.
```bash
source env.sh
make -f Makefile.gcc core # 7 components (UDPStream interface FIRST, then UDPStreamer, UDPStreamerClient, GAMs, TCPLogger, DebugService)
make -f Makefile.gcc apps # StreamHub standalone app → Build/x86-linux/StreamHub/StreamHub.ex
make -f Makefile.gcc test # GTest + Integration test binaries + component test libs
make -f Makefile.gcc all # core + apps + test
make -f Makefile.gcc clean
# Single component:
make -C Source/Components/GAMs/SineArrayGAM -f Makefile.gcc
```
Build output → `Build/x86-linux/` mirroring `PACKAGE` paths (both `libX.so` and
`X.so` are produced). `compile_commands.json` (repo root, gitignored) feeds
LSP/clangd; CMake clients export their own into `Client/*/build/`.
### Non-MARTe2 clients (no env.sh needed)
```bash
cd Common/Client/go && go build ./...
cd Client/debugger && go build ./...
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
cd Client/streamhub-qt && cmake -B build && cmake --build build
```
### Key scripts
| Script | Purpose |
|---|---|
| `./run_streamhub.sh` | Demo stack: build + launch MARTe2 app + StreamHub, optional web (`-w`) / ImGui (`-g`) clients. Flags `-m/-c` MARTe2 dirs, `-b TARGET`, `-p WS_PORT`, `-n MAX_POINTS` (actual default 1000000, header says 10000), `-s` skip build. Generates temp hub cfg with `+History`/`+Recorder` blocks in `/tmp`. Ctrl-C kills all. |
| `./Test/E2E/suite/run_e2e.sh` | Full E2E: 57-scenario matrix + stress + unit suites + gcov coverage + Typst PDF report. Flags: `--skip-build`, `--only <id>`, `--pdf-only`, `--skip-coverage`, `--skip-stress`, `--skip-datasources`, `--skip-recorder`, `--skip-debug`, `--skip-tcplogger` |
| `./Test/E2E/suite/run_stress.sh` | Capacity harness: sweeps one load axis at a time (`--axis`), hard gates survival+liveness, soft gates RSS+zoom-p95 |
## Code Conventions & Common Patterns
- **No STL in `Source/Components/**` (and StreamHub)**: use `StreamString` (not
`std::string`), `FastPollingMutexSem`/`EventSem` (not `std::mutex`/threads),
fixed arrays / MARTe2 `Vector<T>` (not `std::vector`), `REPORT_ERROR` /
`REPORT_ERROR_STATIC` macros (no exceptions). C stdlib is fine. Heap
`new`/`delete[]` is normal. STL/C++17 is fine in `Client/streamhub/` and
`Client/streamhub-qt/`.
- **RT hot-path rule**: `FastPollingMutexSem` on real-time hot paths, never OS
mutexes; RT cycle must not block on the scheduler.
- **Class registration**: `CLASS_REGISTER_DECLARATION()` in the class `public:`
section of the header; `CLASS_REGISTER(Name, "1.0")` at the end of the `.cpp`
inside `namespace MARTe`. Every component `.cpp` ends with it.
- **EUPL v1.1 license headers** on all C++ sources and `Makefile.inc` — preserve
on new files.
- **Per-component build**: each dir has one-line `Makefile.gcc` wrapper
(`include Makefile.inc`) + `Makefile.inc` declaring `OBJSX`, `PACKAGE`,
`ROOT_DIR`, `INCLUDES` (re-declared per file, ~12 MARTe2 layer dirs),
`LIBRARIES`, including `MakeStdLibDefs.$(TARGET)` then
`MakeStdLibRules.$(TARGET)`. Generated `depends.x86-linux` (gcc -MM) is
committed but **never hand-edited** — delete to regenerate.
- **Qt client**: `QT_NO_KEYWORDS` is required (reused `Protocol.h` structs have
members named `signals`); Qt classes use `Q_SIGNALS`/`Q_SLOTS`/`Q_EMIT`. Run
with long options: `--host HOST --port 8090` (single-dash misparsed). Single
GUI thread, 60 Hz QTimer repaint.
- **StreamHub config** is *not* a MARTe2 `RealTimeApplication`: `Hub = { WSPort
MaxPoints PushRate MaxPushPoints RingTemporal RingScalar RingMaxMB AllowedOrigins
+Recorder{...} Sources={id={Label Addr Port}} }`. `AllowedOrigins` is the
WebSocket Origin allowlist — without it a browser serving the SPA from a
different port than the hub is rejected 403.
`+History` keys: `Directory` (required),
`DurationHours` (1), `Decimation` (1), `FlushIntervalSec` (5),
`MinDiskFreeMB` (500). `.shist` files: 64-byte header ('SHR1') + circular
(t,v) float64 pairs.
- **UDPStreamer config**: `Port` (44500; multicast data = `DataPort`, default
`Port+1`), `MaxPayloadSize` (1400), `PublishingMode` `Strict`/`Accumulate`,
per-signal `Signals={Name={Type,Unit,NumberOfDimensions,NumberOfElements,
TimeMode}}` with `TimeMode` `PacketTime`/`FirstSample`/`LastSample`/`FullArray`;
multicast needs `MulticastGroup` + `Interface`.
## Important Files
- `env.sh` — environment; source first, always.
- `Makefile.gcc` / `Makefile.inc` (root) — build orchestration.
- `Common/UDP/UDPSProtocol.h` — canonical wire format; changing it triggers the
4-way mirror checklist above.
- `Source/Applications/StreamHub/main.cpp` — hub entry (`[-cfg file.cfg]
[-port N] [-maxPoints N]`); hub **must be heap-allocated** (~128 MB, exceeds
the 8 MB stack).
- `Test/Configurations/*.cfg` — MARTe2 app configs (`$App = { Class =
RealTimeApplication }` with `+Functions`, `+DataSources`, `+States`, `+Timings`
blocks); `streamhub_demo.cfg` and `TestApp.cfg` are good templates.
- `Test/E2E/suite/{scenarios,gen_data,gen_cfg,validate_waveform,stress}.py` —
declarative scenario matrix and generators consumed identically by the Go
chain-client and validators.
- `Client/debugger/main.go` — `-addr :7777` default, `-enable-dangerous-commands`
safety gate (CR-4) for FORCE/PAUSE/RESUME/STEP/BREAK/MSG.
## Runtime/Tooling Preferences
- **OS**: Linux x86_64 (`TARGET=x86-linux`). External deps live outside this
repo: `MARTe2_DIR` (default `~/workspace/MARTe2`) and
`MARTe2_Components_DIR` (default `~/workspace/MARTe2-components`) — edit
`env.sh` if they differ. `env.sh`'s `LD_LIBRARY_PATH` does **not** cover
UDPStreamerClient/UDPStream lib dirs.
- **C++**: MARTe2 `Makefile.gcc` wrapper system, gtest-1.7.0 for tests.
- **Go**: `go 1.21`; modules use `replace marte2/common => ../../Common/Client/go`
(`gorilla/websocket` v1.5.1). Go binaries are gitignored.
- **ImGui client**: needs SDL2; CMake FetchContent pins Dear ImGui **v1.91.8** +
ImPlot **v0.17** (`implot_items.cpp` is a slow -O3 TU, ~2 min rebuild).
- **Qt client**: Qt6 preferred, Qt5 fallback, Widgets + WebSockets, custom
QPainter plotting (no QtCharts).
- **E2E report**: `typst compile E2E_Report.typ`; Python 3 + numpy for the suite.
- Remove `vgore.*` core dumps when you see them; they are not gitignored.
## Testing & QA
Four test layers; `env.sh` + built stack required for all but the standalone
ones. Only `tests_py.py`, Go tests, and the built C++ test binaries run
standalone.
```bash
./Build/x86-linux/GTest/MainGTest.ex --gtest_filter='Name*' # C++ GTest
./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex # legacy DebugService runtime tests
cd Test/E2E/suite/client && go test ./... # Go chain-client unit tests
cd Test/E2E/suite && python3 -m unittest tests_py # framework logic, standalone
```
- **GTest**: `MainGTest.ex` currently holds only `DebugServiceGTest`
(TraceRingBuffer SPSC, DebugSignalInfo, BreakOp). Component GTests
(`UDPStreamerGTest.cpp` ~46 cases, `StreamHubTest.a`, `UDPStreamerClientTest.a`)
compile **as libraries only — no standalone executable**.
- **Legacy IntegrationTests.ex**: 9 printf-narrated DebugService runtime tests,
always returns 0; `collect.py` parses stdout blocks.
- **E2E suite** (`run_e2e.sh`): 57 curated scenarios (s01–s57) across kinds
`chain`/`direct`/`recorder`/`debug`/`debug_pause_resume`/`tcplogger`, driven
against live MARTeApp.ex + StreamHub.ex + Go chain-client. `scenarios.py` is a
curated covering set: **every configurable UDPStreamer option value appears in
≥1 scenario** — add a scenario when adding an option.
- **Oracle gates** (`validate_waveform.py`): **fidelity** (every received value
within `tol` of ground truth; 0 for un-quantised ints, float epsilon for
un-quantised floats, `quant_step/2 + 1e-6·range` for quantised) is the
**correctness gate**. **Shape** is a *gross* sanity gate + tracked metric
(`corr >= 0.5`, `nRMSE <= 0.30` relaxed by quant step, frequency searched
±5% band); a correct sinusoid yields corr ~0.82–0.98, wrong frequency
collapses to ~0.00. Do **not** tighten shape into a correctness gate —
timestamp calibration (Phase-A) is pending.
- **Stress** (`run_stress.sh`): 7 axes (signal size/count/fan-out/sources/WS
clients/zoom rate), hard gates survival+liveness, soft gates RSS+zoom-p95.
- **Coverage**: `--cpp-coverage` rebuilds with gcov, captures via `lcov`
restricted to `Source/*` + `Test/*`, then restores a clean build.
- Artifacts → `Build/x86-linux/E2E/chain/`: `results.json` (XFAIL/XPASS for
`known_issue` markers), `report_data.json`, `history.jsonl`, `trend_*.png`,
`E2E_Report.pdf`; stress → `stress/stress_results.json`.
## Ports Reference (defaults)
| Port | Protocol | Component | Purpose |
|---|---|---|---|
| 44500 | UDP | UDPStreamer | scalar signals (unicast control + data) |
| 44501/44502 | UDP | UDPStreamer | packed arrays (FirstSample/LastSample, FullArray) |
| 44503 | UDP | UDPStreamer | multicast data (group 239.0.0.1) |
| 8080 | TCP | DebugService | text command channel (one client at a time, newline-terminated) |
| 8081 | UDP | DebugService | trace telemetry (UDPS format) |
| 8082 | TCP | TcpLogger | REPORT_ERROR log forward |
| 8090 | TCP/WS | StreamHub | WebSocket (commands + binary data) |
| 7777 | TCP | Client/debugger | debug web UI (older docs say 9090; current flag is `-addr`) |
| 8080 | TCP | Client/udpstreamer, Client/webui | web UI listen (collides with DebugService in combined demos — scripts adjust) |
ARCHITECTURE.md
+64 -4
View File
@@ -314,7 +314,7 @@ Hub-side trigger with the web client's semantics (config: signal key
```
IDLE →[arm]→ ARMED
ARMED →[edge crossing]→ COLLECTING (latches trigTime, pre/postSec)
COLLECTING →[post window + margin elapsed]→ TRIGGERED (broadcast binary v2 capture)
COLLECTING →[every source produced past the window]→ TRIGGERED (broadcast binary v2 capture)
TRIGGERED →[auto-rearm (normal, ~200 ms) | rearm (single)]→ ARMED
any →[disarm]→ IDLE
```
@@ -325,6 +325,30 @@ sample of the configured signal. The capture is assembled in the push loop from
LTTB-capped at 20 000 points/signal, and broadcast as a binary version-2 frame.
A `stopped` flag (`trigStop`) freezes auto-rearm.
COLLECTING is left on the **data's** clock, not `clock_gettime()`: `trigTime`
comes from sample timestamps, and a source that free-runs on its own clock sits
seconds away from wall time, so a wall-clock deadline chops exactly that offset
off every capture's tail. `UDPSourceSession::ProducerNewestTime()` reports how
far a source has produced — counting only signals actually timestamped from a
time signal, since PACKET-timed ones (the time array itself included) are
stamped on arrival and would just report "now".
Sources are harvested **one at a time**, each as soon as *it* passes
`trigTime + postSec + 0.15 s` (`BeginTriggerCapture` / `HarvestTriggerCapture` /
`FinishTriggerCapture`, the frame accumulating across push ticks). Making every
source wait for the slowest lets the leaders' rings roll past the pre-trigger
region before it is ever read. A 2 s wall-clock watchdog per capture bounds the
wait for a source that stopped advancing; it is harvested short, with a warning
naming the source and how far it got.
Because `RingTemporal` only holds ~1 s at 1 MSps, `setTrigger` publishes the
requested window and the push loop calls `GrowRingsForTrigger()`: each ring
measures its own rate (`Count() / TimeSpan()` — UDPS sources usually report
`samplingRate = 0`) and is grown in place to `rate × (window + 0.5 s) × 1.2`,
clamped per signal to `RingMaxMB`. `SignalRingBuffer::Grow()` preserves
contents *and* `totalWritten`, so live push cursors stay valid. Without this a
long window only ever captures its tail.
### Configuration File (MARTe2 cfg format)
```
@@ -334,9 +358,11 @@ Hub = {
PushRate = 30 // push loop Hz
MaxPushPoints = 50 // LTTB cap per signal per tick
StatsRate = 1 // stats broadcast Hz
RingTemporal = 1000000 // ring capacity (points) for multi-element signals
RingTemporal = 1000000 // initial ring capacity (points) for multi-element signals
RingScalar = 100000 // ring capacity (points) for scalar signals
RingMaxMB = 128 // per-signal ceiling when a trigger window grows a ring
SourcesFile = "streamhub_sources.json" // dynamic-source persistence
AllowedOrigins = "http://127.0.0.1:8099,http://localhost:8099" // see below
Sources = {
App1 = {
Label = "MARTe2 App 1"
@@ -358,6 +384,16 @@ Sources added at runtime (`addSource`) get generated ids `s1, s2, …`;
`saveSources` persists them to `SourcesFile` (JSON array of
`{label, addr, multicastGroup?, dataPort?}`), reloaded at start-up.
`AllowedOrigins` is a comma/space-separated allowlist of `scheme://host[:port]`
values accepted in the WebSocket `Origin` header (max 8 entries, 128 chars
each), matching the Go hub's option. Without it the handshake only accepts an
`Origin` whose host matches the request `Host` — so a browser that loaded the
SPA from a *different* port than the hub (the `run_streamhub.sh` layout, SPA on
8099 and hub on 8090) is rejected with 403. Non-browser clients send no `Origin`
and are unaffected. This is the CSWSH guard of RFC 6455 §10.2: browsers attach
cookies to cross-origin WebSocket handshakes, so `Origin` is the only thing
distinguishing a legitimate page from an attacker's.
### Build
```bash
@@ -380,7 +416,9 @@ binary frames carry data push payloads.
| `ping` | — | Hub replies `{"type":"pong"}` |
| `addSource` | `label`, `addr` (`"host:port"`), `multicastGroup?`, `dataPort?` | Connect to a new UDPS source; hub assigns id `s1, s2, …` |
| `removeSource` | `id` | Disconnect and remove a source |
| `saveSources` | — | Persist the current dynamic source list to `SourcesFile` (JSON) |
| `saveSources` | — | Persist the dynamic source list **and** the calibration table to `SourcesFile`; replies `configSaved` |
| `setCalibration` | `source` (label), `signal` (base name), `scale`, `offset`, `unit` | Record `value = raw × scale + offset` for one signal; metadata only, the hub never applies it. Identity entries are deleted. Replies with a `calibration` broadcast |
| `reloadConfig` | — | Re-read `SourcesFile`: calibration replaced wholesale, missing sources added, live sources never touched; replies `configReloaded` |
| `getSources` | — | Trigger `sources` broadcast |
| `getConfig` | `sourceId` | Trigger `config` broadcast for one source |
| `getStats` | — | Trigger `stats` broadcast |
@@ -399,11 +437,33 @@ binary frames carry data push payloads.
| `sources` | `sources:[{id, label, addr:"host:port", state}]` | On connect; after add/remove/getSources; on first CONFIG |
| `config` | `sourceId`, `publishMode`, `signals:[{name, typeCode, quantType, numDimensions, numRows, numCols, rangeMin, rangeMax, timeMode, samplingRate, timeSignalIdx, unit}]` | After CONFIG received from source |
| `stats` | `sources:{id:{state, totalReceived, totalLost, rateHz, rateStdHz, fragsPerCycle, bytesPerCycle, cycleAvgMs, cycleStdMs, cycleMinMs, cycleMaxMs, cycleHistMin, cycleHistMax, cycleHist:[20]}}` | At `StatsRate` Hz |
| `triggerState` | `state` (`"idle"`\|`"armed"`\|`"collecting"`\|`"triggered"`), `mode`, `stopped`, `trigTime?` | On any trigger FSM transition |
| `triggerState` | `state` (`"idle"`\|`"armed"`\|`"collecting"`\|`"triggered"`), `mode`, `stopped`, `trigTime?`, `preSec?`, `postSec?` | On any trigger FSM transition |
| `zoom` | `reqId`, `signals:{"src:sig":{t:[…], v:[…]}}` (`t` printed `%.17g`, `v` `%.9g`) | Unicast reply to `zoom` |
| `maxPointsUpdated` | `maxPoints` | After ring buffer resize |
| `calibration` | `cal:[{source, signal, scale, offset, unit}]` | On connect; after an accepted `setCalibration`; after a successful `reloadConfig` |
| `configSaved` | `ok`, `path`, `error?` | In reply to `saveSources` |
| `configReloaded` | `ok`, `path`, `error?` | In reply to `reloadConfig` |
| `pong` | — | In reply to `ping` |
### Config File Format
`SourcesFile` is a flat JSON array of flat objects; `addr` marks a source,
`signal` marks a calibration entry.
```json
[
{"label": "wave", "addr": "127.0.0.1:44500"},
{"source": "wave", "signal": "Adc", "scale": 0.00030518, "offset": -1.25, "unit": "V"}
]
```
Flatness is a hard constraint: `StreamHub::LoadSourcesFile` scans from each `{`
to the next `}`, so a nested object would truncate the parse. Both hubs read and
write this format identically, and pre-calibration files load unchanged.
Calibration is applied **client-side only**. Rings, history, `zoom` replies, both
binary frames and the trigger comparator are all in raw units.
### Binary Push Frame (version 1, hub → client, binary WS frame)
Little-endian throughout. Sent at `PushRate` Hz per source; contains **only
BUG_FIX_PLAN.md
+221
View File
@@ -0,0 +1,221 @@
# Bug Fix Plan — Security & Correctness Remediation
**Date:** 2026-06-26
**Based on:** `BUG_REPORT.md`
**Scope:** `Source/` and `Client/`
This plan organizes the ~60 findings from the audit into prioritized, dependency-ordered phases. Each phase is independently shippable. Phases are ordered by risk reduction: Critical remote-exploitable issues first, then High crash/OOB issues, then Medium robustness/DoS, then Low hardening.
---
## Guiding principles
1. **Fix root causes, not symptoms.** The integer-overflow-in-bounds-check pattern appears in 6+ places — fix the pattern, not each instance ad hoc. Introduce a shared `boundsCheck(off, count, elemBytes, bufLen)` helper (C++) and a `validateCount(count, elemSize, bufLen)` helper (Go) and use them everywhere.
2. **Defense in depth.** Origin checks + auth + input validation — not just one layer.
3. **No regressions.** After each phase, run the existing test suites (`make -f Makefile.gcc test`, `python3 -m unittest tests_py`, `go test ./...` in each Go module) and the E2E suite (`./Test/E2E/chain/run_chain_e2e.sh --skip-build`).
4. **Minimal blast radius.** Each fix is surgical to the file/function listed in the bug report. No refactors beyond what the fix requires.
---
## Phase 1 — Critical remote-exploitable fixes (ship first)
**Goal:** Eliminate drive-by takeover and remote heap corruption. All fixes are small and localized.
| # | Bug | File(s) | Fix | Est. effort | Depends on |
|---|-----|---------|-----|-------------|------------|
| 1.1 | CR-1: 1-byte heap OOB write in WS frame NUL-term | `WSServer.cpp:251` | Change `kRecvBuf = WS_MAX_RECV_PAYLOAD + 14u` → `+ 14u + 1u` | 5 min | — |
| 1.2 | CR-2: XSS via unescaped `src.addr` | `Client/udpstreamer/static/app.js:3503`; `Client/debugger/static/app.js:3549` | Wrap `src.addr` with existing `escHtml()` in `_statsKV` calls (or inside `_statsKV` itself) | 10 min | — |
| 1.3 | CR-3: WebSocket CSRF (Origin check disabled) | `Common/Client/go/wshub/hub.go:128`; `Source/Applications/StreamHub/WSServer.cpp:186-239` | **Go:** Replace `CheckOrigin: func(r *http.Request) bool { return true }` with a same-origin check (compare `Origin` header host to `Host` header). Add a configurable allowlist env var for non-local deployments. **C++:** Parse `Origin` header in `UpgradeHTTP`; reject if present and host doesn't match the listen address. | 30 min | — |
| 1.4 | CR-4: Unauthenticated command injection to MARTe2 | `Client/debugger/martecontrol.go:217-263` | Add an allowlist of permitted MARTe2 commands (`DISCOVER`, `TREE`, `INFO`, `LS`, `VALUE`, `TRACE`, `UNTRACE`); reject `FORCE`, `UNFORCE`, `PAUSE`, `RESUME`, `STEP`, `BREAK`, `MSG` unless an explicit `--enable-dangerous-commands` flag is set. Log all forwarded commands. | 1 h | 1.3 |
| 1.5 | CR-5: No auth on DebugService TCP | `DebugService.cpp:276` | (a) Bind TCP server to localhost by default (add `BindAddress` config key, default `127.0.0.1`). (b) Add an optional `AuthToken` config key; if set, require the first line from a client to be `AUTH <token>` before accepting commands. | 2 h | — |
**Validation:** `bash -n` on shell scripts; `go build ./...` in each Go module; `make -f Makefile.gcc core apps`; manual test: open browser console on a cross-origin page and confirm WS to `localhost:8090` is rejected; confirm a crafted 65536-byte WS frame no longer corrupts.
**Commit:** `fix(security): critical remote-exploitable fixes (CR-1..CR-5)`
---
## Phase 2 — High-severity crash / OOB / UAF fixes
**Goal:** Eliminate remote crash and memory-corruption vectors. These are the integer-overflow and concurrency bugs.
### 2A — Integer-overflow bounds checks (uniform pattern)
| # | Bug | File(s) | Fix |
|---|-----|---------|-----|
| 2A.1 | HI-1: DATA bounds check overflow | `UDPSourceSession.cpp:358`; `UDPStreamerClient.cpp:520` | Replace `off + elemsToRead * wireElemBytes > size` with 64-bit arithmetic. Add a `validateBounds(off, count, elemBytes, size)` static helper in `UDPSProtocol.h` and use it in both files. |
| 2A.2 | HI-2: Go unbounded allocations | `protocol.go:121, 229, 325` | Add `validateCount(count, elemSize, bufLen)` in `protocol.go`; call before every `make([]T, n)` that uses a network-derived count. Cap `NumElements()` at 1M. |
| 2A.3 | HI-3: `accumFill` overflow + size calc | `UDPStreamer.cpp:700, 738, 757, 857-860` | (a) Add `if (accumFill >= maxBatchCount) { flush; }` before the write at line 857. (b) Use `uint64` for `maxBatchCount * totalSrcBytes` size calculations. |
| 2A.4 | MD-4: `numRows * numCols` overflow | `UDPSourceSession.cpp:240, 346`; `protocol.go:121` | Use `static_cast<uint64>(numRows) * static_cast<uint64>(numCols)`; cap at 1M. |
| 2A.5 | MD-15: `pairCount * 16u` overflow | `Client/streamhub/Protocol.cpp:77, 117` | Check `pairCount > (len - off) / 16` before multiplication; use `ull` suffix. |
| 2A.6 | HI-6: `FD_SET` overflow | `UDPSServer.cpp:273, 308`; `UDPSClient.cpp:383` | Add `if (fd < FD_SETSIZE)` guard before each `FD_SET`; otherwise skip that client this cycle (or switch to `poll()`, which the codebase already uses elsewhere). |
**Est. effort:** 3 h (pattern is repetitive once the helper exists)
### 2B — Use-after-free and concurrency
| # | Bug | File(s) | Fix |
|---|-----|---------|-----|
| 2B.1 | HI-5: Broadcast vs FreeSlot UAF | `WSServer.cpp:345-366, 432-445` | `FreeSlot` must acquire `clients[idx].writeMutex` before setting `active=false` and deleting `sock`. This ensures `BroadcastText`/`BroadcastBinary` cannot dereference a freed socket. |
| 2B.2 | HI-9: TraceRingBuffer not thread-safe | `DebugCore.h:79-142` | Replace `volatile uint32 readIndex/writeIndex` with `Atomic<uint32>` (MARTe2 `Atomic::Load`/`Atomic::Store`). Ensure `Push` writes data before storing `writeIndex` (release ordering); `Pop` loads `writeIndex` before reading data (acquire ordering). |
| 2B.3 | HI-4: `ProcessSignal` unclamped memcpy + `forcedMask` OOB | `DebugServiceBase.cpp:310, 313-318` | (a) Clamp `size` to `sizeof(signalInfo->forcedValue)` (1024). (b) Cap the array-forcing loop at `min(nEl, 256)`. (c) Validate `nEl <= 256` in `RegisterSignal`. |
| 2B.4 | HI-7: Weak PRNG for WS handshake | `WSClient.cpp:29-31` | Replace `srand(time(nullptr))` + `rand()` with `std::random_device` or `getrandom()`/`/dev/urandom` read. |
| 2B.5 | HI-8: Global registry patching | `DebugServiceBase.cpp:217-242` | (a) Save original builders before patching (`item->GetObjectBuilder()`); store in a static array for restore on destruction. (b) Add a `PatchRegistry` config flag (default `true` for back-compat; document the implication). (c) Guard against double-patching (skip if already patched). |
**Est. effort:** 4 h
**Validation:** `make -f Makefile.gcc test` + `./Build/x86-linux/GTest/MainGTest.ex` + `./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex` + `python3 -m unittest tests_py` (in `Test/E2E/chain/`) + `go test ./...` (in each Go module). Craft a UDP packet with `numSamples=0x20000001` and confirm no crash. Run the E2E suite: `./Test/E2E/chain/run_chain_e2e.sh --skip-build`.
**Commit:** `fix(security): high-severity crash/OOB/UAF fixes (HI-1..HI-9)`
---
## Phase 3 — Medium-severity robustness / DoS / parser fixes
**Goal:** Harden input validation, fix reassembly logic, and improve WS RFC compliance.
### 3A — UDPS protocol hardening
| # | Bug | File(s) | Fix |
|---|-----|---------|-----|
| 3A.1 | MD-1: `recvMask` too small | `UDPSClient.cpp:544, 592-594` | Enlarge `recvMask` to 64 bytes (512 bits) to match the `totalFragments <= 512` cap. |
| 3A.2 | MD-2: No type matching in reassembly | `UDPSClient.cpp:548-555` | Add `type` field to `ReassemblySlot`; key on `counter && type`. |
| 3A.3 | MD-3: Signal name not null-terminated | `UDPSourceSession.cpp:219-223` | After `memcpy`, force `name[63]='\0'` and `unit[31]='\0'`. |
| 3A.4 | MD-6: No auth on UDP CONNECT | `UDPSServer.cpp:655-723` | Document trust boundary in `Docs/Protocol.md`. Optional: add a `ConnectToken` config key. |
| 3A.5 | MD-13: Reassembler unbounded map growth (Go) | `reassembler.go:41-89` | Add `maxSets = 1024` cap; reject new sets when full. |
| 3A.6 | LO-1: `totalFrags` overflow | `UDPSServer.cpp:541-542` | Validate `payloadSize <= maxPayloadSize * 65535` before the calculation. |
| 3A.7 | LO-17: `bufMutex.Create` unchecked | `UDPStreamer.cpp:119`; `UDPStreamerClient.cpp:149` | Check return value; `REPORT_ERROR` on failure. |
| 3A.8 | LO-19: Reassembler ticker panic | `reassembler.go:93` | Guard `if r.expiry <= 0 { r.expiry = 2 * time.Second }`. |
**Est. effort:** 2 h
### 3B — WebSocket and JSON robustness (C++ clients)
| # | Bug | File(s) | Fix |
|---|-----|---------|-----|
| 3B.1 | MD-16: `readU16`/`readU32` silent failure | `Client/streamhub/Protocol.cpp:21-38` | Change `readU16`/`readU32`/`readF64` to return `bool` (or set an `ok` flag); `ParseBinaryFrame` fails fast on any truncated read. |
| 3B.2 | MD-17: JSON injection in command builders | `Client/streamhub/Protocol.cpp:183-213` | Add a `jsonEscape(str)` helper; use it for all `%s` string interpolations. Switch to `std::string` to avoid truncation. |
| 3B.3 | MD-18: `strstr`-based JSON parsing | `Client/streamhub/Protocol.cpp:296-310, 495, 510` | Migrate `ParseSources`, `ParseZoom`, `ParseStats` to a real JSON parser. **ImGui:** add a minimal JSON parser or vendor a single-header library (e.g. nlohmann/json). **Qt:** use `QJsonDocument`. |
| 3B.4 | MD-19: WS RFC 6455 violations | `WSClient.cpp:204-223` | (a) Reject control frames with `payloadLen > 125`. (b) Implement `CONTINUATION` opcode reassembly (or at least log and drop with a clear message). (c) Echo `CLOSE` frame. |
| 3B.5 | MD-20: Handshake no timeout | `WSClient.cpp:290-301` | Set `SO_RCVTIMEO` to 5s on the socket before the handshake loop. |
| 3B.6 | MD-5: SHA1 latent overflow | `SHA1.h:50`; `WSFrame_client.h:113` | Add `if (len > 119u) return;` guard; use `uint64_t bitLen`; use `std::vector` instead of `new[]`/`delete[]`. |
| 3B.7 | MD-24: `parseCapture` panic | `Test/E2E/chain/client/main.go:140-171` | Add bounds checks before each read, mirroring `parsePush`. |
**Est. effort:** 4 h (3B.3 is the largest item — JSON parser migration)
### 3C — Go hub and debugger hardening
| # | Bug | File(s) | Fix |
|---|-----|---------|-----|
| 3C.1 | MD-10: No WS client cap | `hub.go:367-377` | Track `len(h.clients)`; reject above configurable max (default 32). |
| 3C.2 | MD-11: Silent data loss | `hub.go:346-358` | Add a `droppedCount` atomic counter per channel; expose via `Snapshot()`. |
| 3C.3 | MD-12: SSRF via `addSource` | `hub.go:83-96`; `sources.go:62-67` | Validate `addr` against a configurable allowlist (default: localhost + private RFC1918 ranges; reject link-local/metadata endpoints like `169.254.169.254`). |
| 3C.4 | MD-14: Index panic | `martecontrol.go:543` | Use `strings.TrimPrefix(line, "OK SERVICE_INFO ")` with a length check. |
| 3C.5 | LO-14: `stopCh` double-close | `martecontrol.go:182-189` | Use `sync.Once` for closing `stopCh`. |
| 3C.6 | LO-10: `unsafe.Pointer` aliasing | `hub.go:588-594` | Replace `float64ToBytes` with `binary.LittleEndian` put operations. |
| 3C.7 | LO-11: `+Inf` in JSON | `stats.go:115-116` | Guard `if avg > 0 { si.RateHz = 1.0 / avg } else { si.RateHz = 0 }`. |
**Est. effort:** 2 h
### 3D — TcpLogger and DebugService fixes
| # | Bug | File(s) | Fix |
|---|-----|---------|-----|
| 3D.1 | MD-7: `StringHelper::Copy` overflow | `TcpLogger.cpp:87` | Replace with `strncpy(entry.description, description, MAX_ERROR_MESSAGE_SIZE-1); entry.description[MAX_ERROR_MESSAGE_SIZE-1]='\0';` |
| 3D.2 | MD-8: `volatile` indices + lost wakeup | `TcpLogger.cpp:83-153, 157-158` | Use `Atomic::Load`/`Store` for `writeIdx`/`readIdx`; use `eventSem.ResetWait()` instead of `Wait`+`Reset`. |
| 3D.3 | MD-9: `printf` on RT thread | `TcpLogger.cpp:75-76` | Add a `MirrorToStdout` config key (default `false`); guard the `printf`/`fflush` behind it. |
| 3D.4 | MD-21: Stack buffer + shadowed member | `DebugService.cpp:438, 489` | Remove the local `udpsSampleBuf` (use the member); heap-allocate `cfgBuf`. |
| 3D.5 | MD-23: `configValidated` read without lock | `UDPStreamerClient.cpp:463` | Mark `volatile` or acquire `bufMutex` before reading. |
| 3D.6 | MD-22: Spinlock on RT path | `UDPStreamer.cpp:856, 947-976` | Minimize the RT-side critical section: swap a pointer instead of `memcpy` under the lock. Move the `memcpy` outside the lock (double-buffer pattern). |
| 3D.7 | LO-7: JSON escaping in DISCOVER | `DebugServiceBase.cpp:900-906` | Use the existing `EscapeJson` helper for signal names. |
| 3D.8 | LO-8: `EvaluateBreak` only element 0 | `DebugBrokerWrapper.h:61-86` | Document the limitation in the function comment. |
| 3D.9 | LO-9: `fprintf(stderr)` on init | `DebugBrokerWrapper.h:195-197` | Replace with `REPORT_ERROR`. |
**Est. effort:** 3 h
**Validation:** Full test suites + E2E. For 3B.3 (JSON parser migration), add unit tests for crafted JSON inputs (nested quotes, escaped chars, truncated payloads). For 3A.1/3A.2, add a unit test that sends duplicate high-index fragments and mixed-type same-counter fragments.
**Commit:** `fix(robustness): medium-severity input validation, parser, and DoS fixes (MD-1..MD-24)`
---
## Phase 4 — Low-severity hardening and documentation
**Goal:** Clean up latent bugs, fix doc mismatches, add missing hardening. These are non-urgent but improve code health.
| # | Bug | File(s) | Fix |
|---|-----|---------|-----|
| 4.1 | LO-2: `Stop()` TOCTOU | `WSServer.cpp:104-134` | Replace `Sleep(200ms)` with thread join. |
| 4.2 | LO-3: Spinlock priority inversion | `UDPSourceSession.h`; `WSServer.h` | Document that `FastPollingMutexSem` is only for very short critical sections on same-core RT configs. Consider `MutexSem` for non-RT-contended paths. |
| 4.3 | LO-4: `SignalBuffer` mod-0 | `SignalBuffer.h:36-41` | Guard `push`/`readLast`/`readRange` against `capacity == 0`. |
| 4.4 | LO-5: Misleading "Thread-safe" comment | `SignalBuffer.h:18` | Remove the claim or add internal locking. |
| 4.5 | LO-6: GAM type validation + doc | `SineArrayGAM.cpp:81`; `TimeArrayGAM.cpp:54`; `TimeArrayGAM.h:8,27` | Add `GetSignalType` checks; update `TimeArrayGAM.h` doc from `uint32` to `uint64`. |
| 4.6 | LO-12: Directory listing | `Client/webui/main.go:26` | Disable directory listings (return 404 for directories). |
| 4.7 | LO-13: No security headers | `Client/debugger/main.go:55`; `Client/udpstreamer/main.go`; `Client/webui/main.go` | Add a middleware that sets `X-Content-Type-Options: nosniff`, `X-Frame-Options: DENY`, `Content-Security-Policy: default-src 'self'`. |
| 4.8 | LO-15: `host_`/`port_` race | `WSClient.cpp:48-58` | Protect with `sendMutex_` or make `atomic<uint16_t>` + `std::string` guarded by a small mutex. |
| 4.9 | LO-16: `ReadExactTCP` edge case | `UDPSClient.cpp:474-487` | Add a max-iterations guard. |
| 4.10 | LO-18: `RangeMin < RangeMax` validation | `UDPStreamer.cpp:403-404` | Validate when `quantType != None`; `REPORT_ERROR` if `rangeMax <= rangeMin`. |
**Est. effort:** 2 h
**Commit:** `fix(hardening): low-severity fixes, doc corrections, security headers (LO-1..LO-19)`
---
## Phase 5 — Cross-cutting refactors (optional, post-hardening)
These are not bug fixes but structural improvements that prevent the recurrence of the bug classes found in this audit.
| # | Refactor | Rationale | Est. effort |
|---|----------|-----------|-------------|
| 5.1 | Shared `validateBounds` / `validateCount` helpers | Centralizes the integer-overflow-prevention pattern; prevents future copy-paste bugs | 1 h |
| 5.2 | Real JSON parser in C++ clients (nlohmann/json or Qt's QJsonDocument) | Eliminates the entire class of `strstr`/`snprintf` JSON bugs (MD-16, MD-17, MD-18) | 4 h |
| 5.3 | `poll()`/`epoll` everywhere (replace all `select`+`FD_SET`) | Eliminates the `FD_SETSIZE` limitation entirely (HI-6) | 2 h |
| 5.4 | Auth framework for DebugService + web UIs | Token-based auth shared between the Go web UIs and the C++ DebugService; eliminates the "no auth anywhere" theme | 1 d |
| 5.5 | Fuzzing harness for UDPS protocol parsers | `libFuzzer` or `go-fuzz` harnesses that feed random bytes to `ParseConfig`/`ParseData`/`DecodeElems`/`ParseBinaryFrame`; catches future overflow variants | 1 d |
| 5.6 | Thread-sanitizer and address-sanitizer CI runs | `make CXXFLAGS="-fsanitize=address,undefined"`; `go test -race`; catches UAF and races automatically | 4 h |
---
## Verification checklist (run after each phase)
```bash
source env.sh
# C++ build + tests
make -f Makefile.gcc clean
make -f Makefile.gcc core apps test
./Build/x86-linux/GTest/MainGTest.ex
./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex
# Go tests (each module)
cd Common/Client/go && go vet ./... && go test ./... && cd -
cd Client/debugger && go vet ./... && go build ./... && cd -
cd Client/udpstreamer && go vet ./... && go build ./... && cd -
cd Test/E2E/chain/client && go vet ./... && go test ./... && cd -
# Python framework tests
cd Test/E2E/chain && python3 -m unittest tests_py && cd -
# Full E2E suite
./Test/E2E/chain/run_chain_e2e.sh --skip-build
# ASan/UBSan smoke test (after Phase 2+)
make -f Makefile.gcc clean
make -f Makefile.gcc CXXFLAGS="-fsanitize=address,undefined -g" core apps
./Build/x86-linux/GTest/MainGTest.ex
```
---
## Timeline summary
| Phase | Scope | Est. effort | Risk reduction |
|-------|-------|-------------|----------------|
| 1 | Critical remote-exploitable (CR-1..CR-5) | ~4 h | Eliminates drive-by takeover + heap corruption |
| 2 | High crash/OOB/UAF (HI-1..HI-9) | ~7 h | Eliminates remote crash + memory corruption |
| 3 | Medium robustness/DoS/parser (MD-1..MD-24) | ~11 h | Hardens input validation + RFC compliance |
| 4 | Low hardening/doc (LO-1..LO-19) | ~2 h | Code health + defense in depth |
| 5 | Cross-cutting refactors (optional) | ~3 d | Prevents recurrence of bug classes |
**Total (Phases 1-4):** ~24 h of focused work. Phase 5 is optional and can be scheduled separately.
BUG_REPORT.md
+1011
View File
File diff suppressed because it is too large Load Diff
CLAUDE.md
+37 -3
View File
@@ -30,6 +30,9 @@ make -C Source/Components/DataSources/UDPStreamer -f Makefile.gcc
cd Common/Client/go && go build ./...
cd Client/debugger && go build ./...
# Standalone C UDPS client library (no MARTe2, libc + BSD sockets only)
cd Common/Client/c && make && make cxxcheck
# ImGui desktop client (not a MARTe2 component; needs SDL2)
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
@@ -37,9 +40,40 @@ cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --buil
cd Client/streamhub-qt && cmake -B build && cmake --build build
```
End-to-end demo scripts (build + launch full stack, see headers for ports/options): `./run_combined_test.sh`, `./run_streamhub.sh`.
End-to-end demo script (build + launch full stack, see header for ports/options): `./run_streamhub.sh`.
**Streaming-chain E2E suite** (`Test/E2E/chain/`): `./run_chain_e2e.sh [--skip-build] [--only <id>] [--cpp-coverage]` drives the full chain per scenario (`scenarios.py`) — generates typed/shaped input + both cfgs, runs MARTe2+StreamHub, records via the Go `chain-client` (live/zoom/window/trigger), and validates the recorded waveform against an analytic/fed oracle (`validate_waveform.py`: fidelity gates correctness, sine shape-fit is a gross-sanity gate + tracked metric pending Phase-A timestamp calibration). It then runs the unit suites + coverage (`collect.py`: C++ GTest, Go, Python; `--cpp-coverage` does an instrumented `--coverage` rebuild, captures with lcov restricted to `Source/*`+`Test/*`, then restores the clean build), consolidates everything into `report_data.json` with per-field progression/regression vs the previous run and trend plots (`report_build.py`, history in `Build/x86-linux/E2E/chain/history.jsonl`), and compiles a Typst PDF (`E2E_Report.typ`). Python framework unit tests: `python3 -m unittest tests_py` (in `Test/E2E/chain/`).
**Streaming-chain E2E suite** (`Test/E2E/suite/`):
```bash
./Test/E2E/suite/run_e2e.sh [flags]
```
Flags:
| Flag | Effect |
|---|---|
| `--skip-build` | Skip C++ component rebuild |
| `--only <id>` | Run a single scenario by ID |
| `--pdf-only` | Just compile the Typst PDF report |
| `--cpp-coverage` | Instrumented gcov rebuild + lcov capture (on by default) |
| `--skip-coverage` | Disable the coverage pass |
| `--skip-stress` | Skip the stress matrix |
| `--skip-datasources` | Skip `direct` scenarios |
| `--skip-recorder` | Skip `recorder` scenarios |
| `--skip-debug` | Skip `debug` and `debug_pause_resume` scenarios |
| `--skip-tcplogger` | Skip `tcplogger` scenarios |
Scenario kinds (defined in `scenarios.py`):
- **chain** — full streaming pipeline: MARTe2 → UDPStreamer → StreamHub → Go `chain-client` (live/zoom/window/trigger). Validates recorded waveform against analytic/fed oracle (`validate_waveform.py`: fidelity gates correctness, sine shape-fit is a gross-sanity gate + tracked metric).
- **direct** — MARTe2 FileReader → FileWriter round-trip, validates binary output.
- **recorder** — MARTe2 → StreamHub with history recorder, validates recorded `.bin` file.
- **debug / debug_pause_resume** — DebugService scenarios via the Go `debugclient`.
- **tcplogger** — TcpLogger scenarios via the Go `debugclient`.
After scenarios, the suite runs unit tests + coverage (`collect.py`: C++ GTest, Go, Python; coverage uses lcov restricted to `Source/*` — the `Test/` harness is excluded), consolidates everything into `report_data.json` with per-field progression/regression vs the previous run and trend plots (`report_build.py`, history in `Build/x86-linux/E2E/chain/history.jsonl`), and compiles a Typst PDF (`E2E_Report.typ`). Artifacts go to `Build/x86-linux/E2E/chain/` (report, logs, PDF) and `/tmp/chain_e2e/` (scratch). Results are aggregated into `results.json` with XFAIL/XPASS handling for known issues.
Python framework unit tests: `python3 -m unittest tests_py` (in `Test/E2E/suite/`).
Build output goes to `Build/x86-linux/` (shared libs per component, `.ex` executables).
@@ -50,7 +84,7 @@ Two independent data paths:
1. **Streaming path**: `UDPStreamer` DataSource serialises signals each RT cycle to UDPS binary packets (UDP 44500, unicast/multicast) → `StreamHub` (`Source/Applications/StreamHub/`, headless C++ app: ring buffers, LTTB decimation, trigger FSM) → WebSocket 8090 → browser (`Client/udpstreamer`, Go), native ImGui client (`Client/streamhub`), or native Qt client (`Client/streamhub-qt`).
2. **Debug path**: `DebugService` patches the `ClassRegistryDatabase` at `Initialise()` so subsequent `ConfigureApplication()` instantiates `DebugBrokerWrapper<T>` around all `MemoryMap*Broker` types — no application changes. RT hot path goes through `DebugServiceI` (abstract singleton in `DebugServiceI.h`) for forcing/tracing/breakpoints. Exposes TCP 8080 (text commands), UDP 8081 (trace telemetry), works with `TcpLogger` on 8082. Web UI: `Client/debugger` (Go).
**Shared wire format**: `Common/UDP/UDPSProtocol.h` defines the UDPS binary protocol (17-byte packed header, 136-byte signal descriptors, little-endian). It is deliberately MARTe2-free so it's shared by C++ producers (`UDPStreamer`, `DebugService`), the C++ consumer (`Source/Components/Interfaces/UDPStream/UDPSClient`), and the Go decoder (`Common/Client/go/udpsprotocol`). Changes to the protocol must be mirrored across all of these, plus the JS client parsers.
**Shared wire format**: `Common/UDP/UDPSProtocol.h` defines the UDPS binary protocol (17-byte packed header, 136-byte signal descriptors, little-endian). It is deliberately MARTe2-free so it's shared by C++ producers (`UDPStreamer`, `DebugService`), the C++ consumer (`Source/Components/Interfaces/UDPStream/UDPSClient`), the Go decoder (`Common/Client/go/udpsprotocol`), and the standalone C client (`Common/Client/c`, which redeclares the constants rather than including this header, so it stays MARTe-free). Changes to the protocol must be mirrored across all of these, plus the JS client parsers.
**StreamHub WebSocket protocol**: JSON text frames for commands/events, binary frames for data pushes — spec in `ARCHITECTURE.md` §6. The Go hub (`Client/udpstreamer`) and C++ StreamHub implement the identical protocol; both clients (browser JS and ImGui) must stay compatible with both.
Client/debugger/controller/cmd_gate_test.go
+54
View File
@@ -0,0 +1,54 @@
package controller
import (
"testing"
)
// TestIsDangerousCommand_Force — FORCE is dangerous.
func TestIsDangerousCommand_Force(t *testing.T) {
if !isDangerousCommand("FORCE signal 1.0") {
t.Error("FORCE should be dangerous")
}
}
// TestIsDangerousCommand_Pause — PAUSE is dangerous.
func TestIsDangerousCommand_Pause(t *testing.T) {
if !isDangerousCommand("PAUSE") {
t.Error("PAUSE should be dangerous")
}
}
// TestIsDangerousCommand_Msg — MSG is dangerous.
func TestIsDangerousCommand_Msg(t *testing.T) {
if !isDangerousCommand("MSG target func") {
t.Error("MSG should be dangerous")
}
}
// TestIsDangerousCommand_CaseInsensitive — case-insensitive.
func TestIsDangerousCommand_CaseInsensitive(t *testing.T) {
if !isDangerousCommand("force signal 1.0") {
t.Error("lowercase force should be dangerous")
}
}
// TestIsDangerousCommand_SafeCommand — DISCOVER is not dangerous.
func TestIsDangerousCommand_SafeCommand(t *testing.T) {
if isDangerousCommand("DISCOVER") {
t.Error("DISCOVER should not be dangerous")
}
}
// TestIsDangerousCommand_TraceNotDangerous — TRACE is not dangerous (read-only).
func TestIsDangerousCommand_TraceNotDangerous(t *testing.T) {
if isDangerousCommand("TRACE signal 1") {
t.Error("TRACE should not be dangerous")
}
}
// TestIsDangerousCommand_Empty — empty command is not dangerous.
func TestIsDangerousCommand_Empty(t *testing.T) {
if isDangerousCommand("") {
t.Error("empty command should not be dangerous")
}
}
Client/debugger/martecontrol.go → Client/debugger/controller/martecontrol.go
+112 -29
View File
@@ -1,4 +1,9 @@
package main
// Package controller implements MarteController, the shared TCP/UDP client
// logic that drives a running MARTe2 DebugService+TCPLogger instance. It is
// consumed both by the Client/debugger browser-facing WebSocket server
// (package main, via NewMarteController) and headlessly by the
// Test/E2E/suite/debugclient E2E test tool (via NewHeadlessMarteController).
package controller
import (
"bufio"
@@ -17,6 +22,40 @@ import (
"marte2/common/wshub"
)
// ---------------------------------------------------------------------------
// Command safety gate (CR-4)
// ---------------------------------------------------------------------------
// DangerousCommandsEnabled gates commands that mutate the RT application state
// (FORCE, PAUSE, RESUME, STEP, BREAK, MSG). Set via --enable-dangerous-commands.
var DangerousCommandsEnabled = false
// dangerousCommands is the set of MARTe2 commands that can change signal values
// or alter execution flow. Without --enable-dangerous-commands these are blocked
// from the browser WebSocket path.
var dangerousCommands = map[string]bool{
"FORCE": true,
"UNFORCE": true,
"PAUSE": true,
"RESUME": true,
"STEP": true,
"BREAK": true,
"UNBREAK": true,
"MSG": true,
"LOAD": true,
"UNLOAD": true,
}
// isDangerousCommand returns true if the command's first word is in the
// dangerous set (case-insensitive).
func isDangerousCommand(cmd string) bool {
parts := strings.Fields(cmd)
if len(parts) == 0 {
return false
}
return dangerousCommands[strings.ToUpper(parts[0])]
}
// ---------------------------------------------------------------------------
// Signal metadata (populated by DISCOVER)
// ---------------------------------------------------------------------------
@@ -47,7 +86,8 @@ func broadcastHub(hub *wshub.Hub, v any) {
// ---------------------------------------------------------------------------
type MarteController struct {
hub *wshub.Hub
hub *wshub.Hub
sink func(v any)
mu sync.Mutex
tcpConn net.Conn
@@ -101,6 +141,7 @@ func NewMarteController(hub *wshub.Hub) *MarteController {
forcedState: make(map[string]string),
stopCh: make(chan struct{}),
}
mc.sink = func(v any) { broadcastHub(mc.hub, v) }
// Register the new-client hook so connection + forced/traced state is
// replayed to any browser that connects (or reconnects) while the server
// already holds a live MARTe2 TCP session.
@@ -108,6 +149,20 @@ func NewMarteController(hub *wshub.Hub) *MarteController {
return mc
}
// NewHeadlessMarteController creates a MarteController with no WebSocket hub,
// routing all events through sink instead (used by the debugclient E2E test tool).
func NewHeadlessMarteController(sink func(v any)) *MarteController {
mc := &MarteController{
hub: nil,
signals: make(map[uint32]*SignalMeta),
tracedNames: make(map[string]bool),
forcedState: make(map[string]string),
stopCh: make(chan struct{}),
}
mc.sink = sink
return mc
}
func (m *MarteController) IsConnected() bool {
return atomic.LoadInt32(&m.connected) == 1
}
@@ -166,10 +221,13 @@ func (m *MarteController) Connect(host string, cmdPort, udpPort, logPort int) {
m.stopCh = make(chan struct{})
m.mu.Unlock()
// Update source state so the browser shows "connecting".
m.hub.SetSourceState("debug", "connecting")
// Update source state so the browser shows "connecting". No-op headless
// (m.hub == nil for NewHeadlessMarteController instances).
if m.hub != nil {
m.hub.SetSourceState("debug", "connecting")
}
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "log", "time": time.Now().Format("15:04:05.000"),
"level": "INFO", "message": fmt.Sprintf("Connecting to %s cmd=%d udp=%d log=%d", host, cmdPort, udpPort, logPort),
})
@@ -198,7 +256,9 @@ func (m *MarteController) Disconnect() {
m.baseTsSet = false
m.basesMu.Unlock()
m.discoverAcc = nil
m.hub.SetSourceState("debug", "disconnected")
if m.hub != nil {
m.hub.SetSourceState("debug", "disconnected")
}
}
func (m *MarteController) stopped() bool {
@@ -256,10 +316,25 @@ func (m *MarteController) HandleBrowserCommand(msg []byte) {
return
}
cmd, _ := data["cmd"].(string)
if cmd != "" {
m.trackForcedCmd(cmd)
m.SendCommand(cmd)
if cmd == "" {
return
}
// Gate dangerous commands (FORCE/UNFORCE/PAUSE/RESUME/STEP/BREAK/MSG)
// behind an explicit opt-in flag. Without it, only read-only commands
// (DISCOVER, TREE, INFO, LS, VALUE, TRACE, UNTRACE, STEP_STATUS) are
// forwarded to the MARTe2 TCP control connection.
if isDangerousCommand(cmd) {
if !DangerousCommandsEnabled {
m.sink(map[string]any{
"type": "log", "time": time.Now().Format("15:04:05.000"),
"level": "WARNING",
"message": fmt.Sprintf("Blocked dangerous command (requires --enable-dangerous-commands): %s", cmd),
})
return
}
}
m.trackForcedCmd(cmd)
m.SendCommand(cmd)
}
}
@@ -272,7 +347,7 @@ func (m *MarteController) runTCP(host string, port int) {
for !m.stopped() {
conn, err := net.DialTimeout("tcp", addr, 5*time.Second)
if err != nil {
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "log", "time": time.Now().Format("15:04:05.000"),
"level": "WARNING", "message": fmt.Sprintf("TCP %s: %v — retrying…", addr, err),
})
@@ -287,7 +362,7 @@ func (m *MarteController) runTCP(host string, port int) {
m.mu.Unlock()
atomic.StoreInt32(&m.connected, 1)
broadcastHub(m.hub, map[string]any{"type": "connected"})
m.sink(map[string]any{"type": "connected"})
// Send SERVICE_INFO to auto-discover ports
m.writeCmd("SERVICE_INFO")
@@ -297,7 +372,7 @@ func (m *MarteController) runTCP(host string, port int) {
m.readLoop(conn)
atomic.StoreInt32(&m.connected, 0)
broadcastHub(m.hub, map[string]any{"type": "disconnected"})
m.sink(map[string]any{"type": "disconnected"})
m.mu.Lock()
m.tcpConn = nil
@@ -323,7 +398,7 @@ func (m *MarteController) writeCmd(cmd string) {
silent := cmd == "STEP_STATUS" || cmd == "INFO"
if !silent {
log.Printf("[→MARTe] %s", cmd)
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "log", "time": time.Now().Format("15:04:05.000"),
"level": "CMD", "message": fmt.Sprintf("→ %s", cmd),
})
@@ -465,7 +540,7 @@ func (m *MarteController) handleJSONResponse(tag, data string) {
silent := tag == "STEP_STATUS" || tag == "INFO"
if !silent {
log.Printf("[←MARTe] %s %d bytes", tag, len(data))
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "log", "time": time.Now().Format("15:04:05.000"),
"level": "RESP", "message": fmt.Sprintf("← %s (%d B)", tag, len(data)),
})
@@ -500,25 +575,27 @@ func (m *MarteController) handleJSONResponse(tag, data string) {
raw := m.rawSigs
m.rawSigsMu.RUnlock()
if len(raw) > 0 {
m.hub.UpdateConfigForSource("debug", m.translateSignalNames(raw))
if m.hub != nil {
m.hub.UpdateConfigForSource("debug", m.translateSignalNames(raw))
}
} else {
m.synthesizeHubConfig(all)
}
// Re-marshal the merged list so the browser gets a single consistent blob.
merged, _ := json.Marshal(discoverResp{Signals: all})
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "response", "tag": "DISCOVER", "data": string(merged),
})
return
case "TREE":
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "tree_node",
"data": data,
})
return
}
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "response",
"tag": tag,
"data": data,
@@ -537,13 +614,13 @@ func (m *MarteController) handleTextLine(line string) {
fmt.Sscanf(p[8:], "%d", &newLog)
}
}
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "response",
"tag": "SERVICE_INFO",
"data": line[len("OK SERVICE_INFO "):],
})
if newUDP > 0 || newLog > 0 {
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "service_config",
"udp_port": newUDP,
"log_port": newLog,
@@ -567,7 +644,7 @@ func (m *MarteController) handleTextLine(line string) {
}
}
}
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "text_line",
"data": line,
})
@@ -774,7 +851,9 @@ func (m *MarteController) synthesizeHubConfig(sigs []discoverSignalJSON) {
// buffer and limiting live streaming to the fraction of a second that
// accumulated before the DISCOVER response arrived.
translated := m.translateSignalNames(sigInfos)
m.hub.UpdateConfigForSource("debug", translated)
if m.hub != nil {
m.hub.UpdateConfigForSource("debug", translated)
}
}
// ---------------------------------------------------------------------------
@@ -801,7 +880,7 @@ func (m *MarteController) runDebugUDP(host string, port int) {
if err != nil {
msg := fmt.Sprintf("UDP bind on %s failed: %v — rebuild DebugService C++ and restart", addr, err)
log.Printf("[debug-udp] %s", msg)
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "log", "time": time.Now().Format("15:04:05.000"),
"level": "ERROR", "message": msg,
})
@@ -812,7 +891,7 @@ func (m *MarteController) runDebugUDP(host string, port int) {
conn.SetReadBuffer(10 * 1024 * 1024)
log.Printf("[debug-udp] listening on %s for UDPS packets", addr)
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "log", "time": time.Now().Format("15:04:05.000"),
"level": "INFO", "message": fmt.Sprintf("UDP listener bound on %s", addr),
})
@@ -865,8 +944,10 @@ func (m *MarteController) runDebugUDP(host string, port int) {
sigs = m.translateSignalNames(sigs)
currentSigs = sigs
currentPublishMode = pm
m.hub.UpdateConfigForSource("debug", sigs)
m.hub.SetSourceState("debug", "connected")
if m.hub != nil {
m.hub.UpdateConfigForSource("debug", sigs)
m.hub.SetSourceState("debug", "connected")
}
case udpsprotocol.PktData:
if len(currentSigs) == 0 {
@@ -888,8 +969,10 @@ func (m *MarteController) runDebugUDP(host string, port int) {
log.Printf("[debug-udp] parse data: %v", err)
continue
}
for _, s := range samples {
m.hub.PushDataForSource("debug", s)
if m.hub != nil {
for _, s := range samples {
m.hub.PushDataForSource("debug", s)
}
}
}
}
@@ -923,7 +1006,7 @@ func (m *MarteController) runLog(host string, port int) {
}
level := rest[:idx]
msg := rest[idx+1:]
broadcastHub(m.hub, map[string]any{
m.sink(map[string]any{
"type": "log",
"time": time.Now().Format("15:04:05.000"),
"level": level,
Client/debugger/main.go
+5 -1
View File
@@ -10,6 +10,8 @@ import (
"net/http"
"os"
"marte2debugger/controller"
"marte2/common/wshub"
)
@@ -21,13 +23,15 @@ var staticFiles embed.FS
func main() {
addr := flag.String("addr", ":7777", "HTTP listen address")
sourcesFile := flag.String("sources-file", "", "JSON file for persistent source list")
flag.BoolVar(&controller.DangerousCommandsEnabled, "enable-dangerous-commands", false,
"Allow FORCE/PAUSE/RESUME/STEP/BREAK/MSG commands from the browser (CR-4 safety gate)")
flag.Parse()
hub := wshub.NewHub()
sm := wshub.NewSourceManager(hub, *sourcesFile)
hub.SetSourceManager(sm)
ctrl := NewMarteController(hub)
ctrl := controller.NewMarteController(hub)
go hub.Run()
Client/debugger/static/app.js
+1 -1
View File
@@ -3511,7 +3511,7 @@ function _fmtHz(v) { return v != null && isFinite(v) && v > 0 ? v.toFixed(2) + '
function _fmtKB(v) { return v != null && isFinite(v) ? (v / 1024).toFixed(2) + ' KB' : '—'; }
function _statsKV(label, value, cls) {
return `<div class="stats-kv"><span class="stats-k">${label}</span><span class="stats-v${cls ? ' ' + cls : ''}">${value}</span></div>`;
return `<div class="stats-kv"><span class="stats-k">${escHtml(label)}</span><span class="stats-v${cls ? ' ' + cls : ''}">${escHtml(value)}</span></div>`;
}
function _histHTML(si) {
Client/streamhub-qt/Hub.cpp
+9 -1
View File
@@ -160,7 +160,15 @@ void Hub::onTriggerState(const std::string& json) {
trigger_.trigTime = msg.trigTime;
trigger_.hasTrigTime = true;
}
if (msg.state == "idle") { trigger_.hasTrigTime = false; }
if (msg.hasWindow) {
trigger_.firedPreS = msg.preSec;
trigger_.firedPostS = msg.postSec;
trigger_.hasFiredWin = true;
}
if (msg.state == "idle") {
trigger_.hasTrigTime = false;
trigger_.hasFiredWin = false;
}
Q_EMIT triggerStateChanged();
}
Client/streamhub-qt/Model.h
+5
View File
@@ -60,6 +60,11 @@ struct TriggerCfgState {
bool stopped = false;
bool hasTrigTime = false;
double trigTime = 0.0;
/* Window the hub latched at fire time. Not the same as windowSec/prePercent
* above, which are editable and may have moved on since the trigger fired. */
bool hasFiredWin = false;
double firedPreS = 0.0;
double firedPostS = 0.0;
};
/** Per-signal vertical scale state (oscilloscope style). */
Client/streamhub-qt/PlotWidget.cpp
+233 -77
View File
@@ -78,6 +78,49 @@ static double normalizeY(double raw, const VScale& vs) {
return (raw - vs.resolvedOffset) / vs.resolvedDiv + vs.screenPos;
}
/* Resolve the one scale every trace shares in unified mode: same rules as the
* per-signal version applied to the union of the plot — range takes the union
* of the declared ranges, auto fits the union of the data. */
static void resolveUnifiedVScale(VScale& vs,
const std::vector<PlotAssignment>& slots,
const std::vector<Source>& sources,
const std::vector<std::vector<double> >& vStore) {
if (vs.mode == 2) {
vs.resolvedDiv = std::max(vs.divValue, 1e-30);
vs.resolvedOffset = vs.offset;
return;
}
double mn = 1e300, mx = -1e300;
if (vs.mode == 1) {
for (const auto& a : slots) {
if (a.sourceIdx < 0 || a.sourceIdx >= (int)sources.size()) continue;
if (a.signalIdx < 0 ||
a.signalIdx >= (int)sources[a.sourceIdx].signals.size()) continue;
const auto& m = sources[a.sourceIdx].signals[a.signalIdx].meta;
if (!(m.rangeMax > m.rangeMin)) continue;
if (m.rangeMin < mn) mn = m.rangeMin;
if (m.rangeMax > mx) mx = m.rangeMax;
}
if (mx > mn) {
vs.resolvedDiv = std::max((mx - mn) / 8.0, 1e-30);
vs.resolvedOffset = (mn + mx) / 2.0;
return;
}
mn = 1e300; mx = -1e300; /* no usable range: fall through to auto */
}
for (const auto& vv : vStore) {
for (double v : vv) {
if (!std::isfinite(v)) continue;
if (v < mn) mn = v;
if (v > mx) mx = v;
}
}
if (!std::isfinite(mn) || mn > mx) { mn = -1.0; mx = 1.0; }
if (mn == mx) { mn -= 1.0; mx += 1.0; }
vs.resolvedDiv = std::max((mx - mn) / 6.0, 1e-30);
vs.resolvedOffset = (mx + mn) / 2.0;
}
static bool dataMinMax(const std::vector<double>& v, double& mn, double& mx) {
mn = 1e300; mx = -1e300;
for (double x : v) { if (std::isfinite(x)) { if (x < mn) mn = x; if (x > mx) mx = x; } }
@@ -189,6 +232,50 @@ void PlotCanvas::drawMarker(QPainter& p, double cx, double cy, int marker, doubl
}
}
/** @brief What the plot renders on the trigger-relative axis, if anything. */
struct TrigView {
bool rel = false; /* render against t - trig instead of wall clock */
bool fromCap = false; /* data comes from the capture frame, not the ring */
double trigT = 0.0;
double preS = 0.0;
double postS = 0.0;
};
/* Two ways to end up in trigger-relative time. Either a v2 capture frame has
* arrived, or a trigger has fired and its window is still filling. In the
* second case the hub sends nothing until the whole window has been produced —
* several seconds for a long window at a high rate — so the trace is drawn from
* the local rings onto the final axis, growing left to right. Filling wins
* over the last capture: once a new trigger fires the old waveform is history.
* A capture latches its own pre/post at fire time, so later edits in the
* trigger bar must not move the axis of a finished capture. */
static TrigView resolveTrigView(Hub* hub, const GlobalView* gv, bool paused) {
TrigView tv;
if (!gv->trigView) { return tv; }
const TriggerCfgState& t = hub->trigger();
if (!paused && t.status == "collecting" && t.hasTrigTime) {
tv.rel = true;
tv.trigT = t.trigTime;
/* Prefer the window the hub latched at fire time; the local config is
* only a fallback for hubs that do not report it, and may have been
* edited since the trigger fired. */
tv.preS = t.hasFiredWin ? t.firedPreS
: t.windowSec * t.prePercent * 0.01;
tv.postS = t.hasFiredWin ? t.firedPostS : t.windowSec - tv.preS;
return tv;
}
const CaptureFrame* cap = hub->capture();
if (cap != nullptr) {
tv.rel = true;
tv.fromCap = true;
tv.trigT = cap->trigTime;
tv.preS = cap->preSec;
tv.postS = cap->postSec;
}
return tv;
}
void PlotCanvas::paintEvent(QPaintEvent*) {
QPainter p(this);
p.setRenderHint(QPainter::Antialiasing, true);
@@ -205,13 +292,14 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
p.fillRect(rect(), col::base());
p.fillRect(r, col::crust());
const CaptureFrame* cap = hub->capture();
const bool trigView = (cap != nullptr) && gv->trigView;
auto& zc = hub->zoomCache(w_->plotIdx_);
auto& hc = hub->histZoomCache(w_->plotIdx_);
const bool paused = w_->paused_;
bool& live = w_->live_;
const TrigView tv = resolveTrigView(hub, gv, paused);
const CaptureFrame* cap = hub->capture();
/* ── pause snapshot ─────────────────────────────────────────────────── */
auto& snap = w_->snap_;
if (paused) {
@@ -239,15 +327,15 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
/* ── gather data per slot ───────────────────────────────────────────── */
std::vector<std::vector<double>> tStore(slots.size()), vStore(slots.size());
const bool liveHiRes = !trigView && live && !paused &&
const bool liveHiRes = !tv.rel && live && !paused &&
gv->windowSec <= kLiveHiResMaxWin && zc.valid &&
(zc.t1 - zc.t0) >= gv->windowSec * 0.9 && (wallNow - zc.t1) < 3.0;
const bool useZoomData = !trigView && !paused && zc.valid &&
const bool useZoomData = !tv.rel && !paused && zc.valid &&
(liveHiRes ||
(!live && zc.t0 <= w_->plotXMin_ + 1e-9 && zc.t1 >= w_->plotXMax_ - 1e-9));
bool useHistData = !trigView && !paused && !live && hc.valid &&
bool useHistData = !tv.rel && !paused && !live && hc.valid &&
hc.t0 <= w_->plotXMin_ + 1e-9 && hc.t1 >= w_->plotXMax_ - 1e-9;
if (useHistData) {
bool any = false;
@@ -271,17 +359,25 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
const auto& sig = sources[a.sourceIdx].signals[a.signalIdx];
const std::string key = hub->slotKey(a);
if (trigView) {
if (tv.fromCap) {
for (const auto& cs : cap->signals) {
if (cs.key != key) continue;
size_t n = std::min(cs.t.size(), cs.v.size());
tStore[si].reserve(n); vStore[si].reserve(n);
for (size_t i = 0; i < n; i++) {
tStore[si].push_back(cs.t[i] - cap->trigTime);
tStore[si].push_back(cs.t[i] - tv.trigT);
vStore[si].push_back(cs.v[i]);
}
break;
}
} else if (tv.rel) {
/* Filling: local ring, clipped to the (absolute) trigger window and
* shifted onto the trigger-relative axis. */
sig.buf.readRange(tv.trigT - tv.preS, tv.trigT + tv.postS,
tStore[si], vStore[si]);
for (size_t i = 0; i < tStore[si].size(); i++) {
tStore[si][i] -= tv.trigT;
}
} else if (useZoomData) {
bool found = false;
for (const auto& zs : zc.pts) {
@@ -302,11 +398,18 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
resolveVScale(a, sig, vStore[si]);
}
if (w_->vMode_ == 3) {
resolveUnifiedVScale(w_->uniVS_, slots, sources, vStore);
}
/* ── X range ────────────────────────────────────────────────────────── */
double xMin, xMax;
if (trigView) {
if (tv.rel) {
if (w_->trigZoomed_) { xMin = w_->plotXMin_; xMax = w_->plotXMax_; }
else { xMin = -cap->preSec; xMax = cap->postSec; }
/* Full window from the start, even while filling: a trace growing into
* a fixed axis reads as progress, whereas an axis that grows with the
* data shifts the whole trace every frame. */
else { xMin = -tv.preS; xMax = tv.postS; }
} else if (live && !paused) {
if (liveHiRes) { xMax = zc.t1; xMin = zc.t1 - gv->windowSec; }
else { xMax = wallNow; xMin = wallNow - gv->windowSec; }
@@ -319,19 +422,25 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
/* ── grid + ticks ───────────────────────────────────────────────────── */
p.setPen(QPen(QColor(0x31,0x32,0x44,160), 1.0));
/* Y grid: 9 division lines */
const auto& av = (w_->vMode_ == 0 && w_->activeSlot_ >= 0 &&
w_->activeSlot_ < (int)slots.size())
? slots[w_->activeSlot_].vs : VScale();
/* Which scale labels the axis: the active signal's in normal mode, the one
* the whole plot shares in unified mode (where nothing has to be selected).
* Banded modes have no single scale, so they keep the plain division numbers. */
const VScale* axisVS = nullptr;
if (w_->vMode_ == 0 && w_->activeSlot_ >= 0 &&
w_->activeSlot_ < (int)slots.size()) {
axisVS = &slots[w_->activeSlot_].vs;
} else if (w_->vMode_ == 3) {
axisVS = &w_->uniVS_;
}
p.setFont(QFont(font().family(), 8));
for (int d = -4; d <= 4; d++) {
double y = yToPx(d, r);
p.setPen(QPen(QColor(0x31,0x32,0x44, d==0?220:120), d==0?1.2:1.0));
p.drawLine(QPointF(r.left(), y), QPointF(r.right(), y));
QString lbl;
if (w_->vMode_ == 0 && w_->activeSlot_ >= 0 &&
w_->activeSlot_ < (int)slots.size()) {
double rawVal = av.resolvedOffset + (d - av.screenPos) * av.resolvedDiv;
lbl = fmtVal(rawVal);
if (axisVS != nullptr) {
lbl = fmtVal(axisVS->resolvedOffset +
(d - axisVS->screenPos) * axisVS->resolvedDiv);
} else {
lbl = QString::number(d);
}
@@ -346,7 +455,7 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
p.setPen(QPen(QColor(0x31,0x32,0x44,120), 1.0));
p.drawLine(QPointF(x, r.top()), QPointF(x, r.bottom()));
p.setPen(QColor(0xa6,0xad,0xc8));
QString xl = trigView ? fmtVal(xv) + "s" : QString::number(xv, 'f', 3);
QString xl = tv.rel ? fmtVal(xv) + "s" : QString::number(xv, 'f', 3);
int flags = (t==0?Qt::AlignLeft:(t==10?Qt::AlignRight:Qt::AlignHCenter))
| Qt::AlignTop;
p.drawText(QRectF(x-40, r.bottom()+2, 80, 14), flags, xl);
@@ -396,8 +505,10 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
if (w_->vMode_ == 1) bandNormalize(vDec, vNorm, myKi, nTraces, true);
else if (w_->vMode_ == 2) bandNormalize(vDec, vNorm, myKi, nTraces, a.vs.digitalInMixed);
else {
/* unified shares one scale, normal gives each trace its own */
const VScale& nvs = (w_->vMode_ == 3) ? w_->uniVS_ : a.vs;
vNorm.resize(nOut);
for (size_t k = 0; k < nOut; k++) vNorm[k] = normalizeY(vDec[k], a.vs);
for (size_t k = 0; k < nOut; k++) vNorm[k] = normalizeY(vDec[k], nvs);
}
QColor c = sig.color;
@@ -423,7 +534,7 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
}
/* trigger instant marker at t=0 */
if (trigView) {
if (tv.rel) {
double x = xToPx(0.0, xMin, xMax, r);
p.setPen(QPen(QColor(255,255,0,200), 1.5, Qt::DashLine));
p.drawLine(QPointF(x, r.top()), QPointF(x, r.bottom()));
@@ -476,8 +587,7 @@ void PlotCanvas::wheelEvent(QWheelEvent* e) {
Hub* hub = w_->hub_;
GlobalView* gv = w_->gv_;
auto& slots = w_->slots_;
const CaptureFrame* cap = hub->capture();
const bool trigView = (cap != nullptr) && gv->trigView;
const TrigView tv = resolveTrigView(hub, gv, w_->paused_);
bool& live = w_->live_;
double dy = e->angleDelta().y();
@@ -488,43 +598,51 @@ void PlotCanvas::wheelEvent(QWheelEvent* e) {
const double now = nowSec();
auto enterTrigZoom = [&]() {
if (trigView && !w_->trigZoomed_) {
w_->setStoredX(-cap->preSec, cap->postSec);
if (tv.rel && !w_->trigZoomed_) {
w_->setStoredX(-tv.preS, tv.postS);
w_->trigZoomed_ = true;
}
};
auto xZoomStored = [&](double f) {
if (trigView) enterTrigZoom();
if (tv.rel) enterTrigZoom();
if (now - w_->lastHistPushMs_ > 0.6) { w_->pushZoomHist(); w_->lastHistPushMs_ = now; }
double cx = (w_->plotXMin_ + w_->plotXMax_) * 0.5;
double half = (w_->plotXMax_ - w_->plotXMin_) * 0.5 * f;
w_->setStoredX(cx - half, cx + half);
};
auto makeManual = [&](PlotAssignment& a) {
if (a.vs.mode != 2) {
a.vs.divValue = std::max(a.vs.resolvedDiv, 1e-30);
a.vs.offset = a.vs.resolvedOffset;
a.vs.mode = 2;
/* Seed manual from the resolved values so the gesture sticks. */
auto makeManual = [&](VScale& vs) {
if (vs.mode != 2) {
vs.divValue = std::max(vs.resolvedDiv, 1e-30);
vs.offset = vs.resolvedOffset;
vs.mode = 2;
}
};
/* Scroll adjusts the scale the axis is labelled with: the active signal's in
* normal mode, the plot's shared one in unified mode (nothing to select). */
VScale* wheelVS = nullptr;
if (w_->vMode_ == 3) {
wheelVS = &w_->uniVS_;
} else if (w_->activeSlot_ >= 0 && w_->activeSlot_ < (int)slots.size()) {
wheelVS = &slots[w_->activeSlot_].vs;
}
if (ctrl) {
if (!trigView && live) gv->windowSec = std::clamp(gv->windowSec*factor, 1e-4, 3600.0);
if (!tv.rel && live) gv->windowSec = std::clamp(gv->windowSec*factor, 1e-4, 3600.0);
else xZoomStored(factor);
} else if (shift) {
if (w_->activeSlot_ >= 0 && w_->activeSlot_ < (int)slots.size()) {
auto& a = slots[w_->activeSlot_];
makeManual(a);
a.vs.screenPos += (dy > 0) ? 0.5 : -0.5;
if (wheelVS != nullptr) {
makeManual(*wheelVS);
wheelVS->screenPos += (dy > 0) ? 0.5 : -0.5;
}
} else {
if (w_->activeSlot_ >= 0 && w_->activeSlot_ < (int)slots.size()) {
auto& a = slots[w_->activeSlot_];
makeManual(a);
a.vs.divValue = std::max(a.vs.divValue * factor, 1e-30);
if (wheelVS != nullptr) {
makeManual(*wheelVS);
wheelVS->divValue = std::max(wheelVS->divValue * factor, 1e-30);
} else {
if (!trigView && live) gv->windowSec = std::clamp(gv->windowSec*factor, 1e-4, 3600.0);
if (!tv.rel && live) gv->windowSec = std::clamp(gv->windowSec*factor, 1e-4, 3600.0);
else xZoomStored(factor);
}
}
@@ -549,8 +667,7 @@ void PlotCanvas::mouseMoveEvent(QMouseEvent* e) {
GlobalView* gv = w_->gv_;
Hub* hub = w_->hub_;
const QRectF r = plotRect();
const CaptureFrame* cap = hub->capture();
const bool trigView = (cap != nullptr) && gv->trigView;
const TrigView tv = resolveTrigView(hub, gv, w_->paused_);
bool& live = w_->live_;
if (dragCursor_ != 0) {
@@ -560,11 +677,11 @@ void PlotCanvas::mouseMoveEvent(QMouseEvent* e) {
return;
}
if (panning_) {
if (trigView && !w_->trigZoomed_) {
w_->setStoredX(-cap->preSec, cap->postSec);
if (tv.rel && !w_->trigZoomed_) {
w_->setStoredX(-tv.preS, tv.postS);
w_->trigZoomed_ = true;
}
if (!trigView && live) { w_->initPlotX(nowSec()); live = false; }
if (!tv.rel && live) { w_->initPlotX(nowSec()); live = false; }
double dxPix = e->pos().x() - lastPos_.x();
lastPos_ = e->pos();
double xRange = w_->plotXMax_ - w_->plotXMin_;
@@ -677,11 +794,10 @@ void PlotWidget::onCaptureReceived() {
void PlotWidget::tick() {
Hub* hub = hub_;
GlobalView* gv = gv_;
const CaptureFrame* cap = hub->capture();
const bool trigView = (cap != nullptr) && gv->trigView;
const TrigView tv = resolveTrigView(hub, gv, paused_);
const double now = nowSec();
if (!trigView && !paused_) {
if (!tv.rel && !paused_) {
std::string csv;
for (const auto& a : slots_) {
std::string k = hub->slotKey(a);
@@ -736,9 +852,13 @@ void PlotWidget::rebuildHeader() {
auto* b = new QToolButton(header_);
b->setCheckable(true);
b->setChecked(activeSlot_ == i);
b->setText(QString("%1 %2/div")
.arg(QString::fromStdString(sig.meta.name))
.arg(fmtVal(a.vs.resolvedDiv)));
/* In unified mode every badge would repeat the same div value, which
* the header's Y-Scale button already shows — so show just the name. */
b->setText(vMode_ == 3
? QString::fromStdString(sig.meta.name)
: QString("%1 %2/div")
.arg(QString::fromStdString(sig.meta.name))
.arg(fmtVal(a.vs.resolvedDiv)));
QColor c = sig.color;
QString fg = (activeSlot_ == i) ? "#11111b" : "#11111b";
QColor bg = (activeSlot_ == i) ? col::blue() : c;
@@ -797,11 +917,17 @@ void PlotWidget::rebuildHeader() {
headerLay_->addWidget(fit);
}
/* N / D / M */
const char* vl[3] = {"N", "D", "M"};
for (int vm = 0; vm < 3; vm++) {
/* N / U / D / M */
const char* vl[4] = {"N", "U", "D", "M"};
const char* vtip[4] = {"Normal: one vertical scale per signal",
"Unified: one vertical scale shared by every signal",
"Digital", "Mixed"};
const int vmode[4] = {0, 3, 1, 2};
for (int i = 0; i < 4; i++) {
const int vm = vmode[i];
auto* vb = new QToolButton(header_);
vb->setText(vl[vm]);
vb->setText(vl[i]);
vb->setToolTip(vtip[i]);
vb->setCheckable(true);
vb->setChecked(vMode_ == vm);
connect(vb, &QToolButton::clicked, this, [this, vm]() {
@@ -810,9 +936,57 @@ void PlotWidget::rebuildHeader() {
headerLay_->addWidget(vb);
}
/* Unified mode's single scale belongs to the plot, not to any one signal,
* so it is edited from here rather than from a badge's context menu. */
if (vMode_ == 3) {
auto* yb = new QToolButton(header_);
yb->setText(QString("Y-Scale: %1/div").arg(fmtVal(uniVS_.resolvedDiv)));
yb->setToolTip("Vertical scale shared by every signal in this plot");
connect(yb, &QToolButton::clicked, this, [this, yb]() {
showUnifiedVScaleMenu(yb->mapToGlobal(QPoint(0, yb->height())));
});
headerLay_->addWidget(yb);
}
headerLay_->addStretch(1);
}
/** Populate @a vs with the Auto/Range/Manual entries driving @a evs. */
void PlotWidget::buildVScaleMenu(QMenu* vs, VScale& evs) {
const char* modes[] = {"Auto", "Range", "Manual"};
for (int mm = 0; mm < 3; mm++) {
QAction* act = vs->addAction(modes[mm]);
act->setCheckable(true); act->setChecked(evs.mode == mm);
connect(act, &QAction::triggered, this, [this, &evs, mm]() {
evs.mode = mm; rebuildHeader(); canvas_->update();
});
}
vs->addSeparator();
vs->addAction("Manual V/div…", [this, &evs]() {
bool ok; double v = QInputDialog::getDouble(this, "V/div", "Units per division",
evs.mode==2?evs.divValue:evs.resolvedDiv, -1e12, 1e12, 6, &ok);
if (ok) { evs.divValue = v; evs.mode = 2; rebuildHeader(); canvas_->update(); }
});
vs->addAction("Offset…", [this, &evs]() {
bool ok; double v = QInputDialog::getDouble(this, "Offset", "Center value",
evs.mode==2?evs.offset:evs.resolvedOffset, -1e12, 1e12, 6, &ok);
if (ok) { evs.offset = v; evs.mode = 2; rebuildHeader(); canvas_->update(); }
});
vs->addAction("Position (div)…", [this, &evs]() {
bool ok; double v = QInputDialog::getDouble(this, "Position", "Divisions from center",
evs.screenPos, -8, 8, 2, &ok);
if (ok) { evs.screenPos = v; canvas_->update(); }
});
}
void PlotWidget::showUnifiedVScaleMenu(const QPoint& globalPos) {
QMenu m;
m.addAction("Y-Scale — all signals")->setEnabled(false);
m.addSeparator();
buildVScaleMenu(&m, uniVS_);
m.exec(globalPos);
}
void PlotWidget::showBadgeMenu(int slotIdx, const QPoint& globalPos) {
auto& sources = hub_->sources();
if (slotIdx < 0 || slotIdx >= (int)slots_.size()) return;
@@ -846,30 +1020,12 @@ void PlotWidget::showBadgeMenu(int slotIdx, const QPoint& globalPos) {
connect(dg, &QAction::toggled, this, [&](bool on){ a.vs.digitalInMixed = on; canvas_->update(); });
}
m.addSeparator();
QMenu* vs = m.addMenu("V-scale");
const char* modes[] = {"Auto", "Range", "Manual"};
for (int mm = 0; mm < 3; mm++) {
QAction* act = vs->addAction(modes[mm]);
act->setCheckable(true); act->setChecked(a.vs.mode == mm);
connect(act, &QAction::triggered, this, [&, mm]() { a.vs.mode = mm; rebuildHeader(); canvas_->update(); });
/* In unified mode the plot has one scale for every trace, so it is edited
* from the header's Y-Scale button instead of from any one signal. */
if (vMode_ != 3) {
m.addSeparator();
buildVScaleMenu(m.addMenu("V-scale"), a.vs);
}
vs->addSeparator();
vs->addAction("Manual V/div…", [&]() {
bool ok; double v = QInputDialog::getDouble(this, "V/div", "Units per division",
a.vs.mode==2?a.vs.divValue:a.vs.resolvedDiv, -1e12, 1e12, 6, &ok);
if (ok) { a.vs.divValue = v; a.vs.mode = 2; rebuildHeader(); canvas_->update(); }
});
vs->addAction("Offset…", [&]() {
bool ok; double v = QInputDialog::getDouble(this, "Offset", "Center value",
a.vs.mode==2?a.vs.offset:a.vs.resolvedOffset, -1e12, 1e12, 6, &ok);
if (ok) { a.vs.offset = v; a.vs.mode = 2; rebuildHeader(); canvas_->update(); }
});
vs->addAction("Position (div)…", [&]() {
bool ok; double v = QInputDialog::getDouble(this, "Position", "Divisions from center",
a.vs.screenPos, -8, 8, 2, &ok);
if (ok) { a.vs.screenPos = v; canvas_->update(); }
});
m.addSeparator();
m.addAction("Remove from plot", [&]() {
Client/streamhub-qt/PlotWidget.h
+5 -1
View File
@@ -21,6 +21,7 @@
class QHBoxLayout;
class QToolButton;
class QLabel;
class QMenu;
namespace shq {
@@ -69,6 +70,8 @@ private:
friend class PlotCanvas;
void rebuildHeader();
void buildVScaleMenu(QMenu* vs, VScale& evs);
void showUnifiedVScaleMenu(const QPoint& globalPos);
void showBadgeMenu(int slotIdx, const QPoint& globalPos);
void pushZoomHist();
void initPlotX(double tMax);
@@ -87,7 +90,8 @@ private:
bool paused_ = false;
double plotXMin_ = 0.0;
double plotXMax_ = 0.0;
int vMode_ = 0; /* 0 normal 1 digital 2 mixed */
int vMode_ = 0; /* 0 normal 1 digital 2 mixed 3 unified */
VScale uniVS_; /* the one scale every trace shares in mode 3 */
int activeSlot_ = -1;
bool trigZoomed_ = false;
Client/streamhub/App.cpp
+6
View File
@@ -825,11 +825,17 @@ void App::onTriggerState(const std::string& json) {
trigger_.trigTime = msg.trigTime;
trigger_.hasTrigTime = true;
}
if (msg.hasWindow) {
trigger_.firedPreS = msg.preSec;
trigger_.firedPostS = msg.postSec;
trigger_.hasFiredWin = true;
}
/* Double-buffer semantics: the last recorded capture stays on display
* (even while re-armed/collecting) and is only replaced when a new
* capture frame has been fully received and parsed (handleBinary v2). */
if (msg.state == "idle") {
trigger_.hasTrigTime = false;
trigger_.hasFiredWin = false;
}
}
Client/streamhub/App.h
+11 -2
View File
@@ -61,6 +61,11 @@ struct TriggerState {
bool stopped = false;
bool hasTrigTime = false;
double trigTime = 0.0;
/* Window the hub latched at fire time. Not the same as windowSec/prePercent
* above, which are editable and may have moved on since the trigger fired. */
bool hasFiredWin = false;
double firedPreS = 0.0;
double firedPostS = 0.0;
};
/** Per-signal vertical scale state (oscilloscope style). */
@@ -183,9 +188,12 @@ public:
plotXMax_[i] = tMax;
}
/** @brief Per-plot vertical normalisation: 0=normal 1=digital 2=mixed. */
/** @brief Per-plot vertical normalisation: 0=normal 1=digital 2=mixed 3=unified. */
int& plotVMode(int i) { return plotVMode_[i]; }
/** @brief The one scale every trace shares in unified mode (vMode 3). */
VScale& plotUnifiedVS(int i) { return plotUniVS_[i]; }
/* ---- Cursors A/B (global: shared & synchronised across all plots) ---- */
bool& cursorsOn() { return cursorsOn_; }
double& cursorA() { return cursorA_; }
@@ -302,7 +310,8 @@ private:
double windowSec_ = 10.0; /* live scroll window width */
double plotXMin_[kMaxPlotSlots] = {}; /* stored X min for non-live mode */
double plotXMax_[kMaxPlotSlots] = {}; /* stored X max for non-live mode */
int plotVMode_[kMaxPlotSlots] = {}; /* 0=normal 1=digital 2=mixed */
int plotVMode_[kMaxPlotSlots] = {}; /* 0=normal 1=digital 2=mixed 3=unified */
VScale plotUniVS_[kMaxPlotSlots]; /* shared scale used by vMode 3 */
/* Cursors (global) */
bool cursorsOn_ = false;
Client/streamhub/PlotPanel.cpp
+192 -61
View File
@@ -85,6 +85,49 @@ static double normalizeY(double raw, const VScale& vs) {
return (raw - vs.resolvedOffset) / vs.resolvedDiv + vs.screenPos;
}
/** Resolve the one scale every trace shares in unified mode.
*
* Same rules as the per-signal version, applied to the union of the plot:
* range takes the union of the declared ranges, auto fits the union of the
* data. Signals whose slot is empty contribute nothing. */
static void resolveUnifiedVScale(VScale& vs,
const std::vector<PlotAssignment>& slots,
const std::vector<Source>& sources,
const std::vector<std::vector<double> >& vStore) {
if (vs.mode == 2) { /* manual */
vs.resolvedDiv = std::max(vs.divValue, 1e-30);
vs.resolvedOffset = vs.offset;
return;
}
double mn = 1e300, mx = -1e300;
if (vs.mode == 1) { /* range: union of every declared range */
for (const auto& a : slots) {
if (a.sourceIdx < 0 || a.sourceIdx >= (int)sources.size()) continue;
const auto& m = sources[a.sourceIdx].signals[a.signalIdx].meta;
if (!(m.rangeMax > m.rangeMin)) continue;
if (m.rangeMin < mn) mn = m.rangeMin;
if (m.rangeMax > mx) mx = m.rangeMax;
}
if (mx > mn) {
vs.resolvedDiv = std::max((mx - mn) / 8.0, 1e-30);
vs.resolvedOffset = (mn + mx) / 2.0;
return;
}
mn = 1e300; mx = -1e300; /* no usable range: fall through to auto */
}
for (const auto& vv : vStore) {
for (double v : vv) {
if (!std::isfinite(v)) continue;
if (v < mn) mn = v;
if (v > mx) mx = v;
}
}
if (!std::isfinite(mn) || mn > mx) { mn = -1.0; mx = 1.0; }
if (mn == mx) { mn -= 1.0; mx += 1.0; }
vs.resolvedDiv = std::max((mx - mn) / 6.0, 1e-30);
vs.resolvedOffset = (mx + mn) / 2.0;
}
/** Min/max of a vector (returns false if empty/non-finite). */
static bool dataMinMax(const std::vector<double>& v, double& mn, double& mx) {
mn = 1e300; mx = -1e300;
@@ -149,9 +192,36 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
const double wallNow = std::chrono::duration<double>(
std::chrono::system_clock::now().time_since_epoch()).count();
/* Trigger view: render the hub capture relative to the trigger instant */
/* Trigger view: render the hub capture relative to the trigger instant.
*
* Two ways to end up in trigger-relative time. Either a v2 capture frame
* has arrived (trigView), or a trigger has fired and its window is still
* filling (trigFill). In the second case the hub sends nothing until the
* whole window has been produced — several seconds for a long window at a
* high rate — so the trace is drawn from this client's own rings on the
* final axis, growing left to right. Filling wins over the previous
* capture: once a new trigger fires, the stale waveform is history. */
const CaptureFrame* cap = app.capture();
const bool trigView = (cap != nullptr) && app.showTrigBar();
const TriggerState& trg = app.trigger();
/* Prefer the window the hub latched at fire time; the local config is only
* a fallback for hubs that do not report it, and may have been edited
* since the trigger fired. */
const double fillPreS = trg.hasFiredWin ? trg.firedPreS
: trg.windowSec * trg.prePercent * 0.01;
const double fillPostS = trg.hasFiredWin ? trg.firedPostS
: trg.windowSec - fillPreS;
const bool trigFill = app.showTrigBar() && !paused &&
trg.status == "collecting" && trg.hasTrigTime;
const bool trigView = (cap != nullptr) && app.showTrigBar() && !trigFill;
const bool trigRel = trigView || trigFill;
/* Window edges of whatever is on screen. A capture latches its own
* pre/post at fire time, so later edits in the trigger bar must not move
* the axis of a finished capture. */
const double trigT = trigView ? cap->trigTime : trg.trigTime;
const double trigPreS = trigView ? cap->preSec : fillPreS;
const double trigPostS = trigView ? cap->postSec : fillPostS;
/* Hi-res zoom cache for this plot */
auto& zc = app.zoomCache(plotIdx);
@@ -194,13 +264,13 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
* from decimated pushes) undersamples the visible range. Periodically
* fetch a fresh ~2400-pt slice from the hub raw ring and anchor the X
* axis to the fetched slice (scope-style refresh at the fetch rate). */
const bool liveHiRes = !trigView && live && !paused &&
const bool liveHiRes = !trigRel && live && !paused &&
app.windowSec() <= kLiveHiResMaxWin &&
zc.valid &&
(zc.t1 - zc.t0) >= app.windowSec() * 0.9 &&
(wallNow - zc.t1) < 3.0;
const bool useZoomData = !trigView && !paused && zc.valid &&
const bool useZoomData = !trigRel && !paused && zc.valid &&
(liveHiRes ||
(!live &&
zc.t0 <= app.plotXMin(plotIdx) + 1e-9 &&
@@ -215,7 +285,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
const bool haveHistCover = hc.valid &&
hc.t0 <= app.plotXMin(plotIdx) + 1e-9 &&
hc.t1 >= app.plotXMax(plotIdx) - 1e-9;
bool useHistData = !trigView && !paused && !live && haveHistCover;
bool useHistData = !trigRel && !paused && !live && haveHistCover;
if (useHistData) {
/* Check that at least one signal has actual data points */
bool anyData = false;
@@ -231,7 +301,12 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
* copied tens of MB per signal per frame. A 10% margin keeps a sample on
* each side so the later fine clip still has its boundary points. */
double visT0, visT1;
if (live) { visT1 = wallNow; visT0 = wallNow - app.windowSec(); }
if (trigFill) {
/* Absolute bounds of the trigger window: the ring is indexed on the
* hub clock, the axis on trigger-relative time. */
visT0 = trigT - trigPreS; visT1 = trigT + trigPostS;
}
else if (live) { visT1 = wallNow; visT0 = wallNow - app.windowSec(); }
else { visT1 = app.plotXMax(plotIdx); visT0 = app.plotXMin(plotIdx); }
{
double margin = (visT1 - visT0) * 0.1;
@@ -272,6 +347,15 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
}
break;
}
} else if (trigFill) {
/* Live ring, clipped to the (absolute) window and shifted onto the
* trigger-relative axis. visT0/visT1 already carry a margin, so
* clip here rather than reusing readBase. */
(void) sig.buf.readRange(trigT - trigPreS, trigT + trigPostS,
tStore[si], vStore[si]);
for (size_t i = 0; i < tStore[si].size(); i++) {
tStore[si][i] -= trigT;
}
} else if (useZoomData) {
bool found = false;
for (const auto& zs : zc.signals) {
@@ -302,6 +386,11 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
resolveVScale(a, sig, vStore[si]);
}
VScale& uniVS = app.plotUnifiedVS(plotIdx);
if (vMode == 3) {
resolveUnifiedVScale(uniVS, slots, sources, vStore);
}
/* clamp active slot */
if (actSlot >= (int)slots.size()) actSlot = -1;
@@ -326,9 +415,11 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
ImVec4(0.067f,0.067f,0.106f,1.f));
char badge[80];
/* show vscale info: resolved div value */
/* Show the div value actually in force: the plot's shared one in
* unified mode, this signal's otherwise. */
char dvbuf[16];
fmtVal(dvbuf, sizeof(dvbuf), a.vs.resolvedDiv);
fmtVal(dvbuf, sizeof(dvbuf),
(vMode == 3) ? uniVS.resolvedDiv : a.vs.resolvedDiv);
snprintf(badge, sizeof(badge), "%s %s/div##b%d",
sig.meta.name.c_str(), dvbuf, i);
@@ -398,7 +489,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
}
/* Back / Fit / Reset (zoom history) */
if (!live || (trigView && app.trigZoomed(plotIdx))) {
if (!live || (trigRel && app.trigZoomed(plotIdx))) {
ImGui::SameLine();
auto& hist = app.zoomHist(plotIdx);
if (hist.empty()) { ImGui::BeginDisabled(); }
@@ -408,7 +499,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
}
if (hist.empty()) { ImGui::EndDisabled(); }
ImGui::SameLine();
if (trigView) {
if (trigRel) {
/* Reset to full capture window */
if (ImGui::SmallButton(ICON_FA_EXPAND " Reset##zr")) {
app.trigZoomed(plotIdx) = false;
@@ -440,10 +531,15 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* Norm/Dig/Mix mode — compact toggle buttons matching SmallButton height */
ImGui::SameLine();
{
static const char* kVLabels[] = {"N", "D", "M"};
static const char* kVTooltips[] = {"Normal", "Digital", "Mixed"};
for (int vm = 0; vm < 3; vm++) {
char vmId[16]; snprintf(vmId, sizeof(vmId), "%s##vm%d_%d", kVLabels[vm], plotIdx, vm);
static const char* kVLabels[] = {"N", "U", "D", "M"};
static const char* kVTooltips[] = {
"Normal: one vertical scale per signal",
"Unified: one vertical scale shared by every signal",
"Digital", "Mixed" };
static const int kVModes[] = {0, 3, 1, 2};
for (int i = 0; i < 4; i++) {
const int vm = kVModes[i];
char vmId[16]; snprintf(vmId, sizeof(vmId), "%s##vm%d_%d", kVLabels[i], plotIdx, vm);
bool sel = (vMode == vm);
if (sel) {
ImGui::PushStyleColor(ImGuiCol_Button, ImVec4(0.537f,0.706f,0.980f,0.4f));
@@ -451,28 +547,41 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
}
if (ImGui::SmallButton(vmId)) { vMode = vm; }
if (sel) { ImGui::PopStyleColor(2); }
if (ImGui::IsItemHovered()) { ImGui::SetTooltip("%s", kVTooltips[vm]); }
if (vm < 2) { ImGui::SameLine(0.f, 1.f); }
if (ImGui::IsItemHovered()) { ImGui::SetTooltip("%s", kVTooltips[i]); }
if (i < 3) { ImGui::SameLine(0.f, 1.f); }
}
}
/* ── VScale toolbar (shown when an active signal is selected) ───────── */
/* ── VScale toolbar ──────────────────────────────────────────────────── *
* Normal mode edits the active signal's scale; unified mode edits the one
* scale the whole plot shares, so it needs no selection. */
VScale *toolVS = static_cast<VScale *>(0);
if (vMode == 0 && actSlot >= 0 && actSlot < (int)slots.size()) {
auto& a = slots[actSlot];
toolVS = &slots[actSlot].vs;
} else if (vMode == 3) {
toolVS = &uniVS;
}
if (toolVS != static_cast<VScale *>(0)) {
VScale& tvs = *toolVS;
ImGui::PushStyleVar(ImGuiStyleVar_ItemSpacing, ImVec2(4.f,2.f));
if (vMode == 3) {
ImGui::TextDisabled("all signals");
ImGui::SameLine(0.f,10.f);
}
/* mode buttons */
static const char* kModeLabels[] = {"Auto","Range","Manual"};
for (int m = 0; m < 3; m++) {
bool sel = (a.vs.mode == m);
bool sel = (tvs.mode == m);
if (sel) {
ImGui::PushStyleColor(ImGuiCol_Button,
ImVec4(0.537f,0.706f,0.980f,0.3f));
ImGui::PushStyleColor(ImGuiCol_Text,
ImVec4(0.537f,0.706f,0.980f,1.f));
}
if (ImGui::SmallButton(kModeLabels[m])) { a.vs.mode = m; }
if (ImGui::SmallButton(kModeLabels[m])) { tvs.mode = m; }
if (sel) ImGui::PopStyleColor(2);
if (m < 2) ImGui::SameLine(0.f,2.f);
}
@@ -480,23 +589,23 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* resolved info */
char rbuf[24], obuf[24];
fmtVal(rbuf, sizeof(rbuf), a.vs.resolvedDiv);
fmtVal(obuf, sizeof(obuf), a.vs.resolvedOffset);
fmtVal(rbuf, sizeof(rbuf), tvs.resolvedDiv);
fmtVal(obuf, sizeof(obuf), tvs.resolvedOffset);
if (a.vs.mode == 2) { /* manual: editable */
if (tvs.mode == 2) { /* manual: editable */
ImGui::SetNextItemWidth(70.f);
ImGui::InputDouble("V/div##vd", &a.vs.divValue, 0,0,"%.4g");
ImGui::InputDouble("V/div##vd", &tvs.divValue, 0,0,"%.4g");
ImGui::SameLine(0.f,4.f);
ImGui::SetNextItemWidth(80.f);
ImGui::InputDouble("Offset##vo", &a.vs.offset, 0,0,"%.4g");
ImGui::InputDouble("Offset##vo", &tvs.offset, 0,0,"%.4g");
} else {
ImGui::TextDisabled("%s/div @%s", rbuf, obuf);
}
ImGui::SameLine(0.f,10.f);
ImGui::SetNextItemWidth(50.f);
float sp = (float)a.vs.screenPos;
float sp = (float)tvs.screenPos;
if (ImGui::InputFloat("Pos(div)##vp", &sp, 0,0,"%.1f")) {
a.vs.screenPos = sp;
tvs.screenPos = sp;
}
ImGui::PopStyleVar();
@@ -539,7 +648,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
if (ImPlot::BeginPlot(plotId, ImVec2(-1.f,-1.f), plotFlags)) {
/* Both axes locked so ImPlot never overrides our explicit limits. */
ImPlot::SetupAxes(trigView ? "t - trig (s)" : "Time (s)", nullptr,
ImPlot::SetupAxes(trigRel ? "t - trig (s)" : "Time (s)", nullptr,
ImPlotAxisFlags_Lock,
ImPlotAxisFlags_Lock);
@@ -549,13 +658,17 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* X axis: trig view → capture window (zoomable); live → wall clock; else stored */
double xMin, xMax;
bool& trigZm = app.trigZoomed(plotIdx);
if (trigView) {
if (trigRel) {
if (trigZm) {
xMin = app.plotXMin(plotIdx);
xMax = app.plotXMax(plotIdx);
} else {
xMin = -cap->preSec;
xMax = cap->postSec;
/* Full window from the start, even while filling: a trace that
* grows into a fixed axis reads as progress; an axis that
* grows with the data makes the whole trace shift every
* frame and the time base meaningless. */
xMin = -trigPreS;
xMax = trigPostS;
}
} else if (live && !paused) {
if (liveHiRes) {
@@ -568,7 +681,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} else {
xMin = app.plotXMin(plotIdx); xMax = app.plotXMax(plotIdx);
}
if (trigView || (live && !paused) || !live) {
if (trigRel || (live && !paused) || !live) {
if (xMax > xMin) {
ImPlot::SetupAxisLimits(ImAxis_X1, xMin, xMax, ImGuiCond_Always);
}
@@ -579,8 +692,16 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
static char yTickBufs[9][20];
static const char* yTickLabels[9];
const VScale *axisVS = static_cast<const VScale *>(0);
if (vMode == 0 && actSlot >= 0 && actSlot < (int)slots.size()) {
const auto& av = slots[actSlot].vs;
axisVS = &slots[actSlot].vs;
} else if (vMode == 3) {
/* Unified: the shared scale labels the axis for every trace at
* once, so no signal has to be selected first. */
axisVS = &uniVS;
}
if (axisVS != static_cast<const VScale *>(0)) {
const VScale& av = *axisVS;
for (int d = 0; d < 9; d++) {
double divPos = yTickVals[d];
double rawVal = av.resolvedOffset + (divPos - av.screenPos) * av.resolvedDiv;
@@ -636,15 +757,15 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* Helper: enter zoomed mode for trigger view (seed from capture window) */
auto enterTrigZoom = [&]() {
if (trigView && !trigZm) {
app.setPlotX(plotIdx, -cap->preSec, cap->postSec);
if (trigRel && !trigZm) {
app.setPlotX(plotIdx, -trigPreS, trigPostS);
trigZm = true;
}
};
/* Helper: X-zoom the stored range by factor around center */
auto xZoomStored = [&](double factor) {
if (trigView) { enterTrigZoom(); }
if (trigRel) { enterTrigZoom(); }
if (now - lastHistPush[plotIdx] > 0.6) {
app.pushZoomHist(plotIdx);
lastHistPush[plotIdx] = now;
@@ -659,37 +780,45 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
const double zoomOut = 1.25;
double factor = (wheel > 0.f) ? zoomIn : zoomOut;
/* Scroll adjusts the scale the axis is labelled with: the
* active signal's in normal mode, the plot's shared one in
* unified mode (where there is nothing to select). */
VScale *wheelVS = static_cast<VScale *>(0);
if (vMode == 3) {
wheelVS = &uniVS;
} else if (actSlot >= 0 && actSlot < (int)slots.size()) {
wheelVS = &slots[actSlot].vs;
}
/* Seed manual from the resolved values so the gesture sticks. */
auto latchManual = [](VScale& v) {
if (v.mode != 2) {
v.divValue = std::max(v.resolvedDiv, 1e-30);
v.offset = v.resolvedOffset;
v.mode = 2;
}
};
if (ctrl) {
/* ── X zoom ─────────────────────────────────────────── */
if (!trigView && live) {
if (!trigRel && live) {
app.setWindowSec(app.windowSec() * factor);
} else {
xZoomStored(factor);
}
} else if (shift) {
/* ── Y offset of active signal ───────────────────────── */
if (actSlot >= 0 && actSlot < (int)slots.size()) {
auto& a = slots[actSlot];
if (a.vs.mode != 2) {
a.vs.divValue = std::max(a.vs.resolvedDiv, 1e-30);
a.vs.offset = a.vs.resolvedOffset;
a.vs.mode = 2;
}
a.vs.screenPos += (wheel > 0.f) ? 0.5 : -0.5;
/* ── Y pan ───────────────────────────────────────────── */
if (wheelVS != static_cast<VScale *>(0)) {
latchManual(*wheelVS);
wheelVS->screenPos += (wheel > 0.f) ? 0.5 : -0.5;
}
} else {
/* ── Y zoom of active signal ─────────────────────────── */
if (actSlot >= 0 && actSlot < (int)slots.size()) {
auto& a = slots[actSlot];
if (a.vs.mode != 2) {
a.vs.divValue = std::max(a.vs.resolvedDiv, 1e-30);
a.vs.offset = a.vs.resolvedOffset;
a.vs.mode = 2;
}
a.vs.divValue = std::max(a.vs.divValue * factor, 1e-30);
/* ── Y zoom ──────────────────────────────────────────── */
if (wheelVS != static_cast<VScale *>(0)) {
latchManual(*wheelVS);
wheelVS->divValue = std::max(wheelVS->divValue * factor, 1e-30);
} else {
/* No active signal: plain scroll → X zoom */
if (!trigView && live) {
if (!trigRel && live) {
app.setWindowSec(app.windowSec() * factor);
} else {
xZoomStored(factor);
@@ -701,8 +830,8 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* Right-drag → X pan. Transition live→non-live on drag start;
* in trigger view, enter trigger-zoom mode. */
if (ImGui::IsMouseDragging(ImGuiMouseButton_Right)) {
if (trigView) { enterTrigZoom(); }
if (!trigView && live) {
if (trigRel) { enterTrigZoom(); }
if (!trigRel && live) {
app.initPlotX(plotIdx, wallNow);
live = false;
lastHistPush[plotIdx] = now;
@@ -721,7 +850,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
}
/* ── Hi-res WS zoom requests (suppressed while paused) ──────────── */
if (!trigView && !paused) {
if (!trigRel && !paused) {
std::string csv;
for (const auto& a : slots) {
std::string k = app.slotKey(a);
@@ -821,9 +950,11 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} else if (vMode == 2) { /* mixed */
bandNormalize(vDec, vNorm, myKi, nTraces, a.vs.digitalInMixed);
} else {
/* unified shares one scale, normal gives each trace its own */
const VScale& nvs = (vMode == 3) ? uniVS : a.vs;
vNorm.resize(nOut);
for (size_t k = 0; k < nOut; k++) {
vNorm[k] = normalizeY(vDec[k], a.vs);
vNorm[k] = normalizeY(vDec[k], nvs);
}
}
@@ -836,7 +967,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
}
/* Trigger instant marker (capture view: t = 0) */
if (trigView) {
if (trigRel) {
double t0m = 0.0;
ImPlot::DragLineX(900, &t0m, ImVec4(1.f,1.f,0.f,0.8f),
1.5f, ImPlotDragToolFlags_NoInputs);
Client/streamhub/Protocol.cpp
+6
View File
@@ -458,6 +458,12 @@ bool ParseTriggerState(const std::string& json, TriggerStateMsg& out) {
double tt = 0.0;
out.hasTrigTime = jsonGetDouble(json.c_str(), "trigTime", tt);
out.trigTime = tt;
double pre = 0.0, post = 0.0;
out.hasWindow = jsonGetDouble(json.c_str(), "preSec", pre) &&
jsonGetDouble(json.c_str(), "postSec", post);
out.preSec = pre;
out.postSec = post;
return true;
}
Client/streamhub/Protocol.h
+5
View File
@@ -109,6 +109,11 @@ struct TriggerStateMsg {
bool stopped = false;
bool hasTrigTime = false;
double trigTime = 0.0;
/* Window latched at fire time, sent alongside trigTime. Older hubs omit
* it, hence hasWindow — fall back to the local trigger config then. */
bool hasWindow = false;
double preSec = 0.0;
double postSec = 0.0;
};
/*---------------------------------------------------------------------------*/
Client/streamhub/WSClient.cpp
+13 -4
View File
@@ -18,18 +18,27 @@
#include <cstdlib>
#include <ctime>
#include <chrono>
#include <random>
namespace StreamHubClient {
/* ── Helpers ─────────────────────────────────────────────────────────────── */
static std::string base64Key() {
/* Generate 16 random bytes and base64-encode them */
/* HI-7: use /dev/urandom (CSPRNG) instead of srand(time)/rand() */
uint8_t raw[16];
srand(static_cast<unsigned>(time(nullptr)));
for (int i = 0; i < 16; i++) {
raw[i] = static_cast<uint8_t>(rand() & 0xFF);
int fd = open("/dev/urandom", O_RDONLY);
if (fd < 0 || read(fd, raw, sizeof(raw)) != static_cast<ssize_t>(sizeof(raw))) {
/* Fallback: std::random_device (still better than srand/rand) */
std::random_device rd;
for (size_t i = 0; i < sizeof(raw); i += sizeof(unsigned)) {
unsigned val = rd();
for (size_t j = 0; j < sizeof(unsigned) && i + j < sizeof(raw); j++) {
raw[i + j] = static_cast<uint8_t>(val >> (j * 8));
}
}
}
if (fd >= 0) { close(fd); }
char out[32];
WS_Base64Encode(raw, 16, out);
return std::string(out);
Client/streamhub/build/StreamHubClient
BIN
View File
Binary file not shown.
Client/udpstreamer/main.go
+56 -3
View File
@@ -9,6 +9,9 @@ import (
"log"
"net/http"
"os"
"os/signal"
"path/filepath"
"syscall"
"marte2/common/wshub"
)
@@ -21,20 +24,56 @@ var staticFiles embed.FS
// multiFlag allows a flag to be repeated: --source a --source b
type multiFlag []string
func (f *multiFlag) String() string { return fmt.Sprintf("%v", []string(*f)) }
func (f *multiFlag) Set(v string) error { *f = append(*f, v); return nil }
func (f *multiFlag) String() string { return fmt.Sprintf("%v", []string(*f)) }
func (f *multiFlag) Set(v string) error { *f = append(*f, v); return nil }
// defaultHistoryDir is where samples are archived unless -history-dir says
// otherwise. History is on by default because it is what holds a trigger
// capture at full resolution: the in-memory rings roll past a captured window
// within seconds of it being taken, and a zoom after that has nothing but the
// capture's own decimated copy to draw. Per-signal files are bounded by
// -history-max-mpts, so the default costs a fixed amount of space.
func defaultHistoryDir() string {
return filepath.Join(os.TempDir(), "udpstreamer-history")
}
func main() {
var sourceArgs multiFlag
flag.Var(&sourceArgs, "source", `Data source in the form [label@]host:port[/multicastGroup:dataPort] (repeatable)`)
sourcesFile := flag.String("sources-file", "", "JSON file for persistent source list (load on start, save target)")
listenAddr := flag.String("addr", ":8080", "HTTP listen address")
histDir := flag.String("history-dir", defaultHistoryDir(), "Directory for disk-backed signal history (empty disables it)")
histWindow := flag.Float64("history-window-sec", 0, "Timespan the history files hold before any client says what it displays (0 keeps the 10 s default); the hub re-sizes them to the live or trigger window afterwards")
histDecim := flag.Int("history-decimation", 1, "Keep every Nth sample in the history files")
histFlush := flag.Int("history-flush-sec", 5, "Seconds between history header flushes")
histMinFree := flag.Int("history-min-free-mb", 500, "Pause history writing below this much free disk (negative disables the check)")
histMaxMPts := flag.Float64("history-max-mpts", 0, "Per-signal history budget in millions of points, also settable in the web UI (0 keeps the 16 MPts / 256 MB default)")
ringMPts := flag.Float64("ring-mpts", 0, "Per-signal in-memory buffer in millions of points (0 keeps the 10 MPts / 160 MB default)")
flag.Parse()
hub := wshub.NewHub()
// The budget bounds memory, not the window: a window too long to hold at the
// source rate is buffered as min/max pairs rather than truncated to the tail.
hub.SetRingBudget(int(*ringMPts * 1e6))
sm := wshub.NewSourceManager(hub, *sourcesFile)
hub.SetSourceManager(sm)
if err := hub.EnableHistory(wshub.HistoryConfig{
Directory: *histDir,
WindowSec: *histWindow,
Decimation: *histDecim,
FlushIntervalSec: *histFlush,
MinDiskFreeMB: *histMinFree,
MaxPointsPerSignal: int(*histMaxMPts * 1e6),
}); err != nil {
log.Fatalf("history: %v", err)
}
if *histDir == "" {
log.Print("history disabled: zooming into a trigger capture will fall back " +
"to the capture's own decimated copy once the rings roll past it")
} else {
log.Printf("history: %s", *histDir)
}
go hub.Run()
// Load sources from file first (if specified), then add any CLI --source flags.
@@ -60,7 +99,21 @@ func main() {
})
log.Printf("UDPStreamer WebUI listening on %s (build=%s)", *listenAddr, buildVersion)
if err := http.ListenAndServe(*listenAddr, nil); err != nil {
// Serve in the background so Ctrl-C can flush the history files: the
// samples written since the last periodic flush are on disk but are not
// yet accounted for in the file headers, so exiting outright loses them.
srvErr := make(chan error, 1)
go func() { srvErr <- http.ListenAndServe(*listenAddr, nil) }()
sig := make(chan os.Signal, 1)
signal.Notify(sig, os.Interrupt, syscall.SIGTERM)
select {
case err := <-srvErr:
hub.CloseHistory()
log.Fatalf("http: %v", err)
case s := <-sig:
log.Printf("received %s, flushing history", s)
hub.CloseHistory()
}
}
Client/udpstreamer/static/app.js
+1337 -303
View File
File diff suppressed because it is too large Load Diff
Client/udpstreamer/static/calibration.js
+161
View File
@@ -0,0 +1,161 @@
// Per-signal affine calibration: value = raw * scale + offset, with an optional
// unit override. Pure and dependency-free so it can be unit-tested under Node;
// in the browser it defines the global `Calib`.
(function (root) {
'use strict';
var MAX_UNIT_LEN = 16;
var IDENTITY = Object.freeze({scale: 1, offset: 0, unit: ''});
// The table is keyed by (source label, base signal name). U+0000 cannot occur
// in either, so it is an unambiguous separator.
function calKey(source, signal) {
return source + '\u0000' + signal;
}
// 'Adc[3]' -> 'Adc'. One calibration covers every element of an array signal.
function baseSignalName(name) {
var s = String(name == null ? '' : name);
var open = s.lastIndexOf('[');
if (open >= 0 && s.charAt(s.length - 1) === ']') {
var idx = s.slice(open + 1, s.length - 1);
if (idx.length > 0 && /^[0-9]+$/.test(idx)) return s.slice(0, open);
}
return s;
}
function isFiniteNum(v) {
return typeof v === 'number' && isFinite(v);
}
// Mirrors the hub-side validation exactly (Go: CalConfig.Normalise,
// C++: StreamHub::HandleSetCalibration). Returns null when the entry must be
// rejected, so a caller can revert an input field to its last accepted value.
function normaliseCal(obj) {
if (obj === null || typeof obj !== 'object') return null;
var source = String(obj.source == null ? '' : obj.source).trim();
var signal = baseSignalName(String(obj.signal == null ? '' : obj.signal).trim());
if (source === '' || signal === '') return null;
var scale = obj.scale === undefined ? 1 : obj.scale;
var offset = obj.offset === undefined ? 0 : obj.offset;
if (!isFiniteNum(scale) || scale === 0) return null;
if (!isFiniteNum(offset)) return null;
var unit = String(obj.unit == null ? '' : obj.unit).trim();
// Cap at MAX_UNIT_LEN UTF-8 bytes, matching both hubs' Normalise() exactly.
// TextEncoder/TextDecoder are available natively in all modern browsers and
// Node v11+; no build step or bundler is needed.
var enc = new TextEncoder();
var bytes = enc.encode(unit);
if (bytes.length > MAX_UNIT_LEN) {
// Truncate to MAX_UNIT_LEN bytes, then repair any split UTF-8 rune at the
// tail. Mirrors Go's utf8.DecodeLastRuneInString walk-back loop and the
// C++ repair loop in StreamHub::SetCalibrationEntry:
// Drop continuation bytes (10xxxxxx) from the tail until the last byte
// is either an ASCII byte (< 0x80) or a lead byte whose sequence is
// complete (i.e. all expected continuation bytes are present).
//
// A single pass suffices here, where the C++ needs a loop. The C++ input
// is a raw const char* straight off the wire and may hold arbitrary bytes;
// `bytes` here comes from TextEncoder, which always emits well-formed
// UTF-8 (unpaired surrogates become U+FFFD = EF BF BD, and no byte is ever
// >= 0xF8). Truncating well-formed UTF-8 can therefore strand at most a
// lead byte plus three continuation bytes, which one pass fully repairs.
var b = bytes.slice(0, MAX_UNIT_LEN);
var len = b.length;
// Walk back over continuation bytes (up to 3) to find the lead byte of
// the last sequence.
var cont = 0;
while (cont < 3 && cont < len && (b[len - 1 - cont] & 0xC0) === 0x80) {
cont++;
}
if (cont < len) {
var lead = b[len - 1 - cont];
// Determine expected sequence length from the lead byte.
var seqLen = lead < 0x80 ? 1 : // 0xxxxxxx ASCII
(lead & 0xE0) === 0xC0 ? 2 : // 110xxxxx
(lead & 0xF0) === 0xE0 ? 3 : // 1110xxxx
(lead & 0xF8) === 0xF0 ? 4 : // 11110xxx
1; // orphaned continuation byte — treat as 1
var haveBytes = cont + 1; // lead + continuation bytes present
if (haveBytes < seqLen) {
// Incomplete sequence: drop the lead byte and all its continuations.
len = len - haveBytes;
}
}
unit = new TextDecoder().decode(b.slice(0, len));
}
return {source: source, signal: signal, scale: scale, offset: offset, unit: unit};
}
function isIdentity(cal) {
return cal.scale === 1 && cal.offset === 0 && cal.unit === '';
}
function applyCal(raw, cal) {
return raw * cal.scale + cal.offset;
}
function invertCal(value, cal) {
return (value - cal.offset) / cal.scale;
}
// A negative scale swaps the ends of a range, so re-order after calibrating.
function calRange(min, max, cal) {
var a = applyCal(min, cal), b = applyCal(max, cal);
return a <= b ? [a, b] : [b, a];
}
function CalTable() {
this._m = Object.create(null);
}
CalTable.prototype.get = function (source, signal) {
var e = this._m[calKey(source, baseSignalName(signal))];
return e === undefined ? IDENTITY : e;
};
// Returns false when the entry was rejected as invalid. An entry that reduces
// to the identity is deleted rather than stored, so a Reset cleans the table
// (and, once saved, the config file) instead of filling it with no-ops.
CalTable.prototype.set = function (entry) {
var c = normaliseCal(entry);
if (c === null) return false;
var k = calKey(c.source, c.signal);
if (isIdentity(c)) delete this._m[k];
else this._m[k] = c;
return true;
};
CalTable.prototype.replaceAll = function (list) {
this._m = Object.create(null);
if (!list) return;
for (var i = 0; i < list.length; i++) this.set(list[i]);
};
CalTable.prototype.list = function () {
var out = [], k;
for (k in this._m) out.push(this._m[k]);
out.sort(function (a, b) {
if (a.source !== b.source) return a.source < b.source ? -1 : 1;
if (a.signal !== b.signal) return a.signal < b.signal ? -1 : 1;
return 0;
});
return out;
};
var api = {
MAX_UNIT_LEN: MAX_UNIT_LEN,
IDENTITY: IDENTITY,
calKey: calKey,
baseSignalName: baseSignalName,
normaliseCal: normaliseCal,
isIdentity: isIdentity,
applyCal: applyCal,
invertCal: invertCal,
calRange: calRange,
CalTable: CalTable,
};
if (typeof module !== 'undefined' && module.exports) module.exports = api;
else root.Calib = api;
})(typeof globalThis !== 'undefined' ? globalThis : this);
Client/udpstreamer/static/decimate-worker.js
+51
View File
@@ -0,0 +1,51 @@
'use strict';
// Min/max (peak-envelope) decimation — O(n). Runs off-main-thread to avoid
// blocking the render loop.
//
// The range is split into threshold/2 equal buckets and each contributes its
// smallest and largest sample, in the order the two occurred — the way an
// oscilloscope draws a trace it cannot show pixel-for-pixel.
//
// This replaced LTTB, which picks the sample forming the largest triangle with
// its neighbours: a plausible-looking shape, but it silently drops a one-sample
// spike whenever a smoother neighbour scores higher — exactly the sample worth
// looking at. The envelope cannot drop it, because a spike is by definition its
// bucket's min or max. Every output point is a real sample at its real
// timestamp; nothing is interpolated or averaged.
//
// Kept identical to minMaxDecimate() in Common/Client/go/wshub/hub.go and to
// decimate() in app.js, so a trace looks the same whichever thinned it.
function decimate(t, v, threshold) {
const len = t.length;
if (len <= threshold || threshold < 4) {
// Copy to new arrays so we can transfer them back without detaching the input.
return { t: new Float64Array(t), v: new Float64Array(v) };
}
const buckets = threshold >> 1;
const outT = new Float64Array(threshold);
const outV = new Float64Array(threshold);
let n = 0;
for (let b = 0; b < buckets; b++) {
const lo = Math.floor(b * len / buckets);
const hi = (b === buckets - 1) ? len : Math.floor((b + 1) * len / buckets);
if (lo >= hi) continue;
let iMin = lo, iMax = lo;
for (let j = lo + 1; j < hi; j++) {
if (v[j] < v[iMin]) iMin = j;
if (v[j] > v[iMax]) iMax = j;
}
// Emit in time order so the result plots as one ascending trace.
if (iMin > iMax) { const s = iMin; iMin = iMax; iMax = s; }
outT[n] = t[iMin]; outV[n] = v[iMin]; n++;
// A bucket whose samples are all equal has one extreme, not two.
if (iMax !== iMin) { outT[n] = t[iMax]; outV[n] = v[iMax]; n++; }
}
// slice() so the transferred buffers are exactly the used length.
return { t: outT.slice(0, n), v: outV.slice(0, n) };
}
self.onmessage = function({ data: { id, t, v, threshold } }) {
const result = decimate(t, v, threshold);
// Transfer the output buffers back to the main thread zero-copy.
self.postMessage({ id, t: result.t, v: result.v }, [result.t.buffer, result.v.buffer]);
};
Client/udpstreamer/static/index.html
+78 -13
View File
@@ -21,20 +21,34 @@
<span id="cur-ta">A: —</span><span class="cur-sep">│</span>
<span id="cur-tb">B: —</span><span class="cur-sep">│</span>
<span id="cur-dt">ΔT: —</span>
<span id="ruler-readout" style="display:none">
<span class="cur-sep">│</span>
<span id="cur-y1">Y1: —</span><span class="cur-sep">│</span>
<span id="cur-y2">Y2: —</span><span class="cur-sep">│</span>
<span id="cur-dy">ΔY: —</span>
</span>
</div>
<span class="ctrl-label" id="lbl-window">Window:</span>
<select id="window-select" class="ctrl-select">
<option value="1">1 s</option><option value="5" selected>5 s</option>
<option value="10">10 s</option><option value="30">30 s</option>
<option value="60">60 s</option>
<option value="1">1 s</option><option value="2">2 s</option>
<option value="5" selected>5 s</option><option value="10">10 s</option>
<option value="15">15 s</option><option value="30">30 s</option>
<option value="60">60 s</option><option value="120">2 min</option>
<option value="300">5 min</option><option value="600">10 min</option>
</select>
<button id="btn-cursor" class="ctrl-btn" style="display:none">Cursor</button>
<button id="btn-cursor" class="ctrl-btn">Cursors</button>
<button id="btn-cursor-reset" class="ctrl-btn" style="display:none" title="Bring cursors A/B back into the visible window">↔ Reset</button>
<button id="btn-ruler" class="ctrl-btn" title="Horizontal value rulers">Rulers</button>
<button id="btn-zoom-back" class="ctrl-btn" style="display:none">← Back</button>
<button id="btn-zoom-fit" class="ctrl-btn">Fit</button>
<button id="btn-csv-all" class="ctrl-btn" title="Export all signals to CSV">⬇ CSV</button>
<button id="btn-sync-resume" class="ctrl-btn resume-btn" style="display:none">↺ Auto</button>
<button id="btn-trigger" class="ctrl-btn">⚡ Trigger</button>
<button id="btn-pause-global" class="ctrl-btn">⏸ Pause</button>
<label class="ctrl-check" title="Snap jittery inter-frame timestamps to ideal spacing (eliminates overlaps/gaps from software-dispatch jitter)">
<input type="checkbox" id="cb-monotonic">
Sync TS
</label>
</div>
<!-- ── Trigger bar ───────────────────────────────────────────── -->
<div id="trigbar">
@@ -60,10 +74,19 @@
<div class="trig-group">
<span class="trig-label">Window</span>
<select id="trig-window" class="trig-select">
<option value="0.0001">100 μs</option><option value="0.001">1 ms</option>
<option value="0.01">10 ms</option><option value="0.1">100 ms</option>
<option value="0.5">500 ms</option><option value="1" selected>1 s</option>
<option value="0.0001">100 μs</option><option value="0.0002">200 μs</option>
<option value="0.0005">500 μs</option><option value="0.001">1 ms</option>
<option value="0.002">2 ms</option><option value="0.005">5 ms</option>
<option value="0.01">10 ms</option><option value="0.02">20 ms</option>
<option value="0.05">50 ms</option><option value="0.1">100 ms</option>
<option value="0.2">200 ms</option><option value="0.5">500 ms</option>
<option value="1" selected>1 s</option><option value="2">2 s</option>
<option value="5">5 s</option><option value="10">10 s</option>
<option value="20">20 s</option><option value="30">30 s</option>
<option value="60">60 s</option>
<option value="120">2 m</option>
<option value="300">5 m</option>
<option value="600">10 m</option>
</select>
</div>
<div class="trig-sep"></div>
@@ -73,6 +96,11 @@
<span class="trig-range-val" id="trig-pre-val">20%</span>
</div>
<div class="trig-sep"></div>
<div class="trig-group">
<span class="trig-label" title="Re-arm delay after a capture — prevents double triggering">Holdoff</span>
<input id="trig-holdoff" class="trig-input" type="number" min="0" max="60" step="0.01" value="0.2">
<span class="trig-label">s</span>
</div>
<div class="trig-group">
<span class="trig-label">Mode</span>
<select id="trig-mode" class="trig-select">
@@ -83,6 +111,7 @@
<div class="trig-sep"></div>
<div class="trig-group" style="gap:8px">
<span id="trig-status-badge">IDLE</span>
<button id="btn-trig-force" title="Capture now, ignoring the threshold">Force</button>
<button id="btn-trig-stop" style="display:none">Stop</button>
<button id="btn-trig-rearm">Rearm</button>
</div>
@@ -113,10 +142,30 @@
<span id="status-text">Disconnected</span>
<span id="sb-tsage"></span>
<button id="btn-stats" class="ctrl-btn" style="height:16px;padding:0 7px;font-size:10px;line-height:1">📊 Stats</button>
<span id="history-badge" style="display:none;font-size:10px;color:#f9e2af;margin-left:8px"></span>
<button id="history-badge" style="display:none" title="Disk history — click to set the per-signal budget"></button>
</div>
<span id="build-version"></span>
</div>
<!-- ── History budget popup ──────────────────────────────────── -->
<div id="history-panel" style="display:none">
<div class="ctx-menu-header">Disk history budget</div>
<div class="ctx-row">
<label>Budget</label>
<input type="number" id="hist-budget" class="ctx-num" min="0.001" step="1">
<span class="ctx-range-val">MPts/signal</span>
</div>
<div class="hist-note">
The budget buys resolution, not duration: a signal too fast to store
sample-for-sample is archived as a min/max envelope wide enough to fit,
so the configured window is always covered.
</div>
<div id="hist-signal-res"></div>
<div class="hist-note hist-warn">Applying re-creates the history files — archived data is lost.</div>
<div class="ctx-row" style="margin:0;justify-content:flex-end">
<button class="ctx-btn" id="btn-hist-cancel">Cancel</button>
<button class="ctx-btn" id="btn-hist-apply">Apply</button>
</div>
</div>
<div id="layout-menu"></div>
<!-- ── Signal style context menu ─────────────────────────────── -->
<div id="sig-ctx-menu" style="display:none">
@@ -161,7 +210,8 @@
</div>
<!-- ── Array index picker (trigger signal) ──────────────────────── -->
<div id="array-idx-picker" style="display:none">
<div class="ctx-menu-header">Element index: <span id="aip-sig" class="ctx-menu-key"></span></div>
<div class="ctx-menu-header">Element index:
<span id="aip-sig" class="ctx-menu-key"></span></div>
<div class="ctx-row">
<label>Index</label>
<input type="number" id="aip-idx" class="ctx-num" min="0" step="1" value="0">
@@ -175,7 +225,8 @@
<!-- ── VScale toolbar (moved into plot card when active) ─────────── -->
<div id="vscale-menu" style="display:none">
<div class="vstb-header">
<span class="vstb-label">V-Scale: <span id="vscale-menu-key" class="ctx-menu-key"></span></span>
<span class="vstb-label"><span id="vscale-menu-title">V-Scale</span>:
<span id="vscale-menu-key" class="ctx-menu-key"></span></span>
<div class="ctx-btns" id="vscale-mode-btns">
<button class="ctx-btn active" data-mode="auto">Auto</button>
<button class="ctx-btn" data-mode="range">Range</button>
@@ -185,9 +236,9 @@
<label class="vstb-lbl">V/div</label>
<input type="number" id="vscale-vdiv" class="ctx-num" min="1e-30" step="any" value="1">
</div>
<div id="vscale-pos-row" style="display:none;align-items:center;gap:4px">
<label class="vstb-lbl">Pos</label>
<input type="number" id="vscale-pos" class="ctx-num" step="0.1" value="0">
<div id="vscale-offset-row" style="display:none;align-items:center;gap:4px">
<label class="vstb-lbl" title="Raw value at screen centre — unbounded, may lie outside the plotted range">Offset</label>
<input type="number" id="vscale-offset" class="ctx-num" step="any" value="0">
</div>
<div id="vscale-type-row" style="display:none;align-items:center;gap:4px">
<label class="vstb-lbl">Type</label>
@@ -196,9 +247,23 @@
<button class="ctx-btn" data-type="digital">Digital</button>
</div>
</div>
<div class="vstb-sep"></div>
<div id="vscale-cal-row" style="display:flex;align-items:center;gap:4px">
<label class="vstb-lbl" id="vscale-cal-lbl" title="Data calibration: value = raw × Scale + Offset. Applies to the plot, cursors, hover readout, CSV export and trigger threshold.">Cal</label>
<label class="vstb-lbl">Scale</label>
<input type="number" id="vscale-cal-scale" class="ctx-num ctx-num-sm" step="any" value="1">
<label class="vstb-lbl">Offset</label>
<input type="number" id="vscale-cal-offset" class="ctx-num ctx-num-sm" step="any" value="0">
<label class="vstb-lbl">Unit</label>
<input type="text" id="vscale-cal-unit" class="ctx-num ctx-num-xs" placeholder="—">
<button class="ctx-btn" id="btn-cal-reset" title="Clear this signal's calibration">Reset</button>
</div>
<button id="btn-vscale-close" class="vstb-close" title="Close">✕</button>
</div>
</div>
<!-- Follows the mouse over a plot: time + per-trace values. -->
<div id="hover-readout" style="display:none"></div>
<script src="/calibration.js"></script>
<script src="/app.js"></script>
</body>
</html>
Client/udpstreamer/static/lttb-worker.js
-39
View File
@@ -1,39 +0,0 @@
'use strict';
// LTTB (Largest Triangle Three Buckets) decimation — O(n).
// Runs off-main-thread to avoid blocking the render loop.
function lttb(t, v, threshold) {
const len = t.length;
if (len <= threshold) {
// Copy to new arrays so we can transfer them back without detaching the input.
return { t: new Float64Array(t), v: new Float64Array(v) };
}
const outT = new Float64Array(threshold);
const outV = new Float64Array(threshold);
outT[0] = t[0]; outV[0] = v[0];
outT[threshold - 1] = t[len - 1]; outV[threshold - 1] = v[len - 1];
const every = (len - 2) / (threshold - 2);
let a = 0;
for (let i = 0; i < threshold - 2; i++) {
const avgS = Math.floor((i + 1) * every) + 1;
const avgE = Math.min(Math.floor((i + 2) * every) + 1, len);
let avgT = 0, avgV = 0, n = 0;
for (let j = avgS; j < avgE; j++) { avgT += t[j]; avgV += v[j]; n++; }
if (n) { avgT /= n; avgV /= n; }
const rS = Math.floor(i * every) + 1;
const rE = Math.min(Math.floor((i + 1) * every) + 1, len);
let maxA = -1, next = rS;
const aT = t[a], aV = v[a];
for (let j = rS; j < rE; j++) {
const area = Math.abs((aT - avgT) * (v[j] - aV) - (aT - t[j]) * (avgV - aV));
if (area > maxA) { maxA = area; next = j; }
}
outT[i + 1] = t[next]; outV[i + 1] = v[next]; a = next;
}
return { t: outT, v: outV };
}
self.onmessage = function({ data: { id, t, v, threshold } }) {
const result = lttb(t, v, threshold);
// Transfer the output buffers back to the main thread zero-copy.
self.postMessage({ id, t: result.t, v: result.v }, [result.t.buffer, result.v.buffer]);
};
Client/udpstreamer/static/style.css
+73 -19
View File
@@ -52,6 +52,23 @@ html, body { height:100%; background:var(--bg); color:var(--text);
#cursor-readout.visible { display:flex; }
#cur-ta { color:var(--sky); } #cur-tb { color:var(--yellow); }
#cur-dt { color:var(--subtext1); } .cur-sep { color:var(--surface2); }
#ruler-readout { display:inline-flex; align-items:center; gap:8px; }
#cur-y1 { color:var(--green); } #cur-y2 { color:var(--red); }
#cur-dy { color:var(--subtext1); }
/* Mouse-over time/value tooltip */
#hover-readout {
position:fixed; z-index:60; pointer-events:none;
background:var(--surface0); border:1px solid var(--surface1);
border-radius:5px; padding:4px 8px;
font-size:11px; font-family:monospace; white-space:nowrap;
box-shadow:0 4px 12px rgba(0,0,0,0.45);
}
#hover-readout .hov-time { color:var(--subtext1); margin-bottom:3px; }
#hover-readout .hov-row { display:flex; align-items:center; gap:6px; }
#hover-readout .hov-dot { width:8px; height:8px; border-radius:50%; flex-shrink:0; }
#hover-readout .hov-name { color:var(--subtext0); }
#hover-readout .hov-val { color:var(--text); margin-left:auto; padding-left:10px; }
.topbar-vsep { width:1px; height:22px; background:var(--surface0); flex-shrink:0; margin:0 2px; }
#layout-btns { display:flex; gap:2px; align-items:center; flex-shrink:0; }
@@ -74,6 +91,12 @@ button.ctrl-btn.trig-active { background:rgba(203,166,247,0.15); border-color:va
button.ctrl-btn.cursor-a { border-color:var(--sky); color:var(--sky); }
button.ctrl-btn.cursor-b { border-color:var(--yellow); color:var(--yellow); }
button.ctrl-btn.resume-btn { border-color:var(--teal); color:var(--teal); }
label.ctrl-check {
display:flex; align-items:center; gap:4px; flex-shrink:0;
font-size:12px; color:var(--subtext0); cursor:pointer; white-space:nowrap;
}
label.ctrl-check input { margin:0; cursor:pointer; accent-color:var(--accent); }
label.ctrl-check:has(input:checked) { color:var(--accent); }
/* ── Trigger bar ──────────────────────────────────────────────── */
#trigbar {
@@ -118,10 +141,17 @@ input[type=range].trig-range::-webkit-slider-thumb {
#trig-status-badge.armed { background:rgba(166,227,161,0.12); border-color:var(--green); color:var(--green); }
#trig-status-badge.waiting { background:rgba(249,226,175,0.12); border-color:var(--yellow); color:var(--yellow); }
#trig-status-badge.triggered { background:rgba(203,166,247,0.15); border-color:var(--mauve); color:var(--mauve); }
#btn-trig-rearm, #btn-trig-stop {
#btn-trig-force, #btn-trig-rearm, #btn-trig-stop {
border:none; border-radius:5px;
padding:4px 12px; font-size:12px; font-weight:600; cursor:pointer; display:none;
padding:4px 12px; font-size:12px; font-weight:600; cursor:pointer;
}
#btn-trig-rearm, #btn-trig-stop { display:none; }
#btn-trig-force {
background:var(--surface0); color:var(--text);
border:1px solid var(--surface1);
transition:background var(--transition),border-color var(--transition),color var(--transition);
}
#btn-trig-force:hover { background:var(--surface1); border-color:var(--mauve); color:var(--mauve); }
#btn-trig-rearm { background:var(--mauve); color:var(--crust); }
#btn-trig-stop { background:var(--surface1); color:var(--yellow); border:1px solid var(--yellow); }
#btn-trig-rearm:hover, #btn-trig-stop:hover { opacity:0.85; }
@@ -169,23 +199,6 @@ input[type=range].trig-range::-webkit-slider-thumb {
.sig-name { flex:1; font-size:13px; color:var(--text); overflow:hidden; text-overflow:ellipsis; white-space:nowrap; }
.sig-unit { font-size:11px; color:var(--subtext0); font-style:italic; }
.type-badge { font-size:10px; background:var(--surface1); color:var(--subtext1); padding:1px 5px; border-radius:3px; white-space:nowrap; }
.array-group {}
.array-header {
padding:6px 14px 6px 10px; cursor:pointer; border-radius:6px; margin:1px 6px;
transition:background var(--transition); display:flex; align-items:center; gap:6px; user-select:none;
}
.array-header:hover { background:var(--surface0); }
.array-arrow { font-size:10px; color:var(--subtext0); transition:transform var(--transition); display:inline-block; }
.array-header.open .array-arrow { transform:rotate(90deg); }
.array-children { display:none; padding-left:16px; }
.array-header.open + .array-children { display:block; }
.array-child {
padding:4px 14px 4px 8px; cursor:grab; border-radius:6px; margin:1px 6px;
transition:background var(--transition); display:flex; align-items:center; gap:8px;
user-select:none; color:var(--subtext1); font-size:12px;
}
.array-child:hover { background:var(--surface0); }
.array-child:active { cursor:grabbing; }
/* ── Main area ────────────────────────────────────────────────── */
#main { flex:1; display:flex; flex-direction:column; overflow:hidden; min-width:0; }
@@ -301,6 +314,32 @@ input[type=range].trig-range::-webkit-slider-thumb {
border:1px solid var(--mauve);
}
/* ── History budget ───────────────────────────────────────────── */
#history-badge {
font-size:10px; color:var(--yellow); margin-left:8px; cursor:pointer;
background:transparent; border:1px solid transparent; border-radius:4px;
padding:1px 5px; white-space:nowrap;
}
#history-badge:hover { border-color:var(--yellow); background:rgba(249,226,175,0.10); }
#history-panel {
position:fixed; z-index:300;
background:var(--mantle); border:1px solid var(--surface1); border-radius:var(--radius);
box-shadow:0 8px 24px rgba(0,0,0,0.6); padding:10px; width:290px;
}
.hist-note { font-size:10px; color:var(--overlay0); line-height:1.4; margin:6px 0; }
.hist-warn { color:var(--peach); }
#hist-signal-res {
font-size:10px; font-family:monospace; color:var(--subtext0);
max-height:120px; overflow-y:auto;
border-top:1px solid var(--surface0); border-bottom:1px solid var(--surface0);
padding:5px 0;
}
.hist-res-row { display:flex; justify-content:space-between; gap:8px; }
.hist-res-row .hist-res-key {
overflow:hidden; text-overflow:ellipsis; white-space:nowrap; color:var(--subtext1);
}
.hist-res-row .hist-res-val { color:var(--mauve); flex-shrink:0; }
/* ── Signal style context menu ────────────────────────────────── */
#sig-ctx-menu {
position:fixed; z-index:300;
@@ -363,6 +402,11 @@ input[type=range].trig-range::-webkit-slider-thumb {
}
.vstb-close:hover { color:var(--red); }
.plot-vscale-bar { display:none; }
.vstb-sep { width:1px; height:16px; background:var(--surface1); flex-shrink:0; }
.ctx-num-sm { width:70px; }
.ctx-num-xs { width:46px; }
#vscale-cal-lbl { color:var(--mauve); font-weight:600; }
.cal-invalid { border-color:var(--red) !important; }
/* ── Per-plot cursor value readout (in plot card header) ────────── */
.plot-cursor-ro {
@@ -450,6 +494,16 @@ input[type=range].trig-range::-webkit-slider-thumb {
.add-src-btn:hover { background:rgba(137,180,250,0.15); border-color:var(--accent); }
.save-src-btn { color:var(--green); }
.save-src-btn:hover { background:rgba(166,227,161,0.1); border-color:var(--green); }
.cfg-btn-row { display:flex; gap:6px; }
.cfg-btn-row .add-src-btn { flex:1; }
.reload-src-btn { color:var(--peach); }
.reload-src-btn:hover { background:rgba(250,179,135,0.1); border-color:var(--peach); }
.cfg-status {
font-size:10px; line-height:1.3; padding:2px 0; min-height:13px;
overflow-wrap:anywhere;
}
.cfg-status.ok { color:var(--green); }
.cfg-status.err { color:var(--red); }
/* ── Stats panel ─────────────────────────────────────────────── */
#stats-panel {
Client/udpstreamer/test/calibration.test.js
+158
View File
@@ -0,0 +1,158 @@
const test = require('node:test');
const assert = require('node:assert');
const C = require('../static/calibration.js');
test('baseSignalName strips an element suffix', () => {
assert.strictEqual(C.baseSignalName('Adc'), 'Adc');
assert.strictEqual(C.baseSignalName('Adc[3]'), 'Adc');
assert.strictEqual(C.baseSignalName('Adc[12]'), 'Adc');
assert.strictEqual(C.baseSignalName('A[1]B'), 'A[1]B');
assert.strictEqual(C.baseSignalName(''), '');
// A name that is entirely the suffix "[0]" must reduce to the empty string,
// matching Go (arrayIndexSuffix regexp) and C++ (strchr truncation) behaviour.
assert.strictEqual(C.baseSignalName('[0]'), '');
});
test('calKey is stable and separates the two fields', () => {
assert.strictEqual(C.calKey('a', 'b'), C.calKey('a', 'b'));
assert.notStrictEqual(C.calKey('ab', 'c'), C.calKey('a', 'bc'));
});
test('normaliseCal accepts a valid entry and fills defaults', () => {
assert.deepStrictEqual(
C.normaliseCal({source: ' wave ', signal: ' Adc ', scale: 2, offset: -1, unit: ' V '}),
{source: 'wave', signal: 'Adc', scale: 2, offset: -1, unit: 'V'});
assert.deepStrictEqual(
C.normaliseCal({source: 'wave', signal: 'Adc'}),
{source: 'wave', signal: 'Adc', scale: 1, offset: 0, unit: ''});
});
test('normaliseCal strips an element suffix from the signal name', () => {
assert.strictEqual(C.normaliseCal({source: 'w', signal: 'Adc[3]'}).signal, 'Adc');
});
test('normaliseCal truncates an over-long unit', () => {
const long = 'abcdefghijklmnopqrstuvwxyz';
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: long}).unit,
long.slice(0, C.MAX_UNIT_LEN));
});
test('normaliseCal leaves short non-ASCII units untouched', () => {
// 'Ω' is U+03A9, 2 UTF-8 bytes — well within 16 bytes.
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: 'Ω'}).unit, 'Ω');
// 'µs' is U+00B5 + U+0073, 3 UTF-8 bytes.
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: 'µs'}).unit, 'µs');
// '°C' is U+00B0 + U+0043, 3 UTF-8 bytes.
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: '°C'}).unit, '°C');
});
test('normaliseCal truncates an over-long ASCII unit to exactly 16 bytes', () => {
// 20 ASCII characters — each 1 byte, so cut at character 16.
const long = 'abcdefghijklmnopqrst'; // 20 chars
const result = C.normaliseCal({source: 'w', signal: 's', unit: long}).unit;
assert.strictEqual(result, 'abcdefghijklmnop'); // first 16 bytes/chars
assert.strictEqual(new TextEncoder().encode(result).length, 16);
});
test('normaliseCal cuts a mid-rune byte boundary back to the last complete rune', () => {
// Each 'Ω' (U+03A9) is 2 UTF-8 bytes (CE A9).
// 8 × 'Ω' = 16 bytes exactly — fits without truncation.
const fits = 'ΩΩΩΩΩΩΩΩ';
assert.strictEqual(new TextEncoder().encode(fits).length, 16);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: fits}).unit, fits);
// 9 × 'Ω' = 18 bytes. Slicing at 16 bytes lands in the middle of the 9th
// 'Ω' (only 1 of its 2 bytes is in the window) so only 8 'Ω' should survive.
// No U+FFFD replacement character must appear.
const toolong = 'ΩΩΩΩΩΩΩΩΩ';
const result = C.normaliseCal({source: 'w', signal: 's', unit: toolong}).unit;
assert.strictEqual(result, 'ΩΩΩΩΩΩΩΩ');
assert.ok(!result.includes('\uFFFD'), 'must not contain U+FFFD replacement character');
assert.strictEqual(new TextEncoder().encode(result).length, 16);
});
test('normaliseCal leaves a unit that is exactly 16 bytes ending on a complete multi-byte rune untouched', () => {
// 'abcdefgΩhijklµ' → 7 ASCII + 'Ω' (2 bytes) + 5 ASCII + 'µ' (2 bytes) = 16 bytes
const u = 'abcdefgΩhijklµ';
assert.strictEqual(new TextEncoder().encode(u).length, 16);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: u}).unit, u);
});
test('normaliseCal rejects invalid entries', () => {
assert.strictEqual(C.normaliseCal(null), null);
assert.strictEqual(C.normaliseCal({signal: 's'}), null);
assert.strictEqual(C.normaliseCal({source: 'w'}), null);
assert.strictEqual(C.normaliseCal({source: ' ', signal: 's'}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', scale: 0}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', scale: NaN}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', scale: Infinity}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', offset: NaN}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', scale: '2'}), null);
// A signal name that is entirely an array-index suffix reduces to the empty
// string after stripping, so the entry must be rejected — matching Go and C++.
assert.strictEqual(C.normaliseCal({source: 'w', signal: '[0]'}), null);
});
test('applyCal and invertCal round-trip', () => {
const cal = {scale: 0.5, offset: -1.25, unit: 'V'};
assert.strictEqual(C.applyCal(10, cal), 3.75);
assert.strictEqual(C.invertCal(3.75, cal), 10);
assert.strictEqual(C.applyCal(7, C.IDENTITY), 7);
assert.strictEqual(C.invertCal(7, C.IDENTITY), 7);
});
test('applyCal passes non-finite samples through untouched', () => {
assert.ok(Number.isNaN(C.applyCal(NaN, {scale: 2, offset: 1, unit: ''})));
});
test('calRange re-orders when the scale is negative', () => {
assert.deepStrictEqual(C.calRange(0, 10, {scale: 2, offset: 1, unit: ''}), [1, 21]);
assert.deepStrictEqual(C.calRange(0, 10, {scale: -2, offset: 1, unit: ''}), [-19, 1]);
});
test('CalTable.get returns IDENTITY for an unknown signal', () => {
const t = new C.CalTable();
assert.deepStrictEqual(t.get('w', 'Adc'), C.IDENTITY);
});
test('CalTable.get resolves an element name to its base signal', () => {
const t = new C.CalTable();
t.set({source: 'w', signal: 'Adc', scale: 3, offset: 0, unit: ''});
assert.strictEqual(t.get('w', 'Adc[7]').scale, 3);
});
test('CalTable.set stores, overwrites, and deletes identity entries', () => {
const t = new C.CalTable();
assert.strictEqual(t.set({source: 'w', signal: 'Adc', scale: 2}), true);
assert.strictEqual(t.get('w', 'Adc').scale, 2);
t.set({source: 'w', signal: 'Adc', scale: 5});
assert.strictEqual(t.get('w', 'Adc').scale, 5);
assert.strictEqual(t.list().length, 1);
// Resetting to identity removes the entry entirely.
assert.strictEqual(t.set({source: 'w', signal: 'Adc', scale: 1, offset: 0, unit: ''}), true);
assert.strictEqual(t.list().length, 0);
// An invalid entry is refused and changes nothing.
assert.strictEqual(t.set({source: 'w', signal: 'Adc', scale: 0}), false);
assert.strictEqual(t.list().length, 0);
});
test('CalTable.replaceAll drops the previous contents', () => {
const t = new C.CalTable();
t.set({source: 'w', signal: 'Old', scale: 2});
t.replaceAll([
{source: 'w', signal: 'B', scale: 2},
{source: 'w', signal: 'A', scale: 3},
{source: 'w', signal: 'Bad', scale: 0},
{source: 'w', signal: 'Ident', scale: 1, offset: 0, unit: ''},
]);
assert.deepStrictEqual(t.list().map(e => e.signal), ['A', 'B']);
});
test('CalTable.list is sorted by source then signal', () => {
const t = new C.CalTable();
t.set({source: 'z', signal: 'a', scale: 2});
t.set({source: 'a', signal: 'z', scale: 2});
t.set({source: 'a', signal: 'b', scale: 2});
assert.deepStrictEqual(t.list().map(e => e.source + '/' + e.signal),
['a/b', 'a/z', 'z/a']);
});
Common/Client/c/.gitignore
+5
View File
@@ -0,0 +1,5 @@
*.o
*.a
udps_dump
.cxxcheck
.cxxcheck.cpp
Common/Client/c/Makefile
+46
View File
@@ -0,0 +1,46 @@
# UDPS C client library — standalone, no MARTe2, no external dependencies.
#
# make build libudpsclient.a and the example
# make example build only the example
# make cxxcheck verify the header is usable from C++
# make clean
CC ?= cc
CXX ?= c++
AR ?= ar
CFLAGS ?= -O2 -g
WARN = -Wall -Wextra -Wpedantic
STD = -std=c99
CPPFLAGS += -I.
# Old glibc (< 2.17) keeps clock_gettime in librt; harmless to add there.
LDLIBS ?=
LIB = libudpsclient.a
OBJ = udps_client.o
EXAMPLE = udps_dump
.PHONY: all example cxxcheck clean
all: $(LIB) $(EXAMPLE)
$(LIB): $(OBJ)
$(AR) rcs $@ $^
udps_client.o: udps_client.c udps_client.h
$(CC) $(STD) $(WARN) $(CFLAGS) $(CPPFLAGS) -c -o $@ $<
example: $(EXAMPLE)
$(EXAMPLE): example/udps_dump.c $(LIB)
$(CC) $(STD) $(WARN) $(CFLAGS) $(CPPFLAGS) -o $@ $< $(LIB) $(LDLIBS)
# The header is C++-safe; this target keeps it that way.
cxxcheck: udps_client.h
echo '#include "udps_client.h"' > .cxxcheck.cpp
echo 'int main() { udps_client_config_t c; udps_client_config_init(&c); return 0; }' >> .cxxcheck.cpp
$(CXX) -std=c++11 -Wall -Wextra $(CPPFLAGS) -o .cxxcheck .cxxcheck.cpp $(LIB) $(LDLIBS)
./.cxxcheck && rm -f .cxxcheck .cxxcheck.cpp
clean:
rm -f $(LIB) $(OBJ) $(EXAMPLE) .cxxcheck .cxxcheck.cpp
Common/Client/c/example/udps_dump.c
+260
View File
@@ -0,0 +1,260 @@
/**
* @file udps_dump.c
* @brief Example UDPS client: connects to a UDPStreamer and prints what arrives.
*
* Build with the Makefile in the parent directory, then for a unicast stream:
*
* ./udps_dump --host 127.0.0.1 --port 44500
*
* or, for a multicast one:
*
* ./udps_dump --host 127.0.0.1 --port 44500 \
* --multicast 239.0.0.1 --iface 127.0.0.1
*
* Ctrl-C prints a summary of what was received.
*/
#define _POSIX_C_SOURCE 200809L
#include "udps_client.h"
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
static volatile sig_atomic_t g_stop = 0;
static void on_sigint(int sig) {
(void)sig;
g_stop = 1;
}
typedef struct {
double print_interval; /**< Seconds between frame printouts. */
double last_print;
uint64_t frames;
uint64_t max_frames;
} dump_state_t;
static double now_wall(void) {
struct timespec ts;
clock_gettime(CLOCK_REALTIME, &ts);
return (double)ts.tv_sec + (double)ts.tv_nsec * 1e-9;
}
static const char *time_mode_name(uint8_t m) {
switch (m) {
case UDPS_TIME_PACKET: return "packet";
case UDPS_TIME_FULL_ARRAY: return "full-array";
case UDPS_TIME_FIRST_SAMPLE: return "first-sample";
case UDPS_TIME_LAST_SAMPLE: return "last-sample";
default: return "?";
}
}
static const char *publish_mode_name(uint8_t m) {
switch (m) {
case UDPS_PUBLISH_STRICT: return "strict";
case UDPS_PUBLISH_ACCUMULATE: return "accumulate";
case UDPS_PUBLISH_DECIMATE: return "decimate";
default: return "?";
}
}
static void on_config(const udps_signal_t *sigs, uint32_t n, uint8_t mode,
void *user) {
uint32_t i;
(void)user;
printf("\nCONFIG: %u signal(s), publish mode %s\n", (unsigned)n,
publish_mode_name(mode));
printf(" %-3s %-24s %-8s %-10s %-8s %-10s %s\n", "#", "name", "type",
"shape", "unit", "rate[Hz]", "time-mode");
for (i = 0u; i < n; i++) {
char shape[32];
const udps_signal_t *s = &sigs[i];
if (s->num_cols > 1u) {
snprintf(shape, sizeof shape, "%ux%u", (unsigned)s->num_rows,
(unsigned)s->num_cols);
} else {
snprintf(shape, sizeof shape, "%u",
(unsigned)udps_signal_num_elements(s));
}
printf(" %-3u %-24s %-8s %-10s %-8s %-10.6g %s%s\n", (unsigned)i,
s->name, udps_type_name(s->type_code), shape,
(s->unit[0] != '\0') ? s->unit : "-", s->sampling_rate,
time_mode_name(s->time_mode),
(s->quant_type != UDPS_QUANT_NONE) ? " (quantised)" : "");
}
fflush(stdout);
}
static void on_data(const udps_frame_t *f, void *user) {
dump_state_t *st = (dump_state_t *)user;
uint32_t i;
double now;
st->frames++;
now = now_wall();
if ((now - st->last_print) < st->print_interval) {
return; /* Streams run far faster than a terminal can be read. */
}
st->last_print = now;
printf("\nframe #%lu t=%.6f samples=%u (%lu frames so far)\n",
(unsigned long)f->counter, f->recv_time, (unsigned)f->num_samples,
(unsigned long)st->frames);
for (i = 0u; i < f->num_signals; i++) {
const double *v = f->values[i].values;
uint32_t cnt = f->values[i].count;
double lo, hi;
uint32_t k;
if (cnt == 0u) {
continue;
}
lo = hi = v[0];
for (k = 1u; k < cnt; k++) {
if (v[k] < lo) {
lo = v[k];
}
if (v[k] > hi) {
hi = v[k];
}
}
printf(" %-24s n=%-6u first=%-12.6g last=%-12.6g min=%-12.6g max=%-12.6g %s\n",
f->signals[i].name, (unsigned)cnt, v[0], v[cnt - 1u], lo, hi,
f->signals[i].unit);
}
fflush(stdout);
}
static void on_event(udps_event_t ev, const char *detail, void *user) {
(void)user;
switch (ev) {
case UDPS_EVENT_CONNECTED:
printf("[connected to %s]\n", detail ? detail : "");
break;
case UDPS_EVENT_DISCONNECTED:
printf("[disconnected: %s]\n", detail ? detail : "");
break;
case UDPS_EVENT_ERROR:
fprintf(stderr, "[error] %s\n", detail ? detail : "");
break;
default:
break;
}
fflush(stdout);
}
static void usage(const char *argv0) {
printf("Usage: %s --host ADDR --port N [options]\n"
"\n"
" --host ADDR server address (default 127.0.0.1)\n"
" --port N server UDP port, or TCP control port in multicast\n"
" mode (default 44500)\n"
" --multicast GROUP join GROUP for data instead of unicast\n"
" --iface ADDR local interface address for the multicast join\n"
" --data-port N multicast data port (default: --port + 1)\n"
" --silence SEC reconnect after SEC without data (default 1, 0 off)\n"
" --interval SEC seconds between printouts (default 1)\n"
" --frames N exit after N frames (default: run until Ctrl-C)\n"
" --help this text\n",
argv0);
}
int main(int argc, char **argv) {
udps_client_config_t cfg;
udps_client_t *cli;
dump_state_t st;
udps_stats_t stats;
struct sigaction sa;
const char *host = "127.0.0.1";
int i;
udps_client_config_init(&cfg);
cfg.server_port = 44500u;
memset(&st, 0, sizeof st);
st.print_interval = 1.0;
for (i = 1; i < argc; i++) {
const char *a = argv[i];
const char *next = (i + 1 < argc) ? argv[i + 1] : NULL;
if (strcmp(a, "--help") == 0) {
usage(argv[0]);
return 0;
}
if (next == NULL) {
fprintf(stderr, "missing value for %s\n", a);
return 2;
}
if (strcmp(a, "--host") == 0) {
host = next;
} else if (strcmp(a, "--port") == 0) {
cfg.server_port = (uint16_t)atoi(next);
} else if (strcmp(a, "--multicast") == 0) {
cfg.multicast_group = next;
} else if (strcmp(a, "--iface") == 0) {
cfg.interface_addr = next;
} else if (strcmp(a, "--data-port") == 0) {
cfg.data_port = (uint16_t)atoi(next);
} else if (strcmp(a, "--silence") == 0) {
cfg.silence_timeout_s = atof(next);
} else if (strcmp(a, "--interval") == 0) {
st.print_interval = atof(next);
} else if (strcmp(a, "--frames") == 0) {
st.max_frames = (uint64_t)strtoull(next, NULL, 10);
} else {
fprintf(stderr, "unknown option %s\n", a);
usage(argv[0]);
return 2;
}
i++;
}
cfg.server_addr = host;
cli = udps_client_create(&cfg);
if (cli == NULL) {
fprintf(stderr, "could not create client for %s:%u\n", host,
(unsigned)cfg.server_port);
return 1;
}
udps_client_set_callbacks(cli, on_config, on_data, on_event, &st);
memset(&sa, 0, sizeof sa);
sa.sa_handler = on_sigint;
(void)sigaction(SIGINT, &sa, NULL);
(void)sigaction(SIGTERM, &sa, NULL);
printf("listening to %s:%u%s%s ... (Ctrl-C to stop)\n", host,
(unsigned)cfg.server_port,
cfg.multicast_group ? " via multicast " : "",
cfg.multicast_group ? cfg.multicast_group : "");
while (!g_stop && (st.max_frames == 0u || st.frames < st.max_frames)) {
/* All the work — connecting, receiving, decoding, reconnecting — and
* every callback happens inside this call. */
(void)udps_client_poll(cli, 200);
}
udps_client_stats(cli, &stats);
printf("\n--- summary ---\n"
"packets %lu\n"
"bytes %.1f MiB\n"
"frames %lu\n"
"configs %lu\n"
"gaps %lu (datagrams lost)\n"
"dropped %lu (fragments)\n"
"reconnects %lu\n",
(unsigned long)stats.packets_received,
(double)stats.bytes_received / (1024.0 * 1024.0),
(unsigned long)stats.frames_delivered,
(unsigned long)stats.config_updates,
(unsigned long)stats.counter_gaps,
(unsigned long)stats.fragments_dropped,
(unsigned long)stats.reconnects);
udps_client_destroy(cli);
return 0;
}
Common/Client/c/udps_client.c
+1209
View File
File diff suppressed because it is too large Load Diff
Common/Client/c/udps_client.h
+324
View File
@@ -0,0 +1,324 @@
#ifndef UDPS_CLIENT_H
#define UDPS_CLIENT_H
/**
* @file udps_client.h
* @brief Standalone UDPS (UDPStreamer) receiver library — C99, no MARTe2.
*
* Depends only on libc and BSD sockets, so it can be dropped into any C or C++
* program that needs to consume a UDPStreamer / DebugService stream. The wire
* format is specified in Docs/Protocol.md; the library reference (and a worked
* example) is Docs/UDPS-C-Client.md.
*
* Usage in one paragraph: fill a udps_client_config_t, create a client, install
* callbacks, then call udps_client_poll() in a loop. The client owns the
* connection state machine — it sends CONNECT, reassembles fragmented packets,
* decodes CONFIG and DATA, sends keepalives, and reconnects when the server
* goes silent. Nothing is done behind your back: no threads are created and
* every callback runs inside your call to udps_client_poll().
*
* Threading: a udps_client_t must be used from one thread at a time.
*/
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
/*---------------------------------------------------------------------------*/
/* Protocol constants */
/*---------------------------------------------------------------------------*/
/** Magic number: ASCII 'UDPS' stored little-endian. */
#define UDPS_MAGIC 0x53504455u
/** Size of the packed packet header on the wire. */
#define UDPS_HEADER_SIZE 17u
/** Size of one serialised signal descriptor in a CONFIG payload. */
#define UDPS_SIGNAL_DESC_SIZE 136u
/** Value of udps_signal_t::time_signal_idx when the signal has no time reference. */
#define UDPS_NO_TIME_SIGNAL 0xFFFFFFFFu
/** Upper bound on elements per signal; larger descriptors are rejected. */
#define UDPS_MAX_ELEMENTS (1u << 20)
/** Packet types (udps_header_t::type). */
enum {
UDPS_PKT_DATA = 0, /**< Server -> client: signal samples. */
UDPS_PKT_CONFIG = 1, /**< Server -> client: signal metadata. */
UDPS_PKT_ACK = 2, /**< Client -> server: keepalive. */
UDPS_PKT_CONNECT = 3, /**< Client -> server: open a session. */
UDPS_PKT_DISCONNECT = 4 /**< Either direction: close a session. */
};
/** Sample type codes (udps_signal_t::type_code). */
enum {
UDPS_T_UINT8 = 0,
UDPS_T_INT8 = 1,
UDPS_T_UINT16 = 2,
UDPS_T_INT16 = 3,
UDPS_T_UINT32 = 4,
UDPS_T_INT32 = 5,
UDPS_T_UINT64 = 6,
UDPS_T_INT64 = 7,
UDPS_T_FLOAT32 = 8,
UDPS_T_FLOAT64 = 9,
UDPS_T_UNKNOWN = 255
};
/** Quantisation codes (udps_signal_t::quant_type). */
enum {
UDPS_QUANT_NONE = 0, /**< Raw values in the signal's own type. */
UDPS_QUANT_UINT8 = 1, /**< [range_min, range_max] mapped onto uint8. */
UDPS_QUANT_INT8 = 2,
UDPS_QUANT_UINT16 = 3,
UDPS_QUANT_INT16 = 4
};
/** Time-reference modes (udps_signal_t::time_mode). */
enum {
UDPS_TIME_PACKET = 0, /**< No per-element time; use packet arrival. */
UDPS_TIME_FULL_ARRAY = 1, /**< The time signal carries one stamp per element. */
UDPS_TIME_FIRST_SAMPLE = 2, /**< Time signal (scalar) stamps element 0. */
UDPS_TIME_LAST_SAMPLE = 3 /**< Time signal (scalar) stamps element N-1. */
};
/** Publishing modes (udps_frame_t::publish_mode). */
enum {
UDPS_PUBLISH_STRICT = 0, /**< One packet per RT cycle. */
UDPS_PUBLISH_ACCUMULATE = 1, /**< A batch of cycles per packet. */
UDPS_PUBLISH_DECIMATE = 2 /**< One packet every N cycles. */
};
/*---------------------------------------------------------------------------*/
/* Data model */
/*---------------------------------------------------------------------------*/
/** Decoded 17-byte packet header. */
typedef struct {
uint32_t magic;
uint8_t type;
uint32_t counter; /**< Same for every fragment of one update. */
uint16_t fragment_idx;
uint16_t total_fragments; /**< 1 when the update fits in one datagram. */
uint32_t payload_bytes;
} udps_header_t;
/** Metadata for one streamed signal, as carried by the CONFIG payload. */
typedef struct {
char name[65]; /**< NUL-terminated. */
uint8_t type_code; /**< UDPS_T_*. */
uint8_t quant_type; /**< UDPS_QUANT_*. */
uint8_t num_dimensions; /**< 0 scalar, 1 vector, 2 matrix. */
uint32_t num_rows;
uint32_t num_cols;
double range_min; /**< Physical range, used to dequantise. */
double range_max;
uint8_t time_mode; /**< UDPS_TIME_*. */
double sampling_rate; /**< Hz; 0 when unknown. */
uint32_t time_signal_idx;/**< Index into the signal list, or UDPS_NO_TIME_SIGNAL. */
char unit[33]; /**< NUL-terminated. */
} udps_signal_t;
/**
* @brief Decoded values of one signal within a frame.
*
* Values are always physical doubles: quantised signals are already expanded
* back onto [range_min, range_max]. @c count is @c num_samples for a scalar
* signal in Accumulate mode (one value per batched cycle) and the signal's
* element count in every other case.
*/
typedef struct {
const double *values;
uint32_t count;
} udps_signal_values_t;
/** One fully decoded DATA packet. */
typedef struct {
uint32_t counter; /**< Packet counter; gaps mean lost datagrams. */
uint64_t hrt; /**< Producer's high-resolution timer at send. */
double recv_time; /**< Wall-clock seconds (CLOCK_REALTIME) at arrival. */
uint8_t publish_mode; /**< UDPS_PUBLISH_*. */
uint32_t num_samples; /**< Batched cycles; 1 unless Accumulate. */
uint32_t num_signals;
const udps_signal_t *signals; /**< num_signals entries, CONFIG order. */
const udps_signal_values_t *values; /**< num_signals entries, same order. */
} udps_frame_t;
/** Connection lifecycle events reported through udps_event_cb. */
typedef enum {
UDPS_EVENT_CONNECTED, /**< Sockets are up and CONNECT was sent. */
UDPS_EVENT_DISCONNECTED, /**< Session dropped; the client will retry. */
UDPS_EVENT_ERROR /**< Recoverable problem; detail says what. */
} udps_event_t;
/** Cumulative counters, never reset. */
typedef struct {
uint64_t packets_received; /**< Datagrams (and TCP frames) accepted. */
uint64_t bytes_received;
uint64_t frames_delivered; /**< DATA packets decoded and handed to you. */
uint64_t config_updates;
uint64_t fragments_dropped; /**< Duplicate, stale or unplaceable fragments. */
uint64_t counter_gaps; /**< DATA packets missing from the sequence. */
uint64_t reconnects;
} udps_stats_t;
/*---------------------------------------------------------------------------*/
/* Client */
/*---------------------------------------------------------------------------*/
typedef struct udps_client udps_client_t;
/** Called whenever a CONFIG packet redefines the signal set. */
typedef void (*udps_config_cb)(const udps_signal_t *signals,
uint32_t num_signals,
uint8_t publish_mode,
void *user);
/**
* @brief Called for every decoded DATA packet.
*
* The frame and everything it points at are owned by the client and are only
* valid until the callback returns — copy anything you need to keep.
*/
typedef void (*udps_data_cb)(const udps_frame_t *frame, void *user);
/** Called on connection state changes and on recoverable errors. */
typedef void (*udps_event_cb)(udps_event_t event, const char *detail, void *user);
/**
* @brief Transport configuration.
*
* Zero-initialise with udps_client_config_init(), then override what you need.
* Set @c multicast_group to switch from unicast to multicast: in unicast the
* client sends CONNECT over UDP and receives everything on its ephemeral port;
* in multicast it joins the group for DATA and opens a TCP control connection
* to @c server_port for CONNECT and CONFIG.
*/
typedef struct {
const char *server_addr; /**< IPv4 dotted quad. Required. */
uint16_t server_port; /**< UDP port (unicast) or TCP port (multicast). Required. */
const char *multicast_group;/**< IPv4 group; NULL selects unicast. */
const char *interface_addr; /**< Local IPv4 of the interface to join on. NULL = default route. */
uint16_t data_port; /**< Multicast data port; 0 means server_port + 1. */
double silence_timeout_s; /**< Reconnect after this long without data. 0 disables. */
double reconnect_delay_s; /**< Wait between reconnect attempts. */
double keepalive_interval_s;/**< Unicast ACK period. 0 disables. */
uint32_t recv_buffer_bytes; /**< SO_RCVBUF; large bursts need a large value. */
uint32_t max_packet_bytes; /**< Ceiling on one reassembled payload. */
} udps_client_config_t;
/** Fills @p cfg with the defaults documented in Docs/UDPS-C-Client.md. */
void udps_client_config_init(udps_client_config_t *cfg);
/**
* @brief Creates a client. No socket is opened until the first poll.
* @return NULL if @p cfg is invalid or memory ran out.
*/
udps_client_t *udps_client_create(const udps_client_config_t *cfg);
/** Closes the session (sending DISCONNECT if connected) and frees the client. */
void udps_client_destroy(udps_client_t *client);
/** Installs the callbacks. Any of them may be NULL. */
void udps_client_set_callbacks(udps_client_t *client,
udps_config_cb on_config,
udps_data_cb on_data,
udps_event_cb on_event,
void *user);
/**
* @brief Drives the client: connects if needed, then waits for and processes
* packets for at most @p timeout_ms milliseconds.
*
* Callbacks fire from inside this call. A negative @p timeout_ms blocks until
* something happens. Call it in a loop; it is the only function that does work.
*
* @return the number of packets processed (0 on timeout), or -1 if the session
* broke. -1 is not fatal: the next call retries after reconnect_delay_s.
*/
int udps_client_poll(udps_client_t *client, int timeout_ms);
/** Non-zero once the sockets are up (which does not yet imply CONFIG arrived). */
int udps_client_is_connected(const udps_client_t *client);
/**
* @brief The current signal set, or NULL before the first CONFIG.
* @param num_signals Out; may be NULL.
*/
const udps_signal_t *udps_client_signals(const udps_client_t *client,
uint32_t *num_signals);
/** The publishing mode from the last CONFIG (UDPS_PUBLISH_*). */
uint8_t udps_client_publish_mode(const udps_client_t *client);
/** Copies the counters into @p out. */
void udps_client_stats(const udps_client_t *client, udps_stats_t *out);
/** Human-readable description of the last failure. Never NULL. */
const char *udps_client_last_error(const udps_client_t *client);
/*---------------------------------------------------------------------------*/
/* Stateless helpers */
/*---------------------------------------------------------------------------*/
/** Elements in one sample of @p signal (rows x cols, at least 1). */
uint32_t udps_signal_num_elements(const udps_signal_t *signal);
/** Short name of a type code, e.g. "float32". Never NULL. */
const char *udps_type_name(uint8_t type_code);
/**
* @brief Decodes a packet header.
* @return 0 on success, -1 if @p len is too small or the magic is wrong.
*/
int udps_parse_header(const void *buf, size_t len, udps_header_t *out);
/**
* @brief Decodes a reassembled CONFIG payload.
* @param signals Out array of at most @p max_signals entries.
* @param num_signals Out; the number actually written.
* @param publish_mode Out; may be NULL.
* @return 0 on success, -1 if the payload is malformed or does not fit.
*/
int udps_parse_config(const void *payload,
size_t len,
udps_signal_t *signals,
uint32_t max_signals,
uint32_t *num_signals,
uint8_t *publish_mode);
/**
* @brief One value out of a frame.
* @param sample Accumulate batch slot; ignored for non-scalar signals.
* @param elem Element within the sample; ignored for accumulated scalars.
* @return the value, or 0.0 if any index is out of range.
*/
double udps_frame_value(const udps_frame_t *frame,
uint32_t signal_idx,
uint32_t sample,
uint32_t elem);
/**
* @brief Arrival-anchored estimate of the wall-clock time of one element.
*
* Exact only for streams that declare a sampling rate: the packet is assumed to
* have arrived as its last element was produced, and earlier elements are dated
* backwards by 1/sampling_rate. Signals with UDPS_TIME_PACKET, or without a
* sampling rate, all report the arrival time. When the stream carries a time
* signal (time_signal_idx != UDPS_NO_TIME_SIGNAL) that signal is the accurate
* source — read it like any other signal instead of using this helper.
*/
double udps_frame_element_time(const udps_frame_t *frame,
uint32_t signal_idx,
uint32_t elem);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UDPS_CLIENT_H */
Common/Client/go/udpsprotocol/protocol.go
+29 -1
View File
@@ -99,6 +99,20 @@ func BuildDisconnectPacket() []byte {
})
}
// BuildAckPacket returns a 17-byte ACK datagram. Unicast clients send it
// periodically as a keepalive: UDPSServer refreshes the client's last-seen
// without re-sending CONFIG (which a repeated CONNECT would trigger).
func BuildAckPacket() []byte {
return buildHeader(PacketHeader{
Magic: MagicUDPS,
Type: PktACK,
Counter: 0,
FragmentIdx: 0,
TotalFragments: 1,
PayloadBytes: 0,
})
}
// ─── Signal descriptor (136 bytes) ───────────────────────────────────────────
// SignalInfo holds the parsed metadata for one signal.
@@ -127,7 +141,12 @@ func (s SignalInfo) NumElements() int {
if c == 0 {
c = 1
}
return r * c
/* HI-2: cap at 1M to prevent integer overflow / OOM from crafted packets */
n := r * c
if n < 0 || n > 1024*1024 {
return 1024 * 1024
}
return n
}
// rawTypeSize returns the byte size for one element of the raw (unquantised) type.
@@ -227,6 +246,11 @@ func ParseConfig(payload []byte) ([]SignalInfo, uint8, error) {
return nil, 0, fmt.Errorf("config payload too short")
}
numSigs := binary.LittleEndian.Uint32(payload[0:4])
/* HI-2: validate numSigs against payload length before allocating */
maxSigs := uint32(len(payload) / SigDescSize)
if numSigs > maxSigs {
return nil, 0, fmt.Errorf("config claims %d signals but payload can hold at most %d", numSigs, maxSigs)
}
offset := 4
sigs := make([]SignalInfo, 0, numSigs)
for i := uint32(0); i < numSigs; i++ {
@@ -327,6 +351,10 @@ func ParseData(payload []byte, sigs []SignalInfo, publishMode uint8, arrivalTime
if numSamples == 0 {
return []DataSample{}, nil
}
/* HI-2: sanity-cap numSamples to prevent OOM from crafted packets */
if numSamples < 0 || numSamples > 1024*1024 {
return nil, fmt.Errorf("accumulate numSamples %d out of range", numSamples)
}
// Parse per-signal data blocks (all slots for a signal are contiguous).
accumVals := make(map[string][]float64, len(sigs)) // scalars: numSamples values
Common/Client/go/udpsprotocol/protocol_test.go
+97
View File
@@ -0,0 +1,97 @@
package udpsprotocol
import (
"encoding/binary"
"math"
"testing"
"time"
)
// TestParseConfig_HugeNumSigs_NoOOM — a CONFIG payload claiming 0xFFFFFFFF signals
// must return an error, not panic/OOM.
func TestParseConfig_HugeNumSigs_NoOOM(t *testing.T) {
// 4 bytes: numSigs = 0xFFFFFFFF, then nothing else
payload := make([]byte, 4)
binary.LittleEndian.PutUint32(payload[0:4], 0xFFFFFFFF)
sigs, _, err := ParseConfig(payload)
if err == nil {
t.Fatal("expected error for huge numSigs, got nil")
}
if sigs != nil {
t.Fatalf("expected nil sigs, got %d", len(sigs))
}
}
// TestParseConfig_ValidSmallConfig — a minimal valid CONFIG parses correctly.
func TestParseConfig_ValidSmallConfig(t *testing.T) {
// 1 signal, then publish mode
payload := make([]byte, 4+SigDescSize+1)
binary.LittleEndian.PutUint32(payload[0:4], 1)
// Set typeCode to float32 (8) at offset 64
payload[4+64] = 8
// numRows=1, numCols=1 at offsets 67, 71
binary.LittleEndian.PutUint32(payload[4+67:4+71], 1)
binary.LittleEndian.PutUint32(payload[4+71:4+75], 1)
// publish mode = 0 (Strict)
payload[4+SigDescSize] = 0
sigs, pm, err := ParseConfig(payload)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(sigs) != 1 {
t.Fatalf("expected 1 signal, got %d", len(sigs))
}
if pm != PublishModeStrict {
t.Fatalf("expected Strict mode, got %d", pm)
}
}
// TestNumElements_OverflowCapped — huge numRows*numCols is capped, no panic.
func TestNumElements_OverflowCapped(t *testing.T) {
s := SignalInfo{NumRows: 0xFFFFFFFF, NumCols: 0xFFFFFFFF}
n := s.NumElements()
if n <= 0 || n > 1024*1024 {
t.Fatalf("expected capped value 1M, got %d", n)
}
}
// TestNumElements_Normal — normal values work correctly.
func TestNumElements_Normal(t *testing.T) {
s := SignalInfo{NumRows: 3, NumCols: 4}
if n := s.NumElements(); n != 12 {
t.Fatalf("expected 12, got %d", n)
}
}
// TestParseData_HugeNumSamples_NoOOM — an Accumulate DATA packet with
// numSamples=0xFFFFFFFF must return an error, not OOM.
func TestParseData_HugeNumSamples_NoOOM(t *testing.T) {
sigs := []SignalInfo{
{Name: "test", TypeCode: 8, NumRows: 1, NumCols: 1, QuantType: QuantNone},
}
payload := make([]byte, 12)
binary.LittleEndian.PutUint64(payload[0:8], 0) // HRT
binary.LittleEndian.PutUint32(payload[8:12], 0xFFFFFFFF)
_, err := ParseData(payload, sigs, PublishModeAccumulate, time.Now())
if err == nil {
t.Fatal("expected error for huge numSamples, got nil")
}
}
// TestParseData_ValidStrict — a valid Strict DATA packet parses without error.
func TestParseData_ValidStrict(t *testing.T) {
sigs := []SignalInfo{
{Name: "test", TypeCode: 8, NumRows: 1, NumCols: 1, QuantType: QuantNone},
}
// 8 HRT + 4 bytes float32
payload := make([]byte, 12)
binary.LittleEndian.PutUint64(payload[0:8], 1000)
binary.LittleEndian.PutUint32(payload[8:12], math.Float32bits(3.14))
samples, err := ParseData(payload, sigs, PublishModeStrict, time.Now())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if len(samples) != 1 {
t.Fatalf("expected 1 sample, got %d", len(samples))
}
}
Common/Client/go/wshub/calibration.go
+234
View File
@@ -0,0 +1,234 @@
package wshub
import (
"encoding/json"
"log"
"math"
"regexp"
"sort"
"strings"
"sync"
"unicode/utf8"
)
// arrayIndexSuffix matches a trailing "[digits]" at the very end of a signal
// name, used to strip array-element suffixes so one entry covers the whole
// array. The regexp is anchored to the end of the string and requires digits,
// so it only removes a well-formed trailing element index: "Adc[3]" → "Adc",
// "[0]" → "", "A[1]B" → "A[1]B" (no match).
//
// Known difference vs C++: the C++ hub uses strchr(signal,'[') which finds the
// FIRST '[' anywhere in the name, so C++ reduces "A[1]B" to "A" while this
// regexp leaves it unchanged. Both implementations agree on the common cases
// ("Name[i]" and "[i]" alone) that arise from real UDPS signal names.
var arrayIndexSuffix = regexp.MustCompile(`\[\d+\]$`)
// maxUnitLen bounds the calibration unit override. Mirrored by kMaxUnitLen in
// the C++ StreamHub and MAX_UNIT_LEN in the SPA's calibration.js.
const maxUnitLen = 16
// CalConfig is one per-signal affine calibration: y = raw*Scale + Offset.
//
// The key is (Source label, base Signal name). It is deliberately the source
// *label* and not the runtime id ("s1", "s2"): ids are assigned in add-order at
// startup, so a calibration keyed by id would rebind to a different source
// whenever the source list order changed.
type CalConfig struct {
Source string `json:"source"`
Signal string `json:"signal"`
Scale float64 `json:"scale"`
Offset float64 `json:"offset"`
Unit string `json:"unit,omitempty"`
}
// calKey builds the calTable map key. NUL cannot occur in either component,
// so the concatenation is unambiguous.
func calKey(source, signal string) string { return source + "\x00" + signal }
// Normalise trims and validates the entry in place, reporting whether it is
// usable. A zero or non-finite Scale is rejected because it makes the
// calibration non-invertible, which the trigger threshold path depends on.
func (c *CalConfig) Normalise() bool {
c.Source = strings.TrimSpace(c.Source)
c.Signal = strings.TrimSpace(c.Signal)
// Strip a trailing "[digits]" suffix so one entry covers an entire array
// signal. "Adc[3]" → "Adc". Must run before the empty check below so
// that "[0]" → "" → rejected, matching C++ and JS behaviour.
c.Signal = arrayIndexSuffix.ReplaceAllString(c.Signal, "")
if c.Source == "" || c.Signal == "" {
return false
}
if math.IsNaN(c.Scale) || math.IsInf(c.Scale, 0) || c.Scale == 0 {
return false
}
if math.IsNaN(c.Offset) || math.IsInf(c.Offset, 0) {
return false
}
c.Unit = strings.TrimSpace(c.Unit)
if len(c.Unit) > maxUnitLen {
c.Unit = c.Unit[:maxUnitLen]
// The byte cut may land mid-rune. Drop any trailing partial rune so
// the result is always valid UTF-8; json.Marshal would otherwise emit
// replacement characters and break the save→load round-trip.
for {
r, size := utf8.DecodeLastRuneInString(c.Unit)
if r != utf8.RuneError || size != 1 {
break
}
c.Unit = c.Unit[:len(c.Unit)-1]
}
}
return true
}
// IsIdentity reports whether the entry carries no information and can be
// dropped rather than stored and persisted.
func (c CalConfig) IsIdentity() bool {
return c.Scale == 1 && c.Offset == 0 && c.Unit == ""
}
// calTable is the hub's calibration store, safe for concurrent use.
type calTable struct {
mu sync.RWMutex
entries map[string]CalConfig
}
func newCalTable() *calTable {
return &calTable{entries: make(map[string]CalConfig)}
}
// Set validates and stores one entry, reporting whether it was accepted.
// Storing an identity entry removes any existing one for that key.
func (t *calTable) Set(c CalConfig) bool {
if !c.Normalise() {
return false
}
t.mu.Lock()
defer t.mu.Unlock()
if c.IsIdentity() {
delete(t.entries, calKey(c.Source, c.Signal))
} else {
t.entries[calKey(c.Source, c.Signal)] = c
}
return true
}
// Replace swaps the whole table for the given entries, silently dropping the
// invalid and identity ones. Used by config load and reload.
func (t *calTable) Replace(list []CalConfig) {
next := make(map[string]CalConfig, len(list))
for _, c := range list {
if !c.Normalise() || c.IsIdentity() {
continue
}
next[calKey(c.Source, c.Signal)] = c
}
t.mu.Lock()
t.entries = next
t.mu.Unlock()
}
// List returns the entries sorted by source then signal, so both the wire
// message and the config file have a stable order.
func (t *calTable) List() []CalConfig {
t.mu.RLock()
out := make([]CalConfig, 0, len(t.entries))
for _, c := range t.entries {
out = append(out, c)
}
t.mu.RUnlock()
sort.Slice(out, func(i, j int) bool {
if out[i].Source != out[j].Source {
return out[i].Source < out[j].Source
}
return out[i].Signal < out[j].Signal
})
return out
}
// ─── Config file codec ────────────────────────────────────────────────────────
// configFileEntry is the union of a source block and a calibration block.
//
// The file is one FLAT array of FLAT objects — never a nested one. The C++
// StreamHub's LoadSourcesFile is a hand-rolled scanner that takes each "{" up
// to the next "}" as one object, so a nested block would truncate the parse.
// Scale and Offset are pointers so that an absent field can be told apart from
// an explicit zero and defaulted to the identity values.
type configFileEntry struct {
// Source fields.
Label string `json:"label,omitempty"`
Addr string `json:"addr,omitempty"`
MulticastGroup string `json:"multicastGroup,omitempty"`
DataPort int `json:"dataPort,omitempty"`
// Calibration fields.
Source string `json:"source,omitempty"`
Signal string `json:"signal,omitempty"`
Scale *float64 `json:"scale,omitempty"`
Offset *float64 `json:"offset,omitempty"`
Unit string `json:"unit,omitempty"`
}
// parseConfigFile splits the flat array into sources and calibration entries.
// A block with "addr" is a source, one with "signal" is a calibration; anything
// else is skipped with a warning.
func parseConfigFile(data []byte) ([]SourceConfig, []CalConfig, error) {
var raw []configFileEntry
if err := json.Unmarshal(data, &raw); err != nil {
return nil, nil, err
}
srcs := make([]SourceConfig, 0, len(raw))
cals := make([]CalConfig, 0, len(raw))
for _, e := range raw {
switch {
case e.Addr != "":
srcs = append(srcs, SourceConfig{
Label: e.Label,
Addr: e.Addr,
MulticastGroup: e.MulticastGroup,
DataPort: e.DataPort,
})
case e.Signal != "":
c := CalConfig{Source: e.Source, Signal: e.Signal, Scale: 1, Offset: 0, Unit: e.Unit}
if e.Scale != nil {
c.Scale = *e.Scale
}
if e.Offset != nil {
c.Offset = *e.Offset
}
if !c.Normalise() {
log.Printf("wshub: skipping invalid calibration entry %q/%q", e.Source, e.Signal)
continue
}
cals = append(cals, c)
default:
log.Printf("wshub: skipping unrecognised config block")
}
}
return srcs, cals, nil
}
// encodeConfigFile renders the sources followed by the calibration entries as
// one flat array, in the indented shape the existing files already use.
func encodeConfigFile(srcs []SourceConfig, cals []CalConfig) ([]byte, error) {
out := make([]configFileEntry, 0, len(srcs)+len(cals))
for _, s := range srcs {
out = append(out, configFileEntry{
Label: s.Label,
Addr: s.Addr,
MulticastGroup: s.MulticastGroup,
DataPort: s.DataPort,
})
}
for _, c := range cals {
scale, offset := c.Scale, c.Offset
out = append(out, configFileEntry{
Source: c.Source,
Signal: c.Signal,
Scale: &scale,
Offset: &offset,
Unit: c.Unit,
})
}
return json.MarshalIndent(out, "", " ")
}
Common/Client/go/wshub/calibration_test.go
+237
View File
@@ -0,0 +1,237 @@
package wshub
import (
"math"
"strings"
"testing"
"unicode/utf8"
)
func TestCalConfigNormalise(t *testing.T) {
cases := []struct {
name string
in CalConfig
want bool
wantUnit string
}{
{"plain", CalConfig{Source: "wave", Signal: "Adc", Scale: 2, Offset: -1, Unit: "V"}, true, "V"},
{"trims", CalConfig{Source: " wave ", Signal: " Adc ", Scale: 1, Unit: " V "}, true, "V"},
{"emptySource", CalConfig{Signal: "Adc", Scale: 1}, false, ""},
{"emptySignal", CalConfig{Source: "wave", Scale: 1}, false, ""},
{"zeroScale", CalConfig{Source: "wave", Signal: "Adc", Scale: 0}, false, ""},
{"nanScale", CalConfig{Source: "wave", Signal: "Adc", Scale: math.NaN()}, false, ""},
{"infScale", CalConfig{Source: "wave", Signal: "Adc", Scale: math.Inf(1)}, false, ""},
{"nanOffset", CalConfig{Source: "wave", Signal: "Adc", Scale: 1, Offset: math.NaN()}, false, ""},
{"infOffset", CalConfig{Source: "wave", Signal: "Adc", Scale: 1, Offset: math.Inf(-1)}, false, ""},
{"negScaleOK", CalConfig{Source: "wave", Signal: "Adc", Scale: -1}, true, ""},
{"longUnit", CalConfig{Source: "wave", Signal: "Adc", Scale: 1,
Unit: "0123456789abcdefGHIJ"}, true, "0123456789abcdef"},
// Finding 1: array-element suffix stripping for cross-implementation parity.
{"arrayIndex3", CalConfig{Source: "wave", Signal: "Adc[3]", Scale: 1}, true, ""},
{"arrayIndex12", CalConfig{Source: "wave", Signal: "Adc[12]", Scale: 1}, true, ""},
{"arrayNoSuffix", CalConfig{Source: "wave", Signal: "Adc", Scale: 1}, true, ""},
{"arrayMidBracket", CalConfig{Source: "wave", Signal: "A[1]B", Scale: 1}, true, ""},
{"arrayNonNumeric", CalConfig{Source: "wave", Signal: "Adc[x]", Scale: 1}, true, ""},
{"arrayZeroOnly", CalConfig{Source: "wave", Signal: "[0]", Scale: 1}, false, ""},
}
for _, c := range cases {
got := c.in
if ok := got.Normalise(); ok != c.want {
t.Errorf("%s: Normalise() = %v, want %v", c.name, ok, c.want)
continue
}
if c.want && got.Unit != c.wantUnit {
t.Errorf("%s: Unit = %q, want %q", c.name, got.Unit, c.wantUnit)
}
}
if len("0123456789abcdef") != maxUnitLen {
t.Fatalf("test assumes maxUnitLen == 16, got %d", maxUnitLen)
}
// Verify stripped Signal values for array-index cases.
arraySignalCases := []struct {
input string
want string
}{
{"Adc[3]", "Adc"},
{"Adc[12]", "Adc"},
{"Adc", "Adc"},
{"A[1]B", "A[1]B"},
{"Adc[x]", "Adc[x]"},
}
for _, ac := range arraySignalCases {
got := CalConfig{Source: "wave", Signal: ac.input, Scale: 1}
got.Normalise()
if got.Signal != ac.want {
t.Errorf("Signal strip %q: got %q, want %q", ac.input, got.Signal, ac.want)
}
}
// Finding 2: UTF-8 unit truncation must not split a multi-byte rune.
// "°" is U+00B0, encoded as 2 bytes in UTF-8.
degree := "°"
if len(degree) != 2 {
t.Fatalf("test expects '°' to be 2 bytes, got %d", len(degree))
}
unit16 := strings.Repeat(degree, 8) // exactly 16 bytes — must survive intact
c8 := CalConfig{Source: "wave", Signal: "Adc", Scale: 1, Unit: unit16}
c8.Normalise()
if c8.Unit != unit16 {
t.Errorf("16-byte degree unit mangled: got %q, want %q", c8.Unit, unit16)
}
if !utf8.ValidString(c8.Unit) {
t.Errorf("16-byte degree unit is not valid UTF-8: %q", c8.Unit)
}
unit18 := strings.Repeat(degree, 9) // 18 bytes — must truncate to 8 degrees (16 bytes), not 16 bytes with a broken half-rune
c9 := CalConfig{Source: "wave", Signal: "Adc", Scale: 1, Unit: unit18}
c9.Normalise()
if c9.Unit != unit16 {
t.Errorf("18-byte degree unit truncated to %q, want %q", c9.Unit, unit16)
}
if !utf8.ValidString(c9.Unit) {
t.Errorf("truncated degree unit is not valid UTF-8: %q", c9.Unit)
}
}
func TestCalTableSetListAndIdentityRemoval(t *testing.T) {
tab := newCalTable()
if !tab.Set(CalConfig{Source: "b", Signal: "Y", Scale: 3, Offset: 1, Unit: "A"}) {
t.Fatal("Set(b/Y) rejected")
}
if !tab.Set(CalConfig{Source: "a", Signal: "X", Scale: 2}) {
t.Fatal("Set(a/X) rejected")
}
if tab.Set(CalConfig{Source: "a", Signal: "Z", Scale: 0}) {
t.Error("Set with scale=0 accepted, want rejected")
}
got := tab.List()
if len(got) != 2 {
t.Fatalf("List() = %d entries, want 2", len(got))
}
// Sorted by source then signal.
if got[0].Source != "a" || got[1].Source != "b" {
t.Errorf("List() order = %q,%q, want a,b", got[0].Source, got[1].Source)
}
// An identity entry removes the stored one.
if !tab.Set(CalConfig{Source: "a", Signal: "X", Scale: 1, Offset: 0, Unit: ""}) {
t.Fatal("identity Set rejected")
}
if got := tab.List(); len(got) != 1 || got[0].Source != "b" {
t.Errorf("after identity Set, List() = %+v, want only b/Y", got)
}
}
func TestCalTableReplace(t *testing.T) {
tab := newCalTable()
tab.Set(CalConfig{Source: "old", Signal: "X", Scale: 5})
tab.Replace([]CalConfig{
{Source: "new", Signal: "Y", Scale: 2},
{Source: "bad", Signal: "Z", Scale: 0}, // invalid → dropped
{Source: "id", Signal: "W", Scale: 1, Offset: 0, Unit: ""}, // identity → dropped
})
got := tab.List()
if len(got) != 1 || got[0].Source != "new" {
t.Fatalf("List() = %+v, want only new/Y", got)
}
}
func TestParseConfigFileCurrentFormat(t *testing.T) {
// A file written by the current binaries — sources only, spaces after colons.
data := []byte(`[
{
"label": "wave",
"addr": "127.0.0.1:44500"
},
{
"label": "mc",
"addr": "127.0.0.1:44501",
"multicastGroup": "239.0.0.1",
"dataPort": 44502
}
]`)
srcs, cals, err := parseConfigFile(data)
if err != nil {
t.Fatalf("parseConfigFile: %v", err)
}
if len(srcs) != 2 || len(cals) != 0 {
t.Fatalf("got %d sources / %d cals, want 2 / 0", len(srcs), len(cals))
}
if srcs[1].MulticastGroup != "239.0.0.1" || srcs[1].DataPort != 44502 {
t.Errorf("multicast source = %+v", srcs[1])
}
}
func TestParseConfigFileMixed(t *testing.T) {
data := []byte(`[
{"label":"wave","addr":"127.0.0.1:44500"},
{"source":"wave","signal":"Adc","scale":0.00030518,"offset":-1.25,"unit":"V"},
{"source":"wave","signal":"Bare"},
{"source":"wave","signal":"Bad","scale":0},
{"nonsense":true}
]`)
srcs, cals, err := parseConfigFile(data)
if err != nil {
t.Fatalf("parseConfigFile: %v", err)
}
if len(srcs) != 1 {
t.Fatalf("got %d sources, want 1", len(srcs))
}
if len(cals) != 2 {
t.Fatalf("got %d cals, want 2 (Adc and Bare; Bad is invalid)", len(cals))
}
if cals[0].Scale != 0.00030518 || cals[0].Offset != -1.25 || cals[0].Unit != "V" {
t.Errorf("Adc = %+v", cals[0])
}
// Absent scale/offset default to the identity values, not to zero.
if cals[1].Signal != "Bare" || cals[1].Scale != 1 || cals[1].Offset != 0 {
t.Errorf("Bare = %+v, want scale 1 / offset 0", cals[1])
}
}
func TestParseConfigFileMalformed(t *testing.T) {
if _, _, err := parseConfigFile([]byte("not json")); err == nil {
t.Error("parseConfigFile(garbage) = nil error, want error")
}
}
func TestEncodeConfigFileRoundTrip(t *testing.T) {
srcs := []SourceConfig{
{Label: "wave", Addr: "127.0.0.1:44500"},
{Label: "mc", Addr: "127.0.0.1:44501", MulticastGroup: "239.0.0.1", DataPort: 44502},
}
cals := []CalConfig{
{Source: "wave", Signal: "Adc", Scale: 0.5, Offset: 0, Unit: "V"},
}
data, err := encodeConfigFile(srcs, cals)
if err != nil {
t.Fatalf("encodeConfigFile: %v", err)
}
gotSrcs, gotCals, err := parseConfigFile(data)
if err != nil {
t.Fatalf("parseConfigFile(encoded): %v\n%s", err, data)
}
if len(gotSrcs) != 2 || len(gotCals) != 1 {
t.Fatalf("round-trip gave %d sources / %d cals, want 2 / 1\n%s",
len(gotSrcs), len(gotCals), data)
}
if gotSrcs[1] != srcs[1] {
t.Errorf("source round-trip: got %+v, want %+v", gotSrcs[1], srcs[1])
}
if gotCals[0] != cals[0] {
t.Errorf("cal round-trip: got %+v, want %+v", gotCals[0], cals[0])
}
// offset 0 must survive as an explicit field, not be dropped by omitempty.
if !bytesContains(data, []byte(`"offset": 0`)) {
t.Errorf("encoded file lost the zero offset:\n%s", data)
}
}
func bytesContains(hay, needle []byte) bool {
for i := 0; i+len(needle) <= len(hay); i++ {
if string(hay[i:i+len(needle)]) == string(needle) {
return true
}
}
return false
}
Common/Client/go/wshub/capture_coverage_test.go
+168
View File
@@ -0,0 +1,168 @@
package wshub
import (
"encoding/binary"
"math"
"testing"
)
// decodeCaptureSpan pulls the time extent of one signal out of a v2 frame.
func decodeCaptureSpan(t *testing.T, buf []byte, key string) (first, last float64, n int) {
t.Helper()
off := 1 + 8 + 8 + 8
nSig := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
for i := 0; i < nSig; i++ {
kl := int(binary.LittleEndian.Uint16(buf[off:]))
off += 2
k := string(buf[off : off+kl])
off += kl
cnt := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
if k == key && cnt > 0 {
first = math.Float64frombits(binary.LittleEndian.Uint64(buf[off:]))
last = math.Float64frombits(binary.LittleEndian.Uint64(buf[off+(cnt-1)*8:]))
n = cnt
}
off += cnt * 16
}
return
}
// The rings only reach back over the window once they have rolled over at the
// current bucket, which takes as long as the window itself — so a window widened
// mid-run leaves the first captures asking for history the rings never stored.
// The archive kept it, and the capture must come back whole.
func TestCaptureBackfillsItsHeadFromTheArchive(t *testing.T) {
h := NewHub()
hw, key := newTestHistory(t, HistoryConfig{
WindowSec: 60, Decimation: 1, MinDiskFreeMB: -1,
}, 1000)
h.hist = hw
// 20 s of 1 kSps, archived in full…
ts, vs := ramp(1000, 0.001, 20000)
hw.write(key, ts, vs)
// …but a ring that only ever holds the last 5 s of it.
rb := newSigRing(5000)
rb.write(ts, vs)
h.rings[key] = rb
// A 15 s window, of which the ring has the newest third.
const t0, t1 = 1005.0, 1020.0
buf := h.buildTriggerCapture(1015, 10, 5)
if buf == nil {
t.Fatal("no capture frame built")
}
first, last, n := decodeCaptureSpan(t, buf, key)
if first > t0+0.05 {
t.Errorf("capture starts at %.3f, want the window's start %.3f — the archive holds it",
first, t0)
}
if last < t1-0.05 {
t.Errorf("capture ends at %.3f, want %.3f", last, t1)
}
if n < 100 {
t.Errorf("capture has %d points, too few for a 15 s window at 1 kSps", n)
}
// The join between the two sources must not break time order, or every
// binary search over the capture — client-side and in the hold — misreads it.
ct, _, ok := h.capture.slice(key, t0, t1)
if !ok {
t.Fatal("the hold declined the window it just published")
}
for i := 1; i < len(ct); i++ {
if ct[i] < ct[i-1] {
t.Fatalf("capture time goes backwards at %d: %.6f then %.6f", i, ct[i-1], ct[i])
}
}
}
// A capture that neither source could fill must not answer for the stretch it is
// missing: the client has to fall through to the archive instead of redrawing
// the same hole on every zoom.
func TestHoldDeclinesTheStretchACaptureNeverGot(t *testing.T) {
h := NewHub()
ts, vs := ramp(1000, 0.001, 20000)
rb := newSigRing(5000) // the newest 5 s only, and no archive to fill from
rb.write(ts, vs)
h.rings["src:sig"] = rb
if buf := h.buildTriggerCapture(1015, 10, 5); buf == nil {
t.Fatal("no capture frame built")
}
if _, _, ok := h.capture.slice("src:sig", 1005, 1020); ok {
t.Error("the hold answered for 15 s it only has the last 5 s of")
}
// What it does hold, it still serves.
if _, _, ok := h.capture.slice("src:sig", 1016, 1019); !ok {
t.Error("the hold declined a range well inside its data")
}
}
// TestCaptureCoverageAcrossShots walks a whole acquisition the way Run() does —
// ingest, retune, dueCapture, rearm — and reports how much of each window the
// capture actually came back with.
func TestCaptureCoverageAcrossShots(t *testing.T) {
const (
key = "s1:Ch1"
rate = 100e3 // scaled 10x down from the 1 MSps producer
budget = 400_000
window = 120.0
prePct = 20.0
batchSec = 1.0 / 30.0
simSec = 900.0
)
h := NewHub()
h.SetRingBudget(budget)
h.rings[key] = newSigRing(ringCapInitial)
h.trigger.SetConfig(trigConfig{signalKey: key, edge: "rising", threshold: 0,
windowSec: window, prePercent: prePct, mode: "normal", holdoffSec: 0.2})
rateHz := float64(rate)
nBatch := int(rateHz * batchSec)
ts := make([]float64, nBatch)
vs := make([]float64, nBatch)
armed := false
shots := 0
for now := 0.0; now < simSec; now += batchSec {
for i := range ts {
ts[i] = now + float64(i)/rateHz
// 0.05 Hz sine: one rising zero crossing every 20 s.
vs[i] = math.Sin(2 * math.Pi * 0.05 * ts[i])
}
h.ingest(key, 1, ts, vs)
h.retuneRings(now)
// Arm once the stream is going, as a user would.
if !armed && now > 5 {
h.trigger.Arm()
armed = true
}
if trigTime, pre, post, ok := h.trigger.dueCapture(now + batchSec); ok {
buf := h.buildTriggerCapture(trigTime, pre, post)
if buf == nil {
t.Fatalf("shot at t=%.1f produced no frame", trigTime)
}
first, last, n := decodeCaptureSpan(t, buf, key)
t0, t1 := trigTime-pre, trigTime+post
_, ringSpan := h.rings[key].stats()
shots++
t.Logf("shot %d fired t=%.1f window [%.1f,%.1f] got [%.1f,%.1f] "+
"= %.0f%% (%d pts, bucket %d, ring span %.1f s)",
shots, trigTime, t0, t1, first, last,
100*(last-first)/(t1-t0), n, h.rings[key].bucketSize(), ringSpan)
h.trigger.markTriggered(now + batchSec)
} else if h.trigger.dueRearm(now + batchSec) {
h.trigger.Arm()
}
}
if shots < 3 {
t.Fatalf("only %d shots in %.0f s", shots, simSec)
}
}
Common/Client/go/wshub/capturehold.go
+83
View File
@@ -0,0 +1,83 @@
package wshub
import (
"sort"
"sync"
)
// capturedWindow is one delivered trigger capture, held at the resolution the
// rings had when it was taken. Nothing mutates it after publication, so readers
// may sub-slice it without copying.
type capturedWindow struct {
t0, t1 float64
sigs map[string]sigData
}
// captureHold is the read half of the trigger double buffer; the rings are the
// write half.
//
// The rings keep rolling while the trigger re-arms and collects the next shot,
// so within seconds of a capture they no longer hold the window the user is
// looking at — a zoom into it came back with only the newest sliver, or with
// nothing. Publishing the window here at capture time gives the viewer a
// snapshot that the re-arming acquisition cannot overwrite: the swap happens
// only when the *next* capture is complete, which is also the moment the client
// stops displaying this one.
type captureHold struct {
mu sync.RWMutex
cur *capturedWindow
}
// publish swaps in a new capture, retiring the previous one. Readers that
// already hold a pointer to the retired window keep reading it safely.
func (ch *captureHold) publish(t0, t1 float64, sigs map[string]sigData) {
if len(sigs) == 0 {
return
}
w := &capturedWindow{t0: t0, t1: t1, sigs: sigs}
ch.mu.Lock()
ch.cur = w
ch.mu.Unlock()
}
// clear drops the held capture, releasing its memory.
func (ch *captureHold) clear() {
ch.mu.Lock()
ch.cur = nil
ch.mu.Unlock()
}
// slice answers [a, b] for one signal out of the held capture, reporting
// whether it could.
//
// It declines any range reaching outside the captured window: that is a live
// zoom or a pan off the capture, and only the rings still track the stream.
// Inside the window the hold is never worse than the rings — retuning does not
// rewrite stored samples, so a ring that still covers the range holds the very
// same points — which is why no trigger-state gating is needed here.
func (ch *captureHold) slice(key string, a, b float64) ([]float64, []float64, bool) {
ch.mu.RLock()
w := ch.cur
ch.mu.RUnlock()
if w == nil || a < w.t0 || b > w.t1 {
return nil, nil, false
}
sd, ok := w.sigs[key]
if !ok || len(sd.T) == 0 {
return nil, nil, false
}
// The window is what was asked for; this signal's samples are what could be
// found. A capture whose front was never recoverable must not answer for the
// stretch it is missing — the client would redraw the same hole on every
// zoom and every "fit" instead of falling back to the archive.
tol := shortCaptureTol * (w.t1 - w.t0)
if sd.T[0] > a+tol || sd.T[len(sd.T)-1] < b-tol {
return nil, nil, false
}
lo := sort.SearchFloat64s(sd.T, a)
hi := lo + sort.Search(len(sd.T)-lo, func(i int) bool { return sd.T[lo+i] > b })
if hi <= lo {
return nil, nil, false
}
return sd.T[lo:hi], sd.V[lo:hi], true
}
Common/Client/go/wshub/capturehold_test.go
+128
View File
@@ -0,0 +1,128 @@
package wshub
import "testing"
func heldRamp(t0, dt float64, n int) sigData {
sd := sigData{T: make([]float64, n), V: make([]float64, n)}
for i := range sd.T {
sd.T[i] = t0 + float64(i)*dt
sd.V[i] = float64(i)
}
return sd
}
func TestCaptureHoldServesRangesInsideTheWindow(t *testing.T) {
var ch captureHold
ch.publish(0, 10, map[string]sigData{"s1:sig": heldRamp(0, 0.1, 101)})
gt, gv, ok := ch.slice("s1:sig", 2, 3)
if !ok {
t.Fatal("held capture declined a range inside its window")
}
if gt[0] < 2 || gt[len(gt)-1] > 3 {
t.Fatalf("range %v..%v escapes the request 2..3", gt[0], gt[len(gt)-1])
}
if len(gt) != len(gv) {
t.Fatalf("t/v length mismatch: %d vs %d", len(gt), len(gv))
}
if gv[0] != 20 {
t.Fatalf("first value %v, want the sample at t=2", gv[0])
}
}
// A range poking outside the capture is a live zoom: only the rings still track
// the stream, so the hold must stand aside rather than answer a clipped range.
func TestCaptureHoldDeclinesRangesOutsideTheWindow(t *testing.T) {
var ch captureHold
ch.publish(0, 10, map[string]sigData{"s1:sig": heldRamp(0, 0.1, 101)})
for _, r := range [][2]float64{{-1, 5}, {5, 11}, {20, 30}, {-5, -1}} {
if _, _, ok := ch.slice("s1:sig", r[0], r[1]); ok {
t.Fatalf("held capture answered %v..%v, which is not inside 0..10", r[0], r[1])
}
}
if _, _, ok := ch.slice("other:sig", 2, 3); ok {
t.Fatal("held capture answered for a signal it does not hold")
}
}
func TestCaptureHoldZeroValueAndClearDecline(t *testing.T) {
var ch captureHold
if _, _, ok := ch.slice("s1:sig", 0, 1); ok {
t.Fatal("empty hold answered a request")
}
ch.publish(0, 10, map[string]sigData{"s1:sig": heldRamp(0, 0.1, 101)})
ch.clear()
if _, _, ok := ch.slice("s1:sig", 2, 3); ok {
t.Fatal("cleared hold still answered a request")
}
}
// The point of the double buffer: the window a client is exploring survives the
// re-armed acquisition rolling the rings past it, and is replaced only when the
// next shot completes.
func TestZoomIntoACaptureSurvivesTheRingRollingPast(t *testing.T) {
h := NewHub()
rb := newSigRing(4000)
h.rings["s1:sig"] = rb
// 2 s of 1 kSps, then fire a trigger over [0.5, 1.5].
ts, vs := make([]float64, 2000), make([]float64, 2000)
for i := range ts {
ts[i], vs[i] = float64(i)*1e-3, float64(i)
}
rb.write(ts, vs)
if msg := h.buildTriggerCapture(1.0, 0.5, 0.5); msg == nil {
t.Fatal("buildTriggerCapture produced no frame")
}
// The trigger re-arms and the stream runs on until the captured window has
// been overwritten several times over.
for pass := 0; pass < 5; pass++ {
for i := range ts {
ts[i] += 2.0
}
rb.write(ts, vs)
}
if rt, _ := rb.slice(0.5, 1.5); len(rt) != 0 {
t.Fatalf("ring still holds %d points of the captured window; the test is not exercising the hold", len(rt))
}
got := h.zoomSlice(0.8, 0.9, []string{"s1:sig"}, 1<<30)
sd, ok := got["s1:sig"]
if !ok {
t.Fatal("zoom into the held capture returned nothing")
}
if len(sd.T) != 101 {
t.Fatalf("zoom returned %d points, want the 101 samples in 0.8..0.9", len(sd.T))
}
if sd.V[0] != 800 || sd.V[len(sd.V)-1] != 900 {
t.Fatalf("zoom returned values %v..%v, want 800..900", sd.V[0], sd.V[len(sd.V)-1])
}
// A live zoom outside the held window still reaches the rings.
if live := h.zoomSlice(11.0, 11.1, []string{"s1:sig"}, 1<<30); len(live["s1:sig"].T) == 0 {
t.Fatal("live zoom outside the capture was swallowed by the hold")
}
}
// A shot that yields nothing must not blank the window already on screen.
func TestEmptyCaptureKeepsThePreviousHold(t *testing.T) {
h := NewHub()
rb := newSigRing(4000)
h.rings["s1:sig"] = rb
ts, vs := make([]float64, 2000), make([]float64, 2000)
for i := range ts {
ts[i], vs[i] = float64(i)*1e-3, float64(i)
}
rb.write(ts, vs)
h.buildTriggerCapture(1.0, 0.5, 0.5)
// A window the rings have no samples for at all.
if msg := h.buildTriggerCapture(500.0, 0.5, 0.5); msg != nil {
t.Fatal("capture of an empty window produced a frame")
}
if _, _, ok := h.capture.slice("s1:sig", 0.8, 0.9); !ok {
t.Fatal("empty capture dropped the previously held window")
}
}
Common/Client/go/wshub/history.go
+1339
View File
File diff suppressed because it is too large Load Diff
Common/Client/go/wshub/history_test.go
+949
View File
@@ -0,0 +1,949 @@
package wshub
import (
"encoding/binary"
"math"
"os"
"path/filepath"
"testing"
"marte2/common/udpsprotocol"
)
// newTestHistory opens a writer in a temp dir with one signal file of the given
// declared rate, and returns the writer plus that signal's key.
func newTestHistory(t *testing.T, cfg HistoryConfig, rate float64) (*historyWriter, string) {
t.Helper()
if cfg.Directory == "" {
cfg.Directory = t.TempDir()
}
hw, err := newHistoryWriter(cfg)
if err != nil {
t.Fatalf("newHistoryWriter: %v", err)
}
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: rate},
})
t.Cleanup(hw.close)
return hw, "src:sig"
}
func ramp(t0 float64, dt float64, n int) ([]float64, []float64) {
ts := make([]float64, n)
vs := make([]float64, n)
for i := range ts {
ts[i] = t0 + float64(i)*dt
vs[i] = float64(i)
}
return ts, vs
}
// A budget that cannot hold the window at full rate must buy the window by
// widening the min/max bucket, not by archiving a shorter stretch: a user
// looking at 600 s wants 600 s of it archived, coarser if need be.
func TestHistCapacityKeepsWindowByBucketing(t *testing.T) {
const mega = 1 << 20
cases := []struct {
name string
window float64
rate float64
maxPts int
wantBucket int
}{
// 60 s of 1 kSps is 60 k samples — well inside 1 MPt, so stored verbatim.
{"slow signal keeps full resolution", 60, 1000, mega, 1},
// 600 s of 1 MSps is 600 M samples against 16 Mi points: at 2 points per
// bucket and the headroom, ceil(2 × 1.25 × 600e6 / 16Mi) = 90 per bucket.
{"fast signal is enveloped", 600, 1e6, 16 * mega, 90},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
capacity, bucket := histCapacityFor(c.window, c.rate, 1, c.maxPts)
if bucket != c.wantBucket {
t.Errorf("bucket = %d, want %d", bucket, c.wantBucket)
}
if capacity > uint32(c.maxPts) {
t.Errorf("capacity %d exceeds the %d-point budget", capacity, c.maxPts)
}
// The whole window has to fit, which is the entire point.
if covered := histCoverageSec(capacity, bucket, 1, c.rate); covered < c.window {
t.Errorf("archive covers %.1f s, want the %.1f s window", covered, c.window)
}
})
}
}
// The file exists to serve the window, so it must track it: a client that widens
// what it displays must not be left reading an archive sized for the old span.
func TestHistorySetWindowResizesFiles(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 10}, 1000)
before := hw.files[key]
if before.bucket != 1 || histCoverageSec(before.capacity, 1, 1, 1000) < 10 {
t.Fatalf("initial geometry = cap %d bucket %d, want 10 s verbatim",
before.capacity, before.bucket)
}
if !hw.setWindow(600) {
t.Fatal("setWindow reported no change for a 60× wider window")
}
after := hw.files[key]
if after == before {
t.Fatal("the file was not re-created")
}
if cov := histCoverageSec(after.capacity, after.bucket, 1, 1000); cov < 600 {
t.Fatalf("archive covers %.1f s, want the new 600 s window", cov)
}
// Same window again: nothing to do, and re-creating the file would throw the
// archive away for nothing.
if hw.setWindow(600) {
t.Fatal("setWindow re-sized for an unchanged window")
}
// A nudge inside the hysteresis band must not either.
if hw.setWindow(610) {
t.Fatal("setWindow re-sized for a 2 % window change")
}
if hw.files[key] != after {
t.Fatal("the file was re-created despite the hysteresis")
}
}
// The archive is what a zoom beyond the rings reads, so a spike that only the
// archive still holds must survive being written to it.
func TestHistoryBucketedWriteKeepsPeaks(t *testing.T) {
// 1 kSps for 1 s = 1000 samples, plus headroom, into a 100-point budget →
// buckets of ceil(2 × 1.25 × 1000 / 100) = 25.
hw, key := newTestHistory(t, HistoryConfig{
WindowSec: 1, MinDiskFreeMB: -1, MaxPointsPerSignal: 100,
}, 1000)
hf := hw.files[key]
if hf.bucket != 25 {
t.Fatalf("bucket = %d, want 25", hf.bucket)
}
ts := make([]float64, 1000)
vs := make([]float64, 1000)
for i := range ts {
ts[i] = float64(i) * 0.001
}
vs[137] = 7.5 // a one-sample positive spike
vs[500] = -3.5 // and a negative one
hw.write(key, ts, vs)
rt, rv := hw.readRange(key, 0, 1, 1000)
if len(rt) == 0 {
t.Fatal("nothing archived")
}
hi, lo := false, false
for i := range rv {
if rv[i] == 7.5 && rt[i] == ts[137] {
hi = true
}
if rv[i] == -3.5 && rt[i] == ts[500] {
lo = true
}
}
if !hi || !lo {
t.Errorf("archive lost a spike (positive kept=%v, negative kept=%v)", hi, lo)
}
// A partial bucket is not written until it completes, so the last few
// samples may be missing; everything before them must be there.
if hf.count == 0 || hf.count > hf.capacity {
t.Errorf("archived %d points into a %d-point file", hf.count, hf.capacity)
}
}
func TestHistoryDisabledWithoutDirectory(t *testing.T) {
hw, err := newHistoryWriter(HistoryConfig{})
if err != nil {
t.Fatalf("newHistoryWriter: %v", err)
}
if hw != nil {
t.Fatal("empty Directory must disable history")
}
// Every method must stay usable on the nil writer, which is how the hub
// avoids guarding each call site.
if hw.enabled() {
t.Fatal("nil writer reports enabled")
}
hw.write("src:sig", []float64{1}, []float64{1})
hw.flushHeaders()
hw.close()
if rt, _ := hw.readRange("src:sig", 0, 1, 10); rt != nil {
t.Fatal("nil writer returned data")
}
if len(hw.info()) != 0 {
t.Fatal("nil writer returned info entries")
}
}
func TestHistoryWriteReadRoundTrip(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{}, 100)
ts, vs := ramp(10, 0.01, 500)
hw.write(key, ts, vs)
rt, rv := hw.readRange(key, 10.5, 11.0, 10000)
if len(rt) != 51 { // inclusive both ends, 0.01 s spacing
t.Fatalf("read %d points, want 51", len(rt))
}
if rt[0] < 10.5-1e-9 || rt[len(rt)-1] > 11.0+1e-9 {
t.Fatalf("range [%v, %v] escapes the request", rt[0], rt[len(rt)-1])
}
for i := range rt {
wantV := math.Round((rt[i] - 10) / 0.01)
if math.Abs(rv[i]-wantV) > 1e-6 {
t.Fatalf("point %d: value %v, want %v", i, rv[i], wantV)
}
}
}
func TestHistoryReadRangeOutsideDataIsEmpty(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{}, 100)
ts, vs := ramp(10, 0.01, 100)
hw.write(key, ts, vs)
if rt, _ := hw.readRange(key, 100, 200, 1000); len(rt) != 0 {
t.Fatalf("read %d points past the newest sample", len(rt))
}
if rt, _ := hw.readRange(key, 0, 5, 1000); len(rt) != 0 {
t.Fatalf("read %d points before the oldest sample", len(rt))
}
if rt, _ := hw.readRange("src:missing", 10, 11, 1000); rt != nil {
t.Fatal("unknown key returned data")
}
if rt, _ := hw.readRange(key, 11, 10, 1000); rt != nil {
t.Fatal("inverted range returned data")
}
}
// Once the file has wrapped, the oldest samples must be gone and the retained
// window must still read back contiguously across the wrap point.
func TestHistoryWrapAround(t *testing.T) {
// A sub-second window at 1 Sps sizes below the 1000-pair floor, which is a
// cheap capacity to wrap.
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 0.36}, 1)
hf := hw.files[key]
if hf.capacity != histMinCapacity {
t.Fatalf("capacity = %d, want the %d floor", hf.capacity, histMinCapacity)
}
// 2.5 fills, in batches that do not align with the capacity so the wrap
// lands mid-batch.
total := 2500
ts, vs := ramp(0, 1, total)
for i := 0; i < total; i += 333 {
end := i + 333
if end > total {
end = total
}
hw.write(key, ts[i:end], vs[i:end])
}
if hf.count != histMinCapacity {
t.Fatalf("count = %d, want a full %d", hf.count, histMinCapacity)
}
wantOldest := float64(total - histMinCapacity)
if hf.tOldest != wantOldest {
t.Fatalf("tOldest = %v, want %v", hf.tOldest, wantOldest)
}
if hf.tNewest != float64(total-1) {
t.Fatalf("tNewest = %v, want %v", hf.tNewest, float64(total-1))
}
rt, rv := hw.readRange(key, wantOldest, float64(total-1), 10000)
if len(rt) != histMinCapacity {
t.Fatalf("read %d points, want the full %d", len(rt), histMinCapacity)
}
for i := range rt {
want := wantOldest + float64(i)
if rt[i] != want || rv[i] != want {
t.Fatalf("point %d = (%v, %v), want (%v, %v)", i, rt[i], rv[i], want, want)
}
}
// The evicted samples must not come back.
if et, _ := hw.readRange(key, 0, wantOldest-1, 10000); len(et) != 0 {
t.Fatalf("read %d evicted points", len(et))
}
}
// A single batch larger than the file keeps its tail, not its head.
func TestHistoryOversizedBatchKeepsTail(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 0.36}, 1)
ts, vs := ramp(0, 1, 3000)
hw.write(key, ts, vs)
hf := hw.files[key]
if hf.count != histMinCapacity {
t.Fatalf("count = %d, want %d", hf.count, histMinCapacity)
}
if hf.tNewest != 2999 {
t.Fatalf("tNewest = %v, want 2999", hf.tNewest)
}
rt, _ := hw.readRange(key, 2000, 2999, 10000)
if len(rt) != histMinCapacity || rt[0] != 2000 {
t.Fatalf("retained window starts at %v with %d points, want 2000 / %d",
rt[0], len(rt), histMinCapacity)
}
}
func TestHistoryDecimation(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{Decimation: 4}, 100)
// Two batches, so the decimation phase must carry across the call boundary
// rather than restarting.
ts, vs := ramp(0, 0.01, 100)
hw.write(key, ts[:37], vs[:37])
hw.write(key, ts[37:], vs[37:])
rt, _ := hw.readRange(key, -1, 1e9, 10000)
if len(rt) != 25 {
t.Fatalf("kept %d of 100 points at decimation 4, want 25", len(rt))
}
for i := 1; i < len(rt); i++ {
if d := rt[i] - rt[i-1]; math.Abs(d-0.04) > 1e-9 {
t.Fatalf("spacing at %d = %v, want 0.04", i, d)
}
}
}
// The input slices are shared with the zoom ring and the trigger, so decimation
// must not touch them.
func TestHistoryWriteDoesNotMutateInput(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{Decimation: 3}, 100)
ts, vs := ramp(0, 0.01, 30)
tCopy := append([]float64(nil), ts...)
vCopy := append([]float64(nil), vs...)
hw.write(key, ts, vs)
for i := range ts {
if ts[i] != tCopy[i] || vs[i] != vCopy[i] {
t.Fatalf("write mutated input at %d", i)
}
}
}
// Reopening the same directory must pick the file back up with its contents,
// which is the whole point of persisting the header.
func TestHistoryReopenPreservesData(t *testing.T) {
dir := t.TempDir()
cfg := HistoryConfig{Directory: dir, WindowSec: 0.36}
sigs := []udpsprotocol.SignalInfo{{Name: "sig", TypeCode: 8, SamplingRate: 1}}
hw, err := newHistoryWriter(cfg)
if err != nil {
t.Fatalf("newHistoryWriter: %v", err)
}
hw.onSourceConfigured("src", sigs)
ts, vs := ramp(0, 1, 400)
hw.write("src:sig", ts, vs)
hw.close()
hw2, err := newHistoryWriter(cfg)
if err != nil {
t.Fatalf("reopen: %v", err)
}
defer hw2.close()
hw2.onSourceConfigured("src", sigs)
hf := hw2.files["src:sig"]
if hf.count != 400 || hf.head != 400 {
t.Fatalf("reopened count=%d head=%d, want 400/400", hf.count, hf.head)
}
rt, rv := hw2.readRange("src:sig", 100, 199, 10000)
if len(rt) != 100 || rt[0] != 100 || rv[0] != 100 {
t.Fatalf("reopened read = %d points starting (%v, %v)", len(rt), rt[0], rv[0])
}
// Appending after the reopen must continue where the file left off.
ts2, vs2 := ramp(400, 1, 50)
hw2.write("src:sig", ts2, vs2)
if hf.tNewest != 449 {
t.Fatalf("tNewest after append = %v, want 449", hf.tNewest)
}
}
// A file sized for a different rate cannot be reused, so it must be recreated
// rather than reopened with a mismatched capacity.
func TestHistoryReopenWithDifferentCapacityRecreates(t *testing.T) {
dir := t.TempDir()
cfg := HistoryConfig{Directory: dir, WindowSec: 3600}
hw, _ := newHistoryWriter(cfg)
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 10},
})
firstCap := hw.files["src:sig"].capacity
hw.write("src:sig", []float64{1, 2}, []float64{1, 2})
hw.close()
hw2, _ := newHistoryWriter(cfg)
defer hw2.close()
hw2.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 100}, // 10× the rate
})
hf := hw2.files["src:sig"]
if hf.capacity == firstCap {
t.Fatalf("capacity unchanged at %d despite a 10x rate change", firstCap)
}
if hf.count != 0 {
t.Fatalf("recreated file kept %d samples", hf.count)
}
}
// A corrupt header must not be trusted: the file gets rebuilt instead.
func TestHistoryCorruptHeaderRecreates(t *testing.T) {
dir := t.TempDir()
cfg := HistoryConfig{Directory: dir, WindowSec: 0.36}
sigs := []udpsprotocol.SignalInfo{{Name: "sig", TypeCode: 8, SamplingRate: 1}}
hw, _ := newHistoryWriter(cfg)
hw.onSourceConfigured("src", sigs)
hw.write("src:sig", []float64{1, 2, 3}, []float64{1, 2, 3})
hw.close()
path := filepath.Join(dir, "src", "sig.shist")
f, err := os.OpenFile(path, os.O_RDWR, 0o644)
if err != nil {
t.Fatalf("open: %v", err)
}
if _, err := f.WriteAt([]byte("XXXX"), 0); err != nil { // clobber the magic
t.Fatalf("clobber: %v", err)
}
f.Close()
hw2, _ := newHistoryWriter(cfg)
defer hw2.close()
hw2.onSourceConfigured("src", sigs)
if got := hw2.files["src:sig"].count; got != 0 {
t.Fatalf("count = %d, want a recreated empty file", got)
}
}
// Raising the budget from the UI has to buy resolution: same duration, a
// narrower min/max bucket. Lowering it again must not overrun the new budget.
func TestSetBudgetRebucketsAtTheSameDuration(t *testing.T) {
// 100 s of 100 kSps is 10 M samples, well past either budget.
hw, key := newTestHistory(t, HistoryConfig{
WindowSec: 100, MaxPointsPerSignal: 100_000,
}, 1e5)
before := hw.files[key]
if before.bucket <= 1 {
t.Fatalf("bucket = %d, want the signal enveloped to fit the budget", before.bucket)
}
if got := hw.setBudget(1_000_000); got != 1_000_000 {
t.Fatalf("setBudget = %d, want 1000000", got)
}
after := hw.files[key]
if after == before {
t.Fatal("the file was not re-created")
}
if after.bucket >= before.bucket {
t.Fatalf("bucket %d → %d, want a finer envelope for a 10× budget",
before.bucket, after.bucket)
}
if after.capacity > 1_000_000 {
t.Fatalf("capacity = %d, over the 1 MPts budget", after.capacity)
}
// The point of the envelope: the duration is covered whatever the budget.
if cov := float64(after.capacity) * float64(after.bucket) / 2 / 1e5; cov < 99 {
t.Fatalf("coverage = %.1f s, want ~100 s", cov)
}
if got := hw.setBudget(100_000); got != 100_000 {
t.Fatalf("setBudget back = %d, want 100000", got)
}
if c := hw.files[key].capacity; c > 100_000 {
t.Fatalf("capacity = %d, over the restored 100 kPts budget", c)
}
}
// A budget that leaves a signal's geometry alone must leave its archive alone
// too — re-creating files nobody asked to resize would throw away history.
func TestSetBudgetKeepsUnaffectedFiles(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{
WindowSec: 1, MaxPointsPerSignal: 16 << 20,
}, 1000)
ts, vs := ramp(0, 0.001, 100)
hw.write(key, ts, vs)
hw.setBudget(8 << 20) // still far more than the 1000 points this signal needs
hf := hw.files[key]
if hf.bucket != 1 {
t.Fatalf("bucket = %d, want the slow signal still archived verbatim", hf.bucket)
}
if hf.count != 100 {
t.Fatalf("count = %d, want the 100 archived samples kept", hf.count)
}
}
// Time-reference signals are the clock for the others, so archiving them would
// just waste disk.
func TestHistorySkipsTimeSignals(t *testing.T) {
dir := t.TempDir()
hw, _ := newHistoryWriter(HistoryConfig{Directory: dir})
defer hw.close()
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "TimeArray", TypeCode: histTypeCodeUint64, SamplingRate: 1000},
{Name: "data", TypeCode: 8, SamplingRate: 1000},
})
if _, ok := hw.files["src:TimeArray"]; ok {
t.Fatal("uint64 time signal was archived")
}
if _, ok := hw.files["src:data"]; !ok {
t.Fatal("data signal was not archived")
}
}
// A second CONFIG for the same source must not throw away the history already
// collected for signals it re-declares.
func TestHistoryReconfigureKeepsExistingFile(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 0.36}, 1)
hw.write(key, []float64{1, 2, 3}, []float64{1, 2, 3})
before := hw.files[key]
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 1},
{Name: "sig2", TypeCode: 8, SamplingRate: 1},
})
if hw.files[key] != before {
t.Fatal("re-CONFIG replaced the existing signal file")
}
if before.count != 3 {
t.Fatalf("count = %d, want the 3 already written", before.count)
}
if _, ok := hw.files["src:sig2"]; !ok {
t.Fatal("newly declared signal was not opened")
}
}
// The C++ UDPStreamer declares samplingRate=0, so sizing the file on the spot
// would use a guess that is three orders of magnitude out at 1 MSps.
func TestHistoryDefersSignalsWithoutDeclaredRate(t *testing.T) {
dir := t.TempDir()
hw, _ := newHistoryWriter(HistoryConfig{Directory: dir})
defer hw.close()
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "fast", TypeCode: 8, SamplingRate: 0},
{Name: "known", TypeCode: 8, SamplingRate: 100},
})
if _, ok := hw.files["src:fast"]; ok {
t.Fatal("undeclared-rate signal was sized before its rate was measured")
}
if got := hw.pendingKeys(); len(got) != 1 || got[0] != "src:fast" {
t.Fatalf("pendingKeys = %v, want [src:fast]", got)
}
if _, ok := hw.files["src:known"]; !ok {
t.Fatal("declared-rate signal was deferred")
}
// Data for a deferred signal is dropped, not misfiled.
hw.write("src:fast", []float64{1}, []float64{1})
// A repeated CONFIG must not queue it twice.
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "fast", TypeCode: 8, SamplingRate: 0},
})
if got := hw.pendingKeys(); len(got) != 1 {
t.Fatalf("pendingKeys = %v after re-CONFIG, want one entry", got)
}
if !hw.openPending("src:fast", 100000) {
t.Fatal("openPending refused a measured rate")
}
hf, ok := hw.files["src:fast"]
if !ok {
t.Fatal("file not opened after the rate was measured")
}
// The default window × 100 kSps, enveloped if it does not fit the budget.
wantCap, wantBucket := histCapacityFor(defaultLiveWindowSec, 100000, 1, histDefaultMaxPoints)
if hf.capacity != wantCap || hf.bucket != wantBucket {
t.Fatalf("capacity/bucket = %d/%d, want %d/%d", hf.capacity, hf.bucket, wantCap, wantBucket)
}
if len(hw.pendingKeys()) != 0 {
t.Fatal("signal still pending after being opened")
}
if hw.openPending("src:fast", 100000) {
t.Fatal("openPending reopened an already-open signal")
}
}
func TestOpenPendingHistoryFilesUsesMeasuredRate(t *testing.T) {
h := NewHub()
if err := h.EnableHistory(HistoryConfig{Directory: t.TempDir(), WindowSec: 3.6}); err != nil {
t.Fatalf("EnableHistory: %v", err)
}
defer h.CloseHistory()
h.hist.onSourceConfigured("s1", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 0},
})
rb := newSigRing(200000)
h.rings["s1:sig"] = rb
// Too little data to measure a rate from: the sweep must wait rather than
// size the file from a burst.
fillRing(rb, 0, 100000, 100) // 1 ms of data
h.openPendingHistoryFiles(100)
if len(h.hist.pendingKeys()) != 1 {
t.Fatal("sweep sized the file from a sub-millisecond sample")
}
fillRing(rb, 0, 100000, 100000) // 1 s at 100 kSps
h.openPendingHistoryFiles(200)
hf, ok := h.hist.files["s1:sig"]
if !ok {
t.Fatal("file not opened once the rate was measurable")
}
// 3.6 s at ~100 kSps, plus headroom, ≈ 450 000 pairs; a fixed 1 kHz guess
// would have produced the 1000-sample floor instead.
if hf.capacity < 400_000 || hf.capacity > 500_000 {
t.Fatalf("capacity = %d, want ~450000 from the measured 100 kSps", hf.capacity)
}
}
func TestOpenPendingHistoryFilesIsThrottled(t *testing.T) {
h := NewHub()
if err := h.EnableHistory(HistoryConfig{Directory: t.TempDir()}); err != nil {
t.Fatalf("EnableHistory: %v", err)
}
defer h.CloseHistory()
h.hist.onSourceConfigured("s1", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 0},
})
h.openPendingHistoryFiles(100) // no ring yet: nothing to measure
rb := newSigRing(20000)
fillRing(rb, 0, 1000, 20000)
h.rings["s1:sig"] = rb
h.openPendingHistoryFiles(100.5)
if len(h.hist.files) != 0 {
t.Fatal("sweep ran inside the throttle window")
}
h.openPendingHistoryFiles(200)
if len(h.hist.files) != 1 {
t.Fatal("sweep did not run after the throttle window elapsed")
}
}
// A hub without history must tolerate the sweep, since Run() calls it every tick.
func TestOpenPendingHistoryFilesNoopWithoutHistory(t *testing.T) {
h := NewHub()
h.openPendingHistoryFiles(100)
}
func TestHistoryInfoShape(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 0.36}, 1)
// Reported before any data arrives, so clients can enable their history UI.
inf := hw.info()
if e, ok := inf[key]; !ok || e.Count != 0 || e.Capacity != histMinCapacity {
t.Fatalf("pre-data info = %+v (present=%v)", inf[key], ok)
}
ts, vs := ramp(5, 1, 10)
hw.write(key, ts, vs)
e := hw.info()[key]
if e.Count != 10 || e.T0 != 5 || e.T1 != 14 {
t.Fatalf("info = %+v, want count=10 t0=5 t1=14", e)
}
}
func TestHistoryHeaderIsPersistedOnFlush(t *testing.T) {
dir := t.TempDir()
hw, key := newTestHistory(t, HistoryConfig{Directory: dir, WindowSec: 0.36, Decimation: 2}, 1)
ts, vs := ramp(0, 1, 20)
hw.write(key, ts, vs)
hw.flushHeaders()
hdr, err := os.ReadFile(filepath.Join(dir, "src", "sig.shist"))
if err != nil {
t.Fatalf("read: %v", err)
}
if string(hdr[0:4]) != "SHR1" {
t.Fatalf("magic = %q", hdr[0:4])
}
if v := binary.LittleEndian.Uint32(hdr[4:]); v != histVersion {
t.Fatalf("version = %d, want %d", v, histVersion)
}
if c := binary.LittleEndian.Uint32(hdr[8:]); c != histMinCapacity {
t.Fatalf("capacity = %d, want %d", c, histMinCapacity)
}
if h := binary.LittleEndian.Uint32(hdr[12:]); h != 10 {
t.Fatalf("head = %d, want 10 (20 samples, decimation 2)", h)
}
if n := binary.LittleEndian.Uint32(hdr[16:]); n != 10 {
t.Fatalf("count = %d, want 10", n)
}
if d := binary.LittleEndian.Uint32(hdr[20:]); d != 2 {
t.Fatalf("decimation = %d, want 2", d)
}
if got := math.Float64frombits(binary.LittleEndian.Uint64(hdr[32:])); got != 19 {
t.Fatalf("tNewest = %v, want 19", got)
}
// The data region must be pre-allocated in full, not grown as it fills.
if want := int64(histHeaderSize) + histMinCapacity*histPairSize; int64(len(hdr)) != want {
t.Fatalf("file size = %d, want the pre-allocated %d", len(hdr), want)
}
}
func TestSanitizeHistName(t *testing.T) {
cases := map[string]string{
"Signal_1": "Signal_1",
"GAM.Out[0]": "GAM.Out[0]",
"a/b": "a_b",
"../../etc/pass": ".._.._etc_pass",
"": "_",
".": "_",
"..": "_",
"with space": "with_space",
"nul\x00byte": "nul_byte",
}
for in, want := range cases {
if got := sanitizeHistName(in); got != want {
t.Errorf("sanitizeHistName(%q) = %q, want %q", in, got, want)
}
}
}
// A producer-supplied name must never place a file outside the history dir.
func TestHistoryNameCannotEscapeDirectory(t *testing.T) {
dir := t.TempDir()
hw, _ := newHistoryWriter(HistoryConfig{Directory: dir})
defer hw.close()
hw.onSourceConfigured("../evil", []udpsprotocol.SignalInfo{
{Name: "../../pwned", TypeCode: 8, SamplingRate: 1},
})
found := false
err := filepath.Walk(dir, func(p string, info os.FileInfo, err error) error {
if err == nil && !info.IsDir() {
found = true
}
return err
})
if err != nil {
t.Fatalf("walk: %v", err)
}
if !found {
t.Fatal("no file created inside the history directory")
}
if _, err := os.Stat(filepath.Join(dir, "..", "..", "pwned.shist")); err == nil {
t.Fatal("a file escaped the history directory")
}
}
func TestHistCapacityFor(t *testing.T) {
cases := []struct {
window float64
rate float64
decim int
maxPts int
want uint32
wantBucket int
}{
// window × rate / decimation, plus the 1.25 headroom.
{600, 1000, 1, 0, 750_000, 1},
{600, 1000, 10, 0, 75_000, 1},
{10, 100, 1, 0, 1250, 1},
{600, 0.001, 1, 0, histMinCapacity, 1}, // absurdly slow → the floor
// Absurdly fast: bounded by histMaxCapacity, and the window is bought with
// a correspondingly absurd bucket rather than by storing less of it.
{600, 1e9, 1, 0, 1_073_729_421, 1397},
{math.NaN(), 1000, 1, 0, histMinCapacity, 1},
{600, math.NaN(), 1, 0, histMinCapacity, 1},
// A budget envelopes a fast signal without touching a slow one, and the
// window is kept either way.
{600, 1e6, 1, 16 << 20, 16_666_667, 90},
{600, 1000, 1, 16 << 20, 750_000, 1},
}
for _, c := range cases {
got, bucket := histCapacityFor(c.window, c.rate, c.decim, c.maxPts)
if got != c.want || bucket != c.wantBucket {
t.Errorf("histCapacityFor(%v, %v, %d, %d) = %d/%d, want %d/%d",
c.window, c.rate, c.decim, c.maxPts, got, bucket, c.want, c.wantBucket)
}
}
}
func TestHistoryConfigDefaults(t *testing.T) {
c := HistoryConfig{}.withDefaults()
if c.WindowSec != defaultLiveWindowSec || c.Decimation != 1 || c.FlushIntervalSec != 5 || c.MinDiskFreeMB != 500 {
t.Fatalf("defaults = %+v", c)
}
// A negative value is the explicit "no disk guard", so it must survive
// defaulting rather than being turned back into 500.
if got := (HistoryConfig{MinDiskFreeMB: -1}).withDefaults().MinDiskFreeMB; got != -1 {
t.Fatalf("MinDiskFreeMB = %d, want the -1 that disables the guard", got)
}
}
func TestHistoryWritePausedWhenDiskLow(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{}, 100)
hw.diskLow = true
hw.write(key, []float64{1, 2, 3}, []float64{1, 2, 3})
if hw.files[key].count != 0 {
t.Fatalf("count = %d, want 0 while the disk guard is tripped", hw.files[key].count)
}
hw.diskLow = false
hw.write(key, []float64{1, 2, 3}, []float64{1, 2, 3})
if hw.files[key].count != 3 {
t.Fatalf("count = %d, want 3 once writing resumes", hw.files[key].count)
}
}
func TestHistoryReadRangeRespectsMaxOut(t *testing.T) {
// A window wide enough that the whole ramp is still on disk when it is read.
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 50}, 100)
ts, vs := ramp(0, 0.01, 5000)
hw.write(key, ts, vs)
rt, rv := hw.readRange(key, -1, 1e9, 100)
if len(rt) != 100 || len(rv) != 100 {
t.Fatalf("read %d/%d points, want the 100 cap", len(rt), len(rv))
}
}
func TestHistoryReadRangeSpansWholeRange(t *testing.T) {
// A capped read must thin the range out, not return its first maxOut
// samples: a client asking for 100 points over 50 s and getting the first
// second of it draws a flat line and falls back to its coarse copy.
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 50}, 100)
ts, vs := ramp(0, 0.01, 5000)
hw.write(key, ts, vs)
rt, _ := hw.readRange(key, 0, 49.99, 100)
if len(rt) == 0 {
t.Fatal("no points read")
}
if got := rt[len(rt)-1] - rt[0]; got < 0.95*49.99 {
t.Fatalf("read spans %.2f s of the 49.99 s asked; a capped read must "+
"cover the whole range", got)
}
}
func TestHistoryReadRangeUncappedIsExact(t *testing.T) {
// Below the cap every sample in the range comes back, so a zoom deep enough
// to fit is served at full resolution.
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 50}, 100)
ts, vs := ramp(0, 0.01, 5000)
hw.write(key, ts, vs)
rt, rv := hw.readRange(key, 1, 1.99, 1000)
if len(rt) != 100 {
t.Fatalf("read %d points, want the 100 samples in [1, 1.99]", len(rt))
}
if rv[0] != 100 || rv[len(rv)-1] != 199 {
t.Fatalf("values %.0f..%.0f, want 100..199", rv[0], rv[len(rv)-1])
}
}
// The capture copy is what makes a trigger window zoomable long after the
// circular archive has wrapped over it.
func TestCaptureRangeOutlivesTheArchive(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{}, 0.001) // floor capacity: 1000
hf := hw.files[key]
if hf.capacity != histMinCapacity {
t.Fatalf("capacity = %d, want the %d floor", hf.capacity, histMinCapacity)
}
ts, vs := ramp(0, 1, 1000) // t = 0..999, exactly full
hw.write(key, ts, vs)
hw.captureRange(500, 600)
// Wrap the archive right over the captured window.
ts2, vs2 := ramp(1000, 1, 1000)
hw.write(key, ts2, vs2)
if hf.tOldest != 1000 || hf.tNewest != 1999 {
t.Fatalf("archive holds [%v, %v], want [1000, 1999]: capturing must not "+
"stop or divert the archive", hf.tOldest, hf.tNewest)
}
rt, rv := hw.readRange(key, 500, 600, 1000)
if len(rt) != 101 {
t.Fatalf("read %d captured samples in [500, 600], want 101", len(rt))
}
if rv[0] != 500 || rv[len(rv)-1] != 600 {
t.Fatalf("captured values %.0f..%.0f, want 500..600", rv[0], rv[len(rv)-1])
}
// A range the capture does not hold is still answered by the archive.
if at, _ := hw.readRange(key, 1500, 1600, 1000); len(at) != 101 {
t.Fatalf("read %d archived samples in [1500, 1600], want 101", len(at))
}
// The next capture replaces the last one, and only then.
hw.captureRange(1500, 1600)
if ct, _ := hw.readRange(key, 500, 600, 1000); len(ct) != 0 {
t.Fatalf("read %d samples of a replaced capture, want 0", len(ct))
}
}
// Delivering a capture copies its window out of the archive, and the archive
// keeps rolling so the next capture's pre-trigger window is there when it fires.
func TestTriggerCaptureCopiesWindowToDisk(t *testing.T) {
h := NewHub()
if err := h.EnableHistory(HistoryConfig{
Directory: t.TempDir(), WindowSec: 36, MinDiskFreeMB: -1,
}); err != nil {
t.Fatalf("EnableHistory: %v", err)
}
t.Cleanup(h.CloseHistory)
h.hist.onSourceConfigured("s1", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 1000},
})
h.rings["s1:sig"] = newSigRing(10000)
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", edge: "rising", threshold: 0,
windowSec: 1, prePercent: 20, mode: "single"})
h.trigger.Arm()
// Cross the threshold, then cover the post-trigger window so the capture
// comes due on the next tick.
h.ingest("s1:sig", 1, []float64{5.0, 5.001}, []float64{-1, 1})
h.ingest("s1:sig", 1, []float64{6.0}, []float64{1})
h.triggerTick()
if h.trigger.State() != trigTriggered {
t.Fatalf("state = %q, want triggered", h.trigger.State())
}
cf := h.hist.captures["s1:sig"]
if cf == nil {
t.Fatal("capture delivered but its window was not copied to disk")
}
// The window is [trigTime-0.2, trigTime+0.8] around the 5.001 crossing, so
// the sample at 6.0 falls outside it.
if cf.count != 2 || cf.tOldest != 5.0 || cf.tNewest != 5.001 {
t.Fatalf("capture holds %d samples in [%v, %v], want 2 in [5, 5.001]",
cf.count, cf.tOldest, cf.tNewest)
}
// Copying the window leaves the archive rolling, so the next capture's
// pre-trigger window — written before its trigger fires — is there for it.
h.ingest("s1:sig", 1, []float64{7.0}, []float64{1})
if got := h.hist.files["s1:sig"].count; got != 4 {
t.Fatalf("archived %d samples, want 4: capturing must not stop writing", got)
}
// Rearming does not discard the capture: it stays on screen until the next
// trigger replaces it.
h.trigger.Arm()
h.triggerTick()
if h.hist.captures["s1:sig"] != cf {
t.Fatal("rearming discarded the capture the client is still showing")
}
}
func TestHistSearch(t *testing.T) {
vals := []float64{0, 1, 2, 3, 4, 5}
at := func(i uint32) float64 { return vals[i] }
if got := histSearch(0, 6, func(i uint32) bool { return at(i) < 3 }); got != 3 {
t.Fatalf("lower bound = %d, want 3", got)
}
if got := histSearch(0, 6, func(i uint32) bool { return at(i) <= 3 }); got != 4 {
t.Fatalf("upper bound = %d, want 4", got)
}
if got := histSearch(0, 6, func(i uint32) bool { return at(i) < -1 }); got != 0 {
t.Fatalf("all-false = %d, want 0", got)
}
if got := histSearch(0, 6, func(i uint32) bool { return at(i) < 100 }); got != 6 {
t.Fatalf("all-true = %d, want 6", got)
}
}
Common/Client/go/wshub/hub.go
+544 -139
View File
@@ -9,6 +9,7 @@ import (
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"unsafe"
@@ -27,6 +28,29 @@ type wsClient struct {
hub *Hub
conn *websocket.Conn
send chan wsMessage
// window is the timespan this client is displaying, in seconds, held as
// float64 bits. The retune sweep sizes the rings from the widest window in
// use, so it must be readable from the hub goroutine while readPump writes
// it. Zero means the client has not said, and the default applies.
window atomic.Uint64
}
func (c *wsClient) setDisplayWindowSec(s float64) {
c.window.Store(math.Float64bits(s))
}
func (c *wsClient) displayWindowSec() float64 {
return math.Float64frombits(c.window.Load())
}
// sendText enqueues one JSON frame for this client, dropping it if the client
// is not draining its queue.
func (c *wsClient) sendText(msg []byte) {
select {
case c.send <- wsMessage{websocket.TextMessage, msg}:
default:
}
}
func (c *wsClient) writePump() {
@@ -107,7 +131,49 @@ func (c *wsClient) readPump() {
case c.hub.commandCh <- hubCmd{op: "wsSaveSources"}:
default:
}
case "setCalibration":
source, _ := env["source"].(string)
signal, _ := env["signal"].(string)
scale, hasScale := env["scale"].(float64)
if !hasScale {
scale = 1
}
offset, _ := env["offset"].(float64)
unit, _ := env["unit"].(string)
select {
case c.hub.commandCh <- hubCmd{op: "wsSetCalibration", cal: CalConfig{
Source: source, Signal: signal,
Scale: scale, Offset: offset, Unit: unit,
}}:
default:
}
case "reloadConfig":
select {
case c.hub.commandCh <- hubCmd{op: "wsReloadConfig"}:
default:
}
case "setWindow":
// Sizes the zoom rings: the hub cannot know how far back a
// client is plotting, and a window it has not been told
// about is a window the buffers may not reach.
if sec, ok := env["seconds"].(float64); ok && sec > 0 && !math.IsInf(sec, 0) {
c.setDisplayWindowSec(sec)
}
case "setMonotonic":
enabled, _ := env["enabled"].(bool)
select {
case c.hub.commandCh <- hubCmd{op: "setMonotonic", enabled: enabled}:
default:
}
case "zoom":
c.hub.handleWSZoom(c, env)
default:
if c.hub.handleTriggerCommand(t, env) {
break
}
if c.hub.handleHistoryCommand(c, t, env) {
break
}
// Unrecognized message type — forward to DebugCh
select {
case c.hub.DebugCh <- msg:
@@ -122,10 +188,48 @@ func (c *wsClient) readPump() {
// ─── Hub ─────────────────────────────────────────────────────────────────────
// allowedOrigins is the set of Origin values (scheme://host[:port]) that are
// accepted for WebSocket upgrades. If empty, same-origin is enforced by
// comparing the Origin's host to the HTTP Host header.
var allowedOrigins []string
// SetAllowedOrigins configures the WebSocket Origin allowlist. Pass an empty
// slice to enforce same-origin only (the default).
func SetAllowedOrigins(origins []string) {
allowedOrigins = origins
}
// checkOrigin validates the Origin header against the allowlist, falling back
// to a same-origin check (Origin host == Host header) when no allowlist is
// configured. Requests with no Origin header (non-browser clients) are allowed.
func checkOrigin(r *http.Request) bool {
origin := r.Header.Get("Origin")
if origin == "" {
return true // non-browser client
}
// Check explicit allowlist first.
for _, allowed := range allowedOrigins {
if origin == allowed {
return true
}
}
// Fall back to same-origin: compare the Origin's host to the Host header.
// Origin format: "scheme://host[:port]" — strip scheme.
host := origin
if idx := strings.Index(host, "://"); idx >= 0 {
host = host[idx+3:]
}
// Strip path if present.
if idx := strings.Index(host, "/"); idx >= 0 {
host = host[:idx]
}
return host == r.Host
}
var upgrader = websocket.Upgrader{
ReadBufferSize: 4096,
WriteBufferSize: 64 * 1024,
CheckOrigin: func(r *http.Request) bool { return true },
CheckOrigin: checkOrigin,
}
// sourceHubState holds all data for one active data source.
@@ -144,6 +248,14 @@ type sourceHubState struct {
// per signal name. Used by the default (TimeModePacket, n>1) path to estimate
// per-element dt when only one packet arrives in a 30 Hz tick.
lastPktNs map[string]int64
// Monotonic timestamp snapping state (all accessed from Run() goroutine):
// lastFrameMeasured — uncorrected measured anchor of the previous frame.
// lastFrameEndT — corrected anchor after snapping.
// gapEMA — exponential moving average of the measured inter-frame gap.
lastFrameMeasured map[string]float64
lastFrameEndT map[string]float64
gapEMA map[string]float64
}
// taggedSample is a DataSample annotated with its source ID.
@@ -154,8 +266,9 @@ type taggedSample struct {
// hubCmd carries a command to the Run() goroutine.
type hubCmd struct {
op string // "addSource","removeSource","setSourceState","updateConfig",
// "wsAddSource","wsRemoveSource","wsSaveSources"
op string // "addSource","removeSource","setSourceState","updateConfig",
// "wsAddSource","wsRemoveSource","wsSaveSources",
// "wsSetCalibration","wsReloadConfig"
sourceID string
label string
addr string
@@ -163,6 +276,8 @@ type hubCmd struct {
sigs []udpsprotocol.SignalInfo
multicastGroup string
dataPort int
enabled bool // "setMonotonic" toggle
cal CalConfig // "wsSetCalibration" payload
}
// Hub is the central broker between UDP clients and WebSocket clients.
@@ -181,26 +296,42 @@ type Hub struct {
sm *SourceManager // set after construction; used for WS-initiated source changes
// cal holds the per-signal calibration table. It is metadata only: the
// rings, the history and the trigger comparator all keep raw samples.
cal *calTable
// Ring buffers for hi-res zoom data.
// ringsMu protects the map structure; each sigRing has its own RWMutex for data.
ringsMu sync.RWMutex
rings map[string]*sigRing // "sourceId:signalKey" → ring
// lastZoomAt tracks the last time a zoom request was served.
// Ring buffer writes are skipped when no zoom has been requested
// in the last 10 s, saving substantial CPU on LTTB + ring writes.
lastZoomAt time.Time
zoomAtMu sync.Mutex
// hist is the disk-backed archive behind long time windows, which hold far
// more samples than the in-memory rings can. nil when history is disabled.
// histOpenAt throttles the sweep that opens the files of signals whose
// producer declared no sampling rate; both are touched only from Run().
hist *historyWriter
histOpenAt float64
statsMu sync.RWMutex
statsMap map[string]*SourceStat
// onClientConnect, if set, is called each time a new WebSocket client
// registers. The callback receives a send function that delivers a message
// directly to that client. It is invoked synchronously from Run(), so it
// must not block.
// trigger is the hub-side trigger FSM driving the oscilloscope capture mode.
// ringTuneAt throttles the sweep that keeps each ring's depth and min/max
// bucket matched to the window being displayed; both are touched only from
// Run(). ringBudgetPts is that sweep's per-signal budget; set before Run().
trigger *triggerEngine
ringTuneAt float64
// capture is the trigger double buffer's read half: the last delivered
// capture window, kept out of the rings' way so the shot being viewed
// survives the re-arm that immediately follows it.
capture captureHold
ringBudgetPts int
onClientConnectMu sync.RWMutex
onClientConnect func(send func([]byte))
// monotonicTS, when true, snaps small inter-frame timestamp deviations
// (< monotonicTolerance) to the ideal gap to eliminate jitter.
monotonicTS bool
}
// NewHub creates an initialised Hub.
@@ -215,9 +346,49 @@ func NewHub() *Hub {
DebugCh: make(chan []byte, 256),
rings: make(map[string]*sigRing),
statsMap: make(map[string]*SourceStat),
trigger: newTriggerEngine(),
cal: newCalTable(),
}
}
// SetRingBudget overrides the per-signal in-memory buffer budget, in points.
// Non-positive values restore the default. It must be called before Run().
// Each point costs 16 bytes, so the budget is the memory bound per temporal
// signal. It does not limit how long a window can be held: a window too long
// to fit at full rate is stored as min/max pairs instead (see retuneRings).
func (h *Hub) SetRingBudget(n int) {
if n <= 0 {
n = defaultRingPts
}
if n < ringCapInitial {
n = ringCapInitial
}
h.ringBudgetPts = n
}
func (h *Hub) ringBudget() int {
if h.ringBudgetPts <= 0 {
return defaultRingPts
}
return h.ringBudgetPts
}
// EnableHistory turns on the disk-backed history archive. It must be called
// before Run(). A HistoryConfig with an empty Directory leaves history off.
func (h *Hub) EnableHistory(cfg HistoryConfig) error {
hw, err := newHistoryWriter(cfg)
if err != nil {
return err
}
h.hist = hw
return nil
}
// CloseHistory flushes and closes the history files. Without it the samples
// written since the last periodic flush are on disk but unaccounted for in the
// file headers, so a restart would not see them.
func (h *Hub) CloseHistory() { h.hist.close() }
// SetOnClientConnect registers a callback invoked synchronously (from Run())
// each time a new WebSocket client connects. The callback receives a send
// function that enqueues one message to that specific client.
@@ -232,6 +403,22 @@ func (h *Hub) SetSourceManager(sm *SourceManager) {
h.sm = sm
}
// ingest routes one batch of full-resolution samples for a signal to every
// consumer that needs them at full rate: the in-memory zoom ring, the disk
// history and the trigger comparator. The live push is decimated separately by
// the caller. The ring and the archive may reduce what they store to fit their
// budget, but they are handed every sample so the reduction sees the extrema.
func (h *Hub) ingest(key string, nElem int, t, v []float64) {
if len(t) == 0 {
return
}
if rb := h.getRing(key); rb != nil {
rb.write(t, v)
}
h.hist.write(key, t, v)
h.trigger.feed(key, nElem, t, v)
}
// getRing returns the ring buffer for a fully-prefixed signal key, or nil.
func (h *Hub) getRing(key string) *sigRing {
h.ringsMu.RLock()
@@ -240,44 +427,11 @@ func (h *Hub) getRing(key string) *sigRing {
return rb
}
// shouldWriteRing returns true if zoom was requested within the last 10 seconds.
func (h *Hub) shouldWriteRing() bool {
h.zoomAtMu.Lock()
ok := time.Since(h.lastZoomAt) < 10*time.Second
h.zoomAtMu.Unlock()
return ok
}
// HandleZoom serves GET /api/zoom?... It also records the access time
// so the ring buffer knows zoom is active and worth populating.
func (h *Hub) HandleZoom(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
t0, err0 := strconv.ParseFloat(q.Get("t0"), 64)
t1, err1 := strconv.ParseFloat(q.Get("t1"), 64)
if err0 != nil || err1 != nil || t1 <= t0 {
http.Error(w, "invalid t0/t1", http.StatusBadRequest)
return
}
var n int
if nStr := q.Get("n"); nStr == "" {
n = 2400
} else {
n, _ = strconv.Atoi(nStr)
if n <= 0 {
n = 1 << 30 // no decimation
} else if n < 10 {
n = 2400
}
}
if n > 0 {
h.zoomAtMu.Lock()
h.lastZoomAt = time.Now()
h.zoomAtMu.Unlock()
}
keys := strings.Split(q.Get("signals"), ",")
// zoomSlice extracts [t0, t1] for the named signals, decimating each to at most
// n points. A range inside the last trigger capture is served from the held
// copy of it, which the re-arming acquisition cannot overwrite; everything else
// comes from the live rings.
func (h *Hub) zoomSlice(t0, t1 float64, keys []string, n int) map[string]sigData {
h.ringsMu.RLock()
refs := make(map[string]*sigRing, len(keys))
for _, k := range keys {
@@ -293,18 +447,76 @@ func (h *Hub) HandleZoom(w http.ResponseWriter, r *http.Request) {
result := make(map[string]sigData, len(refs))
for k, rb := range refs {
rt, rv := rb.slice(t0, t1)
rt, rv, ok := h.capture.slice(k, t0, t1)
if !ok {
rt, rv = rb.slice(t0, t1)
}
if len(rt) == 0 {
continue
}
dt, dv := lttbDecimate(rt, rv, n)
dt, dv := minMaxDecimate(rt, rv, n)
result[k] = sigData{T: dt, V: dv}
}
return result
}
// zoomPoints normalises the client's requested point budget: absent → 2400,
// non-positive → every sample in the range, implausibly small → 2400.
func zoomPoints(n int, present bool) int {
switch {
case !present:
return 2400
case n <= 0:
return 1 << 30 // no decimation
case n < 10:
return 2400
}
return n
}
// handleWSZoom answers a browser {"type":"zoom","reqId":..,"t0":..,"t1":..,
// "n":..,"signals":"a,b"} request, unicasting {"type":"zoom","reqId":..,
// "signals":{...}} back to the requesting client. This is the path the web SPA
// actually uses; /api/zoom is the equivalent HTTP entry point.
func (h *Hub) handleWSZoom(c *wsClient, env map[string]interface{}) {
t0, ok0 := env["t0"].(float64)
t1, ok1 := env["t1"].(float64)
if !ok0 || !ok1 || t1 <= t0 {
return
}
nF, nOK := env["n"].(float64)
n := zoomPoints(int(nF), nOK)
sigCSV, _ := env["signals"].(string)
reply, err := json.Marshal(map[string]any{
"type": "zoom",
"reqId": env["reqId"],
"signals": h.zoomSlice(t0, t1, strings.Split(sigCSV, ","), n),
})
if err != nil {
log.Printf("hub: ws zoom encode: %v", err)
return
}
c.sendText(reply)
}
// HandleZoom serves GET /api/zoom?...
func (h *Hub) HandleZoom(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
t0, err0 := strconv.ParseFloat(q.Get("t0"), 64)
t1, err1 := strconv.ParseFloat(q.Get("t1"), 64)
if err0 != nil || err1 != nil || t1 <= t0 {
http.Error(w, "invalid t0/t1", http.StatusBadRequest)
return
}
nStr := q.Get("n")
nVal, _ := strconv.Atoi(nStr)
n := zoomPoints(nVal, nStr != "")
w.Header().Set("Content-Type", "application/json")
if err := json.NewEncoder(w).Encode(map[string]any{
"type": "zoom",
"signals": result,
"signals": h.zoomSlice(t0, t1, strings.Split(q.Get("signals"), ","), n),
}); err != nil {
log.Printf("hub: zoom encode: %v", err)
}
@@ -392,6 +604,26 @@ func buildSourcesMsg(sm map[string]*sourceHubState) []byte {
return msg
}
// buildCalibrationMsg serialises the calibration table as a "calibration"
// message. It is its own frame rather than a field on "sources" because the
// C++ BroadcastSources serialises into a fixed 4096-byte buffer that a
// calibration table would overflow.
func buildCalibrationMsg(t *calTable) []byte {
list := t.List() // never nil: the SPA replaces its table wholesale on receipt
msg, _ := json.Marshal(map[string]any{"type": "calibration", "cal": list})
return msg
}
// buildConfigAckMsg serialises a configSaved / configReloaded acknowledgement.
func buildConfigAckMsg(msgType, path string, err error) []byte {
m := map[string]any{"type": msgType, "ok": err == nil, "path": path}
if err != nil {
m["error"] = err.Error()
}
msg, _ := json.Marshal(m)
return msg
}
// Run is the hub's main goroutine. Must be started with go hub.Run().
func (h *Hub) Run() {
ticker := time.NewTicker(time.Second / 30)
@@ -400,6 +632,16 @@ func (h *Hub) Run() {
statsTicker := time.NewTicker(time.Second)
defer statsTicker.Stop()
// Header flushes are what make the archived samples findable again; the
// data region is written as it arrives. Ticks are ignored when history is
// off, so a disabled writer costs one no-op call per period.
flushPeriod := time.Duration(5) * time.Second
if h.hist.enabled() {
flushPeriod = time.Duration(h.hist.cfg.FlushIntervalSec) * time.Second
}
flushTicker := time.NewTicker(flushPeriod)
defer flushTicker.Stop()
sourcesMap := make(map[string]*sourceHubState)
var sourcesMsg []byte
@@ -417,11 +659,36 @@ func (h *Hub) Run() {
h.clients[c] = true
// Send current state to the new client.
if sourcesMsg != nil {
select { case c.send <- wsMessage{websocket.TextMessage, sourcesMsg}: default: }
select {
case c.send <- wsMessage{websocket.TextMessage, sourcesMsg}:
default:
}
}
for _, src := range sourcesMap {
if src.configJS != nil {
select { case c.send <- wsMessage{websocket.TextMessage, src.configJS}: default: }
select {
case c.send <- wsMessage{websocket.TextMessage, src.configJS}:
default:
}
}
}
select {
case c.send <- wsMessage{websocket.TextMessage, h.trigger.stateMsg()}:
default:
}
monoMsg, _ := json.Marshal(map[string]any{"type": "monotonicState", "enabled": h.monotonicTS})
select {
case c.send <- wsMessage{websocket.TextMessage, monoMsg}:
default:
}
calMsg := buildCalibrationMsg(h.cal)
select {
case c.send <- wsMessage{websocket.TextMessage, calMsg}:
default:
}
if h.hist.enabled() {
if msg := h.buildHistoryInfoMsg(); msg != nil {
c.sendText(msg)
}
}
// Notify the application layer so it can replay any persistent state
@@ -431,7 +698,10 @@ func (h *Hub) Run() {
h.onClientConnectMu.RUnlock()
if fn != nil {
fn(func(msg []byte) {
select { case c.send <- wsMessage{websocket.TextMessage, msg}: default: }
select {
case c.send <- wsMessage{websocket.TextMessage, msg}:
default:
}
})
}
@@ -443,19 +713,25 @@ func (h *Hub) Run() {
case msg := <-h.broadcastCh:
for c := range h.clients {
select { case c.send <- wsMessage{websocket.TextMessage, msg}: default: }
select {
case c.send <- wsMessage{websocket.TextMessage, msg}:
default:
}
}
case cmd := <-h.commandCh:
switch cmd.op {
case "addSource":
sourcesMap[cmd.sourceID] = &sourceHubState{
id: cmd.sourceID,
label: cmd.label,
addr: cmd.addr,
connState: "connecting",
timeSigCalib: make(map[string]float64),
lastPktNs: make(map[string]int64),
id: cmd.sourceID,
label: cmd.label,
addr: cmd.addr,
connState: "connecting",
timeSigCalib: make(map[string]float64),
lastPktNs: make(map[string]int64),
lastFrameEndT: make(map[string]float64),
lastFrameMeasured: make(map[string]float64),
gapEMA: make(map[string]float64),
}
h.statsMu.Lock()
h.statsMap[cmd.sourceID] = &SourceStat{}
@@ -491,6 +767,7 @@ func (h *Hub) Run() {
}
src.signals = cmd.sigs
src.configSeq++
src.lastFrameEndT = make(map[string]float64)
cfgMsg, err := json.Marshal(map[string]any{
"type": "config",
"sourceId": cmd.sourceID,
@@ -514,16 +791,29 @@ func (h *Hub) Run() {
ne := sig.NumElements()
isTemporal := ne > 1 && sig.TimeMode != udpsprotocol.TimeModePacket
if isTemporal {
h.rings[pfxUpd+sig.Name] = newSigRing(ringCapTemporal)
h.rings[pfxUpd+sig.Name] = newSigRing(ringCapInitial)
} else if ne == 1 {
h.rings[pfxUpd+sig.Name] = newSigRing(ringCapScalar)
} else {
// n>1, TimeModePacket snapshot-waveform: each packet contributes n
// elements, so use the temporal capacity to hold enough history.
h.rings[pfxUpd+sig.Name] = newSigRing(ringCapTemporal)
// elements, so this is a fast stream too and gets the same budget.
h.rings[pfxUpd+sig.Name] = newSigRing(ringCapInitial)
}
}
h.ringsMu.Unlock()
// The held capture describes rings that no longer exist. A
// restarted producer can even replay the same timestamps, so
// keeping it would answer zooms with the old run's samples.
h.capture.clear()
// Opening the archive files touches the filesystem, so keep it
// off the Run() goroutine; the write path simply drops samples
// for a key whose file is not open yet.
if h.hist.enabled() {
go func(id string, sigs []udpsprotocol.SignalInfo) {
h.hist.onSourceConfigured(id, sigs)
h.broadcast(h.buildHistoryInfoMsg())
}(cmd.sourceID, cmd.sigs)
}
case "wsAddSource":
if h.sm != nil {
@@ -539,10 +829,48 @@ func (h *Hub) Run() {
case "wsSaveSources":
if h.sm != nil {
if err := h.sm.Save(); err != nil {
log.Printf("hub: save sources: %v", err)
}
// Save writes to disk; run it off the Run() goroutine so a
// slow filesystem can never stall the hub loop.
go func(sm *SourceManager) {
err := sm.Save()
if err != nil {
log.Printf("hub: save config: %v", err)
}
h.broadcast(buildConfigAckMsg("configSaved", sm.Path(), err))
}(h.sm)
}
case "wsSetCalibration":
if h.cal.Set(cmd.cal) {
h.broadcast(buildCalibrationMsg(h.cal))
} else {
// No broadcast: the offending client reverts to the last
// value it was sent.
log.Printf("hub: rejected calibration %q/%q (scale=%v offset=%v)",
cmd.cal.Source, cmd.cal.Signal, cmd.cal.Scale, cmd.cal.Offset)
}
case "wsReloadConfig":
if h.sm != nil {
// Reload calls sm.Add(), which sends on commandCh; from the
// Run() goroutine that send would hit the non-blocking
// default and be dropped, so it must run elsewhere.
go func(sm *SourceManager) {
err := sm.Reload()
if err != nil {
log.Printf("hub: reload config: %v", err)
}
h.broadcast(buildConfigAckMsg("configReloaded", sm.Path(), err))
if err == nil {
h.broadcast(buildCalibrationMsg(h.cal))
}
}(h.sm)
}
case "setMonotonic":
h.monotonicTS = cmd.enabled
monoMsg, _ := json.Marshal(map[string]any{"type": "monotonicState", "enabled": h.monotonicTS})
h.broadcast(monoMsg)
}
case ts := <-h.dataCh:
@@ -554,10 +882,15 @@ func (h *Hub) Run() {
continue
}
src, ok := sourcesMap[srcID]
if !ok || len(src.signals) == 0 || len(h.clients) == 0 {
if !ok || len(src.signals) == 0 {
pending[srcID] = pending[srcID][:0]
continue
}
// Built even with no clients connected: this is also what feeds
// the rings, the disk history and the trigger, none of which may
// stop just because nobody is watching. It also keeps the push
// cursors advancing, so the first client to connect does not get
// a backlog burst. Matches the C++ StreamHub.
msg := h.buildBinaryDataMessageForSource(src, samples)
pending[srcID] = pending[srcID][:0]
if msg != nil {
@@ -569,6 +902,10 @@ func (h *Hub) Run() {
}
}
}
h.triggerTick()
case <-flushTicker.C:
h.hist.flushHeaders()
case <-statsTicker.C:
h.statsMu.RLock()
@@ -602,53 +939,91 @@ func writeFloat64s(buf []byte, off int, f []float64) int {
// ─── Data serialisation ───────────────────────────────────────────────────────
// maxPushPoints bounds the live push only. The zoom rings deliberately store
// every sample: decimating on the way in would cap the resolution a zoom can
// ever recover, and the browser already decimates for display.
const maxPushPoints = 50
const maxRingPoints = 20_000
const ringCapTemporal = 6_000_000
// Ring geometry, in samples per signal (16 bytes each).
//
// defaultRingPts is the per-signal memory budget for temporal (array) signals:
// what the hub may spend keeping one signal available for zoom and for trigger
// captures. 10 M points is 160 MB. The budget buys resolution, not span —
// retuneRings buckets the input so the display window fits whatever the source
// rate is.
//
// ringCapInitial is where a ring starts, so a source that is configured but
// never sends costs nothing; the first retune sweep grows it to the budget.
//
// ringCapScalar sizes scalar signals, which arrive at the packet rate and would
// squander a budget meant for megasample streams.
const defaultRingPts = 10_000_000
const ringCapInitial = 250_000
const ringCapScalar = 100_000
// lttbDecimate reduces (tIn, vIn) to at most threshold representative points
// using the Largest-Triangle-Three-Buckets algorithm.
func lttbDecimate(tIn, vIn []float64, threshold int) ([]float64, []float64) {
// monotonicTolerance is the maximum inter-frame timestamp deviation (seconds)
// treated as jitter and snapped to the ideal gap. Larger deviations are
// preserved as genuine discontinuities (missing frames, rate changes).
const monotonicTolerance = 0.005 // 5 ms
// monotonicEMAAlpha is the smoothing factor for the inter-frame gap EMA.
// 0.01 gives a time constant of ~100 frames (~1 s at 100 Hz): fast enough to
// track real rate changes, slow enough to average out per-frame jitter.
const monotonicEMAAlpha = 0.01
// minMaxDecimate reduces (tIn, vIn) to at most threshold points the way an
// oscilloscope draws a trace it cannot show pixel-for-pixel: the range is split
// into threshold/2 equal buckets and each contributes its smallest and largest
// sample, in the order the two occurred.
//
// This is what replaced LTTB on every path here. LTTB picks the sample that
// makes the largest triangle with its neighbours, which reads as a plausible
// shape but silently drops a one-sample spike whenever a smoother neighbour
// scores higher — precisely the sample the user is looking for. The envelope
// cannot drop it: a spike is by definition its bucket's min or max. The cost is
// that a flat trace is drawn as a band rather than a line, which is how a scope
// behaves too.
//
// Both output arrays hold real samples with their real timestamps; nothing is
// interpolated or averaged.
func minMaxDecimate(tIn, vIn []float64, threshold int) ([]float64, []float64) {
n := len(tIn)
if n <= threshold || threshold < 3 {
// Below four there is no room for a single min/max pair plus endpoints.
if n <= threshold || threshold < 4 {
return tIn, vIn
}
outT := make([]float64, threshold)
outV := make([]float64, threshold)
outT[0], outV[0] = tIn[0], vIn[0]
outT[threshold-1], outV[threshold-1] = tIn[n-1], vIn[n-1]
every := float64(n-2) / float64(threshold-2)
a := 0
for i := 0; i < threshold-2; i++ {
avgS := int(float64(i+1)*every) + 1
avgE := int(float64(i+2)*every) + 1
if avgE > n {
avgE = n
buckets := threshold / 2
outT := make([]float64, 0, threshold)
outV := make([]float64, 0, threshold)
for b := 0; b < buckets; b++ {
lo := b * n / buckets
hi := (b + 1) * n / buckets
if b == buckets-1 {
hi = n
}
avgT, avgV, cnt := 0.0, 0.0, 0
for j := avgS; j < avgE; j++ {
avgT += tIn[j]; avgV += vIn[j]; cnt++
if lo >= hi {
continue
}
if cnt > 0 {
avgT /= float64(cnt); avgV /= float64(cnt)
}
rS := int(float64(i)*every) + 1
rE := int(float64(i+1)*every) + 1
if rE > n {
rE = n
}
maxArea, next := -1.0, rS
aT, aV := tIn[a], vIn[a]
for j := rS; j < rE; j++ {
area := math.Abs((aT-avgT)*(vIn[j]-aV) - (aT-tIn[j])*(avgV-aV))
if area > maxArea {
maxArea = area; next = j
iMin, iMax := lo, lo
for j := lo + 1; j < hi; j++ {
if vIn[j] < vIn[iMin] {
iMin = j
}
if vIn[j] > vIn[iMax] {
iMax = j
}
}
outT[i+1], outV[i+1] = tIn[next], vIn[next]
a = next
// Emit in time order so the result plots as one ascending trace.
if iMin > iMax {
iMin, iMax = iMax, iMin
}
outT = append(outT, tIn[iMin])
outV = append(outV, vIn[iMin])
// A bucket whose samples are all equal has one extreme, not two.
if iMax != iMin {
outT = append(outT, tIn[iMax])
outV = append(outV, vIn[iMax])
}
}
return outT, outV
}
@@ -672,11 +1047,13 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
if src.configSeq != src.configSeqAtCalib {
src.configSeqAtCalib = src.configSeq
src.timeSigCalib = make(map[string]float64)
src.lastFrameEndT = make(map[string]float64)
src.lastFrameMeasured = make(map[string]float64)
src.gapEMA = make(map[string]float64)
}
sigs := src.signals
pfx := src.id + ":"
writeRing := h.shouldWriteRing()
type pairBuf struct {
t, v []float64
@@ -728,6 +1105,25 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
anchorTime = float64(s.WallTime.UnixNano()) / 1e9
anchorIsFirstSample = false
}
if h.monotonicTS && dt > 0 {
nominalGap := float64(n) * dt
measuredAnchor := anchorTime
if prevMeasured, ok := src.lastFrameMeasured[sig.Name]; ok {
measuredGap := measuredAnchor - prevMeasured
prevEMA, hasEMA := src.gapEMA[sig.Name]
if !hasEMA {
prevEMA = nominalGap
}
src.gapEMA[sig.Name] = prevEMA*(1-monotonicEMAAlpha) + measuredGap*monotonicEMAAlpha
smoothedGap := src.gapEMA[sig.Name]
deviation := math.Abs(measuredGap - smoothedGap)
if deviation > 0 && deviation < monotonicTolerance {
anchorTime = src.lastFrameEndT[sig.Name] + smoothedGap
}
}
src.lastFrameMeasured[sig.Name] = measuredAnchor
src.lastFrameEndT[sig.Name] = anchorTime
}
for k := 0; k < n; k++ {
var t float64
if anchorIsFirstSample {
@@ -739,13 +1135,8 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
allV = append(allV, vals[k])
}
}
if writeRing {
ringT, ringV := lttbDecimate(allT, allV, maxRingPoints)
if rb := h.getRing(pfx + sig.Name); rb != nil {
rb.write(ringT, ringV)
}
}
decimT, decimV := lttbDecimate(allT, allV, maxPushPoints)
h.ingest(pfx+sig.Name, n, allT, allV)
decimT, decimV := minMaxDecimate(allT, allV, maxPushPoints)
pairs[sig.Name] = pairBuf{t: decimT, v: decimV}
case sig.TimeMode == udpsprotocol.TimeModeFullArray:
@@ -787,13 +1178,8 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
allV = append(allV, vals[k])
}
}
if writeRing {
ringT, ringV := lttbDecimate(allT, allV, maxRingPoints)
if rb := h.getRing(pfx + sig.Name); rb != nil {
rb.write(ringT, ringV)
}
}
decimT, decimV := lttbDecimate(allT, allV, maxPushPoints)
h.ingest(pfx+sig.Name, n, allT, allV)
decimT, decimV := minMaxDecimate(allT, allV, maxPushPoints)
pairs[sig.Name] = pairBuf{t: decimT, v: decimV}
case n == 1:
@@ -807,25 +1193,19 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
ts = append(ts, float64(s.WallTime.UnixNano())/1e9)
vs = append(vs, vals[0])
}
if writeRing {
if rb := h.getRing(pfx + sig.Name); rb != nil {
rb.write(ts, vs)
}
}
h.ingest(pfx+sig.Name, 1, ts, vs)
pairs[sig.Name] = pairBuf{t: ts, v: vs}
default:
// n > 1, TimeModePacket: C++ sends samplingRate=0 so we interpolate
// per-element timestamps from wall-clock differences between packets.
//
// Three fixes vs the naïve approach:
// Two fixes vs the naïve approach:
// 1. Use src.lastPktNs[name] for the single-packet case so dt is
// estimated from the actual inter-packet gap, not 1/n.
// 2. Send all n elements to the browser without LTTB so sinusoidal
// waveforms are not degraded (packets arrive at ≤30 Hz, bandwidth
// is trivially acceptable).
// 3. Always write the ring buffer regardless of shouldWriteRing() so
// the first zoom request immediately returns full-resolution data.
allT := make([]float64, 0, len(batch)*n)
allV := make([]float64, 0, len(batch)*n)
for bi, s := range batch {
@@ -838,19 +1218,38 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
var dtSec float64
if bi+1 < len(batch) {
// Two consecutive packets in this tick → exact dt.
dtSec = (float64(batch[bi+1].WallTime.UnixNano())-float64(wallNs))/1e9/float64(n)
dtSec = (float64(batch[bi+1].WallTime.UnixNano()) - float64(wallNs)) / 1e9 / float64(n)
} else if bi > 0 {
// Last of multiple packets → use diff from previous.
dtSec = (float64(wallNs)-float64(batch[bi-1].WallTime.UnixNano()))/1e9/float64(n)
dtSec = (float64(wallNs) - float64(batch[bi-1].WallTime.UnixNano())) / 1e9 / float64(n)
} else if prevNs, ok2 := src.lastPktNs[sig.Name]; ok2 && prevNs > 0 && wallNs > prevNs {
// Single packet this tick → gap from the previous tick's packet.
dtSec = (float64(wallNs)-float64(prevNs))/1e9/float64(n)
dtSec = (float64(wallNs) - float64(prevNs)) / 1e9 / float64(n)
} else {
// Truly first packet ever — inter-packet timing unknown.
// Skip to avoid poisoning the ring with wrongly-spaced timestamps;
// lastPktNs will be recorded below so the next packet uses correct dt.
continue
}
if h.monotonicTS && dtSec > 0 {
nominalGap := float64(n) * dtSec
measuredStart := wallSec
if prevMeasured, ok := src.lastFrameMeasured[sig.Name]; ok {
measuredGap := measuredStart - prevMeasured
prevEMA, hasEMA := src.gapEMA[sig.Name]
if !hasEMA {
prevEMA = nominalGap
}
src.gapEMA[sig.Name] = prevEMA*(1-monotonicEMAAlpha) + measuredGap*monotonicEMAAlpha
smoothedGap := src.gapEMA[sig.Name]
deviation := math.Abs(measuredGap - smoothedGap)
if deviation > 0 && deviation < monotonicTolerance {
wallSec = src.lastFrameEndT[sig.Name] + smoothedGap
}
}
src.lastFrameMeasured[sig.Name] = measuredStart
src.lastFrameEndT[sig.Name] = wallSec
}
for j := 0; j < n; j++ {
allT = append(allT, wallSec+float64(j)*dtSec)
allV = append(allV, vals[j])
@@ -860,13 +1259,19 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
src.lastPktNs[sig.Name] = batch[len(batch)-1].WallTime.UnixNano()
}
if len(allT) > 0 {
// Ring: always populate (fix 3), LTTB only if it actually reduces size.
ringT, ringV := lttbDecimate(allT, allV, maxRingPoints)
if rb := h.getRing(pfx + sig.Name); rb != nil {
rb.write(ringT, ringV)
h.ingest(pfx+sig.Name, n, allT, allV)
// Live push: never below one packet's worth of elements, or LTTB
// would flatten the snapshot waveform itself; never above it
// either, since anything more is just packets that piled up
// during the tick. Pushing every point unconditionally does not
// survive a fast producer: a 5 kHz x 1000-element array is 5M
// points/s on the wire and the client queue never drains.
thr := maxPushPoints
if n > thr {
thr = n
}
// Live push: send all points without LTTB (fix 2).
pairs[sig.Name] = pairBuf{t: allT, v: allV}
decimT, decimV := minMaxDecimate(allT, allV, thr)
pairs[sig.Name] = pairBuf{t: decimT, v: decimV}
}
}
}
Common/Client/go/wshub/hub_calibration_test.go
+132
View File
@@ -0,0 +1,132 @@
package wshub
import (
"encoding/json"
"errors"
"testing"
"time"
)
func TestBuildCalibrationMsg(t *testing.T) {
tab := newCalTable()
tab.Set(CalConfig{Source: "wave", Signal: "Adc", Scale: 0.5, Offset: -1.25, Unit: "V"})
var got struct {
Type string `json:"type"`
Cal []CalConfig `json:"cal"`
}
raw := buildCalibrationMsg(tab)
if err := json.Unmarshal(raw, &got); err != nil {
t.Fatalf("unmarshal %s: %v", raw, err)
}
if got.Type != "calibration" {
t.Errorf("type = %q, want calibration", got.Type)
}
if len(got.Cal) != 1 || got.Cal[0] != (CalConfig{
Source: "wave", Signal: "Adc", Scale: 0.5, Offset: -1.25, Unit: "V"}) {
t.Errorf("cal = %+v", got.Cal)
}
}
func TestBuildCalibrationMsgEmptyTableIsEmptyArray(t *testing.T) {
// The SPA replaces its table wholesale on every calibration message, so an
// empty table must serialise as [] and not as null.
raw := buildCalibrationMsg(newCalTable())
var got struct {
Cal []CalConfig `json:"cal"`
}
if err := json.Unmarshal(raw, &got); err != nil {
t.Fatalf("unmarshal %s: %v", raw, err)
}
if got.Cal == nil {
t.Errorf("cal = null, want []; raw = %s", raw)
}
}
func TestBuildConfigAckMsg(t *testing.T) {
ok := buildConfigAckMsg("configSaved", "/tmp/x.json", nil)
var m map[string]any
if err := json.Unmarshal(ok, &m); err != nil {
t.Fatal(err)
}
if m["type"] != "configSaved" || m["ok"] != true || m["path"] != "/tmp/x.json" {
t.Errorf("success ack = %s", ok)
}
if _, has := m["error"]; has {
t.Errorf("success ack carries an error field: %s", ok)
}
bad := buildConfigAckMsg("configReloaded", "", errors.New("boom"))
m = nil
if err := json.Unmarshal(bad, &m); err != nil {
t.Fatal(err)
}
if m["type"] != "configReloaded" || m["ok"] != false || m["error"] != "boom" {
t.Errorf("failure ack = %s", bad)
}
}
func TestHubSetCalibrationCommand(t *testing.T) {
h := NewHub()
go h.Run()
// Register a client before sending commands so broadcasts are observable.
sendCh := make(chan wsMessage, 64)
c := &wsClient{hub: h, send: sendCh}
h.register <- c
sleepMillis(20) // let Run() process the register and flush initial state msgs
drainSendCh(sendCh) // discard state-sync messages (sources, trigger, cal, ...)
h.commandCh <- hubCmd{op: "wsSetCalibration", cal: CalConfig{
Source: "wave", Signal: "Adc", Scale: 4, Offset: 1, Unit: "V"}}
if raw := waitMsg(t, sendCh, "calibration"); raw == nil {
t.Fatal("no calibration broadcast after a valid setCalibration")
}
if got := h.cal.List(); len(got) != 1 || got[0].Scale != 4 {
t.Fatalf("table = %+v, want one entry with scale 4", got)
}
// An invalid entry is rejected and emits no broadcast at all.
h.commandCh <- hubCmd{op: "wsSetCalibration", cal: CalConfig{
Source: "wave", Signal: "Adc", Scale: 0}}
if raw := waitMsg(t, sendCh, "calibration"); raw != nil {
t.Errorf("invalid setCalibration broadcast %s", raw)
}
if got := h.cal.List(); len(got) != 1 || got[0].Scale != 4 {
t.Errorf("table changed after a rejected setCalibration: %+v", got)
}
h.unregister <- c
}
// drainSendCh reads all currently buffered messages from the channel.
func drainSendCh(ch chan wsMessage) {
for {
select {
case <-ch:
default:
return
}
}
}
// waitMsg waits up to ~250 ms for a message of the given type on sendCh.
func waitMsg(t *testing.T, sendCh chan wsMessage, msgType string) []byte {
t.Helper()
deadline := time.After(250 * time.Millisecond)
for {
select {
case msg := <-sendCh:
var env struct {
Type string `json:"type"`
}
if json.Unmarshal(msg.data, &env) == nil && env.Type == msgType {
return msg.data
}
case <-deadline:
return nil
}
}
}
func sleepMillis(n int) { time.Sleep(time.Duration(n) * time.Millisecond) }
Common/Client/go/wshub/keepalive_test.go
+69
View File
@@ -0,0 +1,69 @@
package wshub
import (
"net"
"testing"
"time"
"marte2/common/udpsprotocol"
)
// TestUDPClientSendsPeriodicKeepAliveAcks verifies that a unicast UDPClient
// re-sends ACK datagrams from the SAME socket at keepAliveInterval. The
// UDPSServer refreshes a unicast client's last-seen only on client->server
// traffic; without this keepalive it evicts the client after ClientTimeout
// (default 30 s) and the stream dies.
func TestUDPClientSendsPeriodicKeepAliveAcks(t *testing.T) {
srv, err := net.ListenUDP("udp4", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")})
if err != nil {
t.Fatal(err)
}
defer srv.Close()
c := NewUDPClient(srv.LocalAddr().String(), "ka1", NewHub(), "", 0)
c.keepAliveInterval = 150 * time.Millisecond
go c.Run()
defer c.Stop()
buf := make([]byte, 512)
// 1) CONNECT from the client's ephemeral socket.
srv.SetReadDeadline(time.Now().Add(3 * time.Second))
n, clientAddr, err := srv.ReadFromUDP(buf)
if err != nil {
t.Fatalf("expected CONNECT: %v", err)
}
hdr, err := udpsprotocol.ParseHeader(buf[:n])
if err != nil {
t.Fatalf("parse CONNECT: %v", err)
}
if hdr.Type != udpsprotocol.PktConnect {
t.Fatalf("first packet type = %d, want CONNECT (%d)", hdr.Type, udpsprotocol.PktConnect)
}
// 2) Collect ACKs for ~1 s: must be periodic and from the SAME socket
// (a new ephemeral socket would be registered as a new client).
deadline := time.Now().Add(time.Second)
acks := 0
for time.Now().Before(deadline) {
srv.SetReadDeadline(deadline)
n, addr, err := srv.ReadFromUDP(buf)
if err != nil {
break
}
hdr, err := udpsprotocol.ParseHeader(buf[:n])
if err != nil {
continue
}
if hdr.Type != udpsprotocol.PktACK {
t.Fatalf("unexpected packet type %d from %s", hdr.Type, addr)
}
if addr.String() != clientAddr.String() {
t.Fatalf("ACK from %s, want same socket as CONNECT (%s)", addr, clientAddr)
}
acks++
}
if acks < 3 {
t.Fatalf("expected >= 3 keepalive ACKs in 1 s, got %d", acks)
}
}
Common/Client/go/wshub/multicast_iface_test.go
+116
View File
@@ -0,0 +1,116 @@
//go:build linux
package wshub
import (
"net"
"syscall"
"testing"
"time"
)
// setMulticastIf pins a socket's outgoing multicast interface (IP_MULTICAST_IF),
// which is exactly what UDPStreamer/UDPSServer does with its `Interface` key.
func setMulticastIf(t *testing.T, conn *net.UDPConn, ip [4]byte) {
t.Helper()
rc, err := conn.SyscallConn()
if err != nil {
t.Fatalf("SyscallConn: %v", err)
}
var sockErr error
if err := rc.Control(func(fd uintptr) {
sockErr = syscall.SetsockoptInet4Addr(int(fd), syscall.IPPROTO_IP, syscall.IP_MULTICAST_IF, ip)
}); err != nil {
t.Fatalf("Control: %v", err)
}
if sockErr != nil {
t.Fatalf("IP_MULTICAST_IF: %v", sockErr)
}
}
func TestInterfaceForIPResolvesLoopback(t *testing.T) {
ifi := interfaceForIP(net.ParseIP("127.0.0.1"))
if ifi == nil {
t.Fatal("no interface resolved for 127.0.0.1")
}
if ifi.Flags&net.FlagLoopback == 0 {
t.Fatalf("resolved %q for 127.0.0.1, which is not a loopback interface", ifi.Name)
}
}
func TestInterfaceForIPUnknownAddressIsNil(t *testing.T) {
// Unspecified and unassigned addresses must fall back to "let the kernel
// choose" rather than resolving to an arbitrary interface.
if ifi := interfaceForIP(net.IPv4zero); ifi != nil {
t.Fatalf("0.0.0.0 resolved to %q, want nil", ifi.Name)
}
if ifi := interfaceForIP(nil); ifi != nil {
t.Fatalf("nil IP resolved to %q, want nil", ifi.Name)
}
if ifi := interfaceForIP(net.ParseIP("203.0.113.42")); ifi != nil {
t.Fatalf("unassigned address resolved to %q, want nil", ifi.Name)
}
}
// TestMulticastJoinOnControlInterfaceReceivesData is the regression test for the
// bug that left the web UI permanently blank: the hub joined the group with a
// nil interface, so imr_interface stayed INADDR_ANY and the kernel picked the
// default-route interface. A UDPStreamer configured with Interface = "127.0.0.1"
// sends out the loopback instead, and every datagram was silently dropped.
//
// The sender here mimics that server exactly (IP_MULTICAST_IF = 127.0.0.1); the
// receiver joins the way runMulticastSession now does, via the interface that
// owns the control connection's local address.
func TestMulticastJoinOnControlInterfaceReceivesData(t *testing.T) {
const group = "239.255.13.37"
ifi := interfaceForIP(net.ParseIP("127.0.0.1"))
if ifi == nil {
t.Skip("no loopback interface available")
}
rx, err := net.ListenMulticastUDP("udp4", ifi, &net.UDPAddr{IP: net.ParseIP(group), Port: 0})
if err != nil {
t.Fatalf("join %s on %s: %v", group, ifi.Name, err)
}
defer rx.Close()
port := rx.LocalAddr().(*net.UDPAddr).Port
tx, err := net.ListenUDP("udp4", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")})
if err != nil {
t.Fatalf("sender socket: %v", err)
}
defer tx.Close()
setMulticastIf(t, tx, [4]byte{127, 0, 0, 1})
payload := []byte("UDPS-multicast-probe")
dst := &net.UDPAddr{IP: net.ParseIP(group), Port: port}
// Datagrams are lossy even on loopback if the join has not settled, so send
// a few and accept the first that lands.
done := make(chan struct{})
defer close(done)
go func() {
for {
select {
case <-done:
return
default:
}
tx.WriteToUDP(payload, dst)
time.Sleep(20 * time.Millisecond)
}
}()
buf := make([]byte, 128)
if err := rx.SetReadDeadline(time.Now().Add(3 * time.Second)); err != nil {
t.Fatal(err)
}
n, _, err := rx.ReadFromUDP(buf)
if err != nil {
t.Fatalf("no multicast received on %s within 3s: %v", ifi.Name, err)
}
if got := string(buf[:n]); got != string(payload) {
t.Fatalf("payload = %q, want %q", got, payload)
}
}
Common/Client/go/wshub/origin_test.go
+72
View File
@@ -0,0 +1,72 @@
package wshub
import (
"net/http"
"testing"
)
// TestCheckOrigin_NoOriginHeader — non-browser clients (no Origin) are allowed.
func TestCheckOrigin_NoOriginHeader(t *testing.T) {
r := &http.Request{Header: http.Header{}}
if !checkOrigin(r) {
t.Fatal("non-browser client (no Origin header) should be allowed")
}
}
// TestCheckOrigin_SameOrigin — Origin host matching Host header is allowed.
func TestCheckOrigin_SameOrigin(t *testing.T) {
r := &http.Request{
Header: http.Header{
"Origin": []string{"http://localhost:8090"},
},
Host: "localhost:8090",
}
if !checkOrigin(r) {
t.Fatal("same-origin request should be allowed")
}
}
// TestCheckOrigin_CrossOriginBlocked — different Origin host is rejected.
func TestCheckOrigin_CrossOriginBlocked(t *testing.T) {
r := &http.Request{
Header: http.Header{
"Origin": []string{"http://evil.example.com:8090"},
},
Host: "localhost:8090",
}
if checkOrigin(r) {
t.Fatal("cross-origin request should be blocked")
}
}
// TestCheckOrigin_Allowlist — explicitly allowed origins pass even if cross-origin.
func TestCheckOrigin_Allowlist(t *testing.T) {
SetAllowedOrigins([]string{"http://evil.example.com:8090"})
defer SetAllowedOrigins(nil) // reset
r := &http.Request{
Header: http.Header{
"Origin": []string{"http://evil.example.com:8090"},
},
Host: "localhost:8090",
}
if !checkOrigin(r) {
t.Fatal("allowlisted origin should be allowed")
}
}
// TestCheckOrigin_AllowlistDoesNotMatch — non-allowlisted cross-origin is blocked.
func TestCheckOrigin_AllowlistDoesNotMatch(t *testing.T) {
SetAllowedOrigins([]string{"http://good.example.com"})
defer SetAllowedOrigins(nil)
r := &http.Request{
Header: http.Header{
"Origin": []string{"http://evil.example.com"},
},
Host: "localhost:8090",
}
if checkOrigin(r) {
t.Fatal("non-allowlisted cross-origin should be blocked")
}
}
Common/Client/go/wshub/ringbuf.go
+317 -9
View File
@@ -1,6 +1,10 @@
package wshub
import "sync"
import (
"log"
"math"
"sync"
)
// sigRing is a fixed-capacity circular buffer storing (time, value) pairs.
// Writes come from the Hub.Run() goroutine; reads come from HTTP handler goroutines.
@@ -10,30 +14,334 @@ type sigRing struct {
t, v []float64
cap int
head, size int // next write position; current fill
// bucket is how many source samples collapse into one min/max pair on the
// way in. 1 stores the stream verbatim. Raising it trades resolution for
// the timespan a fixed capacity covers, which is what lets a long display
// window fit in a fixed per-signal memory budget.
bucket int
// In-progress bucket. accN counts source samples seen since the last pair
// was emitted; the four acc fields are the extrema and when they occurred.
accN int
accTMin, accVMin float64
accTMax, accVMax float64
// Source-sample accounting, kept because size and the stored timespan no
// longer give the source rate once bucket > 1. Reset every
// srcRateWindowSec so a producer restart or a rate change is not averaged
// against the whole run.
srcCount int64
srcT0, srcT1 float64
haveSrc bool
}
// srcRateWindowSec bounds how long a source-rate measurement accumulates before
// starting over. Long enough to average out per-frame jitter, short enough that
// a rate change is reflected within a few seconds.
const srcRateWindowSec = 10.0
func newSigRing(capacity int) *sigRing {
return &sigRing{
t: make([]float64, capacity),
v: make([]float64, capacity),
cap: capacity,
t: make([]float64, capacity),
v: make([]float64, capacity),
cap: capacity,
bucket: 1,
}
}
// write appends (tArr[i], vArr[i]) pairs, overwriting oldest entries when full.
// With bucket > 1 each group of bucket samples contributes only its minimum and
// its maximum, in the order the two occurred.
func (rb *sigRing) write(tArr, vArr []float64) {
rb.mu.Lock()
defer rb.mu.Unlock()
if n := len(tArr); n > 0 {
if !rb.haveSrc || tArr[n-1]-rb.srcT0 > srcRateWindowSec || tArr[0] < rb.srcT0 {
rb.srcT0, rb.srcCount, rb.haveSrc = tArr[0], 0, true
}
rb.srcT1 = tArr[n-1]
rb.srcCount += int64(n)
}
if rb.bucket <= 1 {
for i := 0; i < len(tArr); i++ {
rb.pushLocked(tArr[i], vArr[i])
}
return
}
for i := 0; i < len(tArr); i++ {
rb.t[rb.head] = tArr[i]
rb.v[rb.head] = vArr[i]
rb.head = (rb.head + 1) % rb.cap
if rb.size < rb.cap {
rb.size++
t, v := tArr[i], vArr[i]
if rb.accN == 0 {
rb.accTMin, rb.accVMin, rb.accTMax, rb.accVMax = t, v, t, v
} else {
if v < rb.accVMin {
rb.accTMin, rb.accVMin = t, v
}
if v > rb.accVMax {
rb.accTMax, rb.accVMax = t, v
}
}
rb.accN++
if rb.accN >= rb.bucket {
rb.flushBucketLocked()
}
}
}
func (rb *sigRing) pushLocked(t, v float64) {
rb.t[rb.head] = t
rb.v[rb.head] = v
rb.head = (rb.head + 1) % rb.cap
if rb.size < rb.cap {
rb.size++
}
}
// flushBucketLocked emits the accumulated extrema oldest-first. Time order
// matters: every read binary-searches rb.t, so the stored timestamps must stay
// non-decreasing.
func (rb *sigRing) flushBucketLocked() {
if rb.accN == 0 {
return
}
if rb.accTMin <= rb.accTMax {
rb.pushLocked(rb.accTMin, rb.accVMin)
rb.pushLocked(rb.accTMax, rb.accVMax)
} else {
rb.pushLocked(rb.accTMax, rb.accVMax)
rb.pushLocked(rb.accTMin, rb.accVMin)
}
rb.accN = 0
}
// setBucket changes the min/max reduction applied to incoming samples and
// reports whether it changed. Samples already stored keep the resolution they
// were written at; the ring converges on the new one as it rolls.
func (rb *sigRing) setBucket(n int) bool {
if n < 1 {
n = 1
}
rb.mu.Lock()
defer rb.mu.Unlock()
if n == rb.bucket {
return false
}
// Emit what the old bucket had collected rather than dropping it.
rb.flushBucketLocked()
rb.bucket = n
return true
}
func (rb *sigRing) bucketSize() int {
rb.mu.RLock()
defer rb.mu.RUnlock()
return rb.bucket
}
// sourceRate is the measured rate of the incoming stream in samples per second,
// or 0 while there is too little to extrapolate from. Unlike stats() it counts
// source samples, so it is unaffected by bucketing.
func (rb *sigRing) sourceRate() float64 {
rb.mu.RLock()
defer rb.mu.RUnlock()
if rb.srcCount < 2 || rb.srcT1 <= rb.srcT0 {
return 0
}
return float64(rb.srcCount-1) / (rb.srcT1 - rb.srcT0)
}
// stats reports the current fill and the timespan it covers, so callers can
// estimate the stream's sample rate without copying the data out.
func (rb *sigRing) stats() (count int, span float64) {
rb.mu.RLock()
defer rb.mu.RUnlock()
if rb.size < 2 {
return rb.size, 0
}
start := 0
if rb.size == rb.cap {
start = rb.head
}
oldest := rb.t[start]
newest := rb.t[(start+rb.size-1)%rb.cap]
return rb.size, newest - oldest
}
func (rb *sigRing) capacity() int {
rb.mu.RLock()
defer rb.mu.RUnlock()
return rb.cap
}
// ─── Ring tuning ─────────────────────────────────────────────────────────────
// ringHeadroom oversizes a reduced ring's span. It absorbs rate jitter and
// keeps the tail of a trigger window in the buffer long enough for the capture
// to read it. It applies only once the window no longer fits verbatim: at the
// boundary, spending a whole extra bucket step to buy 25 % more span would cost
// half the resolution.
const ringHeadroom = 1.25
// ringTuneIntervalSec throttles the retune sweep. The source rate only settles
// once data flows, so the sweep repeats rather than running once.
const ringTuneIntervalSec = 1.0
// defaultLiveWindowSec is the window assumed when no client has said what it is
// displaying — the native clients never do, and a browser has not yet at the
// moment the first samples land.
const defaultLiveWindowSec = 10.0
// ringBucketFor is how many source samples must collapse into one min/max pair
// for `window` seconds at `rate` samples/s to fit in `capacity` points.
//
// rate*window <= capacity → 1, the buffer stays verbatim and reaches further
// back than the window, which is free zoom headroom
// rate*window > capacity → >1, so the whole window fits at reduced resolution
//
// A bucket costs two points (its minimum and its maximum), hence the factor 2.
func ringBucketFor(rate, window float64, capacity int) int {
if capacity <= 0 || rate <= 0 || window <= 0 {
return 1
}
need := rate * window
if need <= float64(capacity) {
return 1
}
return int(math.Ceil(2 * need * ringHeadroom / float64(capacity)))
}
// ringCoverage is how many source samples a ring of `capacity` points holds at
// the given bucket. A bucket of 2 stores both of its samples, so it covers no
// more ground than a bucket of 1.
func ringCoverage(bucket, capacity int) int {
if bucket <= 2 {
return capacity
}
return capacity / 2 * bucket
}
// activeWindowSec is the timespan the buffers must cover. An armed trigger owns
// it: its pre-window has to already be in the ring when the trigger fires or
// there is nothing to back-fill the capture from. Otherwise it is the widest
// window any connected client is displaying.
func (h *Hub) activeWindowSec() float64 {
if h.trigger != nil && h.trigger.Active() {
if cfg := h.trigger.Config(); cfg.windowSec > 0 {
return cfg.windowSec
}
}
widest := 0.0
for c := range h.clients {
if w := c.displayWindowSec(); w > widest {
widest = w
}
}
if widest <= 0 {
return defaultLiveWindowSec
}
return widest
}
// retuneRings keeps every ring matched to the window being displayed: grown
// towards the per-signal budget, and bucketed so the window fits inside it.
//
// A fixed sample-count ring covers a fraction of a second at a megasample rate,
// which is why long windows used to come back with only their tail populated —
// in live mode as much as under a trigger. Spending the budget on min/max pairs
// rather than on a bigger allocation is what makes an arbitrarily long window
// work within a fixed memory bound.
//
// Called from Hub.Run() only, so reading h.clients here needs no lock.
func (h *Hub) retuneRings(nowSec float64) {
if nowSec < h.ringTuneAt {
return
}
h.ringTuneAt = nowSec + ringTuneIntervalSec
window := h.activeWindowSec()
if window <= 0 {
return
}
// The archive answers for the same window as the rings — it is what a zoom
// or a capture falls back on once they have rolled past it — so it is sized
// from the same number.
if h.hist.setWindow(window) {
if msg := h.buildHistoryInfoMsg(); msg != nil {
h.broadcast(msg)
}
}
budget := h.ringBudget()
h.ringsMu.RLock()
keys := make([]string, 0, len(h.rings))
rings := make([]*sigRing, 0, len(h.rings))
for k, rb := range h.rings {
keys = append(keys, k)
rings = append(rings, rb)
}
h.ringsMu.RUnlock()
for i, rb := range rings {
rate := rb.sourceRate()
if rate <= 0 {
continue
}
// Claim the whole budget before deciding on a bucket: memory is what
// buys resolution, so it is spent first and reduced from only if the
// window still does not fit.
if rb.capacity() < budget && rate*window > float64(rb.capacity()) {
rb.grow(budget)
}
cur := rb.bucketSize()
need := rate * window
covered := float64(ringCoverage(cur, rb.capacity()))
// Hysteresis. Retuning up and retuning down must not share a threshold:
// a bucket step doubles or halves the span, so a rate jittering across
// the boundary would otherwise flip the resolution every second. Hold
// the current bucket while it covers the window without covering more
// than twice it.
if covered >= need && covered <= 2*need {
continue
}
want := ringBucketFor(rate, window, rb.capacity())
if !rb.setBucket(want) {
continue
}
if want > 1 {
log.Printf("hub: ring %s stores min/max over %d samples: %.0f s at %.0f kSps does not fit in %d points",
keys[i], want, window, rate/1e3, rb.capacity())
} else {
log.Printf("hub: ring %s back to full resolution: %.0f s at %.0f kSps fits in %d points",
keys[i], window, rate/1e3, rb.capacity())
}
}
}
// grow enlarges the buffer to newCap, keeping every sample it currently holds.
// Shrinking is refused: it would discard history a pending capture may need.
func (rb *sigRing) grow(newCap int) bool {
rb.mu.Lock()
defer rb.mu.Unlock()
if newCap <= rb.cap {
return false
}
nt := make([]float64, newCap)
nv := make([]float64, newCap)
start := 0
if rb.size == rb.cap {
start = rb.head
}
for i := 0; i < rb.size; i++ {
p := (start + i) % rb.cap
nt[i], nv[i] = rb.t[p], rb.v[p]
}
rb.t, rb.v = nt, nv
rb.cap = newCap
rb.head = rb.size // size < newCap, so no wrap
return true
}
// slice returns copies of all (t, v) pairs whose timestamp falls in [t0, t1].
// The returned slices are safe to use after the call without holding any lock.
func (rb *sigRing) slice(t0, t1 float64) ([]float64, []float64) {
Common/Client/go/wshub/ringbuf_test.go
+147
View File
@@ -0,0 +1,147 @@
package wshub
import (
"math"
"testing"
)
// dump returns the ring's contents oldest-first, which is what every reader
// sees through slice() but is easier to assert on directly.
func dump(rb *sigRing) ([]float64, []float64) {
return rb.slice(math.Inf(-1), math.Inf(1))
}
func TestRingBucketStoresMinMaxPairsInTimeOrder(t *testing.T) {
rb := newSigRing(100)
rb.setBucket(4)
// Two buckets. In the first the minimum comes before the maximum, in the
// second the order is reversed, so the emitted pairs must not be sorted by
// value — a ring whose timestamps are not monotonic breaks slice()'s
// binary search.
ts := []float64{0, 1, 2, 3, 4, 5, 6, 7}
vs := []float64{-5, 0, 0, 9, 9, 0, 0, -5}
rb.write(ts, vs)
gotT, gotV := dump(rb)
wantT := []float64{0, 3, 4, 7}
wantV := []float64{-5, 9, 9, -5}
if len(gotT) != len(wantT) {
t.Fatalf("stored %d points, want %d", len(gotT), len(wantT))
}
for i := range wantT {
if gotT[i] != wantT[i] || gotV[i] != wantV[i] {
t.Fatalf("point %d = (%v,%v), want (%v,%v)", i, gotT[i], gotV[i], wantT[i], wantV[i])
}
}
}
func TestRingBucketExtendsTheSpanAFixedCapacityCovers(t *testing.T) {
const cap = 200
// 2000 samples at 1 kHz is 2 s, ten times what the capacity holds verbatim.
ts := make([]float64, 2000)
vs := make([]float64, 2000)
for i := range ts {
ts[i] = float64(i) * 1e-3
vs[i] = math.Sin(float64(i))
}
full := newSigRing(cap)
full.write(ts, vs)
if _, span := full.stats(); span > 0.25 {
t.Fatalf("full-rate ring spans %.3f s, expected ~0.2 s", span)
}
// bucket 20 turns 20 samples into 2 points, so the same capacity reaches
// 10x further: 200/2*20 = 2000 samples = 2 s.
bucketed := newSigRing(cap)
bucketed.setBucket(20)
bucketed.write(ts, vs)
count, span := bucketed.stats()
if count != cap {
t.Fatalf("bucketed ring holds %d points, want the full %d", count, cap)
}
if span < 1.9 {
t.Fatalf("bucketed ring spans %.3f s, want the whole ~2 s", span)
}
}
func TestRingSourceRateIsUnaffectedByBucketing(t *testing.T) {
rb := newSigRing(1000)
rb.setBucket(50)
ts := make([]float64, 5000)
vs := make([]float64, 5000)
for i := range ts {
ts[i] = float64(i) * 1e-4 // 10 kHz
}
rb.write(ts, vs)
got := rb.sourceRate()
if math.Abs(got-10000) > 10 {
t.Fatalf("sourceRate = %.1f, want ~10000", got)
}
}
func TestSetBucketFlushesThePartialBucket(t *testing.T) {
rb := newSigRing(100)
rb.setBucket(10)
// Three samples: not enough to close a bucket of 10, so nothing is stored
// yet and they would be silently dropped by a re-bucket that just reset the
// accumulator.
rb.write([]float64{0, 1, 2}, []float64{7, -7, 0})
if n, _ := rb.stats(); n != 0 {
t.Fatalf("partial bucket already emitted %d points", n)
}
rb.setBucket(2)
gotT, gotV := dump(rb)
if len(gotT) != 2 || gotT[0] != 0 || gotV[0] != 7 || gotT[1] != 1 || gotV[1] != -7 {
t.Fatalf("flushed pair = %v/%v, want t=[0 1] v=[7 -7]", gotT, gotV)
}
}
func TestActiveWindowSecFallsBackToTheDefault(t *testing.T) {
h := NewHub()
if got := h.activeWindowSec(); got != defaultLiveWindowSec {
t.Fatalf("activeWindowSec with no clients = %v, want %v", got, defaultLiveWindowSec)
}
}
// Clients disagree about how far back they are plotting, and a buffer sized for
// the narrowest one leaves the others with nothing to zoom into.
func TestActiveWindowSecTakesTheWidestClientWindow(t *testing.T) {
h := NewHub()
narrow, wide, silent := &wsClient{}, &wsClient{}, &wsClient{}
narrow.setDisplayWindowSec(1)
wide.setDisplayWindowSec(120)
h.clients[narrow], h.clients[wide], h.clients[silent] = true, true, true
if got := h.activeWindowSec(); got != 120 {
t.Fatalf("activeWindowSec = %v, want the widest 120", got)
}
}
// An armed trigger owns the window: its pre-window has to be in the buffer
// before the trigger fires or the capture has nothing to back-fill from.
func TestActiveWindowSecPrefersTheArmedTrigger(t *testing.T) {
h := NewHub()
c := &wsClient{}
c.setDisplayWindowSec(1)
h.clients[c] = true
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 45, mode: "normal"})
if got := h.activeWindowSec(); got != 45 {
t.Fatalf("activeWindowSec = %v, want the trigger's 45", got)
}
}
func TestSetBucketToOneRestoresVerbatimStorage(t *testing.T) {
rb := newSigRing(100)
rb.setBucket(4)
rb.setBucket(1)
ts := []float64{0, 1, 2, 3}
vs := []float64{1, 2, 3, 4}
rb.write(ts, vs)
gotT, _ := dump(rb)
if len(gotT) != 4 {
t.Fatalf("stored %d points, want all 4", len(gotT))
}
}
Common/Client/go/wshub/sources.go
+196 -27
View File
@@ -1,12 +1,12 @@
package wshub
import (
"encoding/json"
"fmt"
"io"
"log"
"net"
"os"
"sort"
"strconv"
"strings"
"sync"
@@ -95,11 +95,16 @@ func (sm *SourceManager) Remove(id string) {
}
}
// Save writes the current source list to filePath.
func (sm *SourceManager) Save() error {
if sm.filePath == "" {
return fmt.Errorf("no sources-file configured")
}
// Path returns the configured config-file path ("" when none).
func (sm *SourceManager) Path() string {
sm.mu.RLock()
defer sm.mu.RUnlock()
return sm.filePath
}
// snapshotSources returns the current sources sorted by label, so the written
// file is byte-stable across runs (the map iteration order is not).
func (sm *SourceManager) snapshotSources() []SourceConfig {
sm.mu.RLock()
cfgs := make([]SourceConfig, 0, len(sm.sources))
for _, ms := range sm.sources {
@@ -111,26 +116,86 @@ func (sm *SourceManager) Save() error {
})
}
sm.mu.RUnlock()
sort.Slice(cfgs, func(i, j int) bool {
if cfgs[i].Label != cfgs[j].Label {
return cfgs[i].Label < cfgs[j].Label
}
return cfgs[i].Addr < cfgs[j].Addr
})
return cfgs
}
data, err := json.MarshalIndent(cfgs, "", " ")
// Save writes the current source list and calibration table to filePath as one
// flat JSON array.
func (sm *SourceManager) Save() error {
path := sm.Path()
if path == "" {
return fmt.Errorf("no sources-file configured")
}
data, err := encodeConfigFile(sm.snapshotSources(), sm.hub.cal.List())
if err != nil {
return err
}
return os.WriteFile(sm.filePath, data, 0644)
return os.WriteFile(path, data, 0644)
}
// Load reads sources from path and adds them.
// Load reads the config file at path, replaces the calibration table with its
// contents and starts every source it lists.
func (sm *SourceManager) Load(path string) error {
data, err := os.ReadFile(path)
if err != nil {
return err
}
var cfgs []SourceConfig
if err := json.Unmarshal(data, &cfgs); err != nil {
srcs, cals, err := parseConfigFile(data)
if err != nil {
return err
}
sm.mu.Lock()
sm.filePath = path
for _, cfg := range cfgs {
sm.mu.Unlock()
sm.hub.cal.Replace(cals)
for _, cfg := range srcs {
sm.Add(cfg.Label, cfg.Addr, cfg.MulticastGroup, cfg.DataPort)
}
return nil
}
// Reload re-reads the config file. The calibration table is replaced wholesale
// and sources listed in the file that are not already running are started; no
// live source is ever stopped, restarted or reconnected, because a reload must
// not interrupt streaming. The asymmetry is deliberate: calibration is cheap
// to reapply, a source is a live UDP session.
func (sm *SourceManager) Reload() error {
path := sm.Path()
if path == "" {
return fmt.Errorf("no sources-file configured")
}
data, err := os.ReadFile(path)
if err != nil {
return err
}
srcs, cals, err := parseConfigFile(data)
if err != nil {
return err
}
sm.hub.cal.Replace(cals)
sm.mu.RLock()
live := make(map[string]bool, len(sm.sources))
for _, ms := range sm.sources {
live[ms.label+"\x00"+ms.addr] = true
}
sm.mu.RUnlock()
for _, cfg := range srcs {
label := cfg.Label
if label == "" {
label = cfg.Addr // Add() applies the same default
}
if live[label+"\x00"+cfg.Addr] {
continue
}
sm.Add(cfg.Label, cfg.Addr, cfg.MulticastGroup, cfg.DataPort)
}
return nil
@@ -172,27 +237,34 @@ const (
reconnectDelay = 2 * time.Second
readBufSize = 65536
udpRcvBufSize = 8 * 1024 * 1024
// keepAliveInterval is the unicast keepalive period. The UDPStreamer
// server evicts silent unicast clients after its ClientTimeout (default
// 30 s); an ACK from the same socket refreshes its last-seen without
// triggering a CONFIG resend (a CONNECT would).
keepAliveInterval = 15 * time.Second
)
// UDPClient manages the connection to one MARTe2 streamer source.
type UDPClient struct {
serverAddr string
sourceID string
hub *Hub
multicastGroup string
dataPort int
stopCh chan struct{}
serverAddr string
sourceID string
hub *Hub
multicastGroup string
dataPort int
keepAliveInterval time.Duration
stopCh chan struct{}
}
// NewUDPClient creates a UDPClient bound to a specific source ID.
func NewUDPClient(serverAddr, sourceID string, hub *Hub, multicastGroup string, dataPort int) *UDPClient {
return &UDPClient{
serverAddr: serverAddr,
sourceID: sourceID,
hub: hub,
multicastGroup: multicastGroup,
dataPort: dataPort,
stopCh: make(chan struct{}),
serverAddr: serverAddr,
sourceID: sourceID,
hub: hub,
multicastGroup: multicastGroup,
dataPort: dataPort,
keepAliveInterval: keepAliveInterval,
stopCh: make(chan struct{}),
}
}
@@ -253,6 +325,20 @@ func (u *UDPClient) runSession() error {
return err
}
log.Printf("[%s] udp: sent CONNECT", u.sourceID)
lastData := time.Now()
lastKeepAlive := time.Now()
// sendKeepAliveIfDue sends an ACK if the keepalive interval has elapsed.
// ACK refreshes the server's last-seen without re-sending CONFIG (which a
// repeated CONNECT would trigger).
sendKeepAliveIfDue := func() error {
if u.keepAliveInterval > 0 && time.Since(lastKeepAlive) >= u.keepAliveInterval {
if _, err := conn.WriteToUDP(udpsprotocol.BuildAckPacket(), serverAddr); err != nil {
return err
}
lastKeepAlive = time.Now()
}
return nil
}
reassembler := udpsprotocol.NewReassembler(2 * time.Second)
buf := make([]byte, readBufSize)
@@ -260,14 +346,34 @@ func (u *UDPClient) runSession() error {
var currentPublishMode uint8
for {
conn.SetReadDeadline(time.Now().Add(silenceTimeout))
// Wake up at least every keepalive interval so ACKs are sent even
// when the server is idle; the read deadline also doubles as the
// silence detector (no data for silenceTimeout = server gone).
wakeup := silenceTimeout
if u.keepAliveInterval > 0 && u.keepAliveInterval < wakeup {
wakeup = u.keepAliveInterval
}
conn.SetReadDeadline(time.Now().Add(wakeup))
n, _, err := conn.ReadFromUDP(buf)
arrivalTime := time.Now()
if err != nil {
if ne, ok := err.(net.Error); ok && ne.Timeout() {
if time.Since(lastData) >= silenceTimeout {
// True silence: stream is dead — Run() reconnects.
conn.WriteToUDP(udpsprotocol.BuildDisconnectPacket(), serverAddr)
return err
}
// Short wakeup: keepalive if due, then keep waiting.
if kaErr := sendKeepAliveIfDue(); kaErr != nil {
return kaErr
}
continue
}
conn.WriteToUDP(udpsprotocol.BuildDisconnectPacket(), serverAddr)
return err
}
lastData = arrivalTime
if n < udpsprotocol.HeaderSize {
log.Printf("[%s] udp: short datagram (%d bytes), skipping", u.sourceID, n)
@@ -334,9 +440,59 @@ func (u *UDPClient) runSession() error {
return nil
default:
}
if kaErr := sendKeepAliveIfDue(); kaErr != nil {
return kaErr
}
}
}
// interfaceForIP returns the interface that owns the given local address, or
// nil if no interface matches (in which case callers fall back to letting the
// kernel choose).
func interfaceForIP(ip net.IP) *net.Interface {
if ip == nil || ip.IsUnspecified() {
return nil
}
ifaces, err := net.Interfaces()
if err != nil {
return nil
}
for i := range ifaces {
addrs, err := ifaces[i].Addrs()
if err != nil {
continue
}
for _, a := range addrs {
var aIP net.IP
switch v := a.(type) {
case *net.IPNet:
aIP = v.IP
case *net.IPAddr:
aIP = v.IP
}
if aIP != nil && aIP.Equal(ip) {
return &ifaces[i]
}
}
}
return nil
}
// interfaceForConn returns the interface a connection's local endpoint sits on.
func interfaceForConn(c net.Conn) *net.Interface {
if c == nil {
return nil
}
switch a := c.LocalAddr().(type) {
case *net.TCPAddr:
return interfaceForIP(a.IP)
case *net.UDPAddr:
return interfaceForIP(a.IP)
}
return nil
}
// runMulticastSession handles the multicast mode session.
func (u *UDPClient) runMulticastSession() error {
tcpAddr, err := net.ResolveTCPAddr("tcp4", u.serverAddr)
@@ -390,7 +546,15 @@ func (u *UDPClient) runMulticastSession() error {
return &net.AddrError{Err: "invalid multicast group IP", Addr: u.multicastGroup}
}
mcastAddr := &net.UDPAddr{IP: mcastIP, Port: mcastPort}
mcastConn, err := net.ListenMulticastUDP("udp4", nil, mcastAddr)
// Join on the interface that reaches the control connection. The UDPStreamer
// pins its multicast sends to its configured Interface (IP_MULTICAST_IF), so
// a join with a nil interface — which leaves imr_interface at INADDR_ANY and
// lets the kernel pick the default-route interface — silently receives
// nothing whenever that is not the sending interface. The local address of
// the control connection is the interface the server is reachable on, which
// is the sending interface in every single-homed and same-host deployment.
ifi := interfaceForConn(tcpConn)
mcastConn, err := net.ListenMulticastUDP("udp4", ifi, mcastAddr)
if err != nil {
return err
}
@@ -398,7 +562,12 @@ func (u *UDPClient) runMulticastSession() error {
if err := mcastConn.SetReadBuffer(udpRcvBufSize); err != nil {
log.Printf("[%s] multicast SetReadBuffer: %v", u.sourceID, err)
}
log.Printf("[%s] joined multicast %s:%s", u.sourceID, u.multicastGroup, strconv.Itoa(mcastPort))
ifName := "default"
if ifi != nil {
ifName = ifi.Name
}
log.Printf("[%s] joined multicast %s:%s on interface %s",
u.sourceID, u.multicastGroup, strconv.Itoa(mcastPort), ifName)
tcpDone := make(chan error, 1)
go func() {
Common/Client/go/wshub/sources_test.go
+131
View File
@@ -0,0 +1,131 @@
package wshub
import (
"os"
"path/filepath"
"testing"
)
// newTestManager builds a hub + manager pair with no goroutines running.
func newTestManager(t *testing.T) (*Hub, *SourceManager, string) {
t.Helper()
path := filepath.Join(t.TempDir(), "sources.json")
h := NewHub()
sm := NewSourceManager(h, path)
h.SetSourceManager(sm)
return h, sm, path
}
func TestSaveWritesSourcesAndCalibration(t *testing.T) {
h, sm, path := newTestManager(t)
// Register two sources without starting any UDP client.
sm.mu.Lock()
sm.sources["s1"] = &managedSource{id: "s1", label: "wave", addr: "127.0.0.1:44500"}
sm.sources["s2"] = &managedSource{
id: "s2", label: "mc", addr: "127.0.0.1:44501",
multicastGroup: "239.0.0.1", dataPort: 44502,
}
sm.mu.Unlock()
if !h.cal.Set(CalConfig{Source: "wave", Signal: "Adc", Scale: 0.5, Offset: -1.25, Unit: "V"}) {
t.Fatal("calibration rejected")
}
if err := sm.Save(); err != nil {
t.Fatalf("Save: %v", err)
}
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
srcs, cals, err := parseConfigFile(data)
if err != nil {
t.Fatalf("parseConfigFile: %v\n%s", err, data)
}
if len(srcs) != 2 {
t.Fatalf("got %d sources, want 2\n%s", len(srcs), data)
}
// Save sorts by label so the file is byte-stable across runs.
if srcs[0].Label != "mc" || srcs[1].Label != "wave" {
t.Errorf("source order = %q,%q, want mc,wave", srcs[0].Label, srcs[1].Label)
}
if len(cals) != 1 || cals[0].Signal != "Adc" || cals[0].Scale != 0.5 {
t.Fatalf("calibration round-trip failed: %+v\n%s", cals, data)
}
}
func TestSaveWithoutFilePathFails(t *testing.T) {
h := NewHub()
sm := NewSourceManager(h, "")
h.SetSourceManager(sm)
if err := sm.Save(); err == nil {
t.Error("Save() with no path = nil error, want error")
}
}
func TestLoadSeedsCalibrationTable(t *testing.T) {
h, sm, path := newTestManager(t)
// No "addr" blocks: Load must not start any UDP client during the test.
if err := os.WriteFile(path, []byte(`[
{"source":"wave","signal":"Adc","scale":0.25,"offset":2,"unit":"mV"},
{"source":"wave","signal":"Dac","scale":2}
]`), 0o644); err != nil {
t.Fatal(err)
}
if err := sm.Load(path); err != nil {
t.Fatalf("Load: %v", err)
}
got := h.cal.List()
if len(got) != 2 {
t.Fatalf("List() = %d entries, want 2", len(got))
}
if got[0].Signal != "Adc" || got[0].Unit != "mV" || got[0].Offset != 2 {
t.Errorf("Adc = %+v", got[0])
}
if sm.Path() != path {
t.Errorf("Path() = %q, want %q", sm.Path(), path)
}
}
func TestReloadReplacesCalibrationAndKeepsLiveSources(t *testing.T) {
h, sm, path := newTestManager(t)
// A live source that the file does not mention must survive the reload.
sm.mu.Lock()
sm.sources["s1"] = &managedSource{id: "s1", label: "live", addr: "127.0.0.1:44999"}
sm.mu.Unlock()
// A stale calibration that the file does not mention must be dropped.
h.cal.Set(CalConfig{Source: "stale", Signal: "Old", Scale: 9})
if err := os.WriteFile(path, []byte(`[
{"source":"wave","signal":"Adc","scale":0.5}
]`), 0o644); err != nil {
t.Fatal(err)
}
if err := sm.Reload(); err != nil {
t.Fatalf("Reload: %v", err)
}
got := h.cal.List()
if len(got) != 1 || got[0].Source != "wave" {
t.Fatalf("after Reload, calibration = %+v, want only wave/Adc", got)
}
sm.mu.RLock()
_, alive := sm.sources["s1"]
n := len(sm.sources)
sm.mu.RUnlock()
if !alive || n != 1 {
t.Errorf("live source count = %d (s1 alive=%v), want 1 / true", n, alive)
}
}
func TestReloadWithoutFilePathFails(t *testing.T) {
h := NewHub()
sm := NewSourceManager(h, "")
h.SetSourceManager(sm)
if err := sm.Reload(); err == nil {
t.Error("Reload() with no path = nil error, want error")
}
}
Common/Client/go/wshub/stats.go
+3 -2
View File
@@ -32,8 +32,9 @@ type SourceStat struct {
}
// RecordFragment is called for every UDP datagram of a DATA packet.
// complete: this fragment completed the DATA reassembly.
// nBytes: raw datagram size (header+payload).
//
// complete: this fragment completed the DATA reassembly.
// nBytes: raw datagram size (header+payload).
func (s *SourceStat) RecordFragment(counter uint32, nBytes int, arrivalNs int64, complete bool) {
s.mu.Lock()
defer s.mu.Unlock()
Common/Client/go/wshub/trigger.go
+781
View File
@@ -0,0 +1,781 @@
package wshub
import (
"encoding/binary"
"encoding/json"
"log"
"math"
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/gorilla/websocket"
)
// Trigger FSM states, matching the C++ StreamHub TriggerEngine and the strings
// expected by the web SPA's "triggerState" handler.
const (
trigIdle = "idle"
trigArmed = "armed"
trigCollecting = "collecting"
trigTriggered = "triggered"
)
// captureMarginSec is the extra delay past the post-trigger window before the
// capture is extracted, so the rings have received the last samples.
const captureMarginSec = 0.15
// captureStallSec is how long the stream may be silent before a collecting
// trigger gives up waiting for the rest of its window and delivers what it has.
const captureStallSec = 2.0
// autoRearmDelaySec is the pause between a completed capture and the automatic
// rearm in "normal" mode.
const autoRearmDelaySec = 0.2
// trigCapturePts caps the points sent per signal in a capture frame. A window
// of 60 s at 1 MSps is 60 M raw samples — ~960 MB per signal on the wire, which
// no client can take and which the send path would simply drop. Matches the C++
// StreamHub's kTrigCapturePts.
const trigCapturePts = 20000
// shortCaptureTol is the fraction of the window a capture may miss at its front
// before it is reported. One min/max bucket of slack, not a quality target.
const shortCaptureTol = 0.01
// maxTriggerWindowSec bounds the capture window, matching the longest option
// the web UI offers. It is not a resolution limit: retuneRings buckets the
// rings so any window fits the per-signal memory budget, at the cost of storing
// min/max pairs rather than every sample.
const maxTriggerWindowSec = 600.0
// trigConfig is the client-settable part of the trigger.
type trigConfig struct {
signalKey string // "src:sig" or "src:sig[i]"
edge string // "rising" | "falling" | "both"
threshold float64
windowSec float64
prePercent float64
mode string // "normal" | "single"
holdoffSec float64 // rearm delay after a capture (double-trigger guard)
}
// triggerEngine implements the hub-side trigger FSM. Its methods are safe to
// call from the WebSocket read goroutines and from Hub.Run() concurrently.
type triggerEngine struct {
mu sync.Mutex
cfg trigConfig
// Parsed form of cfg.signalKey, refreshed by SetConfig.
baseKey string // "src:sig"
elemIdx int // -1 when the key has no "[i]" suffix
state string
stopped bool
// sentState is the state carried by the last stateMsg handed out. The
// armed→collecting transition happens inside feed(), on the ingest path,
// so the hub cannot see it by sampling State() across a tick — by the time
// the tick runs, ingest has already moved the FSM.
sentState string
// sentFill is the pre-fill fraction carried by the last stateMsg, so a
// trigger that is armed but still filling can report progress.
sentFill float64
// How far back the trigger signal's ring reaches and how fast that is
// growing (seconds of span per second of wall clock), refreshed by the hub.
// bufKnown is false when there is no ring to measure, which disables the
// fill gate rather than blocking the trigger on a measurement that will
// never arrive; bufRateOK is false until two measurements exist.
bufSpan float64
bufGrowth float64
bufKnown bool
bufRateOK bool
// Reference point the growth is measured against.
bufRefSpan, bufRefWall float64
prevValue float64
prevValid bool
lastT float64
lastTOK bool
// lastFeedWall is the wall clock at the last feed(), used only to notice a
// stalled stream — the window itself is measured on the sample clock.
lastFeedWall float64
trigTime float64
firedPre float64
firedPost float64
firedValid bool
rearmAt float64 // wall-clock seconds; 0 when no rearm is pending
}
func newTriggerEngine() *triggerEngine {
return &triggerEngine{
cfg: trigConfig{edge: "rising", windowSec: 1, prePercent: 20, mode: "normal", holdoffSec: autoRearmDelaySec},
elemIdx: -1,
state: trigIdle,
}
}
// parseSignalKey splits "src:sig[3]" into ("src:sig", 3). A key without an
// element suffix yields an index of -1.
func parseSignalKey(key string) (string, int) {
if !strings.HasSuffix(key, "]") {
return key, -1
}
open := strings.LastIndexByte(key, '[')
if open < 0 {
return key, -1
}
idx, err := strconv.Atoi(key[open+1 : len(key)-1])
if err != nil || idx < 0 {
return key, -1
}
return key[:open], idx
}
func (te *triggerEngine) SetConfig(cfg trigConfig) {
te.mu.Lock()
defer te.mu.Unlock()
// Clamp to the bounds the web UI offers.
if cfg.windowSec < 1e-4 {
cfg.windowSec = 1e-4
}
if cfg.windowSec > maxTriggerWindowSec {
cfg.windowSec = maxTriggerWindowSec
}
if cfg.prePercent < 0 {
cfg.prePercent = 0
}
if cfg.prePercent > 100 {
cfg.prePercent = 100
}
if cfg.holdoffSec < 0 {
cfg.holdoffSec = 0
}
if cfg.holdoffSec > 60 {
cfg.holdoffSec = 60
}
te.cfg = cfg
base, idx := parseSignalKey(cfg.signalKey)
if base != te.baseKey {
// The buffer measurement belongs to the old signal's ring.
te.bufKnown, te.bufRateOK = false, false
}
te.baseKey, te.elemIdx = base, idx
te.prevValid = false
te.prevValue = 0
}
func (te *triggerEngine) Config() trigConfig {
te.mu.Lock()
defer te.mu.Unlock()
return te.cfg
}
func (te *triggerEngine) Arm() {
te.mu.Lock()
te.state = trigArmed
te.prevValid = false
te.prevValue = 0
te.rearmAt = 0
te.mu.Unlock()
}
func (te *triggerEngine) Disarm() {
te.mu.Lock()
te.state = trigIdle
te.stopped = false
te.prevValid = false
te.prevValue = 0
te.firedValid = false
te.rearmAt = 0
te.mu.Unlock()
}
func (te *triggerEngine) SetStopped(v bool) {
te.mu.Lock()
te.stopped = v
if v {
te.rearmAt = 0
}
te.mu.Unlock()
}
func (te *triggerEngine) Stopped() bool {
te.mu.Lock()
defer te.mu.Unlock()
return te.stopped
}
func (te *triggerEngine) State() string {
te.mu.Lock()
defer te.mu.Unlock()
return te.state
}
// Active reports whether a trigger signal is configured. The rings must stay
// populated from that moment on: a capture reaches back over the pre-trigger
// window, so waiting until the trigger arms would leave that window empty.
func (te *triggerEngine) Active() bool {
te.mu.Lock()
defer te.mu.Unlock()
return te.baseKey != ""
}
// baseSignalKey is the configured trigger signal without its "[i]" suffix, or
// "" when no trigger signal is set.
func (te *triggerEngine) baseSignalKey() string {
te.mu.Lock()
defer te.mu.Unlock()
return te.baseKey
}
// bufGrowthIntervalSec is the shortest baseline the span growth is measured
// over. The hub refreshes 30 times a second and the span moves in steps as
// batches land, so a shorter baseline measures the batching, not the trend.
const bufGrowthIntervalSec = 0.5
// bufGrowthSmooth is the weight of a new growth measurement in the running
// estimate.
const bufGrowthSmooth = 0.5
// setBuffered records how far back the trigger signal's ring reaches, at wall
// clock now, and derives how fast that is growing. Pass known=false when there
// is no such ring.
func (te *triggerEngine) setBuffered(span float64, known bool, now float64) {
te.mu.Lock()
defer te.mu.Unlock()
if !known {
te.bufKnown, te.bufRateOK = false, false
return
}
if !te.bufKnown {
te.bufKnown = true
te.bufRefSpan, te.bufRefWall = span, now
}
te.bufSpan = span
dt := now - te.bufRefWall
if dt < bufGrowthIntervalSec {
return
}
g := (span - te.bufRefSpan) / dt
// A ring that is not full grows one second of span per second; one that is
// full grows by whatever its incoming samples free up. Neither can exceed 1,
// and a shrinking ring is simply not growing.
if g < 0 {
g = 0
} else if g > 1 {
g = 1
}
if te.bufRateOK {
g = te.bufGrowth + bufGrowthSmooth*(g-te.bufGrowth)
}
te.bufGrowth, te.bufRateOK = g, true
te.bufRefSpan, te.bufRefWall = span, now
}
// fillNeedLocked is how far back the buffer must reach before an edge may be
// accepted, so that the capture is still whole when it is harvested a
// post-window later.
//
// What has to hold at harvest time is that the buffer spans the whole window:
// its newest sample is then trigTime+post, so anything less has lost the front
// of the capture. The buffer keeps filling while the post-window is collected,
// though, so the shortfall it may start with is exactly what it will make up in
// that time — measured, not assumed:
//
// need = windowSec − growth × postSec, floored at the pre-trigger window
//
// A ring that is still filling grows a second per second, which reduces this to
// the pre-trigger window: everything after the trigger is yet to be recorded
// anyway. A full one grows only as fast as its incoming samples free space —
// re-bucketing to a longer window replaces dense old samples with sparse new
// ones — and it is that case, growth well below 1, where firing on the
// pre-window alone delivers a capture whose front has been overwritten by the
// time it is read. In the steady state growth is 0 and need is the whole
// window, which a ring tuned for that window already exceeds, so nothing waits.
func (te *triggerEngine) fillNeedLocked() float64 {
pre := te.cfg.windowSec * te.cfg.prePercent / 100
growth := 0.0 // until measured, assume the buffer will not fill on its own
if te.bufRateOK {
growth = te.bufGrowth
}
need := te.cfg.windowSec - growth*(te.cfg.windowSec-pre)
if need < pre {
need = pre
}
return need
}
// fillLocked is how much of that requirement is met, as a fraction in [0, 1].
// It is 1 whenever the gate does not apply: nothing needed, or no ring to
// measure.
func (te *triggerEngine) fillLocked() float64 {
need := te.fillNeedLocked()
if need <= 0 || !te.bufKnown || te.bufSpan >= need*(1-shortCaptureTol) {
return 1
}
if te.bufSpan <= 0 {
return 0
}
return te.bufSpan / need
}
// latchWindowLocked freezes the pre/post split at fire time so later config
// edits do not change how the capture is rendered.
func (te *triggerEngine) latchWindowLocked(t float64) {
te.state = trigCollecting
te.trigTime = t
te.firedPre = te.cfg.windowSec * te.cfg.prePercent / 100
te.firedPost = te.cfg.windowSec - te.firedPre
te.firedValid = true
te.rearmAt = 0
}
// Force fires the trigger immediately at the most recent sample time (falling
// back to the current wall clock when no sample has been seen yet).
func (te *triggerEngine) Force() {
te.mu.Lock()
defer te.mu.Unlock()
if te.state == trigCollecting {
return
}
t := float64(time.Now().UnixNano()) / 1e9
if te.lastTOK {
t = te.lastT
}
te.latchWindowLocked(t)
}
// feed passes a batch of full-resolution samples for one signal to the FSM.
// key is the fully-prefixed "src:sig" name; nElem is the signal's element count
// so that an "[i]"-suffixed configuration can select a single column out of the
// flattened element-major batch.
func (te *triggerEngine) feed(key string, nElem int, t, v []float64) {
if len(t) == 0 || len(t) != len(v) {
return
}
te.mu.Lock()
defer te.mu.Unlock()
if key != te.baseKey {
return
}
te.lastT = t[len(t)-1]
te.lastTOK = true
te.lastFeedWall = float64(time.Now().UnixNano()) / 1e9
if te.state != trigArmed {
return
}
step, start := 1, 0
if te.elemIdx >= 0 && nElem > 1 {
if te.elemIdx >= nElem {
return
}
step, start = nElem, te.elemIdx
}
// Hold off while the buffer does not reach back far enough. Firing now would
// deliver a capture whose front is simply missing — the ring never held it —
// which is what made the first shot after a window change come back short.
// Track the level meanwhile, so the first edge once the buffer is deep
// enough is still measured against the right previous sample.
if te.fillLocked() < 1 {
for i := start; i < len(v); i += step {
te.prevValue, te.prevValid = v[i], true
}
return
}
thr := te.cfg.threshold
for i := start; i < len(t); i += step {
if !te.prevValid {
te.prevValue = v[i]
te.prevValid = true
continue
}
up := te.prevValue < thr && v[i] >= thr
down := te.prevValue > thr && v[i] <= thr
te.prevValue = v[i]
fired := false
switch te.cfg.edge {
case "falling":
fired = down
case "both":
fired = up || down
default:
fired = up
}
if fired {
te.latchWindowLocked(t[i])
return
}
}
}
// dueCapture reports whether a collecting trigger's post-window has elapsed and
// returns the latched window.
//
// The window is measured on the sample clock, not the wall clock: trigTime is a
// sample timestamp, and a stream whose timestamps lag real time (a busy
// producer, a buffered link) would otherwise be cut short by exactly that lag —
// an 8 s lag turned a 60 s window into a 36 s capture. Waiting for the samples
// themselves also means the ring really holds the window by the time it is read.
func (te *triggerEngine) dueCapture(nowSec float64) (trigTime, pre, post float64, ok bool) {
te.mu.Lock()
defer te.mu.Unlock()
if te.state != trigCollecting || !te.firedValid {
return 0, 0, 0, false
}
deadline := te.trigTime + te.firedPost + captureMarginSec
switch {
case te.lastTOK && te.lastT >= deadline:
// The samples have covered the window.
case !te.lastTOK && nowSec >= deadline:
// No sample ever seen, so trigTime came from the wall clock (Force).
case te.lastFeedWall > 0 && nowSec-te.lastFeedWall >= captureStallSec:
// The stream has dried up; deliver what was collected rather than
// leaving the client stuck in "collecting" forever.
default:
return 0, 0, 0, false
}
return te.trigTime, te.firedPre, te.firedPost, true
}
// markTriggered completes a capture and schedules the automatic rearm when the
// engine runs in "normal" mode.
func (te *triggerEngine) markTriggered(nowSec float64) {
te.mu.Lock()
if te.state == trigCollecting {
te.state = trigTriggered
if te.cfg.mode != "single" && !te.stopped {
te.rearmAt = nowSec + te.cfg.holdoffSec
}
}
te.mu.Unlock()
}
// dueRearm reports whether a pending automatic rearm has come due, consuming it.
func (te *triggerEngine) dueRearm(nowSec float64) bool {
te.mu.Lock()
defer te.mu.Unlock()
if te.state != trigTriggered || te.rearmAt == 0 || nowSec < te.rearmAt {
return false
}
te.rearmAt = 0
return !te.stopped
}
// stateUnsent reports whether the FSM has moved since the last stateMsg was
// built, i.e. whether clients still have to be told.
func (te *triggerEngine) stateUnsent() bool {
te.mu.Lock()
defer te.mu.Unlock()
if te.state != te.sentState {
return true
}
// An armed trigger waiting for its buffer is otherwise indistinguishable
// from one that is ignoring edges, so the filling itself is news. Coarse
// steps only: this is checked 30 times a second.
if te.state == trigArmed {
f := te.fillLocked()
return math.Abs(f-te.sentFill) >= 0.02 || (f >= 1 && te.sentFill < 1)
}
return false
}
// stateMsg builds the JSON "triggerState" broadcast for the current FSM state.
func (te *triggerEngine) stateMsg() []byte {
te.mu.Lock()
te.sentState = te.state
te.sentFill = te.fillLocked()
m := map[string]any{
"type": "triggerState",
"state": te.state,
"mode": te.cfg.mode,
"stopped": te.stopped,
}
if te.state == trigArmed && te.sentFill < 1 {
// Armed but holding off: the buffer does not yet reach back far enough
// to deliver the window, so edges are being ignored on purpose.
m["bufferFill"] = te.sentFill
m["bufferNeedSec"] = te.fillNeedLocked()
}
if te.firedValid {
// The window latched at fire time. Clients draw the filling capture on
// this axis before the v2 frame arrives, and config edits between arm
// and fire would otherwise leave them inferring the wrong window from
// their own copy of the config.
m["trigTime"] = te.trigTime
m["preSec"] = te.firedPre
m["postSec"] = te.firedPost
}
te.mu.Unlock()
msg, _ := json.Marshal(m)
return msg
}
/* ─── Hub integration ─────────────────────────────────────────────────────── */
// broadcastTriggerState pushes the current FSM state to every client.
func (h *Hub) broadcastTriggerState() {
h.broadcast(h.trigger.stateMsg())
}
// handleTriggerCommand processes a trigger-related browser message. It returns
// false when the message type is not a trigger command.
func (h *Hub) handleTriggerCommand(t string, env map[string]interface{}) bool {
switch t {
case "setTrigger":
cfg := h.trigger.Config()
if s, ok := env["signal"].(string); ok {
cfg.signalKey = s
}
if s, ok := env["edge"].(string); ok {
cfg.edge = s
}
if s, ok := env["mode"].(string); ok {
cfg.mode = s
}
if f, ok := env["threshold"].(float64); ok {
cfg.threshold = f
}
if f, ok := env["windowSec"].(float64); ok {
cfg.windowSec = f
}
if f, ok := env["prePercent"].(float64); ok {
cfg.prePercent = f
}
if f, ok := env["holdoffSec"].(float64); ok {
cfg.holdoffSec = f
}
h.trigger.SetConfig(cfg)
case "arm", "rearm":
h.trigger.Arm()
case "disarm":
h.trigger.Disarm()
case "trigStop":
stopped := !h.trigger.Stopped()
if b, ok := env["stopped"].(bool); ok {
stopped = b
}
h.trigger.SetStopped(stopped)
case "forceTrigger":
h.trigger.Force()
default:
return false
}
// Measure the buffer now rather than waiting for the next tick: ingest runs
// on the source goroutine and a 1 MSps stream crosses the threshold many
// times within one 33 ms tick, so an arm serviced here would otherwise fire
// on a stale (or missing) measurement before the gate ever saw the new
// configuration.
h.refreshTriggerFill()
h.broadcastTriggerState()
return true
}
// refreshTriggerFill tells the FSM how far back the trigger signal's ring
// reaches, which is what lets an armed trigger hold off until a capture taken
// now would come back whole.
//
// The ring is the right yardstick even though a short capture is back-filled
// from the archive: the archive is sized for the same window and starts over
// whenever that window changes, so it holds no more of the stretch being waited
// for than the ring does. It can only add to what the capture finds.
//
// Called both from the push tick and from the client goroutine handling a
// trigger command; all the state it derives lives in the engine, behind the
// engine's lock.
func (h *Hub) refreshTriggerFill() {
if h.trigger == nil {
return
}
now := float64(time.Now().UnixNano()) / 1e9
var rb *sigRing
if key := h.trigger.baseSignalKey(); key != "" {
rb = h.getRing(key)
}
if rb == nil {
// Nothing to measure. Do not gate on a signal the hub does not carry:
// that would leave the trigger armed forever, which is worse than a
// short capture.
h.trigger.setBuffered(0, false, now)
return
}
_, span := rb.stats()
h.trigger.setBuffered(span, true, now)
}
// triggerTick services the trigger FSM; called from Hub.Run() on every push tick.
func (h *Hub) triggerTick() {
nowSec := float64(time.Now().UnixNano()) / 1e9
h.retuneRings(nowSec)
h.openPendingHistoryFiles(nowSec)
h.refreshTriggerFill()
if trigTime, pre, post, ok := h.trigger.dueCapture(nowSec); ok {
if msg := h.buildTriggerCapture(trigTime, pre, post); msg != nil {
dropped := 0
for c := range h.clients {
select {
case c.send <- wsMessage{websocket.BinaryMessage, msg}:
default:
dropped++
}
}
// A dropped capture is invisible to the user — the trigger fires,
// the state goes to "triggered" and no waveform ever arrives — so
// say so rather than leaving it to be guessed at.
if dropped > 0 {
log.Printf("wshub: trigger capture (%d B) dropped for %d client(s): send queue full",
len(msg), dropped)
}
}
h.trigger.markTriggered(nowSec)
// A capture is only zoomable for as long as its samples still exist at
// full resolution somewhere, and the rings roll past the window within
// seconds of it being taken. Lift the window out of the archive into a
// file of its own, where nothing overwrites it until the next trigger.
h.hist.captureRange(trigTime-pre, trigTime+post)
} else if h.trigger.dueRearm(nowSec) {
h.trigger.Arm()
}
if h.trigger.stateUnsent() {
h.broadcastTriggerState()
}
}
// backfillCaptureHead prepends the front of [t0, t1] that the ring no longer
// holds, read from the disk archive. It returns its input unchanged when the
// ring already reaches t0, when history is off, or when the archive has nothing
// for that range.
//
// The rings are sized for the window, but they only have to *become* that long:
// they are min/max buckets that cover the configured window once they have
// rolled over completely at the current bucket, which takes as long as the
// window itself. Widen the window and arm, and the first captures ask for more
// history than the ring has ever stored — the frame then starts late and the
// user sees a blank front half. The archive is written straight through, at the
// geometry its file was created with, so unless that file was re-sized too it
// has kept the stretch the ring is still converging on.
func (h *Hub) backfillCaptureHead(key string, t0, t1 float64, st, sv []float64) ([]float64, []float64) {
window := t1 - t0
if !h.hist.enabled() || window <= 0 {
return st, sv
}
gapEnd := t1
if len(st) > 0 {
gapEnd = st[0]
}
gap := gapEnd - t0
if gap <= shortCaptureTol*window {
return st, sv
}
// Budget the read by the share of the window being back-filled. The frame is
// decimated to trigCapturePts either way, so a bigger read would buy nothing
// but disk seeks — on the hub's own goroutine, between two push ticks.
maxOut := int(float64(trigCapturePts)*gap/window) + 2
ht, hv := h.hist.readRange(key, t0, gapEnd, maxOut)
if len(ht) == 0 {
return st, sv
}
// Drop anything at or past the ring's first sample: the two sources overlap
// around the join, and the frame's timestamps must stay ascending.
n := len(ht)
if len(st) > 0 {
n = sort.SearchFloat64s(ht, st[0])
}
if n == 0 {
return st, sv
}
outT := make([]float64, 0, n+len(st))
outV := make([]float64, 0, n+len(sv))
outT = append(append(outT, ht[:n]...), st...)
outV = append(append(outV, hv[:n]...), sv...)
return outT, outV
}
// buildTriggerCapture extracts [trigTime-pre, trigTime+post] from every ring
// buffer and encodes the version-2 binary capture frame:
//
// [u8 2][f64 trigTime][f64 preSec][f64 postSec][u32 nSig]
// {[u16 keyLen][fullKey][u32 N][t f64×N][v f64×N]}
func (h *Hub) buildTriggerCapture(trigTime, pre, post float64) []byte {
t0, t1 := trigTime-pre, trigTime+post
type sigSlice struct {
key string
t, v []float64
}
h.ringsMu.RLock()
keys := make([]string, 0, len(h.rings))
rings := make([]*sigRing, 0, len(h.rings))
for k, rb := range h.rings {
keys = append(keys, k)
rings = append(rings, rb)
}
h.ringsMu.RUnlock()
slices := make([]sigSlice, 0, len(keys))
held := make(map[string]sigData, len(keys))
total := 1 + 8 + 8 + 8 + 4
for i, k := range keys {
st, sv := rings[i].slice(t0, t1)
st, sv = h.backfillCaptureHead(k, t0, t1, st, sv)
if len(st) == 0 {
continue
}
// Neither the ring nor the archive reached t0. Nothing can recover that
// data, so name it rather than leaving the user to wonder why the front
// of their window is blank.
if lost := st[0] - t0; lost > shortCaptureTol*(t1-t0) {
cnt, span := rings[i].stats()
log.Printf("wshub: capture %s is short by %.2f s of %.2f s: ring holds %.2f s (%d pts, min/max over %d)",
k, lost, t1-t0, span, cnt, rings[i].bucketSize())
}
// Take the second half of the double buffer here, before the frame is
// decimated: the client gets 20 000 points to draw, but a zoom into
// them has to come back with the underlying samples, and the rings will
// have rolled past them by the time it is asked for.
held[k] = sigData{T: st, V: sv}
// Decimate before framing: a long window at a high sample rate is
// hundreds of megabytes raw, which the send path would silently drop.
// The min/max envelope keeps every peak in the window, so a glitch is
// still on screen at the zoomed-out view that first shows it.
st, sv = minMaxDecimate(st, sv, trigCapturePts)
slices = append(slices, sigSlice{key: k, t: st, v: sv})
total += 2 + len(k) + 4 + len(st)*16
}
if len(slices) == 0 {
return nil
}
// Swap only now that the capture is known good. A shot that yielded nothing
// must leave the previous window on screen rather than blanking it.
h.capture.publish(t0, t1, held)
buf := make([]byte, total)
buf[0] = 2
off := 1
binary.LittleEndian.PutUint64(buf[off:], math.Float64bits(trigTime))
off += 8
binary.LittleEndian.PutUint64(buf[off:], math.Float64bits(pre))
off += 8
binary.LittleEndian.PutUint64(buf[off:], math.Float64bits(post))
off += 8
binary.LittleEndian.PutUint32(buf[off:], uint32(len(slices)))
off += 4
for _, s := range slices {
binary.LittleEndian.PutUint16(buf[off:], uint16(len(s.key)))
off += 2
copy(buf[off:], s.key)
off += len(s.key)
binary.LittleEndian.PutUint32(buf[off:], uint32(len(s.t)))
off += 4
off = writeFloat64s(buf, off, s.t)
off = writeFloat64s(buf, off, s.v)
}
return buf
}
Common/Client/go/wshub/trigger_capture_test.go
+278
View File
@@ -0,0 +1,278 @@
package wshub
import (
"encoding/binary"
"math"
"testing"
)
// fillRing writes n samples at the given rate starting at t0.
func fillRing(rb *sigRing, t0 float64, rate float64, n int) {
ts := make([]float64, n)
vs := make([]float64, n)
for i := range ts {
ts[i] = t0 + float64(i)/rate
vs[i] = math.Sin(float64(i))
}
rb.write(ts, vs)
}
func TestRingGrowPreservesSamples(t *testing.T) {
rb := newSigRing(100)
// Overflow the ring so the retained window starts mid-buffer.
fillRing(rb, 0, 1000, 250)
beforeT, beforeV := rb.slice(-1e9, 1e9)
if len(beforeT) != 100 {
t.Fatalf("pre-grow fill = %d, want 100", len(beforeT))
}
if !rb.grow(1000) {
t.Fatal("grow(1000) returned false")
}
if rb.capacity() != 1000 {
t.Fatalf("capacity = %d, want 1000", rb.capacity())
}
afterT, afterV := rb.slice(-1e9, 1e9)
if len(afterT) != len(beforeT) {
t.Fatalf("post-grow fill = %d, want %d", len(afterT), len(beforeT))
}
for i := range beforeT {
if afterT[i] != beforeT[i] || afterV[i] != beforeV[i] {
t.Fatalf("sample %d changed across grow", i)
}
}
// Further writes must keep landing in order rather than wrapping early.
fillRing(rb, 1.0, 1000, 500)
if n, _ := rb.stats(); n != 600 {
t.Fatalf("fill after grow = %d, want 600", n)
}
// Shrinking is refused.
if rb.grow(10) {
t.Fatal("grow(10) shrank the ring")
}
}
func TestRingStatsMeasuresRate(t *testing.T) {
rb := newSigRing(10000)
fillRing(rb, 0, 1000, 1000) // 1 kHz
n, span := rb.stats()
if n != 1000 {
t.Fatalf("count = %d, want 1000", n)
}
rate := float64(n) / span
if math.Abs(rate-1001) > 5 { // n samples span (n-1) intervals
t.Fatalf("rate = %v, want ~1000", rate)
}
}
// A long trigger window must grow the rings to hold it: a fixed sample-count
// ring covers a fraction of a second at a high rate, which is what made 60 s
// captures come back with only their tail populated.
func TestRetuneRingsCoversTriggerWindow(t *testing.T) {
h := NewHub()
rb := newSigRing(6000) // 6 s at 1 kHz — far short of a 60 s window
fillRing(rb, 0, 1000, 6000)
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", edge: "rising",
windowSec: 60, prePercent: 20, mode: "normal"})
h.retuneRings(1000)
// 60 s at 1 kHz is 60 k samples: growing to the budget holds them verbatim.
if got := rb.capacity(); got < 60000 {
t.Fatalf("capacity = %d, want >= 60000 to hold a 60 s window", got)
}
if got := rb.bucketSize(); got != 1 {
t.Fatalf("bucket = %d, want 1: the window fits at full rate", got)
}
}
// Past the budget the window is kept by reducing resolution, not by dropping
// its head — the whole point of the min/max buckets.
func TestRetuneRingsBucketsWhenTheWindowExceedsTheBudget(t *testing.T) {
h := NewHub()
rb := newSigRing(1000)
fillRing(rb, 0, 1e6, 100_000) // 1 MSps
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 60, mode: "normal"})
h.retuneRings(1000)
if got := rb.capacity(); got != defaultRingPts {
t.Fatalf("capacity = %d, want the budget %d", got, defaultRingPts)
}
// 60 s at 1 MSps is 60 M samples in a 10 M-point buffer, so each stored
// pair must cover at least 12 source samples.
bucket := rb.bucketSize()
if bucket < 12 {
t.Fatalf("bucket = %d, too fine to fit 60 M samples in %d points", bucket, rb.capacity())
}
if covered := float64(rb.capacity()) / 2 * float64(bucket) / 1e6; covered < 60 {
t.Fatalf("buffer covers %.1f s, want the whole 60 s window", covered)
}
}
// A raised budget buys resolution back: the same window is held verbatim.
func TestRetuneRingsHonoursRaisedBudget(t *testing.T) {
h := NewHub()
h.SetRingBudget(80_000_000)
rb := newSigRing(1000)
fillRing(rb, 0, 1e6, 100_000) // 1 MSps
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 60, mode: "normal"})
h.retuneRings(1000)
if got := rb.bucketSize(); got != 1 {
t.Fatalf("bucket = %d, want 1: 60 M samples fit in an 80 M-point buffer", got)
}
}
func TestSetRingBudgetBounds(t *testing.T) {
h := NewHub()
h.SetRingBudget(0)
if got := h.ringBudget(); got != defaultRingPts {
t.Fatalf("ringBudget after 0 = %d, want the default %d", got, defaultRingPts)
}
// Never below the depth a freshly configured ring already has, or the
// budget would ask for a shrink the ring refuses anyway.
h.SetRingBudget(10)
if got := h.ringBudget(); got != ringCapInitial {
t.Fatalf("ringBudget after 10 = %d, want the floor %d", got, ringCapInitial)
}
}
func TestRetuneRingsIsThrottled(t *testing.T) {
h := NewHub()
h.SetRingBudget(250_000)
rb := newSigRing(250_000)
fillRing(rb, 0, 1e6, 100_000)
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 10, mode: "normal"})
h.retuneRings(100)
first := rb.bucketSize()
if first <= 1 {
t.Fatalf("bucket = %d, expected a reduction for 10 s at 1 MSps in 250 k points", first)
}
// Same second: the sweep must not run again even though a bigger window
// is now configured.
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 600, mode: "normal"})
h.retuneRings(100.5)
if rb.bucketSize() != first {
t.Fatalf("sweep ran inside the throttle window")
}
h.retuneRings(200)
if rb.bucketSize() <= first {
t.Fatalf("sweep did not run after the throttle window elapsed")
}
}
// With no trigger armed and no client saying otherwise, the rings are sized for
// the default live window — live mode needs the buffers just as much as a
// capture does.
func TestRetuneRingsSizesForTheLiveWindow(t *testing.T) {
h := NewHub()
rb := newSigRing(1000)
fillRing(rb, 0, 1e6, 100_000) // 1 MSps: 10 s does not fit in 1000 points
h.rings["s1:sig"] = rb
// No signal configured → trigger inactive, so the live window governs.
h.trigger.SetConfig(trigConfig{windowSec: 600, mode: "normal"})
h.retuneRings(100)
if got := rb.capacity(); got != defaultRingPts {
t.Fatalf("capacity = %d, want the budget %d", got, defaultRingPts)
}
// defaultLiveWindowSec at 1 MSps is exactly the budget, so no reduction.
if got := rb.bucketSize(); got != 1 {
t.Fatalf("bucket = %d, want 1 for the default live window", got)
}
}
func TestRingBucketForCoversTheWindow(t *testing.T) {
cases := []struct {
rate, window float64
capacity int
want int
}{
{1000, 10, 1_000_000, 1}, // 10 k samples in 1 M points: verbatim
{1e6, 10, 10_000_000, 1}, // exactly the budget: still verbatim
{1e6, 60, 10_000_000, 15}, // 60 M samples, 1.25x headroom
{1e6, 600, 10_000_000, 150}, // 600 s still fits, at 1/150 resolution
{0, 10, 1_000_000, 1}, // no rate measured yet
{1000, 0, 1_000_000, 1}, // no window
}
for _, c := range cases {
if got := ringBucketFor(c.rate, c.window, c.capacity); got != c.want {
t.Errorf("ringBucketFor(%v, %v, %d) = %d, want %d",
c.rate, c.window, c.capacity, got, c.want)
}
}
}
// decodeCapture pulls the per-signal point counts out of a v2 capture frame.
func decodeCapture(t *testing.T, buf []byte) map[string]int {
t.Helper()
if buf[0] != 2 {
t.Fatalf("frame version = %d, want 2", buf[0])
}
off := 1 + 8 + 8 + 8
nSig := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
out := make(map[string]int, nSig)
for i := 0; i < nSig; i++ {
kl := int(binary.LittleEndian.Uint16(buf[off:]))
off += 2
key := string(buf[off : off+kl])
off += kl
n := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
off += n * 16
out[key] = n
}
if off != len(buf) {
t.Fatalf("decoded %d of %d bytes", off, len(buf))
}
return out
}
// A 60 s window at a high rate is hundreds of megabytes raw; the capture frame
// must be decimated so it can actually reach a client.
func TestBuildTriggerCaptureDecimates(t *testing.T) {
h := NewHub()
rb := newSigRing(200000)
fillRing(rb, 0, 100000, 200000) // 2 s at 100 kSps
h.rings["s1:sig"] = rb
buf := h.buildTriggerCapture(1.0, 1.0, 1.0)
if buf == nil {
t.Fatal("no capture frame built")
}
counts := decodeCapture(t, buf)
n := counts["s1:sig"]
if n != trigCapturePts {
t.Fatalf("captured %d points, want the %d-point cap", n, trigCapturePts)
}
}
// Short captures must stay full resolution — decimation only kicks in above
// the cap.
func TestBuildTriggerCaptureKeepsSmallWindowsIntact(t *testing.T) {
h := NewHub()
rb := newSigRing(10000)
fillRing(rb, 0, 1000, 10000) // 10 s at 1 kHz
h.rings["s1:sig"] = rb
buf := h.buildTriggerCapture(1.0, 0.5, 0.5)
if buf == nil {
t.Fatal("no capture frame built")
}
counts := decodeCapture(t, buf)
if n := counts["s1:sig"]; n < 990 || n > 1010 {
t.Fatalf("captured %d points, want ~1000 undecimated", n)
}
}
Common/Client/go/wshub/trigger_test.go
+507
View File
@@ -0,0 +1,507 @@
package wshub
import (
"encoding/json"
"math"
"testing"
"time"
)
func TestParseSignalKey(t *testing.T) {
cases := []struct {
in string
base string
idx int
}{
{"src:sig", "src:sig", -1},
{"src:sig[0]", "src:sig", 0},
{"src:sig[3]", "src:sig", 3},
{"src:sig[x]", "src:sig[x]", -1},
{"src:sig]", "src:sig]", -1},
}
for _, c := range cases {
base, idx := parseSignalKey(c.in)
if base != c.base || idx != c.idx {
t.Errorf("parseSignalKey(%q) = (%q,%d), want (%q,%d)",
c.in, base, idx, c.base, c.idx)
}
}
}
func armed(key, edge string, thr float64) *triggerEngine {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: key, edge: edge, threshold: thr,
windowSec: 1, prePercent: 20, mode: "normal", holdoffSec: autoRearmDelaySec})
te.Arm()
return te
}
func TestFeedRisingEdge(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{1, 2, 3, 4}, []float64{0, 0.2, 0.9, 1.0})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
// Fires at the sample that crossed, i.e. t=3.
trigTime, pre, post, ok := te.dueCapture(1e9)
if !ok || trigTime != 3 {
t.Fatalf("dueCapture = (%v,%v), want trigTime 3", trigTime, ok)
}
if pre != 0.2 || post != 0.8 {
t.Errorf("pre/post = %v/%v, want 0.2/0.8", pre, post)
}
}
func TestFeedFallingEdgeIgnoresRising(t *testing.T) {
te := armed("src:sig", "falling", 0.5)
te.feed("src:sig", 1, []float64{1, 2, 3}, []float64{0, 0.9, 1.0})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed (no falling edge)", te.State())
}
te.feed("src:sig", 1, []float64{4, 5}, []float64{0.6, 0.1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
}
func TestFeedIgnoresOtherSignals(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:other", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed", te.State())
}
}
func TestFeedArrayElementSelection(t *testing.T) {
// 2-element signal, element-major: [e0,e1, e0,e1, ...]. Only element 1
// crosses the threshold.
te := armed("src:sig[1]", "rising", 0.5)
tt := []float64{1, 1, 2, 2}
vv := []float64{0, 0, 0, 1}
te.feed("src:sig", 2, tt, vv)
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
// Element 0 never crosses, so a config on [0] must not fire.
te2 := armed("src:sig[0]", "rising", 0.5)
te2.feed("src:sig", 2, tt, vv)
if te2.State() != trigArmed {
t.Fatalf("state = %q, want armed", te2.State())
}
}
func TestForceUsesLastSampleTime(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising", threshold: 1e9,
windowSec: 2, prePercent: 50, mode: "single"})
te.Arm()
te.feed("src:sig", 1, []float64{10, 11, 12}, []float64{0, 0, 0})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed (threshold unreachable)", te.State())
}
te.Force()
// post = 1 s, so the capture waits for samples past t = 12 + 1 + 0.15.
te.feed("src:sig", 1, []float64{13.2}, []float64{0})
trigTime, pre, post, ok := te.dueCapture(1e9)
if !ok || trigTime != 12 || pre != 1 || post != 1 {
t.Fatalf("dueCapture = (%v,%v,%v,%v), want (12,1,1,true)",
trigTime, pre, post, ok)
}
}
func TestForceFromIdle(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising",
windowSec: 1, prePercent: 20, mode: "normal"})
te.Force()
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
}
func TestCaptureMarginDelaysExtraction(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1}) // fires at t=1
// post = 0.8 s; capture is due once the samples reach 1 + 0.8 + 0.15.
te.feed("src:sig", 1, []float64{1.9}, []float64{0})
if _, _, _, ok := te.dueCapture(1e9); ok {
t.Error("capture extracted before the margin elapsed")
}
te.feed("src:sig", 1, []float64{1.96}, []float64{0})
if _, _, _, ok := te.dueCapture(1e9); !ok {
t.Error("capture not extracted after the margin elapsed")
}
}
// A stream whose timestamps run behind real time must still yield the whole
// window: measuring the post-window on the wall clock cut the capture short by
// exactly the lag (an 8 s lag turned a 60 s window into a 36 s one).
func TestCaptureWaitsForLaggingStream(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1}) // fires at t=1
wallNow := float64(time.Now().UnixNano()) / 1e9
// Wall clock is far past the post-window, but the samples are not.
te.feed("src:sig", 1, []float64{1.5}, []float64{0})
if _, _, _, ok := te.dueCapture(wallNow); ok {
t.Error("capture extracted while the stream was still short of the window")
}
te.feed("src:sig", 1, []float64{2.0}, []float64{0})
if _, _, _, ok := te.dueCapture(wallNow); !ok {
t.Error("capture not extracted once the samples covered the window")
}
}
// A dead stream must not leave the client stuck in "collecting" forever.
func TestCaptureCompletesWhenStreamStalls(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1}) // fires at t=1
wallNow := float64(time.Now().UnixNano()) / 1e9
if _, _, _, ok := te.dueCapture(wallNow + captureStallSec/2); ok {
t.Error("capture extracted before the stall timeout")
}
if _, _, _, ok := te.dueCapture(wallNow + captureStallSec + 0.1); !ok {
t.Error("capture not extracted after the stream stalled")
}
}
func TestAutoRearmNormalMode(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1})
te.markTriggered(100)
if te.State() != trigTriggered {
t.Fatalf("state = %q, want triggered", te.State())
}
if te.dueRearm(100.1) {
t.Error("rearmed before the delay elapsed")
}
if !te.dueRearm(100.3) {
t.Error("did not rearm after the delay elapsed")
}
if te.dueRearm(200) {
t.Error("rearm was not consumed")
}
}
func TestNoAutoRearmInSingleMode(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising", threshold: 0.5,
windowSec: 1, prePercent: 20, mode: "single"})
te.Arm()
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1})
te.markTriggered(100)
if te.dueRearm(200) {
t.Error("single mode must not auto-rearm")
}
}
func TestStoppedSuppressesRearm(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1})
te.SetStopped(true)
te.markTriggered(100)
if te.dueRearm(200) {
t.Error("stopped engine must not rearm")
}
}
func TestSetConfigClamps(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 1000, prePercent: 500, holdoffSec: 120})
if cfg := te.Config(); cfg.windowSec != 600 || cfg.prePercent != 100 || cfg.holdoffSec != 60 {
t.Errorf("upper clamp = %v/%v/%v, want 600/100/60", cfg.windowSec, cfg.prePercent, cfg.holdoffSec)
}
// The web UI's longest option must survive intact — it used to be clamped
// to 60 s, so a 10 min capture silently came back one minute long.
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 600, prePercent: 20, holdoffSec: 1})
if cfg := te.Config(); cfg.windowSec != 600 {
t.Errorf("windowSec = %v, want the requested 600", cfg.windowSec)
}
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 0, prePercent: -5, holdoffSec: -1})
if cfg := te.Config(); cfg.windowSec != 1e-4 || cfg.prePercent != 0 || cfg.holdoffSec != 0 {
t.Errorf("lower clamp = %v/%v/%v, want 1e-4/0/0", cfg.windowSec, cfg.prePercent, cfg.holdoffSec)
}
}
func TestHoldoffControlsRearmDelay(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising", threshold: 0.5,
windowSec: 1, prePercent: 20, mode: "normal", holdoffSec: 5})
te.Arm()
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1})
te.markTriggered(100)
if te.dueRearm(104.9) {
t.Error("rearmed before the configured holdoff elapsed")
}
if !te.dueRearm(105.1) {
t.Error("did not rearm after the configured holdoff elapsed")
}
}
func TestActiveTracksConfiguredSignal(t *testing.T) {
te := newTriggerEngine()
if te.Active() {
t.Error("a fresh engine must not be active")
}
te.SetConfig(trigConfig{signalKey: "src:sig", windowSec: 1})
if !te.Active() {
t.Error("engine must be active once a signal is configured")
}
// Rings must keep filling after a capture completes, not just while armed.
te.Disarm()
if !te.Active() {
t.Error("engine must stay active after disarm while a signal is set")
}
}
// The armed→collecting transition happens inside feed(), on the ingest path,
// which the hub runs before triggerTick in the same loop iteration. Clients need
// that state — it carries trigTime and the latched window, without which they
// cannot draw the window filling and sit frozen until the capture arrives.
func TestCollectingIsBroadcast(t *testing.T) {
h := NewHub()
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", edge: "rising", threshold: 0,
windowSec: 10, prePercent: 20, mode: "single"})
h.trigger.Arm()
h.triggerTick()
drainStates(t, h)
// Fire, but stay well inside the post-trigger window: the capture is still
// seconds away and this is exactly when the client has nothing to draw.
h.ingest("s1:sig", 1, []float64{5.0, 5.001}, []float64{-1, 1})
h.triggerTick()
states := drainStates(t, h)
found := false
for _, m := range states {
if m["state"] == trigCollecting {
found = true
if m["trigTime"] != 5.001 {
t.Errorf("collecting broadcast has trigTime %v, want 5.001", m["trigTime"])
}
if m["preSec"] != 2.0 || m["postSec"] != 8.0 {
t.Errorf("collecting broadcast has pre=%v post=%v, want 2 and 8",
m["preSec"], m["postSec"])
}
}
}
if !found {
t.Fatalf("no collecting broadcast after the trigger fired, got %v", states)
}
}
// setFill hands the engine a buffer span and a growth rate, as the hub's
// per-tick measurements would: a reference point and a second one a second
// later. It forgets any earlier measurement first, so the rate is the one
// asked for rather than a blend with it.
func setFill(te *triggerEngine, span, growth, now float64) {
te.setBuffered(0, false, now)
te.setBuffered(span-growth, true, now)
te.setBuffered(span, true, now+1)
}
// What has to hold is that the buffer spans the whole window by the time the
// capture is read, one post-window after the trigger fires — so whatever it
// will fill in on its own during that time need not be there yet.
func TestFillNeed(t *testing.T) {
cases := []struct {
window, prePercent, growth, want float64
}{
{100, 20, 1, 20}, // still filling: only the pre-window has to exist
{100, 20, 0.5, 60}, // half speed: 40 s of the 80 s post-window fills in
{100, 20, 0, 100}, // not growing at all: it must already be all there
{100, 0, 0.9, 10}, // no pre-window, but the buffer still has to keep up
{100, 100, 1, 100}, // all pre-window: nothing fills in after the trigger
}
for _, c := range cases {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", windowSec: c.window, prePercent: c.prePercent})
setFill(te, 1e6, c.growth, 100) // span large enough not to matter
te.mu.Lock()
got := te.fillNeedLocked()
te.mu.Unlock()
if math.Abs(got-c.want) > 1e-6 {
t.Errorf("fillNeed(window %v, pre %v%%, growth %v) = %v, want %v",
c.window, c.prePercent, c.growth, got, c.want)
}
}
}
// A trigger that fires before its pre-window has been buffered can only produce
// a capture whose front half never existed. It must wait instead.
func TestFillGateHoldsFire(t *testing.T) {
te := armed("src:sig", "rising", 0.5) // window 1 s, pre 20 % → 0.2 s needed
setFill(te, 0.05, 1, 100)
te.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed: only 0.05 s of the 0.2 s pre-window is buffered", te.State())
}
// The level was still tracked, so the next crossing is a real edge and not a
// re-detection of the one that was held off.
setFill(te, 0.25, 1, 200)
te.feed("src:sig", 1, []float64{3, 4}, []float64{1, 1})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed: no crossing, the signal stayed high", te.State())
}
te.feed("src:sig", 1, []float64{5, 6}, []float64{0, 1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting once the pre-window is buffered", te.State())
}
te.feed("src:sig", 1, []float64{7, 8}, []float64{1, 1}) // carry the sample clock past the window
if trigTime, _, _, ok := te.dueCapture(1e9); !ok || trigTime != 6 {
t.Errorf("dueCapture = (%v,%v), want trigTime 6", trigTime, ok)
}
}
// A ring that is full and re-bucketing for a longer window fills slower than
// real time — it drops dense old samples to take sparse new ones — so more of
// the window has to be there before an edge may be accepted.
func TestFillGateAccountsForSlowGrowth(t *testing.T) {
te := armed("src:sig", "rising", 0.5) // window 1 s, pre 20 % → post 0.8 s
// At half speed only 0.4 s of the post-window fills in, so 0.6 s is needed.
setFill(te, 0.5, 0.5, 100)
te.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed: 0.5 s buffered of the 0.6 s needed", te.State())
}
// The same 0.5 s in a ring still filling at full speed is plenty: everything
// after the trigger is yet to be recorded anyway.
te2 := armed("src:sig", "rising", 0.5)
setFill(te2, 0.5, 1, 100)
te2.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te2.State() != trigCollecting {
t.Fatalf("state = %q, want collecting: the buffer keeps up with the stream", te2.State())
}
setFill(te, 0.65, 0.5, 200)
te.feed("src:sig", 1, []float64{3, 4}, []float64{0, 1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting once the buffer will span the window", te.State())
}
}
func TestFillGateInactiveWithoutMeasurement(t *testing.T) {
// No ring for the configured signal: gating would leave the trigger armed
// forever, which is worse than a short capture.
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting: nothing measured, so nothing to gate on", te.State())
}
// Nor is there anything to wait for when the buffer keeps up and the whole
// window is still to come.
te = newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising", threshold: 0.5,
windowSec: 1, prePercent: 0, mode: "normal"})
te.Arm()
setFill(te, 0, 1, 100)
te.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting with a 0 %% pre-window", te.State())
}
}
// Force is the user overriding the trigger, so it overrides the gate too.
func TestForceIgnoresFillGate(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
setFill(te, 0, 0, 100)
te.Force()
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
}
// seedFillNow is setFill against the real clock, for tests that then let the
// hub take its own measurements: its ticks land inside the growth measurement
// interval, so they refresh the span and leave the seeded rate alone.
func seedFillNow(te *triggerEngine, span, growth float64) {
now := float64(time.Now().UnixNano()) / 1e9
te.setBuffered(0, false, now-1)
te.setBuffered(span-growth, true, now-1)
te.setBuffered(span, true, now)
}
// While it holds off, the trigger looks identical to one that is ignoring
// edges. The state broadcast has to say it is filling, and keep saying so.
func TestFillProgressIsBroadcast(t *testing.T) {
h := NewHub()
rb := newSigRing(1000)
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", edge: "rising", threshold: 0,
windowSec: 10, prePercent: 50, mode: "single"}) // 5 s of pre-window
h.trigger.Arm()
rb.write([]float64{0, 1}, []float64{-1, -1})
// Filling at the rate of the stream, so only the pre-window is needed.
seedFillNow(h.trigger, 1, 1)
h.triggerTick()
states := drainStates(t, h)
if len(states) == 0 {
t.Fatal("no state broadcast while the trigger was filling")
}
last := states[len(states)-1]
if last["state"] != trigArmed {
t.Fatalf("state = %v, want armed", last["state"])
}
if f, _ := last["bufferFill"].(float64); f < 0.19 || f > 0.21 {
t.Errorf("bufferFill = %v, want ~0.2 (1 s of 5 s)", last["bufferFill"])
}
if last["bufferNeedSec"] != 5.0 {
t.Errorf("bufferNeedSec = %v, want 5", last["bufferNeedSec"])
}
// An edge now is ignored: there is no 5 s of history to capture.
h.ingest("s1:sig", 1, []float64{1.5, 2.0}, []float64{-1, 1})
if h.trigger.State() != trigArmed {
t.Fatalf("state = %q, want armed: the pre-window is only 20 %% buffered", h.trigger.State())
}
// Progress is news even though the state has not moved.
rb.write([]float64{2, 3}, []float64{-1, -1})
h.triggerTick()
if states = drainStates(t, h); len(states) == 0 {
t.Fatal("no state broadcast as the pre-window filled further")
}
if f, _ := states[len(states)-1]["bufferFill"].(float64); f < 0.59 || f > 0.61 {
t.Errorf("bufferFill = %v, want ~0.6 (3 s of 5 s)", states[len(states)-1]["bufferFill"])
}
// Full: the gate opens, the fill disappears from the message and the next
// edge fires.
rb.write([]float64{4, 5.2}, []float64{-1, -1})
h.triggerTick()
states = drainStates(t, h)
if len(states) == 0 {
t.Fatal("no state broadcast when the pre-window filled")
}
if _, ok := states[len(states)-1]["bufferFill"]; ok {
t.Errorf("bufferFill still reported once the pre-window is buffered: %v", states[len(states)-1])
}
h.ingest("s1:sig", 1, []float64{5.3, 5.4}, []float64{-1, 1})
if h.trigger.State() != trigCollecting {
t.Fatalf("state = %q, want collecting once the pre-window is buffered", h.trigger.State())
}
}
// drainStates decodes every triggerState frame the hub has queued for
// broadcast. Hub.Run is what normally drains this queue, and it is not running
// in these tests.
func drainStates(t *testing.T, h *Hub) []map[string]any {
t.Helper()
var out []map[string]any
for {
select {
case msg := <-h.broadcastCh:
var m map[string]any
if err := json.Unmarshal(msg, &m); err != nil {
continue
}
if m["type"] == "triggerState" {
out = append(out, m)
}
default:
return out
}
}
}
Common/Client/go/wshub/zoom_test.go
+120
View File
@@ -0,0 +1,120 @@
package wshub
import (
"math"
"testing"
)
// A scope's envelope must not lose a spike, however narrow, and must stay in
// time order so it can be plotted as a single trace.
func TestMinMaxDecimateKeepsExtremes(t *testing.T) {
const n = 10000
ts := make([]float64, n)
vs := make([]float64, n)
for i := range ts {
ts[i] = float64(i) * 1e-6
vs[i] = math.Sin(float64(i) * 0.01)
}
// A one-sample spike in each direction: exactly what plain decimation drops.
vs[4321] = 12.5
vs[6789] = -9.75
dt, dv := minMaxDecimate(ts, vs, 200)
if len(dt) > 200 || len(dt) != len(dv) {
t.Fatalf("got %d t / %d v points, want <= 200 of each", len(dt), len(dv))
}
hiSeen, loSeen := false, false
for i := range dv {
switch dv[i] {
case 12.5:
hiSeen = true
if dt[i] != ts[4321] {
t.Errorf("spike kept at t=%v, want %v: timestamps must be the real ones", dt[i], ts[4321])
}
case -9.75:
loSeen = true
}
if i > 0 && dt[i] < dt[i-1] {
t.Fatalf("output is not time-ordered at %d: %v after %v", i, dt[i], dt[i-1])
}
}
if !hiSeen || !loSeen {
t.Errorf("envelope lost a spike (max kept=%v, min kept=%v)", hiSeen, loSeen)
}
}
func TestMinMaxDecimatePassesShortInputThrough(t *testing.T) {
ts := []float64{1, 2, 3}
vs := []float64{4, 5, 6}
dt, dv := minMaxDecimate(ts, vs, 200)
if len(dt) != 3 || dv[2] != 6 {
t.Errorf("input below the budget was altered: %v / %v", dt, dv)
}
// A flat bucket contributes one point, not two: nothing is invented.
flatT := make([]float64, 100)
flatV := make([]float64, 100)
for i := range flatT {
flatT[i] = float64(i)
}
if ft, _ := minMaxDecimate(flatT, flatV, 10); len(ft) != 5 {
t.Errorf("flat input decimated to %d points, want 5 (one per bucket)", len(ft))
}
}
func TestZoomPoints(t *testing.T) {
cases := []struct {
n int
present bool
want int
}{
{0, false, 2400}, // absent → default budget
{2400, true, 2400}, // explicit budget honoured
{0, true, 1 << 30}, // 0 → every sample in range
{-1, true, 1 << 30}, // negative → every sample in range
{5, true, 2400}, // implausibly small → default budget
}
for _, c := range cases {
if got := zoomPoints(c.n, c.present); got != c.want {
t.Errorf("zoomPoints(%d,%v) = %d, want %d", c.n, c.present, got, c.want)
}
}
}
func TestZoomSliceReturnsFullResolution(t *testing.T) {
h := NewHub()
rb := newSigRing(1000)
ts := make([]float64, 500)
vs := make([]float64, 500)
for i := range ts {
ts[i] = float64(i) * 0.001 // 1 kHz
vs[i] = float64(i)
}
rb.write(ts, vs)
h.rings["s1:sig"] = rb
// A budget larger than the range must return every sample untouched.
res := h.zoomSlice(0.100, 0.199, []string{"s1:sig"}, 1<<30)
sd, ok := res["s1:sig"]
if !ok {
t.Fatal("signal missing from zoom result")
}
if len(sd.T) != 100 {
t.Fatalf("got %d points, want 100", len(sd.T))
}
if sd.V[0] != 100 || sd.V[99] != 199 {
t.Errorf("value range = %v..%v, want 100..199", sd.V[0], sd.V[99])
}
// A small budget decimates but keeps the endpoints.
dec := h.zoomSlice(0.100, 0.199, []string{"s1:sig"}, 20)
if len(dec["s1:sig"].T) != 20 {
t.Errorf("decimated to %d points, want 20", len(dec["s1:sig"].T))
}
}
func TestZoomSliceUnknownSignal(t *testing.T) {
h := NewHub()
if res := h.zoomSlice(0, 1, []string{"nope", ""}, 100); len(res) != 0 {
t.Errorf("got %d entries, want 0", len(res))
}
}
Docs/E2E-Suite.md
+282
View File
@@ -0,0 +1,282 @@
# E2E Test Suite
The streaming-chain end-to-end suite (`Test/E2E/suite/`) validates the full data path from
MARTe2 real-time application through the UDPS wire protocol to StreamHub and client consumers.
It also covers the debug/trace path (DebugService, TCPLogger) and the direct
UDPStreamer-to-UDPStreamerClient round-trip.
## Overview
The suite is driven by a single orchestrator script:
```bash
source env.sh
./Test/E2E/suite/run_e2e.sh [flags]
```
For each scenario defined in `scenarios.py`, the orchestrator:
1. **Generates input data** (`gen_data.py`) — deterministic typed/shaped binary in MARTe2
FileReader format, plus a ground-truth dict for the validator.
2. **Generates configs** (`gen_cfg.py`) — MARTe2 app config (LinuxTimer + FileReader + IOGAM +
UDPStreamer) and StreamHub config, per scenario.
3. **Launches the server stack** — MARTe2 app + StreamHub (for chain/recorder scenarios) or
MARTe2 app alone (for direct/debug scenarios).
4. **Drives mock clients** — the Go `chain-client` (chain scenarios) or `debugclient`
(debug/tcplogger scenarios) connects, records data, and runs behavioural checks.
5. **Validates** (`validate_waveform.py`) — compares the recorded stream against the analytic
ground truth and/or the fed-reference tap file.
6. **Renders plots** (`plots.py`) — waveform, trigger, and zoom overlay PNGs per scenario.
7. **Runs unit tests + coverage** (`collect.py`) — C++ GTest, Go, and Python suites with
optional lcov C++ line coverage.
8. **Runs stress matrix** (`stress_run.py` / `stress.py`) — capacity sweeps (signal size,
count, fan-out, zoom rate) with survival/liveness/RSS/latency gates.
9. **Builds the report** (`report_build.py`) — consolidates everything into
`report_data.json` with regression tracking against the previous run, trend plots, and a
Typst PDF (`E2E_Report.typ`).
---
## Flags
| Flag | Effect |
| -------------------- | -------------------------------------------------------- |
| `--skip-build` | Skip C++ component rebuild |
| `--only <id>` | Run a single scenario by ID |
| `--pdf-only` | Just compile the Typst PDF report (no tests) |
| `--cpp-coverage` | Instrumented gcov rebuild + lcov capture (on by default) |
| `--skip-coverage` | Disable the coverage pass |
| `--skip-stress` | Skip the stress matrix |
| `--skip-datasources` | Skip `direct` scenarios |
| `--skip-recorder` | Skip `recorder` scenarios |
| `--skip-debug` | Skip `debug` and `debug_pause_resume` scenarios |
| `--skip-tcplogger` | Skip `tcplogger` scenarios |
---
## Scenario Kinds
### chain
Full streaming pipeline: MARTe2 (FileReader -> IOGAM -> UDPStreamer) -> StreamHub -> Go
`chain-client`. The client records the live binary stream and runs behavioural checks
(live, zoom, window, trigger). The validator compares the recording against the analytic
ground truth (fidelity, sine shape fit, continuity) and optionally a fed-reference tap.
### direct
MARTe2 FileReader -> UDPStreamer -> UDPStreamerClient -> FileWriter round-trip. Validates that
the written binary matches the input binary (bit-exact for each signal type).
### recorder
MARTe2 -> UDPStreamer -> StreamHub with BinaryRecorder enabled. Validates the `.bin` file
written to disk by the recorder against the original input.
### debug / debug_pause_resume
DebugService scenarios exercising FORCE, TRACE, and BREAK commands over TCP (port 8080) with
trace telemetry on UDP (port 8081). The Go `debugclient` scripts a fixed command sequence and
verifies real acknowledgements. The `debug_pause_resume` variant additionally verifies that
PAUSE halts the RT loop and RESUME restarts it via live VALUE polling.
### tcplogger
TCPLogger delivery: verifies that a triggered DebugService event produces a log line on the
TCPLogger TCP port (8082/9090).
---
## Validation Oracles
Each chain scenario specifies an `oracle` mode:
- **analytic** — ground truth is reconstructed from `gen_data.py`'s deterministic formulas
(sine, ramp, counter, time_us, time_ns). No reference file needed.
- **fed** — a second IOGAM branch in the MARTe config taps the same signals into a FileWriter
("tap file"). The validator compares recordings against this tap.
- **both** — both oracles are applied.
Per-signal checks (`validate_waveform.py`):
| Check | Description |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Fidelity** | Every received value within tolerance of some ground-truth value. Tolerance is 0 for raw integers, float epsilon for raw floats, `quant_step/2 + 1e-6*range` for quantised floats. |
| **Shape** | Sine signals (>= 8 points): least-squares fit of `a*sin(wt)+b*cos(wt)+c`. Requires correlation >= 0.99 and low normalised RMSE (relaxed by quant step). |
| **Fed reference** | When `--tap` is given, each received value must also match the tap. |
| **Continuity** | Flags inter-sample gaps > 10x median spacing. Fails when summed gap duration exceeds 5% of capture span. |
---
## Client Checks
The Go `chain-client` (`Test/E2E/suite/client/`) performs behavioural checks specified per
scenario in `client_checks`:
| Check | What it verifies |
| --------- | ----------------------------------------------------------------------------------------------- |
| `live` | WebSocket connection succeeds and live binary pushes arrive with monotonic timestamps. |
| `zoom` | A `zoom` WS command returns a valid binary response covering the requested time range. |
| `window` | A `window` WS command returns data within the specified time bounds. |
| `trigger` | A `trigger` WS command on the specified signal fires and returns data around the trigger point. |
---
## Stress Matrix
The stress module (`stress.py` + `stress_run.py`) exercises capacity by sweeping one load axis
at a time:
| Axis | What is scaled |
| ------------------ | ------------------------------------------------------------ |
| Signal size | Bytes per packet (array element count) |
| Signal count | Number of signals per source |
| Subscriber fan-out | Number of StreamHub instances subscribing to one UDPStreamer |
| WS client count | Parallel WebSocket clients on one StreamHub |
| Zoom request rate | Concurrent zoom queries per second per client |
Gates:
- **Survival** (hard) — neither server crashed or hung.
- **Liveness** (hard) — every client received monotonic, timestamped pushes.
- **Peak RSS** (soft) — MARTe and StreamHub memory stayed under case ceilings.
- **Zoom p95 latency** (soft) — round-trip zoom query latency under load.
Results are written to `stress_results.json` with axis/level for scaling-curve plots.
---
## Artifacts
| Path | Content |
| -------------------------------------------- | ------------------------------------------------------------------- |
| `Build/x86-linux/E2E/chain/results.json` | Per-scenario status (PASS/FAIL/SKIP/XFAIL/XPASS) + waveform metrics |
| `Build/x86-linux/E2E/chain/report_data.json` | Full report data including regression diffs |
| `Build/x86-linux/E2E/chain/history.jsonl` | One-line-per-run headline metrics for trend tracking |
| `Build/x86-linux/E2E/chain/trend_*.png` | Pass-rate / coverage / fidelity / memory trend plots |
| `Build/x86-linux/E2E/chain/E2E_Report.pdf` | Compiled Typst PDF report |
| `Build/x86-linux/E2E/chain/unit_tests.json` | Per-suite test results (GTest, Go, Python) |
| `Build/x86-linux/E2E/chain/coverage.json` | Per-language coverage percentages |
| `Build/x86-linux/E2E/chain/stress/` | Stress matrix results |
| `Build/x86-linux/E2E/chain/hub_<id>.log` | StreamHub stdout/stderr per scenario |
| `Build/x86-linux/E2E/chain/marte_<id>.log` | MARTe2 app stdout/stderr per scenario |
| `Build/x86-linux/E2E/chain/client_<id>.log` | Client stdout/stderr per scenario |
| `/tmp/chain_e2e/` | Scratch: input binaries, configs, recordings, metrics, plots |
---
## XFAIL / XPASS Handling
Scenarios may carry a `known_issue` marker (a human-readable string describing a documented,
not-yet-fixed chain gap). When present:
- A raw **FAIL** is reclassified as **XFAIL** (expected failure) — does not break the green
baseline.
- A raw **PASS** becomes **XPASS** (unexpectedly fixed) — surfaced as a failure to prompt
removal of the stale marker.
Overall status is PASS when there are no hard FAILs and no XPASSes.
---
## Framework Files
| File | Role |
| ---------------------- | --------------------------------------------------------------- |
| `run_e2e.sh` | Top-level orchestrator (build, run scenarios, coverage, report) |
| `scenarios.py` | Declarative scenario matrix + validation |
| `gen_data.py` | Deterministic input binary generator |
| `gen_cfg.py` | MARTe2 + StreamHub config generator |
| `validate_waveform.py` | Waveform comparison (fidelity, shape, continuity) |
| `plots.py` | Per-scenario PNG figure renderer |
| `collect.py` | Unit test runner + coverage collector (GTest, Go, Python, lcov) |
| `report_build.py` | Report data consolidator + trend plots + history |
| `stress.py` | Declarative stress case matrix |
| `stress_run.py` | Stress matrix orchestrator |
| `proc_perf.py` | Live-process CPU/RSS snapshot from `/proc` |
| `E2E_Report.typ` | Typst template for the PDF report |
| `tests_py.py` | Python framework unit tests (`python3 -m unittest tests_py`) |
| `client/main.go` | Go chain-client (live record + zoom/window/trigger checks) |
| `debugclient/main.go` | Go debug/tcplogger client (command scripting + verification) |
---
## Scenario Matrix
| ID | Kind | Description |
| ----------------------------- | ------------------ | ---------------------------------------------------------------------------------------- |
| `s01_scalar_uint32` | chain | Single uint32 scalar counter, Strict unicast (type fidelity) |
| `s02_array_float32_fullarray` | chain | 100-elem float32 array, FullArray time mode, uint64 ns time array |
| `s03_quant_uint16` | chain | float32 scalar quantised to uint16 over [-5,5], Strict unicast |
| `s04_int8_scalar` | chain | int8 scalar counter, type fidelity |
| `s05_uint8_scalar` | chain | uint8 scalar counter, type fidelity |
| `s06_int16_scalar` | chain | int16 scalar ramp, type fidelity |
| `s07_uint16_scalar` | chain | uint16 scalar ramp, type fidelity |
| `s08_int32_scalar` | chain | int32 scalar counter, type fidelity |
| `s09_int64_scalar` | chain | int64 scalar counter, type fidelity |
| `s10_uint64_scalar` | chain | uint64 scalar counter, type fidelity |
| `s11_float64_scalar` | chain | float64 scalar sine 5 Hz (double-precision path) |
| `s12_f32_arr8` | chain | float32 8-elem array sine 5 Hz |
| `s13_f32_arr32` | chain | float32 32-elem array sine 10 Hz |
| `s14_f64_arr64` | chain | float64 64-elem array ramp |
| `s15_i16_arr16` | chain | int16 16-elem array counter |
| `s16_f32_arr256` | chain | float32 256-elem array sine 5 Hz (large frame) |
| `s17_lastsample` | chain | float32 8-elem LastSample, uint64 ns scalar anchor |
| `s18_firstsample` | chain | float32 8-elem FirstSample, uint32 us scalar anchor |
| `s19_fullarray_f64` | chain | float64 50-elem FullArray sine 5 Hz, uint64 ns time |
| `s20_quant_uint8` | chain | float32 scalar quant uint8 [-1,1] sine 5 Hz |
| `s21_quant_int8` | chain | float32 scalar quant int8 [-10,10] sine 5 Hz |
| `s22_quant_int16` | chain | float32 scalar quant int16 [-100,100] ramp |
| `s23_quant_f64_arr` | chain | float64 16-elem quant uint16 [-2,2] sine 5 Hz |
| `s24_accumulate` | chain | float32 scalar sine 5 Hz, Accumulate @50 Hz refresh |
| `s25_decimate4` | chain | float32 scalar sine 5 Hz, Decimate ratio 4 |
| `s26_decimate10_arr` | chain | float32 8-elem counter, Decimate ratio 10 |
| `s27_frag_f64_128` | chain | float64 128-elem ramp, MaxPayload 512 (fragmented) |
| `s28_frag_f32_100` | chain | float32 100-elem sine 5 Hz, MaxPayload 256 (fragmented) |
| `s29_mcast_scalar` | chain | multicast float32 scalar sine 5 Hz |
| `s30_mcast_arr_fullarray` | chain | multicast float32 32-elem FullArray sine 5 Hz |
| `s31_two_src` | chain | two unicast sources: float32 sine + uint32 counter |
| `s32_three_src` | chain | three unicast sources: int16 ramp / float64 sine / uint8 counter |
| `s33_dec_arr_quant` | chain | Decimate 2 + 16-elem quant uint16 sine 5 Hz |
| `s34_acc_fullarray` | chain | Accumulate @100 Hz: accumulated scalar + 32-elem FullArray sine passenger |
| `s35_mcast_decimate` | chain | multicast + Decimate ratio 5, float32 scalar sine 5 Hz |
| `s36_big_frag_dec` | chain | float64 64-elem ramp, MaxPayload 256 + Decimate 4 |
| `s37_trig_ramp_i32` | chain | trigger on int32 ramp scalar |
| `s38_trig_f64_sine` | chain | trigger on float64 sine 5 Hz scalar |
| `s39_uint8_arr32` | chain | uint8 32-elem array counter (wrap fidelity) |
| `s40_int8_arr16` | chain | int8 16-elem array counter (wrap fidelity) |
| `s41_f32_unit` | chain | float32 scalar ramp with Unit=V |
| `s42_f64_counter` | chain | float64 scalar counter (large integer values) |
| `s43_fullarray_quant` | chain | float32 16-elem FullArray quant uint16 sine 5 Hz |
| `s44_window_check` | chain | float32 sine 5 Hz scalar, window time-range check |
| `s45_decimate_multisig` | chain | Decimate ratio 2 over a 2-signal source |
| `s46_accumulate_arr` | chain | Accumulate @200 Hz: accumulated scalar sine + 16-elem array passenger |
| `s47_mcast_multisrc` | chain | multicast, two sources (scalar each) |
| `s48_f64_arr_big_payload` | chain | float64 100-elem ramp, MaxPayload 65490 (single frame) |
| `s49_mixed_quant_raw` | chain | one source: quant uint8 sine + raw float32 sine |
| `s50_trig_quant` | chain | trigger on quantised uint16 sine 10 Hz |
| `s51_8x1msps_100hz` | chain | 8x float32 10k-elem arrays @1 MSps, FirstSample, 100 Hz packets (~32 MB/s) |
| `s52_direct_unicast` | direct | Direct UDPStreamer->UDPStreamerClient round-trip, unicast |
| `s53_direct_multicast` | direct | Direct UDPStreamer->UDPStreamerClient round-trip, multicast |
| `s54_recorder` | recorder | StreamHub BinaryRecorder disk-output round-trip |
| `s55_debug_force_trace_break` | debug | DebugService FORCE/TRACE/BREAK over real TCP 8080 + UDP 8081 |
| `s56_tcplogger_delivery` | tcplogger | TCPLogger delivers a log line for a triggered DebugService event |
| `s57_debug_pause_resume` | debug_pause_resume | DebugService PAUSE/RESUME halts and resumes the RT loop, verified via live VALUE polling |
---
## Coverage Goals
The chain scenario matrix is a curated covering set: every configurable UDPStreamer option
value appears in at least one scenario:
- **All 10 MARTe2 types**: int8, uint8, int16, uint16, int32, uint32, int64, uint64, float32, float64
- **Scalar and array shapes**: elements 1, 8, 16, 32, 50, 64, 100, 128, 256, 1000, 10000
- **All four TimeModes**: PacketTime, FullArray, FirstSample, LastSample
- **All five QuantizedTypes**: none, uint8, int8, uint16, int16
- **All three PublishingModes**: Strict, Accumulate, Decimate
- **Both network modes**: unicast and multicast
- **Fragmentation**: small MaxPayloadSize forcing multi-fragment datagrams
- **Multi-source**: 1, 2, and 3 independent UDPStreamer feeds into one StreamHub
- **High-risk interactions**: decimate+quant+array, accumulate+fullarray, multicast+decimate,
fragmentation+decimate, mixed quant+raw signals
Docs/StreamHub-API.md
+177 -9
View File
@@ -46,8 +46,61 @@ Reply (unicast): `{"type":"pong"}`.
```json
{"type":"saveSources"}
```
Persists the current dynamically-added source list to the hub's `SourcesFile`
(JSON array of `{label,addr,multicastGroup,dataPort}`); it is reloaded at startup.
Writes the hub's `SourcesFile`: the current dynamically-added source list **and**
the calibration table, as one flat JSON array (see [§4](#4-config-file-format)).
The hub replies with [`configSaved`](#configsaved). Despite the name, this
command persists the whole config, not just the sources.
### `setCalibration`
```json
{"type":"setCalibration","source":"wave","signal":"Adc","scale":0.00030518,"offset":-1.25,"unit":"V"}
```
Records an affine calibration `value = raw × scale + offset` for one signal,
keyed by the source's **label** (not its runtime id) and the **base** signal
name — one entry covers every element of an array signal.
| Field | Type | Default | Validation |
|---|---|---|---|
| `source` | string | — | non-empty after trimming |
| `signal` | string | — | non-empty after trimming; any trailing `[i]` is stripped |
| `scale` | number | `1` | finite and non-zero |
| `offset` | number | `0` | finite |
| `unit` | string | `""` | trimmed, truncated to 16 UTF-8 bytes (a multi-byte character such as `°C` consumes more than one byte; truncation never splits a character); empty = use the streamer's own unit |
Calibration is **metadata only**: the hub stores and redistributes it but never
applies it. Ring buffers, recorded history, the `zoom` reply, both binary frames
and the trigger comparator all stay in raw units — a client that ignores
calibration behaves exactly as before.
An entry that reduces to the identity (`scale = 1`, `offset = 0`, `unit = ""`) is
**deleted** rather than stored, so a reset leaves no residue in the config file.
On acceptance the hub broadcasts [`calibration`](#calibration) to every client. A
rejected entry produces **no** broadcast, so the offending client reverts to the
last value it was told.
### `reloadConfig`
```json
{"type":"reloadConfig"}
```
Re-reads `SourcesFile` and then:
- **replaces** the calibration table wholesale with the file's contents;
- **adds** any source in the file that is not already active;
- **never** removes, restarts or reconnects a live source.
The asymmetry is deliberate: calibration is cheap to reapply, whereas a source is
a live UDP session that must not be interrupted. An unsaved source the user added
keeps streaming.
The hub replies with [`configReloaded`](#configreloaded), followed on success by a
`calibration` broadcast. Whether a `sources` broadcast follows depends on the
hub implementation: the Go hub emits `sources` only when the file adds at least
one new source (each `sm.Add()` call triggers it individually), while the C++
hub always emits `sources` unconditionally after a successful reload. Clients
must therefore tolerate an unsolicited `sources` frame after any reload.
### `getSources` / `getConfig` / `getStats`
@@ -67,8 +120,11 @@ Force a broadcast of the corresponding event.
- `signal` — full key `src:sig`, or `src:sig[i]` to trigger on element *i* of a
multi-element PACKET signal.
- `edge` — `"rising"`, `"falling"` or `"both"`.
- `windowSec` — total capture window (clamped to 1e-4 … 10 s).
`preSec = windowSec * prePercent / 100`, `postSec = windowSec − preSec`.
- `windowSec` — total capture window. `preSec = windowSec * prePercent / 100`,
`postSec = windowSec − preSec`. Clamped to 1e-4 … 600 s by the Go hub and to
1e-4 … 60 s by the C++ one: the Go rings store min/max pairs once a window
outgrows their memory budget, so a long window costs resolution, while the C++
rings are fixed-capacity and would return the window truncated instead.
- `mode` — `"normal"` (auto-rearm ~200 ms after capture) or `"single"`
(stays TRIGGERED until `rearm`).
@@ -122,6 +178,33 @@ Every transition is broadcast as a `triggerState` event.
Request the hub to send a `historyInfo` event (unicast). Also sent automatically
on client connect.
### `setHistoryBudget` (Go hub only)
```json
{"type":"setHistoryBudget","maxMPtsPerSignal":16.0}
```
Sets the per-signal archive budget in millions of stored points, the runtime
equivalent of `-history-max-mpts`. `0` restores the 16 MPts default; the hub
clamps to its own ceiling. Every archive file is re-created at the new size —
**the archived samples are lost**, because a file's capacity and min/max bucket
width are fixed at creation. Broadcasts `historyInfo` rather than answering the
requester alone: every client's view of what history exists has been invalidated.
### `setWindow` (Go hub only)
```json
{"type":"setWindow","seconds":60}
```
Reports how far back this client is plotting. The hub sizes its in-memory
buffers for the **widest** window any connected client has reported (10 s if
none has), bucketing each ring as min/max pairs when the window is too long to
hold verbatim — see *In-memory buffer policy* in
[StreamHub-Developer.md](StreamHub-Developer.md). While a trigger is armed the
trigger's own window wins. No reply; send it on connect and whenever the
timescale changes. A window the hub is not told about is a window whose start
may already have rolled out of the ring, leaving a `zoom` over it nothing to
answer with.
### `setMaxPoints`
```json
@@ -176,6 +259,20 @@ Sent at `StatsRate` Hz (default 1 Hz):
`state` ∈ `idle | armed | collecting | triggered`; `trigTime` present once a
trigger has fired.
The Go hub adds `bufferFill` (0…1) and `bufferNeedSec` while `state` is `armed`
**and** its buffers do not yet reach back far enough to deliver a whole window.
Edges are ignored until they do, so that no capture arrives with a front that
was never recorded; the fields are absent once the requirement is met.
`bufferNeedSec` is how far back the hub must reach *now*, which is less than the
window by however much its buffers will fill in on their own while the
post-trigger window is collected: the pre-trigger span while they keep up with
the stream, and up to the whole `windowSec` when they do not (a full ring
re-bucketing for a longer window fills slower than real time, so the front of a
capture recedes while it is being collected).
The event is re-broadcast as the fraction grows, so a client can show the
progress instead of an armed trigger that appears to be ignoring the signal.
`forceTrigger` fires regardless.
### `zoom` (reply)
```json
@@ -190,18 +287,26 @@ trigger has fired.
Sent on client connect (if history is enabled) and on `historyInfo` command:
```json
{"type":"historyInfo","enabled":true,"durationHours":1.0,"decimation":10,
{"type":"historyInfo","enabled":true,"windowSec":600.0,"decimation":10,
"maxMPtsPerSignal":16.777216,
"signals":{
"scalar:Sine1":{"t0":1765360000.0,"t1":1765370000.0,"count":360000,"capacity":360000},
"scalar:Sine2":{"t0":1765360000.0,"t1":1765370000.0,"count":360000,"capacity":360000}}}
"scalar:Sine1":{"t0":1765360000.0,"t1":1765370000.0,"count":360000,"capacity":360000,"bucket":1},
"scalar:Sine2":{"t0":1765360000.0,"t1":1765370000.0,"count":360000,"capacity":360000,"bucket":1}}}
```
- `enabled` — `true` if the `+History` config block is present and valid.
- `durationHours` — configured history duration.
- `windowSec` — the timespan the files are sized to hold, i.e. the live or
trigger window the clients are displaying (Go hub). The C++ StreamHub instead
keeps a fixed retention period and reports it as `durationHours`.
- `decimation` — samples-to-disk decimation factor (1 = every sample).
- `maxMPtsPerSignal` — current per-signal budget, in millions of stored points
(Go hub only; see `setHistoryBudget`).
- `signals` — per-signal metadata keyed by `"sourceId:signalName"`:
- `t0`/`t1` — oldest/newest timestamp stored on disk (Unix seconds).
- `count` — number of valid entries currently in the circular file.
- `capacity` — total capacity of the circular file.
- `bucket` — source samples per stored min/max pair (Go hub only); `1` means
the signal is archived verbatim, higher means it is stored as an envelope
because it is too fast to fit the budget at full resolution.
### `historyZoom` (reply)
@@ -219,6 +324,41 @@ If history is not enabled: `{"type":"historyZoom","error":"history not enabled"}
{"type":"maxPointsUpdated","maxPoints":50000}
```
### `calibration`
```json
{"type":"calibration","cal":[
{"source":"wave","signal":"Adc","scale":0.00030518,"offset":-1.25,"unit":"V"}
]}
```
The complete calibration table. Broadcast when a client connects (as an empty
array when nothing is calibrated), after every accepted `setCalibration`, and
after a successful `reloadConfig`. It is a separate frame rather than a field on
`sources` because `sources` is serialised into a fixed 4 KiB buffer.
### `configSaved`
```json
{"type":"configSaved","ok":true,"path":"/etc/streamhub/sources.json"}
{"type":"configSaved","ok":false,"path":"","error":"no sources file configured"}
```
Broadcast in reply to `saveSources`. `path` is always present (empty when the hub
has no config file configured); `error` only when `ok` is false. The exact error
text is not part of the protocol contract and differs between hubs (the Go hub
uses `"no sources-file configured"`, the C++ hub `"no sources file configured"`).
### `configReloaded`
```json
{"type":"configReloaded","ok":true,"path":"/etc/streamhub/sources.json"}
{"type":"configReloaded","ok":false,"path":"/etc/streamhub/sources.json","error":"cannot read sources file"}
```
Broadcast in reply to `reloadConfig`; same shape as `configSaved`. On success
it is followed by a `calibration` broadcast. Whether a `sources` broadcast also
follows is hub-specific: the Go hub sends it only if the reload added at least
one new source; the C++ hub sends it unconditionally. Clients must tolerate an
unsolicited `sources` frame after any reload.
---
## 3. Binary frames (hub → client)
@@ -270,7 +410,33 @@ per signal:
---
## 4. Limits
## 4. Config file format
`SourcesFile` (C++ `SourcesFile` config key, Go `-sources-file` flag) is a flat
JSON array of flat objects. A block containing `addr` is a source; a block
containing `signal` is a calibration entry; anything else is skipped with a
warning.
```json
[
{"label": "wave", "addr": "127.0.0.1:44500"},
{"label": "mc", "addr": "127.0.0.1:44501", "multicastGroup": "239.0.0.1", "dataPort": 44502},
{"source": "wave", "signal": "Adc", "scale": 0.00030518, "offset": -1.25, "unit": "V"}
]
```
**Every object must stay flat.** The C++ `StreamHub::LoadSourcesFile` parser
takes each `{` up to the next `}` as one object, so a nested object anywhere in
the file would truncate the parse at the inner brace. A nested
`"calibration": {…}` inside a source entry is therefore not an option, and this
is why calibration entries are siblings of sources rather than children.
Files written by hub versions predating calibration load unchanged, and a file
written by either hub loads in the other.
---
## 5. Limits
| Limit | Value |
|-------|-------|
@@ -278,3 +444,5 @@ per signal:
| UDPS source sessions | 32 |
| Max received WS payload | 64 KiB |
| Max sent WS payload | 4 MiB |
| Calibration entries | 256 (C++ hub, `kMaxCalibration`); unbounded (Go hub) |
| Calibration unit override | 16 UTF-8 bytes |
Docs/StreamHub-Developer.md
+263 -24
View File
@@ -60,8 +60,20 @@ Each session calibrates per time-source:
`packetT = pktCalibOffset + hrt/hrtFreq`.
- Each referenced time signal gets its own offset on first value;
`timerToSec = 1e-9` for `uint64` time signals, `1e-6` otherwise.
- Re-anchoring on reconnect, CONFIG change, or if computed time drifts > 2 s
from wall clock (source restart / remote-vs-local HRT frequency drift).
- The time-signal offset is **snapped** only on a genuine discontinuity in the
source: reconnect, CONFIG change, or the source clock jumping backward (a
looping/rewinding producer such as a rewinding `FileReader`).
- Plain *drift* — a source free-running on its own clock, or remote-vs-local HRT
frequency error — is **slewed**, not snapped. Past a 2 s threshold the offset
is nudged toward wall clock by at most 10 % of the packet's own duration.
Snapping instead would shift the whole published timeline in one step and so
tear a hole of exactly the drift into a stream that is in fact continuous;
a source drifting past the threshold repeatedly used to produce a train of
2 s holes. Drift is the honest reading, and the trade-off `TimeArrayGAM`'s
`Anchor = Continuous` explicitly asks for: a producer that cannot sustain its
nominal sample rate will fall progressively behind wall clock, and the hub
reports that rather than hiding it. The Go hub anchors once and never
re-anchors, so it never had the hole.
Per `timeMode`:
@@ -94,16 +106,50 @@ Hub-side, web-client semantics (`setTrigger` fields in
[StreamHub-API.md](StreamHub-API.md)):
```
IDLE --arm--> ARMED --edge crossing--> COLLECTING --wallNow ≥ trigTime+postSec+0.15s--> TRIGGERED
IDLE --arm--> ARMED --edge crossing--> COLLECTING --every source past trigTime+postSec+0.15s--> TRIGGERED
TRIGGERED --rearm (single) / auto ~200ms (normal, unless stopped)--> ARMED
any --disarm--> IDLE
```
`UDPSourceSession` calls `TriggerEngine::CheckSample` for every decoded sample
of the configured signal (signal index cached per config epoch). On
finalisation the push loop reads `[trigTime−preSec, trigTime+postSec]` from all
rings, LTTB-caps to 20 000 pts/signal and broadcasts a binary **version 2**
capture frame; every FSM transition broadcasts a `triggerState` event.
of the configured signal (signal index cached per config epoch). Each source is
read `[trigTime−preSec, trigTime+postSec]`, LTTB-capped to 20 000 pts/signal and
appended to a binary **version 2** capture frame; every FSM transition
broadcasts a `triggerState` event.
Once fired, that event carries `trigTime` **and** the window latched at fire
time (`preSec`/`postSec`). Clients draw the still-filling capture from their own
buffers on that axis long before the v2 frame arrives — for a long window at a
high rate the hub stays silent for seconds — and the trigger bar's window and
pre-% are editable, so without the latched values a client would place the
filling trace on whatever window the operator happened to be typing. Older hubs
omit both fields; clients fall back to their local config.
The COLLECTING deadline is on the **data's** clock, via
`UDPSourceSession::ProducerNewestTime()` — `trigTime` comes from sample
timestamps, and a source free-running on its own clock sits seconds away from
`clock_gettime()`, so a wall-clock deadline chops exactly that offset off every
capture's tail. Only signals actually timestamped from a time signal count
toward that reading: PACKET-timed ones (including the time array itself) are
stamped on arrival and would just report "now".
Sources are harvested independently — `BeginTriggerCapture`,
`HarvestTriggerCapture` per source as *it* becomes ready, `FinishTriggerCapture`
once all are in — with the frame accumulating in `capBuf_` across push ticks.
Waiting for the slowest source before reading any of them lets the leaders'
rings roll past the pre-trigger region first, losing the head of their traces. A
2 s wall-clock watchdog bounds the wait for a source that stopped advancing: it
is harvested short, with a warning naming the source and how far it got.
`setTrigger` also records the requested window, and each stats tick the push
loop runs `GrowRingsForTrigger()`. A ring whose measured rate
(`Count() / TimeSpan()`, since UDPS sources usually advertise
`samplingRate = 0`) cannot hold `window + 0.5 s` is grown in place to
`rate × (window + 0.5) × 1.2` points, clamped to `RingMaxMB` per signal.
`SignalRingBuffer::Grow()` copies oldest→newest and leaves `count` /
`totalWritten` untouched so the per-client push cursors survive the resize.
Rings never shrink; a hub left with a 5 s window on a 5 MSps source will sit at
the ceiling.
## 6. Configuration
@@ -113,9 +159,16 @@ MaxPoints = 20000 // legacy global cap (overridable with -maxPoints)
PushRate = 30 // Hz
MaxPushPoints = 50 // per signal per push
StatsRate = 1 // Hz
RingTemporal = 1000000 // ring capacity, temporal signals (pts)
RingTemporal = 1000000 // initial ring capacity, temporal signals (pts)
RingScalar = 100000 // ring capacity, scalar/PACKET signals (pts)
RingMaxMB = 128 // per-signal growth ceiling (MiB) for trigger windows
SourcesFile = "streamhub_sources.json" // saveSources persistence
AllowedOrigins = "http://127.0.0.1:8099,http://localhost:8099"
// comma/space-separated WebSocket Origin allowlist (max 8 × 128 chars).
// Without it the handshake only accepts an Origin whose host matches
// the request Host, so a browser serving the SPA from another port
// (run_streamhub.sh: SPA 8099, hub 8090) gets 403. Non-browser
// clients send no Origin and are unaffected.
Sources = {
Src1 = { Label = "PSU" Addr = "127.0.0.1" Port = 44500
MulticastGroup = "239.0.0.1" DataPort = 44503 } // multicast optional
@@ -145,6 +198,54 @@ Per-signal file capacity is computed at source CONFIG time:
`capacity = ceil(DurationHours × 3600 × samplingRate / Decimation)`, minimum
1000 pairs.
The Go hub (`Client/udpstreamer`) carries the same archive and the same file
format, configured with flags instead of a config node: `-history-dir`
(defaults to `<tmp>/udpstreamer-history`; empty disables),
`-history-window-sec`, `-history-decimation`, `-history-flush-sec`,
`-history-min-free-mb` (negative disables the check; 0 means the 500 MB default,
where the C++ `MinDiskFreeMB = 0` disables it) and `-history-max-mpts`, a
per-signal budget in millions of stored points, defaulting to 16 MPts (256 MB).
The budget exists because the timespan alone cannot bound the file: 600 s of a
1 MSps signal is 9.6 GB.
**The Go hub sizes its files from the window, not from a retention period.** The
archive exists to answer a zoom or a trigger capture after the in-memory rings
have rolled past it, and neither ever asks for more than the live or trigger
window — so a file holds `windowSec × rate` samples (plus 25 % headroom, since a
capture is read back a window after its first sample was written), and never
hours of them. Retaining an hour instead meant a 1 s live window was archived at
a thousandth of the resolution the same budget could have bought.
The budget is therefore spent on resolution, not on span. A signal too fast to
archive sample-for-sample within it is stored as a **min/max envelope**: `bucket`
source samples collapse to their two extremes, with `bucket` the narrowest that
makes the window fit. The `.shist` header's `decimation` field carries
`bucket × Decimation`, so a reader knows the stored resolution, and a file is
only reopened when it matches.
`Hub.retuneRings` re-sizes the files once a second alongside the rings, from the
same `activeWindowSec()`. A file's capacity and bucket are fixed at creation, so
a re-size discards what it held; two rules keep that rare. A file is only grown
when it no longer covers the window, and only shrunk when it is enveloped
(`bucket > 1`), covers more than twice the window, and a narrower bucket is
actually available — a file already at full resolution is left alone however
short the window becomes, so arming a 1 s trigger does not throw away the
seconds the capture is about to ask for. `historyInfo` is re-broadcast whenever a
re-size happens.
The budget is also settable at runtime from the web UI (the history badge in the
status bar) via the `setHistoryBudget` WS command; `historyInfo` reports it as
`maxMPtsPerSignal` and reports each signal's `bucket`. Changing it re-creates the
files, so the archived samples are lost — a file's capacity and bucket width are
fixed at creation and an existing envelope cannot be re-bucketed into a different
one.
History is on by default in the Go hub because it is what holds a trigger capture
at full resolution — see *Trigger captures* below. Signals whose producer
declares `samplingRate = 0` — every UDPS source — are not sized from a guess: the
file is opened only once the hub has measured the rate off the live stream, which
it retries once a second.
### `.shist` binary file format
Each signal gets one file: `<Directory>/<sourceId>/<signalName>.shist`.
@@ -178,13 +279,155 @@ reopened — head/count/time bounds are restored from the on-disk header.
`historyZoom` requests (see [StreamHub-API.md](StreamHub-API.md)) call
`HistoryWriter::ReadRange` which performs binary search over the circular file
using `pread` to locate the `[t0, t1]` window, then copies matching pairs.
If the result exceeds the requested `n`, LTTB decimation is applied (same
`LTTBDecimate` as in-memory zoom).
If the result exceeds the requested `n`, decimation is applied (same decimator
as in-memory zoom: `LTTBDecimate` in the C++ hub, `minMaxDecimate` in the Go
one).
Both the web SPA and ImGui client issue `historyZoom` in parallel with regular
`zoom` and merge the results: history covers the older part of the visible
window, the in-memory ring covers the recent part.
A range wider than the read budget is thinned across its whole width with a
stride, not truncated at the front: answering a 10 s query with its first
few milliseconds reads as an empty plot to a client and sends it back to its
own coarse copy of the data.
### In-memory buffer policy (Go hub)
Each temporal signal gets one ring holding a fixed **budget** of `(t, v)` pairs:
10 M points, 160 MB, settable with `-ring-mpts`. Scalar signals keep a flat
100 000-packet ring, where a megasample budget would be waste. Rings start at
250 k points and are grown to the budget on demand, so a source that is
configured but never sends costs nothing.
Like the disk archive, the budget buys **resolution, not span**. Once a second
`retuneRings` compares the measured source rate against the window being
displayed and picks each ring's min/max `bucket`:
| condition | bucket | effect |
|---|---|---|
| `Sps × window ≤ budget` | 1 | stored verbatim; the ring reaches further back than the window, which is free zoom headroom |
| `Sps × window > budget` | `⌈2 × Sps × window × 1.25 ÷ capacity⌉` | `bucket` samples collapse to their two extremes, so the whole window fits |
A bucket costs two points (its minimum and its maximum), hence the factor 2 —
and why a bucket of 2 covers no more ground than a bucket of 1.
The window is the **trigger's** while a trigger is armed: its pre-window has to
already be in the ring when the trigger fires, or the capture has nothing to
back-fill from. Otherwise it is the widest window any connected client has
reported with the `setWindow` command, defaulting to 10 s for clients that never
send one. Sizing for the live window matters as much as for a capture: a fixed
sample-count ring covers ~6 s at 1 MSps, so a zoom on a 60 s timescale used to
come back with only its tail.
Retuning is hysteretic — a bucket is held while it covers the window without
covering more than twice it. Sharing one threshold for up and down makes a rate
jittering across a bucket boundary halve and double the stored resolution every
second.
Live pushes, the disk archive and the trigger comparator all see every sample:
`ingest` hands the raw batch to each, and only the ring's own copy is reduced.
### Trigger captures (Go hub)
A trigger capture is delivered as a decimated snapshot (20 000 points), so a
zoom into it has to come from full-resolution storage. The rings are tuned to
~1.25× the trigger window, so they roll past a captured window shortly after the
capture — and the trigger rearms and starts refilling them immediately.
**In-memory double buffer.** The rings are the write half; `captureHold`
(`capturehold.go`) is the read half. As `buildTriggerCapture` lifts each signal's
window out of its ring it publishes the *undecimated* slice into the hold, and
`zoomSlice` answers from the hold rather than the ring for any range the held
window fully contains. The swap happens only once the next capture is complete —
which is also the moment the client stops displaying the previous one — so the
shot being explored is never overwritten by the acquisition running behind it. A
capture that came back empty does not swap, so it cannot blank the window on
screen.
**Waiting for the buffer.** An armed trigger ignores edges until its buffers
reach back far enough for a capture taken now to come back whole (`fillLocked`
in `trigger.go`, fed by `refreshTriggerFill` from the trigger signal's own ring
— once per tick, and again on every trigger command so that an `arm` cannot
fire on a stale measurement). Firing earlier can only produce a capture whose
front was never recorded, which is what made the first shot after a widened
window come back short.
What must hold is that the buffer spans the whole window *at harvest time* — its
newest sample is then `trigTime + post`, so anything less has lost the front of
the capture. It keeps filling while the post-window is collected, so the
shortfall it may start with is what it will make up in that time, measured
rather than assumed:
```
need = windowSec − growth × postSec (floored at the pre-trigger window)
```
`growth` is the ring's span growth in seconds per second, sampled over at least
`bufGrowthIntervalSec` and smoothed. The three regimes fall out of the one
formula:
| ring | growth | needs |
|---|---|---|
| still filling | 1 | the pre-trigger window — everything after the trigger is yet to be recorded anyway |
| full, re-bucketing for a longer window | 0…1 | in between: it drops dense old samples to take sparse new ones, so it fills slower than real time and the front of the capture recedes while the post-window elapses |
| full, settled | 0 | the whole window — which a ring tuned for that window already exceeds, so nothing actually waits |
Measured at 1 MSps, widening 10 s → 30 s with 50 % pre: growth settles at ~0.65,
so `need` converges on ~20.4 s of the 30 s and the trigger fires ~12 s after
arming with a capture that is 100 % complete. Requiring the whole window instead
would have waited 26 s for the same result.
The gate measures the trigger signal's ring, not the narrowest of all of them: a
signal that never reaches back that far would otherwise stop the trigger from
ever firing. It is disabled outright when there is no ring to measure or nothing
is needed, and `forceTrigger` overrides it. While it holds off, `triggerState`
carries `bufferFill`/`bufferNeedSec` and is re-broadcast as the fraction climbs,
so the UI shows `ARMED 42%` rather than a trigger that looks stuck.
**Back-filling a short capture.** A ring only spans the window once it has
rolled over completely at its current min/max bucket, which takes as long as the
window itself; widen the window, or arm right after setting it, and the first
captures start late and the client draws a blank front half.
`backfillCaptureHead` (`trigger.go`) therefore prepends whatever of
`[t0, ring's first sample)` the archive still holds, budgeting the read by the
share of the window being filled and trimming the overlap so the frame's
timestamps stay ascending. It needs history enabled; without it the capture is
simply short, and the hub logs by how much. The hold declines any range its own
samples do not actually cover, so a stretch neither source could supply falls
through to the archive instead of being redrawn as the same hole on every zoom
and every *fit*.
The hold declines ranges reaching outside its window: those are live zooms, and
only the rings still track the stream. Inside the window it needs no
trigger-state gating, because retuning never rewrites stored samples — a ring
that still covers the range holds the very same points. It is cleared when
`updateConfig` rebuilds the rings, since a restarted producer can replay the same
timestamps.
Budget: the hold costs one window per signal on top of the ring budget, up to a
further ~0.8 × `-ring-mpts`. Nothing is held until the first capture fires.
**On disk.** The archive covers what the hold cannot: ranges wider than the
capture window, and sessions where the hub restarted. It is circular and sized
from that same window, so it too wraps over a captured shot within a window of
delivering it. When a capture is delivered, the hub therefore copies
`[trigTime − pre, trigTime + post]` out of each `.shist` into a
`<signalName>.cap` file, laid out as a full non-wrapping `.shist`
(`capacity == count`, `head == 0`) so the same `readRange` reads it. A
`historyZoom` whose range the capture file fully contains is answered from it;
anything wider is answered from the archive. The copy is replaced by the next
trigger and by nothing else — rearming keeps it, because the client is still
showing that capture.
Budget: a capture costs one window's worth of disk per signal on top of
`-history-max-mpts`.
Protecting the window in place instead — pinning the region and refusing to
wrap onto it — does not work, and was tried: a capture held for longer than the
archive covers stops the archive dead, and the resulting hole lands exactly
where the *next* capture's pre-trigger window belongs.
## 7. Build & test
```bash
@@ -199,28 +442,24 @@ make -f Makefile.gcc test
# SignalRingBuffer (ReadSince / binary-search ReadRange / wrap),
# TriggerEngine FSM, LTTB — sources in Test/Applications/StreamHub/
./run_e2e_test.sh # full-stack E2E (see below)
cd Test/E2E/suite && ./run_e2e.sh # full-stack E2E (see below)
./run_streamhub.sh -w -g # interactive demo stack
```
### End-to-end test
`./run_e2e_test.sh` builds everything, launches the demo MARTe2 application
(`Test/Configurations/streamhub_demo.cfg`: 3 UDPStreamers — multicast scalars,
FirstSample/LastSample arrays, FullArray + uint64 ns time array) plus a
StreamHub on port 8095 (with history enabled in `/tmp/streamhub_e2e_history`),
then runs the Go WS client `Test/E2E/streamhub` which verifies:
`sources`/`config` events, ≥10 binary v1 pushes with wall-clock and strictly
monotonic time on all streams, `stats` shape, a `zoom` round-trip (reqId echo,
unicast), `historyInfo` broadcast (enabled, duration, decimation, signal count),
a `historyZoom` round-trip (reqId echo, signal data), and a complete trigger
cycle (setTrigger → arm → binary v2 capture → triggered → disarm). Logs land
in `/tmp/streamhub_e2e_{marte,hub}.log`. Exit 0 iff every check passes.
`Test/E2E/suite/run_e2e.sh` is the unified E2E suite covering the whole
streaming + debug chain (`chain`/`direct`/`recorder`/`debug`/`tcplogger`
scenario kinds, see `Test/E2E/suite/scenarios.py`), including StreamHub live
push, zoom, window and trigger checks via the Go `chain-client`. It builds
everything, runs the scenario matrix plus the stress matrix, and produces a
consolidated `report_data.json` + Typst PDF report
(`Test/E2E/suite/E2E_Report.typ`). See the script's `--help` for options.
When changing the WS protocol, update **in lockstep**: this hub, the Go hub
(`Common/Client/go/wshub`), the browser SPA (`Client/udpstreamer/static`), the
ImGui client (`Client/streamhub/Protocol.cpp`), the E2E client
(`Test/E2E/streamhub`), and [StreamHub-API.md](StreamHub-API.md).
ImGui client (`Client/streamhub/Protocol.cpp`), the E2E `chain-client`
(`Test/E2E/suite/client`), and [StreamHub-API.md](StreamHub-API.md).
## 8. Gotchas
Docs/UDPS-C-Client.md
+287
View File
@@ -0,0 +1,287 @@
# UDPS C Client Library
`Common/Client/c/` is a standalone receiver for the UDPS streaming protocol: it connects to a
`UDPStreamer` DataSource (or any other UDPS producer, such as `DebugService`), decodes the
signals, and hands them to your callbacks as plain `double`s.
It has **no MARTe2 dependency** and no third-party dependencies at all — just libc and BSD
sockets. Two files, `udps_client.h` and `udps_client.c`, drop into any C or C++ project.
The wire format itself is specified in [Protocol.md](Protocol.md); this document covers the
library. The producer side is documented in [UDPStreamer.md](UDPStreamer.md).
---
## Build
```bash
cd Common/Client/c
make # libudpsclient.a + the udps_dump example
make cxxcheck # verifies the header compiles and links from C++
make clean
```
Or just add the two files to your own build:
```bash
cc -std=c99 -O2 -c udps_client.c
```
Requirements: a C99 compiler and POSIX sockets. On glibc older than 2.17 add `-lrt`
(`clock_gettime` lived in librt back then). The header is wrapped in `extern "C"`, so C++
callers include it directly.
---
## Quick start
```c
#include "udps_client.h"
#include <stdio.h>
static void on_data(const udps_frame_t *f, void *user) {
(void)user;
/* Signals are in CONFIG order; values are already physical doubles. */
printf("#%u %s = %g\n", f->counter, f->signals[0].name, f->values[0].values[0]);
}
int main(void) {
udps_client_config_t cfg;
udps_client_t *cli;
udps_client_config_init(&cfg);
cfg.server_addr = "127.0.0.1";
cfg.server_port = 44500;
cli = udps_client_create(&cfg);
udps_client_set_callbacks(cli, NULL, on_data, NULL, NULL);
for (;;) {
udps_client_poll(cli, 200); /* connects, receives, decodes, reconnects */
}
udps_client_destroy(cli);
return 0;
}
```
`udps_client_poll()` is the only function that does work. It never spawns a thread, and every
callback runs inside it — so if your program already has an event loop, call it from there and
you need no synchronisation at all. A client must be used from one thread at a time.
---
## Connection model
The library implements both transports of the protocol and picks one from the configuration:
| | Unicast (`multicast_group == NULL`) | Multicast (`multicast_group` set) |
|---|---|---|
| CONNECT | UDP datagram to `server_addr:server_port` | over a TCP connection to `server_addr:server_port` |
| CONFIG | UDP, back to the client's ephemeral port | over the same TCP connection |
| DATA | UDP, same ephemeral port | UDP multicast on `data_port` |
| Keepalive | ACK every `keepalive_interval_s` | not needed (the TCP session is the liveness signal) |
In multicast mode the group is joined *before* CONNECT is sent, because the server multicasts
CONFIG as soon as it sees a client — a group joined afterwards would miss it.
The client reconnects on its own: if nothing arrives for `silence_timeout_s` it sends
DISCONNECT, closes the sockets, waits `reconnect_delay_s`, and starts over. `udps_client_poll()`
returns `-1` when that happens, which is informational, not fatal.
---
## Configuration
Always start from `udps_client_config_init()` — it fills in the defaults below — then override
what you need. Strings are copied into the client, so they need not outlive `udps_client_create()`.
| Field | Default | Meaning |
|---|---|---|
| `server_addr` | — (required) | Server IPv4 address; a hostname is resolved if it is not a dotted quad. |
| `server_port` | — (required) | Server UDP port, or the TCP control port in multicast mode. |
| `multicast_group` | `NULL` | IPv4 group to join. Non-`NULL` selects the multicast transport. |
| `interface_addr` | `NULL` | Local IPv4 **address** (not a name, e.g. `"127.0.0.1"`) of the interface to join on. Defaults to the default route, which silently receives nothing if the server sends elsewhere. |
| `data_port` | `server_port + 1` | Multicast data port. Must match the producer's `DataPort`. |
| `silence_timeout_s` | `1.0` | Reconnect after this long without data. `0` disables the check — use it for streams that are idle by design. |
| `reconnect_delay_s` | `2.0` | Wait between reconnection attempts. |
| `keepalive_interval_s` | `15.0` | Unicast ACK period. The server evicts silent clients after its `ClientTimeout` (30 s by default). `0` disables. |
| `recv_buffer_bytes` | 4 MiB | `SO_RCVBUF`. The Linux default (~208 KiB) is overrun by fast producers and the kernel drops datagrams silently. |
| `max_packet_bytes` | 1 MiB | Ceiling on one reassembled payload; a reassembly buffer of this size is allocated per in-flight update (4 at most). |
---
## API
### Lifecycle
```c
void udps_client_config_init(udps_client_config_t *cfg);
udps_client_t *udps_client_create(const udps_client_config_t *cfg);
void udps_client_set_callbacks(udps_client_t *c, udps_config_cb, udps_data_cb,
udps_event_cb, void *user);
int udps_client_poll(udps_client_t *c, int timeout_ms);
void udps_client_destroy(udps_client_t *c);
```
`udps_client_create()` returns `NULL` on a bad address or an invalid configuration; no socket is
opened until the first poll. `udps_client_poll()` returns the number of packets processed, `0` on
timeout, or `-1` if the session broke — pass a negative `timeout_ms` to block. `destroy` sends
DISCONNECT before closing.
### Callbacks
```c
void on_config(const udps_signal_t *signals, uint32_t n, uint8_t publish_mode, void *user);
void on_data (const udps_frame_t *frame, void *user);
void on_event (udps_event_t event, const char *detail, void *user);
```
`on_config` fires on every CONFIG packet: the signal set can change at runtime, so treat it as a
reset of everything you cached. `on_event` reports `UDPS_EVENT_CONNECTED`,
`UDPS_EVENT_DISCONNECTED` and `UDPS_EVENT_ERROR` with a human-readable `detail`.
> **The frame and everything it points at are owned by the client and are valid only until
> `on_data` returns.** The decode buffers are reused by the next packet. Copy what you keep.
### Inspection
```c
int udps_client_is_connected(const udps_client_t *c);
const udps_signal_t *udps_client_signals(const udps_client_t *c, uint32_t *n);
uint8_t udps_client_publish_mode(const udps_client_t *c);
void udps_client_stats(const udps_client_t *c, udps_stats_t *out);
const char *udps_client_last_error(const udps_client_t *c);
```
### Helpers
```c
uint32_t udps_signal_num_elements(const udps_signal_t *s);
const char *udps_type_name(uint8_t type_code);
int udps_parse_header(const void *buf, size_t len, udps_header_t *out);
int udps_parse_config(const void *payload, size_t len, udps_signal_t *sigs,
uint32_t max_signals, uint32_t *n, uint8_t *publish_mode);
double udps_frame_value(const udps_frame_t *f, uint32_t sig, uint32_t sample, uint32_t elem);
double udps_frame_element_time(const udps_frame_t *f, uint32_t sig, uint32_t elem);
```
`udps_parse_header` and `udps_parse_config` are stateless and socket-free, so captured or
replayed traffic can be decoded without a client.
---
## Reading a frame
```c
typedef struct {
uint32_t counter; /* gaps in this sequence are lost datagrams */
uint64_t hrt; /* producer's high-resolution timer at send */
double recv_time; /* CLOCK_REALTIME seconds at arrival */
uint8_t publish_mode;
uint32_t num_samples; /* batched RT cycles; 1 unless Accumulate */
uint32_t num_signals;
const udps_signal_t *signals; /* CONFIG order */
const udps_signal_values_t *values; /* same order */
} udps_frame_t;
```
`values[i].values` is an array of `values[i].count` physical `double`s. Quantised signals are
already expanded back onto `[range_min, range_max]`, and integer types are widened — the decoded
form does not depend on the wire type, so a consumer need not branch on `type_code` at all.
**Element count.** `count` is the signal's element count (`num_rows × num_cols`), *except* for a
scalar signal in Accumulate mode, where the producer batches several RT cycles into one packet
and `count == num_samples` — one value per cycle. Arrays are not batched: they appear once and
apply to the whole packet. `udps_frame_value(f, sig, sample, elem)` applies that rule for you.
**Timestamps.** The protocol does not put a timestamp on every element; how to date them depends
on the signal's `time_mode` (see [Protocol.md](Protocol.md#time-mode-codes)):
| `time_mode` | Where the time comes from |
|---|---|
| `UDPS_TIME_PACKET` | No per-element time. Use `recv_time`. |
| `UDPS_TIME_FULL_ARRAY` | The signal at `time_signal_idx` holds one timestamp per element — read it like any other signal. |
| `UDPS_TIME_FIRST_SAMPLE` / `UDPS_TIME_LAST_SAMPLE` | The signal at `time_signal_idx` is a scalar stamping element 0 (or N−1); the rest follow at `1/sampling_rate`. |
The time signal is a raw producer-side counter (µs, or ns when it is a `uint64`), not wall clock,
so plotting it against real time needs a one-off calibration against `recv_time` — that is what
the Go hub does. `udps_frame_element_time()` skips all that and returns an arrival-anchored
estimate: good enough for a quick plot, but when a time signal exists, it is the accurate source.
---
## Diagnosing loss
```c
udps_stats_t s;
udps_client_stats(cli, &s);
```
| Counter | Meaning |
|---|---|
| `packets_received`, `bytes_received` | Accepted datagrams and TCP frames. |
| `frames_delivered` | DATA packets decoded and passed to `on_data`. |
| `config_updates` | CONFIG packets applied. |
| `counter_gaps` | Missing packet counters — datagrams lost on the wire or in the kernel. |
| `fragments_dropped` | Duplicate, stale or unplaceable fragments; a non-zero value with `counter_gaps` means fragmented updates are arriving incomplete. |
| `reconnects` | Sessions re-established after a silence timeout. |
Persistent loss on a fast stream is almost always the receive buffer: raise `recv_buffer_bytes`
(and `net.core.rmem_max`, which caps it). A fragmented producer is more fragile than one sending
whole cycles, because losing any fragment discards the whole update — if you control the
producer, sizing `MaxPayloadSize` above one cycle removes that failure mode entirely.
---
## Example program
`example/udps_dump.c` connects, prints the signal table on CONFIG, then a throttled summary of
each frame, and a receive-statistics report on Ctrl-C.
```bash
# unicast
./udps_dump --host 127.0.0.1 --port 44500
# multicast
./udps_dump --host 127.0.0.1 --port 44500 --multicast 239.0.0.1 --iface 127.0.0.1
# quieter, and stop after 500 frames
./udps_dump --host 127.0.0.1 --port 44500 --interval 5 --frames 500
```
| Flag | Meaning |
|---|---|
| `--host ADDR` | Server address (default `127.0.0.1`). |
| `--port N` | Server UDP port, or TCP control port in multicast mode (default 44500). |
| `--multicast GROUP` | Join `GROUP` for data instead of using unicast. |
| `--iface ADDR` | Local interface address for the multicast join. |
| `--data-port N` | Multicast data port (default `--port + 1`). |
| `--silence SEC` | Reconnect after `SEC` without data; `0` disables. |
| `--interval SEC` | Seconds between printouts (default 1). |
| `--frames N` | Exit after `N` frames. |
Against the repository's own producer (`./run_udp_producer.sh -n 2`, two 1 Msps channels of
1000-element `float32` arrays at 1 kHz) the output looks like:
```
CONFIG: 3 signal(s), publish mode strict
# name type shape unit rate[Hz] time-mode
0 TimeArray uint64 1x1000 ns 0 packet
1 Ch1 float32 1x1000 V 0 full-array
2 Ch2 float32 1x1000 V 0 full-array
frame #2289897 t=1787409851.723466 samples=1
Ch1 n=1000 first=-6.9e-10 last=-0.00628 min=-1 max=1 V
Ch2 n=1000 first=0.5 last=0.49975 min=-0.5 max=0.5 V
```
---
## Limitations
- IPv4 only, matching the protocol and the producer.
- One thread per client; there is no internal locking.
- The receive path allocates only when a CONFIG grows the signal set or a frame grows the decode
arena, so a steady stream is allocation-free — but this is not a hard real-time component.
- DATA arriving before the first CONFIG is dropped: without descriptors it cannot be decoded.
This is normal for a few packets after joining a multicast group.
Docs/UDPStreamer.md
+197 -31
View File
@@ -8,7 +8,9 @@ thread.
## Key Features
- **Zero-copy RT path** — `Synchronise()` only locks, copies signal memory, and posts a semaphore.
- **Single-client model** — one client at a time; a new CONNECT replaces the previous session.
- **Unicast and multicast** — unicast (default): single client at a time, new CONNECT replaces
the previous session. Multicast: multiple clients receive data simultaneously by joining
a multicast group; control traffic uses a TCP listener.
- **Packet fragmentation** — large payloads are split into ≤ `MaxPayloadSize`-byte datagrams,
each with a header carrying fragment index and total count so the client can reassemble them.
- **Signal quantization** — `float32`/`float64` signals can be linearly quantized to
@@ -16,6 +18,8 @@ thread.
- **Temporal arrays** — signals with `NumberOfElements > 1` can carry per-sample time
metadata via `TimeMode` and `TimeSignal`, enabling high-frequency burst transmission
(e.g. 1 000 samples per RT cycle at 1 MSps).
- **Publishing modes** — `Strict` (one packet per RT cycle), `Accumulate` (batch N snapshots
then flush on size or time limit), `Decimate` (send every Nth cycle).
---
@@ -26,10 +30,23 @@ thread.
Class = UDPStreamer
// Network
Port = 44500 // UDP port the server listens on (default: 44500)
Port = 44500 // UDP port (unicast) or TCP control port (multicast)
MaxPayloadSize = 1400 // Maximum bytes per UDP datagram (default: 1400)
// Must be > 17 (header size). Tune for MTU.
// Multicast (optional — omit for unicast mode)
MulticastGroup = "239.0.0.1" // IPv4 multicast address (224.0.0.0/4)
Interface = "192.168.1.10" // Local IPv4 address of the outgoing interface
// (mandatory when MulticastGroup is set)
DataPort = 44501 // UDP port for multicast DATA (default: Port+1)
// Publishing mode (optional)
PublishingMode = "Strict" // Strict | Accumulate | Decimate
// For Accumulate mode:
MinRefreshRate = 120.0 // Flush frequency in Hz (required for Accumulate)
// For Decimate mode:
Ratio = 10 // Send 1 packet every N RT cycles (required for Decimate)
// Background thread (optional)
CPUMask = 0x2 // CPU affinity mask for the network thread
StackSize = 1048576 // Stack size in bytes (default: 1 MiB)
@@ -66,34 +83,40 @@ thread.
### Top-level Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `Port` | uint16 | 44500 | UDP server port |
| `MaxPayloadSize` | uint32 | 1400 | Max payload bytes per UDP datagram (min 18) |
| `CPUMask` | uint32 | 0 (any) | Background thread CPU affinity |
| `StackSize` | uint32 | 1 048 576 | Background thread stack size in bytes |
| Parameter | Type | Default | Description |
| ---------------- | ------ | ---------------- | --------------------------------------------------------------------------- |
| `Port` | uint16 | 44500 | UDP server port (unicast) or TCP control port (multicast). Values ≤ 1024 produce a warning. |
| `MaxPayloadSize` | uint32 | 1400 | Max payload bytes per UDP datagram (min 18) |
| `MulticastGroup` | string | *(absent)* | IPv4 multicast address (e.g. `"239.0.0.1"`). Must be in 224.0.0.0/4. Absent or empty = unicast mode. |
| `Interface` | string | *(absent)* | Local IPv4 address of the interface multicast DATA leaves from, in dotted-quad form (e.g. `"192.168.1.10"`, or `"127.0.0.1"` for loopback-only testing). **Not** an interface name — `"eth0"` is rejected. **Mandatory** when `MulticastGroup` is set. |
| `DataPort` | uint16 | Port+1 | UDP port for multicast DATA datagrams. Ignored in unicast mode. |
| `PublishingMode` | string | Strict | `Strict`: send every RT cycle. `Accumulate`: batch until size/time limit. `Decimate`: send every Nth cycle. |
| `MinRefreshRate` | float64| — | Flush frequency in Hz. **Required** when `PublishingMode` = `Accumulate`. |
| `Ratio` | uint32 | — | Send 1 packet every `Ratio` RT cycles. **Required** when `PublishingMode` = `Decimate`. |
| `CPUMask` | uint32 | 0xFFFFFFFF (any) | Background thread CPU affinity bitmask |
| `StackSize` | uint32 | MARTe2 default | Background thread stack size in bytes |
### Per-signal Parameters
| Parameter | Type | Default | Applies to |
|-----------|------|---------|------------|
| `Unit` | string | `""` | Any type — informational, forwarded to client in CONFIG |
| `RangeMin` | float64 | 0.0 | float32/float64 with `QuantizedType` |
| `RangeMax` | float64 | 1.0 | float32/float64 with `QuantizedType` |
| `QuantizedType` | string | `none` | float32/float64 only |
| `TimeMode` | string | `PacketTime` | Signals with `NumberOfElements > 1` |
| `TimeSignal` | string | — | Required when `TimeMode` ≠ `PacketTime` |
| `SamplingRate` | float64 | 0.0 | Required when `TimeMode` = `FirstSample` or `LastSample` |
| Parameter | Type | Default | Applies to |
| --------------- | ------- | ------------ | -------------------------------------------------------- |
| `Unit` | string | `""` | Any type — informational, forwarded to client in CONFIG |
| `RangeMin` | float64 | 0.0 | float32/float64 with `QuantizedType` |
| `RangeMax` | float64 | 1.0 | float32/float64 with `QuantizedType` |
| `QuantizedType` | string | `none` | float32/float64 only |
| `TimeMode` | string | `PacketTime` | Signals with `NumberOfElements > 1` |
| `TimeSignal` | string | — | Required when `TimeMode` ≠ `PacketTime` |
| `SamplingRate` | float64 | 0.0 | Required when `TimeMode` = `FirstSample` or `LastSample` |
### Quantization Types
| Value | Wire type | Bit depth | Notes |
|-------|-----------|-----------|-------|
| `none` | same as source | — | Raw copy, no quantization |
| `uint8` | uint8 | 8-bit | Maps `[RangeMin, RangeMax]` → `[0, 255]` |
| `int8` | int8 | 8-bit | Maps `[RangeMin, RangeMax]` → `[-127, 127]` |
| `uint16` | uint16 | 16-bit | Maps `[RangeMin, RangeMax]` → `[0, 65 535]` |
| `int16` | int16 | 16-bit | Maps `[RangeMin, RangeMax]` → `[-32 767, 32 767]` |
| Value | Wire type | Bit depth | Notes |
| -------- | -------------- | --------- | ------------------------------------------------- |
| `none` | same as source | — | Raw copy, no quantization |
| `uint8` | uint8 | 8-bit | Maps `[RangeMin, RangeMax]` → `[0, 255]` |
| `int8` | int8 | 8-bit | Maps `[RangeMin, RangeMax]` → `[-127, 127]` |
| `uint16` | uint16 | 16-bit | Maps `[RangeMin, RangeMax]` → `[0, 65 535]` |
| `int16` | int16 | 16-bit | Maps `[RangeMin, RangeMax]` → `[-32 767, 32 767]` |
Quantization formula (unsigned, e.g. uint16):
@@ -104,12 +127,75 @@ wire_value = (uint16)(normalized × 65535)
### Time Modes
| Value | Meaning | Requirements |
|-------|---------|--------------|
| `PacketTime` | The HRT counter captured at `Synchronise()` time is used as the packet timestamp. No per-signal time metadata. | — |
| `FullArray` | `TimeSignal` carries one timestamp per element (same `NumberOfElements`). | `TimeSignal` must have the same `NumberOfElements`. |
| `FirstSample` | `TimeSignal` is a scalar giving the timestamp of element `[0]`. Elements `[1..N-1]` are inferred at `1/SamplingRate` intervals. | Scalar `TimeSignal`; `SamplingRate > 0`. |
| `LastSample` | Same as `FirstSample` but `TimeSignal` is the timestamp of element `[N-1]`. | Scalar `TimeSignal`; `SamplingRate > 0`. |
| Value | Meaning | Requirements |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------- |
| `PacketTime` | The HRT counter captured at `Synchronise()` time is used as the packet timestamp. No per-signal time metadata. | — |
| `FullArray` | `TimeSignal` carries one timestamp per element (same `NumberOfElements`). | `TimeSignal` must have the same `NumberOfElements`. |
| `FirstSample` | `TimeSignal` is a scalar giving the timestamp of element `[0]`. Elements `[1..N-1]` are inferred at `1/SamplingRate` intervals. | Scalar `TimeSignal`; `SamplingRate > 0`. |
| `LastSample` | Same as `FirstSample` but `TimeSignal` is the timestamp of element `[N-1]`. | Scalar `TimeSignal`; `SamplingRate > 0`. |
---
## Network Modes
### Unicast (default)
The server opens a single UDP socket on `Port`. The client initiates the session by sending a
CONNECT packet to that port. The server replies with a CONFIG packet on the same socket and
subsequently sends DATA packets directly to the client's address. One client at a time; a new
CONNECT evicts the previous client.
### Multicast
Enabled by setting `MulticastGroup` to a valid IPv4 multicast address (224.0.0.0/4).
The `Interface` parameter is **mandatory**. It is the local IPv4 address of the
interface DATA datagrams leave from, given in dotted-quad form — it sets
`IP_MULTICAST_IF` on the data socket and is parsed with `inet_addr()`, so an
interface *name* such as `"eth0"` is rejected and `Initialise` fails.
Receivers must join the group on the matching interface. A receiver that joins
with `INADDR_ANY` lets the kernel pick the default-route interface, and it will
silently receive nothing if that is not the interface named by `Interface`.
The server opens a TCP listener on `Port` for control traffic and a UDP socket aimed at
`MulticastGroup:DataPort` for data traffic. The client:
1. Connects to `Port` via TCP and sends a CONNECT packet.
2. Receives the CONFIG packet over TCP.
3. Joins the multicast group (`MulticastGroup:DataPort`) to receive DATA packets.
Multiple clients may receive data simultaneously by joining the same group.
---
## Publishing Modes
### Strict (default)
Sends one DATA packet for every `Synchronise()` call (every RT cycle). Simplest and lowest
latency.
### Accumulate
Batches multiple RT-cycle snapshots into a single DATA packet. All signals (scalars and arrays)
are accumulated: one full snapshot per RT cycle. The batch is flushed when either:
- **Size condition**: adding one more sample would exceed `MaxPayloadSize`.
- **Time condition**: `1/MinRefreshRate` seconds have elapsed since the last flush.
The maximum batch count is computed automatically from `MaxPayloadSize` and the total wire size
of all signals. Scalar signals with `Unit="us"` or `"ns"` are auto-promoted as the per-sample
FullArray time reference for all other scalars.
Requires `MinRefreshRate` (Hz) to be set.
### Decimate
Sends one DATA packet every `Ratio` RT cycles, dropping intermediate cycles. Only the most
recent snapshot at the Nth cycle is sent.
Requires `Ratio` (≥ 1) to be set. `Ratio = 1` is equivalent to `Strict` mode (a warning is
logged).
---
@@ -151,7 +237,7 @@ PrepareNextState() ← opens UDP server socket, starts background threa
---
## Example: minimal scalar streaming
## Example: minimal scalar streaming (unicast)
```
+Data = {
@@ -168,6 +254,26 @@ PrepareNextState() ← opens UDP server socket, starts background threa
}
```
## Example: multicast with accumulation
```
+Streamer = {
Class = UDPStreamer
Port = 44500 // TCP control port
MulticastGroup = "239.0.0.1" // Enables multicast mode
Interface = "192.168.1.10" // Local IP of the outgoing interface (mandatory)
DataPort = 44501 // UDP data port (default: Port+1)
MaxPayloadSize = 1400
PublishingMode = "Accumulate"
MinRefreshRate = 60.0 // Flush at least 60 times/s
Signals = {
Time = { Type = uint32; Unit = "us" }
Voltage = { Type = float32; Unit = "V"; RangeMin = -10.0; RangeMax = 10.0; QuantizedType = uint16 }
}
}
```
## Example: high-frequency burst
```
@@ -192,9 +298,69 @@ PrepareNextState() ← opens UDP server socket, starts background threa
}
```
---
## UDPStreamerClient DataSource
`UDPStreamerClient` is a MARTe2 **input** DataSource that receives signals from a `UDPStreamer`
server. Transport, fragment reassembly, and auto-reconnect are delegated to `UDPSClient`; the
DataSource only decodes CONFIG/DATA payloads into real-time signal memory.
### Configuration
```
+ClientDS = {
Class = UDPStreamerClient
ServerAddress = "192.168.1.10" // UDPStreamer server IP
Port = 44500 // Server port
// Multicast (optional — omit for unicast)
MulticastGroup = "239.0.0.1"
DataPort = 44501 // UDP data port (default: Port+1)
Interface = "192.168.1.10" // See table below
MaxPayloadSize = 1400
Signals = {
Counter = { Type = uint32 }
}
}
```
### Parameters
| Parameter | Type | Default | Description |
| --------------- | ------- | ---------- | ----------- |
| `ServerAddress` | string | 127.0.0.1 | IPv4 address of the `UDPStreamer` server. |
| `Port` | uint16 | 44500 | Server UDP port (unicast) or TCP control port (multicast). |
| `MulticastGroup`| string | *(absent)* | IPv4 multicast address. Presence enables multicast mode. |
| `DataPort` | uint16 | Port+1 | UDP port for multicast DATA datagrams. |
| `Interface` | string | *(absent)* | Local IPv4 dotted-quad address (e.g. `"127.0.0.1"`) of the interface to join the multicast group on. **Optional**: omitting it uses the default-route interface (INADDR_ANY), which silently receives nothing if the server sends on a different interface. Not an interface name — `"eth0"` is invalid. |
| `MaxPayloadSize`| uint32 | 1400 | Max payload bytes per datagram (must match the server). |
| `SilenceTimeout`| float32 | 1.0 | Seconds of no data before auto-reconnect. 0 disables. |
| `KeepAliveInterval` | uint32 | 15 | Seconds between unicast keepalive ACKs. 0 disables. |
| `CPUMask` | uint32 | 0xFFFFFFFF | CPU affinity for the background receiver thread. |
| `StackSize` | uint32 | default | Stack size in bytes for the receiver thread. |
With `MaxPayloadSize = 1400`, a single 1000-element float32 signal produces:
```
payload = 8 B (HRT timestamp) + 4 B (T0/uint32) + 4000 B (float32×1000) = 4012 B
fragments = ceil(4012 / 1383) = 3
```
## Example: decimated output
```
+Streamer = {
Class = UDPStreamer
Port = 44500
PublishingMode = "Decimate"
Ratio = 10 // Send 1 packet every 10 RT cycles
Signals = {
Time = { Type = uint32; Unit = "us" }
Position = { Type = float64; Unit = "mm" }
}
}
```
Docs/WebUI.md
+37
View File
@@ -80,6 +80,23 @@ Signals received in the CONFIG packet are listed in the sidebar:
- **Spatial arrays** — `TimeMode = PacketTime` arrays are shown as an expandable
group; individual elements (`Ch1[0]`, `Ch1[1]`, …) can be dragged independently.
The unit badge next to each signal shows the calibration's unit override when one
is set, and the streamer's own unit otherwise.
At the bottom of the sidebar, the collapsible **Sources & Config** section holds:
- the `host:port`, label, multicast group and data port inputs plus **Connect**,
which adds a source at runtime;
- **Save** — writes the source list and the whole calibration table to the hub's
config file;
- **Reload** — re-reads that file. Calibration is replaced wholesale (so unsaved
edits are discarded), sources present in the file but not running are added,
and no running source is stopped or reconnected. The hub may send an updated
`sources` list even when nothing changed (see the API doc for the per-hub
difference);
- a status line showing the written path on success or the hub's error text on
failure.
Click the sidebar toggle button (☰) to collapse/expand the signal list.
### Adding Plots
@@ -143,11 +160,31 @@ plot header showing per-signal vertical scale controls:
| **V/div** | Volts (or units) per division |
| **Pos (div)** | Screen position in divisions (draggable offset marker on Y axis) |
| **Type** (Mixed mode only) | Toggle between **Analog** and **Digital** for this signal |
| **Cal · Scale** | Data calibration gain. `value = raw × Scale + Offset` |
| **Cal · Offset** | Data calibration bias, in calibrated units |
| **Cal · Unit** | Overrides the unit reported by the streamer (max 16 UTF-8 bytes; a multi-byte character such as `°C` counts as more than one byte) |
| **Reset** | Clears this signal's calibration (`Scale = 1`, `Offset = 0`, no unit override) |
| **✕** | Close the toolbar and deselect the signal |
Offset markers (small triangles on the Y axis) show each signal's position and can
be dragged to reposition signals without opening the toolbar.
**Calibration vs. V/div and Offset.** They are different things. V/div and Offset
are a *display* transform: they move and stretch the trace on screen. Calibration
changes *the value itself* — the plot, the Y-axis tick labels, the cursor and
hover readouts, the CSV export and the trigger threshold all report
`raw × Scale + Offset` in the calibrated unit. V/div is then read as "calibrated
units per division" and Offset as "the calibrated value at screen centre".
The calibration header names the **base** signal and its element count, because
one entry covers every element of an array — opening the toolbar on `Adc[3]` and
editing the calibration moves all of `Adc`.
Calibration is keyed by the source's **label**, is shared with every other
browser connected to the same hub, and is not persisted until you press **Save**
in the Sources & Config section. It is mirrored to `localStorage` so it survives
a page reload even against a hub with no config file.
### Plot Controls
| Control | Action |
Makefile.gcc
+2
View File
@@ -25,6 +25,7 @@ core:
test:
$(MAKE) -C Test/Components/DataSources/UDPStreamer -f Makefile.gcc
$(MAKE) -C Test/Components/DataSources/UDPStreamerClient -f Makefile.gcc
$(MAKE) -C Test/Applications/StreamHub -f Makefile.gcc
$(MAKE) -C Test/GTest -f Makefile.gcc
$(MAKE) -C Test/Integration -f Makefile.gcc
@@ -39,6 +40,7 @@ clean:
$(MAKE) -C Source/Components/Interfaces/TCPLogger -f Makefile.gcc clean
$(MAKE) -C Source/Components/Interfaces/DebugService -f Makefile.gcc clean
$(MAKE) -C Test/Components/DataSources/UDPStreamer -f Makefile.gcc clean
$(MAKE) -C Test/Components/DataSources/UDPStreamerClient -f Makefile.gcc clean
$(MAKE) -C Test/Applications/StreamHub -f Makefile.gcc clean
$(MAKE) -C Test/GTest -f Makefile.gcc clean
$(MAKE) -C Test/Integration -f Makefile.gcc clean
README.md
+48 -31
View File
@@ -9,15 +9,15 @@ for control applications built with [MARTe2](https://vcis.f4e.europa.eu/marte2-d
This repository integrates two complementary capabilities:
| Capability | Component | Purpose |
|---|---|---|
| **Signal streaming** | `UDPStreamer` DataSource | Continuously stream selected signals to a browser-based oscilloscope over UDP |
| **Signal debugging** | `DebugService` Interface | On-demand signal tracing, value forcing, and conditional breakpoints — zero application code changes required |
| **Sine generation** | `SineArrayGAM` | Generate continuous sine-wave arrays for testing and simulation |
| **Time stamping** | `TimeArrayGAM` | Provide time-reference arrays aligned to an RT cycle |
| **Log forwarding** | `TCPLogger` Interface | Forward `REPORT_ERROR` log events to TCP clients in real time |
| **Integrated client** | `Common/Client/go` | Go packages for UDPS protocol and WebSocket hub |
| **Debug web client** | `Client/debugger` | Browser-based debug UI communicating with `DebugService` |
| Capability | Component | Purpose |
| --------------------- | ------------------------ | ------------------------------------------------------------------------------------------------------------- |
| **Signal streaming** | `UDPStreamer` DataSource | Continuously stream selected signals to a browser-based oscilloscope over UDP |
| **Signal debugging** | `DebugService` Interface | On-demand signal tracing, value forcing, and conditional breakpoints — zero application code changes required |
| **Sine generation** | `SineArrayGAM` | Generate continuous sine-wave arrays for testing and simulation |
| **Time stamping** | `TimeArrayGAM` | Provide time-reference arrays aligned to an RT cycle |
| **Log forwarding** | `TCPLogger` Interface | Forward `REPORT_ERROR` log events to TCP clients in real time |
| **Integrated client** | `Common/Client/go` | Go packages for UDPS protocol and WebSocket hub |
| **Debug web client** | `Client/debugger` | Browser-based debug UI communicating with `DebugService` |
---
@@ -49,9 +49,9 @@ MARTe_Integrated_components/
### UDPStreamer DataSource
Streams MARTe2 signals over UDP using the UDPS binary protocol. Clients register by
Streams MARTe2 signals over UDP using the UDPS binary protocol. Clients register by
sending a `CONNECT` packet; the server then sends `CONFIG` (signal metadata) and continuous
`DATA` packets. Features:
`DATA` packets. Features:
- Optional 16-bit quantization (configurable per signal: `QuantizedType`)
- Packed high-frequency bursts (`NumberOfElements > 1` with `SamplingRate`)
@@ -61,26 +61,42 @@ See `Docs/UDPStreamer.md` and `Docs/Protocol.md`.
### SineArrayGAM
Generates a continuous float32 sine-wave array every RT cycle. Used as a signal
source for testing and demo applications. Configurable: `Frequency`, `Amplitude`,
Generates a continuous float32 sine-wave array every RT cycle. Used as a signal
source for testing and demo applications. Configurable: `Frequency`, `Amplitude`,
`Phase`, `SamplingRate`, `NumberOfElements`.
See `Docs/SineArrayGAM.md`.
### TimeArrayGAM
Generates a time-reference float64 array. Each element holds the timestamp of the
Generates a time-reference uint64 array. Each element holds the timestamp of the
corresponding sample in a packed burst, computed from the RT cycle timestamp and the
configured `SamplingRate`.
`Anchor` selects how the burst is placed in time:
| `Anchor` | `out[k]` |
|---|---|
| `FirstSample` | `input + k · period` |
| `LastSample` | `input − (N−1−k) · period` |
| `Continuous` | `input(first cycle) + (n + k) · period` |
`FirstSample`/`LastSample` re-read the timer each cycle, so a lost RT cycle
(`LinuxTimer` re-phases with `counter += nCycles`) punches a whole-period hole
into the time base even though only one array of samples was produced. Use
`Continuous` when the data signal is itself contiguous (`SineArrayGAM` never
skips phase): it latches the timer once and then advances an internal sample
counter by `N` per cycle, like an acquisition card running off its own clock.
### DebugService Interface
Instruments a running MARTe2 application **without modifying its source code**. On
Instruments a running MARTe2 application **without modifying its source code**. On
`Initialise()` it patches the `ClassRegistryDatabase` to wrap all standard
`MemoryMap*Broker` types. When `RealTimeApplication::ConfigureApplication()` runs
`MemoryMap*Broker` types. When `RealTimeApplication::ConfigureApplication()` runs
afterward the application transparently uses the wrapped brokers.
Capabilities accessible over TCP (port 8080 by default):
- `DISCOVER` — enumerate all signals with type and alias metadata
- `TRACE` — enable/disable high-speed UDP telemetry per signal (with decimation)
- `FORCE` / `UNFORCE` — inject persistent values into signals on the RT path
@@ -98,7 +114,7 @@ See `Docs/DebugService.md`.
### TCPLogger Interface
A `LoggerConsumerI` that forwards every MARTe2 `REPORT_ERROR` call to up to 8 TCP
clients on a configurable port. Works as a sidecar to `DebugService`.
clients on a configurable port. Works as a sidecar to `DebugService`.
### StreamHub Application
@@ -108,7 +124,7 @@ UDPStreamer sources and serves them to oscilloscope clients over WebSocket
hub-side trigger engine, per-window zoom. Clients: browser SPA
(`Client/webui` + `Client/udpstreamer/static`) and native ImGui desktop client
(`Client/streamhub`). Demo: `./run_streamhub.sh -w -g`; E2E test:
`./run_e2e_test.sh`.
`Test/E2E/suite/run_e2e.sh`.
See `Docs/StreamHub-UserGuide.md`, `Docs/StreamHub-API.md` and
`Docs/StreamHub-Developer.md`.
@@ -116,7 +132,7 @@ See `Docs/StreamHub-UserGuide.md`, `Docs/StreamHub-API.md` and
### UDPS Protocol
The `Common/UDP/UDPSProtocol.h` header defines the shared binary wire format used by
both `UDPStreamer` and `DebugService`. It is intentionally free of MARTe2-specific
both `UDPStreamer` and `DebugService`. It is intentionally free of MARTe2-specific
dependencies so it can also be used by Go clients (via `Common/Client/go/udpsprotocol`).
See `Docs/Protocol.md`.
@@ -224,18 +240,19 @@ Open `http://localhost:9090`, explore the object tree, trace signals, force valu
## Documentation
| Document | Contents |
|---|---|
| `Docs/Protocol.md` | UDPS binary wire protocol specification |
| `Docs/UDPStreamer.md` | UDPStreamer DataSource configuration reference |
| `Docs/SineArrayGAM.md` | SineArrayGAM configuration reference |
| `Docs/DebugService.md` | DebugService TCP API and architecture |
| `Docs/Tutorial.md` | Step-by-step tutorial covering both components |
| `Docs/WebUI.md` | Web client user guide |
| `Docs/StreamHub-UserGuide.md` | StreamHub oscilloscope user guide (web + ImGui clients) |
| `Docs/StreamHub-API.md` | StreamHub WebSocket protocol (commands, events, binary frames) |
| `Docs/StreamHub-Developer.md` | StreamHub internals, threading, time base, build & E2E tests |
| `ARCHITECTURE.md` | System architecture overview |
| Document | Contents |
| ----------------------------- | -------------------------------------------------------------- |
| `Docs/Protocol.md` | UDPS binary wire protocol specification |
| `Docs/UDPStreamer.md` | UDPStreamer DataSource configuration reference |
| `Docs/UDPS-C-Client.md` | Standalone C/C++ UDPS receiver library (`Common/Client/c`) |
| `Docs/SineArrayGAM.md` | SineArrayGAM configuration reference |
| `Docs/DebugService.md` | DebugService TCP API and architecture |
| `Docs/Tutorial.md` | Step-by-step tutorial covering both components |
| `Docs/WebUI.md` | Web client user guide |
| `Docs/StreamHub-UserGuide.md` | StreamHub oscilloscope user guide (web + ImGui clients) |
| `Docs/StreamHub-API.md` | StreamHub WebSocket protocol (commands, events, binary frames) |
| `Docs/StreamHub-Developer.md` | StreamHub internals, threading, time base, build & E2E tests |
| `ARCHITECTURE.md` | System architecture overview |
---
Source/Applications/StreamHub/SignalRingBuffer.h
+96
View File
@@ -78,6 +78,32 @@ public:
/** @return Current number of stored points (≤ capacity). */
uint32 Count() const;
/** @return Allocated capacity in points. */
uint32 Capacity() const;
/**
* @brief Enlarge the buffer to @p newCap points, keeping the stored data
* and the TotalWritten() counter (unlike Allocate(), which resets both so
* every reader cursor and every retained sample is lost).
* @return true if the buffer now holds at least @p newCap points.
*/
bool Grow(uint32 newCap);
/**
* @brief Wall-clock span currently retained, i.e. newest minus oldest
* timestamp. 0 when fewer than two points are stored.
*/
float64 TimeSpan() const;
/**
* @brief Timestamp of the most recently stored point, 0 when empty.
*
* This is the source's own time base, which is *not* the hub's wall clock:
* use it, never clock_gettime(), whenever a decision depends on how far
* the data itself has advanced.
*/
float64 NewestTime() const;
/** @brief Discard all stored points. */
void Clear();
@@ -138,6 +164,69 @@ inline bool SignalRingBuffer::Allocate(uint32 maxPts) {
return true;
}
inline bool SignalRingBuffer::Grow(uint32 newCap) {
if (newCap <= capacity) { return true; }
/* Allocate outside the lock; readers may be active. */
float64 *newT = new float64[newCap];
float64 *newV = new float64[newCap];
if ((newT == static_cast<float64 *>(0)) ||
(newV == static_cast<float64 *>(0))) {
delete[] newT;
delete[] newV;
return false;
}
(void) mutex.FastLock();
if (newCap > capacity) {
/* Copy oldest-to-newest so the new buffer starts unwrapped. */
const uint32 avail = count;
for (uint32 i = 0u; i < avail; i++) {
const uint32 idx = (head + capacity - avail + i) % capacity;
newT[i] = tBuf[idx];
newV[i] = vBuf[idx];
}
float64 *oldT = tBuf;
float64 *oldV = vBuf;
tBuf = newT;
vBuf = newV;
capacity = newCap;
head = avail;
/* count and totalWritten are unchanged: no sample is gained or lost,
* so push cursors stay valid across the resize. */
mutex.FastUnLock();
delete[] oldT;
delete[] oldV;
return true;
}
mutex.FastUnLock();
delete[] newT;
delete[] newV;
return true;
}
inline float64 SignalRingBuffer::TimeSpan() const {
(void) mutex.FastLock();
float64 span = 0.0;
if ((capacity > 0u) && (count > 1u)) {
const uint32 oldest = (head + capacity - count) % capacity;
const uint32 newest = (head + capacity - 1u) % capacity;
span = tBuf[newest] - tBuf[oldest];
}
mutex.FastUnLock();
return (span > 0.0) ? span : 0.0;
}
inline float64 SignalRingBuffer::NewestTime() const {
(void) mutex.FastLock();
float64 t = 0.0;
if ((capacity > 0u) && (count > 0u)) {
t = tBuf[(head + capacity - 1u) % capacity];
}
mutex.FastUnLock();
return t;
}
inline void SignalRingBuffer::Write(float64 t, float64 v) {
(void) mutex.FastLock();
if (capacity > 0u) {
@@ -288,6 +377,13 @@ inline MARTe::uint64 SignalRingBuffer::TotalWritten() const {
return tw;
}
inline uint32 SignalRingBuffer::Capacity() const {
(void) mutex.FastLock();
const uint32 c = capacity;
mutex.FastUnLock();
return c;
}
inline uint32 SignalRingBuffer::Count() const {
(void) mutex.FastLock();
uint32 c = count;
Source/Applications/StreamHub/StreamHub.cpp
+914 -156
View File
File diff suppressed because it is too large Load Diff
Source/Applications/StreamHub/StreamHub.h
+112 -11
View File
@@ -44,6 +44,32 @@ using MARTe::StructuredDataI;
/** Maximum number of simultaneously connected UDPStreamer sources. */
static const uint32 kMaxSessions = 32u;
/** Maximum number of stored per-signal calibration entries. */
static const uint32 kMaxCalibration = 256u;
/** Maximum length of a calibration unit override (mirrors the Go maxUnitLen). */
static const uint32 kMaxUnitLen = 16u;
/**
* @brief One per-signal affine calibration: y = raw*scale + offset.
*
* Keyed by the source LABEL (not the runtime "sN" id, which is assigned in
* add-order and would rebind if the source list were reordered) and by the
* BASE signal name (no "[i]" suffix: one entry covers a whole array signal).
*
* Fixed-size char arrays are used deliberately: they avoid per-entry heap
* churn (no StreamString allocation per calibration slot), keep the type free
* of STL, and make a CalibrationEntry snapshot trivially copyable under the
* calibration mutex lock.
*/
struct CalibrationEntry {
char source[128]; ///< Source label
char signal[128]; ///< Base signal name (no "[i]" suffix)
char unit[17]; ///< Unit override (max kMaxUnitLen bytes + NUL)
MARTe::float64 scale;
MARTe::float64 offset;
};
/**
* @brief Top-level StreamHub orchestrator.
*
@@ -65,7 +91,7 @@ public:
* WSPort (uint32, default 8090)
* MaxPoints (uint32, default 20000) — ring buffer capacity per signal
* PushRate (uint32, default 30) — push loop rate in Hz
* MaxPushPoints (uint32, default 500) — LTTB threshold for live push
* MaxPushPoints (uint32, default 50) — LTTB threshold for live push
* StatsRate (uint32, default 1) — stats broadcast rate in Hz
* +Sources { +<id> { Label=...; Addr=...; Port=... } }
*
@@ -108,6 +134,12 @@ private:
/** Broadcast {"type":"config","sourceId":...} for one session. */
void BroadcastConfig(uint32 sessionIdx);
/** Broadcast {"type":"calibration","cal":[...]} to all clients. */
void BroadcastCalibration();
/** Broadcast {"type":"configSaved"|"configReloaded","ok":...} to all clients. */
void BroadcastConfigAck(const char *msgType, bool ok, const char *errText);
/* ---- Trigger (push loop side) ----------------------------------------- */
/**
@@ -120,12 +152,40 @@ private:
void BroadcastTriggerState();
/**
* @brief Build and broadcast the version=2 binary capture frame:
* [u8 2][f64 trigTime][f64 preSec][f64 postSec][u32 nSig]
* {[u16 keyLen][fullKey][u32 N][t f64×N][v f64×N]}
* @brief Size every ring so it retains the current trigger window.
* Called from the push loop once per stats tick; a no-op once the rings
* are large enough. Rates are measured from the rings themselves because
* most sources advertise samplingRate = 0.
*/
void BroadcastTriggerCapture(float64 trigTime, float64 preSec,
float64 postSec);
void GrowRingsForTrigger();
/** @return Largest ring capacity currently allocated across all sessions. */
uint32 CurrentMaxRingCapacity() const;
/**
* @brief How far source @p i has produced, in the trigger's time base;
* @p wallNowS when it publishes no producer clock (its samples are then
* stamped on arrival, so they share the hub's wall clock).
*/
float64 SourceFrontierTime(uint32 i, float64 wallNowS) const;
/* ---- Trigger capture assembly ---------------------------------------
* Sources are harvested one at a time, each as soon as *it* has produced
* past the end of the window, rather than all together once the slowest
* has. Sources free-run on their own clocks and can lag each other by
* seconds; making every source wait for the slowest lets the leaders' ring
* buffers roll past the pre-trigger region before it is ever read. */
/** @brief Start a version=2 capture frame:
* [u8 2][f64 trigTime][f64 preSec][f64 postSec][u32 nSig]. */
void BeginTriggerCapture(float64 trigTime, float64 preSec, float64 postSec);
/** @brief Append session @p i's signals to the pending frame, each as
* {[u16 keyLen][fullKey][u32 N][t f64×N][v f64×N]}. */
void HarvestTriggerCapture(uint32 i, float64 t0, float64 t1);
/** @brief Patch nSig, broadcast the pending frame and release it. */
void FinishTriggerCapture();
/* ---- Command handlers (called from OnWSCommand) ---------------------- */
@@ -140,11 +200,14 @@ private:
void HandleRearm();
void HandleTrigStop(const char *json);
void HandleSetTrigger(const char *json);
void HandleForceTrigger();
void HandleZoom(const char *json, uint32 slotIdx);
void HandleHistoryZoom(const char *json, uint32 slotIdx);
void HandleHistoryInfo(uint32 slotIdx);
void HandleSetMaxPoints(const char *json);
void HandlePing(uint32 slotIdx);
void HandleSetCalibration(const char *json);
void HandleReloadConfig();
/* ---- Binary recorder commands --------------------------------------- */
@@ -171,10 +234,32 @@ private:
const char *mcGroup, uint16 dataPort);
/**
* @brief Load sources from sourcesFile_ (JSON array of
* {"label","addr","multicastGroup","dataPort"}) and start them.
* @brief Load sources and calibration from sourcesFile_ (a flat JSON array
* of {"label","addr","multicastGroup","dataPort"} source blocks and
* {"source","signal","scale","offset","unit"} calibration blocks).
* @param skipActive when true, a source whose "host:port" is already
* streaming is left alone instead of being started a second time.
* @param clearCalibration when true, the calibration table is cleared
* after a successful fread (never before), so a transient I/O failure
* does not silently wipe user calibration data.
* @return true if the file was read.
*/
void LoadSourcesFile();
bool LoadSourcesFile(bool skipActive, bool clearCalibration = false);
/** @return true if a session for this "host:port" is already active. */
bool SourceIsActive(const char *addrPort);
/**
* @brief Store or replace one calibration entry. An identity entry
* (scale 1, offset 0, empty unit) removes any stored one instead.
* @return true if the entry was valid (and therefore stored or removed).
*/
bool SetCalibrationEntry(const char *source, const char *signal,
MARTe::float64 scale, MARTe::float64 offset,
const char *unit);
/** Drop every calibration entry (used by reload, which replaces wholesale). */
void ClearCalibration();
/* ---- Tiny JSON helpers ----------------------------------------------- */
@@ -214,11 +299,17 @@ private:
uint32 pushRateHz_;
uint32 maxPushPoints_;
uint32 statsRateHz_;
uint32 ringTemporal_; ///< Ring capacity for multi-element (waveform) signals
uint32 ringTemporal_; ///< Initial ring capacity for multi-element (waveform) signals
uint32 ringScalar_; ///< Ring capacity for scalar signals
uint32 ringMaxPts_; ///< Ceiling a ring may be grown to for a trigger window
volatile float64 trigRetentionSec_; ///< Retention the current trigger window needs
StreamString sourcesFile_; ///< Persistent dynamic source list (JSON)
uint32 nextSourceId_; ///< Counter for generated session ids ("sN")
CalibrationEntry *calibration_; ///< Heap-allocated array[kMaxCalibration]
uint32 numCalibration_;
FastPollingMutexSem calibrationMutex_; ///< Serializes calibration reads/writes
/* Push loop state */
volatile bool running_;
uint32 tickCount_; ///< incremented each push tick
@@ -231,7 +322,9 @@ private:
static const uint32 kPushBufSize = 8u * 1024u * 1024u;
uint8 *pushBuf_;
/* Decimated output scratch (LTTB): maxPushPoints × 2 arrays per signal */
/* Decimated output scratch (LTTB). Sized like the read scratch rather
* than maxPushPoints_: a PACKET-timed array raises its own threshold to
* one packet's worth of elements, which can exceed maxPushPoints_. */
float64 *lttbT_;
float64 *lttbV_;
@@ -250,6 +343,14 @@ private:
TrigState lastTrigState_; ///< Last broadcast FSM state
bool rearmPending_; ///< Normal-mode auto-rearm scheduled
float64 rearmAtWallS_; ///< Wall time of the scheduled auto-rearm
float64 collectStartWallS_; ///< Wall time COLLECTING began (watchdog only)
/* Capture frame under assembly across ticks (push thread only) */
MARTe::uint8 *capBuf_; ///< Pending frame, NULL when idle
uint32 capCap_; ///< Allocated size of capBuf_
uint32 capOff_; ///< Bytes written so far
uint32 capNSig_; ///< Signals appended so far
bool capHarvested_[kMaxSessions]; ///< Session already appended
};
} /* namespace StreamHub */
Source/Applications/StreamHub/TriggerEngine.cpp
+34 -1
View File
@@ -14,6 +14,8 @@ TriggerEngine::TriggerEngine()
stopped_(false),
prevValue_(0.0),
prevValid_(false),
lastTime_(0.0),
lastTimeValid_(false),
trigTime_(0.0),
firedPreSec_(0.0),
firedPostSec_(0.0),
@@ -25,9 +27,16 @@ void TriggerEngine::SetConfig(const TriggerConfig &cfg) {
config_ = cfg;
/* Clamp to web UI bounds */
if (config_.windowSec < 1.0e-4) { config_.windowSec = 1.0e-4; }
if (config_.windowSec > 10.0) { config_.windowSec = 10.0; }
/* 60 s where the Go hub allows 600. Deliberate: these rings are
* fixed-capacity and store every sample, so a window they cannot hold is
* harvested truncated and silently decimated to kTrigCapturePts. The Go
* hub stores min/max pairs instead once a window outgrows its budget, so
* there a long window costs resolution rather than coverage. */
if (config_.windowSec > 60.0) { config_.windowSec = 60.0; }
if (config_.prePercent < 0.0) { config_.prePercent = 0.0; }
if (config_.prePercent > 100.0) { config_.prePercent = 100.0; }
if (config_.holdoffSec < 0.0) { config_.holdoffSec = 0.0; }
if (config_.holdoffSec > 60.0) { config_.holdoffSec = 60.0; }
epoch_++;
prevValid_ = false;
prevValue_ = 0.0;
@@ -82,6 +91,11 @@ bool TriggerEngine::GetStopped() const {
void TriggerEngine::CheckSample(float64 t, float64 v) {
(void) mutex_.FastLock();
/* Track the newest watched timestamp in every state so Force() has a
* reference time to latch the capture window around. */
lastTime_ = t;
lastTimeValid_ = true;
if (state_ != kTrigArmed) {
mutex_.FastUnLock();
return;
@@ -122,6 +136,25 @@ void TriggerEngine::CheckSample(float64 t, float64 v) {
mutex_.FastUnLock();
}
bool TriggerEngine::Force() {
(void) mutex_.FastLock();
bool ok = lastTimeValid_ && (state_ != kTrigCollecting);
if (ok) {
state_ = kTrigCollecting;
trigTime_ = lastTime_;
firedPreSec_ = config_.windowSec * config_.prePercent / 100.0;
firedPostSec_ = config_.windowSec - firedPreSec_;
firedValid_ = true;
REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information,
"TriggerEngine: forced at t=%.6f (pre=%.4fs post=%.4fs)",
trigTime_, firedPreSec_, firedPostSec_);
}
mutex_.FastUnLock();
return ok;
}
TrigState TriggerEngine::GetState() const {
(void) mutex_.FastLock();
TrigState ret = state_;
Source/Applications/StreamHub/TriggerEngine.h
+15 -2
View File
@@ -62,9 +62,10 @@ struct TriggerConfig {
StreamString signalKey; ///< Full key: "src:sig" or "src:sig[i]"
TrigEdge edge; ///< Rising / falling / both
float64 threshold; ///< Trigger threshold (physical units)
float64 windowSec; ///< Capture window length [1e-4 .. 10] s
float64 windowSec; ///< Capture window length [1e-4 .. 60] s
float64 prePercent; ///< Pre-trigger part of the window [0 .. 100] %
TrigAcqMode mode; ///< Normal (auto-rearm) or single
float64 holdoffSec; ///< Rearm delay after a capture [0 .. 60] s
};
/**
@@ -106,6 +107,15 @@ public:
*/
void CheckSample(float64 t, float64 v);
/**
* @brief Fire the trigger unconditionally at the most recent sample time of
* the watched signal, latching the pre/post window exactly as CheckSample
* does. Any state except COLLECTING → COLLECTING.
* @return false when no sample has been seen yet, or a capture is already
* being collected.
*/
bool Force();
/** @return Current FSM state. */
TrigState GetState() const;
@@ -127,6 +137,8 @@ private:
bool stopped_;
float64 prevValue_; ///< Last sample (edge detection)
bool prevValid_; ///< First-sample guard in ARMED state
float64 lastTime_; ///< Timestamp of the newest watched sample
bool lastTimeValid_;///< true once a watched sample has been seen
float64 trigTime_; ///< Latched trigger time (Unix s)
float64 firedPreSec_; ///< Window pre-part latched at fire time
float64 firedPostSec_; ///< Window post-part latched at fire time
@@ -138,7 +150,8 @@ inline TriggerConfig::TriggerConfig()
threshold(0.0),
windowSec(1.0),
prePercent(20.0),
mode(kTrigNormal) {
mode(kTrigNormal),
holdoffSec(0.2) {
}
} /* namespace StreamHub */
Source/Applications/StreamHub/UDPSourceSession.cpp
+110 -11
View File
@@ -220,6 +220,9 @@ void UDPSourceSession::ParseConfigPayload(const uint8 *payload, uint32 size) {
memcpy(&sigDescs_[i],
payload + 4u + i * UDPS_SIGNAL_DESC_SIZE,
UDPS_SIGNAL_DESC_SIZE);
/* MD-3: force null-termination of name/unit to prevent intra-struct OOB read */
sigDescs_[i].name[sizeof(sigDescs_[i].name) - 1u] = '\0';
sigDescs_[i].unit[sizeof(sigDescs_[i].unit) - 1u] = '\0';
}
publishMode_ = payload[4u + numSigs * UDPS_SIGNAL_DESC_SIZE];
numSignals_ = numSigs;
@@ -237,8 +240,11 @@ void UDPSourceSession::ParseConfigPayload(const uint8 *payload, uint32 size) {
/* (Re)allocate the time-signal decode scratch to the largest element count. */
uint32 maxElems = 1u;
for (uint32 i = 0u; i < numSigs; i++) {
uint32 ne = sigDescs_[i].numRows * sigDescs_[i].numCols;
if (ne > maxElems) { maxElems = ne; }
uint64 ne = static_cast<uint64>(sigDescs_[i].numRows) *
static_cast<uint64>(sigDescs_[i].numCols);
if (ne == 0u) { ne = 1u; }
if (ne > 0x100000u) { ne = 0x100000u; /* sanity cap */ }
if (ne > maxElems) { maxElems = static_cast<uint32>(ne); }
}
if (maxElems > timeScratchLen_) {
delete[] timeScratch_;
@@ -289,6 +295,83 @@ void UDPSourceSession::AllocateRingBuffers() {
}
}
bool UDPSourceSession::GrowRingsForSeconds(float64 seconds, uint32 maxPts) {
if ((seconds <= 0.0) || (maxPts == 0u)) { return false; }
(void) metaMutex_.FastLock();
const uint32 nSigs = numSignals_;
metaMutex_.FastUnLock();
bool grew = false;
for (uint32 i = 0u; i < nSigs; i++) {
const uint32 count = rings_[i].Count();
const float64 span = rings_[i].TimeSpan();
/* Need a decent sample of the stream before extrapolating a rate;
* a couple of packets' worth of span is enough at any rate. */
if ((count < 2u) || (span <= 0.0)) { continue; }
const float64 rate = static_cast<float64>(count) / span;
/* 20 % headroom absorbs rate jitter and the capture margin. */
float64 need = rate * seconds * 1.2;
if (need > static_cast<float64>(maxPts)) {
need = static_cast<float64>(maxPts);
}
const uint32 needPts = static_cast<uint32>(need);
if (needPts > rings_[i].Capacity()) {
if (rings_[i].Grow(needPts)) { grew = true; }
}
}
return grew;
}
uint32 UDPSourceSession::GetMaxRingCapacity() const {
(void) metaMutex_.FastLock();
const uint32 nSigs = numSignals_;
metaMutex_.FastUnLock();
uint32 maxCap = 0u;
for (uint32 i = 0u; i < nSigs; i++) {
const uint32 c = rings_[i].Capacity();
if (c > maxCap) { maxCap = c; }
}
return maxCap;
}
float64 UDPSourceSession::ProducerNewestTime() const {
/* Mirror exactly the ParseDataPayload branches that timestamp from the
* referenced time signal; every other branch stamps on arrival and so
* would report "now" in the hub's clock, not the producer's. The time
* signal itself is one of those — it is PACKET-timed. */
(void) metaMutex_.FastLock();
const uint32 nSigs = numSignals_;
bool producerTimed[UDPSS_MAX_SIGNALS];
for (uint32 i = 0u; i < nSigs; i++) {
const UDPSSignalDescriptor &d = sigDescs_[i];
uint64 ne = static_cast<uint64>(d.numRows) *
static_cast<uint64>(d.numCols);
if (ne == 0u) { ne = 1u; }
const bool hasTimeSig = (d.timeSignalIdx != UDPS_NO_TIME_SIGNAL) &&
(d.timeSignalIdx < nSigs);
const bool isFirstLast = (ne > 1u) &&
((d.timeMode == UDPS_TIMEMODE_FIRST_SAMPLE) ||
(d.timeMode == UDPS_TIMEMODE_LAST_SAMPLE));
const bool isFullArray = (d.timeMode == UDPS_TIMEMODE_FULL_ARRAY);
producerTimed[i] = hasTimeSig && (isFullArray || isFirstLast);
}
metaMutex_.FastUnLock();
/* Signals of one source share a packet, so they advance together; the max
* is "how far this source has produced" without stalling on a signal that
* simply is not being sent. */
float64 newest = 0.0;
for (uint32 i = 0u; i < nSigs; i++) {
if (!producerTimed[i]) { continue; }
const float64 t = rings_[i].NewestTime();
if (t > newest) { newest = t; }
}
return newest;
}
/*---------------------------------------------------------------------------*/
/* DATA parsing */
/*---------------------------------------------------------------------------*/
@@ -343,22 +426,36 @@ void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size) {
uint32 off = offset;
for (uint32 s = 0u; s < nSigs; s++) {
const UDPSSignalDescriptor &desc = descs[s];
uint32 numElements = desc.numRows * desc.numCols;
if (numElements == 0u) { numElements = 1u; }
/* HI-1: use 64-bit multiply to avoid overflow on attacker-controlled numRows/numCols */
uint64 numElements64 = static_cast<uint64>(desc.numRows) *
static_cast<uint64>(desc.numCols);
if (numElements64 == 0u) { numElements64 = 1u; }
if (numElements64 > 0x100000u) { return; /* sanity cap: 1M elements */ }
uint32 numElements = static_cast<uint32>(numElements64);
uint32 wireElemBytes = (desc.quantType != UDPS_QUANT_NONE)
? QuantWireBytes(desc.quantType)
: MARTe::UDPSTypeCodeByteSize(desc.typeCode);
if (wireElemBytes == 0u) { return; }
uint32 elemsToRead = ((pm == UDPS_PUBLISH_ACCUMULATE) && (numElements == 1u))
? numSamples
: numElements;
/* Accumulate mode batches one full snapshot (all elements) per RT
* cycle for every signal (scalar or array) — see UDPStreamer's
* SerializeAccumulated. HI-1: 64-bit multiply to avoid overflow on
* attacker-controlled numSamples. */
uint64 elemsToRead64 = (pm == UDPS_PUBLISH_ACCUMULATE)
? (numElements64 * static_cast<uint64>(numSamples))
: numElements64;
if (elemsToRead64 > 0x100000u) { return; /* sanity cap: 1M elements */ }
uint32 elemsToRead = static_cast<uint32>(elemsToRead64);
if (off + elemsToRead * wireElemBytes > size) { return; }
/* HI-1: 64-bit bounds check to prevent uint32 multiply overflow */
uint64 bytesNeeded = static_cast<uint64>(off) +
static_cast<uint64>(elemsToRead) *
static_cast<uint64>(wireElemBytes);
if (bytesNeeded > static_cast<uint64>(size)) { return; }
sigOff[s] = off;
sigElems[s] = elemsToRead;
off += elemsToRead * wireElemBytes;
off += static_cast<uint32>(elemsToRead * wireElemBytes);
}
/* The decode scratch is sized at CONFIG time to the largest per-signal
@@ -389,8 +486,10 @@ void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size) {
for (uint32 s = 0u; s < nSigs; s++) {
const UDPSSignalDescriptor &desc = descs[s];
uint32 numElements = desc.numRows * desc.numCols;
if (numElements == 0u) { numElements = 1u; }
uint64 ne64 = static_cast<uint64>(desc.numRows) *
static_cast<uint64>(desc.numCols);
if (ne64 == 0u) { ne64 = 1u; }
uint32 numElements = static_cast<uint32>(ne64);
const uint32 nElems = sigElems[s];
const bool isFirstLast = (numElements > 1u) &&
Source/Applications/StreamHub/UDPSourceSession.h
+57 -8
View File
@@ -154,6 +154,37 @@ public:
*/
void SetRingCapacities(uint32 temporal, uint32 scalar);
/**
* @brief Grow every ring so it can retain at least @p seconds of history.
*
* The required capacity is seconds × the rate measured from the ring
* itself (count / time span), because most sources advertise
* samplingRate = 0. Signals whose ring has not filled enough to measure a
* rate are left alone; the caller is expected to retry.
*
* @param seconds Retention target.
* @param maxPts Per-signal ceiling, so a multi-Msps source cannot be
* asked to allocate an unbounded amount of memory.
* @return true if at least one ring was enlarged.
*/
bool GrowRingsForSeconds(float64 seconds, uint32 maxPts);
/** @return Largest ring capacity currently allocated in this session. */
uint32 GetMaxRingCapacity() const;
/**
* @brief Newest timestamp this source has produced on its *own* clock, or
* 0 when it publishes no producer-timed signal (or has no data yet).
*
* Only signals that reference a time signal count: PACKET-timed signals
* are stamped on arrival and so live in the hub's wall-clock domain, not
* the producer's, even when they come from the very same source. A source
* free-running on its own clock sits seconds away from wall time and drifts,
* so anything waiting for a capture window to fill must compare against
* this, never clock_gettime().
*/
float64 ProducerNewestTime() const;
/**
* @brief Attach the (shared) hub trigger engine.
* Every decoded sample of the trigger's configured signal — resolved
@@ -241,24 +272,42 @@ private:
* signal @p tIdx given the first decoded timer value @p timer0S of the
* current packet and the arrival wall time @p wallNowS.
*
* Re-anchors the offset (offset = wallNowS − timer0S) when (a) it is the
* first packet, (b) the source clock jumped backward versus the previous
* packet (a looping/rewinding producer), or (c) the computed wall time has
* drifted past kRecalibThresholdS from the true arrival wall time.
* Snaps the offset to wallNowS − timer0S only when there is a genuine
* discontinuity in the source: the first packet, or a backward jump of the
* source clock (a looping/rewinding producer).
*
* A source that free-runs on its own clock also *drifts* against wall time,
* without any discontinuity. Snapping that away would shift the whole
* published timeline in one step and so tear a hole of exactly the drift
* into a stream that is in fact continuous, which is worse than the drift
* itself. Past kRecalibThresholdS the offset is therefore slewed instead:
* nudged towards wall time by at most kMaxSlewFraction of the packet's own
* duration, so the seam can never exceed a fraction of one packet.
*
* @return the calibration offset to add to timer-seconds for this signal.
*/
inline float64 CalibrateTimeSignal(uint32 tIdx, float64 timer0S,
float64 wallNowS) {
static const float64 kRecalibThresholdS = 2.0;
static const float64 kMaxSlewFraction = 0.1;
const bool reset = timeSigLastValid_[tIdx] &&
(timer0S < timeSigLastTimerS_[tIdx]);
const float64 drift = (timeSigCalib_[tIdx] + timer0S) - wallNowS;
const float64 absDrift = (drift < 0.0) ? -drift : drift;
if ((!timeSigCalibValid_[tIdx]) || reset ||
(absDrift > kRecalibThresholdS)) {
if ((!timeSigCalibValid_[tIdx]) || reset) {
timeSigCalib_[tIdx] = wallNowS - timer0S;
timeSigCalibValid_[tIdx] = true;
}
else {
const float64 drift = (timeSigCalib_[tIdx] + timer0S) - wallNowS;
const float64 absDrift = (drift < 0.0) ? -drift : drift;
if (absDrift > kRecalibThresholdS) {
const float64 pktSpan = timer0S - timeSigLastTimerS_[tIdx];
const float64 maxStep = pktSpan * kMaxSlewFraction;
float64 step = -drift;
if (step > maxStep) { step = maxStep; }
if (step < -maxStep) { step = -maxStep; }
timeSigCalib_[tIdx] += step;
}
}
timeSigLastTimerS_[tIdx] = timer0S;
timeSigLastValid_[tIdx] = true;
return timeSigCalib_[tIdx];
Source/Applications/StreamHub/WSServer.cpp
+129 -17
View File
@@ -8,6 +8,7 @@
#include "SHA1.h"
#include "Base64.h"
#include "AdvancedErrorManagement.h"
#include "Select.h"
#include "Sleep.h"
#include "Threads.h"
#include "TimeoutType.h"
@@ -57,8 +58,10 @@ static const char *FindSubstr(const char *s, const char *pattern) {
WSServer::WSServer()
: numClients(0u),
liveReadThreads(0u),
callback(static_cast<WSCommandCallback *>(0)),
running(false),
numAllowedOrigins(0u),
acceptTid(MARTe::InvalidThreadIdentifier) {
for (uint32 i = 0u; i < WS_MAX_CLIENTS; i++) {
@@ -66,6 +69,20 @@ WSServer::WSServer()
clients[i].active = false;
clients[i].readTid = MARTe::InvalidThreadIdentifier;
}
for (uint32 i = 0u; i < WS_MAX_ORIGINS; i++) {
allowedOrigins[i][0] = '\0';
}
}
bool WSServer::AddAllowedOrigin(const char *origin) {
if ((origin == static_cast<const char *>(0)) || (origin[0] == '\0')) {
return false;
}
if (numAllowedOrigins >= WS_MAX_ORIGINS) { return false; }
if (strlen(origin) >= WS_MAX_ORIGIN_LEN) { return false; }
strcpy(allowedOrigins[numAllowedOrigins], origin);
numAllowedOrigins++;
return true;
}
WSServer::~WSServer() {
@@ -104,9 +121,9 @@ bool WSServer::Start(uint16 port, WSCommandCallback *cb) {
bool WSServer::Stop() {
if (!running) { return true; }
running = false;
Sleep::MSec(200u);
/* Close all client connections — their read threads will exit on error */
/* Close all client connections — their read threads wake out of select()
* and unwind through FreeSlot. */
(void) clientsMutex.FastLock();
for (uint32 i = 0u; i < WS_MAX_CLIENTS; i++) {
if (clients[i].active && (clients[i].sock != static_cast<BasicTCPSocket *>(0))) {
@@ -114,10 +131,23 @@ bool WSServer::Stop() {
}
}
clientsMutex.FastUnLock();
Sleep::MSec(200u);
/* The accept loop polls WaitConnection with a 500 ms timeout, so it is out
* of the listener by now. */
Sleep::MSec(600u);
tcpListener.Close();
Sleep::MSec(100u);
/* Wait for the read threads: they hold pointers to the sockets freed
* below. Bounded — leaking a socket at exit beats deleting one that a
* wedged thread is still reading from. */
static const uint32 kReadJoinMs = 3000u;
for (uint32 waited = 0u; waited < kReadJoinMs; waited += 20u) {
(void) clientsMutex.FastLock();
const uint32 live = liveReadThreads;
clientsMutex.FastUnLock();
if (live == 0u) { break; }
Sleep::MSec(20u);
}
/* Free any remaining slots */
(void) clientsMutex.FastLock();
@@ -170,6 +200,10 @@ void WSServer::AcceptLoop() {
}
/* Start per-client read thread */
(void) clientsMutex.FastLock();
liveReadThreads++;
clientsMutex.FastUnLock();
ClientThreadArg *arg = new ClientThreadArg();
arg->srv = this;
arg->slot = slot;
@@ -199,6 +233,68 @@ bool WSServer::UpgradeHTTP(BasicTCPSocket *sock) {
if (strstr(hdrBuf, "\r\n\r\n") != static_cast<char *>(0)) { break; }
}
/* Origin validation (CSWSH / CSRF defence, RFC 6455 §10.2).
* If an Origin header is present it must either be on the configured
* allowlist or its host must match the Host header (same-origin).
* Non-browser clients (no Origin) are allowed. */
const char *originHdr = FindSubstr(hdrBuf, "Origin:");
if (originHdr != static_cast<const char *>(0)) {
originHdr += 7; /* skip "Origin:" */
while (*originHdr == ' ') { originHdr++; }
/* Full origin value "scheme://host[:port]", for the allowlist. */
char originFull[WS_MAX_ORIGIN_LEN];
uint32 ofLen = 0u;
while ((originHdr[ofLen] != '\r') && (originHdr[ofLen] != '\n') &&
(originHdr[ofLen] != '\0') && (ofLen < (WS_MAX_ORIGIN_LEN - 1u))) {
originFull[ofLen] = originHdr[ofLen];
ofLen++;
}
originFull[ofLen] = '\0';
bool allowed = false;
for (uint32 i = 0u; (i < numAllowedOrigins) && !allowed; i++) {
if (strcmp(originFull, allowedOrigins[i]) == 0) { allowed = true; }
}
/* Extract the host part of Origin: "scheme://host[:port]" */
char originHost[256];
uint32 ohLen = 0u;
const char *op = originHdr;
/* Skip scheme:// */
const char *schemeEnd = strstr(op, "://");
if (schemeEnd != static_cast<const char *>(0)) { op = schemeEnd + 3; }
while (*op != '\r' && *op != '\n' && *op != '\0' &&
*op != '/' && ohLen < 255u) {
originHost[ohLen++] = *op++;
}
originHost[ohLen] = '\0';
/* Extract Host header value */
const char *hostHdr = FindSubstr(hdrBuf, "Host:");
if (!allowed && (hostHdr != static_cast<const char *>(0))) {
hostHdr += 5; /* skip "Host:" */
while (*hostHdr == ' ') { hostHdr++; }
char hostVal[256];
uint32 hvLen = 0u;
while (*hostHdr != '\r' && *hostHdr != '\n' &&
*hostHdr != '\0' && hvLen < 255u) {
hostVal[hvLen++] = *hostHdr++;
}
hostVal[hvLen] = '\0';
if (strcmp(originHost, hostVal) != 0) {
/* Cross-origin — reject the upgrade */
const char *forbidden =
"HTTP/1.1 403 Forbidden\r\n"
"Content-Type: text/plain\r\n"
"Connection: close\r\n"
"\r\nOrigin not allowed\r\n";
uint32 forbLen = static_cast<uint32>(strlen(forbidden));
(void) sock->Write(forbidden, forbLen);
return false;
}
}
}
/* Find Sec-WebSocket-Key */
const char *keyHdr = FindSubstr(hdrBuf, "Sec-WebSocket-Key:");
if (keyHdr == static_cast<const char *>(0)) { return false; }
@@ -246,29 +342,37 @@ void WSServer::ClientReadLoop(uint32 slotIdx) {
WSClientSlot &slot = clients[slotIdx];
BasicTCPSocket *sock = slot.sock;
/* Receive buffer (grows as needed by simple state machine) */
static const uint32 kRecvBuf = WS_MAX_RECV_PAYLOAD + 14u;
/* Receive buffer: WS_MAX_RECV_PAYLOAD + max header (14: 2 + 8 ext-length +
* 4 mask) + 1 spare byte for in-place NUL-termination of the payload. */
static const uint32 kRecvBuf = WS_MAX_RECV_PAYLOAD + 14u + 1u;
uint8 *buf = new uint8[kRecvBuf];
uint32 filled = 0u;
while (running && slot.active) {
/* Read more bytes (with short timeout so we can check running) */
uint32 want = kRecvBuf - filled;
if (want == 0u) {
/* Buffer full — discard old frame (shouldn't happen with reasonable clients) */
filled = 0u;
continue;
}
bool ok = sock->Read(reinterpret_cast<char *>(buf + filled), want,
TimeoutType(500u));
if (!ok) {
/* Timeout or error — check running and retry */
if (!running) { break; }
if (want == kRecvBuf) {
/* Zero bytes read — connection likely closed */
break;
}
continue;
/* Wait for readability before reading. BasicTCPSocket::Read reports a
* timeout and a closed peer identically (false, zero bytes), so polling
* it on its own cannot end the loop: once the client goes away recv
* returns immediately and forever, and the thread spins at 100% CPU
* until it starves the rest of the hub. select() tells the two apart —
* readable followed by no data is end of stream. A wait consumes the
* handle set, hence a fresh Select each pass. */
MARTe::Select sel;
if (!sel.AddReadHandle(*sock)) { break; }
const MARTe::int32 ready = sel.WaitUntil(TimeoutType(500u));
if (ready == 0) { continue; } /* idle client — recheck running */
if (ready < 0) { break; } /* socket closed or errored */
/* Readable: this returns at once, and only fails at end of stream. */
if (!sock->Read(reinterpret_cast<char *>(buf + filled), want,
TimeoutType(500u))) {
break;
}
filled += want;
@@ -336,6 +440,10 @@ client_done:
callback->OnWSClientDisconnected();
}
FreeSlot(slotIdx);
(void) clientsMutex.FastLock();
if (liveReadThreads > 0u) { liveReadThreads--; }
clientsMutex.FastUnLock();
}
/*---------------------------------------------------------------------------*/
@@ -431,6 +539,9 @@ uint32 WSServer::AllocSlot(BasicTCPSocket *sock) {
void WSServer::FreeSlot(uint32 idx) {
if (idx >= WS_MAX_CLIENTS) { return; }
/* HI-5: acquire writeMutex before modifying active/sock to prevent
* use-after-free when BroadcastText/BroadcastBinary are iterating. */
(void) clients[idx].writeMutex.FastLock();
(void) clientsMutex.FastLock();
if (clients[idx].active) {
clients[idx].active = false;
@@ -442,6 +553,7 @@ void WSServer::FreeSlot(uint32 idx) {
if (numClients > 0u) { numClients--; }
}
clientsMutex.FastUnLock();
clients[idx].writeMutex.FastUnLock();
}
} /* namespace StreamHub */
Source/Applications/StreamHub/WSServer.h
+25 -1
View File
@@ -34,6 +34,12 @@ static const uint32 WS_MAX_RECV_PAYLOAD = 65536u;
/** Maximum WebSocket frame payload we will send (data frames can be large). */
static const uint32 WS_MAX_SEND_PAYLOAD = 4u * 1024u * 1024u; /* 4 MiB */
/** Maximum entries in the Origin allowlist. */
static const uint32 WS_MAX_ORIGINS = 8u;
/** Maximum length of one allowlisted Origin ("scheme://host[:port]"). */
static const uint32 WS_MAX_ORIGIN_LEN = 128u;
/**
* @brief Callback interface — implemented by StreamHub.
*/
@@ -77,6 +83,20 @@ public:
*/
bool Start(uint16 port, WSCommandCallback *cb);
/**
* @brief Add an Origin that is accepted for the WebSocket upgrade.
*
* With an empty allowlist (the default) only same-origin requests pass:
* the Origin's host must equal the Host header, which excludes the usual
* deployment where the SPA is served by a separate web server on another
* port. Add that server's origin (e.g. "http://localhost:8080") to allow
* it. Requests without an Origin header (non-browser clients) always pass.
*
* @param origin "scheme://host[:port]", compared verbatim.
* @return false if the allowlist is full or the string is too long.
*/
bool AddAllowedOrigin(const char *origin);
/**
* @brief Stop accept thread; close all client connections; close listener.
*/
@@ -119,11 +139,15 @@ private:
BasicTCPSocket tcpListener;
WSClientSlot clients[WS_MAX_CLIENTS];
uint32 numClients;
mutable FastPollingMutexSem clientsMutex; ///< Protects numClients and clients[] array
uint32 liveReadThreads; ///< Read threads not yet unwound; Stop() waits on it
mutable FastPollingMutexSem clientsMutex; ///< Protects numClients, liveReadThreads and clients[] array
WSCommandCallback *callback;
volatile bool running;
char allowedOrigins[WS_MAX_ORIGINS][WS_MAX_ORIGIN_LEN];
uint32 numAllowedOrigins;
MARTe::ThreadIdentifier acceptTid;
};
Source/Components/DataSources/UDPStreamer/UDPStreamer.cpp
+1120 -1135
View File
File diff suppressed because it is too large Load Diff
Source/Components/DataSources/UDPStreamer/UDPStreamer.h
+13 -13
View File
@@ -103,7 +103,7 @@ struct UDPStreamerSignalInfo {
uint32 srcByteSize; /**< Bytes in MARTe2 memory */
uint32 wireByteSize; /**< Bytes on the wire (may differ when quantized) */
uint32 bufferOffset; /**< Byte offset in the flat MemoryDataSourceI memory buffer */
bool accumulated; /**< True when this scalar was expanded to flushCount elements in Auto accumulation mode */
bool accumulated; /**< True when this signal is batched (one snapshot per RT cycle) in Accumulate mode */
};
/**
@@ -140,16 +140,17 @@ struct UDPStreamerSignalInfo {
* fragmented into multiple datagrams if payload exceeds MaxPayloadSize).
*
* @par Top-level configuration parameters
* | Parameter | Type | Default | Description |
* |-----------------|---------|---------|-------------|
* | Port | uint16 | 44500 | TCP control port (multicast) or UDP server port (unicast). Values ≤ 1024 produce a warning. |
* | MulticastGroup | string | *(absent)* | **Enables multicast mode.** IPv4 multicast address, e.g. `"239.0.0.1"`. Must be in 224.0.0.0/4. Absent or empty = unicast. |
* | DataPort | uint16 | Port+1 | UDP port for multicast DATA datagrams. Ignored in unicast mode. Must be non-zero and differ from Port. |
* | MaxPayloadSize | uint32 | 1400 | Maximum bytes of signal payload per UDP datagram (excluding the 17-byte header). Larger signals are fragmented. |
* | PublishingMode | string | Strict | `Strict`: send one packet every Synchronise() call. `Auto`: rate-limited; flush only when MinRefreshRate interval has elapsed. |
* | MinRefreshRate | float64 | — | Required when PublishingMode = Auto. Flush frequency in Hz (e.g. 120.0). |
* | MaxBatchSize | uint32 | 1 | Optional when PublishingMode = Auto. Number of RT cycles to accumulate before flushing one packet. Scalar signals are expanded to arrays of MaxBatchSize elements; the first scalar with Unit="us" or "ns" is auto-promoted as the per-sample FullArray timestamp reference for all other scalars. When omitted or 1, the most-recent single value is sent at MinRefreshRate. |
* | CPUMask | uint32 | 0xFFFFFFFF | CPU affinity bitmask for the background thread. |
* | Parameter | Type | Default | Description |
* |-----------------|---------|------------------|-------------|
* | Port | uint16 | 44500 | TCP control port (multicast) or UDP server port (unicast). Values ≤ 1024 produce a warning. |
* | MulticastGroup | string | *(absent)* | **Enables multicast mode.** IPv4 multicast address, e.g. `"239.0.0.1"`. Must be in 224.0.0.0/4. Absent or empty = unicast. |
* | Interface | string | *(absent)* | Multicast binded interface **ONLY FOR MULTICAST** |
* | DataPort | uint16 | Port+1 | UDP port for multicast DATA datagrams. Ignored in unicast mode. Must be non-zero and differ from Port. |
* | MaxPayloadSize | uint32 | 1400 | Maximum bytes of signal payload per UDP datagram (excluding the 17-byte header). Larger signals are fragmented. |
* | PublishingMode | string | Strict | `Strict`: send one packet every Synchronise() call. `Auto`: rate-limited; flush only when MinRefreshRate interval has elapsed. |
* | MinRefreshRate | float64 | — | Required when PublishingMode = Auto. Flush frequency in Hz (e.g. 120.0). |
* | MaxBatchSize | uint32 | 1 | Optional when PublishingMode = Auto. Number of RT cycles to accumulate before flushing one packet. Scalar signals are expanded to arrays of MaxBatchSize elements; the first scalar with Unit="us" or "ns" is auto-promoted as the per-sample FullArray timestamp reference for all other scalars. When omitted or 1, the most-recent single value is sent at MinRefreshRate. |
* | CPUMask | uint32 | 0xFFFFFFFF | CPU affinity bitmask for the background thread. |
* | StackSize | uint32 | (MARTe2 default) | Stack size in bytes for the background thread. |
*
* @par Per-signal configuration parameters
@@ -358,8 +359,7 @@ private:
uint64 flushPeriodTicks; /**< HRT ticks per flush interval (computed from minRefreshRate) */
/* Accumulate mode — dynamic batch parameters */
uint32 maxBatchCount; /**< Max snapshots that fit in MaxPayloadSize (Accumulate) */
uint32 singleCycleWireBytes; /**< Wire bytes for all accumulated signals per snapshot */
uint32 fixedWireBytes; /**< Wire bytes for non-accumulated signals (arrays, once per packet) */
uint32 singleCycleWireBytes; /**< Wire bytes for ALL signals (scalar and array) per snapshot */
volatile uint64 lastPublishTs; /**< HRT counter of last successful flush (Accumulate mode) */
uint8 *accumBuffer; /**< Heap: [maxBatchCount × totalSrcBytes] linear fill */
uint64 *accumTimestamps; /**< Heap: [maxBatchCount] HRT counter per snapshot */
Source/Components/DataSources/UDPStreamer/depends.x86-linux
+23 -30
View File
@@ -1,48 +1,48 @@
../../../..//Build/x86-linux/Components/DataSources/UDPStreamer/UDPStreamer.o: UDPStreamer.cpp \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/AdvancedErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassProperties.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/ErrorType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Architecture/x86_gcc/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/FractionalInteger.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitBoolean.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TemplateParametersVerificator.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitRange.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamString.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HeapManager.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HeapI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/ErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/ErrorInformation.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/ErrorType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitBoolean.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitRange.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StreamI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/TimeoutType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HighResolutionTimer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/TimeStamp.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Architecture/x86_gcc/HighResolutionTimerA.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/HighResolutionTimer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../HighResolutionTimer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/HighResolutionTimerCalibrator.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/TimeStamp.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamMemoryReference.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/CharBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitBoolean.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/ErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/BufferedStreamI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/TimeoutType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/AnyType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassProperties.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/FractionalInteger.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassRegistryDatabase.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GlobalObjectsDatabase.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GlobalObjectI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/StandardHeap.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../HeapI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../ErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../Generic/StandardHeap_Generic.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/FastPollingMutexSem.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Atomic.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Architecture/x86_gcc/AtomicA.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/ErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Sleep.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassRegistryItem.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CString.h \
@@ -53,7 +53,6 @@
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BasicType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/FractionalInteger.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitRange.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitBoolean.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/ZeroTerminatedArray.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/LinkedListable.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/LinkedListHolderT.h \
@@ -70,17 +69,18 @@
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StaticListHolder.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Matrix.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Vector.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HeapManager.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HeapI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/MemoryOperationsHelper.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/FormatDescriptor.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/IOBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/CharBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StreamI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StringHelper.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamStringIOBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/AdvancedErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassProperties.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamMemoryReference.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L4Configuration/ConfigurationDatabase.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L4Configuration/AnyObject.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/Object.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StringHelper.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/StructuredDataI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/AnyType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/CLASSREGISTER.h \
@@ -104,8 +104,6 @@
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/StructuredDataI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L4Configuration/TypeConversion.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Vector.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamString.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamStringIOBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedThreadI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedServiceMethodBinderI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/ExecutionInfo.h \
@@ -116,18 +114,12 @@
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitSet.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedServiceMethodBinderT.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HighResolutionTimer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/MemoryMapSynchronisedOutputBroker.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/MemoryMapOutputBroker.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/MemoryMapBroker.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/BrokerI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/DataSourceI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/StatefulI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/ExecutableI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Sleep.h \
UDPStreamer.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedServiceMethodBinderI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L1Portability/EventSem.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/MemoryDataSourceI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/DataSourceI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/StatefulI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/SingleThreadService.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedServiceI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedThreadI.h \
@@ -141,5 +133,6 @@
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HandleI.h \
/home/martino/workspace/MARTe2/Source/Core/FileSystem/L1Portability/Environment/Linux/SocketCore.h \
/home/martino/workspace/MARTe2/Source/Core/FileSystem/L1Portability/BasicUDPSocket.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Sleep.h \
/home/martino/workspace/MARTe2/Source/Core/FileSystem/L1Portability/InternetHost.h \
../../../..//Common/UDP/UDPSProtocol.h
Source/Components/DataSources/UDPStreamer/dependsRaw.x86-linux
+23 -30
View File
@@ -1,48 +1,48 @@
UDPStreamer.o: UDPStreamer.cpp \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/AdvancedErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassProperties.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/ErrorType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Architecture/x86_gcc/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/FractionalInteger.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitBoolean.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TemplateParametersVerificator.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitRange.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamString.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HeapManager.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HeapI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/ErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/ErrorInformation.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/ErrorType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitBoolean.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitRange.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StreamI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/TimeoutType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HighResolutionTimer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/TimeStamp.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Architecture/x86_gcc/HighResolutionTimerA.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/HighResolutionTimer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../HighResolutionTimer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/HighResolutionTimerCalibrator.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/TimeStamp.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamMemoryReference.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/CharBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitBoolean.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/ErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/BufferedStreamI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/TimeoutType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/AnyType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassProperties.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/FractionalInteger.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassRegistryDatabase.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GlobalObjectsDatabase.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GlobalObjectI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/StandardHeap.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../HeapI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../ErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../../GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/../Generic/StandardHeap_Generic.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/FastPollingMutexSem.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Atomic.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Architecture/x86_gcc/AtomicA.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/ErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Sleep.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassRegistryItem.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CString.h \
@@ -53,7 +53,6 @@ UDPStreamer.o: UDPStreamer.cpp \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BasicType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/FractionalInteger.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitRange.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitBoolean.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/ZeroTerminatedArray.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/LinkedListable.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/LinkedListHolderT.h \
@@ -70,17 +69,18 @@ UDPStreamer.o: UDPStreamer.cpp \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StaticListHolder.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Matrix.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Vector.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HeapManager.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HeapI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/MemoryOperationsHelper.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/FormatDescriptor.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/IOBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/CharBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StreamI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StringHelper.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamStringIOBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/AdvancedErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassProperties.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamMemoryReference.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L4Configuration/ConfigurationDatabase.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L4Configuration/AnyObject.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/Object.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StringHelper.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/StructuredDataI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/AnyType.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/CLASSREGISTER.h \
@@ -104,8 +104,6 @@ UDPStreamer.o: UDPStreamer.cpp \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/StructuredDataI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L4Configuration/TypeConversion.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Vector.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamString.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamStringIOBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedThreadI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedServiceMethodBinderI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/ExecutionInfo.h \
@@ -116,18 +114,12 @@ UDPStreamer.o: UDPStreamer.cpp \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/BitSet.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedServiceMethodBinderT.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HighResolutionTimer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/MemoryMapSynchronisedOutputBroker.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/MemoryMapOutputBroker.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/MemoryMapBroker.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/BrokerI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/DataSourceI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/StatefulI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/ExecutableI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Sleep.h \
UDPStreamer.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedServiceMethodBinderI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L1Portability/EventSem.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/MemoryDataSourceI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/DataSourceI.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L5GAMs/StatefulI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/SingleThreadService.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedServiceI.h \
/home/martino/workspace/MARTe2/Source/Core/Scheduler/L3Services/EmbeddedThreadI.h \
@@ -141,5 +133,6 @@ UDPStreamer.o: UDPStreamer.cpp \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/HandleI.h \
/home/martino/workspace/MARTe2/Source/Core/FileSystem/L1Portability/Environment/Linux/SocketCore.h \
/home/martino/workspace/MARTe2/Source/Core/FileSystem/L1Portability/BasicUDPSocket.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Sleep.h \
/home/martino/workspace/MARTe2/Source/Core/FileSystem/L1Portability/InternetHost.h \
../../../..//Common/UDP/UDPSProtocol.h
Source/Components/DataSources/UDPStreamerClient/UDPStreamerClient.cpp
+36 -3
View File
@@ -55,6 +55,12 @@ static const uint16 UDPS_CLIENT_DEFAULT_DP_OFFSET = 1u;
/** Default max payload per UDP datagram (bytes). */
static const uint32 UDPS_CLIENT_DEFAULT_MAX_PAYLOAD = 1400u;
/** Default unicast keepalive interval (seconds); 0 disables. */
static const uint32 UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S = 15u;
/** Default silence timeout before reconnect (seconds); sub-second values allowed. */
static const float32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 1.0f;
/** Bytes prepended to each DATA payload for the HRT packet timestamp. */
static const uint32 UDPS_CLIENT_TIMESTAMP_BYTES = 8u;
@@ -129,6 +135,8 @@ UDPStreamerClient::UDPStreamerClient() :
serverAddress = UDPS_CLIENT_DEFAULT_ADDR;
port = UDPS_CLIENT_DEFAULT_PORT;
maxPayloadSize = UDPS_CLIENT_DEFAULT_MAX_PAYLOAD;
keepAliveInterval = UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S;
silenceTimeout = UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S;
cpuMask = 0xFFFFFFFFu;
stackSize = THREADS_DEFAULT_STACKSIZE;
dataPort = UDPS_CLIENT_DEFAULT_PORT + UDPS_CLIENT_DEFAULT_DP_OFFSET;
@@ -201,6 +209,18 @@ bool UDPStreamerClient::Initialise(StructuredDataI &data) {
}
}
if (ok) {
if (!data.Read("KeepAliveInterval", keepAliveInterval)) {
keepAliveInterval = UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S;
}
}
if (ok) {
if (!data.Read("SilenceTimeout", silenceTimeout)) {
silenceTimeout = UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S;
}
}
if (ok) {
if (!data.Read("CPUMask", cpuMask)) {
cpuMask = 0xFFFFFFFFu;
@@ -224,10 +244,14 @@ bool UDPStreamerClient::Initialise(StructuredDataI &data) {
dp = port + UDPS_CLIENT_DEFAULT_DP_OFFSET;
}
dataPort = dp;
StreamString ifaceStr = "";
(void) data.Read("Interface", ifaceStr);
multicastInterface = ifaceStr;
REPORT_ERROR(ErrorManagement::Information,
"Multicast mode: group=%s, server=%s, controlPort=%u, dataPort=%u.",
"Multicast mode: group=%s, server=%s, controlPort=%u, dataPort=%u, interface=%s.",
multicastGroup.Buffer(), serverAddress.Buffer(),
static_cast<uint32>(port), static_cast<uint32>(dataPort));
static_cast<uint32>(port), static_cast<uint32>(dataPort),
(multicastInterface.Size() > 0u) ? multicastInterface.Buffer() : "default");
}
else {
useMulticast = false;
@@ -243,8 +267,13 @@ bool UDPStreamerClient::Initialise(StructuredDataI &data) {
if (ok && useMulticast) {
ok = cdb.Write("MulticastGroup", multicastGroup);
if (ok) { ok = cdb.Write("DataPort", static_cast<uint32>(dataPort)); }
if (ok && (multicastInterface.Size() > 0u)) {
ok = cdb.Write("Interface", multicastInterface);
}
}
if (ok) { ok = cdb.Write("MaxPayloadSize", maxPayloadSize); }
if (ok) { ok = cdb.Write("KeepAliveInterval", keepAliveInterval); }
if (ok) { ok = cdb.Write("SilenceTimeout", silenceTimeout); }
if (ok) { ok = cdb.Write("CPUMask", cpuMask); }
if (ok) { ok = cdb.Write("StackSize", stackSize); }
if (ok) { ok = cdb.MoveToRoot(); }
@@ -517,7 +546,11 @@ void UDPStreamerClient::DecodeSnapshot(const uint8 *payload, uint32 size,
const bool accScalar = (publishMode == UDPS_PUBLISH_ACCUMULATE) && (ne == 1u);
const uint32 elemsToRead = accScalar ? numSamples : ne;
if ((off + (elemsToRead * wireElemBytes)) > size) { return; }
/* HI-1: 64-bit bounds check to prevent uint32 multiply overflow */
uint64 bytesNeeded = static_cast<uint64>(off) +
static_cast<uint64>(elemsToRead) *
static_cast<uint64>(wireElemBytes);
if (bytesNeeded > static_cast<uint64>(size)) { return; }
uint8 *d = dst + info.bufferOffset;
Source/Components/DataSources/UDPStreamerClient/UDPStreamerClient.h
+6 -3
View File
@@ -174,11 +174,14 @@ private:
StreamString serverAddress; /**< Server IP address. */
uint16 port; /**< Server port. */
uint32 maxPayloadSize; /**< Max payload bytes per datagram. */
uint32 keepAliveInterval; /**< Seconds between unicast keepalive ACKs (0 disables). */
float32 silenceTimeout; /**< Seconds of no data before reconnect (sub-second allowed, 0 disables). */
uint32 cpuMask; /**< Background thread CPU affinity. */
uint32 stackSize; /**< Background thread stack size. */
StreamString multicastGroup; /**< Multicast group IP; empty = unicast. */
uint16 dataPort; /**< UDP port for DATA datagrams (multicast). */
bool useMulticast; /**< True when MulticastGroup is set. */
StreamString multicastGroup; /**< Multicast group IP; empty = unicast. */
StreamString multicastInterface; /**< Local IPv4 address for multicast join; empty = INADDR_ANY. */
uint16 dataPort; /**< UDP port for DATA datagrams (multicast). */
bool useMulticast; /**< True when MulticastGroup is set. */
/* Signal metadata */
uint32 numSigs; /**< Number of signals. */
Source/Components/GAMs/TimeArrayGAM/TimeArrayGAM.cpp
+23 -2
View File
@@ -16,6 +16,10 @@ TimeArrayGAM::TimeArrayGAM() :
GAM(),
samplingRate(1000000.0),
anchorIsFirst(true),
anchorIsCont(false),
contStarted(false),
contOriginNs(0u),
contSamples(0u),
nElements(0u),
inputTime(NULL_PTR(uint32 *)),
outputBuf(NULL_PTR(uint64 *)) {
@@ -42,9 +46,12 @@ bool TimeArrayGAM::Initialise(StructuredDataI &data) {
else if (anchor == "LastSample") {
anchorIsFirst = false;
}
else if (anchor == "Continuous") {
anchorIsCont = true;
}
else {
REPORT_ERROR(ErrorManagement::InitialisationError,
"TimeArrayGAM: Anchor must be 'FirstSample' or 'LastSample'.");
"TimeArrayGAM: Anchor must be 'FirstSample', 'LastSample' or 'Continuous'.");
ok = false;
}
}
@@ -88,7 +95,21 @@ bool TimeArrayGAM::Execute() {
/* Input is uint32 microseconds (LinuxTimer); convert to nanoseconds. */
uint64 anchorNs = static_cast<uint64>(*inputTime) * 1000u;
if (anchorIsFirst) {
if (anchorIsCont) {
/* Latch the timer once, then run off an internal sample counter so a
* lost RT cycle (LinuxTimer re-phases with counter += nCycles) cannot
* punch a hole into an otherwise contiguous sample stream. */
if (!contStarted) {
contOriginNs = anchorNs;
contStarted = true;
}
for (uint32 k = 0u; k < nElements; k++) {
outputBuf[k] = contOriginNs +
(contSamples + static_cast<uint64>(k)) * periodNs;
}
contSamples += static_cast<uint64>(nElements);
}
else if (anchorIsFirst) {
/* out[k] = anchorNs + k * periodNs */
for (uint32 k = 0u; k < nElements; k++) {
outputBuf[k] = anchorNs + static_cast<uint64>(k) * periodNs;
Source/Components/GAMs/TimeArrayGAM/TimeArrayGAM.h
+14 -1
View File
@@ -10,6 +10,15 @@
*
* Anchor = FirstSample: out[k] = input + k * period_us
* Anchor = LastSample: out[k] = input - (N-1-k) * period_us
* Anchor = Continuous: out[k] = input(first cycle) + (n + k) * period_us
*
* FirstSample/LastSample re-read the timer every cycle, so they propagate any
* cycle the RT thread loses: LinuxTimer re-phases (counter += nCycles) and the
* emitted time base jumps by a whole period while only one array of samples is
* produced, leaving a hole. Continuous anchors once and then advances an
* internal sample counter by N per cycle, which is what an acquisition card
* with its own clock does — use it when the data signal is itself contiguous
* (SineArrayGAM, for instance, never skips phase on a lost cycle).
*
* The resulting time array is suitable as the TimeSignal for a UDPStreamer signal
* configured with TimeMode = FullArray, providing exact per-sample timestamps.
@@ -19,7 +28,7 @@
* +TimeArrayGAM1 = {
* Class = TimeArrayGAM
* SamplingRate = 1000000.0 // Sample rate in Hz (must match data signal)
* Anchor = FirstSample // FirstSample (default) or LastSample
* Anchor = FirstSample // FirstSample (default), LastSample or Continuous
* InputSignals = {
* Time = { DataSource = DDB; Type = uint32 }
* }
@@ -54,6 +63,10 @@ public:
private:
float64 samplingRate; /**< Sample rate [Hz] */
bool anchorIsFirst; /**< true = FirstSample anchor, false = LastSample */
bool anchorIsCont; /**< true = Continuous anchor (internal sample counter) */
bool contStarted; /**< Continuous: origin has been latched */
uint64 contOriginNs; /**< Continuous: timer value latched on the first cycle */
uint64 contSamples; /**< Continuous: samples emitted so far */
uint32 nElements; /**< Number of output elements */
uint32 *inputTime; /**< Pointer to scalar input (microseconds, uint32 from LinuxTimer) */
uint64 *outputBuf; /**< Pointer to output array (nanoseconds, uint64) */
Source/Components/Interfaces/DebugService/DebugCore.h
+14 -10
View File
@@ -78,8 +78,9 @@ public:
bool Push(uint32 signalID, uint64 timestamp, void* data, uint32 size) {
uint32 packetSize = 4 + 8 + 4 + size; // ID + TS + Size + Data
uint32 read = readIndex;
uint32 write = writeIndex;
/* HI-9: use atomic loads for cross-thread index reads */
uint32 read = __atomic_load_n(&readIndex, __ATOMIC_ACQUIRE);
uint32 write = __atomic_load_n(&writeIndex, __ATOMIC_ACQUIRE);
uint32 available = 0;
if (read <= write) {
@@ -96,13 +97,15 @@ public:
WriteToBuffer(&tempWrite, &size, 4);
WriteToBuffer(&tempWrite, data, size);
writeIndex = tempWrite;
// HI-9: release store so data writes are visible before index update
__atomic_store_n(&writeIndex, tempWrite, __ATOMIC_RELEASE);
return true;
}
bool Pop(uint32 &signalID, uint64 &timestamp, void* dataBuffer, uint32 &size, uint32 maxSize) {
uint32 read = readIndex;
uint32 write = writeIndex;
/* HI-9: acquire-load writeIndex to see data written by Push */
uint32 read = __atomic_load_n(&readIndex, __ATOMIC_ACQUIRE);
uint32 write = __atomic_load_n(&writeIndex, __ATOMIC_ACQUIRE);
if (read == write) return false;
uint32 tempRead = read;
@@ -124,9 +127,9 @@ public:
// locate the next entry safely, so fall back to discarding everything
// to avoid reading garbage as sample headers on future Pop() calls.
if (tempSize >= bufferSize) {
readIndex = write; // corrupt ring — discard all
__atomic_store_n(&readIndex, write, __ATOMIC_RELEASE); // corrupt ring — discard all
} else {
readIndex = (tempRead + tempSize) % bufferSize;
__atomic_store_n(&readIndex, (tempRead + tempSize) % bufferSize, __ATOMIC_RELEASE);
}
return false;
}
@@ -137,13 +140,14 @@ public:
timestamp = tempTs;
size = tempSize;
readIndex = tempRead;
// HI-9: release-store readIndex after reading data
__atomic_store_n(&readIndex, tempRead, __ATOMIC_RELEASE);
return true;
}
uint32 Count() {
uint32 read = readIndex;
uint32 write = writeIndex;
uint32 read = __atomic_load_n(&readIndex, __ATOMIC_ACQUIRE);
uint32 write = __atomic_load_n(&writeIndex, __ATOMIC_ACQUIRE);
if (write >= read) return write - read;
return bufferSize - (read - write);
}
Source/Components/Interfaces/DebugService/DebugService.cpp
+129 -21
View File
@@ -13,6 +13,7 @@
#include "Threads.h"
#include "TimeoutType.h"
#include "UDPSProtocol.h"
#include <string.h>
namespace MARTe {
@@ -122,6 +123,12 @@ bool DebugService::Initialise(StructuredDataI &data) {
suppressTimeoutLogs = (suppress == 1u);
}
StreamString tempToken;
if (data.Read("AuthToken", tempToken)) {
authToken = tempToken;
}
clientAuthenticated = (authToken.Size() == 0u);
// Capture only the local subtree — do NOT call MoveToRoot() on the shared CDB.
(void)data.Copy(fullConfig);
@@ -281,6 +288,8 @@ ErrorManagement::ErrorType DebugService::Server(ExecutionInfo &info) {
cmdCountInWindow = 0u;
cmdWindowStartMs = nowMs;
lastDataTimeMs = nowMs;
/* CR-5: require auth if an AuthToken is configured. */
clientAuthenticated = (authToken.Size() == 0u);
}
} else {
if (nowMs - lastDataTimeMs > CLIENT_IDLE_TIMEOUT_MS) {
@@ -351,23 +360,101 @@ ErrorManagement::ErrorType DebugService::Server(ExecutionInfo &info) {
uint32 cmdLen = len;
command.Write(raw + lineStart, cmdLen);
// Dispatch via base HandleCommand, write response to socket.
StreamString out;
HandleCommand(command, out);
if (out.Size() > 0u) {
const char8 *wPtr = out.Buffer();
uint32 remaining = (uint32)out.Size();
lastDataTimeMs = (uint64)((float64)HighResolutionTimer::Counter() *
HighResolutionTimer::Period() * 1000.0);
while (remaining > 0u) {
uint32 wrote = remaining;
if (!activeClient->Write(wPtr, wrote) || wrote == 0u) {
break;
/* CR-5: Auth token gate. If an AuthToken is
* configured, the client must send
* "AUTH <token>" before any other command. */
if (authToken.Size() > 0u) {
const char8 *cmdPtr = command.Buffer();
if (cmdLen >= 5u &&
strncmp(cmdPtr, "AUTH ", 5u) == 0) {
const char8 *recvToken = cmdPtr + 5u;
uint32 recvLen = cmdLen - 5u;
/* Strip trailing \r if present */
if (recvLen > 0u &&
recvToken[recvLen - 1u] == '\r') {
recvLen--;
}
wPtr += wrote;
remaining -= wrote;
if (recvLen == authToken.Size() &&
strncmp(recvToken,
authToken.Buffer(),
recvLen) == 0) {
clientAuthenticated = true;
const char8 *okResp =
"OK AUTHENTICATED\n";
uint32 respSz =
static_cast<uint32>(
strlen(okResp));
(void) activeClient->Write(
okResp, respSz);
} else {
const char8 *badResp =
"ERR AUTH_FAILED\n";
uint32 respSz =
static_cast<uint32>(
strlen(badResp));
(void) activeClient->Write(
badResp, respSz);
}
} else if (!clientAuthenticated) {
const char8 *needAuth =
"ERR AUTH_REQUIRED\n";
uint32 respSz =
static_cast<uint32>(
strlen(needAuth));
(void) activeClient->Write(
needAuth, respSz);
} else {
// Dispatch via base HandleCommand,
// write response to socket.
StreamString out;
HandleCommand(command, out);
if (out.Size() > 0u) {
const char8 *wPtr = out.Buffer();
uint32 remaining =
(uint32)out.Size();
lastDataTimeMs =
(uint64)((float64)
HighResolutionTimer::Counter() *
HighResolutionTimer::Period() *
1000.0);
while (remaining > 0u) {
uint32 wrote = remaining;
if (!activeClient->Write(
wPtr, wrote) ||
wrote == 0u) {
break;
}
wPtr += wrote;
remaining -= wrote;
lastDataTimeMs =
(uint64)((float64)
HighResolutionTimer::Counter() *
HighResolutionTimer::Period() *
1000.0);
}
}
}
} else {
// No auth token configured — back-compat.
// Dispatch via base HandleCommand, write
// response to socket.
StreamString out;
HandleCommand(command, out);
if (out.Size() > 0u) {
const char8 *wPtr = out.Buffer();
uint32 remaining = (uint32)out.Size();
lastDataTimeMs = (uint64)((float64)HighResolutionTimer::Counter() *
HighResolutionTimer::Period() * 1000.0);
while (remaining > 0u) {
uint32 wrote = remaining;
if (!activeClient->Write(wPtr, wrote) || wrote == 0u) {
break;
}
wPtr += wrote;
remaining -= wrote;
lastDataTimeMs = (uint64)((float64)HighResolutionTimer::Counter() *
HighResolutionTimer::Period() * 1000.0);
}
}
}
}
@@ -433,6 +520,10 @@ ErrorManagement::ErrorType DebugService::Streamer(ExecutionInfo &info) {
// b) Drain traceBuffer — pack each sample into udpsDataPayload
bool anyData = false;
bool pendingInDrain[UDPS_MAX_SLOTS];
for (uint32 i = 0u; i < udpsNumSlots; i++) {
pendingInDrain[i] = false;
}
uint32 id, size;
uint64 ts;
uint8 udpsSampleBuf[UDPS_MAX_SAMPLE_BYTES];
@@ -441,6 +532,18 @@ ErrorManagement::ErrorType DebugService::Streamer(ExecutionInfo &info) {
// Find matching slot by internalID
for (uint32 i = 0u; i < udpsNumSlots; i++) {
if (udpsSlots[i].internalID == id) {
if (pendingInDrain[i]) {
// This slot already holds an unflushed sample from earlier
// in this same drain pass — flush it now instead of
// silently overwriting it, or lossless tracing would drop
// a real sample whenever the Streamer thread falls behind
// by more than one RT cycle.
FlushUdpsFrame();
for (uint32 j = 0u; j < udpsNumSlots; j++) {
pendingInDrain[j] = false;
}
anyData = false;
}
if ((udpsDataPayload != NULL_PTR(uint8 *)) &&
(8u + udpsSlots[i].wireOffset + udpsSlots[i].wireSize <= udpsDataPayloadSize)) {
uint32 copySize = size;
@@ -448,6 +551,7 @@ ErrorManagement::ErrorType DebugService::Streamer(ExecutionInfo &info) {
memcpy(udpsDataPayload + 8u + udpsSlots[i].wireOffset, udpsSampleBuf, copySize);
udpsSlots[i].everFilled = true;
}
pendingInDrain[i] = true;
anyData = true;
break;
}
@@ -455,18 +559,22 @@ ErrorManagement::ErrorType DebugService::Streamer(ExecutionInfo &info) {
}
// c) If we have data, stamp with HRT and send via udpsServer
if (anyData && udpsNumSlots > 0u && udpsDataPayload != NULL_PTR(uint8 *)) {
if (anyData) {
FlushUdpsFrame();
} else {
Sleep::MSec(1u);
}
return ErrorManagement::NoError;
}
void DebugService::FlushUdpsFrame() {
if (udpsNumSlots > 0u && udpsDataPayload != NULL_PTR(uint8 *)) {
uint64 hrt = HighResolutionTimer::Counter();
memcpy(udpsDataPayload, &hrt, 8u);
udpsPacketCounter++;
(void)udpsServer.SendData(udpsPacketCounter, udpsDataPayload, udpsDataPayloadSize);
}
if (!anyData) {
Sleep::MSec(1u);
}
return ErrorManagement::NoError;
}
// ---------------------------------------------------------------------------
Source/Components/Interfaces/DebugService/DebugService.h
+21
View File
@@ -66,6 +66,20 @@ private:
*/
bool SendUDPSConfig();
/**
* @brief Stamp the current udpsDataPayload with an HRT timestamp and send
* it as one UDPS DATA packet.
* @details Factored out of Streamer() so a single drain pass of
* traceBuffer can flush more than once per tick — see the
* pendingInDrain guard in Streamer(): without an eager flush, a
* slot that is written twice within the same drain pass (e.g.
* because the Streamer thread was briefly descheduled and two
* RT cycles' worth of samples piled up in traceBuffer) would
* silently overwrite-and-lose the first of the two samples,
* defeating lossless tracing.
*/
void FlushUdpsFrame();
// -----------------------------------------------------------------------
// TCP/UDP transport configuration
// -----------------------------------------------------------------------
@@ -76,6 +90,13 @@ private:
bool isServer;
bool suppressTimeoutLogs;
/** Optional authentication token (CR-5). If set (non-empty), the first
* command from a new TCP client must be "AUTH <token>". All other
* commands are rejected until the client authenticates. If empty
* (default), no authentication is required (back-compat). */
StreamString authToken;
bool clientAuthenticated;
BasicTCPSocket tcpServer;
UDPSServer udpsServer; ///< Handles fragmentation and multi-client sending
Source/Components/Interfaces/DebugService/DebugServiceBase.cpp
+56 -160
View File
@@ -181,6 +181,12 @@ static void BuildCDBFromContainer(ReferenceContainer *container,
}
}
/* HI-8: Guard against double-patching (e.g. two DebugService instances).
* Once the registry has been patched, subsequent PatchRegistry() calls are
* no-ops. Original builders are not saved/restored — the debug wrappers
* persist for the process lifetime (intentional for transparent debugging). */
static bool registryPatched = false;
static void PatchItemInternal(const char8 *originalName,
ObjectBuilder *debugBuilder) {
ClassRegistryItem *item =
@@ -215,6 +221,14 @@ DebugServiceBase::~DebugServiceBase() {
// ---------------------------------------------------------------------------
void DebugServiceBase::PatchRegistry() {
/* HI-8: skip if already patched (prevents double-patch leak when multiple
* DebugService instances are created). */
if (registryPatched) {
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"PatchRegistry: registry already patched — skipping (double-patch guard).");
return;
}
registryPatched = true;
PatchItemInternal("MemoryMapInputBroker",
new DebugMemoryMapInputBrokerBuilder());
PatchItemInternal("MemoryMapOutputBroker",
@@ -305,13 +319,20 @@ void DebugServiceBase::ProcessSignal(DebugSignalInfo *signalInfo, uint32 size,
return;
if (signalInfo->isForcing) {
uint32 nEl = signalInfo->numberOfElements;
/* HI-4: clamp size to forcedValue buffer to prevent OOB read */
uint32 forceSize = size;
if (forceSize > static_cast<uint32>(sizeof(signalInfo->forcedValue))) {
forceSize = static_cast<uint32>(sizeof(signalInfo->forcedValue));
}
if (nEl <= 1u) {
// Scalar — single memcpy.
memcpy(signalInfo->memoryAddress, signalInfo->forcedValue, size);
// Scalar — single memcpy (clamped to forcedValue bounds).
memcpy(signalInfo->memoryAddress, signalInfo->forcedValue, forceSize);
} else {
// Array — copy only the elements whose bit is set in forcedMask.
uint32 elemBytes = size / nEl;
for (uint32 e = 0u; e < nEl; e++) {
// HI-4: cap loop at 256 elements (forcedMask is 32 bytes = 256 bits).
uint32 elemBytes = forceSize / nEl;
uint32 nElCapped = (nEl > 256u) ? 256u : nEl;
for (uint32 e = 0u; e < nElCapped; e++) {
if (signalInfo->forcedMask[e >> 3u] & (uint8)(1u << (e & 7u))) {
memcpy((uint8 *)signalInfo->memoryAddress + e * elemBytes,
signalInfo->forcedValue + e * elemBytes,
@@ -1070,9 +1091,9 @@ void DebugServiceBase::InfoNode(const char8 *path, StreamString &out) {
Reference ref = ObjectRegistryDatabase::Instance()->Find(path);
out += "{";
if (ref.IsValid()) {
out += "\"Name\":\"";
out += "\"Name\": \"";
EscapeJson(ref->GetName(), out);
out += "\",\"Class\":\"";
out += "\", \"Class\": \"";
EscapeJson(ref->GetClassProperties()->GetName(), out);
out += "\"";
ConfigurationDatabase db;
@@ -1089,7 +1110,7 @@ void DebugServiceBase::InfoNode(const char8 *path, StreamString &out) {
if (TypeConvert(st, at)) {
out += "\"";
EscapeJson(cn, out);
out += "\":\"";
out += "\": \"";
EscapeJson(buf, out);
out += "\"";
if (i < nc - 1u)
@@ -1108,8 +1129,8 @@ void DebugServiceBase::InfoNode(const char8 *path, StreamString &out) {
DebugSignalInfo *s = signals[aliases[i].signalIndex];
const char8 *tn =
TypeDescriptor::GetTypeNameFromTypeDescriptor(s->type);
out.Printf("\"Name\":\"%s\",\"Class\":\"Signal\",\"Type\":\"%s\","
"\"ID\":%u",
out.Printf("\"Name\": \"%s\", \"Class\": \"Signal\", \"Type\": \"%s\", "
"\"ID\": %u",
s->name.Buffer(), tn ? tn : "Unknown", s->internalID);
enrichAlias = aliases[i].name;
found = true;
@@ -1120,29 +1141,39 @@ void DebugServiceBase::InfoNode(const char8 *path, StreamString &out) {
if (found)
EnrichWithConfig(enrichAlias.Buffer(), out);
else
out += "\"Error\":\"Object not found\"";
out += "\"Error\": \"Object not found\"";
}
out += "}\nOK INFO\n";
}
void DebugServiceBase::ListNodes(const char8 *path, StreamString &out) {
Reference ref =
bool isRoot =
(path == NULL_PTR(const char8 *) || StringHelper::Length(path) == 0 ||
StringHelper::Compare(path, "/") == 0)
? ObjectRegistryDatabase::Instance()
: ObjectRegistryDatabase::Instance()->Find(path);
StringHelper::Compare(path, "/") == 0);
// NOTE: ObjectRegistryDatabase::Instance() is a raw, long-lived singleton
// pointer that is never itself owned by a Reference. Wrapping it in a
// Reference here (as previously done via a ternary) would increment its
// reference count and then delete it when the local Reference goes out of
// scope, destroying the registry. Keep the root case as a raw pointer.
ReferenceContainer *rc = NULL_PTR(ReferenceContainer *);
Reference ref;
if (isRoot) {
rc = ObjectRegistryDatabase::Instance();
} else {
ref = ObjectRegistryDatabase::Instance()->Find(path);
if (ref.IsValid()) {
rc = dynamic_cast<ReferenceContainer *>(ref.operator->());
}
}
out.Printf("Nodes under %s:\n", path ? path : "/");
if (ref.IsValid()) {
ReferenceContainer *rc =
dynamic_cast<ReferenceContainer *>(ref.operator->());
if (rc != NULL_PTR(ReferenceContainer *)) {
uint32 n = rc->Size();
for (uint32 i = 0u; i < n; i++) {
Reference c = rc->Get(i);
if (c.IsValid()) {
out.Printf(" %s [%s]\n", c->GetName(),
c->GetClassProperties()->GetName());
}
if (rc != NULL_PTR(ReferenceContainer *)) {
uint32 n = rc->Size();
for (uint32 i = 0u; i < n; i++) {
Reference c = rc->Get(i);
if (c.IsValid()) {
out.Printf(" %s [%s]\n", c->GetName(),
c->GetClassProperties()->GetName());
}
}
} else {
@@ -1177,141 +1208,6 @@ void DebugServiceBase::RebuildConfigFromRegistry() {
RebuildTransportConfig();
}
// ---------------------------------------------------------------------------
// Tree export
// ---------------------------------------------------------------------------
uint32 DebugServiceBase::ExportTree(ReferenceContainer *container,
StreamString &json,
const char8 *pathPrefix) {
if (container == NULL_PTR(ReferenceContainer *))
return 0u;
uint32 size = container->Size();
uint32 valid = 0u;
for (uint32 i = 0u; i < size; i++) {
Reference child = container->Get(i);
if (!child.IsValid())
continue;
if (valid > 0u)
json += ",\n";
const char8 *cname = child->GetName();
if (cname == NULL_PTR(const char8 *))
cname = "unnamed";
StreamString cp;
if (pathPrefix != NULL_PTR(const char8 *))
cp.Printf("%s.%s", pathPrefix, cname);
else
cp = cname;
StreamString nj;
nj += "{\"Name\":\"";
EscapeJson(cname, nj);
nj += "\",\"Class\":\"";
EscapeJson(child->GetClassProperties()->GetName(), nj);
nj += "\"";
ReferenceContainer *inner =
dynamic_cast<ReferenceContainer *>(child.operator->());
DataSourceI *ds = dynamic_cast<DataSourceI *>(child.operator->());
GAM *gam = dynamic_cast<GAM *>(child.operator->());
if (inner != NULL_PTR(ReferenceContainer *) ||
ds != NULL_PTR(DataSourceI *) || gam != NULL_PTR(GAM *)) {
nj += ",\"Children\":[\n";
uint32 sc = 0u;
if (inner != NULL_PTR(ReferenceContainer *))
sc += ExportTree(inner, nj, cp.Buffer());
if (ds != NULL_PTR(DataSourceI *)) {
uint32 ns = ds->GetNumberOfSignals();
for (uint32 j = 0u; j < ns; j++) {
if (sc > 0u) {
nj += ",\n";
}
sc++;
StreamString sn;
(void)ds->GetSignalName(j, sn);
const char8 *st = TypeDescriptor::GetTypeNameFromTypeDescriptor(
ds->GetSignalType(j));
uint8 d = 0u;
(void)ds->GetSignalNumberOfDimensions(j, d);
uint32 el = 0u;
(void)ds->GetSignalNumberOfElements(j, el);
StreamString sfp;
sfp.Printf("%s.%s", cp.Buffer(), sn.Buffer());
bool tr = false, fo = false;
(void)IsInstrumented(sfp.Buffer(), tr, fo);
nj += "{\"Name\":\"";
EscapeJson(sn.Buffer(), nj);
nj += "\",\"Class\":\"Signal\",\"Type\":\"";
EscapeJson(st ? st : "Unknown", nj);
nj.Printf("\",\"Dimensions\":%u,\"Elements\":%u,"
"\"IsTraceable\":%s,\"IsForcable\":%s}",
d, el, tr ? "true" : "false", fo ? "true" : "false");
}
}
if (gam != NULL_PTR(GAM *)) {
uint32 nIn = gam->GetNumberOfInputSignals();
for (uint32 j = 0u; j < nIn; j++) {
if (sc > 0u) {
nj += ",\n";
}
sc++;
StreamString sn;
(void)gam->GetSignalName(InputSignals, j, sn);
const char8 *st = TypeDescriptor::GetTypeNameFromTypeDescriptor(
gam->GetSignalType(InputSignals, j));
uint32 d = 0u;
(void)gam->GetSignalNumberOfDimensions(InputSignals, j, d);
uint32 el = 0u;
(void)gam->GetSignalNumberOfElements(InputSignals, j, el);
StreamString sfp;
sfp.Printf("%s.In.%s", cp.Buffer(), sn.Buffer());
bool tr = false, fo = false;
(void)IsInstrumented(sfp.Buffer(), tr, fo);
nj += "{\"Name\":\"In.";
EscapeJson(sn.Buffer(), nj);
nj += "\",\"Class\":\"InputSignal\",\"Type\":\"";
EscapeJson(st ? st : "Unknown", nj);
nj.Printf("\",\"Dimensions\":%u,\"Elements\":%u,"
"\"IsTraceable\":%s,\"IsForcable\":%s}",
d, el, tr ? "true" : "false", fo ? "true" : "false");
}
uint32 nOut = gam->GetNumberOfOutputSignals();
for (uint32 j = 0u; j < nOut; j++) {
if (sc > 0u) {
nj += ",\n";
}
sc++;
StreamString sn;
(void)gam->GetSignalName(OutputSignals, j, sn);
const char8 *st = TypeDescriptor::GetTypeNameFromTypeDescriptor(
gam->GetSignalType(OutputSignals, j));
uint32 d = 0u;
(void)gam->GetSignalNumberOfDimensions(OutputSignals, j, d);
uint32 el = 0u;
(void)gam->GetSignalNumberOfElements(OutputSignals, j, el);
StreamString sfp;
sfp.Printf("%s.Out.%s", cp.Buffer(), sn.Buffer());
bool tr = false, fo = false;
(void)IsInstrumented(sfp.Buffer(), tr, fo);
nj += "{\"Name\":\"Out.";
EscapeJson(sn.Buffer(), nj);
nj += "\",\"Class\":\"OutputSignal\",\"Type\":\"";
EscapeJson(st ? st : "Unknown", nj);
nj.Printf("\",\"Dimensions\":%u,\"Elements\":%u,"
"\"IsTraceable\":%s,\"IsForcable\":%s}",
d, el, tr ? "true" : "false", fo ? "true" : "false");
}
}
nj += "\n]";
}
nj += "}";
json += nj;
valid++;
}
return valid;
}
// ---------------------------------------------------------------------------
// EnrichWithConfig
// ---------------------------------------------------------------------------
Source/Components/Interfaces/DebugService/DebugServiceBase.h
-2
View File
@@ -174,8 +174,6 @@ protected:
void UpdateBrokersBreakStatus();
void PatchRegistry();
uint32 ExportTree(ReferenceContainer *container, StreamString &json,
const char8 *pathPrefix);
void ExportTreeNode(const char8 *path, StreamString &out);
void EnrichWithConfig(const char8 *path, StreamString &json);
static void JsonifyDatabase(ConfigurationDatabase &db, StreamString &json);
Source/Components/Interfaces/UDPStream/UDPSClient.cpp
+87 -5
View File
@@ -9,6 +9,7 @@
#include "MemoryOperationsHelper.h"
#include <sys/select.h>
#include <sys/socket.h>
#include <errno.h>
namespace MARTe {
@@ -23,14 +24,17 @@ UDPSClient::UDPSClient()
useMulticast(false),
silenceTimeoutTicks(0u),
reconnectDelayTicks(0u),
keepAliveIntervalTicks(0u),
maxPayloadSize(UDPS_CLIENT_DEFAULT_MAX_PAYLOAD),
cpuMask(0xFFFFFFFFu),
stackSize(65536u),
recvBufferSize(UDPS_CLIENT_DEFAULT_RECV_BUFFER),
listener(NULL_PTR(UDPSClientListener *)),
threadService(*this),
connected(false),
lastDataTicks(0u),
disconnectTick(0u),
lastKeepAliveTicks(0u),
localPort(0u),
lastGcTicks(0u) {
@@ -81,16 +85,25 @@ bool UDPSClient::Initialise(StructuredDataI &data) {
uint32 dpU32 = static_cast<uint32>(serverPort) + 1u;
(void) data.Read("DataPort", dpU32);
dataPort = static_cast<uint16>(dpU32);
StreamString iface;
(void) data.Read("Interface", iface);
multicastInterface = iface;
}
uint32 silenceS = UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S;
float32 silenceS = UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S;
(void) data.Read("SilenceTimeout", silenceS);
silenceTimeoutTicks = static_cast<uint64>(silenceS) * HighResolutionTimer::Frequency();
/* float64 math: the tick rate (~1e9) exceeds float32's 24-bit mantissa */
silenceTimeoutTicks = static_cast<uint64>(static_cast<float64>(silenceS) *
static_cast<float64>(HighResolutionTimer::Frequency()));
uint32 reconnectS = UDPS_CLIENT_DEFAULT_RECONNECT_DELAY_S;
(void) data.Read("ReconnectDelay", reconnectS);
reconnectDelayTicks = static_cast<uint64>(reconnectS) * HighResolutionTimer::Frequency();
uint32 keepAliveS = UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S;
(void) data.Read("KeepAliveInterval", keepAliveS);
keepAliveIntervalTicks = static_cast<uint64>(keepAliveS) * HighResolutionTimer::Frequency();
uint32 mps = UDPS_CLIENT_DEFAULT_MAX_PAYLOAD;
(void) data.Read("MaxPayloadSize", mps);
maxPayloadSize = mps;
@@ -98,6 +111,9 @@ bool UDPSClient::Initialise(StructuredDataI &data) {
(void) data.Read("CPUMask", cpuMask);
(void) data.Read("StackSize", stackSize);
recvBufferSize = UDPS_CLIENT_DEFAULT_RECV_BUFFER;
(void) data.Read("RecvBufferSize", recvBufferSize);
return true;
}
@@ -180,6 +196,18 @@ ErrorManagement::ErrorType UDPSClient::Execute(ExecutionInfo &info) {
}
}
// Unicast keepalive: UDPSServer evicts silent unicast clients after its
// ClientTimeout (default 30 s). Re-sending CONNECT would also re-trigger
// a CONFIG resend; an ACK refreshes the server's last-seen with no side
// effects, so it is the keepalive packet of choice. Multicast clients
// hold a persistent TCP control connection and need no keepalive.
if (!useMulticast && (keepAliveIntervalTicks > 0u)) {
if ((now - lastKeepAliveTicks) >= keepAliveIntervalTicks) {
SendKeepAlive();
lastKeepAliveTicks = now;
}
}
// Periodic GC of stale reassembly slots (~every 1 s)
uint64 gcFreq = HighResolutionTimer::Frequency();
if ((now - lastGcTicks) >= gcFreq) {
@@ -199,6 +227,7 @@ bool UDPSClient::Connect() {
if (ok) {
connected = true;
lastDataTicks = HighResolutionTimer::Counter();
lastKeepAliveTicks = lastDataTicks;
if (listener != NULL_PTR(UDPSClientListener *)) {
listener->OnUDPSConnected();
}
@@ -209,6 +238,23 @@ bool UDPSClient::Connect() {
return ok;
}
void UDPSClient::SetRecvBufferSize(BasicUDPSocket &sock) {
/* BasicUDPSocket exposes no SO_RCVBUF API; the OS default (Linux
* rmem_default, typically ~208 KiB) is easily overrun by high-throughput
* sources, causing silent kernel-level datagram drops. Work around this
* by calling setsockopt() directly on the raw handle. Best-effort: a
* failure here just leaves the OS default in place. */
Handle fd = sock.GetReadHandle();
if (fd >= 0) {
int32 sz = static_cast<int32>(recvBufferSize);
if (setsockopt(fd, SOL_SOCKET, SO_RCVBUF, &sz, sizeof(sz)) != 0) {
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: Could not set SO_RCVBUF to %u bytes.",
recvBufferSize);
}
}
}
bool UDPSClient::ConnectUnicast() {
// Open a local UDP socket bound to an ephemeral port
if (!recvSocket.Open()) {
@@ -216,6 +262,7 @@ bool UDPSClient::ConnectUnicast() {
"UDPSClient: Could not open receive socket.");
return false;
}
SetRecvBufferSize(recvSocket);
if (!recvSocket.Listen(0u)) {
REPORT_ERROR_STATIC(ErrorManagement::Warning,
@@ -256,6 +303,7 @@ bool UDPSClient::ConnectMulticast() {
"UDPSClient: Could not open multicast socket.");
return false;
}
SetRecvBufferSize(mcastSocket);
bool ok = mcastSocket.Listen(dataPort);
if (!ok) {
@@ -266,7 +314,12 @@ bool UDPSClient::ConnectMulticast() {
return false;
}
ok = mcastSocket.Join(multicastGroup.Buffer());
if (multicastInterface.Size() > 0u) {
ok = mcastSocket.Join(multicastGroup.Buffer(), multicastInterface.Buffer());
}
else {
ok = mcastSocket.Join(multicastGroup.Buffer());
}
if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: Could not join multicast group %s.",
@@ -275,8 +328,9 @@ bool UDPSClient::ConnectMulticast() {
return false;
}
REPORT_ERROR_STATIC(ErrorManagement::Information,
"UDPSClient: Joined multicast group %s on port %u.",
multicastGroup.Buffer(), static_cast<uint32>(dataPort));
"UDPSClient: Joined multicast group %s on port %u via interface %s.",
multicastGroup.Buffer(), static_cast<uint32>(dataPort),
(multicastInterface.Size() > 0u) ? multicastInterface.Buffer() : "default");
// Now open the TCP control connection and announce ourselves
if (!tcpSocket.Open()) {
@@ -354,6 +408,25 @@ void UDPSClient::Disconnect() {
}
}
// ---------------------------------------------------------------------------
// Private: SendKeepAlive
// ---------------------------------------------------------------------------
void UDPSClient::SendKeepAlive() {
if (useMulticast || !recvSocket.IsValid()) {
return;
}
uint8 ackPkt[UDPS_HEADER_SIZE];
UDPSBuildHeader(ackPkt, UDPS_TYPE_ACK, 0u, 0u, 1u, 0u);
InternetHost serverDest(serverPort, serverAddr.Buffer());
(void) recvSocket.SetDestination(serverDest);
uint32 sendSize = UDPS_HEADER_SIZE;
if (!recvSocket.Write(reinterpret_cast<const char8 *>(ackPkt), sendSize)) {
/* Non-fatal: if the server is truly gone, the silence timeout
* triggers the usual disconnect + reconnect. */
}
}
// ---------------------------------------------------------------------------
// Private: ReceiveAndProcess
// ---------------------------------------------------------------------------
@@ -378,6 +451,15 @@ bool UDPSClient::ReceiveAndProcess() {
return false;
}
/* HI-6: guard against FD_SETSIZE overflow */
if (fd < 0 || fd >= FD_SETSIZE ||
(tcpFd >= 0 && tcpFd >= FD_SETSIZE)) {
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: fd >= FD_SETSIZE (%d/%d) — skipping select.",
fd, tcpFd);
return false;
}
fd_set rset;
FD_ZERO(&rset);
FD_SET(fd, &rset);
Source/Components/Interfaces/UDPStream/UDPSClient.h
+30 -3
View File
@@ -86,15 +86,26 @@ public:
* 256-fragment span the recvMask[32] tracks at typical chunk sizes. */
static const uint32 UDPS_CLIENT_MAX_PACKET_BYTES = 1048576u; // 1 MiB
/** Default silence timeout before reconnect (seconds). */
static const uint32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 5u;
/** Default silence timeout before reconnect (seconds); sub-second values allowed. */
static const float32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 1.0f;
/** Default delay between reconnect attempts (seconds). */
static const uint32 UDPS_CLIENT_DEFAULT_RECONNECT_DELAY_S = 2u;
/** Default unicast keepalive interval (seconds). UDPSServer evicts silent
* unicast clients after its ClientTimeout (default 30 s); the client
* re-sends an ACK on this interval to stay registered. 0 disables. */
static const uint32 UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S = 15u;
/** Default maximum payload size (bytes, excluding 17-byte header). */
static const uint32 UDPS_CLIENT_DEFAULT_MAX_PAYLOAD = 1400u;
/** Default OS UDP receive socket buffer size (bytes). The Linux default
* (rmem_default, typically ~208 KiB) is easily overrun by high-throughput
* sources (e.g. multi-hundred-KiB bursts every few ms), causing silent
* kernel-level datagram drops. 4 MiB gives generous burst headroom. */
static const uint32 UDPS_CLIENT_DEFAULT_RECV_BUFFER = 4194304u; // 4 MiB
UDPSClient();
virtual ~UDPSClient();
@@ -105,12 +116,19 @@ public:
* - ServerAddr (char*) Server IPv4 address. Required.
* - Port (uint16) Server UDP port (unicast) or TCP listen port (multicast). Required.
* - MulticastGroup (char*) IPv4 multicast address; presence enables multicast mode.
* - Interface (char*) Local IPv4 dotted-quad address (e.g. "127.0.0.1") of the interface on
* which to join the multicast group. Optional; omitting it uses the
* default-route interface (INADDR_ANY), which silently receives nothing
* if the server sends on a different interface.
* - DataPort (uint16) UDP multicast data port (defaults to Port+1).
* - SilenceTimeout (uint32) Seconds of no data before reconnect. Default 5.
* - SilenceTimeout (float32) Seconds of no data before reconnect. Default 1.0.
* Sub-second values allowed; 0 disables the check.
* - ReconnectDelay (uint32) Seconds to wait between reconnect attempts. Default 2.
* - KeepAliveInterval (uint32) Seconds between unicast keepalive ACKs. Default 15. 0 disables.
* - MaxPayloadSize (uint32) Max payload bytes per datagram, excluding header. Default 1400.
* - CPUMask (uint32) CPU affinity mask for the receive thread. Default 0xFFFFFFFF.
* - StackSize (uint32) Stack size for the receive thread. Default 65536.
* - RecvBufferSize (uint32) OS UDP receive socket buffer size (bytes). Default 4 MiB.
*/
bool Initialise(StructuredDataI &data);
@@ -160,11 +178,16 @@ private:
// -------------------------------------------------------------------------
bool Connect();
void Disconnect();
/** Send a keepalive ACK to the server (unicast only, same socket). */
void SendKeepAlive();
bool ReceiveAndProcess();
bool ConnectUnicast();
bool ConnectMulticast();
/** Set the OS receive buffer size (SO_RCVBUF) on a UDP socket's raw handle. */
void SetRecvBufferSize(BasicUDPSocket &sock);
void ProcessDatagram(const uint8 *buf, uint32 size);
/** Read one full UDPS frame (header + payload) from the TCP control socket. */
bool ReceiveTCPFrame();
@@ -181,13 +204,16 @@ private:
StreamString serverAddr;
uint16 serverPort;
StreamString multicastGroup;
StreamString multicastInterface;
uint16 dataPort;
bool useMulticast;
uint64 silenceTimeoutTicks;
uint64 reconnectDelayTicks;
uint64 keepAliveIntervalTicks; ///< 0 = keepalive disabled
uint32 maxPayloadSize;
uint32 cpuMask;
uint32 stackSize;
uint32 recvBufferSize;
// -------------------------------------------------------------------------
// Runtime state
@@ -197,6 +223,7 @@ private:
bool connected;
uint64 lastDataTicks; ///< Ticks at last received DATA/CONFIG
uint64 disconnectTick; ///< Ticks when we disconnected (for delay)
uint64 lastKeepAliveTicks; ///< Ticks at last keepalive ACK sent
// Unicast
BasicUDPSocket recvSocket; ///< Bound to ephemeral port; receives DATA
Source/Components/Interfaces/UDPStream/UDPSServer.cpp
+674 -675
View File
File diff suppressed because it is too large Load Diff
Source/Components/Interfaces/UDPStream/UDPSServer.h
+1
View File
@@ -235,6 +235,7 @@ private:
uint16 port;
uint32 maxPayloadSize;
StreamString multicastGroup;
StreamString interface;
uint16 dataPort;
bool useMulticast;
uint64 clientTimeoutTicks; ///< 0 = disabled
Test/Applications/StreamHub/BoundsCheckTest.cpp
+73
View File
@@ -0,0 +1,73 @@
/**
* @file BoundsCheckTest.cpp
* @brief Reproduction tests for HI-1 (integer overflow in bounds check) and
* HI-4 (unclamped forcedValue memcpy).
*
* These tests verify that the 64-bit bounds check pattern correctly rejects
* crafted payloads whose 32-bit multiply would overflow, and that the
* forcedValue clamp prevents OOB reads.
*/
#include <gtest/gtest.h>
#include "GeneralDefinitions.h"
// HI-1: Verify that a 64-bit bounds check rejects a payload where
// elemsToRead * wireElemBytes would overflow uint32.
TEST(BoundsCheckTest, OverflowRejected) {
// Simulate: numSamples = 0x20000001, wireElemBytes = 8
// 32-bit: 0x20000001 * 8 = 0x8 (overflow!)
// 64-bit: 0x100000008 (correctly large, > any reasonable payload size)
MARTe::uint32 elemsToRead = 0x20000001u;
MARTe::uint32 wireElemBytes = 8u;
MARTe::uint32 off = 12u; // after HRT + numSamples
MARTe::uint32 size = 1400u; // typical max payload
// This is the FIXED pattern (64-bit):
MARTe::uint64 bytesNeeded = static_cast<MARTe::uint64>(off) +
static_cast<MARTe::uint64>(elemsToRead) *
static_cast<MARTe::uint64>(wireElemBytes);
// Should reject (bytesNeeded >> size)
EXPECT_GT(bytesNeeded, static_cast<MARTe::uint64>(size))
<< "64-bit check should detect overflow that 32-bit would miss";
// Verify the OLD (buggy) 32-bit pattern would have passed:
MARTe::uint32 oldCheck = off + (elemsToRead * wireElemBytes);
// On 32-bit: 0x20000001 * 8 = 0x100000008 truncated to 0x8
// off + 0x8 = 20, which is < 1400, so the old check would pass (bug!)
// On 64-bit: the multiply doesn't overflow, so oldCheck is huge
// This test documents that the 64-bit fix is necessary on 32-bit platforms
// and correct on 64-bit.
(void) oldCheck;
}
// HI-1: Verify a normal (non-overflow) case passes the 64-bit check.
TEST(BoundsCheckTest, NormalCasePasses) {
MARTe::uint32 elemsToRead = 100u;
MARTe::uint32 wireElemBytes = 4u;
MARTe::uint32 off = 12u;
MARTe::uint32 size = 500u;
MARTe::uint64 bytesNeeded = static_cast<MARTe::uint64>(off) +
static_cast<MARTe::uint64>(elemsToRead) *
static_cast<MARTe::uint64>(wireElemBytes);
EXPECT_LE(bytesNeeded, static_cast<MARTe::uint64>(size))
<< "normal case should pass the bounds check";
}
// HI-1: Verify numRows * numCols overflow is detected.
TEST(BoundsCheckTest, NumRowsNumColsOverflow) {
MARTe::uint32 numRows = 0x10000u;
MARTe::uint32 numCols = 0x10000u;
// 32-bit: 0x10000 * 0x10000 = 0 (overflow!)
MARTe::uint32 oldResult = numRows * numCols;
EXPECT_EQ(oldResult, 0u) << "32-bit multiply should overflow to 0";
// 64-bit fix:
MARTe::uint64 newResult = static_cast<MARTe::uint64>(numRows) *
static_cast<MARTe::uint64>(numCols);
EXPECT_EQ(newResult, static_cast<MARTe::uint64>(0x100000000ULL))
<< "64-bit multiply should give correct result";
EXPECT_GT(newResult, static_cast<MARTe::uint64>(0x100000u))
<< "should exceed the sanity cap, triggering rejection";
}
Test/Applications/StreamHub/Makefile.inc
+1 -1
View File
@@ -22,7 +22,7 @@
#
#############################################################
OBJSX = TriggerEngineSrc.x BinaryRecorderSrc.x SignalRingBufferGTest.x TriggerEngineGTest.x LTTBGTest.x BinaryRecorderGTest.x
OBJSX = TriggerEngineSrc.x BinaryRecorderSrc.x SignalRingBufferGTest.x TriggerEngineGTest.x LTTBGTest.x BinaryRecorderGTest.x BoundsCheckTest.x WSServerBufferTest.x
PACKAGE=Applications
ROOT_DIR=../../..
Test/Applications/StreamHub/TriggerEngineGTest.cpp
+1 -1
View File
@@ -121,7 +121,7 @@ TEST(TriggerEngineGTest, TestConfigClamping) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.0, 100.0, 150.0));
TriggerConfig cfg = eng.GetConfig();
EXPECT_DOUBLE_EQ(10.0, cfg.windowSec);
EXPECT_DOUBLE_EQ(60.0, cfg.windowSec);
EXPECT_DOUBLE_EQ(100.0, cfg.prePercent);
eng.SetConfig(MakeConfig(kEdgeRising, 0.0, 1.0e-6, -5.0));
Test/Applications/StreamHub/WSServerBufferTest.cpp
+92
View File
@@ -0,0 +1,92 @@
/**
* @file WSServerBufferTest.cpp
* @brief Reproduction test for CR-1: 1-byte heap OOB write in WSServer.
*
* Verifies that the receive buffer allocated in ClientReadLoop is large enough
* to hold a maximal masked WebSocket frame (14-byte header + 65536 payload)
* plus one extra byte for in-place NUL-termination, without overflowing.
*
* Build: linked into the GTest harness alongside MainGTest.cpp.
*/
#include "WSFrame.h"
#include <gtest/gtest.h>
#include <cstdlib>
#include <cstring>
using namespace StreamHub;
// Mirror the WSServer.h constant (TEST_WS_MAX_RECV_PAYLOAD = 65536).
static const uint32 TEST_WS_MAX_RECV_PAYLOAD = 65536u;
// Test: A maximal masked WebSocket frame (64-bit extended length, masked)
// with payloadLen = TEST_WS_MAX_RECV_PAYLOAD must fit within kRecvBuf, and
// payload[plen] must be a valid in-bounds index (for NUL-termination).
TEST(WSServerBufferTest, MaximalFrameFitsInRecvBuffer) {
// Reproduce the exact buffer sizing logic from WSServer::ClientReadLoop.
const uint32 kRecvBuf = TEST_WS_MAX_RECV_PAYLOAD + 14u + 1u;
uint8 *buf = new uint8[kRecvBuf];
// Build a maximal masked frame: FIN + TEXT, payloadLen=65536 (64-bit ext),
// mask=1.
uint8 frame[14 + 65536];
frame[0] = WS_FIN_BIT | WS_OPCODE_TEXT; // FIN + TEXT
frame[1] = WS_MASK_BIT | 127u; // masked + 64-bit length
// 8-byte extended length = 65536
uint64 plen = TEST_WS_MAX_RECV_PAYLOAD;
for (int i = 7; i >= 0; i--) {
frame[2 + i] = static_cast<uint8>(plen & 0xFFu);
plen >>= 8u;
}
// 4-byte mask key
frame[10] = 0xAA; frame[11] = 0xBB; frame[12] = 0xCC; frame[13] = 0xDD;
// Payload (doesn't matter, just fill with zeros)
memset(frame + 14, 0, 65536);
// Copy into buf (simulating a TCP read)
ASSERT_LE(sizeof(frame), static_cast<size_t>(kRecvBuf));
memcpy(buf, frame, sizeof(frame));
// Parse the header
WSFrameHeader hdr;
ASSERT_TRUE(WSParseHeader(buf, sizeof(frame), hdr));
ASSERT_EQ(hdr.headerSize, 14u);
ASSERT_EQ(hdr.payloadLen, static_cast<uint64>(TEST_WS_MAX_RECV_PAYLOAD));
ASSERT_TRUE(hdr.masked);
// Unmask
uint8 *payload = buf + hdr.headerSize;
WSUnmask(payload, static_cast<uint32>(hdr.payloadLen), hdr.maskKey);
// The critical check: payload[plen] must be within the buffer.
// Before the fix, kRecvBuf was 65550 and payload[65536] = buf[65550]
// was one byte past the end. After the fix (+1), it's in bounds.
uint32 plenIdx = static_cast<uint32>(hdr.payloadLen);
ASSERT_LT(hdr.headerSize + plenIdx, kRecvBuf)
<< "payload[plen] would be out of bounds — buffer overflow!";
// Simulate the NUL-termination that WSServer does:
uint8 savedByte = payload[plenIdx];
payload[plenIdx] = '\0';
// Verify it's within bounds (no ASan/heap overflow)
EXPECT_EQ(payload[plenIdx], '\0');
payload[plenIdx] = savedByte;
delete[] buf;
}
// Test: Verify the old (buggy) buffer size would have overflowed.
// This documents the bug for future readers.
TEST(WSServerBufferTest, OldBufferSizeWouldOverflow) {
const uint32 oldKRecvBuf = TEST_WS_MAX_RECV_PAYLOAD + 14u; // the buggy size
const uint32 headerSize = 14u;
const uint32 plen = TEST_WS_MAX_RECV_PAYLOAD;
// headerSize + plen == oldKRecvBuf, so payload[plen] = buf[oldKRecvBuf]
// is one byte past the end.
ASSERT_EQ(headerSize + plen, oldKRecvBuf)
<< "Expected the old buffer to be exactly full (no room for NUL term)";
// The fix adds +1:
const uint32 newKRecvBuf = TEST_WS_MAX_RECV_PAYLOAD + 14u + 1u;
ASSERT_LT(headerSize + plen, newKRecvBuf)
<< "New buffer must have room for the NUL-termination byte";
}
Test/Applications/StreamHub/depends.x86-linux
+46
View File
@@ -133,6 +133,30 @@
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/IOBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/CharBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StreamI.h
../../../Build/x86-linux/Applications/StreamHub/BoundsCheckTest.o: BoundsCheckTest.cpp \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-internal.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-port.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-message.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-string.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-filepath.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-type-util.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-death-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-death-test-internal.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-param-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-param-util.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-linked_ptr.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-printers.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-param-util-generated.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest_prod.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-test-part.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-typed-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest_pred_impl.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Architecture/x86_gcc/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/GeneralDefinitions.h
../../../Build/x86-linux/Applications/StreamHub/LTTBGTest.o: LTTBGTest.cpp ../../../Source/Applications/StreamHub/LTTB.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Architecture/x86_gcc/CompilerTypes.h \
@@ -379,3 +403,25 @@
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/AdvancedErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassProperties.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamMemoryReference.h
../../../Build/x86-linux/Applications/StreamHub/WSServerBufferTest.o: WSServerBufferTest.cpp \
../../../Source/Applications/StreamHub/WSFrame.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Architecture/x86_gcc/CompilerTypes.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-internal.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-port.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-message.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-string.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-filepath.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-type-util.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-death-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-death-test-internal.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-param-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-param-util.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-linked_ptr.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-printers.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-param-util-generated.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest_prod.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-test-part.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-typed-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest_pred_impl.h
Test/Applications/StreamHub/dependsRaw.x86-linux
+46
View File
@@ -133,6 +133,30 @@ BinaryRecorderSrc.o: BinaryRecorderSrc.cpp \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/IOBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/CharBuffer.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/StreamI.h
BoundsCheckTest.o: BoundsCheckTest.cpp \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-internal.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-port.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-message.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-string.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-filepath.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-type-util.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-death-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-death-test-internal.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-param-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-param-util.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-linked_ptr.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-printers.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-param-util-generated.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest_prod.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-test-part.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-typed-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest_pred_impl.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/GeneralDefinitions.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/TypeCharacteristics.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Architecture/x86_gcc/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L1Portability/Environment/Linux/GeneralDefinitions.h
LTTBGTest.o: LTTBGTest.cpp ../../../Source/Applications/StreamHub/LTTB.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Architecture/x86_gcc/CompilerTypes.h \
@@ -379,3 +403,25 @@ TriggerEngineSrc.o: TriggerEngineSrc.cpp \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/AdvancedErrorManagement.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L2Objects/ClassProperties.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L3Streams/StreamMemoryReference.h
WSServerBufferTest.o: WSServerBufferTest.cpp \
../../../Source/Applications/StreamHub/WSFrame.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/CompilerTypes.h \
/home/martino/workspace/MARTe2/Source/Core/BareMetal/L0Types/Architecture/x86_gcc/CompilerTypes.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-internal.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-port.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-message.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-string.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-filepath.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-type-util.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-death-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-death-test-internal.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-param-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-param-util.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-linked_ptr.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-printers.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/internal/gtest-param-util-generated.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest_prod.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-test-part.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest-typed-test.h \
/home/martino/workspace/MARTe2/Lib/gtest-1.7.0/include/gtest/gtest_pred_impl.h
Test/Components/DataSources/UDPStreamer/UDPStreamerTest.cpp
+13 -1
View File
@@ -1559,6 +1559,7 @@ bool UDPStreamerTest::TestInitialise_MulticastMode_Valid() {
ConfigurationDatabase cdb;
cdb.Write("Port", 44710u);
cdb.Write("MulticastGroup", "239.0.0.1");
cdb.Write("Interface", "127.0.0.1");
cdb.Write("DataPort", 44711u);
cdb.CreateRelative("Signals");
cdb.MoveToRoot();
@@ -1574,6 +1575,7 @@ bool UDPStreamerTest::TestInitialise_MulticastMode_DefaultDataPort() {
ConfigurationDatabase cdb;
cdb.Write("Port", 44712u);
cdb.Write("MulticastGroup", "239.0.0.1");
cdb.Write("Interface", "127.0.0.1");
/* DataPort intentionally omitted: should default to 44713 */
cdb.CreateRelative("Signals");
cdb.MoveToRoot();
@@ -1588,6 +1590,9 @@ bool UDPStreamerTest::TestInitialise_MulticastMode_InvalidDataPort() {
ConfigurationDatabase cdb;
cdb.Write("Port", 44714u);
cdb.Write("MulticastGroup", "239.0.0.1");
/* Interface is mandatory for multicast; supply it so the rejection below
* is provably caused by DataPort == Port and not by a missing Interface. */
cdb.Write("Interface", "127.0.0.1");
cdb.Write("DataPort", 44714u); /* same as Port — must be rejected */
cdb.CreateRelative("Signals");
cdb.MoveToRoot();
@@ -1618,6 +1623,7 @@ bool UDPStreamerTest::TestPrepareNextState_Multicast() {
" Class = UDPStreamer\n"
" Port = 44716\n"
" MulticastGroup = \"239.0.0.1\"\n"
" Interface = \"127.0.0.1\"\n"
" DataPort = 44717\n"
" MaxPayloadSize = 1400\n"
" Signals = {\n"
@@ -1695,6 +1701,7 @@ bool UDPStreamerTest::TestExecute_MulticastConnectDataDisconnect() {
" Class = UDPStreamer\n"
" Port = 44720\n"
" MulticastGroup = \"239.0.0.1\"\n"
" Interface = \"127.0.0.1\"\n"
" DataPort = 44721\n"
" MaxPayloadSize = 1400\n"
" Signals = {\n"
@@ -1786,7 +1793,12 @@ bool UDPStreamerTest::TestExecute_MulticastConnectDataDisconnect() {
ok = mcastReader.Listen(44721u);
}
if (ok) {
ok = mcastReader.Join("239.0.0.1");
/* Join on the same interface the streamer sends from (Interface =
* 127.0.0.1 sets IP_MULTICAST_IF on the server's data socket). The
* single-argument Join() would pass INADDR_ANY, letting the kernel
* pick the default-route interface, and the datagram would never
* reach this socket. */
ok = mcastReader.Join("239.0.0.1", "127.0.0.1");
}
/* Step 4: Trigger Synchronise() to generate a DATA packet */
Test/Components/DataSources/UDPStreamerClient/Makefile.gcc
+1
View File
@@ -0,0 +1 @@
include Makefile.inc
Test/Components/DataSources/UDPStreamerClient/Makefile.inc
+59
View File
@@ -0,0 +1,59 @@
#############################################################
#
# Copyright 2015 F4E | European Joint Undertaking for ITER
# and the Development of Fusion Energy ('Fusion for Energy')
#
# Licensed under the EUPL, Version 1.1 or - as soon they
# will be approved by the European Commission - subsequent
# versions of the EUPL (the "Licence");
# You may not use this work except in compliance with the
# Licence.
# You may obtain a copy of the Licence at:
#
# http://ec.europa.eu/idabc/eupl
#
# Unless required by applicable law or agreed to in
# writing, software distributed under the Licence is
# distributed on an "AS IS" basis,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
# express or implied.
# See the Licence for the specific language governing
# permissions and limitations under the Licence.
#
#############################################################
OBJSX = UDPStreamerClientTest.x UDPStreamerClientGTest.x
PACKAGE=Components/DataSources
ROOT_DIR=../../../..
MAKEDEFAULTDIR=$(MARTe2_DIR)/MakeDefaults
include $(MAKEDEFAULTDIR)/MakeStdLibDefs.$(TARGET)
INCLUDES += -I.
INCLUDES += -I$(ROOT_DIR)/Source/Components/Interfaces/UDPStream
INCLUDES += -I$(MARTe2_DIR)/Source/Core/BareMetal/L0Types
INCLUDES += -I$(MARTe2_DIR)/Source/Core/BareMetal/L1Portability
INCLUDES += -I$(MARTe2_DIR)/Source/Core/BareMetal/L2Objects
INCLUDES += -I$(MARTe2_DIR)/Source/Core/BareMetal/L3Streams
INCLUDES += -I$(MARTe2_DIR)/Source/Core/BareMetal/L4Messages
INCLUDES += -I$(MARTe2_DIR)/Source/Core/BareMetal/L4Configuration
INCLUDES += -I$(MARTe2_DIR)/Source/Core/BareMetal/L5GAMs
INCLUDES += -I$(MARTe2_DIR)/Source/Core/Scheduler/L1Portability
INCLUDES += -I$(MARTe2_DIR)/Source/Core/Scheduler/L3Services
INCLUDES += -I$(MARTe2_DIR)/Source/Core/Scheduler/L4Messages
INCLUDES += -I$(MARTe2_DIR)/Source/Core/Scheduler/L4StateMachine
INCLUDES += -I$(MARTe2_DIR)/Source/Core/Scheduler/L5GAMs
INCLUDES += -I$(MARTe2_DIR)/Source/Core/FileSystem/L1Portability
INCLUDES += -I$(MARTe2_DIR)/Source/Core/FileSystem/L3Streams
INCLUDES += -I$(MARTe2_DIR)/Lib/gtest-1.7.0/include
INCLUDES += -I$(ROOT_DIR)/Common/UDP
INCLUDES += -I$(ROOT_DIR)/Source/Components/DataSources/UDPStreamerClient
all: $(OBJS) \
$(BUILD_DIR)/UDPStreamerClientTest$(LIBEXT)
echo $(OBJS)
include depends.$(TARGET)
include $(MAKEDEFAULTDIR)/MakeStdLibRules.$(TARGET)
Test/Components/DataSources/UDPStreamerClient/UDPStreamerClientGTest.cpp
+152
View File
@@ -0,0 +1,152 @@
/**
* @file UDPStreamerClientGTest.cpp
* @brief Source file for class UDPStreamerClientGTest
* @date 01/07/2026
* @author Martino Ferrari
*
* @copyright Copyright 2015 F4E | European Joint Undertaking for ITER and
* the Development of Fusion Energy ('Fusion for Energy').
* Licensed under the EUPL, Version 1.1 or - as soon they will be approved
* by the European Commission - subsequent versions of the EUPL (the "Licence")
* You may not use this work except in compliance with the Licence.
* You may obtain a copy of the Licence at: http://ec.europa.eu/idabc/eupl
*
* @warning Unless required by applicable law or agreed to in writing,
* software distributed under the Licence is distributed on an "AS IS"
* basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
* or implied. See the Licence permissions and limitations under the Licence.
*
* @details This source file contains the GTest wrapper for all UDPStreamerClient tests.
*/
#define DLL_API
/*---------------------------------------------------------------------------*/
/* Standard header includes */
/*---------------------------------------------------------------------------*/
#include "gtest/gtest.h"
#include <limits.h>
/*---------------------------------------------------------------------------*/
/* Project header includes */
/*---------------------------------------------------------------------------*/
#include "UDPStreamerClientTest.h"
/*---------------------------------------------------------------------------*/
/* Method definitions */
/*---------------------------------------------------------------------------*/
TEST(UDPStreamerClientGTest, TestInitialise_Valid) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestInitialise_Valid());
}
TEST(UDPStreamerClientGTest, TestInitialise_DefaultServerAddress) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestInitialise_DefaultServerAddress());
}
TEST(UDPStreamerClientGTest, TestInitialise_DefaultPort) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestInitialise_DefaultPort());
}
TEST(UDPStreamerClientGTest, TestInitialise_SilenceTimeoutFloat) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestInitialise_SilenceTimeoutFloat());
}
TEST(UDPStreamerClientGTest, TestInitialise_MulticastMode_Valid) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestInitialise_MulticastMode_Valid());
}
TEST(UDPStreamerClientGTest, TestInitialise_MulticastMode_DefaultDataPort) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestInitialise_MulticastMode_DefaultDataPort());
}
TEST(UDPStreamerClientGTest, TestSetConfiguredDatabase_MultipleSignals) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestSetConfiguredDatabase_MultipleSignals());
}
TEST(UDPStreamerClientGTest, TestPrepareNextState_StartsReceiver) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestPrepareNextState_StartsReceiver());
}
TEST(UDPStreamerClientGTest, TestSynchronise_NoData) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestSynchronise_NoData());
}
TEST(UDPStreamerClientGTest, TestOnUDPSConfig_BasicAccept) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSConfig_BasicAccept());
}
TEST(UDPStreamerClientGTest, TestOnUDPSConfig_SignalCountMismatch) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSConfig_SignalCountMismatch());
}
TEST(UDPStreamerClientGTest, TestOnUDPSConfig_NameMismatch) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSConfig_NameMismatch());
}
TEST(UDPStreamerClientGTest, TestOnUDPSConfig_ElementCountMismatch) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSConfig_ElementCountMismatch());
}
TEST(UDPStreamerClientGTest, TestOnUDPSConfig_TooSmallPayload) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSConfig_TooSmallPayload());
}
TEST(UDPStreamerClientGTest, TestOnUDPSData_QuantizedUint16) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSData_QuantizedUint16());
}
TEST(UDPStreamerClientGTest, TestOnUDPSData_QuantizedInt8) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSData_QuantizedInt8());
}
TEST(UDPStreamerClientGTest, TestOnUDPSData_MultipleSignalsOrder) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSData_MultipleSignalsOrder());
}
TEST(UDPStreamerClientGTest, TestOnUDPSData_AccumulateMode) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSData_AccumulateMode());
}
TEST(UDPStreamerClientGTest, TestOnUDPSData_TooSmallPayload) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSData_TooSmallPayload());
}
TEST(UDPStreamerClientGTest, TestOnUDPSData_BeforeConfig) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSData_BeforeConfig());
}
TEST(UDPStreamerClientGTest, TestOnUDPSDisconnected_InvalidatesConfig) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestOnUDPSDisconnected_InvalidatesConfig());
}
TEST(UDPStreamerClientGTest, TestExecute_ConnectConfigDataEndToEnd) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestExecute_ConnectConfigDataEndToEnd());
}
TEST(UDPStreamerClientGTest, TestExecute_MulticastReceivesDataOnInterface) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestExecute_MulticastReceivesDataOnInterface());
}

Some files were not shown because too many files have changed in this diff Show More