martino/MARTe-Integrated-Components
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: martino/MARTe-Integrated-Components:1ddb4fe356ec260735395c36ea145a83d98553a8
Branches Tags
martino/MARTe-Integrated-Components:main martino/MARTe-Integrated-Components:feature/udpscope martino/MARTe-Integrated-Components:feature/stress-report-integration
..
compare: martino/MARTe-Integrated-Components:main
Branches Tags
martino/MARTe-Integrated-Components:main martino/MARTe-Integrated-Components:feature/udpscope martino/MARTe-Integrated-Components:feature/stress-report-integration
42 Commits
1ddb4fe356 .. main
Author SHA1 Message Date
Martino FerrariandClaude Opus 4.6 5562877c99 fix(udps): publish the producer's HRT frequency so timestamps survive the hop 
DATA packets timestamp with the raw value of the producer's high-resolution
counter, and the wire never said how fast that counter runs. The hub divided by
its own timer's frequency instead, which is only the same number while producer
and hub share a machine — on x86 it is the TSC frequency and differs from model
to model. Off-box, every accumulated batch was therefore laid out over the wrong
span of time: the samples in it drift away from where they belong and start
colliding with the next packet's, which is the "same" symptom as a stale time
base even though nothing is out of order.

CONFIG now carries the rate as a trailing uint64, alongside the publish-mode
byte and read the same tolerant way: absent or zero means the producer did not
say, and the hub falls back to its own timer as before. Anything below 1 kHz is
not a high-resolution timer and is refused, so a mis-parsed payload cannot
stretch a millisecond batch across seconds.

The Accumulate DATA payload is unchanged, so this costs nothing per packet and
the period *within* a batch is still estimated from the gap between packets.

The Go, C and browser parsers already ignore trailer bytes they do not know,
so they read the new CONFIG unchanged; none of them uses the HRT timestamp.

Also corrects the Accumulate DATA layout in all three protocol documents: they
described it as one snapshot per array signal, where it has always been one per
accumulated cycle.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-09-02 02:49:50 +02:00
Martino FerrariandClaude Opus 4.6 092fd3c775 fix(streamhub): stop the trigger going deaf between captures 
TriggerEngine::CheckSample returned early in every state but ARMED, so an
edge arriving while a capture was being collected or handed out was
dropped, and the automatic rearm then waited for a FRESH edge. The engine
was therefore blind from its own trigger point until the capture had been
harvested — a post-window — and for the holdoff on top of that.

On a sparse pulse train that rounds the capture spacing up to a whole
pulse period: at the default 1 s window the blind stretch is 1 s, so a
1 Hz train was caught at 0.5 Hz and a wider window lost whole multiples.

The comparator now keeps running through COLLECTING and TRIGGERED and
remembers the first edge at or past trigTime + max(postSec, holdoffSec).
The holdoff guards against re-triggering on the ringing of the same
event and is measured from the trigger point, so it overlaps the
post-window rather than adding to it. Rearm() fires on the remembered
edge; it also keeps the tracked level, so the first sample after it has
a real predecessor instead of being spent seeding one.

Arm() stays the operator's arm and discards the held edge — they asked
for the next event, not one already been and gone — and SetConfig() and
Disarm() drop it too, since it was never judged against the new window.

This is the same defect and the same remedy already validated in the Go
hub (wshub/trigger.go, trigger_sporadic_test.go); the C++ hub had been
left with the original semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-09-02 01:45:13 +02:00
Martino FerrariandClaude Opus 4.6 fbae7d712c fix(udps): stop packets being dated from an earlier time base 
Reported as samples sporadically carrying a previous packet's timestamp:
holes on one side of the stream and collisions on the other, in both the
Go and the MARTe2 receiver. That it appeared in both is what located it
-- the shared cause is upstream of either client. Four independent
defects, all of which end in a packet's values being placed at a time
that is not theirs.

Reassembly slot exhaustion (the "Reassembly slots full; evicting oldest"
flood). Chunk size was learnt only from fragment 0, so an out-of-order
burst destroyed a packet whose bytes had all arrived and left the slot
occupied until the 2 s GC. Slots were keyed on the counter alone, but
DATA and CONFIG number independently, so equal counters merged the two
streams. The 32-byte received-mask covered 256 of the 512 fragments the
client accepts, so a duplicate above 255 was counted as new and the
packet was delivered with a hole of stale bytes in it. And one datagram
was read per Execute(), which cannot drain a fast producer. Fixed with a
pendingTail deferral, (counter, type) keying, a 64-byte mask, a
256-datagram drain, counter-age slot reclamation, and a 1 Hz aggregated
warning in place of the per-eviction flood.

UDPStreamer dropping whole Accumulate batches. EventSem::ResetWait is
Reset-then-Wait, so a Post() landing while the sender thread was inside
ServiceClients()/SendData() was destroyed by the next Reset. The batch
was then skipped with dataReady false, readyFill was never cleared, and
the following flush overwrote it: an entire run of RT cycles never
reached the wire. The record of pending work now lives in the buffers
rather than in the semaphore edge, which also removes up to
UDPS_DATA_WAIT_MS of latency; genuine backpressure overwrites are
counted and reported. Against the unfixed code the new test sees
2999/3000 batches never consumed.

Period inflation after loss. Accumulated scalars carry no SamplingRate,
so the receiver derives dt from the sender-clock gap -- but dividing it
by the previous packet's sample count is only right while nothing is
lost. One loss doubles the reported period, which spreads a batch a full
batch past its own end and into the range the next packet claims. That
is the hole and the collision, exactly. Inferring the cycle count from
the estimate's own period is not a way out: it has a stable fixed point
wherever gap/dt is an integer, so a real rate change locks it at the old
one for good (AccumDtGTest.FollowsSustainedRateChange).

The packet counter removes the ambiguity, so all three receivers now
order on it: a DATA packet that does not advance the counter is dropped
rather than delivered, because its values are older than data already
handed over. Ordering is on the signed difference so it survives the
uint32 wrap, and the sequence resets on reconnect, where the producer's
counter restarts independently of ours. The loss count that falls out of
the same delta feeds the period estimate as cycles = prevN * (1 + lost),
which reduces exactly to gap/prevN when nothing is lost and therefore
still tracks a genuine rate change. UDPSClient::AcceptDataCounter (C++),
udpsprotocol.SequenceGate (Go), decode_data (C).

The C client's existing gap counter was wrap-unsafe and let a stale
packet rewind last_counter, which made every subsequent gap wrong; it
uses the same code now. Docs/Protocol.md gains an Ordering DATA section
stating the requirement for any receiver, including ones outside this
repository.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-09-02 01:18:49 +02:00
Martino FerrariandClaude Opus 4.6 f334995865 feat(webui): sporadic-trigger capture, CSV export and UI rework 
Brings the Go hub and web SPA work developed on feature/udpscope onto
main, without the udpscope client itself.

The trigger engine could not capture a sporadic event: it armed on the
live tail only, so a burst shorter than one push window was already past
by the time the FSM looked for it. It now searches the ring history for
the crossing, which also makes a capture reproducible from the same data
rather than dependent on push timing (wshub/trigger.go, ringbuf.go,
history.go).

Adds CSV/JSON export of the visible window (wshub/export.go) and reworks
the SPA: per-signal axis controls, a readable trigger panel, and a fix
for the flicker caused by repainting on every push instead of on a frame
tick (static/app.js, index.html, style.css).

BUFFER_AND_TRIGGER.md documents the ring/decimation/trigger interaction,
which is otherwise only inferable from the three files that implement it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-09-02 01:18:46 +02:00
Martino FerrariandClaude Opus 4.6 cf815e1d3f docs: implementation plan for the UDPScope direct-UDPS oscilloscope 
Eighteen TDD tasks covering the build scaffold, pane tree, time base, frame
decoding, trigger FSM, threading, UI, persistence and export, so the scope can
be built task-by-task with a reviewable deliverable at each step.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-27 16:35:41 +02:00
Martino FerrariandClaude Opus 4.6 52958cf8bc docs: design for UDPScope, a direct-to-streamer ImGui oscilloscope 
A bench scope that attaches straight to one UDPStreamer through the
standalone C client, so it can be dropped on a machine with no StreamHub,
no Go and no browser. Records the decisions that are easy to get wrong:
the time-base rules must follow UDPSourceSession rather than the C
library's arrival-time estimate, decimation must be min/max rather than
LTTB so glitches survive, and the ring must be sized past the trigger
window by an explicit harvest margin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-27 11:28:54 +02:00
Martino FerrariandClaude Opus 4.6 7c5eb31a52 feat: standalone C/C++ UDPS client library 
Consuming a UDPStreamer feed so far meant either linking MARTe2 (UDPSClient)
or writing Go (Common/Client/go/udpsprotocol). Common/Client/c fills the gap
for plain C/C++ integrators: two files depending on nothing but libc and BSD
sockets, covering the whole receive path — CONNECT, fragment reassembly,
CONFIG/DATA decoding with dequantisation, keepalives and silence-triggered
reconnect. No threads are spawned; udps_client_poll() does all the work and
runs every callback, so it drops into an existing event loop unsynchronised.

Verified against run_udp_producer.sh at 1 Msps: unicast (120 MiB, no loss) and
multicast with 12-fragment cycles (116k datagrams, no loss).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-22 16:51:40 +02:00
Martino Ferrari e03c60db25 Implemented and fixed many issues 2026-08-21 23:24:48 +02:00
Martino FerrariandClaude Sonnet 4.6 14d5351a81 fix: UDPSClient uses two-arg Join so multicast receiver lands on the right interface 
UDPSClient::ConnectMulticast was calling the single-arg BasicUDPSocket::Join,
which forwards NULL as the local interface and lets the kernel bind to
INADDR_ANY.  On a multi-homed host (or when the server sends on loopback via
Interface = "127.0.0.1") the client joins the wrong interface and silently
receives nothing.

Fix: read the optional Interface key inside the useMulticast block in
UDPSClient::Initialise; in ConnectMulticast call the two-arg
Join(group, interface) when Interface is set, and fall back to the one-arg
call otherwise to preserve the existing INADDR_ANY behaviour for configs that
omit it.

Forward the new optional Interface key through UDPStreamerClient (read from
DataSource config, written into the UDPSClient ConfigurationDatabase only
when non-empty).  Extend the "Joined multicast group" log to report the
interface name or "default".

Regression test TestExecute_MulticastReceivesDataOnInterface: mock TCP
control listener + multicast UDP DATA socket with IP_MULTICAST_IF set to
127.0.0.1, verifying a uint32 value of 424242 reaches DataSource signal
memory.  Confirmed FAILED without the Join fix and PASSED with it.

Suite: 133/133 (was 132/132 before this commit).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 08:23:41 +02:00
Martino FerrariandClaude Opus 4.6 61a2aa3988 docs: Interface takes an interface IP, not an interface name 
UDPSServer parses Interface with inet_addr() to set IP_MULTICAST_IF, so
a name like "eth0" yields INADDR_NONE and Initialise fails. Every
example in UDPStreamer.md used "eth0", so anyone following the docs hit
that failure; the .cfg files in Test/ already used a dotted-quad.

Replace the examples with 192.168.1.10, state the dotted-quad
requirement in the parameter table and the Multicast section, and note
that receivers must join on the matching interface or they silently
receive nothing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 08:09:21 +02:00
Martino FerrariandClaude Opus 4.6 c827ecefff test: fix UDPStreamer multicast tests broken by mandatory Interface 
Commit 3e0a481 made Interface mandatory for multicast in both
UDPStreamer::Initialise and UDPSServer::Initialise and updated
Docs/UDPStreamer.md, but left the GTest configs untouched. All four
multicast tests have failed since.

Add Interface to the five multicast configs. The field is parsed with
inet_addr(), so it takes a dotted-quad, not an interface name; 127.0.0.1
keeps the tests self-contained and off the LAN.

TestInitialise_MulticastMode_InvalidDataPort was passing for the wrong
reason: UDPStreamer.cpp rejected it on the missing Interface before the
DataPort == Port check could run. It now proves what its name claims.

TestExecute_MulticastConnectDataDisconnect additionally needed the
two-argument Join(): the single-argument form passes INADDR_ANY, so the
reader joined the default-route interface while the server sent on
loopback via IP_MULTICAST_IF, and the DATA datagram never arrived.

GTest: 132/132 (was 128/132), multicast subset stable over 3 runs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 08:06:35 +02:00
Martino FerrariandClaude Opus 4.6 72c286db33 chore: untrack SDD scratch reports 
These subagent handoff artifacts were committed before
.superpowers/sdd/.gitignore took effect. They are ephemeral
scratch, not project content.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 08:01:09 +02:00
Martino Ferrari 7aba1260be Merge branch 'feature/signal-calibration-config' 
Per-signal data calibration (value = raw * scale + offset) with an optional
unit override, plus sources+calibration persistence in a single config file,
implemented identically in the Go hub and the C++ StreamHub.
 2026-08-17 08:00:09 +02:00
Martino FerrariandClaude Sonnet 4.6 1f8592f854 fix: address all 9 findings from final calibration code review 
- calibration.js: fix baseSignalName('[0]') parity with Go/C++ (>= 0 not > 0)
- calibration.test.js: add assertions for '[0]' edge case in two existing tests
- app.js: remove stale typeof guard around refreshVScaleMenu (always defined)
- app.js: call refreshTrigThresholdField on trig-signal change (both assignment sites)
- index.html: drop maxlength='16' on unit input; normaliseCal is the sole enforcer
- configcheck/main.go: delete dead nextOneOf function (no callers)
- hub_calibration_test.go: delete orphaned waitBroadcast comment (function never existed)
- calibration.go: correct arrayIndexSuffix comment to document known Go/C++ difference
- Docs/StreamHub-API.md: add calibration entry count and unit byte limits to §5 table
- spec: fix configReloaded missing path field, '16 chars'→'16 UTF-8 bytes', StreamString→char[], chain scenario→configcheck program, four→five new frames

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:57:28 +02:00
Martino FerrariandClaude Sonnet 4.6 42f5a726af fix: three StreamHub C++ defects from final code review 
Finding 1 (HandleReloadConfig data loss): move ClearCalibration inside
LoadSourcesFile so the table is wiped only after a successful fread.
Adds a clearCalibration bool parameter (default false); the reload path
passes true, the startup path passes false.

Finding 2 (JSON injection via unit/source/signal): add JsonEscape()
static helper (escapes \", \\, \n \r \t, and \u00XX for other control
chars). Applied at all three emission sites: BroadcastCalibration,
HandleSaveSources, and BroadcastSources (label). Teach JsonGetString to
unescape the same set on read, so values round-trip correctly.

Finding 3 (%.17g verbosity): add ShortFloat() static helper that tries
%.15g then %.16g then %.17g, stopping at the first precision whose
strtod() output compares equal to the original. Applied at both float
emission sites. 0.1 now prints as "0.1", not "0.10000000000000001".

Minor: fix two inaccurate comments in StreamHub.h — the CalibrationEntry
rationale (not a 133 MB / address-limit issue; the real reason is no
per-entry heap churn, STL-free, trivially copyable) and "chars" to "bytes"
for the unit cap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:50:01 +02:00
Martino FerrariandClaude Sonnet 4.6 686fc2ce7d docs: fix three review findings in calibration documentation 
- Finding 1 (Critical): replace "16 chars" with "16 UTF-8 bytes" in the
  setCalibration unit field description (StreamHub-API.md) and the Cal·Unit
  toolbar row (WebUI.md); note that multi-byte characters consume more than
  one byte and that truncation never splits a character.

- Finding 2 (Important): correct the claim that a sources broadcast after
  reloadConfig is conditional on new sources being added — that is true only
  of the Go hub. The C++ hub calls BroadcastSources() unconditionally on
  success. Both the reloadConfig command description and the configReloaded
  event description in StreamHub-API.md are updated; the Reload bullet in
  WebUI.md is updated with a brief note. Clients must tolerate an unsolicited
  sources frame after any reload.

- Finding 3 (Minor): the configSaved failure example used "no SourcesFile
  configured", which matches neither hub. Corrected to the C++ form
  "no sources file configured" and added a note that the exact error text
  is not part of the protocol contract (Go uses "no sources-file configured").

Source evidence: calibration.go (maxUnitLen, len(), rune-repair loop),
StreamHub.cpp (kMaxUnitLen, byte strncpy, HandleReloadConfig unconditional
BroadcastSources, HandleSaveSources error string).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:34:48 +02:00
Martino FerrariandClaude Sonnet 4.6 d26b78b7f6 docs: document per-signal calibration and config save/reload 
Update Docs/StreamHub-API.md (new setCalibration/reloadConfig commands,
calibration/configSaved/configReloaded events, §4 config file format),
ARCHITECTURE.md §6 (updated command/event tables and Config File Format
subsection), Docs/WebUI.md (Cal row in V-Scale Toolbar, Sources & Config
sidebar section). Also corrects the spec's Validation sentence to match
the shipped cal-invalid border behaviour instead of silent field revert.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:27:25 +02:00
Martino FerrariandClaude Sonnet 4.6 e37eec0276 webui: add the Sources & Config sidebar section 
Renames the "Add Source" section to "Sources & Config" and replaces the
fire-and-forget "Save list" button with Save and Reload, plus a one-line
status area that renders the hub's configSaved/configReloaded ack.

onConfigAck replaces the no-op stub Task 7 left behind. It branches on the
frame type because configSaved carries a path and configReloaded does not,
and surfaces the hub's error text on failure rather than failing silently.
The status is kept outside the DOM because buildSidebar() recreates this
section on every sources broadcast.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 07:23:08 +02:00
Martino FerrariandClaude Sonnet 4.6 5917ab1bf2 Restore calibration parity: revert unit-stripping from normaliseCal, fix CSV quoting at export 
Finding 1: revert the comma/quote strip added in 3cb998c from
normaliseCal() in calibration.js. The strip broke byte-identical parity
with Go CalConfig.Normalise and C++ StreamHub::SetCalibrationEntry, both
of which only trim whitespace and cap at 16 UTF-8 bytes. Delete the
companion test that asserted the now-removed behaviour (suite returns to
18 tests).

Finding 2: fix the actual CSV-safety problem at the point of use in
exportAllCSV() in app.js. Header cells (time column and signal columns)
are now RFC 4180-quoted: wrapped in double quotes with any embedded
double quote doubled. This safely handles units or signal names that
contain commas or quotes without touching normaliseCal.

Finding 3: update two stale comments in app.js that called the trigger
threshold or rawFromNorm result 'raw' — Task 9 moved trig.threshold into
calibrated units throughout, so the comments now say 'calibrated'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 01:15:13 +02:00
Martino FerrariandClaude Sonnet 4.6 3cb998c5da webui: calibrate CSV export, trigger threshold and unit display 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:55:07 +02:00
Martino FerrariandClaude Sonnet 4.6 4908c039a5 fix(webui): keep cal-invalid styling on focused field during hub broadcast 
refreshVScaleMenu() was unconditionally removing the cal-invalid class
from all three calibration inputs, which silently cleared the red-border
error indicator on a field the user was editing whenever a hub
calibration broadcast arrived. Apply the same focused-element guard
already used for .value writes so a rejected value's error styling
persists until the user corrects it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:52:04 +02:00
Martino FerrariandClaude Sonnet 4.6 4e3a90b2c6 webui: add calibration editor to the V-Scale toolbar 
Adds the Cal row (Scale / Offset / Unit / Reset) to #vscale-menu, its
CSS, and the refreshVScaleMenu() / commitCal() logic that wires it to
calTable and the hub via setCalibrationWS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:48:39 +02:00
Martino Ferrari 0e9ec61226 webui: apply per-signal calibration to the whole display path 2026-08-17 00:43:18 +02:00
Martino FerrariandClaude Opus 4.6 7312dd0ca0 docs(calibration.js): explain why one UTF-8 repair pass suffices in JS 
TextEncoder always emits well-formed UTF-8, so truncation can strand at
most a lead byte plus three continuations — one repair pass covers it.
The C++ hub needs a loop because its input is raw bytes off the wire.
Also drops a dead variable from the byte-boundary test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 00:41:00 +02:00
Martino FerrariandClaude Opus 4.6 48d62c1f80 fix(calibration.js): cap unit at 16 UTF-8 bytes, matching both hubs 
normaliseCal was using String.length/.slice() (UTF-16 code units), so
multi-byte characters like °, Ω, µ could slip through oversized. Now uses
TextEncoder to slice at 16 bytes, then repairs any incomplete trailing
UTF-8 sequence by walking back over continuation bytes to find the lead
byte and dropping the incomplete rune — exactly mirroring Go's
utf8.DecodeLastRuneInString loop and the C++ walk-back in
StreamHub::SetCalibrationEntry. Adds 4 new test cases covering the
non-ASCII/boundary scenarios, and corrects the canonical test command in
the task-6 report to 'cd Client/udpstreamer && node --test'.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-17 00:36:01 +02:00
Martino FerrariandClaude Sonnet 4.6 21d084d2ea webui: add pure calibration module with unit tests 
Implements Calib JS module (calibration.js) with affine transform primitives,
CalTable, and normaliseCal matching Go CalConfig.Normalise semantics; 14 node
--test cases all pass. Loads before app.js in index.html.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:29:53 +02:00
Martino FerrariandClaude Sonnet 4.6 b1c2a34eea test(configcheck): harden frame-strictness, sort-order, and document BroadcastSources difference 
Fix round 1 review findings:

1. Frame-matching strategy: add protocolFrameTypes set; next() now fails immediately
   on an out-of-order protocol frame instead of silently discarding it. Ambient
   frames (data, stats, triggerState, monotonicState) are logged and skipped.
   nextSkipping() accepts explicitly-listed protocol frames that legitimately differ
   in order across hubs (connect-time "sources" before "calibration" in C++).

2. BroadcastSources documented exception: the extra "sources" frame C++ emits after
   reload is accepted and logged as [KNOWN DIFFERENCE] with full rationale; the
   report recommendation to remove it has been retracted (it is required for correct
   client-side source-list updates after reload-with-new-sources).

3. Sort-order constraint exercised: three calibration entries submitted in
   reverse-sort order (src2/Beta, src1/Zeta, src1/Alpha); each broadcast and the
   saved config file are asserted to be sorted by source then signal.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:26:12 +02:00
Martino FerrariandClaude Sonnet 4.6 73ba725d8f test: add cross-hub calibration and config-persistence parity checker 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:14:15 +02:00
Martino FerrariandClaude Sonnet 4.6 bdc74f5fd2 StreamHub: loop UTF-8 tail repair to match Go CalConfig.Normalise() 
The single-pass repair left invalid bytes when the candidate lead byte
had class 0 (illegal 0xF8-0xFF bytes, or a bare continuation byte
reached after the 3-byte backward-scan cap). Convert to a loop with a
`cut` flag mirroring Go's loop: each iteration either makes no cut
(exits) or strictly reduces ulen by >= 1 byte (terminates in <= 16
iterations). Also treat expected==0 as a cut target, matching Go's
behaviour of stripping any byte that decodes as an invalid one-byte
sequence.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:06:45 +02:00
Martino FerrariandClaude Sonnet 4.6 93e00d0c21 fix(StreamHub): correct UTF-8 tail repair to run only after truncation 
The previous walk-back in SetCalibrationEntry was unconditional, corrupting
short valid units ending in multi-byte characters (e.g. Omega, mu, degree).
Also failed to drop an orphaned lead byte left after stripping continuation
bytes. Restructured to use a 256-byte staging buffer so truncation can be
detected, then repair runs only in the truncation branch. Algorithm now
matches Go CalConfig.Normalise() exactly: scan back over continuation bytes
(up to 3), find the lead byte, derive expected sequence length, cut if
incomplete. Covers all cases: orphaned continuation, orphaned lead, cut on
lead byte.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-17 00:02:33 +02:00
Martino FerrariandClaude Sonnet 4.6 2f9b135c62 task-4-report: append Fix round 1 section 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 23:55:49 +02:00
Martino FerrariandClaude Sonnet 4.6 957be793ae StreamHub: fix calibration trim, UTF-8 unit truncation, and sort order 
Finding 1: Add TrimInPlace helper; apply trim→strip-[i]→empty-check order
in SetCalibrationEntry (matching Go Normalise()), so whitespace-padded
source/signal from WS clients normalise identically to config-file loads.

Finding 2: Trim unit before truncating to kMaxUnitLen, then walk back
continuation bytes (0x80-0xBF) to avoid leaving a partial UTF-8 rune,
matching Go's utf8.DecodeLastRuneInString loop.

Finding 3: BroadcastCalibration and HandleSaveSources now emit entries
sorted by source then signal (insertion sort over an index array, no STL),
producing byte-identical calibration frames and config files to the Go hub.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 23:55:46 +02:00
Martino FerrariandClaude Sonnet 4.6 cdafb877a3 StreamHub: per-signal calibration, config reload, whitespace-tolerant JSON 
- Add CalibrationEntry (heap-allocated array[256], char[] fields to stay within
  the 133 MB struct's canonical address limit) plus calibrationMutex_ and
  numCalibration_.
- Implement SetCalibrationEntry/ClearCalibration, BroadcastCalibration,
  BroadcastConfigAck, HandleSetCalibration, HandleReloadConfig.
- Wire setCalibration and reloadConfig into OnWSCommand dispatch.
- Broadcast calibration to each newly connected client after triggerState.
- Fix JSON round-trip bug: replace JsonGetString/JsonGetBool helpers with a
  shared whitespace-tolerant JsonFindValue (tolerates "key" : "value" as
  written by HandleSaveSources); add JsonIsFinite (no <cmath>).
- Extend LoadSourcesFile(bool skipActive) to also parse calibration blocks;
  SourceIsActive checks live sessions before starting a duplicate.
- Extend HandleSaveSources to persist calibration blocks; emit configSaved ack.
- Reload semantics: calibration replaced wholesale, sources added only.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 19:56:44 +02:00
Martino FerrariandClaude Sonnet 4.6 ffe7cb1cc5 wshub: add setCalibration/reloadConfig frames and config acks 
Wire five new WebSocket frames into hub.go: setCalibration (client→hub),
calibration (hub→client broadcast), reloadConfig (client→hub), configSaved and
configReloaded (hub→client acks with ok/path/error). Extend hubCmd with cal
field, add buildCalibrationMsg and buildConfigAckMsg builders, send calibration
on client connect, and run Save/Reload on separate goroutines to avoid blocking
the Run() select loop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 19:36:09 +02:00
Martino FerrariandClaude Sonnet 4.6 dfd257cfd9 wshub: persist calibration alongside sources; add config reload 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 19:27:23 +02:00
Martino FerrariandClaude Sonnet 4.6 66efd74dd5 wshub: fix CalConfig.Normalise parity gaps vs C++ twin and JS client 
Finding 1: strip trailing [digits] array-element suffix from Signal so one
calibration entry covers an entire array signal, matching the C++ strchr
truncation and the JS equivalent.

Finding 2: after the 16-byte Unit truncation, drop any trailing partial UTF-8
rune so json.Marshal never emits replacement characters; keeps byte limit in
sync with C++ strncpy(u, unit, kMaxUnitLen).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-08-16 19:21:41 +02:00
Martino Ferrari 47f1567a26 wshub: add per-signal calibration store and flat config-file codec 2026-08-16 19:16:22 +02:00
Martino Ferrari 6d26e8191c docs: implementation plan for per-signal calibration and persistent hub config 2026-08-16 19:14:34 +02:00
Martino FerrariandClaude Opus 4.6 5ab1721df5 docs: spec per-signal calibration and persistent hub config 
Design for a per-signal affine calibration (scale/offset/unit) applied
client-side and stored server-side alongside the source list, so signals in
raw units can be read in engineering units and the setup survives a reload.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-08-16 18:17:07 +02:00
Martino Ferrari 2370848994 added trigger 2026-08-13 10:28:56 +02:00
Martino Ferrari ff5ad22447 included jitter correction on client 2026-08-13 10:28:43 +02:00
Martino Ferrari a49ab5ba25 Added silence timeout as floating point 2026-08-10 17:18:50 +02:00
105 changed files with 33637 additions and 1214 deletions
Expand all files Collapse all files
AGENTS.md
+6 -3
View File
@@ -58,7 +58,7 @@ UDP 8081 telemetry, TcpLogger 8082 (REPORT_ERROR → "LOG <LEVEL> <desc>" lines)
|---|---| |---|---|
| `Source/Components/DataSources/UDPStreamer/` | Output DataSource; UDP I/O on bg thread, RT thread only spinlock+memcpy in `Synchronise()` | | `Source/Components/DataSources/UDPStreamer/` | Output DataSource; UDP I/O on bg thread, RT thread only spinlock+memcpy in `Synchronise()` |
| `Source/Components/DataSources/UDPStreamerClient/` | Input DataSource (shared `UDPSClient`), double-buffered ready/scratch | | `Source/Components/DataSources/UDPStreamerClient/` | Input DataSource (shared `UDPSClient`), double-buffered ready/scratch |
| `Source/Components/GAMs/` | `SineArrayGAM` (float32 sine, continuous phase), `TimeArrayGAM` (us-timer → per-sample timestamp array) | | `Source/Components/GAMs/` | `SineArrayGAM` (float32 sine, continuous phase), `TimeArrayGAM` (us-timer → per-sample timestamp array; `Anchor = FirstSample|LastSample|Continuous`, use `Continuous` for contiguous sources so a lost RT cycle cannot hole the time base) |
| `Source/Components/Interfaces/DebugService/` | Registry patching, `DebugBrokerWrapper.h`, TCP/UDP services | | `Source/Components/Interfaces/DebugService/` | Registry patching, `DebugBrokerWrapper.h`, TCP/UDP services |
| `Source/Components/Interfaces/TCPLogger/` | `LoggerConsumerI` forwarding `REPORT_ERROR` to ≤8 TCP clients | | `Source/Components/Interfaces/TCPLogger/` | `LoggerConsumerI` forwarding `REPORT_ERROR` to ≤8 TCP clients |
| `Source/Components/Interfaces/UDPStream/` | Plain-C++ helpers (not MARTe2 Objects): `UDPSClient` (auto-reconnect + fragment reassembly), `UDPSServer` (not thread-safe — owner's Execute thread only) | | `Source/Components/Interfaces/UDPStream/` | Plain-C++ helpers (not MARTe2 Objects): `UDPSClient` (auto-reconnect + fragment reassembly), `UDPSServer` (not thread-safe — owner's Execute thread only) |
@@ -140,8 +140,11 @@ cd Client/streamhub-qt && cmake -B build && cmake --build build
with long options: `--host HOST --port 8090` (single-dash misparsed). Single with long options: `--host HOST --port 8090` (single-dash misparsed). Single
GUI thread, 60 Hz QTimer repaint. GUI thread, 60 Hz QTimer repaint.
- **StreamHub config** is *not* a MARTe2 `RealTimeApplication`: `Hub = { WSPort - **StreamHub config** is *not* a MARTe2 `RealTimeApplication`: `Hub = { WSPort
MaxPoints PushRate MaxPushPoints RingTemporal RingScalar +Recorder{...} MaxPoints PushRate MaxPushPoints RingTemporal RingScalar RingMaxMB AllowedOrigins
Sources={id={Label Addr Port}} }`. `+History` keys: `Directory` (required), +Recorder{...} Sources={id={Label Addr Port}} }`. `AllowedOrigins` is the
WebSocket Origin allowlist — without it a browser serving the SPA from a
different port than the hub is rejected 403.
`+History` keys: `Directory` (required),
`DurationHours` (1), `Decimation` (1), `FlushIntervalSec` (5), `DurationHours` (1), `Decimation` (1), `FlushIntervalSec` (5),
`MinDiskFreeMB` (500). `.shist` files: 64-byte header ('SHR1') + circular `MinDiskFreeMB` (500). `.shist` files: 64-byte header ('SHR1') + circular
(t,v) float64 pairs. (t,v) float64 pairs.
ARCHITECTURE.md
+75 -8
View File
@@ -117,9 +117,16 @@ Sent when the signal set changes or a client connects:
``` ```
[uint32 numSigs] [uint32 numSigs]
numSigs × UDPSSignalDescriptor (136 bytes each, packed) numSigs × UDPSSignalDescriptor (136 bytes each, packed)
[uint8 publishMode] 0=Strict/Decimate, 1=Accumulate [uint8 publishMode] 0=Strict, 1=Accumulate, 2=Decimate
[uint64 hrtFrequency] producer's HRT ticks per second; 0 = unknown
``` ```
Everything after the descriptors is an optional trailer: a receiver accepts a
payload that stops early and ignores bytes it does not know. `hrtFrequency` is
what lets a receiver on another host turn the raw counter in DATA into seconds
— without it the only option is the receiver's own timer, which agrees with the
producer only when the two share a machine.
### DATA Payload (Strict / Decimate modes) ### DATA Payload (Strict / Decimate modes)
``` ```
@@ -130,9 +137,9 @@ per-signal data in CONFIG order (quantised or raw, no inter-signal padding)
### DATA Payload (Accumulate mode) ### DATA Payload (Accumulate mode)
``` ```
[uint64 HRT timestamp] [uint64 HRT timestamp of the first slot in the batch]
[uint32 numSamples] [uint32 numSamples] RT cycles accumulated into this packet
for each signal: if scalar → numSamples elements; else → NumElements once for each signal, in CONFIG order: numSamples × NumElements values
``` ```
### Quantization / Dequantization ### Quantization / Dequantization
@@ -314,7 +321,7 @@ Hub-side trigger with the web client's semantics (config: signal key
``` ```
IDLE →[arm]→ ARMED IDLE →[arm]→ ARMED
ARMED →[edge crossing]→ COLLECTING (latches trigTime, pre/postSec) ARMED →[edge crossing]→ COLLECTING (latches trigTime, pre/postSec)
COLLECTING →[post window + margin elapsed]→ TRIGGERED (broadcast binary v2 capture) COLLECTING →[every source produced past the window]→ TRIGGERED (broadcast binary v2 capture)
TRIGGERED →[auto-rearm (normal, ~200 ms) | rearm (single)]→ ARMED TRIGGERED →[auto-rearm (normal, ~200 ms) | rearm (single)]→ ARMED
any →[disarm]→ IDLE any →[disarm]→ IDLE
``` ```
@@ -325,6 +332,30 @@ sample of the configured signal. The capture is assembled in the push loop from
LTTB-capped at 20 000 points/signal, and broadcast as a binary version-2 frame. LTTB-capped at 20 000 points/signal, and broadcast as a binary version-2 frame.
A `stopped` flag (`trigStop`) freezes auto-rearm. A `stopped` flag (`trigStop`) freezes auto-rearm.
COLLECTING is left on the **data's** clock, not `clock_gettime()`: `trigTime`
comes from sample timestamps, and a source that free-runs on its own clock sits
seconds away from wall time, so a wall-clock deadline chops exactly that offset
off every capture's tail. `UDPSourceSession::ProducerNewestTime()` reports how
far a source has produced — counting only signals actually timestamped from a
time signal, since PACKET-timed ones (the time array itself included) are
stamped on arrival and would just report "now".
Sources are harvested **one at a time**, each as soon as *it* passes
`trigTime + postSec + 0.15 s` (`BeginTriggerCapture` / `HarvestTriggerCapture` /
`FinishTriggerCapture`, the frame accumulating across push ticks). Making every
source wait for the slowest lets the leaders' rings roll past the pre-trigger
region before it is ever read. A 2 s wall-clock watchdog per capture bounds the
wait for a source that stopped advancing; it is harvested short, with a warning
naming the source and how far it got.
Because `RingTemporal` only holds ~1 s at 1 MSps, `setTrigger` publishes the
requested window and the push loop calls `GrowRingsForTrigger()`: each ring
measures its own rate (`Count() / TimeSpan()` — UDPS sources usually report
`samplingRate = 0`) and is grown in place to `rate × (window + 0.5 s) × 1.2`,
clamped per signal to `RingMaxMB`. `SignalRingBuffer::Grow()` preserves
contents *and* `totalWritten`, so live push cursors stay valid. Without this a
long window only ever captures its tail.
### Configuration File (MARTe2 cfg format) ### Configuration File (MARTe2 cfg format)
``` ```
@@ -334,9 +365,11 @@ Hub = {
PushRate = 30 // push loop Hz PushRate = 30 // push loop Hz
MaxPushPoints = 50 // LTTB cap per signal per tick MaxPushPoints = 50 // LTTB cap per signal per tick
StatsRate = 1 // stats broadcast Hz StatsRate = 1 // stats broadcast Hz
RingTemporal = 1000000 // ring capacity (points) for multi-element signals RingTemporal = 1000000 // initial ring capacity (points) for multi-element signals
RingScalar = 100000 // ring capacity (points) for scalar signals RingScalar = 100000 // ring capacity (points) for scalar signals
RingMaxMB = 128 // per-signal ceiling when a trigger window grows a ring
SourcesFile = "streamhub_sources.json" // dynamic-source persistence SourcesFile = "streamhub_sources.json" // dynamic-source persistence
AllowedOrigins = "http://127.0.0.1:8099,http://localhost:8099" // see below
Sources = { Sources = {
App1 = { App1 = {
Label = "MARTe2 App 1" Label = "MARTe2 App 1"
@@ -358,6 +391,16 @@ Sources added at runtime (`addSource`) get generated ids `s1, s2, …`;
`saveSources` persists them to `SourcesFile` (JSON array of `saveSources` persists them to `SourcesFile` (JSON array of
`{label, addr, multicastGroup?, dataPort?}`), reloaded at start-up. `{label, addr, multicastGroup?, dataPort?}`), reloaded at start-up.
`AllowedOrigins` is a comma/space-separated allowlist of `scheme://host[:port]`
values accepted in the WebSocket `Origin` header (max 8 entries, 128 chars
each), matching the Go hub's option. Without it the handshake only accepts an
`Origin` whose host matches the request `Host` — so a browser that loaded the
SPA from a *different* port than the hub (the `run_streamhub.sh` layout, SPA on
8099 and hub on 8090) is rejected with 403. Non-browser clients send no `Origin`
and are unaffected. This is the CSWSH guard of RFC 6455 §10.2: browsers attach
cookies to cross-origin WebSocket handshakes, so `Origin` is the only thing
distinguishing a legitimate page from an attacker's.
### Build ### Build
```bash ```bash
@@ -380,7 +423,9 @@ binary frames carry data push payloads.
| `ping` | — | Hub replies `{"type":"pong"}` | | `ping` | — | Hub replies `{"type":"pong"}` |
| `addSource` | `label`, `addr` (`"host:port"`), `multicastGroup?`, `dataPort?` | Connect to a new UDPS source; hub assigns id `s1, s2, …` | | `addSource` | `label`, `addr` (`"host:port"`), `multicastGroup?`, `dataPort?` | Connect to a new UDPS source; hub assigns id `s1, s2, …` |
| `removeSource` | `id` | Disconnect and remove a source | | `removeSource` | `id` | Disconnect and remove a source |
| `saveSources` | — | Persist the current dynamic source list to `SourcesFile` (JSON) | | `saveSources` | — | Persist the dynamic source list **and** the calibration table to `SourcesFile`; replies `configSaved` |
| `setCalibration` | `source` (label), `signal` (base name), `scale`, `offset`, `unit` | Record `value = raw × scale + offset` for one signal; metadata only, the hub never applies it. Identity entries are deleted. Replies with a `calibration` broadcast |
| `reloadConfig` | — | Re-read `SourcesFile`: calibration replaced wholesale, missing sources added, live sources never touched; replies `configReloaded` |
| `getSources` | — | Trigger `sources` broadcast | | `getSources` | — | Trigger `sources` broadcast |
| `getConfig` | `sourceId` | Trigger `config` broadcast for one source | | `getConfig` | `sourceId` | Trigger `config` broadcast for one source |
| `getStats` | — | Trigger `stats` broadcast | | `getStats` | — | Trigger `stats` broadcast |
@@ -399,11 +444,33 @@ binary frames carry data push payloads.
| `sources` | `sources:[{id, label, addr:"host:port", state}]` | On connect; after add/remove/getSources; on first CONFIG | | `sources` | `sources:[{id, label, addr:"host:port", state}]` | On connect; after add/remove/getSources; on first CONFIG |
| `config` | `sourceId`, `publishMode`, `signals:[{name, typeCode, quantType, numDimensions, numRows, numCols, rangeMin, rangeMax, timeMode, samplingRate, timeSignalIdx, unit}]` | After CONFIG received from source | | `config` | `sourceId`, `publishMode`, `signals:[{name, typeCode, quantType, numDimensions, numRows, numCols, rangeMin, rangeMax, timeMode, samplingRate, timeSignalIdx, unit}]` | After CONFIG received from source |
| `stats` | `sources:{id:{state, totalReceived, totalLost, rateHz, rateStdHz, fragsPerCycle, bytesPerCycle, cycleAvgMs, cycleStdMs, cycleMinMs, cycleMaxMs, cycleHistMin, cycleHistMax, cycleHist:[20]}}` | At `StatsRate` Hz | | `stats` | `sources:{id:{state, totalReceived, totalLost, rateHz, rateStdHz, fragsPerCycle, bytesPerCycle, cycleAvgMs, cycleStdMs, cycleMinMs, cycleMaxMs, cycleHistMin, cycleHistMax, cycleHist:[20]}}` | At `StatsRate` Hz |
| `triggerState` | `state` (`"idle"`\|`"armed"`\|`"collecting"`\|`"triggered"`), `mode`, `stopped`, `trigTime?` | On any trigger FSM transition | | `triggerState` | `state` (`"idle"`\|`"armed"`\|`"collecting"`\|`"triggered"`), `mode`, `stopped`, `trigTime?`, `preSec?`, `postSec?` | On any trigger FSM transition |
| `zoom` | `reqId`, `signals:{"src:sig":{t:[…], v:[…]}}` (`t` printed `%.17g`, `v` `%.9g`) | Unicast reply to `zoom` | | `zoom` | `reqId`, `signals:{"src:sig":{t:[…], v:[…]}}` (`t` printed `%.17g`, `v` `%.9g`) | Unicast reply to `zoom` |
| `maxPointsUpdated` | `maxPoints` | After ring buffer resize | | `maxPointsUpdated` | `maxPoints` | After ring buffer resize |
| `calibration` | `cal:[{source, signal, scale, offset, unit}]` | On connect; after an accepted `setCalibration`; after a successful `reloadConfig` |
| `configSaved` | `ok`, `path`, `error?` | In reply to `saveSources` |
| `configReloaded` | `ok`, `path`, `error?` | In reply to `reloadConfig` |
| `pong` | — | In reply to `ping` | | `pong` | — | In reply to `ping` |
### Config File Format
`SourcesFile` is a flat JSON array of flat objects; `addr` marks a source,
`signal` marks a calibration entry.
```json
[
{"label": "wave", "addr": "127.0.0.1:44500"},
{"source": "wave", "signal": "Adc", "scale": 0.00030518, "offset": -1.25, "unit": "V"}
]
```
Flatness is a hard constraint: `StreamHub::LoadSourcesFile` scans from each `{`
to the next `}`, so a nested object would truncate the parse. Both hubs read and
write this format identically, and pre-calibration files load unchanged.
Calibration is applied **client-side only**. Rings, history, `zoom` replies, both
binary frames and the trigger comparator are all in raw units.
### Binary Push Frame (version 1, hub → client, binary WS frame) ### Binary Push Frame (version 1, hub → client, binary WS frame)
Little-endian throughout. Sent at `PushRate` Hz per source; contains **only Little-endian throughout. Sent at `PushRate` Hz per source; contains **only
CLAUDE.md
+4 -1
View File
@@ -30,6 +30,9 @@ make -C Source/Components/DataSources/UDPStreamer -f Makefile.gcc
cd Common/Client/go && go build ./... cd Common/Client/go && go build ./...
cd Client/debugger && go build ./... cd Client/debugger && go build ./...
# Standalone C UDPS client library (no MARTe2, libc + BSD sockets only)
cd Common/Client/c && make && make cxxcheck
# ImGui desktop client (not a MARTe2 component; needs SDL2) # ImGui desktop client (not a MARTe2 component; needs SDL2)
cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build cd Client/streamhub && cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build
@@ -81,7 +84,7 @@ Two independent data paths:
1. **Streaming path**: `UDPStreamer` DataSource serialises signals each RT cycle to UDPS binary packets (UDP 44500, unicast/multicast) → `StreamHub` (`Source/Applications/StreamHub/`, headless C++ app: ring buffers, LTTB decimation, trigger FSM) → WebSocket 8090 → browser (`Client/udpstreamer`, Go), native ImGui client (`Client/streamhub`), or native Qt client (`Client/streamhub-qt`). 1. **Streaming path**: `UDPStreamer` DataSource serialises signals each RT cycle to UDPS binary packets (UDP 44500, unicast/multicast) → `StreamHub` (`Source/Applications/StreamHub/`, headless C++ app: ring buffers, LTTB decimation, trigger FSM) → WebSocket 8090 → browser (`Client/udpstreamer`, Go), native ImGui client (`Client/streamhub`), or native Qt client (`Client/streamhub-qt`).
2. **Debug path**: `DebugService` patches the `ClassRegistryDatabase` at `Initialise()` so subsequent `ConfigureApplication()` instantiates `DebugBrokerWrapper<T>` around all `MemoryMap*Broker` types — no application changes. RT hot path goes through `DebugServiceI` (abstract singleton in `DebugServiceI.h`) for forcing/tracing/breakpoints. Exposes TCP 8080 (text commands), UDP 8081 (trace telemetry), works with `TcpLogger` on 8082. Web UI: `Client/debugger` (Go). 2. **Debug path**: `DebugService` patches the `ClassRegistryDatabase` at `Initialise()` so subsequent `ConfigureApplication()` instantiates `DebugBrokerWrapper<T>` around all `MemoryMap*Broker` types — no application changes. RT hot path goes through `DebugServiceI` (abstract singleton in `DebugServiceI.h`) for forcing/tracing/breakpoints. Exposes TCP 8080 (text commands), UDP 8081 (trace telemetry), works with `TcpLogger` on 8082. Web UI: `Client/debugger` (Go).
**Shared wire format**: `Common/UDP/UDPSProtocol.h` defines the UDPS binary protocol (17-byte packed header, 136-byte signal descriptors, little-endian). It is deliberately MARTe2-free so it's shared by C++ producers (`UDPStreamer`, `DebugService`), the C++ consumer (`Source/Components/Interfaces/UDPStream/UDPSClient`), and the Go decoder (`Common/Client/go/udpsprotocol`). Changes to the protocol must be mirrored across all of these, plus the JS client parsers. **Shared wire format**: `Common/UDP/UDPSProtocol.h` defines the UDPS binary protocol (17-byte packed header, 136-byte signal descriptors, little-endian). It is deliberately MARTe2-free so it's shared by C++ producers (`UDPStreamer`, `DebugService`), the C++ consumer (`Source/Components/Interfaces/UDPStream/UDPSClient`), the Go decoder (`Common/Client/go/udpsprotocol`), and the standalone C client (`Common/Client/c`, which redeclares the constants rather than including this header, so it stays MARTe-free). Changes to the protocol must be mirrored across all of these, plus the JS client parsers.
**StreamHub WebSocket protocol**: JSON text frames for commands/events, binary frames for data pushes — spec in `ARCHITECTURE.md` §6. The Go hub (`Client/udpstreamer`) and C++ StreamHub implement the identical protocol; both clients (browser JS and ImGui) must stay compatible with both. **StreamHub WebSocket protocol**: JSON text frames for commands/events, binary frames for data pushes — spec in `ARCHITECTURE.md` §6. The Go hub (`Client/udpstreamer`) and C++ StreamHub implement the identical protocol; both clients (browser JS and ImGui) must stay compatible with both.
Client/streamhub-qt/Hub.cpp
+9 -1
View File
@@ -160,7 +160,15 @@ void Hub::onTriggerState(const std::string& json) {
trigger_.trigTime = msg.trigTime; trigger_.trigTime = msg.trigTime;
trigger_.hasTrigTime = true; trigger_.hasTrigTime = true;
} }
if (msg.state == "idle") { trigger_.hasTrigTime = false; } if (msg.hasWindow) {
trigger_.firedPreS = msg.preSec;
trigger_.firedPostS = msg.postSec;
trigger_.hasFiredWin = true;
}
if (msg.state == "idle") {
trigger_.hasTrigTime = false;
trigger_.hasFiredWin = false;
}
Q_EMIT triggerStateChanged(); Q_EMIT triggerStateChanged();
} }
Client/streamhub-qt/Model.h
+5
View File
@@ -60,6 +60,11 @@ struct TriggerCfgState {
bool stopped = false; bool stopped = false;
bool hasTrigTime = false; bool hasTrigTime = false;
double trigTime = 0.0; double trigTime = 0.0;
/* Window the hub latched at fire time. Not the same as windowSec/prePercent
* above, which are editable and may have moved on since the trigger fired. */
bool hasFiredWin = false;
double firedPreS = 0.0;
double firedPostS = 0.0;
}; };
/** Per-signal vertical scale state (oscilloscope style). */ /** Per-signal vertical scale state (oscilloscope style). */
Client/streamhub-qt/PlotWidget.cpp
+233 -77
View File
@@ -78,6 +78,49 @@ static double normalizeY(double raw, const VScale& vs) {
return (raw - vs.resolvedOffset) / vs.resolvedDiv + vs.screenPos; return (raw - vs.resolvedOffset) / vs.resolvedDiv + vs.screenPos;
} }
/* Resolve the one scale every trace shares in unified mode: same rules as the
* per-signal version applied to the union of the plot — range takes the union
* of the declared ranges, auto fits the union of the data. */
static void resolveUnifiedVScale(VScale& vs,
const std::vector<PlotAssignment>& slots,
const std::vector<Source>& sources,
const std::vector<std::vector<double> >& vStore) {
if (vs.mode == 2) {
vs.resolvedDiv = std::max(vs.divValue, 1e-30);
vs.resolvedOffset = vs.offset;
return;
}
double mn = 1e300, mx = -1e300;
if (vs.mode == 1) {
for (const auto& a : slots) {
if (a.sourceIdx < 0 || a.sourceIdx >= (int)sources.size()) continue;
if (a.signalIdx < 0 ||
a.signalIdx >= (int)sources[a.sourceIdx].signals.size()) continue;
const auto& m = sources[a.sourceIdx].signals[a.signalIdx].meta;
if (!(m.rangeMax > m.rangeMin)) continue;
if (m.rangeMin < mn) mn = m.rangeMin;
if (m.rangeMax > mx) mx = m.rangeMax;
}
if (mx > mn) {
vs.resolvedDiv = std::max((mx - mn) / 8.0, 1e-30);
vs.resolvedOffset = (mn + mx) / 2.0;
return;
}
mn = 1e300; mx = -1e300; /* no usable range: fall through to auto */
}
for (const auto& vv : vStore) {
for (double v : vv) {
if (!std::isfinite(v)) continue;
if (v < mn) mn = v;
if (v > mx) mx = v;
}
}
if (!std::isfinite(mn) || mn > mx) { mn = -1.0; mx = 1.0; }
if (mn == mx) { mn -= 1.0; mx += 1.0; }
vs.resolvedDiv = std::max((mx - mn) / 6.0, 1e-30);
vs.resolvedOffset = (mx + mn) / 2.0;
}
static bool dataMinMax(const std::vector<double>& v, double& mn, double& mx) { static bool dataMinMax(const std::vector<double>& v, double& mn, double& mx) {
mn = 1e300; mx = -1e300; mn = 1e300; mx = -1e300;
for (double x : v) { if (std::isfinite(x)) { if (x < mn) mn = x; if (x > mx) mx = x; } } for (double x : v) { if (std::isfinite(x)) { if (x < mn) mn = x; if (x > mx) mx = x; } }
@@ -189,6 +232,50 @@ void PlotCanvas::drawMarker(QPainter& p, double cx, double cy, int marker, doubl
} }
} }
/** @brief What the plot renders on the trigger-relative axis, if anything. */
struct TrigView {
bool rel = false; /* render against t - trig instead of wall clock */
bool fromCap = false; /* data comes from the capture frame, not the ring */
double trigT = 0.0;
double preS = 0.0;
double postS = 0.0;
};
/* Two ways to end up in trigger-relative time. Either a v2 capture frame has
* arrived, or a trigger has fired and its window is still filling. In the
* second case the hub sends nothing until the whole window has been produced —
* several seconds for a long window at a high rate — so the trace is drawn from
* the local rings onto the final axis, growing left to right. Filling wins
* over the last capture: once a new trigger fires the old waveform is history.
* A capture latches its own pre/post at fire time, so later edits in the
* trigger bar must not move the axis of a finished capture. */
static TrigView resolveTrigView(Hub* hub, const GlobalView* gv, bool paused) {
TrigView tv;
if (!gv->trigView) { return tv; }
const TriggerCfgState& t = hub->trigger();
if (!paused && t.status == "collecting" && t.hasTrigTime) {
tv.rel = true;
tv.trigT = t.trigTime;
/* Prefer the window the hub latched at fire time; the local config is
* only a fallback for hubs that do not report it, and may have been
* edited since the trigger fired. */
tv.preS = t.hasFiredWin ? t.firedPreS
: t.windowSec * t.prePercent * 0.01;
tv.postS = t.hasFiredWin ? t.firedPostS : t.windowSec - tv.preS;
return tv;
}
const CaptureFrame* cap = hub->capture();
if (cap != nullptr) {
tv.rel = true;
tv.fromCap = true;
tv.trigT = cap->trigTime;
tv.preS = cap->preSec;
tv.postS = cap->postSec;
}
return tv;
}
void PlotCanvas::paintEvent(QPaintEvent*) { void PlotCanvas::paintEvent(QPaintEvent*) {
QPainter p(this); QPainter p(this);
p.setRenderHint(QPainter::Antialiasing, true); p.setRenderHint(QPainter::Antialiasing, true);
@@ -205,13 +292,14 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
p.fillRect(rect(), col::base()); p.fillRect(rect(), col::base());
p.fillRect(r, col::crust()); p.fillRect(r, col::crust());
const CaptureFrame* cap = hub->capture();
const bool trigView = (cap != nullptr) && gv->trigView;
auto& zc = hub->zoomCache(w_->plotIdx_); auto& zc = hub->zoomCache(w_->plotIdx_);
auto& hc = hub->histZoomCache(w_->plotIdx_); auto& hc = hub->histZoomCache(w_->plotIdx_);
const bool paused = w_->paused_; const bool paused = w_->paused_;
bool& live = w_->live_; bool& live = w_->live_;
const TrigView tv = resolveTrigView(hub, gv, paused);
const CaptureFrame* cap = hub->capture();
/* ── pause snapshot ─────────────────────────────────────────────────── */ /* ── pause snapshot ─────────────────────────────────────────────────── */
auto& snap = w_->snap_; auto& snap = w_->snap_;
if (paused) { if (paused) {
@@ -239,15 +327,15 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
/* ── gather data per slot ───────────────────────────────────────────── */ /* ── gather data per slot ───────────────────────────────────────────── */
std::vector<std::vector<double>> tStore(slots.size()), vStore(slots.size()); std::vector<std::vector<double>> tStore(slots.size()), vStore(slots.size());
const bool liveHiRes = !trigView && live && !paused && const bool liveHiRes = !tv.rel && live && !paused &&
gv->windowSec <= kLiveHiResMaxWin && zc.valid && gv->windowSec <= kLiveHiResMaxWin && zc.valid &&
(zc.t1 - zc.t0) >= gv->windowSec * 0.9 && (wallNow - zc.t1) < 3.0; (zc.t1 - zc.t0) >= gv->windowSec * 0.9 && (wallNow - zc.t1) < 3.0;
const bool useZoomData = !trigView && !paused && zc.valid && const bool useZoomData = !tv.rel && !paused && zc.valid &&
(liveHiRes || (liveHiRes ||
(!live && zc.t0 <= w_->plotXMin_ + 1e-9 && zc.t1 >= w_->plotXMax_ - 1e-9)); (!live && zc.t0 <= w_->plotXMin_ + 1e-9 && zc.t1 >= w_->plotXMax_ - 1e-9));
bool useHistData = !trigView && !paused && !live && hc.valid && bool useHistData = !tv.rel && !paused && !live && hc.valid &&
hc.t0 <= w_->plotXMin_ + 1e-9 && hc.t1 >= w_->plotXMax_ - 1e-9; hc.t0 <= w_->plotXMin_ + 1e-9 && hc.t1 >= w_->plotXMax_ - 1e-9;
if (useHistData) { if (useHistData) {
bool any = false; bool any = false;
@@ -271,17 +359,25 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
const auto& sig = sources[a.sourceIdx].signals[a.signalIdx]; const auto& sig = sources[a.sourceIdx].signals[a.signalIdx];
const std::string key = hub->slotKey(a); const std::string key = hub->slotKey(a);
if (trigView) { if (tv.fromCap) {
for (const auto& cs : cap->signals) { for (const auto& cs : cap->signals) {
if (cs.key != key) continue; if (cs.key != key) continue;
size_t n = std::min(cs.t.size(), cs.v.size()); size_t n = std::min(cs.t.size(), cs.v.size());
tStore[si].reserve(n); vStore[si].reserve(n); tStore[si].reserve(n); vStore[si].reserve(n);
for (size_t i = 0; i < n; i++) { for (size_t i = 0; i < n; i++) {
tStore[si].push_back(cs.t[i] - cap->trigTime); tStore[si].push_back(cs.t[i] - tv.trigT);
vStore[si].push_back(cs.v[i]); vStore[si].push_back(cs.v[i]);
} }
break; break;
} }
} else if (tv.rel) {
/* Filling: local ring, clipped to the (absolute) trigger window and
* shifted onto the trigger-relative axis. */
sig.buf.readRange(tv.trigT - tv.preS, tv.trigT + tv.postS,
tStore[si], vStore[si]);
for (size_t i = 0; i < tStore[si].size(); i++) {
tStore[si][i] -= tv.trigT;
}
} else if (useZoomData) { } else if (useZoomData) {
bool found = false; bool found = false;
for (const auto& zs : zc.pts) { for (const auto& zs : zc.pts) {
@@ -302,11 +398,18 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
resolveVScale(a, sig, vStore[si]); resolveVScale(a, sig, vStore[si]);
} }
if (w_->vMode_ == 3) {
resolveUnifiedVScale(w_->uniVS_, slots, sources, vStore);
}
/* ── X range ────────────────────────────────────────────────────────── */ /* ── X range ────────────────────────────────────────────────────────── */
double xMin, xMax; double xMin, xMax;
if (trigView) { if (tv.rel) {
if (w_->trigZoomed_) { xMin = w_->plotXMin_; xMax = w_->plotXMax_; } if (w_->trigZoomed_) { xMin = w_->plotXMin_; xMax = w_->plotXMax_; }
else { xMin = -cap->preSec; xMax = cap->postSec; } /* Full window from the start, even while filling: a trace growing into
* a fixed axis reads as progress, whereas an axis that grows with the
* data shifts the whole trace every frame. */
else { xMin = -tv.preS; xMax = tv.postS; }
} else if (live && !paused) { } else if (live && !paused) {
if (liveHiRes) { xMax = zc.t1; xMin = zc.t1 - gv->windowSec; } if (liveHiRes) { xMax = zc.t1; xMin = zc.t1 - gv->windowSec; }
else { xMax = wallNow; xMin = wallNow - gv->windowSec; } else { xMax = wallNow; xMin = wallNow - gv->windowSec; }
@@ -319,19 +422,25 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
/* ── grid + ticks ───────────────────────────────────────────────────── */ /* ── grid + ticks ───────────────────────────────────────────────────── */
p.setPen(QPen(QColor(0x31,0x32,0x44,160), 1.0)); p.setPen(QPen(QColor(0x31,0x32,0x44,160), 1.0));
/* Y grid: 9 division lines */ /* Y grid: 9 division lines */
const auto& av = (w_->vMode_ == 0 && w_->activeSlot_ >= 0 && /* Which scale labels the axis: the active signal's in normal mode, the one
w_->activeSlot_ < (int)slots.size()) * the whole plot shares in unified mode (where nothing has to be selected).
? slots[w_->activeSlot_].vs : VScale(); * Banded modes have no single scale, so they keep the plain division numbers. */
const VScale* axisVS = nullptr;
if (w_->vMode_ == 0 && w_->activeSlot_ >= 0 &&
w_->activeSlot_ < (int)slots.size()) {
axisVS = &slots[w_->activeSlot_].vs;
} else if (w_->vMode_ == 3) {
axisVS = &w_->uniVS_;
}
p.setFont(QFont(font().family(), 8)); p.setFont(QFont(font().family(), 8));
for (int d = -4; d <= 4; d++) { for (int d = -4; d <= 4; d++) {
double y = yToPx(d, r); double y = yToPx(d, r);
p.setPen(QPen(QColor(0x31,0x32,0x44, d==0?220:120), d==0?1.2:1.0)); p.setPen(QPen(QColor(0x31,0x32,0x44, d==0?220:120), d==0?1.2:1.0));
p.drawLine(QPointF(r.left(), y), QPointF(r.right(), y)); p.drawLine(QPointF(r.left(), y), QPointF(r.right(), y));
QString lbl; QString lbl;
if (w_->vMode_ == 0 && w_->activeSlot_ >= 0 && if (axisVS != nullptr) {
w_->activeSlot_ < (int)slots.size()) { lbl = fmtVal(axisVS->resolvedOffset +
double rawVal = av.resolvedOffset + (d - av.screenPos) * av.resolvedDiv; (d - axisVS->screenPos) * axisVS->resolvedDiv);
lbl = fmtVal(rawVal);
} else { } else {
lbl = QString::number(d); lbl = QString::number(d);
} }
@@ -346,7 +455,7 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
p.setPen(QPen(QColor(0x31,0x32,0x44,120), 1.0)); p.setPen(QPen(QColor(0x31,0x32,0x44,120), 1.0));
p.drawLine(QPointF(x, r.top()), QPointF(x, r.bottom())); p.drawLine(QPointF(x, r.top()), QPointF(x, r.bottom()));
p.setPen(QColor(0xa6,0xad,0xc8)); p.setPen(QColor(0xa6,0xad,0xc8));
QString xl = trigView ? fmtVal(xv) + "s" : QString::number(xv, 'f', 3); QString xl = tv.rel ? fmtVal(xv) + "s" : QString::number(xv, 'f', 3);
int flags = (t==0?Qt::AlignLeft:(t==10?Qt::AlignRight:Qt::AlignHCenter)) int flags = (t==0?Qt::AlignLeft:(t==10?Qt::AlignRight:Qt::AlignHCenter))
| Qt::AlignTop; | Qt::AlignTop;
p.drawText(QRectF(x-40, r.bottom()+2, 80, 14), flags, xl); p.drawText(QRectF(x-40, r.bottom()+2, 80, 14), flags, xl);
@@ -396,8 +505,10 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
if (w_->vMode_ == 1) bandNormalize(vDec, vNorm, myKi, nTraces, true); if (w_->vMode_ == 1) bandNormalize(vDec, vNorm, myKi, nTraces, true);
else if (w_->vMode_ == 2) bandNormalize(vDec, vNorm, myKi, nTraces, a.vs.digitalInMixed); else if (w_->vMode_ == 2) bandNormalize(vDec, vNorm, myKi, nTraces, a.vs.digitalInMixed);
else { else {
/* unified shares one scale, normal gives each trace its own */
const VScale& nvs = (w_->vMode_ == 3) ? w_->uniVS_ : a.vs;
vNorm.resize(nOut); vNorm.resize(nOut);
for (size_t k = 0; k < nOut; k++) vNorm[k] = normalizeY(vDec[k], a.vs); for (size_t k = 0; k < nOut; k++) vNorm[k] = normalizeY(vDec[k], nvs);
} }
QColor c = sig.color; QColor c = sig.color;
@@ -423,7 +534,7 @@ void PlotCanvas::paintEvent(QPaintEvent*) {
} }
/* trigger instant marker at t=0 */ /* trigger instant marker at t=0 */
if (trigView) { if (tv.rel) {
double x = xToPx(0.0, xMin, xMax, r); double x = xToPx(0.0, xMin, xMax, r);
p.setPen(QPen(QColor(255,255,0,200), 1.5, Qt::DashLine)); p.setPen(QPen(QColor(255,255,0,200), 1.5, Qt::DashLine));
p.drawLine(QPointF(x, r.top()), QPointF(x, r.bottom())); p.drawLine(QPointF(x, r.top()), QPointF(x, r.bottom()));
@@ -476,8 +587,7 @@ void PlotCanvas::wheelEvent(QWheelEvent* e) {
Hub* hub = w_->hub_; Hub* hub = w_->hub_;
GlobalView* gv = w_->gv_; GlobalView* gv = w_->gv_;
auto& slots = w_->slots_; auto& slots = w_->slots_;
const CaptureFrame* cap = hub->capture(); const TrigView tv = resolveTrigView(hub, gv, w_->paused_);
const bool trigView = (cap != nullptr) && gv->trigView;
bool& live = w_->live_; bool& live = w_->live_;
double dy = e->angleDelta().y(); double dy = e->angleDelta().y();
@@ -488,43 +598,51 @@ void PlotCanvas::wheelEvent(QWheelEvent* e) {
const double now = nowSec(); const double now = nowSec();
auto enterTrigZoom = [&]() { auto enterTrigZoom = [&]() {
if (trigView && !w_->trigZoomed_) { if (tv.rel && !w_->trigZoomed_) {
w_->setStoredX(-cap->preSec, cap->postSec); w_->setStoredX(-tv.preS, tv.postS);
w_->trigZoomed_ = true; w_->trigZoomed_ = true;
} }
}; };
auto xZoomStored = [&](double f) { auto xZoomStored = [&](double f) {
if (trigView) enterTrigZoom(); if (tv.rel) enterTrigZoom();
if (now - w_->lastHistPushMs_ > 0.6) { w_->pushZoomHist(); w_->lastHistPushMs_ = now; } if (now - w_->lastHistPushMs_ > 0.6) { w_->pushZoomHist(); w_->lastHistPushMs_ = now; }
double cx = (w_->plotXMin_ + w_->plotXMax_) * 0.5; double cx = (w_->plotXMin_ + w_->plotXMax_) * 0.5;
double half = (w_->plotXMax_ - w_->plotXMin_) * 0.5 * f; double half = (w_->plotXMax_ - w_->plotXMin_) * 0.5 * f;
w_->setStoredX(cx - half, cx + half); w_->setStoredX(cx - half, cx + half);
}; };
auto makeManual = [&](PlotAssignment& a) { /* Seed manual from the resolved values so the gesture sticks. */
if (a.vs.mode != 2) { auto makeManual = [&](VScale& vs) {
a.vs.divValue = std::max(a.vs.resolvedDiv, 1e-30); if (vs.mode != 2) {
a.vs.offset = a.vs.resolvedOffset; vs.divValue = std::max(vs.resolvedDiv, 1e-30);
a.vs.mode = 2; vs.offset = vs.resolvedOffset;
vs.mode = 2;
} }
}; };
/* Scroll adjusts the scale the axis is labelled with: the active signal's in
* normal mode, the plot's shared one in unified mode (nothing to select). */
VScale* wheelVS = nullptr;
if (w_->vMode_ == 3) {
wheelVS = &w_->uniVS_;
} else if (w_->activeSlot_ >= 0 && w_->activeSlot_ < (int)slots.size()) {
wheelVS = &slots[w_->activeSlot_].vs;
}
if (ctrl) { if (ctrl) {
if (!trigView && live) gv->windowSec = std::clamp(gv->windowSec*factor, 1e-4, 3600.0); if (!tv.rel && live) gv->windowSec = std::clamp(gv->windowSec*factor, 1e-4, 3600.0);
else xZoomStored(factor); else xZoomStored(factor);
} else if (shift) { } else if (shift) {
if (w_->activeSlot_ >= 0 && w_->activeSlot_ < (int)slots.size()) { if (wheelVS != nullptr) {
auto& a = slots[w_->activeSlot_]; makeManual(*wheelVS);
makeManual(a); wheelVS->screenPos += (dy > 0) ? 0.5 : -0.5;
a.vs.screenPos += (dy > 0) ? 0.5 : -0.5;
} }
} else { } else {
if (w_->activeSlot_ >= 0 && w_->activeSlot_ < (int)slots.size()) { if (wheelVS != nullptr) {
auto& a = slots[w_->activeSlot_]; makeManual(*wheelVS);
makeManual(a); wheelVS->divValue = std::max(wheelVS->divValue * factor, 1e-30);
a.vs.divValue = std::max(a.vs.divValue * factor, 1e-30);
} else { } else {
if (!trigView && live) gv->windowSec = std::clamp(gv->windowSec*factor, 1e-4, 3600.0); if (!tv.rel && live) gv->windowSec = std::clamp(gv->windowSec*factor, 1e-4, 3600.0);
else xZoomStored(factor); else xZoomStored(factor);
} }
} }
@@ -549,8 +667,7 @@ void PlotCanvas::mouseMoveEvent(QMouseEvent* e) {
GlobalView* gv = w_->gv_; GlobalView* gv = w_->gv_;
Hub* hub = w_->hub_; Hub* hub = w_->hub_;
const QRectF r = plotRect(); const QRectF r = plotRect();
const CaptureFrame* cap = hub->capture(); const TrigView tv = resolveTrigView(hub, gv, w_->paused_);
const bool trigView = (cap != nullptr) && gv->trigView;
bool& live = w_->live_; bool& live = w_->live_;
if (dragCursor_ != 0) { if (dragCursor_ != 0) {
@@ -560,11 +677,11 @@ void PlotCanvas::mouseMoveEvent(QMouseEvent* e) {
return; return;
} }
if (panning_) { if (panning_) {
if (trigView && !w_->trigZoomed_) { if (tv.rel && !w_->trigZoomed_) {
w_->setStoredX(-cap->preSec, cap->postSec); w_->setStoredX(-tv.preS, tv.postS);
w_->trigZoomed_ = true; w_->trigZoomed_ = true;
} }
if (!trigView && live) { w_->initPlotX(nowSec()); live = false; } if (!tv.rel && live) { w_->initPlotX(nowSec()); live = false; }
double dxPix = e->pos().x() - lastPos_.x(); double dxPix = e->pos().x() - lastPos_.x();
lastPos_ = e->pos(); lastPos_ = e->pos();
double xRange = w_->plotXMax_ - w_->plotXMin_; double xRange = w_->plotXMax_ - w_->plotXMin_;
@@ -677,11 +794,10 @@ void PlotWidget::onCaptureReceived() {
void PlotWidget::tick() { void PlotWidget::tick() {
Hub* hub = hub_; Hub* hub = hub_;
GlobalView* gv = gv_; GlobalView* gv = gv_;
const CaptureFrame* cap = hub->capture(); const TrigView tv = resolveTrigView(hub, gv, paused_);
const bool trigView = (cap != nullptr) && gv->trigView;
const double now = nowSec(); const double now = nowSec();
if (!trigView && !paused_) { if (!tv.rel && !paused_) {
std::string csv; std::string csv;
for (const auto& a : slots_) { for (const auto& a : slots_) {
std::string k = hub->slotKey(a); std::string k = hub->slotKey(a);
@@ -736,9 +852,13 @@ void PlotWidget::rebuildHeader() {
auto* b = new QToolButton(header_); auto* b = new QToolButton(header_);
b->setCheckable(true); b->setCheckable(true);
b->setChecked(activeSlot_ == i); b->setChecked(activeSlot_ == i);
b->setText(QString("%1 %2/div") /* In unified mode every badge would repeat the same div value, which
.arg(QString::fromStdString(sig.meta.name)) * the header's Y-Scale button already shows — so show just the name. */
.arg(fmtVal(a.vs.resolvedDiv))); b->setText(vMode_ == 3
? QString::fromStdString(sig.meta.name)
: QString("%1 %2/div")
.arg(QString::fromStdString(sig.meta.name))
.arg(fmtVal(a.vs.resolvedDiv)));
QColor c = sig.color; QColor c = sig.color;
QString fg = (activeSlot_ == i) ? "#11111b" : "#11111b"; QString fg = (activeSlot_ == i) ? "#11111b" : "#11111b";
QColor bg = (activeSlot_ == i) ? col::blue() : c; QColor bg = (activeSlot_ == i) ? col::blue() : c;
@@ -797,11 +917,17 @@ void PlotWidget::rebuildHeader() {
headerLay_->addWidget(fit); headerLay_->addWidget(fit);
} }
/* N / D / M */ /* N / U / D / M */
const char* vl[3] = {"N", "D", "M"}; const char* vl[4] = {"N", "U", "D", "M"};
for (int vm = 0; vm < 3; vm++) { const char* vtip[4] = {"Normal: one vertical scale per signal",
"Unified: one vertical scale shared by every signal",
"Digital", "Mixed"};
const int vmode[4] = {0, 3, 1, 2};
for (int i = 0; i < 4; i++) {
const int vm = vmode[i];
auto* vb = new QToolButton(header_); auto* vb = new QToolButton(header_);
vb->setText(vl[vm]); vb->setText(vl[i]);
vb->setToolTip(vtip[i]);
vb->setCheckable(true); vb->setCheckable(true);
vb->setChecked(vMode_ == vm); vb->setChecked(vMode_ == vm);
connect(vb, &QToolButton::clicked, this, [this, vm]() { connect(vb, &QToolButton::clicked, this, [this, vm]() {
@@ -810,9 +936,57 @@ void PlotWidget::rebuildHeader() {
headerLay_->addWidget(vb); headerLay_->addWidget(vb);
} }
/* Unified mode's single scale belongs to the plot, not to any one signal,
* so it is edited from here rather than from a badge's context menu. */
if (vMode_ == 3) {
auto* yb = new QToolButton(header_);
yb->setText(QString("Y-Scale: %1/div").arg(fmtVal(uniVS_.resolvedDiv)));
yb->setToolTip("Vertical scale shared by every signal in this plot");
connect(yb, &QToolButton::clicked, this, [this, yb]() {
showUnifiedVScaleMenu(yb->mapToGlobal(QPoint(0, yb->height())));
});
headerLay_->addWidget(yb);
}
headerLay_->addStretch(1); headerLay_->addStretch(1);
} }
/** Populate @a vs with the Auto/Range/Manual entries driving @a evs. */
void PlotWidget::buildVScaleMenu(QMenu* vs, VScale& evs) {
const char* modes[] = {"Auto", "Range", "Manual"};
for (int mm = 0; mm < 3; mm++) {
QAction* act = vs->addAction(modes[mm]);
act->setCheckable(true); act->setChecked(evs.mode == mm);
connect(act, &QAction::triggered, this, [this, &evs, mm]() {
evs.mode = mm; rebuildHeader(); canvas_->update();
});
}
vs->addSeparator();
vs->addAction("Manual V/div…", [this, &evs]() {
bool ok; double v = QInputDialog::getDouble(this, "V/div", "Units per division",
evs.mode==2?evs.divValue:evs.resolvedDiv, -1e12, 1e12, 6, &ok);
if (ok) { evs.divValue = v; evs.mode = 2; rebuildHeader(); canvas_->update(); }
});
vs->addAction("Offset…", [this, &evs]() {
bool ok; double v = QInputDialog::getDouble(this, "Offset", "Center value",
evs.mode==2?evs.offset:evs.resolvedOffset, -1e12, 1e12, 6, &ok);
if (ok) { evs.offset = v; evs.mode = 2; rebuildHeader(); canvas_->update(); }
});
vs->addAction("Position (div)…", [this, &evs]() {
bool ok; double v = QInputDialog::getDouble(this, "Position", "Divisions from center",
evs.screenPos, -8, 8, 2, &ok);
if (ok) { evs.screenPos = v; canvas_->update(); }
});
}
void PlotWidget::showUnifiedVScaleMenu(const QPoint& globalPos) {
QMenu m;
m.addAction("Y-Scale — all signals")->setEnabled(false);
m.addSeparator();
buildVScaleMenu(&m, uniVS_);
m.exec(globalPos);
}
void PlotWidget::showBadgeMenu(int slotIdx, const QPoint& globalPos) { void PlotWidget::showBadgeMenu(int slotIdx, const QPoint& globalPos) {
auto& sources = hub_->sources(); auto& sources = hub_->sources();
if (slotIdx < 0 || slotIdx >= (int)slots_.size()) return; if (slotIdx < 0 || slotIdx >= (int)slots_.size()) return;
@@ -846,30 +1020,12 @@ void PlotWidget::showBadgeMenu(int slotIdx, const QPoint& globalPos) {
connect(dg, &QAction::toggled, this, [&](bool on){ a.vs.digitalInMixed = on; canvas_->update(); }); connect(dg, &QAction::toggled, this, [&](bool on){ a.vs.digitalInMixed = on; canvas_->update(); });
} }
m.addSeparator(); /* In unified mode the plot has one scale for every trace, so it is edited
QMenu* vs = m.addMenu("V-scale"); * from the header's Y-Scale button instead of from any one signal. */
const char* modes[] = {"Auto", "Range", "Manual"}; if (vMode_ != 3) {
for (int mm = 0; mm < 3; mm++) { m.addSeparator();
QAction* act = vs->addAction(modes[mm]); buildVScaleMenu(m.addMenu("V-scale"), a.vs);
act->setCheckable(true); act->setChecked(a.vs.mode == mm);
connect(act, &QAction::triggered, this, [&, mm]() { a.vs.mode = mm; rebuildHeader(); canvas_->update(); });
} }
vs->addSeparator();
vs->addAction("Manual V/div…", [&]() {
bool ok; double v = QInputDialog::getDouble(this, "V/div", "Units per division",
a.vs.mode==2?a.vs.divValue:a.vs.resolvedDiv, -1e12, 1e12, 6, &ok);
if (ok) { a.vs.divValue = v; a.vs.mode = 2; rebuildHeader(); canvas_->update(); }
});
vs->addAction("Offset…", [&]() {
bool ok; double v = QInputDialog::getDouble(this, "Offset", "Center value",
a.vs.mode==2?a.vs.offset:a.vs.resolvedOffset, -1e12, 1e12, 6, &ok);
if (ok) { a.vs.offset = v; a.vs.mode = 2; rebuildHeader(); canvas_->update(); }
});
vs->addAction("Position (div)…", [&]() {
bool ok; double v = QInputDialog::getDouble(this, "Position", "Divisions from center",
a.vs.screenPos, -8, 8, 2, &ok);
if (ok) { a.vs.screenPos = v; canvas_->update(); }
});
m.addSeparator(); m.addSeparator();
m.addAction("Remove from plot", [&]() { m.addAction("Remove from plot", [&]() {
Client/streamhub-qt/PlotWidget.h
+5 -1
View File
@@ -21,6 +21,7 @@
class QHBoxLayout; class QHBoxLayout;
class QToolButton; class QToolButton;
class QLabel; class QLabel;
class QMenu;
namespace shq { namespace shq {
@@ -69,6 +70,8 @@ private:
friend class PlotCanvas; friend class PlotCanvas;
void rebuildHeader(); void rebuildHeader();
void buildVScaleMenu(QMenu* vs, VScale& evs);
void showUnifiedVScaleMenu(const QPoint& globalPos);
void showBadgeMenu(int slotIdx, const QPoint& globalPos); void showBadgeMenu(int slotIdx, const QPoint& globalPos);
void pushZoomHist(); void pushZoomHist();
void initPlotX(double tMax); void initPlotX(double tMax);
@@ -87,7 +90,8 @@ private:
bool paused_ = false; bool paused_ = false;
double plotXMin_ = 0.0; double plotXMin_ = 0.0;
double plotXMax_ = 0.0; double plotXMax_ = 0.0;
int vMode_ = 0; /* 0 normal 1 digital 2 mixed */ int vMode_ = 0; /* 0 normal 1 digital 2 mixed 3 unified */
VScale uniVS_; /* the one scale every trace shares in mode 3 */
int activeSlot_ = -1; int activeSlot_ = -1;
bool trigZoomed_ = false; bool trigZoomed_ = false;
Client/streamhub/App.cpp
+6
View File
@@ -825,11 +825,17 @@ void App::onTriggerState(const std::string& json) {
trigger_.trigTime = msg.trigTime; trigger_.trigTime = msg.trigTime;
trigger_.hasTrigTime = true; trigger_.hasTrigTime = true;
} }
if (msg.hasWindow) {
trigger_.firedPreS = msg.preSec;
trigger_.firedPostS = msg.postSec;
trigger_.hasFiredWin = true;
}
/* Double-buffer semantics: the last recorded capture stays on display /* Double-buffer semantics: the last recorded capture stays on display
* (even while re-armed/collecting) and is only replaced when a new * (even while re-armed/collecting) and is only replaced when a new
* capture frame has been fully received and parsed (handleBinary v2). */ * capture frame has been fully received and parsed (handleBinary v2). */
if (msg.state == "idle") { if (msg.state == "idle") {
trigger_.hasTrigTime = false; trigger_.hasTrigTime = false;
trigger_.hasFiredWin = false;
} }
} }
Client/streamhub/App.h
+11 -2
View File
@@ -61,6 +61,11 @@ struct TriggerState {
bool stopped = false; bool stopped = false;
bool hasTrigTime = false; bool hasTrigTime = false;
double trigTime = 0.0; double trigTime = 0.0;
/* Window the hub latched at fire time. Not the same as windowSec/prePercent
* above, which are editable and may have moved on since the trigger fired. */
bool hasFiredWin = false;
double firedPreS = 0.0;
double firedPostS = 0.0;
}; };
/** Per-signal vertical scale state (oscilloscope style). */ /** Per-signal vertical scale state (oscilloscope style). */
@@ -183,9 +188,12 @@ public:
plotXMax_[i] = tMax; plotXMax_[i] = tMax;
} }
/** @brief Per-plot vertical normalisation: 0=normal 1=digital 2=mixed. */ /** @brief Per-plot vertical normalisation: 0=normal 1=digital 2=mixed 3=unified. */
int& plotVMode(int i) { return plotVMode_[i]; } int& plotVMode(int i) { return plotVMode_[i]; }
/** @brief The one scale every trace shares in unified mode (vMode 3). */
VScale& plotUnifiedVS(int i) { return plotUniVS_[i]; }
/* ---- Cursors A/B (global: shared & synchronised across all plots) ---- */ /* ---- Cursors A/B (global: shared & synchronised across all plots) ---- */
bool& cursorsOn() { return cursorsOn_; } bool& cursorsOn() { return cursorsOn_; }
double& cursorA() { return cursorA_; } double& cursorA() { return cursorA_; }
@@ -302,7 +310,8 @@ private:
double windowSec_ = 10.0; /* live scroll window width */ double windowSec_ = 10.0; /* live scroll window width */
double plotXMin_[kMaxPlotSlots] = {}; /* stored X min for non-live mode */ double plotXMin_[kMaxPlotSlots] = {}; /* stored X min for non-live mode */
double plotXMax_[kMaxPlotSlots] = {}; /* stored X max for non-live mode */ double plotXMax_[kMaxPlotSlots] = {}; /* stored X max for non-live mode */
int plotVMode_[kMaxPlotSlots] = {}; /* 0=normal 1=digital 2=mixed */ int plotVMode_[kMaxPlotSlots] = {}; /* 0=normal 1=digital 2=mixed 3=unified */
VScale plotUniVS_[kMaxPlotSlots]; /* shared scale used by vMode 3 */
/* Cursors (global) */ /* Cursors (global) */
bool cursorsOn_ = false; bool cursorsOn_ = false;
Client/streamhub/PlotPanel.cpp
+192 -61
View File
@@ -85,6 +85,49 @@ static double normalizeY(double raw, const VScale& vs) {
return (raw - vs.resolvedOffset) / vs.resolvedDiv + vs.screenPos; return (raw - vs.resolvedOffset) / vs.resolvedDiv + vs.screenPos;
} }
/** Resolve the one scale every trace shares in unified mode.
*
* Same rules as the per-signal version, applied to the union of the plot:
* range takes the union of the declared ranges, auto fits the union of the
* data. Signals whose slot is empty contribute nothing. */
static void resolveUnifiedVScale(VScale& vs,
const std::vector<PlotAssignment>& slots,
const std::vector<Source>& sources,
const std::vector<std::vector<double> >& vStore) {
if (vs.mode == 2) { /* manual */
vs.resolvedDiv = std::max(vs.divValue, 1e-30);
vs.resolvedOffset = vs.offset;
return;
}
double mn = 1e300, mx = -1e300;
if (vs.mode == 1) { /* range: union of every declared range */
for (const auto& a : slots) {
if (a.sourceIdx < 0 || a.sourceIdx >= (int)sources.size()) continue;
const auto& m = sources[a.sourceIdx].signals[a.signalIdx].meta;
if (!(m.rangeMax > m.rangeMin)) continue;
if (m.rangeMin < mn) mn = m.rangeMin;
if (m.rangeMax > mx) mx = m.rangeMax;
}
if (mx > mn) {
vs.resolvedDiv = std::max((mx - mn) / 8.0, 1e-30);
vs.resolvedOffset = (mn + mx) / 2.0;
return;
}
mn = 1e300; mx = -1e300; /* no usable range: fall through to auto */
}
for (const auto& vv : vStore) {
for (double v : vv) {
if (!std::isfinite(v)) continue;
if (v < mn) mn = v;
if (v > mx) mx = v;
}
}
if (!std::isfinite(mn) || mn > mx) { mn = -1.0; mx = 1.0; }
if (mn == mx) { mn -= 1.0; mx += 1.0; }
vs.resolvedDiv = std::max((mx - mn) / 6.0, 1e-30);
vs.resolvedOffset = (mx + mn) / 2.0;
}
/** Min/max of a vector (returns false if empty/non-finite). */ /** Min/max of a vector (returns false if empty/non-finite). */
static bool dataMinMax(const std::vector<double>& v, double& mn, double& mx) { static bool dataMinMax(const std::vector<double>& v, double& mn, double& mx) {
mn = 1e300; mx = -1e300; mn = 1e300; mx = -1e300;
@@ -149,9 +192,36 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
const double wallNow = std::chrono::duration<double>( const double wallNow = std::chrono::duration<double>(
std::chrono::system_clock::now().time_since_epoch()).count(); std::chrono::system_clock::now().time_since_epoch()).count();
/* Trigger view: render the hub capture relative to the trigger instant */ /* Trigger view: render the hub capture relative to the trigger instant.
*
* Two ways to end up in trigger-relative time. Either a v2 capture frame
* has arrived (trigView), or a trigger has fired and its window is still
* filling (trigFill). In the second case the hub sends nothing until the
* whole window has been produced — several seconds for a long window at a
* high rate — so the trace is drawn from this client's own rings on the
* final axis, growing left to right. Filling wins over the previous
* capture: once a new trigger fires, the stale waveform is history. */
const CaptureFrame* cap = app.capture(); const CaptureFrame* cap = app.capture();
const bool trigView = (cap != nullptr) && app.showTrigBar(); const TriggerState& trg = app.trigger();
/* Prefer the window the hub latched at fire time; the local config is only
* a fallback for hubs that do not report it, and may have been edited
* since the trigger fired. */
const double fillPreS = trg.hasFiredWin ? trg.firedPreS
: trg.windowSec * trg.prePercent * 0.01;
const double fillPostS = trg.hasFiredWin ? trg.firedPostS
: trg.windowSec - fillPreS;
const bool trigFill = app.showTrigBar() && !paused &&
trg.status == "collecting" && trg.hasTrigTime;
const bool trigView = (cap != nullptr) && app.showTrigBar() && !trigFill;
const bool trigRel = trigView || trigFill;
/* Window edges of whatever is on screen. A capture latches its own
* pre/post at fire time, so later edits in the trigger bar must not move
* the axis of a finished capture. */
const double trigT = trigView ? cap->trigTime : trg.trigTime;
const double trigPreS = trigView ? cap->preSec : fillPreS;
const double trigPostS = trigView ? cap->postSec : fillPostS;
/* Hi-res zoom cache for this plot */ /* Hi-res zoom cache for this plot */
auto& zc = app.zoomCache(plotIdx); auto& zc = app.zoomCache(plotIdx);
@@ -194,13 +264,13 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
* from decimated pushes) undersamples the visible range. Periodically * from decimated pushes) undersamples the visible range. Periodically
* fetch a fresh ~2400-pt slice from the hub raw ring and anchor the X * fetch a fresh ~2400-pt slice from the hub raw ring and anchor the X
* axis to the fetched slice (scope-style refresh at the fetch rate). */ * axis to the fetched slice (scope-style refresh at the fetch rate). */
const bool liveHiRes = !trigView && live && !paused && const bool liveHiRes = !trigRel && live && !paused &&
app.windowSec() <= kLiveHiResMaxWin && app.windowSec() <= kLiveHiResMaxWin &&
zc.valid && zc.valid &&
(zc.t1 - zc.t0) >= app.windowSec() * 0.9 && (zc.t1 - zc.t0) >= app.windowSec() * 0.9 &&
(wallNow - zc.t1) < 3.0; (wallNow - zc.t1) < 3.0;
const bool useZoomData = !trigView && !paused && zc.valid && const bool useZoomData = !trigRel && !paused && zc.valid &&
(liveHiRes || (liveHiRes ||
(!live && (!live &&
zc.t0 <= app.plotXMin(plotIdx) + 1e-9 && zc.t0 <= app.plotXMin(plotIdx) + 1e-9 &&
@@ -215,7 +285,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
const bool haveHistCover = hc.valid && const bool haveHistCover = hc.valid &&
hc.t0 <= app.plotXMin(plotIdx) + 1e-9 && hc.t0 <= app.plotXMin(plotIdx) + 1e-9 &&
hc.t1 >= app.plotXMax(plotIdx) - 1e-9; hc.t1 >= app.plotXMax(plotIdx) - 1e-9;
bool useHistData = !trigView && !paused && !live && haveHistCover; bool useHistData = !trigRel && !paused && !live && haveHistCover;
if (useHistData) { if (useHistData) {
/* Check that at least one signal has actual data points */ /* Check that at least one signal has actual data points */
bool anyData = false; bool anyData = false;
@@ -231,7 +301,12 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
* copied tens of MB per signal per frame. A 10% margin keeps a sample on * copied tens of MB per signal per frame. A 10% margin keeps a sample on
* each side so the later fine clip still has its boundary points. */ * each side so the later fine clip still has its boundary points. */
double visT0, visT1; double visT0, visT1;
if (live) { visT1 = wallNow; visT0 = wallNow - app.windowSec(); } if (trigFill) {
/* Absolute bounds of the trigger window: the ring is indexed on the
* hub clock, the axis on trigger-relative time. */
visT0 = trigT - trigPreS; visT1 = trigT + trigPostS;
}
else if (live) { visT1 = wallNow; visT0 = wallNow - app.windowSec(); }
else { visT1 = app.plotXMax(plotIdx); visT0 = app.plotXMin(plotIdx); } else { visT1 = app.plotXMax(plotIdx); visT0 = app.plotXMin(plotIdx); }
{ {
double margin = (visT1 - visT0) * 0.1; double margin = (visT1 - visT0) * 0.1;
@@ -272,6 +347,15 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} }
break; break;
} }
} else if (trigFill) {
/* Live ring, clipped to the (absolute) window and shifted onto the
* trigger-relative axis. visT0/visT1 already carry a margin, so
* clip here rather than reusing readBase. */
(void) sig.buf.readRange(trigT - trigPreS, trigT + trigPostS,
tStore[si], vStore[si]);
for (size_t i = 0; i < tStore[si].size(); i++) {
tStore[si][i] -= trigT;
}
} else if (useZoomData) { } else if (useZoomData) {
bool found = false; bool found = false;
for (const auto& zs : zc.signals) { for (const auto& zs : zc.signals) {
@@ -302,6 +386,11 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
resolveVScale(a, sig, vStore[si]); resolveVScale(a, sig, vStore[si]);
} }
VScale& uniVS = app.plotUnifiedVS(plotIdx);
if (vMode == 3) {
resolveUnifiedVScale(uniVS, slots, sources, vStore);
}
/* clamp active slot */ /* clamp active slot */
if (actSlot >= (int)slots.size()) actSlot = -1; if (actSlot >= (int)slots.size()) actSlot = -1;
@@ -326,9 +415,11 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
ImVec4(0.067f,0.067f,0.106f,1.f)); ImVec4(0.067f,0.067f,0.106f,1.f));
char badge[80]; char badge[80];
/* show vscale info: resolved div value */ /* Show the div value actually in force: the plot's shared one in
* unified mode, this signal's otherwise. */
char dvbuf[16]; char dvbuf[16];
fmtVal(dvbuf, sizeof(dvbuf), a.vs.resolvedDiv); fmtVal(dvbuf, sizeof(dvbuf),
(vMode == 3) ? uniVS.resolvedDiv : a.vs.resolvedDiv);
snprintf(badge, sizeof(badge), "%s %s/div##b%d", snprintf(badge, sizeof(badge), "%s %s/div##b%d",
sig.meta.name.c_str(), dvbuf, i); sig.meta.name.c_str(), dvbuf, i);
@@ -398,7 +489,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} }
/* Back / Fit / Reset (zoom history) */ /* Back / Fit / Reset (zoom history) */
if (!live || (trigView && app.trigZoomed(plotIdx))) { if (!live || (trigRel && app.trigZoomed(plotIdx))) {
ImGui::SameLine(); ImGui::SameLine();
auto& hist = app.zoomHist(plotIdx); auto& hist = app.zoomHist(plotIdx);
if (hist.empty()) { ImGui::BeginDisabled(); } if (hist.empty()) { ImGui::BeginDisabled(); }
@@ -408,7 +499,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} }
if (hist.empty()) { ImGui::EndDisabled(); } if (hist.empty()) { ImGui::EndDisabled(); }
ImGui::SameLine(); ImGui::SameLine();
if (trigView) { if (trigRel) {
/* Reset to full capture window */ /* Reset to full capture window */
if (ImGui::SmallButton(ICON_FA_EXPAND " Reset##zr")) { if (ImGui::SmallButton(ICON_FA_EXPAND " Reset##zr")) {
app.trigZoomed(plotIdx) = false; app.trigZoomed(plotIdx) = false;
@@ -440,10 +531,15 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* Norm/Dig/Mix mode — compact toggle buttons matching SmallButton height */ /* Norm/Dig/Mix mode — compact toggle buttons matching SmallButton height */
ImGui::SameLine(); ImGui::SameLine();
{ {
static const char* kVLabels[] = {"N", "D", "M"}; static const char* kVLabels[] = {"N", "U", "D", "M"};
static const char* kVTooltips[] = {"Normal", "Digital", "Mixed"}; static const char* kVTooltips[] = {
for (int vm = 0; vm < 3; vm++) { "Normal: one vertical scale per signal",
char vmId[16]; snprintf(vmId, sizeof(vmId), "%s##vm%d_%d", kVLabels[vm], plotIdx, vm); "Unified: one vertical scale shared by every signal",
"Digital", "Mixed" };
static const int kVModes[] = {0, 3, 1, 2};
for (int i = 0; i < 4; i++) {
const int vm = kVModes[i];
char vmId[16]; snprintf(vmId, sizeof(vmId), "%s##vm%d_%d", kVLabels[i], plotIdx, vm);
bool sel = (vMode == vm); bool sel = (vMode == vm);
if (sel) { if (sel) {
ImGui::PushStyleColor(ImGuiCol_Button, ImVec4(0.537f,0.706f,0.980f,0.4f)); ImGui::PushStyleColor(ImGuiCol_Button, ImVec4(0.537f,0.706f,0.980f,0.4f));
@@ -451,28 +547,41 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} }
if (ImGui::SmallButton(vmId)) { vMode = vm; } if (ImGui::SmallButton(vmId)) { vMode = vm; }
if (sel) { ImGui::PopStyleColor(2); } if (sel) { ImGui::PopStyleColor(2); }
if (ImGui::IsItemHovered()) { ImGui::SetTooltip("%s", kVTooltips[vm]); } if (ImGui::IsItemHovered()) { ImGui::SetTooltip("%s", kVTooltips[i]); }
if (vm < 2) { ImGui::SameLine(0.f, 1.f); } if (i < 3) { ImGui::SameLine(0.f, 1.f); }
} }
} }
/* ── VScale toolbar (shown when an active signal is selected) ───────── */ /* ── VScale toolbar ──────────────────────────────────────────────────── *
* Normal mode edits the active signal's scale; unified mode edits the one
* scale the whole plot shares, so it needs no selection. */
VScale *toolVS = static_cast<VScale *>(0);
if (vMode == 0 && actSlot >= 0 && actSlot < (int)slots.size()) { if (vMode == 0 && actSlot >= 0 && actSlot < (int)slots.size()) {
auto& a = slots[actSlot]; toolVS = &slots[actSlot].vs;
} else if (vMode == 3) {
toolVS = &uniVS;
}
if (toolVS != static_cast<VScale *>(0)) {
VScale& tvs = *toolVS;
ImGui::PushStyleVar(ImGuiStyleVar_ItemSpacing, ImVec2(4.f,2.f)); ImGui::PushStyleVar(ImGuiStyleVar_ItemSpacing, ImVec2(4.f,2.f));
if (vMode == 3) {
ImGui::TextDisabled("all signals");
ImGui::SameLine(0.f,10.f);
}
/* mode buttons */ /* mode buttons */
static const char* kModeLabels[] = {"Auto","Range","Manual"}; static const char* kModeLabels[] = {"Auto","Range","Manual"};
for (int m = 0; m < 3; m++) { for (int m = 0; m < 3; m++) {
bool sel = (a.vs.mode == m); bool sel = (tvs.mode == m);
if (sel) { if (sel) {
ImGui::PushStyleColor(ImGuiCol_Button, ImGui::PushStyleColor(ImGuiCol_Button,
ImVec4(0.537f,0.706f,0.980f,0.3f)); ImVec4(0.537f,0.706f,0.980f,0.3f));
ImGui::PushStyleColor(ImGuiCol_Text, ImGui::PushStyleColor(ImGuiCol_Text,
ImVec4(0.537f,0.706f,0.980f,1.f)); ImVec4(0.537f,0.706f,0.980f,1.f));
} }
if (ImGui::SmallButton(kModeLabels[m])) { a.vs.mode = m; } if (ImGui::SmallButton(kModeLabels[m])) { tvs.mode = m; }
if (sel) ImGui::PopStyleColor(2); if (sel) ImGui::PopStyleColor(2);
if (m < 2) ImGui::SameLine(0.f,2.f); if (m < 2) ImGui::SameLine(0.f,2.f);
} }
@@ -480,23 +589,23 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* resolved info */ /* resolved info */
char rbuf[24], obuf[24]; char rbuf[24], obuf[24];
fmtVal(rbuf, sizeof(rbuf), a.vs.resolvedDiv); fmtVal(rbuf, sizeof(rbuf), tvs.resolvedDiv);
fmtVal(obuf, sizeof(obuf), a.vs.resolvedOffset); fmtVal(obuf, sizeof(obuf), tvs.resolvedOffset);
if (a.vs.mode == 2) { /* manual: editable */ if (tvs.mode == 2) { /* manual: editable */
ImGui::SetNextItemWidth(70.f); ImGui::SetNextItemWidth(70.f);
ImGui::InputDouble("V/div##vd", &a.vs.divValue, 0,0,"%.4g"); ImGui::InputDouble("V/div##vd", &tvs.divValue, 0,0,"%.4g");
ImGui::SameLine(0.f,4.f); ImGui::SameLine(0.f,4.f);
ImGui::SetNextItemWidth(80.f); ImGui::SetNextItemWidth(80.f);
ImGui::InputDouble("Offset##vo", &a.vs.offset, 0,0,"%.4g"); ImGui::InputDouble("Offset##vo", &tvs.offset, 0,0,"%.4g");
} else { } else {
ImGui::TextDisabled("%s/div @%s", rbuf, obuf); ImGui::TextDisabled("%s/div @%s", rbuf, obuf);
} }
ImGui::SameLine(0.f,10.f); ImGui::SameLine(0.f,10.f);
ImGui::SetNextItemWidth(50.f); ImGui::SetNextItemWidth(50.f);
float sp = (float)a.vs.screenPos; float sp = (float)tvs.screenPos;
if (ImGui::InputFloat("Pos(div)##vp", &sp, 0,0,"%.1f")) { if (ImGui::InputFloat("Pos(div)##vp", &sp, 0,0,"%.1f")) {
a.vs.screenPos = sp; tvs.screenPos = sp;
} }
ImGui::PopStyleVar(); ImGui::PopStyleVar();
@@ -539,7 +648,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
if (ImPlot::BeginPlot(plotId, ImVec2(-1.f,-1.f), plotFlags)) { if (ImPlot::BeginPlot(plotId, ImVec2(-1.f,-1.f), plotFlags)) {
/* Both axes locked so ImPlot never overrides our explicit limits. */ /* Both axes locked so ImPlot never overrides our explicit limits. */
ImPlot::SetupAxes(trigView ? "t - trig (s)" : "Time (s)", nullptr, ImPlot::SetupAxes(trigRel ? "t - trig (s)" : "Time (s)", nullptr,
ImPlotAxisFlags_Lock, ImPlotAxisFlags_Lock,
ImPlotAxisFlags_Lock); ImPlotAxisFlags_Lock);
@@ -549,13 +658,17 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* X axis: trig view → capture window (zoomable); live → wall clock; else stored */ /* X axis: trig view → capture window (zoomable); live → wall clock; else stored */
double xMin, xMax; double xMin, xMax;
bool& trigZm = app.trigZoomed(plotIdx); bool& trigZm = app.trigZoomed(plotIdx);
if (trigView) { if (trigRel) {
if (trigZm) { if (trigZm) {
xMin = app.plotXMin(plotIdx); xMin = app.plotXMin(plotIdx);
xMax = app.plotXMax(plotIdx); xMax = app.plotXMax(plotIdx);
} else { } else {
xMin = -cap->preSec; /* Full window from the start, even while filling: a trace that
xMax = cap->postSec; * grows into a fixed axis reads as progress; an axis that
* grows with the data makes the whole trace shift every
* frame and the time base meaningless. */
xMin = -trigPreS;
xMax = trigPostS;
} }
} else if (live && !paused) { } else if (live && !paused) {
if (liveHiRes) { if (liveHiRes) {
@@ -568,7 +681,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} else { } else {
xMin = app.plotXMin(plotIdx); xMax = app.plotXMax(plotIdx); xMin = app.plotXMin(plotIdx); xMax = app.plotXMax(plotIdx);
} }
if (trigView || (live && !paused) || !live) { if (trigRel || (live && !paused) || !live) {
if (xMax > xMin) { if (xMax > xMin) {
ImPlot::SetupAxisLimits(ImAxis_X1, xMin, xMax, ImGuiCond_Always); ImPlot::SetupAxisLimits(ImAxis_X1, xMin, xMax, ImGuiCond_Always);
} }
@@ -579,8 +692,16 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
static char yTickBufs[9][20]; static char yTickBufs[9][20];
static const char* yTickLabels[9]; static const char* yTickLabels[9];
const VScale *axisVS = static_cast<const VScale *>(0);
if (vMode == 0 && actSlot >= 0 && actSlot < (int)slots.size()) { if (vMode == 0 && actSlot >= 0 && actSlot < (int)slots.size()) {
const auto& av = slots[actSlot].vs; axisVS = &slots[actSlot].vs;
} else if (vMode == 3) {
/* Unified: the shared scale labels the axis for every trace at
* once, so no signal has to be selected first. */
axisVS = &uniVS;
}
if (axisVS != static_cast<const VScale *>(0)) {
const VScale& av = *axisVS;
for (int d = 0; d < 9; d++) { for (int d = 0; d < 9; d++) {
double divPos = yTickVals[d]; double divPos = yTickVals[d];
double rawVal = av.resolvedOffset + (divPos - av.screenPos) * av.resolvedDiv; double rawVal = av.resolvedOffset + (divPos - av.screenPos) * av.resolvedDiv;
@@ -636,15 +757,15 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* Helper: enter zoomed mode for trigger view (seed from capture window) */ /* Helper: enter zoomed mode for trigger view (seed from capture window) */
auto enterTrigZoom = [&]() { auto enterTrigZoom = [&]() {
if (trigView && !trigZm) { if (trigRel && !trigZm) {
app.setPlotX(plotIdx, -cap->preSec, cap->postSec); app.setPlotX(plotIdx, -trigPreS, trigPostS);
trigZm = true; trigZm = true;
} }
}; };
/* Helper: X-zoom the stored range by factor around center */ /* Helper: X-zoom the stored range by factor around center */
auto xZoomStored = [&](double factor) { auto xZoomStored = [&](double factor) {
if (trigView) { enterTrigZoom(); } if (trigRel) { enterTrigZoom(); }
if (now - lastHistPush[plotIdx] > 0.6) { if (now - lastHistPush[plotIdx] > 0.6) {
app.pushZoomHist(plotIdx); app.pushZoomHist(plotIdx);
lastHistPush[plotIdx] = now; lastHistPush[plotIdx] = now;
@@ -659,37 +780,45 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
const double zoomOut = 1.25; const double zoomOut = 1.25;
double factor = (wheel > 0.f) ? zoomIn : zoomOut; double factor = (wheel > 0.f) ? zoomIn : zoomOut;
/* Scroll adjusts the scale the axis is labelled with: the
* active signal's in normal mode, the plot's shared one in
* unified mode (where there is nothing to select). */
VScale *wheelVS = static_cast<VScale *>(0);
if (vMode == 3) {
wheelVS = &uniVS;
} else if (actSlot >= 0 && actSlot < (int)slots.size()) {
wheelVS = &slots[actSlot].vs;
}
/* Seed manual from the resolved values so the gesture sticks. */
auto latchManual = [](VScale& v) {
if (v.mode != 2) {
v.divValue = std::max(v.resolvedDiv, 1e-30);
v.offset = v.resolvedOffset;
v.mode = 2;
}
};
if (ctrl) { if (ctrl) {
/* ── X zoom ─────────────────────────────────────────── */ /* ── X zoom ─────────────────────────────────────────── */
if (!trigView && live) { if (!trigRel && live) {
app.setWindowSec(app.windowSec() * factor); app.setWindowSec(app.windowSec() * factor);
} else { } else {
xZoomStored(factor); xZoomStored(factor);
} }
} else if (shift) { } else if (shift) {
/* ── Y offset of active signal ───────────────────────── */ /* ── Y pan ───────────────────────────────────────────── */
if (actSlot >= 0 && actSlot < (int)slots.size()) { if (wheelVS != static_cast<VScale *>(0)) {
auto& a = slots[actSlot]; latchManual(*wheelVS);
if (a.vs.mode != 2) { wheelVS->screenPos += (wheel > 0.f) ? 0.5 : -0.5;
a.vs.divValue = std::max(a.vs.resolvedDiv, 1e-30);
a.vs.offset = a.vs.resolvedOffset;
a.vs.mode = 2;
}
a.vs.screenPos += (wheel > 0.f) ? 0.5 : -0.5;
} }
} else { } else {
/* ── Y zoom of active signal ─────────────────────────── */ /* ── Y zoom ──────────────────────────────────────────── */
if (actSlot >= 0 && actSlot < (int)slots.size()) { if (wheelVS != static_cast<VScale *>(0)) {
auto& a = slots[actSlot]; latchManual(*wheelVS);
if (a.vs.mode != 2) { wheelVS->divValue = std::max(wheelVS->divValue * factor, 1e-30);
a.vs.divValue = std::max(a.vs.resolvedDiv, 1e-30);
a.vs.offset = a.vs.resolvedOffset;
a.vs.mode = 2;
}
a.vs.divValue = std::max(a.vs.divValue * factor, 1e-30);
} else { } else {
/* No active signal: plain scroll → X zoom */ /* No active signal: plain scroll → X zoom */
if (!trigView && live) { if (!trigRel && live) {
app.setWindowSec(app.windowSec() * factor); app.setWindowSec(app.windowSec() * factor);
} else { } else {
xZoomStored(factor); xZoomStored(factor);
@@ -701,8 +830,8 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
/* Right-drag → X pan. Transition live→non-live on drag start; /* Right-drag → X pan. Transition live→non-live on drag start;
* in trigger view, enter trigger-zoom mode. */ * in trigger view, enter trigger-zoom mode. */
if (ImGui::IsMouseDragging(ImGuiMouseButton_Right)) { if (ImGui::IsMouseDragging(ImGuiMouseButton_Right)) {
if (trigView) { enterTrigZoom(); } if (trigRel) { enterTrigZoom(); }
if (!trigView && live) { if (!trigRel && live) {
app.initPlotX(plotIdx, wallNow); app.initPlotX(plotIdx, wallNow);
live = false; live = false;
lastHistPush[plotIdx] = now; lastHistPush[plotIdx] = now;
@@ -721,7 +850,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} }
/* ── Hi-res WS zoom requests (suppressed while paused) ──────────── */ /* ── Hi-res WS zoom requests (suppressed while paused) ──────────── */
if (!trigView && !paused) { if (!trigRel && !paused) {
std::string csv; std::string csv;
for (const auto& a : slots) { for (const auto& a : slots) {
std::string k = app.slotKey(a); std::string k = app.slotKey(a);
@@ -821,9 +950,11 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} else if (vMode == 2) { /* mixed */ } else if (vMode == 2) { /* mixed */
bandNormalize(vDec, vNorm, myKi, nTraces, a.vs.digitalInMixed); bandNormalize(vDec, vNorm, myKi, nTraces, a.vs.digitalInMixed);
} else { } else {
/* unified shares one scale, normal gives each trace its own */
const VScale& nvs = (vMode == 3) ? uniVS : a.vs;
vNorm.resize(nOut); vNorm.resize(nOut);
for (size_t k = 0; k < nOut; k++) { for (size_t k = 0; k < nOut; k++) {
vNorm[k] = normalizeY(vDec[k], a.vs); vNorm[k] = normalizeY(vDec[k], nvs);
} }
} }
@@ -836,7 +967,7 @@ void RenderPlotPanel(App& app, int plotIdx, bool& paused) {
} }
/* Trigger instant marker (capture view: t = 0) */ /* Trigger instant marker (capture view: t = 0) */
if (trigView) { if (trigRel) {
double t0m = 0.0; double t0m = 0.0;
ImPlot::DragLineX(900, &t0m, ImVec4(1.f,1.f,0.f,0.8f), ImPlot::DragLineX(900, &t0m, ImVec4(1.f,1.f,0.f,0.8f),
1.5f, ImPlotDragToolFlags_NoInputs); 1.5f, ImPlotDragToolFlags_NoInputs);
Client/streamhub/Protocol.cpp
+6
View File
@@ -458,6 +458,12 @@ bool ParseTriggerState(const std::string& json, TriggerStateMsg& out) {
double tt = 0.0; double tt = 0.0;
out.hasTrigTime = jsonGetDouble(json.c_str(), "trigTime", tt); out.hasTrigTime = jsonGetDouble(json.c_str(), "trigTime", tt);
out.trigTime = tt; out.trigTime = tt;
double pre = 0.0, post = 0.0;
out.hasWindow = jsonGetDouble(json.c_str(), "preSec", pre) &&
jsonGetDouble(json.c_str(), "postSec", post);
out.preSec = pre;
out.postSec = post;
return true; return true;
} }
Client/streamhub/Protocol.h
+5
View File
@@ -109,6 +109,11 @@ struct TriggerStateMsg {
bool stopped = false; bool stopped = false;
bool hasTrigTime = false; bool hasTrigTime = false;
double trigTime = 0.0; double trigTime = 0.0;
/* Window latched at fire time, sent alongside trigTime. Older hubs omit
* it, hence hasWindow — fall back to the local trigger config then. */
bool hasWindow = false;
double preSec = 0.0;
double postSec = 0.0;
}; };
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
Client/udpstreamer/BUFFER_AND_TRIGGER.md
+425
View File
@@ -0,0 +1,425 @@
# Buffer time-window & trigger logic in `Client/udpstreamer`
How the UDP Scope client acquires, buffers, times and triggers waveforms.
The pipeline has two halves that must be read together:
- the **Go hub** (`Common/Client/go/wshub/`) — owns the UDP sockets, the
full-resolution sample storage, the disk history, and the trigger FSM;
- the **browser SPA** (`static/app.js`) — owns the display buffers, the rolling
window, and the trigger capture rendering.
The same SPA is also served by `Client/webui` and talks to the C++ StreamHub,
which mirrors the Go hub's behaviour (same trigger FSM states, same binary
frames). Everything below describes the Go-hub path; the wire contracts are
identical on both.
---
## 1. End-to-end data flow
```
MARTe2 RT app ──UDP/UDPS──▶ sources.go: runSession()
│ CONFIG + DATA packets, 17-byte header, HRT timestamp per frame
▼
udpsprotocol.ParseData() → []DataSample{HRTTimestamp, WallTime, Values}
▼
Hub.Run() dataCh → pending[sourceID] (drained every 30 Hz tick)
▼
buildBinaryDataMessageForSource()
├─ rebuild per-sample timestamps from TimeMode / calibration / monotonic snap
├─ h.ingest(key, n, t, v) ← FULL rate: ring.write + hist.write + trigger.feed
└─ minMaxDecimate(…, maxPushPoints=50) → WS binary v1 frame to clients
▼
browser: onBinaryData() → pushBuffer() into per-signal circular buffers
▼
renderDirtyPlots() (rAF loop) → buildUPlotData() → uPlot
```
The 30 Hz push is the **only** live path to the browser and it is decimated to
≤50 points/signal/tick. Everything that needs full resolution — zoom, trigger
captures, disk history — is fed independently through `ingest()` and never goes
over the wire until asked.
---
## 2. Hub-side buffers: `sigRing` (`wshub/ringbuf.go`)
One ring per `"sourceId:signalName"` key. A fixed-capacity circular buffer of
Float64 `(t, v)` pairs with a `sync.RWMutex` (writes from `Hub.Run()`, reads
from HTTP/WS handler goroutines).
### 2.1 Min/max bucketing
`bucket` is how many source samples collapse into **one min/max pair** on the
way in:
- `bucket == 1` — the stream is stored verbatim;
- `bucket > 1` — each group contributes its minimum and its maximum, emitted in
time order (`flushBucketLocked`), so the stored timestamps stay
non-decreasing (reads binary-search `rb.t`).
Bucketing is what lets an arbitrarily long window fit a fixed per-signal memory
budget at a megasample rate. Samples already stored keep the resolution they
were written at; the ring converges on a new bucket as it rolls
(`setBucket`).
### 2.2 Source-rate measurement
`sigRing` keeps its own source-sample accounting (`srcCount`, `srcT0`, `srcT1`,
reset every `srcRateWindowSec = 10 s`), because once `bucket > 1` neither `size`
nor the stored timespan measures the real incoming rate. `sourceRate()` is used
by the tuning sweep and by the history writer.
### 2.3 Ring tuning (`retuneRings`, every 1 s)
`activeWindowSec()` decides how far back the rings must reach:
1. an **armed trigger** owns the window: `cfg.windowSec + captureLagSec`
(`captureLagSec = captureMarginSec + 1/30 ≈ 0.183 s` — the capture is read
out a post-window + margin + one push tick after the trigger, so the rings
must hold that much extra or the front of the capture has already rolled);
2. otherwise the **widest window any connected client is displaying**
(`wsClient.displayWindowSec`, set by the SPA's `setWindow` command), with a
`defaultLiveWindowSec = 10 s` fallback while nobody has said;
Then per ring, with `budget = ringBudget()` (default `defaultRingPts = 10 M`,
floor `ringCapInitial = 250 k`):
- grow to the budget first (`grow()` preserves all samples, never shrinks);
- compute the needed bucket with `ringBucketFor(rate, window, capacity)`
(`ceil(2·rate·window·ringHeadroom / capacity)`, `ringHeadroom = 1.25`);
- apply it with **hysteresis**: keep the current bucket while its coverage is
between `need` and `2·need`, so a rate jittering across the boundary does not
flip the resolution every second.
The history archive is re-sized from the same window (`hist.setWindow`) so a
zoom or capture that outlives the rings can fall back to it.
### 2.4 Reading: `slice(t0, t1)`
Binary search for `t0` then `t1` over the circular layout, returning copies of
the pairs in `[t0, t1]`. Safe to use without holding the lock.
---
## 3. Disk history (`wshub/history.go`)
Optional (`EnableHistory`, `CloseHistory`), enabled by the hub configuration.
Every sample goes to disk through `ingest → hist.write` at full rate, in files
sized for the *current* window (not a retention period). It exists to back
three things the rings cannot:
- **zoom past the window**: `readRange(key, t0, t1, maxOut)`;
- **captures the rings have rolled past**: `captureRange(trigTime−pre, trigTime+post)`
lifts each capture into a file of its own so nothing overwrites it before the
next trigger;
- **short captures**: `backfillCaptureHead` prepends the front of the window the
ring no longer holds (the ring only *becomes* as long as the window after a
re-tune; the archive was written straight through).
---
## 4. Live push to the browser
`Hub.Run()` drains `pending[sourceID]` on a 30 Hz ticker. Even with no client
connected the frame is built: that is what keeps feeding rings, history and the
trigger, and keeps push cursors advancing so a late client does not get a
backlog burst.
`buildBinaryDataMessageForSource` reconstructs per-sample timestamps per signal
`TimeMode`:
| Mode | Timestamp reconstruction |
|---|---|
| `FirstSample` / `LastSample` | scalar TimeSignal value × `timerToSec` (µs→s or ns→s for u64), calibrated once against `WallTime`; samples spaced by `1/SamplingRate` |
| `FullArray` | per-element TimeSignal array, calibrated once against `WallTime` |
| scalar (`n == 1`) | `WallTime` of the UDP arrival |
| `PacketTime` (default, n>1) | inter-packet wall-clock gaps divided by n (single-packet ticks use the gap from the previous tick) |
**Monotonic snapping** (optional, `setMonotonic` command / "Sync TS" checkbox):
when enabled, the inter-frame anchor gap is smoothed with an EMA
(`monotonicEMAAlpha = 0.01`, initialised from the nominal `n·dt`) and small
deviations (< `monotonicTolerance = 5 ms`) are snapped to the smoothed gap,
removing the software-dispatch jitter overlaps/gaps described in the StreamHub
docs while tracking the true hardware rate (no accumulated drift).
The live frame is a **binary v1** WS message:
```
[u8 1][u8 srcIdLen][srcId][u32 nSigs]
{[u16 keyLen][key][u32 N][f64 t×N][f64 v×N]}
```
with each signal min/max-decimated to `maxPushPoints = 50` (`minMaxDecimate`:
the range is split into `threshold/2` buckets, each contributing its min and max
in time order — a scope-style envelope that keeps glitches on screen).
---
## 5. Browser-side buffers (`static/app.js`)
### 5.1 Capacity & growth
- `MAX_CAP = 2 000 000` — hard ceiling per buffer (~32 MB/signal at Float64 t+v);
- `DEFAULT_CAP = 100 000` — starting size for scalars;
- `TEMPORAL_CAP = 500 000` — starting size for array signals (the hub pushes
≤50 pts/signal/tick, so this already covers ~5 min);
- `growBufferForWindow(buf, windowSec)` — **sizes from the buffer's own span**,
not the signal's sampling rate: the incoming rate here is the hub-decimated
~1.5 kpts/s regardless of the source rate, so rate-based sizing overshot by
three orders of magnitude. Grows only when the buffer is full, to
`windowSec × 1.5` headroom, capped at `MAX_CAP`.
- `growBuffer` copies all existing samples into a larger array (preserving
circular order).
### 5.2 The window
`windowSec` (default 5 s, options 1 s … 10 min) is the rolling viewport.
Changing it:
1. updates `windowSec`;
2. `sendWindow()` → WS `setWindow` → hub `displayWindowSec` → ring re-tune;
3. grows every local buffer via `growBufferForWindow`;
4. evicts the decimation cache (a different window invalidates all cached
renderings).
The rolling "now" anchor is **data-driven, not wall-clock**:
`computePlotNow(p)` takes the newest timestamp of each contributing source and
uses the min-of-max over sources that are still active (a source lagging the
fastest by more than `windowSec` is treated as stale and excluded). This keeps
the window tracking real data regardless of clock skew between hub and browser.
### 5.3 Slicing & rendering
- `getBufferSliceRange(buf, t0, t1)` — binary search on the circular layout,
O(log n + window size);
- `getBufferSliceRangeWithBrackets` — same plus one point on each side so lines
still cross a nearly-empty zoom window;
- `supplementWithBrackets` — same bracketing for sparse server-fetched zoom data.
`buildLiveData(p)`:
1. slices every trace in `[t0, t1]`;
2. picks the **master** signal: highest `SamplingRate`, then most points;
3. decimates the master to ~2× plot width (`DECIM_MIN = 200` floor) via a
background worker (`decimateAsync`, stale-while-revalidate cache keyed per
plot/range/data-generation);
4. resamples every other trace onto the master grid with `resampleLinear`;
5. normalises Y (`applyVScaleNorm`: calibration `v·scale+offset`, then
`(y − offset)/div`).
### 5.4 Zoom
A zoom pins `p.xRange` and asks the hub for hi-res data over the exact range
(WS `zoom` request or HTTP `/api/zoom`). The hub answers from the full-res
rings — or from the **held copy of the last trigger capture** (`captureHold`
double buffer) while that window is still relevant — decimated to the requested
point budget. The browser prefers the fetched data when it exists, falls back
to its own circular buffers otherwise, and always brackets with local points.
---
## 6. Hub-side trigger FSM (`wshub/trigger.go`)
### 6.1 States and configuration
```
idle ──arm──▶ armed ──edge──▶ collecting ──window elapsed──▶ triggered
▲ ▲ (pre/post latched) │
│ └───────────── rearm (normal mode, after holdoff) ◀─────────┘
└────────────── disarm / single mode stays triggered
```
Configuration (`trigConfig`, client-settable via WS `setTrigger`):
| field | meaning | clamp |
|---|---|---|
| `signalKey` | `"src:sig"` or `"src:sig[i]"` | — |
| `edge` | `rising` / `falling` / `both` | — |
| `threshold` | **raw** units (SPA converts calibrated → raw) | — |
| `windowSec` | capture window | `[1e-4, 600]` |
| `prePercent` | pre-trigger share | `[0, 100]` |
| `mode` | `normal` (auto-rearm) / `single` | — |
| `holdoffSec` | re-arm delay after a capture, double-trigger guard | `[0, 60]` |
### 6.2 Edge detection (`feed`)
Called from `ingest` with every full-resolution batch for the trigger signal.
Level tracking (`prevValue`/`prevValid`) compares consecutive samples against
the threshold; `[i]`-suffixed keys stride the flattened batch by `nElem` to
watch one column. On a qualifying edge in `armed` state: `latchWindowLocked`
freezes `trigTime` and the pre/post split (so later config edits cannot move a
capture's axis).
### 6.3 Buffer-fill gate
Before accepting an edge, the FSM checks that the trigger signal's ring reaches
back far enough that the capture will come back whole (`fillLocked`):
```
need = windowSec − growth × postSec, floored at the pre-window
```
`growth` is the measured span-growth rate of the ring (`setBuffered`, refreshed
by `refreshTriggerFill` from the tick and from trigger commands). A still-filling
ring grows 1 s of span per second, so the gate reduces to the pre-window; a
full ring at a long window needs the whole window. While holding off, the level
is still tracked so the first edge after the gate opens is measured against the
right predecessor. The SPA shows the hold-off as an armed trigger with a
`bufferFill %` badge.
### 6.4 Window timing and the pending edge
`dueCapture` waits for the window on the **sample clock**, not the wall clock:
`lastT ≥ trigTime + post + captureMarginSec(0.15)`. This avoids cutting a
capture short when the stream's timestamps lag real time. Three ways it fires:
1. the samples themselves covered the window;
2. wall-clock fallback when no sample was ever seen (Force from idle);
3. `captureStallSec = 2 s` of stream silence — deliver what was collected
rather than leaving the client stuck in "collecting".
While a capture is in flight the comparator keeps running. The **first**
qualifying edge at/after `notBefore = trigTime + max(post, holdoffSec)` is
remembered (`pendingT`/`pendingValid`) and fired immediately on the automatic
`rearm()`. Without this the trigger was deaf through the whole post-window +
holdoff, which rounded sparse pulse trains up to whole periods (a 1 Hz train at
a 1 s window was caught at 0.5 Hz).
### 6.5 Holdoff and rearm
`markTriggered` moves `collecting → triggered` and, in `normal` mode (not
stopped), schedules `rearmAt = now + cfg.holdoffSec`. `dueRearm` consumes it;
`rearm()` re-arms immediately on a pending edge or returns to `armed`. The
holdoff is measured from the trigger point, overlapping the post-window rather
than adding to it.
`Force()` fires immediately at the most recent sample time (wall clock if no
sample yet) — the "Force" button.
### 6.6 Capture assembly (`buildTriggerCapture`)
On a due capture the hub builds the **binary v2** frame:
```
[u8 2][f64 trigTime][f64 preSec][f64 postSec][u32 nSig]
{[u16 keyLen][fullKey][u32 N][f64 t×N][f64 v×N]}
```
For every ring:
1. `slice(trigTime−pre, trigTime+post)`;
2. `backfillCaptureHead` from disk history for the front the ring lost;
3. if it is still short by more than `shortCaptureTol = 1 %` of the window,
log it explicitly (nothing can recover data the ring never held);
4. keep the **full-resolution** slice in the `captureHold` double buffer
(so a zoom into the capture can be answered after the rings roll past);
5. min/max-decimate to `trigCapturePts = 20 000` per signal for the wire —
a 60 s window at 1 MSps is ~960 MB raw per signal and would be dropped by
the send path anyway.
The double buffer is published (`capture.publish`) only once the frame is known
good, so a shot that yielded nothing leaves the previous capture on screen.
Dropped frames (client send-queue full) are logged.
`triggerTick` (every push tick) drives the whole FSM: re-tune rings → open
pending history files → refresh the fill measurement → due capture (send + mark
triggered + `hist.captureRange`) or due rearm → broadcast state only when it
changed (`stateUnsent`).
### 6.7 WS commands
| message | effect |
|---|---|
| `setTrigger {signal, edge, threshold, windowSec, prePercent, mode, holdoffSec}` | replace config |
| `arm` / `rearm` | explicit arm (discards pending edge) |
| `disarm` | → idle |
| `trigStop {stopped}` | pause/resume auto-rearm |
| `forceTrigger` | fire now |
Every command also refreshes the buffer-fill measurement synchronously — at
1 MSps the ring crosses the fill threshold many times inside one 33 ms tick, so
waiting for the next tick would fire on a stale measurement.
---
## 7. Browser-side trigger (`static/app.js`)
### 7.1 State handling
`onTriggerState(msg)` tracks the FSM broadcast:
- **armed** — shows `bufferFill %` while the hub is holding off on the fill
gate, so a trigger that is not yet fireable does not look broken;
- **collecting** — clears the previous snapshot, latches `trigTime` (and
`preSec`/`postSec` if the hub sent them), and lets live data sweep into the
trigger axis (see 7.3);
- **triggered / idle** — bookkeeping for the Rearm/Stop buttons.
### 7.2 Capture handling
`onTriggerCapture` parses the v2 frame into `trig.snapshot[key] = {t, v}` plus
the latched `_preS`/`_postS`. It is **ignored when the client did not enable
the trigger** (`trig.enabled`), because the hub keeps an armed trigger across
client sessions and applying a foreign capture would clobber this client's zoom
and scales. On receipt: the horizontal zoom is dropped so the whole capture is
visible, but **vertical scales (V/div, offset) persist** — they are user
settings and must survive from shot to shot.
### 7.3 Rendering modes (`buildUPlotData`)
| state | renderer | source |
|---|---|---|
| collecting, no snapshot yet | `buildTrigFillData` | live buffers, drawn on the *final* trigger axis (relative seconds, `[-pre, +post]`) so the trace sweeps in from the left |
| armed, not fired | freeze last frame | — |
| snapshot present | `buildTrigData` | the capture (or a hi-res zoom reply that covers ≥98 % of the view, else the snapshot) |
| otherwise | `buildLiveData` | rolling window |
`buildTrigData` converts to trigger-relative time (`t − trigT`), picks the
master by rate/count, decimates (cached per range+source-tag), resamples the
other traces, and normalises Y.
### 7.4 Threshold in calibrated units
The trigger threshold is held in **calibrated units** (what the user sees on
the Y axis). `sendTrigConfig()` inverts it through the signal's calibration
before sending: `raw = (calibrated − offset)/scale`, so the hub's raw
comparator fires exactly when `GAIN·signal + OFFSET` crosses the threshold.
The threshold line (`drawTriggerMarker`) maps the same calibrated threshold
through the signal's vscale: `y_norm = (threshold − offset)/div`.
---
## 8. Key constants
| constant | value | file |
|---|---|---|
| push rate | 30 Hz | `hub.go` |
| `maxPushPoints` (live) | 50 pts/signal/tick | `hub.go` |
| `trigCapturePts` (capture) | 20 000 pts/signal | `trigger.go` |
| `captureMarginSec` | 0.15 s | `trigger.go` |
| `captureStallSec` | 2.0 s | `trigger.go` |
| `autoRearmDelaySec` (default holdoff) | 0.2 s | `trigger.go` |
| `maxTriggerWindowSec` | 600 s | `trigger.go` |
| `ringBudget` default | 10 000 000 pts/signal | `hub.go` |
| `ringCapInitial` | 250 000 | `hub.go` |
| `ringCapScalar` | 100 000 | `hub.go` |
| `ringHeadroom` | 1.25 | `ringbuf.go` |
| `defaultLiveWindowSec` | 10 s | `ringbuf.go` |
| `captureLagSec` | 0.15 + 1/30 ≈ 0.183 s | `ringbuf.go` |
| `monotonicTolerance` | 5 ms | `hub.go` |
| `monotonicEMAAlpha` | 0.01 | `hub.go` |
| `MAX_CAP` (browser) | 2 000 000 pts/signal | `app.js` |
| `DEFAULT_CAP` / `TEMPORAL_CAP` | 100 000 / 500 000 | `app.js` |
| `DECIM_MIN` | 200 | `app.js` |
---
## 9. The C++ StreamHub mirror
The Go hub and the C++ StreamHub implement the same WS contracts and must stay
in sync (`AGENTS.md`): same `triggerState` FSM strings, same v2 capture frame,
same command set (`setTrigger` including `holdoffSec`, `arm`, `disarm`,
`trigStop`, `forceTrigger`), same `trigCapturePts`/`kTrigCapturePts` cap, and
the same ring/history windowing intent (`Source/Applications/StreamHub/`). A
protocol change on one side must be mirrored on the other.
Client/udpstreamer/go.mod
+11 -1
View File
@@ -1,12 +1,22 @@
module udpstreamer-webui module udpstreamer-webui
go 1.21 go 1.24.9
require marte2/common v0.0.0 require marte2/common v0.0.0
require ( require (
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.1 // indirect github.com/gorilla/websocket v1.5.1 // indirect
github.com/klauspost/compress v1.17.9 // indirect
github.com/parquet-go/bitpack v1.0.0 // indirect
github.com/parquet-go/jsonlite v1.0.0 // indirect
github.com/parquet-go/parquet-go v0.32.0 // indirect
github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/twpayne/go-geom v1.6.1 // indirect
golang.org/x/net v0.17.0 // indirect golang.org/x/net v0.17.0 // indirect
golang.org/x/sys v0.38.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
) )
replace marte2/common => ../../Common/Client/go replace marte2/common => ../../Common/Client/go
Client/udpstreamer/go.sum
+34
View File
@@ -1,4 +1,38 @@
github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7OputlJIzU=
github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU=
github.com/alecthomas/assert/v2 v2.10.0 h1:jjRCHsj6hBJhkmhznrCzoNpbA3zqy0fYiUcYZP/GkPY=
github.com/alecthomas/assert/v2 v2.10.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc=
github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY= github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY=
github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY= github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA=
github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw=
github.com/parquet-go/bitpack v1.0.0 h1:AUqzlKzPPXf2bCdjfj4sTeacrUwsT7NlcYDMUQxPcQA=
github.com/parquet-go/bitpack v1.0.0/go.mod h1:XnVk9TH+O40eOOmvpAVZ7K2ocQFrQwysLMnc6M/8lgs=
github.com/parquet-go/jsonlite v1.0.0 h1:87QNdi56wOfsE5bdgas0vRzHPxfJgzrXGml1zZdd7VU=
github.com/parquet-go/jsonlite v1.0.0/go.mod h1:nDjpkpL4EOtqs6NQugUsi0Rleq9sW/OtC1NnZEnxzF0=
github.com/parquet-go/parquet-go v0.32.0 h1:NWDqTUHfrCS4cJP/Fj2HlxvqsrVedWG3sayMkf+znzM=
github.com/parquet-go/parquet-go v0.32.0/go.mod h1:navtkAYr2LGoJVp141oXPlO/sxLvaOe3la2JEoD8+rg=
github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ=
github.com/pierrec/lz4/v4 v4.1.21/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/twpayne/go-geom v1.6.1 h1:iLE+Opv0Ihm/ABIcvQFGIiFBXd76oBIar9drAwHFhR4=
github.com/twpayne/go-geom v1.6.1/go.mod h1:Kr+Nly6BswFsKM5sd31YaoWS5PeDDH2NftJTK7Gd028=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM= golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE= golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
Client/udpstreamer/main.go
+57 -3
View File
@@ -9,6 +9,9 @@ import (
"log" "log"
"net/http" "net/http"
"os" "os"
"os/signal"
"path/filepath"
"syscall"
"marte2/common/wshub" "marte2/common/wshub"
) )
@@ -21,20 +24,56 @@ var staticFiles embed.FS
// multiFlag allows a flag to be repeated: --source a --source b // multiFlag allows a flag to be repeated: --source a --source b
type multiFlag []string type multiFlag []string
func (f *multiFlag) String() string { return fmt.Sprintf("%v", []string(*f)) } func (f *multiFlag) String() string { return fmt.Sprintf("%v", []string(*f)) }
func (f *multiFlag) Set(v string) error { *f = append(*f, v); return nil } func (f *multiFlag) Set(v string) error { *f = append(*f, v); return nil }
// defaultHistoryDir is where samples are archived unless -history-dir says
// otherwise. History is on by default because it is what holds a trigger
// capture at full resolution: the in-memory rings roll past a captured window
// within seconds of it being taken, and a zoom after that has nothing but the
// capture's own decimated copy to draw. Per-signal files are bounded by
// -history-max-mpts, so the default costs a fixed amount of space.
func defaultHistoryDir() string {
return filepath.Join(os.TempDir(), "udpstreamer-history")
}
func main() { func main() {
var sourceArgs multiFlag var sourceArgs multiFlag
flag.Var(&sourceArgs, "source", `Data source in the form [label@]host:port[/multicastGroup:dataPort] (repeatable)`) flag.Var(&sourceArgs, "source", `Data source in the form [label@]host:port[/multicastGroup:dataPort] (repeatable)`)
sourcesFile := flag.String("sources-file", "", "JSON file for persistent source list (load on start, save target)") sourcesFile := flag.String("sources-file", "", "JSON file for persistent source list (load on start, save target)")
listenAddr := flag.String("addr", ":8080", "HTTP listen address") listenAddr := flag.String("addr", ":8080", "HTTP listen address")
histDir := flag.String("history-dir", defaultHistoryDir(), "Directory for disk-backed signal history (empty disables it)")
histWindow := flag.Float64("history-window-sec", 0, "Timespan the history files hold before any client says what it displays (0 keeps the 10 s default); the hub re-sizes them to the live or trigger window afterwards")
histDecim := flag.Int("history-decimation", 1, "Keep every Nth sample in the history files")
histFlush := flag.Int("history-flush-sec", 5, "Seconds between history header flushes")
histMinFree := flag.Int("history-min-free-mb", 500, "Pause history writing below this much free disk (negative disables the check)")
histMaxMPts := flag.Float64("history-max-mpts", 0, "Per-signal history budget in millions of points, also settable in the web UI (0 keeps the 16 MPts / 256 MB default)")
ringMPts := flag.Float64("ring-mpts", 0, "Per-signal in-memory buffer in millions of points (0 keeps the 10 MPts / 160 MB default)")
flag.Parse() flag.Parse()
hub := wshub.NewHub() hub := wshub.NewHub()
// The budget bounds memory, not the window: a window too long to hold at the
// source rate is buffered as min/max pairs rather than truncated to the tail.
hub.SetRingBudget(int(*ringMPts * 1e6))
sm := wshub.NewSourceManager(hub, *sourcesFile) sm := wshub.NewSourceManager(hub, *sourcesFile)
hub.SetSourceManager(sm) hub.SetSourceManager(sm)
if err := hub.EnableHistory(wshub.HistoryConfig{
Directory: *histDir,
WindowSec: *histWindow,
Decimation: *histDecim,
FlushIntervalSec: *histFlush,
MinDiskFreeMB: *histMinFree,
MaxPointsPerSignal: int(*histMaxMPts * 1e6),
}); err != nil {
log.Fatalf("history: %v", err)
}
if *histDir == "" {
log.Print("history disabled: zooming into a trigger capture will fall back " +
"to the capture's own decimated copy once the rings roll past it")
} else {
log.Printf("history: %s", *histDir)
}
go hub.Run() go hub.Run()
// Load sources from file first (if specified), then add any CLI --source flags. // Load sources from file first (if specified), then add any CLI --source flags.
@@ -55,12 +94,27 @@ func main() {
http.Handle("/", http.FileServer(http.FS(sub))) http.Handle("/", http.FileServer(http.FS(sub)))
http.HandleFunc("/ws", hub.HandleWebSocket) http.HandleFunc("/ws", hub.HandleWebSocket)
http.HandleFunc("/api/zoom", hub.HandleZoom) http.HandleFunc("/api/zoom", hub.HandleZoom)
http.HandleFunc("/api/export", hub.HandleExport)
http.HandleFunc("/version", func(w http.ResponseWriter, r *http.Request) { http.HandleFunc("/version", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, buildVersion) fmt.Fprint(w, buildVersion)
}) })
log.Printf("UDPStreamer WebUI listening on %s (build=%s)", *listenAddr, buildVersion) log.Printf("UDPStreamer WebUI listening on %s (build=%s)", *listenAddr, buildVersion)
if err := http.ListenAndServe(*listenAddr, nil); err != nil {
// Serve in the background so Ctrl-C can flush the history files: the
// samples written since the last periodic flush are on disk but are not
// yet accounted for in the file headers, so exiting outright loses them.
srvErr := make(chan error, 1)
go func() { srvErr <- http.ListenAndServe(*listenAddr, nil) }()
sig := make(chan os.Signal, 1)
signal.Notify(sig, os.Interrupt, syscall.SIGTERM)
select {
case err := <-srvErr:
hub.CloseHistory()
log.Fatalf("http: %v", err) log.Fatalf("http: %v", err)
case s := <-sig:
log.Printf("received %s, flushing history", s)
hub.CloseHistory()
} }
} }
Client/udpstreamer/static/app.js
+1882 -378
View File
File diff suppressed because it is too large Load Diff
Client/udpstreamer/static/calibration.js
+161
View File
@@ -0,0 +1,161 @@
// Per-signal affine calibration: value = raw * scale + offset, with an optional
// unit override. Pure and dependency-free so it can be unit-tested under Node;
// in the browser it defines the global `Calib`.
(function (root) {
'use strict';
var MAX_UNIT_LEN = 16;
var IDENTITY = Object.freeze({scale: 1, offset: 0, unit: ''});
// The table is keyed by (source label, base signal name). U+0000 cannot occur
// in either, so it is an unambiguous separator.
function calKey(source, signal) {
return source + '\u0000' + signal;
}
// 'Adc[3]' -> 'Adc'. One calibration covers every element of an array signal.
function baseSignalName(name) {
var s = String(name == null ? '' : name);
var open = s.lastIndexOf('[');
if (open >= 0 && s.charAt(s.length - 1) === ']') {
var idx = s.slice(open + 1, s.length - 1);
if (idx.length > 0 && /^[0-9]+$/.test(idx)) return s.slice(0, open);
}
return s;
}
function isFiniteNum(v) {
return typeof v === 'number' && isFinite(v);
}
// Mirrors the hub-side validation exactly (Go: CalConfig.Normalise,
// C++: StreamHub::HandleSetCalibration). Returns null when the entry must be
// rejected, so a caller can revert an input field to its last accepted value.
function normaliseCal(obj) {
if (obj === null || typeof obj !== 'object') return null;
var source = String(obj.source == null ? '' : obj.source).trim();
var signal = baseSignalName(String(obj.signal == null ? '' : obj.signal).trim());
if (source === '' || signal === '') return null;
var scale = obj.scale === undefined ? 1 : obj.scale;
var offset = obj.offset === undefined ? 0 : obj.offset;
if (!isFiniteNum(scale) || scale === 0) return null;
if (!isFiniteNum(offset)) return null;
var unit = String(obj.unit == null ? '' : obj.unit).trim();
// Cap at MAX_UNIT_LEN UTF-8 bytes, matching both hubs' Normalise() exactly.
// TextEncoder/TextDecoder are available natively in all modern browsers and
// Node v11+; no build step or bundler is needed.
var enc = new TextEncoder();
var bytes = enc.encode(unit);
if (bytes.length > MAX_UNIT_LEN) {
// Truncate to MAX_UNIT_LEN bytes, then repair any split UTF-8 rune at the
// tail. Mirrors Go's utf8.DecodeLastRuneInString walk-back loop and the
// C++ repair loop in StreamHub::SetCalibrationEntry:
// Drop continuation bytes (10xxxxxx) from the tail until the last byte
// is either an ASCII byte (< 0x80) or a lead byte whose sequence is
// complete (i.e. all expected continuation bytes are present).
//
// A single pass suffices here, where the C++ needs a loop. The C++ input
// is a raw const char* straight off the wire and may hold arbitrary bytes;
// `bytes` here comes from TextEncoder, which always emits well-formed
// UTF-8 (unpaired surrogates become U+FFFD = EF BF BD, and no byte is ever
// >= 0xF8). Truncating well-formed UTF-8 can therefore strand at most a
// lead byte plus three continuation bytes, which one pass fully repairs.
var b = bytes.slice(0, MAX_UNIT_LEN);
var len = b.length;
// Walk back over continuation bytes (up to 3) to find the lead byte of
// the last sequence.
var cont = 0;
while (cont < 3 && cont < len && (b[len - 1 - cont] & 0xC0) === 0x80) {
cont++;
}
if (cont < len) {
var lead = b[len - 1 - cont];
// Determine expected sequence length from the lead byte.
var seqLen = lead < 0x80 ? 1 : // 0xxxxxxx ASCII
(lead & 0xE0) === 0xC0 ? 2 : // 110xxxxx
(lead & 0xF0) === 0xE0 ? 3 : // 1110xxxx
(lead & 0xF8) === 0xF0 ? 4 : // 11110xxx
1; // orphaned continuation byte — treat as 1
var haveBytes = cont + 1; // lead + continuation bytes present
if (haveBytes < seqLen) {
// Incomplete sequence: drop the lead byte and all its continuations.
len = len - haveBytes;
}
}
unit = new TextDecoder().decode(b.slice(0, len));
}
return {source: source, signal: signal, scale: scale, offset: offset, unit: unit};
}
function isIdentity(cal) {
return cal.scale === 1 && cal.offset === 0 && cal.unit === '';
}
function applyCal(raw, cal) {
return raw * cal.scale + cal.offset;
}
function invertCal(value, cal) {
return (value - cal.offset) / cal.scale;
}
// A negative scale swaps the ends of a range, so re-order after calibrating.
function calRange(min, max, cal) {
var a = applyCal(min, cal), b = applyCal(max, cal);
return a <= b ? [a, b] : [b, a];
}
function CalTable() {
this._m = Object.create(null);
}
CalTable.prototype.get = function (source, signal) {
var e = this._m[calKey(source, baseSignalName(signal))];
return e === undefined ? IDENTITY : e;
};
// Returns false when the entry was rejected as invalid. An entry that reduces
// to the identity is deleted rather than stored, so a Reset cleans the table
// (and, once saved, the config file) instead of filling it with no-ops.
CalTable.prototype.set = function (entry) {
var c = normaliseCal(entry);
if (c === null) return false;
var k = calKey(c.source, c.signal);
if (isIdentity(c)) delete this._m[k];
else this._m[k] = c;
return true;
};
CalTable.prototype.replaceAll = function (list) {
this._m = Object.create(null);
if (!list) return;
for (var i = 0; i < list.length; i++) this.set(list[i]);
};
CalTable.prototype.list = function () {
var out = [], k;
for (k in this._m) out.push(this._m[k]);
out.sort(function (a, b) {
if (a.source !== b.source) return a.source < b.source ? -1 : 1;
if (a.signal !== b.signal) return a.signal < b.signal ? -1 : 1;
return 0;
});
return out;
};
var api = {
MAX_UNIT_LEN: MAX_UNIT_LEN,
IDENTITY: IDENTITY,
calKey: calKey,
baseSignalName: baseSignalName,
normaliseCal: normaliseCal,
isIdentity: isIdentity,
applyCal: applyCal,
invertCal: invertCal,
calRange: calRange,
CalTable: CalTable,
};
if (typeof module !== 'undefined' && module.exports) module.exports = api;
else root.Calib = api;
})(typeof globalThis !== 'undefined' ? globalThis : this);
Client/udpstreamer/static/decimate-worker.js
+51
View File
@@ -0,0 +1,51 @@
'use strict';
// Min/max (peak-envelope) decimation — O(n). Runs off-main-thread to avoid
// blocking the render loop.
//
// The range is split into threshold/2 equal buckets and each contributes its
// smallest and largest sample, in the order the two occurred — the way an
// oscilloscope draws a trace it cannot show pixel-for-pixel.
//
// This replaced LTTB, which picks the sample forming the largest triangle with
// its neighbours: a plausible-looking shape, but it silently drops a one-sample
// spike whenever a smoother neighbour scores higher — exactly the sample worth
// looking at. The envelope cannot drop it, because a spike is by definition its
// bucket's min or max. Every output point is a real sample at its real
// timestamp; nothing is interpolated or averaged.
//
// Kept identical to minMaxDecimate() in Common/Client/go/wshub/hub.go and to
// decimate() in app.js, so a trace looks the same whichever thinned it.
function decimate(t, v, threshold) {
const len = t.length;
if (len <= threshold || threshold < 4) {
// Copy to new arrays so we can transfer them back without detaching the input.
return { t: new Float64Array(t), v: new Float64Array(v) };
}
const buckets = threshold >> 1;
const outT = new Float64Array(threshold);
const outV = new Float64Array(threshold);
let n = 0;
for (let b = 0; b < buckets; b++) {
const lo = Math.floor(b * len / buckets);
const hi = (b === buckets - 1) ? len : Math.floor((b + 1) * len / buckets);
if (lo >= hi) continue;
let iMin = lo, iMax = lo;
for (let j = lo + 1; j < hi; j++) {
if (v[j] < v[iMin]) iMin = j;
if (v[j] > v[iMax]) iMax = j;
}
// Emit in time order so the result plots as one ascending trace.
if (iMin > iMax) { const s = iMin; iMin = iMax; iMax = s; }
outT[n] = t[iMin]; outV[n] = v[iMin]; n++;
// A bucket whose samples are all equal has one extreme, not two.
if (iMax !== iMin) { outT[n] = t[iMax]; outV[n] = v[iMax]; n++; }
}
// slice() so the transferred buffers are exactly the used length.
return { t: outT.slice(0, n), v: outV.slice(0, n) };
}
self.onmessage = function({ data: { id, t, v, threshold } }) {
const result = decimate(t, v, threshold);
// Transfer the output buffers back to the main thread zero-copy.
self.postMessage({ id, t: result.t, v: result.v }, [result.t.buffer, result.v.buffer]);
};
Client/udpstreamer/static/index.html
+83 -14
View File
@@ -21,20 +21,38 @@
<span id="cur-ta">A: —</span><span class="cur-sep">│</span> <span id="cur-ta">A: —</span><span class="cur-sep">│</span>
<span id="cur-tb">B: —</span><span class="cur-sep">│</span> <span id="cur-tb">B: —</span><span class="cur-sep">│</span>
<span id="cur-dt">ΔT: —</span> <span id="cur-dt">ΔT: —</span>
<span id="ruler-readout" style="display:none">
<span class="cur-sep">│</span>
<span id="cur-y1">Y1: —</span><span class="cur-sep">│</span>
<span id="cur-y2">Y2: —</span><span class="cur-sep">│</span>
<span id="cur-dy">ΔY: —</span>
</span>
</div> </div>
<span class="ctrl-label" id="lbl-window">Window:</span> <span class="ctrl-label" id="lbl-window">Window:</span>
<select id="window-select" class="ctrl-select"> <select id="window-select" class="ctrl-select">
<option value="1">1 s</option><option value="5" selected>5 s</option> <option value="1">1 s</option><option value="2">2 s</option>
<option value="10">10 s</option><option value="30">30 s</option> <option value="5" selected>5 s</option><option value="10">10 s</option>
<option value="60">60 s</option> <option value="15">15 s</option><option value="30">30 s</option>
<option value="60">60 s</option><option value="120">2 min</option>
<option value="300">5 min</option><option value="600">10 min</option>
</select> </select>
<button id="btn-cursor" class="ctrl-btn" style="display:none">Cursor</button> <button id="btn-cursor" class="ctrl-btn">Cursors</button>
<button id="btn-cursor-reset" class="ctrl-btn" style="display:none" title="Bring cursors A/B back into the visible window">↔ Reset</button>
<button id="btn-ruler" class="ctrl-btn" title="Horizontal value rulers">Rulers</button>
<button id="btn-zoom-back" class="ctrl-btn" style="display:none">← Back</button> <button id="btn-zoom-back" class="ctrl-btn" style="display:none">← Back</button>
<button id="btn-zoom-fit" class="ctrl-btn">Fit</button> <button id="btn-zoom-fit" class="ctrl-btn">Fit</button>
<button id="btn-csv-all" class="ctrl-btn" title="Export all signals to CSV">⬇ CSV</button> <select id="export-select" class="ctrl-select" title="Export the visible signals">
<option value="" disabled selected>⬇ Export</option>
<option value="csv" title="Export the visible signals as CSV (decimated to a bounded row count)">CSV</option>
<option value="parquet" title="Export every stored sample (full resolution, no holes) as Parquet — requires the Go hub">Parquet</option>
</select>
<button id="btn-sync-resume" class="ctrl-btn resume-btn" style="display:none">↺ Auto</button> <button id="btn-sync-resume" class="ctrl-btn resume-btn" style="display:none">↺ Auto</button>
<button id="btn-trigger" class="ctrl-btn">⚡ Trigger</button> <button id="btn-trigger" class="ctrl-btn">⚡ Trigger</button>
<button id="btn-pause-global" class="ctrl-btn">⏸ Pause</button> <button id="btn-pause-global" class="ctrl-btn">⏸ Pause</button>
<label class="ctrl-check" title="Snap jittery inter-frame timestamps to ideal spacing (eliminates overlaps/gaps from software-dispatch jitter)">
<input type="checkbox" id="cb-monotonic">
Sync TS
</label>
</div> </div>
<!-- ── Trigger bar ───────────────────────────────────────────── --> <!-- ── Trigger bar ───────────────────────────────────────────── -->
<div id="trigbar"> <div id="trigbar">
@@ -60,10 +78,19 @@
<div class="trig-group"> <div class="trig-group">
<span class="trig-label">Window</span> <span class="trig-label">Window</span>
<select id="trig-window" class="trig-select"> <select id="trig-window" class="trig-select">
<option value="0.0001">100 μs</option><option value="0.001">1 ms</option> <option value="0.0001">100 μs</option><option value="0.0002">200 μs</option>
<option value="0.01">10 ms</option><option value="0.1">100 ms</option> <option value="0.0005">500 μs</option><option value="0.001">1 ms</option>
<option value="0.5">500 ms</option><option value="1" selected>1 s</option> <option value="0.002">2 ms</option><option value="0.005">5 ms</option>
<option value="0.01">10 ms</option><option value="0.02">20 ms</option>
<option value="0.05">50 ms</option><option value="0.1">100 ms</option>
<option value="0.2">200 ms</option><option value="0.5">500 ms</option>
<option value="1" selected>1 s</option><option value="2">2 s</option>
<option value="5">5 s</option><option value="10">10 s</option> <option value="5">5 s</option><option value="10">10 s</option>
<option value="20">20 s</option><option value="30">30 s</option>
<option value="60">60 s</option>
<option value="120">2 m</option>
<option value="300">5 m</option>
<option value="600">10 m</option>
</select> </select>
</div> </div>
<div class="trig-sep"></div> <div class="trig-sep"></div>
@@ -73,6 +100,11 @@
<span class="trig-range-val" id="trig-pre-val">20%</span> <span class="trig-range-val" id="trig-pre-val">20%</span>
</div> </div>
<div class="trig-sep"></div> <div class="trig-sep"></div>
<div class="trig-group">
<span class="trig-label" title="Re-arm delay after a capture — prevents double triggering">Holdoff</span>
<input id="trig-holdoff" class="trig-input" type="number" min="0" max="60" step="0.01" value="0.2">
<span class="trig-label">s</span>
</div>
<div class="trig-group"> <div class="trig-group">
<span class="trig-label">Mode</span> <span class="trig-label">Mode</span>
<select id="trig-mode" class="trig-select"> <select id="trig-mode" class="trig-select">
@@ -83,6 +115,7 @@
<div class="trig-sep"></div> <div class="trig-sep"></div>
<div class="trig-group" style="gap:8px"> <div class="trig-group" style="gap:8px">
<span id="trig-status-badge">IDLE</span> <span id="trig-status-badge">IDLE</span>
<button id="btn-trig-force" title="Capture now, ignoring the threshold">Force</button>
<button id="btn-trig-stop" style="display:none">Stop</button> <button id="btn-trig-stop" style="display:none">Stop</button>
<button id="btn-trig-rearm">Rearm</button> <button id="btn-trig-rearm">Rearm</button>
</div> </div>
@@ -113,10 +146,30 @@
<span id="status-text">Disconnected</span> <span id="status-text">Disconnected</span>
<span id="sb-tsage"></span> <span id="sb-tsage"></span>
<button id="btn-stats" class="ctrl-btn" style="height:16px;padding:0 7px;font-size:10px;line-height:1">📊 Stats</button> <button id="btn-stats" class="ctrl-btn" style="height:16px;padding:0 7px;font-size:10px;line-height:1">📊 Stats</button>
<span id="history-badge" style="display:none;font-size:10px;color:#f9e2af;margin-left:8px"></span> <button id="history-badge" style="display:none" title="Disk history — click to set the per-signal budget"></button>
</div> </div>
<span id="build-version"></span> <span id="build-version"></span>
</div> </div>
<!-- ── History budget popup ──────────────────────────────────── -->
<div id="history-panel" style="display:none">
<div class="ctx-menu-header">Disk history budget</div>
<div class="ctx-row">
<label>Budget</label>
<input type="number" id="hist-budget" class="ctx-num" min="0.001" step="1">
<span class="ctx-range-val">MPts/signal</span>
</div>
<div class="hist-note">
The budget buys resolution, not duration: a signal too fast to store
sample-for-sample is archived as a min/max envelope wide enough to fit,
so the configured window is always covered.
</div>
<div id="hist-signal-res"></div>
<div class="hist-note hist-warn">Applying re-creates the history files — archived data is lost.</div>
<div class="ctx-row" style="margin:0;justify-content:flex-end">
<button class="ctx-btn" id="btn-hist-cancel">Cancel</button>
<button class="ctx-btn" id="btn-hist-apply">Apply</button>
</div>
</div>
<div id="layout-menu"></div> <div id="layout-menu"></div>
<!-- ── Signal style context menu ─────────────────────────────── --> <!-- ── Signal style context menu ─────────────────────────────── -->
<div id="sig-ctx-menu" style="display:none"> <div id="sig-ctx-menu" style="display:none">
@@ -161,7 +214,8 @@
</div> </div>
<!-- ── Array index picker (trigger signal) ──────────────────────── --> <!-- ── Array index picker (trigger signal) ──────────────────────── -->
<div id="array-idx-picker" style="display:none"> <div id="array-idx-picker" style="display:none">
<div class="ctx-menu-header">Element index: <span id="aip-sig" class="ctx-menu-key"></span></div> <div class="ctx-menu-header">Element index:
<span id="aip-sig" class="ctx-menu-key"></span></div>
<div class="ctx-row"> <div class="ctx-row">
<label>Index</label> <label>Index</label>
<input type="number" id="aip-idx" class="ctx-num" min="0" step="1" value="0"> <input type="number" id="aip-idx" class="ctx-num" min="0" step="1" value="0">
@@ -175,7 +229,8 @@
<!-- ── VScale toolbar (moved into plot card when active) ─────────── --> <!-- ── VScale toolbar (moved into plot card when active) ─────────── -->
<div id="vscale-menu" style="display:none"> <div id="vscale-menu" style="display:none">
<div class="vstb-header"> <div class="vstb-header">
<span class="vstb-label">V-Scale: <span id="vscale-menu-key" class="ctx-menu-key"></span></span> <span class="vstb-label"><span id="vscale-menu-title">V-Scale</span>:
<span id="vscale-menu-key" class="ctx-menu-key"></span></span>
<div class="ctx-btns" id="vscale-mode-btns"> <div class="ctx-btns" id="vscale-mode-btns">
<button class="ctx-btn active" data-mode="auto">Auto</button> <button class="ctx-btn active" data-mode="auto">Auto</button>
<button class="ctx-btn" data-mode="range">Range</button> <button class="ctx-btn" data-mode="range">Range</button>
@@ -185,9 +240,9 @@
<label class="vstb-lbl">V/div</label> <label class="vstb-lbl">V/div</label>
<input type="number" id="vscale-vdiv" class="ctx-num" min="1e-30" step="any" value="1"> <input type="number" id="vscale-vdiv" class="ctx-num" min="1e-30" step="any" value="1">
</div> </div>
<div id="vscale-pos-row" style="display:none;align-items:center;gap:4px"> <div id="vscale-offset-row" style="display:none;align-items:center;gap:4px">
<label class="vstb-lbl">Pos</label> <label class="vstb-lbl" title="Raw value at screen centre — unbounded, may lie outside the plotted range">Offset</label>
<input type="number" id="vscale-pos" class="ctx-num" step="0.1" value="0"> <input type="number" id="vscale-offset" class="ctx-num" step="any" value="0">
</div> </div>
<div id="vscale-type-row" style="display:none;align-items:center;gap:4px"> <div id="vscale-type-row" style="display:none;align-items:center;gap:4px">
<label class="vstb-lbl">Type</label> <label class="vstb-lbl">Type</label>
@@ -196,9 +251,23 @@
<button class="ctx-btn" data-type="digital">Digital</button> <button class="ctx-btn" data-type="digital">Digital</button>
</div> </div>
</div> </div>
<div class="vstb-sep"></div>
<div id="vscale-cal-row" style="display:flex;align-items:center;gap:4px">
<label class="vstb-lbl" id="vscale-cal-lbl" title="Data calibration: value = raw × Scale + Offset. Applies to the plot, cursors, hover readout, CSV export and trigger threshold.">Cal</label>
<label class="vstb-lbl">Scale</label>
<input type="number" id="vscale-cal-scale" class="ctx-num ctx-num-sm" step="any" value="1">
<label class="vstb-lbl">Offset</label>
<input type="number" id="vscale-cal-offset" class="ctx-num ctx-num-sm" step="any" value="0">
<label class="vstb-lbl">Unit</label>
<input type="text" id="vscale-cal-unit" class="ctx-num ctx-num-xs" placeholder="—">
<button class="ctx-btn" id="btn-cal-reset" title="Clear this signal's calibration">Reset</button>
</div>
<button id="btn-vscale-close" class="vstb-close" title="Close">✕</button> <button id="btn-vscale-close" class="vstb-close" title="Close">✕</button>
</div> </div>
</div> </div>
<!-- Follows the mouse over a plot: time + per-trace values. -->
<div id="hover-readout" style="display:none"></div>
<script src="/calibration.js"></script>
<script src="/app.js"></script> <script src="/app.js"></script>
</body> </body>
</html> </html>
Client/udpstreamer/static/lttb-worker.js
-39
View File
@@ -1,39 +0,0 @@
'use strict';
// LTTB (Largest Triangle Three Buckets) decimation — O(n).
// Runs off-main-thread to avoid blocking the render loop.
function lttb(t, v, threshold) {
const len = t.length;
if (len <= threshold) {
// Copy to new arrays so we can transfer them back without detaching the input.
return { t: new Float64Array(t), v: new Float64Array(v) };
}
const outT = new Float64Array(threshold);
const outV = new Float64Array(threshold);
outT[0] = t[0]; outV[0] = v[0];
outT[threshold - 1] = t[len - 1]; outV[threshold - 1] = v[len - 1];
const every = (len - 2) / (threshold - 2);
let a = 0;
for (let i = 0; i < threshold - 2; i++) {
const avgS = Math.floor((i + 1) * every) + 1;
const avgE = Math.min(Math.floor((i + 2) * every) + 1, len);
let avgT = 0, avgV = 0, n = 0;
for (let j = avgS; j < avgE; j++) { avgT += t[j]; avgV += v[j]; n++; }
if (n) { avgT /= n; avgV /= n; }
const rS = Math.floor(i * every) + 1;
const rE = Math.min(Math.floor((i + 1) * every) + 1, len);
let maxA = -1, next = rS;
const aT = t[a], aV = v[a];
for (let j = rS; j < rE; j++) {
const area = Math.abs((aT - avgT) * (v[j] - aV) - (aT - t[j]) * (avgV - aV));
if (area > maxA) { maxA = area; next = j; }
}
outT[i + 1] = t[next]; outV[i + 1] = v[next]; a = next;
}
return { t: outT, v: outV };
}
self.onmessage = function({ data: { id, t, v, threshold } }) {
const result = lttb(t, v, threshold);
// Transfer the output buffers back to the main thread zero-copy.
self.postMessage({ id, t: result.t, v: result.v }, [result.t.buffer, result.v.buffer]);
};
Client/udpstreamer/static/style.css
+81 -19
View File
@@ -14,6 +14,11 @@
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; } *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
html, body { height:100%; background:var(--bg); color:var(--text); html, body { height:100%; background:var(--bg); color:var(--text);
font-family:'Segoe UI',system-ui,sans-serif; font-size:14px; overflow:hidden; } font-family:'Segoe UI',system-ui,sans-serif; font-size:14px; overflow:hidden; }
/* Uniform 0.9x compaction — scales every element (fonts, bars, plots,
spacing) while reflowing layout. `zoom` (Chrome/Edge/Safari, Firefox 126+)
is preferred over `transform: scale` because it reflows, so fixed-position
bars and JS-computed offsets stay aligned. */
html { zoom: 0.9; }
::-webkit-scrollbar { width:6px; } ::-webkit-scrollbar { width:6px; }
::-webkit-scrollbar-track { background:var(--mantle); } ::-webkit-scrollbar-track { background:var(--mantle); }
::-webkit-scrollbar-thumb { background:var(--surface1); border-radius:3px; } ::-webkit-scrollbar-thumb { background:var(--surface1); border-radius:3px; }
@@ -52,6 +57,23 @@ html, body { height:100%; background:var(--bg); color:var(--text);
#cursor-readout.visible { display:flex; } #cursor-readout.visible { display:flex; }
#cur-ta { color:var(--sky); } #cur-tb { color:var(--yellow); } #cur-ta { color:var(--sky); } #cur-tb { color:var(--yellow); }
#cur-dt { color:var(--subtext1); } .cur-sep { color:var(--surface2); } #cur-dt { color:var(--subtext1); } .cur-sep { color:var(--surface2); }
#ruler-readout { display:inline-flex; align-items:center; gap:8px; }
#cur-y1 { color:var(--green); } #cur-y2 { color:var(--red); }
#cur-dy { color:var(--subtext1); }
/* Mouse-over time/value tooltip */
#hover-readout {
position:fixed; z-index:60; pointer-events:none;
background:var(--surface0); border:1px solid var(--surface1);
border-radius:5px; padding:4px 8px;
font-size:11px; font-family:monospace; white-space:nowrap;
box-shadow:0 4px 12px rgba(0,0,0,0.45);
}
#hover-readout .hov-time { color:var(--subtext1); margin-bottom:3px; }
#hover-readout .hov-row { display:flex; align-items:center; gap:6px; }
#hover-readout .hov-dot { width:8px; height:8px; border-radius:50%; flex-shrink:0; }
#hover-readout .hov-name { color:var(--subtext0); }
#hover-readout .hov-val { color:var(--text); margin-left:auto; padding-left:10px; }
.topbar-vsep { width:1px; height:22px; background:var(--surface0); flex-shrink:0; margin:0 2px; } .topbar-vsep { width:1px; height:22px; background:var(--surface0); flex-shrink:0; margin:0 2px; }
#layout-btns { display:flex; gap:2px; align-items:center; flex-shrink:0; } #layout-btns { display:flex; gap:2px; align-items:center; flex-shrink:0; }
@@ -74,6 +96,12 @@ button.ctrl-btn.trig-active { background:rgba(203,166,247,0.15); border-color:va
button.ctrl-btn.cursor-a { border-color:var(--sky); color:var(--sky); } button.ctrl-btn.cursor-a { border-color:var(--sky); color:var(--sky); }
button.ctrl-btn.cursor-b { border-color:var(--yellow); color:var(--yellow); } button.ctrl-btn.cursor-b { border-color:var(--yellow); color:var(--yellow); }
button.ctrl-btn.resume-btn { border-color:var(--teal); color:var(--teal); } button.ctrl-btn.resume-btn { border-color:var(--teal); color:var(--teal); }
label.ctrl-check {
display:flex; align-items:center; gap:4px; flex-shrink:0;
font-size:12px; color:var(--subtext0); cursor:pointer; white-space:nowrap;
}
label.ctrl-check input { margin:0; cursor:pointer; accent-color:var(--accent); }
label.ctrl-check:has(input:checked) { color:var(--accent); }
/* ── Trigger bar ──────────────────────────────────────────────── */ /* ── Trigger bar ──────────────────────────────────────────────── */
#trigbar { #trigbar {
@@ -118,10 +146,17 @@ input[type=range].trig-range::-webkit-slider-thumb {
#trig-status-badge.armed { background:rgba(166,227,161,0.12); border-color:var(--green); color:var(--green); } #trig-status-badge.armed { background:rgba(166,227,161,0.12); border-color:var(--green); color:var(--green); }
#trig-status-badge.waiting { background:rgba(249,226,175,0.12); border-color:var(--yellow); color:var(--yellow); } #trig-status-badge.waiting { background:rgba(249,226,175,0.12); border-color:var(--yellow); color:var(--yellow); }
#trig-status-badge.triggered { background:rgba(203,166,247,0.15); border-color:var(--mauve); color:var(--mauve); } #trig-status-badge.triggered { background:rgba(203,166,247,0.15); border-color:var(--mauve); color:var(--mauve); }
#btn-trig-rearm, #btn-trig-stop { #btn-trig-force, #btn-trig-rearm, #btn-trig-stop {
border:none; border-radius:5px; border:none; border-radius:5px;
padding:4px 12px; font-size:12px; font-weight:600; cursor:pointer; display:none; padding:4px 12px; font-size:12px; font-weight:600; cursor:pointer;
} }
#btn-trig-rearm, #btn-trig-stop { display:none; }
#btn-trig-force {
background:var(--surface0); color:var(--text);
border:1px solid var(--surface1);
transition:background var(--transition),border-color var(--transition),color var(--transition);
}
#btn-trig-force:hover { background:var(--surface1); border-color:var(--mauve); color:var(--mauve); }
#btn-trig-rearm { background:var(--mauve); color:var(--crust); } #btn-trig-rearm { background:var(--mauve); color:var(--crust); }
#btn-trig-stop { background:var(--surface1); color:var(--yellow); border:1px solid var(--yellow); } #btn-trig-stop { background:var(--surface1); color:var(--yellow); border:1px solid var(--yellow); }
#btn-trig-rearm:hover, #btn-trig-stop:hover { opacity:0.85; } #btn-trig-rearm:hover, #btn-trig-stop:hover { opacity:0.85; }
@@ -169,23 +204,6 @@ input[type=range].trig-range::-webkit-slider-thumb {
.sig-name { flex:1; font-size:13px; color:var(--text); overflow:hidden; text-overflow:ellipsis; white-space:nowrap; } .sig-name { flex:1; font-size:13px; color:var(--text); overflow:hidden; text-overflow:ellipsis; white-space:nowrap; }
.sig-unit { font-size:11px; color:var(--subtext0); font-style:italic; } .sig-unit { font-size:11px; color:var(--subtext0); font-style:italic; }
.type-badge { font-size:10px; background:var(--surface1); color:var(--subtext1); padding:1px 5px; border-radius:3px; white-space:nowrap; } .type-badge { font-size:10px; background:var(--surface1); color:var(--subtext1); padding:1px 5px; border-radius:3px; white-space:nowrap; }
.array-group {}
.array-header {
padding:6px 14px 6px 10px; cursor:pointer; border-radius:6px; margin:1px 6px;
transition:background var(--transition); display:flex; align-items:center; gap:6px; user-select:none;
}
.array-header:hover { background:var(--surface0); }
.array-arrow { font-size:10px; color:var(--subtext0); transition:transform var(--transition); display:inline-block; }
.array-header.open .array-arrow { transform:rotate(90deg); }
.array-children { display:none; padding-left:16px; }
.array-header.open + .array-children { display:block; }
.array-child {
padding:4px 14px 4px 8px; cursor:grab; border-radius:6px; margin:1px 6px;
transition:background var(--transition); display:flex; align-items:center; gap:8px;
user-select:none; color:var(--subtext1); font-size:12px;
}
.array-child:hover { background:var(--surface0); }
.array-child:active { cursor:grabbing; }
/* ── Main area ────────────────────────────────────────────────── */ /* ── Main area ────────────────────────────────────────────────── */
#main { flex:1; display:flex; flex-direction:column; overflow:hidden; min-width:0; } #main { flex:1; display:flex; flex-direction:column; overflow:hidden; min-width:0; }
@@ -243,6 +261,9 @@ input[type=range].trig-range::-webkit-slider-thumb {
#plot-grid.l2x3 { grid-template-columns:1fr 1fr; grid-template-rows:1fr 1fr 1fr; } #plot-grid.l2x3 { grid-template-columns:1fr 1fr; grid-template-rows:1fr 1fr 1fr; }
#plot-grid.l1x4 { grid-template-columns:1fr; grid-template-rows:1fr 1fr 1fr 1fr; } #plot-grid.l1x4 { grid-template-columns:1fr; grid-template-rows:1fr 1fr 1fr 1fr; }
#plot-grid.l4x1 { grid-template-columns:1fr 1fr 1fr 1fr; grid-template-rows:1fr; } #plot-grid.l4x1 { grid-template-columns:1fr 1fr 1fr 1fr; grid-template-rows:1fr; }
/* 1+2 layout: one plot spanning the top row, two side by side below. */
#plot-grid.l1p2 { grid-template-columns:1fr 1fr; grid-template-rows:1fr 1fr; }
#plot-grid.l1p2 .plot-card:first-child { grid-column: 1 / -1; }
/* ── Plot card ────────────────────────────────────────────────── */ /* ── Plot card ────────────────────────────────────────────────── */
.plot-card { .plot-card {
@@ -301,6 +322,32 @@ input[type=range].trig-range::-webkit-slider-thumb {
border:1px solid var(--mauve); border:1px solid var(--mauve);
} }
/* ── History budget ───────────────────────────────────────────── */
#history-badge {
font-size:10px; color:var(--yellow); margin-left:8px; cursor:pointer;
background:transparent; border:1px solid transparent; border-radius:4px;
padding:1px 5px; white-space:nowrap;
}
#history-badge:hover { border-color:var(--yellow); background:rgba(249,226,175,0.10); }
#history-panel {
position:fixed; z-index:300;
background:var(--mantle); border:1px solid var(--surface1); border-radius:var(--radius);
box-shadow:0 8px 24px rgba(0,0,0,0.6); padding:10px; width:290px;
}
.hist-note { font-size:10px; color:var(--overlay0); line-height:1.4; margin:6px 0; }
.hist-warn { color:var(--peach); }
#hist-signal-res {
font-size:10px; font-family:monospace; color:var(--subtext0);
max-height:120px; overflow-y:auto;
border-top:1px solid var(--surface0); border-bottom:1px solid var(--surface0);
padding:5px 0;
}
.hist-res-row { display:flex; justify-content:space-between; gap:8px; }
.hist-res-row .hist-res-key {
overflow:hidden; text-overflow:ellipsis; white-space:nowrap; color:var(--subtext1);
}
.hist-res-row .hist-res-val { color:var(--mauve); flex-shrink:0; }
/* ── Signal style context menu ────────────────────────────────── */ /* ── Signal style context menu ────────────────────────────────── */
#sig-ctx-menu { #sig-ctx-menu {
position:fixed; z-index:300; position:fixed; z-index:300;
@@ -363,6 +410,11 @@ input[type=range].trig-range::-webkit-slider-thumb {
} }
.vstb-close:hover { color:var(--red); } .vstb-close:hover { color:var(--red); }
.plot-vscale-bar { display:none; } .plot-vscale-bar { display:none; }
.vstb-sep { width:1px; height:16px; background:var(--surface1); flex-shrink:0; }
.ctx-num-sm { width:70px; }
.ctx-num-xs { width:46px; }
#vscale-cal-lbl { color:var(--mauve); font-weight:600; }
.cal-invalid { border-color:var(--red) !important; }
/* ── Per-plot cursor value readout (in plot card header) ────────── */ /* ── Per-plot cursor value readout (in plot card header) ────────── */
.plot-cursor-ro { .plot-cursor-ro {
@@ -450,6 +502,16 @@ input[type=range].trig-range::-webkit-slider-thumb {
.add-src-btn:hover { background:rgba(137,180,250,0.15); border-color:var(--accent); } .add-src-btn:hover { background:rgba(137,180,250,0.15); border-color:var(--accent); }
.save-src-btn { color:var(--green); } .save-src-btn { color:var(--green); }
.save-src-btn:hover { background:rgba(166,227,161,0.1); border-color:var(--green); } .save-src-btn:hover { background:rgba(166,227,161,0.1); border-color:var(--green); }
.cfg-btn-row { display:flex; gap:6px; }
.cfg-btn-row .add-src-btn { flex:1; }
.reload-src-btn { color:var(--peach); }
.reload-src-btn:hover { background:rgba(250,179,135,0.1); border-color:var(--peach); }
.cfg-status {
font-size:10px; line-height:1.3; padding:2px 0; min-height:13px;
overflow-wrap:anywhere;
}
.cfg-status.ok { color:var(--green); }
.cfg-status.err { color:var(--red); }
/* ── Stats panel ─────────────────────────────────────────────── */ /* ── Stats panel ─────────────────────────────────────────────── */
#stats-panel { #stats-panel {
Client/udpstreamer/test/calibration.test.js
+158
View File
@@ -0,0 +1,158 @@
const test = require('node:test');
const assert = require('node:assert');
const C = require('../static/calibration.js');
test('baseSignalName strips an element suffix', () => {
assert.strictEqual(C.baseSignalName('Adc'), 'Adc');
assert.strictEqual(C.baseSignalName('Adc[3]'), 'Adc');
assert.strictEqual(C.baseSignalName('Adc[12]'), 'Adc');
assert.strictEqual(C.baseSignalName('A[1]B'), 'A[1]B');
assert.strictEqual(C.baseSignalName(''), '');
// A name that is entirely the suffix "[0]" must reduce to the empty string,
// matching Go (arrayIndexSuffix regexp) and C++ (strchr truncation) behaviour.
assert.strictEqual(C.baseSignalName('[0]'), '');
});
test('calKey is stable and separates the two fields', () => {
assert.strictEqual(C.calKey('a', 'b'), C.calKey('a', 'b'));
assert.notStrictEqual(C.calKey('ab', 'c'), C.calKey('a', 'bc'));
});
test('normaliseCal accepts a valid entry and fills defaults', () => {
assert.deepStrictEqual(
C.normaliseCal({source: ' wave ', signal: ' Adc ', scale: 2, offset: -1, unit: ' V '}),
{source: 'wave', signal: 'Adc', scale: 2, offset: -1, unit: 'V'});
assert.deepStrictEqual(
C.normaliseCal({source: 'wave', signal: 'Adc'}),
{source: 'wave', signal: 'Adc', scale: 1, offset: 0, unit: ''});
});
test('normaliseCal strips an element suffix from the signal name', () => {
assert.strictEqual(C.normaliseCal({source: 'w', signal: 'Adc[3]'}).signal, 'Adc');
});
test('normaliseCal truncates an over-long unit', () => {
const long = 'abcdefghijklmnopqrstuvwxyz';
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: long}).unit,
long.slice(0, C.MAX_UNIT_LEN));
});
test('normaliseCal leaves short non-ASCII units untouched', () => {
// 'Ω' is U+03A9, 2 UTF-8 bytes — well within 16 bytes.
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: 'Ω'}).unit, 'Ω');
// 'µs' is U+00B5 + U+0073, 3 UTF-8 bytes.
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: 'µs'}).unit, 'µs');
// '°C' is U+00B0 + U+0043, 3 UTF-8 bytes.
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: '°C'}).unit, '°C');
});
test('normaliseCal truncates an over-long ASCII unit to exactly 16 bytes', () => {
// 20 ASCII characters — each 1 byte, so cut at character 16.
const long = 'abcdefghijklmnopqrst'; // 20 chars
const result = C.normaliseCal({source: 'w', signal: 's', unit: long}).unit;
assert.strictEqual(result, 'abcdefghijklmnop'); // first 16 bytes/chars
assert.strictEqual(new TextEncoder().encode(result).length, 16);
});
test('normaliseCal cuts a mid-rune byte boundary back to the last complete rune', () => {
// Each 'Ω' (U+03A9) is 2 UTF-8 bytes (CE A9).
// 8 × 'Ω' = 16 bytes exactly — fits without truncation.
const fits = 'ΩΩΩΩΩΩΩΩ';
assert.strictEqual(new TextEncoder().encode(fits).length, 16);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: fits}).unit, fits);
// 9 × 'Ω' = 18 bytes. Slicing at 16 bytes lands in the middle of the 9th
// 'Ω' (only 1 of its 2 bytes is in the window) so only 8 'Ω' should survive.
// No U+FFFD replacement character must appear.
const toolong = 'ΩΩΩΩΩΩΩΩΩ';
const result = C.normaliseCal({source: 'w', signal: 's', unit: toolong}).unit;
assert.strictEqual(result, 'ΩΩΩΩΩΩΩΩ');
assert.ok(!result.includes('\uFFFD'), 'must not contain U+FFFD replacement character');
assert.strictEqual(new TextEncoder().encode(result).length, 16);
});
test('normaliseCal leaves a unit that is exactly 16 bytes ending on a complete multi-byte rune untouched', () => {
// 'abcdefgΩhijklµ' → 7 ASCII + 'Ω' (2 bytes) + 5 ASCII + 'µ' (2 bytes) = 16 bytes
const u = 'abcdefgΩhijklµ';
assert.strictEqual(new TextEncoder().encode(u).length, 16);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', unit: u}).unit, u);
});
test('normaliseCal rejects invalid entries', () => {
assert.strictEqual(C.normaliseCal(null), null);
assert.strictEqual(C.normaliseCal({signal: 's'}), null);
assert.strictEqual(C.normaliseCal({source: 'w'}), null);
assert.strictEqual(C.normaliseCal({source: ' ', signal: 's'}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', scale: 0}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', scale: NaN}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', scale: Infinity}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', offset: NaN}), null);
assert.strictEqual(C.normaliseCal({source: 'w', signal: 's', scale: '2'}), null);
// A signal name that is entirely an array-index suffix reduces to the empty
// string after stripping, so the entry must be rejected — matching Go and C++.
assert.strictEqual(C.normaliseCal({source: 'w', signal: '[0]'}), null);
});
test('applyCal and invertCal round-trip', () => {
const cal = {scale: 0.5, offset: -1.25, unit: 'V'};
assert.strictEqual(C.applyCal(10, cal), 3.75);
assert.strictEqual(C.invertCal(3.75, cal), 10);
assert.strictEqual(C.applyCal(7, C.IDENTITY), 7);
assert.strictEqual(C.invertCal(7, C.IDENTITY), 7);
});
test('applyCal passes non-finite samples through untouched', () => {
assert.ok(Number.isNaN(C.applyCal(NaN, {scale: 2, offset: 1, unit: ''})));
});
test('calRange re-orders when the scale is negative', () => {
assert.deepStrictEqual(C.calRange(0, 10, {scale: 2, offset: 1, unit: ''}), [1, 21]);
assert.deepStrictEqual(C.calRange(0, 10, {scale: -2, offset: 1, unit: ''}), [-19, 1]);
});
test('CalTable.get returns IDENTITY for an unknown signal', () => {
const t = new C.CalTable();
assert.deepStrictEqual(t.get('w', 'Adc'), C.IDENTITY);
});
test('CalTable.get resolves an element name to its base signal', () => {
const t = new C.CalTable();
t.set({source: 'w', signal: 'Adc', scale: 3, offset: 0, unit: ''});
assert.strictEqual(t.get('w', 'Adc[7]').scale, 3);
});
test('CalTable.set stores, overwrites, and deletes identity entries', () => {
const t = new C.CalTable();
assert.strictEqual(t.set({source: 'w', signal: 'Adc', scale: 2}), true);
assert.strictEqual(t.get('w', 'Adc').scale, 2);
t.set({source: 'w', signal: 'Adc', scale: 5});
assert.strictEqual(t.get('w', 'Adc').scale, 5);
assert.strictEqual(t.list().length, 1);
// Resetting to identity removes the entry entirely.
assert.strictEqual(t.set({source: 'w', signal: 'Adc', scale: 1, offset: 0, unit: ''}), true);
assert.strictEqual(t.list().length, 0);
// An invalid entry is refused and changes nothing.
assert.strictEqual(t.set({source: 'w', signal: 'Adc', scale: 0}), false);
assert.strictEqual(t.list().length, 0);
});
test('CalTable.replaceAll drops the previous contents', () => {
const t = new C.CalTable();
t.set({source: 'w', signal: 'Old', scale: 2});
t.replaceAll([
{source: 'w', signal: 'B', scale: 2},
{source: 'w', signal: 'A', scale: 3},
{source: 'w', signal: 'Bad', scale: 0},
{source: 'w', signal: 'Ident', scale: 1, offset: 0, unit: ''},
]);
assert.deepStrictEqual(t.list().map(e => e.signal), ['A', 'B']);
});
test('CalTable.list is sorted by source then signal', () => {
const t = new C.CalTable();
t.set({source: 'z', signal: 'a', scale: 2});
t.set({source: 'a', signal: 'z', scale: 2});
t.set({source: 'a', signal: 'b', scale: 2});
assert.deepStrictEqual(t.list().map(e => e.source + '/' + e.signal),
['a/b', 'a/z', 'z/a']);
});
Common/Client/c/.gitignore
+5
View File
@@ -0,0 +1,5 @@
*.o
*.a
udps_dump
.cxxcheck
.cxxcheck.cpp
Common/Client/c/Makefile
+46
View File
@@ -0,0 +1,46 @@
# UDPS C client library — standalone, no MARTe2, no external dependencies.
#
# make build libudpsclient.a and the example
# make example build only the example
# make cxxcheck verify the header is usable from C++
# make clean
CC ?= cc
CXX ?= c++
AR ?= ar
CFLAGS ?= -O2 -g
WARN = -Wall -Wextra -Wpedantic
STD = -std=c99
CPPFLAGS += -I.
# Old glibc (< 2.17) keeps clock_gettime in librt; harmless to add there.
LDLIBS ?=
LIB = libudpsclient.a
OBJ = udps_client.o
EXAMPLE = udps_dump
.PHONY: all example cxxcheck clean
all: $(LIB) $(EXAMPLE)
$(LIB): $(OBJ)
$(AR) rcs $@ $^
udps_client.o: udps_client.c udps_client.h
$(CC) $(STD) $(WARN) $(CFLAGS) $(CPPFLAGS) -c -o $@ $<
example: $(EXAMPLE)
$(EXAMPLE): example/udps_dump.c $(LIB)
$(CC) $(STD) $(WARN) $(CFLAGS) $(CPPFLAGS) -o $@ $< $(LIB) $(LDLIBS)
# The header is C++-safe; this target keeps it that way.
cxxcheck: udps_client.h
echo '#include "udps_client.h"' > .cxxcheck.cpp
echo 'int main() { udps_client_config_t c; udps_client_config_init(&c); return 0; }' >> .cxxcheck.cpp
$(CXX) -std=c++11 -Wall -Wextra $(CPPFLAGS) -o .cxxcheck .cxxcheck.cpp $(LIB) $(LDLIBS)
./.cxxcheck && rm -f .cxxcheck .cxxcheck.cpp
clean:
rm -f $(LIB) $(OBJ) $(EXAMPLE) .cxxcheck .cxxcheck.cpp
Common/Client/c/example/udps_dump.c
+260
View File
@@ -0,0 +1,260 @@
/**
* @file udps_dump.c
* @brief Example UDPS client: connects to a UDPStreamer and prints what arrives.
*
* Build with the Makefile in the parent directory, then for a unicast stream:
*
* ./udps_dump --host 127.0.0.1 --port 44500
*
* or, for a multicast one:
*
* ./udps_dump --host 127.0.0.1 --port 44500 \
* --multicast 239.0.0.1 --iface 127.0.0.1
*
* Ctrl-C prints a summary of what was received.
*/
#define _POSIX_C_SOURCE 200809L
#include "udps_client.h"
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
static volatile sig_atomic_t g_stop = 0;
static void on_sigint(int sig) {
(void)sig;
g_stop = 1;
}
typedef struct {
double print_interval; /**< Seconds between frame printouts. */
double last_print;
uint64_t frames;
uint64_t max_frames;
} dump_state_t;
static double now_wall(void) {
struct timespec ts;
clock_gettime(CLOCK_REALTIME, &ts);
return (double)ts.tv_sec + (double)ts.tv_nsec * 1e-9;
}
static const char *time_mode_name(uint8_t m) {
switch (m) {
case UDPS_TIME_PACKET: return "packet";
case UDPS_TIME_FULL_ARRAY: return "full-array";
case UDPS_TIME_FIRST_SAMPLE: return "first-sample";
case UDPS_TIME_LAST_SAMPLE: return "last-sample";
default: return "?";
}
}
static const char *publish_mode_name(uint8_t m) {
switch (m) {
case UDPS_PUBLISH_STRICT: return "strict";
case UDPS_PUBLISH_ACCUMULATE: return "accumulate";
case UDPS_PUBLISH_DECIMATE: return "decimate";
default: return "?";
}
}
static void on_config(const udps_signal_t *sigs, uint32_t n, uint8_t mode,
void *user) {
uint32_t i;
(void)user;
printf("\nCONFIG: %u signal(s), publish mode %s\n", (unsigned)n,
publish_mode_name(mode));
printf(" %-3s %-24s %-8s %-10s %-8s %-10s %s\n", "#", "name", "type",
"shape", "unit", "rate[Hz]", "time-mode");
for (i = 0u; i < n; i++) {
char shape[32];
const udps_signal_t *s = &sigs[i];
if (s->num_cols > 1u) {
snprintf(shape, sizeof shape, "%ux%u", (unsigned)s->num_rows,
(unsigned)s->num_cols);
} else {
snprintf(shape, sizeof shape, "%u",
(unsigned)udps_signal_num_elements(s));
}
printf(" %-3u %-24s %-8s %-10s %-8s %-10.6g %s%s\n", (unsigned)i,
s->name, udps_type_name(s->type_code), shape,
(s->unit[0] != '\0') ? s->unit : "-", s->sampling_rate,
time_mode_name(s->time_mode),
(s->quant_type != UDPS_QUANT_NONE) ? " (quantised)" : "");
}
fflush(stdout);
}
static void on_data(const udps_frame_t *f, void *user) {
dump_state_t *st = (dump_state_t *)user;
uint32_t i;
double now;
st->frames++;
now = now_wall();
if ((now - st->last_print) < st->print_interval) {
return; /* Streams run far faster than a terminal can be read. */
}
st->last_print = now;
printf("\nframe #%lu t=%.6f samples=%u (%lu frames so far)\n",
(unsigned long)f->counter, f->recv_time, (unsigned)f->num_samples,
(unsigned long)st->frames);
for (i = 0u; i < f->num_signals; i++) {
const double *v = f->values[i].values;
uint32_t cnt = f->values[i].count;
double lo, hi;
uint32_t k;
if (cnt == 0u) {
continue;
}
lo = hi = v[0];
for (k = 1u; k < cnt; k++) {
if (v[k] < lo) {
lo = v[k];
}
if (v[k] > hi) {
hi = v[k];
}
}
printf(" %-24s n=%-6u first=%-12.6g last=%-12.6g min=%-12.6g max=%-12.6g %s\n",
f->signals[i].name, (unsigned)cnt, v[0], v[cnt - 1u], lo, hi,
f->signals[i].unit);
}
fflush(stdout);
}
static void on_event(udps_event_t ev, const char *detail, void *user) {
(void)user;
switch (ev) {
case UDPS_EVENT_CONNECTED:
printf("[connected to %s]\n", detail ? detail : "");
break;
case UDPS_EVENT_DISCONNECTED:
printf("[disconnected: %s]\n", detail ? detail : "");
break;
case UDPS_EVENT_ERROR:
fprintf(stderr, "[error] %s\n", detail ? detail : "");
break;
default:
break;
}
fflush(stdout);
}
static void usage(const char *argv0) {
printf("Usage: %s --host ADDR --port N [options]\n"
"\n"
" --host ADDR server address (default 127.0.0.1)\n"
" --port N server UDP port, or TCP control port in multicast\n"
" mode (default 44500)\n"
" --multicast GROUP join GROUP for data instead of unicast\n"
" --iface ADDR local interface address for the multicast join\n"
" --data-port N multicast data port (default: --port + 1)\n"
" --silence SEC reconnect after SEC without data (default 1, 0 off)\n"
" --interval SEC seconds between printouts (default 1)\n"
" --frames N exit after N frames (default: run until Ctrl-C)\n"
" --help this text\n",
argv0);
}
int main(int argc, char **argv) {
udps_client_config_t cfg;
udps_client_t *cli;
dump_state_t st;
udps_stats_t stats;
struct sigaction sa;
const char *host = "127.0.0.1";
int i;
udps_client_config_init(&cfg);
cfg.server_port = 44500u;
memset(&st, 0, sizeof st);
st.print_interval = 1.0;
for (i = 1; i < argc; i++) {
const char *a = argv[i];
const char *next = (i + 1 < argc) ? argv[i + 1] : NULL;
if (strcmp(a, "--help") == 0) {
usage(argv[0]);
return 0;
}
if (next == NULL) {
fprintf(stderr, "missing value for %s\n", a);
return 2;
}
if (strcmp(a, "--host") == 0) {
host = next;
} else if (strcmp(a, "--port") == 0) {
cfg.server_port = (uint16_t)atoi(next);
} else if (strcmp(a, "--multicast") == 0) {
cfg.multicast_group = next;
} else if (strcmp(a, "--iface") == 0) {
cfg.interface_addr = next;
} else if (strcmp(a, "--data-port") == 0) {
cfg.data_port = (uint16_t)atoi(next);
} else if (strcmp(a, "--silence") == 0) {
cfg.silence_timeout_s = atof(next);
} else if (strcmp(a, "--interval") == 0) {
st.print_interval = atof(next);
} else if (strcmp(a, "--frames") == 0) {
st.max_frames = (uint64_t)strtoull(next, NULL, 10);
} else {
fprintf(stderr, "unknown option %s\n", a);
usage(argv[0]);
return 2;
}
i++;
}
cfg.server_addr = host;
cli = udps_client_create(&cfg);
if (cli == NULL) {
fprintf(stderr, "could not create client for %s:%u\n", host,
(unsigned)cfg.server_port);
return 1;
}
udps_client_set_callbacks(cli, on_config, on_data, on_event, &st);
memset(&sa, 0, sizeof sa);
sa.sa_handler = on_sigint;
(void)sigaction(SIGINT, &sa, NULL);
(void)sigaction(SIGTERM, &sa, NULL);
printf("listening to %s:%u%s%s ... (Ctrl-C to stop)\n", host,
(unsigned)cfg.server_port,
cfg.multicast_group ? " via multicast " : "",
cfg.multicast_group ? cfg.multicast_group : "");
while (!g_stop && (st.max_frames == 0u || st.frames < st.max_frames)) {
/* All the work — connecting, receiving, decoding, reconnecting — and
* every callback happens inside this call. */
(void)udps_client_poll(cli, 200);
}
udps_client_stats(cli, &stats);
printf("\n--- summary ---\n"
"packets %lu\n"
"bytes %.1f MiB\n"
"frames %lu\n"
"configs %lu\n"
"gaps %lu (datagrams lost)\n"
"dropped %lu (fragments)\n"
"reconnects %lu\n",
(unsigned long)stats.packets_received,
(double)stats.bytes_received / (1024.0 * 1024.0),
(unsigned long)stats.frames_delivered,
(unsigned long)stats.config_updates,
(unsigned long)stats.counter_gaps,
(unsigned long)stats.fragments_dropped,
(unsigned long)stats.reconnects);
udps_client_destroy(cli);
return 0;
}
Common/Client/c/udps_client.c
+1230
View File
File diff suppressed because it is too large Load Diff
Common/Client/c/udps_client.h
+340
View File
@@ -0,0 +1,340 @@
#ifndef UDPS_CLIENT_H
#define UDPS_CLIENT_H
/**
* @file udps_client.h
* @brief Standalone UDPS (UDPStreamer) receiver library — C99, no MARTe2.
*
* Depends only on libc and BSD sockets, so it can be dropped into any C or C++
* program that needs to consume a UDPStreamer / DebugService stream. The wire
* format is specified in Docs/Protocol.md; the library reference (and a worked
* example) is Docs/UDPS-C-Client.md.
*
* Usage in one paragraph: fill a udps_client_config_t, create a client, install
* callbacks, then call udps_client_poll() in a loop. The client owns the
* connection state machine — it sends CONNECT, reassembles fragmented packets,
* decodes CONFIG and DATA, sends keepalives, and reconnects when the server
* goes silent. Nothing is done behind your back: no threads are created and
* every callback runs inside your call to udps_client_poll().
*
* Threading: a udps_client_t must be used from one thread at a time.
*/
#include <stddef.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
/*---------------------------------------------------------------------------*/
/* Protocol constants */
/*---------------------------------------------------------------------------*/
/** Magic number: ASCII 'UDPS' stored little-endian. */
#define UDPS_MAGIC 0x53504455u
/** Size of the packed packet header on the wire. */
#define UDPS_HEADER_SIZE 17u
/** Size of one serialised signal descriptor in a CONFIG payload. */
#define UDPS_SIGNAL_DESC_SIZE 136u
/** Value of udps_signal_t::time_signal_idx when the signal has no time reference. */
#define UDPS_NO_TIME_SIGNAL 0xFFFFFFFFu
/** Upper bound on elements per signal; larger descriptors are rejected. */
#define UDPS_MAX_ELEMENTS (1u << 20)
/** Packet types (udps_header_t::type). */
enum {
UDPS_PKT_DATA = 0, /**< Server -> client: signal samples. */
UDPS_PKT_CONFIG = 1, /**< Server -> client: signal metadata. */
UDPS_PKT_ACK = 2, /**< Client -> server: keepalive. */
UDPS_PKT_CONNECT = 3, /**< Client -> server: open a session. */
UDPS_PKT_DISCONNECT = 4 /**< Either direction: close a session. */
};
/** Sample type codes (udps_signal_t::type_code). */
enum {
UDPS_T_UINT8 = 0,
UDPS_T_INT8 = 1,
UDPS_T_UINT16 = 2,
UDPS_T_INT16 = 3,
UDPS_T_UINT32 = 4,
UDPS_T_INT32 = 5,
UDPS_T_UINT64 = 6,
UDPS_T_INT64 = 7,
UDPS_T_FLOAT32 = 8,
UDPS_T_FLOAT64 = 9,
UDPS_T_UNKNOWN = 255
};
/** Quantisation codes (udps_signal_t::quant_type). */
enum {
UDPS_QUANT_NONE = 0, /**< Raw values in the signal's own type. */
UDPS_QUANT_UINT8 = 1, /**< [range_min, range_max] mapped onto uint8. */
UDPS_QUANT_INT8 = 2,
UDPS_QUANT_UINT16 = 3,
UDPS_QUANT_INT16 = 4
};
/** Time-reference modes (udps_signal_t::time_mode). */
enum {
UDPS_TIME_PACKET = 0, /**< No per-element time; use packet arrival. */
UDPS_TIME_FULL_ARRAY = 1, /**< The time signal carries one stamp per element. */
UDPS_TIME_FIRST_SAMPLE = 2, /**< Time signal (scalar) stamps element 0. */
UDPS_TIME_LAST_SAMPLE = 3 /**< Time signal (scalar) stamps element N-1. */
};
/** Publishing modes (udps_frame_t::publish_mode). */
enum {
UDPS_PUBLISH_STRICT = 0, /**< One packet per RT cycle. */
UDPS_PUBLISH_ACCUMULATE = 1, /**< A batch of cycles per packet. */
UDPS_PUBLISH_DECIMATE = 2 /**< One packet every N cycles. */
};
/*---------------------------------------------------------------------------*/
/* Data model */
/*---------------------------------------------------------------------------*/
/** Decoded 17-byte packet header. */
typedef struct {
uint32_t magic;
uint8_t type;
uint32_t counter; /**< Same for every fragment of one update. */
uint16_t fragment_idx;
uint16_t total_fragments; /**< 1 when the update fits in one datagram. */
uint32_t payload_bytes;
} udps_header_t;
/** Metadata for one streamed signal, as carried by the CONFIG payload. */
typedef struct {
char name[65]; /**< NUL-terminated. */
uint8_t type_code; /**< UDPS_T_*. */
uint8_t quant_type; /**< UDPS_QUANT_*. */
uint8_t num_dimensions; /**< 0 scalar, 1 vector, 2 matrix. */
uint32_t num_rows;
uint32_t num_cols;
double range_min; /**< Physical range, used to dequantise. */
double range_max;
uint8_t time_mode; /**< UDPS_TIME_*. */
double sampling_rate; /**< Hz; 0 when unknown. */
uint32_t time_signal_idx;/**< Index into the signal list, or UDPS_NO_TIME_SIGNAL. */
char unit[33]; /**< NUL-terminated. */
} udps_signal_t;
/**
* @brief Decoded values of one signal within a frame.
*
* Values are always physical doubles: quantised signals are already expanded
* back onto [range_min, range_max]. @c count is @c num_samples for a scalar
* signal in Accumulate mode (one value per batched cycle) and the signal's
* element count in every other case.
*/
typedef struct {
const double *values;
uint32_t count;
} udps_signal_values_t;
/** One fully decoded DATA packet. */
typedef struct {
uint32_t counter; /**< Packet counter; gaps mean lost datagrams. */
/**
* DATA packets missing immediately before this one, from the counter.
*
* Needed to space samples correctly: the elapsed time since the previous
* frame covers the lost packets' cycles too, so dividing it by this
* frame's sample count alone gives a period too long by exactly
* @c lost + 1, which walks the samples past their own end and into the
* range the next frame claims.
*/
uint32_t lost;
uint64_t hrt; /**< Producer's high-resolution timer at send. */
double recv_time; /**< Wall-clock seconds (CLOCK_REALTIME) at arrival. */
uint8_t publish_mode; /**< UDPS_PUBLISH_*. */
uint32_t num_samples; /**< Batched cycles; 1 unless Accumulate. */
uint32_t num_signals;
const udps_signal_t *signals; /**< num_signals entries, CONFIG order. */
const udps_signal_values_t *values; /**< num_signals entries, same order. */
} udps_frame_t;
/** Connection lifecycle events reported through udps_event_cb. */
typedef enum {
UDPS_EVENT_CONNECTED, /**< Sockets are up and CONNECT was sent. */
UDPS_EVENT_DISCONNECTED, /**< Session dropped; the client will retry. */
UDPS_EVENT_ERROR /**< Recoverable problem; detail says what. */
} udps_event_t;
/** Cumulative counters, never reset. */
typedef struct {
uint64_t packets_received; /**< Datagrams (and TCP frames) accepted. */
uint64_t bytes_received;
uint64_t frames_delivered; /**< DATA packets decoded and handed to you. */
uint64_t config_updates;
uint64_t fragments_dropped; /**< Duplicate, stale or unplaceable fragments. */
uint64_t counter_gaps; /**< DATA packets missing from the sequence. */
/**
* DATA packets dropped for not advancing the counter: reordered or
* duplicated on the wire. Delivering one would stamp its values with a
* time base older than data already handed over.
*/
uint64_t stale_packets;
uint64_t reconnects;
} udps_stats_t;
/*---------------------------------------------------------------------------*/
/* Client */
/*---------------------------------------------------------------------------*/
typedef struct udps_client udps_client_t;
/** Called whenever a CONFIG packet redefines the signal set. */
typedef void (*udps_config_cb)(const udps_signal_t *signals,
uint32_t num_signals,
uint8_t publish_mode,
void *user);
/**
* @brief Called for every decoded DATA packet.
*
* The frame and everything it points at are owned by the client and are only
* valid until the callback returns — copy anything you need to keep.
*/
typedef void (*udps_data_cb)(const udps_frame_t *frame, void *user);
/** Called on connection state changes and on recoverable errors. */
typedef void (*udps_event_cb)(udps_event_t event, const char *detail, void *user);
/**
* @brief Transport configuration.
*
* Zero-initialise with udps_client_config_init(), then override what you need.
* Set @c multicast_group to switch from unicast to multicast: in unicast the
* client sends CONNECT over UDP and receives everything on its ephemeral port;
* in multicast it joins the group for DATA and opens a TCP control connection
* to @c server_port for CONNECT and CONFIG.
*/
typedef struct {
const char *server_addr; /**< IPv4 dotted quad. Required. */
uint16_t server_port; /**< UDP port (unicast) or TCP port (multicast). Required. */
const char *multicast_group;/**< IPv4 group; NULL selects unicast. */
const char *interface_addr; /**< Local IPv4 of the interface to join on. NULL = default route. */
uint16_t data_port; /**< Multicast data port; 0 means server_port + 1. */
double silence_timeout_s; /**< Reconnect after this long without data. 0 disables. */
double reconnect_delay_s; /**< Wait between reconnect attempts. */
double keepalive_interval_s;/**< Unicast ACK period. 0 disables. */
uint32_t recv_buffer_bytes; /**< SO_RCVBUF; large bursts need a large value. */
uint32_t max_packet_bytes; /**< Ceiling on one reassembled payload. */
} udps_client_config_t;
/** Fills @p cfg with the defaults documented in Docs/UDPS-C-Client.md. */
void udps_client_config_init(udps_client_config_t *cfg);
/**
* @brief Creates a client. No socket is opened until the first poll.
* @return NULL if @p cfg is invalid or memory ran out.
*/
udps_client_t *udps_client_create(const udps_client_config_t *cfg);
/** Closes the session (sending DISCONNECT if connected) and frees the client. */
void udps_client_destroy(udps_client_t *client);
/** Installs the callbacks. Any of them may be NULL. */
void udps_client_set_callbacks(udps_client_t *client,
udps_config_cb on_config,
udps_data_cb on_data,
udps_event_cb on_event,
void *user);
/**
* @brief Drives the client: connects if needed, then waits for and processes
* packets for at most @p timeout_ms milliseconds.
*
* Callbacks fire from inside this call. A negative @p timeout_ms blocks until
* something happens. Call it in a loop; it is the only function that does work.
*
* @return the number of packets processed (0 on timeout), or -1 if the session
* broke. -1 is not fatal: the next call retries after reconnect_delay_s.
*/
int udps_client_poll(udps_client_t *client, int timeout_ms);
/** Non-zero once the sockets are up (which does not yet imply CONFIG arrived). */
int udps_client_is_connected(const udps_client_t *client);
/**
* @brief The current signal set, or NULL before the first CONFIG.
* @param num_signals Out; may be NULL.
*/
const udps_signal_t *udps_client_signals(const udps_client_t *client,
uint32_t *num_signals);
/** The publishing mode from the last CONFIG (UDPS_PUBLISH_*). */
uint8_t udps_client_publish_mode(const udps_client_t *client);
/** Copies the counters into @p out. */
void udps_client_stats(const udps_client_t *client, udps_stats_t *out);
/** Human-readable description of the last failure. Never NULL. */
const char *udps_client_last_error(const udps_client_t *client);
/*---------------------------------------------------------------------------*/
/* Stateless helpers */
/*---------------------------------------------------------------------------*/
/** Elements in one sample of @p signal (rows x cols, at least 1). */
uint32_t udps_signal_num_elements(const udps_signal_t *signal);
/** Short name of a type code, e.g. "float32". Never NULL. */
const char *udps_type_name(uint8_t type_code);
/**
* @brief Decodes a packet header.
* @return 0 on success, -1 if @p len is too small or the magic is wrong.
*/
int udps_parse_header(const void *buf, size_t len, udps_header_t *out);
/**
* @brief Decodes a reassembled CONFIG payload.
* @param signals Out array of at most @p max_signals entries.
* @param num_signals Out; the number actually written.
* @param publish_mode Out; may be NULL.
* @return 0 on success, -1 if the payload is malformed or does not fit.
*/
int udps_parse_config(const void *payload,
size_t len,
udps_signal_t *signals,
uint32_t max_signals,
uint32_t *num_signals,
uint8_t *publish_mode);
/**
* @brief One value out of a frame.
* @param sample Accumulate batch slot; ignored for non-scalar signals.
* @param elem Element within the sample; ignored for accumulated scalars.
* @return the value, or 0.0 if any index is out of range.
*/
double udps_frame_value(const udps_frame_t *frame,
uint32_t signal_idx,
uint32_t sample,
uint32_t elem);
/**
* @brief Arrival-anchored estimate of the wall-clock time of one element.
*
* Exact only for streams that declare a sampling rate: the packet is assumed to
* have arrived as its last element was produced, and earlier elements are dated
* backwards by 1/sampling_rate. Signals with UDPS_TIME_PACKET, or without a
* sampling rate, all report the arrival time. When the stream carries a time
* signal (time_signal_idx != UDPS_NO_TIME_SIGNAL) that signal is the accurate
* source — read it like any other signal instead of using this helper.
*/
double udps_frame_element_time(const udps_frame_t *frame,
uint32_t signal_idx,
uint32_t elem);
#ifdef __cplusplus
} /* extern "C" */
#endif
#endif /* UDPS_CLIENT_H */
Common/Client/go/go.mod
+14 -2
View File
@@ -1,7 +1,19 @@
module marte2/common module marte2/common
go 1.21 go 1.24.9
require github.com/gorilla/websocket v1.5.1 require github.com/gorilla/websocket v1.5.1
require golang.org/x/net v0.17.0 // indirect require (
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/klauspost/compress v1.17.9 // indirect
github.com/parquet-go/bitpack v1.0.0 // indirect
github.com/parquet-go/jsonlite v1.0.0 // indirect
github.com/parquet-go/parquet-go v0.32.0 // indirect
github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/twpayne/go-geom v1.6.1 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/sys v0.38.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
)
Common/Client/go/go.sum
+21
View File
@@ -1,4 +1,25 @@
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY= github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY=
github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY= github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY=
github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA=
github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw=
github.com/parquet-go/bitpack v1.0.0 h1:AUqzlKzPPXf2bCdjfj4sTeacrUwsT7NlcYDMUQxPcQA=
github.com/parquet-go/bitpack v1.0.0/go.mod h1:XnVk9TH+O40eOOmvpAVZ7K2ocQFrQwysLMnc6M/8lgs=
github.com/parquet-go/jsonlite v1.0.0 h1:87QNdi56wOfsE5bdgas0vRzHPxfJgzrXGml1zZdd7VU=
github.com/parquet-go/jsonlite v1.0.0/go.mod h1:nDjpkpL4EOtqs6NQugUsi0Rleq9sW/OtC1NnZEnxzF0=
github.com/parquet-go/parquet-go v0.32.0 h1:NWDqTUHfrCS4cJP/Fj2HlxvqsrVedWG3sayMkf+znzM=
github.com/parquet-go/parquet-go v0.32.0/go.mod h1:navtkAYr2LGoJVp141oXPlO/sxLvaOe3la2JEoD8+rg=
github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ=
github.com/pierrec/lz4/v4 v4.1.21/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/twpayne/go-geom v1.6.1 h1:iLE+Opv0Ihm/ABIcvQFGIiFBXd76oBIar9drAwHFhR4=
github.com/twpayne/go-geom v1.6.1/go.mod h1:Kr+Nly6BswFsKM5sd31YaoWS5PeDDH2NftJTK7Gd028=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM= golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE= golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
Common/Client/go/udpsprotocol/accumulate_array_test.go
+146
View File
@@ -0,0 +1,146 @@
package udpsprotocol
// Accumulate mode ships one full snapshot of EVERY signal per RT cycle —
// arrays included. See UDPStreamer.cpp pass 5 ("ALL signals (scalars and
// arrays alike) are tagged accumulated = true") and SerializeAccumulated,
// which writes, for each signal in CONFIG order, numSamples consecutive
// snapshots of that signal's full element set.
//
// The tests below build a payload byte-for-byte the way the C++ producer
// does, so a decoding regression shows up here rather than as a mangled
// waveform three components downstream.
import (
"encoding/binary"
"math"
"testing"
"time"
)
// buildAccumulatePayload lays out an Accumulate DATA payload exactly as
// UDPStreamer::SerializeAccumulated does:
//
// [8 HRT][4 numSamples] then, per signal, numSamples × NumElements float64.
//
// slots[i][k] holds signal i's element set for cycle k.
func buildAccumulatePayload(hrt uint64, slots [][][]float64) []byte {
numSamples := 0
if len(slots) > 0 {
numSamples = len(slots[0])
}
out := make([]byte, 12)
binary.LittleEndian.PutUint64(out[0:8], hrt)
binary.LittleEndian.PutUint32(out[8:12], uint32(numSamples))
for _, sig := range slots {
for _, elems := range sig {
for _, v := range elems {
var b [8]byte
binary.LittleEndian.PutUint64(b[:], math.Float64bits(v))
out = append(out, b[:]...)
}
}
}
return out
}
// TestParseDataAccumulateGivesEachSlotItsOwnArray pins the array case: with an
// accumulated batch, slot k's array signal must decode to the values the
// producer captured on cycle k, not to some other cycle's. Handing every slot
// slot 0's array would stamp one cycle's data with every slot's timestamp —
// the same samples drawn repeatedly at advancing times, with the cycles they
// displaced missing entirely.
func TestParseDataAccumulateGivesEachSlotItsOwnArray(t *testing.T) {
sigs := []SignalInfo{
{Name: "Time", TypeCode: 9, NumRows: 1, NumCols: 1, QuantType: QuantNone},
{Name: "Wave", TypeCode: 9, NumRows: 4, NumCols: 1, QuantType: QuantNone},
}
// Three RT cycles. "Wave" carries a different ramp each cycle so a
// mix-up is unambiguous.
timeSlots := [][]float64{{10}, {20}, {30}}
waveSlots := [][]float64{
{1, 2, 3, 4},
{5, 6, 7, 8},
{9, 10, 11, 12},
}
payload := buildAccumulatePayload(777, [][][]float64{timeSlots, waveSlots})
samples, err := ParseData(payload, sigs, PublishModeAccumulate, time.Now())
if err != nil {
t.Fatalf("ParseData: %v", err)
}
if len(samples) != 3 {
t.Fatalf("expected 3 slots, got %d", len(samples))
}
for k, s := range samples {
got := s.Values["Wave"]
want := waveSlots[k]
if len(got) != len(want) {
t.Fatalf("slot %d: Wave has %d elements, want %d", k, len(got), len(want))
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("slot %d: Wave = %v, want %v (slot %d's data has been "+
"served for this slot's timestamp)", k, got, want,
indexOfSlot(waveSlots, got))
}
}
if tv := s.Values["Time"]; len(tv) != 1 || tv[0] != timeSlots[k][0] {
t.Fatalf("slot %d: Time = %v, want %v", k, tv, timeSlots[k])
}
}
}
// TestParseDataAccumulateConsumesTheWholeArrayBlock catches the same defect
// from the other side: a signal following an array must be read at the right
// offset. Under-reading the array block slides every later signal backwards
// into the array's tail, which decodes as plausible-looking but wrong values
// rather than as an error.
func TestParseDataAccumulateConsumesTheWholeArrayBlock(t *testing.T) {
sigs := []SignalInfo{
{Name: "Wave", TypeCode: 9, NumRows: 4, NumCols: 1, QuantType: QuantNone},
{Name: "Tail", TypeCode: 9, NumRows: 1, NumCols: 1, QuantType: QuantNone},
}
waveSlots := [][]float64{
{1, 2, 3, 4},
{5, 6, 7, 8},
{9, 10, 11, 12},
}
tailSlots := [][]float64{{100}, {200}, {300}}
payload := buildAccumulatePayload(0, [][][]float64{waveSlots, tailSlots})
samples, err := ParseData(payload, sigs, PublishModeAccumulate, time.Now())
if err != nil {
t.Fatalf("ParseData: %v", err)
}
if len(samples) != 3 {
t.Fatalf("expected 3 slots, got %d", len(samples))
}
for k, s := range samples {
tv := s.Values["Tail"]
if len(tv) != 1 || tv[0] != tailSlots[k][0] {
t.Fatalf("slot %d: Tail = %v, want %v — the array block before it "+
"was not fully consumed", k, tv, tailSlots[k])
}
}
}
// indexOfSlot reports which slot's data a decoded array actually matches, so a
// failure message can name the culprit instead of just showing numbers.
func indexOfSlot(slots [][]float64, got []float64) int {
for k, want := range slots {
if len(want) != len(got) {
continue
}
same := true
for i := range want {
if want[i] != got[i] {
same = false
break
}
}
if same {
return k
}
}
return -1
}
Common/Client/go/udpsprotocol/protocol.go
+38 -33
View File
@@ -290,28 +290,37 @@ type DataSample struct {
HRTTimestamp uint64 HRTTimestamp uint64
WallTime time.Time // wall-clock time at UDP arrival; used as x-axis WallTime time.Time // wall-clock time at UDP arrival; used as x-axis
Values map[string][]float64 // key = signal name, value = []float64 with NumElements entries Values map[string][]float64 // key = signal name, value = []float64 with NumElements entries
// Lost is the number of DATA packets missing between the previous sample
// and this one, taken from the producer's packet counter (see
// SequenceGate). Consumers that derive a per-element period from the
// inter-packet gap need it: the gap widens with every lost packet, and
// dividing it by this packet's element count alone reports a period too
// long by exactly that factor — which walks the packet's elements past
// their own end and into the range the next packet claims.
Lost uint32
} }
// parseElems reads n elements for sig from payload at offset, advancing offset. // parseElems reads n elements for sig from payload at offset, advancing offset.
// Returns the slice of float64 values and the new offset. // Returns the slice of float64 values and the new offset.
func parseElems(payload []byte, offset, n int, sig SignalInfo) ([]float64, int, error) { func parseElems(payload []byte, offset, n int, sig SignalInfo) ([]float64, int, error) {
sz := rawTypeSize(sig.TypeCode)
if sig.QuantType != QuantNone {
sz = quantSize(sig.QuantType)
}
// Bounds-check before allocating. In Accumulate mode n is numSamples ×
// NumElements, so a malformed packet could otherwise ask for an allocation
// far larger than its own payload could ever justify.
if n < 0 || n > (len(payload)-offset)/sz {
return nil, offset, fmt.Errorf("data payload truncated for signal %q", sig.Name)
}
elems := make([]float64, n) elems := make([]float64, n)
needed := n * sz
if sig.QuantType == QuantNone { if sig.QuantType == QuantNone {
sz := rawTypeSize(sig.TypeCode)
needed := n * sz
if offset+needed > len(payload) {
return nil, offset, fmt.Errorf("data payload truncated for signal %q", sig.Name)
}
for i := 0; i < n; i++ { for i := 0; i < n; i++ {
elems[i] = readRawElement(payload, offset+i*sz, sig.TypeCode) elems[i] = readRawElement(payload, offset+i*sz, sig.TypeCode)
} }
offset += needed offset += needed
} else { } else {
sz := quantSize(sig.QuantType)
needed := n * sz
if offset+needed > len(payload) {
return nil, offset, fmt.Errorf("data payload truncated (quant) for signal %q", sig.Name)
}
for i := 0; i < n; i++ { for i := 0; i < n; i++ {
var raw uint16 var raw uint16
if sz == 1 { if sz == 1 {
@@ -331,7 +340,13 @@ func parseElems(payload []byte, offset, n int, sig SignalInfo) ([]float64, int,
// //
// For PublishModeAccumulate the payload format is: // For PublishModeAccumulate the payload format is:
// //
// [8 HRT][4 numSamples][for each signal: accumulated scalars → numSamples elems; arrays → NumElements elems] // [8 HRT][4 numSamples][for each signal: numSamples × NumElements elems]
//
// Every signal is accumulated, arrays included: the producer captures one full
// snapshot of the whole signal set per RT cycle and lays the cycles out
// contiguously per signal (UDPStreamer::SerializeAccumulated). Reading only
// NumElements for an array would hand every slot the first cycle's data and
// slide all later signals into that array's tail.
// //
// The function returns one DataSample per accumulated snapshot so the hub can // The function returns one DataSample per accumulated snapshot so the hub can
// process each slot independently with its own timestamp. // process each slot independently with its own timestamp.
@@ -357,28 +372,18 @@ func ParseData(payload []byte, sigs []SignalInfo, publishMode uint8, arrivalTime
} }
// Parse per-signal data blocks (all slots for a signal are contiguous). // Parse per-signal data blocks (all slots for a signal are contiguous).
accumVals := make(map[string][]float64, len(sigs)) // scalars: numSamples values accumVals := make(map[string][]float64, len(sigs)) // numSamples × NumElements
fixedVals := make(map[string][]float64, len(sigs)) // arrays: NumElements values accumElems := make(map[string]int, len(sigs))
for _, sig := range sigs { for _, sig := range sigs {
n := sig.NumElements() n := sig.NumElements()
if n == 1 { elems, newOff, err := parseElems(payload, offset, numSamples*n, sig)
// Accumulated scalar: read numSamples back-to-back elements. if err != nil {
elems, newOff, err := parseElems(payload, offset, numSamples, sig) return nil, err
if err != nil {
return nil, err
}
offset = newOff
accumVals[sig.Name] = elems
} else {
// Fixed array (non-accumulated): one set of NumElements values.
elems, newOff, err := parseElems(payload, offset, n, sig)
if err != nil {
return nil, err
}
offset = newOff
fixedVals[sig.Name] = elems
} }
offset = newOff
accumVals[sig.Name] = elems
accumElems[sig.Name] = n
} }
// Build one DataSample per slot. // Build one DataSample per slot.
@@ -386,10 +391,10 @@ func ParseData(payload []byte, sigs []SignalInfo, publishMode uint8, arrivalTime
for k := 0; k < numSamples; k++ { for k := 0; k < numSamples; k++ {
vals := make(map[string][]float64, len(sigs)) vals := make(map[string][]float64, len(sigs))
for sigName, av := range accumVals { for sigName, av := range accumVals {
vals[sigName] = []float64{av[k]} n := accumElems[sigName]
} // Sub-slice of the decoded block; the hub treats values as
for sigName, fv := range fixedVals { // read-only, so no copy is needed.
vals[sigName] = fv // shared read-only reference; hub does not modify vals[sigName] = av[k*n : (k+1)*n : (k+1)*n]
} }
samples[k] = DataSample{HRTTimestamp: hrt, WallTime: arrivalTime, Values: vals} samples[k] = DataSample{HRTTimestamp: hrt, WallTime: arrivalTime, Values: vals}
} }
Common/Client/go/udpsprotocol/sequence.go
+49
View File
@@ -0,0 +1,49 @@
package udpsprotocol
// SequenceGate orders DATA packets by the producer's packet counter.
//
// Reassembly completes in arrival order, not counter order, so a packet that
// was delayed or duplicated on the wire is handed up after a newer one has
// already been consumed. Its samples then carry an older time base than the
// data already in the ring: they land on top of samples that are already
// there, and the span they should have filled stays empty. That is a hole on
// one side and a collision on the other, from a packet that is entirely
// well-formed — the counter is the only thing that distinguishes it.
//
// A SequenceGate is not safe for concurrent use; each receive loop owns one.
type SequenceGate struct {
last uint32
valid bool
// Stale counts packets rejected for not advancing the counter (reordered
// or duplicated), for diagnostics.
Stale uint64
}
// Reset forgets the sequence. Call it on (re)connect: the producer's counter
// restarts independently of ours, so a counter carried over from the previous
// connection would reject the whole new stream.
func (g *SequenceGate) Reset() {
g.last = 0
g.valid = false
}
// Accept reports whether a DATA packet with this counter should be delivered,
// and how many packets went missing immediately before it.
//
// The counter is a wrapping uint32, so ordering is done on the signed
// difference: a plain comparison would call the first packet after the wrap
// stale and reject everything from then on.
func (g *SequenceGate) Accept(counter uint32) (ok bool, lost uint32) {
if !g.valid {
g.valid = true
g.last = counter
return true, 0
}
delta := int32(counter - g.last)
if delta <= 0 {
g.Stale++
return false, 0
}
g.last = counter
return true, uint32(delta) - 1
}
Common/Client/go/udpsprotocol/sequence_test.go
+82
View File
@@ -0,0 +1,82 @@
package udpsprotocol
import "testing"
// A packet older than one already delivered carries an older time base. Its
// samples land on top of data that is already in the ring and leave the span
// they should have filled empty, so it must not get through.
func TestSequenceGateRejectsStaleAndDuplicate(t *testing.T) {
var g SequenceGate
if ok, lost := g.Accept(10); !ok || lost != 0 {
t.Fatalf("first packet: got (%v, %d), want (true, 0)", ok, lost)
}
if ok, _ := g.Accept(11); !ok {
t.Fatal("counter 11 advances past 10 and must be accepted")
}
if ok, _ := g.Accept(9); ok {
t.Error("counter 9 is older than the delivered 11 and must be dropped")
}
if ok, _ := g.Accept(11); ok {
t.Error("a repeat of the delivered counter must be dropped")
}
if g.Stale != 2 {
t.Errorf("Stale = %d, want 2", g.Stale)
}
// The rejections must not have moved the sequence on.
if ok, lost := g.Accept(12); !ok || lost != 0 {
t.Errorf("after rejections: got (%v, %d), want (true, 0)", ok, lost)
}
}
// The loss count is what lets a consumer tell a widened gap from a slowed
// producer, so it must exclude the packet being delivered and must not persist
// into the next one.
func TestSequenceGateReportsLoss(t *testing.T) {
var g SequenceGate
g.Accept(100)
if _, lost := g.Accept(104); lost != 3 {
t.Errorf("101..103 missing: lost = %d, want 3", lost)
}
if _, lost := g.Accept(105); lost != 0 {
t.Errorf("consecutive packet: lost = %d, want 0", lost)
}
if g.Stale != 0 {
t.Errorf("Stale = %d, want 0", g.Stale)
}
}
// The counter is a wrapping uint32. Ordering it by plain comparison would call
// every packet after the wrap older than 0xFFFFFFFF and kill the stream.
func TestSequenceGateSurvivesWraparound(t *testing.T) {
var g SequenceGate
for _, c := range []uint32{0xFFFFFFFD, 0xFFFFFFFE, 0xFFFFFFFF, 0, 1, 2} {
ok, lost := g.Accept(c)
if !ok {
t.Fatalf("counter %#x rejected across the wrap", c)
}
if lost != 0 {
t.Errorf("counter %#x: lost = %d, want 0", c, lost)
}
}
// Loss must still be measured correctly across the wrap.
var h SequenceGate
h.Accept(0xFFFFFFFE)
if _, lost := h.Accept(1); lost != 2 {
t.Errorf("0xFFFFFFFF and 0 missing: lost = %d, want 2", lost)
}
}
// A reconnect restarts the producer's counter independently of ours; a carried
// over counter would reject the entire new stream.
func TestSequenceGateResetAcceptsLowerCounter(t *testing.T) {
var g SequenceGate
g.Accept(5000)
g.Reset()
if ok, lost := g.Accept(3); !ok || lost != 0 {
t.Errorf("after Reset: got (%v, %d), want (true, 0)", ok, lost)
}
}
Common/Client/go/wshub/calibration.go
+234
View File
@@ -0,0 +1,234 @@
package wshub
import (
"encoding/json"
"log"
"math"
"regexp"
"sort"
"strings"
"sync"
"unicode/utf8"
)
// arrayIndexSuffix matches a trailing "[digits]" at the very end of a signal
// name, used to strip array-element suffixes so one entry covers the whole
// array. The regexp is anchored to the end of the string and requires digits,
// so it only removes a well-formed trailing element index: "Adc[3]" → "Adc",
// "[0]" → "", "A[1]B" → "A[1]B" (no match).
//
// Known difference vs C++: the C++ hub uses strchr(signal,'[') which finds the
// FIRST '[' anywhere in the name, so C++ reduces "A[1]B" to "A" while this
// regexp leaves it unchanged. Both implementations agree on the common cases
// ("Name[i]" and "[i]" alone) that arise from real UDPS signal names.
var arrayIndexSuffix = regexp.MustCompile(`\[\d+\]$`)
// maxUnitLen bounds the calibration unit override. Mirrored by kMaxUnitLen in
// the C++ StreamHub and MAX_UNIT_LEN in the SPA's calibration.js.
const maxUnitLen = 16
// CalConfig is one per-signal affine calibration: y = raw*Scale + Offset.
//
// The key is (Source label, base Signal name). It is deliberately the source
// *label* and not the runtime id ("s1", "s2"): ids are assigned in add-order at
// startup, so a calibration keyed by id would rebind to a different source
// whenever the source list order changed.
type CalConfig struct {
Source string `json:"source"`
Signal string `json:"signal"`
Scale float64 `json:"scale"`
Offset float64 `json:"offset"`
Unit string `json:"unit,omitempty"`
}
// calKey builds the calTable map key. NUL cannot occur in either component,
// so the concatenation is unambiguous.
func calKey(source, signal string) string { return source + "\x00" + signal }
// Normalise trims and validates the entry in place, reporting whether it is
// usable. A zero or non-finite Scale is rejected because it makes the
// calibration non-invertible, which the trigger threshold path depends on.
func (c *CalConfig) Normalise() bool {
c.Source = strings.TrimSpace(c.Source)
c.Signal = strings.TrimSpace(c.Signal)
// Strip a trailing "[digits]" suffix so one entry covers an entire array
// signal. "Adc[3]" → "Adc". Must run before the empty check below so
// that "[0]" → "" → rejected, matching C++ and JS behaviour.
c.Signal = arrayIndexSuffix.ReplaceAllString(c.Signal, "")
if c.Source == "" || c.Signal == "" {
return false
}
if math.IsNaN(c.Scale) || math.IsInf(c.Scale, 0) || c.Scale == 0 {
return false
}
if math.IsNaN(c.Offset) || math.IsInf(c.Offset, 0) {
return false
}
c.Unit = strings.TrimSpace(c.Unit)
if len(c.Unit) > maxUnitLen {
c.Unit = c.Unit[:maxUnitLen]
// The byte cut may land mid-rune. Drop any trailing partial rune so
// the result is always valid UTF-8; json.Marshal would otherwise emit
// replacement characters and break the save→load round-trip.
for {
r, size := utf8.DecodeLastRuneInString(c.Unit)
if r != utf8.RuneError || size != 1 {
break
}
c.Unit = c.Unit[:len(c.Unit)-1]
}
}
return true
}
// IsIdentity reports whether the entry carries no information and can be
// dropped rather than stored and persisted.
func (c CalConfig) IsIdentity() bool {
return c.Scale == 1 && c.Offset == 0 && c.Unit == ""
}
// calTable is the hub's calibration store, safe for concurrent use.
type calTable struct {
mu sync.RWMutex
entries map[string]CalConfig
}
func newCalTable() *calTable {
return &calTable{entries: make(map[string]CalConfig)}
}
// Set validates and stores one entry, reporting whether it was accepted.
// Storing an identity entry removes any existing one for that key.
func (t *calTable) Set(c CalConfig) bool {
if !c.Normalise() {
return false
}
t.mu.Lock()
defer t.mu.Unlock()
if c.IsIdentity() {
delete(t.entries, calKey(c.Source, c.Signal))
} else {
t.entries[calKey(c.Source, c.Signal)] = c
}
return true
}
// Replace swaps the whole table for the given entries, silently dropping the
// invalid and identity ones. Used by config load and reload.
func (t *calTable) Replace(list []CalConfig) {
next := make(map[string]CalConfig, len(list))
for _, c := range list {
if !c.Normalise() || c.IsIdentity() {
continue
}
next[calKey(c.Source, c.Signal)] = c
}
t.mu.Lock()
t.entries = next
t.mu.Unlock()
}
// List returns the entries sorted by source then signal, so both the wire
// message and the config file have a stable order.
func (t *calTable) List() []CalConfig {
t.mu.RLock()
out := make([]CalConfig, 0, len(t.entries))
for _, c := range t.entries {
out = append(out, c)
}
t.mu.RUnlock()
sort.Slice(out, func(i, j int) bool {
if out[i].Source != out[j].Source {
return out[i].Source < out[j].Source
}
return out[i].Signal < out[j].Signal
})
return out
}
// ─── Config file codec ────────────────────────────────────────────────────────
// configFileEntry is the union of a source block and a calibration block.
//
// The file is one FLAT array of FLAT objects — never a nested one. The C++
// StreamHub's LoadSourcesFile is a hand-rolled scanner that takes each "{" up
// to the next "}" as one object, so a nested block would truncate the parse.
// Scale and Offset are pointers so that an absent field can be told apart from
// an explicit zero and defaulted to the identity values.
type configFileEntry struct {
// Source fields.
Label string `json:"label,omitempty"`
Addr string `json:"addr,omitempty"`
MulticastGroup string `json:"multicastGroup,omitempty"`
DataPort int `json:"dataPort,omitempty"`
// Calibration fields.
Source string `json:"source,omitempty"`
Signal string `json:"signal,omitempty"`
Scale *float64 `json:"scale,omitempty"`
Offset *float64 `json:"offset,omitempty"`
Unit string `json:"unit,omitempty"`
}
// parseConfigFile splits the flat array into sources and calibration entries.
// A block with "addr" is a source, one with "signal" is a calibration; anything
// else is skipped with a warning.
func parseConfigFile(data []byte) ([]SourceConfig, []CalConfig, error) {
var raw []configFileEntry
if err := json.Unmarshal(data, &raw); err != nil {
return nil, nil, err
}
srcs := make([]SourceConfig, 0, len(raw))
cals := make([]CalConfig, 0, len(raw))
for _, e := range raw {
switch {
case e.Addr != "":
srcs = append(srcs, SourceConfig{
Label: e.Label,
Addr: e.Addr,
MulticastGroup: e.MulticastGroup,
DataPort: e.DataPort,
})
case e.Signal != "":
c := CalConfig{Source: e.Source, Signal: e.Signal, Scale: 1, Offset: 0, Unit: e.Unit}
if e.Scale != nil {
c.Scale = *e.Scale
}
if e.Offset != nil {
c.Offset = *e.Offset
}
if !c.Normalise() {
log.Printf("wshub: skipping invalid calibration entry %q/%q", e.Source, e.Signal)
continue
}
cals = append(cals, c)
default:
log.Printf("wshub: skipping unrecognised config block")
}
}
return srcs, cals, nil
}
// encodeConfigFile renders the sources followed by the calibration entries as
// one flat array, in the indented shape the existing files already use.
func encodeConfigFile(srcs []SourceConfig, cals []CalConfig) ([]byte, error) {
out := make([]configFileEntry, 0, len(srcs)+len(cals))
for _, s := range srcs {
out = append(out, configFileEntry{
Label: s.Label,
Addr: s.Addr,
MulticastGroup: s.MulticastGroup,
DataPort: s.DataPort,
})
}
for _, c := range cals {
scale, offset := c.Scale, c.Offset
out = append(out, configFileEntry{
Source: c.Source,
Signal: c.Signal,
Scale: &scale,
Offset: &offset,
Unit: c.Unit,
})
}
return json.MarshalIndent(out, "", " ")
}
Common/Client/go/wshub/calibration_test.go
+237
View File
@@ -0,0 +1,237 @@
package wshub
import (
"math"
"strings"
"testing"
"unicode/utf8"
)
func TestCalConfigNormalise(t *testing.T) {
cases := []struct {
name string
in CalConfig
want bool
wantUnit string
}{
{"plain", CalConfig{Source: "wave", Signal: "Adc", Scale: 2, Offset: -1, Unit: "V"}, true, "V"},
{"trims", CalConfig{Source: " wave ", Signal: " Adc ", Scale: 1, Unit: " V "}, true, "V"},
{"emptySource", CalConfig{Signal: "Adc", Scale: 1}, false, ""},
{"emptySignal", CalConfig{Source: "wave", Scale: 1}, false, ""},
{"zeroScale", CalConfig{Source: "wave", Signal: "Adc", Scale: 0}, false, ""},
{"nanScale", CalConfig{Source: "wave", Signal: "Adc", Scale: math.NaN()}, false, ""},
{"infScale", CalConfig{Source: "wave", Signal: "Adc", Scale: math.Inf(1)}, false, ""},
{"nanOffset", CalConfig{Source: "wave", Signal: "Adc", Scale: 1, Offset: math.NaN()}, false, ""},
{"infOffset", CalConfig{Source: "wave", Signal: "Adc", Scale: 1, Offset: math.Inf(-1)}, false, ""},
{"negScaleOK", CalConfig{Source: "wave", Signal: "Adc", Scale: -1}, true, ""},
{"longUnit", CalConfig{Source: "wave", Signal: "Adc", Scale: 1,
Unit: "0123456789abcdefGHIJ"}, true, "0123456789abcdef"},
// Finding 1: array-element suffix stripping for cross-implementation parity.
{"arrayIndex3", CalConfig{Source: "wave", Signal: "Adc[3]", Scale: 1}, true, ""},
{"arrayIndex12", CalConfig{Source: "wave", Signal: "Adc[12]", Scale: 1}, true, ""},
{"arrayNoSuffix", CalConfig{Source: "wave", Signal: "Adc", Scale: 1}, true, ""},
{"arrayMidBracket", CalConfig{Source: "wave", Signal: "A[1]B", Scale: 1}, true, ""},
{"arrayNonNumeric", CalConfig{Source: "wave", Signal: "Adc[x]", Scale: 1}, true, ""},
{"arrayZeroOnly", CalConfig{Source: "wave", Signal: "[0]", Scale: 1}, false, ""},
}
for _, c := range cases {
got := c.in
if ok := got.Normalise(); ok != c.want {
t.Errorf("%s: Normalise() = %v, want %v", c.name, ok, c.want)
continue
}
if c.want && got.Unit != c.wantUnit {
t.Errorf("%s: Unit = %q, want %q", c.name, got.Unit, c.wantUnit)
}
}
if len("0123456789abcdef") != maxUnitLen {
t.Fatalf("test assumes maxUnitLen == 16, got %d", maxUnitLen)
}
// Verify stripped Signal values for array-index cases.
arraySignalCases := []struct {
input string
want string
}{
{"Adc[3]", "Adc"},
{"Adc[12]", "Adc"},
{"Adc", "Adc"},
{"A[1]B", "A[1]B"},
{"Adc[x]", "Adc[x]"},
}
for _, ac := range arraySignalCases {
got := CalConfig{Source: "wave", Signal: ac.input, Scale: 1}
got.Normalise()
if got.Signal != ac.want {
t.Errorf("Signal strip %q: got %q, want %q", ac.input, got.Signal, ac.want)
}
}
// Finding 2: UTF-8 unit truncation must not split a multi-byte rune.
// "°" is U+00B0, encoded as 2 bytes in UTF-8.
degree := "°"
if len(degree) != 2 {
t.Fatalf("test expects '°' to be 2 bytes, got %d", len(degree))
}
unit16 := strings.Repeat(degree, 8) // exactly 16 bytes — must survive intact
c8 := CalConfig{Source: "wave", Signal: "Adc", Scale: 1, Unit: unit16}
c8.Normalise()
if c8.Unit != unit16 {
t.Errorf("16-byte degree unit mangled: got %q, want %q", c8.Unit, unit16)
}
if !utf8.ValidString(c8.Unit) {
t.Errorf("16-byte degree unit is not valid UTF-8: %q", c8.Unit)
}
unit18 := strings.Repeat(degree, 9) // 18 bytes — must truncate to 8 degrees (16 bytes), not 16 bytes with a broken half-rune
c9 := CalConfig{Source: "wave", Signal: "Adc", Scale: 1, Unit: unit18}
c9.Normalise()
if c9.Unit != unit16 {
t.Errorf("18-byte degree unit truncated to %q, want %q", c9.Unit, unit16)
}
if !utf8.ValidString(c9.Unit) {
t.Errorf("truncated degree unit is not valid UTF-8: %q", c9.Unit)
}
}
func TestCalTableSetListAndIdentityRemoval(t *testing.T) {
tab := newCalTable()
if !tab.Set(CalConfig{Source: "b", Signal: "Y", Scale: 3, Offset: 1, Unit: "A"}) {
t.Fatal("Set(b/Y) rejected")
}
if !tab.Set(CalConfig{Source: "a", Signal: "X", Scale: 2}) {
t.Fatal("Set(a/X) rejected")
}
if tab.Set(CalConfig{Source: "a", Signal: "Z", Scale: 0}) {
t.Error("Set with scale=0 accepted, want rejected")
}
got := tab.List()
if len(got) != 2 {
t.Fatalf("List() = %d entries, want 2", len(got))
}
// Sorted by source then signal.
if got[0].Source != "a" || got[1].Source != "b" {
t.Errorf("List() order = %q,%q, want a,b", got[0].Source, got[1].Source)
}
// An identity entry removes the stored one.
if !tab.Set(CalConfig{Source: "a", Signal: "X", Scale: 1, Offset: 0, Unit: ""}) {
t.Fatal("identity Set rejected")
}
if got := tab.List(); len(got) != 1 || got[0].Source != "b" {
t.Errorf("after identity Set, List() = %+v, want only b/Y", got)
}
}
func TestCalTableReplace(t *testing.T) {
tab := newCalTable()
tab.Set(CalConfig{Source: "old", Signal: "X", Scale: 5})
tab.Replace([]CalConfig{
{Source: "new", Signal: "Y", Scale: 2},
{Source: "bad", Signal: "Z", Scale: 0}, // invalid → dropped
{Source: "id", Signal: "W", Scale: 1, Offset: 0, Unit: ""}, // identity → dropped
})
got := tab.List()
if len(got) != 1 || got[0].Source != "new" {
t.Fatalf("List() = %+v, want only new/Y", got)
}
}
func TestParseConfigFileCurrentFormat(t *testing.T) {
// A file written by the current binaries — sources only, spaces after colons.
data := []byte(`[
{
"label": "wave",
"addr": "127.0.0.1:44500"
},
{
"label": "mc",
"addr": "127.0.0.1:44501",
"multicastGroup": "239.0.0.1",
"dataPort": 44502
}
]`)
srcs, cals, err := parseConfigFile(data)
if err != nil {
t.Fatalf("parseConfigFile: %v", err)
}
if len(srcs) != 2 || len(cals) != 0 {
t.Fatalf("got %d sources / %d cals, want 2 / 0", len(srcs), len(cals))
}
if srcs[1].MulticastGroup != "239.0.0.1" || srcs[1].DataPort != 44502 {
t.Errorf("multicast source = %+v", srcs[1])
}
}
func TestParseConfigFileMixed(t *testing.T) {
data := []byte(`[
{"label":"wave","addr":"127.0.0.1:44500"},
{"source":"wave","signal":"Adc","scale":0.00030518,"offset":-1.25,"unit":"V"},
{"source":"wave","signal":"Bare"},
{"source":"wave","signal":"Bad","scale":0},
{"nonsense":true}
]`)
srcs, cals, err := parseConfigFile(data)
if err != nil {
t.Fatalf("parseConfigFile: %v", err)
}
if len(srcs) != 1 {
t.Fatalf("got %d sources, want 1", len(srcs))
}
if len(cals) != 2 {
t.Fatalf("got %d cals, want 2 (Adc and Bare; Bad is invalid)", len(cals))
}
if cals[0].Scale != 0.00030518 || cals[0].Offset != -1.25 || cals[0].Unit != "V" {
t.Errorf("Adc = %+v", cals[0])
}
// Absent scale/offset default to the identity values, not to zero.
if cals[1].Signal != "Bare" || cals[1].Scale != 1 || cals[1].Offset != 0 {
t.Errorf("Bare = %+v, want scale 1 / offset 0", cals[1])
}
}
func TestParseConfigFileMalformed(t *testing.T) {
if _, _, err := parseConfigFile([]byte("not json")); err == nil {
t.Error("parseConfigFile(garbage) = nil error, want error")
}
}
func TestEncodeConfigFileRoundTrip(t *testing.T) {
srcs := []SourceConfig{
{Label: "wave", Addr: "127.0.0.1:44500"},
{Label: "mc", Addr: "127.0.0.1:44501", MulticastGroup: "239.0.0.1", DataPort: 44502},
}
cals := []CalConfig{
{Source: "wave", Signal: "Adc", Scale: 0.5, Offset: 0, Unit: "V"},
}
data, err := encodeConfigFile(srcs, cals)
if err != nil {
t.Fatalf("encodeConfigFile: %v", err)
}
gotSrcs, gotCals, err := parseConfigFile(data)
if err != nil {
t.Fatalf("parseConfigFile(encoded): %v\n%s", err, data)
}
if len(gotSrcs) != 2 || len(gotCals) != 1 {
t.Fatalf("round-trip gave %d sources / %d cals, want 2 / 1\n%s",
len(gotSrcs), len(gotCals), data)
}
if gotSrcs[1] != srcs[1] {
t.Errorf("source round-trip: got %+v, want %+v", gotSrcs[1], srcs[1])
}
if gotCals[0] != cals[0] {
t.Errorf("cal round-trip: got %+v, want %+v", gotCals[0], cals[0])
}
// offset 0 must survive as an explicit field, not be dropped by omitempty.
if !bytesContains(data, []byte(`"offset": 0`)) {
t.Errorf("encoded file lost the zero offset:\n%s", data)
}
}
func bytesContains(hay, needle []byte) bool {
for i := 0; i+len(needle) <= len(hay); i++ {
if string(hay[i:i+len(needle)]) == string(needle) {
return true
}
}
return false
}
Common/Client/go/wshub/capture_coverage_test.go
+168
View File
@@ -0,0 +1,168 @@
package wshub
import (
"encoding/binary"
"math"
"testing"
)
// decodeCaptureSpan pulls the time extent of one signal out of a v2 frame.
func decodeCaptureSpan(t *testing.T, buf []byte, key string) (first, last float64, n int) {
t.Helper()
off := 1 + 8 + 8 + 8
nSig := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
for i := 0; i < nSig; i++ {
kl := int(binary.LittleEndian.Uint16(buf[off:]))
off += 2
k := string(buf[off : off+kl])
off += kl
cnt := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
if k == key && cnt > 0 {
first = math.Float64frombits(binary.LittleEndian.Uint64(buf[off:]))
last = math.Float64frombits(binary.LittleEndian.Uint64(buf[off+(cnt-1)*8:]))
n = cnt
}
off += cnt * 16
}
return
}
// The rings only reach back over the window once they have rolled over at the
// current bucket, which takes as long as the window itself — so a window widened
// mid-run leaves the first captures asking for history the rings never stored.
// The archive kept it, and the capture must come back whole.
func TestCaptureBackfillsItsHeadFromTheArchive(t *testing.T) {
h := NewHub()
hw, key := newTestHistory(t, HistoryConfig{
WindowSec: 60, Decimation: 1, MinDiskFreeMB: -1,
}, 1000)
h.hist = hw
// 20 s of 1 kSps, archived in full…
ts, vs := ramp(1000, 0.001, 20000)
hw.write(key, ts, vs)
// …but a ring that only ever holds the last 5 s of it.
rb := newSigRing(5000)
rb.write(ts, vs)
h.rings[key] = rb
// A 15 s window, of which the ring has the newest third.
const t0, t1 = 1005.0, 1020.0
buf := h.buildTriggerCapture(1015, 10, 5)
if buf == nil {
t.Fatal("no capture frame built")
}
first, last, n := decodeCaptureSpan(t, buf, key)
if first > t0+0.05 {
t.Errorf("capture starts at %.3f, want the window's start %.3f — the archive holds it",
first, t0)
}
if last < t1-0.05 {
t.Errorf("capture ends at %.3f, want %.3f", last, t1)
}
if n < 100 {
t.Errorf("capture has %d points, too few for a 15 s window at 1 kSps", n)
}
// The join between the two sources must not break time order, or every
// binary search over the capture — client-side and in the hold — misreads it.
ct, _, ok := h.capture.slice(key, t0, t1)
if !ok {
t.Fatal("the hold declined the window it just published")
}
for i := 1; i < len(ct); i++ {
if ct[i] < ct[i-1] {
t.Fatalf("capture time goes backwards at %d: %.6f then %.6f", i, ct[i-1], ct[i])
}
}
}
// A capture that neither source could fill must not answer for the stretch it is
// missing: the client has to fall through to the archive instead of redrawing
// the same hole on every zoom.
func TestHoldDeclinesTheStretchACaptureNeverGot(t *testing.T) {
h := NewHub()
ts, vs := ramp(1000, 0.001, 20000)
rb := newSigRing(5000) // the newest 5 s only, and no archive to fill from
rb.write(ts, vs)
h.rings["src:sig"] = rb
if buf := h.buildTriggerCapture(1015, 10, 5); buf == nil {
t.Fatal("no capture frame built")
}
if _, _, ok := h.capture.slice("src:sig", 1005, 1020); ok {
t.Error("the hold answered for 15 s it only has the last 5 s of")
}
// What it does hold, it still serves.
if _, _, ok := h.capture.slice("src:sig", 1016, 1019); !ok {
t.Error("the hold declined a range well inside its data")
}
}
// TestCaptureCoverageAcrossShots walks a whole acquisition the way Run() does —
// ingest, retune, dueCapture, rearm — and reports how much of each window the
// capture actually came back with.
func TestCaptureCoverageAcrossShots(t *testing.T) {
const (
key = "s1:Ch1"
rate = 100e3 // scaled 10x down from the 1 MSps producer
budget = 400_000
window = 120.0
prePct = 20.0
batchSec = 1.0 / 30.0
simSec = 900.0
)
h := NewHub()
h.SetRingBudget(budget)
h.rings[key] = newSigRing(ringCapInitial)
h.trigger.SetConfig(trigConfig{signalKey: key, edge: "rising", threshold: 0,
windowSec: window, prePercent: prePct, mode: "normal", holdoffSec: 0.2})
rateHz := float64(rate)
nBatch := int(rateHz * batchSec)
ts := make([]float64, nBatch)
vs := make([]float64, nBatch)
armed := false
shots := 0
for now := 0.0; now < simSec; now += batchSec {
for i := range ts {
ts[i] = now + float64(i)/rateHz
// 0.05 Hz sine: one rising zero crossing every 20 s.
vs[i] = math.Sin(2 * math.Pi * 0.05 * ts[i])
}
h.ingest(key, 1, ts, vs)
h.retuneRings(now)
// Arm once the stream is going, as a user would.
if !armed && now > 5 {
h.trigger.Arm()
armed = true
}
if trigTime, pre, post, ok := h.trigger.dueCapture(now + batchSec); ok {
buf := h.buildTriggerCapture(trigTime, pre, post)
if buf == nil {
t.Fatalf("shot at t=%.1f produced no frame", trigTime)
}
first, last, n := decodeCaptureSpan(t, buf, key)
t0, t1 := trigTime-pre, trigTime+post
_, ringSpan := h.rings[key].stats()
shots++
t.Logf("shot %d fired t=%.1f window [%.1f,%.1f] got [%.1f,%.1f] "+
"= %.0f%% (%d pts, bucket %d, ring span %.1f s)",
shots, trigTime, t0, t1, first, last,
100*(last-first)/(t1-t0), n, h.rings[key].bucketSize(), ringSpan)
h.trigger.markTriggered(now + batchSec)
} else if h.trigger.dueRearm(now + batchSec) {
h.trigger.Arm()
}
}
if shots < 3 {
t.Fatalf("only %d shots in %.0f s", shots, simSec)
}
}
Common/Client/go/wshub/capture_shortwindow_test.go
+75
View File
@@ -0,0 +1,75 @@
package wshub
import (
"math"
"testing"
)
// A short window at a high sample rate fits in a ring's initial capacity, so the
// retune sweep used to leave it there — and a ring holding exactly the window has
// already rolled past the front of a capture by the time that capture is read,
// which happens a post-window plus captureMarginSec after the trigger fires.
//
// 1 MSps over a 200 ms window: 200 k points fit in the 250 k initial ring, and
// every shot came back missing its first 123 ms.
func TestCaptureWholeAtHighRateShortWindow(t *testing.T) {
const (
key = "s1:Ch1"
rate = 1e6
window = 0.2
prePct = 20.0
batchSec = 1.0 / 30.0
simSec = 6.0
)
h := NewHub()
h.SetRingBudget(defaultRingPts)
h.rings[key] = newSigRing(ringCapInitial)
h.trigger.SetConfig(trigConfig{signalKey: key, edge: "rising", threshold: 0,
windowSec: window, prePercent: prePct, mode: "normal", holdoffSec: 0.2})
rateHz := float64(rate)
nBatch := int(rateHz * batchSec)
ts := make([]float64, nBatch)
vs := make([]float64, nBatch)
armed, shots := false, 0
for now := 0.0; now < simSec; now += batchSec {
for i := range ts {
ts[i] = now + float64(i)/rateHz
vs[i] = math.Sin(2 * math.Pi * 5 * ts[i]) // a rising crossing every 200 ms
}
h.ingest(key, 1, ts, vs)
h.retuneRings(now)
h.refreshTriggerFill()
if !armed && now > 2 {
h.trigger.Arm()
armed = true
}
trigTime, pre, post, ok := h.trigger.dueCapture(now + batchSec)
if !ok {
if h.trigger.dueRearm(now + batchSec) {
h.trigger.Arm()
}
continue
}
t0 := trigTime - pre
buf := h.buildTriggerCapture(trigTime, pre, post)
if buf == nil {
t.Fatalf("shot at t=%.4f produced no frame at all", trigTime)
}
first, last, n := decodeCaptureSpan(t, buf, key)
shots++
if lost := first - t0; lost > shortCaptureTol*window {
_, span := h.rings[key].stats()
t.Errorf("shot at t=%.4f is missing %.0f ms at the front of its %.0f ms window "+
"(got [%.4f,%.4f], %d pts; ring holds %.4f s in %d points)",
trigTime, 1e3*lost, 1e3*window, first, last, n, span, h.rings[key].capacity())
}
h.trigger.markTriggered(now + batchSec)
}
if shots < 3 {
t.Fatalf("only %d shots in %.0f s", shots, simSec)
}
}
Common/Client/go/wshub/capturehold.go
+83
View File
@@ -0,0 +1,83 @@
package wshub
import (
"sort"
"sync"
)
// capturedWindow is one delivered trigger capture, held at the resolution the
// rings had when it was taken. Nothing mutates it after publication, so readers
// may sub-slice it without copying.
type capturedWindow struct {
t0, t1 float64
sigs map[string]sigData
}
// captureHold is the read half of the trigger double buffer; the rings are the
// write half.
//
// The rings keep rolling while the trigger re-arms and collects the next shot,
// so within seconds of a capture they no longer hold the window the user is
// looking at — a zoom into it came back with only the newest sliver, or with
// nothing. Publishing the window here at capture time gives the viewer a
// snapshot that the re-arming acquisition cannot overwrite: the swap happens
// only when the *next* capture is complete, which is also the moment the client
// stops displaying this one.
type captureHold struct {
mu sync.RWMutex
cur *capturedWindow
}
// publish swaps in a new capture, retiring the previous one. Readers that
// already hold a pointer to the retired window keep reading it safely.
func (ch *captureHold) publish(t0, t1 float64, sigs map[string]sigData) {
if len(sigs) == 0 {
return
}
w := &capturedWindow{t0: t0, t1: t1, sigs: sigs}
ch.mu.Lock()
ch.cur = w
ch.mu.Unlock()
}
// clear drops the held capture, releasing its memory.
func (ch *captureHold) clear() {
ch.mu.Lock()
ch.cur = nil
ch.mu.Unlock()
}
// slice answers [a, b] for one signal out of the held capture, reporting
// whether it could.
//
// It declines any range reaching outside the captured window: that is a live
// zoom or a pan off the capture, and only the rings still track the stream.
// Inside the window the hold is never worse than the rings — retuning does not
// rewrite stored samples, so a ring that still covers the range holds the very
// same points — which is why no trigger-state gating is needed here.
func (ch *captureHold) slice(key string, a, b float64) ([]float64, []float64, bool) {
ch.mu.RLock()
w := ch.cur
ch.mu.RUnlock()
if w == nil || a < w.t0 || b > w.t1 {
return nil, nil, false
}
sd, ok := w.sigs[key]
if !ok || len(sd.T) == 0 {
return nil, nil, false
}
// The window is what was asked for; this signal's samples are what could be
// found. A capture whose front was never recoverable must not answer for the
// stretch it is missing — the client would redraw the same hole on every
// zoom and every "fit" instead of falling back to the archive.
tol := shortCaptureTol * (w.t1 - w.t0)
if sd.T[0] > a+tol || sd.T[len(sd.T)-1] < b-tol {
return nil, nil, false
}
lo := sort.SearchFloat64s(sd.T, a)
hi := lo + sort.Search(len(sd.T)-lo, func(i int) bool { return sd.T[lo+i] > b })
if hi <= lo {
return nil, nil, false
}
return sd.T[lo:hi], sd.V[lo:hi], true
}
Common/Client/go/wshub/capturehold_test.go
+128
View File
@@ -0,0 +1,128 @@
package wshub
import "testing"
func heldRamp(t0, dt float64, n int) sigData {
sd := sigData{T: make([]float64, n), V: make([]float64, n)}
for i := range sd.T {
sd.T[i] = t0 + float64(i)*dt
sd.V[i] = float64(i)
}
return sd
}
func TestCaptureHoldServesRangesInsideTheWindow(t *testing.T) {
var ch captureHold
ch.publish(0, 10, map[string]sigData{"s1:sig": heldRamp(0, 0.1, 101)})
gt, gv, ok := ch.slice("s1:sig", 2, 3)
if !ok {
t.Fatal("held capture declined a range inside its window")
}
if gt[0] < 2 || gt[len(gt)-1] > 3 {
t.Fatalf("range %v..%v escapes the request 2..3", gt[0], gt[len(gt)-1])
}
if len(gt) != len(gv) {
t.Fatalf("t/v length mismatch: %d vs %d", len(gt), len(gv))
}
if gv[0] != 20 {
t.Fatalf("first value %v, want the sample at t=2", gv[0])
}
}
// A range poking outside the capture is a live zoom: only the rings still track
// the stream, so the hold must stand aside rather than answer a clipped range.
func TestCaptureHoldDeclinesRangesOutsideTheWindow(t *testing.T) {
var ch captureHold
ch.publish(0, 10, map[string]sigData{"s1:sig": heldRamp(0, 0.1, 101)})
for _, r := range [][2]float64{{-1, 5}, {5, 11}, {20, 30}, {-5, -1}} {
if _, _, ok := ch.slice("s1:sig", r[0], r[1]); ok {
t.Fatalf("held capture answered %v..%v, which is not inside 0..10", r[0], r[1])
}
}
if _, _, ok := ch.slice("other:sig", 2, 3); ok {
t.Fatal("held capture answered for a signal it does not hold")
}
}
func TestCaptureHoldZeroValueAndClearDecline(t *testing.T) {
var ch captureHold
if _, _, ok := ch.slice("s1:sig", 0, 1); ok {
t.Fatal("empty hold answered a request")
}
ch.publish(0, 10, map[string]sigData{"s1:sig": heldRamp(0, 0.1, 101)})
ch.clear()
if _, _, ok := ch.slice("s1:sig", 2, 3); ok {
t.Fatal("cleared hold still answered a request")
}
}
// The point of the double buffer: the window a client is exploring survives the
// re-armed acquisition rolling the rings past it, and is replaced only when the
// next shot completes.
func TestZoomIntoACaptureSurvivesTheRingRollingPast(t *testing.T) {
h := NewHub()
rb := newSigRing(4000)
h.rings["s1:sig"] = rb
// 2 s of 1 kSps, then fire a trigger over [0.5, 1.5].
ts, vs := make([]float64, 2000), make([]float64, 2000)
for i := range ts {
ts[i], vs[i] = float64(i)*1e-3, float64(i)
}
rb.write(ts, vs)
if msg := h.buildTriggerCapture(1.0, 0.5, 0.5); msg == nil {
t.Fatal("buildTriggerCapture produced no frame")
}
// The trigger re-arms and the stream runs on until the captured window has
// been overwritten several times over.
for pass := 0; pass < 5; pass++ {
for i := range ts {
ts[i] += 2.0
}
rb.write(ts, vs)
}
if rt, _ := rb.slice(0.5, 1.5); len(rt) != 0 {
t.Fatalf("ring still holds %d points of the captured window; the test is not exercising the hold", len(rt))
}
got := h.zoomSlice(0.8, 0.9, []string{"s1:sig"}, 1<<30)
sd, ok := got["s1:sig"]
if !ok {
t.Fatal("zoom into the held capture returned nothing")
}
if len(sd.T) != 101 {
t.Fatalf("zoom returned %d points, want the 101 samples in 0.8..0.9", len(sd.T))
}
if sd.V[0] != 800 || sd.V[len(sd.V)-1] != 900 {
t.Fatalf("zoom returned values %v..%v, want 800..900", sd.V[0], sd.V[len(sd.V)-1])
}
// A live zoom outside the held window still reaches the rings.
if live := h.zoomSlice(11.0, 11.1, []string{"s1:sig"}, 1<<30); len(live["s1:sig"].T) == 0 {
t.Fatal("live zoom outside the capture was swallowed by the hold")
}
}
// A shot that yields nothing must not blank the window already on screen.
func TestEmptyCaptureKeepsThePreviousHold(t *testing.T) {
h := NewHub()
rb := newSigRing(4000)
h.rings["s1:sig"] = rb
ts, vs := make([]float64, 2000), make([]float64, 2000)
for i := range ts {
ts[i], vs[i] = float64(i)*1e-3, float64(i)
}
rb.write(ts, vs)
h.buildTriggerCapture(1.0, 0.5, 0.5)
// A window the rings have no samples for at all.
if msg := h.buildTriggerCapture(500.0, 0.5, 0.5); msg != nil {
t.Fatal("capture of an empty window produced a frame")
}
if _, _, ok := h.capture.slice("s1:sig", 0.8, 0.9); !ok {
t.Fatal("empty capture dropped the previously held window")
}
}
Common/Client/go/wshub/export.go
+119
View File
@@ -0,0 +1,119 @@
package wshub
import (
"fmt"
"net/http"
"sort"
"strconv"
"strings"
"time"
"github.com/parquet-go/parquet-go"
)
// ExportSample is one row of the binary export: a single stored sample, in
// long ("tidy") form, keyed by source and signal with its own timestamp.
//
// Keeping each signal's samples as its own rows — rather than resampling onto a
// shared time grid — is what makes the export hole-free: per-signal streams of
// different lengths export exactly as stored, nothing is fabricated, and
// nothing is dropped.
type ExportSample struct {
Source string `parquet:"source"`
Signal string `parquet:"signal"`
Time float64 `parquet:"time"`
Value float64 `parquet:"value"`
}
// exportChunkRows bounds each batched write and, via MaxRowsPerRowGroup, the
// size of each parquet row group: memory stays bounded however large the
// export is, because a finished row group is flushed to the HTTP stream.
const exportChunkRows = 65536
// exportWriteBuffer is the parquet writer's output buffer: larger than the
// 32KiB default means fewer writes on the HTTP stream for a multi-GB export.
const exportWriteBuffer = 1 << 20
// HandleExport serves GET /api/export?t0=..&t1=..[&signals=a,b] as a Parquet
// file containing every stored sample of the named signals in [t0, t1].
//
// Unlike /api/zoom there is no decimation: the file holds the full contents of
// the rings. At rates above the ring budget those contents are min/max buckets
// (the finest resolution the hub retains); at lower rates they are verbatim.
func (h *Hub) HandleExport(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
t0, err0 := strconv.ParseFloat(q.Get("t0"), 64)
t1, err1 := strconv.ParseFloat(q.Get("t1"), 64)
if err0 != nil || err1 != nil || t1 <= t0 {
http.Error(w, "invalid t0/t1", http.StatusBadRequest)
return
}
var keys []string
if s := strings.TrimSpace(q.Get("signals")); s != "" {
keys = strings.Split(s, ",")
for i := range keys {
keys[i] = strings.TrimSpace(keys[i])
}
}
// Snapshot the rings we will read. A signal removed mid-export must not
// silently drop rows from the file.
h.ringsMu.RLock()
refs := make(map[string]*sigRing)
if keys == nil {
for k, rb := range h.rings {
refs[k] = rb
}
} else {
for _, k := range keys {
if rb, ok := h.rings[k]; ok {
refs[k] = rb
}
}
}
h.ringsMu.RUnlock()
if len(refs) == 0 {
http.Error(w, "no signals", http.StatusNotFound)
return
}
// Deterministic column order.
names := make([]string, 0, len(refs))
for k := range refs {
names = append(names, k)
}
sort.Strings(names)
w.Header().Set("Content-Type", "application/vnd.apache.parquet")
w.Header().Set("Content-Disposition",
fmt.Sprintf("attachment; filename=\"signals_%d.parquet\"", time.Now().Unix()))
writer := parquet.NewGenericWriter[ExportSample](w,
parquet.MaxRowsPerRowGroup(exportChunkRows),
parquet.WriteBufferSize(exportWriteBuffer),
)
batch := make([]ExportSample, 0, exportChunkRows)
for _, key := range names {
st, sv := refs[key].slice(t0, t1)
colon := strings.IndexByte(key, ':')
source, signal := key, key
if colon >= 0 {
source = key[:colon]
signal = key[colon+1:]
}
for i := range st {
batch = append(batch, ExportSample{Source: source, Signal: signal, Time: st[i], Value: sv[i]})
if len(batch) >= exportChunkRows {
if _, err := writer.Write(batch); err != nil {
// Client went away or the stream broke; stop writing.
return
}
batch = batch[:0]
}
}
}
if len(batch) > 0 {
_, _ = writer.Write(batch)
}
_ = writer.Close()
}
Common/Client/go/wshub/export_test.go
+87
View File
@@ -0,0 +1,87 @@
package wshub
import (
"bytes"
"net/http/httptest"
"testing"
"github.com/parquet-go/parquet-go"
)
func TestHandleExportParquetFullResolution(t *testing.T) {
h := NewHub()
// Two signals with different lengths and offset time bases: the export must
// keep every sample of each, on its own timestamps (no holes, no
// resampling, no decimation).
sig1 := newSigRing(10000)
sig2 := newSigRing(10000)
t1, v1 := make([]float64, 1000), make([]float64, 1000)
for i := range t1 {
t1[i] = float64(i) * 0.001
v1[i] = float64(i) * 2
}
sig1.write(t1, v1)
t2, v2 := make([]float64, 500), make([]float64, 500)
for i := range t2 {
t2[i] = 0.1 + float64(i)*0.002
v2[i] = -float64(i)
}
sig2.write(t2, v2)
h.rings["s1:Ch1"] = sig1
h.rings["s1:Ch2"] = sig2
req := httptest.NewRequest("GET", "/api/export?t0=0&t1=2&signals=s1:Ch1,s1:Ch2", nil)
rec := httptest.NewRecorder()
h.HandleExport(rec, req)
if rec.Code != 200 {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
reader := parquet.NewGenericReader[ExportSample](bytes.NewReader(rec.Body.Bytes()))
defer reader.Close()
var got []ExportSample
buf := make([]ExportSample, 1000)
for {
n, err := reader.Read(buf)
got = append(got, buf[:n]...)
if err != nil {
break
}
}
if len(got) != 1500 {
t.Fatalf("rows = %d, want 1500 (every sample of both signals)", len(got))
}
ch1 := filterExportSamples(got, "s1", "Ch1")
ch2 := filterExportSamples(got, "s1", "Ch2")
if len(ch1) != 1000 || len(ch2) != 500 {
t.Fatalf("ch1=%d ch2=%d rows, want 1000/500 (no holes, no resampling)", len(ch1), len(ch2))
}
if ch1[0].Time != 0 || ch1[0].Value != 0 || ch1[999].Time != 0.999 || ch1[999].Value != 1998 {
t.Fatalf("ch1 endpoints wrong: first=%+v last=%+v", ch1[0], ch1[999])
}
if ch2[0].Time != 0.1 || ch2[499].Time != 0.1+499*0.002 || ch2[499].Value != -499 {
t.Fatalf("ch2 endpoints wrong: first=%+v last=%+v", ch2[0], ch2[499])
}
}
func filterExportSamples(rows []ExportSample, source, signal string) []ExportSample {
out := make([]ExportSample, 0, len(rows))
for _, r := range rows {
if r.Source == source && r.Signal == signal {
out = append(out, r)
}
}
return out
}
func TestHandleExportParquetBadRange(t *testing.T) {
h := NewHub()
h.rings["s1:Ch1"] = newSigRing(10)
req := httptest.NewRequest("GET", "/api/export?t0=2&t1=1", nil)
rec := httptest.NewRecorder()
h.HandleExport(rec, req)
if rec.Code != 400 {
t.Fatalf("status = %d, want 400 for inverted range", rec.Code)
}
}
Common/Client/go/wshub/history.go
+1348
View File
File diff suppressed because it is too large Load Diff
Common/Client/go/wshub/history_test.go
+949
View File
@@ -0,0 +1,949 @@
package wshub
import (
"encoding/binary"
"math"
"os"
"path/filepath"
"testing"
"marte2/common/udpsprotocol"
)
// newTestHistory opens a writer in a temp dir with one signal file of the given
// declared rate, and returns the writer plus that signal's key.
func newTestHistory(t *testing.T, cfg HistoryConfig, rate float64) (*historyWriter, string) {
t.Helper()
if cfg.Directory == "" {
cfg.Directory = t.TempDir()
}
hw, err := newHistoryWriter(cfg)
if err != nil {
t.Fatalf("newHistoryWriter: %v", err)
}
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: rate},
})
t.Cleanup(hw.close)
return hw, "src:sig"
}
func ramp(t0 float64, dt float64, n int) ([]float64, []float64) {
ts := make([]float64, n)
vs := make([]float64, n)
for i := range ts {
ts[i] = t0 + float64(i)*dt
vs[i] = float64(i)
}
return ts, vs
}
// A budget that cannot hold the window at full rate must buy the window by
// widening the min/max bucket, not by archiving a shorter stretch: a user
// looking at 600 s wants 600 s of it archived, coarser if need be.
func TestHistCapacityKeepsWindowByBucketing(t *testing.T) {
const mega = 1 << 20
cases := []struct {
name string
window float64
rate float64
maxPts int
wantBucket int
}{
// 60 s of 1 kSps is 60 k samples — well inside 1 MPt, so stored verbatim.
{"slow signal keeps full resolution", 60, 1000, mega, 1},
// 600 s of 1 MSps is 600 M samples against 16 Mi points: at 2 points per
// bucket and the headroom, ceil(2 × 1.25 × 600e6 / 16Mi) = 90 per bucket.
{"fast signal is enveloped", 600, 1e6, 16 * mega, 90},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
capacity, bucket := histCapacityFor(c.window, c.rate, 1, c.maxPts)
if bucket != c.wantBucket {
t.Errorf("bucket = %d, want %d", bucket, c.wantBucket)
}
if capacity > uint32(c.maxPts) {
t.Errorf("capacity %d exceeds the %d-point budget", capacity, c.maxPts)
}
// The whole window has to fit, which is the entire point.
if covered := histCoverageSec(capacity, bucket, 1, c.rate); covered < c.window {
t.Errorf("archive covers %.1f s, want the %.1f s window", covered, c.window)
}
})
}
}
// The file exists to serve the window, so it must track it: a client that widens
// what it displays must not be left reading an archive sized for the old span.
func TestHistorySetWindowResizesFiles(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 10}, 1000)
before := hw.files[key]
if before.bucket != 1 || histCoverageSec(before.capacity, 1, 1, 1000) < 10 {
t.Fatalf("initial geometry = cap %d bucket %d, want 10 s verbatim",
before.capacity, before.bucket)
}
if !hw.setWindow(600) {
t.Fatal("setWindow reported no change for a 60× wider window")
}
after := hw.files[key]
if after == before {
t.Fatal("the file was not re-created")
}
if cov := histCoverageSec(after.capacity, after.bucket, 1, 1000); cov < 600 {
t.Fatalf("archive covers %.1f s, want the new 600 s window", cov)
}
// Same window again: nothing to do, and re-creating the file would throw the
// archive away for nothing.
if hw.setWindow(600) {
t.Fatal("setWindow re-sized for an unchanged window")
}
// A nudge inside the hysteresis band must not either.
if hw.setWindow(610) {
t.Fatal("setWindow re-sized for a 2 % window change")
}
if hw.files[key] != after {
t.Fatal("the file was re-created despite the hysteresis")
}
}
// The archive is what a zoom beyond the rings reads, so a spike that only the
// archive still holds must survive being written to it.
func TestHistoryBucketedWriteKeepsPeaks(t *testing.T) {
// 1 kSps for 1 s = 1000 samples, plus headroom, into a 100-point budget →
// buckets of ceil(2 × 1.25 × 1000 / 100) = 25.
hw, key := newTestHistory(t, HistoryConfig{
WindowSec: 1, MinDiskFreeMB: -1, MaxPointsPerSignal: 100,
}, 1000)
hf := hw.files[key]
if hf.bucket != 25 {
t.Fatalf("bucket = %d, want 25", hf.bucket)
}
ts := make([]float64, 1000)
vs := make([]float64, 1000)
for i := range ts {
ts[i] = float64(i) * 0.001
}
vs[137] = 7.5 // a one-sample positive spike
vs[500] = -3.5 // and a negative one
hw.write(key, ts, vs)
rt, rv := hw.readRange(key, 0, 1, 1000)
if len(rt) == 0 {
t.Fatal("nothing archived")
}
hi, lo := false, false
for i := range rv {
if rv[i] == 7.5 && rt[i] == ts[137] {
hi = true
}
if rv[i] == -3.5 && rt[i] == ts[500] {
lo = true
}
}
if !hi || !lo {
t.Errorf("archive lost a spike (positive kept=%v, negative kept=%v)", hi, lo)
}
// A partial bucket is not written until it completes, so the last few
// samples may be missing; everything before them must be there.
if hf.count == 0 || hf.count > hf.capacity {
t.Errorf("archived %d points into a %d-point file", hf.count, hf.capacity)
}
}
func TestHistoryDisabledWithoutDirectory(t *testing.T) {
hw, err := newHistoryWriter(HistoryConfig{})
if err != nil {
t.Fatalf("newHistoryWriter: %v", err)
}
if hw != nil {
t.Fatal("empty Directory must disable history")
}
// Every method must stay usable on the nil writer, which is how the hub
// avoids guarding each call site.
if hw.enabled() {
t.Fatal("nil writer reports enabled")
}
hw.write("src:sig", []float64{1}, []float64{1})
hw.flushHeaders()
hw.close()
if rt, _ := hw.readRange("src:sig", 0, 1, 10); rt != nil {
t.Fatal("nil writer returned data")
}
if len(hw.info()) != 0 {
t.Fatal("nil writer returned info entries")
}
}
func TestHistoryWriteReadRoundTrip(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{}, 100)
ts, vs := ramp(10, 0.01, 500)
hw.write(key, ts, vs)
rt, rv := hw.readRange(key, 10.5, 11.0, 10000)
if len(rt) != 51 { // inclusive both ends, 0.01 s spacing
t.Fatalf("read %d points, want 51", len(rt))
}
if rt[0] < 10.5-1e-9 || rt[len(rt)-1] > 11.0+1e-9 {
t.Fatalf("range [%v, %v] escapes the request", rt[0], rt[len(rt)-1])
}
for i := range rt {
wantV := math.Round((rt[i] - 10) / 0.01)
if math.Abs(rv[i]-wantV) > 1e-6 {
t.Fatalf("point %d: value %v, want %v", i, rv[i], wantV)
}
}
}
func TestHistoryReadRangeOutsideDataIsEmpty(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{}, 100)
ts, vs := ramp(10, 0.01, 100)
hw.write(key, ts, vs)
if rt, _ := hw.readRange(key, 100, 200, 1000); len(rt) != 0 {
t.Fatalf("read %d points past the newest sample", len(rt))
}
if rt, _ := hw.readRange(key, 0, 5, 1000); len(rt) != 0 {
t.Fatalf("read %d points before the oldest sample", len(rt))
}
if rt, _ := hw.readRange("src:missing", 10, 11, 1000); rt != nil {
t.Fatal("unknown key returned data")
}
if rt, _ := hw.readRange(key, 11, 10, 1000); rt != nil {
t.Fatal("inverted range returned data")
}
}
// Once the file has wrapped, the oldest samples must be gone and the retained
// window must still read back contiguously across the wrap point.
func TestHistoryWrapAround(t *testing.T) {
// A sub-second window at 1 Sps sizes below the 1000-pair floor, which is a
// cheap capacity to wrap.
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 0.36}, 1)
hf := hw.files[key]
if hf.capacity != histMinCapacity {
t.Fatalf("capacity = %d, want the %d floor", hf.capacity, histMinCapacity)
}
// 2.5 fills, in batches that do not align with the capacity so the wrap
// lands mid-batch.
total := 2500
ts, vs := ramp(0, 1, total)
for i := 0; i < total; i += 333 {
end := i + 333
if end > total {
end = total
}
hw.write(key, ts[i:end], vs[i:end])
}
if hf.count != histMinCapacity {
t.Fatalf("count = %d, want a full %d", hf.count, histMinCapacity)
}
wantOldest := float64(total - histMinCapacity)
if hf.tOldest != wantOldest {
t.Fatalf("tOldest = %v, want %v", hf.tOldest, wantOldest)
}
if hf.tNewest != float64(total-1) {
t.Fatalf("tNewest = %v, want %v", hf.tNewest, float64(total-1))
}
rt, rv := hw.readRange(key, wantOldest, float64(total-1), 10000)
if len(rt) != histMinCapacity {
t.Fatalf("read %d points, want the full %d", len(rt), histMinCapacity)
}
for i := range rt {
want := wantOldest + float64(i)
if rt[i] != want || rv[i] != want {
t.Fatalf("point %d = (%v, %v), want (%v, %v)", i, rt[i], rv[i], want, want)
}
}
// The evicted samples must not come back.
if et, _ := hw.readRange(key, 0, wantOldest-1, 10000); len(et) != 0 {
t.Fatalf("read %d evicted points", len(et))
}
}
// A single batch larger than the file keeps its tail, not its head.
func TestHistoryOversizedBatchKeepsTail(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 0.36}, 1)
ts, vs := ramp(0, 1, 3000)
hw.write(key, ts, vs)
hf := hw.files[key]
if hf.count != histMinCapacity {
t.Fatalf("count = %d, want %d", hf.count, histMinCapacity)
}
if hf.tNewest != 2999 {
t.Fatalf("tNewest = %v, want 2999", hf.tNewest)
}
rt, _ := hw.readRange(key, 2000, 2999, 10000)
if len(rt) != histMinCapacity || rt[0] != 2000 {
t.Fatalf("retained window starts at %v with %d points, want 2000 / %d",
rt[0], len(rt), histMinCapacity)
}
}
func TestHistoryDecimation(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{Decimation: 4}, 100)
// Two batches, so the decimation phase must carry across the call boundary
// rather than restarting.
ts, vs := ramp(0, 0.01, 100)
hw.write(key, ts[:37], vs[:37])
hw.write(key, ts[37:], vs[37:])
rt, _ := hw.readRange(key, -1, 1e9, 10000)
if len(rt) != 25 {
t.Fatalf("kept %d of 100 points at decimation 4, want 25", len(rt))
}
for i := 1; i < len(rt); i++ {
if d := rt[i] - rt[i-1]; math.Abs(d-0.04) > 1e-9 {
t.Fatalf("spacing at %d = %v, want 0.04", i, d)
}
}
}
// The input slices are shared with the zoom ring and the trigger, so decimation
// must not touch them.
func TestHistoryWriteDoesNotMutateInput(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{Decimation: 3}, 100)
ts, vs := ramp(0, 0.01, 30)
tCopy := append([]float64(nil), ts...)
vCopy := append([]float64(nil), vs...)
hw.write(key, ts, vs)
for i := range ts {
if ts[i] != tCopy[i] || vs[i] != vCopy[i] {
t.Fatalf("write mutated input at %d", i)
}
}
}
// Reopening the same directory must pick the file back up with its contents,
// which is the whole point of persisting the header.
func TestHistoryReopenPreservesData(t *testing.T) {
dir := t.TempDir()
cfg := HistoryConfig{Directory: dir, WindowSec: 0.36}
sigs := []udpsprotocol.SignalInfo{{Name: "sig", TypeCode: 8, SamplingRate: 1}}
hw, err := newHistoryWriter(cfg)
if err != nil {
t.Fatalf("newHistoryWriter: %v", err)
}
hw.onSourceConfigured("src", sigs)
ts, vs := ramp(0, 1, 400)
hw.write("src:sig", ts, vs)
hw.close()
hw2, err := newHistoryWriter(cfg)
if err != nil {
t.Fatalf("reopen: %v", err)
}
defer hw2.close()
hw2.onSourceConfigured("src", sigs)
hf := hw2.files["src:sig"]
if hf.count != 400 || hf.head != 400 {
t.Fatalf("reopened count=%d head=%d, want 400/400", hf.count, hf.head)
}
rt, rv := hw2.readRange("src:sig", 100, 199, 10000)
if len(rt) != 100 || rt[0] != 100 || rv[0] != 100 {
t.Fatalf("reopened read = %d points starting (%v, %v)", len(rt), rt[0], rv[0])
}
// Appending after the reopen must continue where the file left off.
ts2, vs2 := ramp(400, 1, 50)
hw2.write("src:sig", ts2, vs2)
if hf.tNewest != 449 {
t.Fatalf("tNewest after append = %v, want 449", hf.tNewest)
}
}
// A file sized for a different rate cannot be reused, so it must be recreated
// rather than reopened with a mismatched capacity.
func TestHistoryReopenWithDifferentCapacityRecreates(t *testing.T) {
dir := t.TempDir()
cfg := HistoryConfig{Directory: dir, WindowSec: 3600}
hw, _ := newHistoryWriter(cfg)
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 10},
})
firstCap := hw.files["src:sig"].capacity
hw.write("src:sig", []float64{1, 2}, []float64{1, 2})
hw.close()
hw2, _ := newHistoryWriter(cfg)
defer hw2.close()
hw2.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 100}, // 10× the rate
})
hf := hw2.files["src:sig"]
if hf.capacity == firstCap {
t.Fatalf("capacity unchanged at %d despite a 10x rate change", firstCap)
}
if hf.count != 0 {
t.Fatalf("recreated file kept %d samples", hf.count)
}
}
// A corrupt header must not be trusted: the file gets rebuilt instead.
func TestHistoryCorruptHeaderRecreates(t *testing.T) {
dir := t.TempDir()
cfg := HistoryConfig{Directory: dir, WindowSec: 0.36}
sigs := []udpsprotocol.SignalInfo{{Name: "sig", TypeCode: 8, SamplingRate: 1}}
hw, _ := newHistoryWriter(cfg)
hw.onSourceConfigured("src", sigs)
hw.write("src:sig", []float64{1, 2, 3}, []float64{1, 2, 3})
hw.close()
path := filepath.Join(dir, "src", "sig.shist")
f, err := os.OpenFile(path, os.O_RDWR, 0o644)
if err != nil {
t.Fatalf("open: %v", err)
}
if _, err := f.WriteAt([]byte("XXXX"), 0); err != nil { // clobber the magic
t.Fatalf("clobber: %v", err)
}
f.Close()
hw2, _ := newHistoryWriter(cfg)
defer hw2.close()
hw2.onSourceConfigured("src", sigs)
if got := hw2.files["src:sig"].count; got != 0 {
t.Fatalf("count = %d, want a recreated empty file", got)
}
}
// Raising the budget from the UI has to buy resolution: same duration, a
// narrower min/max bucket. Lowering it again must not overrun the new budget.
func TestSetBudgetRebucketsAtTheSameDuration(t *testing.T) {
// 100 s of 100 kSps is 10 M samples, well past either budget.
hw, key := newTestHistory(t, HistoryConfig{
WindowSec: 100, MaxPointsPerSignal: 100_000,
}, 1e5)
before := hw.files[key]
if before.bucket <= 1 {
t.Fatalf("bucket = %d, want the signal enveloped to fit the budget", before.bucket)
}
if got := hw.setBudget(1_000_000); got != 1_000_000 {
t.Fatalf("setBudget = %d, want 1000000", got)
}
after := hw.files[key]
if after == before {
t.Fatal("the file was not re-created")
}
if after.bucket >= before.bucket {
t.Fatalf("bucket %d → %d, want a finer envelope for a 10× budget",
before.bucket, after.bucket)
}
if after.capacity > 1_000_000 {
t.Fatalf("capacity = %d, over the 1 MPts budget", after.capacity)
}
// The point of the envelope: the duration is covered whatever the budget.
if cov := float64(after.capacity) * float64(after.bucket) / 2 / 1e5; cov < 99 {
t.Fatalf("coverage = %.1f s, want ~100 s", cov)
}
if got := hw.setBudget(100_000); got != 100_000 {
t.Fatalf("setBudget back = %d, want 100000", got)
}
if c := hw.files[key].capacity; c > 100_000 {
t.Fatalf("capacity = %d, over the restored 100 kPts budget", c)
}
}
// A budget that leaves a signal's geometry alone must leave its archive alone
// too — re-creating files nobody asked to resize would throw away history.
func TestSetBudgetKeepsUnaffectedFiles(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{
WindowSec: 1, MaxPointsPerSignal: 16 << 20,
}, 1000)
ts, vs := ramp(0, 0.001, 100)
hw.write(key, ts, vs)
hw.setBudget(8 << 20) // still far more than the 1000 points this signal needs
hf := hw.files[key]
if hf.bucket != 1 {
t.Fatalf("bucket = %d, want the slow signal still archived verbatim", hf.bucket)
}
if hf.count != 100 {
t.Fatalf("count = %d, want the 100 archived samples kept", hf.count)
}
}
// Time-reference signals are the clock for the others, so archiving them would
// just waste disk.
func TestHistorySkipsTimeSignals(t *testing.T) {
dir := t.TempDir()
hw, _ := newHistoryWriter(HistoryConfig{Directory: dir})
defer hw.close()
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "TimeArray", TypeCode: histTypeCodeUint64, SamplingRate: 1000},
{Name: "data", TypeCode: 8, SamplingRate: 1000},
})
if _, ok := hw.files["src:TimeArray"]; ok {
t.Fatal("uint64 time signal was archived")
}
if _, ok := hw.files["src:data"]; !ok {
t.Fatal("data signal was not archived")
}
}
// A second CONFIG for the same source must not throw away the history already
// collected for signals it re-declares.
func TestHistoryReconfigureKeepsExistingFile(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 0.36}, 1)
hw.write(key, []float64{1, 2, 3}, []float64{1, 2, 3})
before := hw.files[key]
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 1},
{Name: "sig2", TypeCode: 8, SamplingRate: 1},
})
if hw.files[key] != before {
t.Fatal("re-CONFIG replaced the existing signal file")
}
if before.count != 3 {
t.Fatalf("count = %d, want the 3 already written", before.count)
}
if _, ok := hw.files["src:sig2"]; !ok {
t.Fatal("newly declared signal was not opened")
}
}
// The C++ UDPStreamer declares samplingRate=0, so sizing the file on the spot
// would use a guess that is three orders of magnitude out at 1 MSps.
func TestHistoryDefersSignalsWithoutDeclaredRate(t *testing.T) {
dir := t.TempDir()
hw, _ := newHistoryWriter(HistoryConfig{Directory: dir})
defer hw.close()
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "fast", TypeCode: 8, SamplingRate: 0},
{Name: "known", TypeCode: 8, SamplingRate: 100},
})
if _, ok := hw.files["src:fast"]; ok {
t.Fatal("undeclared-rate signal was sized before its rate was measured")
}
if got := hw.pendingKeys(); len(got) != 1 || got[0] != "src:fast" {
t.Fatalf("pendingKeys = %v, want [src:fast]", got)
}
if _, ok := hw.files["src:known"]; !ok {
t.Fatal("declared-rate signal was deferred")
}
// Data for a deferred signal is dropped, not misfiled.
hw.write("src:fast", []float64{1}, []float64{1})
// A repeated CONFIG must not queue it twice.
hw.onSourceConfigured("src", []udpsprotocol.SignalInfo{
{Name: "fast", TypeCode: 8, SamplingRate: 0},
})
if got := hw.pendingKeys(); len(got) != 1 {
t.Fatalf("pendingKeys = %v after re-CONFIG, want one entry", got)
}
if !hw.openPending("src:fast", 100000) {
t.Fatal("openPending refused a measured rate")
}
hf, ok := hw.files["src:fast"]
if !ok {
t.Fatal("file not opened after the rate was measured")
}
// The default window × 100 kSps, enveloped if it does not fit the budget.
wantCap, wantBucket := histCapacityFor(defaultLiveWindowSec, 100000, 1, histDefaultMaxPoints)
if hf.capacity != wantCap || hf.bucket != wantBucket {
t.Fatalf("capacity/bucket = %d/%d, want %d/%d", hf.capacity, hf.bucket, wantCap, wantBucket)
}
if len(hw.pendingKeys()) != 0 {
t.Fatal("signal still pending after being opened")
}
if hw.openPending("src:fast", 100000) {
t.Fatal("openPending reopened an already-open signal")
}
}
func TestOpenPendingHistoryFilesUsesMeasuredRate(t *testing.T) {
h := NewHub()
if err := h.EnableHistory(HistoryConfig{Directory: t.TempDir(), WindowSec: 3.6}); err != nil {
t.Fatalf("EnableHistory: %v", err)
}
defer h.CloseHistory()
h.hist.onSourceConfigured("s1", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 0},
})
rb := newSigRing(200000)
h.rings["s1:sig"] = rb
// Too little data to measure a rate from: the sweep must wait rather than
// size the file from a burst.
fillRing(rb, 0, 100000, 100) // 1 ms of data
h.openPendingHistoryFiles(100)
if len(h.hist.pendingKeys()) != 1 {
t.Fatal("sweep sized the file from a sub-millisecond sample")
}
fillRing(rb, 0, 100000, 100000) // 1 s at 100 kSps
h.openPendingHistoryFiles(200)
hf, ok := h.hist.files["s1:sig"]
if !ok {
t.Fatal("file not opened once the rate was measurable")
}
// 3.6 s at ~100 kSps, plus headroom, ≈ 450 000 pairs; a fixed 1 kHz guess
// would have produced the 1000-sample floor instead.
if hf.capacity < 400_000 || hf.capacity > 500_000 {
t.Fatalf("capacity = %d, want ~450000 from the measured 100 kSps", hf.capacity)
}
}
func TestOpenPendingHistoryFilesIsThrottled(t *testing.T) {
h := NewHub()
if err := h.EnableHistory(HistoryConfig{Directory: t.TempDir()}); err != nil {
t.Fatalf("EnableHistory: %v", err)
}
defer h.CloseHistory()
h.hist.onSourceConfigured("s1", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 0},
})
h.openPendingHistoryFiles(100) // no ring yet: nothing to measure
rb := newSigRing(20000)
fillRing(rb, 0, 1000, 20000)
h.rings["s1:sig"] = rb
h.openPendingHistoryFiles(100.5)
if len(h.hist.files) != 0 {
t.Fatal("sweep ran inside the throttle window")
}
h.openPendingHistoryFiles(200)
if len(h.hist.files) != 1 {
t.Fatal("sweep did not run after the throttle window elapsed")
}
}
// A hub without history must tolerate the sweep, since Run() calls it every tick.
func TestOpenPendingHistoryFilesNoopWithoutHistory(t *testing.T) {
h := NewHub()
h.openPendingHistoryFiles(100)
}
func TestHistoryInfoShape(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 0.36}, 1)
// Reported before any data arrives, so clients can enable their history UI.
inf := hw.info()
if e, ok := inf[key]; !ok || e.Count != 0 || e.Capacity != histMinCapacity {
t.Fatalf("pre-data info = %+v (present=%v)", inf[key], ok)
}
ts, vs := ramp(5, 1, 10)
hw.write(key, ts, vs)
e := hw.info()[key]
if e.Count != 10 || e.T0 != 5 || e.T1 != 14 {
t.Fatalf("info = %+v, want count=10 t0=5 t1=14", e)
}
}
func TestHistoryHeaderIsPersistedOnFlush(t *testing.T) {
dir := t.TempDir()
hw, key := newTestHistory(t, HistoryConfig{Directory: dir, WindowSec: 0.36, Decimation: 2}, 1)
ts, vs := ramp(0, 1, 20)
hw.write(key, ts, vs)
hw.flushHeaders()
hdr, err := os.ReadFile(filepath.Join(dir, "src", "sig.shist"))
if err != nil {
t.Fatalf("read: %v", err)
}
if string(hdr[0:4]) != "SHR1" {
t.Fatalf("magic = %q", hdr[0:4])
}
if v := binary.LittleEndian.Uint32(hdr[4:]); v != histVersion {
t.Fatalf("version = %d, want %d", v, histVersion)
}
if c := binary.LittleEndian.Uint32(hdr[8:]); c != histMinCapacity {
t.Fatalf("capacity = %d, want %d", c, histMinCapacity)
}
if h := binary.LittleEndian.Uint32(hdr[12:]); h != 10 {
t.Fatalf("head = %d, want 10 (20 samples, decimation 2)", h)
}
if n := binary.LittleEndian.Uint32(hdr[16:]); n != 10 {
t.Fatalf("count = %d, want 10", n)
}
if d := binary.LittleEndian.Uint32(hdr[20:]); d != 2 {
t.Fatalf("decimation = %d, want 2", d)
}
if got := math.Float64frombits(binary.LittleEndian.Uint64(hdr[32:])); got != 19 {
t.Fatalf("tNewest = %v, want 19", got)
}
// The data region must be pre-allocated in full, not grown as it fills.
if want := int64(histHeaderSize) + histMinCapacity*histPairSize; int64(len(hdr)) != want {
t.Fatalf("file size = %d, want the pre-allocated %d", len(hdr), want)
}
}
func TestSanitizeHistName(t *testing.T) {
cases := map[string]string{
"Signal_1": "Signal_1",
"GAM.Out[0]": "GAM.Out[0]",
"a/b": "a_b",
"../../etc/pass": ".._.._etc_pass",
"": "_",
".": "_",
"..": "_",
"with space": "with_space",
"nul\x00byte": "nul_byte",
}
for in, want := range cases {
if got := sanitizeHistName(in); got != want {
t.Errorf("sanitizeHistName(%q) = %q, want %q", in, got, want)
}
}
}
// A producer-supplied name must never place a file outside the history dir.
func TestHistoryNameCannotEscapeDirectory(t *testing.T) {
dir := t.TempDir()
hw, _ := newHistoryWriter(HistoryConfig{Directory: dir})
defer hw.close()
hw.onSourceConfigured("../evil", []udpsprotocol.SignalInfo{
{Name: "../../pwned", TypeCode: 8, SamplingRate: 1},
})
found := false
err := filepath.Walk(dir, func(p string, info os.FileInfo, err error) error {
if err == nil && !info.IsDir() {
found = true
}
return err
})
if err != nil {
t.Fatalf("walk: %v", err)
}
if !found {
t.Fatal("no file created inside the history directory")
}
if _, err := os.Stat(filepath.Join(dir, "..", "..", "pwned.shist")); err == nil {
t.Fatal("a file escaped the history directory")
}
}
func TestHistCapacityFor(t *testing.T) {
cases := []struct {
window float64
rate float64
decim int
maxPts int
want uint32
wantBucket int
}{
// window × rate / decimation, plus the 1.25 headroom.
{600, 1000, 1, 0, 750_000, 1},
{600, 1000, 10, 0, 75_000, 1},
{10, 100, 1, 0, 1250, 1},
{600, 0.001, 1, 0, histMinCapacity, 1}, // absurdly slow → the floor
// Absurdly fast: bounded by histMaxCapacity, and the window is bought with
// a correspondingly absurd bucket rather than by storing less of it.
{600, 1e9, 1, 0, 1_073_729_421, 1397},
{math.NaN(), 1000, 1, 0, histMinCapacity, 1},
{600, math.NaN(), 1, 0, histMinCapacity, 1},
// A budget envelopes a fast signal without touching a slow one, and the
// window is kept either way.
{600, 1e6, 1, 16 << 20, 16_666_667, 90},
{600, 1000, 1, 16 << 20, 750_000, 1},
}
for _, c := range cases {
got, bucket := histCapacityFor(c.window, c.rate, c.decim, c.maxPts)
if got != c.want || bucket != c.wantBucket {
t.Errorf("histCapacityFor(%v, %v, %d, %d) = %d/%d, want %d/%d",
c.window, c.rate, c.decim, c.maxPts, got, bucket, c.want, c.wantBucket)
}
}
}
func TestHistoryConfigDefaults(t *testing.T) {
c := HistoryConfig{}.withDefaults()
if c.WindowSec != defaultLiveWindowSec || c.Decimation != 1 || c.FlushIntervalSec != 5 || c.MinDiskFreeMB != 500 {
t.Fatalf("defaults = %+v", c)
}
// A negative value is the explicit "no disk guard", so it must survive
// defaulting rather than being turned back into 500.
if got := (HistoryConfig{MinDiskFreeMB: -1}).withDefaults().MinDiskFreeMB; got != -1 {
t.Fatalf("MinDiskFreeMB = %d, want the -1 that disables the guard", got)
}
}
func TestHistoryWritePausedWhenDiskLow(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{}, 100)
hw.diskLow = true
hw.write(key, []float64{1, 2, 3}, []float64{1, 2, 3})
if hw.files[key].count != 0 {
t.Fatalf("count = %d, want 0 while the disk guard is tripped", hw.files[key].count)
}
hw.diskLow = false
hw.write(key, []float64{1, 2, 3}, []float64{1, 2, 3})
if hw.files[key].count != 3 {
t.Fatalf("count = %d, want 3 once writing resumes", hw.files[key].count)
}
}
func TestHistoryReadRangeRespectsMaxOut(t *testing.T) {
// A window wide enough that the whole ramp is still on disk when it is read.
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 50}, 100)
ts, vs := ramp(0, 0.01, 5000)
hw.write(key, ts, vs)
rt, rv := hw.readRange(key, -1, 1e9, 100)
if len(rt) != 100 || len(rv) != 100 {
t.Fatalf("read %d/%d points, want the 100 cap", len(rt), len(rv))
}
}
func TestHistoryReadRangeSpansWholeRange(t *testing.T) {
// A capped read must thin the range out, not return its first maxOut
// samples: a client asking for 100 points over 50 s and getting the first
// second of it draws a flat line and falls back to its coarse copy.
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 50}, 100)
ts, vs := ramp(0, 0.01, 5000)
hw.write(key, ts, vs)
rt, _ := hw.readRange(key, 0, 49.99, 100)
if len(rt) == 0 {
t.Fatal("no points read")
}
if got := rt[len(rt)-1] - rt[0]; got < 0.95*49.99 {
t.Fatalf("read spans %.2f s of the 49.99 s asked; a capped read must "+
"cover the whole range", got)
}
}
func TestHistoryReadRangeUncappedIsExact(t *testing.T) {
// Below the cap every sample in the range comes back, so a zoom deep enough
// to fit is served at full resolution.
hw, key := newTestHistory(t, HistoryConfig{WindowSec: 50}, 100)
ts, vs := ramp(0, 0.01, 5000)
hw.write(key, ts, vs)
rt, rv := hw.readRange(key, 1, 1.99, 1000)
if len(rt) != 100 {
t.Fatalf("read %d points, want the 100 samples in [1, 1.99]", len(rt))
}
if rv[0] != 100 || rv[len(rv)-1] != 199 {
t.Fatalf("values %.0f..%.0f, want 100..199", rv[0], rv[len(rv)-1])
}
}
// The capture copy is what makes a trigger window zoomable long after the
// circular archive has wrapped over it.
func TestCaptureRangeOutlivesTheArchive(t *testing.T) {
hw, key := newTestHistory(t, HistoryConfig{}, 0.001) // floor capacity: 1000
hf := hw.files[key]
if hf.capacity != histMinCapacity {
t.Fatalf("capacity = %d, want the %d floor", hf.capacity, histMinCapacity)
}
ts, vs := ramp(0, 1, 1000) // t = 0..999, exactly full
hw.write(key, ts, vs)
hw.captureRange(500, 600)
// Wrap the archive right over the captured window.
ts2, vs2 := ramp(1000, 1, 1000)
hw.write(key, ts2, vs2)
if hf.tOldest != 1000 || hf.tNewest != 1999 {
t.Fatalf("archive holds [%v, %v], want [1000, 1999]: capturing must not "+
"stop or divert the archive", hf.tOldest, hf.tNewest)
}
rt, rv := hw.readRange(key, 500, 600, 1000)
if len(rt) != 101 {
t.Fatalf("read %d captured samples in [500, 600], want 101", len(rt))
}
if rv[0] != 500 || rv[len(rv)-1] != 600 {
t.Fatalf("captured values %.0f..%.0f, want 500..600", rv[0], rv[len(rv)-1])
}
// A range the capture does not hold is still answered by the archive.
if at, _ := hw.readRange(key, 1500, 1600, 1000); len(at) != 101 {
t.Fatalf("read %d archived samples in [1500, 1600], want 101", len(at))
}
// The next capture replaces the last one, and only then.
hw.captureRange(1500, 1600)
if ct, _ := hw.readRange(key, 500, 600, 1000); len(ct) != 0 {
t.Fatalf("read %d samples of a replaced capture, want 0", len(ct))
}
}
// Delivering a capture copies its window out of the archive, and the archive
// keeps rolling so the next capture's pre-trigger window is there when it fires.
func TestTriggerCaptureCopiesWindowToDisk(t *testing.T) {
h := NewHub()
if err := h.EnableHistory(HistoryConfig{
Directory: t.TempDir(), WindowSec: 36, MinDiskFreeMB: -1,
}); err != nil {
t.Fatalf("EnableHistory: %v", err)
}
t.Cleanup(h.CloseHistory)
h.hist.onSourceConfigured("s1", []udpsprotocol.SignalInfo{
{Name: "sig", TypeCode: 8, SamplingRate: 1000},
})
h.rings["s1:sig"] = newSigRing(10000)
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", edge: "rising", threshold: 0,
windowSec: 1, prePercent: 20, mode: "single"})
h.trigger.Arm()
// Cross the threshold, then cover the post-trigger window so the capture
// comes due on the next tick.
h.ingest("s1:sig", 1, []float64{5.0, 5.001}, []float64{-1, 1})
h.ingest("s1:sig", 1, []float64{6.0}, []float64{1})
h.triggerTick()
if h.trigger.State() != trigTriggered {
t.Fatalf("state = %q, want triggered", h.trigger.State())
}
cf := h.hist.captures["s1:sig"]
if cf == nil {
t.Fatal("capture delivered but its window was not copied to disk")
}
// The window is [trigTime-0.2, trigTime+0.8] around the 5.001 crossing, so
// the sample at 6.0 falls outside it.
if cf.count != 2 || cf.tOldest != 5.0 || cf.tNewest != 5.001 {
t.Fatalf("capture holds %d samples in [%v, %v], want 2 in [5, 5.001]",
cf.count, cf.tOldest, cf.tNewest)
}
// Copying the window leaves the archive rolling, so the next capture's
// pre-trigger window — written before its trigger fires — is there for it.
h.ingest("s1:sig", 1, []float64{7.0}, []float64{1})
if got := h.hist.files["s1:sig"].count; got != 4 {
t.Fatalf("archived %d samples, want 4: capturing must not stop writing", got)
}
// Rearming does not discard the capture: it stays on screen until the next
// trigger replaces it.
h.trigger.Arm()
h.triggerTick()
if h.hist.captures["s1:sig"] != cf {
t.Fatal("rearming discarded the capture the client is still showing")
}
}
func TestHistSearch(t *testing.T) {
vals := []float64{0, 1, 2, 3, 4, 5}
at := func(i uint32) float64 { return vals[i] }
if got := histSearch(0, 6, func(i uint32) bool { return at(i) < 3 }); got != 3 {
t.Fatalf("lower bound = %d, want 3", got)
}
if got := histSearch(0, 6, func(i uint32) bool { return at(i) <= 3 }); got != 4 {
t.Fatalf("upper bound = %d, want 4", got)
}
if got := histSearch(0, 6, func(i uint32) bool { return at(i) < -1 }); got != 0 {
t.Fatalf("all-false = %d, want 0", got)
}
if got := histSearch(0, 6, func(i uint32) bool { return at(i) < 100 }); got != 6 {
t.Fatalf("all-true = %d, want 6", got)
}
}
Common/Client/go/wshub/hub.go
+514 -138
View File
@@ -9,6 +9,7 @@ import (
"strconv" "strconv"
"strings" "strings"
"sync" "sync"
"sync/atomic"
"time" "time"
"unsafe" "unsafe"
@@ -27,6 +28,29 @@ type wsClient struct {
hub *Hub hub *Hub
conn *websocket.Conn conn *websocket.Conn
send chan wsMessage send chan wsMessage
// window is the timespan this client is displaying, in seconds, held as
// float64 bits. The retune sweep sizes the rings from the widest window in
// use, so it must be readable from the hub goroutine while readPump writes
// it. Zero means the client has not said, and the default applies.
window atomic.Uint64
}
func (c *wsClient) setDisplayWindowSec(s float64) {
c.window.Store(math.Float64bits(s))
}
func (c *wsClient) displayWindowSec() float64 {
return math.Float64frombits(c.window.Load())
}
// sendText enqueues one JSON frame for this client, dropping it if the client
// is not draining its queue.
func (c *wsClient) sendText(msg []byte) {
select {
case c.send <- wsMessage{websocket.TextMessage, msg}:
default:
}
} }
func (c *wsClient) writePump() { func (c *wsClient) writePump() {
@@ -107,7 +131,49 @@ func (c *wsClient) readPump() {
case c.hub.commandCh <- hubCmd{op: "wsSaveSources"}: case c.hub.commandCh <- hubCmd{op: "wsSaveSources"}:
default: default:
} }
case "setCalibration":
source, _ := env["source"].(string)
signal, _ := env["signal"].(string)
scale, hasScale := env["scale"].(float64)
if !hasScale {
scale = 1
}
offset, _ := env["offset"].(float64)
unit, _ := env["unit"].(string)
select {
case c.hub.commandCh <- hubCmd{op: "wsSetCalibration", cal: CalConfig{
Source: source, Signal: signal,
Scale: scale, Offset: offset, Unit: unit,
}}:
default:
}
case "reloadConfig":
select {
case c.hub.commandCh <- hubCmd{op: "wsReloadConfig"}:
default:
}
case "setWindow":
// Sizes the zoom rings: the hub cannot know how far back a
// client is plotting, and a window it has not been told
// about is a window the buffers may not reach.
if sec, ok := env["seconds"].(float64); ok && sec > 0 && !math.IsInf(sec, 0) {
c.setDisplayWindowSec(sec)
}
case "setMonotonic":
enabled, _ := env["enabled"].(bool)
select {
case c.hub.commandCh <- hubCmd{op: "setMonotonic", enabled: enabled}:
default:
}
case "zoom":
c.hub.handleWSZoom(c, env)
default: default:
if c.hub.handleTriggerCommand(t, env) {
break
}
if c.hub.handleHistoryCommand(c, t, env) {
break
}
// Unrecognized message type — forward to DebugCh // Unrecognized message type — forward to DebugCh
select { select {
case c.hub.DebugCh <- msg: case c.hub.DebugCh <- msg:
@@ -182,6 +248,14 @@ type sourceHubState struct {
// per signal name. Used by the default (TimeModePacket, n>1) path to estimate // per signal name. Used by the default (TimeModePacket, n>1) path to estimate
// per-element dt when only one packet arrives in a 30 Hz tick. // per-element dt when only one packet arrives in a 30 Hz tick.
lastPktNs map[string]int64 lastPktNs map[string]int64
// Monotonic timestamp snapping state (all accessed from Run() goroutine):
// lastFrameMeasured — uncorrected measured anchor of the previous frame.
// lastFrameEndT — corrected anchor after snapping.
// gapEMA — exponential moving average of the measured inter-frame gap.
lastFrameMeasured map[string]float64
lastFrameEndT map[string]float64
gapEMA map[string]float64
} }
// taggedSample is a DataSample annotated with its source ID. // taggedSample is a DataSample annotated with its source ID.
@@ -192,8 +266,9 @@ type taggedSample struct {
// hubCmd carries a command to the Run() goroutine. // hubCmd carries a command to the Run() goroutine.
type hubCmd struct { type hubCmd struct {
op string // "addSource","removeSource","setSourceState","updateConfig", op string // "addSource","removeSource","setSourceState","updateConfig",
// "wsAddSource","wsRemoveSource","wsSaveSources" // "wsAddSource","wsRemoveSource","wsSaveSources",
// "wsSetCalibration","wsReloadConfig"
sourceID string sourceID string
label string label string
addr string addr string
@@ -201,6 +276,8 @@ type hubCmd struct {
sigs []udpsprotocol.SignalInfo sigs []udpsprotocol.SignalInfo
multicastGroup string multicastGroup string
dataPort int dataPort int
enabled bool // "setMonotonic" toggle
cal CalConfig // "wsSetCalibration" payload
} }
// Hub is the central broker between UDP clients and WebSocket clients. // Hub is the central broker between UDP clients and WebSocket clients.
@@ -219,26 +296,42 @@ type Hub struct {
sm *SourceManager // set after construction; used for WS-initiated source changes sm *SourceManager // set after construction; used for WS-initiated source changes
// cal holds the per-signal calibration table. It is metadata only: the
// rings, the history and the trigger comparator all keep raw samples.
cal *calTable
// Ring buffers for hi-res zoom data. // Ring buffers for hi-res zoom data.
// ringsMu protects the map structure; each sigRing has its own RWMutex for data. // ringsMu protects the map structure; each sigRing has its own RWMutex for data.
ringsMu sync.RWMutex ringsMu sync.RWMutex
rings map[string]*sigRing // "sourceId:signalKey" → ring rings map[string]*sigRing // "sourceId:signalKey" → ring
// lastZoomAt tracks the last time a zoom request was served. // hist is the disk-backed archive behind long time windows, which hold far
// Ring buffer writes are skipped when no zoom has been requested // more samples than the in-memory rings can. nil when history is disabled.
// in the last 10 s, saving substantial CPU on LTTB + ring writes. // histOpenAt throttles the sweep that opens the files of signals whose
lastZoomAt time.Time // producer declared no sampling rate; both are touched only from Run().
zoomAtMu sync.Mutex hist *historyWriter
histOpenAt float64
statsMu sync.RWMutex statsMu sync.RWMutex
statsMap map[string]*SourceStat statsMap map[string]*SourceStat
// onClientConnect, if set, is called each time a new WebSocket client // trigger is the hub-side trigger FSM driving the oscilloscope capture mode.
// registers. The callback receives a send function that delivers a message // ringTuneAt throttles the sweep that keeps each ring's depth and min/max
// directly to that client. It is invoked synchronously from Run(), so it // bucket matched to the window being displayed; both are touched only from
// must not block. // Run(). ringBudgetPts is that sweep's per-signal budget; set before Run().
trigger *triggerEngine
ringTuneAt float64
// capture is the trigger double buffer's read half: the last delivered
// capture window, kept out of the rings' way so the shot being viewed
// survives the re-arm that immediately follows it.
capture captureHold
ringBudgetPts int
onClientConnectMu sync.RWMutex onClientConnectMu sync.RWMutex
onClientConnect func(send func([]byte)) onClientConnect func(send func([]byte))
// monotonicTS, when true, snaps small inter-frame timestamp deviations
// (< monotonicTolerance) to the ideal gap to eliminate jitter.
monotonicTS bool
} }
// NewHub creates an initialised Hub. // NewHub creates an initialised Hub.
@@ -253,9 +346,49 @@ func NewHub() *Hub {
DebugCh: make(chan []byte, 256), DebugCh: make(chan []byte, 256),
rings: make(map[string]*sigRing), rings: make(map[string]*sigRing),
statsMap: make(map[string]*SourceStat), statsMap: make(map[string]*SourceStat),
trigger: newTriggerEngine(),
cal: newCalTable(),
} }
} }
// SetRingBudget overrides the per-signal in-memory buffer budget, in points.
// Non-positive values restore the default. It must be called before Run().
// Each point costs 16 bytes, so the budget is the memory bound per temporal
// signal. It does not limit how long a window can be held: a window too long
// to fit at full rate is stored as min/max pairs instead (see retuneRings).
func (h *Hub) SetRingBudget(n int) {
if n <= 0 {
n = defaultRingPts
}
if n < ringCapInitial {
n = ringCapInitial
}
h.ringBudgetPts = n
}
func (h *Hub) ringBudget() int {
if h.ringBudgetPts <= 0 {
return defaultRingPts
}
return h.ringBudgetPts
}
// EnableHistory turns on the disk-backed history archive. It must be called
// before Run(). A HistoryConfig with an empty Directory leaves history off.
func (h *Hub) EnableHistory(cfg HistoryConfig) error {
hw, err := newHistoryWriter(cfg)
if err != nil {
return err
}
h.hist = hw
return nil
}
// CloseHistory flushes and closes the history files. Without it the samples
// written since the last periodic flush are on disk but unaccounted for in the
// file headers, so a restart would not see them.
func (h *Hub) CloseHistory() { h.hist.close() }
// SetOnClientConnect registers a callback invoked synchronously (from Run()) // SetOnClientConnect registers a callback invoked synchronously (from Run())
// each time a new WebSocket client connects. The callback receives a send // each time a new WebSocket client connects. The callback receives a send
// function that enqueues one message to that specific client. // function that enqueues one message to that specific client.
@@ -270,6 +403,22 @@ func (h *Hub) SetSourceManager(sm *SourceManager) {
h.sm = sm h.sm = sm
} }
// ingest routes one batch of full-resolution samples for a signal to every
// consumer that needs them at full rate: the in-memory zoom ring, the disk
// history and the trigger comparator. The live push is decimated separately by
// the caller. The ring and the archive may reduce what they store to fit their
// budget, but they are handed every sample so the reduction sees the extrema.
func (h *Hub) ingest(key string, nElem int, t, v []float64) {
if len(t) == 0 {
return
}
if rb := h.getRing(key); rb != nil {
rb.write(t, v)
}
h.hist.write(key, t, v)
h.trigger.feed(key, nElem, t, v)
}
// getRing returns the ring buffer for a fully-prefixed signal key, or nil. // getRing returns the ring buffer for a fully-prefixed signal key, or nil.
func (h *Hub) getRing(key string) *sigRing { func (h *Hub) getRing(key string) *sigRing {
h.ringsMu.RLock() h.ringsMu.RLock()
@@ -278,44 +427,11 @@ func (h *Hub) getRing(key string) *sigRing {
return rb return rb
} }
// shouldWriteRing returns true if zoom was requested within the last 10 seconds. // zoomSlice extracts [t0, t1] for the named signals, decimating each to at most
func (h *Hub) shouldWriteRing() bool { // n points. A range inside the last trigger capture is served from the held
h.zoomAtMu.Lock() // copy of it, which the re-arming acquisition cannot overwrite; everything else
ok := time.Since(h.lastZoomAt) < 10*time.Second // comes from the live rings.
h.zoomAtMu.Unlock() func (h *Hub) zoomSlice(t0, t1 float64, keys []string, n int) map[string]sigData {
return ok
}
// HandleZoom serves GET /api/zoom?... It also records the access time
// so the ring buffer knows zoom is active and worth populating.
func (h *Hub) HandleZoom(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
t0, err0 := strconv.ParseFloat(q.Get("t0"), 64)
t1, err1 := strconv.ParseFloat(q.Get("t1"), 64)
if err0 != nil || err1 != nil || t1 <= t0 {
http.Error(w, "invalid t0/t1", http.StatusBadRequest)
return
}
var n int
if nStr := q.Get("n"); nStr == "" {
n = 2400
} else {
n, _ = strconv.Atoi(nStr)
if n <= 0 {
n = 1 << 30 // no decimation
} else if n < 10 {
n = 2400
}
}
if n > 0 {
h.zoomAtMu.Lock()
h.lastZoomAt = time.Now()
h.zoomAtMu.Unlock()
}
keys := strings.Split(q.Get("signals"), ",")
h.ringsMu.RLock() h.ringsMu.RLock()
refs := make(map[string]*sigRing, len(keys)) refs := make(map[string]*sigRing, len(keys))
for _, k := range keys { for _, k := range keys {
@@ -331,18 +447,76 @@ func (h *Hub) HandleZoom(w http.ResponseWriter, r *http.Request) {
result := make(map[string]sigData, len(refs)) result := make(map[string]sigData, len(refs))
for k, rb := range refs { for k, rb := range refs {
rt, rv := rb.slice(t0, t1) rt, rv, ok := h.capture.slice(k, t0, t1)
if !ok {
rt, rv = rb.slice(t0, t1)
}
if len(rt) == 0 { if len(rt) == 0 {
continue continue
} }
dt, dv := lttbDecimate(rt, rv, n) dt, dv := minMaxDecimate(rt, rv, n)
result[k] = sigData{T: dt, V: dv} result[k] = sigData{T: dt, V: dv}
} }
return result
}
// zoomPoints normalises the client's requested point budget: absent → 2400,
// non-positive → every sample in the range, implausibly small → 2400.
func zoomPoints(n int, present bool) int {
switch {
case !present:
return 2400
case n <= 0:
return 1 << 30 // no decimation
case n < 10:
return 2400
}
return n
}
// handleWSZoom answers a browser {"type":"zoom","reqId":..,"t0":..,"t1":..,
// "n":..,"signals":"a,b"} request, unicasting {"type":"zoom","reqId":..,
// "signals":{...}} back to the requesting client. This is the path the web SPA
// actually uses; /api/zoom is the equivalent HTTP entry point.
func (h *Hub) handleWSZoom(c *wsClient, env map[string]interface{}) {
t0, ok0 := env["t0"].(float64)
t1, ok1 := env["t1"].(float64)
if !ok0 || !ok1 || t1 <= t0 {
return
}
nF, nOK := env["n"].(float64)
n := zoomPoints(int(nF), nOK)
sigCSV, _ := env["signals"].(string)
reply, err := json.Marshal(map[string]any{
"type": "zoom",
"reqId": env["reqId"],
"signals": h.zoomSlice(t0, t1, strings.Split(sigCSV, ","), n),
})
if err != nil {
log.Printf("hub: ws zoom encode: %v", err)
return
}
c.sendText(reply)
}
// HandleZoom serves GET /api/zoom?...
func (h *Hub) HandleZoom(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
t0, err0 := strconv.ParseFloat(q.Get("t0"), 64)
t1, err1 := strconv.ParseFloat(q.Get("t1"), 64)
if err0 != nil || err1 != nil || t1 <= t0 {
http.Error(w, "invalid t0/t1", http.StatusBadRequest)
return
}
nStr := q.Get("n")
nVal, _ := strconv.Atoi(nStr)
n := zoomPoints(nVal, nStr != "")
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
if err := json.NewEncoder(w).Encode(map[string]any{ if err := json.NewEncoder(w).Encode(map[string]any{
"type": "zoom", "type": "zoom",
"signals": result, "signals": h.zoomSlice(t0, t1, strings.Split(q.Get("signals"), ","), n),
}); err != nil { }); err != nil {
log.Printf("hub: zoom encode: %v", err) log.Printf("hub: zoom encode: %v", err)
} }
@@ -430,6 +604,26 @@ func buildSourcesMsg(sm map[string]*sourceHubState) []byte {
return msg return msg
} }
// buildCalibrationMsg serialises the calibration table as a "calibration"
// message. It is its own frame rather than a field on "sources" because the
// C++ BroadcastSources serialises into a fixed 4096-byte buffer that a
// calibration table would overflow.
func buildCalibrationMsg(t *calTable) []byte {
list := t.List() // never nil: the SPA replaces its table wholesale on receipt
msg, _ := json.Marshal(map[string]any{"type": "calibration", "cal": list})
return msg
}
// buildConfigAckMsg serialises a configSaved / configReloaded acknowledgement.
func buildConfigAckMsg(msgType, path string, err error) []byte {
m := map[string]any{"type": msgType, "ok": err == nil, "path": path}
if err != nil {
m["error"] = err.Error()
}
msg, _ := json.Marshal(m)
return msg
}
// Run is the hub's main goroutine. Must be started with go hub.Run(). // Run is the hub's main goroutine. Must be started with go hub.Run().
func (h *Hub) Run() { func (h *Hub) Run() {
ticker := time.NewTicker(time.Second / 30) ticker := time.NewTicker(time.Second / 30)
@@ -438,6 +632,16 @@ func (h *Hub) Run() {
statsTicker := time.NewTicker(time.Second) statsTicker := time.NewTicker(time.Second)
defer statsTicker.Stop() defer statsTicker.Stop()
// Header flushes are what make the archived samples findable again; the
// data region is written as it arrives. Ticks are ignored when history is
// off, so a disabled writer costs one no-op call per period.
flushPeriod := time.Duration(5) * time.Second
if h.hist.enabled() {
flushPeriod = time.Duration(h.hist.cfg.FlushIntervalSec) * time.Second
}
flushTicker := time.NewTicker(flushPeriod)
defer flushTicker.Stop()
sourcesMap := make(map[string]*sourceHubState) sourcesMap := make(map[string]*sourceHubState)
var sourcesMsg []byte var sourcesMsg []byte
@@ -455,11 +659,36 @@ func (h *Hub) Run() {
h.clients[c] = true h.clients[c] = true
// Send current state to the new client. // Send current state to the new client.
if sourcesMsg != nil { if sourcesMsg != nil {
select { case c.send <- wsMessage{websocket.TextMessage, sourcesMsg}: default: } select {
case c.send <- wsMessage{websocket.TextMessage, sourcesMsg}:
default:
}
} }
for _, src := range sourcesMap { for _, src := range sourcesMap {
if src.configJS != nil { if src.configJS != nil {
select { case c.send <- wsMessage{websocket.TextMessage, src.configJS}: default: } select {
case c.send <- wsMessage{websocket.TextMessage, src.configJS}:
default:
}
}
}
select {
case c.send <- wsMessage{websocket.TextMessage, h.trigger.stateMsg()}:
default:
}
monoMsg, _ := json.Marshal(map[string]any{"type": "monotonicState", "enabled": h.monotonicTS})
select {
case c.send <- wsMessage{websocket.TextMessage, monoMsg}:
default:
}
calMsg := buildCalibrationMsg(h.cal)
select {
case c.send <- wsMessage{websocket.TextMessage, calMsg}:
default:
}
if h.hist.enabled() {
if msg := h.buildHistoryInfoMsg(); msg != nil {
c.sendText(msg)
} }
} }
// Notify the application layer so it can replay any persistent state // Notify the application layer so it can replay any persistent state
@@ -469,7 +698,10 @@ func (h *Hub) Run() {
h.onClientConnectMu.RUnlock() h.onClientConnectMu.RUnlock()
if fn != nil { if fn != nil {
fn(func(msg []byte) { fn(func(msg []byte) {
select { case c.send <- wsMessage{websocket.TextMessage, msg}: default: } select {
case c.send <- wsMessage{websocket.TextMessage, msg}:
default:
}
}) })
} }
@@ -481,19 +713,25 @@ func (h *Hub) Run() {
case msg := <-h.broadcastCh: case msg := <-h.broadcastCh:
for c := range h.clients { for c := range h.clients {
select { case c.send <- wsMessage{websocket.TextMessage, msg}: default: } select {
case c.send <- wsMessage{websocket.TextMessage, msg}:
default:
}
} }
case cmd := <-h.commandCh: case cmd := <-h.commandCh:
switch cmd.op { switch cmd.op {
case "addSource": case "addSource":
sourcesMap[cmd.sourceID] = &sourceHubState{ sourcesMap[cmd.sourceID] = &sourceHubState{
id: cmd.sourceID, id: cmd.sourceID,
label: cmd.label, label: cmd.label,
addr: cmd.addr, addr: cmd.addr,
connState: "connecting", connState: "connecting",
timeSigCalib: make(map[string]float64), timeSigCalib: make(map[string]float64),
lastPktNs: make(map[string]int64), lastPktNs: make(map[string]int64),
lastFrameEndT: make(map[string]float64),
lastFrameMeasured: make(map[string]float64),
gapEMA: make(map[string]float64),
} }
h.statsMu.Lock() h.statsMu.Lock()
h.statsMap[cmd.sourceID] = &SourceStat{} h.statsMap[cmd.sourceID] = &SourceStat{}
@@ -529,6 +767,7 @@ func (h *Hub) Run() {
} }
src.signals = cmd.sigs src.signals = cmd.sigs
src.configSeq++ src.configSeq++
src.lastFrameEndT = make(map[string]float64)
cfgMsg, err := json.Marshal(map[string]any{ cfgMsg, err := json.Marshal(map[string]any{
"type": "config", "type": "config",
"sourceId": cmd.sourceID, "sourceId": cmd.sourceID,
@@ -552,16 +791,29 @@ func (h *Hub) Run() {
ne := sig.NumElements() ne := sig.NumElements()
isTemporal := ne > 1 && sig.TimeMode != udpsprotocol.TimeModePacket isTemporal := ne > 1 && sig.TimeMode != udpsprotocol.TimeModePacket
if isTemporal { if isTemporal {
h.rings[pfxUpd+sig.Name] = newSigRing(ringCapTemporal) h.rings[pfxUpd+sig.Name] = newSigRing(ringCapInitial)
} else if ne == 1 { } else if ne == 1 {
h.rings[pfxUpd+sig.Name] = newSigRing(ringCapScalar) h.rings[pfxUpd+sig.Name] = newSigRing(ringCapScalar)
} else { } else {
// n>1, TimeModePacket snapshot-waveform: each packet contributes n // n>1, TimeModePacket snapshot-waveform: each packet contributes n
// elements, so use the temporal capacity to hold enough history. // elements, so this is a fast stream too and gets the same budget.
h.rings[pfxUpd+sig.Name] = newSigRing(ringCapTemporal) h.rings[pfxUpd+sig.Name] = newSigRing(ringCapInitial)
} }
} }
h.ringsMu.Unlock() h.ringsMu.Unlock()
// The held capture describes rings that no longer exist. A
// restarted producer can even replay the same timestamps, so
// keeping it would answer zooms with the old run's samples.
h.capture.clear()
// Opening the archive files touches the filesystem, so keep it
// off the Run() goroutine; the write path simply drops samples
// for a key whose file is not open yet.
if h.hist.enabled() {
go func(id string, sigs []udpsprotocol.SignalInfo) {
h.hist.onSourceConfigured(id, sigs)
h.broadcast(h.buildHistoryInfoMsg())
}(cmd.sourceID, cmd.sigs)
}
case "wsAddSource": case "wsAddSource":
if h.sm != nil { if h.sm != nil {
@@ -577,10 +829,48 @@ func (h *Hub) Run() {
case "wsSaveSources": case "wsSaveSources":
if h.sm != nil { if h.sm != nil {
if err := h.sm.Save(); err != nil { // Save writes to disk; run it off the Run() goroutine so a
log.Printf("hub: save sources: %v", err) // slow filesystem can never stall the hub loop.
} go func(sm *SourceManager) {
err := sm.Save()
if err != nil {
log.Printf("hub: save config: %v", err)
}
h.broadcast(buildConfigAckMsg("configSaved", sm.Path(), err))
}(h.sm)
} }
case "wsSetCalibration":
if h.cal.Set(cmd.cal) {
h.broadcast(buildCalibrationMsg(h.cal))
} else {
// No broadcast: the offending client reverts to the last
// value it was sent.
log.Printf("hub: rejected calibration %q/%q (scale=%v offset=%v)",
cmd.cal.Source, cmd.cal.Signal, cmd.cal.Scale, cmd.cal.Offset)
}
case "wsReloadConfig":
if h.sm != nil {
// Reload calls sm.Add(), which sends on commandCh; from the
// Run() goroutine that send would hit the non-blocking
// default and be dropped, so it must run elsewhere.
go func(sm *SourceManager) {
err := sm.Reload()
if err != nil {
log.Printf("hub: reload config: %v", err)
}
h.broadcast(buildConfigAckMsg("configReloaded", sm.Path(), err))
if err == nil {
h.broadcast(buildCalibrationMsg(h.cal))
}
}(h.sm)
}
case "setMonotonic":
h.monotonicTS = cmd.enabled
monoMsg, _ := json.Marshal(map[string]any{"type": "monotonicState", "enabled": h.monotonicTS})
h.broadcast(monoMsg)
} }
case ts := <-h.dataCh: case ts := <-h.dataCh:
@@ -592,10 +882,15 @@ func (h *Hub) Run() {
continue continue
} }
src, ok := sourcesMap[srcID] src, ok := sourcesMap[srcID]
if !ok || len(src.signals) == 0 || len(h.clients) == 0 { if !ok || len(src.signals) == 0 {
pending[srcID] = pending[srcID][:0] pending[srcID] = pending[srcID][:0]
continue continue
} }
// Built even with no clients connected: this is also what feeds
// the rings, the disk history and the trigger, none of which may
// stop just because nobody is watching. It also keeps the push
// cursors advancing, so the first client to connect does not get
// a backlog burst. Matches the C++ StreamHub.
msg := h.buildBinaryDataMessageForSource(src, samples) msg := h.buildBinaryDataMessageForSource(src, samples)
pending[srcID] = pending[srcID][:0] pending[srcID] = pending[srcID][:0]
if msg != nil { if msg != nil {
@@ -607,6 +902,10 @@ func (h *Hub) Run() {
} }
} }
} }
h.triggerTick()
case <-flushTicker.C:
h.hist.flushHeaders()
case <-statsTicker.C: case <-statsTicker.C:
h.statsMu.RLock() h.statsMu.RLock()
@@ -640,53 +939,91 @@ func writeFloat64s(buf []byte, off int, f []float64) int {
// ─── Data serialisation ─────────────────────────────────────────────────────── // ─── Data serialisation ───────────────────────────────────────────────────────
// maxPushPoints bounds the live push only. The zoom rings deliberately store
// every sample: decimating on the way in would cap the resolution a zoom can
// ever recover, and the browser already decimates for display.
const maxPushPoints = 50 const maxPushPoints = 50
const maxRingPoints = 20_000
const ringCapTemporal = 6_000_000 // Ring geometry, in samples per signal (16 bytes each).
//
// defaultRingPts is the per-signal memory budget for temporal (array) signals:
// what the hub may spend keeping one signal available for zoom and for trigger
// captures. 10 M points is 160 MB. The budget buys resolution, not span —
// retuneRings buckets the input so the display window fits whatever the source
// rate is.
//
// ringCapInitial is where a ring starts, so a source that is configured but
// never sends costs nothing; the first retune sweep grows it to the budget.
//
// ringCapScalar sizes scalar signals, which arrive at the packet rate and would
// squander a budget meant for megasample streams.
const defaultRingPts = 10_000_000
const ringCapInitial = 250_000
const ringCapScalar = 100_000 const ringCapScalar = 100_000
// lttbDecimate reduces (tIn, vIn) to at most threshold representative points // monotonicTolerance is the maximum inter-frame timestamp deviation (seconds)
// using the Largest-Triangle-Three-Buckets algorithm. // treated as jitter and snapped to the ideal gap. Larger deviations are
func lttbDecimate(tIn, vIn []float64, threshold int) ([]float64, []float64) { // preserved as genuine discontinuities (missing frames, rate changes).
const monotonicTolerance = 0.005 // 5 ms
// monotonicEMAAlpha is the smoothing factor for the inter-frame gap EMA.
// 0.01 gives a time constant of ~100 frames (~1 s at 100 Hz): fast enough to
// track real rate changes, slow enough to average out per-frame jitter.
const monotonicEMAAlpha = 0.01
// minMaxDecimate reduces (tIn, vIn) to at most threshold points the way an
// oscilloscope draws a trace it cannot show pixel-for-pixel: the range is split
// into threshold/2 equal buckets and each contributes its smallest and largest
// sample, in the order the two occurred.
//
// This is what replaced LTTB on every path here. LTTB picks the sample that
// makes the largest triangle with its neighbours, which reads as a plausible
// shape but silently drops a one-sample spike whenever a smoother neighbour
// scores higher — precisely the sample the user is looking for. The envelope
// cannot drop it: a spike is by definition its bucket's min or max. The cost is
// that a flat trace is drawn as a band rather than a line, which is how a scope
// behaves too.
//
// Both output arrays hold real samples with their real timestamps; nothing is
// interpolated or averaged.
func minMaxDecimate(tIn, vIn []float64, threshold int) ([]float64, []float64) {
n := len(tIn) n := len(tIn)
if n <= threshold || threshold < 3 { // Below four there is no room for a single min/max pair plus endpoints.
if n <= threshold || threshold < 4 {
return tIn, vIn return tIn, vIn
} }
outT := make([]float64, threshold) buckets := threshold / 2
outV := make([]float64, threshold) outT := make([]float64, 0, threshold)
outT[0], outV[0] = tIn[0], vIn[0] outV := make([]float64, 0, threshold)
outT[threshold-1], outV[threshold-1] = tIn[n-1], vIn[n-1] for b := 0; b < buckets; b++ {
lo := b * n / buckets
every := float64(n-2) / float64(threshold-2) hi := (b + 1) * n / buckets
a := 0 if b == buckets-1 {
for i := 0; i < threshold-2; i++ { hi = n
avgS := int(float64(i+1)*every) + 1
avgE := int(float64(i+2)*every) + 1
if avgE > n {
avgE = n
} }
avgT, avgV, cnt := 0.0, 0.0, 0 if lo >= hi {
for j := avgS; j < avgE; j++ { continue
avgT += tIn[j]; avgV += vIn[j]; cnt++
} }
if cnt > 0 { iMin, iMax := lo, lo
avgT /= float64(cnt); avgV /= float64(cnt) for j := lo + 1; j < hi; j++ {
} if vIn[j] < vIn[iMin] {
rS := int(float64(i)*every) + 1 iMin = j
rE := int(float64(i+1)*every) + 1 }
if rE > n { if vIn[j] > vIn[iMax] {
rE = n iMax = j
}
maxArea, next := -1.0, rS
aT, aV := tIn[a], vIn[a]
for j := rS; j < rE; j++ {
area := math.Abs((aT-avgT)*(vIn[j]-aV) - (aT-tIn[j])*(avgV-aV))
if area > maxArea {
maxArea = area; next = j
} }
} }
outT[i+1], outV[i+1] = tIn[next], vIn[next] // Emit in time order so the result plots as one ascending trace.
a = next if iMin > iMax {
iMin, iMax = iMax, iMin
}
outT = append(outT, tIn[iMin])
outV = append(outV, vIn[iMin])
// A bucket whose samples are all equal has one extreme, not two.
if iMax != iMin {
outT = append(outT, tIn[iMax])
outV = append(outV, vIn[iMax])
}
} }
return outT, outV return outT, outV
} }
@@ -710,11 +1047,13 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
if src.configSeq != src.configSeqAtCalib { if src.configSeq != src.configSeqAtCalib {
src.configSeqAtCalib = src.configSeq src.configSeqAtCalib = src.configSeq
src.timeSigCalib = make(map[string]float64) src.timeSigCalib = make(map[string]float64)
src.lastFrameEndT = make(map[string]float64)
src.lastFrameMeasured = make(map[string]float64)
src.gapEMA = make(map[string]float64)
} }
sigs := src.signals sigs := src.signals
pfx := src.id + ":" pfx := src.id + ":"
writeRing := h.shouldWriteRing()
type pairBuf struct { type pairBuf struct {
t, v []float64 t, v []float64
@@ -766,6 +1105,25 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
anchorTime = float64(s.WallTime.UnixNano()) / 1e9 anchorTime = float64(s.WallTime.UnixNano()) / 1e9
anchorIsFirstSample = false anchorIsFirstSample = false
} }
if h.monotonicTS && dt > 0 {
nominalGap := float64(n) * dt
measuredAnchor := anchorTime
if prevMeasured, ok := src.lastFrameMeasured[sig.Name]; ok {
measuredGap := measuredAnchor - prevMeasured
prevEMA, hasEMA := src.gapEMA[sig.Name]
if !hasEMA {
prevEMA = nominalGap
}
src.gapEMA[sig.Name] = prevEMA*(1-monotonicEMAAlpha) + measuredGap*monotonicEMAAlpha
smoothedGap := src.gapEMA[sig.Name]
deviation := math.Abs(measuredGap - smoothedGap)
if deviation > 0 && deviation < monotonicTolerance {
anchorTime = src.lastFrameEndT[sig.Name] + smoothedGap
}
}
src.lastFrameMeasured[sig.Name] = measuredAnchor
src.lastFrameEndT[sig.Name] = anchorTime
}
for k := 0; k < n; k++ { for k := 0; k < n; k++ {
var t float64 var t float64
if anchorIsFirstSample { if anchorIsFirstSample {
@@ -777,13 +1135,8 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
allV = append(allV, vals[k]) allV = append(allV, vals[k])
} }
} }
if writeRing { h.ingest(pfx+sig.Name, n, allT, allV)
ringT, ringV := lttbDecimate(allT, allV, maxRingPoints) decimT, decimV := minMaxDecimate(allT, allV, maxPushPoints)
if rb := h.getRing(pfx + sig.Name); rb != nil {
rb.write(ringT, ringV)
}
}
decimT, decimV := lttbDecimate(allT, allV, maxPushPoints)
pairs[sig.Name] = pairBuf{t: decimT, v: decimV} pairs[sig.Name] = pairBuf{t: decimT, v: decimV}
case sig.TimeMode == udpsprotocol.TimeModeFullArray: case sig.TimeMode == udpsprotocol.TimeModeFullArray:
@@ -825,13 +1178,8 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
allV = append(allV, vals[k]) allV = append(allV, vals[k])
} }
} }
if writeRing { h.ingest(pfx+sig.Name, n, allT, allV)
ringT, ringV := lttbDecimate(allT, allV, maxRingPoints) decimT, decimV := minMaxDecimate(allT, allV, maxPushPoints)
if rb := h.getRing(pfx + sig.Name); rb != nil {
rb.write(ringT, ringV)
}
}
decimT, decimV := lttbDecimate(allT, allV, maxPushPoints)
pairs[sig.Name] = pairBuf{t: decimT, v: decimV} pairs[sig.Name] = pairBuf{t: decimT, v: decimV}
case n == 1: case n == 1:
@@ -845,25 +1193,19 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
ts = append(ts, float64(s.WallTime.UnixNano())/1e9) ts = append(ts, float64(s.WallTime.UnixNano())/1e9)
vs = append(vs, vals[0]) vs = append(vs, vals[0])
} }
if writeRing { h.ingest(pfx+sig.Name, 1, ts, vs)
if rb := h.getRing(pfx + sig.Name); rb != nil {
rb.write(ts, vs)
}
}
pairs[sig.Name] = pairBuf{t: ts, v: vs} pairs[sig.Name] = pairBuf{t: ts, v: vs}
default: default:
// n > 1, TimeModePacket: C++ sends samplingRate=0 so we interpolate // n > 1, TimeModePacket: C++ sends samplingRate=0 so we interpolate
// per-element timestamps from wall-clock differences between packets. // per-element timestamps from wall-clock differences between packets.
// //
// Three fixes vs the naïve approach: // Two fixes vs the naïve approach:
// 1. Use src.lastPktNs[name] for the single-packet case so dt is // 1. Use src.lastPktNs[name] for the single-packet case so dt is
// estimated from the actual inter-packet gap, not 1/n. // estimated from the actual inter-packet gap, not 1/n.
// 2. Send all n elements to the browser without LTTB so sinusoidal // 2. Send all n elements to the browser without LTTB so sinusoidal
// waveforms are not degraded (packets arrive at ≤30 Hz, bandwidth // waveforms are not degraded (packets arrive at ≤30 Hz, bandwidth
// is trivially acceptable). // is trivially acceptable).
// 3. Always write the ring buffer regardless of shouldWriteRing() so
// the first zoom request immediately returns full-resolution data.
allT := make([]float64, 0, len(batch)*n) allT := make([]float64, 0, len(batch)*n)
allV := make([]float64, 0, len(batch)*n) allV := make([]float64, 0, len(batch)*n)
for bi, s := range batch { for bi, s := range batch {
@@ -874,21 +1216,49 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
wallNs := s.WallTime.UnixNano() wallNs := s.WallTime.UnixNano()
wallSec := float64(wallNs) / 1e9 wallSec := float64(wallNs) / 1e9
var dtSec float64 var dtSec float64
// A gap spans the elements of every packet that went missing
// inside it as well as this packet's own, so the divisor has
// to widen with it. Without this a single loss halves the
// apparent rate and the elements overrun into the next
// packet's range. The loss count belongs to the packet the
// gap ends at.
if bi+1 < len(batch) { if bi+1 < len(batch) {
// Two consecutive packets in this tick → exact dt. // Two consecutive packets in this tick → exact dt.
dtSec = (float64(batch[bi+1].WallTime.UnixNano())-float64(wallNs))/1e9/float64(n) span := float64(n) * float64(1+batch[bi+1].Lost)
dtSec = (float64(batch[bi+1].WallTime.UnixNano()) - float64(wallNs)) / 1e9 / span
} else if bi > 0 { } else if bi > 0 {
// Last of multiple packets → use diff from previous. // Last of multiple packets → use diff from previous.
dtSec = (float64(wallNs)-float64(batch[bi-1].WallTime.UnixNano()))/1e9/float64(n) span := float64(n) * float64(1+s.Lost)
dtSec = (float64(wallNs) - float64(batch[bi-1].WallTime.UnixNano())) / 1e9 / span
} else if prevNs, ok2 := src.lastPktNs[sig.Name]; ok2 && prevNs > 0 && wallNs > prevNs { } else if prevNs, ok2 := src.lastPktNs[sig.Name]; ok2 && prevNs > 0 && wallNs > prevNs {
// Single packet this tick → gap from the previous tick's packet. // Single packet this tick → gap from the previous tick's packet.
dtSec = (float64(wallNs)-float64(prevNs))/1e9/float64(n) span := float64(n) * float64(1+s.Lost)
dtSec = (float64(wallNs) - float64(prevNs)) / 1e9 / span
} else { } else {
// Truly first packet ever — inter-packet timing unknown. // Truly first packet ever — inter-packet timing unknown.
// Skip to avoid poisoning the ring with wrongly-spaced timestamps; // Skip to avoid poisoning the ring with wrongly-spaced timestamps;
// lastPktNs will be recorded below so the next packet uses correct dt. // lastPktNs will be recorded below so the next packet uses correct dt.
continue continue
} }
if h.monotonicTS && dtSec > 0 {
nominalGap := float64(n) * dtSec
measuredStart := wallSec
if prevMeasured, ok := src.lastFrameMeasured[sig.Name]; ok {
measuredGap := measuredStart - prevMeasured
prevEMA, hasEMA := src.gapEMA[sig.Name]
if !hasEMA {
prevEMA = nominalGap
}
src.gapEMA[sig.Name] = prevEMA*(1-monotonicEMAAlpha) + measuredGap*monotonicEMAAlpha
smoothedGap := src.gapEMA[sig.Name]
deviation := math.Abs(measuredGap - smoothedGap)
if deviation > 0 && deviation < monotonicTolerance {
wallSec = src.lastFrameEndT[sig.Name] + smoothedGap
}
}
src.lastFrameMeasured[sig.Name] = measuredStart
src.lastFrameEndT[sig.Name] = wallSec
}
for j := 0; j < n; j++ { for j := 0; j < n; j++ {
allT = append(allT, wallSec+float64(j)*dtSec) allT = append(allT, wallSec+float64(j)*dtSec)
allV = append(allV, vals[j]) allV = append(allV, vals[j])
@@ -898,13 +1268,19 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
src.lastPktNs[sig.Name] = batch[len(batch)-1].WallTime.UnixNano() src.lastPktNs[sig.Name] = batch[len(batch)-1].WallTime.UnixNano()
} }
if len(allT) > 0 { if len(allT) > 0 {
// Ring: always populate (fix 3), LTTB only if it actually reduces size. h.ingest(pfx+sig.Name, n, allT, allV)
ringT, ringV := lttbDecimate(allT, allV, maxRingPoints) // Live push: never below one packet's worth of elements, or LTTB
if rb := h.getRing(pfx + sig.Name); rb != nil { // would flatten the snapshot waveform itself; never above it
rb.write(ringT, ringV) // either, since anything more is just packets that piled up
// during the tick. Pushing every point unconditionally does not
// survive a fast producer: a 5 kHz x 1000-element array is 5M
// points/s on the wire and the client queue never drains.
thr := maxPushPoints
if n > thr {
thr = n
} }
// Live push: send all points without LTTB (fix 2). decimT, decimV := minMaxDecimate(allT, allV, thr)
pairs[sig.Name] = pairBuf{t: allT, v: allV} pairs[sig.Name] = pairBuf{t: decimT, v: decimV}
} }
} }
} }
Common/Client/go/wshub/hub_calibration_test.go
+132
View File
@@ -0,0 +1,132 @@
package wshub
import (
"encoding/json"
"errors"
"testing"
"time"
)
func TestBuildCalibrationMsg(t *testing.T) {
tab := newCalTable()
tab.Set(CalConfig{Source: "wave", Signal: "Adc", Scale: 0.5, Offset: -1.25, Unit: "V"})
var got struct {
Type string `json:"type"`
Cal []CalConfig `json:"cal"`
}
raw := buildCalibrationMsg(tab)
if err := json.Unmarshal(raw, &got); err != nil {
t.Fatalf("unmarshal %s: %v", raw, err)
}
if got.Type != "calibration" {
t.Errorf("type = %q, want calibration", got.Type)
}
if len(got.Cal) != 1 || got.Cal[0] != (CalConfig{
Source: "wave", Signal: "Adc", Scale: 0.5, Offset: -1.25, Unit: "V"}) {
t.Errorf("cal = %+v", got.Cal)
}
}
func TestBuildCalibrationMsgEmptyTableIsEmptyArray(t *testing.T) {
// The SPA replaces its table wholesale on every calibration message, so an
// empty table must serialise as [] and not as null.
raw := buildCalibrationMsg(newCalTable())
var got struct {
Cal []CalConfig `json:"cal"`
}
if err := json.Unmarshal(raw, &got); err != nil {
t.Fatalf("unmarshal %s: %v", raw, err)
}
if got.Cal == nil {
t.Errorf("cal = null, want []; raw = %s", raw)
}
}
func TestBuildConfigAckMsg(t *testing.T) {
ok := buildConfigAckMsg("configSaved", "/tmp/x.json", nil)
var m map[string]any
if err := json.Unmarshal(ok, &m); err != nil {
t.Fatal(err)
}
if m["type"] != "configSaved" || m["ok"] != true || m["path"] != "/tmp/x.json" {
t.Errorf("success ack = %s", ok)
}
if _, has := m["error"]; has {
t.Errorf("success ack carries an error field: %s", ok)
}
bad := buildConfigAckMsg("configReloaded", "", errors.New("boom"))
m = nil
if err := json.Unmarshal(bad, &m); err != nil {
t.Fatal(err)
}
if m["type"] != "configReloaded" || m["ok"] != false || m["error"] != "boom" {
t.Errorf("failure ack = %s", bad)
}
}
func TestHubSetCalibrationCommand(t *testing.T) {
h := NewHub()
go h.Run()
// Register a client before sending commands so broadcasts are observable.
sendCh := make(chan wsMessage, 64)
c := &wsClient{hub: h, send: sendCh}
h.register <- c
sleepMillis(20) // let Run() process the register and flush initial state msgs
drainSendCh(sendCh) // discard state-sync messages (sources, trigger, cal, ...)
h.commandCh <- hubCmd{op: "wsSetCalibration", cal: CalConfig{
Source: "wave", Signal: "Adc", Scale: 4, Offset: 1, Unit: "V"}}
if raw := waitMsg(t, sendCh, "calibration"); raw == nil {
t.Fatal("no calibration broadcast after a valid setCalibration")
}
if got := h.cal.List(); len(got) != 1 || got[0].Scale != 4 {
t.Fatalf("table = %+v, want one entry with scale 4", got)
}
// An invalid entry is rejected and emits no broadcast at all.
h.commandCh <- hubCmd{op: "wsSetCalibration", cal: CalConfig{
Source: "wave", Signal: "Adc", Scale: 0}}
if raw := waitMsg(t, sendCh, "calibration"); raw != nil {
t.Errorf("invalid setCalibration broadcast %s", raw)
}
if got := h.cal.List(); len(got) != 1 || got[0].Scale != 4 {
t.Errorf("table changed after a rejected setCalibration: %+v", got)
}
h.unregister <- c
}
// drainSendCh reads all currently buffered messages from the channel.
func drainSendCh(ch chan wsMessage) {
for {
select {
case <-ch:
default:
return
}
}
}
// waitMsg waits up to ~250 ms for a message of the given type on sendCh.
func waitMsg(t *testing.T, sendCh chan wsMessage, msgType string) []byte {
t.Helper()
deadline := time.After(250 * time.Millisecond)
for {
select {
case msg := <-sendCh:
var env struct {
Type string `json:"type"`
}
if json.Unmarshal(msg.data, &env) == nil && env.Type == msgType {
return msg.data
}
case <-deadline:
return nil
}
}
}
func sleepMillis(n int) { time.Sleep(time.Duration(n) * time.Millisecond) }
Common/Client/go/wshub/multicast_iface_test.go
+116
View File
@@ -0,0 +1,116 @@
//go:build linux
package wshub
import (
"net"
"syscall"
"testing"
"time"
)
// setMulticastIf pins a socket's outgoing multicast interface (IP_MULTICAST_IF),
// which is exactly what UDPStreamer/UDPSServer does with its `Interface` key.
func setMulticastIf(t *testing.T, conn *net.UDPConn, ip [4]byte) {
t.Helper()
rc, err := conn.SyscallConn()
if err != nil {
t.Fatalf("SyscallConn: %v", err)
}
var sockErr error
if err := rc.Control(func(fd uintptr) {
sockErr = syscall.SetsockoptInet4Addr(int(fd), syscall.IPPROTO_IP, syscall.IP_MULTICAST_IF, ip)
}); err != nil {
t.Fatalf("Control: %v", err)
}
if sockErr != nil {
t.Fatalf("IP_MULTICAST_IF: %v", sockErr)
}
}
func TestInterfaceForIPResolvesLoopback(t *testing.T) {
ifi := interfaceForIP(net.ParseIP("127.0.0.1"))
if ifi == nil {
t.Fatal("no interface resolved for 127.0.0.1")
}
if ifi.Flags&net.FlagLoopback == 0 {
t.Fatalf("resolved %q for 127.0.0.1, which is not a loopback interface", ifi.Name)
}
}
func TestInterfaceForIPUnknownAddressIsNil(t *testing.T) {
// Unspecified and unassigned addresses must fall back to "let the kernel
// choose" rather than resolving to an arbitrary interface.
if ifi := interfaceForIP(net.IPv4zero); ifi != nil {
t.Fatalf("0.0.0.0 resolved to %q, want nil", ifi.Name)
}
if ifi := interfaceForIP(nil); ifi != nil {
t.Fatalf("nil IP resolved to %q, want nil", ifi.Name)
}
if ifi := interfaceForIP(net.ParseIP("203.0.113.42")); ifi != nil {
t.Fatalf("unassigned address resolved to %q, want nil", ifi.Name)
}
}
// TestMulticastJoinOnControlInterfaceReceivesData is the regression test for the
// bug that left the web UI permanently blank: the hub joined the group with a
// nil interface, so imr_interface stayed INADDR_ANY and the kernel picked the
// default-route interface. A UDPStreamer configured with Interface = "127.0.0.1"
// sends out the loopback instead, and every datagram was silently dropped.
//
// The sender here mimics that server exactly (IP_MULTICAST_IF = 127.0.0.1); the
// receiver joins the way runMulticastSession now does, via the interface that
// owns the control connection's local address.
func TestMulticastJoinOnControlInterfaceReceivesData(t *testing.T) {
const group = "239.255.13.37"
ifi := interfaceForIP(net.ParseIP("127.0.0.1"))
if ifi == nil {
t.Skip("no loopback interface available")
}
rx, err := net.ListenMulticastUDP("udp4", ifi, &net.UDPAddr{IP: net.ParseIP(group), Port: 0})
if err != nil {
t.Fatalf("join %s on %s: %v", group, ifi.Name, err)
}
defer rx.Close()
port := rx.LocalAddr().(*net.UDPAddr).Port
tx, err := net.ListenUDP("udp4", &net.UDPAddr{IP: net.ParseIP("127.0.0.1")})
if err != nil {
t.Fatalf("sender socket: %v", err)
}
defer tx.Close()
setMulticastIf(t, tx, [4]byte{127, 0, 0, 1})
payload := []byte("UDPS-multicast-probe")
dst := &net.UDPAddr{IP: net.ParseIP(group), Port: port}
// Datagrams are lossy even on loopback if the join has not settled, so send
// a few and accept the first that lands.
done := make(chan struct{})
defer close(done)
go func() {
for {
select {
case <-done:
return
default:
}
tx.WriteToUDP(payload, dst)
time.Sleep(20 * time.Millisecond)
}
}()
buf := make([]byte, 128)
if err := rx.SetReadDeadline(time.Now().Add(3 * time.Second)); err != nil {
t.Fatal(err)
}
n, _, err := rx.ReadFromUDP(buf)
if err != nil {
t.Fatalf("no multicast received on %s within 3s: %v", ifi.Name, err)
}
if got := string(buf[:n]); got != string(payload) {
t.Fatalf("payload = %q, want %q", got, payload)
}
}
Common/Client/go/wshub/packet_dt_loss_test.go
+139
View File
@@ -0,0 +1,139 @@
package wshub
import (
"math"
"testing"
"time"
"marte2/common/udpsprotocol"
)
// pktSignal is a 4-element array with no time signal and no declared sampling
// rate, i.e. the TimeModePacket path where dt has to be inferred from the gap
// between packets.
func pktSignal(name string) udpsprotocol.SignalInfo {
return udpsprotocol.SignalInfo{
Name: name,
TypeCode: 8, // float64
NumDimensions: 1,
NumRows: 4,
NumCols: 1,
TimeMode: udpsprotocol.TimeModePacket,
TimeSignalIdx: udpsprotocol.NoTimeSignal,
}
}
// newPacketDtHub builds a Hub with a ring for one packet-timed array signal and
// returns both. It does not start Run(): buildBinaryDataMessageForSource is
// called directly so the timestamps it produces can be read back verbatim.
func newPacketDtHub(t *testing.T, sigName string) (*Hub, *sourceHubState, *sigRing) {
t.Helper()
h := NewHub()
src := &sourceHubState{
id: "s1",
signals: []udpsprotocol.SignalInfo{pktSignal(sigName)},
timeSigCalib: map[string]float64{},
lastPktNs: map[string]int64{},
lastFrameMeasured: map[string]float64{},
lastFrameEndT: map[string]float64{},
gapEMA: map[string]float64{},
}
rb := newSigRing(4096)
h.rings["s1:"+sigName] = rb
return h, src, rb
}
// packet builds a one-signal batch entry arriving at t0 with the given loss
// count; the values are irrelevant, only the timestamps are under test.
func packet(sigName string, at time.Time, lost uint32, n int) udpsprotocol.DataSample {
vals := make([]float64, n)
return udpsprotocol.DataSample{WallTime: at, Values: map[string][]float64{sigName: vals}, Lost: lost}
}
// ringTimes returns the timestamps written to the ring, in order.
func ringTimes(rb *sigRing) []float64 {
rb.mu.RLock()
defer rb.mu.RUnlock()
out := make([]float64, 0, rb.size)
start := (rb.head - rb.size + rb.cap) % rb.cap
for i := 0; i < rb.size; i++ {
out = append(out, rb.t[(start+i)%rb.cap])
}
return out
}
// A lost packet widens the inter-packet gap without adding elements to the
// packet that follows it. Dividing the gap by that packet's element count
// alone reports a period too long by exactly the number of packets missing,
// which walks the elements past their own end and into the range the next
// packet claims: they collide there, and the span they vacated stays empty.
func TestPacketDtIgnoresLostPacketWidening(t *testing.T) {
const sig = "Wave"
const n = 4
const dt = 1 * time.Millisecond
base := time.Unix(1700000000, 0)
h, src, rb := newPacketDtHub(t, sig)
// One clean packet establishes lastPktNs.
h.buildBinaryDataMessageForSource(src, []udpsprotocol.DataSample{
packet(sig, base, 0, n)})
// The next producer packet is lost, so the one after it arrives a full
// extra batch later and reports Lost=1.
arrival := base.Add(2 * n * dt)
h.buildBinaryDataMessageForSource(src, []udpsprotocol.DataSample{
packet(sig, arrival, 1, n)})
ts := ringTimes(rb)
if len(ts) != n {
t.Fatalf("ring holds %d points, want %d (the first packet is skipped: no gap yet)", len(ts), n)
}
got := ts[1] - ts[0]
if !nearSec(got, dt.Seconds()) {
t.Errorf("dt = %v s, want %v s (the gap spans two batches, not one)", got, dt.Seconds())
}
// Elements run forward from the packet's own arrival, so a doubled dt
// would stretch this batch across two batch periods and into the range the
// next packet claims.
span := ts[len(ts)-1] - ts[0]
if !nearSec(span, float64(n-1)*dt.Seconds()) {
t.Errorf("batch spans %v s, want %v s: it overruns into the next packet's range",
span, float64(n-1)*dt.Seconds())
}
}
// nearSec compares two intervals in seconds. The hub carries timestamps as
// float64 seconds derived from UnixNano, whose spacing near the current epoch
// is a couple of hundred nanoseconds, so exact equality is not available. The
// defect under test moves the period by a factor of two, three orders of
// magnitude outside this tolerance.
func nearSec(got, want float64) bool { return math.Abs(got-want) <= 1e-6 }
// The correction must be driven by the reported loss and nothing else: with no
// packet missing the period still comes straight from the gap, so a producer
// that genuinely slows down is followed rather than second-guessed.
func TestPacketDtFollowsGapWhenNothingIsLost(t *testing.T) {
const sig = "Wave"
const n = 4
base := time.Unix(1700000000, 0)
h, src, rb := newPacketDtHub(t, sig)
h.buildBinaryDataMessageForSource(src, []udpsprotocol.DataSample{
packet(sig, base, 0, n)})
// Same widened gap as the test above, but reported as no loss: the
// producer really is running at half the rate.
slowDt := 2 * time.Millisecond
h.buildBinaryDataMessageForSource(src, []udpsprotocol.DataSample{
packet(sig, base.Add(n*slowDt), 0, n)})
ts := ringTimes(rb)
if len(ts) != n {
t.Fatalf("ring holds %d points, want %d", len(ts), n)
}
if got := ts[1] - ts[0]; !nearSec(got, slowDt.Seconds()) {
t.Errorf("dt = %v s, want %v s: a real rate change must be followed", got, slowDt.Seconds())
}
}
Common/Client/go/wshub/ringbuf.go
+330 -9
View File
@@ -1,6 +1,10 @@
package wshub package wshub
import "sync" import (
"log"
"math"
"sync"
)
// sigRing is a fixed-capacity circular buffer storing (time, value) pairs. // sigRing is a fixed-capacity circular buffer storing (time, value) pairs.
// Writes come from the Hub.Run() goroutine; reads come from HTTP handler goroutines. // Writes come from the Hub.Run() goroutine; reads come from HTTP handler goroutines.
@@ -10,30 +14,347 @@ type sigRing struct {
t, v []float64 t, v []float64
cap int cap int
head, size int // next write position; current fill head, size int // next write position; current fill
// bucket is how many source samples collapse into one min/max pair on the
// way in. 1 stores the stream verbatim. Raising it trades resolution for
// the timespan a fixed capacity covers, which is what lets a long display
// window fit in a fixed per-signal memory budget.
bucket int
// In-progress bucket. accN counts source samples seen since the last pair
// was emitted; the four acc fields are the extrema and when they occurred.
accN int
accTMin, accVMin float64
accTMax, accVMax float64
// Source-sample accounting, kept because size and the stored timespan no
// longer give the source rate once bucket > 1. Reset every
// srcRateWindowSec so a producer restart or a rate change is not averaged
// against the whole run.
srcCount int64
srcT0, srcT1 float64
haveSrc bool
} }
// srcRateWindowSec bounds how long a source-rate measurement accumulates before
// starting over. Long enough to average out per-frame jitter, short enough that
// a rate change is reflected within a few seconds.
const srcRateWindowSec = 10.0
func newSigRing(capacity int) *sigRing { func newSigRing(capacity int) *sigRing {
return &sigRing{ return &sigRing{
t: make([]float64, capacity), t: make([]float64, capacity),
v: make([]float64, capacity), v: make([]float64, capacity),
cap: capacity, cap: capacity,
bucket: 1,
} }
} }
// write appends (tArr[i], vArr[i]) pairs, overwriting oldest entries when full. // write appends (tArr[i], vArr[i]) pairs, overwriting oldest entries when full.
// With bucket > 1 each group of bucket samples contributes only its minimum and
// its maximum, in the order the two occurred.
func (rb *sigRing) write(tArr, vArr []float64) { func (rb *sigRing) write(tArr, vArr []float64) {
rb.mu.Lock() rb.mu.Lock()
defer rb.mu.Unlock() defer rb.mu.Unlock()
if n := len(tArr); n > 0 {
if !rb.haveSrc || tArr[n-1]-rb.srcT0 > srcRateWindowSec || tArr[0] < rb.srcT0 {
rb.srcT0, rb.srcCount, rb.haveSrc = tArr[0], 0, true
}
rb.srcT1 = tArr[n-1]
rb.srcCount += int64(n)
}
if rb.bucket <= 1 {
for i := 0; i < len(tArr); i++ {
rb.pushLocked(tArr[i], vArr[i])
}
return
}
for i := 0; i < len(tArr); i++ { for i := 0; i < len(tArr); i++ {
rb.t[rb.head] = tArr[i] t, v := tArr[i], vArr[i]
rb.v[rb.head] = vArr[i] if rb.accN == 0 {
rb.head = (rb.head + 1) % rb.cap rb.accTMin, rb.accVMin, rb.accTMax, rb.accVMax = t, v, t, v
if rb.size < rb.cap { } else {
rb.size++ if v < rb.accVMin {
rb.accTMin, rb.accVMin = t, v
}
if v > rb.accVMax {
rb.accTMax, rb.accVMax = t, v
}
}
rb.accN++
if rb.accN >= rb.bucket {
rb.flushBucketLocked()
} }
} }
} }
func (rb *sigRing) pushLocked(t, v float64) {
rb.t[rb.head] = t
rb.v[rb.head] = v
rb.head = (rb.head + 1) % rb.cap
if rb.size < rb.cap {
rb.size++
}
}
// flushBucketLocked emits the accumulated extrema oldest-first. Time order
// matters: every read binary-searches rb.t, so the stored timestamps must stay
// non-decreasing.
func (rb *sigRing) flushBucketLocked() {
if rb.accN == 0 {
return
}
if rb.accTMin <= rb.accTMax {
rb.pushLocked(rb.accTMin, rb.accVMin)
rb.pushLocked(rb.accTMax, rb.accVMax)
} else {
rb.pushLocked(rb.accTMax, rb.accVMax)
rb.pushLocked(rb.accTMin, rb.accVMin)
}
rb.accN = 0
}
// setBucket changes the min/max reduction applied to incoming samples and
// reports whether it changed. Samples already stored keep the resolution they
// were written at; the ring converges on the new one as it rolls.
func (rb *sigRing) setBucket(n int) bool {
if n < 1 {
n = 1
}
rb.mu.Lock()
defer rb.mu.Unlock()
if n == rb.bucket {
return false
}
// Emit what the old bucket had collected rather than dropping it.
rb.flushBucketLocked()
rb.bucket = n
return true
}
func (rb *sigRing) bucketSize() int {
rb.mu.RLock()
defer rb.mu.RUnlock()
return rb.bucket
}
// sourceRate is the measured rate of the incoming stream in samples per second,
// or 0 while there is too little to extrapolate from. Unlike stats() it counts
// source samples, so it is unaffected by bucketing.
func (rb *sigRing) sourceRate() float64 {
rb.mu.RLock()
defer rb.mu.RUnlock()
if rb.srcCount < 2 || rb.srcT1 <= rb.srcT0 {
return 0
}
return float64(rb.srcCount-1) / (rb.srcT1 - rb.srcT0)
}
// stats reports the current fill and the timespan it covers, so callers can
// estimate the stream's sample rate without copying the data out.
func (rb *sigRing) stats() (count int, span float64) {
rb.mu.RLock()
defer rb.mu.RUnlock()
if rb.size < 2 {
return rb.size, 0
}
start := 0
if rb.size == rb.cap {
start = rb.head
}
oldest := rb.t[start]
newest := rb.t[(start+rb.size-1)%rb.cap]
return rb.size, newest - oldest
}
func (rb *sigRing) capacity() int {
rb.mu.RLock()
defer rb.mu.RUnlock()
return rb.cap
}
// ─── Ring tuning ─────────────────────────────────────────────────────────────
// ringHeadroom oversizes a reduced ring's span. It absorbs rate jitter and
// keeps the tail of a trigger window in the buffer long enough for the capture
// to read it. It applies only once the window no longer fits verbatim: at the
// boundary, spending a whole extra bucket step to buy 25 % more span would cost
// half the resolution.
const ringHeadroom = 1.25
// ringTuneIntervalSec throttles the retune sweep. The source rate only settles
// once data flows, so the sweep repeats rather than running once.
const ringTuneIntervalSec = 1.0
// defaultLiveWindowSec is the window assumed when no client has said what it is
// displaying — the native clients never do, and a browser has not yet at the
// moment the first samples land.
const defaultLiveWindowSec = 10.0
// ringBucketFor is how many source samples must collapse into one min/max pair
// for `window` seconds at `rate` samples/s to fit in `capacity` points.
//
// rate*window <= capacity → 1, the buffer stays verbatim and reaches further
// back than the window, which is free zoom headroom
// rate*window > capacity → >1, so the whole window fits at reduced resolution
//
// A bucket costs two points (its minimum and its maximum), hence the factor 2.
func ringBucketFor(rate, window float64, capacity int) int {
if capacity <= 0 || rate <= 0 || window <= 0 {
return 1
}
need := rate * window
if need <= float64(capacity) {
return 1
}
return int(math.Ceil(2 * need * ringHeadroom / float64(capacity)))
}
// ringCoverage is how many source samples a ring of `capacity` points holds at
// the given bucket. A bucket of 2 stores both of its samples, so it covers no
// more ground than a bucket of 1.
func ringCoverage(bucket, capacity int) int {
if bucket <= 2 {
return capacity
}
return capacity / 2 * bucket
}
// captureLagSec is how much further back than the window itself a ring has to
// reach to deliver a capture of it.
//
// A capture is not read out when its last sample arrives but captureMarginSec
// later, and then only on the next push tick — so by the time the window is
// extracted, its oldest sample is that much deeper in the ring. A ring holding
// exactly the window has already overwritten the front of its own capture, which
// is what made every shot at a short window come back missing its head. The
// pre/post split does not enter into it: the harvest is a post-window after the
// trigger and the read reaches a pre-window before it, so the two sum to the
// window whatever the split.
const captureLagSec = captureMarginSec + 1.0/30.0
// activeWindowSec is the timespan the buffers must cover. An armed trigger owns
// it: its pre-window has to already be in the ring when the trigger fires or
// there is nothing to back-fill the capture from. Otherwise it is the widest
// window any connected client is displaying.
func (h *Hub) activeWindowSec() float64 {
if h.trigger != nil && h.trigger.Active() {
if cfg := h.trigger.Config(); cfg.windowSec > 0 {
return cfg.windowSec + captureLagSec
}
}
widest := 0.0
for c := range h.clients {
if w := c.displayWindowSec(); w > widest {
widest = w
}
}
if widest <= 0 {
return defaultLiveWindowSec
}
return widest
}
// retuneRings keeps every ring matched to the window being displayed: grown
// towards the per-signal budget, and bucketed so the window fits inside it.
//
// A fixed sample-count ring covers a fraction of a second at a megasample rate,
// which is why long windows used to come back with only their tail populated —
// in live mode as much as under a trigger. Spending the budget on min/max pairs
// rather than on a bigger allocation is what makes an arbitrarily long window
// work within a fixed memory bound.
//
// Called from Hub.Run() only, so reading h.clients here needs no lock.
func (h *Hub) retuneRings(nowSec float64) {
if nowSec < h.ringTuneAt {
return
}
h.ringTuneAt = nowSec + ringTuneIntervalSec
window := h.activeWindowSec()
if window <= 0 {
return
}
// The archive answers for the same window as the rings — it is what a zoom
// or a capture falls back on once they have rolled past it — so it is sized
// from the same number.
if h.hist.setWindow(window) {
if msg := h.buildHistoryInfoMsg(); msg != nil {
h.broadcast(msg)
}
}
budget := h.ringBudget()
h.ringsMu.RLock()
keys := make([]string, 0, len(h.rings))
rings := make([]*sigRing, 0, len(h.rings))
for k, rb := range h.rings {
keys = append(keys, k)
rings = append(rings, rb)
}
h.ringsMu.RUnlock()
for i, rb := range rings {
rate := rb.sourceRate()
if rate <= 0 {
continue
}
// Claim the whole budget before deciding on a bucket: memory is what
// buys resolution, so it is spent first and reduced from only if the
// window still does not fit.
if rb.capacity() < budget && rate*window > float64(rb.capacity()) {
rb.grow(budget)
}
cur := rb.bucketSize()
need := rate * window
covered := float64(ringCoverage(cur, rb.capacity()))
// Hysteresis. Retuning up and retuning down must not share a threshold:
// a bucket step doubles or halves the span, so a rate jittering across
// the boundary would otherwise flip the resolution every second. Hold
// the current bucket while it covers the window without covering more
// than twice it.
if covered >= need && covered <= 2*need {
continue
}
want := ringBucketFor(rate, window, rb.capacity())
if !rb.setBucket(want) {
continue
}
if want > 1 {
log.Printf("hub: ring %s stores min/max over %d samples: %.0f s at %.0f kSps does not fit in %d points",
keys[i], want, window, rate/1e3, rb.capacity())
} else {
log.Printf("hub: ring %s back to full resolution: %.0f s at %.0f kSps fits in %d points",
keys[i], window, rate/1e3, rb.capacity())
}
}
}
// grow enlarges the buffer to newCap, keeping every sample it currently holds.
// Shrinking is refused: it would discard history a pending capture may need.
func (rb *sigRing) grow(newCap int) bool {
rb.mu.Lock()
defer rb.mu.Unlock()
if newCap <= rb.cap {
return false
}
nt := make([]float64, newCap)
nv := make([]float64, newCap)
start := 0
if rb.size == rb.cap {
start = rb.head
}
for i := 0; i < rb.size; i++ {
p := (start + i) % rb.cap
nt[i], nv[i] = rb.t[p], rb.v[p]
}
rb.t, rb.v = nt, nv
rb.cap = newCap
rb.head = rb.size // size < newCap, so no wrap
return true
}
// slice returns copies of all (t, v) pairs whose timestamp falls in [t0, t1]. // slice returns copies of all (t, v) pairs whose timestamp falls in [t0, t1].
// The returned slices are safe to use after the call without holding any lock. // The returned slices are safe to use after the call without holding any lock.
func (rb *sigRing) slice(t0, t1 float64) ([]float64, []float64) { func (rb *sigRing) slice(t0, t1 float64) ([]float64, []float64) {
Common/Client/go/wshub/ringbuf_test.go
+150
View File
@@ -0,0 +1,150 @@
package wshub
import (
"math"
"testing"
)
// dump returns the ring's contents oldest-first, which is what every reader
// sees through slice() but is easier to assert on directly.
func dump(rb *sigRing) ([]float64, []float64) {
return rb.slice(math.Inf(-1), math.Inf(1))
}
func TestRingBucketStoresMinMaxPairsInTimeOrder(t *testing.T) {
rb := newSigRing(100)
rb.setBucket(4)
// Two buckets. In the first the minimum comes before the maximum, in the
// second the order is reversed, so the emitted pairs must not be sorted by
// value — a ring whose timestamps are not monotonic breaks slice()'s
// binary search.
ts := []float64{0, 1, 2, 3, 4, 5, 6, 7}
vs := []float64{-5, 0, 0, 9, 9, 0, 0, -5}
rb.write(ts, vs)
gotT, gotV := dump(rb)
wantT := []float64{0, 3, 4, 7}
wantV := []float64{-5, 9, 9, -5}
if len(gotT) != len(wantT) {
t.Fatalf("stored %d points, want %d", len(gotT), len(wantT))
}
for i := range wantT {
if gotT[i] != wantT[i] || gotV[i] != wantV[i] {
t.Fatalf("point %d = (%v,%v), want (%v,%v)", i, gotT[i], gotV[i], wantT[i], wantV[i])
}
}
}
func TestRingBucketExtendsTheSpanAFixedCapacityCovers(t *testing.T) {
const cap = 200
// 2000 samples at 1 kHz is 2 s, ten times what the capacity holds verbatim.
ts := make([]float64, 2000)
vs := make([]float64, 2000)
for i := range ts {
ts[i] = float64(i) * 1e-3
vs[i] = math.Sin(float64(i))
}
full := newSigRing(cap)
full.write(ts, vs)
if _, span := full.stats(); span > 0.25 {
t.Fatalf("full-rate ring spans %.3f s, expected ~0.2 s", span)
}
// bucket 20 turns 20 samples into 2 points, so the same capacity reaches
// 10x further: 200/2*20 = 2000 samples = 2 s.
bucketed := newSigRing(cap)
bucketed.setBucket(20)
bucketed.write(ts, vs)
count, span := bucketed.stats()
if count != cap {
t.Fatalf("bucketed ring holds %d points, want the full %d", count, cap)
}
if span < 1.9 {
t.Fatalf("bucketed ring spans %.3f s, want the whole ~2 s", span)
}
}
func TestRingSourceRateIsUnaffectedByBucketing(t *testing.T) {
rb := newSigRing(1000)
rb.setBucket(50)
ts := make([]float64, 5000)
vs := make([]float64, 5000)
for i := range ts {
ts[i] = float64(i) * 1e-4 // 10 kHz
}
rb.write(ts, vs)
got := rb.sourceRate()
if math.Abs(got-10000) > 10 {
t.Fatalf("sourceRate = %.1f, want ~10000", got)
}
}
func TestSetBucketFlushesThePartialBucket(t *testing.T) {
rb := newSigRing(100)
rb.setBucket(10)
// Three samples: not enough to close a bucket of 10, so nothing is stored
// yet and they would be silently dropped by a re-bucket that just reset the
// accumulator.
rb.write([]float64{0, 1, 2}, []float64{7, -7, 0})
if n, _ := rb.stats(); n != 0 {
t.Fatalf("partial bucket already emitted %d points", n)
}
rb.setBucket(2)
gotT, gotV := dump(rb)
if len(gotT) != 2 || gotT[0] != 0 || gotV[0] != 7 || gotT[1] != 1 || gotV[1] != -7 {
t.Fatalf("flushed pair = %v/%v, want t=[0 1] v=[7 -7]", gotT, gotV)
}
}
func TestActiveWindowSecFallsBackToTheDefault(t *testing.T) {
h := NewHub()
if got := h.activeWindowSec(); got != defaultLiveWindowSec {
t.Fatalf("activeWindowSec with no clients = %v, want %v", got, defaultLiveWindowSec)
}
}
// Clients disagree about how far back they are plotting, and a buffer sized for
// the narrowest one leaves the others with nothing to zoom into.
func TestActiveWindowSecTakesTheWidestClientWindow(t *testing.T) {
h := NewHub()
narrow, wide, silent := &wsClient{}, &wsClient{}, &wsClient{}
narrow.setDisplayWindowSec(1)
wide.setDisplayWindowSec(120)
h.clients[narrow], h.clients[wide], h.clients[silent] = true, true, true
if got := h.activeWindowSec(); got != 120 {
t.Fatalf("activeWindowSec = %v, want the widest 120", got)
}
}
// An armed trigger owns the window: its pre-window has to be in the buffer
// before the trigger fires or the capture has nothing to back-fill from. The
// buffers must reach back past the window itself, because the capture is read
// out a margin and a tick after its last sample lands.
func TestActiveWindowSecPrefersTheArmedTrigger(t *testing.T) {
h := NewHub()
c := &wsClient{}
c.setDisplayWindowSec(1)
h.clients[c] = true
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 45, mode: "normal"})
if got := h.activeWindowSec(); got != 45+captureLagSec {
t.Fatalf("activeWindowSec = %v, want the trigger's 45 plus the %v harvest lag",
got, captureLagSec)
}
}
func TestSetBucketToOneRestoresVerbatimStorage(t *testing.T) {
rb := newSigRing(100)
rb.setBucket(4)
rb.setBucket(1)
ts := []float64{0, 1, 2, 3}
vs := []float64{1, 2, 3, 4}
rb.write(ts, vs)
gotT, _ := dump(rb)
if len(gotT) != 4 {
t.Fatalf("stored %d points, want all 4", len(gotT))
}
}
Common/Client/go/wshub/sources.go
+160 -14
View File
@@ -1,12 +1,12 @@
package wshub package wshub
import ( import (
"encoding/json"
"fmt" "fmt"
"io" "io"
"log" "log"
"net" "net"
"os" "os"
"sort"
"strconv" "strconv"
"strings" "strings"
"sync" "sync"
@@ -95,11 +95,16 @@ func (sm *SourceManager) Remove(id string) {
} }
} }
// Save writes the current source list to filePath. // Path returns the configured config-file path ("" when none).
func (sm *SourceManager) Save() error { func (sm *SourceManager) Path() string {
if sm.filePath == "" { sm.mu.RLock()
return fmt.Errorf("no sources-file configured") defer sm.mu.RUnlock()
} return sm.filePath
}
// snapshotSources returns the current sources sorted by label, so the written
// file is byte-stable across runs (the map iteration order is not).
func (sm *SourceManager) snapshotSources() []SourceConfig {
sm.mu.RLock() sm.mu.RLock()
cfgs := make([]SourceConfig, 0, len(sm.sources)) cfgs := make([]SourceConfig, 0, len(sm.sources))
for _, ms := range sm.sources { for _, ms := range sm.sources {
@@ -111,26 +116,86 @@ func (sm *SourceManager) Save() error {
}) })
} }
sm.mu.RUnlock() sm.mu.RUnlock()
sort.Slice(cfgs, func(i, j int) bool {
if cfgs[i].Label != cfgs[j].Label {
return cfgs[i].Label < cfgs[j].Label
}
return cfgs[i].Addr < cfgs[j].Addr
})
return cfgs
}
data, err := json.MarshalIndent(cfgs, "", " ") // Save writes the current source list and calibration table to filePath as one
// flat JSON array.
func (sm *SourceManager) Save() error {
path := sm.Path()
if path == "" {
return fmt.Errorf("no sources-file configured")
}
data, err := encodeConfigFile(sm.snapshotSources(), sm.hub.cal.List())
if err != nil { if err != nil {
return err return err
} }
return os.WriteFile(sm.filePath, data, 0644) return os.WriteFile(path, data, 0644)
} }
// Load reads sources from path and adds them. // Load reads the config file at path, replaces the calibration table with its
// contents and starts every source it lists.
func (sm *SourceManager) Load(path string) error { func (sm *SourceManager) Load(path string) error {
data, err := os.ReadFile(path) data, err := os.ReadFile(path)
if err != nil { if err != nil {
return err return err
} }
var cfgs []SourceConfig srcs, cals, err := parseConfigFile(data)
if err := json.Unmarshal(data, &cfgs); err != nil { if err != nil {
return err return err
} }
sm.mu.Lock()
sm.filePath = path sm.filePath = path
for _, cfg := range cfgs { sm.mu.Unlock()
sm.hub.cal.Replace(cals)
for _, cfg := range srcs {
sm.Add(cfg.Label, cfg.Addr, cfg.MulticastGroup, cfg.DataPort)
}
return nil
}
// Reload re-reads the config file. The calibration table is replaced wholesale
// and sources listed in the file that are not already running are started; no
// live source is ever stopped, restarted or reconnected, because a reload must
// not interrupt streaming. The asymmetry is deliberate: calibration is cheap
// to reapply, a source is a live UDP session.
func (sm *SourceManager) Reload() error {
path := sm.Path()
if path == "" {
return fmt.Errorf("no sources-file configured")
}
data, err := os.ReadFile(path)
if err != nil {
return err
}
srcs, cals, err := parseConfigFile(data)
if err != nil {
return err
}
sm.hub.cal.Replace(cals)
sm.mu.RLock()
live := make(map[string]bool, len(sm.sources))
for _, ms := range sm.sources {
live[ms.label+"\x00"+ms.addr] = true
}
sm.mu.RUnlock()
for _, cfg := range srcs {
label := cfg.Label
if label == "" {
label = cfg.Addr // Add() applies the same default
}
if live[label+"\x00"+cfg.Addr] {
continue
}
sm.Add(cfg.Label, cfg.Addr, cfg.MulticastGroup, cfg.DataPort) sm.Add(cfg.Label, cfg.Addr, cfg.MulticastGroup, cfg.DataPort)
} }
return nil return nil
@@ -276,6 +341,9 @@ func (u *UDPClient) runSession() error {
} }
reassembler := udpsprotocol.NewReassembler(2 * time.Second) reassembler := udpsprotocol.NewReassembler(2 * time.Second)
// Per-session: the producer's counter restarts independently of ours, so
// the gate must not carry a counter over from the previous connection.
var gate udpsprotocol.SequenceGate
buf := make([]byte, readBufSize) buf := make([]byte, readBufSize)
var currentSigs []udpsprotocol.SignalInfo var currentSigs []udpsprotocol.SignalInfo
var currentPublishMode uint8 var currentPublishMode uint8
@@ -349,11 +417,20 @@ func (u *UDPClient) runSession() error {
if len(currentSigs) == 0 { if len(currentSigs) == 0 {
continue continue
} }
fresh, lost := gate.Accept(hdr.Counter)
if !fresh {
continue
}
samples, err := udpsprotocol.ParseData(complete, currentSigs, currentPublishMode, arrivalTime) samples, err := udpsprotocol.ParseData(complete, currentSigs, currentPublishMode, arrivalTime)
if err != nil { if err != nil {
log.Printf("[%s] udp: parse data: %v", u.sourceID, err) log.Printf("[%s] udp: parse data: %v", u.sourceID, err)
continue continue
} }
// The gap precedes the packet, so it belongs to its first slot only;
// the slots after it are consecutive cycles of the same batch.
if len(samples) > 0 {
samples[0].Lost = lost
}
for _, s := range samples { for _, s := range samples {
u.hub.PushDataForSource(u.sourceID, s) u.hub.PushDataForSource(u.sourceID, s)
} }
@@ -382,6 +459,52 @@ func (u *UDPClient) runSession() error {
} }
} }
// interfaceForIP returns the interface that owns the given local address, or
// nil if no interface matches (in which case callers fall back to letting the
// kernel choose).
func interfaceForIP(ip net.IP) *net.Interface {
if ip == nil || ip.IsUnspecified() {
return nil
}
ifaces, err := net.Interfaces()
if err != nil {
return nil
}
for i := range ifaces {
addrs, err := ifaces[i].Addrs()
if err != nil {
continue
}
for _, a := range addrs {
var aIP net.IP
switch v := a.(type) {
case *net.IPNet:
aIP = v.IP
case *net.IPAddr:
aIP = v.IP
}
if aIP != nil && aIP.Equal(ip) {
return &ifaces[i]
}
}
}
return nil
}
// interfaceForConn returns the interface a connection's local endpoint sits on.
func interfaceForConn(c net.Conn) *net.Interface {
if c == nil {
return nil
}
switch a := c.LocalAddr().(type) {
case *net.TCPAddr:
return interfaceForIP(a.IP)
case *net.UDPAddr:
return interfaceForIP(a.IP)
}
return nil
}
// runMulticastSession handles the multicast mode session. // runMulticastSession handles the multicast mode session.
func (u *UDPClient) runMulticastSession() error { func (u *UDPClient) runMulticastSession() error {
tcpAddr, err := net.ResolveTCPAddr("tcp4", u.serverAddr) tcpAddr, err := net.ResolveTCPAddr("tcp4", u.serverAddr)
@@ -435,7 +558,15 @@ func (u *UDPClient) runMulticastSession() error {
return &net.AddrError{Err: "invalid multicast group IP", Addr: u.multicastGroup} return &net.AddrError{Err: "invalid multicast group IP", Addr: u.multicastGroup}
} }
mcastAddr := &net.UDPAddr{IP: mcastIP, Port: mcastPort} mcastAddr := &net.UDPAddr{IP: mcastIP, Port: mcastPort}
mcastConn, err := net.ListenMulticastUDP("udp4", nil, mcastAddr) // Join on the interface that reaches the control connection. The UDPStreamer
// pins its multicast sends to its configured Interface (IP_MULTICAST_IF), so
// a join with a nil interface — which leaves imr_interface at INADDR_ANY and
// lets the kernel pick the default-route interface — silently receives
// nothing whenever that is not the sending interface. The local address of
// the control connection is the interface the server is reachable on, which
// is the sending interface in every single-homed and same-host deployment.
ifi := interfaceForConn(tcpConn)
mcastConn, err := net.ListenMulticastUDP("udp4", ifi, mcastAddr)
if err != nil { if err != nil {
return err return err
} }
@@ -443,7 +574,12 @@ func (u *UDPClient) runMulticastSession() error {
if err := mcastConn.SetReadBuffer(udpRcvBufSize); err != nil { if err := mcastConn.SetReadBuffer(udpRcvBufSize); err != nil {
log.Printf("[%s] multicast SetReadBuffer: %v", u.sourceID, err) log.Printf("[%s] multicast SetReadBuffer: %v", u.sourceID, err)
} }
log.Printf("[%s] joined multicast %s:%s", u.sourceID, u.multicastGroup, strconv.Itoa(mcastPort)) ifName := "default"
if ifi != nil {
ifName = ifi.Name
}
log.Printf("[%s] joined multicast %s:%s on interface %s",
u.sourceID, u.multicastGroup, strconv.Itoa(mcastPort), ifName)
tcpDone := make(chan error, 1) tcpDone := make(chan error, 1)
go func() { go func() {
@@ -465,6 +601,9 @@ func (u *UDPClient) runMulticastSession() error {
}() }()
reassembler := udpsprotocol.NewReassembler(2 * time.Second) reassembler := udpsprotocol.NewReassembler(2 * time.Second)
// Per-session, as in runSession(): a counter from the previous connection
// would reject the whole new stream.
var gate udpsprotocol.SequenceGate
buf := make([]byte, readBufSize) buf := make([]byte, readBufSize)
for { for {
@@ -505,11 +644,18 @@ func (u *UDPClient) runMulticastSession() error {
if len(currentSigs) == 0 { if len(currentSigs) == 0 {
continue continue
} }
fresh, lost := gate.Accept(hdr.Counter)
if !fresh {
continue
}
samples, parseErr := udpsprotocol.ParseData(complete, currentSigs, currentPublishMode, arrivalTime) samples, parseErr := udpsprotocol.ParseData(complete, currentSigs, currentPublishMode, arrivalTime)
if parseErr != nil { if parseErr != nil {
log.Printf("[%s] multicast: parse data: %v", u.sourceID, parseErr) log.Printf("[%s] multicast: parse data: %v", u.sourceID, parseErr)
continue continue
} }
if len(samples) > 0 {
samples[0].Lost = lost
}
for _, s := range samples { for _, s := range samples {
u.hub.PushDataForSource(u.sourceID, s) u.hub.PushDataForSource(u.sourceID, s)
} }
Common/Client/go/wshub/sources_test.go
+131
View File
@@ -0,0 +1,131 @@
package wshub
import (
"os"
"path/filepath"
"testing"
)
// newTestManager builds a hub + manager pair with no goroutines running.
func newTestManager(t *testing.T) (*Hub, *SourceManager, string) {
t.Helper()
path := filepath.Join(t.TempDir(), "sources.json")
h := NewHub()
sm := NewSourceManager(h, path)
h.SetSourceManager(sm)
return h, sm, path
}
func TestSaveWritesSourcesAndCalibration(t *testing.T) {
h, sm, path := newTestManager(t)
// Register two sources without starting any UDP client.
sm.mu.Lock()
sm.sources["s1"] = &managedSource{id: "s1", label: "wave", addr: "127.0.0.1:44500"}
sm.sources["s2"] = &managedSource{
id: "s2", label: "mc", addr: "127.0.0.1:44501",
multicastGroup: "239.0.0.1", dataPort: 44502,
}
sm.mu.Unlock()
if !h.cal.Set(CalConfig{Source: "wave", Signal: "Adc", Scale: 0.5, Offset: -1.25, Unit: "V"}) {
t.Fatal("calibration rejected")
}
if err := sm.Save(); err != nil {
t.Fatalf("Save: %v", err)
}
data, err := os.ReadFile(path)
if err != nil {
t.Fatalf("ReadFile: %v", err)
}
srcs, cals, err := parseConfigFile(data)
if err != nil {
t.Fatalf("parseConfigFile: %v\n%s", err, data)
}
if len(srcs) != 2 {
t.Fatalf("got %d sources, want 2\n%s", len(srcs), data)
}
// Save sorts by label so the file is byte-stable across runs.
if srcs[0].Label != "mc" || srcs[1].Label != "wave" {
t.Errorf("source order = %q,%q, want mc,wave", srcs[0].Label, srcs[1].Label)
}
if len(cals) != 1 || cals[0].Signal != "Adc" || cals[0].Scale != 0.5 {
t.Fatalf("calibration round-trip failed: %+v\n%s", cals, data)
}
}
func TestSaveWithoutFilePathFails(t *testing.T) {
h := NewHub()
sm := NewSourceManager(h, "")
h.SetSourceManager(sm)
if err := sm.Save(); err == nil {
t.Error("Save() with no path = nil error, want error")
}
}
func TestLoadSeedsCalibrationTable(t *testing.T) {
h, sm, path := newTestManager(t)
// No "addr" blocks: Load must not start any UDP client during the test.
if err := os.WriteFile(path, []byte(`[
{"source":"wave","signal":"Adc","scale":0.25,"offset":2,"unit":"mV"},
{"source":"wave","signal":"Dac","scale":2}
]`), 0o644); err != nil {
t.Fatal(err)
}
if err := sm.Load(path); err != nil {
t.Fatalf("Load: %v", err)
}
got := h.cal.List()
if len(got) != 2 {
t.Fatalf("List() = %d entries, want 2", len(got))
}
if got[0].Signal != "Adc" || got[0].Unit != "mV" || got[0].Offset != 2 {
t.Errorf("Adc = %+v", got[0])
}
if sm.Path() != path {
t.Errorf("Path() = %q, want %q", sm.Path(), path)
}
}
func TestReloadReplacesCalibrationAndKeepsLiveSources(t *testing.T) {
h, sm, path := newTestManager(t)
// A live source that the file does not mention must survive the reload.
sm.mu.Lock()
sm.sources["s1"] = &managedSource{id: "s1", label: "live", addr: "127.0.0.1:44999"}
sm.mu.Unlock()
// A stale calibration that the file does not mention must be dropped.
h.cal.Set(CalConfig{Source: "stale", Signal: "Old", Scale: 9})
if err := os.WriteFile(path, []byte(`[
{"source":"wave","signal":"Adc","scale":0.5}
]`), 0o644); err != nil {
t.Fatal(err)
}
if err := sm.Reload(); err != nil {
t.Fatalf("Reload: %v", err)
}
got := h.cal.List()
if len(got) != 1 || got[0].Source != "wave" {
t.Fatalf("after Reload, calibration = %+v, want only wave/Adc", got)
}
sm.mu.RLock()
_, alive := sm.sources["s1"]
n := len(sm.sources)
sm.mu.RUnlock()
if !alive || n != 1 {
t.Errorf("live source count = %d (s1 alive=%v), want 1 / true", n, alive)
}
}
func TestReloadWithoutFilePathFails(t *testing.T) {
h := NewHub()
sm := NewSourceManager(h, "")
h.SetSourceManager(sm)
if err := sm.Reload(); err == nil {
t.Error("Reload() with no path = nil error, want error")
}
}
Common/Client/go/wshub/stats.go
+3 -2
View File
@@ -32,8 +32,9 @@ type SourceStat struct {
} }
// RecordFragment is called for every UDP datagram of a DATA packet. // RecordFragment is called for every UDP datagram of a DATA packet.
// complete: this fragment completed the DATA reassembly. //
// nBytes: raw datagram size (header+payload). // complete: this fragment completed the DATA reassembly.
// nBytes: raw datagram size (header+payload).
func (s *SourceStat) RecordFragment(counter uint32, nBytes int, arrivalNs int64, complete bool) { func (s *SourceStat) RecordFragment(counter uint32, nBytes int, arrivalNs int64, complete bool) {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
Common/Client/go/wshub/trigger.go
+887
View File
@@ -0,0 +1,887 @@
package wshub
import (
"encoding/binary"
"encoding/json"
"log"
"math"
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/gorilla/websocket"
)
// Trigger FSM states, matching the C++ StreamHub TriggerEngine and the strings
// expected by the web SPA's "triggerState" handler.
const (
trigIdle = "idle"
trigArmed = "armed"
trigCollecting = "collecting"
trigTriggered = "triggered"
)
// captureMarginSec is the extra delay past the post-trigger window before the
// capture is extracted, so the rings have received the last samples.
const captureMarginSec = 0.15
// captureStallSec is how long the stream may be silent before a collecting
// trigger gives up waiting for the rest of its window and delivers what it has.
const captureStallSec = 2.0
// autoRearmDelaySec is the pause between a completed capture and the automatic
// rearm in "normal" mode.
const autoRearmDelaySec = 0.2
// trigCapturePts caps the points sent per signal in a capture frame. A window
// of 60 s at 1 MSps is 60 M raw samples — ~960 MB per signal on the wire, which
// no client can take and which the send path would simply drop. Matches the C++
// StreamHub's kTrigCapturePts.
const trigCapturePts = 20000
// shortCaptureTol is the fraction of the window a capture may miss at its front
// before it is reported. One min/max bucket of slack, not a quality target.
const shortCaptureTol = 0.01
// maxTriggerWindowSec bounds the capture window, matching the longest option
// the web UI offers. It is not a resolution limit: retuneRings buckets the
// rings so any window fits the per-signal memory budget, at the cost of storing
// min/max pairs rather than every sample.
const maxTriggerWindowSec = 600.0
// trigConfig is the client-settable part of the trigger.
type trigConfig struct {
signalKey string // "src:sig" or "src:sig[i]"
edge string // "rising" | "falling" | "both"
threshold float64
windowSec float64
prePercent float64
mode string // "normal" | "single"
holdoffSec float64 // rearm delay after a capture (double-trigger guard)
}
// triggerEngine implements the hub-side trigger FSM. Its methods are safe to
// call from the WebSocket read goroutines and from Hub.Run() concurrently.
type triggerEngine struct {
mu sync.Mutex
cfg trigConfig
// Parsed form of cfg.signalKey, refreshed by SetConfig.
baseKey string // "src:sig"
elemIdx int // -1 when the key has no "[i]" suffix
state string
stopped bool
// sentState is the state carried by the last stateMsg handed out. The
// armed→collecting transition happens inside feed(), on the ingest path,
// so the hub cannot see it by sampling State() across a tick — by the time
// the tick runs, ingest has already moved the FSM.
sentState string
// sentFill is the pre-fill fraction carried by the last stateMsg, so a
// trigger that is armed but still filling can report progress.
sentFill float64
// How far back the trigger signal's ring reaches and how fast that is
// growing (seconds of span per second of wall clock), refreshed by the hub.
// bufKnown is false when there is no ring to measure, which disables the
// fill gate rather than blocking the trigger on a measurement that will
// never arrive; bufRateOK is false until two measurements exist.
bufSpan float64
bufGrowth float64
bufKnown bool
bufRateOK bool
// bufCoverage is the maximum span (seconds) the ring can reach at its
// current bucket and capacity — the gate must never demand more than this,
// or a ring whose coverage is below the window can never satisfy it. 0 =
// unknown (no measurable rate).
bufCoverage float64
// bufArchived is true when the disk history already spans the trigger
// window, so a short capture's front can be back-filled from it.
bufArchived bool
// Reference point the growth is measured against.
bufRefSpan, bufRefWall float64
prevValue float64
prevValid bool
lastT float64
lastTOK bool
// lastFeedWall is the wall clock at the last feed(), used only to notice a
// stalled stream — the window itself is measured on the sample clock.
lastFeedWall float64
trigTime float64
firedPre float64
firedPost float64
firedValid bool
// The edge to fire on as soon as the FSM rearms, in sample time. Recorded
// while a capture is still being collected or handed out, for edges late
// enough that a capture of them would not overlap the one in flight.
//
// Without this the trigger is deaf from its own trigger point until the
// capture has been harvested — a post-window plus captureMarginSec — and
// then for the holdoff on top of that, and afterwards waits for a FRESH
// edge. On a sparse pulse train that rounds the capture spacing up to a
// whole pulse period: at the default 1 s window the blind stretch comes to
// 1.15 s, so a 1 Hz train was caught at 0.5 Hz and a wider window lost whole
// multiples. Remembering the edge instead makes the blind stretch exactly
// the post-window it has to be, since the capture is built from the edge's
// own timestamp and the ring still holds everything around it.
pendingT float64
pendingValid bool
rearmAt float64 // wall-clock seconds; 0 when no rearm is pending
}
func newTriggerEngine() *triggerEngine {
return &triggerEngine{
cfg: trigConfig{edge: "rising", windowSec: 1, prePercent: 20, mode: "normal", holdoffSec: autoRearmDelaySec},
elemIdx: -1,
state: trigIdle,
}
}
// parseSignalKey splits "src:sig[3]" into ("src:sig", 3). A key without an
// element suffix yields an index of -1.
func parseSignalKey(key string) (string, int) {
if !strings.HasSuffix(key, "]") {
return key, -1
}
open := strings.LastIndexByte(key, '[')
if open < 0 {
return key, -1
}
idx, err := strconv.Atoi(key[open+1 : len(key)-1])
if err != nil || idx < 0 {
return key, -1
}
return key[:open], idx
}
func (te *triggerEngine) SetConfig(cfg trigConfig) {
te.mu.Lock()
defer te.mu.Unlock()
// Clamp to the bounds the web UI offers.
if cfg.windowSec < 1e-4 {
cfg.windowSec = 1e-4
}
if cfg.windowSec > maxTriggerWindowSec {
cfg.windowSec = maxTriggerWindowSec
}
if cfg.prePercent < 0 {
cfg.prePercent = 0
}
if cfg.prePercent > 100 {
cfg.prePercent = 100
}
if cfg.holdoffSec < 0 {
cfg.holdoffSec = 0
}
if cfg.holdoffSec > 60 {
cfg.holdoffSec = 60
}
te.cfg = cfg
base, idx := parseSignalKey(cfg.signalKey)
if base != te.baseKey {
// The buffer measurement belongs to the old signal's ring.
te.bufKnown, te.bufRateOK = false, false
te.bufCoverage, te.bufArchived = 0, false
}
te.baseKey, te.elemIdx = base, idx
te.prevValid = false
te.prevValue = 0
// An edge held over from the old configuration would be latched against the
// new window, whose fill the gate has not vouched for.
te.pendingValid = false
}
func (te *triggerEngine) Config() trigConfig {
te.mu.Lock()
defer te.mu.Unlock()
return te.cfg
}
// Arm starts a fresh acquisition. It is the user's own arm, so it discards any
// edge remembered during the previous capture: the user asked for the next
// event, not for one that has already been and gone.
func (te *triggerEngine) Arm() {
te.mu.Lock()
te.state = trigArmed
te.prevValid = false
te.prevValue = 0
te.pendingValid = false
te.rearmAt = 0
te.mu.Unlock()
}
// rearm is the automatic arm at the end of a capture. Unlike Arm it honours an
// edge that arrived while the capture was being collected, firing on it at once
// rather than waiting for the next one — see pendingT. It also keeps the level
// tracked through the dead time, so the first sample after rearming is compared
// against its real predecessor instead of being spent seeding one.
func (te *triggerEngine) rearm() {
te.mu.Lock()
te.rearmAt = 0
if te.pendingValid {
t := te.pendingT
te.pendingValid = false
te.latchWindowLocked(t)
} else {
te.state = trigArmed
}
te.mu.Unlock()
}
func (te *triggerEngine) Disarm() {
te.mu.Lock()
te.state = trigIdle
te.stopped = false
te.prevValid = false
te.prevValue = 0
te.firedValid = false
te.pendingValid = false
te.rearmAt = 0
te.mu.Unlock()
}
func (te *triggerEngine) SetStopped(v bool) {
te.mu.Lock()
te.stopped = v
if v {
te.rearmAt = 0
}
te.mu.Unlock()
}
func (te *triggerEngine) Stopped() bool {
te.mu.Lock()
defer te.mu.Unlock()
return te.stopped
}
func (te *triggerEngine) State() string {
te.mu.Lock()
defer te.mu.Unlock()
return te.state
}
// Active reports whether a trigger signal is configured. The rings must stay
// populated from that moment on: a capture reaches back over the pre-trigger
// window, so waiting until the trigger arms would leave that window empty.
func (te *triggerEngine) Active() bool {
te.mu.Lock()
defer te.mu.Unlock()
return te.baseKey != ""
}
// baseSignalKey is the configured trigger signal without its "[i]" suffix, or
// "" when no trigger signal is set.
func (te *triggerEngine) baseSignalKey() string {
te.mu.Lock()
defer te.mu.Unlock()
return te.baseKey
}
// bufGrowthIntervalSec is the shortest baseline the span growth is measured
// over. The hub refreshes 30 times a second and the span moves in steps as
// batches land, so a shorter baseline measures the batching, not the trend.
const bufGrowthIntervalSec = 0.5
// bufGrowthSmooth is the weight of a new growth measurement in the running
// estimate.
const bufGrowthSmooth = 0.5
// setBuffered records how far back the trigger signal's ring reaches, at wall
// clock now, and derives how fast that is growing. coverage is the maximum
// span (seconds) the ring can reach at its current bucket/capacity; archived
// says the disk history already spans the trigger window. Pass known=false when
// there is no ring to measure.
func (te *triggerEngine) setBuffered(span, coverage float64, archived, known bool, now float64) {
te.mu.Lock()
defer te.mu.Unlock()
if !known {
te.bufKnown, te.bufRateOK = false, false
te.bufCoverage, te.bufArchived = 0, false
return
}
if !te.bufKnown {
te.bufKnown = true
te.bufRefSpan, te.bufRefWall = span, now
}
te.bufSpan = span
te.bufCoverage = coverage
te.bufArchived = archived
dt := now - te.bufRefWall
if dt < bufGrowthIntervalSec {
return
}
g := (span - te.bufRefSpan) / dt
// A ring that is not full grows one second of span per second; one that is
// full grows by whatever its incoming samples free up. Neither can exceed 1,
// and a shrinking ring is simply not growing.
if g < 0 {
g = 0
} else if g > 1 {
g = 1
}
if te.bufRateOK {
g = te.bufGrowth + bufGrowthSmooth*(g-te.bufGrowth)
}
te.bufGrowth, te.bufRateOK = g, true
te.bufRefSpan, te.bufRefWall = span, now
}
// fillNeedLocked is how far back the buffer must reach before an edge may be
// accepted, so that the capture is still whole when it is harvested a
// post-window later.
//
// What has to hold at harvest time is that the buffer spans the whole window:
// its newest sample is then trigTime+post, so anything less has lost the front
// of the capture. The buffer keeps filling while the post-window is collected,
// though, so the shortfall it may start with is exactly what it will make up in
// that time — measured, not assumed:
//
// need = windowSec − growth × postSec, floored at the pre-trigger window
//
// A ring that is still filling grows a second per second, which reduces this to
// the pre-trigger window: everything after the trigger is yet to be recorded
// anyway. A full one grows only as fast as its incoming samples free space —
// re-bucketing to a longer window replaces dense old samples with sparse new
// ones — and it is that case, growth well below 1, where firing on the
//
// Two escapes keep an armed trigger from staying deaf forever:
//
// - archived — the disk history already spans the window, so the front of a
// capture can be back-filled from it; the ring only needs to
// hold the pre-window worth of recent data.
// - coverage — never demand more than the ring can physically reach. If its
// coverage saturates below the window (a measured source rate
// that over-estimates the true one), the gate opens once the
// ring is full anyway and a short capture is delivered instead
// of deafness.
func (te *triggerEngine) fillNeedLocked() float64 {
pre := te.cfg.windowSec * te.cfg.prePercent / 100
growth := 0.0 // until measured, assume the buffer will not fill on its own
if te.bufRateOK {
growth = te.bufGrowth
}
need := te.cfg.windowSec - growth*(te.cfg.windowSec-pre)
if need < pre {
need = pre
}
if te.bufArchived {
// The archive back-fills the front; the ring holds the post-trigger
// window live, so the pre-window is all it needs to have reached.
return pre
}
if te.bufCoverage > 0 && need > te.bufCoverage {
need = te.bufCoverage
}
return need
}
// fillLocked is how much of that requirement is met, as a fraction in [0, 1].
// It is 1 whenever the gate does not apply: nothing needed, or no ring to
// measure.
func (te *triggerEngine) fillLocked() float64 {
need := te.fillNeedLocked()
if need <= 0 || !te.bufKnown || te.bufSpan >= need*(1-shortCaptureTol) {
return 1
}
if te.bufSpan <= 0 {
return 0
}
return te.bufSpan / need
}
// latchWindowLocked freezes the pre/post split at fire time so later config
// edits do not change how the capture is rendered.
func (te *triggerEngine) latchWindowLocked(t float64) {
te.state = trigCollecting
te.trigTime = t
te.firedPre = te.cfg.windowSec * te.cfg.prePercent / 100
te.firedPost = te.cfg.windowSec - te.firedPre
te.firedValid = true
te.rearmAt = 0
}
// Force fires the trigger immediately at the most recent sample time (falling
// back to the current wall clock when no sample has been seen yet).
func (te *triggerEngine) Force() {
te.mu.Lock()
defer te.mu.Unlock()
if te.state == trigCollecting {
return
}
t := float64(time.Now().UnixNano()) / 1e9
if te.lastTOK {
t = te.lastT
}
te.latchWindowLocked(t)
}
// feed passes a batch of full-resolution samples for one signal to the FSM.
// key is the fully-prefixed "src:sig" name; nElem is the signal's element count
// so that an "[i]"-suffixed configuration can select a single column out of the
// flattened element-major batch.
func (te *triggerEngine) feed(key string, nElem int, t, v []float64) {
if len(t) == 0 || len(t) != len(v) {
return
}
te.mu.Lock()
defer te.mu.Unlock()
if key != te.baseKey {
return
}
te.lastT = t[len(t)-1]
te.lastTOK = true
te.lastFeedWall = float64(time.Now().UnixNano()) / 1e9
// A capture in flight does not stop the comparator; it only changes what an
// edge does. See pendingT.
inFlight := te.state == trigCollecting || te.state == trigTriggered
if te.state != trigArmed && !inFlight {
return
}
step, start := 1, 0
if te.elemIdx >= 0 && nElem > 1 {
if te.elemIdx >= nElem {
return
}
step, start = nElem, te.elemIdx
}
// Hold off while the buffer does not reach back far enough. Firing now would
// deliver a capture whose front is simply missing — the ring never held it —
// which is what made the first shot after a window change come back short.
// Track the level meanwhile, so the first edge once the buffer is deep
// enough is still measured against the right previous sample.
if !inFlight && te.fillLocked() < 1 {
for i := start; i < len(v); i += step {
te.prevValue, te.prevValid = v[i], true
}
return
}
// The earliest trigger point a new capture may take. The one in flight owns
// everything up to the end of its own post-window, and the holdoff — a guard
// against re-triggering on the ringing of the SAME event — is measured from
// its trigger point too, so the two overlap rather than add.
notBefore := math.Inf(1)
if inFlight && te.firedValid {
notBefore = te.trigTime + math.Max(te.firedPost, te.cfg.holdoffSec)
}
thr := te.cfg.threshold
for i := start; i < len(t); i += step {
if !te.prevValid {
te.prevValue = v[i]
te.prevValid = true
continue
}
up := te.prevValue < thr && v[i] >= thr
down := te.prevValue > thr && v[i] <= thr
te.prevValue = v[i]
fired := false
switch te.cfg.edge {
case "falling":
fired = down
case "both":
fired = up || down
default:
fired = up
}
if !fired {
continue
}
if !inFlight {
te.latchWindowLocked(t[i])
return
}
// Keep the FIRST qualifying edge and go on tracking the level: a later
// one would be no more use, and stopping here would leave prevValue
// stale by the time the FSM rearms.
if !te.pendingValid && t[i] >= notBefore {
te.pendingT, te.pendingValid = t[i], true
}
}
}
// dueCapture reports whether a collecting trigger's post-window has elapsed and
// returns the latched window.
//
// The window is measured on the sample clock, not the wall clock: trigTime is a
// sample timestamp, and a stream whose timestamps lag real time (a busy
// producer, a buffered link) would otherwise be cut short by exactly that lag —
// an 8 s lag turned a 60 s window into a 36 s capture. Waiting for the samples
// themselves also means the ring really holds the window by the time it is read.
func (te *triggerEngine) dueCapture(nowSec float64) (trigTime, pre, post float64, ok bool) {
te.mu.Lock()
defer te.mu.Unlock()
if te.state != trigCollecting || !te.firedValid {
return 0, 0, 0, false
}
deadline := te.trigTime + te.firedPost + captureMarginSec
switch {
case te.lastTOK && te.lastT >= deadline:
// The samples have covered the window.
case !te.lastTOK && nowSec >= deadline:
// No sample ever seen, so trigTime came from the wall clock (Force).
case te.lastFeedWall > 0 && nowSec-te.lastFeedWall >= captureStallSec:
// The stream has dried up; deliver what was collected rather than
// leaving the client stuck in "collecting" forever.
default:
return 0, 0, 0, false
}
return te.trigTime, te.firedPre, te.firedPost, true
}
// markTriggered completes a capture and schedules the automatic rearm when the
// engine runs in "normal" mode.
func (te *triggerEngine) markTriggered(nowSec float64) {
te.mu.Lock()
if te.state == trigCollecting {
te.state = trigTriggered
if te.cfg.mode != "single" && !te.stopped {
te.rearmAt = nowSec + te.cfg.holdoffSec
}
}
te.mu.Unlock()
}
// dueRearm reports whether a pending automatic rearm has come due, consuming it.
func (te *triggerEngine) dueRearm(nowSec float64) bool {
te.mu.Lock()
defer te.mu.Unlock()
if te.state != trigTriggered || te.rearmAt == 0 || nowSec < te.rearmAt {
return false
}
te.rearmAt = 0
return !te.stopped
}
// stateUnsent reports whether the FSM has moved since the last stateMsg was
// built, i.e. whether clients still have to be told.
func (te *triggerEngine) stateUnsent() bool {
te.mu.Lock()
defer te.mu.Unlock()
if te.state != te.sentState {
return true
}
// An armed trigger waiting for its buffer is otherwise indistinguishable
// from one that is ignoring edges, so the filling itself is news. Coarse
// steps only: this is checked 30 times a second.
if te.state == trigArmed {
f := te.fillLocked()
return math.Abs(f-te.sentFill) >= 0.02 || (f >= 1 && te.sentFill < 1)
}
return false
}
// stateMsg builds the JSON "triggerState" broadcast for the current FSM state.
func (te *triggerEngine) stateMsg() []byte {
te.mu.Lock()
te.sentState = te.state
te.sentFill = te.fillLocked()
m := map[string]any{
"type": "triggerState",
"state": te.state,
"mode": te.cfg.mode,
"stopped": te.stopped,
}
if te.state == trigArmed && te.sentFill < 1 {
// Armed but holding off: the buffer does not yet reach back far enough
// to deliver the window, so edges are being ignored on purpose.
m["bufferFill"] = te.sentFill
m["bufferNeedSec"] = te.fillNeedLocked()
}
if te.firedValid {
// The window latched at fire time. Clients draw the filling capture on
// this axis before the v2 frame arrives, and config edits between arm
// and fire would otherwise leave them inferring the wrong window from
// their own copy of the config.
m["trigTime"] = te.trigTime
m["preSec"] = te.firedPre
m["postSec"] = te.firedPost
}
te.mu.Unlock()
msg, _ := json.Marshal(m)
return msg
}
/* ─── Hub integration ─────────────────────────────────────────────────────── */
// broadcastTriggerState pushes the current FSM state to every client.
func (h *Hub) broadcastTriggerState() {
h.broadcast(h.trigger.stateMsg())
}
// handleTriggerCommand processes a trigger-related browser message. It returns
// false when the message type is not a trigger command.
func (h *Hub) handleTriggerCommand(t string, env map[string]interface{}) bool {
switch t {
case "setTrigger":
cfg := h.trigger.Config()
if s, ok := env["signal"].(string); ok {
cfg.signalKey = s
}
if s, ok := env["edge"].(string); ok {
cfg.edge = s
}
if s, ok := env["mode"].(string); ok {
cfg.mode = s
}
if f, ok := env["threshold"].(float64); ok {
cfg.threshold = f
}
if f, ok := env["windowSec"].(float64); ok {
cfg.windowSec = f
}
if f, ok := env["prePercent"].(float64); ok {
cfg.prePercent = f
}
if f, ok := env["holdoffSec"].(float64); ok {
cfg.holdoffSec = f
}
h.trigger.SetConfig(cfg)
case "arm", "rearm":
h.trigger.Arm()
case "disarm":
h.trigger.Disarm()
case "trigStop":
stopped := !h.trigger.Stopped()
if b, ok := env["stopped"].(bool); ok {
stopped = b
}
h.trigger.SetStopped(stopped)
case "forceTrigger":
h.trigger.Force()
default:
return false
}
// Measure the buffer now rather than waiting for the next tick: ingest runs
// on the source goroutine and a 1 MSps stream crosses the threshold many
// times within one 33 ms tick, so an arm serviced here would otherwise fire
// on a stale (or missing) measurement before the gate ever saw the new
// configuration.
h.refreshTriggerFill()
h.broadcastTriggerState()
return true
}
// refreshTriggerFill tells the FSM how far back the trigger signal's ring
// reaches, which is what lets an armed trigger hold off until a capture taken
// now would come back whole.
//
// The ring is the right yardstick even though a short capture is back-filled
// from the archive: the archive is sized for the same window and starts over
// whenever that window changes, so it holds no more of the stretch being waited
// for than the ring does. It can only add to what the capture finds.
//
// Called both from the push tick and from the client goroutine handling a
// trigger command; all the state it derives lives in the engine, behind the
// engine's lock.
func (h *Hub) refreshTriggerFill() {
if h.trigger == nil {
return
}
now := float64(time.Now().UnixNano()) / 1e9
key := h.trigger.baseSignalKey()
var rb *sigRing
if key != "" {
rb = h.getRing(key)
}
if rb == nil {
// Nothing to measure. Do not gate on a signal the hub does not carry:
// that would leave the trigger armed forever, which is worse than a
// short capture.
h.trigger.setBuffered(0, 0, false, false, now)
return
}
_, span := rb.stats()
// Maximum span the ring can ever reach at its current bucket/capacity, in
// seconds. The gate must never demand more than this, or a ring whose
// coverage is below the window (a measured source rate that over-estimates
// the true one) can never satisfy it.
coverage := 0.0
if rate := rb.sourceRate(); rate > 0 {
coverage = float64(ringCoverage(rb.bucketSize(), rb.capacity())) / rate
}
// If the disk archive already spans the trigger window, the front of a
// short capture can be back-filled from it, so the ring need not cover the
// whole window on its own.
archived := h.hist.coversWindow(key, h.trigger.Config().windowSec)
h.trigger.setBuffered(span, coverage, archived, true, now)
}
// triggerTick services the trigger FSM; called from Hub.Run() on every push tick.
func (h *Hub) triggerTick() {
nowSec := float64(time.Now().UnixNano()) / 1e9
h.retuneRings(nowSec)
h.openPendingHistoryFiles(nowSec)
h.refreshTriggerFill()
if trigTime, pre, post, ok := h.trigger.dueCapture(nowSec); ok {
if msg := h.buildTriggerCapture(trigTime, pre, post); msg != nil {
dropped := 0
for c := range h.clients {
select {
case c.send <- wsMessage{websocket.BinaryMessage, msg}:
default:
dropped++
}
}
// A dropped capture is invisible to the user — the trigger fires,
// the state goes to "triggered" and no waveform ever arrives — so
// say so rather than leaving it to be guessed at.
if dropped > 0 {
log.Printf("wshub: trigger capture (%d B) dropped for %d client(s): send queue full",
len(msg), dropped)
}
}
h.trigger.markTriggered(nowSec)
// A capture is only zoomable for as long as its samples still exist at
// full resolution somewhere, and the rings roll past the window within
// seconds of it being taken. Lift the window out of the archive into a
// file of its own, where nothing overwrites it until the next trigger.
h.hist.captureRange(trigTime-pre, trigTime+post)
} else if h.trigger.dueRearm(nowSec) {
h.trigger.rearm()
}
if h.trigger.stateUnsent() {
h.broadcastTriggerState()
}
}
// backfillCaptureHead prepends the front of [t0, t1] that the ring no longer
// holds, read from the disk archive. It returns its input unchanged when the
// ring already reaches t0, when history is off, or when the archive has nothing
// for that range.
//
// The rings are sized for the window, but they only have to *become* that long:
// they are min/max buckets that cover the configured window once they have
// rolled over completely at the current bucket, which takes as long as the
// window itself. Widen the window and arm, and the first captures ask for more
// history than the ring has ever stored — the frame then starts late and the
// user sees a blank front half. The archive is written straight through, at the
// geometry its file was created with, so unless that file was re-sized too it
// has kept the stretch the ring is still converging on.
func (h *Hub) backfillCaptureHead(key string, t0, t1 float64, st, sv []float64) ([]float64, []float64) {
window := t1 - t0
if !h.hist.enabled() || window <= 0 {
return st, sv
}
gapEnd := t1
if len(st) > 0 {
gapEnd = st[0]
}
gap := gapEnd - t0
if gap <= shortCaptureTol*window {
return st, sv
}
// Budget the read by the share of the window being back-filled. The frame is
// decimated to trigCapturePts either way, so a bigger read would buy nothing
// but disk seeks — on the hub's own goroutine, between two push ticks.
maxOut := int(float64(trigCapturePts)*gap/window) + 2
ht, hv := h.hist.readRange(key, t0, gapEnd, maxOut)
if len(ht) == 0 {
return st, sv
}
// Drop anything at or past the ring's first sample: the two sources overlap
// around the join, and the frame's timestamps must stay ascending.
n := len(ht)
if len(st) > 0 {
n = sort.SearchFloat64s(ht, st[0])
}
if n == 0 {
return st, sv
}
outT := make([]float64, 0, n+len(st))
outV := make([]float64, 0, n+len(sv))
outT = append(append(outT, ht[:n]...), st...)
outV = append(append(outV, hv[:n]...), sv...)
return outT, outV
}
// buildTriggerCapture extracts [trigTime-pre, trigTime+post] from every ring
// buffer and encodes the version-2 binary capture frame:
//
// [u8 2][f64 trigTime][f64 preSec][f64 postSec][u32 nSig]
// {[u16 keyLen][fullKey][u32 N][t f64×N][v f64×N]}
func (h *Hub) buildTriggerCapture(trigTime, pre, post float64) []byte {
t0, t1 := trigTime-pre, trigTime+post
type sigSlice struct {
key string
t, v []float64
}
h.ringsMu.RLock()
keys := make([]string, 0, len(h.rings))
rings := make([]*sigRing, 0, len(h.rings))
for k, rb := range h.rings {
keys = append(keys, k)
rings = append(rings, rb)
}
h.ringsMu.RUnlock()
slices := make([]sigSlice, 0, len(keys))
held := make(map[string]sigData, len(keys))
total := 1 + 8 + 8 + 8 + 4
for i, k := range keys {
st, sv := rings[i].slice(t0, t1)
st, sv = h.backfillCaptureHead(k, t0, t1, st, sv)
if len(st) == 0 {
continue
}
// Neither the ring nor the archive reached t0. Nothing can recover that
// data, so name it rather than leaving the user to wonder why the front
// of their window is blank.
if lost := st[0] - t0; lost > shortCaptureTol*(t1-t0) {
cnt, span := rings[i].stats()
log.Printf("wshub: capture %s is short by %.2f s of %.2f s: ring holds %.2f s (%d pts, min/max over %d)",
k, lost, t1-t0, span, cnt, rings[i].bucketSize())
}
// Take the second half of the double buffer here, before the frame is
// decimated: the client gets 20 000 points to draw, but a zoom into
// them has to come back with the underlying samples, and the rings will
// have rolled past them by the time it is asked for.
held[k] = sigData{T: st, V: sv}
// Decimate before framing: a long window at a high sample rate is
// hundreds of megabytes raw, which the send path would silently drop.
// The min/max envelope keeps every peak in the window, so a glitch is
// still on screen at the zoomed-out view that first shows it.
st, sv = minMaxDecimate(st, sv, trigCapturePts)
slices = append(slices, sigSlice{key: k, t: st, v: sv})
total += 2 + len(k) + 4 + len(st)*16
}
if len(slices) == 0 {
return nil
}
// Swap only now that the capture is known good. A shot that yielded nothing
// must leave the previous window on screen rather than blanking it.
h.capture.publish(t0, t1, held)
buf := make([]byte, total)
buf[0] = 2
off := 1
binary.LittleEndian.PutUint64(buf[off:], math.Float64bits(trigTime))
off += 8
binary.LittleEndian.PutUint64(buf[off:], math.Float64bits(pre))
off += 8
binary.LittleEndian.PutUint64(buf[off:], math.Float64bits(post))
off += 8
binary.LittleEndian.PutUint32(buf[off:], uint32(len(slices)))
off += 4
for _, s := range slices {
binary.LittleEndian.PutUint16(buf[off:], uint16(len(s.key)))
off += 2
copy(buf[off:], s.key)
off += len(s.key)
binary.LittleEndian.PutUint32(buf[off:], uint32(len(s.t)))
off += 4
off = writeFloat64s(buf, off, s.t)
off = writeFloat64s(buf, off, s.v)
}
return buf
}
Common/Client/go/wshub/trigger_capture_test.go
+278
View File
@@ -0,0 +1,278 @@
package wshub
import (
"encoding/binary"
"math"
"testing"
)
// fillRing writes n samples at the given rate starting at t0.
func fillRing(rb *sigRing, t0 float64, rate float64, n int) {
ts := make([]float64, n)
vs := make([]float64, n)
for i := range ts {
ts[i] = t0 + float64(i)/rate
vs[i] = math.Sin(float64(i))
}
rb.write(ts, vs)
}
func TestRingGrowPreservesSamples(t *testing.T) {
rb := newSigRing(100)
// Overflow the ring so the retained window starts mid-buffer.
fillRing(rb, 0, 1000, 250)
beforeT, beforeV := rb.slice(-1e9, 1e9)
if len(beforeT) != 100 {
t.Fatalf("pre-grow fill = %d, want 100", len(beforeT))
}
if !rb.grow(1000) {
t.Fatal("grow(1000) returned false")
}
if rb.capacity() != 1000 {
t.Fatalf("capacity = %d, want 1000", rb.capacity())
}
afterT, afterV := rb.slice(-1e9, 1e9)
if len(afterT) != len(beforeT) {
t.Fatalf("post-grow fill = %d, want %d", len(afterT), len(beforeT))
}
for i := range beforeT {
if afterT[i] != beforeT[i] || afterV[i] != beforeV[i] {
t.Fatalf("sample %d changed across grow", i)
}
}
// Further writes must keep landing in order rather than wrapping early.
fillRing(rb, 1.0, 1000, 500)
if n, _ := rb.stats(); n != 600 {
t.Fatalf("fill after grow = %d, want 600", n)
}
// Shrinking is refused.
if rb.grow(10) {
t.Fatal("grow(10) shrank the ring")
}
}
func TestRingStatsMeasuresRate(t *testing.T) {
rb := newSigRing(10000)
fillRing(rb, 0, 1000, 1000) // 1 kHz
n, span := rb.stats()
if n != 1000 {
t.Fatalf("count = %d, want 1000", n)
}
rate := float64(n) / span
if math.Abs(rate-1001) > 5 { // n samples span (n-1) intervals
t.Fatalf("rate = %v, want ~1000", rate)
}
}
// A long trigger window must grow the rings to hold it: a fixed sample-count
// ring covers a fraction of a second at a high rate, which is what made 60 s
// captures come back with only their tail populated.
func TestRetuneRingsCoversTriggerWindow(t *testing.T) {
h := NewHub()
rb := newSigRing(6000) // 6 s at 1 kHz — far short of a 60 s window
fillRing(rb, 0, 1000, 6000)
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", edge: "rising",
windowSec: 60, prePercent: 20, mode: "normal"})
h.retuneRings(1000)
// 60 s at 1 kHz is 60 k samples: growing to the budget holds them verbatim.
if got := rb.capacity(); got < 60000 {
t.Fatalf("capacity = %d, want >= 60000 to hold a 60 s window", got)
}
if got := rb.bucketSize(); got != 1 {
t.Fatalf("bucket = %d, want 1: the window fits at full rate", got)
}
}
// Past the budget the window is kept by reducing resolution, not by dropping
// its head — the whole point of the min/max buckets.
func TestRetuneRingsBucketsWhenTheWindowExceedsTheBudget(t *testing.T) {
h := NewHub()
rb := newSigRing(1000)
fillRing(rb, 0, 1e6, 100_000) // 1 MSps
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 60, mode: "normal"})
h.retuneRings(1000)
if got := rb.capacity(); got != defaultRingPts {
t.Fatalf("capacity = %d, want the budget %d", got, defaultRingPts)
}
// 60 s at 1 MSps is 60 M samples in a 10 M-point buffer, so each stored
// pair must cover at least 12 source samples.
bucket := rb.bucketSize()
if bucket < 12 {
t.Fatalf("bucket = %d, too fine to fit 60 M samples in %d points", bucket, rb.capacity())
}
if covered := float64(rb.capacity()) / 2 * float64(bucket) / 1e6; covered < 60 {
t.Fatalf("buffer covers %.1f s, want the whole 60 s window", covered)
}
}
// A raised budget buys resolution back: the same window is held verbatim.
func TestRetuneRingsHonoursRaisedBudget(t *testing.T) {
h := NewHub()
h.SetRingBudget(80_000_000)
rb := newSigRing(1000)
fillRing(rb, 0, 1e6, 100_000) // 1 MSps
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 60, mode: "normal"})
h.retuneRings(1000)
if got := rb.bucketSize(); got != 1 {
t.Fatalf("bucket = %d, want 1: 60 M samples fit in an 80 M-point buffer", got)
}
}
func TestSetRingBudgetBounds(t *testing.T) {
h := NewHub()
h.SetRingBudget(0)
if got := h.ringBudget(); got != defaultRingPts {
t.Fatalf("ringBudget after 0 = %d, want the default %d", got, defaultRingPts)
}
// Never below the depth a freshly configured ring already has, or the
// budget would ask for a shrink the ring refuses anyway.
h.SetRingBudget(10)
if got := h.ringBudget(); got != ringCapInitial {
t.Fatalf("ringBudget after 10 = %d, want the floor %d", got, ringCapInitial)
}
}
func TestRetuneRingsIsThrottled(t *testing.T) {
h := NewHub()
h.SetRingBudget(250_000)
rb := newSigRing(250_000)
fillRing(rb, 0, 1e6, 100_000)
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 10, mode: "normal"})
h.retuneRings(100)
first := rb.bucketSize()
if first <= 1 {
t.Fatalf("bucket = %d, expected a reduction for 10 s at 1 MSps in 250 k points", first)
}
// Same second: the sweep must not run again even though a bigger window
// is now configured.
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 600, mode: "normal"})
h.retuneRings(100.5)
if rb.bucketSize() != first {
t.Fatalf("sweep ran inside the throttle window")
}
h.retuneRings(200)
if rb.bucketSize() <= first {
t.Fatalf("sweep did not run after the throttle window elapsed")
}
}
// With no trigger armed and no client saying otherwise, the rings are sized for
// the default live window — live mode needs the buffers just as much as a
// capture does.
func TestRetuneRingsSizesForTheLiveWindow(t *testing.T) {
h := NewHub()
rb := newSigRing(1000)
fillRing(rb, 0, 1e6, 100_000) // 1 MSps: 10 s does not fit in 1000 points
h.rings["s1:sig"] = rb
// No signal configured → trigger inactive, so the live window governs.
h.trigger.SetConfig(trigConfig{windowSec: 600, mode: "normal"})
h.retuneRings(100)
if got := rb.capacity(); got != defaultRingPts {
t.Fatalf("capacity = %d, want the budget %d", got, defaultRingPts)
}
// defaultLiveWindowSec at 1 MSps is exactly the budget, so no reduction.
if got := rb.bucketSize(); got != 1 {
t.Fatalf("bucket = %d, want 1 for the default live window", got)
}
}
func TestRingBucketForCoversTheWindow(t *testing.T) {
cases := []struct {
rate, window float64
capacity int
want int
}{
{1000, 10, 1_000_000, 1}, // 10 k samples in 1 M points: verbatim
{1e6, 10, 10_000_000, 1}, // exactly the budget: still verbatim
{1e6, 60, 10_000_000, 15}, // 60 M samples, 1.25x headroom
{1e6, 600, 10_000_000, 150}, // 600 s still fits, at 1/150 resolution
{0, 10, 1_000_000, 1}, // no rate measured yet
{1000, 0, 1_000_000, 1}, // no window
}
for _, c := range cases {
if got := ringBucketFor(c.rate, c.window, c.capacity); got != c.want {
t.Errorf("ringBucketFor(%v, %v, %d) = %d, want %d",
c.rate, c.window, c.capacity, got, c.want)
}
}
}
// decodeCapture pulls the per-signal point counts out of a v2 capture frame.
func decodeCapture(t *testing.T, buf []byte) map[string]int {
t.Helper()
if buf[0] != 2 {
t.Fatalf("frame version = %d, want 2", buf[0])
}
off := 1 + 8 + 8 + 8
nSig := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
out := make(map[string]int, nSig)
for i := 0; i < nSig; i++ {
kl := int(binary.LittleEndian.Uint16(buf[off:]))
off += 2
key := string(buf[off : off+kl])
off += kl
n := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
off += n * 16
out[key] = n
}
if off != len(buf) {
t.Fatalf("decoded %d of %d bytes", off, len(buf))
}
return out
}
// A 60 s window at a high rate is hundreds of megabytes raw; the capture frame
// must be decimated so it can actually reach a client.
func TestBuildTriggerCaptureDecimates(t *testing.T) {
h := NewHub()
rb := newSigRing(200000)
fillRing(rb, 0, 100000, 200000) // 2 s at 100 kSps
h.rings["s1:sig"] = rb
buf := h.buildTriggerCapture(1.0, 1.0, 1.0)
if buf == nil {
t.Fatal("no capture frame built")
}
counts := decodeCapture(t, buf)
n := counts["s1:sig"]
if n != trigCapturePts {
t.Fatalf("captured %d points, want the %d-point cap", n, trigCapturePts)
}
}
// Short captures must stay full resolution — decimation only kicks in above
// the cap.
func TestBuildTriggerCaptureKeepsSmallWindowsIntact(t *testing.T) {
h := NewHub()
rb := newSigRing(10000)
fillRing(rb, 0, 1000, 10000) // 10 s at 1 kHz
h.rings["s1:sig"] = rb
buf := h.buildTriggerCapture(1.0, 0.5, 0.5)
if buf == nil {
t.Fatal("no capture frame built")
}
counts := decodeCapture(t, buf)
if n := counts["s1:sig"]; n < 990 || n > 1010 {
t.Fatalf("captured %d points, want ~1000 undecimated", n)
}
}
Common/Client/go/wshub/trigger_sporadic_test.go
+367
View File
@@ -0,0 +1,367 @@
package wshub
import (
"encoding/binary"
"math"
"testing"
)
/*
Sporadic-signal trigger coverage.
Every other trigger test in this package feeds a periodic waveform, or a
hand-built two-sample batch. Neither can show a trigger that is blind most of
the time: a sine crosses the threshold again a few milliseconds after every
missed crossing, so a trigger losing 80 % of its edges still fires steadily and
looks healthy. A sparse train — 0000000111000000000000, one short burst in a
long flat run — has nothing to fall back on, so every missed edge is a missed
capture and the yield is a direct measure of how long the FSM was deaf.
That deafness is what these tests pin down. It is not a bug in itself: a capture
cannot be harvested before the samples after its trigger point exist, so the
trigger is necessarily blind for its own post-trigger window. What must NOT
happen is for edges arriving after that window to be thrown away as well.
*/
// pulseTrainSim drives a Hub the way Run() does — ingest on one side, the
// trigger tick on the other — on a simulated clock.
type pulseTrainSim struct {
rateHz float64
batchSec float64
pulsePeriod float64
pulseSamples int
simSec float64
windowSec float64
prePercent float64
holdoffSec float64
armAt float64
// When windowChangeAt > 0 the window is switched to windowChangeTo at that
// time and the trigger re-armed, as a user editing the trigger bar would.
windowChangeAt float64
windowChangeTo float64
}
type pulseTrainResult struct {
pulses int // pulse starts presented after the trigger was armed
shots int // captures actually delivered
trigTimes []float64 // the sample time each capture triggered on
worstCov float64 // smallest fraction of its window a capture spanned
holdDeclined int // captures the zoom hold would not answer for
drawnPulses int // pulses visible in the delivered frames
wantPulses int // pulses those frames' windows really contained
gatedPulses int // pulses that arrived armed but with the fill gate shut
}
// yield is the fraction of presented pulses that produced a capture.
func (r pulseTrainResult) yield() float64 {
if r.pulses == 0 {
return 0
}
return float64(r.shots) / float64(r.pulses)
}
// run executes the simulation and returns what the trigger caught.
func (s pulseTrainSim) run(t *testing.T, key string) pulseTrainResult {
t.Helper()
h := NewHub()
h.rings[key] = newSigRing(ringCapInitial)
h.trigger.SetConfig(trigConfig{
signalKey: key, edge: "rising", threshold: 0.5,
windowSec: s.windowSec, prePercent: s.prePercent,
mode: "normal", holdoffSec: s.holdoffSec,
})
res := pulseTrainResult{worstCov: 1}
nBatch := int(s.rateHz * s.batchSec)
ts := make([]float64, nBatch)
vs := make([]float64, nBatch)
armed, changed := false, false
for now := 0.0; now < s.simSec; now += s.batchSec {
nPulseStarts := 0
for i := range ts {
ts[i] = now + float64(i)/s.rateHz
// Position within the current pulse period, in samples.
k := int((ts[i] - math.Floor(ts[i]/s.pulsePeriod)*s.pulsePeriod) * s.rateHz)
if k < s.pulseSamples {
vs[i] = 1
if k == 0 {
nPulseStarts++
}
} else {
vs[i] = 0
}
}
if !armed && now >= s.armAt {
h.trigger.Arm()
armed = true
}
if s.windowChangeAt > 0 && !changed && now >= s.windowChangeAt {
cfg := h.trigger.Config()
cfg.windowSec = s.windowChangeTo
h.trigger.SetConfig(cfg)
h.trigger.Arm()
changed = true
}
if armed {
res.pulses += nPulseStarts
if nPulseStarts > 0 && h.trigger.State() == trigArmed {
h.trigger.mu.Lock()
f := h.trigger.fillLocked()
h.trigger.mu.Unlock()
if f < 1 {
res.gatedPulses += nPulseStarts
}
}
}
h.ingest(key, 1, ts, vs)
// Mirror triggerTick, on the simulated clock.
tick := now + s.batchSec
h.retuneRings(tick)
_, span := h.rings[key].stats()
h.trigger.setBuffered(span, 0, false, true, tick)
if trigTime, pre, post, ok := h.trigger.dueCapture(tick); ok {
if buf := h.buildTriggerCapture(trigTime, pre, post); buf != nil {
res.shots++
res.trigTimes = append(res.trigTimes, trigTime)
first, last, _ := decodeCaptureSpan(t, buf, key)
if cov := (last - first) / (pre + post); cov < res.worstCov {
res.worstCov = cov
}
if _, _, ok := h.capture.slice(key, trigTime-pre, trigTime+post); !ok {
res.holdDeclined++
}
// What the client would actually draw, against what the window
// really contained. A wide window holds several pulses, and a
// frame showing only the one it triggered on has lost the rest
// between the ring, the bucketing and the decimation.
_, fv := decodeCaptureSig(t, buf, key)
res.drawnPulses += countPulses(fv, 0.5)
res.wantPulses += countPulseStarts(trigTime-pre, trigTime+post,
s.pulsePeriod, 1/s.rateHz)
}
h.trigger.markTriggered(tick)
} else if h.trigger.dueRearm(tick) {
h.trigger.rearm()
}
}
return res
}
// decodeCaptureSig pulls one signal's samples out of a v2 capture frame.
func decodeCaptureSig(t *testing.T, buf []byte, key string) (ts, vs []float64) {
t.Helper()
off := 1 + 8 + 8 + 8
nSig := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
for i := 0; i < nSig; i++ {
kl := int(binary.LittleEndian.Uint16(buf[off:]))
off += 2
k := string(buf[off : off+kl])
off += kl
cnt := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
if k == key {
ts = make([]float64, cnt)
vs = make([]float64, cnt)
for j := 0; j < cnt; j++ {
ts[j] = math.Float64frombits(binary.LittleEndian.Uint64(buf[off+j*8:]))
vs[j] = math.Float64frombits(binary.LittleEndian.Uint64(buf[off+cnt*8+j*8:]))
}
}
off += cnt * 16
}
return
}
// countPulses counts runs of samples at or above thr.
func countPulses(v []float64, thr float64) int {
n, in := 0, false
for _, x := range v {
if x >= thr {
if !in {
n, in = n+1, true
}
} else {
in = false
}
}
return n
}
// countPulseStarts is how many pulse starts fall inside [t0, t1]. A pulse
// starting within one sample of t1 is not counted: only its first sample is
// inside the window, and the ring's min/max bucket for it may put that sample's
// extremum just past the edge, which is a boundary artefact rather than a loss.
func countPulseStarts(t0, t1, period, dt float64) int {
n := 0
for k := math.Floor(t0 / period); k*period <= t1; k++ {
if p := k * period; p >= t0 && p < t1-2*dt {
n++
}
}
return n
}
// deadTimeSec is how long the FSM is blind after firing at t: it must acquire
// the post-trigger window before the capture can be harvested, and the holdoff
// guards against re-triggering on the same event. Both are measured from the
// trigger point, so they overlap rather than add.
func deadTimeSec(windowSec, prePercent, holdoffSec float64) float64 {
return math.Max(windowSec*(1-prePercent/100), holdoffSec)
}
// A trigger cannot show two windows at once, so pulses closer together than its
// post-trigger window are necessarily lost. Pulses spaced FURTHER apart than
// that are not: nothing about the acquisition prevents catching every one.
//
// This is the reported failure. The FSM used to go deaf from the trigger point
// until the capture had been harvested (a post-window plus captureMarginSec)
// and the holdoff had then elapsed on top of that, then wait for a fresh edge —
// so the effective spacing was rounded UP to a whole pulse period. At the
// default 1 s window and 0.2 s holdoff the blind stretch came to 1.15 s, which
// is longer than a 1 s pulse period by a hair, and a pulse train at 1 Hz was
// caught at 0.5 Hz. Widening the window made it worse in whole multiples.
func TestSporadicPulsesWiderThanThePostWindowAreAllCaught(t *testing.T) {
const key = "s1:Ch1"
cases := []struct{ window, period float64 }{
{0.5, 0.5}, // post 0.4 s
{1.0, 1.0}, // post 0.8 s — the case the report was made against
{2.0, 2.0}, // post 1.6 s
{5.0, 5.0}, // post 4.0 s
}
for _, c := range cases {
sim := pulseTrainSim{
rateHz: 1000, batchSec: 1.0 / 30.0,
pulsePeriod: c.period, pulseSamples: 3,
simSec: 41 * c.period, windowSec: c.window, prePercent: 20,
holdoffSec: autoRearmDelaySec, armAt: c.period,
}
res := sim.run(t, key)
// Two pulses are always in flight rather than caught: the one that lands
// as the trigger arms, and the one still being collected when the run
// ends.
if got := res.yield(); got < 0.94 {
t.Errorf("window %.1f s, pulse every %.1f s: caught %d of %d (%.0f%%); "+
"the post-trigger window is only %.2f s, so every pulse fits",
c.window, c.period, res.shots, res.pulses, 100*got,
c.window*0.8)
}
}
}
// The loss that remains must be the loss that has to remain. A capture cannot
// start before the previous one's post-window is acquired, and it can only start
// on a pulse, so consecutive captures are a dead time apart rounded UP to the
// next pulse — never further. Any longer gap means an edge that the acquisition
// no longer needed was thrown away anyway.
//
// The bound is stated as dead + period rather than ceil(dead/period)*period
// because when the two divide exactly, whether the pulse at the boundary counts
// comes down to the last bit of the sample timestamp. Both answers are correct;
// a gap beyond either is not.
func TestSporadicCaptureGapsStayWithinTheDeadTime(t *testing.T) {
const key = "s1:Ch1"
for _, window := range []float64{0.5, 1.0, 2.0, 5.0} {
for _, period := range []float64{0.25, 0.5, 1.0, 2.0} {
sim := pulseTrainSim{
rateHz: 1000, batchSec: 1.0 / 30.0,
pulsePeriod: period, pulseSamples: 3,
simSec: 60, windowSec: window, prePercent: 20,
holdoffSec: autoRearmDelaySec, armAt: 1.0,
}
res := sim.run(t, key)
dead := deadTimeSec(window, 20, autoRearmDelaySec)
limit := dead + period + 2*sim.batchSec
worst, worstAt := 0.0, 0.0
for i := 1; i < len(res.trigTimes); i++ {
if g := res.trigTimes[i] - res.trigTimes[i-1]; g > worst {
worst, worstAt = g, res.trigTimes[i-1]
}
}
if worst > limit {
t.Errorf("window %.1f s, pulse every %.2f s: %.2f s between the captures "+
"at %.2f s and %.2f s; the dead time is only %.2f s, so %.2f s is the most "+
"that can be missed",
window, period, worst, worstAt, worstAt+worst, dead, limit)
}
t.Logf("window %.1f s, pulse every %.2f s: %d/%d captures (%.0f%%), "+
"dead time %.2f s, worst gap %.2f s, gated %d",
window, period, res.shots, res.pulses, 100*res.yield(), dead,
worst, res.gatedPulses)
}
}
}
// Whatever the trigger does catch has to come back whole: a window wide enough
// to hold several pulses must show all of them, at every rate, including the
// rates that force the ring into min/max bucketing.
func TestSporadicCaptureShowsEveryPulseInItsWindow(t *testing.T) {
const key = "s1:Ch1"
for _, rate := range []float64{1000, 200e3} {
for _, window := range []float64{1.0, 2.0, 5.0} {
sim := pulseTrainSim{
rateHz: rate, batchSec: 1.0 / 30.0,
pulsePeriod: 0.5, pulseSamples: 3,
simSec: 40, windowSec: window, prePercent: 20,
holdoffSec: autoRearmDelaySec, armAt: 1.0,
}
res := sim.run(t, key)
if res.shots == 0 {
t.Fatalf("rate %.0f window %.1f s: no captures at all", rate, window)
}
// wantPulses excludes the pulse straddling each window's far edge,
// whose bucket may place its extremum just past it, so the frames
// may legitimately draw a few more than that — but never fewer.
if res.drawnPulses < res.wantPulses {
t.Errorf("rate %.0f window %.1f s: frames drew %d pulses, their windows held %d",
rate, window, res.drawnPulses, res.wantPulses)
}
if res.worstCov < 0.98 {
t.Errorf("rate %.0f window %.1f s: worst capture spanned %.0f%% of its window",
rate, window, 100*res.worstCov)
}
if res.holdDeclined > 0 {
t.Errorf("rate %.0f window %.1f s: the zoom hold declined %d of %d captures",
rate, window, res.holdDeclined, res.shots)
}
}
}
}
// Widening the window mid-run is the gesture the report came from. The fill gate
// holds the first shot off until the ring reaches back far enough, which is
// correct; what it must not do is stay shut, nor leave the trigger losing pulses
// once the ring has caught up.
func TestSporadicYieldRecoversAfterAWindowChange(t *testing.T) {
const key = "s1:Ch1"
for _, w := range []float64{1.0, 2.0, 5.0} {
sim := pulseTrainSim{
rateHz: 200e3, batchSec: 1.0 / 30.0,
pulsePeriod: w, pulseSamples: 3,
simSec: 30 * w, windowSec: 0.2, prePercent: 20,
holdoffSec: autoRearmDelaySec, armAt: 1.0,
windowChangeAt: 10 * w, windowChangeTo: w,
}
res := sim.run(t, key)
// Count only what happened after the change settled.
after, want := 0, 0
for _, tt := range res.trigTimes {
if tt > 11*w {
after++
}
}
for p := 11 * w; p < 30*w; p += w {
want++
}
if float64(after) < 0.9*float64(want) {
t.Errorf("window 0.2 -> %.1f s: %d captures in the %d pulses after the change",
w, after, want)
}
}
}
Common/Client/go/wshub/trigger_test.go
+559
View File
@@ -0,0 +1,559 @@
package wshub
import (
"encoding/json"
"math"
"testing"
"time"
)
func TestParseSignalKey(t *testing.T) {
cases := []struct {
in string
base string
idx int
}{
{"src:sig", "src:sig", -1},
{"src:sig[0]", "src:sig", 0},
{"src:sig[3]", "src:sig", 3},
{"src:sig[x]", "src:sig[x]", -1},
{"src:sig]", "src:sig]", -1},
}
for _, c := range cases {
base, idx := parseSignalKey(c.in)
if base != c.base || idx != c.idx {
t.Errorf("parseSignalKey(%q) = (%q,%d), want (%q,%d)",
c.in, base, idx, c.base, c.idx)
}
}
}
func armed(key, edge string, thr float64) *triggerEngine {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: key, edge: edge, threshold: thr,
windowSec: 1, prePercent: 20, mode: "normal", holdoffSec: autoRearmDelaySec})
te.Arm()
return te
}
func TestFeedRisingEdge(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{1, 2, 3, 4}, []float64{0, 0.2, 0.9, 1.0})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
// Fires at the sample that crossed, i.e. t=3.
trigTime, pre, post, ok := te.dueCapture(1e9)
if !ok || trigTime != 3 {
t.Fatalf("dueCapture = (%v,%v), want trigTime 3", trigTime, ok)
}
if pre != 0.2 || post != 0.8 {
t.Errorf("pre/post = %v/%v, want 0.2/0.8", pre, post)
}
}
func TestFeedFallingEdgeIgnoresRising(t *testing.T) {
te := armed("src:sig", "falling", 0.5)
te.feed("src:sig", 1, []float64{1, 2, 3}, []float64{0, 0.9, 1.0})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed (no falling edge)", te.State())
}
te.feed("src:sig", 1, []float64{4, 5}, []float64{0.6, 0.1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
}
func TestFeedIgnoresOtherSignals(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:other", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed", te.State())
}
}
func TestFeedArrayElementSelection(t *testing.T) {
// 2-element signal, element-major: [e0,e1, e0,e1, ...]. Only element 1
// crosses the threshold.
te := armed("src:sig[1]", "rising", 0.5)
tt := []float64{1, 1, 2, 2}
vv := []float64{0, 0, 0, 1}
te.feed("src:sig", 2, tt, vv)
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
// Element 0 never crosses, so a config on [0] must not fire.
te2 := armed("src:sig[0]", "rising", 0.5)
te2.feed("src:sig", 2, tt, vv)
if te2.State() != trigArmed {
t.Fatalf("state = %q, want armed", te2.State())
}
}
func TestForceUsesLastSampleTime(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising", threshold: 1e9,
windowSec: 2, prePercent: 50, mode: "single"})
te.Arm()
te.feed("src:sig", 1, []float64{10, 11, 12}, []float64{0, 0, 0})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed (threshold unreachable)", te.State())
}
te.Force()
// post = 1 s, so the capture waits for samples past t = 12 + 1 + 0.15.
te.feed("src:sig", 1, []float64{13.2}, []float64{0})
trigTime, pre, post, ok := te.dueCapture(1e9)
if !ok || trigTime != 12 || pre != 1 || post != 1 {
t.Fatalf("dueCapture = (%v,%v,%v,%v), want (12,1,1,true)",
trigTime, pre, post, ok)
}
}
func TestForceFromIdle(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising",
windowSec: 1, prePercent: 20, mode: "normal"})
te.Force()
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
}
func TestCaptureMarginDelaysExtraction(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1}) // fires at t=1
// post = 0.8 s; capture is due once the samples reach 1 + 0.8 + 0.15.
te.feed("src:sig", 1, []float64{1.9}, []float64{0})
if _, _, _, ok := te.dueCapture(1e9); ok {
t.Error("capture extracted before the margin elapsed")
}
te.feed("src:sig", 1, []float64{1.96}, []float64{0})
if _, _, _, ok := te.dueCapture(1e9); !ok {
t.Error("capture not extracted after the margin elapsed")
}
}
// A stream whose timestamps run behind real time must still yield the whole
// window: measuring the post-window on the wall clock cut the capture short by
// exactly the lag (an 8 s lag turned a 60 s window into a 36 s one).
func TestCaptureWaitsForLaggingStream(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1}) // fires at t=1
wallNow := float64(time.Now().UnixNano()) / 1e9
// Wall clock is far past the post-window, but the samples are not.
te.feed("src:sig", 1, []float64{1.5}, []float64{0})
if _, _, _, ok := te.dueCapture(wallNow); ok {
t.Error("capture extracted while the stream was still short of the window")
}
te.feed("src:sig", 1, []float64{2.0}, []float64{0})
if _, _, _, ok := te.dueCapture(wallNow); !ok {
t.Error("capture not extracted once the samples covered the window")
}
}
// A dead stream must not leave the client stuck in "collecting" forever.
func TestCaptureCompletesWhenStreamStalls(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1}) // fires at t=1
wallNow := float64(time.Now().UnixNano()) / 1e9
if _, _, _, ok := te.dueCapture(wallNow + captureStallSec/2); ok {
t.Error("capture extracted before the stall timeout")
}
if _, _, _, ok := te.dueCapture(wallNow + captureStallSec + 0.1); !ok {
t.Error("capture not extracted after the stream stalled")
}
}
func TestAutoRearmNormalMode(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1})
te.markTriggered(100)
if te.State() != trigTriggered {
t.Fatalf("state = %q, want triggered", te.State())
}
if te.dueRearm(100.1) {
t.Error("rearmed before the delay elapsed")
}
if !te.dueRearm(100.3) {
t.Error("did not rearm after the delay elapsed")
}
if te.dueRearm(200) {
t.Error("rearm was not consumed")
}
}
func TestNoAutoRearmInSingleMode(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising", threshold: 0.5,
windowSec: 1, prePercent: 20, mode: "single"})
te.Arm()
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1})
te.markTriggered(100)
if te.dueRearm(200) {
t.Error("single mode must not auto-rearm")
}
}
func TestStoppedSuppressesRearm(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1})
te.SetStopped(true)
te.markTriggered(100)
if te.dueRearm(200) {
t.Error("stopped engine must not rearm")
}
}
func TestSetConfigClamps(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 1000, prePercent: 500, holdoffSec: 120})
if cfg := te.Config(); cfg.windowSec != 600 || cfg.prePercent != 100 || cfg.holdoffSec != 60 {
t.Errorf("upper clamp = %v/%v/%v, want 600/100/60", cfg.windowSec, cfg.prePercent, cfg.holdoffSec)
}
// The web UI's longest option must survive intact — it used to be clamped
// to 60 s, so a 10 min capture silently came back one minute long.
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 600, prePercent: 20, holdoffSec: 1})
if cfg := te.Config(); cfg.windowSec != 600 {
t.Errorf("windowSec = %v, want the requested 600", cfg.windowSec)
}
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 0, prePercent: -5, holdoffSec: -1})
if cfg := te.Config(); cfg.windowSec != 1e-4 || cfg.prePercent != 0 || cfg.holdoffSec != 0 {
t.Errorf("lower clamp = %v/%v/%v, want 1e-4/0/0", cfg.windowSec, cfg.prePercent, cfg.holdoffSec)
}
}
func TestHoldoffControlsRearmDelay(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising", threshold: 0.5,
windowSec: 1, prePercent: 20, mode: "normal", holdoffSec: 5})
te.Arm()
te.feed("src:sig", 1, []float64{0, 1}, []float64{0, 1})
te.markTriggered(100)
if te.dueRearm(104.9) {
t.Error("rearmed before the configured holdoff elapsed")
}
if !te.dueRearm(105.1) {
t.Error("did not rearm after the configured holdoff elapsed")
}
}
func TestActiveTracksConfiguredSignal(t *testing.T) {
te := newTriggerEngine()
if te.Active() {
t.Error("a fresh engine must not be active")
}
te.SetConfig(trigConfig{signalKey: "src:sig", windowSec: 1})
if !te.Active() {
t.Error("engine must be active once a signal is configured")
}
// Rings must keep filling after a capture completes, not just while armed.
te.Disarm()
if !te.Active() {
t.Error("engine must stay active after disarm while a signal is set")
}
}
// The armed→collecting transition happens inside feed(), on the ingest path,
// which the hub runs before triggerTick in the same loop iteration. Clients need
// that state — it carries trigTime and the latched window, without which they
// cannot draw the window filling and sit frozen until the capture arrives.
func TestCollectingIsBroadcast(t *testing.T) {
h := NewHub()
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", edge: "rising", threshold: 0,
windowSec: 10, prePercent: 20, mode: "single"})
h.trigger.Arm()
h.triggerTick()
drainStates(t, h)
// Fire, but stay well inside the post-trigger window: the capture is still
// seconds away and this is exactly when the client has nothing to draw.
h.ingest("s1:sig", 1, []float64{5.0, 5.001}, []float64{-1, 1})
h.triggerTick()
states := drainStates(t, h)
found := false
for _, m := range states {
if m["state"] == trigCollecting {
found = true
if m["trigTime"] != 5.001 {
t.Errorf("collecting broadcast has trigTime %v, want 5.001", m["trigTime"])
}
if m["preSec"] != 2.0 || m["postSec"] != 8.0 {
t.Errorf("collecting broadcast has pre=%v post=%v, want 2 and 8",
m["preSec"], m["postSec"])
}
}
}
if !found {
t.Fatalf("no collecting broadcast after the trigger fired, got %v", states)
}
}
// setFill hands the engine a buffer span and a growth rate, as the hub's
// per-tick measurements would: a reference point and a second one a second
// later. It forgets any earlier measurement first, so the rate is the one
// asked for rather than a blend with it.
func setFill(te *triggerEngine, span, growth, now float64) {
te.setBuffered(0, 0, false, false, now)
te.setBuffered(span-growth, 0, false, true, now)
te.setBuffered(span, 0, false, true, now+1)
}
// What has to hold is that the buffer spans the whole window by the time the
// capture is read, one post-window after the trigger fires — so whatever it
// will fill in on its own during that time need not be there yet.
func TestFillNeed(t *testing.T) {
cases := []struct {
window, prePercent, growth, want float64
}{
{100, 20, 1, 20}, // still filling: only the pre-window has to exist
{100, 20, 0.5, 60}, // half speed: 40 s of the 80 s post-window fills in
{100, 20, 0, 100}, // not growing at all: it must already be all there
{100, 0, 0.9, 10}, // no pre-window, but the buffer still has to keep up
{100, 100, 1, 100}, // all pre-window: nothing fills in after the trigger
}
for _, c := range cases {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", windowSec: c.window, prePercent: c.prePercent})
setFill(te, 1e6, c.growth, 100) // span large enough not to matter
te.mu.Lock()
got := te.fillNeedLocked()
te.mu.Unlock()
if math.Abs(got-c.want) > 1e-6 {
t.Errorf("fillNeed(window %v, pre %v%%, growth %v) = %v, want %v",
c.window, c.prePercent, c.growth, got, c.want)
}
}
}
// A trigger that fires before its pre-window has been buffered can only produce
// a capture whose front half never existed. It must wait instead.
func TestFillGateHoldsFire(t *testing.T) {
te := armed("src:sig", "rising", 0.5) // window 1 s, pre 20 % → 0.2 s needed
setFill(te, 0.05, 1, 100)
te.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed: only 0.05 s of the 0.2 s pre-window is buffered", te.State())
}
// The level was still tracked, so the next crossing is a real edge and not a
// re-detection of the one that was held off.
setFill(te, 0.25, 1, 200)
te.feed("src:sig", 1, []float64{3, 4}, []float64{1, 1})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed: no crossing, the signal stayed high", te.State())
}
te.feed("src:sig", 1, []float64{5, 6}, []float64{0, 1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting once the pre-window is buffered", te.State())
}
te.feed("src:sig", 1, []float64{7, 8}, []float64{1, 1}) // carry the sample clock past the window
if trigTime, _, _, ok := te.dueCapture(1e9); !ok || trigTime != 6 {
t.Errorf("dueCapture = (%v,%v), want trigTime 6", trigTime, ok)
}
}
// A ring that is full and re-bucketing for a longer window fills slower than
// real time — it drops dense old samples to take sparse new ones — so more of
// the window has to be there before an edge may be accepted.
func TestFillGateAccountsForSlowGrowth(t *testing.T) {
te := armed("src:sig", "rising", 0.5) // window 1 s, pre 20 % → post 0.8 s
// At half speed only 0.4 s of the post-window fills in, so 0.6 s is needed.
setFill(te, 0.5, 0.5, 100)
te.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigArmed {
t.Fatalf("state = %q, want armed: 0.5 s buffered of the 0.6 s needed", te.State())
}
// The same 0.5 s in a ring still filling at full speed is plenty: everything
// after the trigger is yet to be recorded anyway.
te2 := armed("src:sig", "rising", 0.5)
setFill(te2, 0.5, 1, 100)
te2.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te2.State() != trigCollecting {
t.Fatalf("state = %q, want collecting: the buffer keeps up with the stream", te2.State())
}
setFill(te, 0.65, 0.5, 200)
te.feed("src:sig", 1, []float64{3, 4}, []float64{0, 1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting once the buffer will span the window", te.State())
}
}
func TestFillGateInactiveWithoutMeasurement(t *testing.T) {
// No ring for the configured signal: gating would leave the trigger armed
// forever, which is worse than a short capture.
te := armed("src:sig", "rising", 0.5)
te.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting: nothing measured, so nothing to gate on", te.State())
}
// Nor is there anything to wait for when the buffer keeps up and the whole
// window is still to come.
te = newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "src:sig", edge: "rising", threshold: 0.5,
windowSec: 1, prePercent: 0, mode: "normal"})
te.Arm()
setFill(te, 0, 1, 100)
te.feed("src:sig", 1, []float64{1, 2}, []float64{0, 1})
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting with a 0 %% pre-window", te.State())
}
}
// Force is the user overriding the trigger, so it overrides the gate too.
func TestForceIgnoresFillGate(t *testing.T) {
te := armed("src:sig", "rising", 0.5)
setFill(te, 0, 0, 100)
te.Force()
if te.State() != trigCollecting {
t.Fatalf("state = %q, want collecting", te.State())
}
}
// seedFillNow is setFill against the real clock, for tests that then let the
// hub take its own measurements: its ticks land inside the growth measurement
// interval, so they refresh the span and leave the seeded rate alone.
func seedFillNow(te *triggerEngine, span, growth float64) {
now := float64(time.Now().UnixNano()) / 1e9
te.setBuffered(0, 0, false, false, now-1)
te.setBuffered(span-growth, 0, false, true, now-1)
te.setBuffered(span, 0, false, true, now)
}
// While it holds off, the trigger looks identical to one that is ignoring
// edges. The state broadcast has to say it is filling, and keep saying so.
func TestFillProgressIsBroadcast(t *testing.T) {
h := NewHub()
rb := newSigRing(1000)
h.rings["s1:sig"] = rb
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", edge: "rising", threshold: 0,
windowSec: 10, prePercent: 50, mode: "single"}) // 5 s of pre-window
h.trigger.Arm()
rb.write([]float64{0, 1}, []float64{-1, -1})
// Filling at the rate of the stream, so only the pre-window is needed.
seedFillNow(h.trigger, 1, 1)
h.triggerTick()
states := drainStates(t, h)
if len(states) == 0 {
t.Fatal("no state broadcast while the trigger was filling")
}
last := states[len(states)-1]
if last["state"] != trigArmed {
t.Fatalf("state = %v, want armed", last["state"])
}
if f, _ := last["bufferFill"].(float64); f < 0.19 || f > 0.21 {
t.Errorf("bufferFill = %v, want ~0.2 (1 s of 5 s)", last["bufferFill"])
}
if last["bufferNeedSec"] != 5.0 {
t.Errorf("bufferNeedSec = %v, want 5", last["bufferNeedSec"])
}
// An edge now is ignored: there is no 5 s of history to capture.
h.ingest("s1:sig", 1, []float64{1.5, 2.0}, []float64{-1, 1})
if h.trigger.State() != trigArmed {
t.Fatalf("state = %q, want armed: the pre-window is only 20 %% buffered", h.trigger.State())
}
// Progress is news even though the state has not moved.
rb.write([]float64{2, 3}, []float64{-1, -1})
h.triggerTick()
if states = drainStates(t, h); len(states) == 0 {
t.Fatal("no state broadcast as the pre-window filled further")
}
if f, _ := states[len(states)-1]["bufferFill"].(float64); f < 0.59 || f > 0.61 {
t.Errorf("bufferFill = %v, want ~0.6 (3 s of 5 s)", states[len(states)-1]["bufferFill"])
}
// Full: the gate opens, the fill disappears from the message and the next
// edge fires.
rb.write([]float64{4, 5.2}, []float64{-1, -1})
h.triggerTick()
states = drainStates(t, h)
if len(states) == 0 {
t.Fatal("no state broadcast when the pre-window filled")
}
if _, ok := states[len(states)-1]["bufferFill"]; ok {
t.Errorf("bufferFill still reported once the pre-window is buffered: %v", states[len(states)-1])
}
h.ingest("s1:sig", 1, []float64{5.3, 5.4}, []float64{-1, 1})
if h.trigger.State() != trigCollecting {
t.Fatalf("state = %q, want collecting once the pre-window is buffered", h.trigger.State())
}
}
// drainStates decodes every triggerState frame the hub has queued for
// broadcast. Hub.Run is what normally drains this queue, and it is not running
// in these tests.
func drainStates(t *testing.T, h *Hub) []map[string]any {
t.Helper()
var out []map[string]any
for {
select {
case msg := <-h.broadcastCh:
var m map[string]any
if err := json.Unmarshal(msg, &m); err != nil {
continue
}
if m["type"] == "triggerState" {
out = append(out, m)
}
default:
return out
}
}
}
// A ring whose coverage saturates below the window (measured source rate that
// over-estimates the true one) can never satisfy the full-window need. The
// coverage clamp must open the gate once the ring is full, delivering a short
// capture rather than staying deaf forever.
func TestFillNeedClampedToCoverage(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 60, prePercent: 20, mode: "normal", holdoffSec: 0.2})
setFill(te, 50, 0, 100) // ring full at 50 s, no growth
te.mu.Lock()
te.bufCoverage = 50 // the ring can never reach further back
te.mu.Unlock()
if need := te.fillNeedLocked(); need != 50 {
t.Errorf("need = %v, want 50 (clamped to coverage, not the 60 s window)", need)
}
if f := te.fillLocked(); f < 1 {
t.Errorf("fillLocked = %v, want >= 1: a full ring below the window must still open the gate", f)
}
// Without the clamp the gate would stay shut forever.
te.mu.Lock()
te.bufCoverage = 0
te.mu.Unlock()
if f := te.fillLocked(); f >= 1 {
t.Errorf("baseline: fillLocked = %v, want < 1 without a coverage clamp", f)
}
}
// When the disk archive already spans the window it can back-fill the front of
// a capture, so the gate must only require the ring to have reached the
// pre-window, not the whole window.
func TestFillNeedArchiveLowersToPreWindow(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 60, prePercent: 20, mode: "normal", holdoffSec: 0.2})
setFill(te, 30, 0, 100) // ring holds only 30 s, no growth → need 60 without archive
te.mu.Lock()
te.bufArchived = true
te.mu.Unlock()
if want := 12.0; te.fillNeedLocked() != want { // 60 * 0.20
t.Errorf("need = %v, want %v (archive lowers to the pre-window)", te.fillNeedLocked(), want)
}
// A ring holding just the pre-window opens the gate once archived.
te.mu.Lock()
te.bufSpan = 12
te.mu.Unlock()
if f := te.fillLocked(); f < 1 {
t.Errorf("fillLocked = %v, want >= 1 with pre-window buffered and the archive available", f)
}
}
Common/Client/go/wshub/zoom_test.go
+120
View File
@@ -0,0 +1,120 @@
package wshub
import (
"math"
"testing"
)
// A scope's envelope must not lose a spike, however narrow, and must stay in
// time order so it can be plotted as a single trace.
func TestMinMaxDecimateKeepsExtremes(t *testing.T) {
const n = 10000
ts := make([]float64, n)
vs := make([]float64, n)
for i := range ts {
ts[i] = float64(i) * 1e-6
vs[i] = math.Sin(float64(i) * 0.01)
}
// A one-sample spike in each direction: exactly what plain decimation drops.
vs[4321] = 12.5
vs[6789] = -9.75
dt, dv := minMaxDecimate(ts, vs, 200)
if len(dt) > 200 || len(dt) != len(dv) {
t.Fatalf("got %d t / %d v points, want <= 200 of each", len(dt), len(dv))
}
hiSeen, loSeen := false, false
for i := range dv {
switch dv[i] {
case 12.5:
hiSeen = true
if dt[i] != ts[4321] {
t.Errorf("spike kept at t=%v, want %v: timestamps must be the real ones", dt[i], ts[4321])
}
case -9.75:
loSeen = true
}
if i > 0 && dt[i] < dt[i-1] {
t.Fatalf("output is not time-ordered at %d: %v after %v", i, dt[i], dt[i-1])
}
}
if !hiSeen || !loSeen {
t.Errorf("envelope lost a spike (max kept=%v, min kept=%v)", hiSeen, loSeen)
}
}
func TestMinMaxDecimatePassesShortInputThrough(t *testing.T) {
ts := []float64{1, 2, 3}
vs := []float64{4, 5, 6}
dt, dv := minMaxDecimate(ts, vs, 200)
if len(dt) != 3 || dv[2] != 6 {
t.Errorf("input below the budget was altered: %v / %v", dt, dv)
}
// A flat bucket contributes one point, not two: nothing is invented.
flatT := make([]float64, 100)
flatV := make([]float64, 100)
for i := range flatT {
flatT[i] = float64(i)
}
if ft, _ := minMaxDecimate(flatT, flatV, 10); len(ft) != 5 {
t.Errorf("flat input decimated to %d points, want 5 (one per bucket)", len(ft))
}
}
func TestZoomPoints(t *testing.T) {
cases := []struct {
n int
present bool
want int
}{
{0, false, 2400}, // absent → default budget
{2400, true, 2400}, // explicit budget honoured
{0, true, 1 << 30}, // 0 → every sample in range
{-1, true, 1 << 30}, // negative → every sample in range
{5, true, 2400}, // implausibly small → default budget
}
for _, c := range cases {
if got := zoomPoints(c.n, c.present); got != c.want {
t.Errorf("zoomPoints(%d,%v) = %d, want %d", c.n, c.present, got, c.want)
}
}
}
func TestZoomSliceReturnsFullResolution(t *testing.T) {
h := NewHub()
rb := newSigRing(1000)
ts := make([]float64, 500)
vs := make([]float64, 500)
for i := range ts {
ts[i] = float64(i) * 0.001 // 1 kHz
vs[i] = float64(i)
}
rb.write(ts, vs)
h.rings["s1:sig"] = rb
// A budget larger than the range must return every sample untouched.
res := h.zoomSlice(0.100, 0.199, []string{"s1:sig"}, 1<<30)
sd, ok := res["s1:sig"]
if !ok {
t.Fatal("signal missing from zoom result")
}
if len(sd.T) != 100 {
t.Fatalf("got %d points, want 100", len(sd.T))
}
if sd.V[0] != 100 || sd.V[99] != 199 {
t.Errorf("value range = %v..%v, want 100..199", sd.V[0], sd.V[99])
}
// A small budget decimates but keeps the endpoints.
dec := h.zoomSlice(0.100, 0.199, []string{"s1:sig"}, 20)
if len(dec["s1:sig"].T) != 20 {
t.Errorf("decimated to %d points, want 20", len(dec["s1:sig"].T))
}
}
func TestZoomSliceUnknownSignal(t *testing.T) {
h := NewHub()
if res := h.zoomSlice(0, 1, []string{"nope", ""}, 100); len(res) != 0 {
t.Errorf("got %d entries, want 0", len(res))
}
}
Common/UDP/UDPSProtocol.h
+25 -4
View File
@@ -22,16 +22,23 @@
* CONFIG payload: * CONFIG payload:
* [uint32 numSigs] * [uint32 numSigs]
* numSigs × UDPSSignalDescriptor (136 bytes each, packed) * numSigs × UDPSSignalDescriptor (136 bytes each, packed)
* [uint8 publishMode] (PublishModeStrict / Accumulate / Decimate) * [uint8 publishMode] (PublishModeStrict / Accumulate / Decimate)
* [uint64 hrtFrequency] ticks per second of the producer's HRT
*
* Everything after the descriptors is an optional trailer: a receiver must
* accept a payload that stops early and must ignore bytes it does not know.
* publishMode defaults to Strict when absent, hrtFrequency to
* UDPS_HRT_FREQUENCY_UNKNOWN.
* *
* DATA payload (Strict / Decimate): * DATA payload (Strict / Decimate):
* [uint64 HRT timestamp] * [uint64 HRT timestamp]
* per-signal data in CONFIG order (quantised or raw, no padding) * per-signal data in CONFIG order (quantised or raw, no padding)
* *
* DATA payload (Accumulate): * DATA payload (Accumulate):
* [uint64 HRT timestamp] * [uint64 HRT timestamp of the first slot in the batch]
* [uint32 numSamples] * [uint32 numSamples] RT cycles accumulated into this packet
* for each signal: if scalar → numSamples elements; else → NumElements once * for each signal, in CONFIG order: numSamples × NumElements values
* (signal-major, one full snapshot per accumulated cycle)
*/ */
#ifndef UDPS_PROTOCOL_H_ #ifndef UDPS_PROTOCOL_H_
@@ -123,6 +130,20 @@ static const uint8 UDPS_PUBLISH_STRICT = 0u; ///< One packet per Synchronise
static const uint8 UDPS_PUBLISH_ACCUMULATE = 1u; ///< Variable batch; flush on size or time static const uint8 UDPS_PUBLISH_ACCUMULATE = 1u; ///< Variable batch; flush on size or time
static const uint8 UDPS_PUBLISH_DECIMATE = 2u; ///< One packet per Ratio calls static const uint8 UDPS_PUBLISH_DECIMATE = 2u; ///< One packet per Ratio calls
/*---------------------------------------------------------------------------*/
/* HRT frequency (CONFIG trailing uint64) */
/*---------------------------------------------------------------------------*/
/**
* Sentinel for a CONFIG that carries no HRT frequency, either because the
* trailer is absent (producer older than this field) or because the producer
* could not determine it. DATA timestamps are raw ticks of the producer's
* high-resolution timer, so without this a receiver on another host has no
* way to turn them into seconds and can only fall back to its own timer's
* frequency — which is right only while the two happen to agree.
*/
static const uint64 UDPS_HRT_FREQUENCY_UNKNOWN = 0u;
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
/* CONFIG payload — per-signal descriptor */ /* CONFIG payload — per-signal descriptor */
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
Docs/Protocol.md
+56 -1
View File
@@ -84,8 +84,28 @@ Offset Size Type Field
0xFFFFFFFF = PacketTime (no reference) 0xFFFFFFFF = PacketTime (no reference)
104 32 char[32] unit null-terminated physical unit string 104 32 char[32] unit null-terminated physical unit string
── (total per signal: 136 bytes) ──────────────────────────── ── (total per signal: 136 bytes) ────────────────────────────
── trailer, immediately after the last descriptor ───────────
0 1 uint8 publishMode 0 = Strict, 1 = Accumulate, 2 = Decimate
1 8 uint64 hrtFrequency producer's HRT ticks per second;
0 = unknown
``` ```
### CONFIG trailer
Everything after the descriptors is a trailer that grew field by field, so a
receiver must accept a payload that stops early and must ignore bytes it does
not recognise. An absent `publishMode` means Strict; an absent or zero
`hrtFrequency` means the producer did not publish its tick rate.
`hrtFrequency` is what makes DATA timestamps interpretable off-box. DATA
carries the raw value of the producer's high-resolution counter, and on x86
that counter runs at the TSC frequency — a different number on every model. A
receiver that divides by its own timer's frequency instead is right only while
producer and consumer sit on the same host; anywhere else every batch is laid
out over the wrong span of time. Fall back to the local frequency only when the
field is missing, and reject implausible values (nothing below 1 kHz is a
high-resolution timer).
### Type Codes ### Type Codes
| Code | C type | Bytes/element | | Code | C type | Bytes/element |
@@ -129,7 +149,9 @@ After reassembly, the DATA payload layout is:
``` ```
Offset Size Type Field Offset Size Type Field
────── ──── ────── ──────────────────────────────────────────────────── ────── ──── ────── ────────────────────────────────────────────────────
0 8 uint64 hrtTimestamp hardware reference timer count at Synchronise() 0 8 uint64 hrtTimestamp producer's high-resolution counter at
Synchronise(); divide by the CONFIG
hrtFrequency to get seconds
── for each signal (in config order) ──────────────────────────────────── ── for each signal (in config order) ────────────────────────────────────
varies N×sz — signal data N = numRows×numCols, sz = element size varies N×sz — signal data N = numRows×numCols, sz = element size
(wire size if quantized, raw size otherwise) (wire size if quantized, raw size otherwise)
@@ -171,6 +193,39 @@ the client to reassemble them in any order.
--- ---
## Ordering DATA (required of every receiver)
DATA carries its own `counter` sequence, incremented once per sent packet
(CONFIG is numbered independently). Reassembly completes in arrival order, not
counter order, so a packet reordered or duplicated on the wire surfaces after a
newer one has already been consumed. Its values are well-formed but carry an
older time base: accepting it writes them over samples the consumer already
holds and leaves the span they should have filled empty — a collision on one
side and a hole on the other.
A receiver must therefore drop any DATA packet that does not advance the
counter, and must order it by the *signed* difference:
```c
int32_t delta = (int32_t)(counter - lastCounter); /* survives the uint32 wrap */
if (delta <= 0) { /* stale or duplicate: drop */ }
lost = (uint32_t)delta - 1u; /* packets missing before this one */
```
Comparing the values directly would call the first packet after the wrap stale
and reject the stream from then on.
`lost` matters beyond diagnostics. A consumer that spaces batched samples from
the elapsed time since the previous packet must divide that gap by `lost + 1`
batches; dividing by one batch reports a period too long by exactly that factor
and walks the samples past their own end into the next packet's range. Reset
the sequence on (re)connect: the producer's counter restarts independently.
Implemented in `UDPSClient::AcceptDataCounter` (C++),
`udpsprotocol.SequenceGate` (Go) and `decode_data` (C).
---
## Minimal Python Client Example ## Minimal Python Client Example
```python ```python
Docs/StreamHub-API.md
+177 -9
View File
@@ -46,8 +46,61 @@ Reply (unicast): `{"type":"pong"}`.
```json ```json
{"type":"saveSources"} {"type":"saveSources"}
``` ```
Persists the current dynamically-added source list to the hub's `SourcesFile` Writes the hub's `SourcesFile`: the current dynamically-added source list **and**
(JSON array of `{label,addr,multicastGroup,dataPort}`); it is reloaded at startup. the calibration table, as one flat JSON array (see [§4](#4-config-file-format)).
The hub replies with [`configSaved`](#configsaved). Despite the name, this
command persists the whole config, not just the sources.
### `setCalibration`
```json
{"type":"setCalibration","source":"wave","signal":"Adc","scale":0.00030518,"offset":-1.25,"unit":"V"}
```
Records an affine calibration `value = raw × scale + offset` for one signal,
keyed by the source's **label** (not its runtime id) and the **base** signal
name — one entry covers every element of an array signal.
| Field | Type | Default | Validation |
|---|---|---|---|
| `source` | string | — | non-empty after trimming |
| `signal` | string | — | non-empty after trimming; any trailing `[i]` is stripped |
| `scale` | number | `1` | finite and non-zero |
| `offset` | number | `0` | finite |
| `unit` | string | `""` | trimmed, truncated to 16 UTF-8 bytes (a multi-byte character such as `°C` consumes more than one byte; truncation never splits a character); empty = use the streamer's own unit |
Calibration is **metadata only**: the hub stores and redistributes it but never
applies it. Ring buffers, recorded history, the `zoom` reply, both binary frames
and the trigger comparator all stay in raw units — a client that ignores
calibration behaves exactly as before.
An entry that reduces to the identity (`scale = 1`, `offset = 0`, `unit = ""`) is
**deleted** rather than stored, so a reset leaves no residue in the config file.
On acceptance the hub broadcasts [`calibration`](#calibration) to every client. A
rejected entry produces **no** broadcast, so the offending client reverts to the
last value it was told.
### `reloadConfig`
```json
{"type":"reloadConfig"}
```
Re-reads `SourcesFile` and then:
- **replaces** the calibration table wholesale with the file's contents;
- **adds** any source in the file that is not already active;
- **never** removes, restarts or reconnects a live source.
The asymmetry is deliberate: calibration is cheap to reapply, whereas a source is
a live UDP session that must not be interrupted. An unsaved source the user added
keeps streaming.
The hub replies with [`configReloaded`](#configreloaded), followed on success by a
`calibration` broadcast. Whether a `sources` broadcast follows depends on the
hub implementation: the Go hub emits `sources` only when the file adds at least
one new source (each `sm.Add()` call triggers it individually), while the C++
hub always emits `sources` unconditionally after a successful reload. Clients
must therefore tolerate an unsolicited `sources` frame after any reload.
### `getSources` / `getConfig` / `getStats` ### `getSources` / `getConfig` / `getStats`
@@ -67,8 +120,11 @@ Force a broadcast of the corresponding event.
- `signal` — full key `src:sig`, or `src:sig[i]` to trigger on element *i* of a - `signal` — full key `src:sig`, or `src:sig[i]` to trigger on element *i* of a
multi-element PACKET signal. multi-element PACKET signal.
- `edge` — `"rising"`, `"falling"` or `"both"`. - `edge` — `"rising"`, `"falling"` or `"both"`.
- `windowSec` — total capture window (clamped to 1e-4 … 10 s). - `windowSec` — total capture window. `preSec = windowSec * prePercent / 100`,
`preSec = windowSec * prePercent / 100`, `postSec = windowSec − preSec`. `postSec = windowSec − preSec`. Clamped to 1e-4 … 600 s by the Go hub and to
1e-4 … 60 s by the C++ one: the Go rings store min/max pairs once a window
outgrows their memory budget, so a long window costs resolution, while the C++
rings are fixed-capacity and would return the window truncated instead.
- `mode` — `"normal"` (auto-rearm ~200 ms after capture) or `"single"` - `mode` — `"normal"` (auto-rearm ~200 ms after capture) or `"single"`
(stays TRIGGERED until `rearm`). (stays TRIGGERED until `rearm`).
@@ -122,6 +178,33 @@ Every transition is broadcast as a `triggerState` event.
Request the hub to send a `historyInfo` event (unicast). Also sent automatically Request the hub to send a `historyInfo` event (unicast). Also sent automatically
on client connect. on client connect.
### `setHistoryBudget` (Go hub only)
```json
{"type":"setHistoryBudget","maxMPtsPerSignal":16.0}
```
Sets the per-signal archive budget in millions of stored points, the runtime
equivalent of `-history-max-mpts`. `0` restores the 16 MPts default; the hub
clamps to its own ceiling. Every archive file is re-created at the new size —
**the archived samples are lost**, because a file's capacity and min/max bucket
width are fixed at creation. Broadcasts `historyInfo` rather than answering the
requester alone: every client's view of what history exists has been invalidated.
### `setWindow` (Go hub only)
```json
{"type":"setWindow","seconds":60}
```
Reports how far back this client is plotting. The hub sizes its in-memory
buffers for the **widest** window any connected client has reported (10 s if
none has), bucketing each ring as min/max pairs when the window is too long to
hold verbatim — see *In-memory buffer policy* in
[StreamHub-Developer.md](StreamHub-Developer.md). While a trigger is armed the
trigger's own window wins. No reply; send it on connect and whenever the
timescale changes. A window the hub is not told about is a window whose start
may already have rolled out of the ring, leaving a `zoom` over it nothing to
answer with.
### `setMaxPoints` ### `setMaxPoints`
```json ```json
@@ -176,6 +259,20 @@ Sent at `StatsRate` Hz (default 1 Hz):
`state` ∈ `idle | armed | collecting | triggered`; `trigTime` present once a `state` ∈ `idle | armed | collecting | triggered`; `trigTime` present once a
trigger has fired. trigger has fired.
The Go hub adds `bufferFill` (0…1) and `bufferNeedSec` while `state` is `armed`
**and** its buffers do not yet reach back far enough to deliver a whole window.
Edges are ignored until they do, so that no capture arrives with a front that
was never recorded; the fields are absent once the requirement is met.
`bufferNeedSec` is how far back the hub must reach *now*, which is less than the
window by however much its buffers will fill in on their own while the
post-trigger window is collected: the pre-trigger span while they keep up with
the stream, and up to the whole `windowSec` when they do not (a full ring
re-bucketing for a longer window fills slower than real time, so the front of a
capture recedes while it is being collected).
The event is re-broadcast as the fraction grows, so a client can show the
progress instead of an armed trigger that appears to be ignoring the signal.
`forceTrigger` fires regardless.
### `zoom` (reply) ### `zoom` (reply)
```json ```json
@@ -190,18 +287,26 @@ trigger has fired.
Sent on client connect (if history is enabled) and on `historyInfo` command: Sent on client connect (if history is enabled) and on `historyInfo` command:
```json ```json
{"type":"historyInfo","enabled":true,"durationHours":1.0,"decimation":10, {"type":"historyInfo","enabled":true,"windowSec":600.0,"decimation":10,
"maxMPtsPerSignal":16.777216,
"signals":{ "signals":{
"scalar:Sine1":{"t0":1765360000.0,"t1":1765370000.0,"count":360000,"capacity":360000}, "scalar:Sine1":{"t0":1765360000.0,"t1":1765370000.0,"count":360000,"capacity":360000,"bucket":1},
"scalar:Sine2":{"t0":1765360000.0,"t1":1765370000.0,"count":360000,"capacity":360000}}} "scalar:Sine2":{"t0":1765360000.0,"t1":1765370000.0,"count":360000,"capacity":360000,"bucket":1}}}
``` ```
- `enabled` — `true` if the `+History` config block is present and valid. - `enabled` — `true` if the `+History` config block is present and valid.
- `durationHours` — configured history duration. - `windowSec` — the timespan the files are sized to hold, i.e. the live or
trigger window the clients are displaying (Go hub). The C++ StreamHub instead
keeps a fixed retention period and reports it as `durationHours`.
- `decimation` — samples-to-disk decimation factor (1 = every sample). - `decimation` — samples-to-disk decimation factor (1 = every sample).
- `maxMPtsPerSignal` — current per-signal budget, in millions of stored points
(Go hub only; see `setHistoryBudget`).
- `signals` — per-signal metadata keyed by `"sourceId:signalName"`: - `signals` — per-signal metadata keyed by `"sourceId:signalName"`:
- `t0`/`t1` — oldest/newest timestamp stored on disk (Unix seconds). - `t0`/`t1` — oldest/newest timestamp stored on disk (Unix seconds).
- `count` — number of valid entries currently in the circular file. - `count` — number of valid entries currently in the circular file.
- `capacity` — total capacity of the circular file. - `capacity` — total capacity of the circular file.
- `bucket` — source samples per stored min/max pair (Go hub only); `1` means
the signal is archived verbatim, higher means it is stored as an envelope
because it is too fast to fit the budget at full resolution.
### `historyZoom` (reply) ### `historyZoom` (reply)
@@ -219,6 +324,41 @@ If history is not enabled: `{"type":"historyZoom","error":"history not enabled"}
{"type":"maxPointsUpdated","maxPoints":50000} {"type":"maxPointsUpdated","maxPoints":50000}
``` ```
### `calibration`
```json
{"type":"calibration","cal":[
{"source":"wave","signal":"Adc","scale":0.00030518,"offset":-1.25,"unit":"V"}
]}
```
The complete calibration table. Broadcast when a client connects (as an empty
array when nothing is calibrated), after every accepted `setCalibration`, and
after a successful `reloadConfig`. It is a separate frame rather than a field on
`sources` because `sources` is serialised into a fixed 4 KiB buffer.
### `configSaved`
```json
{"type":"configSaved","ok":true,"path":"/etc/streamhub/sources.json"}
{"type":"configSaved","ok":false,"path":"","error":"no sources file configured"}
```
Broadcast in reply to `saveSources`. `path` is always present (empty when the hub
has no config file configured); `error` only when `ok` is false. The exact error
text is not part of the protocol contract and differs between hubs (the Go hub
uses `"no sources-file configured"`, the C++ hub `"no sources file configured"`).
### `configReloaded`
```json
{"type":"configReloaded","ok":true,"path":"/etc/streamhub/sources.json"}
{"type":"configReloaded","ok":false,"path":"/etc/streamhub/sources.json","error":"cannot read sources file"}
```
Broadcast in reply to `reloadConfig`; same shape as `configSaved`. On success
it is followed by a `calibration` broadcast. Whether a `sources` broadcast also
follows is hub-specific: the Go hub sends it only if the reload added at least
one new source; the C++ hub sends it unconditionally. Clients must tolerate an
unsolicited `sources` frame after any reload.
--- ---
## 3. Binary frames (hub → client) ## 3. Binary frames (hub → client)
@@ -270,7 +410,33 @@ per signal:
--- ---
## 4. Limits ## 4. Config file format
`SourcesFile` (C++ `SourcesFile` config key, Go `-sources-file` flag) is a flat
JSON array of flat objects. A block containing `addr` is a source; a block
containing `signal` is a calibration entry; anything else is skipped with a
warning.
```json
[
{"label": "wave", "addr": "127.0.0.1:44500"},
{"label": "mc", "addr": "127.0.0.1:44501", "multicastGroup": "239.0.0.1", "dataPort": 44502},
{"source": "wave", "signal": "Adc", "scale": 0.00030518, "offset": -1.25, "unit": "V"}
]
```
**Every object must stay flat.** The C++ `StreamHub::LoadSourcesFile` parser
takes each `{` up to the next `}` as one object, so a nested object anywhere in
the file would truncate the parse at the inner brace. A nested
`"calibration": {…}` inside a source entry is therefore not an option, and this
is why calibration entries are siblings of sources rather than children.
Files written by hub versions predating calibration load unchanged, and a file
written by either hub loads in the other.
---
## 5. Limits
| Limit | Value | | Limit | Value |
|-------|-------| |-------|-------|
@@ -278,3 +444,5 @@ per signal:
| UDPS source sessions | 32 | | UDPS source sessions | 32 |
| Max received WS payload | 64 KiB | | Max received WS payload | 64 KiB |
| Max sent WS payload | 4 MiB | | Max sent WS payload | 4 MiB |
| Calibration entries | 256 (C++ hub, `kMaxCalibration`); unbounded (Go hub) |
| Calibration unit override | 16 UTF-8 bytes |
Docs/StreamHub-Developer.md
+273 -11
View File
@@ -60,8 +60,20 @@ Each session calibrates per time-source:
`packetT = pktCalibOffset + hrt/hrtFreq`. `packetT = pktCalibOffset + hrt/hrtFreq`.
- Each referenced time signal gets its own offset on first value; - Each referenced time signal gets its own offset on first value;
`timerToSec = 1e-9` for `uint64` time signals, `1e-6` otherwise. `timerToSec = 1e-9` for `uint64` time signals, `1e-6` otherwise.
- Re-anchoring on reconnect, CONFIG change, or if computed time drifts > 2 s - The time-signal offset is **snapped** only on a genuine discontinuity in the
from wall clock (source restart / remote-vs-local HRT frequency drift). source: reconnect, CONFIG change, or the source clock jumping backward (a
looping/rewinding producer such as a rewinding `FileReader`).
- Plain *drift* — a source free-running on its own clock, or remote-vs-local HRT
frequency error — is **slewed**, not snapped. Past a 2 s threshold the offset
is nudged toward wall clock by at most 10 % of the packet's own duration.
Snapping instead would shift the whole published timeline in one step and so
tear a hole of exactly the drift into a stream that is in fact continuous;
a source drifting past the threshold repeatedly used to produce a train of
2 s holes. Drift is the honest reading, and the trade-off `TimeArrayGAM`'s
`Anchor = Continuous` explicitly asks for: a producer that cannot sustain its
nominal sample rate will fall progressively behind wall clock, and the hub
reports that rather than hiding it. The Go hub anchors once and never
re-anchors, so it never had the hole.
Per `timeMode`: Per `timeMode`:
@@ -94,16 +106,69 @@ Hub-side, web-client semantics (`setTrigger` fields in
[StreamHub-API.md](StreamHub-API.md)): [StreamHub-API.md](StreamHub-API.md)):
``` ```
IDLE --arm--> ARMED --edge crossing--> COLLECTING --wallNow ≥ trigTime+postSec+0.15s--> TRIGGERED IDLE --arm--> ARMED --edge crossing--> COLLECTING --every source past trigTime+postSec+0.15s--> TRIGGERED
TRIGGERED --rearm (single) / auto ~200ms (normal, unless stopped)--> ARMED TRIGGERED --rearm (single) / auto after holdoffSec (normal, unless stopped)--> ARMED
└─ or straight to COLLECTING on a held edge
any --disarm--> IDLE any --disarm--> IDLE
``` ```
`UDPSourceSession` calls `TriggerEngine::CheckSample` for every decoded sample `UDPSourceSession` calls `TriggerEngine::CheckSample` for every decoded sample
of the configured signal (signal index cached per config epoch). On of the configured signal (signal index cached per config epoch).
finalisation the push loop reads `[trigTime−preSec, trigTime+postSec]` from all
rings, LTTB-caps to 20 000 pts/signal and broadcasts a binary **version 2** The comparator keeps running through COLLECTING and TRIGGERED. It cannot fire
capture frame; every FSM transition broadcasts a `triggerState` event. there — the capture in flight owns that stretch — but it remembers the first
edge at or past `trigTime + max(postSec, holdoffSec)`, and `Rearm()` fires on
that remembered edge instead of waiting for a fresh one. Without this the engine
is deaf from its own trigger point until the capture has been harvested and the
holdoff has run, which on a sparse pulse train rounds the capture spacing up to
a whole pulse period: at a 1 s window a 1 Hz train was caught at 0.5 Hz, and a
wider window lost whole multiples. The capture is built from the edge's own
timestamp out of rings that still hold everything around it, so honouring it
costs nothing.
`Arm()` and `Rearm()` differ only in this: `Arm()` is the operator's own arm and
discards the held edge (they asked for the next event), while `Rearm()` is the
automatic end-of-capture arm and consumes it. `Rearm()` also keeps the tracked
level, so the first sample after it is compared against its real predecessor
rather than being spent seeding one. `SetConfig()` and `Disarm()` drop the held
edge as well — it was never judged against the new window. Each source is
read `[trigTime−preSec, trigTime+postSec]`, LTTB-capped to 20 000 pts/signal and
appended to a binary **version 2** capture frame; every FSM transition
broadcasts a `triggerState` event.
Once fired, that event carries `trigTime` **and** the window latched at fire
time (`preSec`/`postSec`). Clients draw the still-filling capture from their own
buffers on that axis long before the v2 frame arrives — for a long window at a
high rate the hub stays silent for seconds — and the trigger bar's window and
pre-% are editable, so without the latched values a client would place the
filling trace on whatever window the operator happened to be typing. Older hubs
omit both fields; clients fall back to their local config.
The COLLECTING deadline is on the **data's** clock, via
`UDPSourceSession::ProducerNewestTime()` — `trigTime` comes from sample
timestamps, and a source free-running on its own clock sits seconds away from
`clock_gettime()`, so a wall-clock deadline chops exactly that offset off every
capture's tail. Only signals actually timestamped from a time signal count
toward that reading: PACKET-timed ones (including the time array itself) are
stamped on arrival and would just report "now".
Sources are harvested independently — `BeginTriggerCapture`,
`HarvestTriggerCapture` per source as *it* becomes ready, `FinishTriggerCapture`
once all are in — with the frame accumulating in `capBuf_` across push ticks.
Waiting for the slowest source before reading any of them lets the leaders'
rings roll past the pre-trigger region first, losing the head of their traces. A
2 s wall-clock watchdog bounds the wait for a source that stopped advancing: it
is harvested short, with a warning naming the source and how far it got.
`setTrigger` also records the requested window, and each stats tick the push
loop runs `GrowRingsForTrigger()`. A ring whose measured rate
(`Count() / TimeSpan()`, since UDPS sources usually advertise
`samplingRate = 0`) cannot hold `window + 0.5 s` is grown in place to
`rate × (window + 0.5) × 1.2` points, clamped to `RingMaxMB` per signal.
`SignalRingBuffer::Grow()` copies oldest→newest and leaves `count` /
`totalWritten` untouched so the per-client push cursors survive the resize.
Rings never shrink; a hub left with a 5 s window on a 5 MSps source will sit at
the ceiling.
## 6. Configuration ## 6. Configuration
@@ -113,9 +178,16 @@ MaxPoints = 20000 // legacy global cap (overridable with -maxPoints)
PushRate = 30 // Hz PushRate = 30 // Hz
MaxPushPoints = 50 // per signal per push MaxPushPoints = 50 // per signal per push
StatsRate = 1 // Hz StatsRate = 1 // Hz
RingTemporal = 1000000 // ring capacity, temporal signals (pts) RingTemporal = 1000000 // initial ring capacity, temporal signals (pts)
RingScalar = 100000 // ring capacity, scalar/PACKET signals (pts) RingScalar = 100000 // ring capacity, scalar/PACKET signals (pts)
RingMaxMB = 128 // per-signal growth ceiling (MiB) for trigger windows
SourcesFile = "streamhub_sources.json" // saveSources persistence SourcesFile = "streamhub_sources.json" // saveSources persistence
AllowedOrigins = "http://127.0.0.1:8099,http://localhost:8099"
// comma/space-separated WebSocket Origin allowlist (max 8 × 128 chars).
// Without it the handshake only accepts an Origin whose host matches
// the request Host, so a browser serving the SPA from another port
// (run_streamhub.sh: SPA 8099, hub 8090) gets 403. Non-browser
// clients send no Origin and are unaffected.
Sources = { Sources = {
Src1 = { Label = "PSU" Addr = "127.0.0.1" Port = 44500 Src1 = { Label = "PSU" Addr = "127.0.0.1" Port = 44500
MulticastGroup = "239.0.0.1" DataPort = 44503 } // multicast optional MulticastGroup = "239.0.0.1" DataPort = 44503 } // multicast optional
@@ -145,6 +217,54 @@ Per-signal file capacity is computed at source CONFIG time:
`capacity = ceil(DurationHours × 3600 × samplingRate / Decimation)`, minimum `capacity = ceil(DurationHours × 3600 × samplingRate / Decimation)`, minimum
1000 pairs. 1000 pairs.
The Go hub (`Client/udpstreamer`) carries the same archive and the same file
format, configured with flags instead of a config node: `-history-dir`
(defaults to `<tmp>/udpstreamer-history`; empty disables),
`-history-window-sec`, `-history-decimation`, `-history-flush-sec`,
`-history-min-free-mb` (negative disables the check; 0 means the 500 MB default,
where the C++ `MinDiskFreeMB = 0` disables it) and `-history-max-mpts`, a
per-signal budget in millions of stored points, defaulting to 16 MPts (256 MB).
The budget exists because the timespan alone cannot bound the file: 600 s of a
1 MSps signal is 9.6 GB.
**The Go hub sizes its files from the window, not from a retention period.** The
archive exists to answer a zoom or a trigger capture after the in-memory rings
have rolled past it, and neither ever asks for more than the live or trigger
window — so a file holds `windowSec × rate` samples (plus 25 % headroom, since a
capture is read back a window after its first sample was written), and never
hours of them. Retaining an hour instead meant a 1 s live window was archived at
a thousandth of the resolution the same budget could have bought.
The budget is therefore spent on resolution, not on span. A signal too fast to
archive sample-for-sample within it is stored as a **min/max envelope**: `bucket`
source samples collapse to their two extremes, with `bucket` the narrowest that
makes the window fit. The `.shist` header's `decimation` field carries
`bucket × Decimation`, so a reader knows the stored resolution, and a file is
only reopened when it matches.
`Hub.retuneRings` re-sizes the files once a second alongside the rings, from the
same `activeWindowSec()`. A file's capacity and bucket are fixed at creation, so
a re-size discards what it held; two rules keep that rare. A file is only grown
when it no longer covers the window, and only shrunk when it is enveloped
(`bucket > 1`), covers more than twice the window, and a narrower bucket is
actually available — a file already at full resolution is left alone however
short the window becomes, so arming a 1 s trigger does not throw away the
seconds the capture is about to ask for. `historyInfo` is re-broadcast whenever a
re-size happens.
The budget is also settable at runtime from the web UI (the history badge in the
status bar) via the `setHistoryBudget` WS command; `historyInfo` reports it as
`maxMPtsPerSignal` and reports each signal's `bucket`. Changing it re-creates the
files, so the archived samples are lost — a file's capacity and bucket width are
fixed at creation and an existing envelope cannot be re-bucketed into a different
one.
History is on by default in the Go hub because it is what holds a trigger capture
at full resolution — see *Trigger captures* below. Signals whose producer
declares `samplingRate = 0` — every UDPS source — are not sized from a guess: the
file is opened only once the hub has measured the rate off the live stream, which
it retries once a second.
### `.shist` binary file format ### `.shist` binary file format
Each signal gets one file: `<Directory>/<sourceId>/<signalName>.shist`. Each signal gets one file: `<Directory>/<sourceId>/<signalName>.shist`.
@@ -178,13 +298,155 @@ reopened — head/count/time bounds are restored from the on-disk header.
`historyZoom` requests (see [StreamHub-API.md](StreamHub-API.md)) call `historyZoom` requests (see [StreamHub-API.md](StreamHub-API.md)) call
`HistoryWriter::ReadRange` which performs binary search over the circular file `HistoryWriter::ReadRange` which performs binary search over the circular file
using `pread` to locate the `[t0, t1]` window, then copies matching pairs. using `pread` to locate the `[t0, t1]` window, then copies matching pairs.
If the result exceeds the requested `n`, LTTB decimation is applied (same If the result exceeds the requested `n`, decimation is applied (same decimator
`LTTBDecimate` as in-memory zoom). as in-memory zoom: `LTTBDecimate` in the C++ hub, `minMaxDecimate` in the Go
one).
Both the web SPA and ImGui client issue `historyZoom` in parallel with regular Both the web SPA and ImGui client issue `historyZoom` in parallel with regular
`zoom` and merge the results: history covers the older part of the visible `zoom` and merge the results: history covers the older part of the visible
window, the in-memory ring covers the recent part. window, the in-memory ring covers the recent part.
A range wider than the read budget is thinned across its whole width with a
stride, not truncated at the front: answering a 10 s query with its first
few milliseconds reads as an empty plot to a client and sends it back to its
own coarse copy of the data.
### In-memory buffer policy (Go hub)
Each temporal signal gets one ring holding a fixed **budget** of `(t, v)` pairs:
10 M points, 160 MB, settable with `-ring-mpts`. Scalar signals keep a flat
100 000-packet ring, where a megasample budget would be waste. Rings start at
250 k points and are grown to the budget on demand, so a source that is
configured but never sends costs nothing.
Like the disk archive, the budget buys **resolution, not span**. Once a second
`retuneRings` compares the measured source rate against the window being
displayed and picks each ring's min/max `bucket`:
| condition | bucket | effect |
|---|---|---|
| `Sps × window ≤ budget` | 1 | stored verbatim; the ring reaches further back than the window, which is free zoom headroom |
| `Sps × window > budget` | `⌈2 × Sps × window × 1.25 ÷ capacity⌉` | `bucket` samples collapse to their two extremes, so the whole window fits |
A bucket costs two points (its minimum and its maximum), hence the factor 2 —
and why a bucket of 2 covers no more ground than a bucket of 1.
The window is the **trigger's** while a trigger is armed: its pre-window has to
already be in the ring when the trigger fires, or the capture has nothing to
back-fill from. Otherwise it is the widest window any connected client has
reported with the `setWindow` command, defaulting to 10 s for clients that never
send one. Sizing for the live window matters as much as for a capture: a fixed
sample-count ring covers ~6 s at 1 MSps, so a zoom on a 60 s timescale used to
come back with only its tail.
Retuning is hysteretic — a bucket is held while it covers the window without
covering more than twice it. Sharing one threshold for up and down makes a rate
jittering across a bucket boundary halve and double the stored resolution every
second.
Live pushes, the disk archive and the trigger comparator all see every sample:
`ingest` hands the raw batch to each, and only the ring's own copy is reduced.
### Trigger captures (Go hub)
A trigger capture is delivered as a decimated snapshot (20 000 points), so a
zoom into it has to come from full-resolution storage. The rings are tuned to
~1.25× the trigger window, so they roll past a captured window shortly after the
capture — and the trigger rearms and starts refilling them immediately.
**In-memory double buffer.** The rings are the write half; `captureHold`
(`capturehold.go`) is the read half. As `buildTriggerCapture` lifts each signal's
window out of its ring it publishes the *undecimated* slice into the hold, and
`zoomSlice` answers from the hold rather than the ring for any range the held
window fully contains. The swap happens only once the next capture is complete —
which is also the moment the client stops displaying the previous one — so the
shot being explored is never overwritten by the acquisition running behind it. A
capture that came back empty does not swap, so it cannot blank the window on
screen.
**Waiting for the buffer.** An armed trigger ignores edges until its buffers
reach back far enough for a capture taken now to come back whole (`fillLocked`
in `trigger.go`, fed by `refreshTriggerFill` from the trigger signal's own ring
— once per tick, and again on every trigger command so that an `arm` cannot
fire on a stale measurement). Firing earlier can only produce a capture whose
front was never recorded, which is what made the first shot after a widened
window come back short.
What must hold is that the buffer spans the whole window *at harvest time* — its
newest sample is then `trigTime + post`, so anything less has lost the front of
the capture. It keeps filling while the post-window is collected, so the
shortfall it may start with is what it will make up in that time, measured
rather than assumed:
```
need = windowSec − growth × postSec (floored at the pre-trigger window)
```
`growth` is the ring's span growth in seconds per second, sampled over at least
`bufGrowthIntervalSec` and smoothed. The three regimes fall out of the one
formula:
| ring | growth | needs |
|---|---|---|
| still filling | 1 | the pre-trigger window — everything after the trigger is yet to be recorded anyway |
| full, re-bucketing for a longer window | 0…1 | in between: it drops dense old samples to take sparse new ones, so it fills slower than real time and the front of the capture recedes while the post-window elapses |
| full, settled | 0 | the whole window — which a ring tuned for that window already exceeds, so nothing actually waits |
Measured at 1 MSps, widening 10 s → 30 s with 50 % pre: growth settles at ~0.65,
so `need` converges on ~20.4 s of the 30 s and the trigger fires ~12 s after
arming with a capture that is 100 % complete. Requiring the whole window instead
would have waited 26 s for the same result.
The gate measures the trigger signal's ring, not the narrowest of all of them: a
signal that never reaches back that far would otherwise stop the trigger from
ever firing. It is disabled outright when there is no ring to measure or nothing
is needed, and `forceTrigger` overrides it. While it holds off, `triggerState`
carries `bufferFill`/`bufferNeedSec` and is re-broadcast as the fraction climbs,
so the UI shows `ARMED 42%` rather than a trigger that looks stuck.
**Back-filling a short capture.** A ring only spans the window once it has
rolled over completely at its current min/max bucket, which takes as long as the
window itself; widen the window, or arm right after setting it, and the first
captures start late and the client draws a blank front half.
`backfillCaptureHead` (`trigger.go`) therefore prepends whatever of
`[t0, ring's first sample)` the archive still holds, budgeting the read by the
share of the window being filled and trimming the overlap so the frame's
timestamps stay ascending. It needs history enabled; without it the capture is
simply short, and the hub logs by how much. The hold declines any range its own
samples do not actually cover, so a stretch neither source could supply falls
through to the archive instead of being redrawn as the same hole on every zoom
and every *fit*.
The hold declines ranges reaching outside its window: those are live zooms, and
only the rings still track the stream. Inside the window it needs no
trigger-state gating, because retuning never rewrites stored samples — a ring
that still covers the range holds the very same points. It is cleared when
`updateConfig` rebuilds the rings, since a restarted producer can replay the same
timestamps.
Budget: the hold costs one window per signal on top of the ring budget, up to a
further ~0.8 × `-ring-mpts`. Nothing is held until the first capture fires.
**On disk.** The archive covers what the hold cannot: ranges wider than the
capture window, and sessions where the hub restarted. It is circular and sized
from that same window, so it too wraps over a captured shot within a window of
delivering it. When a capture is delivered, the hub therefore copies
`[trigTime − pre, trigTime + post]` out of each `.shist` into a
`<signalName>.cap` file, laid out as a full non-wrapping `.shist`
(`capacity == count`, `head == 0`) so the same `readRange` reads it. A
`historyZoom` whose range the capture file fully contains is answered from it;
anything wider is answered from the archive. The copy is replaced by the next
trigger and by nothing else — rearming keeps it, because the client is still
showing that capture.
Budget: a capture costs one window's worth of disk per signal on top of
`-history-max-mpts`.
Protecting the window in place instead — pinning the region and refusing to
wrap onto it — does not work, and was tried: a capture held for longer than the
archive covers stops the archive dead, and the resulting hole lands exactly
where the *next* capture's pre-trigger window belongs.
## 7. Build & test ## 7. Build & test
```bash ```bash
Docs/UDPS-C-Client.md
+296
View File
@@ -0,0 +1,296 @@
# UDPS C Client Library
`Common/Client/c/` is a standalone receiver for the UDPS streaming protocol: it connects to a
`UDPStreamer` DataSource (or any other UDPS producer, such as `DebugService`), decodes the
signals, and hands them to your callbacks as plain `double`s.
It has **no MARTe2 dependency** and no third-party dependencies at all — just libc and BSD
sockets. Two files, `udps_client.h` and `udps_client.c`, drop into any C or C++ project.
The wire format itself is specified in [Protocol.md](Protocol.md); this document covers the
library. The producer side is documented in [UDPStreamer.md](UDPStreamer.md).
---
## Build
```bash
cd Common/Client/c
make # libudpsclient.a + the udps_dump example
make cxxcheck # verifies the header compiles and links from C++
make clean
```
Or just add the two files to your own build:
```bash
cc -std=c99 -O2 -c udps_client.c
```
Requirements: a C99 compiler and POSIX sockets. On glibc older than 2.17 add `-lrt`
(`clock_gettime` lived in librt back then). The header is wrapped in `extern "C"`, so C++
callers include it directly.
---
## Quick start
```c
#include "udps_client.h"
#include <stdio.h>
static void on_data(const udps_frame_t *f, void *user) {
(void)user;
/* Signals are in CONFIG order; values are already physical doubles. */
printf("#%u %s = %g\n", f->counter, f->signals[0].name, f->values[0].values[0]);
}
int main(void) {
udps_client_config_t cfg;
udps_client_t *cli;
udps_client_config_init(&cfg);
cfg.server_addr = "127.0.0.1";
cfg.server_port = 44500;
cli = udps_client_create(&cfg);
udps_client_set_callbacks(cli, NULL, on_data, NULL, NULL);
for (;;) {
udps_client_poll(cli, 200); /* connects, receives, decodes, reconnects */
}
udps_client_destroy(cli);
return 0;
}
```
`udps_client_poll()` is the only function that does work. It never spawns a thread, and every
callback runs inside it — so if your program already has an event loop, call it from there and
you need no synchronisation at all. A client must be used from one thread at a time.
---
## Connection model
The library implements both transports of the protocol and picks one from the configuration:
| | Unicast (`multicast_group == NULL`) | Multicast (`multicast_group` set) |
|---|---|---|
| CONNECT | UDP datagram to `server_addr:server_port` | over a TCP connection to `server_addr:server_port` |
| CONFIG | UDP, back to the client's ephemeral port | over the same TCP connection |
| DATA | UDP, same ephemeral port | UDP multicast on `data_port` |
| Keepalive | ACK every `keepalive_interval_s` | not needed (the TCP session is the liveness signal) |
In multicast mode the group is joined *before* CONNECT is sent, because the server multicasts
CONFIG as soon as it sees a client — a group joined afterwards would miss it.
The client reconnects on its own: if nothing arrives for `silence_timeout_s` it sends
DISCONNECT, closes the sockets, waits `reconnect_delay_s`, and starts over. `udps_client_poll()`
returns `-1` when that happens, which is informational, not fatal.
---
## Configuration
Always start from `udps_client_config_init()` — it fills in the defaults below — then override
what you need. Strings are copied into the client, so they need not outlive `udps_client_create()`.
| Field | Default | Meaning |
|---|---|---|
| `server_addr` | — (required) | Server IPv4 address; a hostname is resolved if it is not a dotted quad. |
| `server_port` | — (required) | Server UDP port, or the TCP control port in multicast mode. |
| `multicast_group` | `NULL` | IPv4 group to join. Non-`NULL` selects the multicast transport. |
| `interface_addr` | `NULL` | Local IPv4 **address** (not a name, e.g. `"127.0.0.1"`) of the interface to join on. Defaults to the default route, which silently receives nothing if the server sends elsewhere. |
| `data_port` | `server_port + 1` | Multicast data port. Must match the producer's `DataPort`. |
| `silence_timeout_s` | `1.0` | Reconnect after this long without data. `0` disables the check — use it for streams that are idle by design. |
| `reconnect_delay_s` | `2.0` | Wait between reconnection attempts. |
| `keepalive_interval_s` | `15.0` | Unicast ACK period. The server evicts silent clients after its `ClientTimeout` (30 s by default). `0` disables. |
| `recv_buffer_bytes` | 4 MiB | `SO_RCVBUF`. The Linux default (~208 KiB) is overrun by fast producers and the kernel drops datagrams silently. |
| `max_packet_bytes` | 1 MiB | Ceiling on one reassembled payload; a reassembly buffer of this size is allocated per in-flight update (4 at most). |
---
## API
### Lifecycle
```c
void udps_client_config_init(udps_client_config_t *cfg);
udps_client_t *udps_client_create(const udps_client_config_t *cfg);
void udps_client_set_callbacks(udps_client_t *c, udps_config_cb, udps_data_cb,
udps_event_cb, void *user);
int udps_client_poll(udps_client_t *c, int timeout_ms);
void udps_client_destroy(udps_client_t *c);
```
`udps_client_create()` returns `NULL` on a bad address or an invalid configuration; no socket is
opened until the first poll. `udps_client_poll()` returns the number of packets processed, `0` on
timeout, or `-1` if the session broke — pass a negative `timeout_ms` to block. `destroy` sends
DISCONNECT before closing.
### Callbacks
```c
void on_config(const udps_signal_t *signals, uint32_t n, uint8_t publish_mode, void *user);
void on_data (const udps_frame_t *frame, void *user);
void on_event (udps_event_t event, const char *detail, void *user);
```
`on_config` fires on every CONFIG packet: the signal set can change at runtime, so treat it as a
reset of everything you cached. `on_event` reports `UDPS_EVENT_CONNECTED`,
`UDPS_EVENT_DISCONNECTED` and `UDPS_EVENT_ERROR` with a human-readable `detail`.
> **The frame and everything it points at are owned by the client and are valid only until
> `on_data` returns.** The decode buffers are reused by the next packet. Copy what you keep.
### Inspection
```c
int udps_client_is_connected(const udps_client_t *c);
const udps_signal_t *udps_client_signals(const udps_client_t *c, uint32_t *n);
uint8_t udps_client_publish_mode(const udps_client_t *c);
void udps_client_stats(const udps_client_t *c, udps_stats_t *out);
const char *udps_client_last_error(const udps_client_t *c);
```
### Helpers
```c
uint32_t udps_signal_num_elements(const udps_signal_t *s);
const char *udps_type_name(uint8_t type_code);
int udps_parse_header(const void *buf, size_t len, udps_header_t *out);
int udps_parse_config(const void *payload, size_t len, udps_signal_t *sigs,
uint32_t max_signals, uint32_t *n, uint8_t *publish_mode);
double udps_frame_value(const udps_frame_t *f, uint32_t sig, uint32_t sample, uint32_t elem);
double udps_frame_element_time(const udps_frame_t *f, uint32_t sig, uint32_t elem);
```
`udps_parse_header` and `udps_parse_config` are stateless and socket-free, so captured or
replayed traffic can be decoded without a client.
---
## Reading a frame
```c
typedef struct {
uint32_t counter; /* gaps in this sequence are lost datagrams */
uint32_t lost; /* DATA packets missing immediately before this one */
uint64_t hrt; /* producer's high-resolution timer at send */
double recv_time; /* CLOCK_REALTIME seconds at arrival */
uint8_t publish_mode;
uint32_t num_samples; /* batched RT cycles; 1 unless Accumulate */
uint32_t num_signals;
const udps_signal_t *signals; /* CONFIG order */
const udps_signal_values_t *values; /* same order */
} udps_frame_t;
```
`values[i].values` is an array of `values[i].count` physical `double`s. Quantised signals are
already expanded back onto `[range_min, range_max]`, and integer types are widened — the decoded
form does not depend on the wire type, so a consumer need not branch on `type_code` at all.
**Element count.** `count` is the signal's element count (`num_rows × num_cols`), *except* for a
scalar signal in Accumulate mode, where the producer batches several RT cycles into one packet
and `count == num_samples` — one value per cycle. Arrays are not batched: they appear once and
apply to the whole packet. `udps_frame_value(f, sig, sample, elem)` applies that rule for you.
**Ordering.** Frames reach `on_data` in counter order: a DATA packet that does not advance the
counter — reordered or duplicated on the wire — is dropped rather than delivered, because its
values carry a time base older than data you already have, and placing them would overwrite live
samples while leaving their own span empty. `lost` reports how many packets went missing just
before the frame. If you space samples yourself from the elapsed time since the previous frame,
divide by `lost + 1` batches, not one: the gap covers the missing packets' cycles too.
**Timestamps.** The protocol does not put a timestamp on every element; how to date them depends
on the signal's `time_mode` (see [Protocol.md](Protocol.md#time-mode-codes)):
| `time_mode` | Where the time comes from |
|---|---|
| `UDPS_TIME_PACKET` | No per-element time. Use `recv_time`. |
| `UDPS_TIME_FULL_ARRAY` | The signal at `time_signal_idx` holds one timestamp per element — read it like any other signal. |
| `UDPS_TIME_FIRST_SAMPLE` / `UDPS_TIME_LAST_SAMPLE` | The signal at `time_signal_idx` is a scalar stamping element 0 (or N−1); the rest follow at `1/sampling_rate`. |
The time signal is a raw producer-side counter (µs, or ns when it is a `uint64`), not wall clock,
so plotting it against real time needs a one-off calibration against `recv_time` — that is what
the Go hub does. `udps_frame_element_time()` skips all that and returns an arrival-anchored
estimate: good enough for a quick plot, but when a time signal exists, it is the accurate source.
---
## Diagnosing loss
```c
udps_stats_t s;
udps_client_stats(cli, &s);
```
| Counter | Meaning |
|---|---|
| `packets_received`, `bytes_received` | Accepted datagrams and TCP frames. |
| `frames_delivered` | DATA packets decoded and passed to `on_data`. |
| `config_updates` | CONFIG packets applied. |
| `counter_gaps` | Missing packet counters — datagrams lost on the wire or in the kernel. |
| `stale_packets` | DATA packets dropped for not advancing the counter: reordered or duplicated on the wire. |
| `fragments_dropped` | Duplicate, stale or unplaceable fragments; a non-zero value with `counter_gaps` means fragmented updates are arriving incomplete. |
| `reconnects` | Sessions re-established after a silence timeout. |
Persistent loss on a fast stream is almost always the receive buffer: raise `recv_buffer_bytes`
(and `net.core.rmem_max`, which caps it). A fragmented producer is more fragile than one sending
whole cycles, because losing any fragment discards the whole update — if you control the
producer, sizing `MaxPayloadSize` above one cycle removes that failure mode entirely.
---
## Example program
`example/udps_dump.c` connects, prints the signal table on CONFIG, then a throttled summary of
each frame, and a receive-statistics report on Ctrl-C.
```bash
# unicast
./udps_dump --host 127.0.0.1 --port 44500
# multicast
./udps_dump --host 127.0.0.1 --port 44500 --multicast 239.0.0.1 --iface 127.0.0.1
# quieter, and stop after 500 frames
./udps_dump --host 127.0.0.1 --port 44500 --interval 5 --frames 500
```
| Flag | Meaning |
|---|---|
| `--host ADDR` | Server address (default `127.0.0.1`). |
| `--port N` | Server UDP port, or TCP control port in multicast mode (default 44500). |
| `--multicast GROUP` | Join `GROUP` for data instead of using unicast. |
| `--iface ADDR` | Local interface address for the multicast join. |
| `--data-port N` | Multicast data port (default `--port + 1`). |
| `--silence SEC` | Reconnect after `SEC` without data; `0` disables. |
| `--interval SEC` | Seconds between printouts (default 1). |
| `--frames N` | Exit after `N` frames. |
Against the repository's own producer (`./run_udp_producer.sh -n 2`, two 1 Msps channels of
1000-element `float32` arrays at 1 kHz) the output looks like:
```
CONFIG: 3 signal(s), publish mode strict
# name type shape unit rate[Hz] time-mode
0 TimeArray uint64 1x1000 ns 0 packet
1 Ch1 float32 1x1000 V 0 full-array
2 Ch2 float32 1x1000 V 0 full-array
frame #2289897 t=1787409851.723466 samples=1
Ch1 n=1000 first=-6.9e-10 last=-0.00628 min=-1 max=1 V
Ch2 n=1000 first=0.5 last=0.49975 min=-0.5 max=0.5 V
```
---
## Limitations
- IPv4 only, matching the protocol and the producer.
- One thread per client; there is no internal locking.
- The receive path allocates only when a CONFIG grows the signal set or a frame grows the decode
arena, so a steady stream is allocation-free — but this is not a hard real-time component.
- DATA arriving before the first CONFIG is dropped: without descriptors it cannot be decoded.
This is normal for a few packets after joining a multicast group.
Docs/UDPStreamer.md
+56 -4
View File
@@ -36,7 +36,8 @@ thread.
// Multicast (optional — omit for unicast mode) // Multicast (optional — omit for unicast mode)
MulticastGroup = "239.0.0.1" // IPv4 multicast address (224.0.0.0/4) MulticastGroup = "239.0.0.1" // IPv4 multicast address (224.0.0.0/4)
Interface = "eth0" // Multicast-bound interface (mandatory when MulticastGroup is set) Interface = "192.168.1.10" // Local IPv4 address of the outgoing interface
// (mandatory when MulticastGroup is set)
DataPort = 44501 // UDP port for multicast DATA (default: Port+1) DataPort = 44501 // UDP port for multicast DATA (default: Port+1)
// Publishing mode (optional) // Publishing mode (optional)
@@ -87,7 +88,7 @@ thread.
| `Port` | uint16 | 44500 | UDP server port (unicast) or TCP control port (multicast). Values ≤ 1024 produce a warning. | | `Port` | uint16 | 44500 | UDP server port (unicast) or TCP control port (multicast). Values ≤ 1024 produce a warning. |
| `MaxPayloadSize` | uint32 | 1400 | Max payload bytes per UDP datagram (min 18) | | `MaxPayloadSize` | uint32 | 1400 | Max payload bytes per UDP datagram (min 18) |
| `MulticastGroup` | string | *(absent)* | IPv4 multicast address (e.g. `"239.0.0.1"`). Must be in 224.0.0.0/4. Absent or empty = unicast mode. | | `MulticastGroup` | string | *(absent)* | IPv4 multicast address (e.g. `"239.0.0.1"`). Must be in 224.0.0.0/4. Absent or empty = unicast mode. |
| `Interface` | string | *(absent)* | Network interface for multicast binding (e.g. `"eth0"`). **Mandatory** when `MulticastGroup` is set. | | `Interface` | string | *(absent)* | Local IPv4 address of the interface multicast DATA leaves from, in dotted-quad form (e.g. `"192.168.1.10"`, or `"127.0.0.1"` for loopback-only testing). **Not** an interface name — `"eth0"` is rejected. **Mandatory** when `MulticastGroup` is set. |
| `DataPort` | uint16 | Port+1 | UDP port for multicast DATA datagrams. Ignored in unicast mode. | | `DataPort` | uint16 | Port+1 | UDP port for multicast DATA datagrams. Ignored in unicast mode. |
| `PublishingMode` | string | Strict | `Strict`: send every RT cycle. `Accumulate`: batch until size/time limit. `Decimate`: send every Nth cycle. | | `PublishingMode` | string | Strict | `Strict`: send every RT cycle. `Accumulate`: batch until size/time limit. `Decimate`: send every Nth cycle. |
| `MinRefreshRate` | float64| — | Flush frequency in Hz. **Required** when `PublishingMode` = `Accumulate`. | | `MinRefreshRate` | float64| — | Flush frequency in Hz. **Required** when `PublishingMode` = `Accumulate`. |
@@ -147,7 +148,14 @@ CONNECT evicts the previous client.
### Multicast ### Multicast
Enabled by setting `MulticastGroup` to a valid IPv4 multicast address (224.0.0.0/4). Enabled by setting `MulticastGroup` to a valid IPv4 multicast address (224.0.0.0/4).
The `Interface` parameter is **mandatory** and specifies the network interface to bind. The `Interface` parameter is **mandatory**. It is the local IPv4 address of the
interface DATA datagrams leave from, given in dotted-quad form — it sets
`IP_MULTICAST_IF` on the data socket and is parsed with `inet_addr()`, so an
interface *name* such as `"eth0"` is rejected and `Initialise` fails.
Receivers must join the group on the matching interface. A receiver that joins
with `INADDR_ANY` lets the kernel pick the default-route interface, and it will
silently receive nothing if that is not the interface named by `Interface`.
The server opens a TCP listener on `Port` for control traffic and a UDP socket aimed at The server opens a TCP listener on `Port` for control traffic and a UDP socket aimed at
`MulticastGroup:DataPort` for data traffic. The client: `MulticastGroup:DataPort` for data traffic. The client:
@@ -253,7 +261,7 @@ PrepareNextState() ← opens UDP server socket, starts background threa
Class = UDPStreamer Class = UDPStreamer
Port = 44500 // TCP control port Port = 44500 // TCP control port
MulticastGroup = "239.0.0.1" // Enables multicast mode MulticastGroup = "239.0.0.1" // Enables multicast mode
Interface = "eth0" // Mandatory for multicast Interface = "192.168.1.10" // Local IP of the outgoing interface (mandatory)
DataPort = 44501 // UDP data port (default: Port+1) DataPort = 44501 // UDP data port (default: Port+1)
MaxPayloadSize = 1400 MaxPayloadSize = 1400
PublishingMode = "Accumulate" PublishingMode = "Accumulate"
@@ -290,6 +298,50 @@ PrepareNextState() ← opens UDP server socket, starts background threa
} }
``` ```
---
## UDPStreamerClient DataSource
`UDPStreamerClient` is a MARTe2 **input** DataSource that receives signals from a `UDPStreamer`
server. Transport, fragment reassembly, and auto-reconnect are delegated to `UDPSClient`; the
DataSource only decodes CONFIG/DATA payloads into real-time signal memory.
### Configuration
```
+ClientDS = {
Class = UDPStreamerClient
ServerAddress = "192.168.1.10" // UDPStreamer server IP
Port = 44500 // Server port
// Multicast (optional — omit for unicast)
MulticastGroup = "239.0.0.1"
DataPort = 44501 // UDP data port (default: Port+1)
Interface = "192.168.1.10" // See table below
MaxPayloadSize = 1400
Signals = {
Counter = { Type = uint32 }
}
}
```
### Parameters
| Parameter | Type | Default | Description |
| --------------- | ------- | ---------- | ----------- |
| `ServerAddress` | string | 127.0.0.1 | IPv4 address of the `UDPStreamer` server. |
| `Port` | uint16 | 44500 | Server UDP port (unicast) or TCP control port (multicast). |
| `MulticastGroup`| string | *(absent)* | IPv4 multicast address. Presence enables multicast mode. |
| `DataPort` | uint16 | Port+1 | UDP port for multicast DATA datagrams. |
| `Interface` | string | *(absent)* | Local IPv4 dotted-quad address (e.g. `"127.0.0.1"`) of the interface to join the multicast group on. **Optional**: omitting it uses the default-route interface (INADDR_ANY), which silently receives nothing if the server sends on a different interface. Not an interface name — `"eth0"` is invalid. |
| `MaxPayloadSize`| uint32 | 1400 | Max payload bytes per datagram (must match the server). |
| `SilenceTimeout`| float32 | 1.0 | Seconds of no data before auto-reconnect. 0 disables. |
| `KeepAliveInterval` | uint32 | 15 | Seconds between unicast keepalive ACKs. 0 disables. |
| `CPUMask` | uint32 | 0xFFFFFFFF | CPU affinity for the background receiver thread. |
| `StackSize` | uint32 | default | Stack size in bytes for the receiver thread. |
With `MaxPayloadSize = 1400`, a single 1000-element float32 signal produces: With `MaxPayloadSize = 1400`, a single 1000-element float32 signal produces:
``` ```
Docs/WebUI.md
+37
View File
@@ -80,6 +80,23 @@ Signals received in the CONFIG packet are listed in the sidebar:
- **Spatial arrays** — `TimeMode = PacketTime` arrays are shown as an expandable - **Spatial arrays** — `TimeMode = PacketTime` arrays are shown as an expandable
group; individual elements (`Ch1[0]`, `Ch1[1]`, …) can be dragged independently. group; individual elements (`Ch1[0]`, `Ch1[1]`, …) can be dragged independently.
The unit badge next to each signal shows the calibration's unit override when one
is set, and the streamer's own unit otherwise.
At the bottom of the sidebar, the collapsible **Sources & Config** section holds:
- the `host:port`, label, multicast group and data port inputs plus **Connect**,
which adds a source at runtime;
- **Save** — writes the source list and the whole calibration table to the hub's
config file;
- **Reload** — re-reads that file. Calibration is replaced wholesale (so unsaved
edits are discarded), sources present in the file but not running are added,
and no running source is stopped or reconnected. The hub may send an updated
`sources` list even when nothing changed (see the API doc for the per-hub
difference);
- a status line showing the written path on success or the hub's error text on
failure.
Click the sidebar toggle button (☰) to collapse/expand the signal list. Click the sidebar toggle button (☰) to collapse/expand the signal list.
### Adding Plots ### Adding Plots
@@ -143,11 +160,31 @@ plot header showing per-signal vertical scale controls:
| **V/div** | Volts (or units) per division | | **V/div** | Volts (or units) per division |
| **Pos (div)** | Screen position in divisions (draggable offset marker on Y axis) | | **Pos (div)** | Screen position in divisions (draggable offset marker on Y axis) |
| **Type** (Mixed mode only) | Toggle between **Analog** and **Digital** for this signal | | **Type** (Mixed mode only) | Toggle between **Analog** and **Digital** for this signal |
| **Cal · Scale** | Data calibration gain. `value = raw × Scale + Offset` |
| **Cal · Offset** | Data calibration bias, in calibrated units |
| **Cal · Unit** | Overrides the unit reported by the streamer (max 16 UTF-8 bytes; a multi-byte character such as `°C` counts as more than one byte) |
| **Reset** | Clears this signal's calibration (`Scale = 1`, `Offset = 0`, no unit override) |
| **✕** | Close the toolbar and deselect the signal | | **✕** | Close the toolbar and deselect the signal |
Offset markers (small triangles on the Y axis) show each signal's position and can Offset markers (small triangles on the Y axis) show each signal's position and can
be dragged to reposition signals without opening the toolbar. be dragged to reposition signals without opening the toolbar.
**Calibration vs. V/div and Offset.** They are different things. V/div and Offset
are a *display* transform: they move and stretch the trace on screen. Calibration
changes *the value itself* — the plot, the Y-axis tick labels, the cursor and
hover readouts, the CSV export and the trigger threshold all report
`raw × Scale + Offset` in the calibrated unit. V/div is then read as "calibrated
units per division" and Offset as "the calibrated value at screen centre".
The calibration header names the **base** signal and its element count, because
one entry covers every element of an array — opening the toolbar on `Adc[3]` and
editing the calibration moves all of `Adc`.
Calibration is keyed by the source's **label**, is shared with every other
browser connected to the same hub, and is not persisted until you press **Save**
in the Sources & Config section. It is mirrored to `localStorage` so it survives
a page reload even against a hub with no config file.
### Plot Controls ### Plot Controls
| Control | Action | | Control | Action |
README.md
+17 -1
View File
@@ -69,10 +69,25 @@ See `Docs/SineArrayGAM.md`.
### TimeArrayGAM ### TimeArrayGAM
Generates a time-reference float64 array. Each element holds the timestamp of the Generates a time-reference uint64 array. Each element holds the timestamp of the
corresponding sample in a packed burst, computed from the RT cycle timestamp and the corresponding sample in a packed burst, computed from the RT cycle timestamp and the
configured `SamplingRate`. configured `SamplingRate`.
`Anchor` selects how the burst is placed in time:
| `Anchor` | `out[k]` |
|---|---|
| `FirstSample` | `input + k · period` |
| `LastSample` | `input − (N−1−k) · period` |
| `Continuous` | `input(first cycle) + (n + k) · period` |
`FirstSample`/`LastSample` re-read the timer each cycle, so a lost RT cycle
(`LinuxTimer` re-phases with `counter += nCycles`) punches a whole-period hole
into the time base even though only one array of samples was produced. Use
`Continuous` when the data signal is itself contiguous (`SineArrayGAM` never
skips phase): it latches the timer once and then advances an internal sample
counter by `N` per cycle, like an acquisition card running off its own clock.
### DebugService Interface ### DebugService Interface
Instruments a running MARTe2 application **without modifying its source code**. On Instruments a running MARTe2 application **without modifying its source code**. On
@@ -229,6 +244,7 @@ Open `http://localhost:9090`, explore the object tree, trace signals, force valu
| ----------------------------- | -------------------------------------------------------------- | | ----------------------------- | -------------------------------------------------------------- |
| `Docs/Protocol.md` | UDPS binary wire protocol specification | | `Docs/Protocol.md` | UDPS binary wire protocol specification |
| `Docs/UDPStreamer.md` | UDPStreamer DataSource configuration reference | | `Docs/UDPStreamer.md` | UDPStreamer DataSource configuration reference |
| `Docs/UDPS-C-Client.md` | Standalone C/C++ UDPS receiver library (`Common/Client/c`) |
| `Docs/SineArrayGAM.md` | SineArrayGAM configuration reference | | `Docs/SineArrayGAM.md` | SineArrayGAM configuration reference |
| `Docs/DebugService.md` | DebugService TCP API and architecture | | `Docs/DebugService.md` | DebugService TCP API and architecture |
| `Docs/Tutorial.md` | Step-by-step tutorial covering both components | | `Docs/Tutorial.md` | Step-by-step tutorial covering both components |
Source/Applications/StreamHub/SignalRingBuffer.h
+96
View File
@@ -78,6 +78,32 @@ public:
/** @return Current number of stored points (≤ capacity). */ /** @return Current number of stored points (≤ capacity). */
uint32 Count() const; uint32 Count() const;
/** @return Allocated capacity in points. */
uint32 Capacity() const;
/**
* @brief Enlarge the buffer to @p newCap points, keeping the stored data
* and the TotalWritten() counter (unlike Allocate(), which resets both so
* every reader cursor and every retained sample is lost).
* @return true if the buffer now holds at least @p newCap points.
*/
bool Grow(uint32 newCap);
/**
* @brief Wall-clock span currently retained, i.e. newest minus oldest
* timestamp. 0 when fewer than two points are stored.
*/
float64 TimeSpan() const;
/**
* @brief Timestamp of the most recently stored point, 0 when empty.
*
* This is the source's own time base, which is *not* the hub's wall clock:
* use it, never clock_gettime(), whenever a decision depends on how far
* the data itself has advanced.
*/
float64 NewestTime() const;
/** @brief Discard all stored points. */ /** @brief Discard all stored points. */
void Clear(); void Clear();
@@ -138,6 +164,69 @@ inline bool SignalRingBuffer::Allocate(uint32 maxPts) {
return true; return true;
} }
inline bool SignalRingBuffer::Grow(uint32 newCap) {
if (newCap <= capacity) { return true; }
/* Allocate outside the lock; readers may be active. */
float64 *newT = new float64[newCap];
float64 *newV = new float64[newCap];
if ((newT == static_cast<float64 *>(0)) ||
(newV == static_cast<float64 *>(0))) {
delete[] newT;
delete[] newV;
return false;
}
(void) mutex.FastLock();
if (newCap > capacity) {
/* Copy oldest-to-newest so the new buffer starts unwrapped. */
const uint32 avail = count;
for (uint32 i = 0u; i < avail; i++) {
const uint32 idx = (head + capacity - avail + i) % capacity;
newT[i] = tBuf[idx];
newV[i] = vBuf[idx];
}
float64 *oldT = tBuf;
float64 *oldV = vBuf;
tBuf = newT;
vBuf = newV;
capacity = newCap;
head = avail;
/* count and totalWritten are unchanged: no sample is gained or lost,
* so push cursors stay valid across the resize. */
mutex.FastUnLock();
delete[] oldT;
delete[] oldV;
return true;
}
mutex.FastUnLock();
delete[] newT;
delete[] newV;
return true;
}
inline float64 SignalRingBuffer::TimeSpan() const {
(void) mutex.FastLock();
float64 span = 0.0;
if ((capacity > 0u) && (count > 1u)) {
const uint32 oldest = (head + capacity - count) % capacity;
const uint32 newest = (head + capacity - 1u) % capacity;
span = tBuf[newest] - tBuf[oldest];
}
mutex.FastUnLock();
return (span > 0.0) ? span : 0.0;
}
inline float64 SignalRingBuffer::NewestTime() const {
(void) mutex.FastLock();
float64 t = 0.0;
if ((capacity > 0u) && (count > 0u)) {
t = tBuf[(head + capacity - 1u) % capacity];
}
mutex.FastUnLock();
return t;
}
inline void SignalRingBuffer::Write(float64 t, float64 v) { inline void SignalRingBuffer::Write(float64 t, float64 v) {
(void) mutex.FastLock(); (void) mutex.FastLock();
if (capacity > 0u) { if (capacity > 0u) {
@@ -288,6 +377,13 @@ inline MARTe::uint64 SignalRingBuffer::TotalWritten() const {
return tw; return tw;
} }
inline uint32 SignalRingBuffer::Capacity() const {
(void) mutex.FastLock();
const uint32 c = capacity;
mutex.FastUnLock();
return c;
}
inline uint32 SignalRingBuffer::Count() const { inline uint32 SignalRingBuffer::Count() const {
(void) mutex.FastLock(); (void) mutex.FastLock();
uint32 c = count; uint32 c = count;
Source/Applications/StreamHub/StreamHub.cpp
+919 -157
View File
File diff suppressed because it is too large Load Diff
Source/Applications/StreamHub/StreamHub.h
+112 -11
View File
@@ -44,6 +44,32 @@ using MARTe::StructuredDataI;
/** Maximum number of simultaneously connected UDPStreamer sources. */ /** Maximum number of simultaneously connected UDPStreamer sources. */
static const uint32 kMaxSessions = 32u; static const uint32 kMaxSessions = 32u;
/** Maximum number of stored per-signal calibration entries. */
static const uint32 kMaxCalibration = 256u;
/** Maximum length of a calibration unit override (mirrors the Go maxUnitLen). */
static const uint32 kMaxUnitLen = 16u;
/**
* @brief One per-signal affine calibration: y = raw*scale + offset.
*
* Keyed by the source LABEL (not the runtime "sN" id, which is assigned in
* add-order and would rebind if the source list were reordered) and by the
* BASE signal name (no "[i]" suffix: one entry covers a whole array signal).
*
* Fixed-size char arrays are used deliberately: they avoid per-entry heap
* churn (no StreamString allocation per calibration slot), keep the type free
* of STL, and make a CalibrationEntry snapshot trivially copyable under the
* calibration mutex lock.
*/
struct CalibrationEntry {
char source[128]; ///< Source label
char signal[128]; ///< Base signal name (no "[i]" suffix)
char unit[17]; ///< Unit override (max kMaxUnitLen bytes + NUL)
MARTe::float64 scale;
MARTe::float64 offset;
};
/** /**
* @brief Top-level StreamHub orchestrator. * @brief Top-level StreamHub orchestrator.
* *
@@ -65,7 +91,7 @@ public:
* WSPort (uint32, default 8090) * WSPort (uint32, default 8090)
* MaxPoints (uint32, default 20000) — ring buffer capacity per signal * MaxPoints (uint32, default 20000) — ring buffer capacity per signal
* PushRate (uint32, default 30) — push loop rate in Hz * PushRate (uint32, default 30) — push loop rate in Hz
* MaxPushPoints (uint32, default 500) — LTTB threshold for live push * MaxPushPoints (uint32, default 50) — LTTB threshold for live push
* StatsRate (uint32, default 1) — stats broadcast rate in Hz * StatsRate (uint32, default 1) — stats broadcast rate in Hz
* +Sources { +<id> { Label=...; Addr=...; Port=... } } * +Sources { +<id> { Label=...; Addr=...; Port=... } }
* *
@@ -108,6 +134,12 @@ private:
/** Broadcast {"type":"config","sourceId":...} for one session. */ /** Broadcast {"type":"config","sourceId":...} for one session. */
void BroadcastConfig(uint32 sessionIdx); void BroadcastConfig(uint32 sessionIdx);
/** Broadcast {"type":"calibration","cal":[...]} to all clients. */
void BroadcastCalibration();
/** Broadcast {"type":"configSaved"|"configReloaded","ok":...} to all clients. */
void BroadcastConfigAck(const char *msgType, bool ok, const char *errText);
/* ---- Trigger (push loop side) ----------------------------------------- */ /* ---- Trigger (push loop side) ----------------------------------------- */
/** /**
@@ -120,12 +152,40 @@ private:
void BroadcastTriggerState(); void BroadcastTriggerState();
/** /**
* @brief Build and broadcast the version=2 binary capture frame: * @brief Size every ring so it retains the current trigger window.
* [u8 2][f64 trigTime][f64 preSec][f64 postSec][u32 nSig] * Called from the push loop once per stats tick; a no-op once the rings
* {[u16 keyLen][fullKey][u32 N][t f64×N][v f64×N]} * are large enough. Rates are measured from the rings themselves because
* most sources advertise samplingRate = 0.
*/ */
void BroadcastTriggerCapture(float64 trigTime, float64 preSec, void GrowRingsForTrigger();
float64 postSec);
/** @return Largest ring capacity currently allocated across all sessions. */
uint32 CurrentMaxRingCapacity() const;
/**
* @brief How far source @p i has produced, in the trigger's time base;
* @p wallNowS when it publishes no producer clock (its samples are then
* stamped on arrival, so they share the hub's wall clock).
*/
float64 SourceFrontierTime(uint32 i, float64 wallNowS) const;
/* ---- Trigger capture assembly ---------------------------------------
* Sources are harvested one at a time, each as soon as *it* has produced
* past the end of the window, rather than all together once the slowest
* has. Sources free-run on their own clocks and can lag each other by
* seconds; making every source wait for the slowest lets the leaders' ring
* buffers roll past the pre-trigger region before it is ever read. */
/** @brief Start a version=2 capture frame:
* [u8 2][f64 trigTime][f64 preSec][f64 postSec][u32 nSig]. */
void BeginTriggerCapture(float64 trigTime, float64 preSec, float64 postSec);
/** @brief Append session @p i's signals to the pending frame, each as
* {[u16 keyLen][fullKey][u32 N][t f64×N][v f64×N]}. */
void HarvestTriggerCapture(uint32 i, float64 t0, float64 t1);
/** @brief Patch nSig, broadcast the pending frame and release it. */
void FinishTriggerCapture();
/* ---- Command handlers (called from OnWSCommand) ---------------------- */ /* ---- Command handlers (called from OnWSCommand) ---------------------- */
@@ -140,11 +200,14 @@ private:
void HandleRearm(); void HandleRearm();
void HandleTrigStop(const char *json); void HandleTrigStop(const char *json);
void HandleSetTrigger(const char *json); void HandleSetTrigger(const char *json);
void HandleForceTrigger();
void HandleZoom(const char *json, uint32 slotIdx); void HandleZoom(const char *json, uint32 slotIdx);
void HandleHistoryZoom(const char *json, uint32 slotIdx); void HandleHistoryZoom(const char *json, uint32 slotIdx);
void HandleHistoryInfo(uint32 slotIdx); void HandleHistoryInfo(uint32 slotIdx);
void HandleSetMaxPoints(const char *json); void HandleSetMaxPoints(const char *json);
void HandlePing(uint32 slotIdx); void HandlePing(uint32 slotIdx);
void HandleSetCalibration(const char *json);
void HandleReloadConfig();
/* ---- Binary recorder commands --------------------------------------- */ /* ---- Binary recorder commands --------------------------------------- */
@@ -171,10 +234,32 @@ private:
const char *mcGroup, uint16 dataPort); const char *mcGroup, uint16 dataPort);
/** /**
* @brief Load sources from sourcesFile_ (JSON array of * @brief Load sources and calibration from sourcesFile_ (a flat JSON array
* {"label","addr","multicastGroup","dataPort"}) and start them. * of {"label","addr","multicastGroup","dataPort"} source blocks and
* {"source","signal","scale","offset","unit"} calibration blocks).
* @param skipActive when true, a source whose "host:port" is already
* streaming is left alone instead of being started a second time.
* @param clearCalibration when true, the calibration table is cleared
* after a successful fread (never before), so a transient I/O failure
* does not silently wipe user calibration data.
* @return true if the file was read.
*/ */
void LoadSourcesFile(); bool LoadSourcesFile(bool skipActive, bool clearCalibration = false);
/** @return true if a session for this "host:port" is already active. */
bool SourceIsActive(const char *addrPort);
/**
* @brief Store or replace one calibration entry. An identity entry
* (scale 1, offset 0, empty unit) removes any stored one instead.
* @return true if the entry was valid (and therefore stored or removed).
*/
bool SetCalibrationEntry(const char *source, const char *signal,
MARTe::float64 scale, MARTe::float64 offset,
const char *unit);
/** Drop every calibration entry (used by reload, which replaces wholesale). */
void ClearCalibration();
/* ---- Tiny JSON helpers ----------------------------------------------- */ /* ---- Tiny JSON helpers ----------------------------------------------- */
@@ -214,11 +299,17 @@ private:
uint32 pushRateHz_; uint32 pushRateHz_;
uint32 maxPushPoints_; uint32 maxPushPoints_;
uint32 statsRateHz_; uint32 statsRateHz_;
uint32 ringTemporal_; ///< Ring capacity for multi-element (waveform) signals uint32 ringTemporal_; ///< Initial ring capacity for multi-element (waveform) signals
uint32 ringScalar_; ///< Ring capacity for scalar signals uint32 ringScalar_; ///< Ring capacity for scalar signals
uint32 ringMaxPts_; ///< Ceiling a ring may be grown to for a trigger window
volatile float64 trigRetentionSec_; ///< Retention the current trigger window needs
StreamString sourcesFile_; ///< Persistent dynamic source list (JSON) StreamString sourcesFile_; ///< Persistent dynamic source list (JSON)
uint32 nextSourceId_; ///< Counter for generated session ids ("sN") uint32 nextSourceId_; ///< Counter for generated session ids ("sN")
CalibrationEntry *calibration_; ///< Heap-allocated array[kMaxCalibration]
uint32 numCalibration_;
FastPollingMutexSem calibrationMutex_; ///< Serializes calibration reads/writes
/* Push loop state */ /* Push loop state */
volatile bool running_; volatile bool running_;
uint32 tickCount_; ///< incremented each push tick uint32 tickCount_; ///< incremented each push tick
@@ -231,7 +322,9 @@ private:
static const uint32 kPushBufSize = 8u * 1024u * 1024u; static const uint32 kPushBufSize = 8u * 1024u * 1024u;
uint8 *pushBuf_; uint8 *pushBuf_;
/* Decimated output scratch (LTTB): maxPushPoints × 2 arrays per signal */ /* Decimated output scratch (LTTB). Sized like the read scratch rather
* than maxPushPoints_: a PACKET-timed array raises its own threshold to
* one packet's worth of elements, which can exceed maxPushPoints_. */
float64 *lttbT_; float64 *lttbT_;
float64 *lttbV_; float64 *lttbV_;
@@ -250,6 +343,14 @@ private:
TrigState lastTrigState_; ///< Last broadcast FSM state TrigState lastTrigState_; ///< Last broadcast FSM state
bool rearmPending_; ///< Normal-mode auto-rearm scheduled bool rearmPending_; ///< Normal-mode auto-rearm scheduled
float64 rearmAtWallS_; ///< Wall time of the scheduled auto-rearm float64 rearmAtWallS_; ///< Wall time of the scheduled auto-rearm
float64 collectStartWallS_; ///< Wall time COLLECTING began (watchdog only)
/* Capture frame under assembly across ticks (push thread only) */
MARTe::uint8 *capBuf_; ///< Pending frame, NULL when idle
uint32 capCap_; ///< Allocated size of capBuf_
uint32 capOff_; ///< Bytes written so far
uint32 capNSig_; ///< Signals appended so far
bool capHarvested_[kMaxSessions]; ///< Session already appended
}; };
} /* namespace StreamHub */ } /* namespace StreamHub */
Source/Applications/StreamHub/TriggerEngine.cpp
+106 -21
View File
@@ -14,10 +14,22 @@ TriggerEngine::TriggerEngine()
stopped_(false), stopped_(false),
prevValue_(0.0), prevValue_(0.0),
prevValid_(false), prevValid_(false),
lastTime_(0.0),
lastTimeValid_(false),
trigTime_(0.0), trigTime_(0.0),
firedPreSec_(0.0), firedPreSec_(0.0),
firedPostSec_(0.0), firedPostSec_(0.0),
firedValid_(false) { firedValid_(false),
pendingTime_(0.0),
pendingValid_(false) {
}
void TriggerEngine::LatchWindowLocked(float64 t) {
state_ = kTrigCollecting;
trigTime_ = t;
firedPreSec_ = config_.windowSec * config_.prePercent / 100.0;
firedPostSec_ = config_.windowSec - firedPreSec_;
firedValid_ = true;
} }
void TriggerEngine::SetConfig(const TriggerConfig &cfg) { void TriggerEngine::SetConfig(const TriggerConfig &cfg) {
@@ -25,12 +37,22 @@ void TriggerEngine::SetConfig(const TriggerConfig &cfg) {
config_ = cfg; config_ = cfg;
/* Clamp to web UI bounds */ /* Clamp to web UI bounds */
if (config_.windowSec < 1.0e-4) { config_.windowSec = 1.0e-4; } if (config_.windowSec < 1.0e-4) { config_.windowSec = 1.0e-4; }
if (config_.windowSec > 10.0) { config_.windowSec = 10.0; } /* 60 s where the Go hub allows 600. Deliberate: these rings are
* fixed-capacity and store every sample, so a window they cannot hold is
* harvested truncated and silently decimated to kTrigCapturePts. The Go
* hub stores min/max pairs instead once a window outgrows its budget, so
* there a long window costs resolution rather than coverage. */
if (config_.windowSec > 60.0) { config_.windowSec = 60.0; }
if (config_.prePercent < 0.0) { config_.prePercent = 0.0; } if (config_.prePercent < 0.0) { config_.prePercent = 0.0; }
if (config_.prePercent > 100.0) { config_.prePercent = 100.0; } if (config_.prePercent > 100.0) { config_.prePercent = 100.0; }
if (config_.holdoffSec < 0.0) { config_.holdoffSec = 0.0; }
if (config_.holdoffSec > 60.0) { config_.holdoffSec = 60.0; }
epoch_++; epoch_++;
prevValid_ = false; prevValid_ = false;
prevValue_ = 0.0; prevValue_ = 0.0;
/* An edge held over from the old configuration would be latched against the
* new window, which it was never judged against. */
pendingValid_ = false;
mutex_.FastUnLock(); mutex_.FastUnLock();
} }
@@ -50,19 +72,40 @@ uint32 TriggerEngine::GetConfigEpoch() const {
void TriggerEngine::Arm() { void TriggerEngine::Arm() {
(void) mutex_.FastLock(); (void) mutex_.FastLock();
state_ = kTrigArmed; state_ = kTrigArmed;
prevValid_ = false; prevValid_ = false;
prevValue_ = 0.0; prevValue_ = 0.0;
pendingValid_ = false;
mutex_.FastUnLock();
}
void TriggerEngine::Rearm() {
(void) mutex_.FastLock();
if (pendingValid_) {
const float64 t = pendingTime_;
pendingValid_ = false;
LatchWindowLocked(t);
REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information,
"TriggerEngine: rearmed onto the edge held at t=%.6f "
"(pre=%.4fs post=%.4fs)",
t, firedPreSec_, firedPostSec_);
}
else {
/* prevValue_/prevValid_ are deliberately kept: the comparator ran right
* through the dead time, so the next sample has a real predecessor. */
state_ = kTrigArmed;
}
mutex_.FastUnLock(); mutex_.FastUnLock();
} }
void TriggerEngine::Disarm() { void TriggerEngine::Disarm() {
(void) mutex_.FastLock(); (void) mutex_.FastLock();
state_ = kTrigIdle; state_ = kTrigIdle;
stopped_ = false; stopped_ = false;
prevValid_ = false; prevValid_ = false;
prevValue_ = 0.0; prevValue_ = 0.0;
firedValid_ = false; firedValid_ = false;
pendingValid_ = false;
mutex_.FastUnLock(); mutex_.FastUnLock();
} }
@@ -82,7 +125,15 @@ bool TriggerEngine::GetStopped() const {
void TriggerEngine::CheckSample(float64 t, float64 v) { void TriggerEngine::CheckSample(float64 t, float64 v) {
(void) mutex_.FastLock(); (void) mutex_.FastLock();
if (state_ != kTrigArmed) { /* Track the newest watched timestamp in every state so Force() has a
* reference time to latch the capture window around. */
lastTime_ = t;
lastTimeValid_ = true;
/* A capture in flight does not stop the comparator; it only changes what an
* edge does. See pendingTime_. */
const bool inFlight = (state_ == kTrigCollecting) || (state_ == kTrigTriggered);
if ((state_ != kTrigArmed) && !inFlight) {
mutex_.FastUnLock(); mutex_.FastUnLock();
return; return;
} }
@@ -107,21 +158,55 @@ void TriggerEngine::CheckSample(float64 t, float64 v) {
} }
if (fired) { if (fired) {
state_ = kTrigCollecting; if (!inFlight) {
trigTime_ = t; /* Latch the window at fire time so later config edits do not
/* Latch the window at fire time so later config edits do not * affect this capture (web client snap._preS/_postS). */
* affect this capture (web client snap._preS/_postS). */ LatchWindowLocked(t);
firedPreSec_ = config_.windowSec * config_.prePercent / 100.0; REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information,
firedPostSec_ = config_.windowSec - firedPreSec_; "TriggerEngine: fired at t=%.6f (pre=%.4fs post=%.4fs)",
firedValid_ = true; t, firedPreSec_, firedPostSec_);
REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information, }
"TriggerEngine: fired at t=%.6f (pre=%.4fs post=%.4fs)", else if (!pendingValid_ && firedValid_) {
t, firedPreSec_, firedPostSec_); /* The earliest trigger point a new capture may take. The one in
* flight owns everything up to the end of its own post-window, and
* the holdoff — a guard against re-triggering on the ringing of the
* SAME event — is measured from its trigger point too, so the two
* overlap rather than add.
*
* Keep only the FIRST qualifying edge: a later one would deliver
* the same capture a pulse further on and skip the one between. */
float64 guard = firedPostSec_;
if (config_.holdoffSec > guard) {
guard = config_.holdoffSec;
}
if (t >= (trigTime_ + guard)) {
pendingTime_ = t;
pendingValid_ = true;
}
}
else {
/* Already holding an edge, or no window latched to measure against. */
}
} }
mutex_.FastUnLock(); mutex_.FastUnLock();
} }
bool TriggerEngine::Force() {
(void) mutex_.FastLock();
bool ok = lastTimeValid_ && (state_ != kTrigCollecting);
if (ok) {
LatchWindowLocked(lastTime_);
REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information,
"TriggerEngine: forced at t=%.6f (pre=%.4fs post=%.4fs)",
trigTime_, firedPreSec_, firedPostSec_);
}
mutex_.FastUnLock();
return ok;
}
TrigState TriggerEngine::GetState() const { TrigState TriggerEngine::GetState() const {
(void) mutex_.FastLock(); (void) mutex_.FastLock();
TrigState ret = state_; TrigState ret = state_;
Source/Applications/StreamHub/TriggerEngine.h
+50 -5
View File
@@ -62,9 +62,10 @@ struct TriggerConfig {
StreamString signalKey; ///< Full key: "src:sig" or "src:sig[i]" StreamString signalKey; ///< Full key: "src:sig" or "src:sig[i]"
TrigEdge edge; ///< Rising / falling / both TrigEdge edge; ///< Rising / falling / both
float64 threshold; ///< Trigger threshold (physical units) float64 threshold; ///< Trigger threshold (physical units)
float64 windowSec; ///< Capture window length [1e-4 .. 10] s float64 windowSec; ///< Capture window length [1e-4 .. 60] s
float64 prePercent; ///< Pre-trigger part of the window [0 .. 100] % float64 prePercent; ///< Pre-trigger part of the window [0 .. 100] %
TrigAcqMode mode; ///< Normal (auto-rearm) or single TrigAcqMode mode; ///< Normal (auto-rearm) or single
float64 holdoffSec; ///< Rearm delay after a capture [0 .. 60] s
}; };
/** /**
@@ -87,9 +88,24 @@ public:
*/ */
uint32 GetConfigEpoch() const; uint32 GetConfigEpoch() const;
/** @brief Arm: any state → ARMED (resets edge detection). */ /**
* @brief Arm: any state → ARMED (resets edge detection).
* This is the user's own arm, so it discards any edge remembered during the
* previous capture: the user asked for the next event, not for one that has
* already been and gone.
*/
void Arm(); void Arm();
/**
* @brief The automatic arm at the end of a capture (normal mode).
* Unlike Arm() it honours an edge seen while the capture was being
* collected, firing on it at once rather than waiting for the next one, and
* it keeps the tracked level so the first sample afterwards is compared
* against its real predecessor. TRIGGERED → COLLECTING when an edge was
* remembered, otherwise → ARMED.
*/
void Rearm();
/** @brief Disarm: any state → IDLE; clears the stopped flag. */ /** @brief Disarm: any state → IDLE; clears the stopped flag. */
void Disarm(); void Disarm();
@@ -101,11 +117,22 @@ public:
/** /**
* @brief Edge-detect one decoded sample of the configured signal. * @brief Edge-detect one decoded sample of the configured signal.
* Receive-thread context. Only acts in ARMED state; on a matching edge * Receive-thread context. In ARMED state a matching edge latches trigTime
* latches trigTime and the pre/post window and moves to COLLECTING. * and the pre/post window and moves to COLLECTING. While a capture is in
* flight (COLLECTING/TRIGGERED) the comparator keeps running and the first
* edge clear of that capture is remembered for the next Rearm().
*/ */
void CheckSample(float64 t, float64 v); void CheckSample(float64 t, float64 v);
/**
* @brief Fire the trigger unconditionally at the most recent sample time of
* the watched signal, latching the pre/post window exactly as CheckSample
* does. Any state except COLLECTING → COLLECTING.
* @return false when no sample has been seen yet, or a capture is already
* being collected.
*/
bool Force();
/** @return Current FSM state. */ /** @return Current FSM state. */
TrigState GetState() const; TrigState GetState() const;
@@ -127,10 +154,27 @@ private:
bool stopped_; bool stopped_;
float64 prevValue_; ///< Last sample (edge detection) float64 prevValue_; ///< Last sample (edge detection)
bool prevValid_; ///< First-sample guard in ARMED state bool prevValid_; ///< First-sample guard in ARMED state
float64 lastTime_; ///< Timestamp of the newest watched sample
bool lastTimeValid_;///< true once a watched sample has been seen
float64 trigTime_; ///< Latched trigger time (Unix s) float64 trigTime_; ///< Latched trigger time (Unix s)
float64 firedPreSec_; ///< Window pre-part latched at fire time float64 firedPreSec_; ///< Window pre-part latched at fire time
float64 firedPostSec_; ///< Window post-part latched at fire time float64 firedPostSec_; ///< Window post-part latched at fire time
bool firedValid_; ///< true after a fire, until Disarm() bool firedValid_; ///< true after a fire, until Disarm()
/**
* The edge to fire on as soon as the FSM rearms, in sample time, recorded
* while a capture is still being collected or handed out. Without it the
* trigger is deaf from its own trigger point until the capture has been
* harvested and the holdoff has run, and then waits for a fresh edge, which
* on a sparse pulse train rounds the capture spacing up to a whole pulse
* period. Remembering the edge instead makes the blind stretch exactly the
* guard interval it has to be, since the capture is built from the edge's
* own timestamp and the rings still hold everything around it.
*/
float64 pendingTime_;
bool pendingValid_;
/** @brief Freeze the pre/post split at fire time; caller holds the mutex. */
void LatchWindowLocked(float64 t);
}; };
inline TriggerConfig::TriggerConfig() inline TriggerConfig::TriggerConfig()
@@ -138,7 +182,8 @@ inline TriggerConfig::TriggerConfig()
threshold(0.0), threshold(0.0),
windowSec(1.0), windowSec(1.0),
prePercent(20.0), prePercent(20.0),
mode(kTrigNormal) { mode(kTrigNormal),
holdoffSec(0.2) {
} }
} /* namespace StreamHub */ } /* namespace StreamHub */
Source/Applications/StreamHub/UDPSourceSession.cpp
+106 -12
View File
@@ -99,6 +99,8 @@ void UDPSourceSession::ResetCalibration() {
lastPktWallValid_[i] = false; lastPktWallValid_[i] = false;
lastPktWallS_[i] = 0.0; lastPktWallS_[i] = 0.0;
accScalarPrevN_[i] = 0u; accScalarPrevN_[i] = 0u;
accScalarDtValid_[i] = false;
accScalarDtEMA_[i] = 0.0;
} }
} }
@@ -197,7 +199,8 @@ void UDPSourceSession::OnUDPSConfig(const uint8 *payload, uint32 payloadSize) {
} }
void UDPSourceSession::OnUDPSData(const uint8 *payload, uint32 payloadSize) { void UDPSourceSession::OnUDPSData(const uint8 *payload, uint32 payloadSize) {
ParseDataPayload(payload, payloadSize); /* Valid only for the duration of this callback. */
ParseDataPayload(payload, payloadSize, client_.GetLastDataGap());
} }
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
@@ -225,6 +228,9 @@ void UDPSourceSession::ParseConfigPayload(const uint8 *payload, uint32 size) {
sigDescs_[i].unit[sizeof(sigDescs_[i].unit) - 1u] = '\0'; sigDescs_[i].unit[sizeof(sigDescs_[i].unit) - 1u] = '\0';
} }
publishMode_ = payload[4u + numSigs * UDPS_SIGNAL_DESC_SIZE]; publishMode_ = payload[4u + numSigs * UDPS_SIGNAL_DESC_SIZE];
hrtFreq_ = UDPSConfigHrtFrequency(
payload, size, numSigs,
static_cast<float64>(MARTe::HighResolutionTimer::Frequency()));
numSignals_ = numSigs; numSignals_ = numSigs;
configured_ = true; configured_ = true;
@@ -273,8 +279,9 @@ void UDPSourceSession::ParseConfigPayload(const uint8 *payload, uint32 size) {
} }
REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information, REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information,
"UDPSourceSession[%s]: CONFIG received — %u signals.", "UDPSourceSession[%s]: CONFIG received — %u signals, "
id_.Buffer(), numSigs); "producer HRT %.0f Hz.",
id_.Buffer(), numSigs, hrtFreq_);
} }
void UDPSourceSession::AllocateRingBuffers() { void UDPSourceSession::AllocateRingBuffers() {
@@ -295,11 +302,89 @@ void UDPSourceSession::AllocateRingBuffers() {
} }
} }
bool UDPSourceSession::GrowRingsForSeconds(float64 seconds, uint32 maxPts) {
if ((seconds <= 0.0) || (maxPts == 0u)) { return false; }
(void) metaMutex_.FastLock();
const uint32 nSigs = numSignals_;
metaMutex_.FastUnLock();
bool grew = false;
for (uint32 i = 0u; i < nSigs; i++) {
const uint32 count = rings_[i].Count();
const float64 span = rings_[i].TimeSpan();
/* Need a decent sample of the stream before extrapolating a rate;
* a couple of packets' worth of span is enough at any rate. */
if ((count < 2u) || (span <= 0.0)) { continue; }
const float64 rate = static_cast<float64>(count) / span;
/* 20 % headroom absorbs rate jitter and the capture margin. */
float64 need = rate * seconds * 1.2;
if (need > static_cast<float64>(maxPts)) {
need = static_cast<float64>(maxPts);
}
const uint32 needPts = static_cast<uint32>(need);
if (needPts > rings_[i].Capacity()) {
if (rings_[i].Grow(needPts)) { grew = true; }
}
}
return grew;
}
uint32 UDPSourceSession::GetMaxRingCapacity() const {
(void) metaMutex_.FastLock();
const uint32 nSigs = numSignals_;
metaMutex_.FastUnLock();
uint32 maxCap = 0u;
for (uint32 i = 0u; i < nSigs; i++) {
const uint32 c = rings_[i].Capacity();
if (c > maxCap) { maxCap = c; }
}
return maxCap;
}
float64 UDPSourceSession::ProducerNewestTime() const {
/* Mirror exactly the ParseDataPayload branches that timestamp from the
* referenced time signal; every other branch stamps on arrival and so
* would report "now" in the hub's clock, not the producer's. The time
* signal itself is one of those — it is PACKET-timed. */
(void) metaMutex_.FastLock();
const uint32 nSigs = numSignals_;
bool producerTimed[UDPSS_MAX_SIGNALS];
for (uint32 i = 0u; i < nSigs; i++) {
const UDPSSignalDescriptor &d = sigDescs_[i];
uint64 ne = static_cast<uint64>(d.numRows) *
static_cast<uint64>(d.numCols);
if (ne == 0u) { ne = 1u; }
const bool hasTimeSig = (d.timeSignalIdx != UDPS_NO_TIME_SIGNAL) &&
(d.timeSignalIdx < nSigs);
const bool isFirstLast = (ne > 1u) &&
((d.timeMode == UDPS_TIMEMODE_FIRST_SAMPLE) ||
(d.timeMode == UDPS_TIMEMODE_LAST_SAMPLE));
const bool isFullArray = (d.timeMode == UDPS_TIMEMODE_FULL_ARRAY);
producerTimed[i] = hasTimeSig && (isFullArray || isFirstLast);
}
metaMutex_.FastUnLock();
/* Signals of one source share a packet, so they advance together; the max
* is "how far this source has produced" without stalling on a signal that
* simply is not being sent. */
float64 newest = 0.0;
for (uint32 i = 0u; i < nSigs; i++) {
if (!producerTimed[i]) { continue; }
const float64 t = rings_[i].NewestTime();
if (t > newest) { newest = t; }
}
return newest;
}
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
/* DATA parsing */ /* DATA parsing */
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size) { void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size,
uint32 lostPackets) {
if (size < 8u) { return; } if (size < 8u) { return; }
/* Copy metadata under lock */ /* Copy metadata under lock */
@@ -491,9 +576,9 @@ void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size) {
* immune to this because it is sampled at acquisition. * immune to this because it is sampled at acquisition.
* *
* hrtTimestamp is the HRT counter of sample 0; hrtFreq_ (the * hrtTimestamp is the HRT counter of sample 0; hrtFreq_ (the
* local HRT frequency, identical to the sender on the same * producer's tick rate, taken from the CONFIG trailer) converts
* host) converts it to seconds, then a one-time calibration * it to seconds, then a one-time calibration maps the sender
* maps the sender clock onto wall-clock. */ * clock onto wall-clock. */
const float64 hrt0Sec = static_cast<float64>(hrtTimestamp) / const float64 hrt0Sec = static_cast<float64>(hrtTimestamp) /
hrtFreq_; hrtFreq_;
if ((!timeSigCalibValid_[s]) || if ((!timeSigCalibValid_[s]) ||
@@ -504,16 +589,25 @@ void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size) {
} }
/* Per-sample dt: samplingRate if present, else derive it from /* Per-sample dt: samplingRate if present, else derive it from
* the sender-HRT gap to the previous packet divided by that * the sender-HRT gap to the previous packet.
* packet's sample count (the flushes carry contiguous RT *
* cycles, so this is exactly one cycle period). */ * The gap is divided by the number of RT cycles it actually
* spans, not by the previous packet's sample count. Those two
* agree only while nothing is lost; once a packet goes missing
* the gap covers cycles the previous count never saw, and
* dividing by that count inflates dt until this packet's
* samples overrun into the next packet's range. lostPackets
* comes from the producer's packet counter, so the divisor
* widens with the gap and dt is unchanged. */
float64 dt; float64 dt;
if (desc.samplingRate > 0.0) { if (desc.samplingRate > 0.0) {
dt = 1.0 / desc.samplingRate; dt = 1.0 / desc.samplingRate;
} else if (lastPktWallValid_[s] && (accScalarPrevN_[s] > 0u) && } else if (lastPktWallValid_[s] && (accScalarPrevN_[s] > 0u) &&
(hrt0Sec > lastPktWallS_[s])) { (hrt0Sec > lastPktWallS_[s])) {
dt = (hrt0Sec - lastPktWallS_[s]) / dt = UDPSEstimateAccumDt(hrt0Sec - lastPktWallS_[s],
static_cast<float64>(accScalarPrevN_[s]); accScalarPrevN_[s], lostPackets,
accScalarDtEMA_[s],
accScalarDtValid_[s]);
} else { } else {
dt = 1.0e-3; /* 1 kHz default until the gap is known */ dt = 1.0e-3; /* 1 kHz default until the gap is known */
} }
Source/Applications/StreamHub/UDPSourceSession.h
+173 -13
View File
@@ -39,6 +39,104 @@ using MARTe::ConfigurationDatabase;
/** Maximum number of signals per source session. */ /** Maximum number of signals per source session. */
static const uint32 UDPSS_MAX_SIGNALS = 256u; static const uint32 UDPSS_MAX_SIGNALS = 256u;
/* Accumulated-scalar dt estimator tuning. */
/** Weight of a new observation; slow enough that one bad gap barely moves it. */
static const float64 UDPSS_DT_EMA_ALPHA = 0.05;
/** Observations outside [lo, hi] x the current estimate are treated as a
* mis-counted gap and discarded rather than smoothed in. */
static const float64 UDPSS_DT_ACCEPT_LO = 0.5;
static const float64 UDPSS_DT_ACCEPT_HI = 2.0;
/**
* @brief Per-sample period of an accumulated scalar packet, robust to loss.
*
* An Accumulate producer batches consecutive RT cycles, so the sender-clock
* gap between two packets' first samples covers exactly as many cycles as the
* earlier packet carried — but only while nothing is lost in between. Over UDP
* (and with a producer that can overwrite a batch the sender never took) that
* assumption fails, and dividing the gap by the previous packet's sample count
* then inflates the period. The packet's own samples are laid out as
* base + e*dt, so an inflated dt walks them past their real end and into the
* span the next packet will claim: samples collide there and leave a hole
* behind them.
*
* The number of packets that went missing is not guessed from the gap — that
* is circular, and an estimator that infers the cycle count from its own
* period has a stable fixed point wherever gap/dt is an integer, so a genuine
* rate change locks it at the old period forever. It comes instead from the
* UDPS packet counter, which the producer increments once per sent packet. The
* gap then spans (1 + lost) batches, each assumed to be prevN cycles, and with
* nothing lost the formula reduces exactly to gap/prevN.
*
* @param gap Sender-clock seconds since the previous packet's first sample.
* Must be > 0.
* @param prevN Samples in the previous packet. Must be > 0.
* @param lost Packets missing between the previous packet and this one,
* from the producer's counter.
* @param[in,out] dtEMA Smoothed period. Seeded on the first call.
* @param[in,out] dtValid False until dtEMA holds an estimate.
* @return The period to space this packet's samples by.
*/
inline float64 UDPSEstimateAccumDt(const float64 gap, const uint32 prevN,
const uint32 lost, float64 &dtEMA,
bool &dtValid) {
float64 cycles = static_cast<float64>(prevN) *
(1.0 + static_cast<float64>(lost));
if (cycles < 1.0) {
cycles = 1.0;
}
const float64 dtObs = gap / cycles;
if (!dtValid) {
dtEMA = dtObs;
dtValid = true;
} else if ((dtObs > (dtEMA * UDPSS_DT_ACCEPT_LO)) &&
(dtObs < (dtEMA * UDPSS_DT_ACCEPT_HI))) {
/* Track slow drift, but ignore observations far outside the current
* estimate: those are the signature of a mis-counted gap, and folding
* one in would drag the estimate towards the very error it exists to
* absorb. */
dtEMA = ((1.0 - UDPSS_DT_EMA_ALPHA) * dtEMA) +
(UDPSS_DT_EMA_ALPHA * dtObs);
}
return dtEMA;
}
/**
* @brief Pick the tick rate to divide a producer's DATA timestamps by.
*
* DATA packets carry the raw value of the producer's high-resolution counter,
* which is meaningless without the rate it runs at. The rate is published in
* the CONFIG trailer, after the descriptors and the publish-mode byte. When it
* is missing — an older producer, or one that could not determine it — the
* only remaining option is this host's own timer, which is right only while
* the two machines agree; on x86 that is the TSC frequency, so it is a
* different number on every model.
*
* @param payload Reassembled CONFIG payload.
* @param size Bytes in @p payload.
* @param numSigs Signal count already read from the payload, capped to what
* the receiver will store.
* @param localFreq This host's HRT frequency, used as the fallback.
* @return Ticks per second to convert DATA timestamps with; never 0.
*/
inline float64 UDPSConfigHrtFrequency(const uint8 *payload, const uint32 size,
const uint32 numSigs,
const float64 localFreq) {
const uint32 offset = 4u + (numSigs * MARTe::UDPS_SIGNAL_DESC_SIZE) + 1u;
if ((payload != NULL_PTR(const uint8 *)) && (size >= (offset + 8u))) {
uint64 wireFreq = 0u;
memcpy(&wireFreq, payload + offset, 8u);
/* Anything below 1 kHz is not a high-resolution timer; the field is
* either absent, unset, or the payload was mis-parsed, and adopting it
* would stretch every timestamp far enough to make the trace useless. */
if (wireFreq >= 1000u) {
return static_cast<float64>(wireFreq);
}
}
return localFreq;
}
/** /**
* @brief One connected UDPStreamer source. * @brief One connected UDPStreamer source.
* *
@@ -154,6 +252,37 @@ public:
*/ */
void SetRingCapacities(uint32 temporal, uint32 scalar); void SetRingCapacities(uint32 temporal, uint32 scalar);
/**
* @brief Grow every ring so it can retain at least @p seconds of history.
*
* The required capacity is seconds × the rate measured from the ring
* itself (count / time span), because most sources advertise
* samplingRate = 0. Signals whose ring has not filled enough to measure a
* rate are left alone; the caller is expected to retry.
*
* @param seconds Retention target.
* @param maxPts Per-signal ceiling, so a multi-Msps source cannot be
* asked to allocate an unbounded amount of memory.
* @return true if at least one ring was enlarged.
*/
bool GrowRingsForSeconds(float64 seconds, uint32 maxPts);
/** @return Largest ring capacity currently allocated in this session. */
uint32 GetMaxRingCapacity() const;
/**
* @brief Newest timestamp this source has produced on its *own* clock, or
* 0 when it publishes no producer-timed signal (or has no data yet).
*
* Only signals that reference a time signal count: PACKET-timed signals
* are stamped on arrival and so live in the hub's wall-clock domain, not
* the producer's, even when they come from the very same source. A source
* free-running on its own clock sits seconds away from wall time and drifts,
* so anything waiting for a capture window to fill must compare against
* this, never clock_gettime().
*/
float64 ProducerNewestTime() const;
/** /**
* @brief Attach the (shared) hub trigger engine. * @brief Attach the (shared) hub trigger engine.
* Every decoded sample of the trigger's configured signal — resolved * Every decoded sample of the trigger's configured signal — resolved
@@ -198,7 +327,13 @@ private:
/* DATA payload parsing */ /* DATA payload parsing */
void ParseConfigPayload(const uint8 *payload, uint32 size); void ParseConfigPayload(const uint8 *payload, uint32 size);
void ParseDataPayload(const uint8 *payload, uint32 size); /**
* @param lostPackets DATA packets missing immediately before this one, from
* the producer's counter; the accumulated-scalar period estimate
* needs it to know how many cycles the sender-clock gap spans.
*/
void ParseDataPayload(const uint8 *payload, uint32 size,
uint32 lostPackets);
void AllocateRingBuffers(); void AllocateRingBuffers();
/** @brief Invalidate all wall-clock calibration state (receive thread only). */ /** @brief Invalidate all wall-clock calibration state (receive thread only). */
@@ -241,24 +376,42 @@ private:
* signal @p tIdx given the first decoded timer value @p timer0S of the * signal @p tIdx given the first decoded timer value @p timer0S of the
* current packet and the arrival wall time @p wallNowS. * current packet and the arrival wall time @p wallNowS.
* *
* Re-anchors the offset (offset = wallNowS − timer0S) when (a) it is the * Snaps the offset to wallNowS − timer0S only when there is a genuine
* first packet, (b) the source clock jumped backward versus the previous * discontinuity in the source: the first packet, or a backward jump of the
* packet (a looping/rewinding producer), or (c) the computed wall time has * source clock (a looping/rewinding producer).
* drifted past kRecalibThresholdS from the true arrival wall time. *
* A source that free-runs on its own clock also *drifts* against wall time,
* without any discontinuity. Snapping that away would shift the whole
* published timeline in one step and so tear a hole of exactly the drift
* into a stream that is in fact continuous, which is worse than the drift
* itself. Past kRecalibThresholdS the offset is therefore slewed instead:
* nudged towards wall time by at most kMaxSlewFraction of the packet's own
* duration, so the seam can never exceed a fraction of one packet.
*
* @return the calibration offset to add to timer-seconds for this signal. * @return the calibration offset to add to timer-seconds for this signal.
*/ */
inline float64 CalibrateTimeSignal(uint32 tIdx, float64 timer0S, inline float64 CalibrateTimeSignal(uint32 tIdx, float64 timer0S,
float64 wallNowS) { float64 wallNowS) {
static const float64 kRecalibThresholdS = 2.0; static const float64 kRecalibThresholdS = 2.0;
static const float64 kMaxSlewFraction = 0.1;
const bool reset = timeSigLastValid_[tIdx] && const bool reset = timeSigLastValid_[tIdx] &&
(timer0S < timeSigLastTimerS_[tIdx]); (timer0S < timeSigLastTimerS_[tIdx]);
const float64 drift = (timeSigCalib_[tIdx] + timer0S) - wallNowS; if ((!timeSigCalibValid_[tIdx]) || reset) {
const float64 absDrift = (drift < 0.0) ? -drift : drift;
if ((!timeSigCalibValid_[tIdx]) || reset ||
(absDrift > kRecalibThresholdS)) {
timeSigCalib_[tIdx] = wallNowS - timer0S; timeSigCalib_[tIdx] = wallNowS - timer0S;
timeSigCalibValid_[tIdx] = true; timeSigCalibValid_[tIdx] = true;
} }
else {
const float64 drift = (timeSigCalib_[tIdx] + timer0S) - wallNowS;
const float64 absDrift = (drift < 0.0) ? -drift : drift;
if (absDrift > kRecalibThresholdS) {
const float64 pktSpan = timer0S - timeSigLastTimerS_[tIdx];
const float64 maxStep = pktSpan * kMaxSlewFraction;
float64 step = -drift;
if (step > maxStep) { step = maxStep; }
if (step < -maxStep) { step = -maxStep; }
timeSigCalib_[tIdx] += step;
}
}
timeSigLastTimerS_[tIdx] = timer0S; timeSigLastTimerS_[tIdx] = timer0S;
timeSigLastValid_[tIdx] = true; timeSigLastValid_[tIdx] = true;
return timeSigCalib_[tIdx]; return timeSigCalib_[tIdx];
@@ -345,13 +498,20 @@ private:
float64 lastPktWallS_[UDPSS_MAX_SIGNALS]; float64 lastPktWallS_[UDPSS_MAX_SIGNALS];
bool lastPktWallValid_[UDPSS_MAX_SIGNALS]; bool lastPktWallValid_[UDPSS_MAX_SIGNALS];
/* Accumulated-scalar timing: HRT counter frequency (local == sender on the /* Accumulated-scalar timing: the producer's HRT counter frequency and the
* same host) and the previous packet's sample count, used to reconstruct * previous packet's sample count, used to reconstruct per-sample timestamps
* per-sample timestamps from the embedded sender HRT instead of the (UDP * from the embedded sender HRT instead of the (UDP burst-sensitive) packet
* burst-sensitive) packet arrival time. */ * arrival time. Seeded from this host's timer and replaced by the rate the
* producer publishes in CONFIG; see UDPSConfigHrtFrequency. */
float64 hrtFreq_; float64 hrtFreq_;
uint32 accScalarPrevN_[UDPSS_MAX_SIGNALS]; uint32 accScalarPrevN_[UDPSS_MAX_SIGNALS];
/* Per-signal state of UDPSEstimateAccumDt (see above): the smoothed
* per-sample period for accumulated scalars whose descriptor carries no
* SamplingRate. */
float64 accScalarDtEMA_[UDPSS_MAX_SIGNALS];
bool accScalarDtValid_[UDPSS_MAX_SIGNALS];
/* Scratch buffers for decoding arrays (receive thread only). */ /* Scratch buffers for decoding arrays (receive thread only). */
float64 *timeScratch_; ///< Time values scratch float64 *timeScratch_; ///< Time values scratch
float64 *valScratch_; ///< Data values scratch float64 *valScratch_; ///< Data values scratch
Source/Applications/StreamHub/WSServer.cpp
+79 -18
View File
@@ -8,6 +8,7 @@
#include "SHA1.h" #include "SHA1.h"
#include "Base64.h" #include "Base64.h"
#include "AdvancedErrorManagement.h" #include "AdvancedErrorManagement.h"
#include "Select.h"
#include "Sleep.h" #include "Sleep.h"
#include "Threads.h" #include "Threads.h"
#include "TimeoutType.h" #include "TimeoutType.h"
@@ -57,8 +58,10 @@ static const char *FindSubstr(const char *s, const char *pattern) {
WSServer::WSServer() WSServer::WSServer()
: numClients(0u), : numClients(0u),
liveReadThreads(0u),
callback(static_cast<WSCommandCallback *>(0)), callback(static_cast<WSCommandCallback *>(0)),
running(false), running(false),
numAllowedOrigins(0u),
acceptTid(MARTe::InvalidThreadIdentifier) { acceptTid(MARTe::InvalidThreadIdentifier) {
for (uint32 i = 0u; i < WS_MAX_CLIENTS; i++) { for (uint32 i = 0u; i < WS_MAX_CLIENTS; i++) {
@@ -66,6 +69,20 @@ WSServer::WSServer()
clients[i].active = false; clients[i].active = false;
clients[i].readTid = MARTe::InvalidThreadIdentifier; clients[i].readTid = MARTe::InvalidThreadIdentifier;
} }
for (uint32 i = 0u; i < WS_MAX_ORIGINS; i++) {
allowedOrigins[i][0] = '\0';
}
}
bool WSServer::AddAllowedOrigin(const char *origin) {
if ((origin == static_cast<const char *>(0)) || (origin[0] == '\0')) {
return false;
}
if (numAllowedOrigins >= WS_MAX_ORIGINS) { return false; }
if (strlen(origin) >= WS_MAX_ORIGIN_LEN) { return false; }
strcpy(allowedOrigins[numAllowedOrigins], origin);
numAllowedOrigins++;
return true;
} }
WSServer::~WSServer() { WSServer::~WSServer() {
@@ -104,9 +121,9 @@ bool WSServer::Start(uint16 port, WSCommandCallback *cb) {
bool WSServer::Stop() { bool WSServer::Stop() {
if (!running) { return true; } if (!running) { return true; }
running = false; running = false;
Sleep::MSec(200u);
/* Close all client connections — their read threads will exit on error */ /* Close all client connections — their read threads wake out of select()
* and unwind through FreeSlot. */
(void) clientsMutex.FastLock(); (void) clientsMutex.FastLock();
for (uint32 i = 0u; i < WS_MAX_CLIENTS; i++) { for (uint32 i = 0u; i < WS_MAX_CLIENTS; i++) {
if (clients[i].active && (clients[i].sock != static_cast<BasicTCPSocket *>(0))) { if (clients[i].active && (clients[i].sock != static_cast<BasicTCPSocket *>(0))) {
@@ -114,10 +131,23 @@ bool WSServer::Stop() {
} }
} }
clientsMutex.FastUnLock(); clientsMutex.FastUnLock();
Sleep::MSec(200u);
/* The accept loop polls WaitConnection with a 500 ms timeout, so it is out
* of the listener by now. */
Sleep::MSec(600u);
tcpListener.Close(); tcpListener.Close();
Sleep::MSec(100u);
/* Wait for the read threads: they hold pointers to the sockets freed
* below. Bounded — leaking a socket at exit beats deleting one that a
* wedged thread is still reading from. */
static const uint32 kReadJoinMs = 3000u;
for (uint32 waited = 0u; waited < kReadJoinMs; waited += 20u) {
(void) clientsMutex.FastLock();
const uint32 live = liveReadThreads;
clientsMutex.FastUnLock();
if (live == 0u) { break; }
Sleep::MSec(20u);
}
/* Free any remaining slots */ /* Free any remaining slots */
(void) clientsMutex.FastLock(); (void) clientsMutex.FastLock();
@@ -170,6 +200,10 @@ void WSServer::AcceptLoop() {
} }
/* Start per-client read thread */ /* Start per-client read thread */
(void) clientsMutex.FastLock();
liveReadThreads++;
clientsMutex.FastUnLock();
ClientThreadArg *arg = new ClientThreadArg(); ClientThreadArg *arg = new ClientThreadArg();
arg->srv = this; arg->srv = this;
arg->slot = slot; arg->slot = slot;
@@ -200,12 +234,28 @@ bool WSServer::UpgradeHTTP(BasicTCPSocket *sock) {
} }
/* Origin validation (CSWSH / CSRF defence, RFC 6455 §10.2). /* Origin validation (CSWSH / CSRF defence, RFC 6455 §10.2).
* If an Origin header is present, its host must match the Host header * If an Origin header is present it must either be on the configured
* (same-origin). Non-browser clients (no Origin) are allowed. */ * allowlist or its host must match the Host header (same-origin).
* Non-browser clients (no Origin) are allowed. */
const char *originHdr = FindSubstr(hdrBuf, "Origin:"); const char *originHdr = FindSubstr(hdrBuf, "Origin:");
if (originHdr != static_cast<const char *>(0)) { if (originHdr != static_cast<const char *>(0)) {
originHdr += 7; /* skip "Origin:" */ originHdr += 7; /* skip "Origin:" */
while (*originHdr == ' ') { originHdr++; } while (*originHdr == ' ') { originHdr++; }
/* Full origin value "scheme://host[:port]", for the allowlist. */
char originFull[WS_MAX_ORIGIN_LEN];
uint32 ofLen = 0u;
while ((originHdr[ofLen] != '\r') && (originHdr[ofLen] != '\n') &&
(originHdr[ofLen] != '\0') && (ofLen < (WS_MAX_ORIGIN_LEN - 1u))) {
originFull[ofLen] = originHdr[ofLen];
ofLen++;
}
originFull[ofLen] = '\0';
bool allowed = false;
for (uint32 i = 0u; (i < numAllowedOrigins) && !allowed; i++) {
if (strcmp(originFull, allowedOrigins[i]) == 0) { allowed = true; }
}
/* Extract the host part of Origin: "scheme://host[:port]" */ /* Extract the host part of Origin: "scheme://host[:port]" */
char originHost[256]; char originHost[256];
uint32 ohLen = 0u; uint32 ohLen = 0u;
@@ -221,7 +271,7 @@ bool WSServer::UpgradeHTTP(BasicTCPSocket *sock) {
/* Extract Host header value */ /* Extract Host header value */
const char *hostHdr = FindSubstr(hdrBuf, "Host:"); const char *hostHdr = FindSubstr(hdrBuf, "Host:");
if (hostHdr != static_cast<const char *>(0)) { if (!allowed && (hostHdr != static_cast<const char *>(0))) {
hostHdr += 5; /* skip "Host:" */ hostHdr += 5; /* skip "Host:" */
while (*hostHdr == ' ') { hostHdr++; } while (*hostHdr == ' ') { hostHdr++; }
char hostVal[256]; char hostVal[256];
@@ -299,23 +349,30 @@ void WSServer::ClientReadLoop(uint32 slotIdx) {
uint32 filled = 0u; uint32 filled = 0u;
while (running && slot.active) { while (running && slot.active) {
/* Read more bytes (with short timeout so we can check running) */
uint32 want = kRecvBuf - filled; uint32 want = kRecvBuf - filled;
if (want == 0u) { if (want == 0u) {
/* Buffer full — discard old frame (shouldn't happen with reasonable clients) */ /* Buffer full — discard old frame (shouldn't happen with reasonable clients) */
filled = 0u; filled = 0u;
continue; continue;
} }
bool ok = sock->Read(reinterpret_cast<char *>(buf + filled), want,
TimeoutType(500u)); /* Wait for readability before reading. BasicTCPSocket::Read reports a
if (!ok) { * timeout and a closed peer identically (false, zero bytes), so polling
/* Timeout or error — check running and retry */ * it on its own cannot end the loop: once the client goes away recv
if (!running) { break; } * returns immediately and forever, and the thread spins at 100% CPU
if (want == kRecvBuf) { * until it starves the rest of the hub. select() tells the two apart —
/* Zero bytes read — connection likely closed */ * readable followed by no data is end of stream. A wait consumes the
break; * handle set, hence a fresh Select each pass. */
} MARTe::Select sel;
continue; if (!sel.AddReadHandle(*sock)) { break; }
const MARTe::int32 ready = sel.WaitUntil(TimeoutType(500u));
if (ready == 0) { continue; } /* idle client — recheck running */
if (ready < 0) { break; } /* socket closed or errored */
/* Readable: this returns at once, and only fails at end of stream. */
if (!sock->Read(reinterpret_cast<char *>(buf + filled), want,
TimeoutType(500u))) {
break;
} }
filled += want; filled += want;
@@ -383,6 +440,10 @@ client_done:
callback->OnWSClientDisconnected(); callback->OnWSClientDisconnected();
} }
FreeSlot(slotIdx); FreeSlot(slotIdx);
(void) clientsMutex.FastLock();
if (liveReadThreads > 0u) { liveReadThreads--; }
clientsMutex.FastUnLock();
} }
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
Source/Applications/StreamHub/WSServer.h
+25 -1
View File
@@ -34,6 +34,12 @@ static const uint32 WS_MAX_RECV_PAYLOAD = 65536u;
/** Maximum WebSocket frame payload we will send (data frames can be large). */ /** Maximum WebSocket frame payload we will send (data frames can be large). */
static const uint32 WS_MAX_SEND_PAYLOAD = 4u * 1024u * 1024u; /* 4 MiB */ static const uint32 WS_MAX_SEND_PAYLOAD = 4u * 1024u * 1024u; /* 4 MiB */
/** Maximum entries in the Origin allowlist. */
static const uint32 WS_MAX_ORIGINS = 8u;
/** Maximum length of one allowlisted Origin ("scheme://host[:port]"). */
static const uint32 WS_MAX_ORIGIN_LEN = 128u;
/** /**
* @brief Callback interface — implemented by StreamHub. * @brief Callback interface — implemented by StreamHub.
*/ */
@@ -77,6 +83,20 @@ public:
*/ */
bool Start(uint16 port, WSCommandCallback *cb); bool Start(uint16 port, WSCommandCallback *cb);
/**
* @brief Add an Origin that is accepted for the WebSocket upgrade.
*
* With an empty allowlist (the default) only same-origin requests pass:
* the Origin's host must equal the Host header, which excludes the usual
* deployment where the SPA is served by a separate web server on another
* port. Add that server's origin (e.g. "http://localhost:8080") to allow
* it. Requests without an Origin header (non-browser clients) always pass.
*
* @param origin "scheme://host[:port]", compared verbatim.
* @return false if the allowlist is full or the string is too long.
*/
bool AddAllowedOrigin(const char *origin);
/** /**
* @brief Stop accept thread; close all client connections; close listener. * @brief Stop accept thread; close all client connections; close listener.
*/ */
@@ -119,11 +139,15 @@ private:
BasicTCPSocket tcpListener; BasicTCPSocket tcpListener;
WSClientSlot clients[WS_MAX_CLIENTS]; WSClientSlot clients[WS_MAX_CLIENTS];
uint32 numClients; uint32 numClients;
mutable FastPollingMutexSem clientsMutex; ///< Protects numClients and clients[] array uint32 liveReadThreads; ///< Read threads not yet unwound; Stop() waits on it
mutable FastPollingMutexSem clientsMutex; ///< Protects numClients, liveReadThreads and clients[] array
WSCommandCallback *callback; WSCommandCallback *callback;
volatile bool running; volatile bool running;
char allowedOrigins[WS_MAX_ORIGINS][WS_MAX_ORIGIN_LEN];
uint32 numAllowedOrigins;
MARTe::ThreadIdentifier acceptTid; MARTe::ThreadIdentifier acceptTid;
}; };
Source/Components/DataSources/UDPStreamer/UDPStreamer.cpp
+118 -44
View File
@@ -107,6 +107,9 @@ UDPStreamer::UDPStreamer()
readyTimestamps = NULL_PTR(uint64 *); readyTimestamps = NULL_PTR(uint64 *);
scratchTimestamps = NULL_PTR(uint64 *); scratchTimestamps = NULL_PTR(uint64 *);
readyFill = 0u; readyFill = 0u;
readySnapshotPending = false;
droppedPublications = 0u;
lastDropReportTicks = 0u;
decimateRatio = 1u; decimateRatio = 1u;
decimateCounter = 0u; decimateCounter = 0u;
@@ -807,7 +810,9 @@ bool UDPStreamer::PrepareNextState(const char8 *const currentStateName,
* receives it immediately. The config is static for the lifetime of this * receives it immediately. The config is static for the lifetime of this
* state. */ * state. */
if (ok) { if (ok) {
uint32 configBufSize = 4u + (numSigs * UDPS_SIGNAL_DESC_SIZE) + 32u + 1u; /* numSigs + descriptors + publishMode + hrtFrequency (+ slack). */
uint32 configBufSize =
4u + (numSigs * UDPS_SIGNAL_DESC_SIZE) + 1u + 8u + 32u;
HeapI *heap = GlobalObjectsDatabase::Instance()->GetStandardHeap(); HeapI *heap = GlobalObjectsDatabase::Instance()->GetStandardHeap();
uint8 *cfgBuf = reinterpret_cast<uint8 *>(heap->Malloc(configBufSize)); uint8 *cfgBuf = reinterpret_cast<uint8 *>(heap->Malloc(configBufSize));
if (cfgBuf != NULL_PTR(uint8 *)) { if (cfgBuf != NULL_PTR(uint8 *)) {
@@ -871,6 +876,11 @@ bool UDPStreamer::Synchronise() {
/* HI-3: if accumFill reached maxBatchCount, force-flush before writing */ /* HI-3: if accumFill reached maxBatchCount, force-flush before writing */
if (accumFill >= maxBatchCount) { if (accumFill >= maxBatchCount) {
uint32 filled = accumFill; uint32 filled = accumFill;
if (readyFill > 0u) {
/* The sender has not taken the previous batch: it is about to be
* overwritten and its cycles will never reach any receiver. */
droppedPublications++;
}
(void)MemoryOperationsHelper::Copy(readyBuffer, accumBuffer, (void)MemoryOperationsHelper::Copy(readyBuffer, accumBuffer,
filled * totalSrcBytes); filled * totalSrcBytes);
(void)MemoryOperationsHelper::Copy( (void)MemoryOperationsHelper::Copy(
@@ -901,6 +911,9 @@ bool UDPStreamer::Synchronise() {
if (sizeCondition || timeCondition) { if (sizeCondition || timeCondition) {
bufMutex.FastLock(TTInfiniteWait); bufMutex.FastLock(TTInfiniteWait);
if (readyFill > 0u) {
droppedPublications++;
}
(void)MemoryOperationsHelper::Copy(readyBuffer, accumBuffer, (void)MemoryOperationsHelper::Copy(readyBuffer, accumBuffer,
filled * totalSrcBytes); filled * totalSrcBytes);
(void)MemoryOperationsHelper::Copy( (void)MemoryOperationsHelper::Copy(
@@ -922,16 +935,24 @@ bool UDPStreamer::Synchronise() {
if (decimateCounter >= decimateRatio) { if (decimateCounter >= decimateRatio) {
decimateCounter = 0u; decimateCounter = 0u;
bufMutex.FastLock(TTInfiniteWait); bufMutex.FastLock(TTInfiniteWait);
if (readySnapshotPending) {
droppedPublications++;
}
(void)MemoryOperationsHelper::Copy(readyBuffer, memory, totalSrcBytes); (void)MemoryOperationsHelper::Copy(readyBuffer, memory, totalSrcBytes);
syncTimestamp = ts; syncTimestamp = ts;
readySnapshotPending = true;
bufMutex.FastUnLock(); bufMutex.FastUnLock();
(void)dataSem.Post(); (void)dataSem.Post();
} }
} else { } else {
/* --- Strict path: post every call --- */ /* --- Strict path: post every call --- */
bufMutex.FastLock(TTInfiniteWait); bufMutex.FastLock(TTInfiniteWait);
if (readySnapshotPending) {
droppedPublications++;
}
(void)MemoryOperationsHelper::Copy(readyBuffer, memory, totalSrcBytes); (void)MemoryOperationsHelper::Copy(readyBuffer, memory, totalSrcBytes);
syncTimestamp = ts; syncTimestamp = ts;
readySnapshotPending = true;
bufMutex.FastUnLock(); bufMutex.FastUnLock();
(void)dataSem.Post(); (void)dataSem.Post();
} }
@@ -955,60 +976,71 @@ ErrorManagement::ErrorType UDPStreamer::Execute(ExecutionInfo &info) {
} }
if (info.GetStage() == ExecutionInfo::MainStage) { if (info.GetStage() == ExecutionInfo::MainStage) {
/* --- Wait for RT thread to post new data --- /* --- Wait for the RT thread to publish new data ---
* ResetWait sleeps the background thread until the RT thread calls * dataSem is only a wake-up hint, never the record of pending work:
* Synchronise() and posts dataSem, or until the timeout expires. * EventSem::ResetWait resets the semaphore before waiting, so a Post that
* Doing this FIRST means the thread spends nearly all its time here * landed while this thread was inside ServiceClients()/SendData() is
* instead of spinning on the non-blocking select() below. * destroyed by the next Reset. Deciding what to send from the wait result
* Command latency is bounded by UDPS_DATA_WAIT_MS (acceptable for * would then skip that publication entirely, and the next flush would
* CONNECT / DISCONNECT). */ * overwrite it — the receiver sees the batch's whole time span missing.
ErrorManagement::ErrorType waitErr = * The buffers therefore carry the state, and are only waited on when they
dataSem.ResetWait(TimeoutType(UDPS_DATA_WAIT_MS)); * are empty (which also avoids paying the wait when work is already
bool dataReady = (waitErr == ErrorManagement::NoError); * queued). */
if (!HasPendingPublication()) {
(void)dataSem.ResetWait(TimeoutType(UDPS_DATA_WAIT_MS));
}
/* --- Poll for incoming control commands (CONNECT / DISCONNECT / ACK) --- /* --- Poll for incoming control commands (CONNECT / DISCONNECT / ACK) ---
*/ */
server.ServiceClients(); server.ServiceClients();
if (dataReady && server.HasClients()) { /* Synchronise() already gates publication to the correct rate (size/time
/* Synchronise() already gates posting dataSem to the correct rate * for Accumulate, every-Nth for Decimate, every call for Strict). The
* (size/time for Accumulate, every-Nth for Decimate, every call for * pending publication is consumed whether or not anyone is listening, so
* Strict). Execute() just sends whatever is in the ready buffers. */ * that a client-less streamer neither spins here nor delivers a stale
if (publishMode == UDPStreamerPublishAccumulate) { * snapshot to the next client that connects. */
/* --- Accumulate batch send --- */ if (publishMode == UDPStreamerPublishAccumulate) {
uint32 fill = 0u; /* --- Accumulate batch send --- */
bufMutex.FastLock(TTInfiniteWait); uint32 fill = 0u;
fill = readyFill; bufMutex.FastLock(TTInfiniteWait);
if (fill > 0u) { fill = readyFill;
(void)MemoryOperationsHelper::Copy(scratchBuffer, readyBuffer, if (fill > 0u) {
fill * totalSrcBytes); (void)MemoryOperationsHelper::Copy(scratchBuffer, readyBuffer,
(void)MemoryOperationsHelper::Copy( fill * totalSrcBytes);
reinterpret_cast<uint8 *>(scratchTimestamps), (void)MemoryOperationsHelper::Copy(
reinterpret_cast<const uint8 *>(readyTimestamps), reinterpret_cast<uint8 *>(scratchTimestamps),
fill * static_cast<uint32>(sizeof(uint64))); reinterpret_cast<const uint8 *>(readyTimestamps),
} fill * static_cast<uint32>(sizeof(uint64)));
bufMutex.FastUnLock(); readyFill = 0u;
}
bufMutex.FastUnLock();
if (fill > 0u) { if ((fill > 0u) && server.HasClients()) {
SerializeAccumulated(scratchBuffer, scratchTimestamps, fill); SerializeAccumulated(scratchBuffer, scratchTimestamps, fill);
uint32 sendBytes = uint32 sendBytes =
UDPS_TIMESTAMP_BYTES + 4u + fill * singleCycleWireBytes; UDPS_TIMESTAMP_BYTES + 4u + fill * singleCycleWireBytes;
packetCounter++; packetCounter++;
if (!server.SendData(packetCounter, wireBuffer, sendBytes)) { if (!server.SendData(packetCounter, wireBuffer, sendBytes)) {
REPORT_ERROR(ErrorManagement::Warning, REPORT_ERROR(ErrorManagement::Warning,
"Failed to send Accumulate DATA packet (counter=%u).", "Failed to send Accumulate DATA packet (counter=%u).",
packetCounter); packetCounter);
}
} }
} else { }
/* --- Single-snapshot send (Strict or Decimate) --- */ } else {
uint64 ts = 0u; /* --- Single-snapshot send (Strict or Decimate) --- */
bufMutex.FastLock(TTInfiniteWait); uint64 ts = 0u;
bool pending = false;
bufMutex.FastLock(TTInfiniteWait);
pending = readySnapshotPending;
if (pending) {
(void)MemoryOperationsHelper::Copy(scratchBuffer, readyBuffer, (void)MemoryOperationsHelper::Copy(scratchBuffer, readyBuffer,
totalSrcBytes); totalSrcBytes);
ts = syncTimestamp; ts = syncTimestamp;
bufMutex.FastUnLock(); readySnapshotPending = false;
}
bufMutex.FastUnLock();
if (pending && server.HasClients()) {
QuantizeAndSerialize(scratchBuffer, ts); QuantizeAndSerialize(scratchBuffer, ts);
packetCounter++; packetCounter++;
@@ -1019,6 +1051,8 @@ ErrorManagement::ErrorType UDPStreamer::Execute(ExecutionInfo &info) {
} }
} }
} }
ReportDroppedPublications();
} }
if (info.GetStage() == ExecutionInfo::TerminationStage) { if (info.GetStage() == ExecutionInfo::TerminationStage) {
@@ -1212,6 +1246,16 @@ bool UDPStreamer::BuildConfigPayload(uint8 *buf, uint32 bufSize,
buf[payloadSize] = static_cast<uint8>(publishMode); buf[payloadSize] = static_cast<uint8>(publishMode);
payloadSize += 1u; payloadSize += 1u;
/* 8 bytes: this host's HRT tick rate. DATA packets carry raw counter
* values, so a receiver on another machine cannot turn them into seconds
* without it. */
if ((payloadSize + 8u) > bufSize) {
return false;
}
uint64 hrtFrequency = HighResolutionTimer::Frequency();
(void)MemoryOperationsHelper::Copy(buf + payloadSize, &hrtFrequency, 8u);
payloadSize += 8u;
return true; return true;
} }
@@ -1327,6 +1371,36 @@ bool UDPStreamer::IsClientConnected() const { return server.HasClients(); }
bool UDPStreamer::IsMulticast() const { return server.IsMulticast(); } bool UDPStreamer::IsMulticast() const { return server.IsMulticast(); }
uint32 UDPStreamer::GetDroppedPublications() const { return droppedPublications; }
bool UDPStreamer::HasPendingPublication() {
bool pending = false;
bufMutex.FastLock(TTInfiniteWait);
pending = (readyFill > 0u) || readySnapshotPending;
bufMutex.FastUnLock();
return pending;
}
void UDPStreamer::ReportDroppedPublications() {
uint64 now = HighResolutionTimer::Counter();
if ((now - lastDropReportTicks) < HighResolutionTimer::Frequency()) {
return;
}
lastDropReportTicks = now;
uint32 dropped = 0u;
bufMutex.FastLock(TTInfiniteWait);
dropped = droppedPublications;
bufMutex.FastUnLock();
if (dropped > 0u) {
REPORT_ERROR(ErrorManagement::Warning,
"Dropped %u unsent publication(s) so far: the sender thread is "
"not keeping up with the RT cycle.",
dropped);
}
}
CLASS_REGISTER(UDPStreamer, "1.0") CLASS_REGISTER(UDPStreamer, "1.0")
} /* namespace MARTe */ } /* namespace MARTe */
Source/Components/DataSources/UDPStreamer/UDPStreamer.h
+30
View File
@@ -322,6 +322,17 @@ public:
*/ */
bool IsMulticast() const; bool IsMulticast() const;
/**
* @brief Number of publications the sender thread never put on the wire.
* @details Synchronise() promotes a snapshot (Strict/Decimate) or a batch
* (Accumulate) to the ready buffer for the sender thread. If the next
* promotion arrives before the sender has taken the previous one, that
* publication is overwritten and its cycles never reach any receiver —
* which a consumer sees as a hole in the time series. Counts those, so the
* loss is measurable rather than inferred from the plot.
*/
uint32 GetDroppedPublications() const;
private: private:
/** /**
* @brief Serializes the CONFIG payload into buf and sets payloadSize. * @brief Serializes the CONFIG payload into buf and sets payloadSize.
@@ -349,6 +360,18 @@ private:
*/ */
static uint8 TypeDescriptorToCode(TypeDescriptor td); static uint8 TypeDescriptorToCode(TypeDescriptor td);
/**
* @brief True when the ready buffer holds data the sender has not taken yet.
* @details Read under bufMutex. The sender must consult this rather than
* rely on the dataSem edge, which ResetWait can destroy.
*/
bool HasPendingPublication();
/**
* @brief Emits at most one warning per second about overwritten publications.
*/
void ReportDroppedPublications();
/* Configuration parameters */ /* Configuration parameters */
uint16 port; /**< UDP server port */ uint16 port; /**< UDP server port */
uint32 maxPayloadSize; /**< Max payload bytes per UDP packet (excluding header) */ uint32 maxPayloadSize; /**< Max payload bytes per UDP packet (excluding header) */
@@ -367,6 +390,13 @@ private:
uint64 *readyTimestamps; /**< Heap: [maxBatchCount] HRT for completed ready batch */ uint64 *readyTimestamps; /**< Heap: [maxBatchCount] HRT for completed ready batch */
uint64 *scratchTimestamps; /**< Heap: [maxBatchCount] background-thread local copy */ uint64 *scratchTimestamps; /**< Heap: [maxBatchCount] background-thread local copy */
uint32 readyFill; /**< Snapshot count in the ready batch */ uint32 readyFill; /**< Snapshot count in the ready batch */
/** Strict/Decimate: readyBuffer holds a snapshot the sender has not taken
* yet. Publication state must live here rather than in dataSem, because
* EventSem::ResetWait resets before waiting and so destroys any Post that
* landed while the sender was busy. */
bool readySnapshotPending;
uint32 droppedPublications; /**< Publications overwritten before being sent */
uint64 lastDropReportTicks; /**< Sender-thread rate limit for the drop warning */
/* Decimate mode */ /* Decimate mode */
uint32 decimateRatio; /**< Send 1 packet every decimateRatio Synchronise() calls */ uint32 decimateRatio; /**< Send 1 packet every decimateRatio Synchronise() calls */
uint32 decimateCounter; /**< Current decimate cycle counter */ uint32 decimateCounter; /**< Current decimate cycle counter */
Source/Components/DataSources/UDPStreamerClient/UDPStreamerClient.cpp
+20 -2
View File
@@ -58,6 +58,9 @@ static const uint32 UDPS_CLIENT_DEFAULT_MAX_PAYLOAD = 1400u;
/** Default unicast keepalive interval (seconds); 0 disables. */ /** Default unicast keepalive interval (seconds); 0 disables. */
static const uint32 UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S = 15u; static const uint32 UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S = 15u;
/** Default silence timeout before reconnect (seconds); sub-second values allowed. */
static const float32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 1.0f;
/** Bytes prepended to each DATA payload for the HRT packet timestamp. */ /** Bytes prepended to each DATA payload for the HRT packet timestamp. */
static const uint32 UDPS_CLIENT_TIMESTAMP_BYTES = 8u; static const uint32 UDPS_CLIENT_TIMESTAMP_BYTES = 8u;
@@ -133,6 +136,7 @@ UDPStreamerClient::UDPStreamerClient() :
port = UDPS_CLIENT_DEFAULT_PORT; port = UDPS_CLIENT_DEFAULT_PORT;
maxPayloadSize = UDPS_CLIENT_DEFAULT_MAX_PAYLOAD; maxPayloadSize = UDPS_CLIENT_DEFAULT_MAX_PAYLOAD;
keepAliveInterval = UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S; keepAliveInterval = UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S;
silenceTimeout = UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S;
cpuMask = 0xFFFFFFFFu; cpuMask = 0xFFFFFFFFu;
stackSize = THREADS_DEFAULT_STACKSIZE; stackSize = THREADS_DEFAULT_STACKSIZE;
dataPort = UDPS_CLIENT_DEFAULT_PORT + UDPS_CLIENT_DEFAULT_DP_OFFSET; dataPort = UDPS_CLIENT_DEFAULT_PORT + UDPS_CLIENT_DEFAULT_DP_OFFSET;
@@ -211,6 +215,12 @@ bool UDPStreamerClient::Initialise(StructuredDataI &data) {
} }
} }
if (ok) {
if (!data.Read("SilenceTimeout", silenceTimeout)) {
silenceTimeout = UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S;
}
}
if (ok) { if (ok) {
if (!data.Read("CPUMask", cpuMask)) { if (!data.Read("CPUMask", cpuMask)) {
cpuMask = 0xFFFFFFFFu; cpuMask = 0xFFFFFFFFu;
@@ -234,10 +244,14 @@ bool UDPStreamerClient::Initialise(StructuredDataI &data) {
dp = port + UDPS_CLIENT_DEFAULT_DP_OFFSET; dp = port + UDPS_CLIENT_DEFAULT_DP_OFFSET;
} }
dataPort = dp; dataPort = dp;
StreamString ifaceStr = "";
(void) data.Read("Interface", ifaceStr);
multicastInterface = ifaceStr;
REPORT_ERROR(ErrorManagement::Information, REPORT_ERROR(ErrorManagement::Information,
"Multicast mode: group=%s, server=%s, controlPort=%u, dataPort=%u.", "Multicast mode: group=%s, server=%s, controlPort=%u, dataPort=%u, interface=%s.",
multicastGroup.Buffer(), serverAddress.Buffer(), multicastGroup.Buffer(), serverAddress.Buffer(),
static_cast<uint32>(port), static_cast<uint32>(dataPort)); static_cast<uint32>(port), static_cast<uint32>(dataPort),
(multicastInterface.Size() > 0u) ? multicastInterface.Buffer() : "default");
} }
else { else {
useMulticast = false; useMulticast = false;
@@ -253,9 +267,13 @@ bool UDPStreamerClient::Initialise(StructuredDataI &data) {
if (ok && useMulticast) { if (ok && useMulticast) {
ok = cdb.Write("MulticastGroup", multicastGroup); ok = cdb.Write("MulticastGroup", multicastGroup);
if (ok) { ok = cdb.Write("DataPort", static_cast<uint32>(dataPort)); } if (ok) { ok = cdb.Write("DataPort", static_cast<uint32>(dataPort)); }
if (ok && (multicastInterface.Size() > 0u)) {
ok = cdb.Write("Interface", multicastInterface);
}
} }
if (ok) { ok = cdb.Write("MaxPayloadSize", maxPayloadSize); } if (ok) { ok = cdb.Write("MaxPayloadSize", maxPayloadSize); }
if (ok) { ok = cdb.Write("KeepAliveInterval", keepAliveInterval); } if (ok) { ok = cdb.Write("KeepAliveInterval", keepAliveInterval); }
if (ok) { ok = cdb.Write("SilenceTimeout", silenceTimeout); }
if (ok) { ok = cdb.Write("CPUMask", cpuMask); } if (ok) { ok = cdb.Write("CPUMask", cpuMask); }
if (ok) { ok = cdb.Write("StackSize", stackSize); } if (ok) { ok = cdb.Write("StackSize", stackSize); }
if (ok) { ok = cdb.MoveToRoot(); } if (ok) { ok = cdb.MoveToRoot(); }
Source/Components/DataSources/UDPStreamerClient/UDPStreamerClient.h
+5 -3
View File
@@ -175,11 +175,13 @@ private:
uint16 port; /**< Server port. */ uint16 port; /**< Server port. */
uint32 maxPayloadSize; /**< Max payload bytes per datagram. */ uint32 maxPayloadSize; /**< Max payload bytes per datagram. */
uint32 keepAliveInterval; /**< Seconds between unicast keepalive ACKs (0 disables). */ uint32 keepAliveInterval; /**< Seconds between unicast keepalive ACKs (0 disables). */
float32 silenceTimeout; /**< Seconds of no data before reconnect (sub-second allowed, 0 disables). */
uint32 cpuMask; /**< Background thread CPU affinity. */ uint32 cpuMask; /**< Background thread CPU affinity. */
uint32 stackSize; /**< Background thread stack size. */ uint32 stackSize; /**< Background thread stack size. */
StreamString multicastGroup; /**< Multicast group IP; empty = unicast. */ StreamString multicastGroup; /**< Multicast group IP; empty = unicast. */
uint16 dataPort; /**< UDP port for DATA datagrams (multicast). */ StreamString multicastInterface; /**< Local IPv4 address for multicast join; empty = INADDR_ANY. */
bool useMulticast; /**< True when MulticastGroup is set. */ uint16 dataPort; /**< UDP port for DATA datagrams (multicast). */
bool useMulticast; /**< True when MulticastGroup is set. */
/* Signal metadata */ /* Signal metadata */
uint32 numSigs; /**< Number of signals. */ uint32 numSigs; /**< Number of signals. */
Source/Components/GAMs/TimeArrayGAM/TimeArrayGAM.cpp
+23 -2
View File
@@ -16,6 +16,10 @@ TimeArrayGAM::TimeArrayGAM() :
GAM(), GAM(),
samplingRate(1000000.0), samplingRate(1000000.0),
anchorIsFirst(true), anchorIsFirst(true),
anchorIsCont(false),
contStarted(false),
contOriginNs(0u),
contSamples(0u),
nElements(0u), nElements(0u),
inputTime(NULL_PTR(uint32 *)), inputTime(NULL_PTR(uint32 *)),
outputBuf(NULL_PTR(uint64 *)) { outputBuf(NULL_PTR(uint64 *)) {
@@ -42,9 +46,12 @@ bool TimeArrayGAM::Initialise(StructuredDataI &data) {
else if (anchor == "LastSample") { else if (anchor == "LastSample") {
anchorIsFirst = false; anchorIsFirst = false;
} }
else if (anchor == "Continuous") {
anchorIsCont = true;
}
else { else {
REPORT_ERROR(ErrorManagement::InitialisationError, REPORT_ERROR(ErrorManagement::InitialisationError,
"TimeArrayGAM: Anchor must be 'FirstSample' or 'LastSample'."); "TimeArrayGAM: Anchor must be 'FirstSample', 'LastSample' or 'Continuous'.");
ok = false; ok = false;
} }
} }
@@ -88,7 +95,21 @@ bool TimeArrayGAM::Execute() {
/* Input is uint32 microseconds (LinuxTimer); convert to nanoseconds. */ /* Input is uint32 microseconds (LinuxTimer); convert to nanoseconds. */
uint64 anchorNs = static_cast<uint64>(*inputTime) * 1000u; uint64 anchorNs = static_cast<uint64>(*inputTime) * 1000u;
if (anchorIsFirst) { if (anchorIsCont) {
/* Latch the timer once, then run off an internal sample counter so a
* lost RT cycle (LinuxTimer re-phases with counter += nCycles) cannot
* punch a hole into an otherwise contiguous sample stream. */
if (!contStarted) {
contOriginNs = anchorNs;
contStarted = true;
}
for (uint32 k = 0u; k < nElements; k++) {
outputBuf[k] = contOriginNs +
(contSamples + static_cast<uint64>(k)) * periodNs;
}
contSamples += static_cast<uint64>(nElements);
}
else if (anchorIsFirst) {
/* out[k] = anchorNs + k * periodNs */ /* out[k] = anchorNs + k * periodNs */
for (uint32 k = 0u; k < nElements; k++) { for (uint32 k = 0u; k < nElements; k++) {
outputBuf[k] = anchorNs + static_cast<uint64>(k) * periodNs; outputBuf[k] = anchorNs + static_cast<uint64>(k) * periodNs;
Source/Components/GAMs/TimeArrayGAM/TimeArrayGAM.h
+14 -1
View File
@@ -10,6 +10,15 @@
* *
* Anchor = FirstSample: out[k] = input + k * period_us * Anchor = FirstSample: out[k] = input + k * period_us
* Anchor = LastSample: out[k] = input - (N-1-k) * period_us * Anchor = LastSample: out[k] = input - (N-1-k) * period_us
* Anchor = Continuous: out[k] = input(first cycle) + (n + k) * period_us
*
* FirstSample/LastSample re-read the timer every cycle, so they propagate any
* cycle the RT thread loses: LinuxTimer re-phases (counter += nCycles) and the
* emitted time base jumps by a whole period while only one array of samples is
* produced, leaving a hole. Continuous anchors once and then advances an
* internal sample counter by N per cycle, which is what an acquisition card
* with its own clock does — use it when the data signal is itself contiguous
* (SineArrayGAM, for instance, never skips phase on a lost cycle).
* *
* The resulting time array is suitable as the TimeSignal for a UDPStreamer signal * The resulting time array is suitable as the TimeSignal for a UDPStreamer signal
* configured with TimeMode = FullArray, providing exact per-sample timestamps. * configured with TimeMode = FullArray, providing exact per-sample timestamps.
@@ -19,7 +28,7 @@
* +TimeArrayGAM1 = { * +TimeArrayGAM1 = {
* Class = TimeArrayGAM * Class = TimeArrayGAM
* SamplingRate = 1000000.0 // Sample rate in Hz (must match data signal) * SamplingRate = 1000000.0 // Sample rate in Hz (must match data signal)
* Anchor = FirstSample // FirstSample (default) or LastSample * Anchor = FirstSample // FirstSample (default), LastSample or Continuous
* InputSignals = { * InputSignals = {
* Time = { DataSource = DDB; Type = uint32 } * Time = { DataSource = DDB; Type = uint32 }
* } * }
@@ -54,6 +63,10 @@ public:
private: private:
float64 samplingRate; /**< Sample rate [Hz] */ float64 samplingRate; /**< Sample rate [Hz] */
bool anchorIsFirst; /**< true = FirstSample anchor, false = LastSample */ bool anchorIsFirst; /**< true = FirstSample anchor, false = LastSample */
bool anchorIsCont; /**< true = Continuous anchor (internal sample counter) */
bool contStarted; /**< Continuous: origin has been latched */
uint64 contOriginNs; /**< Continuous: timer value latched on the first cycle */
uint64 contSamples; /**< Continuous: samples emitted so far */
uint32 nElements; /**< Number of output elements */ uint32 nElements; /**< Number of output elements */
uint32 *inputTime; /**< Pointer to scalar input (microseconds, uint32 from LinuxTimer) */ uint32 *inputTime; /**< Pointer to scalar input (microseconds, uint32 from LinuxTimer) */
uint64 *outputBuf; /**< Pointer to output array (nanoseconds, uint64) */ uint64 *outputBuf; /**< Pointer to output array (nanoseconds, uint64) */
Source/Components/Interfaces/DebugService/DebugService.cpp
+8
View File
@@ -674,6 +674,14 @@ bool DebugService::SendUDPSConfig() {
payloadOffset++; payloadOffset++;
} }
// Write this host's HRT tick rate: DATA packets carry raw counter values,
// which a receiver on another machine cannot convert to seconds without it.
if ((payloadOffset + 8u) <= CFG_BUF_SIZE) {
uint64 hrtFrequency = HighResolutionTimer::Frequency();
memcpy(payload + payloadOffset, &hrtFrequency, 8u);
payloadOffset += 8u;
}
udpsNumSlots = newNumSlots; udpsNumSlots = newNumSlots;
mutex.FastUnLock(); mutex.FastUnLock();
Source/Components/Interfaces/UDPStream/UDPSClient.cpp
+214 -70
View File
@@ -36,7 +36,13 @@ UDPSClient::UDPSClient()
disconnectTick(0u), disconnectTick(0u),
lastKeepAliveTicks(0u), lastKeepAliveTicks(0u),
localPort(0u), localPort(0u),
lastGcTicks(0u) { lastGcTicks(0u),
lastDropWarnTicks(0u),
droppedSinceWarn(0u),
lastDataCounter(0u),
lastDataCounterValid(false),
lastDataGap(0u),
staleDataPackets(0u) {
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) { for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
reassemblySlots[i].counter = 0u; reassemblySlots[i].counter = 0u;
@@ -46,7 +52,11 @@ UDPSClient::UDPSClient()
reassemblySlots[i].active = false; reassemblySlots[i].active = false;
reassemblySlots[i].firstSeenTicks = 0u; reassemblySlots[i].firstSeenTicks = 0u;
reassemblySlots[i].chunkSize = 0u; reassemblySlots[i].chunkSize = 0u;
(void) MemoryOperationsHelper::Set(reassemblySlots[i].recvMask, 0, 32u); reassemblySlots[i].assembledBytes = 0u;
reassemblySlots[i].pendingTailBytes = 0u;
reassemblySlots[i].pendingTailValid = false;
(void) MemoryOperationsHelper::Set(reassemblySlots[i].recvMask, 0,
UDPS_CLIENT_RECV_MASK_BYTES);
} }
} }
@@ -85,11 +95,16 @@ bool UDPSClient::Initialise(StructuredDataI &data) {
uint32 dpU32 = static_cast<uint32>(serverPort) + 1u; uint32 dpU32 = static_cast<uint32>(serverPort) + 1u;
(void) data.Read("DataPort", dpU32); (void) data.Read("DataPort", dpU32);
dataPort = static_cast<uint16>(dpU32); dataPort = static_cast<uint16>(dpU32);
StreamString iface;
(void) data.Read("Interface", iface);
multicastInterface = iface;
} }
uint32 silenceS = UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S; float32 silenceS = UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S;
(void) data.Read("SilenceTimeout", silenceS); (void) data.Read("SilenceTimeout", silenceS);
silenceTimeoutTicks = static_cast<uint64>(silenceS) * HighResolutionTimer::Frequency(); /* float64 math: the tick rate (~1e9) exceeds float32's 24-bit mantissa */
silenceTimeoutTicks = static_cast<uint64>(static_cast<float64>(silenceS) *
static_cast<float64>(HighResolutionTimer::Frequency()));
uint32 reconnectS = UDPS_CLIENT_DEFAULT_RECONNECT_DELAY_S; uint32 reconnectS = UDPS_CLIENT_DEFAULT_RECONNECT_DELAY_S;
(void) data.Read("ReconnectDelay", reconnectS); (void) data.Read("ReconnectDelay", reconnectS);
@@ -223,6 +238,12 @@ bool UDPSClient::Connect() {
connected = true; connected = true;
lastDataTicks = HighResolutionTimer::Counter(); lastDataTicks = HighResolutionTimer::Counter();
lastKeepAliveTicks = lastDataTicks; lastKeepAliveTicks = lastDataTicks;
/* The producer's packetCounter restarts independently of ours, so a
* counter carried over from the previous connection would make the
* sequence gate reject the whole new stream as stale. */
lastDataCounterValid = false;
lastDataCounter = 0u;
lastDataGap = 0u;
if (listener != NULL_PTR(UDPSClientListener *)) { if (listener != NULL_PTR(UDPSClientListener *)) {
listener->OnUDPSConnected(); listener->OnUDPSConnected();
} }
@@ -309,7 +330,12 @@ bool UDPSClient::ConnectMulticast() {
return false; return false;
} }
ok = mcastSocket.Join(multicastGroup.Buffer()); if (multicastInterface.Size() > 0u) {
ok = mcastSocket.Join(multicastGroup.Buffer(), multicastInterface.Buffer());
}
else {
ok = mcastSocket.Join(multicastGroup.Buffer());
}
if (!ok) { if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::Warning, REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: Could not join multicast group %s.", "UDPSClient: Could not join multicast group %s.",
@@ -318,8 +344,9 @@ bool UDPSClient::ConnectMulticast() {
return false; return false;
} }
REPORT_ERROR_STATIC(ErrorManagement::Information, REPORT_ERROR_STATIC(ErrorManagement::Information,
"UDPSClient: Joined multicast group %s on port %u.", "UDPSClient: Joined multicast group %s on port %u via interface %s.",
multicastGroup.Buffer(), static_cast<uint32>(dataPort)); multicastGroup.Buffer(), static_cast<uint32>(dataPort),
(multicastInterface.Size() > 0u) ? multicastInterface.Buffer() : "default");
// Now open the TCP control connection and announce ourselves // Now open the TCP control connection and announce ourselves
if (!tcpSocket.Open()) { if (!tcpSocket.Open()) {
@@ -487,25 +514,45 @@ bool UDPSClient::ReceiveAndProcess() {
return true; // only the TCP socket was readable return true; // only the TCP socket was readable
} }
uint32 recvSize = static_cast<uint32>(sizeof(recvBuf)); /* Drain the socket rather than taking one datagram per Execute() iteration:
bool ok; * a fragmented high-rate source delivers datagrams far faster than the
if (useMulticast) { * select/read round trip retires them, and the resulting kernel-buffer
ok = mcastSocket.Read(reinterpret_cast<char8 *>(recvBuf), recvSize); * overflow shows up as lost fragments — i.e. as packets that can never be
} * reassembled. Bounded so the silence and keepalive checks in Execute()
else { * still run under a sustained flood. */
ok = recvSocket.Read(reinterpret_cast<char8 *>(recvBuf), recvSize); uint32 drained = 0u;
} while (drained < UDPS_CLIENT_MAX_DATAGRAMS_PER_CYCLE) {
uint32 recvSize = static_cast<uint32>(sizeof(recvBuf));
bool ok;
if (useMulticast) {
ok = mcastSocket.Read(reinterpret_cast<char8 *>(recvBuf), recvSize);
}
else {
ok = recvSocket.Read(reinterpret_cast<char8 *>(recvBuf), recvSize);
}
if (!ok || (recvSize < UDPS_HEADER_SIZE)) { if (!ok || (recvSize < UDPS_HEADER_SIZE)) {
REPORT_ERROR_STATIC(ErrorManagement::Warning, REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: ReceiveAndProcess: Read() failed or short packet " "UDPSClient: ReceiveAndProcess: Read() failed or short packet "
"(ok=%s, recvSize=%u, HEADER_SIZE=%u).", "(ok=%s, recvSize=%u, HEADER_SIZE=%u).",
ok ? "true" : "false", recvSize, UDPS_HEADER_SIZE); ok ? "true" : "false", recvSize, UDPS_HEADER_SIZE);
return false; return false;
} }
lastDataTicks = HighResolutionTimer::Counter(); lastDataTicks = HighResolutionTimer::Counter();
ProcessDatagram(recvBuf, recvSize); ProcessDatagram(recvBuf, recvSize);
drained++;
/* Stop as soon as the socket runs dry: Read() would otherwise block. */
fd_set dset;
FD_ZERO(&dset);
FD_SET(fd, &dset);
struct timeval zero;
zero.tv_sec = 0; zero.tv_usec = 0;
if (select(fd + 1, &dset, NULL, NULL, &zero) <= 0) {
break;
}
}
return true; return true;
} }
@@ -588,7 +635,7 @@ void UDPSClient::ProcessDatagram(const uint8 *buf, uint32 size) {
if (hdr->type == UDPS_TYPE_CONFIG) { if (hdr->type == UDPS_TYPE_CONFIG) {
listener->OnUDPSConfig(pl, payloadBytes); listener->OnUDPSConfig(pl, payloadBytes);
} }
else { else if (AcceptDataCounter(hdr->counter)) {
listener->OnUDPSData(pl, payloadBytes); listener->OnUDPSData(pl, payloadBytes);
} }
} }
@@ -605,21 +652,83 @@ void UDPSClient::ProcessDatagram(const uint8 *buf, uint32 size) {
// Private: PlaceFragment // Private: PlaceFragment
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
uint32 UDPSClient::AcquireReassemblySlot(uint32 counter, uint8 type) {
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (!reassemblySlots[i].active) {
return i;
}
}
/* All slots busy. The producer emits packets sequentially, so a slot
* holding an OLDER counter of the SAME stream is provably dead: its
* missing fragments were sent before the ones arriving now and will never
* turn up. Reclaiming it immediately — instead of waiting out the 2 s GC —
* is what keeps a handful of lost fragments from wedging the whole table.
* The counter is a wrapping uint32, so compare via the signed difference. */
uint32 victim = UDPS_CLIENT_MAX_REASSEMBLY_SLOTS;
int32 bestDist = 0;
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (reassemblySlots[i].type != type) {
continue;
}
int32 dist = static_cast<int32>(counter - reassemblySlots[i].counter);
if ((dist > 0) && (dist > bestDist)) {
bestDist = dist;
victim = i;
}
}
if (victim >= UDPS_CLIENT_MAX_REASSEMBLY_SLOTS) {
/* Nothing is provably dead (e.g. the other stream owns every slot):
* fall back to the least recently started. */
uint64 oldestTick = 0xFFFFFFFFFFFFFFFFuLL;
victim = 0u;
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (reassemblySlots[i].firstSeenTicks < oldestTick) {
oldestTick = reassemblySlots[i].firstSeenTicks;
victim = i;
}
}
}
NoteDroppedIncomplete(reassemblySlots[victim].counter);
return victim;
}
void UDPSClient::NoteDroppedIncomplete(uint32 counter) {
droppedSinceWarn++;
uint64 now = HighResolutionTimer::Counter();
if ((now - lastDropWarnTicks) >= HighResolutionTimer::Frequency()) {
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: dropped %u incomplete packet(s) in the last "
"second (latest counter %u); fragments are being lost.",
droppedSinceWarn, counter);
droppedSinceWarn = 0u;
lastDropWarnTicks = now;
}
}
bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr, bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
const uint8 *payload, const uint8 *payload,
uint32 payloadBytes) { uint32 payloadBytes) {
uint32 counter = hdr->counter; uint32 counter = hdr->counter;
uint8 type = hdr->type;
uint16 fragIdx = hdr->fragmentIdx; uint16 fragIdx = hdr->fragmentIdx;
uint16 totalFrags = hdr->totalFragments; uint16 totalFrags = hdr->totalFragments;
if ((fragIdx >= totalFrags) || (totalFrags > 512u)) { if ((fragIdx >= totalFrags) ||
(static_cast<uint32>(totalFrags) > UDPS_CLIENT_MAX_FRAGMENTS)) {
return false; // sanity check return false; // sanity check
} }
// Find existing slot for this counter /* Slots are keyed on (counter, type): DATA and CONFIG carry independent
* counter sequences, so the same counter value legitimately appears on
* both, and matching on the counter alone merges the two streams into one
* slot — one payload is delivered under the wrong type, the other is lost. */
uint32 slot = UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; uint32 slot = UDPS_CLIENT_MAX_REASSEMBLY_SLOTS;
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) { for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (reassemblySlots[i].active && (reassemblySlots[i].counter == counter)) { if (reassemblySlots[i].active && (reassemblySlots[i].counter == counter) &&
(reassemblySlots[i].type == type)) {
slot = i; slot = i;
break; break;
} }
@@ -627,34 +736,20 @@ bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
// Allocate new slot if not found // Allocate new slot if not found
if (slot >= UDPS_CLIENT_MAX_REASSEMBLY_SLOTS) { if (slot >= UDPS_CLIENT_MAX_REASSEMBLY_SLOTS) {
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) { slot = AcquireReassemblySlot(counter, type);
if (!reassemblySlots[i].active) {
slot = i;
break;
}
}
if (slot >= UDPS_CLIENT_MAX_REASSEMBLY_SLOTS) {
// All slots occupied — evict the oldest
uint64 oldestTick = 0xFFFFFFFFFFFFFFFFuLL;
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (reassemblySlots[i].firstSeenTicks < oldestTick) {
oldestTick = reassemblySlots[i].firstSeenTicks;
slot = i;
}
}
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: Reassembly slots full; evicting oldest.");
}
reassemblySlots[slot].counter = counter; reassemblySlots[slot].counter = counter;
reassemblySlots[slot].type = hdr->type; reassemblySlots[slot].type = type;
reassemblySlots[slot].totalFragments = totalFrags; reassemblySlots[slot].totalFragments = totalFrags;
reassemblySlots[slot].receivedFragments = 0u; reassemblySlots[slot].receivedFragments = 0u;
reassemblySlots[slot].active = true; reassemblySlots[slot].active = true;
reassemblySlots[slot].firstSeenTicks = HighResolutionTimer::Counter(); reassemblySlots[slot].firstSeenTicks = HighResolutionTimer::Counter();
reassemblySlots[slot].chunkSize = 0u; reassemblySlots[slot].chunkSize = 0u;
reassemblySlots[slot].assembledBytes = 0u; reassemblySlots[slot].assembledBytes = 0u;
(void) MemoryOperationsHelper::Set(reassemblySlots[slot].recvMask, 0, 32u); reassemblySlots[slot].pendingTailBytes = 0u;
reassemblySlots[slot].pendingTailValid = false;
(void) MemoryOperationsHelper::Set(reassemblySlots[slot].recvMask, 0,
UDPS_CLIENT_RECV_MASK_BYTES);
} }
UDPSReassemblySlot &s = reassemblySlots[slot]; UDPSReassemblySlot &s = reassemblySlots[slot];
@@ -662,27 +757,56 @@ bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
// Skip duplicate // Skip duplicate
uint32 byteIdx = fragIdx / 8u; uint32 byteIdx = fragIdx / 8u;
uint8 bitMask = static_cast<uint8>(1u << (fragIdx % 8u)); uint8 bitMask = static_cast<uint8>(1u << (fragIdx % 8u));
if (byteIdx < 32u) { if ((s.recvMask[byteIdx] & bitMask) != 0u) {
if ((s.recvMask[byteIdx] & bitMask) != 0u) { return false; // already have this fragment
return false; // already have this fragment
}
} }
// Compute placement offset const bool isLastFragment = ((static_cast<uint32>(fragIdx) + 1u) ==
uint32 chunkSize = s.chunkSize; static_cast<uint32>(totalFrags));
if (chunkSize == 0u) {
// Learn chunk size from first non-last fragment /* Every fragment but the last carries a full chunk, so any of them reveals
if (fragIdx == 0u) { * the chunk size — waiting specifically for fragment 0 means a merely
chunkSize = payloadBytes; * reordered burst, with nothing lost, destroys the packet. */
s.chunkSize = chunkSize; if ((s.chunkSize == 0u) && !isLastFragment) {
} s.chunkSize = payloadBytes;
else { }
// Can't place yet without knowing chunk size — drop (rare edge case)
if (s.chunkSize == 0u) {
/* The last fragment arrived before any full-size one: its offset is
* not computable yet, so hold it until the chunk size is known. */
if (payloadBytes > UDPS_CLIENT_PENDING_TAIL_BYTES) {
return false; return false;
} }
if (payloadBytes > 0u) {
(void) MemoryOperationsHelper::Copy(s.pendingTail, payload, payloadBytes);
}
s.pendingTailBytes = payloadBytes;
s.pendingTailValid = true;
s.recvMask[byteIdx] |= bitMask;
s.receivedFragments++;
return false; // totalFrags > 1 here, so this can never complete a packet
} }
uint32 offset = static_cast<uint32>(fragIdx) * chunkSize; // Flush a deferred last fragment now that the chunk size is known.
if (s.pendingTailValid) {
uint32 tailOffset = (static_cast<uint32>(s.totalFragments) - 1u) * s.chunkSize;
if ((tailOffset + s.pendingTailBytes) > static_cast<uint32>(sizeof(s.payload))) {
s.active = false;
NoteDroppedIncomplete(s.counter);
return false; // overflow guard
}
if (s.pendingTailBytes > 0u) {
(void) MemoryOperationsHelper::Copy(s.payload + tailOffset,
s.pendingTail, s.pendingTailBytes);
}
if ((tailOffset + s.pendingTailBytes) > s.assembledBytes) {
s.assembledBytes = tailOffset + s.pendingTailBytes;
}
s.pendingTailValid = false;
s.pendingTailBytes = 0u;
}
uint32 offset = static_cast<uint32>(fragIdx) * s.chunkSize;
if ((offset + payloadBytes) > static_cast<uint32>(sizeof(s.payload))) { if ((offset + payloadBytes) > static_cast<uint32>(sizeof(s.payload))) {
return false; // overflow guard return false; // overflow guard
} }
@@ -698,9 +822,7 @@ bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
s.assembledBytes = offset + payloadBytes; s.assembledBytes = offset + payloadBytes;
} }
if (byteIdx < 32u) { s.recvMask[byteIdx] |= bitMask;
s.recvMask[byteIdx] |= bitMask;
}
s.receivedFragments++; s.receivedFragments++;
// Check if complete // Check if complete
@@ -716,6 +838,30 @@ bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
// Private: DeliverAssembled // Private: DeliverAssembled
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
bool UDPSClient::AcceptDataCounter(uint32 counter) {
if (!lastDataCounterValid) {
lastDataCounterValid = true;
lastDataCounter = counter;
lastDataGap = 0u;
return true;
}
// The counter is a wrapping uint32, so order it by the signed difference:
// that stays correct across the wrap, where a plain comparison would call
// the first packet after it stale and reject the stream from then on.
int32 delta = static_cast<int32>(counter - lastDataCounter);
if (delta <= 0) {
staleDataPackets++;
return false;
}
lastDataGap = static_cast<uint32>(delta) - 1u;
lastDataCounter = counter;
return true;
}
uint32 UDPSClient::GetLastDataGap() const { return lastDataGap; }
uint32 UDPSClient::GetStaleDataPackets() const { return staleDataPackets; }
void UDPSClient::DeliverAssembled(UDPSReassemblySlot &s) { void UDPSClient::DeliverAssembled(UDPSReassemblySlot &s) {
if (listener == NULL_PTR(UDPSClientListener *)) { if (listener == NULL_PTR(UDPSClientListener *)) {
return; return;
@@ -728,7 +874,7 @@ void UDPSClient::DeliverAssembled(UDPSReassemblySlot &s) {
if (s.type == UDPS_TYPE_CONFIG) { if (s.type == UDPS_TYPE_CONFIG) {
listener->OnUDPSConfig(s.payload, totalSize); listener->OnUDPSConfig(s.payload, totalSize);
} }
else { else if (AcceptDataCounter(s.counter)) {
listener->OnUDPSData(s.payload, totalSize); listener->OnUDPSData(s.payload, totalSize);
} }
} }
@@ -746,10 +892,8 @@ void UDPSClient::GcReassemblySlots() {
continue; continue;
} }
if ((now - reassemblySlots[i].firstSeenTicks) > staleThreshold) { if ((now - reassemblySlots[i].firstSeenTicks) > staleThreshold) {
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: Discarding stale reassembly slot (counter %u).",
reassemblySlots[i].counter);
reassemblySlots[i].active = false; reassemblySlots[i].active = false;
NoteDroppedIncomplete(reassemblySlots[i].counter);
} }
} }
} }
Source/Components/Interfaces/UDPStream/UDPSClient.h
+93 -9
View File
@@ -82,12 +82,29 @@ public:
* update for one source is fragmented into MaxPayloadSize chunks; this is * update for one source is fragmented into MaxPayloadSize chunks; this is
* the ceiling on the reassembled total, so it bounds the largest multi- * the ceiling on the reassembled total, so it bounds the largest multi-
* fragment packet the client can deliver. Sized for large array bursts * fragment packet the client can deliver. Sized for large array bursts
* (e.g. 8x10000 float32 ~= 320 KiB) with headroom; stays well within the * (e.g. 8x10000 float32 ~= 320 KiB) with headroom. */
* 256-fragment span the recvMask[32] tracks at typical chunk sizes. */
static const uint32 UDPS_CLIENT_MAX_PACKET_BYTES = 1048576u; // 1 MiB static const uint32 UDPS_CLIENT_MAX_PACKET_BYTES = 1048576u; // 1 MiB
/** Default silence timeout before reconnect (seconds). */ /** Maximum fragment count accepted for one packet. The received-fragment
static const uint32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 5u; * bitmask must cover this whole span: a fragment index the mask cannot
* represent has no duplicate detection, so a duplicated datagram counts
* twice and the packet is delivered with a fragment still missing. */
static const uint32 UDPS_CLIENT_MAX_FRAGMENTS = 512u;
/** Bytes of received-fragment bitmask (one bit per fragment). */
static const uint32 UDPS_CLIENT_RECV_MASK_BYTES =
UDPS_CLIENT_MAX_FRAGMENTS / 8u;
/** Size of the per-slot buffer that holds a last fragment which arrived
* before the chunk size was known. Fragments larger than this cannot be
* deferred and are dropped (the packet then fails to reassemble). */
static const uint32 UDPS_CLIENT_PENDING_TAIL_BYTES = 8192u;
/** Maximum datagrams drained from the socket per Execute() iteration. */
static const uint32 UDPS_CLIENT_MAX_DATAGRAMS_PER_CYCLE = 256u;
/** Default silence timeout before reconnect (seconds); sub-second values allowed. */
static const float32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 1.0f;
/** Default delay between reconnect attempts (seconds). */ /** Default delay between reconnect attempts (seconds). */
static const uint32 UDPS_CLIENT_DEFAULT_RECONNECT_DELAY_S = 2u; static const uint32 UDPS_CLIENT_DEFAULT_RECONNECT_DELAY_S = 2u;
@@ -116,9 +133,13 @@ public:
* - ServerAddr (char*) Server IPv4 address. Required. * - ServerAddr (char*) Server IPv4 address. Required.
* - Port (uint16) Server UDP port (unicast) or TCP listen port (multicast). Required. * - Port (uint16) Server UDP port (unicast) or TCP listen port (multicast). Required.
* - MulticastGroup (char*) IPv4 multicast address; presence enables multicast mode. * - MulticastGroup (char*) IPv4 multicast address; presence enables multicast mode.
* - Interface (char*) Network interface for multicast join (e.g. "lo"). Required when MulticastGroup is set. * - Interface (char*) Local IPv4 dotted-quad address (e.g. "127.0.0.1") of the interface on
* which to join the multicast group. Optional; omitting it uses the
* default-route interface (INADDR_ANY), which silently receives nothing
* if the server sends on a different interface.
* - DataPort (uint16) UDP multicast data port (defaults to Port+1). * - DataPort (uint16) UDP multicast data port (defaults to Port+1).
* - SilenceTimeout (uint32) Seconds of no data before reconnect. Default 5. * - SilenceTimeout (float32) Seconds of no data before reconnect. Default 1.0.
* Sub-second values allowed; 0 disables the check.
* - ReconnectDelay (uint32) Seconds to wait between reconnect attempts. Default 2. * - ReconnectDelay (uint32) Seconds to wait between reconnect attempts. Default 2.
* - KeepAliveInterval (uint32) Seconds between unicast keepalive ACKs. Default 15. 0 disables. * - KeepAliveInterval (uint32) Seconds between unicast keepalive ACKs. Default 15. 0 disables.
* - MaxPayloadSize (uint32) Max payload bytes per datagram, excluding header. Default 1400. * - MaxPayloadSize (uint32) Max payload bytes per datagram, excluding header. Default 1400.
@@ -151,6 +172,22 @@ public:
*/ */
virtual ErrorManagement::ErrorType Execute(ExecutionInfo &info); virtual ErrorManagement::ErrorType Execute(ExecutionInfo &info);
/**
* @brief DATA packets that went missing immediately before the one being
* delivered, from the gap in the producer's packet counter.
* @details Valid for the duration of the OnUDPSData() callback. A listener
* that reconstructs per-sample timestamps needs this: without it, the time
* elapsed since the previous packet looks like it covers only that
* packet's samples, so the inferred sample period comes out too long and
* the samples are spread past where they belong.
*/
uint32 GetLastDataGap() const;
/**
* @brief DATA packets discarded for arriving after a newer one.
*/
uint32 GetStaleDataPackets() const;
private: private:
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
@@ -161,12 +198,19 @@ private:
uint8 type; ///< UDPS_TYPE_DATA or UDPS_TYPE_CONFIG uint8 type; ///< UDPS_TYPE_DATA or UDPS_TYPE_CONFIG
uint16 totalFragments; ///< Expected fragment count uint16 totalFragments; ///< Expected fragment count
uint16 receivedFragments; ///< How many we have so far uint16 receivedFragments; ///< How many we have so far
uint8 recvMask[32]; ///< Bitmask: bit f set iff fragment f received uint8 recvMask[UDPS_CLIENT_RECV_MASK_BYTES]; ///< Bit f set iff fragment f received
uint8 payload[UDPS_CLIENT_MAX_PACKET_BYTES]; ///< Assembled payload buffer uint8 payload[UDPS_CLIENT_MAX_PACKET_BYTES]; ///< Assembled payload buffer
uint64 firstSeenTicks; ///< For GC (2 s stale detection) uint64 firstSeenTicks; ///< For GC (2 s stale detection)
bool active; ///< Slot in use bool active; ///< Slot in use
uint32 chunkSize; ///< Payload bytes per fragment (from first fragment) uint32 chunkSize; ///< Payload bytes per fragment (any non-last fragment)
uint32 assembledBytes; ///< Exact total payload bytes placed so far uint32 assembledBytes; ///< Exact total payload bytes placed so far
/** Last fragment received before chunkSize was known: its offset is
* not yet computable, so it waits here until a full-size fragment
* reveals the chunk size. Only the last fragment can ever be short,
* hence one deferred fragment per slot is enough. */
uint8 pendingTail[UDPS_CLIENT_PENDING_TAIL_BYTES];
uint32 pendingTailBytes;
bool pendingTailValid;
}; };
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
@@ -191,8 +235,39 @@ private:
bool ReadExactTCP(uint8 *dst, uint32 n); bool ReadExactTCP(uint8 *dst, uint32 n);
/** @return true iff this fragment completed the reassembly (payload delivered). */ /** @return true iff this fragment completed the reassembly (payload delivered). */
bool PlaceFragment(const UDPSPacketHeader *hdr, const uint8 *payload, uint32 payloadBytes); bool PlaceFragment(const UDPSPacketHeader *hdr, const uint8 *payload, uint32 payloadBytes);
/**
* @brief Reserve a reassembly slot for (@p counter, @p type), reclaiming
* one if none is free.
* @return the slot index (always valid).
*/
uint32 AcquireReassemblySlot(uint32 counter, uint8 type);
/**
* @brief Account one packet abandoned with fragments missing, and report
* it at most once per second.
* @details Fragment loss on a busy stream is chronic, not exceptional: an
* unconditional message per drop buries every other log line.
*/
void NoteDroppedIncomplete(uint32 counter);
void GcReassemblySlots(); void GcReassemblySlots();
void DeliverAssembled(UDPSReassemblySlot &slot); void DeliverAssembled(UDPSReassemblySlot &slot);
/**
* @brief Sequence gate for DATA packets, applied just before delivery.
* @details The producer numbers DATA packets consecutively, so the counter
* reveals both how many packets went missing and whether this one is late.
* A late packet must not be delivered: its samples predate what the
* listener has already stored, so they land behind the current write
* position and collide with data that is already there — which is what a
* consumer sees as two signals occupying the same instant. Reassembly
* completes in arrival order, not counter order, so this ordering is not
* guaranteed upstream.
*
* Also records the number of packets missing immediately before this one,
* for GetLastDataGap().
*
* @param counter The candidate packet's UDPS counter.
* @return true if the packet is newer than the last delivered one.
*/
bool AcceptDataCounter(uint32 counter);
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
// Configuration // Configuration
@@ -200,6 +275,7 @@ private:
StreamString serverAddr; StreamString serverAddr;
uint16 serverPort; uint16 serverPort;
StreamString multicastGroup; StreamString multicastGroup;
StreamString multicastInterface;
uint16 dataPort; uint16 dataPort;
bool useMulticast; bool useMulticast;
uint64 silenceTimeoutTicks; uint64 silenceTimeoutTicks;
@@ -230,7 +306,15 @@ private:
// Reassembly // Reassembly
UDPSReassemblySlot reassemblySlots[UDPS_CLIENT_MAX_REASSEMBLY_SLOTS]; UDPSReassemblySlot reassemblySlots[UDPS_CLIENT_MAX_REASSEMBLY_SLOTS];
uint64 lastGcTicks; ///< Ticks at last GC run uint64 lastGcTicks; ///< Ticks at last GC run
uint64 lastDropWarnTicks;///< Ticks at last incomplete-packet report
uint32 droppedSinceWarn; ///< Incomplete packets since that report
// DATA sequencing (see AcceptDataCounter)
uint32 lastDataCounter; ///< Counter of the last delivered DATA packet
bool lastDataCounterValid; ///< False until the first DATA packet
uint32 lastDataGap; ///< Packets missing before the current one
uint32 staleDataPackets; ///< DATA packets discarded as late
// Receive scratch buffer // Receive scratch buffer
uint8 recvBuf[65535u + UDPS_HEADER_SIZE]; uint8 recvBuf[65535u + UDPS_HEADER_SIZE];
Test/Applications/StreamHub/AccumDtGTest.cpp
+162
View File
@@ -0,0 +1,162 @@
/**
* @file AccumDtGTest.cpp
* @brief Tests UDPSEstimateAccumDt, the per-sample period estimator used for
* accumulated scalars that carry no SamplingRate.
*
* The estimator exists because the natural formula — sender-clock gap divided
* by the previous packet's sample count — is only correct while no packet is
* lost. When one is, the gap covers cycles that count never saw and the period
* comes out too large, which spreads the packet's samples past their real end
* and into the range the next packet claims. The loss count comes from the
* producer's packet counter rather than being inferred from the gap itself, so
* these tests pin both sides: the estimate must not move when packets go
* missing, and it must still follow a genuine rate change — a cycle count
* inferred from the estimate's own period would lock onto the old one.
*
* @copyright Copyright 2015 F4E | European Joint Undertaking for ITER and
* the Development of Fusion Energy ('Fusion for Energy').
* Licensed under the EUPL, Version 1.1 or - as soon they will be approved
* by the European Commission - subsequent versions of the EUPL (the "Licence")
* You may not use this work except in compliance with the Licence.
* You may obtain a copy of the Licence at: http://ec.europa.eu/idabc/eupl
*
* @warning Unless required by applicable law or agreed to in writing,
* software distributed under the Licence is distributed on an "AS IS"
* basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
* or implied. See the Licence permissions and limitations under the Licence.
*/
#include <gtest/gtest.h>
#include "UDPSourceSession.h"
using MARTe::float64;
using MARTe::uint32;
using StreamHub::UDPSEstimateAccumDt;
namespace {
/** A producer emitting batches of BATCH cycles at a period of DT seconds. */
const float64 kDt = 1.0e-3;
const uint32 kBatch = 10u;
const float64 kGap = kDt * static_cast<float64>(kBatch);
/** Feeds n clean packets and returns the settled estimate. */
float64 Warmup(uint32 n, float64 &dtEMA, bool &dtValid) {
float64 dt = 0.0;
for (uint32 i = 0u; i < n; i++) {
dt = UDPSEstimateAccumDt(kGap, kBatch, 0u, dtEMA, dtValid);
}
return dt;
}
} // namespace
/* The first packet has nothing to go on but the previous sample count, so it
* must fall back to gap/prevN rather than to some fixed default. */
TEST(AccumDtGTest, BootstrapsFromPreviousSampleCount) {
float64 dtEMA = 0.0;
bool dtValid = false;
const float64 dt = UDPSEstimateAccumDt(kGap, kBatch, 0u, dtEMA, dtValid);
EXPECT_TRUE(dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-12);
}
/* A clean stream must hold the period steady, not drift. */
TEST(AccumDtGTest, SteadyStreamStaysOnPeriod) {
float64 dtEMA = 0.0;
bool dtValid = false;
const float64 dt = Warmup(50u, dtEMA, dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-9);
}
/* The regression this whole estimator is for: one packet is lost, so the gap
* doubles while prevN does not. Dividing by prevN would report 2x the true
* period — enough to walk a 10-sample batch a full batch past its own end. */
TEST(AccumDtGTest, LostPacketDoesNotInflatePeriod) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
const float64 dt = UDPSEstimateAccumDt(2.0 * kGap, kBatch, 1u, dtEMA, dtValid);
/* What the naive formula would have produced. */
const float64 naive = (2.0 * kGap) / static_cast<float64>(kBatch);
EXPECT_NEAR(2.0 * kDt, naive, 1.0e-12);
EXPECT_NEAR(kDt, dt, 1.0e-6);
}
/* Several consecutive losses are the same situation, just wider. */
TEST(AccumDtGTest, MultiplePacketLossDoesNotInflatePeriod) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
for (uint32 missing = 1u; missing <= 5u; missing++) {
const float64 span = static_cast<float64>(missing + 1u) * kGap;
const float64 dt = UDPSEstimateAccumDt(span, kBatch, missing, dtEMA,
dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-6) << "after " << missing << " lost packet(s)";
}
}
/* Loss must not leave the estimator poisoned for the packets that follow. */
TEST(AccumDtGTest, RecoversToCleanStreamAfterLoss) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
(void) UDPSEstimateAccumDt(3.0 * kGap, kBatch, 2u, dtEMA, dtValid);
const float64 dt = Warmup(20u, dtEMA, dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-6);
}
/* A real, sustained rate change must still be followed — the estimator is a
* smoother, not a latch. Half the period is exactly on the rejection boundary,
* so use a change that lands inside the accepted band. */
TEST(AccumDtGTest, FollowsSustainedRateChange) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
const float64 newDt = kDt * 0.75;
const float64 newGap = newDt * static_cast<float64>(kBatch);
float64 dt = 0.0;
for (uint32 i = 0u; i < 400u; i++) {
dt = UDPSEstimateAccumDt(newGap, kBatch, 0u, dtEMA, dtValid);
}
EXPECT_NEAR(newDt, dt, 1.0e-6);
}
/* A batch that carries fewer cycles than usual (a time-triggered flush) is not
* loss: the gap shrinks with it, so the period must not shrink too. */
TEST(AccumDtGTest, ShortBatchDoesNotDeflatePeriod) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
const uint32 shortBatch = 3u;
const float64 dt = UDPSEstimateAccumDt(
kDt * static_cast<float64>(shortBatch), shortBatch, 0u, dtEMA, dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-6);
}
/* A gap shorter than one period cannot mean zero cycles; the divisor is
* clamped so the estimate can never be driven to infinity. */
TEST(AccumDtGTest, SubPeriodGapDoesNotExplode) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
const float64 dt = UDPSEstimateAccumDt(kDt * 1.0e-3, 1u, 0u, dtEMA, dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-6);
}
Test/Applications/StreamHub/ConfigHrtFreqGTest.cpp
+151
View File
@@ -0,0 +1,151 @@
/**
* @file ConfigHrtFreqGTest.cpp
* @brief Tests UDPSConfigHrtFrequency, which picks the tick rate used to turn
* a producer's DATA timestamps into seconds.
*
* DATA packets carry the raw value of the producer's high-resolution counter.
* Until the rate was published in CONFIG the hub divided by its own timer's
* frequency, which is only right while the producer runs on the same host —
* on x86 that number is the TSC frequency and differs from model to model, so
* off-box every accumulated batch was laid out over the wrong span of time.
*
* The field is a trailer, so these tests pin both directions: a payload that
* carries it must be believed, and one that stops early — an older producer —
* must still decode against the local fallback rather than against zero.
*
* @copyright Copyright 2015 F4E | European Joint Undertaking for ITER and
* the Development of Fusion Energy ('Fusion for Energy').
* Licensed under the EUPL, Version 1.1 or - as soon they will be approved
* by the European Commission - subsequent versions of the EUPL (the "Licence")
* You may not use this work except in compliance with the Licence.
* You may obtain a copy of the Licence at: http://ec.europa.eu/idabc/eupl
*
* @warning Unless required by applicable law or agreed to in writing,
* software distributed under the Licence is distributed on an "AS IS"
* basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
* or implied. See the Licence permissions and limitations under the Licence.
*/
#include <gtest/gtest.h>
#include <string.h>
#include "UDPSourceSession.h"
using MARTe::uint8;
using MARTe::uint32;
using MARTe::uint64;
using MARTe::float64;
using MARTe::UDPS_SIGNAL_DESC_SIZE;
using StreamHub::UDPSConfigHrtFrequency;
namespace {
/** This hub's own timer rate, i.e. what the code must fall back to. */
const float64 kLocalFreq = 1.0e9;
/** A plausible producer rate that is deliberately not kLocalFreq. */
const uint64 kWireFreq = 2400000000ULL;
const uint32 kNumSigs = 2u;
/** Offset of the CONFIG trailer that follows the publish-mode byte. */
uint32 TrailerOffset(uint32 numSigs) {
return 4u + (numSigs * UDPS_SIGNAL_DESC_SIZE) + 1u;
}
/**
* Builds a CONFIG payload for kNumSigs signals.
* @param withFreq Append the 8-byte HRT frequency trailer.
* @param freq Value to append when @p withFreq.
* @param[out] size Bytes written.
*/
const uint8 *BuildConfig(bool withFreq, uint64 freq, uint32 &size) {
static uint8 buf[4u + (kNumSigs * UDPS_SIGNAL_DESC_SIZE) + 1u + 8u];
(void) memset(buf, 0, sizeof(buf));
(void) memcpy(buf, &kNumSigs, 4u);
size = TrailerOffset(kNumSigs);
if (withFreq) {
(void) memcpy(buf + size, &freq, 8u);
size += 8u;
}
return buf;
}
} // namespace
/* The whole point of the field: a producer that publishes its rate is believed
* even when the hub's own timer runs at a different one. */
TEST(ConfigHrtFreqGTest, AdoptsThePublishedRate) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(true, kWireFreq, size);
EXPECT_DOUBLE_EQ(static_cast<float64>(kWireFreq),
UDPSConfigHrtFrequency(cfg, size, kNumSigs, kLocalFreq));
}
/* A producer older than the field stops after the publish-mode byte. Reading
* past it would take whatever follows in the receive buffer as a frequency. */
TEST(ConfigHrtFreqGTest, FallsBackWhenTheTrailerIsAbsent) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(false, 0u, size);
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(cfg, size, kNumSigs, kLocalFreq));
}
/* A trailer cut short mid-field is not a frequency either; taking the bytes
* that are there would assemble one out of whatever the rest of the buffer
* holds. */
TEST(ConfigHrtFreqGTest, FallsBackOnATruncatedTrailer) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(true, kWireFreq, size);
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(cfg, size - 1u, kNumSigs,
kLocalFreq));
}
/* Zero is the protocol's "I do not know my own rate". Dividing by it yields
* infinities that propagate into every timestamp. */
TEST(ConfigHrtFreqGTest, FallsBackOnTheUnknownSentinel) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(true, MARTe::UDPS_HRT_FREQUENCY_UNKNOWN,
size);
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(cfg, size, kNumSigs, kLocalFreq));
}
/* No high-resolution timer ticks slower than 1 kHz, so a value that low means
* the payload was misread. Adopting it would stretch a millisecond batch
* across whole seconds. */
TEST(ConfigHrtFreqGTest, FallsBackOnAnImplausiblyLowRate) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(true, 999u, size);
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(cfg, size, kNumSigs, kLocalFreq));
}
/* The trailer sits after the descriptors, so its offset moves with the signal
* count; a fixed offset would read descriptor bytes on any other config. */
TEST(ConfigHrtFreqGTest, LocatesTheTrailerAfterTheDescriptors) {
const uint32 numSigs = 7u;
const uint32 size = TrailerOffset(numSigs) + 8u;
uint8 buf[4u + (7u * UDPS_SIGNAL_DESC_SIZE) + 1u + 8u];
/* Fill the descriptor area with a byte pattern that would decode as a
* plausible frequency if the offset were wrong. */
(void) memset(buf, 0x11, sizeof(buf));
(void) memcpy(buf, &numSigs, 4u);
(void) memcpy(buf + TrailerOffset(numSigs), &kWireFreq, 8u);
EXPECT_DOUBLE_EQ(static_cast<float64>(kWireFreq),
UDPSConfigHrtFrequency(buf, size, numSigs, kLocalFreq));
}
/* A null payload must not be dereferenced: CONFIG arrives from the network. */
TEST(ConfigHrtFreqGTest, FallsBackOnANullPayload) {
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(NULL_PTR(const uint8 *), 1024u,
kNumSigs, kLocalFreq));
}
Test/Applications/StreamHub/Makefile.inc
+1 -1
View File
@@ -22,7 +22,7 @@
# #
############################################################# #############################################################
OBJSX = TriggerEngineSrc.x BinaryRecorderSrc.x SignalRingBufferGTest.x TriggerEngineGTest.x LTTBGTest.x BinaryRecorderGTest.x BoundsCheckTest.x WSServerBufferTest.x OBJSX = TriggerEngineSrc.x BinaryRecorderSrc.x SignalRingBufferGTest.x TriggerEngineGTest.x LTTBGTest.x BinaryRecorderGTest.x BoundsCheckTest.x WSServerBufferTest.x AccumDtGTest.x ConfigHrtFreqGTest.x
PACKAGE=Applications PACKAGE=Applications
ROOT_DIR=../../.. ROOT_DIR=../../..
Test/Applications/StreamHub/TriggerEngineGTest.cpp
+178 -1
View File
@@ -121,7 +121,7 @@ TEST(TriggerEngineGTest, TestConfigClamping) {
TriggerEngine eng; TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.0, 100.0, 150.0)); eng.SetConfig(MakeConfig(kEdgeRising, 0.0, 100.0, 150.0));
TriggerConfig cfg = eng.GetConfig(); TriggerConfig cfg = eng.GetConfig();
EXPECT_DOUBLE_EQ(10.0, cfg.windowSec); EXPECT_DOUBLE_EQ(60.0, cfg.windowSec);
EXPECT_DOUBLE_EQ(100.0, cfg.prePercent); EXPECT_DOUBLE_EQ(100.0, cfg.prePercent);
eng.SetConfig(MakeConfig(kEdgeRising, 0.0, 1.0e-6, -5.0)); eng.SetConfig(MakeConfig(kEdgeRising, 0.0, 1.0e-6, -5.0));
@@ -196,6 +196,183 @@ TEST(TriggerEngineGTest, TestRearmResetsEdgeDetection) {
EXPECT_DOUBLE_EQ(4.0, tt); EXPECT_DOUBLE_EQ(4.0, tt);
} }
/* An edge that arrives while a capture is still being collected, or while it is
* being handed out, used to be dropped on the floor: CheckSample returned early
* in every state but ARMED, and the automatic rearm then waited for a FRESH
* edge. The engine is therefore deaf from its own trigger point until the
* capture has been harvested — a post-window — and then for the holdoff on top.
*
* On a sparse pulse train that rounds the capture spacing up to a whole pulse
* period: at the default 1 s window the blind stretch is 1 s, so a 1 Hz train
* was caught at 0.5 Hz and a wider window lost whole multiples. Remembering the
* edge costs nothing, because the capture is built from the edge's own
* timestamp out of a ring that still holds everything around it. */
TEST(TriggerEngineGTest, TestEdgeDuringCaptureFiresOnRearm) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0)); /* post = 0.8 */
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
/* A second pulse, clear of the capture in flight (1.1 + 0.8 = 1.9). */
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0);
eng.MarkTriggered();
eng.Rearm();
EXPECT_EQ(kTrigCollecting, eng.GetState());
float64 tt, pre, post;
ASSERT_TRUE(eng.GetFiredWindow(tt, pre, post));
EXPECT_DOUBLE_EQ(2.5, tt); /* the remembered edge, not the rearm instant */
}
/* The remembered edge must not be one the capture in flight already covers, nor
* one inside the holdoff — that guard exists to stop the ringing of a single
* event re-triggering on itself, and it is measured from the trigger point, so
* the two overlap rather than add. */
TEST(TriggerEngineGTest, TestEdgeInsideOwnCaptureIsNotRemembered) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0)); /* post = 0.8 */
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
/* Inside 1.1 + max(0.8, 0.2 holdoff) = 1.9: the capture owns this stretch. */
eng.CheckSample(1.4, 0.0);
eng.CheckSample(1.5, 1.0);
eng.MarkTriggered();
eng.Rearm();
EXPECT_EQ(kTrigArmed, eng.GetState());
}
/* A holdoff longer than the post-window is what decides the guard interval. */
TEST(TriggerEngineGTest, TestHoldoffOutlastingPostWindowGovernsRearm) {
TriggerEngine eng;
TriggerConfig cfg = MakeConfig(kEdgeRising, 0.5, 1.0, 80.0); /* post = 0.2 */
cfg.holdoffSec = 2.0;
eng.SetConfig(cfg);
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
/* Past the post-window but inside the holdoff (1.1 + 2.0 = 3.1): ignored. */
eng.CheckSample(1.9, 0.0);
eng.CheckSample(2.0, 1.0);
/* Clear of it: remembered. */
eng.CheckSample(3.4, 0.0);
eng.CheckSample(3.5, 1.0);
eng.MarkTriggered();
eng.Rearm();
ASSERT_EQ(kTrigCollecting, eng.GetState());
float64 tt, pre, post;
ASSERT_TRUE(eng.GetFiredWindow(tt, pre, post));
EXPECT_DOUBLE_EQ(3.5, tt);
}
/* Only the first qualifying edge is worth keeping; a later one would deliver
* the same capture a pulse further on and skip the one in between. */
TEST(TriggerEngineGTest, TestFirstQualifyingEdgeWins) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0); /* first past 1.9 */
eng.CheckSample(3.4, 0.0);
eng.CheckSample(3.5, 1.0); /* later, must not displace it */
eng.MarkTriggered();
eng.Rearm();
float64 tt, pre, post;
ASSERT_TRUE(eng.GetFiredWindow(tt, pre, post));
EXPECT_DOUBLE_EQ(2.5, tt);
}
/* Arm() is the user's own arm: it asks for the next event, not for one that has
* already been and gone, so it drops anything remembered. */
TEST(TriggerEngineGTest, TestUserArmDiscardsRememberedEdge) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0);
eng.MarkTriggered();
eng.Arm();
EXPECT_EQ(kTrigArmed, eng.GetState());
}
/* Reconfiguring drops it too: the edge would be latched against a window it was
* never judged against. */
TEST(TriggerEngineGTest, TestSetConfigDiscardsRememberedEdge) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0);
eng.MarkTriggered();
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 2.0, 20.0));
eng.Rearm();
EXPECT_EQ(kTrigArmed, eng.GetState());
}
/* The comparator keeps running through the dead time, so the first sample after
* an automatic rearm is measured against its real predecessor rather than being
* spent seeding one. A rearm landing mid-pulse would otherwise miss that
* pulse's edge as well as the ones it slept through. */
TEST(TriggerEngineGTest, TestRearmKeepsTrackingTheLevel) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
/* Falls back low during the capture: no rising edge, nothing remembered,
* but the level is now known to be 0. */
eng.CheckSample(2.0, 0.0);
eng.MarkTriggered();
eng.Rearm();
ASSERT_EQ(kTrigArmed, eng.GetState());
eng.CheckSample(2.1, 1.0); /* 0.0 → 1.0 across 0.5, on the very first sample */
EXPECT_EQ(kTrigCollecting, eng.GetState());
float64 tt, pre, post;
ASSERT_TRUE(eng.GetFiredWindow(tt, pre, post));
EXPECT_DOUBLE_EQ(2.1, tt);
}
/* Idle is genuinely deaf: nothing is tracked and nothing is remembered, so a
* disarmed hub cannot fire the moment it is armed again. */
TEST(TriggerEngineGTest, TestDisarmDiscardsRememberedEdge) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0);
eng.MarkTriggered();
eng.Disarm();
eng.Rearm();
EXPECT_EQ(kTrigArmed, eng.GetState());
}
TEST(TriggerEngineGTest, TestStoppedFlag) { TEST(TriggerEngineGTest, TestStoppedFlag) {
TriggerEngine eng; TriggerEngine eng;
EXPECT_FALSE(eng.GetStopped()); EXPECT_FALSE(eng.GetStopped());
Test/Components/DataSources/UDPStreamer/UDPStreamerGTest.cpp
+5
View File
@@ -225,3 +225,8 @@ TEST(UDPStreamerGTest, TestExecute_MulticastConnectDataDisconnect) {
UDPStreamerTest test; UDPStreamerTest test;
ASSERT_TRUE(test.TestExecute_MulticastConnectDataDisconnect()); ASSERT_TRUE(test.TestExecute_MulticastConnectDataDisconnect());
} }
TEST(UDPStreamerGTest, TestAccumulate_EveryPublishedCycleReachesTheWire) {
UDPStreamerTest test;
ASSERT_TRUE(test.TestAccumulate_EveryPublishedCycleReachesTheWire());
}
Test/Components/DataSources/UDPStreamer/UDPStreamerTest.cpp
+327 -1
View File
@@ -36,11 +36,13 @@
#include "ConfigurationDatabase.h" #include "ConfigurationDatabase.h"
#include "GAM.h" #include "GAM.h"
#include "GAMScheduler.h" #include "GAMScheduler.h"
#include "HighResolutionTimer.h"
#include "MemoryOperationsHelper.h" #include "MemoryOperationsHelper.h"
#include "ObjectRegistryDatabase.h" #include "ObjectRegistryDatabase.h"
#include "RealTimeApplication.h" #include "RealTimeApplication.h"
#include "Sleep.h" #include "Sleep.h"
#include "StandardParser.h" #include "StandardParser.h"
#include "UDPSClient.h"
#include "UDPStreamer.h" #include "UDPStreamer.h"
#include "UDPStreamerTest.h" #include "UDPStreamerTest.h"
@@ -906,6 +908,23 @@ bool UDPStreamerTest::TestExecute_ConnectDataDisconnect() {
reinterpret_cast<const UDPSPacketHeader *>(recvBuf); reinterpret_cast<const UDPSPacketHeader *>(recvBuf);
ok &= (hdr->magic == UDPS_MAGIC); ok &= (hdr->magic == UDPS_MAGIC);
ok &= (hdr->type == UDPS_TYPE_CONFIG); ok &= (hdr->type == UDPS_TYPE_CONFIG);
/* The CONFIG trailer must carry this host's HRT tick rate: DATA
* packets timestamp with the raw counter, so a receiver on another
* machine has nothing to convert it with otherwise. */
const uint8 *payload = recvBuf + UDPS_HEADER_SIZE;
uint32 numSigs = 0u;
if (ok && (hdr->payloadBytes >= 4u)) {
(void) memcpy(&numSigs, payload, 4u);
}
const uint32 freqOff =
4u + (numSigs * UDPS_SIGNAL_DESC_SIZE) + 1u;
ok &= (hdr->payloadBytes >= (freqOff + 8u));
if (ok) {
uint64 wireFreq = 0u;
(void) memcpy(&wireFreq, payload + freqOff, 8u);
ok &= (wireFreq == HighResolutionTimer::Frequency());
}
} }
} }
@@ -1559,6 +1578,7 @@ bool UDPStreamerTest::TestInitialise_MulticastMode_Valid() {
ConfigurationDatabase cdb; ConfigurationDatabase cdb;
cdb.Write("Port", 44710u); cdb.Write("Port", 44710u);
cdb.Write("MulticastGroup", "239.0.0.1"); cdb.Write("MulticastGroup", "239.0.0.1");
cdb.Write("Interface", "127.0.0.1");
cdb.Write("DataPort", 44711u); cdb.Write("DataPort", 44711u);
cdb.CreateRelative("Signals"); cdb.CreateRelative("Signals");
cdb.MoveToRoot(); cdb.MoveToRoot();
@@ -1574,6 +1594,7 @@ bool UDPStreamerTest::TestInitialise_MulticastMode_DefaultDataPort() {
ConfigurationDatabase cdb; ConfigurationDatabase cdb;
cdb.Write("Port", 44712u); cdb.Write("Port", 44712u);
cdb.Write("MulticastGroup", "239.0.0.1"); cdb.Write("MulticastGroup", "239.0.0.1");
cdb.Write("Interface", "127.0.0.1");
/* DataPort intentionally omitted: should default to 44713 */ /* DataPort intentionally omitted: should default to 44713 */
cdb.CreateRelative("Signals"); cdb.CreateRelative("Signals");
cdb.MoveToRoot(); cdb.MoveToRoot();
@@ -1588,6 +1609,9 @@ bool UDPStreamerTest::TestInitialise_MulticastMode_InvalidDataPort() {
ConfigurationDatabase cdb; ConfigurationDatabase cdb;
cdb.Write("Port", 44714u); cdb.Write("Port", 44714u);
cdb.Write("MulticastGroup", "239.0.0.1"); cdb.Write("MulticastGroup", "239.0.0.1");
/* Interface is mandatory for multicast; supply it so the rejection below
* is provably caused by DataPort == Port and not by a missing Interface. */
cdb.Write("Interface", "127.0.0.1");
cdb.Write("DataPort", 44714u); /* same as Port — must be rejected */ cdb.Write("DataPort", 44714u); /* same as Port — must be rejected */
cdb.CreateRelative("Signals"); cdb.CreateRelative("Signals");
cdb.MoveToRoot(); cdb.MoveToRoot();
@@ -1618,6 +1642,7 @@ bool UDPStreamerTest::TestPrepareNextState_Multicast() {
" Class = UDPStreamer\n" " Class = UDPStreamer\n"
" Port = 44716\n" " Port = 44716\n"
" MulticastGroup = \"239.0.0.1\"\n" " MulticastGroup = \"239.0.0.1\"\n"
" Interface = \"127.0.0.1\"\n"
" DataPort = 44717\n" " DataPort = 44717\n"
" MaxPayloadSize = 1400\n" " MaxPayloadSize = 1400\n"
" Signals = {\n" " Signals = {\n"
@@ -1695,6 +1720,7 @@ bool UDPStreamerTest::TestExecute_MulticastConnectDataDisconnect() {
" Class = UDPStreamer\n" " Class = UDPStreamer\n"
" Port = 44720\n" " Port = 44720\n"
" MulticastGroup = \"239.0.0.1\"\n" " MulticastGroup = \"239.0.0.1\"\n"
" Interface = \"127.0.0.1\"\n"
" DataPort = 44721\n" " DataPort = 44721\n"
" MaxPayloadSize = 1400\n" " MaxPayloadSize = 1400\n"
" Signals = {\n" " Signals = {\n"
@@ -1786,7 +1812,12 @@ bool UDPStreamerTest::TestExecute_MulticastConnectDataDisconnect() {
ok = mcastReader.Listen(44721u); ok = mcastReader.Listen(44721u);
} }
if (ok) { if (ok) {
ok = mcastReader.Join("239.0.0.1"); /* Join on the same interface the streamer sends from (Interface =
* 127.0.0.1 sets IP_MULTICAST_IF on the server's data socket). The
* single-argument Join() would pass INADDR_ANY, letting the kernel
* pick the default-route interface, and the datagram would never
* reach this socket. */
ok = mcastReader.Join("239.0.0.1", "127.0.0.1");
} }
/* Step 4: Trigger Synchronise() to generate a DATA packet */ /* Step 4: Trigger Synchronise() to generate a DATA packet */
@@ -1833,3 +1864,298 @@ bool UDPStreamerTest::TestExecute_MulticastConnectDataDisconnect() {
ObjectRegistryDatabase::Instance()->Purge(); ObjectRegistryDatabase::Instance()->Purge();
return ok; return ok;
} }
/*---------------------------------------------------------------------------*/
/* Accumulate publication continuity */
/*---------------------------------------------------------------------------*/
/* Four float64 scalars, no quantisation: 32 wire bytes per RT cycle.
* With MaxPayloadSize = 60 the accumulate header (8 B HRT + 4 B count) leaves
* room for exactly one cycle, so the size condition flushes on every single
* Synchronise() — the maximum number of hand-offs to the sender thread, each
* one a chance for a promoted batch to be skipped. */
#define ACC_FUNCTIONS_BLOCK \
" +Functions = {\n" \
" Class = ReferenceContainer\n" \
" +Writer = {\n" \
" Class = UDPStreamerTestOutputGAM\n" \
" OutputSignals = {\n" \
" A = {\n" \
" DataSource = Streamer\n" \
" Type = float64\n" \
" }\n" \
" B = {\n" \
" DataSource = Streamer\n" \
" Type = float64\n" \
" }\n" \
" C = {\n" \
" DataSource = Streamer\n" \
" Type = float64\n" \
" }\n" \
" D = {\n" \
" DataSource = Streamer\n" \
" Type = float64\n" \
" }\n" \
" }\n" \
" }\n" \
" }\n"
static const MARTe::char8 *const ACC_CFG_CONTINUITY =
"+Test = {\n"
" Class = RealTimeApplication\n"
ACC_FUNCTIONS_BLOCK
" +Data = {\n"
" Class = ReferenceContainer\n"
" +Streamer = {\n"
" Class = UDPStreamer\n"
" Port = 44680\n"
" MaxPayloadSize = 60\n"
" PublishingMode = Accumulate\n"
" MinRefreshRate = 1000.0\n"
" Signals = {\n"
" A = {\n"
" Type = float64\n"
" }\n"
" B = {\n"
" Type = float64\n"
" }\n"
" C = {\n"
" Type = float64\n"
" }\n"
" D = {\n"
" Type = float64\n"
" }\n"
" }\n"
" }\n"
HF_TAIL_BLOCK;
namespace {
/** Cycles driven by TestAccumulate_EveryPublishedCycleReachesTheWire. */
static const MARTe::uint32 ACC_CONTINUITY_CYCLES = 3000u;
/**
* @brief Records which RT cycles reached the wire, and how often.
*
* The test stamps signal A with the cycle index before every Synchronise(),
* and the config is sized so each Accumulate batch carries exactly one cycle.
* The payload is [8 B HRT][4 B numSamples][A][B][C][D], so A of the single
* slot sits at offset 12 and identifies the cycle unambiguously.
*
* Counting distinct cycles (rather than summing numSamples) is what makes this
* able to tell a lost publication from a re-sent one: a sender that never
* consumes its ready buffer emits the right *number* of packets while
* repeating a stale batch, which shows up here as duplicates plus missing
* cycles instead of a clean tally.
*/
class AccumRampRecorder: public MARTe::UDPSClientListener {
public:
AccumRampRecorder() :
packets(0u), duplicates(0u), malformed(0u) {
mux.Create();
for (MARTe::uint32 i = 0u; i < ACC_CONTINUITY_CYCLES; i++) {
seen[i] = false;
}
}
virtual void OnUDPSData(const MARTe::uint8 *payload, MARTe::uint32 payloadSize) {
MARTe::uint32 n = 0u;
MARTe::float64 v = 0.0;
if (payloadSize >= 20u) {
(void) MARTe::MemoryOperationsHelper::Copy(&n, &payload[8], 4u);
(void) MARTe::MemoryOperationsHelper::Copy(&v, &payload[12], 8u);
}
(void) mux.FastLock();
packets++;
if ((payloadSize < 20u) || (n != 1u)) {
malformed++;
}
else {
MARTe::uint32 idx = static_cast<MARTe::uint32>(v);
if ((static_cast<MARTe::float64>(idx) != v) || (idx >= ACC_CONTINUITY_CYCLES)) {
malformed++;
}
else if (seen[idx]) {
duplicates++;
}
else {
seen[idx] = true;
}
}
mux.FastUnLock();
}
MARTe::uint32 DistinctCycles() {
(void) mux.FastLock();
MARTe::uint32 n = 0u;
for (MARTe::uint32 i = 0u; i < ACC_CONTINUITY_CYCLES; i++) {
if (seen[i]) {
n++;
}
}
mux.FastUnLock();
return n;
}
MARTe::uint32 Packets() {
(void) mux.FastLock();
MARTe::uint32 n = packets;
mux.FastUnLock();
return n;
}
MARTe::uint32 Duplicates() {
(void) mux.FastLock();
MARTe::uint32 n = duplicates;
mux.FastUnLock();
return n;
}
MARTe::uint32 Malformed() {
(void) mux.FastLock();
MARTe::uint32 n = malformed;
mux.FastUnLock();
return n;
}
private:
MARTe::FastPollingMutexSem mux;
bool seen[ACC_CONTINUITY_CYCLES];
MARTe::uint32 packets;
MARTe::uint32 duplicates;
MARTe::uint32 malformed;
};
} // namespace
bool UDPStreamerTest::TestAccumulate_EveryPublishedCycleReachesTheWire() {
using namespace MARTe;
/* One-cycle batches every 200 us: ~5000 small packets/s, which the sender
* thread handles comfortably. The period has to be this short because a
* wake-up can only be swallowed while the sender is mid-send; at 1 ms the
* sender is always back in its wait before the next Synchronise() and the
* defect never fires at all. */
const uint32 CYCLES = ACC_CONTINUITY_CYCLES;
static const float64 CYCLE_SEC = 200e-6;
/* Tolerance, as a fraction of CYCLES, for cycles that never reach the wire.
* It is not zero: this is an ordinary userspace thread on a general-purpose
* kernel, so it can occasionally be descheduled past a 200 us slot, and the
* last batch may still be in the accumulation buffer when the loop ends.
* It is small because the defect this guards against is not marginal — a
* sender that decides what to send from the semaphore edge fails to consume
* essentially every batch (~100% here), so a 1% ceiling separates the two
* regimes with three orders of magnitude to spare. */
const uint32 MAX_LOST = CYCLES / 100u;
ReferenceT<RealTimeApplication> app = LoadApplication(ACC_CFG_CONTINUITY);
bool ok = app.IsValid();
if (ok) {
ok = (app->PrepareNextState("State1") == ErrorManagement::NoError);
}
Sleep::MSec(50u);
AccumRampRecorder counter;
UDPSClient client;
ReferenceT<UDPStreamer> ds;
if (ok) {
ConfigurationDatabase clientCfg;
ok = clientCfg.Write("ServerAddr", "127.0.0.1");
ok = ok && clientCfg.Write("Port", 44680u);
ok = ok && clientCfg.Write("SilenceTimeout", 0.0f);
ok = ok && clientCfg.Write("KeepAliveInterval", 0u);
client.SetListener(&counter);
ok = ok && client.Initialise(clientCfg);
ok = ok && client.Start();
}
/* Wait for the CONNECT to register on the streamer side. */
if (ok) {
ds = ObjectRegistryDatabase::Instance()->Find("Test.Data.Streamer");
ok = ds.IsValid();
}
if (ok) {
uint32 waited = 0u;
while ((waited < 3000u) && !ds->IsClientConnected()) {
Sleep::MSec(20u);
waited += 20u;
}
ok = ds->IsClientConnected();
}
/* Signal A carries the cycle index, so every packet identifies exactly
* which RT cycle produced it. Synchronise() snapshots the DataSource
* memory, so writing straight into it is equivalent to a GAM having
* produced the value. */
float64 *sigA = NULL_PTR(float64 *);
if (ok) {
void *addr = NULL_PTR(void *);
ok = ds->GetSignalMemoryBuffer(0u, 0u, addr);
sigA = reinterpret_cast<float64 *>(addr);
ok = ok && (sigA != NULL_PTR(float64 *));
}
/* Drive the RT cycles. */
if (ok) {
for (uint32 i = 0u; (i < CYCLES) && ok; i++) {
*sigA = static_cast<float64>(i);
ok = ds->Synchronise();
Sleep::Sec(CYCLE_SEC);
}
}
/* Let the last packets drain. */
Sleep::MSec(300u);
uint32 distinct = counter.DistinctCycles();
uint32 packets = counter.Packets();
uint32 duplicates = counter.Duplicates();
uint32 malformed = counter.Malformed();
uint32 dropped = (ds.IsValid()) ? ds->GetDroppedPublications() : 0u;
if (ok) {
ok = (malformed == 0u);
if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::FatalError,
"%u of %u DATA packets did not carry exactly one "
"decodable cycle index.", malformed, packets);
}
}
if (ok) {
/* A cycle that never arrives is a hole in the consumer's time series. */
ok = (distinct + MAX_LOST) >= CYCLES;
if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::FatalError,
"Accumulate lost cycles: %u of %u reached the wire "
"in %u packets (%u duplicates, %u publications "
"overwritten before being sent).",
distinct, CYCLES, packets, duplicates, dropped);
}
}
if (ok) {
/* A cycle that arrives twice means the sender re-sent a ready buffer it
* had already transmitted, which lands the same samples on the receiver
* under two different time bases. */
ok = (duplicates == 0u);
if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::FatalError,
"%u of %u DATA packets repeated a cycle already sent.",
duplicates, packets);
}
}
if (ok) {
/* Same ceiling from the producer's side: it sees the overwrite directly
* and does not depend on the packet reaching the loopback socket. */
ok = (dropped <= MAX_LOST);
if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::FatalError,
"%u of %u publications were overwritten before the "
"sender thread took them.", dropped, CYCLES);
}
}
(void) client.Stop();
ObjectRegistryDatabase::Instance()->Purge();
return ok;
}
Test/Components/DataSources/UDPStreamer/UDPStreamerTest.h
+10
View File
@@ -234,6 +234,16 @@ public:
* @brief Tests full TCP CONNECT → CONFIG → DATA via multicast → DISCONNECT on loopback. * @brief Tests full TCP CONNECT → CONFIG → DATA via multicast → DISCONNECT on loopback.
*/ */
bool TestExecute_MulticastConnectDataDisconnect(); bool TestExecute_MulticastConnectDataDisconnect();
/**
* @brief Tests that Accumulate publishes every RT cycle it batches.
* @details Drives 600 cycles at a rate the sender thread trivially keeps up
* with, and sums the numSamples field of every DATA packet that arrives.
* A batch promoted to the ready buffer but never sent — because the wake-up
* announcing it was swallowed — shows up here as missing cycles, which a
* consumer sees as a hole in the time series.
*/
bool TestAccumulate_EveryPublishedCycleReachesTheWire();
}; };
#endif /* UDPSTREAMERTEST_H_ */ #endif /* UDPSTREAMERTEST_H_ */
Test/Components/DataSources/UDPStreamerClient/UDPStreamerClientGTest.cpp
+10
View File
@@ -51,6 +51,11 @@ TEST(UDPStreamerClientGTest, TestInitialise_DefaultPort) {
ASSERT_TRUE(test.TestInitialise_DefaultPort()); ASSERT_TRUE(test.TestInitialise_DefaultPort());
} }
TEST(UDPStreamerClientGTest, TestInitialise_SilenceTimeoutFloat) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestInitialise_SilenceTimeoutFloat());
}
TEST(UDPStreamerClientGTest, TestInitialise_MulticastMode_Valid) { TEST(UDPStreamerClientGTest, TestInitialise_MulticastMode_Valid) {
UDPStreamerClientTest test; UDPStreamerClientTest test;
ASSERT_TRUE(test.TestInitialise_MulticastMode_Valid()); ASSERT_TRUE(test.TestInitialise_MulticastMode_Valid());
@@ -140,3 +145,8 @@ TEST(UDPStreamerClientGTest, TestExecute_ConnectConfigDataEndToEnd) {
UDPStreamerClientTest test; UDPStreamerClientTest test;
ASSERT_TRUE(test.TestExecute_ConnectConfigDataEndToEnd()); ASSERT_TRUE(test.TestExecute_ConnectConfigDataEndToEnd());
} }
TEST(UDPStreamerClientGTest, TestExecute_MulticastReceivesDataOnInterface) {
UDPStreamerClientTest test;
ASSERT_TRUE(test.TestExecute_MulticastReceivesDataOnInterface());
}
Test/Components/DataSources/UDPStreamerClient/UDPStreamerClientTest.cpp
+228
View File
@@ -26,11 +26,14 @@
/* Standard header includes */ /* Standard header includes */
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
#include <string.h> #include <string.h>
#include <netinet/in.h>
#include <sys/socket.h>
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
/* Project header includes */ /* Project header includes */
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
#include "AdvancedErrorManagement.h" #include "AdvancedErrorManagement.h"
#include "BasicTCPSocket.h"
#include "BasicUDPSocket.h" #include "BasicUDPSocket.h"
#include "ConfigurationDatabase.h" #include "ConfigurationDatabase.h"
#include "GAM.h" #include "GAM.h"
@@ -354,6 +357,51 @@ bool UDPStreamerClientTest::TestInitialise_DefaultPort() {
return ok; return ok;
} }
bool UDPStreamerClientTest::TestInitialise_SilenceTimeoutFloat() {
using namespace MARTe;
static const char8 *const cfg =
"+Test = {\n"
" Class = RealTimeApplication\n"
" +Functions = {\n"
" Class = ReferenceContainer\n"
" +Reader = {\n"
" Class = UDPStreamerClientTestGAM\n"
" InputSignals = { Counter = { DataSource = ClientDS Type = uint32 } }\n"
" OutputSignals = { Counter = { DataSource = DDB Type = uint32 } }\n"
" }\n"
" }\n"
" +Data = {\n"
" Class = ReferenceContainer\n"
" DefaultDataSource = DDB\n"
" +DDB = { Class = GAMDataSource }\n"
" +ClientDS = {\n"
" Class = UDPStreamerClient\n"
" ServerAddress = \"127.0.0.1\"\n"
" Port = 44702\n"
" SilenceTimeout = 0.25\n"
" Signals = { Counter = { Type = uint32 } }\n"
" }\n"
" +Timings = { Class = TimingDataSource }\n"
" }\n"
" +States = {\n"
" Class = ReferenceContainer\n"
" +State1 = {\n"
" Class = RealTimeState\n"
" +Threads = {\n"
" Class = ReferenceContainer\n"
" +Thread1 = { Class = RealTimeThread Functions = { Reader } }\n"
" }\n"
" }\n"
" }\n"
" +Scheduler = { Class = GAMScheduler TimingDataSource = Timings }\n"
"}\n";
ReferenceT<RealTimeApplication> app = LoadApplication(cfg);
bool ok = app.IsValid();
ObjectRegistryDatabase::Instance()->Purge();
return ok;
}
bool UDPStreamerClientTest::TestInitialise_MulticastMode_Valid() { bool UDPStreamerClientTest::TestInitialise_MulticastMode_Valid() {
using namespace MARTe; using namespace MARTe;
static const char8 *const cfg = static const char8 *const cfg =
@@ -1446,3 +1494,183 @@ bool UDPStreamerClientTest::TestExecute_ConnectConfigDataEndToEnd() {
ObjectRegistryDatabase::Instance()->Purge(); ObjectRegistryDatabase::Instance()->Purge();
return ok; return ok;
} }
bool UDPStreamerClientTest::TestExecute_MulticastReceivesDataOnInterface() {
using namespace MARTe;
static const uint16 controlPort = 44730u;
static const uint16 dataPort = 44731u;
static const char8 *const mcGroup = "239.0.0.7";
static const char8 *const mcIface = "127.0.0.1";
static const char8 *const cfg =
"+Test = {\n"
" Class = RealTimeApplication\n"
" +Functions = {\n"
" Class = ReferenceContainer\n"
" +Reader = {\n"
" Class = UDPStreamerClientTestGAM\n"
" InputSignals = {\n"
" Counter = { DataSource = ClientDS Type = uint32 }\n"
" }\n"
" OutputSignals = {\n"
" Counter = { DataSource = DDB Type = uint32 }\n"
" }\n"
" }\n"
" }\n"
" +Data = {\n"
" Class = ReferenceContainer\n"
" DefaultDataSource = DDB\n"
" +DDB = { Class = GAMDataSource }\n"
" +ClientDS = {\n"
" Class = UDPStreamerClient\n"
" ServerAddress = \"127.0.0.1\"\n"
" Port = 44730\n"
" MulticastGroup = \"239.0.0.7\"\n"
" DataPort = 44731\n"
" Interface = \"127.0.0.1\"\n"
" MaxPayloadSize = 1400\n"
" Signals = {\n"
" Counter = { Type = uint32 }\n"
" }\n"
" }\n"
" +Timings = { Class = TimingDataSource }\n"
" }\n"
" +States = {\n"
" Class = ReferenceContainer\n"
" +State1 = {\n"
" Class = RealTimeState\n"
" +Threads = {\n"
" Class = ReferenceContainer\n"
" +Thread1 = {\n"
" Class = RealTimeThread\n"
" Functions = { Reader }\n"
" }\n"
" }\n"
" }\n"
" }\n"
" +Scheduler = {\n"
" Class = GAMScheduler\n"
" TimingDataSource = Timings\n"
" }\n"
"}\n";
ReferenceT<RealTimeApplication> app = LoadApplication(cfg);
bool ok = app.IsValid();
/* Open a TCP listener for the control port BEFORE PrepareNextState so we
* never miss the client's CONNECT. */
BasicTCPSocket tcpListener;
if (ok) {
ok = tcpListener.Open() && tcpListener.Listen(controlPort, 5);
}
if (ok) {
ok = (app->PrepareNextState("State1") == ErrorManagement::NoError);
}
/* Open the multicast data socket aimed at the group, with IP_MULTICAST_IF
* set to 127.0.0.1 so the datagram leaves on loopback — exactly what
* UDPSServer does. */
BasicUDPSocket dataSocket;
if (ok) {
ok = dataSocket.Open();
}
if (ok) {
struct in_addr localIf;
localIf.s_addr = inet_addr(mcIface);
int fd = static_cast<int>(dataSocket.GetWriteHandle());
ok = (setsockopt(fd, IPPROTO_IP, IP_MULTICAST_IF,
&localIf, static_cast<socklen_t>(sizeof(localIf))) == 0);
}
if (ok) {
ok = dataSocket.Connect(mcGroup, dataPort);
}
/* Accept the client's TCP CONNECT. */
BasicTCPSocket *clientConn = NULL_PTR(BasicTCPSocket *);
if (ok) {
clientConn = tcpListener.WaitConnection(TimeoutType(2000u));
ok = (clientConn != NULL_PTR(BasicTCPSocket *));
}
/* Read the CONNECT packet from the accepted TCP connection. */
if (ok) {
uint8 recvBuf[64u];
uint32 recvSize = UDPS_HEADER_SIZE;
ok = clientConn->Read(reinterpret_cast<char8 *>(recvBuf), recvSize);
if (ok) {
const UDPSPacketHeader *hdr = reinterpret_cast<const UDPSPacketHeader *>(recvBuf);
ok = (hdr->magic == UDPS_MAGIC) && (hdr->type == UDPS_TYPE_CONNECT);
}
}
/* Send CONFIG: one scalar "Counter" uint32 signal, unquantised. */
if (ok) {
UDPSSignalDescriptor desc;
(void) memset(&desc, 0, sizeof(desc));
(void) strncpy(desc.name, "Counter", UDPS_MAX_SIGNAL_NAME - 1u);
desc.typeCode = UDPS_TYPECODE_UINT32;
desc.numRows = 1u;
desc.numCols = 1u;
uint8 buf[UDPS_HEADER_SIZE + 4u + UDPS_SIGNAL_DESC_SIZE + 1u];
const uint32 configPayloadBytes = 4u + UDPS_SIGNAL_DESC_SIZE + 1u;
UDPSBuildHeader(buf, UDPS_TYPE_CONFIG, 1u, 0u, 1u, configPayloadBytes);
uint32 numSigs = 1u;
(void) memcpy(buf + UDPS_HEADER_SIZE, &numSigs, 4u);
(void) memcpy(buf + UDPS_HEADER_SIZE + 4u, &desc, UDPS_SIGNAL_DESC_SIZE);
buf[UDPS_HEADER_SIZE + 4u + UDPS_SIGNAL_DESC_SIZE] = UDPS_PUBLISH_STRICT;
uint32 sendSize = static_cast<uint32>(sizeof(buf));
ok = clientConn->Write(reinterpret_cast<const char8 *>(buf), sendSize);
}
Sleep::MSec(100u);
ReferenceT<UDPStreamerClient> ds;
if (ok) {
ds = ObjectRegistryDatabase::Instance()->Find("Test.Data.ClientDS");
ok = ds.IsValid();
}
/* Send DATA over UDP multicast. Each attempt uses a fresh packet counter
* so UDPSClient's reassembly layer does not drop retransmissions. */
bool gotValue = false;
for (uint32 attempt = 0u; ok && (!gotValue) && (attempt < 30u); attempt++) {
SynchroniseThreadArgs *syncArgs = StartSynchroniseThread(ds);
uint8 buf[UDPS_HEADER_SIZE + 8u + 4u];
const uint32 dataPayloadBytes = 8u + 4u;
UDPSBuildHeader(buf, UDPS_TYPE_DATA, 2u + attempt, 0u, 1u, dataPayloadBytes);
(void) memset(buf + UDPS_HEADER_SIZE, 0, 8u);
uint32 value = 424242u;
(void) memcpy(buf + UDPS_HEADER_SIZE + 8u, &value, 4u);
uint32 sendSize = static_cast<uint32>(sizeof(buf));
ok = dataSocket.Write(reinterpret_cast<const char8 *>(buf), sendSize);
if (ok && JoinSynchroniseThread(syncArgs)) {
void *sigMem = NULL_PTR(void *);
if (ds->GetSignalMemoryBuffer(0u, 0u, sigMem)) {
uint32 decoded = 0u;
(void) memcpy(&decoded, sigMem, sizeof(uint32));
gotValue = (decoded == 424242u);
}
}
else if (!ok) {
(void) JoinSynchroniseThread(syncArgs);
}
}
ok = ok && gotValue;
if (clientConn != NULL_PTR(BasicTCPSocket *)) {
(void) clientConn->Close();
delete clientConn;
}
(void) tcpListener.Close();
(void) dataSocket.Close();
Sleep::MSec(50u);
ObjectRegistryDatabase::Instance()->Purge();
return ok;
}
Test/Components/DataSources/UDPStreamerClient/UDPStreamerClientTest.h
+14
View File
@@ -62,6 +62,12 @@ public:
*/ */
bool TestInitialise_MulticastMode_Valid(); bool TestInitialise_MulticastMode_Valid();
/**
* @brief Tests Initialise with a sub-second float32 SilenceTimeout (0.25 s)
* forwarded through to the UDPSClient receiver.
*/
bool TestInitialise_SilenceTimeoutFloat();
/** /**
* @brief Tests that DataPort defaults to Port+1 when MulticastGroup is * @brief Tests that DataPort defaults to Port+1 when MulticastGroup is
* set but DataPort is absent. * set but DataPort is absent.
@@ -152,6 +158,14 @@ public:
* UDP sockets, mirroring the server side of the wire protocol. * UDP sockets, mirroring the server side of the wire protocol.
*/ */
bool TestExecute_ConnectConfigDataEndToEnd(); bool TestExecute_ConnectConfigDataEndToEnd();
/**
* @brief Regression test: verifies that UDPStreamerClient receives multicast
* DATA when Interface is set to "127.0.0.1", ensuring the two-argument
* Join(group, interface) path in UDPSClient::ConnectMulticast is taken.
* This test fails without the UDPSClient multicastInterface fix.
*/
bool TestExecute_MulticastReceivesDataOnInterface();
}; };
#endif /* UDPSTREAMERCLIENTTEST_H_ */ #endif /* UDPSTREAMERCLIENTTEST_H_ */
Test/Configurations/streamhub_demo.cfg
+2 -2
View File
@@ -175,7 +175,7 @@ $App = {
+TimeArrayGAM1 = { +TimeArrayGAM1 = {
Class = TimeArrayGAM Class = TimeArrayGAM
SamplingRate = 1000000.0 SamplingRate = 1000000.0
Anchor = "FirstSample" Anchor = "Continuous"
InputSignals = { InputSignals = {
Time = { Time = {
DataSource = DDB2 DataSource = DDB2
@@ -291,7 +291,7 @@ $App = {
+TimeArrayGAM2 = { +TimeArrayGAM2 = {
Class = TimeArrayGAM Class = TimeArrayGAM
SamplingRate = 5000000.0 SamplingRate = 5000000.0
Anchor = "FirstSample" Anchor = "Continuous"
InputSignals = { InputSignals = {
Time = { Time = {
DataSource = DDB3 DataSource = DDB3
Test/E2E/suite/client/.gitignore
+2
View File
@@ -0,0 +1,2 @@
chain-client
configcheck/configcheck
Test/E2E/suite/client/configcheck/main.go
+403
View File
@@ -0,0 +1,403 @@
// configcheck exercises the calibration and config-persistence WebSocket frames
// against a StreamHub (either the Go hub or the C++ StreamHub) and exits
// non-zero if the hub's replies do not match the protocol.
//
// Frame-strictness rule
// ─────────────────────
// Frames are divided into two categories:
//
// Protocol frames — part of the five command→response sequences under test:
// "calibration", "sources", "configSaved", "configReloaded"
// Ambient frames — unsolicited live traffic the hubs push independently:
// "data", "stats", "triggerState", "monotonicState" (and any unknown type)
//
// The checker is strict about protocol frames: if one arrives when a different
// protocol frame is expected, that is an error (out-of-order or unexpected).
// Ambient frames are logged and skipped without failing the check.
//
// Known documented exception: C++ HandleReloadConfig() emits a "sources" frame
// after "calibration", while the Go hub does not (because Go propagates source
// changes per-add, already broadcasting "sources" when sources are added).
// Both behaviours are correct; the extra "sources" frame from C++ is accepted
// and logged as a deliberate known difference.
package main
import (
"encoding/json"
"flag"
"fmt"
"os"
"sort"
"time"
"github.com/gorilla/websocket"
)
type calEntry struct {
Source string `json:"source"`
Signal string `json:"signal"`
Scale float64 `json:"scale"`
Offset float64 `json:"offset"`
Unit string `json:"unit"`
}
type frame struct {
Type string `json:"type"`
Cal []calEntry `json:"cal"`
OK bool `json:"ok"`
Path string `json:"path"`
Error string `json:"error"`
}
// protocolFrameTypes is the set of frame types that are part of the protocol
// sequences under test. Any frame whose type is in this set but is not the
// one currently expected causes an immediate failure. Frames with types NOT
// in this set are ambient traffic and are silently skipped.
var protocolFrameTypes = map[string]bool{
"calibration": true,
"sources": true,
"configSaved": true,
"configReloaded": true,
}
type conn struct {
ws *websocket.Conn
timeout time.Duration
// readCh is driven by a long-lived background goroutine; nil until startReader.
readCh chan readResult
}
type readResult struct {
f frame
err error
}
// startReader launches a background goroutine that reads all text frames from
// the WebSocket and forwards them on readCh. This avoids setting a read
// deadline on the underlying connection, which permanently poisons gorilla
// websocket v1.5.1 after a timeout fires.
func (c *conn) startReader() {
c.readCh = make(chan readResult, 32)
go func() {
for {
mt, data, err := c.ws.ReadMessage()
if err != nil {
c.readCh <- readResult{err: fmt.Errorf("ws read: %w", err)}
return
}
if mt != websocket.TextMessage {
continue
}
var f frame
if jsonErr := json.Unmarshal(data, &f); jsonErr != nil {
continue
}
c.readCh <- readResult{f: f}
}
}()
}
// next reads from the background reader, skipping ambient frames, until a
// protocol frame arrives or the deadline passes.
//
// If a protocol frame arrives that is NOT the expected one, next returns an
// error immediately — that is the out-of-order/unexpected signal.
func (c *conn) next(want string) (frame, error) {
return c.nextSkipping(want, nil)
}
// nextSkipping is like next but also skips (logs and discards) protocol frames
// whose type is in also. This is used for the initial-connect step where C++
// emits "sources" before "calibration" as part of its state-push sequence,
// whereas the Go hub emits "calibration" first. Passing also=[]string{"sources"}
// lets the checker accept either ordering without letting the connect-time
// sources frame go completely unnoticed.
func (c *conn) nextSkipping(want string, also []string) (frame, error) {
isSkip := make(map[string]bool, len(also))
for _, t := range also {
isSkip[t] = true
}
deadline := time.NewTimer(c.timeout)
defer deadline.Stop()
for {
select {
case <-deadline.C:
return frame{}, fmt.Errorf("timeout waiting for %q", want)
case r, ok := <-c.readCh:
if !ok {
return frame{}, fmt.Errorf("reader closed while waiting for %q", want)
}
if r.err != nil {
return frame{}, fmt.Errorf("read while waiting for %q: %w", want, r.err)
}
if r.f.Type == want {
return r.f, nil
}
// Explicitly-skipped protocol frames (known ordering differences).
if isSkip[r.f.Type] {
fmt.Printf("[skip allowed protocol frame %q while waiting for %q]\n", r.f.Type, want)
continue
}
// Is this an unexpected protocol frame (out-of-order)?
if protocolFrameTypes[r.f.Type] {
return frame{}, fmt.Errorf(
"unexpected protocol frame %q while waiting for %q (out-of-order or spurious emission)",
r.f.Type, want)
}
// Ambient frame — log and skip.
fmt.Printf("[skip ambient %q]\n", r.f.Type)
}
}
}
// nextWithin reads from the background reader until a frame with the wanted
// type arrives within d, returning (frame, true) or (frame{}, false). Unlike
// next() it does NOT return an error on timeout, making it suitable for the
// "must NOT arrive" assertion. Protocol frames with wrong type still skip
// (they will be picked up by the next next() call from the buffer).
func (c *conn) nextWithin(want string, d time.Duration) (frame, bool) {
deadline := time.NewTimer(d)
defer deadline.Stop()
for {
select {
case <-deadline.C:
return frame{}, false
case r, ok := <-c.readCh:
if !ok || r.err != nil {
return frame{}, false
}
if r.f.Type == want {
return r.f, true
}
// For the "must not arrive" check we skip everything else
// (ambient and other protocol frames alike) — we're only
// interested in whether the specific type appears.
}
}
}
func (c *conn) send(v interface{}) error {
data, err := json.Marshal(v)
if err != nil {
return err
}
return c.ws.WriteMessage(websocket.TextMessage, data)
}
func findCal(list []calEntry, source, signal string) (calEntry, bool) {
for _, e := range list {
if e.Source == source && e.Signal == signal {
return e, true
}
}
return calEntry{}, false
}
// isSorted returns true iff the calibration list is sorted by source then signal.
func isSorted(list []calEntry) bool {
for i := 1; i < len(list); i++ {
prev, cur := list[i-1], list[i]
if prev.Source > cur.Source {
return false
}
if prev.Source == cur.Source && prev.Signal > cur.Signal {
return false
}
}
return true
}
func run(url string, timeout time.Duration) error {
ws, _, err := websocket.DefaultDialer.Dial(url, nil)
if err != nil {
return fmt.Errorf("dial %s: %w", url, err)
}
defer ws.Close()
c := &conn{ws: ws, timeout: timeout}
c.startReader()
// ── Step 1: hub sends a calibration frame on connect, even when empty ────
// C++ emits "sources" before "calibration" as part of its initial state
// push; Go emits "calibration" first (Go's sources broadcast is triggered
// per-add when sources are added, not on client-connect in this test where
// no sources exist yet). We explicitly skip "sources" here so the checker
// is not confused by the ordering difference at connect time.
fmt.Println("Step 1: expect calibration on connect")
if _, err := c.nextSkipping("calibration", []string{"sources"}); err != nil {
return fmt.Errorf("on connect: %w", err)
}
// ── Step 2a: set two calibration entries in REVERSE sort order ───────────
// We deliberately set signal "Zeta" before "Alpha" under source "src1",
// and set source "src2" before "src1". The hubs must sort them and the
// checker asserts the received frame and the saved config file are both
// in sorted (source asc, signal asc) order.
wantEntries := []calEntry{
{Source: "src1", Signal: "Alpha", Scale: 2.0, Offset: 0.5, Unit: "m"},
{Source: "src1", Signal: "Zeta", Scale: 0.5, Offset: -1.25, Unit: "V"},
{Source: "src2", Signal: "Beta", Scale: 1.5, Offset: 0.0, Unit: "A"},
}
// Submit in reverse-sort order: src2/Beta, then src1/Zeta, then src1/Alpha.
submitOrder := []calEntry{
wantEntries[2], // src2/Beta
wantEntries[1], // src1/Zeta
wantEntries[0], // src1/Alpha
}
fmt.Println("Step 2: set calibration entries in reverse sort order")
var lastCalFrame frame
for i, e := range submitOrder {
if err := c.send(map[string]interface{}{
"type": "setCalibration",
"source": e.Source,
"signal": e.Signal,
"scale": e.Scale,
"offset": e.Offset,
"unit": e.Unit,
}); err != nil {
return err
}
cf, cerr := c.next("calibration")
if cerr != nil {
return fmt.Errorf("after setCalibration[%d]: %w", i, cerr)
}
if !isSorted(cf.Cal) {
return fmt.Errorf("setCalibration[%d]: calibration frame not sorted by source/signal; got %v", i, cf.Cal)
}
got, ok := findCal(cf.Cal, e.Source, e.Signal)
if !ok {
return fmt.Errorf("setCalibration[%d]: entry %s/%s missing from broadcast", i, e.Source, e.Signal)
}
if got != e {
return fmt.Errorf("setCalibration[%d]: got %+v, want %+v", i, got, e)
}
lastCalFrame = cf
}
// The last broadcast should contain all three entries in sorted order.
if len(lastCalFrame.Cal) != len(wantEntries) {
return fmt.Errorf("after all setCalibration: got %d entries, want %d", len(lastCalFrame.Cal), len(wantEntries))
}
// Confirm sorted order in the final broadcast.
if !isSorted(lastCalFrame.Cal) {
return fmt.Errorf("final calibration broadcast not sorted; got %v", lastCalFrame.Cal)
}
// ── Step 3: invalid entry (scale = 0) must be rejected ───────────────────
fmt.Println("Step 3: setCalibration with scale=0 must be rejected (no broadcast)")
if err := c.send(map[string]interface{}{
"type": "setCalibration", "source": "src1", "signal": "Alpha",
"scale": 0.0, "offset": 0.0, "unit": "",
}); err != nil {
return err
}
if _, accepted := c.nextWithin("calibration", 500*time.Millisecond); accepted {
return fmt.Errorf("setCalibration with scale=0 was accepted, must be rejected")
}
// ── Step 4: saveSources → configSaved ────────────────────────────────────
fmt.Println("Step 4: saveSources → configSaved")
if err := c.send(map[string]string{"type": "saveSources"}); err != nil {
return err
}
savedF, err := c.next("configSaved")
if err != nil {
return err
}
if !savedF.OK {
return fmt.Errorf("configSaved: ok=false, error=%q", savedF.Error)
}
if savedF.Path == "" {
return fmt.Errorf("configSaved: ok=true but path is empty")
}
savedPath := savedF.Path
// ── Step 5: reloadConfig → configReloaded → calibration ─────────────────
// Documented exception: C++ emits an additional "sources" frame after
// "calibration" in HandleReloadConfig(). The Go hub does not emit it
// at that point (it broadcast per-source additions earlier). We accept
// the "sources" frame from C++ but log it as a deliberate known difference.
fmt.Println("Step 5: reloadConfig → configReloaded → calibration")
if err := c.send(map[string]string{"type": "reloadConfig"}); err != nil {
return err
}
reloadedF, err := c.next("configReloaded")
if err != nil {
return err
}
if !reloadedF.OK {
return fmt.Errorf("configReloaded: ok=false, error=%q", reloadedF.Error)
}
calAfterReload, err := c.next("calibration")
if err != nil {
return fmt.Errorf("after reloadConfig, expected calibration: %w", err)
}
if !isSorted(calAfterReload.Cal) {
return fmt.Errorf("reloadConfig calibration not sorted; got %v", calAfterReload.Cal)
}
if len(calAfterReload.Cal) != len(wantEntries) {
return fmt.Errorf("reloadConfig: got %d calibration entries, want %d", len(calAfterReload.Cal), len(wantEntries))
}
for _, want := range wantEntries {
got, ok := findCal(calAfterReload.Cal, want.Source, want.Signal)
if !ok {
return fmt.Errorf("reloadConfig: entry %s/%s did not survive round-trip", want.Source, want.Signal)
}
if got != want {
return fmt.Errorf("reloadConfig: entry %s/%s: got %+v, want %+v", want.Source, want.Signal, got, want)
}
}
// Check for the optional extra "sources" frame from C++ (documented exception).
// We peek with a short timeout; if it arrives we log the known difference.
// If another unexpected protocol frame arrives instead, that is still a failure.
if extraF, arrived := c.nextWithin("sources", 500*time.Millisecond); arrived {
fmt.Printf("[KNOWN DIFFERENCE] C++ hub emitted extra \"sources\" frame after reload "+
"(Go hub does not). This is expected — C++ BroadcastSources() in "+
"HandleReloadConfig() notifies clients of source-list changes after "+
"LoadSourcesFile(skipActive=true); Go hub propagates per-add via commandCh. "+
"Extra frame sources count: %d\n", len(extraF.Cal))
}
// ── Step 6: verify the saved config file is sorted ───────────────────────
// We re-read the saved file and parse it to check sort order.
// This is a file-system check, not a WebSocket check.
fmt.Printf("Step 6: verify saved config file is sorted: %s\n", savedPath)
raw, fileErr := os.ReadFile(savedPath)
if fileErr != nil {
// The config file may not be accessible from this process (e.g. different
// temp dir). Log and skip — the frame sort assertion above already
// provides coverage.
fmt.Printf("[note: cannot read config file %s: %v — skipping file sort check]\n", savedPath, fileErr)
} else {
var fileEntries []calEntry
if jsonErr := json.Unmarshal(raw, &fileEntries); jsonErr != nil {
return fmt.Errorf("config file %s: invalid JSON: %w", savedPath, jsonErr)
}
if !isSorted(fileEntries) {
// Build the expected sorted order for the error message.
sorted := make([]calEntry, len(fileEntries))
copy(sorted, fileEntries)
sort.Slice(sorted, func(i, j int) bool {
if sorted[i].Source != sorted[j].Source {
return sorted[i].Source < sorted[j].Source
}
return sorted[i].Signal < sorted[j].Signal
})
return fmt.Errorf("config file not sorted by source/signal:\n got: %v\n want: %v", fileEntries, sorted)
}
fmt.Printf("Config file has %d entries in sorted order.\n", len(fileEntries))
}
return nil
}
func main() {
url := flag.String("url", "ws://127.0.0.1:8090/ws", "hub WebSocket URL")
timeout := flag.Duration("timeout", 5*time.Second, "per-frame timeout")
flag.Parse()
if err := run(*url, *timeout); err != nil {
fmt.Fprintf(os.Stderr, "configcheck FAIL: %v\n", err)
os.Exit(1)
}
fmt.Println("configcheck OK")
}
Test/GTest/UDPSClientGTest.cpp
+472 -4
View File
@@ -41,6 +41,7 @@
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
#include "BasicUDPSocket.h" #include "BasicUDPSocket.h"
#include "ConfigurationDatabase.h" #include "ConfigurationDatabase.h"
#include "FastPollingMutexSem.h"
#include "InternetHost.h" #include "InternetHost.h"
#include "Sleep.h" #include "Sleep.h"
#include "UDPSClient.h" #include "UDPSClient.h"
@@ -131,6 +132,147 @@ bool WaitForClient(UDPSServer &server, uint32 timeoutMs) {
return false; return false;
} }
/*---------------------------------------------------------------------------*/
/* Fragment-reassembly test harness */
/*---------------------------------------------------------------------------*/
/** Largest reassembled payload the recording listener keeps a copy of. */
const uint32 kMaxRecordedBytes = 8192u;
/** How many reassembled payloads the recording listener keeps. */
const uint32 kMaxRecorded = 16u;
/**
* @brief Listener that records every reassembled DATA/CONFIG payload.
*
* Callbacks run on the UDPSClient receive thread; the test thread reads the
* records after a settle sleep, so both sides take the same lock.
*/
class RecordingListener: public UDPSClientListener {
public:
RecordingListener() :
dataCount(0u), configCount(0u) {
mux.Create();
}
virtual void OnUDPSData(const uint8 *payload, uint32 payloadSize) {
Record(dataPayloads, dataSizes, dataCount, payload, payloadSize);
}
virtual void OnUDPSConfig(const uint8 *payload, uint32 payloadSize) {
Record(configPayloads, configSizes, configCount, payload, payloadSize);
}
uint32 DataCount() {
(void) mux.FastLock();
uint32 n = dataCount;
mux.FastUnLock();
return n;
}
uint32 ConfigCount() {
(void) mux.FastLock();
uint32 n = configCount;
mux.FastUnLock();
return n;
}
/** @return true iff record @p idx matches @p expected byte for byte. */
bool DataMatches(uint32 idx, const uint8 *expected, uint32 expectedSize) {
return Matches(dataPayloads, dataSizes, dataCount, idx, expected,
expectedSize);
}
bool ConfigMatches(uint32 idx, const uint8 *expected, uint32 expectedSize) {
return Matches(configPayloads, configSizes, configCount, idx, expected,
expectedSize);
}
uint32 DataSize(uint32 idx) {
(void) mux.FastLock();
uint32 n = (idx < dataCount) ? dataSizes[idx] : 0u;
mux.FastUnLock();
return n;
}
private:
void Record(uint8 (&dst)[kMaxRecorded][kMaxRecordedBytes],
uint32 (&sizes)[kMaxRecorded], uint32 &count,
const uint8 *payload, uint32 payloadSize) {
(void) mux.FastLock();
if (count < kMaxRecorded) {
sizes[count] = payloadSize;
uint32 n = (payloadSize < kMaxRecordedBytes) ? payloadSize
: kMaxRecordedBytes;
memcpy(dst[count], payload, n);
count++;
}
mux.FastUnLock();
}
bool Matches(uint8 (&src)[kMaxRecorded][kMaxRecordedBytes],
uint32 (&sizes)[kMaxRecorded], uint32 &count, uint32 idx,
const uint8 *expected, uint32 expectedSize) {
(void) mux.FastLock();
bool ok = (idx < count) && (sizes[idx] == expectedSize) &&
(expectedSize <= kMaxRecordedBytes) &&
(memcmp(src[idx], expected, expectedSize) == 0);
mux.FastUnLock();
return ok;
}
FastPollingMutexSem mux;
uint8 dataPayloads[kMaxRecorded][kMaxRecordedBytes];
uint32 dataSizes[kMaxRecorded];
uint32 dataCount;
uint8 configPayloads[kMaxRecorded][kMaxRecordedBytes];
uint32 configSizes[kMaxRecorded];
uint32 configCount;
};
/** Fill @p buf with a position-dependent pattern so misplacement is visible. */
void FillPattern(uint8 *buf, uint32 n, uint8 seed) {
for (uint32 i = 0u; i < n; i++) {
buf[i] = static_cast<uint8>((i * 7u) + seed);
}
}
/** Send one UDPS fragment datagram to 127.0.0.1:@p dstPort. */
bool SendFragment(BasicUDPSocket &sock, uint16 dstPort, uint8 type,
uint32 counter, uint16 fragIdx, uint16 totalFrags,
const uint8 *payload, uint32 payloadBytes) {
uint8 buf[UDPS_HEADER_SIZE + 2048u];
if (payloadBytes > 2048u) {
return false;
}
UDPSBuildHeader(buf, type, counter, fragIdx, totalFrags, payloadBytes);
memcpy(&buf[UDPS_HEADER_SIZE], payload, payloadBytes);
InternetHost dst(dstPort, "127.0.0.1");
(void) sock.SetDestination(dst);
uint32 n = UDPS_HEADER_SIZE + payloadBytes;
return sock.Write(reinterpret_cast<const char8 *>(buf), n);
}
/**
* @brief Bring up a UDPSClient pointed at @p server and learn the ephemeral
* port it receives DATA on (the source port of its CONNECT).
*
* Silence timeout and keepalive are disabled so the session never churns
* underneath the fragments the test injects.
*/
bool StartClientAndLearnPort(UDPSClient &client, ConfigurationDatabase &cfg,
BasicUDPSocket &server, uint16 serverPort,
uint16 &clientPort) {
if (!cfg.Write("ServerAddr", "127.0.0.1")) { return false; }
if (!cfg.Write("Port", static_cast<uint32>(serverPort))) { return false; }
if (!cfg.Write("SilenceTimeout", 0.0f)) { return false; }
if (!cfg.Write("KeepAliveInterval", 0u)) { return false; }
if (!client.Initialise(cfg)) { return false; }
if (!client.Start()) { return false; }
uint8 type = 0xFFu;
if (!WaitDatagram(server, 3000, type, clientPort)) { return false; }
return (type == UDPS_TYPE_CONNECT) && (clientPort != 0u);
}
} // namespace } // namespace
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
@@ -150,7 +292,7 @@ TEST(UDPSClientGTest, TestUnicastKeepAliveSendsPeriodicAck) {
ASSERT_TRUE(cfg.Write("Port", static_cast<uint32>(serverPort))); ASSERT_TRUE(cfg.Write("Port", static_cast<uint32>(serverPort)));
ASSERT_TRUE(cfg.Write("KeepAliveInterval", 1u)); ASSERT_TRUE(cfg.Write("KeepAliveInterval", 1u));
/* SilenceTimeout=0 keeps the session stable for the whole test */ /* SilenceTimeout=0 keeps the session stable for the whole test */
ASSERT_TRUE(cfg.Write("SilenceTimeout", 0u)); ASSERT_TRUE(cfg.Write("SilenceTimeout", 0.0f));
UDPSClient client; UDPSClient client;
ASSERT_TRUE(client.Initialise(cfg)); ASSERT_TRUE(client.Initialise(cfg));
@@ -195,7 +337,7 @@ TEST(UDPSClientGTest, TestKeepAliveDisabledWhenIntervalZero) {
ASSERT_TRUE(cfg.Write("ServerAddr", "127.0.0.1")); ASSERT_TRUE(cfg.Write("ServerAddr", "127.0.0.1"));
ASSERT_TRUE(cfg.Write("Port", static_cast<uint32>(serverPort))); ASSERT_TRUE(cfg.Write("Port", static_cast<uint32>(serverPort)));
ASSERT_TRUE(cfg.Write("KeepAliveInterval", 0u)); ASSERT_TRUE(cfg.Write("KeepAliveInterval", 0u));
ASSERT_TRUE(cfg.Write("SilenceTimeout", 0u)); ASSERT_TRUE(cfg.Write("SilenceTimeout", 0.0f));
UDPSClient client; UDPSClient client;
ASSERT_TRUE(client.Initialise(cfg)); ASSERT_TRUE(client.Initialise(cfg));
@@ -240,7 +382,7 @@ TEST(UDPSClientGTest, TestKeepAlivePreventsServerEviction) {
ASSERT_TRUE(clientCfg.Write("ServerAddr", "127.0.0.1")); ASSERT_TRUE(clientCfg.Write("ServerAddr", "127.0.0.1"));
ASSERT_TRUE(clientCfg.Write("Port", static_cast<uint32>(serverPort))); ASSERT_TRUE(clientCfg.Write("Port", static_cast<uint32>(serverPort)));
ASSERT_TRUE(clientCfg.Write("KeepAliveInterval", 1u)); ASSERT_TRUE(clientCfg.Write("KeepAliveInterval", 1u));
ASSERT_TRUE(clientCfg.Write("SilenceTimeout", 0u)); ASSERT_TRUE(clientCfg.Write("SilenceTimeout", 0.0f));
UDPSClient client; UDPSClient client;
ASSERT_TRUE(client.Initialise(clientCfg)); ASSERT_TRUE(client.Initialise(clientCfg));
ASSERT_TRUE(client.Start()); ASSERT_TRUE(client.Start());
@@ -279,7 +421,7 @@ TEST(UDPSClientGTest, TestServerEvictsWithoutKeepAlive) {
ASSERT_TRUE(clientCfg.Write("ServerAddr", "127.0.0.1")); ASSERT_TRUE(clientCfg.Write("ServerAddr", "127.0.0.1"));
ASSERT_TRUE(clientCfg.Write("Port", static_cast<uint32>(serverPort))); ASSERT_TRUE(clientCfg.Write("Port", static_cast<uint32>(serverPort)));
ASSERT_TRUE(clientCfg.Write("KeepAliveInterval", 0u)); ASSERT_TRUE(clientCfg.Write("KeepAliveInterval", 0u));
ASSERT_TRUE(clientCfg.Write("SilenceTimeout", 0u)); ASSERT_TRUE(clientCfg.Write("SilenceTimeout", 0.0f));
UDPSClient client; UDPSClient client;
ASSERT_TRUE(client.Initialise(clientCfg)); ASSERT_TRUE(client.Initialise(clientCfg));
ASSERT_TRUE(client.Start()); ASSERT_TRUE(client.Start());
@@ -294,3 +436,329 @@ TEST(UDPSClientGTest, TestServerEvictsWithoutKeepAlive) {
client.Stop(); client.Stop();
server.Stop(); server.Stop();
} }
TEST(UDPSClientGTest, TestSilenceTimeoutSubSecondTriggersReconnect) {
/* SilenceTimeout is float32 seconds: a sub-second value must actually
* fire (integer truncation would silently disable the check). */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
ConfigurationDatabase cfg;
ASSERT_TRUE(cfg.Write("ServerAddr", "127.0.0.1"));
ASSERT_TRUE(cfg.Write("Port", static_cast<uint32>(serverPort)));
ASSERT_TRUE(cfg.Write("SilenceTimeout", 0.3f));
ASSERT_TRUE(cfg.Write("ReconnectDelay", 0u)); /* reconnect immediately */
ASSERT_TRUE(cfg.Write("KeepAliveInterval", 0u));
UDPSClient client;
ASSERT_TRUE(client.Initialise(cfg));
ASSERT_TRUE(client.Start());
/* 1) First CONNECT from the client's ephemeral socket */
uint8 type = 0xFFu;
uint16 portA = 0u;
ASSERT_TRUE(WaitDatagram(server, 3000, type, portA));
EXPECT_EQ(type, UDPS_TYPE_CONNECT);
ASSERT_NE(portA, 0u);
/* 2) The server sends nothing: after ~0.3 s the client must disconnect
* and re-announce with a NEW ephemeral socket. Fails if the timeout
* was truncated to 0 (disabled) or left at the old 5 s default. */
uint32 elapsedMs = 0u;
bool reconnected = false;
while ((elapsedMs < 2000u) && !reconnected) {
uint8 t = 0xFFu;
uint16 p = 0u;
bool got = WaitDatagram(server, 500, t, p);
elapsedMs += 500u;
if (!got) {
continue;
}
/* DISCONNECT from the old socket is expected; only a CONNECT from a
* new source port proves the reconnect happened. */
if ((t == UDPS_TYPE_CONNECT) && (p != portA)) {
reconnected = true;
}
}
EXPECT_TRUE(reconnected);
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestReorderedFragmentsAreReassembled) {
/* UDP gives no ordering guarantee: the fragments of one packet may arrive
* in any order, with nothing lost. Reassembly must not depend on fragment
* 0 arriving first — if it does, an out-of-order burst destroys a packet
* whose bytes all arrived, and leaves a slot occupied until the 2 s GC,
* which is how four slots end up permanently full. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
/* 20-byte payload over three 8-byte chunks: the last one is short, which
* is exactly why chunk size has to be learnt from a non-last fragment. */
uint8 expected[20];
FillPattern(expected, sizeof(expected), 3u);
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 7u, 1u, 3u,
&expected[8], 8u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 7u, 2u, 3u,
&expected[16], 4u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 7u, 0u, 3u,
&expected[0], 8u));
Sleep::MSec(400u);
ASSERT_EQ(listener.DataCount(), 1u)
<< "no fragment was lost, yet the packet was not delivered";
EXPECT_EQ(listener.DataSize(0u), 20u);
EXPECT_TRUE(listener.DataMatches(0u, expected, sizeof(expected)));
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestDataAndConfigWithSameCounterDoNotCollide) {
/* DATA and CONFIG carry independent counter sequences, so the same counter
* value legitimately appears on both. A reassembly slot keyed on the
* counter alone merges the two streams: one payload is delivered under the
* wrong type and the other is silently dropped. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
uint8 dataPayload[16];
uint8 cfgPayload[16];
FillPattern(dataPayload, sizeof(dataPayload), 11u);
FillPattern(cfgPayload, sizeof(cfgPayload), 200u);
/* Same counter (42), interleaved, two fragments each. */
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_CONFIG, 42u, 0u, 2u,
&cfgPayload[0], 8u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 42u, 0u, 2u,
&dataPayload[0], 8u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_CONFIG, 42u, 1u, 2u,
&cfgPayload[8], 8u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 42u, 1u, 2u,
&dataPayload[8], 8u));
Sleep::MSec(400u);
EXPECT_EQ(listener.ConfigCount(), 1u);
EXPECT_TRUE(listener.ConfigMatches(0u, cfgPayload, sizeof(cfgPayload)));
ASSERT_EQ(listener.DataCount(), 1u)
<< "the DATA packet was swallowed by the CONFIG slot sharing its counter";
EXPECT_TRUE(listener.DataMatches(0u, dataPayload, sizeof(dataPayload)));
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestDuplicateHighIndexFragmentDoesNotFakeCompletion) {
/* Completion is decided by counting fragments, with a received-bitmask to
* reject duplicates. If the mask is narrower than the fragment count the
* client accepts, a duplicated high-index fragment is counted twice and
* the packet is delivered while a fragment is still missing — a payload
* with a hole of stale bytes, reported as valid. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
/* 300 fragments — past the 256 a 32-byte mask covers, but well inside the
* 512 the client's own sanity check permits. */
const uint16 kTotalFrags = 300u;
const uint32 kChunk = 8u;
const uint32 kLastChunk = 4u;
const uint32 kTotalBytes = ((kTotalFrags - 1u) * kChunk) + kLastChunk;
uint8 expected[((kTotalFrags - 1u) * kChunk) + kLastChunk];
FillPattern(expected, kTotalBytes, 5u);
/* Everything except the final fragment, plus one duplicate above 255. */
for (uint16 f = 0u; f < (kTotalFrags - 1u); f++) {
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 9u, f,
kTotalFrags, &expected[f * kChunk], kChunk));
}
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 9u, 260u,
kTotalFrags, &expected[260u * kChunk], kChunk));
Sleep::MSec(500u);
ASSERT_EQ(listener.DataCount(), 0u)
<< "delivered with a fragment still missing (a duplicate was counted "
"as a new fragment)";
/* The genuinely missing fragment completes it, with the right bytes. */
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 9u,
kTotalFrags - 1u, kTotalFrags,
&expected[(kTotalFrags - 1u) * kChunk],
kLastChunk));
Sleep::MSec(400u);
ASSERT_EQ(listener.DataCount(), 1u);
EXPECT_EQ(listener.DataSize(0u), kTotalBytes);
EXPECT_TRUE(listener.DataMatches(0u, expected, kTotalBytes));
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestStaleDataPacketIsNotDelivered) {
/* A DATA packet that arrives after a newer one has already been delivered
* carries an older time base. Delivering it makes the consumer place its
* samples behind the ones it has: they collide with what is already
* plotted, and the range they should have occupied stays empty. The
* counter is the only thing that tells the two apart, so the client must
* drop anything that does not advance it. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
uint8 pkt[8];
FillPattern(pkt, sizeof(pkt), 1u);
/* 10 and 11 advance the counter; 9 and the repeat of 11 do not. */
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 10u, 0u, 1u,
pkt, sizeof(pkt)));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 11u, 0u, 1u,
pkt, sizeof(pkt)));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 9u, 0u, 1u,
pkt, sizeof(pkt)));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 11u, 0u, 1u,
pkt, sizeof(pkt)));
Sleep::MSec(400u);
EXPECT_EQ(listener.DataCount(), 2u)
<< "a packet older than one already delivered reached the listener";
EXPECT_EQ(client.GetStaleDataPackets(), 2u);
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestCounterGapIsReported) {
/* Consumers that infer a sample period from the sender-clock gap need to
* know how many packets that gap spans; without it a single loss reads as
* a halved rate. The gap comes from the counter, and must exclude the
* packet being delivered. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
uint8 pkt[8];
FillPattern(pkt, sizeof(pkt), 2u);
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 100u, 0u, 1u,
pkt, sizeof(pkt)));
Sleep::MSec(200u);
EXPECT_EQ(client.GetLastDataGap(), 0u) << "the first packet lost nothing";
/* 101, 102 and 103 never arrive. */
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 104u, 0u, 1u,
pkt, sizeof(pkt)));
Sleep::MSec(200u);
EXPECT_EQ(client.GetLastDataGap(), 3u);
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 105u, 0u, 1u,
pkt, sizeof(pkt)));
Sleep::MSec(200u);
EXPECT_EQ(client.GetLastDataGap(), 0u) << "the gap must not persist";
EXPECT_EQ(listener.DataCount(), 3u);
EXPECT_EQ(client.GetStaleDataPackets(), 0u);
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestCounterWraparoundDoesNotRejectStream) {
/* The counter is a uint32 that wraps. Ordering it by plain comparison
* would call every packet after the wrap older than 0xFFFFFFFF and reject
* the stream permanently, so the ordering has to be done on the signed
* difference. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
uint8 pkt[8];
FillPattern(pkt, sizeof(pkt), 4u);
const uint32 counters[4] = { 0xFFFFFFFEu, 0xFFFFFFFFu, 0u, 1u };
for (uint32 i = 0u; i < 4u; i++) {
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA,
counters[i], 0u, 1u, pkt, sizeof(pkt)));
Sleep::MSec(150u);
}
EXPECT_EQ(listener.DataCount(), 4u)
<< "the stream was rejected across the counter wrap";
EXPECT_EQ(client.GetStaleDataPackets(), 0u);
EXPECT_EQ(client.GetLastDataGap(), 0u);
client.Stop();
server.Close();
}
docs/superpowers/plans/2026-08-16-udpstreamer-signal-calibration-and-config.md
+3809
View File
File diff suppressed because it is too large Load Diff
docs/superpowers/plans/2026-08-27-udpscope.md
+10314
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More