Reported as samples sporadically carrying a previous packet's timestamp:
holes on one side of the stream and collisions on the other, in both the
Go and the MARTe2 receiver. That it appeared in both is what located it
-- the shared cause is upstream of either client. Four independent
defects, all of which end in a packet's values being placed at a time
that is not theirs.
Reassembly slot exhaustion (the "Reassembly slots full; evicting oldest"
flood). Chunk size was learnt only from fragment 0, so an out-of-order
burst destroyed a packet whose bytes had all arrived and left the slot
occupied until the 2 s GC. Slots were keyed on the counter alone, but
DATA and CONFIG number independently, so equal counters merged the two
streams. The 32-byte received-mask covered 256 of the 512 fragments the
client accepts, so a duplicate above 255 was counted as new and the
packet was delivered with a hole of stale bytes in it. And one datagram
was read per Execute(), which cannot drain a fast producer. Fixed with a
pendingTail deferral, (counter, type) keying, a 64-byte mask, a
256-datagram drain, counter-age slot reclamation, and a 1 Hz aggregated
warning in place of the per-eviction flood.
UDPStreamer dropping whole Accumulate batches. EventSem::ResetWait is
Reset-then-Wait, so a Post() landing while the sender thread was inside
ServiceClients()/SendData() was destroyed by the next Reset. The batch
was then skipped with dataReady false, readyFill was never cleared, and
the following flush overwrote it: an entire run of RT cycles never
reached the wire. The record of pending work now lives in the buffers
rather than in the semaphore edge, which also removes up to
UDPS_DATA_WAIT_MS of latency; genuine backpressure overwrites are
counted and reported. Against the unfixed code the new test sees
2999/3000 batches never consumed.
Period inflation after loss. Accumulated scalars carry no SamplingRate,
so the receiver derives dt from the sender-clock gap -- but dividing it
by the previous packet's sample count is only right while nothing is
lost. One loss doubles the reported period, which spreads a batch a full
batch past its own end and into the range the next packet claims. That
is the hole and the collision, exactly. Inferring the cycle count from
the estimate's own period is not a way out: it has a stable fixed point
wherever gap/dt is an integer, so a real rate change locks it at the old
one for good (AccumDtGTest.FollowsSustainedRateChange).
The packet counter removes the ambiguity, so all three receivers now
order on it: a DATA packet that does not advance the counter is dropped
rather than delivered, because its values are older than data already
handed over. Ordering is on the signed difference so it survives the
uint32 wrap, and the sequence resets on reconnect, where the producer's
counter restarts independently of ours. The loss count that falls out of
the same delta feeds the period estimate as cycles = prevN * (1 + lost),
which reduces exactly to gap/prevN when nothing is lost and therefore
still tracks a genuine rate change. UDPSClient::AcceptDataCounter (C++),
udpsprotocol.SequenceGate (Go), decode_data (C).
The C client's existing gap counter was wrap-unsafe and let a stale
packet rewind last_counter, which made every subsequent gap wrong; it
uses the same code now. Docs/Protocol.md gains an Ordering DATA section
stating the requirement for any receiver, including ones outside this
repository.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Finding 1 (HandleReloadConfig data loss): move ClearCalibration inside
LoadSourcesFile so the table is wiped only after a successful fread.
Adds a clearCalibration bool parameter (default false); the reload path
passes true, the startup path passes false.
Finding 2 (JSON injection via unit/source/signal): add JsonEscape()
static helper (escapes \", \\, \n \r \t, and \u00XX for other control
chars). Applied at all three emission sites: BroadcastCalibration,
HandleSaveSources, and BroadcastSources (label). Teach JsonGetString to
unescape the same set on read, so values round-trip correctly.
Finding 3 (%.17g verbosity): add ShortFloat() static helper that tries
%.15g then %.16g then %.17g, stopping at the first precision whose
strtod() output compares equal to the original. Applied at both float
emission sites. 0.1 now prints as "0.1", not "0.10000000000000001".
Minor: fix two inaccurate comments in StreamHub.h — the CalibrationEntry
rationale (not a 133 MB / address-limit issue; the real reason is no
per-entry heap churn, STL-free, trivially copyable) and "chars" to "bytes"
for the unit cap.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The single-pass repair left invalid bytes when the candidate lead byte
had class 0 (illegal 0xF8-0xFF bytes, or a bare continuation byte
reached after the 3-byte backward-scan cap). Convert to a loop with a
`cut` flag mirroring Go's loop: each iteration either makes no cut
(exits) or strictly reduces ulen by >= 1 byte (terminates in <= 16
iterations). Also treat expected==0 as a cut target, matching Go's
behaviour of stripping any byte that decodes as an invalid one-byte
sequence.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous walk-back in SetCalibrationEntry was unconditional, corrupting
short valid units ending in multi-byte characters (e.g. Omega, mu, degree).
Also failed to drop an orphaned lead byte left after stripping continuation
bytes. Restructured to use a 256-byte staging buffer so truncation can be
detected, then repair runs only in the truncation branch. Algorithm now
matches Go CalConfig.Normalise() exactly: scan back over continuation bytes
(up to 3), find the lead byte, derive expected sequence length, cut if
incomplete. Covers all cases: orphaned continuation, orphaned lead, cut on
lead byte.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Finding 1: Add TrimInPlace helper; apply trim→strip-[i]→empty-check order
in SetCalibrationEntry (matching Go Normalise()), so whitespace-padded
source/signal from WS clients normalise identically to config-file loads.
Finding 2: Trim unit before truncating to kMaxUnitLen, then walk back
continuation bytes (0x80-0xBF) to avoid leaving a partial UTF-8 rune,
matching Go's utf8.DecodeLastRuneInString loop.
Finding 3: BroadcastCalibration and HandleSaveSources now emit entries
sorted by source then signal (insertion sort over an index array, no STL),
producing byte-identical calibration frames and config files to the Go hub.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add CalibrationEntry (heap-allocated array[256], char[] fields to stay within
the 133 MB struct's canonical address limit) plus calibrationMutex_ and
numCalibration_.
- Implement SetCalibrationEntry/ClearCalibration, BroadcastCalibration,
BroadcastConfigAck, HandleSetCalibration, HandleReloadConfig.
- Wire setCalibration and reloadConfig into OnWSCommand dispatch.
- Broadcast calibration to each newly connected client after triggerState.
- Fix JSON round-trip bug: replace JsonGetString/JsonGetBool helpers with a
shared whitespace-tolerant JsonFindValue (tolerates "key" : "value" as
written by HandleSaveSources); add JsonIsFinite (no <cmath>).
- Extend LoadSourcesFile(bool skipActive) to also parse calibration blocks;
SourceIsActive checks live sessions before starting a duplicate.
- Extend HandleSaveSources to persist calibration blocks; emit configSaved ack.
- Reload semantics: calibration replaced wholesale, sources added only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Parse the optional +Recorder block in Initialise (Enabled/AutoStart/
Directory/MaxFileMB/KeepFiles/StagingMB/FlushIntervalSec/MinDiskFreeMB/
Signals, MB->bytes), apply it to each session before Start (static and
dynamic sources). Drive recorder disk I/O from the push loop via
RecorderFlushTick, and add recStart/recStop/recInfo WS commands plus a
recStatus broadcast/snapshot.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each session owns a BinaryRecorder. The receive thread configures the
recorder layout at CONFIG time and captures each data packet; the push
thread performs all file I/O via RecorderFlushTick. WS subset overrides
are adopted through an epoch-deferred handoff mirroring the trigger path.
Arm/disarm/flush/info are exposed for StreamHub-level control.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a FileWriter-compatible per-source binary recorder: native-type encode
and TypeDescriptor mapping, FileWriter header serialization, subset/quantized/
ACCUMULATE row serialization, and push-thread double-buffer flush with size-cap
rotation, keep-N pruning, fdatasync cadence, disk-free and staging-overflow
guards. Covered by 12 GTests; full suite (83) green.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
