martino/MARTe-Integrated-Components
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(udpscope): keep the packet counter in lockstep with the tick reference

Browse Source 
The counter is the denominator of the very period lastAccHrt is the
numerator of, so any packet that cannot move the tick reference must not
move the counter either. Two paths were violating that: a reordered
datagram rolled the counter back while the reference correctly held
(next burst drawn 0.048x too narrow at distance 20, 83.3% worst spacing
error under 2% sustained reordering), and a stray hrt == 0 packet
advanced the counter from the warm-up branch without a tick to match
(+22.5 ms of future-dating per stray packet).

Rules 1 and 2 now record a counter too. The duplicate-datagram guard is
keyed on one, so an array rule that recorded none was exempt and plotted
every doubly-delivered update twice.

Also: rule 2 divides by the count the anchor actually spans and falls
back to the last period it derived; the counter-gap test is wrap-safe so
2^32 rollover reads as no information rather than 2e9 lost packets.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Martino Ferrari
2026-08-28 06:41:47 +02:00
co-authored by Claude Opus 4.6
parent f97fd825c4
commit 1c61e814c0
4 changed files with 644 additions and 144 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Client/udpscope/FrameDecoder.cpp
+105 -29
View File
@@ -139,8 +139,12 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
* The C client only de-duplicates fragments, so an unfragmented update * The C client only de-duplicates fragments, so an unfragmented update
* reaches us intact both times; emitting it again would double the values * reaches us intact both times; emitting it again would double the values
* and advance the timeline by a burst that never existed. Counter zero is * and advance the timeline by a burst that never existed. Counter zero is
* excluded because a producer that never sets one leaves it there. */ * excluded because a producer that never sets one leaves it there.
if (st.lastEmittedValid && f.counter != 0u && f.counter == st.lastCounter) { *
* Keyed on counterValid, not lastEmittedValid: rules 1 and 2 are exposed to
* the same double delivery and would otherwise plot every array twice, since
* neither of them ever joins rule 3's emitted chain. */
if (st.counterValid && f.counter != 0u && f.counter == st.lastCounter) {
return false; return false;
} }
@@ -166,6 +170,10 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
for (uint32_t e = 0; e < nElems; e++) { for (uint32_t e = 0; e < nElems; e++) {
tsOut[e] = base + tv[e] * tScale; tsOut[e] = base + tv[e] * tScale;
} }
/* Only so the duplicate-datagram guard above has something to compare
* against; nothing in this rule reads it back. */
st.lastCounter = f.counter;
st.counterValid = true;
return true; return true;
} }
@@ -189,18 +197,48 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
* on arrival — immune to the bursty delivery that corrupts everything * on arrival — immune to the bursty delivery that corrupts everything
* arrival-derived. Divide by the counter gap for the same reason rule 3 * arrival-derived. Divide by the counter gap for the same reason rule 3
* does: a lost datagram widens the anchor difference without widening * does: a lost datagram widens the anchor difference without widening
* the array. Until a second packet arrives there is nothing to measure * the array.
* and the elements do stack; that is one packet, not the whole run. */ *
if (!(dt > 0.0) && nElems > 1u && st.prevAnchorValid && * Which array, though, is a question of which end is anchored. For
prodSec > st.prevAnchorProdSec) { * LAST_SAMPLE the anchors bracket THIS packet's elements, so the divisor
const uint32_t gap = (f.counter != 0u && f.counter > st.lastCounter) * is nElems; for FIRST_SAMPLE they bracket the PREVIOUS packet's, so it
? (f.counter - st.lastCounter) : 1u; * is that packet's count. Accumulate mode flushes on a timer, so the
* count really does vary between packets — using the wrong one against a
* 10,10,2,10,20 pattern gave 5x, 0.2x and 0.5x the true period and one
* backward step of 3 ms.
*
* Two packets cannot always be measured. The first has no predecessor,
* and a reordered one has an anchor behind its predecessor's; rather
* than stack the whole array on one instant — the very defect this
* paragraph exists to remove — reuse the last period actually measured,
* exactly as the hrt branch reuses lastHrtDt. Only the genuine first
* packet of a run stacks, and only until the second arrives. */
if (!(dt > 0.0) && nElems > 1u) {
const uint32_t divisor = (d.timeMode == kTimeLastSample)
? nElems : st.prevAnchorCount;
const uint32_t rawGap = f.counter - st.lastCounter;
const bool fwdGap = (f.counter != 0u) && st.counterValid &&
(rawGap != 0u) && (rawGap < 0x80000000u);
if (st.prevAnchorValid && prodSec > st.prevAnchorProdSec &&
divisor > 0u) {
dt = (prodSec - st.prevAnchorProdSec) / dt = (prodSec - st.prevAnchorProdSec) /
(static_cast<double>(nElems) * static_cast<double>(gap)); (static_cast<double>(divisor) *
static_cast<double>(fwdGap ? rawGap : 1u));
st.prevAnchorDt = dt;
} else if (st.prevAnchorDt > 0.0) {
dt = st.prevAnchorDt;
} }
}
/* All four move together or not at all: a reordered packet must not
* leave a newer anchor and an older counter behind for the next one to
* divide one by the other. */
if (!st.prevAnchorValid || prodSec > st.prevAnchorProdSec) {
st.prevAnchorProdSec = prodSec; st.prevAnchorProdSec = prodSec;
st.prevAnchorCount = nElems;
st.prevAnchorValid = true; st.prevAnchorValid = true;
st.lastCounter = f.counter; st.lastCounter = f.counter;
st.counterValid = true;
}
tsOut.resize(nElems); tsOut.resize(nElems);
for (uint32_t e = 0; e < nElems; e++) { for (uint32_t e = 0; e < nElems; e++) {
@@ -386,6 +424,7 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
st.lastEmittedEnd = tsOut[nElems - 1u]; st.lastEmittedEnd = tsOut[nElems - 1u];
st.lastEmittedWall = wallNow; st.lastEmittedWall = wallNow;
st.lastCounter = f.counter; st.lastCounter = f.counter;
st.counterValid = true;
st.prevAccCount = nElems; st.prevAccCount = nElems;
st.lastEmittedValid = true; st.lastEmittedValid = true;
return true; return true;
@@ -417,11 +456,23 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
st.lastAccHrt = f.hrt; st.lastAccHrt = f.hrt;
st.lastAccValid = true; st.lastAccValid = true;
} }
/* Same lockstep rule as the hrt branch below. A packet with hrt == 0
* lands here mid-stream and cannot move the tick reference, so it
* must not move the counter either: advancing the counter alone
* makes the next packet's elapsed span two intervals while its gap
* reports one, drawing that burst twice too wide and ending it
* 22.5 ms in the future at 10 samples per 25 ms packet (6x and
* +112 ms after five such packets). Before any tick reference exists
* nothing is keyed to the counter, so it is free to advance and arm
* the duplicate guard for a producer that never sets hrt at all. */
if (f.hrt != 0u || !st.lastAccValid) {
st.lastCounter = f.counter;
st.counterValid = true;
}
st.prevAccCount = nElems; st.prevAccCount = nElems;
if (!ok) { return false; } if (!ok) { return false; }
st.lastEmittedEnd = tsOut[nElems - 1u]; st.lastEmittedEnd = tsOut[nElems - 1u];
st.lastEmittedWall = wallNow; st.lastEmittedWall = wallNow;
st.lastCounter = f.counter;
st.lastEmittedValid = true; st.lastEmittedValid = true;
return true; return true;
} }
@@ -495,25 +546,43 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
* 11x for ten — which draws the recovery burst that many times too wide * 11x for ten — which draws the recovery burst that many times too wide
* and, because the burst is anchored on its LAST element, ends it in the * and, because the burst is anchored on its LAST element, ends it in the
* FUTURE (measured: +22.5 ms for one loss, +225 ms for ten, at 10 * FUTURE (measured: +22.5 ms for one loss, +225 ms for ten, at 10
* samples per 25 ms packet). At 1% loss that mis-spaced 2.7% of all * samples per 25 ms packet). At 1% loss that mis-spaced 4.0% of all
* samples. The declared branch already reads the counter for exactly * samples. The declared branch already reads the counter for exactly
* this purpose (`lost`, above); the hrt branch must too. * this purpose (`lost`, above); the hrt branch must too.
* *
* Only a FORWARD gap counts. A backward or repeated counter is the * The gap and `elapsed` must be measured from the SAME packet, or the
* reorder case handled above, where elapsed is zero anyway. */ * division mixes references. That is why lastCounter is written under
const uint32_t accGap = (f.counter != 0u && st.lastEmittedValid && * takeHrt below, in lockstep with lastAccHrt: a reordered datagram that
f.counter > st.lastCounter) * rolled lastCounter back while leaving lastAccHrt alone would give the
? (f.counter - st.lastCounter) : 1u; * next in-order packet a gap of d+1 against an elapsed spanning one
* interval, dividing its period by d+1 — measured 0.5x the true spacing
* for a swap of neighbours, 0.048x for a distance of twenty.
*
* Unsigned subtraction wraps, which is what makes this right across the
* counter's own 2^32 rollover. A gap in the top half of the range is not
* a forward gap at all but a backward one seen through the wrap, so it
* is treated as no information rather than as 2 billion lost packets. */
const uint32_t rawGap = f.counter - st.lastCounter;
const bool fwdGap = (f.counter != 0u) && st.counterValid &&
(rawGap != 0u) && (rawGap < 0x80000000u);
const double cycles = static_cast<double>(st.prevAccCount) * const double cycles = static_cast<double>(st.prevAccCount) *
static_cast<double>(accGap); static_cast<double>(fwdGap ? rawGap : 1u);
/* Falling back to kDefaultDt is a last resort, not a default: see /* Falling back to kDefaultDt is a last resort, not a default: see
* SigState::lastHrtDt. The fallback is reached on the first hrt packet * SigState::lastHrtDt. What makes the fallback matter is the producer
* of a producer restart (elapsed is zero because hrt went backwards) and * restart, because that is the one path where elapsed is zero AND
* on a reordered datagram, and in both cases the wrong burst width is * offset.reset() has just forced a re-latch, so the burst width used
* latched into ClockOffset permanently — measured 13.5 ms of standing * here is the one calibrated against — measured 13.5 ms of standing
* displacement at 10 samples per 25 ms packet, 89 ms at 100 per 10 ms, * displacement at 10 samples per 25 ms packet, 89 ms at 100 per 10 ms,
* both below kRecalibThresholdS and so never corrected. */ * both below kRecalibThresholdS and so never corrected. A reorder also
* reaches the fallback but does not re-latch, and its measured standing
* error is 0.013 ms, i.e. nothing.
*
* This is better than kDefaultDt at every cadence except one: a producer
* that restarts having CHANGED its cycle time is remembered wrongly, and
* a tenfold change measured -20 ms against kDefaultDt's -6.8 ms. Both
* are bounded and neither is correct; the remembered period wins the
* case that actually happens. */
double hrtDt; double hrtDt;
if (elapsed > 0.0 && cycles > 0.0) { if (elapsed > 0.0 && cycles > 0.0) {
hrtDt = elapsed / cycles; hrtDt = elapsed / cycles;
@@ -574,16 +643,23 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
/* Keep packetBurst's reference current even though this branch does not /* Keep packetBurst's reference current even though this branch does not
* use it. A single packet with hrt == 0 re-enters the warm-up branch * use it. A single packet with hrt == 0 re-enters the warm-up branch
* above, and packetBurst would otherwise span from whenever this signal * above, and packetBurst would otherwise span from whenever this signal
* last took that branch — the whole session. Measured: after 153 hrt * last took that branch — the whole session. Measured: 153 packets into
* packets, one zero-hrt packet emitted a burst starting 3.8 s in the * a 25 ms stream, one zero-hrt packet emitted a burst starting 2.74 s
* past, growing without bound with session length. */ * behind the trace, and that figure grows with session length. */
st.lastPacketWall = wallNow; st.lastPacketWall = wallNow;
st.lastPacketValid = true; st.lastPacketValid = true;
/* Same duplicate-datagram exposure as the declared branch: a host joined /* In lockstep with lastAccHrt, and for the same reason: these two are
* on two interfaces receives every unfragmented update twice, and the * the numerator and the denominator of the next packet's period. Only a
* guard at the top of timestamps() can only fire if this branch leaves a * packet that defines the new front of producer time may move either.
* counter behind for it to compare against. */ * Advancing the counter alone on a reordered datagram halves the next
* packet's spacing; see the gap comment above. Leaving a counter behind
* at all is what lets the duplicate-datagram guard at the top of
* timestamps() fire — a host joined on two interfaces receives every
* unfragmented update twice, and the original always sets takeHrt. */
if (takeHrt) {
st.lastCounter = f.counter; st.lastCounter = f.counter;
st.counterValid = true;
}
st.lastEmittedValid = true; st.lastEmittedValid = true;
return true; return true;
} }
Client/udpscope/FrameDecoder.h
+21 -11
View File
@@ -53,14 +53,16 @@
* already burst keeps every later update on rule 3 regardless of its length; a * already burst keeps every later update on rule 3 regardless of its length; a
* signal that has never burst is a genuine scalar and is left to rule 5. * signal that has never burst is a genuine scalar and is left to rule 5.
* *
* Two divergences OUTSIDE rule 3 are known and deliberately left as they are. * Divergences OUTSIDE rule 3 exist as well; two are named here only because they
* Rule 1 keys ClockOffset on the consuming signal, where UDPSourceSession.cpp:516 * are the ones easily mistaken for bugs. Rules 1 and 2 key ClockOffset on the
* keys it on the time-signal index, so signals sharing a time signal share an * CONSUMING signal, where UDPSourceSession.cpp:538 and :516 key it on the
* offset there and not here — immaterial, since the mapping they compute is the * time-signal index, so signals sharing a time signal share an offset there and
* same. And a FIRST_SAMPLE/LAST_SAMPLE signal whose time signal is absent falls * not here — immaterial, since the mapping they compute is the same. And a
* through to rule 4 rather than using its declared rate; that is a malformed * FIRST_SAMPLE/LAST_SAMPLE signal whose time signal is absent falls through to
* CONFIG, and spanning arrivals is the more honest answer than trusting a rate * rule 4 rather than using its declared rate; that is a malformed CONFIG, and
* whose anchor never arrived. * spanning arrivals is the more honest answer than trusting a rate whose anchor
* never arrived. Neither this list nor the three above is closed: any other
* difference you find is unexamined, not sanctioned.
*/ */
#pragma once #pragma once
@@ -119,10 +121,13 @@ private:
* latches against, and the resulting displacement is usually too small * latches against, and the resulting displacement is usually too small
* for kRecalibThresholdS to ever heal. Zero until first measured. */ * for kRecalibThresholdS to ever heal. Zero until first measured. */
double lastHrtDt = 0.0; double lastHrtDt = 0.0;
/** Rule 2 only: the previous packet's time-signal anchor, in PRODUCER /** Rule 2 only: the previous packet's time-signal anchor in PRODUCER
* seconds. Consecutive anchors are what lets an array with no declared * seconds, the element count that anchor spanned, and the last period
* sampling rate be spread at all. */ * actually derived from a pair of them. Consecutive anchors are what
* lets an array with no declared sampling rate be spread at all. */
double prevAnchorProdSec = 0.0; double prevAnchorProdSec = 0.0;
uint32_t prevAnchorCount = 0u;
double prevAnchorDt = 0.0;
bool prevAnchorValid = false; bool prevAnchorValid = false;
/** For accumulated scalars (rule 3, either branch): end timestamp of the /** For accumulated scalars (rule 3, either branch): end timestamp of the
* most recently emitted burst, and the packet counter it came from. The * most recently emitted burst, and the packet counter it came from. The
@@ -130,6 +135,11 @@ private:
* the exact duration of any lost datagrams. */ * the exact duration of any lost datagrams. */
double lastEmittedEnd = 0.0; double lastEmittedEnd = 0.0;
uint32_t lastCounter = 0u; uint32_t lastCounter = 0u;
/** Whether lastCounter holds a counter this signal has actually seen.
* Distinct from lastEmittedValid, which is about the emitted TIMELINE:
* rules 1 and 2 keep a counter (so the duplicate-datagram guard can
* fire for them too) without ever joining rule 3's chain. */
bool counterValid = false;
/** ARRIVAL time of the packet that produced lastEmittedEnd. Valid /** ARRIVAL time of the packet that produced lastEmittedEnd. Valid
* exactly when lastEmittedValid is, so it needs no flag of its own. * exactly when lastEmittedValid is, so it needs no flag of its own.
* Deliberately not lastPacketWall, which belongs to packetBurst() and * Deliberately not lastPacketWall, which belongs to packetBurst() and
Client/udpscope/tests/FrameDecoderTest.cpp
+168 -8
View File
@@ -89,6 +89,40 @@ TEST(FrameDecoder, FullArrayTakesOneStampPerElementFromTheTimeSignal) {
EXPECT_NEAR(ts[3], 1000.003, 1e-9); EXPECT_NEAR(ts[3], 1000.003, 1e-9);
} }
// A host joined on two interfaces receives every unfragmented update twice, and
// the second copy is a different signal's problem only if the guard can see it.
// It is keyed on a counter each rule leaves behind, so an array rule that never
// records one is silently exempt — and would plot every array twice, at two
// arrival times, doubling back on the X axis. Rules 1 and 2 join rule 3's
// counter-keeping for this reason alone; neither reads the value back.
TEST(FrameDecoder, ArrayRulesDropADuplicatedDatagram) {
for (uint8_t mode : {kTimeFullArray, kTimeFirstSample}) {
FrameDecoder dec;
dec.setSignals({burst("Sine", mode, 1000.0, 4, 1),
timeSignal("Time", mode == kTimeFullArray ? 4u : 1u)});
FrameBuilder fb;
fb.addSignal({1.0, 2.0, 3.0, 4.0});
if (mode == kTimeFullArray) {
fb.addSignal({5.0e9, 5.001e9, 5.002e9, 5.003e9});
} else {
fb.addSignal({5.0e9});
}
const FrameView& first = fb.build(0, 1000.0, 4, 77u);
dec.beginFrame(first);
std::vector<double> ts;
ASSERT_TRUE(dec.timestamps(first, 0, ts)) << "mode " << int(mode);
/* Same counter, same payload, a fraction of a millisecond later off the
* second interface. */
const FrameView& dup = fb.build(0, 1000.0004, 4, 77u);
dec.beginFrame(dup);
EXPECT_FALSE(dec.timestamps(dup, 0, ts))
<< "mode " << int(mode) << " emitted the duplicate array twice";
}
}
TEST(FrameDecoder, FirstSampleAnchorsElementZeroAndCountsForward) { TEST(FrameDecoder, FirstSampleAnchorsElementZeroAndCountsForward) {
FrameDecoder dec; FrameDecoder dec;
dec.setSignals({burst("Sine", kTimeFirstSample, 1000.0, 4, 1), dec.setSignals({burst("Sine", kTimeFirstSample, 1000.0, 4, 1),
@@ -699,6 +733,41 @@ std::vector<HrtPacket> cleanUndeclaredStream() {
return pkts; return pkts;
} }
/** Runs a schedule of 10-sample bursts; returns one entry per DELIVERED
* datagram, empty where the decoder emitted nothing. Per-packet rather than
* concatenated because the interesting quantity is the spacing INSIDE a
* particular burst, and which burst that is depends on the schedule. */
std::vector<std::vector<double> >
runUndeclaredPerPacket(const std::vector<HrtPacket>& pkts) {
FrameDecoder dec;
dec.setSignals({undeclaredAcc()});
std::vector<std::vector<double> > out;
for (const HrtPacket& p : pkts) {
FrameBuilder fb;
fb.addSignal(std::vector<double>(10, 1.0));
const FrameView& f = fb.build(p.hrt, p.arrival, 10, p.counter);
dec.beginFrame(f);
std::vector<double> ts;
if (!dec.timestamps(f, 0, ts)) { ts.clear(); }
out.push_back(ts);
}
return out;
}
/** Delays the datagram at slot @p k by @p dist delivery slots: the payloads
* behind it each move up one and it lands after them. Only the payload moves —
* the arrival time belongs to the slot, because delivery order is what the
* socket actually saw. */
std::vector<HrtPacket> delayOne(const std::vector<HrtPacket>& clean,
size_t k, size_t dist) {
std::vector<HrtPacket> out = clean;
for (size_t i = 0; i < dist; i++) {
std::swap(out[k + i].hrt, out[k + i + 1].hrt);
std::swap(out[k + i].counter, out[k + i + 1].counter);
}
return out;
}
} /* namespace */ } /* namespace */
// A datagram that overtakes its neighbour arrives with an hrt BEHIND the one // A datagram that overtakes its neighbour arrives with an hrt BEHIND the one
@@ -729,6 +798,77 @@ TEST(FrameDecoder, UndeclaredAccumulatedScalarIgnoresReorderedDatagrams) {
<< "reordering left " << (reorderedEnd - cleanEnd) << " s of offset"; << "reordering left " << (reorderedEnd - cleanEnd) << " s of offset";
} }
// Leaving the hrt reference alone on a reordered datagram is only half the rule:
// the packet counter is the DENOMINATOR of the very period that reference is the
// numerator of, so it has to stay behind too. Rolling lastCounter back while
// lastAccHrt holds gives the next in-order packet a gap of dist+1 against an
// elapsed spanning a single interval, and it derives a period dist+1 times too
// short. The test above cannot see this: it compares end times, and a burst drawn
// too NARROW ends early rather than late, so the damage hides inside the burst.
//
// The bound asserted here is not "the true spacing". A reorder legitimately
// squeezes bursts, because the late datagram's samples belong in the past and
// downstream demands increasing stamps, so the monotonic clamp walks them
// forward instead — and the timeline it leaves ahead of the producer takes a few
// packets to bleed off, squeezing those too. But that clamp has an exact floor:
// its cap is kWallBleedFraction * wallElapsed / nElems, and wallElapsed / nElems
// IS the producer's true period at steady cadence, so no burst it touches can
// ever be narrower than kWallBleedFraction of true. Anything below that floor
// did not come from the clamp; it came from a mis-derived period. That is what
// separates the defect from the design, and it is why the check is a floor
// rather than a target.
TEST(FrameDecoder, UndeclaredAccumulatedScalarKeepsItsSpacingAfterAReorder) {
const double trueDt = 0.0025; /* 10 samples per 25 ms packet */
const double floorDt = 0.5 * trueDt; /* kWallBleedFraction * trueDt */
/* Distance 1 sits exactly ON the floor either way and is here to pin it;
* 5 and 20 are where the defect drops through it, to 0.167x and 0.048x. */
for (size_t dist : {size_t(1), size_t(5), size_t(20)}) {
const std::vector<std::vector<double> > out =
runUndeclaredPerPacket(delayOne(cleanUndeclaredStream(), 150, dist));
/* The late payload lands at slot 150 + dist; the slot after it is the
* first in-order packet to divide by the poisoned counter. */
const size_t after = 150u + dist + 1u;
ASSERT_GE(out[after].size(), 2u) << "distance " << dist;
const double dt = out[after][1] - out[after][0];
EXPECT_GE(dt, floorDt - 1.0e-9)
<< "distance " << dist << " drew its burst at " << dt << " s/sample, "
<< (dt / trueDt) << "x the true spacing";
EXPECT_LE(dt, trueDt + 1.0e-9) << "distance " << dist;
}
}
// The same defect under a network that reorders continuously rather than once.
// Same floor, applied to every burst in the run including the late datagrams'
// own — under sustained reordering there is no quiet packet to exempt, and the
// floor holds for all of them anyway.
TEST(FrameDecoder, UndeclaredAccumulatedScalarKeepsItsSpacingUnderSustainedReordering) {
std::vector<HrtPacket> pkts = cleanUndeclaredStream();
/* Six of 300 datagrams — 2% — delayed by one to five slots. */
for (int n = 0; n < 6; n++) {
pkts = delayOne(pkts, 40u + static_cast<size_t>(n) * 40u,
1u + static_cast<size_t>(n) % 5u);
}
const std::vector<std::vector<double> > out = runUndeclaredPerPacket(pkts);
const double trueDt = 0.0025;
double narrow = 1.0; /* smallest ratio to true seen */
size_t narrowAt = 0u;
for (size_t p = 0; p < out.size(); p++) {
for (size_t e = 1; e < out[p].size(); e++) {
const double ratio = (out[p][e] - out[p][e - 1u]) / trueDt;
if (ratio < narrow) { narrow = ratio; narrowAt = p; }
}
}
/* Bottomed out at 0.167x — an 83.3% spacing error — before the counter moved
* in lockstep with the reference. The clamp's own floor is 0.5x. */
EXPECT_GE(narrow, 0.5 - 1.0e-9)
<< "narrowest burst " << narrow << "x true spacing at packet " << narrowAt;
}
// The counterweight. A producer restart drops hrt from the machine's whole // The counterweight. A producer restart drops hrt from the machine's whole
// uptime back to near zero, and that is the one case where the hrt reference // uptime back to near zero, and that is the one case where the hrt reference
// MUST be allowed to regress: refusing every backward step would leave each // MUST be allowed to regress: refusing every backward step would leave each
@@ -1037,20 +1177,29 @@ TEST(FrameDecoder, UndeclaredAccumulatedScalarReturnsToTheWallClockAfterARestart
// field, so it would be left at whenever this signal last took the warm-up // field, so it would be left at whenever this signal last took the warm-up
// branch — the start of the session — and one stray packet would emit a burst // branch — the start of the session — and one stray packet would emit a burst
// starting seconds in the past, worse the longer the scope has been running. // starting seconds in the past, worse the longer the scope has been running.
//
// The counter must sit out that detour with it, for the same lockstep reason as
// the reorder case: a zero-hrt packet advances the counter but cannot advance
// the tick reference, so the next real packet divides an elapsed spanning one
// interval by a gap reporting two, drawing that burst twice too wide and — since
// the burst is anchored on its LAST element — ending it in the future. Measured
// +22.5 ms for one such packet, +45 ms for two, +112.5 ms for five.
TEST(FrameDecoder, UndeclaredAccumulatedScalarSurvivesAStrayZeroHrtPacket) { TEST(FrameDecoder, UndeclaredAccumulatedScalarSurvivesAStrayZeroHrtPacket) {
FrameDecoder dec;
dec.setSignals({undeclaredAcc()});
const double ticks = 1.0e9; const double ticks = 1.0e9;
const uint64_t bootHrt = static_cast<uint64_t>(86400.0 * ticks); const uint64_t bootHrt = static_cast<uint64_t>(86400.0 * ticks);
const double packetSec = 0.025; const double packetSec = 0.025;
for (int run : {1, 2, 5}) {
FrameDecoder dec;
dec.setSignals({undeclaredAcc()});
double last = 0.0; double last = 0.0;
bool seen = false; bool seen = false;
for (int p = 0; p < 200; p++) { for (int p = 0; p < 200; p++) {
FrameBuilder fb; FrameBuilder fb;
fb.addSignal(std::vector<double>(10, 1.0)); fb.addSignal(std::vector<double>(10, 1.0));
const uint64_t hrt = (p == 153) const bool zero = (p >= 153) && (p < 153 + run);
const uint64_t hrt = zero
? 0u ? 0u
: bootHrt + static_cast<uint64_t>(p * packetSec * ticks); : bootHrt + static_cast<uint64_t>(p * packetSec * ticks);
const double arrival = 700.0 + p * packetSec; const double arrival = 700.0 + p * packetSec;
@@ -1061,15 +1210,26 @@ TEST(FrameDecoder, UndeclaredAccumulatedScalarSurvivesAStrayZeroHrtPacket) {
if (!dec.timestamps(f, 0, ts)) { continue; } if (!dec.timestamps(f, 0, ts)) { continue; }
for (double t : ts) { for (double t : ts) {
if (seen) { if (seen) {
ASSERT_GT(t, last) << "stray zero-hrt packet stepped back " ASSERT_GT(t, last) << "run of " << run
<< " zero-hrt packets stepped back "
<< (last - t) << " s at packet " << p; << (last - t) << " s at packet " << p;
} }
last = t; last = t;
seen = true; seen = true;
} }
/* And it must not land far from where the stream already is: spanning /* And it must not land far from where the stream already is:
* from a session-old reference put the burst 3.8 s in the past. */ * spanning from a session-old reference put the burst 2.74 s in the
if (p > 100) { EXPECT_NEAR(ts.back(), arrival, 0.05) << "at packet " << p; } * past, and a counter that ran on without the tick reference put the
* recovery burst 22.5 ms per stray packet into the future. The
* tolerance is a fifth of a packet period: four and a half times
* tighter than the smallest error it has to reject, and still far
* enough above HrtRateFit's residual not to chase regression noise.
* The 50 ms it replaced admitted every one of those errors. */
if (p > 100) {
EXPECT_NEAR(ts.back(), arrival, packetSec / 5.0)
<< "run of " << run << " at packet " << p;
}
}
} }
} }
docs/superpowers/plans/2026-08-27-udpscope.md
+304 -50
View File
@@ -1521,6 +1521,40 @@ TEST(FrameDecoder, FullArrayTakesOneStampPerElementFromTheTimeSignal) {
EXPECT_NEAR(ts[3], 1000.003, 1e-9); EXPECT_NEAR(ts[3], 1000.003, 1e-9);
} }
// A host joined on two interfaces receives every unfragmented update twice, and
// the second copy is a different signal's problem only if the guard can see it.
// It is keyed on a counter each rule leaves behind, so an array rule that never
// records one is silently exempt — and would plot every array twice, at two
// arrival times, doubling back on the X axis. Rules 1 and 2 join rule 3's
// counter-keeping for this reason alone; neither reads the value back.
TEST(FrameDecoder, ArrayRulesDropADuplicatedDatagram) {
for (uint8_t mode : {kTimeFullArray, kTimeFirstSample}) {
FrameDecoder dec;
dec.setSignals({burst("Sine", mode, 1000.0, 4, 1),
timeSignal("Time", mode == kTimeFullArray ? 4u : 1u)});
FrameBuilder fb;
fb.addSignal({1.0, 2.0, 3.0, 4.0});
if (mode == kTimeFullArray) {
fb.addSignal({5.0e9, 5.001e9, 5.002e9, 5.003e9});
} else {
fb.addSignal({5.0e9});
}
const FrameView& first = fb.build(0, 1000.0, 4, 77u);
dec.beginFrame(first);
std::vector<double> ts;
ASSERT_TRUE(dec.timestamps(first, 0, ts)) << "mode " << int(mode);
/* Same counter, same payload, a fraction of a millisecond later off the
* second interface. */
const FrameView& dup = fb.build(0, 1000.0004, 4, 77u);
dec.beginFrame(dup);
EXPECT_FALSE(dec.timestamps(dup, 0, ts))
<< "mode " << int(mode) << " emitted the duplicate array twice";
}
}
TEST(FrameDecoder, FirstSampleAnchorsElementZeroAndCountsForward) { TEST(FrameDecoder, FirstSampleAnchorsElementZeroAndCountsForward) {
FrameDecoder dec; FrameDecoder dec;
dec.setSignals({burst("Sine", kTimeFirstSample, 1000.0, 4, 1), dec.setSignals({burst("Sine", kTimeFirstSample, 1000.0, 4, 1),
@@ -2131,6 +2165,41 @@ std::vector<HrtPacket> cleanUndeclaredStream() {
return pkts; return pkts;
} }
/** Runs a schedule of 10-sample bursts; returns one entry per DELIVERED
* datagram, empty where the decoder emitted nothing. Per-packet rather than
* concatenated because the interesting quantity is the spacing INSIDE a
* particular burst, and which burst that is depends on the schedule. */
std::vector<std::vector<double> >
runUndeclaredPerPacket(const std::vector<HrtPacket>& pkts) {
FrameDecoder dec;
dec.setSignals({undeclaredAcc()});
std::vector<std::vector<double> > out;
for (const HrtPacket& p : pkts) {
FrameBuilder fb;
fb.addSignal(std::vector<double>(10, 1.0));
const FrameView& f = fb.build(p.hrt, p.arrival, 10, p.counter);
dec.beginFrame(f);
std::vector<double> ts;
if (!dec.timestamps(f, 0, ts)) { ts.clear(); }
out.push_back(ts);
}
return out;
}
/** Delays the datagram at slot @p k by @p dist delivery slots: the payloads
* behind it each move up one and it lands after them. Only the payload moves —
* the arrival time belongs to the slot, because delivery order is what the
* socket actually saw. */
std::vector<HrtPacket> delayOne(const std::vector<HrtPacket>& clean,
size_t k, size_t dist) {
std::vector<HrtPacket> out = clean;
for (size_t i = 0; i < dist; i++) {
std::swap(out[k + i].hrt, out[k + i + 1].hrt);
std::swap(out[k + i].counter, out[k + i + 1].counter);
}
return out;
}
} /* namespace */ } /* namespace */
// A datagram that overtakes its neighbour arrives with an hrt BEHIND the one // A datagram that overtakes its neighbour arrives with an hrt BEHIND the one
@@ -2161,6 +2230,77 @@ TEST(FrameDecoder, UndeclaredAccumulatedScalarIgnoresReorderedDatagrams) {
<< "reordering left " << (reorderedEnd - cleanEnd) << " s of offset"; << "reordering left " << (reorderedEnd - cleanEnd) << " s of offset";
} }
// Leaving the hrt reference alone on a reordered datagram is only half the rule:
// the packet counter is the DENOMINATOR of the very period that reference is the
// numerator of, so it has to stay behind too. Rolling lastCounter back while
// lastAccHrt holds gives the next in-order packet a gap of dist+1 against an
// elapsed spanning a single interval, and it derives a period dist+1 times too
// short. The test above cannot see this: it compares end times, and a burst drawn
// too NARROW ends early rather than late, so the damage hides inside the burst.
//
// The bound asserted here is not "the true spacing". A reorder legitimately
// squeezes bursts, because the late datagram's samples belong in the past and
// downstream demands increasing stamps, so the monotonic clamp walks them
// forward instead — and the timeline it leaves ahead of the producer takes a few
// packets to bleed off, squeezing those too. But that clamp has an exact floor:
// its cap is kWallBleedFraction * wallElapsed / nElems, and wallElapsed / nElems
// IS the producer's true period at steady cadence, so no burst it touches can
// ever be narrower than kWallBleedFraction of true. Anything below that floor
// did not come from the clamp; it came from a mis-derived period. That is what
// separates the defect from the design, and it is why the check is a floor
// rather than a target.
TEST(FrameDecoder, UndeclaredAccumulatedScalarKeepsItsSpacingAfterAReorder) {
const double trueDt = 0.0025; /* 10 samples per 25 ms packet */
const double floorDt = 0.5 * trueDt; /* kWallBleedFraction * trueDt */
/* Distance 1 sits exactly ON the floor either way and is here to pin it;
* 5 and 20 are where the defect drops through it, to 0.167x and 0.048x. */
for (size_t dist : {size_t(1), size_t(5), size_t(20)}) {
const std::vector<std::vector<double> > out =
runUndeclaredPerPacket(delayOne(cleanUndeclaredStream(), 150, dist));
/* The late payload lands at slot 150 + dist; the slot after it is the
* first in-order packet to divide by the poisoned counter. */
const size_t after = 150u + dist + 1u;
ASSERT_GE(out[after].size(), 2u) << "distance " << dist;
const double dt = out[after][1] - out[after][0];
EXPECT_GE(dt, floorDt - 1.0e-9)
<< "distance " << dist << " drew its burst at " << dt << " s/sample, "
<< (dt / trueDt) << "x the true spacing";
EXPECT_LE(dt, trueDt + 1.0e-9) << "distance " << dist;
}
}
// The same defect under a network that reorders continuously rather than once.
// Same floor, applied to every burst in the run including the late datagrams'
// own — under sustained reordering there is no quiet packet to exempt, and the
// floor holds for all of them anyway.
TEST(FrameDecoder, UndeclaredAccumulatedScalarKeepsItsSpacingUnderSustainedReordering) {
std::vector<HrtPacket> pkts = cleanUndeclaredStream();
/* Six of 300 datagrams — 2% — delayed by one to five slots. */
for (int n = 0; n < 6; n++) {
pkts = delayOne(pkts, 40u + static_cast<size_t>(n) * 40u,
1u + static_cast<size_t>(n) % 5u);
}
const std::vector<std::vector<double> > out = runUndeclaredPerPacket(pkts);
const double trueDt = 0.0025;
double narrow = 1.0; /* smallest ratio to true seen */
size_t narrowAt = 0u;
for (size_t p = 0; p < out.size(); p++) {
for (size_t e = 1; e < out[p].size(); e++) {
const double ratio = (out[p][e] - out[p][e - 1u]) / trueDt;
if (ratio < narrow) { narrow = ratio; narrowAt = p; }
}
}
/* Bottomed out at 0.167x — an 83.3% spacing error — before the counter moved
* in lockstep with the reference. The clamp's own floor is 0.5x. */
EXPECT_GE(narrow, 0.5 - 1.0e-9)
<< "narrowest burst " << narrow << "x true spacing at packet " << narrowAt;
}
// The counterweight. A producer restart drops hrt from the machine's whole // The counterweight. A producer restart drops hrt from the machine's whole
// uptime back to near zero, and that is the one case where the hrt reference // uptime back to near zero, and that is the one case where the hrt reference
// MUST be allowed to regress: refusing every backward step would leave each // MUST be allowed to regress: refusing every backward step would leave each
@@ -2469,20 +2609,29 @@ TEST(FrameDecoder, UndeclaredAccumulatedScalarReturnsToTheWallClockAfterARestart
// field, so it would be left at whenever this signal last took the warm-up // field, so it would be left at whenever this signal last took the warm-up
// branch — the start of the session — and one stray packet would emit a burst // branch — the start of the session — and one stray packet would emit a burst
// starting seconds in the past, worse the longer the scope has been running. // starting seconds in the past, worse the longer the scope has been running.
//
// The counter must sit out that detour with it, for the same lockstep reason as
// the reorder case: a zero-hrt packet advances the counter but cannot advance
// the tick reference, so the next real packet divides an elapsed spanning one
// interval by a gap reporting two, drawing that burst twice too wide and — since
// the burst is anchored on its LAST element — ending it in the future. Measured
// +22.5 ms for one such packet, +45 ms for two, +112.5 ms for five.
TEST(FrameDecoder, UndeclaredAccumulatedScalarSurvivesAStrayZeroHrtPacket) { TEST(FrameDecoder, UndeclaredAccumulatedScalarSurvivesAStrayZeroHrtPacket) {
FrameDecoder dec;
dec.setSignals({undeclaredAcc()});
const double ticks = 1.0e9; const double ticks = 1.0e9;
const uint64_t bootHrt = static_cast<uint64_t>(86400.0 * ticks); const uint64_t bootHrt = static_cast<uint64_t>(86400.0 * ticks);
const double packetSec = 0.025; const double packetSec = 0.025;
for (int run : {1, 2, 5}) {
FrameDecoder dec;
dec.setSignals({undeclaredAcc()});
double last = 0.0; double last = 0.0;
bool seen = false; bool seen = false;
for (int p = 0; p < 200; p++) { for (int p = 0; p < 200; p++) {
FrameBuilder fb; FrameBuilder fb;
fb.addSignal(std::vector<double>(10, 1.0)); fb.addSignal(std::vector<double>(10, 1.0));
const uint64_t hrt = (p == 153) const bool zero = (p >= 153) && (p < 153 + run);
const uint64_t hrt = zero
? 0u ? 0u
: bootHrt + static_cast<uint64_t>(p * packetSec * ticks); : bootHrt + static_cast<uint64_t>(p * packetSec * ticks);
const double arrival = 700.0 + p * packetSec; const double arrival = 700.0 + p * packetSec;
@@ -2493,15 +2642,26 @@ TEST(FrameDecoder, UndeclaredAccumulatedScalarSurvivesAStrayZeroHrtPacket) {
if (!dec.timestamps(f, 0, ts)) { continue; } if (!dec.timestamps(f, 0, ts)) { continue; }
for (double t : ts) { for (double t : ts) {
if (seen) { if (seen) {
ASSERT_GT(t, last) << "stray zero-hrt packet stepped back " ASSERT_GT(t, last) << "run of " << run
<< " zero-hrt packets stepped back "
<< (last - t) << " s at packet " << p; << (last - t) << " s at packet " << p;
} }
last = t; last = t;
seen = true; seen = true;
} }
/* And it must not land far from where the stream already is: spanning /* And it must not land far from where the stream already is:
* from a session-old reference put the burst 3.8 s in the past. */ * spanning from a session-old reference put the burst 2.74 s in the
if (p > 100) { EXPECT_NEAR(ts.back(), arrival, 0.05) << "at packet " << p; } * past, and a counter that ran on without the tick reference put the
* recovery burst 22.5 ms per stray packet into the future. The
* tolerance is a fifth of a packet period: four and a half times
* tighter than the smallest error it has to reject, and still far
* enough above HrtRateFit's residual not to chase regression noise.
* The 50 ms it replaced admitted every one of those errors. */
if (p > 100) {
EXPECT_NEAR(ts.back(), arrival, packetSec / 5.0)
<< "run of " << run << " at packet " << p;
}
}
} }
} }
@@ -2688,14 +2848,16 @@ Create `Client/udpscope/FrameDecoder.h`:
* already burst keeps every later update on rule 3 regardless of its length; a * already burst keeps every later update on rule 3 regardless of its length; a
* signal that has never burst is a genuine scalar and is left to rule 5. * signal that has never burst is a genuine scalar and is left to rule 5.
* *
* Two divergences OUTSIDE rule 3 are known and deliberately left as they are. * Divergences OUTSIDE rule 3 exist as well; two are named here only because they
* Rule 1 keys ClockOffset on the consuming signal, where UDPSourceSession.cpp:516 * are the ones easily mistaken for bugs. Rules 1 and 2 key ClockOffset on the
* keys it on the time-signal index, so signals sharing a time signal share an * CONSUMING signal, where UDPSourceSession.cpp:538 and :516 key it on the
* offset there and not here — immaterial, since the mapping they compute is the * time-signal index, so signals sharing a time signal share an offset there and
* same. And a FIRST_SAMPLE/LAST_SAMPLE signal whose time signal is absent falls * not here — immaterial, since the mapping they compute is the same. And a
* through to rule 4 rather than using its declared rate; that is a malformed * FIRST_SAMPLE/LAST_SAMPLE signal whose time signal is absent falls through to
* CONFIG, and spanning arrivals is the more honest answer than trusting a rate * rule 4 rather than using its declared rate; that is a malformed CONFIG, and
* whose anchor never arrived. * spanning arrivals is the more honest answer than trusting a rate whose anchor
* never arrived. Neither this list nor the three above is closed: any other
* difference you find is unexamined, not sanctioned.
*/ */
#pragma once #pragma once
@@ -2754,10 +2916,13 @@ private:
* latches against, and the resulting displacement is usually too small * latches against, and the resulting displacement is usually too small
* for kRecalibThresholdS to ever heal. Zero until first measured. */ * for kRecalibThresholdS to ever heal. Zero until first measured. */
double lastHrtDt = 0.0; double lastHrtDt = 0.0;
/** Rule 2 only: the previous packet's time-signal anchor, in PRODUCER /** Rule 2 only: the previous packet's time-signal anchor in PRODUCER
* seconds. Consecutive anchors are what lets an array with no declared * seconds, the element count that anchor spanned, and the last period
* sampling rate be spread at all. */ * actually derived from a pair of them. Consecutive anchors are what
* lets an array with no declared sampling rate be spread at all. */
double prevAnchorProdSec = 0.0; double prevAnchorProdSec = 0.0;
uint32_t prevAnchorCount = 0u;
double prevAnchorDt = 0.0;
bool prevAnchorValid = false; bool prevAnchorValid = false;
/** For accumulated scalars (rule 3, either branch): end timestamp of the /** For accumulated scalars (rule 3, either branch): end timestamp of the
* most recently emitted burst, and the packet counter it came from. The * most recently emitted burst, and the packet counter it came from. The
@@ -2765,6 +2930,11 @@ private:
* the exact duration of any lost datagrams. */ * the exact duration of any lost datagrams. */
double lastEmittedEnd = 0.0; double lastEmittedEnd = 0.0;
uint32_t lastCounter = 0u; uint32_t lastCounter = 0u;
/** Whether lastCounter holds a counter this signal has actually seen.
* Distinct from lastEmittedValid, which is about the emitted TIMELINE:
* rules 1 and 2 keep a counter (so the duplicate-datagram guard can
* fire for them too) without ever joining rule 3's chain. */
bool counterValid = false;
/** ARRIVAL time of the packet that produced lastEmittedEnd. Valid /** ARRIVAL time of the packet that produced lastEmittedEnd. Valid
* exactly when lastEmittedValid is, so it needs no flag of its own. * exactly when lastEmittedValid is, so it needs no flag of its own.
* Deliberately not lastPacketWall, which belongs to packetBurst() and * Deliberately not lastPacketWall, which belongs to packetBurst() and
@@ -2932,8 +3102,12 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
* The C client only de-duplicates fragments, so an unfragmented update * The C client only de-duplicates fragments, so an unfragmented update
* reaches us intact both times; emitting it again would double the values * reaches us intact both times; emitting it again would double the values
* and advance the timeline by a burst that never existed. Counter zero is * and advance the timeline by a burst that never existed. Counter zero is
* excluded because a producer that never sets one leaves it there. */ * excluded because a producer that never sets one leaves it there.
if (st.lastEmittedValid && f.counter != 0u && f.counter == st.lastCounter) { *
* Keyed on counterValid, not lastEmittedValid: rules 1 and 2 are exposed to
* the same double delivery and would otherwise plot every array twice, since
* neither of them ever joins rule 3's emitted chain. */
if (st.counterValid && f.counter != 0u && f.counter == st.lastCounter) {
return false; return false;
} }
@@ -2959,6 +3133,10 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
for (uint32_t e = 0; e < nElems; e++) { for (uint32_t e = 0; e < nElems; e++) {
tsOut[e] = base + tv[e] * tScale; tsOut[e] = base + tv[e] * tScale;
} }
/* Only so the duplicate-datagram guard above has something to compare
* against; nothing in this rule reads it back. */
st.lastCounter = f.counter;
st.counterValid = true;
return true; return true;
} }
@@ -2982,18 +3160,48 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
* on arrival — immune to the bursty delivery that corrupts everything * on arrival — immune to the bursty delivery that corrupts everything
* arrival-derived. Divide by the counter gap for the same reason rule 3 * arrival-derived. Divide by the counter gap for the same reason rule 3
* does: a lost datagram widens the anchor difference without widening * does: a lost datagram widens the anchor difference without widening
* the array. Until a second packet arrives there is nothing to measure * the array.
* and the elements do stack; that is one packet, not the whole run. */ *
if (!(dt > 0.0) && nElems > 1u && st.prevAnchorValid && * Which array, though, is a question of which end is anchored. For
prodSec > st.prevAnchorProdSec) { * LAST_SAMPLE the anchors bracket THIS packet's elements, so the divisor
const uint32_t gap = (f.counter != 0u && f.counter > st.lastCounter) * is nElems; for FIRST_SAMPLE they bracket the PREVIOUS packet's, so it
? (f.counter - st.lastCounter) : 1u; * is that packet's count. Accumulate mode flushes on a timer, so the
* count really does vary between packets — using the wrong one against a
* 10,10,2,10,20 pattern gave 5x, 0.2x and 0.5x the true period and one
* backward step of 3 ms.
*
* Two packets cannot always be measured. The first has no predecessor,
* and a reordered one has an anchor behind its predecessor's; rather
* than stack the whole array on one instant — the very defect this
* paragraph exists to remove — reuse the last period actually measured,
* exactly as the hrt branch reuses lastHrtDt. Only the genuine first
* packet of a run stacks, and only until the second arrives. */
if (!(dt > 0.0) && nElems > 1u) {
const uint32_t divisor = (d.timeMode == kTimeLastSample)
? nElems : st.prevAnchorCount;
const uint32_t rawGap = f.counter - st.lastCounter;
const bool fwdGap = (f.counter != 0u) && st.counterValid &&
(rawGap != 0u) && (rawGap < 0x80000000u);
if (st.prevAnchorValid && prodSec > st.prevAnchorProdSec &&
divisor > 0u) {
dt = (prodSec - st.prevAnchorProdSec) / dt = (prodSec - st.prevAnchorProdSec) /
(static_cast<double>(nElems) * static_cast<double>(gap)); (static_cast<double>(divisor) *
static_cast<double>(fwdGap ? rawGap : 1u));
st.prevAnchorDt = dt;
} else if (st.prevAnchorDt > 0.0) {
dt = st.prevAnchorDt;
} }
}
/* All four move together or not at all: a reordered packet must not
* leave a newer anchor and an older counter behind for the next one to
* divide one by the other. */
if (!st.prevAnchorValid || prodSec > st.prevAnchorProdSec) {
st.prevAnchorProdSec = prodSec; st.prevAnchorProdSec = prodSec;
st.prevAnchorCount = nElems;
st.prevAnchorValid = true; st.prevAnchorValid = true;
st.lastCounter = f.counter; st.lastCounter = f.counter;
st.counterValid = true;
}
tsOut.resize(nElems); tsOut.resize(nElems);
for (uint32_t e = 0; e < nElems; e++) { for (uint32_t e = 0; e < nElems; e++) {
@@ -3179,6 +3387,7 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
st.lastEmittedEnd = tsOut[nElems - 1u]; st.lastEmittedEnd = tsOut[nElems - 1u];
st.lastEmittedWall = wallNow; st.lastEmittedWall = wallNow;
st.lastCounter = f.counter; st.lastCounter = f.counter;
st.counterValid = true;
st.prevAccCount = nElems; st.prevAccCount = nElems;
st.lastEmittedValid = true; st.lastEmittedValid = true;
return true; return true;
@@ -3210,11 +3419,23 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
st.lastAccHrt = f.hrt; st.lastAccHrt = f.hrt;
st.lastAccValid = true; st.lastAccValid = true;
} }
/* Same lockstep rule as the hrt branch below. A packet with hrt == 0
* lands here mid-stream and cannot move the tick reference, so it
* must not move the counter either: advancing the counter alone
* makes the next packet's elapsed span two intervals while its gap
* reports one, drawing that burst twice too wide and ending it
* 22.5 ms in the future at 10 samples per 25 ms packet (6x and
* +112 ms after five such packets). Before any tick reference exists
* nothing is keyed to the counter, so it is free to advance and arm
* the duplicate guard for a producer that never sets hrt at all. */
if (f.hrt != 0u || !st.lastAccValid) {
st.lastCounter = f.counter;
st.counterValid = true;
}
st.prevAccCount = nElems; st.prevAccCount = nElems;
if (!ok) { return false; } if (!ok) { return false; }
st.lastEmittedEnd = tsOut[nElems - 1u]; st.lastEmittedEnd = tsOut[nElems - 1u];
st.lastEmittedWall = wallNow; st.lastEmittedWall = wallNow;
st.lastCounter = f.counter;
st.lastEmittedValid = true; st.lastEmittedValid = true;
return true; return true;
} }
@@ -3288,25 +3509,43 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
* 11x for ten — which draws the recovery burst that many times too wide * 11x for ten — which draws the recovery burst that many times too wide
* and, because the burst is anchored on its LAST element, ends it in the * and, because the burst is anchored on its LAST element, ends it in the
* FUTURE (measured: +22.5 ms for one loss, +225 ms for ten, at 10 * FUTURE (measured: +22.5 ms for one loss, +225 ms for ten, at 10
* samples per 25 ms packet). At 1% loss that mis-spaced 2.7% of all * samples per 25 ms packet). At 1% loss that mis-spaced 4.0% of all
* samples. The declared branch already reads the counter for exactly * samples. The declared branch already reads the counter for exactly
* this purpose (`lost`, above); the hrt branch must too. * this purpose (`lost`, above); the hrt branch must too.
* *
* Only a FORWARD gap counts. A backward or repeated counter is the * The gap and `elapsed` must be measured from the SAME packet, or the
* reorder case handled above, where elapsed is zero anyway. */ * division mixes references. That is why lastCounter is written under
const uint32_t accGap = (f.counter != 0u && st.lastEmittedValid && * takeHrt below, in lockstep with lastAccHrt: a reordered datagram that
f.counter > st.lastCounter) * rolled lastCounter back while leaving lastAccHrt alone would give the
? (f.counter - st.lastCounter) : 1u; * next in-order packet a gap of d+1 against an elapsed spanning one
* interval, dividing its period by d+1 — measured 0.5x the true spacing
* for a swap of neighbours, 0.048x for a distance of twenty.
*
* Unsigned subtraction wraps, which is what makes this right across the
* counter's own 2^32 rollover. A gap in the top half of the range is not
* a forward gap at all but a backward one seen through the wrap, so it
* is treated as no information rather than as 2 billion lost packets. */
const uint32_t rawGap = f.counter - st.lastCounter;
const bool fwdGap = (f.counter != 0u) && st.counterValid &&
(rawGap != 0u) && (rawGap < 0x80000000u);
const double cycles = static_cast<double>(st.prevAccCount) * const double cycles = static_cast<double>(st.prevAccCount) *
static_cast<double>(accGap); static_cast<double>(fwdGap ? rawGap : 1u);
/* Falling back to kDefaultDt is a last resort, not a default: see /* Falling back to kDefaultDt is a last resort, not a default: see
* SigState::lastHrtDt. The fallback is reached on the first hrt packet * SigState::lastHrtDt. What makes the fallback matter is the producer
* of a producer restart (elapsed is zero because hrt went backwards) and * restart, because that is the one path where elapsed is zero AND
* on a reordered datagram, and in both cases the wrong burst width is * offset.reset() has just forced a re-latch, so the burst width used
* latched into ClockOffset permanently — measured 13.5 ms of standing * here is the one calibrated against — measured 13.5 ms of standing
* displacement at 10 samples per 25 ms packet, 89 ms at 100 per 10 ms, * displacement at 10 samples per 25 ms packet, 89 ms at 100 per 10 ms,
* both below kRecalibThresholdS and so never corrected. */ * both below kRecalibThresholdS and so never corrected. A reorder also
* reaches the fallback but does not re-latch, and its measured standing
* error is 0.013 ms, i.e. nothing.
*
* This is better than kDefaultDt at every cadence except one: a producer
* that restarts having CHANGED its cycle time is remembered wrongly, and
* a tenfold change measured -20 ms against kDefaultDt's -6.8 ms. Both
* are bounded and neither is correct; the remembered period wins the
* case that actually happens. */
double hrtDt; double hrtDt;
if (elapsed > 0.0 && cycles > 0.0) { if (elapsed > 0.0 && cycles > 0.0) {
hrtDt = elapsed / cycles; hrtDt = elapsed / cycles;
@@ -3367,16 +3606,23 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
/* Keep packetBurst's reference current even though this branch does not /* Keep packetBurst's reference current even though this branch does not
* use it. A single packet with hrt == 0 re-enters the warm-up branch * use it. A single packet with hrt == 0 re-enters the warm-up branch
* above, and packetBurst would otherwise span from whenever this signal * above, and packetBurst would otherwise span from whenever this signal
* last took that branch — the whole session. Measured: after 153 hrt * last took that branch — the whole session. Measured: 153 packets into
* packets, one zero-hrt packet emitted a burst starting 3.8 s in the * a 25 ms stream, one zero-hrt packet emitted a burst starting 2.74 s
* past, growing without bound with session length. */ * behind the trace, and that figure grows with session length. */
st.lastPacketWall = wallNow; st.lastPacketWall = wallNow;
st.lastPacketValid = true; st.lastPacketValid = true;
/* Same duplicate-datagram exposure as the declared branch: a host joined /* In lockstep with lastAccHrt, and for the same reason: these two are
* on two interfaces receives every unfragmented update twice, and the * the numerator and the denominator of the next packet's period. Only a
* guard at the top of timestamps() can only fire if this branch leaves a * packet that defines the new front of producer time may move either.
* counter behind for it to compare against. */ * Advancing the counter alone on a reordered datagram halves the next
* packet's spacing; see the gap comment above. Leaving a counter behind
* at all is what lets the duplicate-datagram guard at the top of
* timestamps() fire — a host joined on two interfaces receives every
* unfragmented update twice, and the original always sets takeHrt. */
if (takeHrt) {
st.lastCounter = f.counter; st.lastCounter = f.counter;
st.counterValid = true;
}
st.lastEmittedValid = true; st.lastEmittedValid = true;
return true; return true;
} }
@@ -3411,7 +3657,7 @@ set(CORE_SOURCES
cd Client/udpscope && cmake --build build -j && ./build/udpscope_tests --gtest_filter='FrameDecoder*' cd Client/udpscope && cmake --build build -j && ./build/udpscope_tests --gtest_filter='FrameDecoder*'
``` ```
Expected: PASS, 30 `FrameDecoder` tests — 59 across the whole `udpscope_tests` binary. Expected: PASS, 33 `FrameDecoder` tests — 62 across the whole `udpscope_tests` binary.
If `AccumulatedScalarSurvivesBurstyDelivery` fails, do NOT reach for the hrt fit: with a declared `samplingRate` rule 3 never consults it, precisely because the fit is not ready for the first 32 packets and — since `HrtRateFit` regresses `hrt` against ARRIVAL time — is itself corrupted by the very bursts it would be asked to survive. Check instead that `lastEmittedEnd`, `lastCounter`, `prevAccCount` and `lastEmittedValid` are updated on every emitted burst. If `AccumulatedScalarSurvivesBurstyDelivery` fails, do NOT reach for the hrt fit: with a declared `samplingRate` rule 3 never consults it, precisely because the fit is not ready for the first 32 packets and — since `HrtRateFit` regresses `hrt` against ARRIVAL time — is itself corrupted by the very bursts it would be asked to survive. Check instead that `lastEmittedEnd`, `lastCounter`, `prevAccCount` and `lastEmittedValid` are updated on every emitted burst.
@@ -3443,7 +3689,15 @@ If `UndeclaredAccumulatedScalarKeepsItsSpacingThroughPacketLoss` fails, `hrtDt`
If `UndeclaredAccumulatedScalarReturnsToTheWallClockAfterARestart` fails, the `kDefaultDt` fallback is being used where `lastHrtDt` should be. On the restart packet `hrt` goes backwards, so `elapsed` is zero and no period can be measured — but `st.offset.reset()` on that same packet means it is also the packet that re-latches. `kDefaultDt` is only right at 1 kHz: measured standing displacement is +13.5 ms at 10 samples per 25 ms and -89 ms at 100 per 10 ms, both too small for recalibration to ever heal. Note `AccumulatedScalarSurvivesAProducerRestart` runs at exactly the +13.5 ms cadence and passes right through this, because it asserts only order and spacing. If `UndeclaredAccumulatedScalarReturnsToTheWallClockAfterARestart` fails, the `kDefaultDt` fallback is being used where `lastHrtDt` should be. On the restart packet `hrt` goes backwards, so `elapsed` is zero and no period can be measured — but `st.offset.reset()` on that same packet means it is also the packet that re-latches. `kDefaultDt` is only right at 1 kHz: measured standing displacement is +13.5 ms at 10 samples per 25 ms and -89 ms at 100 per 10 ms, both too small for recalibration to ever heal. Note `AccumulatedScalarSurvivesAProducerRestart` runs at exactly the +13.5 ms cadence and passes right through this, because it asserts only order and spacing.
If `UndeclaredAccumulatedScalarSurvivesAStrayZeroHrtPacket` fails, the hrt branch has stopped keeping `lastPacketWall` current. A packet with `hrt == 0` re-enters the warm-up branch, and `packetBurst` spans from that field — which the hrt branch does not otherwise write, so it would still hold whenever this signal last took the warm-up branch. Measured: after 153 hrt packets, one zero-hrt packet emitted a burst starting 3.8 s in the past, and the displacement grows without bound with session length. If `UndeclaredAccumulatedScalarSurvivesAStrayZeroHrtPacket` fails on the monotonicity assertion, the hrt branch has stopped keeping `lastPacketWall` current. A packet with `hrt == 0` re-enters the warm-up branch, and `packetBurst` spans from that field — which the hrt branch does not otherwise write, so it would still hold whenever this signal last took the warm-up branch. Measured: 153 packets into a 25 ms stream, one zero-hrt packet emitted a burst starting 2.74 s behind the trace, and that figure grows with session length.
If the same test fails instead on the `EXPECT_NEAR(ts.back(), arrival, ...)` assertion, the warm-up branch is advancing `lastCounter` for a packet that could not advance `lastAccHrt`. Those two are the denominator and the numerator of the next packet's period and must move together; a counter that ran on alone makes the recovery burst twice too wide and — burst anchored on its LAST element — ends it in the future, +22.5 ms per stray packet at 10 samples per 25 ms (+45 ms for two, +112.5 ms for five). Guard the write with `f.hrt != 0u || !st.lastAccValid`: before any tick reference exists nothing is keyed to the counter, so it is free to advance and arm the duplicate guard for a producer that never sets `hrt` at all. The test's tolerance is deliberately a fifth of a packet period; the 50 ms it started at admitted every one of those errors and passed for the wrong reason.
The reorder counterpart of that same lockstep rule is covered by `UndeclaredAccumulatedScalarKeepsItsSpacingAfterAReorder` and `...UnderSustainedReordering`. If either fails, `st.lastCounter` is being written outside the `takeHrt` guard: a late datagram rolls the counter back while the tick reference correctly holds, so the next in-order packet divides an `elapsed` spanning one interval by a gap reporting `dist + 1` and derives a period that many times too short — measured 0.500x true spacing at distance 1, 0.167x at 5, 0.048x at 20, and a worst spacing error of 83.3% under 2% sustained reordering.
Read the bound those two assert carefully before "fixing" it. They do NOT assert the true spacing, because a reorder legitimately squeezes bursts: the late datagram's samples belong in the past, downstream demands increasing stamps, so the monotonic clamp walks them forward, and the timeline it leaves ahead of the producer takes several packets to bleed off. What the clamp cannot do is go below its own floor — its cap is `kWallBleedFraction * wallElapsed / nElems`, and `wallElapsed / nElems` IS the true period at steady cadence, so `kWallBleedFraction` (0.5x) is an exact lower bound on anything the clamp touches. A burst narrower than that did not come from the clamp; it came from a mis-derived period. That is also why distance 1 is in the list but proves nothing on its own — it sits exactly ON the floor either way, and only distances 5 and 20 drop through it.
If `ArrayRulesDropADuplicatedDatagram` fails, rules 1 and 2 have stopped recording `lastCounter`/`counterValid`. The duplicate-datagram guard at the top of `timestamps()` is keyed on a counter each rule leaves behind, so a rule that records none is silently exempt from it — and a host joined on two interfaces would then plot every array twice, at two arrival times, doubling back on the X axis. Neither rule reads the value back; they keep it only so the guard can fire. This is also why the guard is keyed on `counterValid` rather than `lastEmittedValid`: the latter is about rule 3's emitted chain, which rules 1 and 2 never join.
If `FirstSampleWithNoRateSpreadsFromConsecutiveAnchors` fails, rule 2 has gone back to `UDPSourceSession.cpp:522`'s behaviour of leaving the step at zero when no rate is declared. That stacks every element of the array on one instant, which a host-local consumer can store but this scope cannot plot, and which contradicts the strictly-increasing invariant rule 3 defends everywhere. The spread is recoverable from consecutive time-signal anchors — the producer's own clock, so immune to the bursty delivery that corrupts anything arrival-derived — and must be divided by the counter gap for the same reason rule 3 divides by it. If `FirstSampleWithNoRateSpreadsFromConsecutiveAnchors` fails, rule 2 has gone back to `UDPSourceSession.cpp:522`'s behaviour of leaving the step at zero when no rate is declared. That stacks every element of the array on one instant, which a host-local consumer can store but this scope cannot plot, and which contradicts the strictly-increasing invariant rule 3 defends everywhere. The spread is recoverable from consecutive time-signal anchors — the producer's own clock, so immune to the bursty delivery that corrupts anything arrival-derived — and must be divided by the counter gap for the same reason rule 3 divides by it.