fix(udps): stop packets being dated from an earlier time base
Reported as samples sporadically carrying a previous packet's timestamp: holes on one side of the stream and collisions on the other, in both the Go and the MARTe2 receiver. That it appeared in both is what located it -- the shared cause is upstream of either client. Four independent defects, all of which end in a packet's values being placed at a time that is not theirs. Reassembly slot exhaustion (the "Reassembly slots full; evicting oldest" flood). Chunk size was learnt only from fragment 0, so an out-of-order burst destroyed a packet whose bytes had all arrived and left the slot occupied until the 2 s GC. Slots were keyed on the counter alone, but DATA and CONFIG number independently, so equal counters merged the two streams. The 32-byte received-mask covered 256 of the 512 fragments the client accepts, so a duplicate above 255 was counted as new and the packet was delivered with a hole of stale bytes in it. And one datagram was read per Execute(), which cannot drain a fast producer. Fixed with a pendingTail deferral, (counter, type) keying, a 64-byte mask, a 256-datagram drain, counter-age slot reclamation, and a 1 Hz aggregated warning in place of the per-eviction flood. UDPStreamer dropping whole Accumulate batches. EventSem::ResetWait is Reset-then-Wait, so a Post() landing while the sender thread was inside ServiceClients()/SendData() was destroyed by the next Reset. The batch was then skipped with dataReady false, readyFill was never cleared, and the following flush overwrote it: an entire run of RT cycles never reached the wire. The record of pending work now lives in the buffers rather than in the semaphore edge, which also removes up to UDPS_DATA_WAIT_MS of latency; genuine backpressure overwrites are counted and reported. Against the unfixed code the new test sees 2999/3000 batches never consumed. Period inflation after loss. Accumulated scalars carry no SamplingRate, so the receiver derives dt from the sender-clock gap -- but dividing it by the previous packet's sample count is only right while nothing is lost. One loss doubles the reported period, which spreads a batch a full batch past its own end and into the range the next packet claims. That is the hole and the collision, exactly. Inferring the cycle count from the estimate's own period is not a way out: it has a stable fixed point wherever gap/dt is an integer, so a real rate change locks it at the old one for good (AccumDtGTest.FollowsSustainedRateChange). The packet counter removes the ambiguity, so all three receivers now order on it: a DATA packet that does not advance the counter is dropped rather than delivered, because its values are older than data already handed over. Ordering is on the signed difference so it survives the uint32 wrap, and the sequence resets on reconnect, where the producer's counter restarts independently of ours. The loss count that falls out of the same delta feeds the period estimate as cycles = prevN * (1 + lost), which reduces exactly to gap/prevN when nothing is lost and therefore still tracks a genuine rate change. UDPSClient::AcceptDataCounter (C++), udpsprotocol.SequenceGate (Go), decode_data (C). The C client's existing gap counter was wrap-unsafe and let a stale packet rewind last_counter, which made every subsequent gap wrong; it uses the same code now. Docs/Protocol.md gains an Ordering DATA section stating the requirement for any receiver, including ones outside this repository. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Martino Ferrari
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
f334995865
commit
fbae7d712c
@@ -82,10 +82,27 @@ public:
|
||||
* update for one source is fragmented into MaxPayloadSize chunks; this is
|
||||
* the ceiling on the reassembled total, so it bounds the largest multi-
|
||||
* fragment packet the client can deliver. Sized for large array bursts
|
||||
* (e.g. 8x10000 float32 ~= 320 KiB) with headroom; stays well within the
|
||||
* 256-fragment span the recvMask[32] tracks at typical chunk sizes. */
|
||||
* (e.g. 8x10000 float32 ~= 320 KiB) with headroom. */
|
||||
static const uint32 UDPS_CLIENT_MAX_PACKET_BYTES = 1048576u; // 1 MiB
|
||||
|
||||
/** Maximum fragment count accepted for one packet. The received-fragment
|
||||
* bitmask must cover this whole span: a fragment index the mask cannot
|
||||
* represent has no duplicate detection, so a duplicated datagram counts
|
||||
* twice and the packet is delivered with a fragment still missing. */
|
||||
static const uint32 UDPS_CLIENT_MAX_FRAGMENTS = 512u;
|
||||
|
||||
/** Bytes of received-fragment bitmask (one bit per fragment). */
|
||||
static const uint32 UDPS_CLIENT_RECV_MASK_BYTES =
|
||||
UDPS_CLIENT_MAX_FRAGMENTS / 8u;
|
||||
|
||||
/** Size of the per-slot buffer that holds a last fragment which arrived
|
||||
* before the chunk size was known. Fragments larger than this cannot be
|
||||
* deferred and are dropped (the packet then fails to reassemble). */
|
||||
static const uint32 UDPS_CLIENT_PENDING_TAIL_BYTES = 8192u;
|
||||
|
||||
/** Maximum datagrams drained from the socket per Execute() iteration. */
|
||||
static const uint32 UDPS_CLIENT_MAX_DATAGRAMS_PER_CYCLE = 256u;
|
||||
|
||||
/** Default silence timeout before reconnect (seconds); sub-second values allowed. */
|
||||
static const float32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 1.0f;
|
||||
|
||||
@@ -155,6 +172,22 @@ public:
|
||||
*/
|
||||
virtual ErrorManagement::ErrorType Execute(ExecutionInfo &info);
|
||||
|
||||
/**
|
||||
* @brief DATA packets that went missing immediately before the one being
|
||||
* delivered, from the gap in the producer's packet counter.
|
||||
* @details Valid for the duration of the OnUDPSData() callback. A listener
|
||||
* that reconstructs per-sample timestamps needs this: without it, the time
|
||||
* elapsed since the previous packet looks like it covers only that
|
||||
* packet's samples, so the inferred sample period comes out too long and
|
||||
* the samples are spread past where they belong.
|
||||
*/
|
||||
uint32 GetLastDataGap() const;
|
||||
|
||||
/**
|
||||
* @brief DATA packets discarded for arriving after a newer one.
|
||||
*/
|
||||
uint32 GetStaleDataPackets() const;
|
||||
|
||||
private:
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
@@ -165,12 +198,19 @@ private:
|
||||
uint8 type; ///< UDPS_TYPE_DATA or UDPS_TYPE_CONFIG
|
||||
uint16 totalFragments; ///< Expected fragment count
|
||||
uint16 receivedFragments; ///< How many we have so far
|
||||
uint8 recvMask[32]; ///< Bitmask: bit f set iff fragment f received
|
||||
uint8 recvMask[UDPS_CLIENT_RECV_MASK_BYTES]; ///< Bit f set iff fragment f received
|
||||
uint8 payload[UDPS_CLIENT_MAX_PACKET_BYTES]; ///< Assembled payload buffer
|
||||
uint64 firstSeenTicks; ///< For GC (2 s stale detection)
|
||||
bool active; ///< Slot in use
|
||||
uint32 chunkSize; ///< Payload bytes per fragment (from first fragment)
|
||||
uint32 chunkSize; ///< Payload bytes per fragment (any non-last fragment)
|
||||
uint32 assembledBytes; ///< Exact total payload bytes placed so far
|
||||
/** Last fragment received before chunkSize was known: its offset is
|
||||
* not yet computable, so it waits here until a full-size fragment
|
||||
* reveals the chunk size. Only the last fragment can ever be short,
|
||||
* hence one deferred fragment per slot is enough. */
|
||||
uint8 pendingTail[UDPS_CLIENT_PENDING_TAIL_BYTES];
|
||||
uint32 pendingTailBytes;
|
||||
bool pendingTailValid;
|
||||
};
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
@@ -195,8 +235,39 @@ private:
|
||||
bool ReadExactTCP(uint8 *dst, uint32 n);
|
||||
/** @return true iff this fragment completed the reassembly (payload delivered). */
|
||||
bool PlaceFragment(const UDPSPacketHeader *hdr, const uint8 *payload, uint32 payloadBytes);
|
||||
/**
|
||||
* @brief Reserve a reassembly slot for (@p counter, @p type), reclaiming
|
||||
* one if none is free.
|
||||
* @return the slot index (always valid).
|
||||
*/
|
||||
uint32 AcquireReassemblySlot(uint32 counter, uint8 type);
|
||||
/**
|
||||
* @brief Account one packet abandoned with fragments missing, and report
|
||||
* it at most once per second.
|
||||
* @details Fragment loss on a busy stream is chronic, not exceptional: an
|
||||
* unconditional message per drop buries every other log line.
|
||||
*/
|
||||
void NoteDroppedIncomplete(uint32 counter);
|
||||
void GcReassemblySlots();
|
||||
void DeliverAssembled(UDPSReassemblySlot &slot);
|
||||
/**
|
||||
* @brief Sequence gate for DATA packets, applied just before delivery.
|
||||
* @details The producer numbers DATA packets consecutively, so the counter
|
||||
* reveals both how many packets went missing and whether this one is late.
|
||||
* A late packet must not be delivered: its samples predate what the
|
||||
* listener has already stored, so they land behind the current write
|
||||
* position and collide with data that is already there — which is what a
|
||||
* consumer sees as two signals occupying the same instant. Reassembly
|
||||
* completes in arrival order, not counter order, so this ordering is not
|
||||
* guaranteed upstream.
|
||||
*
|
||||
* Also records the number of packets missing immediately before this one,
|
||||
* for GetLastDataGap().
|
||||
*
|
||||
* @param counter The candidate packet's UDPS counter.
|
||||
* @return true if the packet is newer than the last delivered one.
|
||||
*/
|
||||
bool AcceptDataCounter(uint32 counter);
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Configuration
|
||||
@@ -235,7 +306,15 @@ private:
|
||||
|
||||
// Reassembly
|
||||
UDPSReassemblySlot reassemblySlots[UDPS_CLIENT_MAX_REASSEMBLY_SLOTS];
|
||||
uint64 lastGcTicks; ///< Ticks at last GC run
|
||||
uint64 lastGcTicks; ///< Ticks at last GC run
|
||||
uint64 lastDropWarnTicks;///< Ticks at last incomplete-packet report
|
||||
uint32 droppedSinceWarn; ///< Incomplete packets since that report
|
||||
|
||||
// DATA sequencing (see AcceptDataCounter)
|
||||
uint32 lastDataCounter; ///< Counter of the last delivered DATA packet
|
||||
bool lastDataCounterValid; ///< False until the first DATA packet
|
||||
uint32 lastDataGap; ///< Packets missing before the current one
|
||||
uint32 staleDataPackets; ///< DATA packets discarded as late
|
||||
|
||||
// Receive scratch buffer
|
||||
uint8 recvBuf[65535u + UDPS_HEADER_SIZE];
|
||||
|
||||
Reference in New Issue
Block a user