Martino FerrariandClaude Opus 4.6
fbae7d712c
Reported as samples sporadically carrying a previous packet's timestamp: holes on one side of the stream and collisions on the other, in both the Go and the MARTe2 receiver. That it appeared in both is what located it -- the shared cause is upstream of either client. Four independent defects, all of which end in a packet's values being placed at a time that is not theirs. Reassembly slot exhaustion (the "Reassembly slots full; evicting oldest" flood). Chunk size was learnt only from fragment 0, so an out-of-order burst destroyed a packet whose bytes had all arrived and left the slot occupied until the 2 s GC. Slots were keyed on the counter alone, but DATA and CONFIG number independently, so equal counters merged the two streams. The 32-byte received-mask covered 256 of the 512 fragments the client accepts, so a duplicate above 255 was counted as new and the packet was delivered with a hole of stale bytes in it. And one datagram was read per Execute(), which cannot drain a fast producer. Fixed with a pendingTail deferral, (counter, type) keying, a 64-byte mask, a 256-datagram drain, counter-age slot reclamation, and a 1 Hz aggregated warning in place of the per-eviction flood. UDPStreamer dropping whole Accumulate batches. EventSem::ResetWait is Reset-then-Wait, so a Post() landing while the sender thread was inside ServiceClients()/SendData() was destroyed by the next Reset. The batch was then skipped with dataReady false, readyFill was never cleared, and the following flush overwrote it: an entire run of RT cycles never reached the wire. The record of pending work now lives in the buffers rather than in the semaphore edge, which also removes up to UDPS_DATA_WAIT_MS of latency; genuine backpressure overwrites are counted and reported. Against the unfixed code the new test sees 2999/3000 batches never consumed. Period inflation after loss. Accumulated scalars carry no SamplingRate, so the receiver derives dt from the sender-clock gap -- but dividing it by the previous packet's sample count is only right while nothing is lost. One loss doubles the reported period, which spreads a batch a full batch past its own end and into the range the next packet claims. That is the hole and the collision, exactly. Inferring the cycle count from the estimate's own period is not a way out: it has a stable fixed point wherever gap/dt is an integer, so a real rate change locks it at the old one for good (AccumDtGTest.FollowsSustainedRateChange). The packet counter removes the ambiguity, so all three receivers now order on it: a DATA packet that does not advance the counter is dropped rather than delivered, because its values are older than data already handed over. Ordering is on the signed difference so it survives the uint32 wrap, and the sequence resets on reconnect, where the producer's counter restarts independently of ours. The loss count that falls out of the same delta feeds the period estimate as cycles = prevN * (1 + lost), which reduces exactly to gap/prevN when nothing is lost and therefore still tracks a genuine rate change. UDPSClient::AcceptDataCounter (C++), udpsprotocol.SequenceGate (Go), decode_data (C). The C client's existing gap counter was wrap-unsafe and let a stale packet rewind last_counter, which made every subsequent gap wrong; it uses the same code now. Docs/Protocol.md gains an Ordering DATA section stating the requirement for any receiver, including ones outside this repository. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
326 lines
14 KiB
C++
326 lines
14 KiB
C++
#ifndef UDPS_CLIENT_H_
|
|
#define UDPS_CLIENT_H_
|
|
|
|
/**
|
|
* @file UDPSClient.h
|
|
* @brief Auto-reconnecting UDPS receiver client (C++ MARTe2 library class).
|
|
*
|
|
* UDPSClient runs its own background thread (via SingleThreadService).
|
|
* On start it connects to a UDPSServer, receives CONFIG + DATA packets, and
|
|
* reassembles fragmented updates. If the server goes silent for longer than
|
|
* SilenceTimeout, it automatically disconnects and retries.
|
|
*
|
|
* Unicast mode: sends CONNECT to server UDP port; receives on an ephemeral UDP port.
|
|
* Multicast mode: TCP to server port for CONFIG; joins UDP multicast group for DATA.
|
|
*
|
|
* Threading: UDPSClientListener callbacks are invoked from the internal receive
|
|
* thread — the implementation must be thread-safe with respect to the caller.
|
|
*/
|
|
|
|
#include "BasicTCPSocket.h"
|
|
#include "BasicUDPSocket.h"
|
|
#include "EmbeddedServiceMethodBinderI.h"
|
|
#include "ExecutionInfo.h"
|
|
#include "HighResolutionTimer.h"
|
|
#include "InternetHost.h"
|
|
#include "SingleThreadService.h"
|
|
#include "StreamString.h"
|
|
#include "StructuredDataI.h"
|
|
#include "UDPSProtocol.h"
|
|
|
|
namespace MARTe {
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Listener interface
|
|
// ---------------------------------------------------------------------------
|
|
|
|
/**
|
|
* @brief Callback interface for UDPSClient events.
|
|
*
|
|
* Implement this interface and pass an instance to UDPSClient::SetListener().
|
|
* All callbacks are invoked from the UDPSClient internal thread.
|
|
*/
|
|
class UDPSClientListener {
|
|
public:
|
|
virtual ~UDPSClientListener() {}
|
|
|
|
/** Called when a fully-reassembled CONFIG payload is available. */
|
|
virtual void OnUDPSConfig(const uint8 *payload, uint32 payloadSize) {}
|
|
|
|
/** Called when a fully-reassembled DATA payload is available. */
|
|
virtual void OnUDPSData(const uint8 *payload, uint32 payloadSize) {}
|
|
|
|
/**
|
|
* @brief Called for every received DATA datagram (fragment).
|
|
* @param counter DATA packet counter from the UDPS header.
|
|
* @param nBytes Raw datagram size (header + payload).
|
|
* @param complete True iff this fragment completed the DATA reassembly.
|
|
*/
|
|
virtual void OnUDPSFragment(uint32 counter, uint32 nBytes, bool complete) {}
|
|
|
|
/** Called when the connection to the server has been established. */
|
|
virtual void OnUDPSConnected() {}
|
|
|
|
/** Called when the connection has been lost or closed. */
|
|
virtual void OnUDPSDisconnected() {}
|
|
};
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// UDPSClient
|
|
// ---------------------------------------------------------------------------
|
|
|
|
/**
|
|
* @brief UDPS receiver client with auto-reconnect and fragment reassembly.
|
|
*/
|
|
class UDPSClient : public EmbeddedServiceMethodBinderI {
|
|
public:
|
|
|
|
/** Maximum number of concurrent in-flight reassembly slots. */
|
|
static const uint32 UDPS_CLIENT_MAX_REASSEMBLY_SLOTS = 4u;
|
|
|
|
/** Maximum size (bytes) of a single reassembled packet payload. A UDPS
|
|
* update for one source is fragmented into MaxPayloadSize chunks; this is
|
|
* the ceiling on the reassembled total, so it bounds the largest multi-
|
|
* fragment packet the client can deliver. Sized for large array bursts
|
|
* (e.g. 8x10000 float32 ~= 320 KiB) with headroom. */
|
|
static const uint32 UDPS_CLIENT_MAX_PACKET_BYTES = 1048576u; // 1 MiB
|
|
|
|
/** Maximum fragment count accepted for one packet. The received-fragment
|
|
* bitmask must cover this whole span: a fragment index the mask cannot
|
|
* represent has no duplicate detection, so a duplicated datagram counts
|
|
* twice and the packet is delivered with a fragment still missing. */
|
|
static const uint32 UDPS_CLIENT_MAX_FRAGMENTS = 512u;
|
|
|
|
/** Bytes of received-fragment bitmask (one bit per fragment). */
|
|
static const uint32 UDPS_CLIENT_RECV_MASK_BYTES =
|
|
UDPS_CLIENT_MAX_FRAGMENTS / 8u;
|
|
|
|
/** Size of the per-slot buffer that holds a last fragment which arrived
|
|
* before the chunk size was known. Fragments larger than this cannot be
|
|
* deferred and are dropped (the packet then fails to reassemble). */
|
|
static const uint32 UDPS_CLIENT_PENDING_TAIL_BYTES = 8192u;
|
|
|
|
/** Maximum datagrams drained from the socket per Execute() iteration. */
|
|
static const uint32 UDPS_CLIENT_MAX_DATAGRAMS_PER_CYCLE = 256u;
|
|
|
|
/** Default silence timeout before reconnect (seconds); sub-second values allowed. */
|
|
static const float32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 1.0f;
|
|
|
|
/** Default delay between reconnect attempts (seconds). */
|
|
static const uint32 UDPS_CLIENT_DEFAULT_RECONNECT_DELAY_S = 2u;
|
|
|
|
/** Default unicast keepalive interval (seconds). UDPSServer evicts silent
|
|
* unicast clients after its ClientTimeout (default 30 s); the client
|
|
* re-sends an ACK on this interval to stay registered. 0 disables. */
|
|
static const uint32 UDPS_CLIENT_DEFAULT_KEEPALIVE_INTERVAL_S = 15u;
|
|
|
|
/** Default maximum payload size (bytes, excluding 17-byte header). */
|
|
static const uint32 UDPS_CLIENT_DEFAULT_MAX_PAYLOAD = 1400u;
|
|
|
|
/** Default OS UDP receive socket buffer size (bytes). The Linux default
|
|
* (rmem_default, typically ~208 KiB) is easily overrun by high-throughput
|
|
* sources (e.g. multi-hundred-KiB bursts every few ms), causing silent
|
|
* kernel-level datagram drops. 4 MiB gives generous burst headroom. */
|
|
static const uint32 UDPS_CLIENT_DEFAULT_RECV_BUFFER = 4194304u; // 4 MiB
|
|
|
|
UDPSClient();
|
|
virtual ~UDPSClient();
|
|
|
|
/**
|
|
* @brief Read configuration from a StructuredDataI node.
|
|
*
|
|
* Expected keys:
|
|
* - ServerAddr (char*) Server IPv4 address. Required.
|
|
* - Port (uint16) Server UDP port (unicast) or TCP listen port (multicast). Required.
|
|
* - MulticastGroup (char*) IPv4 multicast address; presence enables multicast mode.
|
|
* - Interface (char*) Local IPv4 dotted-quad address (e.g. "127.0.0.1") of the interface on
|
|
* which to join the multicast group. Optional; omitting it uses the
|
|
* default-route interface (INADDR_ANY), which silently receives nothing
|
|
* if the server sends on a different interface.
|
|
* - DataPort (uint16) UDP multicast data port (defaults to Port+1).
|
|
* - SilenceTimeout (float32) Seconds of no data before reconnect. Default 1.0.
|
|
* Sub-second values allowed; 0 disables the check.
|
|
* - ReconnectDelay (uint32) Seconds to wait between reconnect attempts. Default 2.
|
|
* - KeepAliveInterval (uint32) Seconds between unicast keepalive ACKs. Default 15. 0 disables.
|
|
* - MaxPayloadSize (uint32) Max payload bytes per datagram, excluding header. Default 1400.
|
|
* - CPUMask (uint32) CPU affinity mask for the receive thread. Default 0xFFFFFFFF.
|
|
* - StackSize (uint32) Stack size for the receive thread. Default 65536.
|
|
* - RecvBufferSize (uint32) OS UDP receive socket buffer size (bytes). Default 4 MiB.
|
|
*/
|
|
bool Initialise(StructuredDataI &data);
|
|
|
|
/**
|
|
* @brief Register the event listener. Must be called before Start().
|
|
*/
|
|
void SetListener(UDPSClientListener *listener);
|
|
|
|
/**
|
|
* @brief Start the receive thread.
|
|
* @return true on success.
|
|
*/
|
|
bool Start();
|
|
|
|
/**
|
|
* @brief Stop the receive thread and close all sockets.
|
|
* @return true on success.
|
|
*/
|
|
bool Stop();
|
|
|
|
/**
|
|
* @brief Internal thread entry point (EmbeddedServiceMethodBinderI).
|
|
* @details Do NOT call directly.
|
|
*/
|
|
virtual ErrorManagement::ErrorType Execute(ExecutionInfo &info);
|
|
|
|
/**
|
|
* @brief DATA packets that went missing immediately before the one being
|
|
* delivered, from the gap in the producer's packet counter.
|
|
* @details Valid for the duration of the OnUDPSData() callback. A listener
|
|
* that reconstructs per-sample timestamps needs this: without it, the time
|
|
* elapsed since the previous packet looks like it covers only that
|
|
* packet's samples, so the inferred sample period comes out too long and
|
|
* the samples are spread past where they belong.
|
|
*/
|
|
uint32 GetLastDataGap() const;
|
|
|
|
/**
|
|
* @brief DATA packets discarded for arriving after a newer one.
|
|
*/
|
|
uint32 GetStaleDataPackets() const;
|
|
|
|
private:
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Fragment reassembly slot
|
|
// -------------------------------------------------------------------------
|
|
struct UDPSReassemblySlot {
|
|
uint32 counter; ///< Packet counter this slot belongs to
|
|
uint8 type; ///< UDPS_TYPE_DATA or UDPS_TYPE_CONFIG
|
|
uint16 totalFragments; ///< Expected fragment count
|
|
uint16 receivedFragments; ///< How many we have so far
|
|
uint8 recvMask[UDPS_CLIENT_RECV_MASK_BYTES]; ///< Bit f set iff fragment f received
|
|
uint8 payload[UDPS_CLIENT_MAX_PACKET_BYTES]; ///< Assembled payload buffer
|
|
uint64 firstSeenTicks; ///< For GC (2 s stale detection)
|
|
bool active; ///< Slot in use
|
|
uint32 chunkSize; ///< Payload bytes per fragment (any non-last fragment)
|
|
uint32 assembledBytes; ///< Exact total payload bytes placed so far
|
|
/** Last fragment received before chunkSize was known: its offset is
|
|
* not yet computable, so it waits here until a full-size fragment
|
|
* reveals the chunk size. Only the last fragment can ever be short,
|
|
* hence one deferred fragment per slot is enough. */
|
|
uint8 pendingTail[UDPS_CLIENT_PENDING_TAIL_BYTES];
|
|
uint32 pendingTailBytes;
|
|
bool pendingTailValid;
|
|
};
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Connection state machine helpers
|
|
// -------------------------------------------------------------------------
|
|
bool Connect();
|
|
void Disconnect();
|
|
/** Send a keepalive ACK to the server (unicast only, same socket). */
|
|
void SendKeepAlive();
|
|
bool ReceiveAndProcess();
|
|
|
|
bool ConnectUnicast();
|
|
bool ConnectMulticast();
|
|
|
|
/** Set the OS receive buffer size (SO_RCVBUF) on a UDP socket's raw handle. */
|
|
void SetRecvBufferSize(BasicUDPSocket &sock);
|
|
|
|
void ProcessDatagram(const uint8 *buf, uint32 size);
|
|
/** Read one full UDPS frame (header + payload) from the TCP control socket. */
|
|
bool ReceiveTCPFrame();
|
|
/** Read exactly n bytes from the TCP control socket. */
|
|
bool ReadExactTCP(uint8 *dst, uint32 n);
|
|
/** @return true iff this fragment completed the reassembly (payload delivered). */
|
|
bool PlaceFragment(const UDPSPacketHeader *hdr, const uint8 *payload, uint32 payloadBytes);
|
|
/**
|
|
* @brief Reserve a reassembly slot for (@p counter, @p type), reclaiming
|
|
* one if none is free.
|
|
* @return the slot index (always valid).
|
|
*/
|
|
uint32 AcquireReassemblySlot(uint32 counter, uint8 type);
|
|
/**
|
|
* @brief Account one packet abandoned with fragments missing, and report
|
|
* it at most once per second.
|
|
* @details Fragment loss on a busy stream is chronic, not exceptional: an
|
|
* unconditional message per drop buries every other log line.
|
|
*/
|
|
void NoteDroppedIncomplete(uint32 counter);
|
|
void GcReassemblySlots();
|
|
void DeliverAssembled(UDPSReassemblySlot &slot);
|
|
/**
|
|
* @brief Sequence gate for DATA packets, applied just before delivery.
|
|
* @details The producer numbers DATA packets consecutively, so the counter
|
|
* reveals both how many packets went missing and whether this one is late.
|
|
* A late packet must not be delivered: its samples predate what the
|
|
* listener has already stored, so they land behind the current write
|
|
* position and collide with data that is already there — which is what a
|
|
* consumer sees as two signals occupying the same instant. Reassembly
|
|
* completes in arrival order, not counter order, so this ordering is not
|
|
* guaranteed upstream.
|
|
*
|
|
* Also records the number of packets missing immediately before this one,
|
|
* for GetLastDataGap().
|
|
*
|
|
* @param counter The candidate packet's UDPS counter.
|
|
* @return true if the packet is newer than the last delivered one.
|
|
*/
|
|
bool AcceptDataCounter(uint32 counter);
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Configuration
|
|
// -------------------------------------------------------------------------
|
|
StreamString serverAddr;
|
|
uint16 serverPort;
|
|
StreamString multicastGroup;
|
|
StreamString multicastInterface;
|
|
uint16 dataPort;
|
|
bool useMulticast;
|
|
uint64 silenceTimeoutTicks;
|
|
uint64 reconnectDelayTicks;
|
|
uint64 keepAliveIntervalTicks; ///< 0 = keepalive disabled
|
|
uint32 maxPayloadSize;
|
|
uint32 cpuMask;
|
|
uint32 stackSize;
|
|
uint32 recvBufferSize;
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Runtime state
|
|
// -------------------------------------------------------------------------
|
|
UDPSClientListener *listener;
|
|
SingleThreadService threadService;
|
|
bool connected;
|
|
uint64 lastDataTicks; ///< Ticks at last received DATA/CONFIG
|
|
uint64 disconnectTick; ///< Ticks when we disconnected (for delay)
|
|
uint64 lastKeepAliveTicks; ///< Ticks at last keepalive ACK sent
|
|
|
|
// Unicast
|
|
BasicUDPSocket recvSocket; ///< Bound to ephemeral port; receives DATA
|
|
uint16 localPort; ///< Ephemeral port we're listening on
|
|
|
|
// Multicast
|
|
BasicTCPSocket tcpSocket; ///< TCP connection to server (for CONNECT + CONFIG)
|
|
BasicUDPSocket mcastSocket; ///< Joined multicast group
|
|
|
|
// Reassembly
|
|
UDPSReassemblySlot reassemblySlots[UDPS_CLIENT_MAX_REASSEMBLY_SLOTS];
|
|
uint64 lastGcTicks; ///< Ticks at last GC run
|
|
uint64 lastDropWarnTicks;///< Ticks at last incomplete-packet report
|
|
uint32 droppedSinceWarn; ///< Incomplete packets since that report
|
|
|
|
// DATA sequencing (see AcceptDataCounter)
|
|
uint32 lastDataCounter; ///< Counter of the last delivered DATA packet
|
|
bool lastDataCounterValid; ///< False until the first DATA packet
|
|
uint32 lastDataGap; ///< Packets missing before the current one
|
|
uint32 staleDataPackets; ///< DATA packets discarded as late
|
|
|
|
// Receive scratch buffer
|
|
uint8 recvBuf[65535u + UDPS_HEADER_SIZE];
|
|
};
|
|
|
|
} // namespace MARTe
|
|
|
|
#endif // UDPS_CLIENT_H_
|