Reported as samples sporadically carrying a previous packet's timestamp: holes on one side of the stream and collisions on the other, in both the Go and the MARTe2 receiver. That it appeared in both is what located it -- the shared cause is upstream of either client. Four independent defects, all of which end in a packet's values being placed at a time that is not theirs. Reassembly slot exhaustion (the "Reassembly slots full; evicting oldest" flood). Chunk size was learnt only from fragment 0, so an out-of-order burst destroyed a packet whose bytes had all arrived and left the slot occupied until the 2 s GC. Slots were keyed on the counter alone, but DATA and CONFIG number independently, so equal counters merged the two streams. The 32-byte received-mask covered 256 of the 512 fragments the client accepts, so a duplicate above 255 was counted as new and the packet was delivered with a hole of stale bytes in it. And one datagram was read per Execute(), which cannot drain a fast producer. Fixed with a pendingTail deferral, (counter, type) keying, a 64-byte mask, a 256-datagram drain, counter-age slot reclamation, and a 1 Hz aggregated warning in place of the per-eviction flood. UDPStreamer dropping whole Accumulate batches. EventSem::ResetWait is Reset-then-Wait, so a Post() landing while the sender thread was inside ServiceClients()/SendData() was destroyed by the next Reset. The batch was then skipped with dataReady false, readyFill was never cleared, and the following flush overwrote it: an entire run of RT cycles never reached the wire. The record of pending work now lives in the buffers rather than in the semaphore edge, which also removes up to UDPS_DATA_WAIT_MS of latency; genuine backpressure overwrites are counted and reported. Against the unfixed code the new test sees 2999/3000 batches never consumed. Period inflation after loss. Accumulated scalars carry no SamplingRate, so the receiver derives dt from the sender-clock gap -- but dividing it by the previous packet's sample count is only right while nothing is lost. One loss doubles the reported period, which spreads a batch a full batch past its own end and into the range the next packet claims. That is the hole and the collision, exactly. Inferring the cycle count from the estimate's own period is not a way out: it has a stable fixed point wherever gap/dt is an integer, so a real rate change locks it at the old one for good (AccumDtGTest.FollowsSustainedRateChange). The packet counter removes the ambiguity, so all three receivers now order on it: a DATA packet that does not advance the counter is dropped rather than delivered, because its values are older than data already handed over. Ordering is on the signed difference so it survives the uint32 wrap, and the sequence resets on reconnect, where the producer's counter restarts independently of ours. The loss count that falls out of the same delta feeds the period estimate as cycles = prevN * (1 + lost), which reduces exactly to gap/prevN when nothing is lost and therefore still tracks a genuine rate change. UDPSClient::AcceptDataCounter (C++), udpsprotocol.SequenceGate (Go), decode_data (C). The C client's existing gap counter was wrap-unsafe and let a stale packet rewind last_counter, which made every subsequent gap wrong; it uses the same code now. Docs/Protocol.md gains an Ordering DATA section stating the requirement for any receiver, including ones outside this repository. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
341 lines
14 KiB
C
341 lines
14 KiB
C
#ifndef UDPS_CLIENT_H
|
|
#define UDPS_CLIENT_H
|
|
|
|
/**
|
|
* @file udps_client.h
|
|
* @brief Standalone UDPS (UDPStreamer) receiver library — C99, no MARTe2.
|
|
*
|
|
* Depends only on libc and BSD sockets, so it can be dropped into any C or C++
|
|
* program that needs to consume a UDPStreamer / DebugService stream. The wire
|
|
* format is specified in Docs/Protocol.md; the library reference (and a worked
|
|
* example) is Docs/UDPS-C-Client.md.
|
|
*
|
|
* Usage in one paragraph: fill a udps_client_config_t, create a client, install
|
|
* callbacks, then call udps_client_poll() in a loop. The client owns the
|
|
* connection state machine — it sends CONNECT, reassembles fragmented packets,
|
|
* decodes CONFIG and DATA, sends keepalives, and reconnects when the server
|
|
* goes silent. Nothing is done behind your back: no threads are created and
|
|
* every callback runs inside your call to udps_client_poll().
|
|
*
|
|
* Threading: a udps_client_t must be used from one thread at a time.
|
|
*/
|
|
|
|
#include <stddef.h>
|
|
#include <stdint.h>
|
|
|
|
#ifdef __cplusplus
|
|
extern "C" {
|
|
#endif
|
|
|
|
/*---------------------------------------------------------------------------*/
|
|
/* Protocol constants */
|
|
/*---------------------------------------------------------------------------*/
|
|
|
|
/** Magic number: ASCII 'UDPS' stored little-endian. */
|
|
#define UDPS_MAGIC 0x53504455u
|
|
|
|
/** Size of the packed packet header on the wire. */
|
|
#define UDPS_HEADER_SIZE 17u
|
|
|
|
/** Size of one serialised signal descriptor in a CONFIG payload. */
|
|
#define UDPS_SIGNAL_DESC_SIZE 136u
|
|
|
|
/** Value of udps_signal_t::time_signal_idx when the signal has no time reference. */
|
|
#define UDPS_NO_TIME_SIGNAL 0xFFFFFFFFu
|
|
|
|
/** Upper bound on elements per signal; larger descriptors are rejected. */
|
|
#define UDPS_MAX_ELEMENTS (1u << 20)
|
|
|
|
/** Packet types (udps_header_t::type). */
|
|
enum {
|
|
UDPS_PKT_DATA = 0, /**< Server -> client: signal samples. */
|
|
UDPS_PKT_CONFIG = 1, /**< Server -> client: signal metadata. */
|
|
UDPS_PKT_ACK = 2, /**< Client -> server: keepalive. */
|
|
UDPS_PKT_CONNECT = 3, /**< Client -> server: open a session. */
|
|
UDPS_PKT_DISCONNECT = 4 /**< Either direction: close a session. */
|
|
};
|
|
|
|
/** Sample type codes (udps_signal_t::type_code). */
|
|
enum {
|
|
UDPS_T_UINT8 = 0,
|
|
UDPS_T_INT8 = 1,
|
|
UDPS_T_UINT16 = 2,
|
|
UDPS_T_INT16 = 3,
|
|
UDPS_T_UINT32 = 4,
|
|
UDPS_T_INT32 = 5,
|
|
UDPS_T_UINT64 = 6,
|
|
UDPS_T_INT64 = 7,
|
|
UDPS_T_FLOAT32 = 8,
|
|
UDPS_T_FLOAT64 = 9,
|
|
UDPS_T_UNKNOWN = 255
|
|
};
|
|
|
|
/** Quantisation codes (udps_signal_t::quant_type). */
|
|
enum {
|
|
UDPS_QUANT_NONE = 0, /**< Raw values in the signal's own type. */
|
|
UDPS_QUANT_UINT8 = 1, /**< [range_min, range_max] mapped onto uint8. */
|
|
UDPS_QUANT_INT8 = 2,
|
|
UDPS_QUANT_UINT16 = 3,
|
|
UDPS_QUANT_INT16 = 4
|
|
};
|
|
|
|
/** Time-reference modes (udps_signal_t::time_mode). */
|
|
enum {
|
|
UDPS_TIME_PACKET = 0, /**< No per-element time; use packet arrival. */
|
|
UDPS_TIME_FULL_ARRAY = 1, /**< The time signal carries one stamp per element. */
|
|
UDPS_TIME_FIRST_SAMPLE = 2, /**< Time signal (scalar) stamps element 0. */
|
|
UDPS_TIME_LAST_SAMPLE = 3 /**< Time signal (scalar) stamps element N-1. */
|
|
};
|
|
|
|
/** Publishing modes (udps_frame_t::publish_mode). */
|
|
enum {
|
|
UDPS_PUBLISH_STRICT = 0, /**< One packet per RT cycle. */
|
|
UDPS_PUBLISH_ACCUMULATE = 1, /**< A batch of cycles per packet. */
|
|
UDPS_PUBLISH_DECIMATE = 2 /**< One packet every N cycles. */
|
|
};
|
|
|
|
/*---------------------------------------------------------------------------*/
|
|
/* Data model */
|
|
/*---------------------------------------------------------------------------*/
|
|
|
|
/** Decoded 17-byte packet header. */
|
|
typedef struct {
|
|
uint32_t magic;
|
|
uint8_t type;
|
|
uint32_t counter; /**< Same for every fragment of one update. */
|
|
uint16_t fragment_idx;
|
|
uint16_t total_fragments; /**< 1 when the update fits in one datagram. */
|
|
uint32_t payload_bytes;
|
|
} udps_header_t;
|
|
|
|
/** Metadata for one streamed signal, as carried by the CONFIG payload. */
|
|
typedef struct {
|
|
char name[65]; /**< NUL-terminated. */
|
|
uint8_t type_code; /**< UDPS_T_*. */
|
|
uint8_t quant_type; /**< UDPS_QUANT_*. */
|
|
uint8_t num_dimensions; /**< 0 scalar, 1 vector, 2 matrix. */
|
|
uint32_t num_rows;
|
|
uint32_t num_cols;
|
|
double range_min; /**< Physical range, used to dequantise. */
|
|
double range_max;
|
|
uint8_t time_mode; /**< UDPS_TIME_*. */
|
|
double sampling_rate; /**< Hz; 0 when unknown. */
|
|
uint32_t time_signal_idx;/**< Index into the signal list, or UDPS_NO_TIME_SIGNAL. */
|
|
char unit[33]; /**< NUL-terminated. */
|
|
} udps_signal_t;
|
|
|
|
/**
|
|
* @brief Decoded values of one signal within a frame.
|
|
*
|
|
* Values are always physical doubles: quantised signals are already expanded
|
|
* back onto [range_min, range_max]. @c count is @c num_samples for a scalar
|
|
* signal in Accumulate mode (one value per batched cycle) and the signal's
|
|
* element count in every other case.
|
|
*/
|
|
typedef struct {
|
|
const double *values;
|
|
uint32_t count;
|
|
} udps_signal_values_t;
|
|
|
|
/** One fully decoded DATA packet. */
|
|
typedef struct {
|
|
uint32_t counter; /**< Packet counter; gaps mean lost datagrams. */
|
|
/**
|
|
* DATA packets missing immediately before this one, from the counter.
|
|
*
|
|
* Needed to space samples correctly: the elapsed time since the previous
|
|
* frame covers the lost packets' cycles too, so dividing it by this
|
|
* frame's sample count alone gives a period too long by exactly
|
|
* @c lost + 1, which walks the samples past their own end and into the
|
|
* range the next frame claims.
|
|
*/
|
|
uint32_t lost;
|
|
uint64_t hrt; /**< Producer's high-resolution timer at send. */
|
|
double recv_time; /**< Wall-clock seconds (CLOCK_REALTIME) at arrival. */
|
|
uint8_t publish_mode; /**< UDPS_PUBLISH_*. */
|
|
uint32_t num_samples; /**< Batched cycles; 1 unless Accumulate. */
|
|
uint32_t num_signals;
|
|
const udps_signal_t *signals; /**< num_signals entries, CONFIG order. */
|
|
const udps_signal_values_t *values; /**< num_signals entries, same order. */
|
|
} udps_frame_t;
|
|
|
|
/** Connection lifecycle events reported through udps_event_cb. */
|
|
typedef enum {
|
|
UDPS_EVENT_CONNECTED, /**< Sockets are up and CONNECT was sent. */
|
|
UDPS_EVENT_DISCONNECTED, /**< Session dropped; the client will retry. */
|
|
UDPS_EVENT_ERROR /**< Recoverable problem; detail says what. */
|
|
} udps_event_t;
|
|
|
|
/** Cumulative counters, never reset. */
|
|
typedef struct {
|
|
uint64_t packets_received; /**< Datagrams (and TCP frames) accepted. */
|
|
uint64_t bytes_received;
|
|
uint64_t frames_delivered; /**< DATA packets decoded and handed to you. */
|
|
uint64_t config_updates;
|
|
uint64_t fragments_dropped; /**< Duplicate, stale or unplaceable fragments. */
|
|
uint64_t counter_gaps; /**< DATA packets missing from the sequence. */
|
|
/**
|
|
* DATA packets dropped for not advancing the counter: reordered or
|
|
* duplicated on the wire. Delivering one would stamp its values with a
|
|
* time base older than data already handed over.
|
|
*/
|
|
uint64_t stale_packets;
|
|
uint64_t reconnects;
|
|
} udps_stats_t;
|
|
|
|
/*---------------------------------------------------------------------------*/
|
|
/* Client */
|
|
/*---------------------------------------------------------------------------*/
|
|
|
|
typedef struct udps_client udps_client_t;
|
|
|
|
/** Called whenever a CONFIG packet redefines the signal set. */
|
|
typedef void (*udps_config_cb)(const udps_signal_t *signals,
|
|
uint32_t num_signals,
|
|
uint8_t publish_mode,
|
|
void *user);
|
|
|
|
/**
|
|
* @brief Called for every decoded DATA packet.
|
|
*
|
|
* The frame and everything it points at are owned by the client and are only
|
|
* valid until the callback returns — copy anything you need to keep.
|
|
*/
|
|
typedef void (*udps_data_cb)(const udps_frame_t *frame, void *user);
|
|
|
|
/** Called on connection state changes and on recoverable errors. */
|
|
typedef void (*udps_event_cb)(udps_event_t event, const char *detail, void *user);
|
|
|
|
/**
|
|
* @brief Transport configuration.
|
|
*
|
|
* Zero-initialise with udps_client_config_init(), then override what you need.
|
|
* Set @c multicast_group to switch from unicast to multicast: in unicast the
|
|
* client sends CONNECT over UDP and receives everything on its ephemeral port;
|
|
* in multicast it joins the group for DATA and opens a TCP control connection
|
|
* to @c server_port for CONNECT and CONFIG.
|
|
*/
|
|
typedef struct {
|
|
const char *server_addr; /**< IPv4 dotted quad. Required. */
|
|
uint16_t server_port; /**< UDP port (unicast) or TCP port (multicast). Required. */
|
|
const char *multicast_group;/**< IPv4 group; NULL selects unicast. */
|
|
const char *interface_addr; /**< Local IPv4 of the interface to join on. NULL = default route. */
|
|
uint16_t data_port; /**< Multicast data port; 0 means server_port + 1. */
|
|
double silence_timeout_s; /**< Reconnect after this long without data. 0 disables. */
|
|
double reconnect_delay_s; /**< Wait between reconnect attempts. */
|
|
double keepalive_interval_s;/**< Unicast ACK period. 0 disables. */
|
|
uint32_t recv_buffer_bytes; /**< SO_RCVBUF; large bursts need a large value. */
|
|
uint32_t max_packet_bytes; /**< Ceiling on one reassembled payload. */
|
|
} udps_client_config_t;
|
|
|
|
/** Fills @p cfg with the defaults documented in Docs/UDPS-C-Client.md. */
|
|
void udps_client_config_init(udps_client_config_t *cfg);
|
|
|
|
/**
|
|
* @brief Creates a client. No socket is opened until the first poll.
|
|
* @return NULL if @p cfg is invalid or memory ran out.
|
|
*/
|
|
udps_client_t *udps_client_create(const udps_client_config_t *cfg);
|
|
|
|
/** Closes the session (sending DISCONNECT if connected) and frees the client. */
|
|
void udps_client_destroy(udps_client_t *client);
|
|
|
|
/** Installs the callbacks. Any of them may be NULL. */
|
|
void udps_client_set_callbacks(udps_client_t *client,
|
|
udps_config_cb on_config,
|
|
udps_data_cb on_data,
|
|
udps_event_cb on_event,
|
|
void *user);
|
|
|
|
/**
|
|
* @brief Drives the client: connects if needed, then waits for and processes
|
|
* packets for at most @p timeout_ms milliseconds.
|
|
*
|
|
* Callbacks fire from inside this call. A negative @p timeout_ms blocks until
|
|
* something happens. Call it in a loop; it is the only function that does work.
|
|
*
|
|
* @return the number of packets processed (0 on timeout), or -1 if the session
|
|
* broke. -1 is not fatal: the next call retries after reconnect_delay_s.
|
|
*/
|
|
int udps_client_poll(udps_client_t *client, int timeout_ms);
|
|
|
|
/** Non-zero once the sockets are up (which does not yet imply CONFIG arrived). */
|
|
int udps_client_is_connected(const udps_client_t *client);
|
|
|
|
/**
|
|
* @brief The current signal set, or NULL before the first CONFIG.
|
|
* @param num_signals Out; may be NULL.
|
|
*/
|
|
const udps_signal_t *udps_client_signals(const udps_client_t *client,
|
|
uint32_t *num_signals);
|
|
|
|
/** The publishing mode from the last CONFIG (UDPS_PUBLISH_*). */
|
|
uint8_t udps_client_publish_mode(const udps_client_t *client);
|
|
|
|
/** Copies the counters into @p out. */
|
|
void udps_client_stats(const udps_client_t *client, udps_stats_t *out);
|
|
|
|
/** Human-readable description of the last failure. Never NULL. */
|
|
const char *udps_client_last_error(const udps_client_t *client);
|
|
|
|
/*---------------------------------------------------------------------------*/
|
|
/* Stateless helpers */
|
|
/*---------------------------------------------------------------------------*/
|
|
|
|
/** Elements in one sample of @p signal (rows x cols, at least 1). */
|
|
uint32_t udps_signal_num_elements(const udps_signal_t *signal);
|
|
|
|
/** Short name of a type code, e.g. "float32". Never NULL. */
|
|
const char *udps_type_name(uint8_t type_code);
|
|
|
|
/**
|
|
* @brief Decodes a packet header.
|
|
* @return 0 on success, -1 if @p len is too small or the magic is wrong.
|
|
*/
|
|
int udps_parse_header(const void *buf, size_t len, udps_header_t *out);
|
|
|
|
/**
|
|
* @brief Decodes a reassembled CONFIG payload.
|
|
* @param signals Out array of at most @p max_signals entries.
|
|
* @param num_signals Out; the number actually written.
|
|
* @param publish_mode Out; may be NULL.
|
|
* @return 0 on success, -1 if the payload is malformed or does not fit.
|
|
*/
|
|
int udps_parse_config(const void *payload,
|
|
size_t len,
|
|
udps_signal_t *signals,
|
|
uint32_t max_signals,
|
|
uint32_t *num_signals,
|
|
uint8_t *publish_mode);
|
|
|
|
/**
|
|
* @brief One value out of a frame.
|
|
* @param sample Accumulate batch slot; ignored for non-scalar signals.
|
|
* @param elem Element within the sample; ignored for accumulated scalars.
|
|
* @return the value, or 0.0 if any index is out of range.
|
|
*/
|
|
double udps_frame_value(const udps_frame_t *frame,
|
|
uint32_t signal_idx,
|
|
uint32_t sample,
|
|
uint32_t elem);
|
|
|
|
/**
|
|
* @brief Arrival-anchored estimate of the wall-clock time of one element.
|
|
*
|
|
* Exact only for streams that declare a sampling rate: the packet is assumed to
|
|
* have arrived as its last element was produced, and earlier elements are dated
|
|
* backwards by 1/sampling_rate. Signals with UDPS_TIME_PACKET, or without a
|
|
* sampling rate, all report the arrival time. When the stream carries a time
|
|
* signal (time_signal_idx != UDPS_NO_TIME_SIGNAL) that signal is the accurate
|
|
* source — read it like any other signal instead of using this helper.
|
|
*/
|
|
double udps_frame_element_time(const udps_frame_t *frame,
|
|
uint32_t signal_idx,
|
|
uint32_t elem);
|
|
|
|
#ifdef __cplusplus
|
|
} /* extern "C" */
|
|
#endif
|
|
|
|
#endif /* UDPS_CLIENT_H */
|