Files
MARTe-Integrated-Components/Docs/UDPS-C-Client.md
Martino FerrariandClaude Opus 4.6 deabd257e5 fix(udps): stop packets being dated from an earlier time base
Reported as samples sporadically carrying a previous packet's timestamp:
holes on one side of the stream and collisions on the other, in both the
Go and the MARTe2 receiver. That it appeared in both is what located it
-- the shared cause is upstream of either client. Four independent
defects, all of which end in a packet's values being placed at a time
that is not theirs.

Reassembly slot exhaustion (the "Reassembly slots full; evicting oldest"
flood). Chunk size was learnt only from fragment 0, so an out-of-order
burst destroyed a packet whose bytes had all arrived and left the slot
occupied until the 2 s GC. Slots were keyed on the counter alone, but
DATA and CONFIG number independently, so equal counters merged the two
streams. The 32-byte received-mask covered 256 of the 512 fragments the
client accepts, so a duplicate above 255 was counted as new and the
packet was delivered with a hole of stale bytes in it. And one datagram
was read per Execute(), which cannot drain a fast producer. Fixed with a
pendingTail deferral, (counter, type) keying, a 64-byte mask, a
256-datagram drain, counter-age slot reclamation, and a 1 Hz aggregated
warning in place of the per-eviction flood.

UDPStreamer dropping whole Accumulate batches. EventSem::ResetWait is
Reset-then-Wait, so a Post() landing while the sender thread was inside
ServiceClients()/SendData() was destroyed by the next Reset. The batch
was then skipped with dataReady false, readyFill was never cleared, and
the following flush overwrote it: an entire run of RT cycles never
reached the wire. The record of pending work now lives in the buffers
rather than in the semaphore edge, which also removes up to
UDPS_DATA_WAIT_MS of latency; genuine backpressure overwrites are
counted and reported. Against the unfixed code the new test sees
2999/3000 batches never consumed.

Period inflation after loss. Accumulated scalars carry no SamplingRate,
so the receiver derives dt from the sender-clock gap -- but dividing it
by the previous packet's sample count is only right while nothing is
lost. One loss doubles the reported period, which spreads a batch a full
batch past its own end and into the range the next packet claims. That
is the hole and the collision, exactly. Inferring the cycle count from
the estimate's own period is not a way out: it has a stable fixed point
wherever gap/dt is an integer, so a real rate change locks it at the old
one for good (AccumDtGTest.FollowsSustainedRateChange).

The packet counter removes the ambiguity, so all three receivers now
order on it: a DATA packet that does not advance the counter is dropped
rather than delivered, because its values are older than data already
handed over. Ordering is on the signed difference so it survives the
uint32 wrap, and the sequence resets on reconnect, where the producer's
counter restarts independently of ours. The loss count that falls out of
the same delta feeds the period estimate as cycles = prevN * (1 + lost),
which reduces exactly to gap/prevN when nothing is lost and therefore
still tracks a genuine rate change. UDPSClient::AcceptDataCounter (C++),
udpsprotocol.SequenceGate (Go), decode_data (C).

The C client's existing gap counter was wrap-unsafe and let a stale
packet rewind last_counter, which made every subsequent gap wrong; it
uses the same code now. Docs/Protocol.md gains an Ordering DATA section
stating the requirement for any receiver, including ones outside this
repository.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-09-02 01:14:40 +02:00

13 KiB
Raw Permalink Blame History

UDPS C Client Library

Common/Client/c/ is a standalone receiver for the UDPS streaming protocol: it connects to a UDPStreamer DataSource (or any other UDPS producer, such as DebugService), decodes the signals, and hands them to your callbacks as plain doubles.

It has no MARTe2 dependency and no third-party dependencies at all — just libc and BSD sockets. Two files, udps_client.h and udps_client.c, drop into any C or C++ project.

The wire format itself is specified in Protocol.md; this document covers the library. The producer side is documented in UDPStreamer.md.


Build

cd Common/Client/c
make            # libudpsclient.a + the udps_dump example
make cxxcheck   # verifies the header compiles and links from C++
make clean

Or just add the two files to your own build:

cc -std=c99 -O2 -c udps_client.c

Requirements: a C99 compiler and POSIX sockets. On glibc older than 2.17 add -lrt (clock_gettime lived in librt back then). The header is wrapped in extern "C", so C++ callers include it directly.


Quick start

#include "udps_client.h"
#include <stdio.h>

static void on_data(const udps_frame_t *f, void *user) {
    (void)user;
    /* Signals are in CONFIG order; values are already physical doubles. */
    printf("#%u  %s = %g\n", f->counter, f->signals[0].name, f->values[0].values[0]);
}

int main(void) {
    udps_client_config_t cfg;
    udps_client_t       *cli;

    udps_client_config_init(&cfg);
    cfg.server_addr = "127.0.0.1";
    cfg.server_port = 44500;

    cli = udps_client_create(&cfg);
    udps_client_set_callbacks(cli, NULL, on_data, NULL, NULL);

    for (;;) {
        udps_client_poll(cli, 200);   /* connects, receives, decodes, reconnects */
    }
    udps_client_destroy(cli);
    return 0;
}

udps_client_poll() is the only function that does work. It never spawns a thread, and every callback runs inside it — so if your program already has an event loop, call it from there and you need no synchronisation at all. A client must be used from one thread at a time.


Connection model

The library implements both transports of the protocol and picks one from the configuration:

Unicast (multicast_group == NULL) Multicast (multicast_group set)
CONNECT UDP datagram to server_addr:server_port over a TCP connection to server_addr:server_port
CONFIG UDP, back to the client's ephemeral port over the same TCP connection
DATA UDP, same ephemeral port UDP multicast on data_port
Keepalive ACK every keepalive_interval_s not needed (the TCP session is the liveness signal)

In multicast mode the group is joined before CONNECT is sent, because the server multicasts CONFIG as soon as it sees a client — a group joined afterwards would miss it.

The client reconnects on its own: if nothing arrives for silence_timeout_s it sends DISCONNECT, closes the sockets, waits reconnect_delay_s, and starts over. udps_client_poll() returns -1 when that happens, which is informational, not fatal.


Configuration

Always start from udps_client_config_init() — it fills in the defaults below — then override what you need. Strings are copied into the client, so they need not outlive udps_client_create().

Field Default Meaning
server_addr — (required) Server IPv4 address; a hostname is resolved if it is not a dotted quad.
server_port — (required) Server UDP port, or the TCP control port in multicast mode.
multicast_group NULL IPv4 group to join. Non-NULL selects the multicast transport.
interface_addr NULL Local IPv4 address (not a name, e.g. "127.0.0.1") of the interface to join on. Defaults to the default route, which silently receives nothing if the server sends elsewhere.
data_port server_port + 1 Multicast data port. Must match the producer's DataPort.
silence_timeout_s 1.0 Reconnect after this long without data. 0 disables the check — use it for streams that are idle by design.
reconnect_delay_s 2.0 Wait between reconnection attempts.
keepalive_interval_s 15.0 Unicast ACK period. The server evicts silent clients after its ClientTimeout (30 s by default). 0 disables.
recv_buffer_bytes 4 MiB SO_RCVBUF. The Linux default (~208 KiB) is overrun by fast producers and the kernel drops datagrams silently.
max_packet_bytes 1 MiB Ceiling on one reassembled payload; a reassembly buffer of this size is allocated per in-flight update (4 at most).

API

Lifecycle

void            udps_client_config_init(udps_client_config_t *cfg);
udps_client_t  *udps_client_create(const udps_client_config_t *cfg);
void            udps_client_set_callbacks(udps_client_t *c, udps_config_cb, udps_data_cb,
                                          udps_event_cb, void *user);
int             udps_client_poll(udps_client_t *c, int timeout_ms);
void            udps_client_destroy(udps_client_t *c);

udps_client_create() returns NULL on a bad address or an invalid configuration; no socket is opened until the first poll. udps_client_poll() returns the number of packets processed, 0 on timeout, or -1 if the session broke — pass a negative timeout_ms to block. destroy sends DISCONNECT before closing.

Callbacks

void on_config(const udps_signal_t *signals, uint32_t n, uint8_t publish_mode, void *user);
void on_data  (const udps_frame_t *frame, void *user);
void on_event (udps_event_t event, const char *detail, void *user);

on_config fires on every CONFIG packet: the signal set can change at runtime, so treat it as a reset of everything you cached. on_event reports UDPS_EVENT_CONNECTED, UDPS_EVENT_DISCONNECTED and UDPS_EVENT_ERROR with a human-readable detail.

The frame and everything it points at are owned by the client and are valid only until on_data returns. The decode buffers are reused by the next packet. Copy what you keep.

Inspection

int                  udps_client_is_connected(const udps_client_t *c);
const udps_signal_t *udps_client_signals(const udps_client_t *c, uint32_t *n);
uint8_t              udps_client_publish_mode(const udps_client_t *c);
void                 udps_client_stats(const udps_client_t *c, udps_stats_t *out);
const char          *udps_client_last_error(const udps_client_t *c);

Helpers

uint32_t    udps_signal_num_elements(const udps_signal_t *s);
const char *udps_type_name(uint8_t type_code);
int         udps_parse_header(const void *buf, size_t len, udps_header_t *out);
int         udps_parse_config(const void *payload, size_t len, udps_signal_t *sigs,
                              uint32_t max_signals, uint32_t *n, uint8_t *publish_mode);
double      udps_frame_value(const udps_frame_t *f, uint32_t sig, uint32_t sample, uint32_t elem);
double      udps_frame_element_time(const udps_frame_t *f, uint32_t sig, uint32_t elem);

udps_parse_header and udps_parse_config are stateless and socket-free, so captured or replayed traffic can be decoded without a client.


Reading a frame

typedef struct {
    uint32_t counter;      /* gaps in this sequence are lost datagrams */
    uint32_t lost;         /* DATA packets missing immediately before this one */
    uint64_t hrt;          /* producer's high-resolution timer at send */
    double   recv_time;    /* CLOCK_REALTIME seconds at arrival */
    uint8_t  publish_mode;
    uint32_t num_samples;  /* batched RT cycles; 1 unless Accumulate */
    uint32_t num_signals;
    const udps_signal_t        *signals;  /* CONFIG order */
    const udps_signal_values_t *values;   /* same order */
} udps_frame_t;

values[i].values is an array of values[i].count physical doubles. Quantised signals are already expanded back onto [range_min, range_max], and integer types are widened — the decoded form does not depend on the wire type, so a consumer need not branch on type_code at all.

Element count. count is the signal's element count (num_rows × num_cols), except for a scalar signal in Accumulate mode, where the producer batches several RT cycles into one packet and count == num_samples — one value per cycle. Arrays are not batched: they appear once and apply to the whole packet. udps_frame_value(f, sig, sample, elem) applies that rule for you.

Ordering. Frames reach on_data in counter order: a DATA packet that does not advance the counter — reordered or duplicated on the wire — is dropped rather than delivered, because its values carry a time base older than data you already have, and placing them would overwrite live samples while leaving their own span empty. lost reports how many packets went missing just before the frame. If you space samples yourself from the elapsed time since the previous frame, divide by lost + 1 batches, not one: the gap covers the missing packets' cycles too.

Timestamps. The protocol does not put a timestamp on every element; how to date them depends on the signal's time_mode (see Protocol.md):

time_mode Where the time comes from
UDPS_TIME_PACKET No per-element time. Use recv_time.
UDPS_TIME_FULL_ARRAY The signal at time_signal_idx holds one timestamp per element — read it like any other signal.
UDPS_TIME_FIRST_SAMPLE / UDPS_TIME_LAST_SAMPLE The signal at time_signal_idx is a scalar stamping element 0 (or N1); the rest follow at 1/sampling_rate.

The time signal is a raw producer-side counter (µs, or ns when it is a uint64), not wall clock, so plotting it against real time needs a one-off calibration against recv_time — that is what the Go hub does. udps_frame_element_time() skips all that and returns an arrival-anchored estimate: good enough for a quick plot, but when a time signal exists, it is the accurate source.


Diagnosing loss

udps_stats_t s;
udps_client_stats(cli, &s);
Counter Meaning
packets_received, bytes_received Accepted datagrams and TCP frames.
frames_delivered DATA packets decoded and passed to on_data.
config_updates CONFIG packets applied.
counter_gaps Missing packet counters — datagrams lost on the wire or in the kernel.
stale_packets DATA packets dropped for not advancing the counter: reordered or duplicated on the wire.
fragments_dropped Duplicate, stale or unplaceable fragments; a non-zero value with counter_gaps means fragmented updates are arriving incomplete.
reconnects Sessions re-established after a silence timeout.

Persistent loss on a fast stream is almost always the receive buffer: raise recv_buffer_bytes (and net.core.rmem_max, which caps it). A fragmented producer is more fragile than one sending whole cycles, because losing any fragment discards the whole update — if you control the producer, sizing MaxPayloadSize above one cycle removes that failure mode entirely.


Example program

example/udps_dump.c connects, prints the signal table on CONFIG, then a throttled summary of each frame, and a receive-statistics report on Ctrl-C.

# unicast
./udps_dump --host 127.0.0.1 --port 44500

# multicast
./udps_dump --host 127.0.0.1 --port 44500 --multicast 239.0.0.1 --iface 127.0.0.1

# quieter, and stop after 500 frames
./udps_dump --host 127.0.0.1 --port 44500 --interval 5 --frames 500
Flag Meaning
--host ADDR Server address (default 127.0.0.1).
--port N Server UDP port, or TCP control port in multicast mode (default 44500).
--multicast GROUP Join GROUP for data instead of using unicast.
--iface ADDR Local interface address for the multicast join.
--data-port N Multicast data port (default --port + 1).
--silence SEC Reconnect after SEC without data; 0 disables.
--interval SEC Seconds between printouts (default 1).
--frames N Exit after N frames.

Against the repository's own producer (./run_udp_producer.sh -n 2, two 1 Msps channels of 1000-element float32 arrays at 1 kHz) the output looks like:

CONFIG: 3 signal(s), publish mode strict
  #   name                     type     shape      unit     rate[Hz]   time-mode
  0   TimeArray                uint64   1x1000     ns       0          packet
  1   Ch1                      float32  1x1000     V        0          full-array
  2   Ch2                      float32  1x1000     V        0          full-array

frame #2289897  t=1787409851.723466  samples=1
  Ch1     n=1000   first=-6.9e-10  last=-0.00628  min=-1     max=1     V
  Ch2     n=1000   first=0.5       last=0.49975   min=-0.5   max=0.5   V

Limitations

  • IPv4 only, matching the protocol and the producer.
  • One thread per client; there is no internal locking.
  • The receive path allocates only when a CONFIG grows the signal set or a frame grows the decode arena, so a steady stream is allocation-free — but this is not a hard real-time component.
  • DATA arriving before the first CONFIG is dropped: without descriptors it cannot be decoded. This is normal for a few packets after joining a multicast group.