Martino FerrariandClaude Opus 4.6
deabd257e5
Reported as samples sporadically carrying a previous packet's timestamp: holes on one side of the stream and collisions on the other, in both the Go and the MARTe2 receiver. That it appeared in both is what located it -- the shared cause is upstream of either client. Four independent defects, all of which end in a packet's values being placed at a time that is not theirs. Reassembly slot exhaustion (the "Reassembly slots full; evicting oldest" flood). Chunk size was learnt only from fragment 0, so an out-of-order burst destroyed a packet whose bytes had all arrived and left the slot occupied until the 2 s GC. Slots were keyed on the counter alone, but DATA and CONFIG number independently, so equal counters merged the two streams. The 32-byte received-mask covered 256 of the 512 fragments the client accepts, so a duplicate above 255 was counted as new and the packet was delivered with a hole of stale bytes in it. And one datagram was read per Execute(), which cannot drain a fast producer. Fixed with a pendingTail deferral, (counter, type) keying, a 64-byte mask, a 256-datagram drain, counter-age slot reclamation, and a 1 Hz aggregated warning in place of the per-eviction flood. UDPStreamer dropping whole Accumulate batches. EventSem::ResetWait is Reset-then-Wait, so a Post() landing while the sender thread was inside ServiceClients()/SendData() was destroyed by the next Reset. The batch was then skipped with dataReady false, readyFill was never cleared, and the following flush overwrote it: an entire run of RT cycles never reached the wire. The record of pending work now lives in the buffers rather than in the semaphore edge, which also removes up to UDPS_DATA_WAIT_MS of latency; genuine backpressure overwrites are counted and reported. Against the unfixed code the new test sees 2999/3000 batches never consumed. Period inflation after loss. Accumulated scalars carry no SamplingRate, so the receiver derives dt from the sender-clock gap -- but dividing it by the previous packet's sample count is only right while nothing is lost. One loss doubles the reported period, which spreads a batch a full batch past its own end and into the range the next packet claims. That is the hole and the collision, exactly. Inferring the cycle count from the estimate's own period is not a way out: it has a stable fixed point wherever gap/dt is an integer, so a real rate change locks it at the old one for good (AccumDtGTest.FollowsSustainedRateChange). The packet counter removes the ambiguity, so all three receivers now order on it: a DATA packet that does not advance the counter is dropped rather than delivered, because its values are older than data already handed over. Ordering is on the signed difference so it survives the uint32 wrap, and the sequence resets on reconnect, where the producer's counter restarts independently of ours. The loss count that falls out of the same delta feeds the period estimate as cycles = prevN * (1 + lost), which reduces exactly to gap/prevN when nothing is lost and therefore still tracks a genuine rate change. UDPSClient::AcceptDataCounter (C++), udpsprotocol.SequenceGate (Go), decode_data (C). The C client's existing gap counter was wrap-unsafe and let a stale packet rewind last_counter, which made every subsequent gap wrong; it uses the same code now. Docs/Protocol.md gains an Ordering DATA section stating the requirement for any receiver, including ones outside this repository. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
297 lines
13 KiB
Markdown
297 lines
13 KiB
Markdown
# UDPS C Client Library
|
||
|
||
`Common/Client/c/` is a standalone receiver for the UDPS streaming protocol: it connects to a
|
||
`UDPStreamer` DataSource (or any other UDPS producer, such as `DebugService`), decodes the
|
||
signals, and hands them to your callbacks as plain `double`s.
|
||
|
||
It has **no MARTe2 dependency** and no third-party dependencies at all — just libc and BSD
|
||
sockets. Two files, `udps_client.h` and `udps_client.c`, drop into any C or C++ project.
|
||
|
||
The wire format itself is specified in [Protocol.md](Protocol.md); this document covers the
|
||
library. The producer side is documented in [UDPStreamer.md](UDPStreamer.md).
|
||
|
||
---
|
||
|
||
## Build
|
||
|
||
```bash
|
||
cd Common/Client/c
|
||
make # libudpsclient.a + the udps_dump example
|
||
make cxxcheck # verifies the header compiles and links from C++
|
||
make clean
|
||
```
|
||
|
||
Or just add the two files to your own build:
|
||
|
||
```bash
|
||
cc -std=c99 -O2 -c udps_client.c
|
||
```
|
||
|
||
Requirements: a C99 compiler and POSIX sockets. On glibc older than 2.17 add `-lrt`
|
||
(`clock_gettime` lived in librt back then). The header is wrapped in `extern "C"`, so C++
|
||
callers include it directly.
|
||
|
||
---
|
||
|
||
## Quick start
|
||
|
||
```c
|
||
#include "udps_client.h"
|
||
#include <stdio.h>
|
||
|
||
static void on_data(const udps_frame_t *f, void *user) {
|
||
(void)user;
|
||
/* Signals are in CONFIG order; values are already physical doubles. */
|
||
printf("#%u %s = %g\n", f->counter, f->signals[0].name, f->values[0].values[0]);
|
||
}
|
||
|
||
int main(void) {
|
||
udps_client_config_t cfg;
|
||
udps_client_t *cli;
|
||
|
||
udps_client_config_init(&cfg);
|
||
cfg.server_addr = "127.0.0.1";
|
||
cfg.server_port = 44500;
|
||
|
||
cli = udps_client_create(&cfg);
|
||
udps_client_set_callbacks(cli, NULL, on_data, NULL, NULL);
|
||
|
||
for (;;) {
|
||
udps_client_poll(cli, 200); /* connects, receives, decodes, reconnects */
|
||
}
|
||
udps_client_destroy(cli);
|
||
return 0;
|
||
}
|
||
```
|
||
|
||
`udps_client_poll()` is the only function that does work. It never spawns a thread, and every
|
||
callback runs inside it — so if your program already has an event loop, call it from there and
|
||
you need no synchronisation at all. A client must be used from one thread at a time.
|
||
|
||
---
|
||
|
||
## Connection model
|
||
|
||
The library implements both transports of the protocol and picks one from the configuration:
|
||
|
||
| | Unicast (`multicast_group == NULL`) | Multicast (`multicast_group` set) |
|
||
|---|---|---|
|
||
| CONNECT | UDP datagram to `server_addr:server_port` | over a TCP connection to `server_addr:server_port` |
|
||
| CONFIG | UDP, back to the client's ephemeral port | over the same TCP connection |
|
||
| DATA | UDP, same ephemeral port | UDP multicast on `data_port` |
|
||
| Keepalive | ACK every `keepalive_interval_s` | not needed (the TCP session is the liveness signal) |
|
||
|
||
In multicast mode the group is joined *before* CONNECT is sent, because the server multicasts
|
||
CONFIG as soon as it sees a client — a group joined afterwards would miss it.
|
||
|
||
The client reconnects on its own: if nothing arrives for `silence_timeout_s` it sends
|
||
DISCONNECT, closes the sockets, waits `reconnect_delay_s`, and starts over. `udps_client_poll()`
|
||
returns `-1` when that happens, which is informational, not fatal.
|
||
|
||
---
|
||
|
||
## Configuration
|
||
|
||
Always start from `udps_client_config_init()` — it fills in the defaults below — then override
|
||
what you need. Strings are copied into the client, so they need not outlive `udps_client_create()`.
|
||
|
||
| Field | Default | Meaning |
|
||
|---|---|---|
|
||
| `server_addr` | — (required) | Server IPv4 address; a hostname is resolved if it is not a dotted quad. |
|
||
| `server_port` | — (required) | Server UDP port, or the TCP control port in multicast mode. |
|
||
| `multicast_group` | `NULL` | IPv4 group to join. Non-`NULL` selects the multicast transport. |
|
||
| `interface_addr` | `NULL` | Local IPv4 **address** (not a name, e.g. `"127.0.0.1"`) of the interface to join on. Defaults to the default route, which silently receives nothing if the server sends elsewhere. |
|
||
| `data_port` | `server_port + 1` | Multicast data port. Must match the producer's `DataPort`. |
|
||
| `silence_timeout_s` | `1.0` | Reconnect after this long without data. `0` disables the check — use it for streams that are idle by design. |
|
||
| `reconnect_delay_s` | `2.0` | Wait between reconnection attempts. |
|
||
| `keepalive_interval_s` | `15.0` | Unicast ACK period. The server evicts silent clients after its `ClientTimeout` (30 s by default). `0` disables. |
|
||
| `recv_buffer_bytes` | 4 MiB | `SO_RCVBUF`. The Linux default (~208 KiB) is overrun by fast producers and the kernel drops datagrams silently. |
|
||
| `max_packet_bytes` | 1 MiB | Ceiling on one reassembled payload; a reassembly buffer of this size is allocated per in-flight update (4 at most). |
|
||
|
||
---
|
||
|
||
## API
|
||
|
||
### Lifecycle
|
||
|
||
```c
|
||
void udps_client_config_init(udps_client_config_t *cfg);
|
||
udps_client_t *udps_client_create(const udps_client_config_t *cfg);
|
||
void udps_client_set_callbacks(udps_client_t *c, udps_config_cb, udps_data_cb,
|
||
udps_event_cb, void *user);
|
||
int udps_client_poll(udps_client_t *c, int timeout_ms);
|
||
void udps_client_destroy(udps_client_t *c);
|
||
```
|
||
|
||
`udps_client_create()` returns `NULL` on a bad address or an invalid configuration; no socket is
|
||
opened until the first poll. `udps_client_poll()` returns the number of packets processed, `0` on
|
||
timeout, or `-1` if the session broke — pass a negative `timeout_ms` to block. `destroy` sends
|
||
DISCONNECT before closing.
|
||
|
||
### Callbacks
|
||
|
||
```c
|
||
void on_config(const udps_signal_t *signals, uint32_t n, uint8_t publish_mode, void *user);
|
||
void on_data (const udps_frame_t *frame, void *user);
|
||
void on_event (udps_event_t event, const char *detail, void *user);
|
||
```
|
||
|
||
`on_config` fires on every CONFIG packet: the signal set can change at runtime, so treat it as a
|
||
reset of everything you cached. `on_event` reports `UDPS_EVENT_CONNECTED`,
|
||
`UDPS_EVENT_DISCONNECTED` and `UDPS_EVENT_ERROR` with a human-readable `detail`.
|
||
|
||
> **The frame and everything it points at are owned by the client and are valid only until
|
||
> `on_data` returns.** The decode buffers are reused by the next packet. Copy what you keep.
|
||
|
||
### Inspection
|
||
|
||
```c
|
||
int udps_client_is_connected(const udps_client_t *c);
|
||
const udps_signal_t *udps_client_signals(const udps_client_t *c, uint32_t *n);
|
||
uint8_t udps_client_publish_mode(const udps_client_t *c);
|
||
void udps_client_stats(const udps_client_t *c, udps_stats_t *out);
|
||
const char *udps_client_last_error(const udps_client_t *c);
|
||
```
|
||
|
||
### Helpers
|
||
|
||
```c
|
||
uint32_t udps_signal_num_elements(const udps_signal_t *s);
|
||
const char *udps_type_name(uint8_t type_code);
|
||
int udps_parse_header(const void *buf, size_t len, udps_header_t *out);
|
||
int udps_parse_config(const void *payload, size_t len, udps_signal_t *sigs,
|
||
uint32_t max_signals, uint32_t *n, uint8_t *publish_mode);
|
||
double udps_frame_value(const udps_frame_t *f, uint32_t sig, uint32_t sample, uint32_t elem);
|
||
double udps_frame_element_time(const udps_frame_t *f, uint32_t sig, uint32_t elem);
|
||
```
|
||
|
||
`udps_parse_header` and `udps_parse_config` are stateless and socket-free, so captured or
|
||
replayed traffic can be decoded without a client.
|
||
|
||
---
|
||
|
||
## Reading a frame
|
||
|
||
```c
|
||
typedef struct {
|
||
uint32_t counter; /* gaps in this sequence are lost datagrams */
|
||
uint32_t lost; /* DATA packets missing immediately before this one */
|
||
uint64_t hrt; /* producer's high-resolution timer at send */
|
||
double recv_time; /* CLOCK_REALTIME seconds at arrival */
|
||
uint8_t publish_mode;
|
||
uint32_t num_samples; /* batched RT cycles; 1 unless Accumulate */
|
||
uint32_t num_signals;
|
||
const udps_signal_t *signals; /* CONFIG order */
|
||
const udps_signal_values_t *values; /* same order */
|
||
} udps_frame_t;
|
||
```
|
||
|
||
`values[i].values` is an array of `values[i].count` physical `double`s. Quantised signals are
|
||
already expanded back onto `[range_min, range_max]`, and integer types are widened — the decoded
|
||
form does not depend on the wire type, so a consumer need not branch on `type_code` at all.
|
||
|
||
**Element count.** `count` is the signal's element count (`num_rows × num_cols`), *except* for a
|
||
scalar signal in Accumulate mode, where the producer batches several RT cycles into one packet
|
||
and `count == num_samples` — one value per cycle. Arrays are not batched: they appear once and
|
||
apply to the whole packet. `udps_frame_value(f, sig, sample, elem)` applies that rule for you.
|
||
|
||
**Ordering.** Frames reach `on_data` in counter order: a DATA packet that does not advance the
|
||
counter — reordered or duplicated on the wire — is dropped rather than delivered, because its
|
||
values carry a time base older than data you already have, and placing them would overwrite live
|
||
samples while leaving their own span empty. `lost` reports how many packets went missing just
|
||
before the frame. If you space samples yourself from the elapsed time since the previous frame,
|
||
divide by `lost + 1` batches, not one: the gap covers the missing packets' cycles too.
|
||
|
||
**Timestamps.** The protocol does not put a timestamp on every element; how to date them depends
|
||
on the signal's `time_mode` (see [Protocol.md](Protocol.md#time-mode-codes)):
|
||
|
||
| `time_mode` | Where the time comes from |
|
||
|---|---|
|
||
| `UDPS_TIME_PACKET` | No per-element time. Use `recv_time`. |
|
||
| `UDPS_TIME_FULL_ARRAY` | The signal at `time_signal_idx` holds one timestamp per element — read it like any other signal. |
|
||
| `UDPS_TIME_FIRST_SAMPLE` / `UDPS_TIME_LAST_SAMPLE` | The signal at `time_signal_idx` is a scalar stamping element 0 (or N−1); the rest follow at `1/sampling_rate`. |
|
||
|
||
The time signal is a raw producer-side counter (µs, or ns when it is a `uint64`), not wall clock,
|
||
so plotting it against real time needs a one-off calibration against `recv_time` — that is what
|
||
the Go hub does. `udps_frame_element_time()` skips all that and returns an arrival-anchored
|
||
estimate: good enough for a quick plot, but when a time signal exists, it is the accurate source.
|
||
|
||
---
|
||
|
||
## Diagnosing loss
|
||
|
||
```c
|
||
udps_stats_t s;
|
||
udps_client_stats(cli, &s);
|
||
```
|
||
|
||
| Counter | Meaning |
|
||
|---|---|
|
||
| `packets_received`, `bytes_received` | Accepted datagrams and TCP frames. |
|
||
| `frames_delivered` | DATA packets decoded and passed to `on_data`. |
|
||
| `config_updates` | CONFIG packets applied. |
|
||
| `counter_gaps` | Missing packet counters — datagrams lost on the wire or in the kernel. |
|
||
| `stale_packets` | DATA packets dropped for not advancing the counter: reordered or duplicated on the wire. |
|
||
| `fragments_dropped` | Duplicate, stale or unplaceable fragments; a non-zero value with `counter_gaps` means fragmented updates are arriving incomplete. |
|
||
| `reconnects` | Sessions re-established after a silence timeout. |
|
||
|
||
Persistent loss on a fast stream is almost always the receive buffer: raise `recv_buffer_bytes`
|
||
(and `net.core.rmem_max`, which caps it). A fragmented producer is more fragile than one sending
|
||
whole cycles, because losing any fragment discards the whole update — if you control the
|
||
producer, sizing `MaxPayloadSize` above one cycle removes that failure mode entirely.
|
||
|
||
---
|
||
|
||
## Example program
|
||
|
||
`example/udps_dump.c` connects, prints the signal table on CONFIG, then a throttled summary of
|
||
each frame, and a receive-statistics report on Ctrl-C.
|
||
|
||
```bash
|
||
# unicast
|
||
./udps_dump --host 127.0.0.1 --port 44500
|
||
|
||
# multicast
|
||
./udps_dump --host 127.0.0.1 --port 44500 --multicast 239.0.0.1 --iface 127.0.0.1
|
||
|
||
# quieter, and stop after 500 frames
|
||
./udps_dump --host 127.0.0.1 --port 44500 --interval 5 --frames 500
|
||
```
|
||
|
||
| Flag | Meaning |
|
||
|---|---|
|
||
| `--host ADDR` | Server address (default `127.0.0.1`). |
|
||
| `--port N` | Server UDP port, or TCP control port in multicast mode (default 44500). |
|
||
| `--multicast GROUP` | Join `GROUP` for data instead of using unicast. |
|
||
| `--iface ADDR` | Local interface address for the multicast join. |
|
||
| `--data-port N` | Multicast data port (default `--port + 1`). |
|
||
| `--silence SEC` | Reconnect after `SEC` without data; `0` disables. |
|
||
| `--interval SEC` | Seconds between printouts (default 1). |
|
||
| `--frames N` | Exit after `N` frames. |
|
||
|
||
Against the repository's own producer (`./run_udp_producer.sh -n 2`, two 1 Msps channels of
|
||
1000-element `float32` arrays at 1 kHz) the output looks like:
|
||
|
||
```
|
||
CONFIG: 3 signal(s), publish mode strict
|
||
# name type shape unit rate[Hz] time-mode
|
||
0 TimeArray uint64 1x1000 ns 0 packet
|
||
1 Ch1 float32 1x1000 V 0 full-array
|
||
2 Ch2 float32 1x1000 V 0 full-array
|
||
|
||
frame #2289897 t=1787409851.723466 samples=1
|
||
Ch1 n=1000 first=-6.9e-10 last=-0.00628 min=-1 max=1 V
|
||
Ch2 n=1000 first=0.5 last=0.49975 min=-0.5 max=0.5 V
|
||
```
|
||
|
||
---
|
||
|
||
## Limitations
|
||
|
||
- IPv4 only, matching the protocol and the producer.
|
||
- One thread per client; there is no internal locking.
|
||
- The receive path allocates only when a CONFIG grows the signal set or a frame grows the decode
|
||
arena, so a steady stream is allocation-free — but this is not a hard real-time component.
|
||
- DATA arriving before the first CONFIG is dropped: without descriptors it cannot be decoded.
|
||
This is normal for a few packets after joining a multicast group.
|