MARTe-Integrated-Components/docs/superpowers/specs/2026-11-05-tcp-streamer-design.md

# TCPStreamer / TCPStreamerClient — design

Date: 2026-11-05

## Goal

A pair of MARTe2 DataSources, `TCPStreamer` (output) and `TCPStreamerClient`
(input), equivalent to `UDPStreamer` / `UDPStreamerClient` but carrying data
over TCP. Plus a feasibility study (section 7) of a single in-process service
that streams data from several DataSources over multiple threads.

## Decisions (agreed)

1. **Wire format**: the existing UDPS framing (17-byte header + CONFIG/DATA
   payload) over the TCP byte stream, `totalFragments` always 1. No new
   protocol; no fragmentation or reassembly.
2. **Slow consumers**: drop for the slow client only. Bounded per-client queue
   (`ClientQueueDepth` frames); a client whose queue is full loses frames
   (counted per client, logged at most once per second); a client stalled
   longer than `ClientStallTimeout` is disconnected. Other clients and the RT
   thread are never affected.
3. **Service**: an in-process MARTe2 `Interface` object (like `DebugService`),
   not a separate daemon. Investigated here, not built in v1.

Decisions made without asking (flagged, easy to reverse):

- v1 scope is the two DataSources plus their transport classes. Consumers
  other than `TCPStreamerClient` (StreamHub, Go decoder, C client, UDPScope)
  keep speaking UDP; adding a TCP ingest to them is a follow-up that needs no
  protocol change because of decision 1.
- The service is a design study only (section 7). v1 is built so the service
  can later reuse the same transport classes.

## 1. Architecture: pluggable transport, shared serialisation

`UDPStreamer` already isolates its transport behind seven calls on `UDPSServer`
(`Initialise`, `Start`, `Stop`, `ServiceClients`, `HasClients`, `SendConfig`,
`SendData`); `UDPStreamerClient` uses `UDPSClient` through four
(`Initialise`, `SetListener`, `Start`, `Stop`). Everything else in those files
(signal parsing, quantisation, Strict/Accumulate/Decimate, double buffering,
drop accounting, DATA decode) is transport-independent and ~1000 lines.

Forking the files would duplicate that. Instead:

- Add two small abstract interfaces next to the existing classes in
  `Source/Components/Interfaces/UDPStream/`:
  - `UDPSServerI` — the seven server calls above.
  - `UDPSClientI` — the four client calls above.
- `UDPSServer` / `UDPSClient` implement them (signature-only change; behaviour
  untouched).
- `UDPStreamer` / `UDPStreamerClient` hold a pointer to the interface, created
  by a protected virtual factory (`CreateServer()` / `CreateClient()`) that
  defaults to the UDP implementation.
- `TCPStreamer : UDPStreamer` and `TCPStreamerClient : UDPStreamerClient`
  override the factory, register as their own MARTe2 classes, and override
  `Initialise` only to parse the TCP-specific keys and reject the UDP-only ones.
- New transport classes `TCPSServer` and `TCPSClient` in the same
  `Interfaces/UDPStream` library.

Risk: this touches working RT code. Mitigation: the change to the UDP classes
is mechanical (pointer instead of member), and the existing GTests and the
E2E suite cover them; run both before and after.

## 2. TCPSServer

Single-threaded, driven from the owner's `Execute()` loop exactly like
`UDPSServer`; no thread per client.

- Listening socket, `SO_REUSEADDR`, non-blocking accept in `ServiceClients()`.
- Per client: non-blocking socket, `TCP_NODELAY`, ring of `ClientQueueDepth`
  frames (pre-allocated at `Start()`, each sized `UDPS_HEADER_SIZE +
  MaxPayloadSize`), partial-write cursor, drop counter, last-progress
  timestamp.
- On accept: enqueue the cached CONFIG first, so a client always sees CONFIG
  before any DATA. `SendConfig()` while clients are connected enqueues the new
  CONFIG on every client (same as UDP) and refreshes the cache.
- CONFIG never goes through the DATA ring: each client has a separate
  pending-CONFIG slot that is always flushed before any queued DATA, so
  CONFIG can't be dropped and a CONFIG change can't be reordered behind
  stale DATA.
- `SendData()`: builds one frame, copies it into each client's DATA ring
  (when full, the oldest queued DATA frame is discarded and counted, so a
  recovering client resumes near live), then attempts a non-blocking flush
  on each client. `ServiceClients()` also flushes, so a client that
  became writable drains without waiting for the next DATA.
- Stall: no bytes written for `ClientStallTimeout` while the queue is
  non-empty → close and evict. Peer close / error on write → evict.
- `Stop()` closes everything and frees buffers.
- No STL, no OS mutexes; allocation only in `Start()`.

Config keys (read by `TCPSServer::Initialise` from the datasource node):

| Key | Default | Meaning |
|---|---|---|
| `Port` | 44500 | TCP listen port |
| `MaxPayloadSize` | 60000 | Max payload bytes per frame; also bounds Accumulate batches |
| `MaxClients` | 16 | Simultaneous clients; extra connections refused |
| `ClientQueueDepth` | 8 | Frames queued per client before dropping |
| `ClientStallTimeout` | 5 | Seconds without progress before disconnect; 0 disables |

`MaxPayloadSize` keeps its UDP meaning only in that Accumulate sizes batches
from it; with no MTU to respect the default is raised, capped at
`UDPS_CLIENT_MAX_PACKET_BYTES` (1 MiB).

## 3. TCPStreamer

Same signal and publishing configuration as `UDPStreamer` (`PublishingMode`,
`MinRefreshRate`, `MaxBatchSize`, per-signal `Unit`/`Range*`/`QuantizedType`/
`TimeMode`/`TimeSignal`/`SamplingRate`, `CPUMask`, `StackSize`). Rejected as
invalid: `MulticastGroup`, `Interface`, `DataPort`. Class name `TCPStreamer`.

The CONFIG payload, including the `hrtFrequency` trailer, is unchanged.
Accessors (`GetPort`, `IsClientConnected`, `GetDroppedPublications`) are
inherited; add `GetClientDrops()` for the per-client queue drops.

## 4. TCPSClient and TCPStreamerClient

`TCPSClient` implements `UDPSClientI` and delivers via the existing
`UDPSClientListener` (`OnUDPSConfig`, `OnUDPSData`, `OnUDPSConnected`,
`OnUDPSDisconnected`), so `UDPStreamerClient`'s decode path is reused as is.

- Own thread (`SingleThreadService`). Connect with a bounded timeout; on
  success `OnUDPSConnected`.
- Frame reader: read the 17-byte header, check magic and
  `payloadBytes <= UDPS_CLIENT_MAX_PACKET_BYTES`, read the payload, dispatch
  by type. Reads are non-blocking with a short `poll`, so `Stop()` is prompt
  and a partial frame survives across iterations.
- Bad magic or oversize length → close and reconnect (stream is
  unrecoverable once out of sync).
- Peer close, read error, or `SilenceTimeout` with no bytes → close,
  `OnUDPSDisconnected`, retry after `ReconnectDelay`.
- No CONNECT/ACK/keepalive traffic: the connection itself is the session, and
  the server sends CONFIG on accept.
- Gaps: TCP is lossless but `TCPSServer` may drop frames; `GetLastDataGap()`
  is computed from the `counter` field exactly as for UDP, so timestamp
  reconstruction keeps working. Stale-packet gating is a no-op (ordered
  stream).

Keys: `ServerAddress` (default `127.0.0.1`), `Port`, `SilenceTimeout`,
`ReconnectDelay`, `CPUMask`, `StackSize`. `TCPStreamerClient` rejects the
multicast keys.

## 5. Error handling

- RT thread path unchanged: spinlock + memcpy in `Synchronise()`, nothing
  network-related.
- All socket failures are per client; a failing client never fails
  `SendData()` for the others, and `SendData()` returns true when at least
  one client accepted the frame or none are connected.
- Drops and stalls are counted and reported rate-limited (1/s), like
  `ReportDroppedPublications`.

## 6. Testing

- GTest for `TCPSServer` + `TCPSClient` over loopback: CONFIG-before-DATA on
  late join; multiple clients; slow client drops while a fast one receives all
  frames; stall disconnect; reconnect after server restart; garbage/oversize
  frame rejection; partial-frame reads.
- Datasource tests under `Test/Components/DataSources/TCPStreamer{,Client}`
  mirroring the UDP ones (Strict, Accumulate, Decimate, quantised signals,
  round trip streamer → client).
- Regression: existing UDP GTests and the E2E suite before and after the
  interface refactor.
- E2E scenario with a TCP chain is deferred with the consumer-side follow-up.

## 7. Single service, many DataSources, many threads (implemented as `TCPStreamService`; feasibility notes below)

**Verdict: feasible, moderate effort, and the v1 design is the right
stepping stone.**

Shape: `TCPStreamService` as a MARTe2 `Interface` singleton (registered like
`DebugService`). Config declares the port and the worker count. `TCPStreamer`
instances gain `Service = <name>` and, instead of owning a `TCPSServer`, call
`service->Publish(streamId, frame)` and `service->PublishConfig(...)`.

Threading:
- One accept thread. N worker threads (config `Workers`), each owning a shard
  of client connections and running the same flush loop as `TCPSServer`
  (poll-driven, non-blocking). Clients are assigned least-loaded at accept.
- Producers (the per-DataSource sender threads from `UDPStreamer::Execute`, not
  the RT threads) publish into a per-stream buffer. Each worker reads those
  buffers; handoff by a per-stream `FastPollingMutexSem`-guarded latest-frame
  slot plus a monotonically increasing sequence number, so a worker that falls
  behind skips to the newest frame rather than queueing — consistent with the
  drop policy. This keeps RT paths free of cross-thread contention beyond what
  exists today.
- Per-client queues stay private to the owning worker, so no locking on the
  send path.

Protocol gap: the UDPS header has no stream identifier. Two ways to carry
several DataSources on one connection:
1. **One port per stream** — no protocol change, but then there is no
   multiplexing and the service only saves threads.
2. **Add a stream id** — a new packet type `UDPS_TYPE_SUBSCRIBE`
   (client → server, lists stream names) and a 2-byte stream-id prefix on
   DATA/CONFIG payloads of multiplexed connections, negotiated by the
   SUBSCRIBE exchange so legacy single-stream clients are unaffected. This is
   a protocol change and must be mirrored across the C++, Go, C and JS
   parsers (see `CLAUDE.md`).

Recommendation: build v1 now; if the service is wanted, do it as a separate
spec choosing option 2, reusing `TCPSServer`'s per-client queue/flush code as
the worker's inner loop. A separate-process daemon was considered and
rejected: it overlaps StreamHub, adds IPC to the RT path and a deployment
unit, with the only gain being survival across application restarts.

Open for that follow-up spec: stream naming and collision rules across
DataSources, per-stream vs per-client queue accounting, and whether
subscribe-time CONFIG filtering is needed.

## 8. As implemented (deviations from sections 2-6)

Decisions taken while building v1; where they differ from the text above,
this section wins.

- Per-client queue: `ClientQueueDepth + 2` slots (up to `depth` DATA, one
  in-flight CONFIG, one pending CONFIG), grown lazily on the sender thread
  rather than pre-allocated at `Start()`. A new CONFIG discards queued frames
  that are not in flight (stale DATA) instead of being "flushed before DATA".
  DATA with a full queue drops the oldest non-in-flight DATA frame, or the
  incoming frame if only the in-flight one is queued.
- Extra key `SendBufferSize` (SO_SNDBUF for client sockets, 0 = OS default).
- `Port = 0` binds an ephemeral port; `TCPStreamer::GetBoundPort()` /
  `TCPSServer::GetPort()` return the bound port once started.
- `SendData()` returns false only for frames larger than 1 MiB
  (`TCPS_MAX_FRAME_PAYLOAD`). Frames may exceed `MaxPayloadSize`; it only
  sizes Accumulate batches.
- Extra accessors: `GetClientCount()`, `GetClientDrops()` (uint64),
  `GetStallEvictions()`.
- UDP-only keys rejected by `TCPStreamer`: `MulticastGroup`, `Interface`,
  `DataPort`, `ClientTimeout`. By `TCPStreamerClient`: `MulticastGroup`,
  `DataPort`, `Interface`, `KeepAliveInterval`.
- `TCPSClient` keys: `ServerAddr`, `Port`, `SilenceTimeout` (default 0 =
  disabled, TCP reports dead peers itself), `ReconnectDelay` (default 2 s),
  `CPUMask`, `StackSize`. `TCPStreamerClient` takes `ServerAddress` as the
  UDP one does.
- No client-side gap or stale-packet handling: the client delivers frames
  exactly as received. A bad magic, oversize length or fragmented frame drops
  the connection and counts a protocol error.
- Tests: `Test/GTest/TCPSServerGTest.cpp`, `TCPSClientGTest.cpp`,
  `TCPStreamerGTest.cpp` (shared helpers in `TCPTestUtil.h`), built into
  `MainGTest.ex`. Not yet covered: Accumulate/Decimate/quantised signals over
  TCP (serialisation is inherited unchanged from `UDPStreamer`), and the E2E
  suite.

## 9. Known gap

- Default `MaxPayloadSize` (60000) is a first guess; confirm against
  Accumulate batch latency during testing.