feat(tcp): TCPStreamer DataSources, in-process TCPStreamService, stream multiplexing and TCPStreamJoin
Add TCPStreamer/TCPStreamerClient (TCP transport behind the UDPSServerI/UDPSClientI seam), an optional single in-process TCPStreamService serving many streams over worker threads, multiplexed SUBSCRIBE (comma list, stream index in fragmentIdx), and TCPStreamJoin, which merges several streams into one sample-and-hold stream paced by a master. StreamHub takes a comma list of streams over one connection. Includes GTests, E2E scenarios s58-s73 and docs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
1 parent
5562877c99
commit
50d85363a3
79 files changed
+12348
-138
No files matched your search
@@ -0,0 +1,114 @@
|
||||
# TCPStreamer / TCPStreamerClient Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add `TCPStreamer` (output) and `TCPStreamerClient` (input) MARTe2 DataSources that carry the existing UDPS framing over TCP, reusing `UDPStreamer`/`UDPStreamerClient` by making their transport pluggable.
|
||||
|
||||
**Architecture:** Two abstract interfaces (`UDPSServerI`, `UDPSClientI`) wrap the seven server calls and four client calls the datasources already use. The base datasources own the transport through a heap pointer that a subclass can swap (`SetServer`/`SetClient`) and expose two virtual hooks (`DefaultMaxPayloadSize`, `InitialiseTransport`). `TCPStreamer : UDPStreamer` and `TCPStreamerClient : UDPStreamerClient` plug in the new `TCPSServer` / `TCPSClient`.
|
||||
|
||||
**Tech Stack:** C++ (MARTe2 conventions, no STL in `Source/`), POSIX sockets, GTest (STL allowed in tests), MARTe2 make system.
|
||||
|
||||
Spec: `docs/superpowers/specs/2026-11-05-tcp-streamer-design.md`.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- No STL, no OS mutexes in `Source/Components/` (spec: "No STL, no OS mutexes; allocation only in `Start()`" — relaxed to "never on the RT thread"; slots grow lazily on the sender thread).
|
||||
- Wire format unchanged: UDPS 17-byte header + CONFIG/DATA payload, `totalFragments` always 1.
|
||||
- RT path (`Synchronise()`) must stay "spinlock + memcpy".
|
||||
- Config keys: `Port` (44500), `MaxPayloadSize` (60000), `MaxClients` (16), `ClientQueueDepth` (8), `ClientStallTimeout` (5 s, 0 disables); client: `ServerAddress`, `Port`, `SilenceTimeout`, `ReconnectDelay`, `CPUMask`, `StackSize`.
|
||||
- `TCPStreamer` rejects `MulticastGroup`, `Interface`, `DataPort`; `TCPStreamerClient` rejects the same.
|
||||
- Do not `git add` unrelated working-tree changes (`Client/debugger/go.*`, `run_*.sh`, `Common/Client/c/Makefile`, ...). Add files by name. Commit only when the user asks.
|
||||
- Build: `source env.sh`, then `make -f Makefile.gcc core test`. Run from repo root. `make clean` first in Task 1 (class layouts change; `Interfaces/UDPStream` has no depends file).
|
||||
|
||||
## File Structure
|
||||
|
||||
| File | Responsibility |
|
||||
|---|---|
|
||||
| `Source/Components/Interfaces/UDPStream/UDPSTransportI.h` (new) | `UDPSServerI`, `UDPSClientI` |
|
||||
| `.../UDPStream/UDPSServer.h`, `UDPSClient.h` (modify) | implement the interfaces |
|
||||
| `.../UDPStream/TCPSServer.{h,cpp}` (new) | multi-client non-blocking TCP server, per-client bounded queue |
|
||||
| `.../UDPStream/TCPSClient.{h,cpp}` (new) | reconnecting TCP frame reader on its own thread |
|
||||
| `.../UDPStream/Makefile.inc` (modify) | add `TCPSServer.x TCPSClient.x` |
|
||||
| `Source/Components/DataSources/UDPStreamer/UDPStreamer.{h,cpp}` (modify) | transport via `UDPSServerI*`, hooks |
|
||||
| `Source/Components/DataSources/UDPStreamerClient/UDPStreamerClient.{h,cpp}` (modify) | transport via `UDPSClientI*`, hooks |
|
||||
| `Source/Components/DataSources/TCPStreamer/` (new) | `TCPStreamer`, Makefiles |
|
||||
| `Source/Components/DataSources/TCPStreamerClient/` (new) | `TCPStreamerClient`, Makefiles |
|
||||
| `Makefile.gcc` (modify) | build/clean the two new components |
|
||||
| `Test/GTest/TCPTestUtil.h` (new) | raw-socket helpers shared by the new tests |
|
||||
| `Test/GTest/TCPSServerGTest.cpp`, `TCPSClientGTest.cpp`, `TCPStreamerGTest.cpp` (new) | tests |
|
||||
| `Test/GTest/Makefile.inc` (modify) | register tests, include paths |
|
||||
| `Docs/TCPStreamer.md`, `CLAUDE.md`, spec (modify/new) | docs |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Pluggable transport seam (pure refactor, UDP behaviour unchanged)
|
||||
|
||||
**Files:** create `UDPSTransportI.h`; modify `UDPSServer.h`, `UDPSClient.h`, `UDPStreamer.{h,cpp}`, `UDPStreamerClient.{h,cpp}`.
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
```cpp
|
||||
class UDPSServerI { virtual ~UDPSServerI(){}
|
||||
virtual bool Initialise(StructuredDataI&)=0; virtual bool Start()=0; virtual bool Stop()=0;
|
||||
virtual void ServiceClients()=0; virtual bool SendConfig(const uint8*,uint32)=0;
|
||||
virtual bool SendData(uint32 counter,const uint8*,uint32)=0;
|
||||
virtual bool HasClients() const=0; virtual bool IsMulticast() const=0; virtual uint16 GetPort() const=0; };
|
||||
class UDPSClientI { virtual ~UDPSClientI(){}
|
||||
virtual bool Initialise(StructuredDataI&)=0; virtual void SetListener(UDPSClientListener*)=0;
|
||||
virtual bool Start()=0; virtual bool Stop()=0; };
|
||||
```
|
||||
`UDPStreamer` protected: `virtual uint32 DefaultMaxPayloadSize() const`, `virtual bool InitialiseTransport(StructuredDataI&)`, `void SetServer(UDPSServerI*)`, `UDPSServerI *Transport() const`.
|
||||
`UDPStreamerClient` protected: `virtual bool InitialiseTransport(StructuredDataI&)`, `void SetClient(UDPSClientI*)`, members `serverAddress, port, silenceTimeout, cpuMask, stackSize`.
|
||||
|
||||
- [ ] **Step 1: Baseline.** `make -f Makefile.gcc clean && make -f Makefile.gcc core test`, then run `./Build/x86-linux/GTest/MainGTest.ex` and record pass/fail counts.
|
||||
- [ ] **Step 2:** Create `UDPSTransportI.h` (code above; forward-declare `class UDPSClientListener;`).
|
||||
- [ ] **Step 3:** `UDPSServer.h`: `#include "UDPSTransportI.h"`, `class UDPSServer : public UDPSServerI`. `UDPSClient.h`: `class UDPSClient : public UDPSClientI, public EmbeddedServiceMethodBinderI`. Signatures already match; no `override` keyword (MARTe2 style).
|
||||
- [ ] **Step 4:** `UDPStreamer`: replace `UDPSServer server;` by `UDPSServerI *server;` (private), constructor `server = new UDPSServer();`, destructor stops then deletes it after the executor is stopped, `server.X` -> `server->X` everywhere, `maxPayloadSize = DefaultMaxPayloadSize()` in `Initialise`, move the `serverCfg` block into `UDPStreamer::InitialiseTransport` (default impl, identical code, calling `server->Initialise`). `GetPort()` keeps returning the member.
|
||||
- [ ] **Step 5:** `UDPStreamerClient`: same pattern with `UDPSClientI *client`; move the "Forward the transport parameters" block into `InitialiseTransport`; destructor `client->Stop()` when `receiverUp`, then delete.
|
||||
- [ ] **Step 6:** `make clean && make core test`; run `MainGTest.ex`: counts must equal Step 1. Run `./Build/x86-linux/Test/Integration/Integration/IntegrationTests.ex` if it passed at baseline.
|
||||
|
||||
### Task 2: `TCPSServer`
|
||||
|
||||
**Files:** create `TCPSServer.{h,cpp}`, `Test/GTest/TCPTestUtil.h`, `Test/GTest/TCPSServerGTest.cpp`; modify `UDPStream/Makefile.inc`, `Test/GTest/Makefile.inc`.
|
||||
|
||||
**Design (see code for the authoritative version):**
|
||||
- Raw POSIX sockets. Listener non-blocking, `SO_REUSEADDR`; `Port=0` binds an ephemeral port and `GetPort()` returns the bound port after `Start()`.
|
||||
- Per client: `Slot[ClientQueueDepth+2]` queue (`buf, cap, len, isConfig`), `frontSent` (bytes of the front frame already written), `dataQueued`, `lastProgressTicks`.
|
||||
- Queue capacity `depth+2` is exact: at most one data-in-flight/old frame, plus two CONFIG frames, plus `depth` DATA frames.
|
||||
- `QueueConfig`: drop every queued frame except an in-flight one, append the CONFIG (CONFIG never dropped, never behind stale DATA).
|
||||
- `QueueData`: while `dataQueued >= depth`, remove the oldest non-in-flight DATA (count a drop); if none is removable drop the new frame.
|
||||
- `Flush`: `send(MSG_NOSIGNAL|MSG_DONTWAIT)` loop; `EAGAIN` stops; other errors evict.
|
||||
- `ServiceClients`: accept (non-blocking), refuse beyond `MaxClients`, enqueue cached CONFIG on accept, detect peer close with `recv`, flush, evict clients with no progress for `ClientStallTimeout`, rate-limited (1/s) drop report.
|
||||
- Extra key `SendBufferSize` (0 = OS default): `SO_SNDBUF` on accepted sockets, which also bounds latency and makes slow-client tests deterministic.
|
||||
- `SendData` returns true unless the frame exceeds 1 MiB; per-client drops are reported through `GetClientDrops()`.
|
||||
|
||||
- [ ] **Step 1:** write `TCPSServerGTest.cpp` first: `Initialise` rejects (`MaxPayloadSize=0`, `ClientQueueDepth=0`, `MaxClients=0`), `ConfigBeforeData_LateJoin`, `MultipleClients_InOrder`, `ReconfigDeliveredToConnectedClient`, `MaxClientsRefused`, `PeerCloseEvicts`, `SlowClientDropsFastClientComplete` (200 x 60000-byte frames, `SendBufferSize=65536`, slow client `SO_RCVBUF=4096` never reads, fast client thread receives all counters 1..200 in order, `GetClientDrops() > 0`), `StallDisconnect` (`ClientStallTimeout=1`).
|
||||
- [ ] **Step 2:** run -> compile failure (no `TCPSServer.h`).
|
||||
- [ ] **Step 3:** implement `TCPSServer`; add to `OBJSX`.
|
||||
- [ ] **Step 4:** `make -f Makefile.gcc core test && ./Build/x86-linux/GTest/MainGTest.ex --gtest_filter='TCPSServer*'` -> all pass.
|
||||
|
||||
### Task 3: `TCPSClient`
|
||||
|
||||
**Files:** create `TCPSClient.{h,cpp}`, `Test/GTest/TCPSClientGTest.cpp`; modify `UDPStream/Makefile.inc`, `Test/GTest/Makefile.inc` (already shared).
|
||||
|
||||
**Design:** `TCPSClient : UDPSClientI, EmbeddedServiceMethodBinderI`; `SingleThreadService`; `Execute` per iteration: if disconnected honour `ReconnectDelay`, then non-blocking `connect` + `poll` (bounded 500 ms) -> `OnUDPSConnected`; if connected `poll(50 ms)`, `recv` into a `HEADER+1 MiB` buffer, parse complete frames, bad magic / `payloadBytes > 1 MiB` / `totalFragments != 1` -> close + `OnUDPSDisconnected`; peer close/read error/`SilenceTimeout` the same. Keys: `ServerAddr`, `Port`, `SilenceTimeout` (float32 s, default 0 = off), `ReconnectDelay` (float32 s, default 2), `CPUMask`, `StackSize`. `Stop()` closes silently (no callbacks).
|
||||
|
||||
- [ ] **Step 1:** tests (`RecordingListener` with `std::mutex`): `ReceivesConfigThenData` (real `TCPSServer`), `ReconnectAfterServerRestart` (fixed port, `ReconnectDelay=0.2`), `RejectsBadMagic`, `RejectsOversizeLength`, `PartialFrameReads` (raw server writing a frame in 3 chunks), `SilenceTimeoutReconnects`, `StopIsPrompt`.
|
||||
- [ ] **Step 2:** run -> compile failure. **Step 3:** implement. **Step 4:** `--gtest_filter='TCPSClient*'` passes.
|
||||
|
||||
### Task 4: `TCPStreamer`, `TCPStreamerClient`, build wiring
|
||||
|
||||
**Files:** create `Source/Components/DataSources/TCPStreamer/{TCPStreamer.h,TCPStreamer.cpp,Makefile.gcc,Makefile.inc}`, same for `TCPStreamerClient`, `Test/GTest/TCPStreamerGTest.cpp`; modify root `Makefile.gcc`, `Test/GTest/Makefile.inc`.
|
||||
|
||||
- [ ] **Step 1:** `TCPStreamerGTest.cpp`: `Constructor_Defaults` (`GetMaxPayloadSize()==60000`, `GetPort()==44500`), `Initialise_RejectsUdpOnlyKeys` (both classes), `Initialise_TcpKeys`, `ClassRegistered`, `StreamerToRawClient` (RealTimeApplication with `TCPStreamer`, raw client sees CONFIG then DATA while the test calls `Synchronise()`), `RoundTripStreamerToClient` (`TCPSServer`-driven client DS: send CONFIG+DATA, `Synchronise()` on a helper thread, check decoded value).
|
||||
- [ ] **Step 2:** run -> compile failure. **Step 3:** implement both classes (each overrides `Initialise` to reject keys, `DefaultMaxPayloadSize`/`InitialiseTransport`, constructor swaps in the TCP transport). Makefile.inc copies UDPStreamer's, with `-lUDPStreamer -lUDPStream` (client: `-lUDPStreamerClient`), packages `Components/DataSources`.
|
||||
- [ ] **Step 4:** add to root `Makefile.gcc` `core` and `clean`; if the `MainGTest` link reports undefined `UDPStreamer` symbols wrap `LIBRARIES_STATIC` in `-Wl,--start-group ... -Wl,--end-group`.
|
||||
- [ ] **Step 5:** full `make core test`, whole `MainGTest.ex` and UDP datasource tests pass.
|
||||
|
||||
### Task 5: Docs and spec sync
|
||||
|
||||
- [ ] `Docs/TCPStreamer.md` (config tables, behaviour, service feasibility pointer), `CLAUDE.md` one-line mention, and update the spec for decisions taken while building: lazy slot growth, `SendBufferSize`, `GetPort()` returns the bound port, `SendData` true unless oversize, client `SilenceTimeout` default 0, no client-side gap handling, Strict frames may exceed `MaxPayloadSize` up to 1 MiB.
|
||||
|
||||
## Self-review
|
||||
|
||||
Spec coverage: pluggable transport (T1), `TCPSServer` incl. CONFIG-first, drop, stall (T2), `TCPSClient` (T3), both datasources + rejection of UDP-only keys + tests + regression (T4), docs/service study already in spec section 7 (T5). Naming checked: `UDPSServerI`, `UDPSClientI`, `SetServer`, `SetClient`, `Transport()`, `DefaultMaxPayloadSize`, `InitialiseTransport` are used identically across tasks.
|
||||
@@ -0,0 +1,249 @@
|
||||
# TCPStreamer / TCPStreamerClient — design
|
||||
|
||||
Date: 2026-11-05
|
||||
|
||||
## Goal
|
||||
|
||||
A pair of MARTe2 DataSources, `TCPStreamer` (output) and `TCPStreamerClient`
|
||||
(input), equivalent to `UDPStreamer` / `UDPStreamerClient` but carrying data
|
||||
over TCP. Plus a feasibility study (section 7) of a single in-process service
|
||||
that streams data from several DataSources over multiple threads.
|
||||
|
||||
## Decisions (agreed)
|
||||
|
||||
1. **Wire format**: the existing UDPS framing (17-byte header + CONFIG/DATA
|
||||
payload) over the TCP byte stream, `totalFragments` always 1. No new
|
||||
protocol; no fragmentation or reassembly.
|
||||
2. **Slow consumers**: drop for the slow client only. Bounded per-client queue
|
||||
(`ClientQueueDepth` frames); a client whose queue is full loses frames
|
||||
(counted per client, logged at most once per second); a client stalled
|
||||
longer than `ClientStallTimeout` is disconnected. Other clients and the RT
|
||||
thread are never affected.
|
||||
3. **Service**: an in-process MARTe2 `Interface` object (like `DebugService`),
|
||||
not a separate daemon. Investigated here, not built in v1.
|
||||
|
||||
Decisions made without asking (flagged, easy to reverse):
|
||||
|
||||
- v1 scope is the two DataSources plus their transport classes. Consumers
|
||||
other than `TCPStreamerClient` (StreamHub, Go decoder, C client, UDPScope)
|
||||
keep speaking UDP; adding a TCP ingest to them is a follow-up that needs no
|
||||
protocol change because of decision 1.
|
||||
- The service is a design study only (section 7). v1 is built so the service
|
||||
can later reuse the same transport classes.
|
||||
|
||||
## 1. Architecture: pluggable transport, shared serialisation
|
||||
|
||||
`UDPStreamer` already isolates its transport behind seven calls on `UDPSServer`
|
||||
(`Initialise`, `Start`, `Stop`, `ServiceClients`, `HasClients`, `SendConfig`,
|
||||
`SendData`); `UDPStreamerClient` uses `UDPSClient` through four
|
||||
(`Initialise`, `SetListener`, `Start`, `Stop`). Everything else in those files
|
||||
(signal parsing, quantisation, Strict/Accumulate/Decimate, double buffering,
|
||||
drop accounting, DATA decode) is transport-independent and ~1000 lines.
|
||||
|
||||
Forking the files would duplicate that. Instead:
|
||||
|
||||
- Add two small abstract interfaces next to the existing classes in
|
||||
`Source/Components/Interfaces/UDPStream/`:
|
||||
- `UDPSServerI` — the seven server calls above.
|
||||
- `UDPSClientI` — the four client calls above.
|
||||
- `UDPSServer` / `UDPSClient` implement them (signature-only change; behaviour
|
||||
untouched).
|
||||
- `UDPStreamer` / `UDPStreamerClient` hold a pointer to the interface, created
|
||||
by a protected virtual factory (`CreateServer()` / `CreateClient()`) that
|
||||
defaults to the UDP implementation.
|
||||
- `TCPStreamer : UDPStreamer` and `TCPStreamerClient : UDPStreamerClient`
|
||||
override the factory, register as their own MARTe2 classes, and override
|
||||
`Initialise` only to parse the TCP-specific keys and reject the UDP-only ones.
|
||||
- New transport classes `TCPSServer` and `TCPSClient` in the same
|
||||
`Interfaces/UDPStream` library.
|
||||
|
||||
Risk: this touches working RT code. Mitigation: the change to the UDP classes
|
||||
is mechanical (pointer instead of member), and the existing GTests and the
|
||||
E2E suite cover them; run both before and after.
|
||||
|
||||
## 2. TCPSServer
|
||||
|
||||
Single-threaded, driven from the owner's `Execute()` loop exactly like
|
||||
`UDPSServer`; no thread per client.
|
||||
|
||||
- Listening socket, `SO_REUSEADDR`, non-blocking accept in `ServiceClients()`.
|
||||
- Per client: non-blocking socket, `TCP_NODELAY`, ring of `ClientQueueDepth`
|
||||
frames (pre-allocated at `Start()`, each sized `UDPS_HEADER_SIZE +
|
||||
MaxPayloadSize`), partial-write cursor, drop counter, last-progress
|
||||
timestamp.
|
||||
- On accept: enqueue the cached CONFIG first, so a client always sees CONFIG
|
||||
before any DATA. `SendConfig()` while clients are connected enqueues the new
|
||||
CONFIG on every client (same as UDP) and refreshes the cache.
|
||||
- CONFIG never goes through the DATA ring: each client has a separate
|
||||
pending-CONFIG slot that is always flushed before any queued DATA, so
|
||||
CONFIG can't be dropped and a CONFIG change can't be reordered behind
|
||||
stale DATA.
|
||||
- `SendData()`: builds one frame, copies it into each client's DATA ring
|
||||
(when full, the oldest queued DATA frame is discarded and counted, so a
|
||||
recovering client resumes near live), then attempts a non-blocking flush
|
||||
on each client. `ServiceClients()` also flushes, so a client that
|
||||
became writable drains without waiting for the next DATA.
|
||||
- Stall: no bytes written for `ClientStallTimeout` while the queue is
|
||||
non-empty → close and evict. Peer close / error on write → evict.
|
||||
- `Stop()` closes everything and frees buffers.
|
||||
- No STL, no OS mutexes; allocation only in `Start()`.
|
||||
|
||||
Config keys (read by `TCPSServer::Initialise` from the datasource node):
|
||||
|
||||
| Key | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `Port` | 44500 | TCP listen port |
|
||||
| `MaxPayloadSize` | 60000 | Max payload bytes per frame; also bounds Accumulate batches |
|
||||
| `MaxClients` | 16 | Simultaneous clients; extra connections refused |
|
||||
| `ClientQueueDepth` | 8 | Frames queued per client before dropping |
|
||||
| `ClientStallTimeout` | 5 | Seconds without progress before disconnect; 0 disables |
|
||||
|
||||
`MaxPayloadSize` keeps its UDP meaning only in that Accumulate sizes batches
|
||||
from it; with no MTU to respect the default is raised, capped at
|
||||
`UDPS_CLIENT_MAX_PACKET_BYTES` (1 MiB).
|
||||
|
||||
## 3. TCPStreamer
|
||||
|
||||
Same signal and publishing configuration as `UDPStreamer` (`PublishingMode`,
|
||||
`MinRefreshRate`, `MaxBatchSize`, per-signal `Unit`/`Range*`/`QuantizedType`/
|
||||
`TimeMode`/`TimeSignal`/`SamplingRate`, `CPUMask`, `StackSize`). Rejected as
|
||||
invalid: `MulticastGroup`, `Interface`, `DataPort`. Class name `TCPStreamer`.
|
||||
|
||||
The CONFIG payload, including the `hrtFrequency` trailer, is unchanged.
|
||||
Accessors (`GetPort`, `IsClientConnected`, `GetDroppedPublications`) are
|
||||
inherited; add `GetClientDrops()` for the per-client queue drops.
|
||||
|
||||
## 4. TCPSClient and TCPStreamerClient
|
||||
|
||||
`TCPSClient` implements `UDPSClientI` and delivers via the existing
|
||||
`UDPSClientListener` (`OnUDPSConfig`, `OnUDPSData`, `OnUDPSConnected`,
|
||||
`OnUDPSDisconnected`), so `UDPStreamerClient`'s decode path is reused as is.
|
||||
|
||||
- Own thread (`SingleThreadService`). Connect with a bounded timeout; on
|
||||
success `OnUDPSConnected`.
|
||||
- Frame reader: read the 17-byte header, check magic and
|
||||
`payloadBytes <= UDPS_CLIENT_MAX_PACKET_BYTES`, read the payload, dispatch
|
||||
by type. Reads are non-blocking with a short `poll`, so `Stop()` is prompt
|
||||
and a partial frame survives across iterations.
|
||||
- Bad magic or oversize length → close and reconnect (stream is
|
||||
unrecoverable once out of sync).
|
||||
- Peer close, read error, or `SilenceTimeout` with no bytes → close,
|
||||
`OnUDPSDisconnected`, retry after `ReconnectDelay`.
|
||||
- No CONNECT/ACK/keepalive traffic: the connection itself is the session, and
|
||||
the server sends CONFIG on accept.
|
||||
- Gaps: TCP is lossless but `TCPSServer` may drop frames; `GetLastDataGap()`
|
||||
is computed from the `counter` field exactly as for UDP, so timestamp
|
||||
reconstruction keeps working. Stale-packet gating is a no-op (ordered
|
||||
stream).
|
||||
|
||||
Keys: `ServerAddress` (default `127.0.0.1`), `Port`, `SilenceTimeout`,
|
||||
`ReconnectDelay`, `CPUMask`, `StackSize`. `TCPStreamerClient` rejects the
|
||||
multicast keys.
|
||||
|
||||
## 5. Error handling
|
||||
|
||||
- RT thread path unchanged: spinlock + memcpy in `Synchronise()`, nothing
|
||||
network-related.
|
||||
- All socket failures are per client; a failing client never fails
|
||||
`SendData()` for the others, and `SendData()` returns true when at least
|
||||
one client accepted the frame or none are connected.
|
||||
- Drops and stalls are counted and reported rate-limited (1/s), like
|
||||
`ReportDroppedPublications`.
|
||||
|
||||
## 6. Testing
|
||||
|
||||
- GTest for `TCPSServer` + `TCPSClient` over loopback: CONFIG-before-DATA on
|
||||
late join; multiple clients; slow client drops while a fast one receives all
|
||||
frames; stall disconnect; reconnect after server restart; garbage/oversize
|
||||
frame rejection; partial-frame reads.
|
||||
- Datasource tests under `Test/Components/DataSources/TCPStreamer{,Client}`
|
||||
mirroring the UDP ones (Strict, Accumulate, Decimate, quantised signals,
|
||||
round trip streamer → client).
|
||||
- Regression: existing UDP GTests and the E2E suite before and after the
|
||||
interface refactor.
|
||||
- E2E scenario with a TCP chain is deferred with the consumer-side follow-up.
|
||||
|
||||
## 7. Single service, many DataSources, many threads (implemented as `TCPStreamService`; feasibility notes below)
|
||||
|
||||
**Verdict: feasible, moderate effort, and the v1 design is the right
|
||||
stepping stone.**
|
||||
|
||||
Shape: `TCPStreamService` as a MARTe2 `Interface` singleton (registered like
|
||||
`DebugService`). Config declares the port and the worker count. `TCPStreamer`
|
||||
instances gain `Service = <name>` and, instead of owning a `TCPSServer`, call
|
||||
`service->Publish(streamId, frame)` and `service->PublishConfig(...)`.
|
||||
|
||||
Threading:
|
||||
- One accept thread. N worker threads (config `Workers`), each owning a shard
|
||||
of client connections and running the same flush loop as `TCPSServer`
|
||||
(poll-driven, non-blocking). Clients are assigned least-loaded at accept.
|
||||
- Producers (the per-DataSource sender threads from `UDPStreamer::Execute`, not
|
||||
the RT threads) publish into a per-stream buffer. Each worker reads those
|
||||
buffers; handoff by a per-stream `FastPollingMutexSem`-guarded latest-frame
|
||||
slot plus a monotonically increasing sequence number, so a worker that falls
|
||||
behind skips to the newest frame rather than queueing — consistent with the
|
||||
drop policy. This keeps RT paths free of cross-thread contention beyond what
|
||||
exists today.
|
||||
- Per-client queues stay private to the owning worker, so no locking on the
|
||||
send path.
|
||||
|
||||
Protocol gap: the UDPS header has no stream identifier. Two ways to carry
|
||||
several DataSources on one connection:
|
||||
1. **One port per stream** — no protocol change, but then there is no
|
||||
multiplexing and the service only saves threads.
|
||||
2. **Add a stream id** — a new packet type `UDPS_TYPE_SUBSCRIBE`
|
||||
(client → server, lists stream names) and a 2-byte stream-id prefix on
|
||||
DATA/CONFIG payloads of multiplexed connections, negotiated by the
|
||||
SUBSCRIBE exchange so legacy single-stream clients are unaffected. This is
|
||||
a protocol change and must be mirrored across the C++, Go, C and JS
|
||||
parsers (see `CLAUDE.md`).
|
||||
|
||||
Recommendation: build v1 now; if the service is wanted, do it as a separate
|
||||
spec choosing option 2, reusing `TCPSServer`'s per-client queue/flush code as
|
||||
the worker's inner loop. A separate-process daemon was considered and
|
||||
rejected: it overlaps StreamHub, adds IPC to the RT path and a deployment
|
||||
unit, with the only gain being survival across application restarts.
|
||||
|
||||
Open for that follow-up spec: stream naming and collision rules across
|
||||
DataSources, per-stream vs per-client queue accounting, and whether
|
||||
subscribe-time CONFIG filtering is needed.
|
||||
|
||||
## 8. As implemented (deviations from sections 2-6)
|
||||
|
||||
Decisions taken while building v1; where they differ from the text above,
|
||||
this section wins.
|
||||
|
||||
- Per-client queue: `ClientQueueDepth + 2` slots (up to `depth` DATA, one
|
||||
in-flight CONFIG, one pending CONFIG), grown lazily on the sender thread
|
||||
rather than pre-allocated at `Start()`. A new CONFIG discards queued frames
|
||||
that are not in flight (stale DATA) instead of being "flushed before DATA".
|
||||
DATA with a full queue drops the oldest non-in-flight DATA frame, or the
|
||||
incoming frame if only the in-flight one is queued.
|
||||
- Extra key `SendBufferSize` (SO_SNDBUF for client sockets, 0 = OS default).
|
||||
- `Port = 0` binds an ephemeral port; `TCPStreamer::GetBoundPort()` /
|
||||
`TCPSServer::GetPort()` return the bound port once started.
|
||||
- `SendData()` returns false only for frames larger than 1 MiB
|
||||
(`TCPS_MAX_FRAME_PAYLOAD`). Frames may exceed `MaxPayloadSize`; it only
|
||||
sizes Accumulate batches.
|
||||
- Extra accessors: `GetClientCount()`, `GetClientDrops()` (uint64),
|
||||
`GetStallEvictions()`.
|
||||
- UDP-only keys rejected by `TCPStreamer`: `MulticastGroup`, `Interface`,
|
||||
`DataPort`, `ClientTimeout`. By `TCPStreamerClient`: `MulticastGroup`,
|
||||
`DataPort`, `Interface`, `KeepAliveInterval`.
|
||||
- `TCPSClient` keys: `ServerAddr`, `Port`, `SilenceTimeout` (default 0 =
|
||||
disabled, TCP reports dead peers itself), `ReconnectDelay` (default 2 s),
|
||||
`CPUMask`, `StackSize`. `TCPStreamerClient` takes `ServerAddress` as the
|
||||
UDP one does.
|
||||
- No client-side gap or stale-packet handling: the client delivers frames
|
||||
exactly as received. A bad magic, oversize length or fragmented frame drops
|
||||
the connection and counts a protocol error.
|
||||
- Tests: `Test/GTest/TCPSServerGTest.cpp`, `TCPSClientGTest.cpp`,
|
||||
`TCPStreamerGTest.cpp` (shared helpers in `TCPTestUtil.h`), built into
|
||||
`MainGTest.ex`. Not yet covered: Accumulate/Decimate/quantised signals over
|
||||
TCP (serialisation is inherited unchanged from `UDPStreamer`), and the E2E
|
||||
suite.
|
||||
|
||||
## 9. Known gap
|
||||
|
||||
- Default `MaxPayloadSize` (60000) is a first guess; confirm against
|
||||
Accumulate batch latency during testing.
|
||||
@@ -0,0 +1,86 @@
|
||||
# TCPStreamService: joining streams (multiplex + merge)
|
||||
|
||||
Date: 2026-11-06. Extends `2026-11-05-tcp-streamer-design.md` section 7.
|
||||
|
||||
## Goal
|
||||
|
||||
Let one consumer take several service streams through a single, cheaper path.
|
||||
Two independent mechanisms were requested ("both"):
|
||||
|
||||
1. **Multiplexed connection**: one TCP socket carries N streams, each keeping its
|
||||
own CONFIG, rate and timeline.
|
||||
2. **Merged stream**: the service builds one new stream whose CONFIG holds the
|
||||
signals of several source streams, emitted at the pace of a master source
|
||||
with sample-and-hold of the others.
|
||||
|
||||
## 1. Multiplexed connection
|
||||
|
||||
### Wire
|
||||
- `SUBSCRIBE` (type 5) payload becomes a comma separated list of stream names
|
||||
(`"fast,slow"`). One name is the old behaviour. Names may not contain `,`.
|
||||
Limits: 64 names, 4096 payload bytes, no duplicates.
|
||||
- The index of a stream is its position in the list.
|
||||
- Every server-to-client frame carries the stream index in `fragmentIdx`
|
||||
(`totalFragments` stays 1). Single-name connections always send 0, so
|
||||
existing clients are unchanged.
|
||||
- Any unknown name closes the whole connection (the client retries), exactly
|
||||
like the single-name case.
|
||||
|
||||
### Server
|
||||
- `TCPSConn` holds up to 64 stream attachments instead of one.
|
||||
- A stream's frame is built once; `Push` copies it into each subscriber's queue
|
||||
and stamps that connection's own index into `fragmentIdx`.
|
||||
- `TCPSFrameQueue` accounts per stream: `ClientQueueDepth` DATA frames *per
|
||||
stream*, drop-oldest of the same stream only, and a CONFIG only discards the
|
||||
stale frames of its own stream. A fast stream can therefore never evict a
|
||||
slow stream's frames.
|
||||
- `Flush` batches queued frames into one `writev()` (up to 16 frames), which is
|
||||
the optimisation for the many-small-frames case; it also applies to
|
||||
single-stream connections.
|
||||
- If a stream unregisters, every connection attached to it is closed.
|
||||
|
||||
### Client
|
||||
- `TCPSClient` accepts `StreamName = "a,b,c"`. It demultiplexes by
|
||||
`fragmentIdx` to one listener per stream and tracks the counter gap per
|
||||
stream.
|
||||
- `TCPSClient::GetStream(i)` returns a `UDPSClientI` facade for stream `i`.
|
||||
Facades reference-count `Start()`/`Stop()` of the shared connection, so
|
||||
existing consumers (StreamHub sessions) work unchanged.
|
||||
- `TCPStreamer` / `TCPStreamerClient` DataSources are single-stream: a comma in
|
||||
`StreamName` is rejected there.
|
||||
|
||||
### StreamHub
|
||||
- A source with `Transport="tcp"` and `StreamName="a,b,c"` creates one session
|
||||
per stream, all sharing one `TCPSClient`. Session labels are
|
||||
`<Label>_<stream>` (just the stream name when no label is given).
|
||||
|
||||
## 2. Merged stream (`TCPStreamJoin`)
|
||||
|
||||
```
|
||||
+Join = { Class = TCPStreamJoin Service = "Svc" StreamName = "all"
|
||||
Sources = "fast,slow" Master = "fast" }
|
||||
```
|
||||
- Root-level object, like the service. It taps the source streams
|
||||
(`TCPSStreamTapI`), and registers its own output stream with the service once
|
||||
every source is registered. Losing a source unregisters the output.
|
||||
- CONFIG: concatenated descriptors, names `<stream>_<signal>` (<= 63 chars),
|
||||
`timeSignalIdx` rebased, publish mode Strict, common `hrtFrequency` (sources
|
||||
with different frequencies are refused).
|
||||
- DATA (one per master frame): `[master timestamp][source0 body][source1 body]...`
|
||||
where a body is that source's last DATA payload without its 8-byte
|
||||
timestamp. Nothing is emitted until every source has delivered one DATA.
|
||||
- Only Strict and Decimate sources can be merged (Accumulate batches have
|
||||
incompatible lengths) and a source body whose size changes without a CONFIG
|
||||
change is ignored. Held values of a slower source are stamped with the
|
||||
master's timestamp unless that source uses a time signal.
|
||||
- Locking: join mutex, then the output stream mutex, then connection `qMutex`.
|
||||
Notifications from the service happen outside its topology lock.
|
||||
- No extra thread: the merged frame is built on the master producer's thread.
|
||||
|
||||
## Testing
|
||||
- GTest: queue per-stream accounting + writev partial writes; service mux
|
||||
(index tagging, fairness, unknown name, duplicate name, comma rejection);
|
||||
client demux/gaps/facades; join (config merge, sample-and-hold, refusal
|
||||
cases).
|
||||
- E2E: a chain scenario with a multiplexed hub source (`tcp_service` + comma
|
||||
list) and one with a merged stream.
|
||||
Reference in new issue
Block a user