feat(tcp): TCPStreamer DataSources, in-process TCPStreamService, stream multiplexing and TCPStreamJoin

Add TCPStreamer/TCPStreamerClient (TCP transport behind the UDPSServerI/UDPSClientI
seam), an optional single in-process TCPStreamService serving many streams over
worker threads, multiplexed SUBSCRIBE (comma list, stream index in fragmentIdx),
and TCPStreamJoin, which merges several streams into one sample-and-hold stream
paced by a master. StreamHub takes a comma list of streams over one connection.
Includes GTests, E2E scenarios s58-s73 and docs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Martino FerrariandClaude Opus 4.6 committed 2026-11-06 13:41:27 +01:00
1 parent 5562877c99
commit 50d85363a3
79 files changed
+12348 -138

No files matched your search

@@ -0,0 +1,86 @@
# TCPStreamService: joining streams (multiplex + merge)
Date: 2026-11-06. Extends `2026-11-05-tcp-streamer-design.md` section 7.
## Goal
Let one consumer take several service streams through a single, cheaper path.
Two independent mechanisms were requested ("both"):
1. **Multiplexed connection**: one TCP socket carries N streams, each keeping its
own CONFIG, rate and timeline.
2. **Merged stream**: the service builds one new stream whose CONFIG holds the
signals of several source streams, emitted at the pace of a master source
with sample-and-hold of the others.
## 1. Multiplexed connection
### Wire
- `SUBSCRIBE` (type 5) payload becomes a comma separated list of stream names
(`"fast,slow"`). One name is the old behaviour. Names may not contain `,`.
Limits: 64 names, 4096 payload bytes, no duplicates.
- The index of a stream is its position in the list.
- Every server-to-client frame carries the stream index in `fragmentIdx`
(`totalFragments` stays 1). Single-name connections always send 0, so
existing clients are unchanged.
- Any unknown name closes the whole connection (the client retries), exactly
like the single-name case.
### Server
- `TCPSConn` holds up to 64 stream attachments instead of one.
- A stream's frame is built once; `Push` copies it into each subscriber's queue
and stamps that connection's own index into `fragmentIdx`.
- `TCPSFrameQueue` accounts per stream: `ClientQueueDepth` DATA frames *per
stream*, drop-oldest of the same stream only, and a CONFIG only discards the
stale frames of its own stream. A fast stream can therefore never evict a
slow stream's frames.
- `Flush` batches queued frames into one `writev()` (up to 16 frames), which is
the optimisation for the many-small-frames case; it also applies to
single-stream connections.
- If a stream unregisters, every connection attached to it is closed.
### Client
- `TCPSClient` accepts `StreamName = "a,b,c"`. It demultiplexes by
`fragmentIdx` to one listener per stream and tracks the counter gap per
stream.
- `TCPSClient::GetStream(i)` returns a `UDPSClientI` facade for stream `i`.
Facades reference-count `Start()`/`Stop()` of the shared connection, so
existing consumers (StreamHub sessions) work unchanged.
- `TCPStreamer` / `TCPStreamerClient` DataSources are single-stream: a comma in
`StreamName` is rejected there.
### StreamHub
- A source with `Transport="tcp"` and `StreamName="a,b,c"` creates one session
per stream, all sharing one `TCPSClient`. Session labels are
`<Label>_<stream>` (just the stream name when no label is given).
## 2. Merged stream (`TCPStreamJoin`)
```
+Join = { Class = TCPStreamJoin Service = "Svc" StreamName = "all"
Sources = "fast,slow" Master = "fast" }
```
- Root-level object, like the service. It taps the source streams
(`TCPSStreamTapI`), and registers its own output stream with the service once
every source is registered. Losing a source unregisters the output.
- CONFIG: concatenated descriptors, names `<stream>_<signal>` (<= 63 chars),
`timeSignalIdx` rebased, publish mode Strict, common `hrtFrequency` (sources
with different frequencies are refused).
- DATA (one per master frame): `[master timestamp][source0 body][source1 body]...`
where a body is that source's last DATA payload without its 8-byte
timestamp. Nothing is emitted until every source has delivered one DATA.
- Only Strict and Decimate sources can be merged (Accumulate batches have
incompatible lengths) and a source body whose size changes without a CONFIG
change is ignored. Held values of a slower source are stamped with the
master's timestamp unless that source uses a time signal.
- Locking: join mutex, then the output stream mutex, then connection `qMutex`.
Notifications from the service happen outside its topology lock.
- No extra thread: the merged frame is built on the master producer's thread.
## Testing
- GTest: queue per-stream accounting + writev partial writes; service mux
(index tagging, fairness, unknown name, duplicate name, comma rejection);
client demux/gaps/facades; join (config merge, sample-and-hold, refusal
cases).
- E2E: a chain scenario with a multiplexed hub source (`tcp_service` + comma
list) and one with a merged stream.