feat(tcp): TCPStreamer DataSources, in-process TCPStreamService, stream multiplexing and TCPStreamJoin
Add TCPStreamer/TCPStreamerClient (TCP transport behind the UDPSServerI/UDPSClientI seam), an optional single in-process TCPStreamService serving many streams over worker threads, multiplexed SUBSCRIBE (comma list, stream index in fragmentIdx), and TCPStreamJoin, which merges several streams into one sample-and-hold stream paced by a master. StreamHub takes a comma list of streams over one connection. Includes GTests, E2E scenarios s58-s73 and docs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
1 parent
5562877c99
commit
50d85363a3
79 files changed
+12348
-138
No files matched your search
@@ -0,0 +1,86 @@
|
||||
# TCPStreamService: joining streams (multiplex + merge)
|
||||
|
||||
Date: 2026-11-06. Extends `2026-11-05-tcp-streamer-design.md` section 7.
|
||||
|
||||
## Goal
|
||||
|
||||
Let one consumer take several service streams through a single, cheaper path.
|
||||
Two independent mechanisms were requested ("both"):
|
||||
|
||||
1. **Multiplexed connection**: one TCP socket carries N streams, each keeping its
|
||||
own CONFIG, rate and timeline.
|
||||
2. **Merged stream**: the service builds one new stream whose CONFIG holds the
|
||||
signals of several source streams, emitted at the pace of a master source
|
||||
with sample-and-hold of the others.
|
||||
|
||||
## 1. Multiplexed connection
|
||||
|
||||
### Wire
|
||||
- `SUBSCRIBE` (type 5) payload becomes a comma separated list of stream names
|
||||
(`"fast,slow"`). One name is the old behaviour. Names may not contain `,`.
|
||||
Limits: 64 names, 4096 payload bytes, no duplicates.
|
||||
- The index of a stream is its position in the list.
|
||||
- Every server-to-client frame carries the stream index in `fragmentIdx`
|
||||
(`totalFragments` stays 1). Single-name connections always send 0, so
|
||||
existing clients are unchanged.
|
||||
- Any unknown name closes the whole connection (the client retries), exactly
|
||||
like the single-name case.
|
||||
|
||||
### Server
|
||||
- `TCPSConn` holds up to 64 stream attachments instead of one.
|
||||
- A stream's frame is built once; `Push` copies it into each subscriber's queue
|
||||
and stamps that connection's own index into `fragmentIdx`.
|
||||
- `TCPSFrameQueue` accounts per stream: `ClientQueueDepth` DATA frames *per
|
||||
stream*, drop-oldest of the same stream only, and a CONFIG only discards the
|
||||
stale frames of its own stream. A fast stream can therefore never evict a
|
||||
slow stream's frames.
|
||||
- `Flush` batches queued frames into one `writev()` (up to 16 frames), which is
|
||||
the optimisation for the many-small-frames case; it also applies to
|
||||
single-stream connections.
|
||||
- If a stream unregisters, every connection attached to it is closed.
|
||||
|
||||
### Client
|
||||
- `TCPSClient` accepts `StreamName = "a,b,c"`. It demultiplexes by
|
||||
`fragmentIdx` to one listener per stream and tracks the counter gap per
|
||||
stream.
|
||||
- `TCPSClient::GetStream(i)` returns a `UDPSClientI` facade for stream `i`.
|
||||
Facades reference-count `Start()`/`Stop()` of the shared connection, so
|
||||
existing consumers (StreamHub sessions) work unchanged.
|
||||
- `TCPStreamer` / `TCPStreamerClient` DataSources are single-stream: a comma in
|
||||
`StreamName` is rejected there.
|
||||
|
||||
### StreamHub
|
||||
- A source with `Transport="tcp"` and `StreamName="a,b,c"` creates one session
|
||||
per stream, all sharing one `TCPSClient`. Session labels are
|
||||
`<Label>_<stream>` (just the stream name when no label is given).
|
||||
|
||||
## 2. Merged stream (`TCPStreamJoin`)
|
||||
|
||||
```
|
||||
+Join = { Class = TCPStreamJoin Service = "Svc" StreamName = "all"
|
||||
Sources = "fast,slow" Master = "fast" }
|
||||
```
|
||||
- Root-level object, like the service. It taps the source streams
|
||||
(`TCPSStreamTapI`), and registers its own output stream with the service once
|
||||
every source is registered. Losing a source unregisters the output.
|
||||
- CONFIG: concatenated descriptors, names `<stream>_<signal>` (<= 63 chars),
|
||||
`timeSignalIdx` rebased, publish mode Strict, common `hrtFrequency` (sources
|
||||
with different frequencies are refused).
|
||||
- DATA (one per master frame): `[master timestamp][source0 body][source1 body]...`
|
||||
where a body is that source's last DATA payload without its 8-byte
|
||||
timestamp. Nothing is emitted until every source has delivered one DATA.
|
||||
- Only Strict and Decimate sources can be merged (Accumulate batches have
|
||||
incompatible lengths) and a source body whose size changes without a CONFIG
|
||||
change is ignored. Held values of a slower source are stamped with the
|
||||
master's timestamp unless that source uses a time signal.
|
||||
- Locking: join mutex, then the output stream mutex, then connection `qMutex`.
|
||||
Notifications from the service happen outside its topology lock.
|
||||
- No extra thread: the merged frame is built on the master producer's thread.
|
||||
|
||||
## Testing
|
||||
- GTest: queue per-stream accounting + writev partial writes; service mux
|
||||
(index tagging, fairness, unknown name, duplicate name, comma rejection);
|
||||
client demux/gaps/facades; join (config merge, sample-and-hold, refusal
|
||||
cases).
|
||||
- E2E: a chain scenario with a multiplexed hub source (`tcp_service` + comma
|
||||
list) and one with a merged stream.
|
||||
Reference in new issue
Block a user