martino/MARTe-Integrated-Components
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: martino/MARTe-Integrated-Components:892e3eae2889c4d3e7f433367856343c42fb6107
Branches Tags
martino/MARTe-Integrated-Components:main martino/MARTe-Integrated-Components:feature/udpscope martino/MARTe-Integrated-Components:feature/stress-report-integration
..
compare: martino/MARTe-Integrated-Components:main
Branches Tags
martino/MARTe-Integrated-Components:main martino/MARTe-Integrated-Components:feature/udpscope martino/MARTe-Integrated-Components:feature/stress-report-integration
4 Commits
892e3eae28 .. main
Author SHA1 Message Date
Martino FerrariandClaude Opus 4.6 5562877c99 fix(udps): publish the producer's HRT frequency so timestamps survive the hop 
DATA packets timestamp with the raw value of the producer's high-resolution
counter, and the wire never said how fast that counter runs. The hub divided by
its own timer's frequency instead, which is only the same number while producer
and hub share a machine — on x86 it is the TSC frequency and differs from model
to model. Off-box, every accumulated batch was therefore laid out over the wrong
span of time: the samples in it drift away from where they belong and start
colliding with the next packet's, which is the "same" symptom as a stale time
base even though nothing is out of order.

CONFIG now carries the rate as a trailing uint64, alongside the publish-mode
byte and read the same tolerant way: absent or zero means the producer did not
say, and the hub falls back to its own timer as before. Anything below 1 kHz is
not a high-resolution timer and is refused, so a mis-parsed payload cannot
stretch a millisecond batch across seconds.

The Accumulate DATA payload is unchanged, so this costs nothing per packet and
the period *within* a batch is still estimated from the gap between packets.

The Go, C and browser parsers already ignore trailer bytes they do not know,
so they read the new CONFIG unchanged; none of them uses the HRT timestamp.

Also corrects the Accumulate DATA layout in all three protocol documents: they
described it as one snapshot per array signal, where it has always been one per
accumulated cycle.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-09-02 02:49:50 +02:00
Martino FerrariandClaude Opus 4.6 092fd3c775 fix(streamhub): stop the trigger going deaf between captures 
TriggerEngine::CheckSample returned early in every state but ARMED, so an
edge arriving while a capture was being collected or handed out was
dropped, and the automatic rearm then waited for a FRESH edge. The engine
was therefore blind from its own trigger point until the capture had been
harvested — a post-window — and for the holdoff on top of that.

On a sparse pulse train that rounds the capture spacing up to a whole
pulse period: at the default 1 s window the blind stretch is 1 s, so a
1 Hz train was caught at 0.5 Hz and a wider window lost whole multiples.

The comparator now keeps running through COLLECTING and TRIGGERED and
remembers the first edge at or past trigTime + max(postSec, holdoffSec).
The holdoff guards against re-triggering on the ringing of the same
event and is measured from the trigger point, so it overlaps the
post-window rather than adding to it. Rearm() fires on the remembered
edge; it also keeps the tracked level, so the first sample after it has
a real predecessor instead of being spent seeding one.

Arm() stays the operator's arm and discards the held edge — they asked
for the next event, not one already been and gone — and SetConfig() and
Disarm() drop it too, since it was never judged against the new window.

This is the same defect and the same remedy already validated in the Go
hub (wshub/trigger.go, trigger_sporadic_test.go); the C++ hub had been
left with the original semantics.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-09-02 01:45:13 +02:00
Martino FerrariandClaude Opus 4.6 fbae7d712c fix(udps): stop packets being dated from an earlier time base 
Reported as samples sporadically carrying a previous packet's timestamp:
holes on one side of the stream and collisions on the other, in both the
Go and the MARTe2 receiver. That it appeared in both is what located it
-- the shared cause is upstream of either client. Four independent
defects, all of which end in a packet's values being placed at a time
that is not theirs.

Reassembly slot exhaustion (the "Reassembly slots full; evicting oldest"
flood). Chunk size was learnt only from fragment 0, so an out-of-order
burst destroyed a packet whose bytes had all arrived and left the slot
occupied until the 2 s GC. Slots were keyed on the counter alone, but
DATA and CONFIG number independently, so equal counters merged the two
streams. The 32-byte received-mask covered 256 of the 512 fragments the
client accepts, so a duplicate above 255 was counted as new and the
packet was delivered with a hole of stale bytes in it. And one datagram
was read per Execute(), which cannot drain a fast producer. Fixed with a
pendingTail deferral, (counter, type) keying, a 64-byte mask, a
256-datagram drain, counter-age slot reclamation, and a 1 Hz aggregated
warning in place of the per-eviction flood.

UDPStreamer dropping whole Accumulate batches. EventSem::ResetWait is
Reset-then-Wait, so a Post() landing while the sender thread was inside
ServiceClients()/SendData() was destroyed by the next Reset. The batch
was then skipped with dataReady false, readyFill was never cleared, and
the following flush overwrote it: an entire run of RT cycles never
reached the wire. The record of pending work now lives in the buffers
rather than in the semaphore edge, which also removes up to
UDPS_DATA_WAIT_MS of latency; genuine backpressure overwrites are
counted and reported. Against the unfixed code the new test sees
2999/3000 batches never consumed.

Period inflation after loss. Accumulated scalars carry no SamplingRate,
so the receiver derives dt from the sender-clock gap -- but dividing it
by the previous packet's sample count is only right while nothing is
lost. One loss doubles the reported period, which spreads a batch a full
batch past its own end and into the range the next packet claims. That
is the hole and the collision, exactly. Inferring the cycle count from
the estimate's own period is not a way out: it has a stable fixed point
wherever gap/dt is an integer, so a real rate change locks it at the old
one for good (AccumDtGTest.FollowsSustainedRateChange).

The packet counter removes the ambiguity, so all three receivers now
order on it: a DATA packet that does not advance the counter is dropped
rather than delivered, because its values are older than data already
handed over. Ordering is on the signed difference so it survives the
uint32 wrap, and the sequence resets on reconnect, where the producer's
counter restarts independently of ours. The loss count that falls out of
the same delta feeds the period estimate as cycles = prevN * (1 + lost),
which reduces exactly to gap/prevN when nothing is lost and therefore
still tracks a genuine rate change. UDPSClient::AcceptDataCounter (C++),
udpsprotocol.SequenceGate (Go), decode_data (C).

The C client's existing gap counter was wrap-unsafe and let a stale
packet rewind last_counter, which made every subsequent gap wrong; it
uses the same code now. Docs/Protocol.md gains an Ordering DATA section
stating the requirement for any receiver, including ones outside this
repository.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-09-02 01:18:49 +02:00
Martino FerrariandClaude Opus 4.6 f334995865 feat(webui): sporadic-trigger capture, CSV export and UI rework 
Brings the Go hub and web SPA work developed on feature/udpscope onto
main, without the udpscope client itself.

The trigger engine could not capture a sporadic event: it armed on the
live tail only, so a burst shorter than one push window was already past
by the time the FSM looked for it. It now searches the ring history for
the crossing, which also makes a capture reproducible from the same data
rather than dependent on push timing (wshub/trigger.go, ringbuf.go,
history.go).

Adds CSV/JSON export of the visible window (wshub/export.go) and reworks
the SPA: per-signal axis controls, a readable trigger panel, and a fix
for the flicker caused by repainting on every push instead of on a frame
tick (static/app.js, index.html, style.css).

BUFFER_AND_TRIGGER.md documents the ring/decimation/trigger interaction,
which is otherwise only inferable from the three files that implement it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-09-02 01:18:46 +02:00
66 changed files with 4587 additions and 2276 deletions
Expand all files Collapse all files
ARCHITECTURE.md
+11 -4
View File
@@ -117,9 +117,16 @@ Sent when the signal set changes or a client connects:
``` ```
[uint32 numSigs] [uint32 numSigs]
numSigs × UDPSSignalDescriptor (136 bytes each, packed) numSigs × UDPSSignalDescriptor (136 bytes each, packed)
[uint8 publishMode] 0=Strict/Decimate, 1=Accumulate [uint8 publishMode] 0=Strict, 1=Accumulate, 2=Decimate
[uint64 hrtFrequency] producer's HRT ticks per second; 0 = unknown
``` ```
Everything after the descriptors is an optional trailer: a receiver accepts a
payload that stops early and ignores bytes it does not know. `hrtFrequency` is
what lets a receiver on another host turn the raw counter in DATA into seconds
— without it the only option is the receiver's own timer, which agrees with the
producer only when the two share a machine.
### DATA Payload (Strict / Decimate modes) ### DATA Payload (Strict / Decimate modes)
``` ```
@@ -130,9 +137,9 @@ per-signal data in CONFIG order (quantised or raw, no inter-signal padding)
### DATA Payload (Accumulate mode) ### DATA Payload (Accumulate mode)
``` ```
[uint64 HRT timestamp] [uint64 HRT timestamp of the first slot in the batch]
[uint32 numSamples] [uint32 numSamples] RT cycles accumulated into this packet
for each signal: if scalar → numSamples elements; else → NumElements once for each signal, in CONFIG order: numSamples × NumElements values
``` ```
### Quantization / Dequantization ### Quantization / Dequantization
Client/udpscope/.gitignore
-2
View File
@@ -1,2 +0,0 @@
build/
compile_commands.json
Client/udpscope/CMakeLists.txt
-132
View File
@@ -1,132 +0,0 @@
cmake_minimum_required(VERSION 3.16)
project(UDPScope CXX C)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_C_STANDARD 99)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
option(UDPSCOPE_BUILD_TESTS "Build the unit tests" ON)
set(STREAMHUB_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../streamhub)
set(CCLIENT_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../../Common/Client/c)
# ── The standalone C UDPS client, compiled in directly ────────────────────────
# Building it here rather than shelling out to its own Makefile keeps this a
# single cmake --build away from a working binary.
add_library(udpsclient STATIC ${CCLIENT_DIR}/udps_client.c)
target_include_directories(udpsclient PUBLIC ${CCLIENT_DIR})
target_compile_options(udpsclient PRIVATE -Wall -Wextra -Wpedantic)
# ── System packages ───────────────────────────────────────────────────────────
find_package(OpenGL REQUIRED)
find_package(SDL2 QUIET CONFIG)
if(NOT SDL2_FOUND)
find_package(PkgConfig REQUIRED)
pkg_check_modules(SDL2 REQUIRED sdl2)
add_library(SDL2::SDL2 INTERFACE IMPORTED)
target_include_directories(SDL2::SDL2 INTERFACE ${SDL2_INCLUDE_DIRS})
target_link_libraries(SDL2::SDL2 INTERFACE ${SDL2_LIBRARIES})
target_compile_options(SDL2::SDL2 INTERFACE ${SDL2_CFLAGS_OTHER})
endif()
# ── Dear ImGui + ImPlot ───────────────────────────────────────────────────────
include(FetchContent)
FetchContent_Declare(imgui
GIT_REPOSITORY https://github.com/ocornut/imgui.git
GIT_TAG v1.91.8
GIT_SHALLOW TRUE)
FetchContent_MakeAvailable(imgui)
FetchContent_Declare(implot
GIT_REPOSITORY https://github.com/epezent/implot.git
GIT_TAG v0.17
GIT_SHALLOW TRUE)
FetchContent_MakeAvailable(implot)
add_library(imgui_lib STATIC
${imgui_SOURCE_DIR}/imgui.cpp
${imgui_SOURCE_DIR}/imgui_draw.cpp
${imgui_SOURCE_DIR}/imgui_tables.cpp
${imgui_SOURCE_DIR}/imgui_widgets.cpp
${imgui_SOURCE_DIR}/backends/imgui_impl_sdl2.cpp
${imgui_SOURCE_DIR}/backends/imgui_impl_opengl3.cpp
${implot_SOURCE_DIR}/implot.cpp
${implot_SOURCE_DIR}/implot_items.cpp)
target_include_directories(imgui_lib PUBLIC
${imgui_SOURCE_DIR} ${imgui_SOURCE_DIR}/backends ${implot_SOURCE_DIR})
target_link_libraries(imgui_lib PUBLIC SDL2::SDL2 OpenGL::GL)
target_compile_options(imgui_lib PRIVATE -w)
# ── Bundled resources, borrowed read-only from the StreamHub client ───────────
set(RESOURCE_DIR ${STREAMHUB_DIR}/resources)
set(FONT_DIR ${RESOURCE_DIR}/fonts)
if(EXISTS ${FONT_DIR}/fa-solid-900.ttf AND EXISTS ${FONT_DIR}/IconsFontAwesome6.h)
set(HAVE_FONT_AWESOME TRUE)
message(STATUS "Font Awesome icons enabled (${FONT_DIR})")
else()
set(HAVE_FONT_AWESOME FALSE)
message(WARNING "Bundled Font Awesome missing — using ASCII icon fallbacks")
endif()
# Guarded: file(COPY) is a hard configure error on a missing source, which
# would defeat the fallback the block above just chose.
if(EXISTS ${FONT_DIR})
file(COPY ${FONT_DIR} DESTINATION ${CMAKE_BINARY_DIR}/resources)
endif()
# ── Core library: everything except main.cpp, so tests can link it ────────────
set(CORE_SOURCES
Decimate.cpp
PaneTree.cpp
TimeBase.cpp
FrameDecoder.cpp
)
add_library(udpscope_core STATIC ${CORE_SOURCES})
target_include_directories(udpscope_core PUBLIC
${CMAKE_CURRENT_SOURCE_DIR}
${STREAMHUB_DIR}) # SignalBuffer.h, reused verbatim
target_link_libraries(udpscope_core PUBLIC udpsclient pthread)
target_compile_options(udpscope_core PRIVATE -Wall -Wextra -Wno-unused-parameter)
# ── Application ───────────────────────────────────────────────────────────────
set(APP_SOURCES
main.cpp
)
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/main.cpp)
add_executable(UDPScope ${APP_SOURCES})
target_link_libraries(UDPScope PRIVATE udpscope_core imgui_lib SDL2::SDL2 OpenGL::GL)
target_compile_definitions(UDPScope PRIVATE APP_RESOURCE_DIR="${RESOURCE_DIR}")
if(HAVE_FONT_AWESOME)
target_include_directories(UDPScope PRIVATE ${FONT_DIR})
target_compile_definitions(UDPScope PRIVATE HAVE_FONT_AWESOME)
endif()
target_compile_options(UDPScope PRIVATE -Wall -Wextra -Wno-unused-parameter)
install(TARGETS UDPScope DESTINATION bin)
install(DIRECTORY ${FONT_DIR} DESTINATION share/udpscope)
endif()
# ── Tests ─────────────────────────────────────────────────────────────────────
if(UDPSCOPE_BUILD_TESTS)
FetchContent_Declare(googletest
GIT_REPOSITORY https://github.com/google/googletest.git
GIT_TAG v1.15.2
GIT_SHALLOW TRUE)
set(gtest_force_shared_crt ON CACHE BOOL "" FORCE)
FetchContent_MakeAvailable(googletest)
file(GLOB TEST_SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/tests/*.cpp)
add_executable(udpscope_tests ${TEST_SOURCES})
target_link_libraries(udpscope_tests PRIVATE udpscope_core GTest::gtest_main)
target_compile_options(udpscope_tests PRIVATE -Wall -Wextra -Wno-unused-parameter)
enable_testing()
include(GoogleTest)
gtest_discover_tests(udpscope_tests)
endif()
Client/udpscope/Decimate.cpp
-47
View File
@@ -1,47 +0,0 @@
#include "Decimate.h"
#include <algorithm>
namespace udpscope {
void MinMaxDecimate(const double* t, const double* v, size_t n,
size_t maxPoints, Series& out) {
out.clear();
if (n == 0 || t == nullptr || v == nullptr) {
return;
}
if (n <= maxPoints || maxPoints < 4) {
out.t.assign(t, t + n);
out.v.assign(v, v + n);
return;
}
/* Two points per bucket, so the bucket count is half the budget. */
const size_t buckets = maxPoints / 2;
out.t.reserve(buckets * 2);
out.v.reserve(buckets * 2);
for (size_t b = 0; b < buckets; b++) {
const size_t begin = (n * b) / buckets;
size_t end = (n * (b + 1)) / buckets;
if (end <= begin) { end = begin + 1; }
if (end > n) { end = n; }
size_t lo = begin, hi = begin;
for (size_t i = begin + 1; i < end; i++) {
if (v[i] < v[lo]) { lo = i; }
if (v[i] > v[hi]) { hi = i; }
}
const size_t first = std::min(lo, hi);
const size_t second = std::max(lo, hi);
out.t.push_back(t[first]);
out.v.push_back(v[first]);
if (second != first) {
out.t.push_back(t[second]);
out.v.push_back(v[second]);
}
}
}
} /* namespace udpscope */
Client/udpscope/Decimate.h
-25
View File
@@ -1,25 +0,0 @@
/**
* @file Decimate.h
* @brief Min/max envelope decimation for screen rendering.
*/
#pragma once
#include "Types.h"
namespace udpscope {
/**
* @brief Reduce n points to at most maxPoints by emitting each bucket's
* minimum and maximum, in time order.
*
* LTTB is deliberately not used. It selects representative points and will
* silently drop a one-sample glitch; on a scope that glitch is usually the
* thing being looked for. The emitted pair stays in time order rather than
* value order because callers binary-search the result by time.
*
* Input shorter than maxPoints is copied through unchanged.
*/
void MinMaxDecimate(const double* t, const double* v, size_t n,
size_t maxPoints, Series& out);
} /* namespace udpscope */
Client/udpscope/FrameDecoder.cpp
-192
View File
@@ -1,192 +0,0 @@
#include "FrameDecoder.h"
#include <cmath>
namespace udpscope {
/** Fallback cycle period before the first inter-packet gap is known. */
static constexpr double kDefaultDt = 1.0e-3;
/**
* How far the forward-chained prediction for an accumulated burst may sit from
* where arrival time says it should be before the chain is abandoned.
*
* A kernel draining a backlog of queued datagrams can legitimately put the
* prediction a few hundred milliseconds ahead of arrival, so the threshold has
* to be well clear of that. Anything larger is not delivery jitter: it is lost
* packets or a declared sampling rate that does not match the producer's real
* one, and both must resynchronise rather than accumulate forever. Same value
* and same reasoning as ClockOffset::kRecalibThresholdS.
*/
static constexpr double kBurstResyncThresholdS = 0.5;
void FrameDecoder::setSignals(const std::vector<SignalMeta>& signals) {
signals_ = signals;
state_.assign(signals_.size(), SigState{});
hrtFit_.reset();
}
void FrameDecoder::reset() {
state_.assign(signals_.size(), SigState{});
hrtFit_.reset();
}
void FrameDecoder::beginFrame(const FrameView& f) {
if (f.hrt != 0u) { hrtFit_.add(f.hrt, f.recvTime); }
}
bool FrameDecoder::packetBurst(uint32_t idx, uint32_t nElems, double wallNow,
std::vector<double>& tsOut) {
SigState& st = state_[idx];
if (!st.lastPacketValid || wallNow <= st.lastPacketWall) {
/* No previous arrival to span from, or time went backwards. Remember
* this one and drop the samples rather than store them at made-up
* spacing. */
st.lastPacketWall = wallNow;
st.lastPacketValid = true;
return false;
}
const double dt = (wallNow - st.lastPacketWall) / static_cast<double>(nElems);
tsOut.resize(nElems);
for (uint32_t e = 0; e < nElems; e++) {
tsOut[e] = st.lastPacketWall + static_cast<double>(e + 1u) * dt;
}
st.lastPacketWall = wallNow;
return true;
}
bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
std::vector<double>& tsOut) {
tsOut.clear();
if (idx >= signals_.size() || idx >= f.numSignals || f.counts == nullptr) {
return false;
}
const SignalMeta& d = signals_[idx];
const uint32_t nElems = f.counts[idx];
if (nElems == 0u) { return false; }
const double wallNow = f.recvTime;
SigState& st = state_[idx];
const bool hasTimeSig = d.hasTimeSignal(f.numSignals);
const uint32_t tIdx = hasTimeSig ? d.timeSignalIdx : 0u;
const double tScale = hasTimeSig
? TimeSignalScale(signals_[tIdx].typeCode)
: 1.0e-6;
/* Rule 1: one stamp per element, straight from the time signal. */
if (d.timeMode == kTimeFullArray && hasTimeSig &&
f.counts[tIdx] >= nElems && f.values[tIdx] != nullptr) {
const double* tv = f.values[tIdx];
const double t0 = tv[0] * tScale;
(void) st.offset.map(t0, wallNow);
const double base = st.offset.offset();
tsOut.resize(nElems);
for (uint32_t e = 0; e < nElems; e++) {
tsOut[e] = base + tv[e] * tScale;
}
return true;
}
/* Rule 2: anchor from the time signal, spread by the sampling rate. */
if ((d.timeMode == kTimeFirstSample || d.timeMode == kTimeLastSample) &&
hasTimeSig && f.counts[tIdx] >= 1u && f.values[tIdx] != nullptr) {
const double anchor = st.offset.map(f.values[tIdx][0] * tScale, wallNow);
const double dt = (d.samplingRate > 0.0) ? (1.0 / d.samplingRate) : 0.0;
tsOut.resize(nElems);
for (uint32_t e = 0; e < nElems; e++) {
tsOut[e] = (d.timeMode == kTimeFirstSample)
? (anchor + static_cast<double>(e) * dt)
: (anchor - static_cast<double>(nElems - 1u - e) * dt);
}
return true;
}
/* Rule 3: accumulated scalar, based on declared sampling rate or hrt.
*
* When samplingRate is declared the inter-element step is exact and we
* anchor from the end of the previous burst rather than from arrival time
* or hrt. This makes the output immune to arrival jitter: even when the
* kernel delivers two packets microseconds apart each burst starts exactly
* one sample period after the previous burst ended.
*
* When samplingRate is absent we must derive dt from the hrt gap, which
* requires the HrtRateFit to be ready. Until then we fall back to
* packetBurst (arrival-time spanning), which is accurate during the normal
* pre-burst delivery phase that precedes the fit becoming ready. */
if (d.numElements() == 1u && nElems > 1u) {
const double dt = (d.samplingRate > 0.0)
? (1.0 / d.samplingRate)
: 0.0;
if (d.samplingRate > 0.0) {
/* Where arrival time says this burst begins: its last element was
* acquired just before the packet landed. */
const double arrivalAnchor =
wallNow - static_cast<double>(nElems - 1u) * dt;
/* Chaining from the end of the previous burst is immune to arrival
* jitter — a kernel draining several queued datagrams microseconds
* apart still yields contiguous timestamps. But a pure chain is
* blind: one lost datagram, or a declared rate that does not match
* the producer's real one, displaces every later sample and never
* recovers. So the chain is a PREDICTION, checked each packet
* against arrival and abandoned when the two disagree by more than
* a delivery backlog can explain. That bounds the error instead of
* letting it accumulate. */
double base = arrivalAnchor;
if (st.lastEmittedValid) {
const double predicted = st.lastEmittedEnd + dt;
if (std::fabs(predicted - arrivalAnchor) <= kBurstResyncThresholdS) {
base = predicted;
}
}
tsOut.resize(nElems);
for (uint32_t e = 0; e < nElems; e++) {
tsOut[e] = base + static_cast<double>(e) * dt;
}
st.lastEmittedEnd = tsOut[nElems - 1u];
st.lastEmittedValid = true;
return true;
}
/* No declared rate: need hrt-derived dt. */
if (!hrtFit_.ready()) {
return packetBurst(idx, nElems, wallNow, tsOut);
}
const double hrtSec = hrtFit_.toSeconds(f.hrt);
const double base = st.offset.map(hrtSec, wallNow);
double hrtDt;
if (st.lastAccValid && st.prevAccCount > 0u && hrtSec > st.lastAccHrtSec) {
/* The flushes carry contiguous RT cycles, so the gap divided by the
* previous packet's sample count is exactly one cycle period. */
hrtDt = (hrtSec - st.lastAccHrtSec) /
static_cast<double>(st.prevAccCount);
} else {
hrtDt = kDefaultDt;
}
tsOut.resize(nElems);
for (uint32_t e = 0; e < nElems; e++) {
tsOut[e] = base + static_cast<double>(e) * hrtDt;
}
st.lastAccHrtSec = hrtSec;
st.lastAccValid = true;
st.prevAccCount = nElems;
return true;
}
/* Rule 4: PACKET burst with no time reference at all. */
if (nElems > 1u) {
return packetBurst(idx, nElems, wallNow, tsOut);
}
/* Rule 5: plain scalar. */
tsOut.assign(1, wallNow);
return true;
}
} /* namespace udpscope */
Client/udpscope/FrameDecoder.h
-71
View File
@@ -1,71 +0,0 @@
/**
* @file FrameDecoder.h
* @brief Per-element timestamp reconstruction for UDPS frames.
*
* The C client's udps_frame_element_time() is explicitly an arrival-anchored
* estimate. It is not sufficient: the kernel frequently delivers several queued
* datagrams in one burst, so two packets are processed microseconds apart even
* though each represents ~10 ms of signal, and arrival-time interpolation then
* crams a packet's samples into that tiny gap — the trace renders as a sawtooth.
* Source/Applications/StreamHub/UDPSourceSession.cpp documents this failure and
* solves it; these are the same rules, computed from udps_frame_t's own fields
* so the scope and StreamHub agree on the same stream.
*/
#pragma once
#include "TimeBase.h"
#include "Types.h"
#include <vector>
namespace udpscope {
class FrameDecoder {
public:
/** Installs the signal table. Clears all per-signal timing history. */
void setSignals(const std::vector<SignalMeta>& signals);
const std::vector<SignalMeta>& signals() const { return signals_; }
/** Call once per frame, before any timestamps() call for that frame. */
void beginFrame(const FrameView& f);
/**
* @brief Timestamps for every value of signal @p idx in this frame.
* @return false when the signal produced nothing usable — an empty slot, or
* the first PACKET burst after connect, which has no previous
* arrival to span from and would otherwise poison the ring with
* wrongly spaced timestamps.
*/
bool timestamps(const FrameView& f, uint32_t idx, std::vector<double>& tsOut);
/** Forgets all timing history; call on reconnect. */
void reset();
private:
bool packetBurst(uint32_t idx, uint32_t nElems, double wallNow,
std::vector<double>& tsOut);
struct SigState {
ClockOffset offset;
double lastPacketWall = 0.0;
bool lastPacketValid = false;
double lastAccHrtSec = 0.0;
bool lastAccValid = false;
uint32_t prevAccCount = 0;
/** For accumulated scalars with a declared sampling rate: end timestamp
* of the most recently emitted burst. The next burst is PREDICTED to
* start one sample period after it — immune to arrival-time jitter —
* but the prediction is discarded when arrival time disagrees with it
* by more than a delivery backlog can explain, so packet loss cannot
* displace the trace permanently. */
double lastEmittedEnd = 0.0;
bool lastEmittedValid = false;
};
std::vector<SignalMeta> signals_;
std::vector<SigState> state_;
HrtRateFit hrtFit_;
};
} /* namespace udpscope */
Client/udpscope/PaneTree.cpp
-157
View File
@@ -1,157 +0,0 @@
#include "PaneTree.h"
#include <algorithm>
#include <cmath>
namespace udpscope {
PaneTree::PaneTree() : root_(new PaneNode()) {}
void PaneTree::setRoot(std::unique_ptr<PaneNode> node) {
if (node) { root_ = std::move(node); }
}
double PaneTree::clampRatio(double ratio, double extent) {
if (extent <= 2.0 * kMinPaneSize) {
return 0.5; /* Too small to honour the minimum on both sides. */
}
const double lo = kMinPaneSize / extent;
return std::min(std::max(ratio, lo), 1.0 - lo);
}
void PaneTree::layoutNode(PaneNode* node, const Rect& r,
std::vector<Placed>& leaves,
std::vector<Splitter>& splitters) {
if (node == nullptr) { return; }
if (node->leaf) {
leaves.push_back(Placed{node, r});
return;
}
if (node->orient == Orient::Columns) {
const double ratio = clampRatio(node->ratio, r.w);
const double wA = r.w * ratio;
layoutNode(node->a.get(), Rect{r.x, r.y, wA, r.h}, leaves, splitters);
layoutNode(node->b.get(), Rect{r.x + wA, r.y, r.w - wA, r.h}, leaves, splitters);
splitters.push_back(Splitter{
node,
Rect{r.x + wA - kSplitterGrab * 0.5, r.y, kSplitterGrab, r.h},
Orient::Columns});
} else {
const double ratio = clampRatio(node->ratio, r.h);
const double hA = r.h * ratio;
layoutNode(node->a.get(), Rect{r.x, r.y, r.w, hA}, leaves, splitters);
layoutNode(node->b.get(), Rect{r.x, r.y + hA, r.w, r.h - hA}, leaves, splitters);
splitters.push_back(Splitter{
node,
Rect{r.x, r.y + hA - kSplitterGrab * 0.5, r.w, kSplitterGrab},
Orient::Rows});
}
}
void PaneTree::layout(const Rect& area,
std::vector<Placed>& leaves,
std::vector<Splitter>& splitters) const {
leaves.clear();
splitters.clear();
layoutNode(root_.get(), area, leaves, splitters);
}
void PaneTree::splitLeaf(PaneNode* leaf, Orient orient) {
if (leaf == nullptr || !leaf->leaf) { return; }
/* Move the existing content into a new first child; the second is empty. */
std::unique_ptr<PaneNode> first(new PaneNode());
first->signals = std::move(leaf->signals);
first->profilePane = leaf->profilePane;
std::unique_ptr<PaneNode> second(new PaneNode());
leaf->leaf = false;
leaf->orient = orient;
leaf->ratio = 0.5;
leaf->signals.clear();
leaf->a = std::move(first);
leaf->b = std::move(second);
}
PaneNode* PaneTree::findParent(PaneNode* node, const PaneNode* child) {
if (node == nullptr || node->leaf) { return nullptr; }
if (node->a.get() == child || node->b.get() == child) { return node; }
if (PaneNode* p = findParent(node->a.get(), child)) { return p; }
return findParent(node->b.get(), child);
}
void PaneTree::closeLeaf(PaneNode* leaf) {
if (leaf == nullptr || !leaf->leaf) { return; }
PaneNode* parent = findParent(root_.get(), leaf);
if (parent == nullptr) {
return; /* The root is the only leaf; a scope with no pane is useless. */
}
std::unique_ptr<PaneNode> survivor =
(parent->a.get() == leaf) ? std::move(parent->b) : std::move(parent->a);
/* Collapse the parent into the survivor in place, so the parent pointer
* held by any caller stays valid. */
parent->leaf = survivor->leaf;
parent->signals = std::move(survivor->signals);
parent->profilePane = survivor->profilePane;
parent->orient = survivor->orient;
parent->ratio = survivor->ratio;
parent->a = std::move(survivor->a);
parent->b = std::move(survivor->b);
}
void PaneTree::setRatio(PaneNode* split, double ratio) {
if (split != nullptr && !split->leaf) {
split->ratio = std::min(std::max(ratio, 0.0), 1.0);
}
}
size_t PaneTree::countLeaves(const PaneNode* node) {
if (node == nullptr) { return 0; }
if (node->leaf) { return 1; }
return countLeaves(node->a.get()) + countLeaves(node->b.get());
}
size_t PaneTree::leafCount() const { return countLeaves(root_.get()); }
const PaneTree::Splitter* PaneTree::hitTestSplitter(
const std::vector<Splitter>& splitters, double px, double py) const {
for (const Splitter& s : splitters) {
if (s.rect.contains(px, py)) { return &s; }
}
return nullptr;
}
Handle PaneTree::hitTestHandle(const Rect& pane, double px, double py) {
if (!pane.contains(px, py)) { return Handle::None; }
const double relX = px - pane.x;
const double relY = py - pane.y;
const double midY = pane.h * 0.5;
const double midX = pane.w * 0.5;
const double half = kHandleSize * 0.5;
/* Close sits in the top-right corner and wins over the edge handles. */
if (relX >= pane.w - kHandleSize && relY <= kHandleSize) {
return Handle::Close;
}
if (relX <= kHandleSize && std::abs(relY - midY) <= half * 3.0) {
return Handle::Left;
}
if (relX >= pane.w - kHandleSize && std::abs(relY - midY) <= half * 3.0) {
return Handle::Right;
}
if (relY <= kHandleSize && std::abs(relX - midX) <= half * 3.0) {
return Handle::Top;
}
if (relY >= pane.h - kHandleSize && std::abs(relX - midX) <= half * 3.0) {
return Handle::Bottom;
}
return Handle::None;
}
} /* namespace udpscope */
Client/udpscope/PaneTree.h
-116
View File
@@ -1,116 +0,0 @@
/**
* @file PaneTree.h
* @brief Binary-space-partition layout of the plot area.
*
* Framework-free: no ImGui, no UDPS. The geometry and the hit-testing are the
* fiddly part of the pane UI and are unit-tested without a window.
*/
#pragma once
#include "Types.h"
#include <memory>
#include <string>
#include <vector>
namespace udpscope {
/** Direction a node splits its rectangle in. */
enum class Orient { Columns, Rows };
/** Vertical scaling strategy for one trace. */
enum class VMode { Auto, Range, Manual };
struct VScale {
VMode mode = VMode::Auto;
double div = 1.0; /**< Units per division, Manual only. */
double offset = 0.0; /**< Centre value, Manual only. */
};
/** One signal drawn in one pane. Signals are named, never indexed. */
struct Assignment {
std::string signalName;
Color color;
float lineWidth = 1.5f;
VScale vs;
};
/** Smallest a pane may be squeezed to, in pixels. */
constexpr double kMinPaneSize = 80.0;
/** Thickness of the splitter drag zone and of the inset handles, in pixels. */
constexpr double kSplitterGrab = 6.0;
constexpr double kHandleSize = 18.0;
/** What the pointer is over inside a pane. */
enum class Handle { None, Left, Right, Top, Bottom, Close };
struct PaneNode {
bool leaf = true;
/* leaf only */
std::vector<Assignment> signals;
bool profilePane = false; /**< Holds vector signals, not time series. */
/* split only */
Orient orient = Orient::Columns;
double ratio = 0.5; /**< First child's share of the parent. */
std::unique_ptr<PaneNode> a, b;
};
class PaneTree {
public:
struct Placed { PaneNode* leaf; Rect rect; };
struct Splitter { PaneNode* node; Rect rect; Orient orient; };
PaneTree();
PaneNode* root() { return root_.get(); }
const PaneNode* root() const { return root_.get(); }
/** Replaces the whole tree, e.g. when loading a session. */
void setRoot(std::unique_ptr<PaneNode> node);
/**
* @brief Walk the tree, producing every leaf's rectangle and every split's
* drag zone.
*/
void layout(const Rect& area,
std::vector<Placed>& leaves,
std::vector<Splitter>& splitters) const;
/** Turn a leaf into a split; the original content stays in the first child. */
void splitLeaf(PaneNode* leaf, Orient orient);
/** Replace the leaf's parent with its sibling. No-op on the last leaf. */
void closeLeaf(PaneNode* leaf);
void setRatio(PaneNode* split, double ratio);
size_t leafCount() const;
/** @return the splitter under the point, or nullptr. */
const Splitter* hitTestSplitter(const std::vector<Splitter>& splitters,
double px, double py) const;
/**
* @brief Which inset handle of @p pane the point is over.
*
* Handles sit inside the pane so they never overlap the splitter drag zone,
* and every pane has all four regardless of whether it touches a window
* edge — a pane in the middle of a 3x3 touches none.
*/
static Handle hitTestHandle(const Rect& pane, double px, double py);
private:
static void layoutNode(PaneNode* node, const Rect& r,
std::vector<Placed>& leaves,
std::vector<Splitter>& splitters);
static size_t countLeaves(const PaneNode* node);
static PaneNode* findParent(PaneNode* node, const PaneNode* child);
static double clampRatio(double ratio, double extent);
std::unique_ptr<PaneNode> root_;
};
} /* namespace udpscope */
Client/udpscope/TimeBase.cpp
-74
View File
@@ -1,74 +0,0 @@
#include "TimeBase.h"
#include <cmath>
namespace udpscope {
/* UDPS_T_UINT64 == 6 in Common/UDP/UDPSProtocol.h. Spelled numerically so this
* translation unit stays free of the C client header. */
static constexpr uint8_t kTypeUint64 = 6u;
double TimeSignalScale(uint8_t typeCode) {
return (typeCode == kTypeUint64) ? 1.0e-9 : 1.0e-6;
}
double ClockOffset::map(double producerSec, double wallSec) {
/* Symmetric on purpose. Delivery jitter of a few tens of ms either side of
* the prediction must not reset the offset or the whole trace wobbles, but a
* producer clock that steps in EITHER direction has to be picked up: a
* restart leaves the prediction behind the wall clock, an NTP correction on
* the producer's host leaves it ahead. A one-sided test silently never fires
* for the second case and the trace sits in the future for the whole run. */
if (!valid_ || std::fabs(wallSec - (offset_ + producerSec)) > kRecalibThresholdS) {
offset_ = wallSec - producerSec;
valid_ = true;
}
return offset_ + producerSec;
}
void HrtRateFit::reset() {
samples_.clear();
n_ = 0;
rate_ = 0.0;
}
void HrtRateFit::add(uint64_t hrt, double wallSec) {
samples_.push_back(Sample{static_cast<double>(hrt), wallSec});
if (samples_.size() > kWindow) { samples_.pop_front(); }
n_++;
if (n_ >= kMinSamples) { refit(); }
}
void HrtRateFit::refit() {
const size_t n = samples_.size();
if (n < 2) { return; }
/* Least squares slope of hrt against wall time. Both are subtracted from
* their first value first: raw hrt counts and epoch seconds are large
* enough that the naive sums lose precision. */
const double h0 = samples_.front().hrt;
const double w0 = samples_.front().wall;
double sw = 0.0, sh = 0.0, sww = 0.0, swh = 0.0;
for (const Sample& s : samples_) {
const double w = s.wall - w0;
const double h = s.hrt - h0;
sw += w;
sh += h;
sww += w * w;
swh += w * h;
}
const double dn = static_cast<double>(n);
const double denom = dn * sww - sw * sw;
if (std::fabs(denom) < 1e-12) { return; }
const double slope = (dn * swh - sw * sh) / denom;
if (slope > 0.0 && std::isfinite(slope)) { rate_ = slope; }
}
double HrtRateFit::toSeconds(uint64_t hrt) const {
if (rate_ <= 0.0) { return 0.0; }
return static_cast<double>(hrt) / rate_;
}
} /* namespace udpscope */
Client/udpscope/TimeBase.h
-87
View File
@@ -1,87 +0,0 @@
/**
* @file TimeBase.h
* @brief Producer-clock to wall-clock reconstruction.
*
* Framework-free. A UDPS stream's accurate timestamps come from a producer
* clock — either a declared time signal or the packet's embedded hrt — and both
* need mapping onto the client's wall clock before they can be plotted.
*/
#pragma once
#include <cstddef>
#include <cstdint>
#include <deque>
namespace udpscope {
/**
* @brief Seconds per count of a time signal, from its type code.
*
* The protocol carries uint64 time signals in nanoseconds and everything else
* in microseconds; this mirrors UDPSourceSession so the two agree on a stream.
*/
double TimeSignalScale(uint8_t typeCode);
/**
* @brief A latched producer-to-wall offset.
*
* Established from the first sample and then held, so network jitter does not
* wobble the trace. Only a drift beyond kRecalibThresholdS — a producer restart
* or re-phase, not delivery noise — forces a new calibration.
*/
class ClockOffset {
public:
static constexpr double kRecalibThresholdS = 0.5;
/** @return producerSec mapped onto wall clock. */
double map(double producerSec, double wallSec);
bool valid() const { return valid_; }
void reset() { valid_ = false; offset_ = 0.0; }
double offset() const { return offset_; }
private:
double offset_ = 0.0;
bool valid_ = false;
};
/**
* @brief Recovers the producer's hrt tick rate by least squares against arrival
* time.
*
* The protocol does not carry the tick rate, and StreamHub's approach of using
* the local MARTe HighResolutionTimer frequency is only valid when the client
* runs on the producer's host. A remote bench scope cannot assume that, so the
* rate is measured: hrt against recv_time is a straight line whose slope is
* ticks per second.
*/
class HrtRateFit {
public:
static constexpr size_t kMinSamples = 32;
static constexpr size_t kWindow = 256;
void add(uint64_t hrt, double wallSec);
bool ready() const { return n_ >= kMinSamples && rate_ > 0.0; }
double ticksPerSecond() const { return rate_; }
/**
* @brief Converts a tick count to seconds on the PRODUCER's own epoch.
*
* The fit recovers the slope only and discards the intercept, so this is
* `hrt / ticksPerSecond()` — not a wall-clock time. A producer's hrt counts
* from its own boot, not from the Unix epoch. Pass the result to
* ClockOffset::map() to land it on the wall clock; latching that arbitrary
* epoch difference is precisely what ClockOffset is for.
*/
double toSeconds(uint64_t hrt) const;
void reset();
private:
void refit();
struct Sample { double hrt; double wall; };
std::deque<Sample> samples_;
size_t n_ = 0;
double rate_ = 0.0;
};
} /* namespace udpscope */
Client/udpscope/Types.h
-103
View File
@@ -1,103 +0,0 @@
/**
* @file Types.h
* @brief Plain data shared across UDPScope modules. No logic, no dependencies.
*/
#pragma once
#include <cstddef>
#include <cstdint>
#include <string>
#include <vector>
namespace udpscope {
/** A time series as two parallel arrays, which is what ImPlot wants. */
struct Series {
std::vector<double> t;
std::vector<double> v;
void clear() { t.clear(); v.clear(); }
size_t size() const { return t.size(); }
bool empty() const { return t.empty(); }
};
/** RGBA in 0..1. Framework-free so PaneTree needs no ImGui. */
struct Color {
float r = 1.f, g = 1.f, b = 1.f, a = 1.f;
};
/** Screen rectangle in pixels. */
struct Rect {
double x = 0.0, y = 0.0, w = 0.0, h = 0.0;
bool contains(double px, double py) const {
return px >= x && px < (x + w) && py >= y && py < (y + h);
}
};
/* Protocol constants, spelled out rather than included, so the framework-free
* modules stay independent of udps_client.h. They mirror Common/UDP/UDPSProtocol.h. */
constexpr uint8_t kTimePacket = 0;
constexpr uint8_t kTimeFullArray = 1;
constexpr uint8_t kTimeFirstSample = 2;
constexpr uint8_t kTimeLastSample = 3;
constexpr uint32_t kNoTimeSignal = 0xFFFFFFFFu;
/** Framework-free mirror of udps_signal_t, plus UI state. */
struct SignalMeta {
std::string name;
uint8_t typeCode = 255;
uint8_t quantType = 0;
uint32_t numRows = 1;
uint32_t numCols = 1;
double rangeMin = 0.0;
double rangeMax = 0.0;
uint8_t timeMode = kTimePacket;
double samplingRate = 0.0;
uint32_t timeSignalIdx = kNoTimeSignal;
std::string unit;
/** User override: treat an ambiguous PACKET array as a profile, not a burst. */
bool profileOverride = false;
uint32_t numElements() const {
const uint64_t n = static_cast<uint64_t>(numRows ? numRows : 1u) *
static_cast<uint64_t>(numCols ? numCols : 1u);
return n == 0u ? 1u : static_cast<uint32_t>(n);
}
bool hasTimeSignal(uint32_t numSignals) const {
return timeSignalIdx != kNoTimeSignal && timeSignalIdx < numSignals;
}
/**
* @brief True when this array should be plotted against element index
* rather than unrolled onto the time axis.
*
* Only PACKET arrays are ambiguous: the producer stamped the whole datagram
* with one time, which is what a genuine vector looks like and also what a
* burst carrying no time metadata looks like. Default is burst, matching
* UDPSourceSession, with this flag as the user's override.
*/
bool isVectorProfile() const {
return profileOverride && numElements() > 1u && timeMode == kTimePacket;
}
};
/**
* @brief Non-owning mirror of udps_frame_t.
*
* Kept separate from the C struct so FrameDecoder can be tested with plain
* arrays and no socket. Points at memory owned by the caller.
*/
struct FrameView {
uint32_t counter = 0;
uint64_t hrt = 0;
double recvTime = 0.0;
uint32_t numSamples = 1;
uint32_t numSignals = 0;
const double* const* values = nullptr; /**< values[i][0..counts[i]) */
const uint32_t* counts = nullptr;
};
} /* namespace udpscope */
Client/udpscope/tests/DecimateTest.cpp
-90
View File
@@ -1,90 +0,0 @@
#include "Decimate.h"
#include <gtest/gtest.h>
#include <algorithm>
#include <vector>
using namespace udpscope;
TEST(MinMaxDecimate, PassesShortInputThroughUnchanged) {
const std::vector<double> t{0.0, 1.0, 2.0};
const std::vector<double> v{5.0, 6.0, 7.0};
Series out;
MinMaxDecimate(t.data(), v.data(), t.size(), 100, out);
EXPECT_EQ(out.t, t);
EXPECT_EQ(out.v, v);
}
// The whole reason for preferring min/max over LTTB: a single-sample spike is
// usually the thing the user is looking for, and it must survive decimation.
TEST(MinMaxDecimate, PreservesAnIsolatedSpike) {
std::vector<double> t(1000), v(1000, 0.0);
for (size_t i = 0; i < t.size(); i++) { t[i] = static_cast<double>(i); }
v[437] = 42.0;
Series out;
MinMaxDecimate(t.data(), v.data(), t.size(), 50, out);
ASSERT_FALSE(out.v.empty());
EXPECT_EQ(*std::max_element(out.v.begin(), out.v.end()), 42.0);
}
TEST(MinMaxDecimate, PreservesTheExtremesOfEveryBucket) {
std::vector<double> t(100), v(100);
for (size_t i = 0; i < t.size(); i++) {
t[i] = static_cast<double>(i);
v[i] = (i % 10 == 3) ? -9.0 : ((i % 10 == 7) ? 9.0 : 0.0);
}
Series out;
MinMaxDecimate(t.data(), v.data(), t.size(), 20, out);
EXPECT_EQ(*std::min_element(out.v.begin(), out.v.end()), -9.0);
EXPECT_EQ(*std::max_element(out.v.begin(), out.v.end()), 9.0);
}
// A ring whose timestamps are not monotonic breaks any later binary search by
// time, so the pair emitted per bucket must be ordered by time, not by value.
TEST(MinMaxDecimate, EmitsPointsInTimeOrder) {
// Two buckets of four. In the first the minimum comes before the maximum,
// in the second the order is reversed. An implementation that emitted
// (min, max) by value rather than by time passes on bucket 0 and fails on
// bucket 1, so this data exercises the swap that a monotonically growing
// ramp never triggers.
const double st[8] = {0, 1, 2, 3, 4, 5, 6, 7};
const double sv[8] = {-5, 0, 0, 9, 9, 0, 0, -5};
Series pair;
MinMaxDecimate(st, sv, 8, 4, pair);
ASSERT_EQ(pair.size(), 4u);
const double wantT[4] = {0, 3, 4, 7};
const double wantV[4] = {-5, 9, 9, -5};
for (size_t i = 0; i < 4; i++) {
EXPECT_EQ(pair.t[i], wantT[i]) << "time at " << i;
EXPECT_EQ(pair.v[i], wantV[i]) << "value at " << i;
}
std::vector<double> t(400), v(400);
for (size_t i = 0; i < t.size(); i++) {
t[i] = static_cast<double>(i);
v[i] = (i % 2 == 0) ? -static_cast<double>(i) : static_cast<double>(i);
}
Series out;
MinMaxDecimate(t.data(), v.data(), t.size(), 40, out);
ASSERT_GT(out.t.size(), 1u);
for (size_t i = 1; i < out.t.size(); i++) {
EXPECT_LE(out.t[i - 1], out.t[i]) << "at index " << i;
}
}
TEST(MinMaxDecimate, HandlesEmptyInput) {
Series out;
out.t.push_back(1.0); // must be cleared
MinMaxDecimate(nullptr, nullptr, 0, 10, out);
EXPECT_TRUE(out.t.empty());
EXPECT_TRUE(out.v.empty());
}
Client/udpscope/tests/FrameDecoderTest.cpp
-315
View File
@@ -1,315 +0,0 @@
#include "FrameDecoder.h"
#include <gtest/gtest.h>
#include <vector>
using namespace udpscope;
namespace {
/** Builds a FrameView over vectors the test owns. */
struct FrameBuilder {
std::vector<std::vector<double>> storage;
std::vector<const double*> ptrs;
std::vector<uint32_t> counts;
FrameView view;
void addSignal(std::vector<double> vals) {
storage.push_back(std::move(vals));
}
const FrameView& build(uint64_t hrt, double recvTime, uint32_t numSamples = 1) {
ptrs.clear();
counts.clear();
for (const auto& s : storage) {
ptrs.push_back(s.data());
counts.push_back(static_cast<uint32_t>(s.size()));
}
view.hrt = hrt;
view.recvTime = recvTime;
view.numSamples = numSamples;
view.numSignals = static_cast<uint32_t>(storage.size());
view.values = ptrs.data();
view.counts = counts.data();
return view;
}
};
SignalMeta burst(const char* name, uint8_t timeMode, double rate,
uint32_t elems, uint32_t timeIdx) {
SignalMeta m;
m.name = name;
m.typeCode = 8; /* float32 */
m.numRows = elems;
m.numCols = 1;
m.timeMode = timeMode;
m.samplingRate = rate;
m.timeSignalIdx = timeIdx;
return m;
}
SignalMeta timeSignal(const char* name, uint32_t elems) {
SignalMeta m;
m.name = name;
m.typeCode = 6; /* uint64 -> nanoseconds */
m.numRows = elems;
m.numCols = 1;
return m;
}
} /* namespace */
TEST(FrameDecoder, FullArrayTakesOneStampPerElementFromTheTimeSignal) {
FrameDecoder dec;
dec.setSignals({burst("Sine", kTimeFullArray, 1000.0, 4, 1),
timeSignal("Time", 4)});
FrameBuilder fb;
fb.addSignal({1.0, 2.0, 3.0, 4.0});
/* Nanoseconds: 5.000, 5.001, 5.002, 5.003 s of producer time. */
fb.addSignal({5.0e9, 5.001e9, 5.002e9, 5.003e9});
const FrameView& f = fb.build(0, 1000.0);
dec.beginFrame(f);
std::vector<double> ts;
ASSERT_TRUE(dec.timestamps(f, 0, ts));
ASSERT_EQ(ts.size(), 4u);
/* Element 0 lands on the arrival time; the rest keep the producer spacing. */
EXPECT_NEAR(ts[0], 1000.000, 1e-9);
EXPECT_NEAR(ts[1], 1000.001, 1e-9);
EXPECT_NEAR(ts[2], 1000.002, 1e-9);
EXPECT_NEAR(ts[3], 1000.003, 1e-9);
}
TEST(FrameDecoder, FirstSampleAnchorsElementZeroAndCountsForward) {
FrameDecoder dec;
dec.setSignals({burst("Sine", kTimeFirstSample, 1000.0, 4, 1),
timeSignal("Time", 1)});
FrameBuilder fb;
fb.addSignal({1.0, 2.0, 3.0, 4.0});
fb.addSignal({7.0e9});
const FrameView& f = fb.build(0, 2000.0);
dec.beginFrame(f);
std::vector<double> ts;
ASSERT_TRUE(dec.timestamps(f, 0, ts));
ASSERT_EQ(ts.size(), 4u);
EXPECT_NEAR(ts[0], 2000.000, 1e-9);
EXPECT_NEAR(ts[3], 2000.003, 1e-9);
}
TEST(FrameDecoder, LastSampleAnchorsTheFinalElementAndCountsBackward) {
FrameDecoder dec;
dec.setSignals({burst("Sine", kTimeLastSample, 1000.0, 4, 1),
timeSignal("Time", 1)});
FrameBuilder fb;
fb.addSignal({1.0, 2.0, 3.0, 4.0});
fb.addSignal({7.0e9});
const FrameView& f = fb.build(0, 3000.0);
dec.beginFrame(f);
std::vector<double> ts;
ASSERT_TRUE(dec.timestamps(f, 0, ts));
ASSERT_EQ(ts.size(), 4u);
EXPECT_NEAR(ts[3], 3000.000, 1e-9);
EXPECT_NEAR(ts[0], 3000.000 - 0.003, 1e-9);
}
TEST(FrameDecoder, PlainScalarUsesArrivalTime) {
FrameDecoder dec;
SignalMeta m;
m.name = "Level";
m.typeCode = 9;
dec.setSignals({m});
FrameBuilder fb;
fb.addSignal({42.0});
const FrameView& f = fb.build(0, 1234.5);
dec.beginFrame(f);
std::vector<double> ts;
ASSERT_TRUE(dec.timestamps(f, 0, ts));
ASSERT_EQ(ts.size(), 1u);
EXPECT_DOUBLE_EQ(ts[0], 1234.5);
}
// This is the failure UDPSourceSession.cpp:560 documents. The kernel delivers
// two queued datagrams microseconds apart even though each carries 10 ms of
// signal. Dating from arrival crams the second packet's samples into that gap
// and the trace becomes a sawtooth; dating from the producer hrt does not.
TEST(FrameDecoder, AccumulatedScalarSurvivesBurstyDelivery) {
FrameDecoder dec;
SignalMeta m;
m.name = "Acc";
m.typeCode = 9;
m.numRows = 1;
m.samplingRate = 1000.0; /* 1 kHz, 10 samples = 10 ms per packet */
dec.setSignals({m});
const double ticks = 1.0e9;
std::vector<double> all;
for (int p = 0; p < 40; p++) {
FrameBuilder fb;
fb.addSignal(std::vector<double>(10, static_cast<double>(p)));
const double producerSec = 100.0 + p * 0.010;
/* Packets 20+ arrive in a burst, all within 50 us of each other. */
const double arrival = (p < 20) ? (500.0 + p * 0.010)
: (500.2 + (p - 20) * 0.00005);
const FrameView& f = fb.build(static_cast<uint64_t>(producerSec * ticks),
arrival, 10);
dec.beginFrame(f);
std::vector<double> ts;
if (dec.timestamps(f, 0, ts)) {
all.insert(all.end(), ts.begin(), ts.end());
}
}
ASSERT_GT(all.size(), 300u);
for (size_t i = 1; i < all.size(); i++) {
EXPECT_GT(all[i], all[i - 1]) << "non-monotonic at " << i;
EXPECT_NEAR(all[i] - all[i - 1], 0.001, 2e-4)
<< "spacing collapsed at " << i << " (sawtooth)";
}
}
// The counterweight to the test above. Suppressing arrival jitter by chaining
// each burst onto the previous one is only safe while the chain is checked: on
// UDP, packets are lost, and a chain that ignores arrival entirely closes the
// hole silently and dates every later sample a full second early — for the rest
// of the run, because nothing ever pulls it back. The prediction has to be
// abandoned once arrival contradicts it by more than a delivery backlog could.
TEST(FrameDecoder, AccumulatedScalarResynchronisesAfterLostPackets) {
FrameDecoder dec;
SignalMeta m;
m.name = "Acc";
m.typeCode = 9;
m.numRows = 1;
m.samplingRate = 1000.0; /* 10 samples = 10 ms per packet */
dec.setSignals({m});
std::vector<double> ts;
for (int p = 0; p < 10; p++) {
FrameBuilder fb;
fb.addSignal(std::vector<double>(10, 1.0));
const FrameView& f = fb.build(0, 500.0 + p * 0.010, 10);
dec.beginFrame(f);
ASSERT_TRUE(dec.timestamps(f, 0, ts));
}
/* Contiguous so far: burst 9 ends at 500.090. */
EXPECT_NEAR(ts[9], 500.090, 1e-9);
/* A full second of packets never arrives. The next one lands at 501.100. */
FrameBuilder fb;
fb.addSignal(std::vector<double>(10, 1.0));
const FrameView& f = fb.build(0, 501.100, 10);
dec.beginFrame(f);
ASSERT_TRUE(dec.timestamps(f, 0, ts));
/* Chaining blindly would put this burst at 500.091..500.100, overlapping
* the gap as though no data were missing. */
EXPECT_NEAR(ts[0], 501.091, 1e-9);
EXPECT_NEAR(ts[9], 501.100, 1e-9);
}
TEST(FrameDecoder, AccumulatedScalarDerivesDtFromTheHrtGapWhenNoRateIsDeclared) {
FrameDecoder dec;
SignalMeta m;
m.name = "Acc";
m.typeCode = 9;
m.samplingRate = 0.0; /* undeclared */
dec.setSignals({m});
const double ticks = 1.0e9;
std::vector<double> last;
for (int p = 0; p < 40; p++) {
FrameBuilder fb;
fb.addSignal(std::vector<double>(10, 1.0));
const double producerSec = 100.0 + p * 0.010; /* 10 ms per packet */
const FrameView& f = fb.build(static_cast<uint64_t>(producerSec * ticks),
700.0 + p * 0.010, 10);
dec.beginFrame(f);
std::vector<double> ts;
if (dec.timestamps(f, 0, ts)) { last = ts; }
}
ASSERT_EQ(last.size(), 10u);
/* 10 ms of producer time across 10 samples is a 1 ms period. */
EXPECT_NEAR(last[1] - last[0], 0.001, 1e-5);
}
// A PACKET burst has no per-element time at all. Elements span
// (lastPacket, thisPacket] — backwards from arrival, because the samples were
// acquired before the packet landed. Forward extrapolation would let a jittered
// packet overlap the next one and break ring monotonicity.
TEST(FrameDecoder, PacketBurstDropsTheFirstFrameThenSpansBackwards) {
FrameDecoder dec;
dec.setSignals({burst("Raw", kTimePacket, 0.0, 5, kNoTimeSignal)});
FrameBuilder fb1;
fb1.addSignal({1.0, 2.0, 3.0, 4.0, 5.0});
const FrameView& f1 = fb1.build(0, 10.0);
dec.beginFrame(f1);
std::vector<double> ts;
EXPECT_FALSE(dec.timestamps(f1, 0, ts))
<< "the first packet has no previous arrival to span from";
FrameBuilder fb2;
fb2.addSignal({6.0, 7.0, 8.0, 9.0, 10.0});
const FrameView& f2 = fb2.build(0, 10.05);
dec.beginFrame(f2);
ASSERT_TRUE(dec.timestamps(f2, 0, ts));
ASSERT_EQ(ts.size(), 5u);
EXPECT_GT(ts[0], 10.0);
EXPECT_NEAR(ts[4], 10.05, 1e-12);
EXPECT_NEAR(ts[1] - ts[0], 0.01, 1e-12);
}
TEST(FrameDecoder, PacketBurstStaysMonotonicUnderJitteredArrivals) {
FrameDecoder dec;
dec.setSignals({burst("Raw", kTimePacket, 0.0, 8, kNoTimeSignal)});
const double jitter[] = {0.0, 0.004, -0.003, 0.006, -0.002, 0.0, 0.005, -0.004};
std::vector<double> all;
for (int p = 0; p < 8; p++) {
FrameBuilder fb;
fb.addSignal(std::vector<double>(8, 1.0));
const FrameView& f = fb.build(0, 20.0 + p * 0.05 + jitter[p]);
dec.beginFrame(f);
std::vector<double> ts;
if (dec.timestamps(f, 0, ts)) {
all.insert(all.end(), ts.begin(), ts.end());
}
}
ASSERT_GT(all.size(), 8u);
for (size_t i = 1; i < all.size(); i++) {
EXPECT_GT(all[i], all[i - 1]) << "packets overlapped at " << i;
}
}
TEST(FrameDecoder, ResetForgetsPerSignalHistory) {
FrameDecoder dec;
dec.setSignals({burst("Raw", kTimePacket, 0.0, 4, kNoTimeSignal)});
FrameBuilder fb;
fb.addSignal({1.0, 2.0, 3.0, 4.0});
const FrameView& f = fb.build(0, 5.0);
dec.beginFrame(f);
std::vector<double> ts;
EXPECT_FALSE(dec.timestamps(f, 0, ts));
const FrameView& f2 = fb.build(0, 5.1);
dec.beginFrame(f2);
EXPECT_TRUE(dec.timestamps(f2, 0, ts));
dec.reset();
const FrameView& f3 = fb.build(0, 5.2);
dec.beginFrame(f3);
EXPECT_FALSE(dec.timestamps(f3, 0, ts))
<< "after reset the next packet is again the first one";
}
Client/udpscope/tests/PaneTreeTest.cpp
-230
View File
@@ -1,230 +0,0 @@
#include "PaneTree.h"
#include <gtest/gtest.h>
using namespace udpscope;
namespace {
const Rect kScreen{0.0, 0.0, 1000.0, 600.0};
std::vector<PaneTree::Placed> leavesOf(const PaneTree& tree, const Rect& area) {
std::vector<PaneTree::Placed> leaves;
std::vector<PaneTree::Splitter> splitters;
tree.layout(area, leaves, splitters);
return leaves;
}
} /* namespace */
TEST(PaneTree, StartsAsOneEmptyLeafFillingTheArea) {
PaneTree tree;
EXPECT_EQ(tree.leafCount(), 1u);
const auto leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 1u);
EXPECT_DOUBLE_EQ(leaves[0].rect.w, 1000.0);
EXPECT_DOUBLE_EQ(leaves[0].rect.h, 600.0);
EXPECT_TRUE(leaves[0].leaf->signals.empty());
}
TEST(PaneTree, SplittingIntoColumnsHalvesTheWidth) {
PaneTree tree;
tree.splitLeaf(tree.root(), Orient::Columns);
const auto leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 2u);
EXPECT_DOUBLE_EQ(leaves[0].rect.w, 500.0);
EXPECT_DOUBLE_EQ(leaves[1].rect.w, 500.0);
EXPECT_DOUBLE_EQ(leaves[0].rect.h, 600.0);
EXPECT_DOUBLE_EQ(leaves[1].rect.x, 500.0);
}
TEST(PaneTree, SplittingIntoRowsHalvesTheHeight) {
PaneTree tree;
tree.splitLeaf(tree.root(), Orient::Rows);
const auto leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 2u);
EXPECT_DOUBLE_EQ(leaves[0].rect.h, 300.0);
EXPECT_DOUBLE_EQ(leaves[1].rect.y, 300.0);
EXPECT_DOUBLE_EQ(leaves[0].rect.w, 1000.0);
}
// The pane being split keeps its content; the new pane is the empty one.
TEST(PaneTree, SplitKeepsTheOriginalContentInTheFirstChild) {
PaneTree tree;
tree.root()->signals.push_back(Assignment{"Voltage", Color{}, 1.5f, VScale{}});
tree.splitLeaf(tree.root(), Orient::Columns);
const auto leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 2u);
ASSERT_EQ(leaves[0].leaf->signals.size(), 1u);
EXPECT_EQ(leaves[0].leaf->signals[0].signalName, "Voltage");
EXPECT_TRUE(leaves[1].leaf->signals.empty());
}
TEST(PaneTree, ClosingALeafGivesItsSpaceToTheSibling) {
PaneTree tree;
tree.splitLeaf(tree.root(), Orient::Columns);
auto leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 2u);
leaves[1].leaf->signals.push_back(Assignment{"Keep", Color{}, 1.5f, VScale{}});
tree.closeLeaf(leaves[0].leaf);
EXPECT_EQ(tree.leafCount(), 1u);
leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 1u);
EXPECT_DOUBLE_EQ(leaves[0].rect.w, 1000.0);
ASSERT_EQ(leaves[0].leaf->signals.size(), 1u);
EXPECT_EQ(leaves[0].leaf->signals[0].signalName, "Keep");
}
TEST(PaneTree, RefusesToCloseTheLastLeaf) {
PaneTree tree;
tree.closeLeaf(tree.root());
EXPECT_EQ(tree.leafCount(), 1u);
}
// A pane in the middle of a 3x3 touches no window edge. It must still be
// splittable, which is why handles are inset inside the pane rather than
// keyed on the window border.
TEST(PaneTree, AnInteriorPaneIsStillSplittable) {
PaneTree tree;
tree.splitLeaf(tree.root(), Orient::Rows); // top / bottom
auto leaves = leavesOf(tree, kScreen);
tree.splitLeaf(leaves[1].leaf, Orient::Rows); // 3 rows
leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 3u);
PaneNode* middle = leaves[1].leaf;
tree.splitLeaf(middle, Orient::Columns);
leaves = leavesOf(tree, kScreen);
tree.splitLeaf(leaves[2].leaf, Orient::Columns);
EXPECT_EQ(tree.leafCount(), 5u);
}
TEST(PaneTree, LayoutReportsOneSplitterPerSplitNode) {
PaneTree tree;
tree.splitLeaf(tree.root(), Orient::Columns);
auto leaves = leavesOf(tree, kScreen);
tree.splitLeaf(leaves[0].leaf, Orient::Rows);
std::vector<PaneTree::Placed> out;
std::vector<PaneTree::Splitter> splitters;
tree.layout(kScreen, out, splitters);
EXPECT_EQ(out.size(), 3u);
EXPECT_EQ(splitters.size(), 2u);
}
TEST(PaneTree, RatioSurvivesALayoutRoundTrip) {
PaneTree tree;
tree.splitLeaf(tree.root(), Orient::Columns);
tree.setRatio(tree.root(), 0.25);
const auto leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 2u);
EXPECT_DOUBLE_EQ(leaves[0].rect.w, 250.0);
EXPECT_DOUBLE_EQ(leaves[1].rect.w, 750.0);
}
TEST(PaneTree, RatioIsClampedSoNeitherPaneGoesBelowTheMinimum) {
PaneTree tree;
tree.splitLeaf(tree.root(), Orient::Columns);
tree.setRatio(tree.root(), 0.001);
const auto leaves = leavesOf(tree, kScreen);
EXPECT_GE(leaves[0].rect.w, kMinPaneSize);
EXPECT_GE(leaves[1].rect.w, kMinPaneSize);
}
TEST(PaneTree, HitTestFindsTheSplitterBetweenTwoPanes) {
PaneTree tree;
tree.splitLeaf(tree.root(), Orient::Columns);
std::vector<PaneTree::Placed> leaves;
std::vector<PaneTree::Splitter> splitters;
tree.layout(kScreen, leaves, splitters);
ASSERT_EQ(splitters.size(), 1u);
const PaneTree::Splitter* hit = tree.hitTestSplitter(splitters, 500.0, 300.0);
ASSERT_NE(hit, nullptr);
EXPECT_EQ(hit->orient, Orient::Columns);
EXPECT_EQ(tree.hitTestSplitter(splitters, 100.0, 300.0), nullptr);
}
TEST(PaneTree, HitTestFindsInsetSplitHandlesAndTheCloseButton) {
const Rect pane{0.0, 0.0, 400.0, 300.0};
EXPECT_EQ(PaneTree::hitTestHandle(pane, 8.0, 150.0), Handle::Left);
EXPECT_EQ(PaneTree::hitTestHandle(pane, 392.0, 150.0), Handle::Right);
EXPECT_EQ(PaneTree::hitTestHandle(pane, 200.0, 8.0), Handle::Top);
EXPECT_EQ(PaneTree::hitTestHandle(pane, 200.0, 292.0), Handle::Bottom);
EXPECT_EQ(PaneTree::hitTestHandle(pane, 392.0, 8.0), Handle::Close);
EXPECT_EQ(PaneTree::hitTestHandle(pane, 200.0, 150.0), Handle::None);
}
// Gap 1: closeLeaf only tested with first child closed; test closing the second child.
TEST(PaneTree, ClosingTheSecondLeafPreservesTheFirstLeafContent) {
PaneTree tree;
tree.splitLeaf(tree.root(), Orient::Columns);
auto leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 2u);
// Assign distinct signals to each leaf
leaves[0].leaf->signals.push_back(Assignment{"Signal_A", Color{}, 1.5f, VScale{}});
leaves[1].leaf->signals.push_back(Assignment{"Signal_B", Color{}, 1.5f, VScale{}});
// Close the second leaf; the first should survive with its content
tree.closeLeaf(leaves[1].leaf);
EXPECT_EQ(tree.leafCount(), 1u);
leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 1u);
ASSERT_EQ(leaves[0].leaf->signals.size(), 1u);
EXPECT_EQ(leaves[0].leaf->signals[0].signalName, "Signal_A");
}
// Gap 2: closeLeaf only tested at depth 1; test at depth 2 (deeper recursion in findParent).
TEST(PaneTree, ClosingALeafAtDepth2PreservesOthersAndUpdatesCount) {
PaneTree tree;
// Build tree: split root (a, b), split b to get depth-2 leaf in the RIGHT subtree
tree.splitLeaf(tree.root(), Orient::Columns); // depth 1: root splits into a, b
auto leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 2u);
tree.splitLeaf(leaves[1].leaf, Orient::Rows); // depth 2: b splits into b.a, b.b
leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 3u);
// Assign distinct signals to each of the three leaves
leaves[0].leaf->signals.push_back(Assignment{"Depth1_Left", Color{}, 1.5f, VScale{}});
leaves[1].leaf->signals.push_back(Assignment{"Depth2_TopRight", Color{}, 1.5f, VScale{}});
leaves[2].leaf->signals.push_back(Assignment{"Depth2_BottomRight", Color{}, 1.5f, VScale{}});
// Close the first depth-2 leaf (leaves[1], which is in the right subtree)
tree.closeLeaf(leaves[1].leaf);
EXPECT_EQ(tree.leafCount(), 2u);
leaves = leavesOf(tree, kScreen);
ASSERT_EQ(leaves.size(), 2u);
// Verify the surviving depth-2 leaf has its signal intact
bool found_left = false;
bool found_bottom_right = false;
for (const auto& leaf : leaves) {
ASSERT_EQ(leaf.leaf->signals.size(), 1u);
if (leaf.leaf->signals[0].signalName == "Depth1_Left") {
found_left = true;
}
if (leaf.leaf->signals[0].signalName == "Depth2_BottomRight") {
found_bottom_right = true;
}
}
EXPECT_TRUE(found_left);
EXPECT_TRUE(found_bottom_right);
}
Client/udpscope/tests/TimeBaseTest.cpp
-117
View File
@@ -1,117 +0,0 @@
#include "TimeBase.h"
#include <gtest/gtest.h>
using namespace udpscope;
TEST(ClockOffset, MapsTheFirstReadingOntoWallClockExactly) {
ClockOffset off;
EXPECT_FALSE(off.valid());
const double wall = 1756291200.5;
EXPECT_DOUBLE_EQ(off.map(10.0, wall), wall);
EXPECT_TRUE(off.valid());
}
// Network delay jitters the arrival time. If the offset chased every packet
// the whole trace would wobble, so it is latched and only corrected on real
// drift.
TEST(ClockOffset, HoldsTheOffsetThroughSmallArrivalJitter) {
ClockOffset off;
off.map(10.0, 1000.0); // offset = 990
// Arrival wanders either side of the prediction. wallSec is a local receive
// timestamp, so it only ever advances — jitter shows up as the gap growing
// and shrinking, never as the clock going backwards.
EXPECT_DOUBLE_EQ(off.map(11.0, 1001.02), 1001.0); // +0.02 late
EXPECT_DOUBLE_EQ(off.map(12.0, 1001.97), 1002.0); // -0.03 early
}
// The threshold has to be symmetric. A producer whose clock steps FORWARD (an
// NTP correction on the producer's host, say) puts the prediction permanently
// ahead of the wall clock — a one-sided "recalibrate only when wall is ahead"
// test never fires for it, and the trace sits in the future for the rest of the
// run.
TEST(ClockOffset, RecalibratesWhenTheProducerClockJumpsForward) {
ClockOffset off;
off.map(10.0, 1000.0); // offset = 990
// Producer leaps 100 s ahead while only 1 s of wall time passes.
EXPECT_DOUBLE_EQ(off.map(111.0, 1001.0), 1001.0);
}
TEST(ClockOffset, RecalibratesWhenDriftExceedsTheThreshold) {
ClockOffset off;
off.map(10.0, 1000.0); // offset = 990
/* Producer clock jumped (restart, re-phase): 5 s of error is not jitter. */
const double mapped = off.map(11.0, 1006.0);
EXPECT_DOUBLE_EQ(mapped, 1006.0);
}
TEST(ClockOffset, ResetForgetsTheCalibration) {
ClockOffset off;
off.map(10.0, 1000.0);
off.reset();
EXPECT_FALSE(off.valid());
EXPECT_DOUBLE_EQ(off.map(50.0, 2000.0), 2000.0);
}
// The tick rate of the producer's high-resolution timer is not carried by the
// protocol, and StreamHub's trick of using the local MARTe timer frequency only
// works on the producer's own host. Recover it from the data instead.
TEST(HrtRateFit, RecoversAKnownTickRate) {
HrtRateFit fit;
const double ticksPerSec = 2.5e9;
EXPECT_FALSE(fit.ready());
for (int i = 0; i < 64; i++) {
const double wall = 1000.0 + i * 0.01;
fit.add(static_cast<uint64_t>(wall * ticksPerSec), wall);
}
ASSERT_TRUE(fit.ready());
EXPECT_NEAR(fit.ticksPerSecond(), ticksPerSec, ticksPerSec * 1e-6);
}
TEST(HrtRateFit, IsNotReadyBeforeTheMinimumSampleCount) {
HrtRateFit fit;
for (size_t i = 0; i < HrtRateFit::kMinSamples - 1; i++) {
fit.add(static_cast<uint64_t>(i) * 1000000u, 1000.0 + i * 0.001);
}
EXPECT_FALSE(fit.ready());
fit.add(static_cast<uint64_t>(HrtRateFit::kMinSamples) * 1000000u,
1000.0 + HrtRateFit::kMinSamples * 0.001);
EXPECT_TRUE(fit.ready());
}
TEST(HrtRateFit, ToSecondsUsesTheFittedRate) {
HrtRateFit fit;
const double ticksPerSec = 1.0e9;
for (int i = 0; i < 64; i++) {
const double wall = 500.0 + i * 0.005;
fit.add(static_cast<uint64_t>(wall * ticksPerSec), wall);
}
ASSERT_TRUE(fit.ready());
EXPECT_NEAR(fit.toSeconds(2000000000ull), 2.0, 1e-4);
}
TEST(HrtRateFit, SurvivesAStalledClock) {
HrtRateFit fit;
for (int i = 0; i < 64; i++) {
fit.add(12345u, 1000.0 + i * 0.01); /* hrt never advances */
}
/* A degenerate fit must not produce a rate that would divide by zero, so it
* must decline to be ready at all. Guarding this behind `if (fit.ready())`
* would make the test vacuous: the branch never runs and a fit that
* declared itself ready with a rate of 0 or NaN would pass unnoticed. */
EXPECT_FALSE(fit.ready());
EXPECT_DOUBLE_EQ(fit.ticksPerSecond(), 0.0);
}
TEST(TimeSignalScale, UsesNanosecondsForUint64AndMicrosecondsOtherwise) {
EXPECT_DOUBLE_EQ(TimeSignalScale(6 /* UDPS_T_UINT64 */), 1.0e-9);
EXPECT_DOUBLE_EQ(TimeSignalScale(9 /* UDPS_T_FLOAT64 */), 1.0e-6);
EXPECT_DOUBLE_EQ(TimeSignalScale(4 /* UDPS_T_UINT32 */), 1.0e-6);
}
Client/udpstreamer/BUFFER_AND_TRIGGER.md
+425
View File
@@ -0,0 +1,425 @@
# Buffer time-window & trigger logic in `Client/udpstreamer`
How the UDP Scope client acquires, buffers, times and triggers waveforms.
The pipeline has two halves that must be read together:
- the **Go hub** (`Common/Client/go/wshub/`) — owns the UDP sockets, the
full-resolution sample storage, the disk history, and the trigger FSM;
- the **browser SPA** (`static/app.js`) — owns the display buffers, the rolling
window, and the trigger capture rendering.
The same SPA is also served by `Client/webui` and talks to the C++ StreamHub,
which mirrors the Go hub's behaviour (same trigger FSM states, same binary
frames). Everything below describes the Go-hub path; the wire contracts are
identical on both.
---
## 1. End-to-end data flow
```
MARTe2 RT app ──UDP/UDPS──▶ sources.go: runSession()
│ CONFIG + DATA packets, 17-byte header, HRT timestamp per frame
▼
udpsprotocol.ParseData() → []DataSample{HRTTimestamp, WallTime, Values}
▼
Hub.Run() dataCh → pending[sourceID] (drained every 30 Hz tick)
▼
buildBinaryDataMessageForSource()
├─ rebuild per-sample timestamps from TimeMode / calibration / monotonic snap
├─ h.ingest(key, n, t, v) ← FULL rate: ring.write + hist.write + trigger.feed
└─ minMaxDecimate(…, maxPushPoints=50) → WS binary v1 frame to clients
▼
browser: onBinaryData() → pushBuffer() into per-signal circular buffers
▼
renderDirtyPlots() (rAF loop) → buildUPlotData() → uPlot
```
The 30 Hz push is the **only** live path to the browser and it is decimated to
≤50 points/signal/tick. Everything that needs full resolution — zoom, trigger
captures, disk history — is fed independently through `ingest()` and never goes
over the wire until asked.
---
## 2. Hub-side buffers: `sigRing` (`wshub/ringbuf.go`)
One ring per `"sourceId:signalName"` key. A fixed-capacity circular buffer of
Float64 `(t, v)` pairs with a `sync.RWMutex` (writes from `Hub.Run()`, reads
from HTTP/WS handler goroutines).
### 2.1 Min/max bucketing
`bucket` is how many source samples collapse into **one min/max pair** on the
way in:
- `bucket == 1` — the stream is stored verbatim;
- `bucket > 1` — each group contributes its minimum and its maximum, emitted in
time order (`flushBucketLocked`), so the stored timestamps stay
non-decreasing (reads binary-search `rb.t`).
Bucketing is what lets an arbitrarily long window fit a fixed per-signal memory
budget at a megasample rate. Samples already stored keep the resolution they
were written at; the ring converges on a new bucket as it rolls
(`setBucket`).
### 2.2 Source-rate measurement
`sigRing` keeps its own source-sample accounting (`srcCount`, `srcT0`, `srcT1`,
reset every `srcRateWindowSec = 10 s`), because once `bucket > 1` neither `size`
nor the stored timespan measures the real incoming rate. `sourceRate()` is used
by the tuning sweep and by the history writer.
### 2.3 Ring tuning (`retuneRings`, every 1 s)
`activeWindowSec()` decides how far back the rings must reach:
1. an **armed trigger** owns the window: `cfg.windowSec + captureLagSec`
(`captureLagSec = captureMarginSec + 1/30 ≈ 0.183 s` — the capture is read
out a post-window + margin + one push tick after the trigger, so the rings
must hold that much extra or the front of the capture has already rolled);
2. otherwise the **widest window any connected client is displaying**
(`wsClient.displayWindowSec`, set by the SPA's `setWindow` command), with a
`defaultLiveWindowSec = 10 s` fallback while nobody has said;
Then per ring, with `budget = ringBudget()` (default `defaultRingPts = 10 M`,
floor `ringCapInitial = 250 k`):
- grow to the budget first (`grow()` preserves all samples, never shrinks);
- compute the needed bucket with `ringBucketFor(rate, window, capacity)`
(`ceil(2·rate·window·ringHeadroom / capacity)`, `ringHeadroom = 1.25`);
- apply it with **hysteresis**: keep the current bucket while its coverage is
between `need` and `2·need`, so a rate jittering across the boundary does not
flip the resolution every second.
The history archive is re-sized from the same window (`hist.setWindow`) so a
zoom or capture that outlives the rings can fall back to it.
### 2.4 Reading: `slice(t0, t1)`
Binary search for `t0` then `t1` over the circular layout, returning copies of
the pairs in `[t0, t1]`. Safe to use without holding the lock.
---
## 3. Disk history (`wshub/history.go`)
Optional (`EnableHistory`, `CloseHistory`), enabled by the hub configuration.
Every sample goes to disk through `ingest → hist.write` at full rate, in files
sized for the *current* window (not a retention period). It exists to back
three things the rings cannot:
- **zoom past the window**: `readRange(key, t0, t1, maxOut)`;
- **captures the rings have rolled past**: `captureRange(trigTime−pre, trigTime+post)`
lifts each capture into a file of its own so nothing overwrites it before the
next trigger;
- **short captures**: `backfillCaptureHead` prepends the front of the window the
ring no longer holds (the ring only *becomes* as long as the window after a
re-tune; the archive was written straight through).
---
## 4. Live push to the browser
`Hub.Run()` drains `pending[sourceID]` on a 30 Hz ticker. Even with no client
connected the frame is built: that is what keeps feeding rings, history and the
trigger, and keeps push cursors advancing so a late client does not get a
backlog burst.
`buildBinaryDataMessageForSource` reconstructs per-sample timestamps per signal
`TimeMode`:
| Mode | Timestamp reconstruction |
|---|---|
| `FirstSample` / `LastSample` | scalar TimeSignal value × `timerToSec` (µs→s or ns→s for u64), calibrated once against `WallTime`; samples spaced by `1/SamplingRate` |
| `FullArray` | per-element TimeSignal array, calibrated once against `WallTime` |
| scalar (`n == 1`) | `WallTime` of the UDP arrival |
| `PacketTime` (default, n>1) | inter-packet wall-clock gaps divided by n (single-packet ticks use the gap from the previous tick) |
**Monotonic snapping** (optional, `setMonotonic` command / "Sync TS" checkbox):
when enabled, the inter-frame anchor gap is smoothed with an EMA
(`monotonicEMAAlpha = 0.01`, initialised from the nominal `n·dt`) and small
deviations (< `monotonicTolerance = 5 ms`) are snapped to the smoothed gap,
removing the software-dispatch jitter overlaps/gaps described in the StreamHub
docs while tracking the true hardware rate (no accumulated drift).
The live frame is a **binary v1** WS message:
```
[u8 1][u8 srcIdLen][srcId][u32 nSigs]
{[u16 keyLen][key][u32 N][f64 t×N][f64 v×N]}
```
with each signal min/max-decimated to `maxPushPoints = 50` (`minMaxDecimate`:
the range is split into `threshold/2` buckets, each contributing its min and max
in time order — a scope-style envelope that keeps glitches on screen).
---
## 5. Browser-side buffers (`static/app.js`)
### 5.1 Capacity & growth
- `MAX_CAP = 2 000 000` — hard ceiling per buffer (~32 MB/signal at Float64 t+v);
- `DEFAULT_CAP = 100 000` — starting size for scalars;
- `TEMPORAL_CAP = 500 000` — starting size for array signals (the hub pushes
≤50 pts/signal/tick, so this already covers ~5 min);
- `growBufferForWindow(buf, windowSec)` — **sizes from the buffer's own span**,
not the signal's sampling rate: the incoming rate here is the hub-decimated
~1.5 kpts/s regardless of the source rate, so rate-based sizing overshot by
three orders of magnitude. Grows only when the buffer is full, to
`windowSec × 1.5` headroom, capped at `MAX_CAP`.
- `growBuffer` copies all existing samples into a larger array (preserving
circular order).
### 5.2 The window
`windowSec` (default 5 s, options 1 s … 10 min) is the rolling viewport.
Changing it:
1. updates `windowSec`;
2. `sendWindow()` → WS `setWindow` → hub `displayWindowSec` → ring re-tune;
3. grows every local buffer via `growBufferForWindow`;
4. evicts the decimation cache (a different window invalidates all cached
renderings).
The rolling "now" anchor is **data-driven, not wall-clock**:
`computePlotNow(p)` takes the newest timestamp of each contributing source and
uses the min-of-max over sources that are still active (a source lagging the
fastest by more than `windowSec` is treated as stale and excluded). This keeps
the window tracking real data regardless of clock skew between hub and browser.
### 5.3 Slicing & rendering
- `getBufferSliceRange(buf, t0, t1)` — binary search on the circular layout,
O(log n + window size);
- `getBufferSliceRangeWithBrackets` — same plus one point on each side so lines
still cross a nearly-empty zoom window;
- `supplementWithBrackets` — same bracketing for sparse server-fetched zoom data.
`buildLiveData(p)`:
1. slices every trace in `[t0, t1]`;
2. picks the **master** signal: highest `SamplingRate`, then most points;
3. decimates the master to ~2× plot width (`DECIM_MIN = 200` floor) via a
background worker (`decimateAsync`, stale-while-revalidate cache keyed per
plot/range/data-generation);
4. resamples every other trace onto the master grid with `resampleLinear`;
5. normalises Y (`applyVScaleNorm`: calibration `v·scale+offset`, then
`(y − offset)/div`).
### 5.4 Zoom
A zoom pins `p.xRange` and asks the hub for hi-res data over the exact range
(WS `zoom` request or HTTP `/api/zoom`). The hub answers from the full-res
rings — or from the **held copy of the last trigger capture** (`captureHold`
double buffer) while that window is still relevant — decimated to the requested
point budget. The browser prefers the fetched data when it exists, falls back
to its own circular buffers otherwise, and always brackets with local points.
---
## 6. Hub-side trigger FSM (`wshub/trigger.go`)
### 6.1 States and configuration
```
idle ──arm──▶ armed ──edge──▶ collecting ──window elapsed──▶ triggered
▲ ▲ (pre/post latched) │
│ └───────────── rearm (normal mode, after holdoff) ◀─────────┘
└────────────── disarm / single mode stays triggered
```
Configuration (`trigConfig`, client-settable via WS `setTrigger`):
| field | meaning | clamp |
|---|---|---|
| `signalKey` | `"src:sig"` or `"src:sig[i]"` | — |
| `edge` | `rising` / `falling` / `both` | — |
| `threshold` | **raw** units (SPA converts calibrated → raw) | — |
| `windowSec` | capture window | `[1e-4, 600]` |
| `prePercent` | pre-trigger share | `[0, 100]` |
| `mode` | `normal` (auto-rearm) / `single` | — |
| `holdoffSec` | re-arm delay after a capture, double-trigger guard | `[0, 60]` |
### 6.2 Edge detection (`feed`)
Called from `ingest` with every full-resolution batch for the trigger signal.
Level tracking (`prevValue`/`prevValid`) compares consecutive samples against
the threshold; `[i]`-suffixed keys stride the flattened batch by `nElem` to
watch one column. On a qualifying edge in `armed` state: `latchWindowLocked`
freezes `trigTime` and the pre/post split (so later config edits cannot move a
capture's axis).
### 6.3 Buffer-fill gate
Before accepting an edge, the FSM checks that the trigger signal's ring reaches
back far enough that the capture will come back whole (`fillLocked`):
```
need = windowSec − growth × postSec, floored at the pre-window
```
`growth` is the measured span-growth rate of the ring (`setBuffered`, refreshed
by `refreshTriggerFill` from the tick and from trigger commands). A still-filling
ring grows 1 s of span per second, so the gate reduces to the pre-window; a
full ring at a long window needs the whole window. While holding off, the level
is still tracked so the first edge after the gate opens is measured against the
right predecessor. The SPA shows the hold-off as an armed trigger with a
`bufferFill %` badge.
### 6.4 Window timing and the pending edge
`dueCapture` waits for the window on the **sample clock**, not the wall clock:
`lastT ≥ trigTime + post + captureMarginSec(0.15)`. This avoids cutting a
capture short when the stream's timestamps lag real time. Three ways it fires:
1. the samples themselves covered the window;
2. wall-clock fallback when no sample was ever seen (Force from idle);
3. `captureStallSec = 2 s` of stream silence — deliver what was collected
rather than leaving the client stuck in "collecting".
While a capture is in flight the comparator keeps running. The **first**
qualifying edge at/after `notBefore = trigTime + max(post, holdoffSec)` is
remembered (`pendingT`/`pendingValid`) and fired immediately on the automatic
`rearm()`. Without this the trigger was deaf through the whole post-window +
holdoff, which rounded sparse pulse trains up to whole periods (a 1 Hz train at
a 1 s window was caught at 0.5 Hz).
### 6.5 Holdoff and rearm
`markTriggered` moves `collecting → triggered` and, in `normal` mode (not
stopped), schedules `rearmAt = now + cfg.holdoffSec`. `dueRearm` consumes it;
`rearm()` re-arms immediately on a pending edge or returns to `armed`. The
holdoff is measured from the trigger point, overlapping the post-window rather
than adding to it.
`Force()` fires immediately at the most recent sample time (wall clock if no
sample yet) — the "Force" button.
### 6.6 Capture assembly (`buildTriggerCapture`)
On a due capture the hub builds the **binary v2** frame:
```
[u8 2][f64 trigTime][f64 preSec][f64 postSec][u32 nSig]
{[u16 keyLen][fullKey][u32 N][f64 t×N][f64 v×N]}
```
For every ring:
1. `slice(trigTime−pre, trigTime+post)`;
2. `backfillCaptureHead` from disk history for the front the ring lost;
3. if it is still short by more than `shortCaptureTol = 1 %` of the window,
log it explicitly (nothing can recover data the ring never held);
4. keep the **full-resolution** slice in the `captureHold` double buffer
(so a zoom into the capture can be answered after the rings roll past);
5. min/max-decimate to `trigCapturePts = 20 000` per signal for the wire —
a 60 s window at 1 MSps is ~960 MB raw per signal and would be dropped by
the send path anyway.
The double buffer is published (`capture.publish`) only once the frame is known
good, so a shot that yielded nothing leaves the previous capture on screen.
Dropped frames (client send-queue full) are logged.
`triggerTick` (every push tick) drives the whole FSM: re-tune rings → open
pending history files → refresh the fill measurement → due capture (send + mark
triggered + `hist.captureRange`) or due rearm → broadcast state only when it
changed (`stateUnsent`).
### 6.7 WS commands
| message | effect |
|---|---|
| `setTrigger {signal, edge, threshold, windowSec, prePercent, mode, holdoffSec}` | replace config |
| `arm` / `rearm` | explicit arm (discards pending edge) |
| `disarm` | → idle |
| `trigStop {stopped}` | pause/resume auto-rearm |
| `forceTrigger` | fire now |
Every command also refreshes the buffer-fill measurement synchronously — at
1 MSps the ring crosses the fill threshold many times inside one 33 ms tick, so
waiting for the next tick would fire on a stale measurement.
---
## 7. Browser-side trigger (`static/app.js`)
### 7.1 State handling
`onTriggerState(msg)` tracks the FSM broadcast:
- **armed** — shows `bufferFill %` while the hub is holding off on the fill
gate, so a trigger that is not yet fireable does not look broken;
- **collecting** — clears the previous snapshot, latches `trigTime` (and
`preSec`/`postSec` if the hub sent them), and lets live data sweep into the
trigger axis (see 7.3);
- **triggered / idle** — bookkeeping for the Rearm/Stop buttons.
### 7.2 Capture handling
`onTriggerCapture` parses the v2 frame into `trig.snapshot[key] = {t, v}` plus
the latched `_preS`/`_postS`. It is **ignored when the client did not enable
the trigger** (`trig.enabled`), because the hub keeps an armed trigger across
client sessions and applying a foreign capture would clobber this client's zoom
and scales. On receipt: the horizontal zoom is dropped so the whole capture is
visible, but **vertical scales (V/div, offset) persist** — they are user
settings and must survive from shot to shot.
### 7.3 Rendering modes (`buildUPlotData`)
| state | renderer | source |
|---|---|---|
| collecting, no snapshot yet | `buildTrigFillData` | live buffers, drawn on the *final* trigger axis (relative seconds, `[-pre, +post]`) so the trace sweeps in from the left |
| armed, not fired | freeze last frame | — |
| snapshot present | `buildTrigData` | the capture (or a hi-res zoom reply that covers ≥98 % of the view, else the snapshot) |
| otherwise | `buildLiveData` | rolling window |
`buildTrigData` converts to trigger-relative time (`t − trigT`), picks the
master by rate/count, decimates (cached per range+source-tag), resamples the
other traces, and normalises Y.
### 7.4 Threshold in calibrated units
The trigger threshold is held in **calibrated units** (what the user sees on
the Y axis). `sendTrigConfig()` inverts it through the signal's calibration
before sending: `raw = (calibrated − offset)/scale`, so the hub's raw
comparator fires exactly when `GAIN·signal + OFFSET` crosses the threshold.
The threshold line (`drawTriggerMarker`) maps the same calibrated threshold
through the signal's vscale: `y_norm = (threshold − offset)/div`.
---
## 8. Key constants
| constant | value | file |
|---|---|---|
| push rate | 30 Hz | `hub.go` |
| `maxPushPoints` (live) | 50 pts/signal/tick | `hub.go` |
| `trigCapturePts` (capture) | 20 000 pts/signal | `trigger.go` |
| `captureMarginSec` | 0.15 s | `trigger.go` |
| `captureStallSec` | 2.0 s | `trigger.go` |
| `autoRearmDelaySec` (default holdoff) | 0.2 s | `trigger.go` |
| `maxTriggerWindowSec` | 600 s | `trigger.go` |
| `ringBudget` default | 10 000 000 pts/signal | `hub.go` |
| `ringCapInitial` | 250 000 | `hub.go` |
| `ringCapScalar` | 100 000 | `hub.go` |
| `ringHeadroom` | 1.25 | `ringbuf.go` |
| `defaultLiveWindowSec` | 10 s | `ringbuf.go` |
| `captureLagSec` | 0.15 + 1/30 ≈ 0.183 s | `ringbuf.go` |
| `monotonicTolerance` | 5 ms | `hub.go` |
| `monotonicEMAAlpha` | 0.01 | `hub.go` |
| `MAX_CAP` (browser) | 2 000 000 pts/signal | `app.js` |
| `DEFAULT_CAP` / `TEMPORAL_CAP` | 100 000 / 500 000 | `app.js` |
| `DECIM_MIN` | 200 | `app.js` |
---
## 9. The C++ StreamHub mirror
The Go hub and the C++ StreamHub implement the same WS contracts and must stay
in sync (`AGENTS.md`): same `triggerState` FSM strings, same v2 capture frame,
same command set (`setTrigger` including `holdoffSec`, `arm`, `disarm`,
`trigStop`, `forceTrigger`), same `trigCapturePts`/`kTrigCapturePts` cap, and
the same ring/history windowing intent (`Source/Applications/StreamHub/`). A
protocol change on one side must be mirrored on the other.
Client/udpstreamer/go.mod
+11 -1
View File
@@ -1,12 +1,22 @@
module udpstreamer-webui module udpstreamer-webui
go 1.21 go 1.24.9
require marte2/common v0.0.0 require marte2/common v0.0.0
require ( require (
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.5.1 // indirect github.com/gorilla/websocket v1.5.1 // indirect
github.com/klauspost/compress v1.17.9 // indirect
github.com/parquet-go/bitpack v1.0.0 // indirect
github.com/parquet-go/jsonlite v1.0.0 // indirect
github.com/parquet-go/parquet-go v0.32.0 // indirect
github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/twpayne/go-geom v1.6.1 // indirect
golang.org/x/net v0.17.0 // indirect golang.org/x/net v0.17.0 // indirect
golang.org/x/sys v0.38.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
) )
replace marte2/common => ../../Common/Client/go replace marte2/common => ../../Common/Client/go
Client/udpstreamer/go.sum
+34
View File
@@ -1,4 +1,38 @@
github.com/DATA-DOG/go-sqlmock v1.5.2 h1:OcvFkGmslmlZibjAjaHm3L//6LiuBgolP7OputlJIzU=
github.com/DATA-DOG/go-sqlmock v1.5.2/go.mod h1:88MAG/4G7SMwSE3CeA0ZKzrT5CiOU3OJ+JlNzwDqpNU=
github.com/alecthomas/assert/v2 v2.10.0 h1:jjRCHsj6hBJhkmhznrCzoNpbA3zqy0fYiUcYZP/GkPY=
github.com/alecthomas/assert/v2 v2.10.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
github.com/alecthomas/repr v0.4.0 h1:GhI2A8MACjfegCPVq9f1FLvIBS+DrQ2KQBFZP1iFzXc=
github.com/alecthomas/repr v0.4.0/go.mod h1:Fr0507jx4eOXV7AlPV6AVZLYrLIuIeSOWtW57eE/O/4=
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY= github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY=
github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY= github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY=
github.com/hexops/gotextdiff v1.0.3 h1:gitA9+qJrrTCsiCl7+kh75nPqQt1cx4ZkudSTLoUqJM=
github.com/hexops/gotextdiff v1.0.3/go.mod h1:pSWU5MAI3yDq+fZBTazCSJysOMbxWL1BSow5/V2vxeg=
github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA=
github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw=
github.com/parquet-go/bitpack v1.0.0 h1:AUqzlKzPPXf2bCdjfj4sTeacrUwsT7NlcYDMUQxPcQA=
github.com/parquet-go/bitpack v1.0.0/go.mod h1:XnVk9TH+O40eOOmvpAVZ7K2ocQFrQwysLMnc6M/8lgs=
github.com/parquet-go/jsonlite v1.0.0 h1:87QNdi56wOfsE5bdgas0vRzHPxfJgzrXGml1zZdd7VU=
github.com/parquet-go/jsonlite v1.0.0/go.mod h1:nDjpkpL4EOtqs6NQugUsi0Rleq9sW/OtC1NnZEnxzF0=
github.com/parquet-go/parquet-go v0.32.0 h1:NWDqTUHfrCS4cJP/Fj2HlxvqsrVedWG3sayMkf+znzM=
github.com/parquet-go/parquet-go v0.32.0/go.mod h1:navtkAYr2LGoJVp141oXPlO/sxLvaOe3la2JEoD8+rg=
github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ=
github.com/pierrec/lz4/v4 v4.1.21/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/twpayne/go-geom v1.6.1 h1:iLE+Opv0Ihm/ABIcvQFGIiFBXd76oBIar9drAwHFhR4=
github.com/twpayne/go-geom v1.6.1/go.mod h1:Kr+Nly6BswFsKM5sd31YaoWS5PeDDH2NftJTK7Gd028=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM= golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE= golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
Client/udpstreamer/main.go
+1
View File
@@ -94,6 +94,7 @@ func main() {
http.Handle("/", http.FileServer(http.FS(sub))) http.Handle("/", http.FileServer(http.FS(sub)))
http.HandleFunc("/ws", hub.HandleWebSocket) http.HandleFunc("/ws", hub.HandleWebSocket)
http.HandleFunc("/api/zoom", hub.HandleZoom) http.HandleFunc("/api/zoom", hub.HandleZoom)
http.HandleFunc("/api/export", hub.HandleExport)
http.HandleFunc("/version", func(w http.ResponseWriter, r *http.Request) { http.HandleFunc("/version", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, buildVersion) fmt.Fprint(w, buildVersion)
}) })
Client/udpstreamer/static/app.js
+581 -111
View File
@@ -476,14 +476,24 @@ let cursorsDirty = false; // if true, redraw all plots to update cursor lines
// Rolling-window anchor used to keep cursors visually fixed while live data scrolls. // Rolling-window anchor used to keep cursors visually fixed while live data scrolls.
let _cursorAnchorNow = null; let _cursorAnchorNow = null;
// Horizontal value rulers — stored in normalized division units (the shared // Horizontal value rulers. The on/off toggle is global, but each plot keeps its
// y scale, -4.5…4.5) so one pair applies to every plot regardless of V/div. // own pair of normalized-division positions (rulerState), so dragging Y1 in
const rulers = { mode: 'off', yA: null, yB: null }; // one plot does not move it in the others.
const rulers = { mode: 'off', plotId: null };
const rulerState = {}; // plotId → { yA, yB }
// Layout — [label, cssClass, cols, rows] function getRulerState(plotId) {
if (!rulerState[plotId]) rulerState[plotId] = { yA: null, yB: null };
return rulerState[plotId];
}
// Layout — [label, cssClass, cols, rows, (optional) plotCount].
// Custom (non-uniform) layouts carry an explicit plotCount; the grid template
// and the spanning cells are defined in style.css under #plot-grid.<class>.
const LAYOUTS = [ const LAYOUTS = [
['1×1', 'l1x1', 1, 1], ['1×2', 'l1x2', 1, 2], ['2×1', 'l2x1', 2, 1], ['1×3', 'l1x3', 1, 3], ['1×1', 'l1x1', 1, 1], ['1×2', 'l1x2', 1, 2], ['2×1', 'l2x1', 2, 1], ['1×3', 'l1x3', 1, 3],
['3×1', 'l3x1', 3, 1], ['2×2', 'l2x2', 2, 2], ['1×4', 'l1x4', 1, 4], ['4×1', 'l4x1', 4, 1], ['3×1', 'l3x1', 3, 1], ['2×2', 'l2x2', 2, 2], ['1×4', 'l1x4', 1, 4], ['4×1', 'l4x1', 4, 1],
['1+2', 'l1p2', 2, 2, 3], // one plot spanning the top row, two below
]; ];
let currentLayout = 'l1x1'; let currentLayout = 'l1x1';
let colFrs = [1]; // fractional column sizes (sum = cols) let colFrs = [1]; // fractional column sizes (sum = cols)
@@ -816,6 +826,7 @@ function onConfig(msg) {
} }
buildSidebar(); buildSidebar();
buildTrigSignalSelect(); buildTrigSignalSelect();
maybeRestoreViewLate();
} }
/* ════════════════════════════════════════════════════════════════ /* ════════════════════════════════════════════════════════════════
@@ -925,12 +936,21 @@ function wsSend(obj) {
function sendWindow() { function sendWindow() {
wsSend({ type: 'setWindow', seconds: windowSec }); wsSend({ type: 'setWindow', seconds: windowSec });
} }
// trig.threshold is held in calibrated units. The hub's comparator runs on raw // trig.threshold is held in calibrated units. The hub.s comparator runs on raw
// samples, so invert on the way out: raw = (calibrated - offset) / scale. // samples, so invert on the way out: raw = (calibrated - offset) / scale.
function sendTrigConfig() { function sendTrigConfig() {
const cal = trig.signal ? calForKey(trig.signal) : Calib.IDENTITY; const cal = trig.signal ? calForKey(trig.signal) : Calib.IDENTITY;
// A negative calibration gain flips the signal on screen (v_cal = v_raw·scale
// + offset with scale < 0), so a calibrated rising edge is a raw FALLING
// edge. The hub compares raw samples, so send the raw direction that matches
// the edge the user picked on the calibrated trace.
let edge = trig.edge;
if (cal.scale < 0) {
if (edge === 'rising') edge = 'falling';
else if (edge === 'falling') edge = 'rising';
}
wsSend({ wsSend({
type: 'setTrigger', signal: trig.signal, edge: trig.edge, type: 'setTrigger', signal: trig.signal, edge: edge,
threshold: Calib.invertCal(trig.threshold, cal), windowSec: trig.windowSec, threshold: Calib.invertCal(trig.threshold, cal), windowSec: trig.windowSec,
prePercent: trig.prePercent, mode: trig.mode, holdoffSec: trig.holdoffSec, prePercent: trig.prePercent, mode: trig.mode, holdoffSec: trig.holdoffSec,
}); });
@@ -1332,7 +1352,11 @@ function decimateAsync(cacheKey, t, v, threshold, gen) {
return result; return result;
} }
} }
return cached || null; // stale entry, or nothing to draw yet // Never hand out a stale decimation: drawing it (at its old timestamps) and
// then the fresh one a frame later is what makes the trace jump/shimmer on
// every push. Return null instead — the caller holds the previous render
// until the worker's fresh result lands (it flags the plot for redraw).
return null;
} }
// Evict stale decimation cache entries for a plot (call when zoom range changes). // Evict stale decimation cache entries for a plot (call when zoom range changes).
@@ -1838,15 +1862,9 @@ function drawCursorLines(u, p) {
if (vNorm === null) return; if (vNorm === null) return;
const cy = u.valToPos(vNorm, 'y', true); const cy = u.valToPos(vNorm, 'y', true);
if (cy < bbox.top || cy > bbox.top + bbox.height) return; if (cy < bbox.top || cy > bbox.top + bbox.height) return;
// Un-transform normalized value back to real units for display // Calibrated value at the cursor time, from the raw source (matches
// y_norm = (y_raw - offset) / divValue → y_raw = y_norm * divValue + offset // the hover and the cursor readouts in every display mode).
const vs = sigVScale[vsKeyFor(p.id, key)]; const vReal = calibratedValueAt(key, val);
let vReal = vNorm;
if (vs) {
const dv = vs._resolvedDiv || vs.divValue || 1;
const ofs = vs._resolvedOffset != null ? vs._resolvedOffset : (vs.offset || 0);
vReal = vNorm * dv + ofs;
}
const tc = getSigStyle(key).color; const tc = getSigStyle(key).color;
// Diamond marker at intersection // Diamond marker at intersection
ctx.fillStyle = tc; ctx.fillStyle = tc;
@@ -1860,7 +1878,7 @@ function drawCursorLines(u, p) {
ctx.closePath(); ctx.closePath();
ctx.fill(); ctx.fill();
// Value text next to diamond (real units) // Value text next to diamond (real units)
const str = Math.abs(vReal) >= 10000 ? vReal.toExponential(2) : parseFloat(vReal.toPrecision(4)).toString(); const str = vReal === null ? '—' : (Math.abs(vReal) >= 10000 ? vReal.toExponential(2) : parseFloat(vReal.toPrecision(4)).toString());
ctx.fillStyle = tc; ctx.fillStyle = tc;
ctx.font = '11px monospace'; ctx.font = '11px monospace';
const currentAlign = ctx.textAlign; const currentAlign = ctx.textAlign;
@@ -1898,6 +1916,8 @@ function rulerRawValue(p, yNorm) {
// Draw the horizontal value rulers (called from the draw hook). // Draw the horizontal value rulers (called from the draw hook).
function drawRulerLines(u, p) { function drawRulerLines(u, p) {
if (rulers.mode !== 'on') return; if (rulers.mode !== 'on') return;
const rs = rulerState[p.id];
if (!rs) return;
const { ctx, bbox } = u; const { ctx, bbox } = u;
if (!bbox) return; if (!bbox) return;
@@ -1926,8 +1946,8 @@ function drawRulerLines(u, p) {
ctx.restore(); ctx.restore();
}; };
drawLine(rulers.yA, 'rgba(166,227,161,0.85)', 'Y1'); drawLine(rs.yA, 'rgba(166,227,161,0.85)', 'Y1');
drawLine(rulers.yB, 'rgba(243,139,168,0.85)', 'Y2'); drawLine(rs.yB, 'rgba(243,139,168,0.85)', 'Y2');
} }
// Compute the rolling-window anchor ("newest common timestamp") for a plot. // Compute the rolling-window anchor ("newest common timestamp") for a plot.
@@ -2133,8 +2153,9 @@ function createUPlot(p) {
const rect = p.uplot.over.getBoundingClientRect(); const rect = p.uplot.over.getBoundingClientRect();
const { min, max } = p.uplot.scales.y; const { min, max } = p.uplot.scales.y;
const toY = val => rect.top + (1 - (val - min) / (max - min)) * rect.height; const toY = val => rect.top + (1 - (val - min) / (max - min)) * rect.height;
if (rulers.yA !== null && Math.abs(clientY - toY(rulers.yA)) <= CURSOR_SNAP_PX) return 'A'; const rs = rulerState[p.id];
if (rulers.yB !== null && Math.abs(clientY - toY(rulers.yB)) <= CURSOR_SNAP_PX) return 'B'; if (rs && rs.yA !== null && Math.abs(clientY - toY(rs.yA)) <= CURSOR_SNAP_PX) return 'A';
if (rs && rs.yB !== null && Math.abs(clientY - toY(rs.yB)) <= CURSOR_SNAP_PX) return 'B';
return null; return null;
} }
@@ -2171,8 +2192,10 @@ function createUPlot(p) {
// Set cursor position immediately on mousedown // Set cursor position immediately on mousedown
if (yTarget) { if (yTarget) {
if (yTarget === 'A') rulers.yA = _rulerValFromEvent(e); rulers.plotId = p.id; // the readout follows the plot whose rulers moved
else rulers.yB = _rulerValFromEvent(e); const rs = getRulerState(p.id);
if (yTarget === 'A') rs.yA = _rulerValFromEvent(e);
else rs.yB = _rulerValFromEvent(e);
} else if (target === 'A') cursors.tA = _cursorValFromEvent(e); } else if (target === 'A') cursors.tA = _cursorValFromEvent(e);
else cursors.tB = _cursorValFromEvent(e); else cursors.tB = _cursorValFromEvent(e);
updateCursorReadout(); updateCursorReadout();
@@ -2180,8 +2203,9 @@ function createUPlot(p) {
const onMove = ev => { const onMove = ev => {
if (yTarget) { if (yTarget) {
if (yTarget === 'A') rulers.yA = _rulerValFromEvent(ev); const rs = getRulerState(p.id);
else rulers.yB = _rulerValFromEvent(ev); if (yTarget === 'A') rs.yA = _rulerValFromEvent(ev);
else rs.yB = _rulerValFromEvent(ev);
} else if (target === 'A') cursors.tA = _cursorValFromEvent(ev); } else if (target === 'A') cursors.tA = _cursorValFromEvent(ev);
else cursors.tB = _cursorValFromEvent(ev); else cursors.tB = _cursorValFromEvent(ev);
updateCursorReadout(); updateCursorReadout();
@@ -2447,8 +2471,12 @@ function buildLiveData(p) {
let dec; let dec;
if (cached) { if (cached) {
dec = cached; dec = cached;
} else if (p.uplot && p.uplot.data && p.uplot.data[0] && p.uplot.data[0].length) {
// Fresh decimation not ready yet — hold the previous render so the trace
// does not flicker between a stale decimation and the fresh one.
return p.uplot.data;
} else { } else {
// Worker job submitted — sync fallback this frame so the plot isn't blank. // First render: worker job submitted, nothing on screen yet — sync.
dec = decimate(masterRaw.t, masterRaw.v, targetPts); dec = decimate(masterRaw.t, masterRaw.v, targetPts);
} }
sharedT = dec.t; sharedT = dec.t;
@@ -2521,7 +2549,14 @@ function buildTrigData(p) {
// same-length snapshot slice for the same range, so it is tagged separately. // same-length snapshot slice for the same range, so it is tagged separately.
const cacheKey = `${p.id}:${masterKey}:${t0.toFixed(6)}:${t1.toFixed(6)}:${masterRaw.t.length}:${usedFetched ? 'hi' : 'snap'}`; const cacheKey = `${p.id}:${masterKey}:${t0.toFixed(6)}:${t1.toFixed(6)}:${masterRaw.t.length}:${usedFetched ? 'hi' : 'snap'}`;
const cachedDec = decimateAsync(cacheKey, masterRaw.t, masterRaw.v, targetPts); const cachedDec = decimateAsync(cacheKey, masterRaw.t, masterRaw.v, targetPts);
const dec = cachedDec || decimate(masterRaw.t, masterRaw.v, targetPts); let dec;
if (cachedDec) {
dec = cachedDec;
} else if (p.uplot && p.uplot.data && p.uplot.data[0] && p.uplot.data[0].length) {
return p.uplot.data; // hold the previous render until the fresh decimation lands
} else {
dec = decimate(masterRaw.t, masterRaw.v, targetPts);
}
// Convert absolute → relative seconds // Convert absolute → relative seconds
const sharedT = new Float64Array(dec.t.length); const sharedT = new Float64Array(dec.t.length);
for (let i = 0; i < dec.t.length; i++) sharedT[i] = dec.t[i] - trigT; for (let i = 0; i < dec.t.length; i++) sharedT[i] = dec.t[i] - trigT;
@@ -2578,8 +2613,15 @@ function buildTrigFillData(p) {
masterV = masterRaw.v; masterV = masterRaw.v;
} else { } else {
const cacheKey = `${p.id}:${masterKey}:trigfill`; const cacheKey = `${p.id}:${masterKey}:trigfill`;
const dec = decimateAsync(cacheKey, masterRaw.t, masterRaw.v, targetPts, _dataGen) || const decd = decimateAsync(cacheKey, masterRaw.t, masterRaw.v, targetPts, _dataGen);
decimate(masterRaw.t, masterRaw.v, targetPts); let dec;
if (decd) {
dec = decd;
} else if (p.uplot && p.uplot.data && p.uplot.data[0] && p.uplot.data[0].length) {
return p.uplot.data; // hold until the fresh decimation is ready
} else {
dec = decimate(masterRaw.t, masterRaw.v, targetPts);
}
sharedAbsT = dec.t; sharedAbsT = dec.t;
masterV = dec.v; masterV = dec.v;
} }
@@ -2753,7 +2795,19 @@ function updateCursorBtnVisibility() {
under one — so a zoom, a pan or a new capture can leave them outside the under one — so a zoom, a pan or a new capture can leave them outside the
viewport entirely, with no way to get them back: they are dragged by grabbing viewport entirely, with no way to get them back: they are dragged by grabbing
their line, and an off-screen line cannot be grabbed. */ their line, and an off-screen line cannot be grabbed. */
function resetRulers() {
// Re-place every plot's rulers at the default ±2 divisions, like
// resetCursors re-places the vertical cursors.
plots.forEach(p => {
const rs = getRulerState(p.id);
rs.yA = -2; rs.yB = 2;
});
updateCursorReadout();
cursorsDirty = true;
}
function resetCursors() { function resetCursors() {
resetRulers();
const refPlot = plots.find(p => p.uplot); const refPlot = plots.find(p => p.uplot);
if (!refPlot) return; if (!refPlot) return;
const { min, max } = refPlot.uplot.scales.x; const { min, max } = refPlot.uplot.scales.x;
@@ -2790,9 +2844,13 @@ document.getElementById('btn-ruler').addEventListener('click', () => {
rulers.mode = rulers.mode === 'off' ? 'on' : 'off'; rulers.mode = rulers.mode === 'off' ? 'on' : 'off';
const btn = document.getElementById('btn-ruler'); const btn = document.getElementById('btn-ruler');
btn.classList.toggle('active', rulers.mode === 'on'); btn.classList.toggle('active', rulers.mode === 'on');
if (rulers.mode === 'on' && rulers.yA === null && rulers.yB === null) { if (rulers.mode === 'on') {
// Auto-place at ±2 divisions from the centre on first use. // Auto-place every plot at ±2 divisions from the centre on first use;
rulers.yA = -2; rulers.yB = 2; // afterwards each plot keeps its own positions.
plots.forEach(pl => {
const rs = getRulerState(pl.id);
if (rs.yA === null && rs.yB === null) { rs.yA = -2; rs.yB = 2; }
});
} }
updateCursorReadout(); updateCursorReadout();
cursorsDirty = true; cursorsDirty = true;
@@ -2809,16 +2867,9 @@ function getValueAtCursor(p, t) {
if (!p.uplot || t === null) return null; if (!p.uplot || t === null) return null;
const key = plotActiveSignal[p.id] || (p.traces.length === 1 ? p.traces[0] : null); const key = plotActiveSignal[p.id] || (p.traces.length === 1 ? p.traces[0] : null);
if (!key) return null; if (!key) return null;
const idx = p.traces.indexOf(key); // Interpolate the raw wire value and apply the calibration explicitly, so
if (idx < 0) return null; // cursor readouts match the hover in every display mode.
const vNorm = interpAtTime(p.uplot, idx + 1, t); return calibratedValueAt(key, t);
if (vNorm === null) return null;
// Un-normalize: y_norm = (y_raw - offset) / divValue
const vs = sigVScale[p.id + ':' + key];
if (!vs) return vNorm;
const dv = vs._resolvedDiv != null ? vs._resolvedDiv : (vs.divValue || 1);
const ofs = vs._resolvedOffset != null ? vs._resolvedOffset : (vs.offset || 0);
return vNorm * dv + ofs;
} }
// Update per-plot cursor value readouts (A, B, ΔV) for all plots. // Update per-plot cursor value readouts (A, B, ΔV) for all plots.
@@ -2851,6 +2902,64 @@ function rawFromNorm(p, key, vNorm) {
const ofs = vs._resolvedOffset != null ? vs._resolvedOffset : (vs.offset || 0); const ofs = vs._resolvedOffset != null ? vs._resolvedOffset : (vs.offset || 0);
return vNorm * dv + ofs; return vNorm * dv + ofs;
} }
// Linear interpolation of a sorted (t, v) pair at absolute time tAbs. Returns
// null outside the data's range — never fabricated, so an export or readout
// cannot invent samples the signal never had.
function interpSortedRaw(t, v, tAbs) {
if (!t || t.length === 0) return null;
if (tAbs < t[0] || tAbs > t[t.length - 1]) return null;
let lo = 0, hi = t.length - 1;
while (lo < hi) { const m = (lo + hi) >> 1; if (t[m] < tAbs) lo = m + 1; else hi = m; }
if (lo === 0) return v[0] ?? null;
const t0 = t[lo - 1], t1 = t[lo];
const v0 = v[lo - 1], v1 = v[lo];
if (v0 == null || v1 == null) return v0 ?? v1 ?? null;
return v0 + (tAbs - t0) / (t1 - t0) * (v1 - v0);
}
// Binary-search linear interpolation of a circular buffer at time t.
function interpCircular(buf, t) {
if (!buf || buf.size === 0) return null;
const { cap, size, head } = buf;
const start = (size === cap) ? head : 0;
const physAt = k => (start + k) % cap;
let lo = 0, hi = size;
while (lo < hi) { const m = (lo + hi) >> 1; if (buf.t[physAt(m)] < t) lo = m + 1; else hi = m; }
if (lo === 0) return buf.v[physAt(0)] ?? null;
if (lo >= size) return buf.v[physAt(size - 1)] ?? null;
const t0 = buf.t[physAt(lo - 1)], t1 = buf.t[physAt(lo)];
const v0 = buf.v[physAt(lo - 1)], v1 = buf.v[physAt(lo)];
if (v0 == null || v1 == null) return v0 ?? v1 ?? null;
return v0 + (t - t0) / (t1 - t0) * (v1 - v0);
}
// Raw (uncalibrated) value of `key` at absolute time tAbs, from the best
// available raw source: trigger snapshot → fetched zoom data → live push
// buffer. All three store wire values, so calibration is applied here, at the
// point of display, exactly once.
function rawAtAbsTime(key, tAbs) {
if (trig.snapshot) {
const s = trig.snapshot[key];
if (s && s.t.length) { const v = interpSortedRaw(s.t, s.v, tAbs); if (v != null) return v; }
}
for (const p of plots) {
const zd = zoomData[p.id];
if (!zd) continue;
const s = zd.signals[key];
if (s && s.t.length) { const v = interpSortedRaw(s.t, s.v, tAbs); if (v != null) return v; }
}
const buf = buffers[key];
if (buf && buf.size) { const v = interpCircular(buf, tAbs); if (v != null) return v; }
return null;
}
// Calibrated value of `key` at axis time t. Under a trigger the axis is
// relative to the trigger instant, so convert to absolute first.
function calibratedValueAt(key, t) {
const tAbs = (inTrigWindow() && trig.trigTime != null) ? trig.trigTime + t : t;
const raw = rawAtAbsTime(key, tAbs);
return raw === null ? null : Calib.applyCal(raw, calForKey(key));
}
function hideHoverReadout() { function hideHoverReadout() {
document.getElementById('hover-readout').style.display = 'none'; document.getElementById('hover-readout').style.display = 'none';
@@ -2870,13 +2979,14 @@ function showHoverReadout(p, e) {
const tStr = inTrigWindow() ? fmtDuration(t, span, true) : fmtLiveTime(t, span); const tStr = inTrigWindow() ? fmtDuration(t, span, true) : fmtLiveTime(t, span);
let html = '<div class="hov-time">' + escHtml(tStr) + '</div>'; let html = '<div class="hov-time">' + escHtml(tStr) + '</div>';
p.traces.forEach((key, idx) => { p.traces.forEach((key, idx) => {
const vNorm = interpAtTime(p.uplot, idx + 1, t);
const name = key.includes(':') ? key.slice(key.indexOf(':') + 1) : key; const name = key.includes(':') ? key.slice(key.indexOf(':') + 1) : key;
// rawFromNorm inverts the vscale transform, which Task 7 made operate on
// calibrated values — so this is already in calibrated units.
const unit = unitForKey(key); const unit = unitForKey(key);
const val = vNorm === null ? '—' // Interpolate the raw wire value and apply the calibration explicitly,
: (_fmtVal(rawFromNorm(p, key, vNorm)) + (unit ? ' ' + unit : '')); // so the hover is correct in every display mode (analog, digital,
// mixed) and independent of the vscale state.
const vCal = calibratedValueAt(key, t);
const val = vCal === null ? '—'
: (_fmtVal(vCal) + (unit ? ' ' + unit : ''));
html += '<div class="hov-row"><span class="hov-dot" style="background:' + html += '<div class="hov-row"><span class="hov-dot" style="background:' +
escHtml(getSigStyle(key).color) + '"></span>' + escHtml(getSigStyle(key).color) + '"></span>' +
'<span class="hov-name">' + escHtml(name) + '</span>' + '<span class="hov-name">' + escHtml(name) + '</span>' +
@@ -2894,17 +3004,25 @@ function showHoverReadout(p, e) {
el.style.top = Math.max(4, y) + 'px'; el.style.top = Math.max(4, y) + 'px';
} }
// Update the Y1/Y2/ΔY ruler readout, expressed in the raw units of the first // Update the Y1/Y2/ΔY ruler readout, expressed in the raw units of the plot
// plot that has an active (or sole) signal. // whose rulers were last moved, falling back to the first plot with a signal.
function updateRulerReadout() { function updateRulerReadout() {
const box = document.getElementById('ruler-readout'); const box = document.getElementById('ruler-readout');
const on = rulers.mode === 'on'; const on = rulers.mode === 'on';
box.style.display = on ? '' : 'none'; box.style.display = on ? '' : 'none';
if (!on) return; if (!on) return;
const ref = plots.find(p => p.uplot && p.traces.length > 0 && let ref = null;
rulerRawValue(p, 0) !== null); if (rulers.plotId !== null) {
const conv = y => (y === null || !ref) ? null : rulerRawValue(ref, y); const pl = plots.find(p => p.id === rulers.plotId);
const vA = conv(rulers.yA), vB = conv(rulers.yB); if (pl && pl.uplot && pl.traces.length > 0) ref = pl;
}
if (!ref) {
ref = plots.find(p => p.uplot && p.traces.length > 0 &&
rulerRawValue(p, 0) !== null) || null;
}
const rs = ref ? rulerState[ref.id] : null;
const conv = y => (y === null || !ref || !rs) ? null : rulerRawValue(ref, y);
const vA = conv(rs ? rs.yA : null), vB = conv(rs ? rs.yB : null);
document.getElementById('cur-y1').textContent = 'Y1: ' + fmtVal(vA); document.getElementById('cur-y1').textContent = 'Y1: ' + fmtVal(vA);
document.getElementById('cur-y2').textContent = 'Y2: ' + fmtVal(vB); document.getElementById('cur-y2').textContent = 'Y2: ' + fmtVal(vB);
document.getElementById('cur-dy').textContent = document.getElementById('cur-dy').textContent =
@@ -3355,18 +3473,36 @@ function initPlotCfgBar(plotId, p) {
/* ════════════════════════════════════════════════════════════════ /* ════════════════════════════════════════════════════════════════
Layout management Layout management
════════════════════════════════════════════════════════════════ */ ════════════════════════════════════════════════════════════════ */
// Returns the number of plot cells in a layout (cols × rows). // Returns the number of plot cells in a layout. Custom layouts carry an
// explicit plotCount; uniform ones are cols × rows.
function layoutPlotCount(cls) { function layoutPlotCount(cls) {
const entry = LAYOUTS.find(l => l[1] === cls);
if (entry) {
if (entry.length >= 5) return entry[4];
return entry[2] * entry[3];
}
const m = cls.match(/^l(\d+)x(\d+)$/); const m = cls.match(/^l(\d+)x(\d+)$/);
return m ? parseInt(m[1]) * parseInt(m[2]) : 1; return m ? parseInt(m[1]) * parseInt(m[2]) : 1;
} }
// Build a small SVG grid thumbnail for a given cols×rows layout. // Build a small SVG grid thumbnail for a layout entry. Custom (non-uniform)
function layoutSVG(cols, rows) { // layouts draw their own cell arrangement.
function layoutSVG(entry) {
const W = 28, H = 20, GAP = 1.5, PAD = 1.5; const W = 28, H = 20, GAP = 1.5, PAD = 1.5;
let rects = '';
if (entry[1] === 'l1p2') {
// 1+2: one full-width cell on top, two side by side below.
const cw = (W - PAD * 2 - GAP) / 2;
const ch = (H - PAD * 2 - GAP) / 2;
const y2 = (PAD + ch + GAP).toFixed(1);
const x2 = (PAD + cw + GAP).toFixed(1);
rects += `<rect x="${PAD}" y="${PAD}" width="${(W - PAD * 2).toFixed(1)}" height="${ch.toFixed(1)}" rx="1.5"/>`;
rects += `<rect x="${PAD}" y="${y2}" width="${cw.toFixed(1)}" height="${ch.toFixed(1)}" rx="1.5"/>`;
rects += `<rect x="${x2}" y="${y2}" width="${cw.toFixed(1)}" height="${ch.toFixed(1)}" rx="1.5"/>`;
} else {
const [, , cols, rows] = entry;
const cw = (W - PAD * 2 - GAP * (cols - 1)) / cols; const cw = (W - PAD * 2 - GAP * (cols - 1)) / cols;
const ch = (H - PAD * 2 - GAP * (rows - 1)) / rows; const ch = (H - PAD * 2 - GAP * (rows - 1)) / rows;
let rects = '';
for (let r = 0; r < rows; r++) { for (let r = 0; r < rows; r++) {
for (let c = 0; c < cols; c++) { for (let c = 0; c < cols; c++) {
const x = (PAD + c * (cw + GAP)).toFixed(1); const x = (PAD + c * (cw + GAP)).toFixed(1);
@@ -3374,6 +3510,7 @@ function layoutSVG(cols, rows) {
rects += `<rect x="${x}" y="${y}" width="${cw.toFixed(1)}" height="${ch.toFixed(1)}" rx="1.5"/>`; rects += `<rect x="${x}" y="${y}" width="${cw.toFixed(1)}" height="${ch.toFixed(1)}" rx="1.5"/>`;
} }
} }
}
return `<svg xmlns="http://www.w3.org/2000/svg" width="${W}" height="${H}" viewBox="0 0 ${W} ${H}">` return `<svg xmlns="http://www.w3.org/2000/svg" width="${W}" height="${H}" viewBox="0 0 ${W} ${H}">`
+ `<rect width="${W}" height="${H}" rx="2" fill="#11111b"/>` + `<rect width="${W}" height="${H}" rx="2" fill="#11111b"/>`
+ `<g fill="#45475a">${rects}</g></svg>`; + `<g fill="#45475a">${rects}</g></svg>`;
@@ -3401,7 +3538,7 @@ function applyLayout(cls) {
// Update button label // Update button label
const btn = document.getElementById('btn-layout'); const btn = document.getElementById('btn-layout');
if (btn) btn.innerHTML = layoutSVG(cols, rows) + ' <span>' + label + '</span> ▾'; if (btn) btn.innerHTML = layoutSVG(entry) + ' <span>' + label + '</span> ▾';
// Update active state in menu // Update active state in menu
document.querySelectorAll('.layout-menu-item') document.querySelectorAll('.layout-menu-item')
@@ -3436,11 +3573,12 @@ function applyLayout(cls) {
function buildLayoutMenu() { function buildLayoutMenu() {
const menu = document.getElementById('layout-menu'); const menu = document.getElementById('layout-menu');
LAYOUTS.forEach(([label, cls, cols, rows]) => { LAYOUTS.forEach(entry => {
const [label, cls] = entry;
const item = document.createElement('button'); const item = document.createElement('button');
item.className = 'layout-menu-item' + (cls === currentLayout ? ' active' : ''); item.className = 'layout-menu-item' + (cls === currentLayout ? ' active' : '');
item.dataset.layout = cls; item.dataset.layout = cls;
item.innerHTML = layoutSVG(cols, rows) + '<span>' + label + '</span>'; item.innerHTML = layoutSVG(entry) + '<span>' + label + '</span>';
item.addEventListener('click', () => { item.addEventListener('click', () => {
applyLayout(cls); applyLayout(cls);
menu.classList.remove('open'); menu.classList.remove('open');
@@ -3467,9 +3605,20 @@ function buildLayoutMenu() {
/* ════════════════════════════════════════════════════════════════ /* ════════════════════════════════════════════════════════════════
Export CSV (all plots) — fetches full-resolution data from ring Export CSV (all plots) — fetches full-resolution data from ring
════════════════════════════════════════════════════════════════ */ ════════════════════════════════════════════════════════════════ */
// Shared busy state for the export dropdown: prevents re-entry and shows
// progress on the selector while a (possibly large) export runs.
let exportBusy = false;
function setExportBusy(busy) {
exportBusy = busy;
const sel = document.getElementById('export-select');
if (!sel) return;
sel.disabled = busy;
const ph = sel.querySelector('option[value=""]');
if (ph) ph.textContent = busy ? '\u23f3 Exporting\u2026' : '\u23ea Export';
}
async function exportAllCSV() { async function exportAllCSV() {
const btn = document.getElementById('btn-csv-all'); if (exportBusy) return;
if (btn.disabled) return;
const inTrigMode = trig.enabled && trig.snapshot !== null; const inTrigMode = trig.enabled && trig.snapshot !== null;
@@ -3482,8 +3631,8 @@ async function exportAllCSV() {
let t0, t1, relOffset = 0; let t0, t1, relOffset = 0;
if (inTrigMode) { if (inTrigMode) {
// Export the full trigger window around the trigger event. // Export the full trigger window around the trigger event.
t0 = trig.trigTime - trigPreSec(); t0 = trig.trigTime - activePreSec();
t1 = trig.trigTime + trigPostSec(); t1 = trig.trigTime + activePostSec();
relOffset = trig.trigTime; relOffset = trig.trigTime;
} else { } else {
// Use the current zoom range if active, else the rolling window. // Use the current zoom range if active, else the rolling window.
@@ -3504,60 +3653,52 @@ async function exportAllCSV() {
t1 = plotNow; t1 = plotNow;
} }
} }
if (!(t1 > t0)) return;
// Show loading state. exportBusy = true;
const origLabel = btn.textContent; // Cap the export. A full window at a megasample rate is hundreds of MB raw
btn.textContent = '⏳ Downloading…'; // (the old exact-timestamp merge exploded into millions of rows and crashed
btn.disabled = true; // the tab); ask the hub for a min/max-decimated envelope — the same scope
// style reduction the live view uses — and cap the number of rows.
const BUDGET = 100000; // max rows per signal
setExportBusy(true);
// Fetch full-resolution ring data (n=0 → no decimation).
let ringSignals = null; let ringSignals = null;
if (!inTrigMode) {
try { try {
ringSignals = await wsZoomRequest(t0, t1, 0, keys); ringSignals = await wsZoomRequest(t0, t1, BUDGET, keys);
} catch (e) { } catch (e) {
console.warn('CSV export: ring fetch failed, falling back to push buffer', e); console.warn('CSV export: ring fetch failed, falling back to local data', e);
} finally {
btn.textContent = origLabel;
btn.disabled = false;
} }
}
setExportBusy(false);
// Build per-signal time/value arrays. // Per-signal raw source: hub ring (whole window, decimated) → trigger
// Priority: ring buffer (full res) → trigger snapshot → push buffer. // snapshot (already \u226420k pts) → local push buffer.
const slices = keys.map(key => { const slices = keys.map(key => {
if (!inTrigMode) {
const rd = ringSignals && ringSignals[key]; const rd = ringSignals && ringSignals[key];
if (rd && rd.t && rd.t.length > 0) { if (rd && rd.t && rd.t.length > 0) return { key, t: rd.t, v: rd.v };
const t = rd.t, v = rd.v;
if (inTrigMode) {
return { t: Array.from(t).map(ts => ts - relOffset), v: Array.from(v) };
} }
return { t: Array.from(t), v: Array.from(v) };
}
// Fallback: push buffer or trigger snapshot.
if (inTrigMode) { if (inTrigMode) {
const raw = trig.snapshot[key] || { t: new Float64Array(0), v: new Float64Array(0) }; const raw = trig.snapshot[key] || { t: new Float64Array(0), v: new Float64Array(0) };
return { t: Array.from(raw.t).map(ts => ts - relOffset), v: Array.from(raw.v) }; return { key, t: raw.t, v: raw.v };
} }
const buf = buffers[key]; if (!buf) return { t: [], v: [] }; const buf = buffers[key];
const sl = getBufferSliceRange(buf, t0, t1); const sl = buf ? getBufferSliceRange(buf, t0, t1) : { t: new Float64Array(0), v: new Float64Array(0) };
return { t: Array.from(sl.t), v: Array.from(sl.v) }; return { key, t: sl.t, v: sl.v };
}); });
const present = slices.filter(s => s.t.length > 0);
if (!present.length) return;
// Merge all timestamps and build aligned rows. // Master time grid = the signal with the most samples; every other signal is
const allT = new Set(); // resampled onto it (linear, no extrapolation). Cells outside a signal's own
slices.forEach(s => s.t.forEach(t => allT.add(t))); // span stay empty rather than being fabricated, so continuous signals export
const sortedT = Array.from(allT).sort((a, b) => a - b); // without holes and no value is invented.
if (!sortedT.length) return; let master = present[0];
present.forEach(s => { if (s.t.length > master.t.length) master = s; });
const lookups = slices.map(s => { const cals = new Map(keys.map(k => [k, calForKey(k)]));
const m = new Map();
s.t.forEach((t, i) => m.set(t, s.v[i]));
return m;
});
// Strip "sourceId:" prefix from column headers for readability, and append
// the effective unit. These values come straight from the ring/history/
// snapshot and never pass through applyVScaleNorm, so calibrate them here.
const cals = keys.map(k => calForKey(k));
const displayKeys = keys.map(k => { const displayKeys = keys.map(k => {
const name = k.includes(':') ? k.split(':').slice(1).join(':') : k; const name = k.includes(':') ? k.split(':').slice(1).join(':') : k;
const u = unitForKey(k); const u = unitForKey(k);
@@ -3566,10 +3707,21 @@ async function exportAllCSV() {
}); });
const timeCol = '"' + (inTrigMode ? 'time_rel_s' : 'time_s') + '"'; const timeCol = '"' + (inTrigMode ? 'time_rel_s' : 'time_s') + '"';
const hdr = [timeCol, ...displayKeys].join(','); const hdr = [timeCol, ...displayKeys].join(',');
const rows = sortedT.map(t =>
[t.toFixed(9), ...lookups.map((lk, i) => const rows = new Array(master.t.length);
lk.has(t) ? Calib.applyCal(lk.get(t), cals[i]) : '')].join(',') for (let i = 0; i < master.t.length; i++) {
); const tAbs = master.t[i];
const cells = present.map(s => {
if (s === master) {
return Calib.applyCal(master.v[i], cals.get(s.key));
}
const v = interpSortedRaw(s.t, s.v, tAbs);
return v === null ? '' : Calib.applyCal(v, cals.get(s.key));
});
const tt = inTrigMode ? tAbs - relOffset : tAbs;
rows[i] = [tt.toFixed(9), ...cells].join(',');
}
const blob = new Blob([hdr + '\n' + rows.join('\n')], { type: 'text/csv' }); const blob = new Blob([hdr + '\n' + rows.join('\n')], { type: 'text/csv' });
const a = document.createElement('a'); const a = document.createElement('a');
a.href = URL.createObjectURL(blob); a.href = URL.createObjectURL(blob);
@@ -3754,6 +3906,10 @@ function deletePlot(plotId) {
let _dbgTick = 0; let _dbgTick = 0;
let _dataGen = 0; // incremented each time new data arrives let _dataGen = 0; // incremented each time new data arrives
function renderDirtyPlots() { function renderDirtyPlots() {
// Schedule the next frame FIRST: an exception below must never kill the
// animation loop, or every plot would freeze until a page refresh.
requestAnimationFrame(renderDirtyPlots);
try {
// Compute global "now" once — shared by all rolling-window plots this frame. // Compute global "now" once — shared by all rolling-window plots this frame.
const globalPlotNow = getGlobalNow(); const globalPlotNow = getGlobalNow();
@@ -3823,7 +3979,7 @@ function renderDirtyPlots() {
plots.forEach(p => { plots.forEach(p => {
if (!p.needsRedraw || !p.uplot || p.traces.length === 0) return; if (!p.needsRedraw || !p.uplot || p.traces.length === 0) return;
try {
const inTrigModeNow = inTrigWindow(); const inTrigModeNow = inTrigWindow();
// The x tick formatter and the cursor-sync group are baked into the uPlot // The x tick formatter and the cursor-sync group are baked into the uPlot
// options at construction. A plot built in live mode therefore keeps // options at construction. A plot built in live mode therefore keeps
@@ -3838,7 +3994,10 @@ function renderDirtyPlots() {
if (isRolling && _dataGen === p.lastDataGen && p.uplot.data && p.uplot.data[0] && p.uplot.data[0].length > 0) { if (isRolling && _dataGen === p.lastDataGen && p.uplot.data && p.uplot.data[0] && p.uplot.data[0].length > 0) {
p.needsRedraw = false; p.needsRedraw = false;
zoomGuard = true; zoomGuard = true;
p.uplot.setScale('x', { min: globalPlotNow - windowSec, max: globalPlotNow }); // Use the same per-plot anchor as the rebuild path, so the rolling window
// does not jump when the frame switches between the two.
const plotNow = computePlotNow(p);
p.uplot.setScale('x', { min: plotNow - windowSec, max: plotNow });
zoomGuard = false; zoomGuard = false;
return; return;
} }
@@ -3875,12 +4034,25 @@ function renderDirtyPlots() {
p.uplot.setScale('x', { min: plotNow - windowSec, max: plotNow }); p.uplot.setScale('x', { min: plotNow - windowSec, max: plotNow });
} }
zoomGuard = false; zoomGuard = false;
p._errCount = 0;
} catch (e) {
// One bad plot must not kill the whole render loop. Track consecutive
// failures and self-heal by rebuilding the uPlot instance.
p._errCount = (p._errCount || 0) + 1;
console.error(`[render] plot ${p.id}:`, e);
p.needsRedraw = true; // retry next frame
if (p._errCount >= 30) {
p._errCount = 0;
try { createUPlot(p); } catch (e2) { console.error(`[render] rebuild plot ${p.id}:`, e2); }
}
}
}); });
// Keep per-plot cursor value readouts in sync with live data. // Keep per-plot cursor value readouts in sync with live data.
if (cursors.mode === 'on') updatePlotCursorReadouts(); if (cursors.mode === 'on') updatePlotCursorReadouts();
} catch (e) {
requestAnimationFrame(renderDirtyPlots); console.error('[render]', e);
}
} }
@@ -3958,6 +4130,7 @@ function onSources(msg) {
}); });
buildSidebar(); buildSidebar();
if (statsOpen) _refreshStatsSelector(); if (statsOpen) _refreshStatsSelector();
maybeRestoreViewLate();
} }
function addSourceWS(label, addr, multicastGroup, dataPort) { function addSourceWS(label, addr, multicastGroup, dataPort) {
@@ -4582,7 +4755,303 @@ initSignalMenu();
const cb = document.getElementById('cb-monotonic'); const cb = document.getElementById('cb-monotonic');
if (cb) cb.checked = localStorage.getItem('udpscope.monotonic') === '1'; if (cb) cb.checked = localStorage.getItem('udpscope.monotonic') === '1';
} }
document.getElementById('btn-csv-all').addEventListener('click', exportAllCSV); // Export every stored sample of the plotted signals as a Parquet file, served
// by the Go hub's /api/export. Full resolution (no decimation) and hole-free
// (each signal keeps its own timestamps — long format). The file can be huge
// (hundreds of MB at high rates), so stream it to disk when the File System
// Access API is available instead of holding it in a Blob.
async function exportParquet() {
if (exportBusy) return;
const inTrigMode = trig.enabled && trig.snapshot !== null;
const keys = [];
plots.forEach(p => p.traces.forEach(k => { if (!keys.includes(k)) keys.push(k); }));
if (!keys.length) return;
// Same range resolution as the CSV export.
let t0, t1;
if (inTrigMode) {
t0 = trig.trigTime - activePreSec();
t1 = trig.trigTime + activePostSec();
} else {
const refPlot = plots.find(p => p.xRange);
if (refPlot) {
[t0, t1] = refPlot.xRange;
} else {
let plotNow = -Infinity;
keys.forEach(k => {
const buf = buffers[k];
if (buf && buf.size > 0) {
const t = buf.t[(buf.head - 1 + buf.cap) % buf.cap];
if (t > plotNow) plotNow = t;
}
});
if (!isFinite(plotNow)) plotNow = Date.now() / 1000;
t0 = plotNow - windowSec;
t1 = plotNow;
}
}
if (!(t1 > t0)) return;
exportBusy = true;
setExportBusy(true);
try {
const url = '/api/export?t0=' + t0.toFixed(9) + '&t1=' + t1.toFixed(9) +
'&signals=' + encodeURIComponent(keys.join(','));
const resp = await fetch(url);
if (!resp.ok) {
alert('Parquet export failed (HTTP ' + resp.status + ').\n\n' +
'The /api/export endpoint is provided by the Go hub; the C++ ' +
'StreamHub does not serve it.');
return;
}
const filename = 'signals_' + Date.now() + '.parquet';
if (window.showSaveFilePicker && resp.body) {
try {
const handle = await window.showSaveFilePicker({
suggestedName: filename,
types: [{ description: 'Parquet', accept: { 'application/vnd.apache.parquet': ['.parquet'] } }],
});
const writable = await handle.createWritable();
await resp.body.pipeTo(writable);
return;
} catch (e) {
if (e && e.name === 'AbortError') return; // user cancelled the picker
console.warn('parquet export: file picker failed, falling back to Blob', e);
}
}
const blob = await resp.blob();
const a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = filename;
a.click();
URL.revokeObjectURL(a.href);
} catch (e) {
console.warn('parquet export failed', e);
alert('Parquet export failed: ' + e.message);
} finally {
setExportBusy(false);
}
}
// Export dropdown: dispatch on selection, then reset to the placeholder so the
// same format can be chosen again.
document.getElementById('export-select').addEventListener('change', () => {
const sel = document.getElementById('export-select');
const fmt = sel.value;
sel.value = '';
if (fmt === 'csv') exportAllCSV();
else if (fmt === 'parquet') exportParquet();
});
/* ════════════════════════════════════════════════════════════════
View-state persistence (cookie)
════════════════════════════════════════════════════════════════ */
// The whole client view — layout, plots (traces/titles/modes), window, trigger
// configuration, rulers, sources — is serialised into one cookie so a reload
// restores the previous view. Cookies are size-limited, so the state degrades
// gracefully (rulers → trigger → sources → traces) when it would not fit.
const VIEW_COOKIE = 'udpscope.view';
const VIEW_COOKIE_MAX = 3500; // encoded chars; browsers cap cookies at ~4 KiB
function packViewState() {
const state = {
v: 1,
windowSec: windowSec,
layout: currentLayout,
plots: plots.map(p => ({
title: p.title,
mode: p.mode,
traces: p.traces.map(k => {
const colon = k.indexOf(':');
const name = colon >= 0 ? k.slice(colon + 1) : k;
return { key: k, label: srcLabelForKey(k), name };
}),
})),
trig: {
enabled: trig.enabled, signal: trig.signal, edge: trig.edge,
threshold: trig.threshold, windowSec: trig.windowSec,
prePercent: trig.prePercent, mode: trig.mode, holdoffSec: trig.holdoffSec,
},
rulers: {
mode: rulers.mode,
plotId: plots.findIndex(p => p.id === rulers.plotId),
states: plots.map(p => {
const rs = rulerState[p.id];
return rs ? { yA: rs.yA, yB: rs.yB } : { yA: null, yB: null };
}),
},
sources: Object.values(sourcesMap).map(s => ({
label: s.label || s.addr || s.id, addr: s.addr,
})),
};
let s = JSON.stringify(state);
const tooBig = () => encodeURIComponent(s).length > VIEW_COOKIE_MAX;
if (tooBig()) { delete state.rulers; s = JSON.stringify(state); }
if (tooBig()) { delete state.trig; s = JSON.stringify(state); }
if (tooBig()) { delete state.sources; s = JSON.stringify(state); }
if (tooBig()) {
state.plots = state.plots.map(p => ({ title: p.title, mode: p.mode }));
s = JSON.stringify(state);
}
if (tooBig()) { state.plots = []; s = JSON.stringify(state); }
return s;
}
// Saves are gated until the saved view has been re-applied (phase 2) or the
// grace timeout fires: otherwise the very first periodic save would overwrite
// the cookie with the not-yet-restored (empty) state and destroy it.
let _viewSaveReady = false;
function saveViewState() {
if (!_viewSaveReady) return;
try {
const s = packViewState();
document.cookie = VIEW_COOKIE + '=' + encodeURIComponent(s) +
'; path=/; max-age=31536000; SameSite=Lax';
} catch (e) {
console.warn('view cookie save failed', e);
}
}
function readViewState() {
try {
const prefix = VIEW_COOKIE + '=';
const m = document.cookie.split('; ').find(c => c.startsWith(prefix));
if (!m) return null;
const st = JSON.parse(decodeURIComponent(m.slice(prefix.length)));
return (st && st.v === 1) ? st : null;
} catch (e) {
return null;
}
}
// Phase 1 (init): layout, plot cards (titles/modes), window, rulers. Traces and
// the trigger need sources/signals loaded, so they are applied in phase 2.
function restoreViewState() {
const st = readViewState();
if (!st) return;
if (st.layout && LAYOUTS.some(l => l[1] === st.layout)) applyLayout(st.layout);
const plotState = st.plots || [];
plotState.forEach((ps, i) => {
const p = plots[i];
if (!p) return;
if (ps.title && ps.title !== 'Plot ' + p.id) {
p.title = ps.title;
const tEl = document.getElementById('ptitle-' + p.id);
if (tEl) tEl.textContent = ps.title;
const inp = document.querySelector('#pcfg-' + p.id + ' .pcfg-title-input');
if (inp) inp.value = ps.title;
}
if (ps.mode && ps.mode !== p.mode) {
p.mode = ps.mode;
document.querySelectorAll('#pcfg-' + p.id + ' .pcfg-mode-btn')
.forEach(b => b.classList.toggle('active', b.dataset.mode === ps.mode));
}
});
if (st.windowSec != null) {
windowSec = st.windowSec;
const sel = document.getElementById('window-select');
if (sel && [...sel.options].some(o => o.value === String(st.windowSec))) {
sel.value = String(st.windowSec);
}
}
if (st.rulers) {
rulers.mode = st.rulers.mode === 'on' ? 'on' : 'off';
const btn = document.getElementById('btn-ruler');
if (btn) btn.classList.toggle('active', rulers.mode === 'on');
(st.rulers.states || []).forEach((rs, i) => {
const p = plots[i];
if (!p || !rs) return;
const cur = getRulerState(p.id);
cur.yA = rs.yA; cur.yB = rs.yB;
});
if (st.rulers.plotId != null && plots[st.rulers.plotId]) {
rulers.plotId = plots[st.rulers.plotId].id;
}
}
}
// Rebuild a saved trace key against the current sources: source ids change
// across restarts, so match by label and fall back to the saved id-key.
function restoreTraceKey(entry) {
const src = Object.values(sourcesMap).find(s => (s.label || s.id) === entry.label);
return src ? (src.id + ':' + entry.name) : entry.key;
}
// Phase 2 (first sources + signals): reconcile sources, re-apply traces, then
// the trigger configuration and the window to the hub.
let _viewLateRestored = false;
function maybeRestoreViewLate() {
if (_viewLateRestored) return;
const st = readViewState();
if (!st) { _viewLateRestored = true; return; }
// Add any saved sources the hub does not already have (it persists its own).
const known = Object.values(sourcesMap).map(s => (s.label || s.addr || s.id) + '\u0000' + s.addr);
let added = false;
(st.sources || []).forEach(sv => {
const k = (sv.label || sv.addr) + '\u0000' + (sv.addr || '');
if (!known.includes(k)) { addSourceWS(sv.label, sv.addr, sv.multicastGroup, sv.dataPort); added = true; }
});
if (added) return; // re-enter when the new sources appear
// Traces and the trigger selector need at least one source with signals.
if (!Object.values(sourcesMap).some(s => (s.signals || []).length > 0)) return;
_viewLateRestored = true;
_viewSaveReady = true;
(st.plots || []).forEach((ps, i) => {
const p = plots[i];
if (!p) return;
(ps.traces || []).forEach(t => addTraceTo(p.id, restoreTraceKey(t)));
});
if (st.windowSec != null) {
windowSec = st.windowSec;
sendWindow();
}
const t = st.trig;
if (t) {
trig.edge = t.edge || trig.edge;
if (t.threshold != null) trig.threshold = t.threshold;
if (t.windowSec != null) trig.windowSec = t.windowSec;
if (t.prePercent != null) trig.prePercent = t.prePercent;
trig.mode = t.mode || trig.mode;
if (t.holdoffSec != null) trig.holdoffSec = t.holdoffSec;
trig.signal = t.signal || '';
const el = id => document.getElementById(id);
if (el('trig-edge')) el('trig-edge').value = trig.edge;
if (el('trig-window')) el('trig-window').value = String(trig.windowSec);
if (el('trig-mode')) el('trig-mode').value = trig.mode;
if (el('trig-holdoff')) el('trig-holdoff').value = trig.holdoffSec;
if (el('trig-pre')) el('trig-pre').value = String(trig.prePercent);
if (el('trig-pre-val')) el('trig-pre-val').textContent = trig.prePercent + '%';
refreshTrigThresholdField();
const selSig = document.getElementById('trig-signal');
if (selSig && trig.signal) {
const base = trig.signal.replace(/\[\d+\]$/, '');
if ([...selSig.options].some(o => o.value === base)) selSig.value = base;
}
if (t.enabled) openTrigBar(true);
else updateTrigStatusBadge('idle');
}
}
// Periodic save keeps the cookie current without wiring every control; the
// pagehide save captures the final state on close/reload.
setInterval(saveViewState, 3000);
window.addEventListener('pagehide', saveViewState);
// Hub down / never connected: stop gating after 10 s so layout, window and
// rulers still persist even though traces and the trigger could not be
// restored.
setTimeout(() => { _viewSaveReady = true; }, 10000);
document.getElementById('history-badge').addEventListener('click', toggleHistoryPanel); document.getElementById('history-badge').addEventListener('click', toggleHistoryPanel);
document.getElementById('btn-hist-cancel').addEventListener('click', toggleHistoryPanel); document.getElementById('btn-hist-cancel').addEventListener('click', toggleHistoryPanel);
document.getElementById('btn-hist-apply').addEventListener('click', applyHistoryBudget); document.getElementById('btn-hist-apply').addEventListener('click', applyHistoryBudget);
@@ -4592,6 +5061,7 @@ document.getElementById('stats-source-sel').addEventListener('change', e => {
statsSelectedSrc = e.target.value || null; statsSelectedSrc = e.target.value || null;
renderStats(); renderStats();
}); });
restoreViewState();
resolveHub().then(connectWS); resolveHub().then(connectWS);
requestAnimationFrame(renderDirtyPlots); requestAnimationFrame(renderDirtyPlots);
fetch('/version').then(r => r.text()).then(v => { fetch('/version').then(r => r.text()).then(v => {
Client/udpstreamer/static/index.html
+5 -1
View File
@@ -41,7 +41,11 @@
<button id="btn-ruler" class="ctrl-btn" title="Horizontal value rulers">Rulers</button> <button id="btn-ruler" class="ctrl-btn" title="Horizontal value rulers">Rulers</button>
<button id="btn-zoom-back" class="ctrl-btn" style="display:none">← Back</button> <button id="btn-zoom-back" class="ctrl-btn" style="display:none">← Back</button>
<button id="btn-zoom-fit" class="ctrl-btn">Fit</button> <button id="btn-zoom-fit" class="ctrl-btn">Fit</button>
<button id="btn-csv-all" class="ctrl-btn" title="Export all signals to CSV">⬇ CSV</button> <select id="export-select" class="ctrl-select" title="Export the visible signals">
<option value="" disabled selected>⬇ Export</option>
<option value="csv" title="Export the visible signals as CSV (decimated to a bounded row count)">CSV</option>
<option value="parquet" title="Export every stored sample (full resolution, no holes) as Parquet — requires the Go hub">Parquet</option>
</select>
<button id="btn-sync-resume" class="ctrl-btn resume-btn" style="display:none">↺ Auto</button> <button id="btn-sync-resume" class="ctrl-btn resume-btn" style="display:none">↺ Auto</button>
<button id="btn-trigger" class="ctrl-btn">⚡ Trigger</button> <button id="btn-trigger" class="ctrl-btn">⚡ Trigger</button>
<button id="btn-pause-global" class="ctrl-btn">⏸ Pause</button> <button id="btn-pause-global" class="ctrl-btn">⏸ Pause</button>
Client/udpstreamer/static/style.css
+8
View File
@@ -14,6 +14,11 @@
*, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; } *, *::before, *::after { box-sizing: border-box; margin: 0; padding: 0; }
html, body { height:100%; background:var(--bg); color:var(--text); html, body { height:100%; background:var(--bg); color:var(--text);
font-family:'Segoe UI',system-ui,sans-serif; font-size:14px; overflow:hidden; } font-family:'Segoe UI',system-ui,sans-serif; font-size:14px; overflow:hidden; }
/* Uniform 0.9x compaction — scales every element (fonts, bars, plots,
spacing) while reflowing layout. `zoom` (Chrome/Edge/Safari, Firefox 126+)
is preferred over `transform: scale` because it reflows, so fixed-position
bars and JS-computed offsets stay aligned. */
html { zoom: 0.9; }
::-webkit-scrollbar { width:6px; } ::-webkit-scrollbar { width:6px; }
::-webkit-scrollbar-track { background:var(--mantle); } ::-webkit-scrollbar-track { background:var(--mantle); }
::-webkit-scrollbar-thumb { background:var(--surface1); border-radius:3px; } ::-webkit-scrollbar-thumb { background:var(--surface1); border-radius:3px; }
@@ -256,6 +261,9 @@ input[type=range].trig-range::-webkit-slider-thumb {
#plot-grid.l2x3 { grid-template-columns:1fr 1fr; grid-template-rows:1fr 1fr 1fr; } #plot-grid.l2x3 { grid-template-columns:1fr 1fr; grid-template-rows:1fr 1fr 1fr; }
#plot-grid.l1x4 { grid-template-columns:1fr; grid-template-rows:1fr 1fr 1fr 1fr; } #plot-grid.l1x4 { grid-template-columns:1fr; grid-template-rows:1fr 1fr 1fr 1fr; }
#plot-grid.l4x1 { grid-template-columns:1fr 1fr 1fr 1fr; grid-template-rows:1fr; } #plot-grid.l4x1 { grid-template-columns:1fr 1fr 1fr 1fr; grid-template-rows:1fr; }
/* 1+2 layout: one plot spanning the top row, two side by side below. */
#plot-grid.l1p2 { grid-template-columns:1fr 1fr; grid-template-rows:1fr 1fr; }
#plot-grid.l1p2 .plot-card:first-child { grid-column: 1 / -1; }
/* ── Plot card ────────────────────────────────────────────────── */ /* ── Plot card ────────────────────────────────────────────────── */
.plot-card { .plot-card {
Common/Client/c/udps_client.c
+26 -5
View File
@@ -482,11 +482,36 @@ static int decode_data(udps_client_t *c, const uint8_t *pl, size_t len,
size_t total = 0u; size_t total = 0u;
size_t written = 0u; size_t written = 0u;
uint32_t i; uint32_t i;
uint32_t lost = 0u;
udps_frame_t frame; udps_frame_t frame;
if (c->num_sigs == 0u) { if (c->num_sigs == 0u) {
return 0; /* DATA before CONFIG: nothing to decode against. */ return 0; /* DATA before CONFIG: nothing to decode against. */
} }
/* Order the sequence before spending anything on the payload.
*
* Reassembly completes in arrival order, not counter order, so a packet
* delayed or duplicated on the wire surfaces after a newer one has already
* been delivered. Its samples carry an older time base: they land on top
* of data the consumer already has and leave the span they should have
* filled empty. Nothing in the payload distinguishes such a packet from a
* good one, only the counter does.
*
* The counter is a wrapping uint32, so it is ordered by the signed
* difference; comparing the values directly would call the first packet
* after the wrap stale and reject the stream from then on. */
if (c->have_counter) {
int32_t delta = (int32_t)(counter - c->last_counter);
if (delta <= 0) {
c->stats.stale_packets++;
return 0;
}
lost = (uint32_t)delta - 1u;
c->stats.counter_gaps += lost;
}
c->last_counter = counter;
c->have_counter = 1;
if (len < 8u) { if (len < 8u) {
return fail(c, "DATA payload too short (%lu bytes)", (unsigned long)len); return fail(c, "DATA payload too short (%lu bytes)", (unsigned long)len);
} }
@@ -528,15 +553,11 @@ static int decode_data(udps_client_t *c, const uint8_t *pl, size_t len,
written += count; written += count;
} }
if (c->have_counter && counter > c->last_counter + 1u) {
c->stats.counter_gaps += counter - c->last_counter - 1u;
}
c->last_counter = counter;
c->have_counter = 1;
c->stats.frames_delivered++; c->stats.frames_delivered++;
if (c->on_data != NULL) { if (c->on_data != NULL) {
frame.counter = counter; frame.counter = counter;
frame.lost = lost;
frame.hrt = rd_u64(pl); frame.hrt = rd_u64(pl);
frame.recv_time = recv_time; frame.recv_time = recv_time;
frame.publish_mode = c->publish_mode; frame.publish_mode = c->publish_mode;
Common/Client/c/udps_client.h
+16
View File
@@ -140,6 +140,16 @@ typedef struct {
/** One fully decoded DATA packet. */ /** One fully decoded DATA packet. */
typedef struct { typedef struct {
uint32_t counter; /**< Packet counter; gaps mean lost datagrams. */ uint32_t counter; /**< Packet counter; gaps mean lost datagrams. */
/**
* DATA packets missing immediately before this one, from the counter.
*
* Needed to space samples correctly: the elapsed time since the previous
* frame covers the lost packets' cycles too, so dividing it by this
* frame's sample count alone gives a period too long by exactly
* @c lost + 1, which walks the samples past their own end and into the
* range the next frame claims.
*/
uint32_t lost;
uint64_t hrt; /**< Producer's high-resolution timer at send. */ uint64_t hrt; /**< Producer's high-resolution timer at send. */
double recv_time; /**< Wall-clock seconds (CLOCK_REALTIME) at arrival. */ double recv_time; /**< Wall-clock seconds (CLOCK_REALTIME) at arrival. */
uint8_t publish_mode; /**< UDPS_PUBLISH_*. */ uint8_t publish_mode; /**< UDPS_PUBLISH_*. */
@@ -164,6 +174,12 @@ typedef struct {
uint64_t config_updates; uint64_t config_updates;
uint64_t fragments_dropped; /**< Duplicate, stale or unplaceable fragments. */ uint64_t fragments_dropped; /**< Duplicate, stale or unplaceable fragments. */
uint64_t counter_gaps; /**< DATA packets missing from the sequence. */ uint64_t counter_gaps; /**< DATA packets missing from the sequence. */
/**
* DATA packets dropped for not advancing the counter: reordered or
* duplicated on the wire. Delivering one would stamp its values with a
* time base older than data already handed over.
*/
uint64_t stale_packets;
uint64_t reconnects; uint64_t reconnects;
} udps_stats_t; } udps_stats_t;
Common/Client/go/go.mod
+14 -2
View File
@@ -1,7 +1,19 @@
module marte2/common module marte2/common
go 1.21 go 1.24.9
require github.com/gorilla/websocket v1.5.1 require github.com/gorilla/websocket v1.5.1
require golang.org/x/net v0.17.0 // indirect require (
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/klauspost/compress v1.17.9 // indirect
github.com/parquet-go/bitpack v1.0.0 // indirect
github.com/parquet-go/jsonlite v1.0.0 // indirect
github.com/parquet-go/parquet-go v0.32.0 // indirect
github.com/pierrec/lz4/v4 v4.1.21 // indirect
github.com/twpayne/go-geom v1.6.1 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/sys v0.38.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
)
Common/Client/go/go.sum
+21
View File
@@ -1,4 +1,25 @@
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY= github.com/gorilla/websocket v1.5.1 h1:gmztn0JnHVt9JZquRuzLw3g4wouNVzKL15iLr/zn/QY=
github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY= github.com/gorilla/websocket v1.5.1/go.mod h1:x3kM2JMyaluk02fnUJpQuwD2dCS5NDG2ZHL0uE0tcaY=
github.com/klauspost/compress v1.17.9 h1:6KIumPrER1LHsvBVuDa0r5xaG0Es51mhhB9BQB2qeMA=
github.com/klauspost/compress v1.17.9/go.mod h1:Di0epgTjJY877eYKx5yC51cX2A2Vl2ibi7bDH9ttBbw=
github.com/parquet-go/bitpack v1.0.0 h1:AUqzlKzPPXf2bCdjfj4sTeacrUwsT7NlcYDMUQxPcQA=
github.com/parquet-go/bitpack v1.0.0/go.mod h1:XnVk9TH+O40eOOmvpAVZ7K2ocQFrQwysLMnc6M/8lgs=
github.com/parquet-go/jsonlite v1.0.0 h1:87QNdi56wOfsE5bdgas0vRzHPxfJgzrXGml1zZdd7VU=
github.com/parquet-go/jsonlite v1.0.0/go.mod h1:nDjpkpL4EOtqs6NQugUsi0Rleq9sW/OtC1NnZEnxzF0=
github.com/parquet-go/parquet-go v0.32.0 h1:NWDqTUHfrCS4cJP/Fj2HlxvqsrVedWG3sayMkf+znzM=
github.com/parquet-go/parquet-go v0.32.0/go.mod h1:navtkAYr2LGoJVp141oXPlO/sxLvaOe3la2JEoD8+rg=
github.com/pierrec/lz4/v4 v4.1.21 h1:yOVMLb6qSIDP67pl/5F7RepeKYu/VmTyEXvuMI5d9mQ=
github.com/pierrec/lz4/v4 v4.1.21/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/twpayne/go-geom v1.6.1 h1:iLE+Opv0Ihm/ABIcvQFGIiFBXd76oBIar9drAwHFhR4=
github.com/twpayne/go-geom v1.6.1/go.mod h1:Kr+Nly6BswFsKM5sd31YaoWS5PeDDH2NftJTK7Gd028=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM= golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE= golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/sys v0.38.0 h1:3yZWxaJjBmCWXqhN1qh02AkOnCQ1poK6oF+a7xWL6Gc=
golang.org/x/sys v0.38.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
Common/Client/go/udpsprotocol/accumulate_array_test.go
+146
View File
@@ -0,0 +1,146 @@
package udpsprotocol
// Accumulate mode ships one full snapshot of EVERY signal per RT cycle —
// arrays included. See UDPStreamer.cpp pass 5 ("ALL signals (scalars and
// arrays alike) are tagged accumulated = true") and SerializeAccumulated,
// which writes, for each signal in CONFIG order, numSamples consecutive
// snapshots of that signal's full element set.
//
// The tests below build a payload byte-for-byte the way the C++ producer
// does, so a decoding regression shows up here rather than as a mangled
// waveform three components downstream.
import (
"encoding/binary"
"math"
"testing"
"time"
)
// buildAccumulatePayload lays out an Accumulate DATA payload exactly as
// UDPStreamer::SerializeAccumulated does:
//
// [8 HRT][4 numSamples] then, per signal, numSamples × NumElements float64.
//
// slots[i][k] holds signal i's element set for cycle k.
func buildAccumulatePayload(hrt uint64, slots [][][]float64) []byte {
numSamples := 0
if len(slots) > 0 {
numSamples = len(slots[0])
}
out := make([]byte, 12)
binary.LittleEndian.PutUint64(out[0:8], hrt)
binary.LittleEndian.PutUint32(out[8:12], uint32(numSamples))
for _, sig := range slots {
for _, elems := range sig {
for _, v := range elems {
var b [8]byte
binary.LittleEndian.PutUint64(b[:], math.Float64bits(v))
out = append(out, b[:]...)
}
}
}
return out
}
// TestParseDataAccumulateGivesEachSlotItsOwnArray pins the array case: with an
// accumulated batch, slot k's array signal must decode to the values the
// producer captured on cycle k, not to some other cycle's. Handing every slot
// slot 0's array would stamp one cycle's data with every slot's timestamp —
// the same samples drawn repeatedly at advancing times, with the cycles they
// displaced missing entirely.
func TestParseDataAccumulateGivesEachSlotItsOwnArray(t *testing.T) {
sigs := []SignalInfo{
{Name: "Time", TypeCode: 9, NumRows: 1, NumCols: 1, QuantType: QuantNone},
{Name: "Wave", TypeCode: 9, NumRows: 4, NumCols: 1, QuantType: QuantNone},
}
// Three RT cycles. "Wave" carries a different ramp each cycle so a
// mix-up is unambiguous.
timeSlots := [][]float64{{10}, {20}, {30}}
waveSlots := [][]float64{
{1, 2, 3, 4},
{5, 6, 7, 8},
{9, 10, 11, 12},
}
payload := buildAccumulatePayload(777, [][][]float64{timeSlots, waveSlots})
samples, err := ParseData(payload, sigs, PublishModeAccumulate, time.Now())
if err != nil {
t.Fatalf("ParseData: %v", err)
}
if len(samples) != 3 {
t.Fatalf("expected 3 slots, got %d", len(samples))
}
for k, s := range samples {
got := s.Values["Wave"]
want := waveSlots[k]
if len(got) != len(want) {
t.Fatalf("slot %d: Wave has %d elements, want %d", k, len(got), len(want))
}
for i := range want {
if got[i] != want[i] {
t.Fatalf("slot %d: Wave = %v, want %v (slot %d's data has been "+
"served for this slot's timestamp)", k, got, want,
indexOfSlot(waveSlots, got))
}
}
if tv := s.Values["Time"]; len(tv) != 1 || tv[0] != timeSlots[k][0] {
t.Fatalf("slot %d: Time = %v, want %v", k, tv, timeSlots[k])
}
}
}
// TestParseDataAccumulateConsumesTheWholeArrayBlock catches the same defect
// from the other side: a signal following an array must be read at the right
// offset. Under-reading the array block slides every later signal backwards
// into the array's tail, which decodes as plausible-looking but wrong values
// rather than as an error.
func TestParseDataAccumulateConsumesTheWholeArrayBlock(t *testing.T) {
sigs := []SignalInfo{
{Name: "Wave", TypeCode: 9, NumRows: 4, NumCols: 1, QuantType: QuantNone},
{Name: "Tail", TypeCode: 9, NumRows: 1, NumCols: 1, QuantType: QuantNone},
}
waveSlots := [][]float64{
{1, 2, 3, 4},
{5, 6, 7, 8},
{9, 10, 11, 12},
}
tailSlots := [][]float64{{100}, {200}, {300}}
payload := buildAccumulatePayload(0, [][][]float64{waveSlots, tailSlots})
samples, err := ParseData(payload, sigs, PublishModeAccumulate, time.Now())
if err != nil {
t.Fatalf("ParseData: %v", err)
}
if len(samples) != 3 {
t.Fatalf("expected 3 slots, got %d", len(samples))
}
for k, s := range samples {
tv := s.Values["Tail"]
if len(tv) != 1 || tv[0] != tailSlots[k][0] {
t.Fatalf("slot %d: Tail = %v, want %v — the array block before it "+
"was not fully consumed", k, tv, tailSlots[k])
}
}
}
// indexOfSlot reports which slot's data a decoded array actually matches, so a
// failure message can name the culprit instead of just showing numbers.
func indexOfSlot(slots [][]float64, got []float64) int {
for k, want := range slots {
if len(want) != len(got) {
continue
}
same := true
for i := range want {
if want[i] != got[i] {
same = false
break
}
}
if same {
return k
}
}
return -1
}
Common/Client/go/udpsprotocol/protocol.go
+33 -28
View File
@@ -290,28 +290,37 @@ type DataSample struct {
HRTTimestamp uint64 HRTTimestamp uint64
WallTime time.Time // wall-clock time at UDP arrival; used as x-axis WallTime time.Time // wall-clock time at UDP arrival; used as x-axis
Values map[string][]float64 // key = signal name, value = []float64 with NumElements entries Values map[string][]float64 // key = signal name, value = []float64 with NumElements entries
// Lost is the number of DATA packets missing between the previous sample
// and this one, taken from the producer's packet counter (see
// SequenceGate). Consumers that derive a per-element period from the
// inter-packet gap need it: the gap widens with every lost packet, and
// dividing it by this packet's element count alone reports a period too
// long by exactly that factor — which walks the packet's elements past
// their own end and into the range the next packet claims.
Lost uint32
} }
// parseElems reads n elements for sig from payload at offset, advancing offset. // parseElems reads n elements for sig from payload at offset, advancing offset.
// Returns the slice of float64 values and the new offset. // Returns the slice of float64 values and the new offset.
func parseElems(payload []byte, offset, n int, sig SignalInfo) ([]float64, int, error) { func parseElems(payload []byte, offset, n int, sig SignalInfo) ([]float64, int, error) {
elems := make([]float64, n)
if sig.QuantType == QuantNone {
sz := rawTypeSize(sig.TypeCode) sz := rawTypeSize(sig.TypeCode)
needed := n * sz if sig.QuantType != QuantNone {
if offset+needed > len(payload) { sz = quantSize(sig.QuantType)
}
// Bounds-check before allocating. In Accumulate mode n is numSamples ×
// NumElements, so a malformed packet could otherwise ask for an allocation
// far larger than its own payload could ever justify.
if n < 0 || n > (len(payload)-offset)/sz {
return nil, offset, fmt.Errorf("data payload truncated for signal %q", sig.Name) return nil, offset, fmt.Errorf("data payload truncated for signal %q", sig.Name)
} }
elems := make([]float64, n)
needed := n * sz
if sig.QuantType == QuantNone {
for i := 0; i < n; i++ { for i := 0; i < n; i++ {
elems[i] = readRawElement(payload, offset+i*sz, sig.TypeCode) elems[i] = readRawElement(payload, offset+i*sz, sig.TypeCode)
} }
offset += needed offset += needed
} else { } else {
sz := quantSize(sig.QuantType)
needed := n * sz
if offset+needed > len(payload) {
return nil, offset, fmt.Errorf("data payload truncated (quant) for signal %q", sig.Name)
}
for i := 0; i < n; i++ { for i := 0; i < n; i++ {
var raw uint16 var raw uint16
if sz == 1 { if sz == 1 {
@@ -331,7 +340,13 @@ func parseElems(payload []byte, offset, n int, sig SignalInfo) ([]float64, int,
// //
// For PublishModeAccumulate the payload format is: // For PublishModeAccumulate the payload format is:
// //
// [8 HRT][4 numSamples][for each signal: accumulated scalars → numSamples elems; arrays → NumElements elems] // [8 HRT][4 numSamples][for each signal: numSamples × NumElements elems]
//
// Every signal is accumulated, arrays included: the producer captures one full
// snapshot of the whole signal set per RT cycle and lays the cycles out
// contiguously per signal (UDPStreamer::SerializeAccumulated). Reading only
// NumElements for an array would hand every slot the first cycle's data and
// slide all later signals into that array's tail.
// //
// The function returns one DataSample per accumulated snapshot so the hub can // The function returns one DataSample per accumulated snapshot so the hub can
// process each slot independently with its own timestamp. // process each slot independently with its own timestamp.
@@ -357,28 +372,18 @@ func ParseData(payload []byte, sigs []SignalInfo, publishMode uint8, arrivalTime
} }
// Parse per-signal data blocks (all slots for a signal are contiguous). // Parse per-signal data blocks (all slots for a signal are contiguous).
accumVals := make(map[string][]float64, len(sigs)) // scalars: numSamples values accumVals := make(map[string][]float64, len(sigs)) // numSamples × NumElements
fixedVals := make(map[string][]float64, len(sigs)) // arrays: NumElements values accumElems := make(map[string]int, len(sigs))
for _, sig := range sigs { for _, sig := range sigs {
n := sig.NumElements() n := sig.NumElements()
if n == 1 { elems, newOff, err := parseElems(payload, offset, numSamples*n, sig)
// Accumulated scalar: read numSamples back-to-back elements.
elems, newOff, err := parseElems(payload, offset, numSamples, sig)
if err != nil { if err != nil {
return nil, err return nil, err
} }
offset = newOff offset = newOff
accumVals[sig.Name] = elems accumVals[sig.Name] = elems
} else { accumElems[sig.Name] = n
// Fixed array (non-accumulated): one set of NumElements values.
elems, newOff, err := parseElems(payload, offset, n, sig)
if err != nil {
return nil, err
}
offset = newOff
fixedVals[sig.Name] = elems
}
} }
// Build one DataSample per slot. // Build one DataSample per slot.
@@ -386,10 +391,10 @@ func ParseData(payload []byte, sigs []SignalInfo, publishMode uint8, arrivalTime
for k := 0; k < numSamples; k++ { for k := 0; k < numSamples; k++ {
vals := make(map[string][]float64, len(sigs)) vals := make(map[string][]float64, len(sigs))
for sigName, av := range accumVals { for sigName, av := range accumVals {
vals[sigName] = []float64{av[k]} n := accumElems[sigName]
} // Sub-slice of the decoded block; the hub treats values as
for sigName, fv := range fixedVals { // read-only, so no copy is needed.
vals[sigName] = fv // shared read-only reference; hub does not modify vals[sigName] = av[k*n : (k+1)*n : (k+1)*n]
} }
samples[k] = DataSample{HRTTimestamp: hrt, WallTime: arrivalTime, Values: vals} samples[k] = DataSample{HRTTimestamp: hrt, WallTime: arrivalTime, Values: vals}
} }
Common/Client/go/udpsprotocol/sequence.go
+49
View File
@@ -0,0 +1,49 @@
package udpsprotocol
// SequenceGate orders DATA packets by the producer's packet counter.
//
// Reassembly completes in arrival order, not counter order, so a packet that
// was delayed or duplicated on the wire is handed up after a newer one has
// already been consumed. Its samples then carry an older time base than the
// data already in the ring: they land on top of samples that are already
// there, and the span they should have filled stays empty. That is a hole on
// one side and a collision on the other, from a packet that is entirely
// well-formed — the counter is the only thing that distinguishes it.
//
// A SequenceGate is not safe for concurrent use; each receive loop owns one.
type SequenceGate struct {
last uint32
valid bool
// Stale counts packets rejected for not advancing the counter (reordered
// or duplicated), for diagnostics.
Stale uint64
}
// Reset forgets the sequence. Call it on (re)connect: the producer's counter
// restarts independently of ours, so a counter carried over from the previous
// connection would reject the whole new stream.
func (g *SequenceGate) Reset() {
g.last = 0
g.valid = false
}
// Accept reports whether a DATA packet with this counter should be delivered,
// and how many packets went missing immediately before it.
//
// The counter is a wrapping uint32, so ordering is done on the signed
// difference: a plain comparison would call the first packet after the wrap
// stale and reject everything from then on.
func (g *SequenceGate) Accept(counter uint32) (ok bool, lost uint32) {
if !g.valid {
g.valid = true
g.last = counter
return true, 0
}
delta := int32(counter - g.last)
if delta <= 0 {
g.Stale++
return false, 0
}
g.last = counter
return true, uint32(delta) - 1
}
Common/Client/go/udpsprotocol/sequence_test.go
+82
View File
@@ -0,0 +1,82 @@
package udpsprotocol
import "testing"
// A packet older than one already delivered carries an older time base. Its
// samples land on top of data that is already in the ring and leave the span
// they should have filled empty, so it must not get through.
func TestSequenceGateRejectsStaleAndDuplicate(t *testing.T) {
var g SequenceGate
if ok, lost := g.Accept(10); !ok || lost != 0 {
t.Fatalf("first packet: got (%v, %d), want (true, 0)", ok, lost)
}
if ok, _ := g.Accept(11); !ok {
t.Fatal("counter 11 advances past 10 and must be accepted")
}
if ok, _ := g.Accept(9); ok {
t.Error("counter 9 is older than the delivered 11 and must be dropped")
}
if ok, _ := g.Accept(11); ok {
t.Error("a repeat of the delivered counter must be dropped")
}
if g.Stale != 2 {
t.Errorf("Stale = %d, want 2", g.Stale)
}
// The rejections must not have moved the sequence on.
if ok, lost := g.Accept(12); !ok || lost != 0 {
t.Errorf("after rejections: got (%v, %d), want (true, 0)", ok, lost)
}
}
// The loss count is what lets a consumer tell a widened gap from a slowed
// producer, so it must exclude the packet being delivered and must not persist
// into the next one.
func TestSequenceGateReportsLoss(t *testing.T) {
var g SequenceGate
g.Accept(100)
if _, lost := g.Accept(104); lost != 3 {
t.Errorf("101..103 missing: lost = %d, want 3", lost)
}
if _, lost := g.Accept(105); lost != 0 {
t.Errorf("consecutive packet: lost = %d, want 0", lost)
}
if g.Stale != 0 {
t.Errorf("Stale = %d, want 0", g.Stale)
}
}
// The counter is a wrapping uint32. Ordering it by plain comparison would call
// every packet after the wrap older than 0xFFFFFFFF and kill the stream.
func TestSequenceGateSurvivesWraparound(t *testing.T) {
var g SequenceGate
for _, c := range []uint32{0xFFFFFFFD, 0xFFFFFFFE, 0xFFFFFFFF, 0, 1, 2} {
ok, lost := g.Accept(c)
if !ok {
t.Fatalf("counter %#x rejected across the wrap", c)
}
if lost != 0 {
t.Errorf("counter %#x: lost = %d, want 0", c, lost)
}
}
// Loss must still be measured correctly across the wrap.
var h SequenceGate
h.Accept(0xFFFFFFFE)
if _, lost := h.Accept(1); lost != 2 {
t.Errorf("0xFFFFFFFF and 0 missing: lost = %d, want 2", lost)
}
}
// A reconnect restarts the producer's counter independently of ours; a carried
// over counter would reject the entire new stream.
func TestSequenceGateResetAcceptsLowerCounter(t *testing.T) {
var g SequenceGate
g.Accept(5000)
g.Reset()
if ok, lost := g.Accept(3); !ok || lost != 0 {
t.Errorf("after Reset: got (%v, %d), want (true, 0)", ok, lost)
}
}
Common/Client/go/wshub/capture_shortwindow_test.go
+75
View File
@@ -0,0 +1,75 @@
package wshub
import (
"math"
"testing"
)
// A short window at a high sample rate fits in a ring's initial capacity, so the
// retune sweep used to leave it there — and a ring holding exactly the window has
// already rolled past the front of a capture by the time that capture is read,
// which happens a post-window plus captureMarginSec after the trigger fires.
//
// 1 MSps over a 200 ms window: 200 k points fit in the 250 k initial ring, and
// every shot came back missing its first 123 ms.
func TestCaptureWholeAtHighRateShortWindow(t *testing.T) {
const (
key = "s1:Ch1"
rate = 1e6
window = 0.2
prePct = 20.0
batchSec = 1.0 / 30.0
simSec = 6.0
)
h := NewHub()
h.SetRingBudget(defaultRingPts)
h.rings[key] = newSigRing(ringCapInitial)
h.trigger.SetConfig(trigConfig{signalKey: key, edge: "rising", threshold: 0,
windowSec: window, prePercent: prePct, mode: "normal", holdoffSec: 0.2})
rateHz := float64(rate)
nBatch := int(rateHz * batchSec)
ts := make([]float64, nBatch)
vs := make([]float64, nBatch)
armed, shots := false, 0
for now := 0.0; now < simSec; now += batchSec {
for i := range ts {
ts[i] = now + float64(i)/rateHz
vs[i] = math.Sin(2 * math.Pi * 5 * ts[i]) // a rising crossing every 200 ms
}
h.ingest(key, 1, ts, vs)
h.retuneRings(now)
h.refreshTriggerFill()
if !armed && now > 2 {
h.trigger.Arm()
armed = true
}
trigTime, pre, post, ok := h.trigger.dueCapture(now + batchSec)
if !ok {
if h.trigger.dueRearm(now + batchSec) {
h.trigger.Arm()
}
continue
}
t0 := trigTime - pre
buf := h.buildTriggerCapture(trigTime, pre, post)
if buf == nil {
t.Fatalf("shot at t=%.4f produced no frame at all", trigTime)
}
first, last, n := decodeCaptureSpan(t, buf, key)
shots++
if lost := first - t0; lost > shortCaptureTol*window {
_, span := h.rings[key].stats()
t.Errorf("shot at t=%.4f is missing %.0f ms at the front of its %.0f ms window "+
"(got [%.4f,%.4f], %d pts; ring holds %.4f s in %d points)",
trigTime, 1e3*lost, 1e3*window, first, last, n, span, h.rings[key].capacity())
}
h.trigger.markTriggered(now + batchSec)
}
if shots < 3 {
t.Fatalf("only %d shots in %.0f s", shots, simSec)
}
}
Common/Client/go/wshub/export.go
+119
View File
@@ -0,0 +1,119 @@
package wshub
import (
"fmt"
"net/http"
"sort"
"strconv"
"strings"
"time"
"github.com/parquet-go/parquet-go"
)
// ExportSample is one row of the binary export: a single stored sample, in
// long ("tidy") form, keyed by source and signal with its own timestamp.
//
// Keeping each signal's samples as its own rows — rather than resampling onto a
// shared time grid — is what makes the export hole-free: per-signal streams of
// different lengths export exactly as stored, nothing is fabricated, and
// nothing is dropped.
type ExportSample struct {
Source string `parquet:"source"`
Signal string `parquet:"signal"`
Time float64 `parquet:"time"`
Value float64 `parquet:"value"`
}
// exportChunkRows bounds each batched write and, via MaxRowsPerRowGroup, the
// size of each parquet row group: memory stays bounded however large the
// export is, because a finished row group is flushed to the HTTP stream.
const exportChunkRows = 65536
// exportWriteBuffer is the parquet writer's output buffer: larger than the
// 32KiB default means fewer writes on the HTTP stream for a multi-GB export.
const exportWriteBuffer = 1 << 20
// HandleExport serves GET /api/export?t0=..&t1=..[&signals=a,b] as a Parquet
// file containing every stored sample of the named signals in [t0, t1].
//
// Unlike /api/zoom there is no decimation: the file holds the full contents of
// the rings. At rates above the ring budget those contents are min/max buckets
// (the finest resolution the hub retains); at lower rates they are verbatim.
func (h *Hub) HandleExport(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
t0, err0 := strconv.ParseFloat(q.Get("t0"), 64)
t1, err1 := strconv.ParseFloat(q.Get("t1"), 64)
if err0 != nil || err1 != nil || t1 <= t0 {
http.Error(w, "invalid t0/t1", http.StatusBadRequest)
return
}
var keys []string
if s := strings.TrimSpace(q.Get("signals")); s != "" {
keys = strings.Split(s, ",")
for i := range keys {
keys[i] = strings.TrimSpace(keys[i])
}
}
// Snapshot the rings we will read. A signal removed mid-export must not
// silently drop rows from the file.
h.ringsMu.RLock()
refs := make(map[string]*sigRing)
if keys == nil {
for k, rb := range h.rings {
refs[k] = rb
}
} else {
for _, k := range keys {
if rb, ok := h.rings[k]; ok {
refs[k] = rb
}
}
}
h.ringsMu.RUnlock()
if len(refs) == 0 {
http.Error(w, "no signals", http.StatusNotFound)
return
}
// Deterministic column order.
names := make([]string, 0, len(refs))
for k := range refs {
names = append(names, k)
}
sort.Strings(names)
w.Header().Set("Content-Type", "application/vnd.apache.parquet")
w.Header().Set("Content-Disposition",
fmt.Sprintf("attachment; filename=\"signals_%d.parquet\"", time.Now().Unix()))
writer := parquet.NewGenericWriter[ExportSample](w,
parquet.MaxRowsPerRowGroup(exportChunkRows),
parquet.WriteBufferSize(exportWriteBuffer),
)
batch := make([]ExportSample, 0, exportChunkRows)
for _, key := range names {
st, sv := refs[key].slice(t0, t1)
colon := strings.IndexByte(key, ':')
source, signal := key, key
if colon >= 0 {
source = key[:colon]
signal = key[colon+1:]
}
for i := range st {
batch = append(batch, ExportSample{Source: source, Signal: signal, Time: st[i], Value: sv[i]})
if len(batch) >= exportChunkRows {
if _, err := writer.Write(batch); err != nil {
// Client went away or the stream broke; stop writing.
return
}
batch = batch[:0]
}
}
}
if len(batch) > 0 {
_, _ = writer.Write(batch)
}
_ = writer.Close()
}
Common/Client/go/wshub/export_test.go
+87
View File
@@ -0,0 +1,87 @@
package wshub
import (
"bytes"
"net/http/httptest"
"testing"
"github.com/parquet-go/parquet-go"
)
func TestHandleExportParquetFullResolution(t *testing.T) {
h := NewHub()
// Two signals with different lengths and offset time bases: the export must
// keep every sample of each, on its own timestamps (no holes, no
// resampling, no decimation).
sig1 := newSigRing(10000)
sig2 := newSigRing(10000)
t1, v1 := make([]float64, 1000), make([]float64, 1000)
for i := range t1 {
t1[i] = float64(i) * 0.001
v1[i] = float64(i) * 2
}
sig1.write(t1, v1)
t2, v2 := make([]float64, 500), make([]float64, 500)
for i := range t2 {
t2[i] = 0.1 + float64(i)*0.002
v2[i] = -float64(i)
}
sig2.write(t2, v2)
h.rings["s1:Ch1"] = sig1
h.rings["s1:Ch2"] = sig2
req := httptest.NewRequest("GET", "/api/export?t0=0&t1=2&signals=s1:Ch1,s1:Ch2", nil)
rec := httptest.NewRecorder()
h.HandleExport(rec, req)
if rec.Code != 200 {
t.Fatalf("status = %d, want 200 (body: %s)", rec.Code, rec.Body.String())
}
reader := parquet.NewGenericReader[ExportSample](bytes.NewReader(rec.Body.Bytes()))
defer reader.Close()
var got []ExportSample
buf := make([]ExportSample, 1000)
for {
n, err := reader.Read(buf)
got = append(got, buf[:n]...)
if err != nil {
break
}
}
if len(got) != 1500 {
t.Fatalf("rows = %d, want 1500 (every sample of both signals)", len(got))
}
ch1 := filterExportSamples(got, "s1", "Ch1")
ch2 := filterExportSamples(got, "s1", "Ch2")
if len(ch1) != 1000 || len(ch2) != 500 {
t.Fatalf("ch1=%d ch2=%d rows, want 1000/500 (no holes, no resampling)", len(ch1), len(ch2))
}
if ch1[0].Time != 0 || ch1[0].Value != 0 || ch1[999].Time != 0.999 || ch1[999].Value != 1998 {
t.Fatalf("ch1 endpoints wrong: first=%+v last=%+v", ch1[0], ch1[999])
}
if ch2[0].Time != 0.1 || ch2[499].Time != 0.1+499*0.002 || ch2[499].Value != -499 {
t.Fatalf("ch2 endpoints wrong: first=%+v last=%+v", ch2[0], ch2[499])
}
}
func filterExportSamples(rows []ExportSample, source, signal string) []ExportSample {
out := make([]ExportSample, 0, len(rows))
for _, r := range rows {
if r.Source == source && r.Signal == signal {
out = append(out, r)
}
}
return out
}
func TestHandleExportParquetBadRange(t *testing.T) {
h := NewHub()
h.rings["s1:Ch1"] = newSigRing(10)
req := httptest.NewRequest("GET", "/api/export?t0=2&t1=1", nil)
rec := httptest.NewRecorder()
h.HandleExport(rec, req)
if rec.Code != 400 {
t.Fatalf("status = %d, want 400 for inverted range", rec.Code)
}
}
Common/Client/go/wshub/history.go
+26 -17
View File
@@ -422,6 +422,26 @@ func (hw *historyWriter) window() float64 {
return hw.windowSec return hw.windowSec
} }
// coversWindow reports whether the archive file for key currently spans at
// least sec seconds. When true, backfillCaptureHead can reconstruct a capture's
// front out of the archive, so the trigger need not wait for the ring to cover
// the whole window on its own.
func (hw *historyWriter) coversWindow(key string, sec float64) bool {
if !hw.enabled() || !(sec > 0) {
return false
}
hw.mu.RLock()
hf, ok := hw.files[key]
hw.mu.RUnlock()
if !ok {
return false
}
hf.mu.RLock()
span := hf.tNewest - hf.tOldest
hf.mu.RUnlock()
return span >= sec
}
// setWindow points the archive at the timespan the clients are looking at, and // setWindow points the archive at the timespan the clients are looking at, and
// re-sizes the files that no longer match it. It reports whether any file's // re-sizes the files that no longer match it. It reports whether any file's
// geometry changed, which invalidates what clients know about the archive. // geometry changed, which invalidates what clients know about the archive.
@@ -842,10 +862,7 @@ func (hf *histFile) readAfter(after, t0, t1 float64, max int) ([]byte, float64,
// The run wraps at most once, so it costs at most two reads. // The run wraps at most once, so it costs at most two reads.
buf := make([]byte, n*histPairSize) buf := make([]byte, n*histPairSize)
start := (oldest + lo) % capacity start := (oldest + lo) % capacity
head := int(capacity-start) * histPairSize head := min(int(capacity-start)*histPairSize, len(buf))
if head > len(buf) {
head = len(buf)
}
if _, err := hf.f.ReadAt(buf[:head], int64(histHeaderSize)+int64(start)*histPairSize); err != nil { if _, err := hf.f.ReadAt(buf[:head], int64(histHeaderSize)+int64(start)*histPairSize); err != nil {
return nil, 0, err return nil, 0, err
} }
@@ -898,10 +915,8 @@ func (hf *histFile) writePairs(t, v []float64) error {
binary.LittleEndian.PutUint64(buf[i*histPairSize+8:], math.Float64bits(v[i])) binary.LittleEndian.PutUint64(buf[i*histPairSize+8:], math.Float64bits(v[i]))
} }
first := int(hf.capacity - hf.head) first := min(int(hf.capacity-hf.head), n)
if first > n {
first = n
}
off := int64(histHeaderSize) + int64(hf.head)*histPairSize off := int64(histHeaderSize) + int64(hf.head)*histPairSize
if _, err := hf.f.WriteAt(buf[:first*histPairSize], off); err != nil { if _, err := hf.f.WriteAt(buf[:first*histPairSize], off); err != nil {
return err return err
@@ -1089,10 +1104,7 @@ func (hw *historyWriter) readRange(key string, t0, t1 float64, maxOut int) ([]fl
// Read in contiguous runs: the range wraps at most once. // Read in contiguous runs: the range wraps at most once.
buf := make([]byte, n*histPairSize) buf := make([]byte, n*histPairSize)
start := (oldest + lo) % capacity start := (oldest + lo) % capacity
first := int(capacity - start) first := min(int(capacity-start), n)
if first > n {
first = n
}
if _, err := hf.f.ReadAt(buf[:first*histPairSize], if _, err := hf.f.ReadAt(buf[:first*histPairSize],
int64(histHeaderSize)+int64(start)*histPairSize); err != nil { int64(histHeaderSize)+int64(start)*histPairSize); err != nil {
return nil, nil return nil, nil
@@ -1265,7 +1277,7 @@ func (h *Hub) handleSetHistoryBudget(env map[string]interface{}) {
// handleHistoryZoom answers a historyZoom request from disk. Same request and // handleHistoryZoom answers a historyZoom request from disk. Same request and
// reply shape as "zoom", so clients can fall back to it transparently when a // reply shape as "zoom", so clients can fall back to it transparently when a
// window reaches further back than the in-memory rings hold. // window reaches further back than the in-memory rings hold.
func (h *Hub) handleHistoryZoom(c *wsClient, env map[string]interface{}) { func (h *Hub) handleHistoryZoom(c *wsClient, env map[string]any) {
if !h.hist.enabled() { if !h.hist.enabled() {
msg, _ := json.Marshal(map[string]any{ msg, _ := json.Marshal(map[string]any{
"type": "historyZoom", "reqId": env["reqId"], "type": "historyZoom", "reqId": env["reqId"],
@@ -1287,10 +1299,7 @@ func (h *Hub) handleHistoryZoom(c *wsClient, env map[string]interface{}) {
// oversampled relative to the plot's point budget and thinned afterwards. // oversampled relative to the plot's point budget and thinned afterwards.
// The cap keeps a request for "no decimation" over a multi-hour window from // The cap keeps a request for "no decimation" over a multi-hour window from
// pulling the whole file into memory. // pulling the whole file into memory.
readCap := n * histReadOversample readCap := min(n*histReadOversample, histDefaultMaxPoints)
if readCap > histMaxReadPoints {
readCap = histMaxReadPoints
}
signals := make(map[string]sigData) signals := make(map[string]sigData)
for _, k := range strings.Split(sigCSV, ",") { for _, k := range strings.Split(sigCSV, ",") {
Common/Client/go/wshub/hub.go
+12 -3
View File
@@ -1216,15 +1216,24 @@ func (h *Hub) buildBinaryDataMessageForSource(src *sourceHubState, batch []udpsp
wallNs := s.WallTime.UnixNano() wallNs := s.WallTime.UnixNano()
wallSec := float64(wallNs) / 1e9 wallSec := float64(wallNs) / 1e9
var dtSec float64 var dtSec float64
// A gap spans the elements of every packet that went missing
// inside it as well as this packet's own, so the divisor has
// to widen with it. Without this a single loss halves the
// apparent rate and the elements overrun into the next
// packet's range. The loss count belongs to the packet the
// gap ends at.
if bi+1 < len(batch) { if bi+1 < len(batch) {
// Two consecutive packets in this tick → exact dt. // Two consecutive packets in this tick → exact dt.
dtSec = (float64(batch[bi+1].WallTime.UnixNano()) - float64(wallNs)) / 1e9 / float64(n) span := float64(n) * float64(1+batch[bi+1].Lost)
dtSec = (float64(batch[bi+1].WallTime.UnixNano()) - float64(wallNs)) / 1e9 / span
} else if bi > 0 { } else if bi > 0 {
// Last of multiple packets → use diff from previous. // Last of multiple packets → use diff from previous.
dtSec = (float64(wallNs) - float64(batch[bi-1].WallTime.UnixNano())) / 1e9 / float64(n) span := float64(n) * float64(1+s.Lost)
dtSec = (float64(wallNs) - float64(batch[bi-1].WallTime.UnixNano())) / 1e9 / span
} else if prevNs, ok2 := src.lastPktNs[sig.Name]; ok2 && prevNs > 0 && wallNs > prevNs { } else if prevNs, ok2 := src.lastPktNs[sig.Name]; ok2 && prevNs > 0 && wallNs > prevNs {
// Single packet this tick → gap from the previous tick's packet. // Single packet this tick → gap from the previous tick's packet.
dtSec = (float64(wallNs) - float64(prevNs)) / 1e9 / float64(n) span := float64(n) * float64(1+s.Lost)
dtSec = (float64(wallNs) - float64(prevNs)) / 1e9 / span
} else { } else {
// Truly first packet ever — inter-packet timing unknown. // Truly first packet ever — inter-packet timing unknown.
// Skip to avoid poisoning the ring with wrongly-spaced timestamps; // Skip to avoid poisoning the ring with wrongly-spaced timestamps;
Common/Client/go/wshub/packet_dt_loss_test.go
+139
View File
@@ -0,0 +1,139 @@
package wshub
import (
"math"
"testing"
"time"
"marte2/common/udpsprotocol"
)
// pktSignal is a 4-element array with no time signal and no declared sampling
// rate, i.e. the TimeModePacket path where dt has to be inferred from the gap
// between packets.
func pktSignal(name string) udpsprotocol.SignalInfo {
return udpsprotocol.SignalInfo{
Name: name,
TypeCode: 8, // float64
NumDimensions: 1,
NumRows: 4,
NumCols: 1,
TimeMode: udpsprotocol.TimeModePacket,
TimeSignalIdx: udpsprotocol.NoTimeSignal,
}
}
// newPacketDtHub builds a Hub with a ring for one packet-timed array signal and
// returns both. It does not start Run(): buildBinaryDataMessageForSource is
// called directly so the timestamps it produces can be read back verbatim.
func newPacketDtHub(t *testing.T, sigName string) (*Hub, *sourceHubState, *sigRing) {
t.Helper()
h := NewHub()
src := &sourceHubState{
id: "s1",
signals: []udpsprotocol.SignalInfo{pktSignal(sigName)},
timeSigCalib: map[string]float64{},
lastPktNs: map[string]int64{},
lastFrameMeasured: map[string]float64{},
lastFrameEndT: map[string]float64{},
gapEMA: map[string]float64{},
}
rb := newSigRing(4096)
h.rings["s1:"+sigName] = rb
return h, src, rb
}
// packet builds a one-signal batch entry arriving at t0 with the given loss
// count; the values are irrelevant, only the timestamps are under test.
func packet(sigName string, at time.Time, lost uint32, n int) udpsprotocol.DataSample {
vals := make([]float64, n)
return udpsprotocol.DataSample{WallTime: at, Values: map[string][]float64{sigName: vals}, Lost: lost}
}
// ringTimes returns the timestamps written to the ring, in order.
func ringTimes(rb *sigRing) []float64 {
rb.mu.RLock()
defer rb.mu.RUnlock()
out := make([]float64, 0, rb.size)
start := (rb.head - rb.size + rb.cap) % rb.cap
for i := 0; i < rb.size; i++ {
out = append(out, rb.t[(start+i)%rb.cap])
}
return out
}
// A lost packet widens the inter-packet gap without adding elements to the
// packet that follows it. Dividing the gap by that packet's element count
// alone reports a period too long by exactly the number of packets missing,
// which walks the elements past their own end and into the range the next
// packet claims: they collide there, and the span they vacated stays empty.
func TestPacketDtIgnoresLostPacketWidening(t *testing.T) {
const sig = "Wave"
const n = 4
const dt = 1 * time.Millisecond
base := time.Unix(1700000000, 0)
h, src, rb := newPacketDtHub(t, sig)
// One clean packet establishes lastPktNs.
h.buildBinaryDataMessageForSource(src, []udpsprotocol.DataSample{
packet(sig, base, 0, n)})
// The next producer packet is lost, so the one after it arrives a full
// extra batch later and reports Lost=1.
arrival := base.Add(2 * n * dt)
h.buildBinaryDataMessageForSource(src, []udpsprotocol.DataSample{
packet(sig, arrival, 1, n)})
ts := ringTimes(rb)
if len(ts) != n {
t.Fatalf("ring holds %d points, want %d (the first packet is skipped: no gap yet)", len(ts), n)
}
got := ts[1] - ts[0]
if !nearSec(got, dt.Seconds()) {
t.Errorf("dt = %v s, want %v s (the gap spans two batches, not one)", got, dt.Seconds())
}
// Elements run forward from the packet's own arrival, so a doubled dt
// would stretch this batch across two batch periods and into the range the
// next packet claims.
span := ts[len(ts)-1] - ts[0]
if !nearSec(span, float64(n-1)*dt.Seconds()) {
t.Errorf("batch spans %v s, want %v s: it overruns into the next packet's range",
span, float64(n-1)*dt.Seconds())
}
}
// nearSec compares two intervals in seconds. The hub carries timestamps as
// float64 seconds derived from UnixNano, whose spacing near the current epoch
// is a couple of hundred nanoseconds, so exact equality is not available. The
// defect under test moves the period by a factor of two, three orders of
// magnitude outside this tolerance.
func nearSec(got, want float64) bool { return math.Abs(got-want) <= 1e-6 }
// The correction must be driven by the reported loss and nothing else: with no
// packet missing the period still comes straight from the gap, so a producer
// that genuinely slows down is followed rather than second-guessed.
func TestPacketDtFollowsGapWhenNothingIsLost(t *testing.T) {
const sig = "Wave"
const n = 4
base := time.Unix(1700000000, 0)
h, src, rb := newPacketDtHub(t, sig)
h.buildBinaryDataMessageForSource(src, []udpsprotocol.DataSample{
packet(sig, base, 0, n)})
// Same widened gap as the test above, but reported as no loss: the
// producer really is running at half the rate.
slowDt := 2 * time.Millisecond
h.buildBinaryDataMessageForSource(src, []udpsprotocol.DataSample{
packet(sig, base.Add(n*slowDt), 0, n)})
ts := ringTimes(rb)
if len(ts) != n {
t.Fatalf("ring holds %d points, want %d", len(ts), n)
}
if got := ts[1] - ts[0]; !nearSec(got, slowDt.Seconds()) {
t.Errorf("dt = %v s, want %v s: a real rate change must be followed", got, slowDt.Seconds())
}
}
Common/Client/go/wshub/ringbuf.go
+14 -1
View File
@@ -221,6 +221,19 @@ func ringCoverage(bucket, capacity int) int {
return capacity / 2 * bucket return capacity / 2 * bucket
} }
// captureLagSec is how much further back than the window itself a ring has to
// reach to deliver a capture of it.
//
// A capture is not read out when its last sample arrives but captureMarginSec
// later, and then only on the next push tick — so by the time the window is
// extracted, its oldest sample is that much deeper in the ring. A ring holding
// exactly the window has already overwritten the front of its own capture, which
// is what made every shot at a short window come back missing its head. The
// pre/post split does not enter into it: the harvest is a post-window after the
// trigger and the read reaches a pre-window before it, so the two sum to the
// window whatever the split.
const captureLagSec = captureMarginSec + 1.0/30.0
// activeWindowSec is the timespan the buffers must cover. An armed trigger owns // activeWindowSec is the timespan the buffers must cover. An armed trigger owns
// it: its pre-window has to already be in the ring when the trigger fires or // it: its pre-window has to already be in the ring when the trigger fires or
// there is nothing to back-fill the capture from. Otherwise it is the widest // there is nothing to back-fill the capture from. Otherwise it is the widest
@@ -228,7 +241,7 @@ func ringCoverage(bucket, capacity int) int {
func (h *Hub) activeWindowSec() float64 { func (h *Hub) activeWindowSec() float64 {
if h.trigger != nil && h.trigger.Active() { if h.trigger != nil && h.trigger.Active() {
if cfg := h.trigger.Config(); cfg.windowSec > 0 { if cfg := h.trigger.Config(); cfg.windowSec > 0 {
return cfg.windowSec return cfg.windowSec + captureLagSec
} }
} }
widest := 0.0 widest := 0.0
Common/Client/go/wshub/ringbuf_test.go
+6 -3
View File
@@ -120,7 +120,9 @@ func TestActiveWindowSecTakesTheWidestClientWindow(t *testing.T) {
} }
// An armed trigger owns the window: its pre-window has to be in the buffer // An armed trigger owns the window: its pre-window has to be in the buffer
// before the trigger fires or the capture has nothing to back-fill from. // before the trigger fires or the capture has nothing to back-fill from. The
// buffers must reach back past the window itself, because the capture is read
// out a margin and a tick after its last sample lands.
func TestActiveWindowSecPrefersTheArmedTrigger(t *testing.T) { func TestActiveWindowSecPrefersTheArmedTrigger(t *testing.T) {
h := NewHub() h := NewHub()
c := &wsClient{} c := &wsClient{}
@@ -128,8 +130,9 @@ func TestActiveWindowSecPrefersTheArmedTrigger(t *testing.T) {
h.clients[c] = true h.clients[c] = true
h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 45, mode: "normal"}) h.trigger.SetConfig(trigConfig{signalKey: "s1:sig", windowSec: 45, mode: "normal"})
if got := h.activeWindowSec(); got != 45 { if got := h.activeWindowSec(); got != 45+captureLagSec {
t.Fatalf("activeWindowSec = %v, want the trigger's 45", got) t.Fatalf("activeWindowSec = %v, want the trigger's 45 plus the %v harvest lag",
got, captureLagSec)
} }
} }
Common/Client/go/wshub/sources.go
+22
View File
@@ -341,6 +341,9 @@ func (u *UDPClient) runSession() error {
} }
reassembler := udpsprotocol.NewReassembler(2 * time.Second) reassembler := udpsprotocol.NewReassembler(2 * time.Second)
// Per-session: the producer's counter restarts independently of ours, so
// the gate must not carry a counter over from the previous connection.
var gate udpsprotocol.SequenceGate
buf := make([]byte, readBufSize) buf := make([]byte, readBufSize)
var currentSigs []udpsprotocol.SignalInfo var currentSigs []udpsprotocol.SignalInfo
var currentPublishMode uint8 var currentPublishMode uint8
@@ -414,11 +417,20 @@ func (u *UDPClient) runSession() error {
if len(currentSigs) == 0 { if len(currentSigs) == 0 {
continue continue
} }
fresh, lost := gate.Accept(hdr.Counter)
if !fresh {
continue
}
samples, err := udpsprotocol.ParseData(complete, currentSigs, currentPublishMode, arrivalTime) samples, err := udpsprotocol.ParseData(complete, currentSigs, currentPublishMode, arrivalTime)
if err != nil { if err != nil {
log.Printf("[%s] udp: parse data: %v", u.sourceID, err) log.Printf("[%s] udp: parse data: %v", u.sourceID, err)
continue continue
} }
// The gap precedes the packet, so it belongs to its first slot only;
// the slots after it are consecutive cycles of the same batch.
if len(samples) > 0 {
samples[0].Lost = lost
}
for _, s := range samples { for _, s := range samples {
u.hub.PushDataForSource(u.sourceID, s) u.hub.PushDataForSource(u.sourceID, s)
} }
@@ -589,6 +601,9 @@ func (u *UDPClient) runMulticastSession() error {
}() }()
reassembler := udpsprotocol.NewReassembler(2 * time.Second) reassembler := udpsprotocol.NewReassembler(2 * time.Second)
// Per-session, as in runSession(): a counter from the previous connection
// would reject the whole new stream.
var gate udpsprotocol.SequenceGate
buf := make([]byte, readBufSize) buf := make([]byte, readBufSize)
for { for {
@@ -629,11 +644,18 @@ func (u *UDPClient) runMulticastSession() error {
if len(currentSigs) == 0 { if len(currentSigs) == 0 {
continue continue
} }
fresh, lost := gate.Accept(hdr.Counter)
if !fresh {
continue
}
samples, parseErr := udpsprotocol.ParseData(complete, currentSigs, currentPublishMode, arrivalTime) samples, parseErr := udpsprotocol.ParseData(complete, currentSigs, currentPublishMode, arrivalTime)
if parseErr != nil { if parseErr != nil {
log.Printf("[%s] multicast: parse data: %v", u.sourceID, parseErr) log.Printf("[%s] multicast: parse data: %v", u.sourceID, parseErr)
continue continue
} }
if len(samples) > 0 {
samples[0].Lost = lost
}
for _, s := range samples { for _, s := range samples {
u.hub.PushDataForSource(u.sourceID, s) u.hub.PushDataForSource(u.sourceID, s)
} }
Common/Client/go/wshub/trigger.go
+119 -13
View File
@@ -92,6 +92,14 @@ type triggerEngine struct {
bufGrowth float64 bufGrowth float64
bufKnown bool bufKnown bool
bufRateOK bool bufRateOK bool
// bufCoverage is the maximum span (seconds) the ring can reach at its
// current bucket and capacity — the gate must never demand more than this,
// or a ring whose coverage is below the window can never satisfy it. 0 =
// unknown (no measurable rate).
bufCoverage float64
// bufArchived is true when the disk history already spans the trigger
// window, so a short capture's front can be back-filled from it.
bufArchived bool
// Reference point the growth is measured against. // Reference point the growth is measured against.
bufRefSpan, bufRefWall float64 bufRefSpan, bufRefWall float64
@@ -108,6 +116,22 @@ type triggerEngine struct {
firedPost float64 firedPost float64
firedValid bool firedValid bool
// The edge to fire on as soon as the FSM rearms, in sample time. Recorded
// while a capture is still being collected or handed out, for edges late
// enough that a capture of them would not overlap the one in flight.
//
// Without this the trigger is deaf from its own trigger point until the
// capture has been harvested — a post-window plus captureMarginSec — and
// then for the holdoff on top of that, and afterwards waits for a FRESH
// edge. On a sparse pulse train that rounds the capture spacing up to a
// whole pulse period: at the default 1 s window the blind stretch comes to
// 1.15 s, so a 1 Hz train was caught at 0.5 Hz and a wider window lost whole
// multiples. Remembering the edge instead makes the blind stretch exactly
// the post-window it has to be, since the capture is built from the edge's
// own timestamp and the ring still holds everything around it.
pendingT float64
pendingValid bool
rearmAt float64 // wall-clock seconds; 0 when no rearm is pending rearmAt float64 // wall-clock seconds; 0 when no rearm is pending
} }
@@ -163,10 +187,14 @@ func (te *triggerEngine) SetConfig(cfg trigConfig) {
if base != te.baseKey { if base != te.baseKey {
// The buffer measurement belongs to the old signal's ring. // The buffer measurement belongs to the old signal's ring.
te.bufKnown, te.bufRateOK = false, false te.bufKnown, te.bufRateOK = false, false
te.bufCoverage, te.bufArchived = 0, false
} }
te.baseKey, te.elemIdx = base, idx te.baseKey, te.elemIdx = base, idx
te.prevValid = false te.prevValid = false
te.prevValue = 0 te.prevValue = 0
// An edge held over from the old configuration would be latched against the
// new window, whose fill the gate has not vouched for.
te.pendingValid = false
} }
func (te *triggerEngine) Config() trigConfig { func (te *triggerEngine) Config() trigConfig {
@@ -175,15 +203,37 @@ func (te *triggerEngine) Config() trigConfig {
return te.cfg return te.cfg
} }
// Arm starts a fresh acquisition. It is the user's own arm, so it discards any
// edge remembered during the previous capture: the user asked for the next
// event, not for one that has already been and gone.
func (te *triggerEngine) Arm() { func (te *triggerEngine) Arm() {
te.mu.Lock() te.mu.Lock()
te.state = trigArmed te.state = trigArmed
te.prevValid = false te.prevValid = false
te.prevValue = 0 te.prevValue = 0
te.pendingValid = false
te.rearmAt = 0 te.rearmAt = 0
te.mu.Unlock() te.mu.Unlock()
} }
// rearm is the automatic arm at the end of a capture. Unlike Arm it honours an
// edge that arrived while the capture was being collected, firing on it at once
// rather than waiting for the next one — see pendingT. It also keeps the level
// tracked through the dead time, so the first sample after rearming is compared
// against its real predecessor instead of being spent seeding one.
func (te *triggerEngine) rearm() {
te.mu.Lock()
te.rearmAt = 0
if te.pendingValid {
t := te.pendingT
te.pendingValid = false
te.latchWindowLocked(t)
} else {
te.state = trigArmed
}
te.mu.Unlock()
}
func (te *triggerEngine) Disarm() { func (te *triggerEngine) Disarm() {
te.mu.Lock() te.mu.Lock()
te.state = trigIdle te.state = trigIdle
@@ -191,6 +241,7 @@ func (te *triggerEngine) Disarm() {
te.prevValid = false te.prevValid = false
te.prevValue = 0 te.prevValue = 0
te.firedValid = false te.firedValid = false
te.pendingValid = false
te.rearmAt = 0 te.rearmAt = 0
te.mu.Unlock() te.mu.Unlock()
} }
@@ -243,13 +294,16 @@ const bufGrowthIntervalSec = 0.5
const bufGrowthSmooth = 0.5 const bufGrowthSmooth = 0.5
// setBuffered records how far back the trigger signal's ring reaches, at wall // setBuffered records how far back the trigger signal's ring reaches, at wall
// clock now, and derives how fast that is growing. Pass known=false when there // clock now, and derives how fast that is growing. coverage is the maximum
// is no such ring. // span (seconds) the ring can reach at its current bucket/capacity; archived
func (te *triggerEngine) setBuffered(span float64, known bool, now float64) { // says the disk history already spans the trigger window. Pass known=false when
// there is no ring to measure.
func (te *triggerEngine) setBuffered(span, coverage float64, archived, known bool, now float64) {
te.mu.Lock() te.mu.Lock()
defer te.mu.Unlock() defer te.mu.Unlock()
if !known { if !known {
te.bufKnown, te.bufRateOK = false, false te.bufKnown, te.bufRateOK = false, false
te.bufCoverage, te.bufArchived = 0, false
return return
} }
if !te.bufKnown { if !te.bufKnown {
@@ -257,6 +311,8 @@ func (te *triggerEngine) setBuffered(span float64, known bool, now float64) {
te.bufRefSpan, te.bufRefWall = span, now te.bufRefSpan, te.bufRefWall = span, now
} }
te.bufSpan = span te.bufSpan = span
te.bufCoverage = coverage
te.bufArchived = archived
dt := now - te.bufRefWall dt := now - te.bufRefWall
if dt < bufGrowthIntervalSec { if dt < bufGrowthIntervalSec {
return return
@@ -294,9 +350,17 @@ func (te *triggerEngine) setBuffered(span float64, known bool, now float64) {
// anyway. A full one grows only as fast as its incoming samples free space — // anyway. A full one grows only as fast as its incoming samples free space —
// re-bucketing to a longer window replaces dense old samples with sparse new // re-bucketing to a longer window replaces dense old samples with sparse new
// ones — and it is that case, growth well below 1, where firing on the // ones — and it is that case, growth well below 1, where firing on the
// pre-window alone delivers a capture whose front has been overwritten by the //
// time it is read. In the steady state growth is 0 and need is the whole // Two escapes keep an armed trigger from staying deaf forever:
// window, which a ring tuned for that window already exceeds, so nothing waits. //
// - archived — the disk history already spans the window, so the front of a
// capture can be back-filled from it; the ring only needs to
// hold the pre-window worth of recent data.
// - coverage — never demand more than the ring can physically reach. If its
// coverage saturates below the window (a measured source rate
// that over-estimates the true one), the gate opens once the
// ring is full anyway and a short capture is delivered instead
// of deafness.
func (te *triggerEngine) fillNeedLocked() float64 { func (te *triggerEngine) fillNeedLocked() float64 {
pre := te.cfg.windowSec * te.cfg.prePercent / 100 pre := te.cfg.windowSec * te.cfg.prePercent / 100
growth := 0.0 // until measured, assume the buffer will not fill on its own growth := 0.0 // until measured, assume the buffer will not fill on its own
@@ -307,6 +371,14 @@ func (te *triggerEngine) fillNeedLocked() float64 {
if need < pre { if need < pre {
need = pre need = pre
} }
if te.bufArchived {
// The archive back-fills the front; the ring holds the post-trigger
// window live, so the pre-window is all it needs to have reached.
return pre
}
if te.bufCoverage > 0 && need > te.bufCoverage {
need = te.bufCoverage
}
return need return need
} }
@@ -366,7 +438,11 @@ func (te *triggerEngine) feed(key string, nElem int, t, v []float64) {
te.lastT = t[len(t)-1] te.lastT = t[len(t)-1]
te.lastTOK = true te.lastTOK = true
te.lastFeedWall = float64(time.Now().UnixNano()) / 1e9 te.lastFeedWall = float64(time.Now().UnixNano()) / 1e9
if te.state != trigArmed {
// A capture in flight does not stop the comparator; it only changes what an
// edge does. See pendingT.
inFlight := te.state == trigCollecting || te.state == trigTriggered
if te.state != trigArmed && !inFlight {
return return
} }
step, start := 1, 0 step, start := 1, 0
@@ -381,12 +457,20 @@ func (te *triggerEngine) feed(key string, nElem int, t, v []float64) {
// which is what made the first shot after a window change come back short. // which is what made the first shot after a window change come back short.
// Track the level meanwhile, so the first edge once the buffer is deep // Track the level meanwhile, so the first edge once the buffer is deep
// enough is still measured against the right previous sample. // enough is still measured against the right previous sample.
if te.fillLocked() < 1 { if !inFlight && te.fillLocked() < 1 {
for i := start; i < len(v); i += step { for i := start; i < len(v); i += step {
te.prevValue, te.prevValid = v[i], true te.prevValue, te.prevValid = v[i], true
} }
return return
} }
// The earliest trigger point a new capture may take. The one in flight owns
// everything up to the end of its own post-window, and the holdoff — a guard
// against re-triggering on the ringing of the SAME event — is measured from
// its trigger point too, so the two overlap rather than add.
notBefore := math.Inf(1)
if inFlight && te.firedValid {
notBefore = te.trigTime + math.Max(te.firedPost, te.cfg.holdoffSec)
}
thr := te.cfg.threshold thr := te.cfg.threshold
for i := start; i < len(t); i += step { for i := start; i < len(t); i += step {
if !te.prevValid { if !te.prevValid {
@@ -406,10 +490,19 @@ func (te *triggerEngine) feed(key string, nElem int, t, v []float64) {
default: default:
fired = up fired = up
} }
if fired { if !fired {
continue
}
if !inFlight {
te.latchWindowLocked(t[i]) te.latchWindowLocked(t[i])
return return
} }
// Keep the FIRST qualifying edge and go on tracking the level: a later
// one would be no more use, and stopping here would leave prevValue
// stale by the time the FSM rearms.
if !te.pendingValid && t[i] >= notBefore {
te.pendingT, te.pendingValid = t[i], true
}
} }
} }
@@ -592,19 +685,32 @@ func (h *Hub) refreshTriggerFill() {
return return
} }
now := float64(time.Now().UnixNano()) / 1e9 now := float64(time.Now().UnixNano()) / 1e9
key := h.trigger.baseSignalKey()
var rb *sigRing var rb *sigRing
if key := h.trigger.baseSignalKey(); key != "" { if key != "" {
rb = h.getRing(key) rb = h.getRing(key)
} }
if rb == nil { if rb == nil {
// Nothing to measure. Do not gate on a signal the hub does not carry: // Nothing to measure. Do not gate on a signal the hub does not carry:
// that would leave the trigger armed forever, which is worse than a // that would leave the trigger armed forever, which is worse than a
// short capture. // short capture.
h.trigger.setBuffered(0, false, now) h.trigger.setBuffered(0, 0, false, false, now)
return return
} }
_, span := rb.stats() _, span := rb.stats()
h.trigger.setBuffered(span, true, now) // Maximum span the ring can ever reach at its current bucket/capacity, in
// seconds. The gate must never demand more than this, or a ring whose
// coverage is below the window (a measured source rate that over-estimates
// the true one) can never satisfy it.
coverage := 0.0
if rate := rb.sourceRate(); rate > 0 {
coverage = float64(ringCoverage(rb.bucketSize(), rb.capacity())) / rate
}
// If the disk archive already spans the trigger window, the front of a
// short capture can be back-filled from it, so the ring need not cover the
// whole window on its own.
archived := h.hist.coversWindow(key, h.trigger.Config().windowSec)
h.trigger.setBuffered(span, coverage, archived, true, now)
} }
// triggerTick services the trigger FSM; called from Hub.Run() on every push tick. // triggerTick services the trigger FSM; called from Hub.Run() on every push tick.
@@ -640,7 +746,7 @@ func (h *Hub) triggerTick() {
// file of its own, where nothing overwrites it until the next trigger. // file of its own, where nothing overwrites it until the next trigger.
h.hist.captureRange(trigTime-pre, trigTime+post) h.hist.captureRange(trigTime-pre, trigTime+post)
} else if h.trigger.dueRearm(nowSec) { } else if h.trigger.dueRearm(nowSec) {
h.trigger.Arm() h.trigger.rearm()
} }
if h.trigger.stateUnsent() { if h.trigger.stateUnsent() {
Common/Client/go/wshub/trigger_sporadic_test.go
+367
View File
@@ -0,0 +1,367 @@
package wshub
import (
"encoding/binary"
"math"
"testing"
)
/*
Sporadic-signal trigger coverage.
Every other trigger test in this package feeds a periodic waveform, or a
hand-built two-sample batch. Neither can show a trigger that is blind most of
the time: a sine crosses the threshold again a few milliseconds after every
missed crossing, so a trigger losing 80 % of its edges still fires steadily and
looks healthy. A sparse train — 0000000111000000000000, one short burst in a
long flat run — has nothing to fall back on, so every missed edge is a missed
capture and the yield is a direct measure of how long the FSM was deaf.
That deafness is what these tests pin down. It is not a bug in itself: a capture
cannot be harvested before the samples after its trigger point exist, so the
trigger is necessarily blind for its own post-trigger window. What must NOT
happen is for edges arriving after that window to be thrown away as well.
*/
// pulseTrainSim drives a Hub the way Run() does — ingest on one side, the
// trigger tick on the other — on a simulated clock.
type pulseTrainSim struct {
rateHz float64
batchSec float64
pulsePeriod float64
pulseSamples int
simSec float64
windowSec float64
prePercent float64
holdoffSec float64
armAt float64
// When windowChangeAt > 0 the window is switched to windowChangeTo at that
// time and the trigger re-armed, as a user editing the trigger bar would.
windowChangeAt float64
windowChangeTo float64
}
type pulseTrainResult struct {
pulses int // pulse starts presented after the trigger was armed
shots int // captures actually delivered
trigTimes []float64 // the sample time each capture triggered on
worstCov float64 // smallest fraction of its window a capture spanned
holdDeclined int // captures the zoom hold would not answer for
drawnPulses int // pulses visible in the delivered frames
wantPulses int // pulses those frames' windows really contained
gatedPulses int // pulses that arrived armed but with the fill gate shut
}
// yield is the fraction of presented pulses that produced a capture.
func (r pulseTrainResult) yield() float64 {
if r.pulses == 0 {
return 0
}
return float64(r.shots) / float64(r.pulses)
}
// run executes the simulation and returns what the trigger caught.
func (s pulseTrainSim) run(t *testing.T, key string) pulseTrainResult {
t.Helper()
h := NewHub()
h.rings[key] = newSigRing(ringCapInitial)
h.trigger.SetConfig(trigConfig{
signalKey: key, edge: "rising", threshold: 0.5,
windowSec: s.windowSec, prePercent: s.prePercent,
mode: "normal", holdoffSec: s.holdoffSec,
})
res := pulseTrainResult{worstCov: 1}
nBatch := int(s.rateHz * s.batchSec)
ts := make([]float64, nBatch)
vs := make([]float64, nBatch)
armed, changed := false, false
for now := 0.0; now < s.simSec; now += s.batchSec {
nPulseStarts := 0
for i := range ts {
ts[i] = now + float64(i)/s.rateHz
// Position within the current pulse period, in samples.
k := int((ts[i] - math.Floor(ts[i]/s.pulsePeriod)*s.pulsePeriod) * s.rateHz)
if k < s.pulseSamples {
vs[i] = 1
if k == 0 {
nPulseStarts++
}
} else {
vs[i] = 0
}
}
if !armed && now >= s.armAt {
h.trigger.Arm()
armed = true
}
if s.windowChangeAt > 0 && !changed && now >= s.windowChangeAt {
cfg := h.trigger.Config()
cfg.windowSec = s.windowChangeTo
h.trigger.SetConfig(cfg)
h.trigger.Arm()
changed = true
}
if armed {
res.pulses += nPulseStarts
if nPulseStarts > 0 && h.trigger.State() == trigArmed {
h.trigger.mu.Lock()
f := h.trigger.fillLocked()
h.trigger.mu.Unlock()
if f < 1 {
res.gatedPulses += nPulseStarts
}
}
}
h.ingest(key, 1, ts, vs)
// Mirror triggerTick, on the simulated clock.
tick := now + s.batchSec
h.retuneRings(tick)
_, span := h.rings[key].stats()
h.trigger.setBuffered(span, 0, false, true, tick)
if trigTime, pre, post, ok := h.trigger.dueCapture(tick); ok {
if buf := h.buildTriggerCapture(trigTime, pre, post); buf != nil {
res.shots++
res.trigTimes = append(res.trigTimes, trigTime)
first, last, _ := decodeCaptureSpan(t, buf, key)
if cov := (last - first) / (pre + post); cov < res.worstCov {
res.worstCov = cov
}
if _, _, ok := h.capture.slice(key, trigTime-pre, trigTime+post); !ok {
res.holdDeclined++
}
// What the client would actually draw, against what the window
// really contained. A wide window holds several pulses, and a
// frame showing only the one it triggered on has lost the rest
// between the ring, the bucketing and the decimation.
_, fv := decodeCaptureSig(t, buf, key)
res.drawnPulses += countPulses(fv, 0.5)
res.wantPulses += countPulseStarts(trigTime-pre, trigTime+post,
s.pulsePeriod, 1/s.rateHz)
}
h.trigger.markTriggered(tick)
} else if h.trigger.dueRearm(tick) {
h.trigger.rearm()
}
}
return res
}
// decodeCaptureSig pulls one signal's samples out of a v2 capture frame.
func decodeCaptureSig(t *testing.T, buf []byte, key string) (ts, vs []float64) {
t.Helper()
off := 1 + 8 + 8 + 8
nSig := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
for i := 0; i < nSig; i++ {
kl := int(binary.LittleEndian.Uint16(buf[off:]))
off += 2
k := string(buf[off : off+kl])
off += kl
cnt := int(binary.LittleEndian.Uint32(buf[off:]))
off += 4
if k == key {
ts = make([]float64, cnt)
vs = make([]float64, cnt)
for j := 0; j < cnt; j++ {
ts[j] = math.Float64frombits(binary.LittleEndian.Uint64(buf[off+j*8:]))
vs[j] = math.Float64frombits(binary.LittleEndian.Uint64(buf[off+cnt*8+j*8:]))
}
}
off += cnt * 16
}
return
}
// countPulses counts runs of samples at or above thr.
func countPulses(v []float64, thr float64) int {
n, in := 0, false
for _, x := range v {
if x >= thr {
if !in {
n, in = n+1, true
}
} else {
in = false
}
}
return n
}
// countPulseStarts is how many pulse starts fall inside [t0, t1]. A pulse
// starting within one sample of t1 is not counted: only its first sample is
// inside the window, and the ring's min/max bucket for it may put that sample's
// extremum just past the edge, which is a boundary artefact rather than a loss.
func countPulseStarts(t0, t1, period, dt float64) int {
n := 0
for k := math.Floor(t0 / period); k*period <= t1; k++ {
if p := k * period; p >= t0 && p < t1-2*dt {
n++
}
}
return n
}
// deadTimeSec is how long the FSM is blind after firing at t: it must acquire
// the post-trigger window before the capture can be harvested, and the holdoff
// guards against re-triggering on the same event. Both are measured from the
// trigger point, so they overlap rather than add.
func deadTimeSec(windowSec, prePercent, holdoffSec float64) float64 {
return math.Max(windowSec*(1-prePercent/100), holdoffSec)
}
// A trigger cannot show two windows at once, so pulses closer together than its
// post-trigger window are necessarily lost. Pulses spaced FURTHER apart than
// that are not: nothing about the acquisition prevents catching every one.
//
// This is the reported failure. The FSM used to go deaf from the trigger point
// until the capture had been harvested (a post-window plus captureMarginSec)
// and the holdoff had then elapsed on top of that, then wait for a fresh edge —
// so the effective spacing was rounded UP to a whole pulse period. At the
// default 1 s window and 0.2 s holdoff the blind stretch came to 1.15 s, which
// is longer than a 1 s pulse period by a hair, and a pulse train at 1 Hz was
// caught at 0.5 Hz. Widening the window made it worse in whole multiples.
func TestSporadicPulsesWiderThanThePostWindowAreAllCaught(t *testing.T) {
const key = "s1:Ch1"
cases := []struct{ window, period float64 }{
{0.5, 0.5}, // post 0.4 s
{1.0, 1.0}, // post 0.8 s — the case the report was made against
{2.0, 2.0}, // post 1.6 s
{5.0, 5.0}, // post 4.0 s
}
for _, c := range cases {
sim := pulseTrainSim{
rateHz: 1000, batchSec: 1.0 / 30.0,
pulsePeriod: c.period, pulseSamples: 3,
simSec: 41 * c.period, windowSec: c.window, prePercent: 20,
holdoffSec: autoRearmDelaySec, armAt: c.period,
}
res := sim.run(t, key)
// Two pulses are always in flight rather than caught: the one that lands
// as the trigger arms, and the one still being collected when the run
// ends.
if got := res.yield(); got < 0.94 {
t.Errorf("window %.1f s, pulse every %.1f s: caught %d of %d (%.0f%%); "+
"the post-trigger window is only %.2f s, so every pulse fits",
c.window, c.period, res.shots, res.pulses, 100*got,
c.window*0.8)
}
}
}
// The loss that remains must be the loss that has to remain. A capture cannot
// start before the previous one's post-window is acquired, and it can only start
// on a pulse, so consecutive captures are a dead time apart rounded UP to the
// next pulse — never further. Any longer gap means an edge that the acquisition
// no longer needed was thrown away anyway.
//
// The bound is stated as dead + period rather than ceil(dead/period)*period
// because when the two divide exactly, whether the pulse at the boundary counts
// comes down to the last bit of the sample timestamp. Both answers are correct;
// a gap beyond either is not.
func TestSporadicCaptureGapsStayWithinTheDeadTime(t *testing.T) {
const key = "s1:Ch1"
for _, window := range []float64{0.5, 1.0, 2.0, 5.0} {
for _, period := range []float64{0.25, 0.5, 1.0, 2.0} {
sim := pulseTrainSim{
rateHz: 1000, batchSec: 1.0 / 30.0,
pulsePeriod: period, pulseSamples: 3,
simSec: 60, windowSec: window, prePercent: 20,
holdoffSec: autoRearmDelaySec, armAt: 1.0,
}
res := sim.run(t, key)
dead := deadTimeSec(window, 20, autoRearmDelaySec)
limit := dead + period + 2*sim.batchSec
worst, worstAt := 0.0, 0.0
for i := 1; i < len(res.trigTimes); i++ {
if g := res.trigTimes[i] - res.trigTimes[i-1]; g > worst {
worst, worstAt = g, res.trigTimes[i-1]
}
}
if worst > limit {
t.Errorf("window %.1f s, pulse every %.2f s: %.2f s between the captures "+
"at %.2f s and %.2f s; the dead time is only %.2f s, so %.2f s is the most "+
"that can be missed",
window, period, worst, worstAt, worstAt+worst, dead, limit)
}
t.Logf("window %.1f s, pulse every %.2f s: %d/%d captures (%.0f%%), "+
"dead time %.2f s, worst gap %.2f s, gated %d",
window, period, res.shots, res.pulses, 100*res.yield(), dead,
worst, res.gatedPulses)
}
}
}
// Whatever the trigger does catch has to come back whole: a window wide enough
// to hold several pulses must show all of them, at every rate, including the
// rates that force the ring into min/max bucketing.
func TestSporadicCaptureShowsEveryPulseInItsWindow(t *testing.T) {
const key = "s1:Ch1"
for _, rate := range []float64{1000, 200e3} {
for _, window := range []float64{1.0, 2.0, 5.0} {
sim := pulseTrainSim{
rateHz: rate, batchSec: 1.0 / 30.0,
pulsePeriod: 0.5, pulseSamples: 3,
simSec: 40, windowSec: window, prePercent: 20,
holdoffSec: autoRearmDelaySec, armAt: 1.0,
}
res := sim.run(t, key)
if res.shots == 0 {
t.Fatalf("rate %.0f window %.1f s: no captures at all", rate, window)
}
// wantPulses excludes the pulse straddling each window's far edge,
// whose bucket may place its extremum just past it, so the frames
// may legitimately draw a few more than that — but never fewer.
if res.drawnPulses < res.wantPulses {
t.Errorf("rate %.0f window %.1f s: frames drew %d pulses, their windows held %d",
rate, window, res.drawnPulses, res.wantPulses)
}
if res.worstCov < 0.98 {
t.Errorf("rate %.0f window %.1f s: worst capture spanned %.0f%% of its window",
rate, window, 100*res.worstCov)
}
if res.holdDeclined > 0 {
t.Errorf("rate %.0f window %.1f s: the zoom hold declined %d of %d captures",
rate, window, res.holdDeclined, res.shots)
}
}
}
}
// Widening the window mid-run is the gesture the report came from. The fill gate
// holds the first shot off until the ring reaches back far enough, which is
// correct; what it must not do is stay shut, nor leave the trigger losing pulses
// once the ring has caught up.
func TestSporadicYieldRecoversAfterAWindowChange(t *testing.T) {
const key = "s1:Ch1"
for _, w := range []float64{1.0, 2.0, 5.0} {
sim := pulseTrainSim{
rateHz: 200e3, batchSec: 1.0 / 30.0,
pulsePeriod: w, pulseSamples: 3,
simSec: 30 * w, windowSec: 0.2, prePercent: 20,
holdoffSec: autoRearmDelaySec, armAt: 1.0,
windowChangeAt: 10 * w, windowChangeTo: w,
}
res := sim.run(t, key)
// Count only what happened after the change settled.
after, want := 0, 0
for _, tt := range res.trigTimes {
if tt > 11*w {
after++
}
}
for p := 11 * w; p < 30*w; p += w {
want++
}
if float64(after) < 0.9*float64(want) {
t.Errorf("window 0.2 -> %.1f s: %d captures in the %d pulses after the change",
w, after, want)
}
}
}
Common/Client/go/wshub/trigger_test.go
+58 -6
View File
@@ -297,9 +297,9 @@ func TestCollectingIsBroadcast(t *testing.T) {
// later. It forgets any earlier measurement first, so the rate is the one // later. It forgets any earlier measurement first, so the rate is the one
// asked for rather than a blend with it. // asked for rather than a blend with it.
func setFill(te *triggerEngine, span, growth, now float64) { func setFill(te *triggerEngine, span, growth, now float64) {
te.setBuffered(0, false, now) te.setBuffered(0, 0, false, false, now)
te.setBuffered(span-growth, true, now) te.setBuffered(span-growth, 0, false, true, now)
te.setBuffered(span, true, now+1) te.setBuffered(span, 0, false, true, now+1)
} }
// What has to hold is that the buffer spans the whole window by the time the // What has to hold is that the buffer spans the whole window by the time the
@@ -417,9 +417,9 @@ func TestForceIgnoresFillGate(t *testing.T) {
// interval, so they refresh the span and leave the seeded rate alone. // interval, so they refresh the span and leave the seeded rate alone.
func seedFillNow(te *triggerEngine, span, growth float64) { func seedFillNow(te *triggerEngine, span, growth float64) {
now := float64(time.Now().UnixNano()) / 1e9 now := float64(time.Now().UnixNano()) / 1e9
te.setBuffered(0, false, now-1) te.setBuffered(0, 0, false, false, now-1)
te.setBuffered(span-growth, true, now-1) te.setBuffered(span-growth, 0, false, true, now-1)
te.setBuffered(span, true, now) te.setBuffered(span, 0, false, true, now)
} }
// While it holds off, the trigger looks identical to one that is ignoring // While it holds off, the trigger looks identical to one that is ignoring
@@ -505,3 +505,55 @@ func drainStates(t *testing.T, h *Hub) []map[string]any {
} }
} }
} }
// A ring whose coverage saturates below the window (measured source rate that
// over-estimates the true one) can never satisfy the full-window need. The
// coverage clamp must open the gate once the ring is full, delivering a short
// capture rather than staying deaf forever.
func TestFillNeedClampedToCoverage(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 60, prePercent: 20, mode: "normal", holdoffSec: 0.2})
setFill(te, 50, 0, 100) // ring full at 50 s, no growth
te.mu.Lock()
te.bufCoverage = 50 // the ring can never reach further back
te.mu.Unlock()
if need := te.fillNeedLocked(); need != 50 {
t.Errorf("need = %v, want 50 (clamped to coverage, not the 60 s window)", need)
}
if f := te.fillLocked(); f < 1 {
t.Errorf("fillLocked = %v, want >= 1: a full ring below the window must still open the gate", f)
}
// Without the clamp the gate would stay shut forever.
te.mu.Lock()
te.bufCoverage = 0
te.mu.Unlock()
if f := te.fillLocked(); f >= 1 {
t.Errorf("baseline: fillLocked = %v, want < 1 without a coverage clamp", f)
}
}
// When the disk archive already spans the window it can back-fill the front of
// a capture, so the gate must only require the ring to have reached the
// pre-window, not the whole window.
func TestFillNeedArchiveLowersToPreWindow(t *testing.T) {
te := newTriggerEngine()
te.SetConfig(trigConfig{signalKey: "s:x", windowSec: 60, prePercent: 20, mode: "normal", holdoffSec: 0.2})
setFill(te, 30, 0, 100) // ring holds only 30 s, no growth → need 60 without archive
te.mu.Lock()
te.bufArchived = true
te.mu.Unlock()
if want := 12.0; te.fillNeedLocked() != want { // 60 * 0.20
t.Errorf("need = %v, want %v (archive lowers to the pre-window)", te.fillNeedLocked(), want)
}
// A ring holding just the pre-window opens the gate once archived.
te.mu.Lock()
te.bufSpan = 12
te.mu.Unlock()
if f := te.fillLocked(); f < 1 {
t.Errorf("fillLocked = %v, want >= 1 with pre-window buffered and the archive available", f)
}
}
Common/UDP/UDPSProtocol.h
+24 -3
View File
@@ -23,15 +23,22 @@
* [uint32 numSigs] * [uint32 numSigs]
* numSigs × UDPSSignalDescriptor (136 bytes each, packed) * numSigs × UDPSSignalDescriptor (136 bytes each, packed)
* [uint8 publishMode] (PublishModeStrict / Accumulate / Decimate) * [uint8 publishMode] (PublishModeStrict / Accumulate / Decimate)
* [uint64 hrtFrequency] ticks per second of the producer's HRT
*
* Everything after the descriptors is an optional trailer: a receiver must
* accept a payload that stops early and must ignore bytes it does not know.
* publishMode defaults to Strict when absent, hrtFrequency to
* UDPS_HRT_FREQUENCY_UNKNOWN.
* *
* DATA payload (Strict / Decimate): * DATA payload (Strict / Decimate):
* [uint64 HRT timestamp] * [uint64 HRT timestamp]
* per-signal data in CONFIG order (quantised or raw, no padding) * per-signal data in CONFIG order (quantised or raw, no padding)
* *
* DATA payload (Accumulate): * DATA payload (Accumulate):
* [uint64 HRT timestamp] * [uint64 HRT timestamp of the first slot in the batch]
* [uint32 numSamples] * [uint32 numSamples] RT cycles accumulated into this packet
* for each signal: if scalar → numSamples elements; else → NumElements once * for each signal, in CONFIG order: numSamples × NumElements values
* (signal-major, one full snapshot per accumulated cycle)
*/ */
#ifndef UDPS_PROTOCOL_H_ #ifndef UDPS_PROTOCOL_H_
@@ -123,6 +130,20 @@ static const uint8 UDPS_PUBLISH_STRICT = 0u; ///< One packet per Synchronise
static const uint8 UDPS_PUBLISH_ACCUMULATE = 1u; ///< Variable batch; flush on size or time static const uint8 UDPS_PUBLISH_ACCUMULATE = 1u; ///< Variable batch; flush on size or time
static const uint8 UDPS_PUBLISH_DECIMATE = 2u; ///< One packet per Ratio calls static const uint8 UDPS_PUBLISH_DECIMATE = 2u; ///< One packet per Ratio calls
/*---------------------------------------------------------------------------*/
/* HRT frequency (CONFIG trailing uint64) */
/*---------------------------------------------------------------------------*/
/**
* Sentinel for a CONFIG that carries no HRT frequency, either because the
* trailer is absent (producer older than this field) or because the producer
* could not determine it. DATA timestamps are raw ticks of the producer's
* high-resolution timer, so without this a receiver on another host has no
* way to turn them into seconds and can only fall back to its own timer's
* frequency — which is right only while the two happen to agree.
*/
static const uint64 UDPS_HRT_FREQUENCY_UNKNOWN = 0u;
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
/* CONFIG payload — per-signal descriptor */ /* CONFIG payload — per-signal descriptor */
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
Docs/Protocol.md
+56 -1
View File
@@ -84,8 +84,28 @@ Offset Size Type Field
0xFFFFFFFF = PacketTime (no reference) 0xFFFFFFFF = PacketTime (no reference)
104 32 char[32] unit null-terminated physical unit string 104 32 char[32] unit null-terminated physical unit string
── (total per signal: 136 bytes) ──────────────────────────── ── (total per signal: 136 bytes) ────────────────────────────
── trailer, immediately after the last descriptor ───────────
0 1 uint8 publishMode 0 = Strict, 1 = Accumulate, 2 = Decimate
1 8 uint64 hrtFrequency producer's HRT ticks per second;
0 = unknown
``` ```
### CONFIG trailer
Everything after the descriptors is a trailer that grew field by field, so a
receiver must accept a payload that stops early and must ignore bytes it does
not recognise. An absent `publishMode` means Strict; an absent or zero
`hrtFrequency` means the producer did not publish its tick rate.
`hrtFrequency` is what makes DATA timestamps interpretable off-box. DATA
carries the raw value of the producer's high-resolution counter, and on x86
that counter runs at the TSC frequency — a different number on every model. A
receiver that divides by its own timer's frequency instead is right only while
producer and consumer sit on the same host; anywhere else every batch is laid
out over the wrong span of time. Fall back to the local frequency only when the
field is missing, and reject implausible values (nothing below 1 kHz is a
high-resolution timer).
### Type Codes ### Type Codes
| Code | C type | Bytes/element | | Code | C type | Bytes/element |
@@ -129,7 +149,9 @@ After reassembly, the DATA payload layout is:
``` ```
Offset Size Type Field Offset Size Type Field
────── ──── ────── ──────────────────────────────────────────────────── ────── ──── ────── ────────────────────────────────────────────────────
0 8 uint64 hrtTimestamp hardware reference timer count at Synchronise() 0 8 uint64 hrtTimestamp producer's high-resolution counter at
Synchronise(); divide by the CONFIG
hrtFrequency to get seconds
── for each signal (in config order) ──────────────────────────────────── ── for each signal (in config order) ────────────────────────────────────
varies N×sz — signal data N = numRows×numCols, sz = element size varies N×sz — signal data N = numRows×numCols, sz = element size
(wire size if quantized, raw size otherwise) (wire size if quantized, raw size otherwise)
@@ -171,6 +193,39 @@ the client to reassemble them in any order.
--- ---
## Ordering DATA (required of every receiver)
DATA carries its own `counter` sequence, incremented once per sent packet
(CONFIG is numbered independently). Reassembly completes in arrival order, not
counter order, so a packet reordered or duplicated on the wire surfaces after a
newer one has already been consumed. Its values are well-formed but carry an
older time base: accepting it writes them over samples the consumer already
holds and leaves the span they should have filled empty — a collision on one
side and a hole on the other.
A receiver must therefore drop any DATA packet that does not advance the
counter, and must order it by the *signed* difference:
```c
int32_t delta = (int32_t)(counter - lastCounter); /* survives the uint32 wrap */
if (delta <= 0) { /* stale or duplicate: drop */ }
lost = (uint32_t)delta - 1u; /* packets missing before this one */
```
Comparing the values directly would call the first packet after the wrap stale
and reject the stream from then on.
`lost` matters beyond diagnostics. A consumer that spaces batched samples from
the elapsed time since the previous packet must divide that gap by `lost + 1`
batches; dividing by one batch reports a period too long by exactly that factor
and walks the samples past their own end into the next packet's range. Reset
the sequence on (re)connect: the producer's counter restarts independently.
Implemented in `UDPSClient::AcceptDataCounter` (C++),
`udpsprotocol.SequenceGate` (Go) and `decode_data` (C).
---
## Minimal Python Client Example ## Minimal Python Client Example
```python ```python
Docs/StreamHub-Developer.md
+21 -2
View File
@@ -107,12 +107,31 @@ Hub-side, web-client semantics (`setTrigger` fields in
``` ```
IDLE --arm--> ARMED --edge crossing--> COLLECTING --every source past trigTime+postSec+0.15s--> TRIGGERED IDLE --arm--> ARMED --edge crossing--> COLLECTING --every source past trigTime+postSec+0.15s--> TRIGGERED
TRIGGERED --rearm (single) / auto ~200ms (normal, unless stopped)--> ARMED TRIGGERED --rearm (single) / auto after holdoffSec (normal, unless stopped)--> ARMED
└─ or straight to COLLECTING on a held edge
any --disarm--> IDLE any --disarm--> IDLE
``` ```
`UDPSourceSession` calls `TriggerEngine::CheckSample` for every decoded sample `UDPSourceSession` calls `TriggerEngine::CheckSample` for every decoded sample
of the configured signal (signal index cached per config epoch). Each source is of the configured signal (signal index cached per config epoch).
The comparator keeps running through COLLECTING and TRIGGERED. It cannot fire
there — the capture in flight owns that stretch — but it remembers the first
edge at or past `trigTime + max(postSec, holdoffSec)`, and `Rearm()` fires on
that remembered edge instead of waiting for a fresh one. Without this the engine
is deaf from its own trigger point until the capture has been harvested and the
holdoff has run, which on a sparse pulse train rounds the capture spacing up to
a whole pulse period: at a 1 s window a 1 Hz train was caught at 0.5 Hz, and a
wider window lost whole multiples. The capture is built from the edge's own
timestamp out of rings that still hold everything around it, so honouring it
costs nothing.
`Arm()` and `Rearm()` differ only in this: `Arm()` is the operator's own arm and
discards the held edge (they asked for the next event), while `Rearm()` is the
automatic end-of-capture arm and consumes it. `Rearm()` also keeps the tracked
level, so the first sample after it is compared against its real predecessor
rather than being spent seeding one. `SetConfig()` and `Disarm()` drop the held
edge as well — it was never judged against the new window. Each source is
read `[trigTime−preSec, trigTime+postSec]`, LTTB-capped to 20 000 pts/signal and read `[trigTime−preSec, trigTime+postSec]`, LTTB-capped to 20 000 pts/signal and
appended to a binary **version 2** capture frame; every FSM transition appended to a binary **version 2** capture frame; every FSM transition
broadcasts a `triggerState` event. broadcasts a `triggerState` event.
Docs/UDPS-C-Client.md
+9
View File
@@ -175,6 +175,7 @@ replayed traffic can be decoded without a client.
```c ```c
typedef struct { typedef struct {
uint32_t counter; /* gaps in this sequence are lost datagrams */ uint32_t counter; /* gaps in this sequence are lost datagrams */
uint32_t lost; /* DATA packets missing immediately before this one */
uint64_t hrt; /* producer's high-resolution timer at send */ uint64_t hrt; /* producer's high-resolution timer at send */
double recv_time; /* CLOCK_REALTIME seconds at arrival */ double recv_time; /* CLOCK_REALTIME seconds at arrival */
uint8_t publish_mode; uint8_t publish_mode;
@@ -194,6 +195,13 @@ scalar signal in Accumulate mode, where the producer batches several RT cycles i
and `count == num_samples` — one value per cycle. Arrays are not batched: they appear once and and `count == num_samples` — one value per cycle. Arrays are not batched: they appear once and
apply to the whole packet. `udps_frame_value(f, sig, sample, elem)` applies that rule for you. apply to the whole packet. `udps_frame_value(f, sig, sample, elem)` applies that rule for you.
**Ordering.** Frames reach `on_data` in counter order: a DATA packet that does not advance the
counter — reordered or duplicated on the wire — is dropped rather than delivered, because its
values carry a time base older than data you already have, and placing them would overwrite live
samples while leaving their own span empty. `lost` reports how many packets went missing just
before the frame. If you space samples yourself from the elapsed time since the previous frame,
divide by `lost + 1` batches, not one: the gap covers the missing packets' cycles too.
**Timestamps.** The protocol does not put a timestamp on every element; how to date them depends **Timestamps.** The protocol does not put a timestamp on every element; how to date them depends
on the signal's `time_mode` (see [Protocol.md](Protocol.md#time-mode-codes)): on the signal's `time_mode` (see [Protocol.md](Protocol.md#time-mode-codes)):
@@ -223,6 +231,7 @@ udps_client_stats(cli, &s);
| `frames_delivered` | DATA packets decoded and passed to `on_data`. | | `frames_delivered` | DATA packets decoded and passed to `on_data`. |
| `config_updates` | CONFIG packets applied. | | `config_updates` | CONFIG packets applied. |
| `counter_gaps` | Missing packet counters — datagrams lost on the wire or in the kernel. | | `counter_gaps` | Missing packet counters — datagrams lost on the wire or in the kernel. |
| `stale_packets` | DATA packets dropped for not advancing the counter: reordered or duplicated on the wire. |
| `fragments_dropped` | Duplicate, stale or unplaceable fragments; a non-zero value with `counter_gaps` means fragmented updates are arriving incomplete. | | `fragments_dropped` | Duplicate, stale or unplaceable fragments; a non-zero value with `counter_gaps` means fragmented updates are arriving incomplete. |
| `reconnects` | Sessions re-established after a silence timeout. | | `reconnects` | Sessions re-established after a silence timeout. |
Source/Applications/StreamHub/StreamHub.cpp
+5 -1
View File
@@ -1736,7 +1736,11 @@ void StreamHub::TriggerTick(float64 wallNowS) {
(wallNowS >= rearmAtWallS_)) { (wallNowS >= rearmAtWallS_)) {
rearmPending_ = false; rearmPending_ = false;
if (!trigger_.GetStopped()) { if (!trigger_.GetStopped()) {
trigger_.Arm(); /* Rearm, not Arm: an edge that arrived while this capture was being
* collected is fired on at once instead of being thrown away, which
* is what kept sparse pulse trains from being caught at their own
* rate. */
trigger_.Rearm();
} }
} }
Source/Applications/StreamHub/TriggerEngine.cpp
+64 -12
View File
@@ -19,7 +19,17 @@ TriggerEngine::TriggerEngine()
trigTime_(0.0), trigTime_(0.0),
firedPreSec_(0.0), firedPreSec_(0.0),
firedPostSec_(0.0), firedPostSec_(0.0),
firedValid_(false) { firedValid_(false),
pendingTime_(0.0),
pendingValid_(false) {
}
void TriggerEngine::LatchWindowLocked(float64 t) {
state_ = kTrigCollecting;
trigTime_ = t;
firedPreSec_ = config_.windowSec * config_.prePercent / 100.0;
firedPostSec_ = config_.windowSec - firedPreSec_;
firedValid_ = true;
} }
void TriggerEngine::SetConfig(const TriggerConfig &cfg) { void TriggerEngine::SetConfig(const TriggerConfig &cfg) {
@@ -40,6 +50,9 @@ void TriggerEngine::SetConfig(const TriggerConfig &cfg) {
epoch_++; epoch_++;
prevValid_ = false; prevValid_ = false;
prevValue_ = 0.0; prevValue_ = 0.0;
/* An edge held over from the old configuration would be latched against the
* new window, which it was never judged against. */
pendingValid_ = false;
mutex_.FastUnLock(); mutex_.FastUnLock();
} }
@@ -62,6 +75,26 @@ void TriggerEngine::Arm() {
state_ = kTrigArmed; state_ = kTrigArmed;
prevValid_ = false; prevValid_ = false;
prevValue_ = 0.0; prevValue_ = 0.0;
pendingValid_ = false;
mutex_.FastUnLock();
}
void TriggerEngine::Rearm() {
(void) mutex_.FastLock();
if (pendingValid_) {
const float64 t = pendingTime_;
pendingValid_ = false;
LatchWindowLocked(t);
REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information,
"TriggerEngine: rearmed onto the edge held at t=%.6f "
"(pre=%.4fs post=%.4fs)",
t, firedPreSec_, firedPostSec_);
}
else {
/* prevValue_/prevValid_ are deliberately kept: the comparator ran right
* through the dead time, so the next sample has a real predecessor. */
state_ = kTrigArmed;
}
mutex_.FastUnLock(); mutex_.FastUnLock();
} }
@@ -72,6 +105,7 @@ void TriggerEngine::Disarm() {
prevValid_ = false; prevValid_ = false;
prevValue_ = 0.0; prevValue_ = 0.0;
firedValid_ = false; firedValid_ = false;
pendingValid_ = false;
mutex_.FastUnLock(); mutex_.FastUnLock();
} }
@@ -96,7 +130,10 @@ void TriggerEngine::CheckSample(float64 t, float64 v) {
lastTime_ = t; lastTime_ = t;
lastTimeValid_ = true; lastTimeValid_ = true;
if (state_ != kTrigArmed) { /* A capture in flight does not stop the comparator; it only changes what an
* edge does. See pendingTime_. */
const bool inFlight = (state_ == kTrigCollecting) || (state_ == kTrigTriggered);
if ((state_ != kTrigArmed) && !inFlight) {
mutex_.FastUnLock(); mutex_.FastUnLock();
return; return;
} }
@@ -121,17 +158,36 @@ void TriggerEngine::CheckSample(float64 t, float64 v) {
} }
if (fired) { if (fired) {
state_ = kTrigCollecting; if (!inFlight) {
trigTime_ = t;
/* Latch the window at fire time so later config edits do not /* Latch the window at fire time so later config edits do not
* affect this capture (web client snap._preS/_postS). */ * affect this capture (web client snap._preS/_postS). */
firedPreSec_ = config_.windowSec * config_.prePercent / 100.0; LatchWindowLocked(t);
firedPostSec_ = config_.windowSec - firedPreSec_;
firedValid_ = true;
REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information, REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information,
"TriggerEngine: fired at t=%.6f (pre=%.4fs post=%.4fs)", "TriggerEngine: fired at t=%.6f (pre=%.4fs post=%.4fs)",
t, firedPreSec_, firedPostSec_); t, firedPreSec_, firedPostSec_);
} }
else if (!pendingValid_ && firedValid_) {
/* The earliest trigger point a new capture may take. The one in
* flight owns everything up to the end of its own post-window, and
* the holdoff — a guard against re-triggering on the ringing of the
* SAME event — is measured from its trigger point too, so the two
* overlap rather than add.
*
* Keep only the FIRST qualifying edge: a later one would deliver
* the same capture a pulse further on and skip the one between. */
float64 guard = firedPostSec_;
if (config_.holdoffSec > guard) {
guard = config_.holdoffSec;
}
if (t >= (trigTime_ + guard)) {
pendingTime_ = t;
pendingValid_ = true;
}
}
else {
/* Already holding an edge, or no window latched to measure against. */
}
}
mutex_.FastUnLock(); mutex_.FastUnLock();
} }
@@ -141,11 +197,7 @@ bool TriggerEngine::Force() {
bool ok = lastTimeValid_ && (state_ != kTrigCollecting); bool ok = lastTimeValid_ && (state_ != kTrigCollecting);
if (ok) { if (ok) {
state_ = kTrigCollecting; LatchWindowLocked(lastTime_);
trigTime_ = lastTime_;
firedPreSec_ = config_.windowSec * config_.prePercent / 100.0;
firedPostSec_ = config_.windowSec - firedPreSec_;
firedValid_ = true;
REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information, REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information,
"TriggerEngine: forced at t=%.6f (pre=%.4fs post=%.4fs)", "TriggerEngine: forced at t=%.6f (pre=%.4fs post=%.4fs)",
trigTime_, firedPreSec_, firedPostSec_); trigTime_, firedPreSec_, firedPostSec_);
Source/Applications/StreamHub/TriggerEngine.h
+35 -3
View File
@@ -88,9 +88,24 @@ public:
*/ */
uint32 GetConfigEpoch() const; uint32 GetConfigEpoch() const;
/** @brief Arm: any state → ARMED (resets edge detection). */ /**
* @brief Arm: any state → ARMED (resets edge detection).
* This is the user's own arm, so it discards any edge remembered during the
* previous capture: the user asked for the next event, not for one that has
* already been and gone.
*/
void Arm(); void Arm();
/**
* @brief The automatic arm at the end of a capture (normal mode).
* Unlike Arm() it honours an edge seen while the capture was being
* collected, firing on it at once rather than waiting for the next one, and
* it keeps the tracked level so the first sample afterwards is compared
* against its real predecessor. TRIGGERED → COLLECTING when an edge was
* remembered, otherwise → ARMED.
*/
void Rearm();
/** @brief Disarm: any state → IDLE; clears the stopped flag. */ /** @brief Disarm: any state → IDLE; clears the stopped flag. */
void Disarm(); void Disarm();
@@ -102,8 +117,10 @@ public:
/** /**
* @brief Edge-detect one decoded sample of the configured signal. * @brief Edge-detect one decoded sample of the configured signal.
* Receive-thread context. Only acts in ARMED state; on a matching edge * Receive-thread context. In ARMED state a matching edge latches trigTime
* latches trigTime and the pre/post window and moves to COLLECTING. * and the pre/post window and moves to COLLECTING. While a capture is in
* flight (COLLECTING/TRIGGERED) the comparator keeps running and the first
* edge clear of that capture is remembered for the next Rearm().
*/ */
void CheckSample(float64 t, float64 v); void CheckSample(float64 t, float64 v);
@@ -143,6 +160,21 @@ private:
float64 firedPreSec_; ///< Window pre-part latched at fire time float64 firedPreSec_; ///< Window pre-part latched at fire time
float64 firedPostSec_; ///< Window post-part latched at fire time float64 firedPostSec_; ///< Window post-part latched at fire time
bool firedValid_; ///< true after a fire, until Disarm() bool firedValid_; ///< true after a fire, until Disarm()
/**
* The edge to fire on as soon as the FSM rearms, in sample time, recorded
* while a capture is still being collected or handed out. Without it the
* trigger is deaf from its own trigger point until the capture has been
* harvested and the holdoff has run, and then waits for a fresh edge, which
* on a sparse pulse train rounds the capture spacing up to a whole pulse
* period. Remembering the edge instead makes the blind stretch exactly the
* guard interval it has to be, since the capture is built from the edge's
* own timestamp and the rings still hold everything around it.
*/
float64 pendingTime_;
bool pendingValid_;
/** @brief Freeze the pre/post split at fire time; caller holds the mutex. */
void LatchWindowLocked(float64 t);
}; };
inline TriggerConfig::TriggerConfig() inline TriggerConfig::TriggerConfig()
Source/Applications/StreamHub/UDPSourceSession.cpp
+29 -12
View File
@@ -99,6 +99,8 @@ void UDPSourceSession::ResetCalibration() {
lastPktWallValid_[i] = false; lastPktWallValid_[i] = false;
lastPktWallS_[i] = 0.0; lastPktWallS_[i] = 0.0;
accScalarPrevN_[i] = 0u; accScalarPrevN_[i] = 0u;
accScalarDtValid_[i] = false;
accScalarDtEMA_[i] = 0.0;
} }
} }
@@ -197,7 +199,8 @@ void UDPSourceSession::OnUDPSConfig(const uint8 *payload, uint32 payloadSize) {
} }
void UDPSourceSession::OnUDPSData(const uint8 *payload, uint32 payloadSize) { void UDPSourceSession::OnUDPSData(const uint8 *payload, uint32 payloadSize) {
ParseDataPayload(payload, payloadSize); /* Valid only for the duration of this callback. */
ParseDataPayload(payload, payloadSize, client_.GetLastDataGap());
} }
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
@@ -225,6 +228,9 @@ void UDPSourceSession::ParseConfigPayload(const uint8 *payload, uint32 size) {
sigDescs_[i].unit[sizeof(sigDescs_[i].unit) - 1u] = '\0'; sigDescs_[i].unit[sizeof(sigDescs_[i].unit) - 1u] = '\0';
} }
publishMode_ = payload[4u + numSigs * UDPS_SIGNAL_DESC_SIZE]; publishMode_ = payload[4u + numSigs * UDPS_SIGNAL_DESC_SIZE];
hrtFreq_ = UDPSConfigHrtFrequency(
payload, size, numSigs,
static_cast<float64>(MARTe::HighResolutionTimer::Frequency()));
numSignals_ = numSigs; numSignals_ = numSigs;
configured_ = true; configured_ = true;
@@ -273,8 +279,9 @@ void UDPSourceSession::ParseConfigPayload(const uint8 *payload, uint32 size) {
} }
REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information, REPORT_ERROR_STATIC(MARTe::ErrorManagement::Information,
"UDPSourceSession[%s]: CONFIG received — %u signals.", "UDPSourceSession[%s]: CONFIG received — %u signals, "
id_.Buffer(), numSigs); "producer HRT %.0f Hz.",
id_.Buffer(), numSigs, hrtFreq_);
} }
void UDPSourceSession::AllocateRingBuffers() { void UDPSourceSession::AllocateRingBuffers() {
@@ -376,7 +383,8 @@ float64 UDPSourceSession::ProducerNewestTime() const {
/* DATA parsing */ /* DATA parsing */
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size) { void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size,
uint32 lostPackets) {
if (size < 8u) { return; } if (size < 8u) { return; }
/* Copy metadata under lock */ /* Copy metadata under lock */
@@ -568,9 +576,9 @@ void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size) {
* immune to this because it is sampled at acquisition. * immune to this because it is sampled at acquisition.
* *
* hrtTimestamp is the HRT counter of sample 0; hrtFreq_ (the * hrtTimestamp is the HRT counter of sample 0; hrtFreq_ (the
* local HRT frequency, identical to the sender on the same * producer's tick rate, taken from the CONFIG trailer) converts
* host) converts it to seconds, then a one-time calibration * it to seconds, then a one-time calibration maps the sender
* maps the sender clock onto wall-clock. */ * clock onto wall-clock. */
const float64 hrt0Sec = static_cast<float64>(hrtTimestamp) / const float64 hrt0Sec = static_cast<float64>(hrtTimestamp) /
hrtFreq_; hrtFreq_;
if ((!timeSigCalibValid_[s]) || if ((!timeSigCalibValid_[s]) ||
@@ -581,16 +589,25 @@ void UDPSourceSession::ParseDataPayload(const uint8 *payload, uint32 size) {
} }
/* Per-sample dt: samplingRate if present, else derive it from /* Per-sample dt: samplingRate if present, else derive it from
* the sender-HRT gap to the previous packet divided by that * the sender-HRT gap to the previous packet.
* packet's sample count (the flushes carry contiguous RT *
* cycles, so this is exactly one cycle period). */ * The gap is divided by the number of RT cycles it actually
* spans, not by the previous packet's sample count. Those two
* agree only while nothing is lost; once a packet goes missing
* the gap covers cycles the previous count never saw, and
* dividing by that count inflates dt until this packet's
* samples overrun into the next packet's range. lostPackets
* comes from the producer's packet counter, so the divisor
* widens with the gap and dt is unchanged. */
float64 dt; float64 dt;
if (desc.samplingRate > 0.0) { if (desc.samplingRate > 0.0) {
dt = 1.0 / desc.samplingRate; dt = 1.0 / desc.samplingRate;
} else if (lastPktWallValid_[s] && (accScalarPrevN_[s] > 0u) && } else if (lastPktWallValid_[s] && (accScalarPrevN_[s] > 0u) &&
(hrt0Sec > lastPktWallS_[s])) { (hrt0Sec > lastPktWallS_[s])) {
dt = (hrt0Sec - lastPktWallS_[s]) / dt = UDPSEstimateAccumDt(hrt0Sec - lastPktWallS_[s],
static_cast<float64>(accScalarPrevN_[s]); accScalarPrevN_[s], lostPackets,
accScalarDtEMA_[s],
accScalarDtValid_[s]);
} else { } else {
dt = 1.0e-3; /* 1 kHz default until the gap is known */ dt = 1.0e-3; /* 1 kHz default until the gap is known */
} }
Source/Applications/StreamHub/UDPSourceSession.h
+116 -5
View File
@@ -39,6 +39,104 @@ using MARTe::ConfigurationDatabase;
/** Maximum number of signals per source session. */ /** Maximum number of signals per source session. */
static const uint32 UDPSS_MAX_SIGNALS = 256u; static const uint32 UDPSS_MAX_SIGNALS = 256u;
/* Accumulated-scalar dt estimator tuning. */
/** Weight of a new observation; slow enough that one bad gap barely moves it. */
static const float64 UDPSS_DT_EMA_ALPHA = 0.05;
/** Observations outside [lo, hi] x the current estimate are treated as a
* mis-counted gap and discarded rather than smoothed in. */
static const float64 UDPSS_DT_ACCEPT_LO = 0.5;
static const float64 UDPSS_DT_ACCEPT_HI = 2.0;
/**
* @brief Per-sample period of an accumulated scalar packet, robust to loss.
*
* An Accumulate producer batches consecutive RT cycles, so the sender-clock
* gap between two packets' first samples covers exactly as many cycles as the
* earlier packet carried — but only while nothing is lost in between. Over UDP
* (and with a producer that can overwrite a batch the sender never took) that
* assumption fails, and dividing the gap by the previous packet's sample count
* then inflates the period. The packet's own samples are laid out as
* base + e*dt, so an inflated dt walks them past their real end and into the
* span the next packet will claim: samples collide there and leave a hole
* behind them.
*
* The number of packets that went missing is not guessed from the gap — that
* is circular, and an estimator that infers the cycle count from its own
* period has a stable fixed point wherever gap/dt is an integer, so a genuine
* rate change locks it at the old period forever. It comes instead from the
* UDPS packet counter, which the producer increments once per sent packet. The
* gap then spans (1 + lost) batches, each assumed to be prevN cycles, and with
* nothing lost the formula reduces exactly to gap/prevN.
*
* @param gap Sender-clock seconds since the previous packet's first sample.
* Must be > 0.
* @param prevN Samples in the previous packet. Must be > 0.
* @param lost Packets missing between the previous packet and this one,
* from the producer's counter.
* @param[in,out] dtEMA Smoothed period. Seeded on the first call.
* @param[in,out] dtValid False until dtEMA holds an estimate.
* @return The period to space this packet's samples by.
*/
inline float64 UDPSEstimateAccumDt(const float64 gap, const uint32 prevN,
const uint32 lost, float64 &dtEMA,
bool &dtValid) {
float64 cycles = static_cast<float64>(prevN) *
(1.0 + static_cast<float64>(lost));
if (cycles < 1.0) {
cycles = 1.0;
}
const float64 dtObs = gap / cycles;
if (!dtValid) {
dtEMA = dtObs;
dtValid = true;
} else if ((dtObs > (dtEMA * UDPSS_DT_ACCEPT_LO)) &&
(dtObs < (dtEMA * UDPSS_DT_ACCEPT_HI))) {
/* Track slow drift, but ignore observations far outside the current
* estimate: those are the signature of a mis-counted gap, and folding
* one in would drag the estimate towards the very error it exists to
* absorb. */
dtEMA = ((1.0 - UDPSS_DT_EMA_ALPHA) * dtEMA) +
(UDPSS_DT_EMA_ALPHA * dtObs);
}
return dtEMA;
}
/**
* @brief Pick the tick rate to divide a producer's DATA timestamps by.
*
* DATA packets carry the raw value of the producer's high-resolution counter,
* which is meaningless without the rate it runs at. The rate is published in
* the CONFIG trailer, after the descriptors and the publish-mode byte. When it
* is missing — an older producer, or one that could not determine it — the
* only remaining option is this host's own timer, which is right only while
* the two machines agree; on x86 that is the TSC frequency, so it is a
* different number on every model.
*
* @param payload Reassembled CONFIG payload.
* @param size Bytes in @p payload.
* @param numSigs Signal count already read from the payload, capped to what
* the receiver will store.
* @param localFreq This host's HRT frequency, used as the fallback.
* @return Ticks per second to convert DATA timestamps with; never 0.
*/
inline float64 UDPSConfigHrtFrequency(const uint8 *payload, const uint32 size,
const uint32 numSigs,
const float64 localFreq) {
const uint32 offset = 4u + (numSigs * MARTe::UDPS_SIGNAL_DESC_SIZE) + 1u;
if ((payload != NULL_PTR(const uint8 *)) && (size >= (offset + 8u))) {
uint64 wireFreq = 0u;
memcpy(&wireFreq, payload + offset, 8u);
/* Anything below 1 kHz is not a high-resolution timer; the field is
* either absent, unset, or the payload was mis-parsed, and adopting it
* would stretch every timestamp far enough to make the trace useless. */
if (wireFreq >= 1000u) {
return static_cast<float64>(wireFreq);
}
}
return localFreq;
}
/** /**
* @brief One connected UDPStreamer source. * @brief One connected UDPStreamer source.
* *
@@ -229,7 +327,13 @@ private:
/* DATA payload parsing */ /* DATA payload parsing */
void ParseConfigPayload(const uint8 *payload, uint32 size); void ParseConfigPayload(const uint8 *payload, uint32 size);
void ParseDataPayload(const uint8 *payload, uint32 size); /**
* @param lostPackets DATA packets missing immediately before this one, from
* the producer's counter; the accumulated-scalar period estimate
* needs it to know how many cycles the sender-clock gap spans.
*/
void ParseDataPayload(const uint8 *payload, uint32 size,
uint32 lostPackets);
void AllocateRingBuffers(); void AllocateRingBuffers();
/** @brief Invalidate all wall-clock calibration state (receive thread only). */ /** @brief Invalidate all wall-clock calibration state (receive thread only). */
@@ -394,13 +498,20 @@ private:
float64 lastPktWallS_[UDPSS_MAX_SIGNALS]; float64 lastPktWallS_[UDPSS_MAX_SIGNALS];
bool lastPktWallValid_[UDPSS_MAX_SIGNALS]; bool lastPktWallValid_[UDPSS_MAX_SIGNALS];
/* Accumulated-scalar timing: HRT counter frequency (local == sender on the /* Accumulated-scalar timing: the producer's HRT counter frequency and the
* same host) and the previous packet's sample count, used to reconstruct * previous packet's sample count, used to reconstruct per-sample timestamps
* per-sample timestamps from the embedded sender HRT instead of the (UDP * from the embedded sender HRT instead of the (UDP burst-sensitive) packet
* burst-sensitive) packet arrival time. */ * arrival time. Seeded from this host's timer and replaced by the rate the
* producer publishes in CONFIG; see UDPSConfigHrtFrequency. */
float64 hrtFreq_; float64 hrtFreq_;
uint32 accScalarPrevN_[UDPSS_MAX_SIGNALS]; uint32 accScalarPrevN_[UDPSS_MAX_SIGNALS];
/* Per-signal state of UDPSEstimateAccumDt (see above): the smoothed
* per-sample period for accumulated scalars whose descriptor carries no
* SamplingRate. */
float64 accScalarDtEMA_[UDPSS_MAX_SIGNALS];
bool accScalarDtValid_[UDPSS_MAX_SIGNALS];
/* Scratch buffers for decoding arrays (receive thread only). */ /* Scratch buffers for decoding arrays (receive thread only). */
float64 *timeScratch_; ///< Time values scratch float64 *timeScratch_; ///< Time values scratch
float64 *valScratch_; ///< Data values scratch float64 *valScratch_; ///< Data values scratch
Source/Components/DataSources/UDPStreamer/UDPStreamer.cpp
+90 -16
View File
@@ -107,6 +107,9 @@ UDPStreamer::UDPStreamer()
readyTimestamps = NULL_PTR(uint64 *); readyTimestamps = NULL_PTR(uint64 *);
scratchTimestamps = NULL_PTR(uint64 *); scratchTimestamps = NULL_PTR(uint64 *);
readyFill = 0u; readyFill = 0u;
readySnapshotPending = false;
droppedPublications = 0u;
lastDropReportTicks = 0u;
decimateRatio = 1u; decimateRatio = 1u;
decimateCounter = 0u; decimateCounter = 0u;
@@ -807,7 +810,9 @@ bool UDPStreamer::PrepareNextState(const char8 *const currentStateName,
* receives it immediately. The config is static for the lifetime of this * receives it immediately. The config is static for the lifetime of this
* state. */ * state. */
if (ok) { if (ok) {
uint32 configBufSize = 4u + (numSigs * UDPS_SIGNAL_DESC_SIZE) + 32u + 1u; /* numSigs + descriptors + publishMode + hrtFrequency (+ slack). */
uint32 configBufSize =
4u + (numSigs * UDPS_SIGNAL_DESC_SIZE) + 1u + 8u + 32u;
HeapI *heap = GlobalObjectsDatabase::Instance()->GetStandardHeap(); HeapI *heap = GlobalObjectsDatabase::Instance()->GetStandardHeap();
uint8 *cfgBuf = reinterpret_cast<uint8 *>(heap->Malloc(configBufSize)); uint8 *cfgBuf = reinterpret_cast<uint8 *>(heap->Malloc(configBufSize));
if (cfgBuf != NULL_PTR(uint8 *)) { if (cfgBuf != NULL_PTR(uint8 *)) {
@@ -871,6 +876,11 @@ bool UDPStreamer::Synchronise() {
/* HI-3: if accumFill reached maxBatchCount, force-flush before writing */ /* HI-3: if accumFill reached maxBatchCount, force-flush before writing */
if (accumFill >= maxBatchCount) { if (accumFill >= maxBatchCount) {
uint32 filled = accumFill; uint32 filled = accumFill;
if (readyFill > 0u) {
/* The sender has not taken the previous batch: it is about to be
* overwritten and its cycles will never reach any receiver. */
droppedPublications++;
}
(void)MemoryOperationsHelper::Copy(readyBuffer, accumBuffer, (void)MemoryOperationsHelper::Copy(readyBuffer, accumBuffer,
filled * totalSrcBytes); filled * totalSrcBytes);
(void)MemoryOperationsHelper::Copy( (void)MemoryOperationsHelper::Copy(
@@ -901,6 +911,9 @@ bool UDPStreamer::Synchronise() {
if (sizeCondition || timeCondition) { if (sizeCondition || timeCondition) {
bufMutex.FastLock(TTInfiniteWait); bufMutex.FastLock(TTInfiniteWait);
if (readyFill > 0u) {
droppedPublications++;
}
(void)MemoryOperationsHelper::Copy(readyBuffer, accumBuffer, (void)MemoryOperationsHelper::Copy(readyBuffer, accumBuffer,
filled * totalSrcBytes); filled * totalSrcBytes);
(void)MemoryOperationsHelper::Copy( (void)MemoryOperationsHelper::Copy(
@@ -922,16 +935,24 @@ bool UDPStreamer::Synchronise() {
if (decimateCounter >= decimateRatio) { if (decimateCounter >= decimateRatio) {
decimateCounter = 0u; decimateCounter = 0u;
bufMutex.FastLock(TTInfiniteWait); bufMutex.FastLock(TTInfiniteWait);
if (readySnapshotPending) {
droppedPublications++;
}
(void)MemoryOperationsHelper::Copy(readyBuffer, memory, totalSrcBytes); (void)MemoryOperationsHelper::Copy(readyBuffer, memory, totalSrcBytes);
syncTimestamp = ts; syncTimestamp = ts;
readySnapshotPending = true;
bufMutex.FastUnLock(); bufMutex.FastUnLock();
(void)dataSem.Post(); (void)dataSem.Post();
} }
} else { } else {
/* --- Strict path: post every call --- */ /* --- Strict path: post every call --- */
bufMutex.FastLock(TTInfiniteWait); bufMutex.FastLock(TTInfiniteWait);
if (readySnapshotPending) {
droppedPublications++;
}
(void)MemoryOperationsHelper::Copy(readyBuffer, memory, totalSrcBytes); (void)MemoryOperationsHelper::Copy(readyBuffer, memory, totalSrcBytes);
syncTimestamp = ts; syncTimestamp = ts;
readySnapshotPending = true;
bufMutex.FastUnLock(); bufMutex.FastUnLock();
(void)dataSem.Post(); (void)dataSem.Post();
} }
@@ -955,25 +976,29 @@ ErrorManagement::ErrorType UDPStreamer::Execute(ExecutionInfo &info) {
} }
if (info.GetStage() == ExecutionInfo::MainStage) { if (info.GetStage() == ExecutionInfo::MainStage) {
/* --- Wait for RT thread to post new data --- /* --- Wait for the RT thread to publish new data ---
* ResetWait sleeps the background thread until the RT thread calls * dataSem is only a wake-up hint, never the record of pending work:
* Synchronise() and posts dataSem, or until the timeout expires. * EventSem::ResetWait resets the semaphore before waiting, so a Post that
* Doing this FIRST means the thread spends nearly all its time here * landed while this thread was inside ServiceClients()/SendData() is
* instead of spinning on the non-blocking select() below. * destroyed by the next Reset. Deciding what to send from the wait result
* Command latency is bounded by UDPS_DATA_WAIT_MS (acceptable for * would then skip that publication entirely, and the next flush would
* CONNECT / DISCONNECT). */ * overwrite it — the receiver sees the batch's whole time span missing.
ErrorManagement::ErrorType waitErr = * The buffers therefore carry the state, and are only waited on when they
dataSem.ResetWait(TimeoutType(UDPS_DATA_WAIT_MS)); * are empty (which also avoids paying the wait when work is already
bool dataReady = (waitErr == ErrorManagement::NoError); * queued). */
if (!HasPendingPublication()) {
(void)dataSem.ResetWait(TimeoutType(UDPS_DATA_WAIT_MS));
}
/* --- Poll for incoming control commands (CONNECT / DISCONNECT / ACK) --- /* --- Poll for incoming control commands (CONNECT / DISCONNECT / ACK) ---
*/ */
server.ServiceClients(); server.ServiceClients();
if (dataReady && server.HasClients()) { /* Synchronise() already gates publication to the correct rate (size/time
/* Synchronise() already gates posting dataSem to the correct rate * for Accumulate, every-Nth for Decimate, every call for Strict). The
* (size/time for Accumulate, every-Nth for Decimate, every call for * pending publication is consumed whether or not anyone is listening, so
* Strict). Execute() just sends whatever is in the ready buffers. */ * that a client-less streamer neither spins here nor delivers a stale
* snapshot to the next client that connects. */
if (publishMode == UDPStreamerPublishAccumulate) { if (publishMode == UDPStreamerPublishAccumulate) {
/* --- Accumulate batch send --- */ /* --- Accumulate batch send --- */
uint32 fill = 0u; uint32 fill = 0u;
@@ -986,10 +1011,11 @@ ErrorManagement::ErrorType UDPStreamer::Execute(ExecutionInfo &info) {
reinterpret_cast<uint8 *>(scratchTimestamps), reinterpret_cast<uint8 *>(scratchTimestamps),
reinterpret_cast<const uint8 *>(readyTimestamps), reinterpret_cast<const uint8 *>(readyTimestamps),
fill * static_cast<uint32>(sizeof(uint64))); fill * static_cast<uint32>(sizeof(uint64)));
readyFill = 0u;
} }
bufMutex.FastUnLock(); bufMutex.FastUnLock();
if (fill > 0u) { if ((fill > 0u) && server.HasClients()) {
SerializeAccumulated(scratchBuffer, scratchTimestamps, fill); SerializeAccumulated(scratchBuffer, scratchTimestamps, fill);
uint32 sendBytes = uint32 sendBytes =
UDPS_TIMESTAMP_BYTES + 4u + fill * singleCycleWireBytes; UDPS_TIMESTAMP_BYTES + 4u + fill * singleCycleWireBytes;
@@ -1003,12 +1029,18 @@ ErrorManagement::ErrorType UDPStreamer::Execute(ExecutionInfo &info) {
} else { } else {
/* --- Single-snapshot send (Strict or Decimate) --- */ /* --- Single-snapshot send (Strict or Decimate) --- */
uint64 ts = 0u; uint64 ts = 0u;
bool pending = false;
bufMutex.FastLock(TTInfiniteWait); bufMutex.FastLock(TTInfiniteWait);
pending = readySnapshotPending;
if (pending) {
(void)MemoryOperationsHelper::Copy(scratchBuffer, readyBuffer, (void)MemoryOperationsHelper::Copy(scratchBuffer, readyBuffer,
totalSrcBytes); totalSrcBytes);
ts = syncTimestamp; ts = syncTimestamp;
readySnapshotPending = false;
}
bufMutex.FastUnLock(); bufMutex.FastUnLock();
if (pending && server.HasClients()) {
QuantizeAndSerialize(scratchBuffer, ts); QuantizeAndSerialize(scratchBuffer, ts);
packetCounter++; packetCounter++;
@@ -1019,6 +1051,8 @@ ErrorManagement::ErrorType UDPStreamer::Execute(ExecutionInfo &info) {
} }
} }
} }
ReportDroppedPublications();
} }
if (info.GetStage() == ExecutionInfo::TerminationStage) { if (info.GetStage() == ExecutionInfo::TerminationStage) {
@@ -1212,6 +1246,16 @@ bool UDPStreamer::BuildConfigPayload(uint8 *buf, uint32 bufSize,
buf[payloadSize] = static_cast<uint8>(publishMode); buf[payloadSize] = static_cast<uint8>(publishMode);
payloadSize += 1u; payloadSize += 1u;
/* 8 bytes: this host's HRT tick rate. DATA packets carry raw counter
* values, so a receiver on another machine cannot turn them into seconds
* without it. */
if ((payloadSize + 8u) > bufSize) {
return false;
}
uint64 hrtFrequency = HighResolutionTimer::Frequency();
(void)MemoryOperationsHelper::Copy(buf + payloadSize, &hrtFrequency, 8u);
payloadSize += 8u;
return true; return true;
} }
@@ -1327,6 +1371,36 @@ bool UDPStreamer::IsClientConnected() const { return server.HasClients(); }
bool UDPStreamer::IsMulticast() const { return server.IsMulticast(); } bool UDPStreamer::IsMulticast() const { return server.IsMulticast(); }
uint32 UDPStreamer::GetDroppedPublications() const { return droppedPublications; }
bool UDPStreamer::HasPendingPublication() {
bool pending = false;
bufMutex.FastLock(TTInfiniteWait);
pending = (readyFill > 0u) || readySnapshotPending;
bufMutex.FastUnLock();
return pending;
}
void UDPStreamer::ReportDroppedPublications() {
uint64 now = HighResolutionTimer::Counter();
if ((now - lastDropReportTicks) < HighResolutionTimer::Frequency()) {
return;
}
lastDropReportTicks = now;
uint32 dropped = 0u;
bufMutex.FastLock(TTInfiniteWait);
dropped = droppedPublications;
bufMutex.FastUnLock();
if (dropped > 0u) {
REPORT_ERROR(ErrorManagement::Warning,
"Dropped %u unsent publication(s) so far: the sender thread is "
"not keeping up with the RT cycle.",
dropped);
}
}
CLASS_REGISTER(UDPStreamer, "1.0") CLASS_REGISTER(UDPStreamer, "1.0")
} /* namespace MARTe */ } /* namespace MARTe */
Source/Components/DataSources/UDPStreamer/UDPStreamer.h
+30
View File
@@ -322,6 +322,17 @@ public:
*/ */
bool IsMulticast() const; bool IsMulticast() const;
/**
* @brief Number of publications the sender thread never put on the wire.
* @details Synchronise() promotes a snapshot (Strict/Decimate) or a batch
* (Accumulate) to the ready buffer for the sender thread. If the next
* promotion arrives before the sender has taken the previous one, that
* publication is overwritten and its cycles never reach any receiver —
* which a consumer sees as a hole in the time series. Counts those, so the
* loss is measurable rather than inferred from the plot.
*/
uint32 GetDroppedPublications() const;
private: private:
/** /**
* @brief Serializes the CONFIG payload into buf and sets payloadSize. * @brief Serializes the CONFIG payload into buf and sets payloadSize.
@@ -349,6 +360,18 @@ private:
*/ */
static uint8 TypeDescriptorToCode(TypeDescriptor td); static uint8 TypeDescriptorToCode(TypeDescriptor td);
/**
* @brief True when the ready buffer holds data the sender has not taken yet.
* @details Read under bufMutex. The sender must consult this rather than
* rely on the dataSem edge, which ResetWait can destroy.
*/
bool HasPendingPublication();
/**
* @brief Emits at most one warning per second about overwritten publications.
*/
void ReportDroppedPublications();
/* Configuration parameters */ /* Configuration parameters */
uint16 port; /**< UDP server port */ uint16 port; /**< UDP server port */
uint32 maxPayloadSize; /**< Max payload bytes per UDP packet (excluding header) */ uint32 maxPayloadSize; /**< Max payload bytes per UDP packet (excluding header) */
@@ -367,6 +390,13 @@ private:
uint64 *readyTimestamps; /**< Heap: [maxBatchCount] HRT for completed ready batch */ uint64 *readyTimestamps; /**< Heap: [maxBatchCount] HRT for completed ready batch */
uint64 *scratchTimestamps; /**< Heap: [maxBatchCount] background-thread local copy */ uint64 *scratchTimestamps; /**< Heap: [maxBatchCount] background-thread local copy */
uint32 readyFill; /**< Snapshot count in the ready batch */ uint32 readyFill; /**< Snapshot count in the ready batch */
/** Strict/Decimate: readyBuffer holds a snapshot the sender has not taken
* yet. Publication state must live here rather than in dataSem, because
* EventSem::ResetWait resets before waiting and so destroys any Post that
* landed while the sender was busy. */
bool readySnapshotPending;
uint32 droppedPublications; /**< Publications overwritten before being sent */
uint64 lastDropReportTicks; /**< Sender-thread rate limit for the drop warning */
/* Decimate mode */ /* Decimate mode */
uint32 decimateRatio; /**< Send 1 packet every decimateRatio Synchronise() calls */ uint32 decimateRatio; /**< Send 1 packet every decimateRatio Synchronise() calls */
uint32 decimateCounter; /**< Current decimate cycle counter */ uint32 decimateCounter; /**< Current decimate cycle counter */
Source/Components/Interfaces/DebugService/DebugService.cpp
+8
View File
@@ -674,6 +674,14 @@ bool DebugService::SendUDPSConfig() {
payloadOffset++; payloadOffset++;
} }
// Write this host's HRT tick rate: DATA packets carry raw counter values,
// which a receiver on another machine cannot convert to seconds without it.
if ((payloadOffset + 8u) <= CFG_BUF_SIZE) {
uint64 hrtFrequency = HighResolutionTimer::Frequency();
memcpy(payload + payloadOffset, &hrtFrequency, 8u);
payloadOffset += 8u;
}
udpsNumSlots = newNumSlots; udpsNumSlots = newNumSlots;
mutex.FastUnLock(); mutex.FastUnLock();
Source/Components/Interfaces/UDPStream/UDPSClient.cpp
+177 -44
View File
@@ -36,7 +36,13 @@ UDPSClient::UDPSClient()
disconnectTick(0u), disconnectTick(0u),
lastKeepAliveTicks(0u), lastKeepAliveTicks(0u),
localPort(0u), localPort(0u),
lastGcTicks(0u) { lastGcTicks(0u),
lastDropWarnTicks(0u),
droppedSinceWarn(0u),
lastDataCounter(0u),
lastDataCounterValid(false),
lastDataGap(0u),
staleDataPackets(0u) {
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) { for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
reassemblySlots[i].counter = 0u; reassemblySlots[i].counter = 0u;
@@ -46,7 +52,11 @@ UDPSClient::UDPSClient()
reassemblySlots[i].active = false; reassemblySlots[i].active = false;
reassemblySlots[i].firstSeenTicks = 0u; reassemblySlots[i].firstSeenTicks = 0u;
reassemblySlots[i].chunkSize = 0u; reassemblySlots[i].chunkSize = 0u;
(void) MemoryOperationsHelper::Set(reassemblySlots[i].recvMask, 0, 32u); reassemblySlots[i].assembledBytes = 0u;
reassemblySlots[i].pendingTailBytes = 0u;
reassemblySlots[i].pendingTailValid = false;
(void) MemoryOperationsHelper::Set(reassemblySlots[i].recvMask, 0,
UDPS_CLIENT_RECV_MASK_BYTES);
} }
} }
@@ -228,6 +238,12 @@ bool UDPSClient::Connect() {
connected = true; connected = true;
lastDataTicks = HighResolutionTimer::Counter(); lastDataTicks = HighResolutionTimer::Counter();
lastKeepAliveTicks = lastDataTicks; lastKeepAliveTicks = lastDataTicks;
/* The producer's packetCounter restarts independently of ours, so a
* counter carried over from the previous connection would make the
* sequence gate reject the whole new stream as stale. */
lastDataCounterValid = false;
lastDataCounter = 0u;
lastDataGap = 0u;
if (listener != NULL_PTR(UDPSClientListener *)) { if (listener != NULL_PTR(UDPSClientListener *)) {
listener->OnUDPSConnected(); listener->OnUDPSConnected();
} }
@@ -498,6 +514,14 @@ bool UDPSClient::ReceiveAndProcess() {
return true; // only the TCP socket was readable return true; // only the TCP socket was readable
} }
/* Drain the socket rather than taking one datagram per Execute() iteration:
* a fragmented high-rate source delivers datagrams far faster than the
* select/read round trip retires them, and the resulting kernel-buffer
* overflow shows up as lost fragments — i.e. as packets that can never be
* reassembled. Bounded so the silence and keepalive checks in Execute()
* still run under a sustained flood. */
uint32 drained = 0u;
while (drained < UDPS_CLIENT_MAX_DATAGRAMS_PER_CYCLE) {
uint32 recvSize = static_cast<uint32>(sizeof(recvBuf)); uint32 recvSize = static_cast<uint32>(sizeof(recvBuf));
bool ok; bool ok;
if (useMulticast) { if (useMulticast) {
@@ -517,6 +541,18 @@ bool UDPSClient::ReceiveAndProcess() {
lastDataTicks = HighResolutionTimer::Counter(); lastDataTicks = HighResolutionTimer::Counter();
ProcessDatagram(recvBuf, recvSize); ProcessDatagram(recvBuf, recvSize);
drained++;
/* Stop as soon as the socket runs dry: Read() would otherwise block. */
fd_set dset;
FD_ZERO(&dset);
FD_SET(fd, &dset);
struct timeval zero;
zero.tv_sec = 0; zero.tv_usec = 0;
if (select(fd + 1, &dset, NULL, NULL, &zero) <= 0) {
break;
}
}
return true; return true;
} }
@@ -599,7 +635,7 @@ void UDPSClient::ProcessDatagram(const uint8 *buf, uint32 size) {
if (hdr->type == UDPS_TYPE_CONFIG) { if (hdr->type == UDPS_TYPE_CONFIG) {
listener->OnUDPSConfig(pl, payloadBytes); listener->OnUDPSConfig(pl, payloadBytes);
} }
else { else if (AcceptDataCounter(hdr->counter)) {
listener->OnUDPSData(pl, payloadBytes); listener->OnUDPSData(pl, payloadBytes);
} }
} }
@@ -616,21 +652,83 @@ void UDPSClient::ProcessDatagram(const uint8 *buf, uint32 size) {
// Private: PlaceFragment // Private: PlaceFragment
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
uint32 UDPSClient::AcquireReassemblySlot(uint32 counter, uint8 type) {
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (!reassemblySlots[i].active) {
return i;
}
}
/* All slots busy. The producer emits packets sequentially, so a slot
* holding an OLDER counter of the SAME stream is provably dead: its
* missing fragments were sent before the ones arriving now and will never
* turn up. Reclaiming it immediately — instead of waiting out the 2 s GC —
* is what keeps a handful of lost fragments from wedging the whole table.
* The counter is a wrapping uint32, so compare via the signed difference. */
uint32 victim = UDPS_CLIENT_MAX_REASSEMBLY_SLOTS;
int32 bestDist = 0;
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (reassemblySlots[i].type != type) {
continue;
}
int32 dist = static_cast<int32>(counter - reassemblySlots[i].counter);
if ((dist > 0) && (dist > bestDist)) {
bestDist = dist;
victim = i;
}
}
if (victim >= UDPS_CLIENT_MAX_REASSEMBLY_SLOTS) {
/* Nothing is provably dead (e.g. the other stream owns every slot):
* fall back to the least recently started. */
uint64 oldestTick = 0xFFFFFFFFFFFFFFFFuLL;
victim = 0u;
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (reassemblySlots[i].firstSeenTicks < oldestTick) {
oldestTick = reassemblySlots[i].firstSeenTicks;
victim = i;
}
}
}
NoteDroppedIncomplete(reassemblySlots[victim].counter);
return victim;
}
void UDPSClient::NoteDroppedIncomplete(uint32 counter) {
droppedSinceWarn++;
uint64 now = HighResolutionTimer::Counter();
if ((now - lastDropWarnTicks) >= HighResolutionTimer::Frequency()) {
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: dropped %u incomplete packet(s) in the last "
"second (latest counter %u); fragments are being lost.",
droppedSinceWarn, counter);
droppedSinceWarn = 0u;
lastDropWarnTicks = now;
}
}
bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr, bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
const uint8 *payload, const uint8 *payload,
uint32 payloadBytes) { uint32 payloadBytes) {
uint32 counter = hdr->counter; uint32 counter = hdr->counter;
uint8 type = hdr->type;
uint16 fragIdx = hdr->fragmentIdx; uint16 fragIdx = hdr->fragmentIdx;
uint16 totalFrags = hdr->totalFragments; uint16 totalFrags = hdr->totalFragments;
if ((fragIdx >= totalFrags) || (totalFrags > 512u)) { if ((fragIdx >= totalFrags) ||
(static_cast<uint32>(totalFrags) > UDPS_CLIENT_MAX_FRAGMENTS)) {
return false; // sanity check return false; // sanity check
} }
// Find existing slot for this counter /* Slots are keyed on (counter, type): DATA and CONFIG carry independent
* counter sequences, so the same counter value legitimately appears on
* both, and matching on the counter alone merges the two streams into one
* slot — one payload is delivered under the wrong type, the other is lost. */
uint32 slot = UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; uint32 slot = UDPS_CLIENT_MAX_REASSEMBLY_SLOTS;
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) { for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (reassemblySlots[i].active && (reassemblySlots[i].counter == counter)) { if (reassemblySlots[i].active && (reassemblySlots[i].counter == counter) &&
(reassemblySlots[i].type == type)) {
slot = i; slot = i;
break; break;
} }
@@ -638,34 +736,20 @@ bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
// Allocate new slot if not found // Allocate new slot if not found
if (slot >= UDPS_CLIENT_MAX_REASSEMBLY_SLOTS) { if (slot >= UDPS_CLIENT_MAX_REASSEMBLY_SLOTS) {
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) { slot = AcquireReassemblySlot(counter, type);
if (!reassemblySlots[i].active) {
slot = i;
break;
}
}
if (slot >= UDPS_CLIENT_MAX_REASSEMBLY_SLOTS) {
// All slots occupied — evict the oldest
uint64 oldestTick = 0xFFFFFFFFFFFFFFFFuLL;
for (uint32 i = 0u; i < UDPS_CLIENT_MAX_REASSEMBLY_SLOTS; i++) {
if (reassemblySlots[i].firstSeenTicks < oldestTick) {
oldestTick = reassemblySlots[i].firstSeenTicks;
slot = i;
}
}
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: Reassembly slots full; evicting oldest.");
}
reassemblySlots[slot].counter = counter; reassemblySlots[slot].counter = counter;
reassemblySlots[slot].type = hdr->type; reassemblySlots[slot].type = type;
reassemblySlots[slot].totalFragments = totalFrags; reassemblySlots[slot].totalFragments = totalFrags;
reassemblySlots[slot].receivedFragments = 0u; reassemblySlots[slot].receivedFragments = 0u;
reassemblySlots[slot].active = true; reassemblySlots[slot].active = true;
reassemblySlots[slot].firstSeenTicks = HighResolutionTimer::Counter(); reassemblySlots[slot].firstSeenTicks = HighResolutionTimer::Counter();
reassemblySlots[slot].chunkSize = 0u; reassemblySlots[slot].chunkSize = 0u;
reassemblySlots[slot].assembledBytes = 0u; reassemblySlots[slot].assembledBytes = 0u;
(void) MemoryOperationsHelper::Set(reassemblySlots[slot].recvMask, 0, 32u); reassemblySlots[slot].pendingTailBytes = 0u;
reassemblySlots[slot].pendingTailValid = false;
(void) MemoryOperationsHelper::Set(reassemblySlots[slot].recvMask, 0,
UDPS_CLIENT_RECV_MASK_BYTES);
} }
UDPSReassemblySlot &s = reassemblySlots[slot]; UDPSReassemblySlot &s = reassemblySlots[slot];
@@ -673,27 +757,56 @@ bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
// Skip duplicate // Skip duplicate
uint32 byteIdx = fragIdx / 8u; uint32 byteIdx = fragIdx / 8u;
uint8 bitMask = static_cast<uint8>(1u << (fragIdx % 8u)); uint8 bitMask = static_cast<uint8>(1u << (fragIdx % 8u));
if (byteIdx < 32u) {
if ((s.recvMask[byteIdx] & bitMask) != 0u) { if ((s.recvMask[byteIdx] & bitMask) != 0u) {
return false; // already have this fragment return false; // already have this fragment
} }
const bool isLastFragment = ((static_cast<uint32>(fragIdx) + 1u) ==
static_cast<uint32>(totalFrags));
/* Every fragment but the last carries a full chunk, so any of them reveals
* the chunk size — waiting specifically for fragment 0 means a merely
* reordered burst, with nothing lost, destroys the packet. */
if ((s.chunkSize == 0u) && !isLastFragment) {
s.chunkSize = payloadBytes;
} }
// Compute placement offset if (s.chunkSize == 0u) {
uint32 chunkSize = s.chunkSize; /* The last fragment arrived before any full-size one: its offset is
if (chunkSize == 0u) { * not computable yet, so hold it until the chunk size is known. */
// Learn chunk size from first non-last fragment if (payloadBytes > UDPS_CLIENT_PENDING_TAIL_BYTES) {
if (fragIdx == 0u) {
chunkSize = payloadBytes;
s.chunkSize = chunkSize;
}
else {
// Can't place yet without knowing chunk size — drop (rare edge case)
return false; return false;
} }
if (payloadBytes > 0u) {
(void) MemoryOperationsHelper::Copy(s.pendingTail, payload, payloadBytes);
}
s.pendingTailBytes = payloadBytes;
s.pendingTailValid = true;
s.recvMask[byteIdx] |= bitMask;
s.receivedFragments++;
return false; // totalFrags > 1 here, so this can never complete a packet
} }
uint32 offset = static_cast<uint32>(fragIdx) * chunkSize; // Flush a deferred last fragment now that the chunk size is known.
if (s.pendingTailValid) {
uint32 tailOffset = (static_cast<uint32>(s.totalFragments) - 1u) * s.chunkSize;
if ((tailOffset + s.pendingTailBytes) > static_cast<uint32>(sizeof(s.payload))) {
s.active = false;
NoteDroppedIncomplete(s.counter);
return false; // overflow guard
}
if (s.pendingTailBytes > 0u) {
(void) MemoryOperationsHelper::Copy(s.payload + tailOffset,
s.pendingTail, s.pendingTailBytes);
}
if ((tailOffset + s.pendingTailBytes) > s.assembledBytes) {
s.assembledBytes = tailOffset + s.pendingTailBytes;
}
s.pendingTailValid = false;
s.pendingTailBytes = 0u;
}
uint32 offset = static_cast<uint32>(fragIdx) * s.chunkSize;
if ((offset + payloadBytes) > static_cast<uint32>(sizeof(s.payload))) { if ((offset + payloadBytes) > static_cast<uint32>(sizeof(s.payload))) {
return false; // overflow guard return false; // overflow guard
} }
@@ -709,9 +822,7 @@ bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
s.assembledBytes = offset + payloadBytes; s.assembledBytes = offset + payloadBytes;
} }
if (byteIdx < 32u) {
s.recvMask[byteIdx] |= bitMask; s.recvMask[byteIdx] |= bitMask;
}
s.receivedFragments++; s.receivedFragments++;
// Check if complete // Check if complete
@@ -727,6 +838,30 @@ bool UDPSClient::PlaceFragment(const UDPSPacketHeader *hdr,
// Private: DeliverAssembled // Private: DeliverAssembled
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
bool UDPSClient::AcceptDataCounter(uint32 counter) {
if (!lastDataCounterValid) {
lastDataCounterValid = true;
lastDataCounter = counter;
lastDataGap = 0u;
return true;
}
// The counter is a wrapping uint32, so order it by the signed difference:
// that stays correct across the wrap, where a plain comparison would call
// the first packet after it stale and reject the stream from then on.
int32 delta = static_cast<int32>(counter - lastDataCounter);
if (delta <= 0) {
staleDataPackets++;
return false;
}
lastDataGap = static_cast<uint32>(delta) - 1u;
lastDataCounter = counter;
return true;
}
uint32 UDPSClient::GetLastDataGap() const { return lastDataGap; }
uint32 UDPSClient::GetStaleDataPackets() const { return staleDataPackets; }
void UDPSClient::DeliverAssembled(UDPSReassemblySlot &s) { void UDPSClient::DeliverAssembled(UDPSReassemblySlot &s) {
if (listener == NULL_PTR(UDPSClientListener *)) { if (listener == NULL_PTR(UDPSClientListener *)) {
return; return;
@@ -739,7 +874,7 @@ void UDPSClient::DeliverAssembled(UDPSReassemblySlot &s) {
if (s.type == UDPS_TYPE_CONFIG) { if (s.type == UDPS_TYPE_CONFIG) {
listener->OnUDPSConfig(s.payload, totalSize); listener->OnUDPSConfig(s.payload, totalSize);
} }
else { else if (AcceptDataCounter(s.counter)) {
listener->OnUDPSData(s.payload, totalSize); listener->OnUDPSData(s.payload, totalSize);
} }
} }
@@ -757,10 +892,8 @@ void UDPSClient::GcReassemblySlots() {
continue; continue;
} }
if ((now - reassemblySlots[i].firstSeenTicks) > staleThreshold) { if ((now - reassemblySlots[i].firstSeenTicks) > staleThreshold) {
REPORT_ERROR_STATIC(ErrorManagement::Warning,
"UDPSClient: Discarding stale reassembly slot (counter %u).",
reassemblySlots[i].counter);
reassemblySlots[i].active = false; reassemblySlots[i].active = false;
NoteDroppedIncomplete(reassemblySlots[i].counter);
} }
} }
} }
Source/Components/Interfaces/UDPStream/UDPSClient.h
+83 -4
View File
@@ -82,10 +82,27 @@ public:
* update for one source is fragmented into MaxPayloadSize chunks; this is * update for one source is fragmented into MaxPayloadSize chunks; this is
* the ceiling on the reassembled total, so it bounds the largest multi- * the ceiling on the reassembled total, so it bounds the largest multi-
* fragment packet the client can deliver. Sized for large array bursts * fragment packet the client can deliver. Sized for large array bursts
* (e.g. 8x10000 float32 ~= 320 KiB) with headroom; stays well within the * (e.g. 8x10000 float32 ~= 320 KiB) with headroom. */
* 256-fragment span the recvMask[32] tracks at typical chunk sizes. */
static const uint32 UDPS_CLIENT_MAX_PACKET_BYTES = 1048576u; // 1 MiB static const uint32 UDPS_CLIENT_MAX_PACKET_BYTES = 1048576u; // 1 MiB
/** Maximum fragment count accepted for one packet. The received-fragment
* bitmask must cover this whole span: a fragment index the mask cannot
* represent has no duplicate detection, so a duplicated datagram counts
* twice and the packet is delivered with a fragment still missing. */
static const uint32 UDPS_CLIENT_MAX_FRAGMENTS = 512u;
/** Bytes of received-fragment bitmask (one bit per fragment). */
static const uint32 UDPS_CLIENT_RECV_MASK_BYTES =
UDPS_CLIENT_MAX_FRAGMENTS / 8u;
/** Size of the per-slot buffer that holds a last fragment which arrived
* before the chunk size was known. Fragments larger than this cannot be
* deferred and are dropped (the packet then fails to reassemble). */
static const uint32 UDPS_CLIENT_PENDING_TAIL_BYTES = 8192u;
/** Maximum datagrams drained from the socket per Execute() iteration. */
static const uint32 UDPS_CLIENT_MAX_DATAGRAMS_PER_CYCLE = 256u;
/** Default silence timeout before reconnect (seconds); sub-second values allowed. */ /** Default silence timeout before reconnect (seconds); sub-second values allowed. */
static const float32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 1.0f; static const float32 UDPS_CLIENT_DEFAULT_SILENCE_TIMEOUT_S = 1.0f;
@@ -155,6 +172,22 @@ public:
*/ */
virtual ErrorManagement::ErrorType Execute(ExecutionInfo &info); virtual ErrorManagement::ErrorType Execute(ExecutionInfo &info);
/**
* @brief DATA packets that went missing immediately before the one being
* delivered, from the gap in the producer's packet counter.
* @details Valid for the duration of the OnUDPSData() callback. A listener
* that reconstructs per-sample timestamps needs this: without it, the time
* elapsed since the previous packet looks like it covers only that
* packet's samples, so the inferred sample period comes out too long and
* the samples are spread past where they belong.
*/
uint32 GetLastDataGap() const;
/**
* @brief DATA packets discarded for arriving after a newer one.
*/
uint32 GetStaleDataPackets() const;
private: private:
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
@@ -165,12 +198,19 @@ private:
uint8 type; ///< UDPS_TYPE_DATA or UDPS_TYPE_CONFIG uint8 type; ///< UDPS_TYPE_DATA or UDPS_TYPE_CONFIG
uint16 totalFragments; ///< Expected fragment count uint16 totalFragments; ///< Expected fragment count
uint16 receivedFragments; ///< How many we have so far uint16 receivedFragments; ///< How many we have so far
uint8 recvMask[32]; ///< Bitmask: bit f set iff fragment f received uint8 recvMask[UDPS_CLIENT_RECV_MASK_BYTES]; ///< Bit f set iff fragment f received
uint8 payload[UDPS_CLIENT_MAX_PACKET_BYTES]; ///< Assembled payload buffer uint8 payload[UDPS_CLIENT_MAX_PACKET_BYTES]; ///< Assembled payload buffer
uint64 firstSeenTicks; ///< For GC (2 s stale detection) uint64 firstSeenTicks; ///< For GC (2 s stale detection)
bool active; ///< Slot in use bool active; ///< Slot in use
uint32 chunkSize; ///< Payload bytes per fragment (from first fragment) uint32 chunkSize; ///< Payload bytes per fragment (any non-last fragment)
uint32 assembledBytes; ///< Exact total payload bytes placed so far uint32 assembledBytes; ///< Exact total payload bytes placed so far
/** Last fragment received before chunkSize was known: its offset is
* not yet computable, so it waits here until a full-size fragment
* reveals the chunk size. Only the last fragment can ever be short,
* hence one deferred fragment per slot is enough. */
uint8 pendingTail[UDPS_CLIENT_PENDING_TAIL_BYTES];
uint32 pendingTailBytes;
bool pendingTailValid;
}; };
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
@@ -195,8 +235,39 @@ private:
bool ReadExactTCP(uint8 *dst, uint32 n); bool ReadExactTCP(uint8 *dst, uint32 n);
/** @return true iff this fragment completed the reassembly (payload delivered). */ /** @return true iff this fragment completed the reassembly (payload delivered). */
bool PlaceFragment(const UDPSPacketHeader *hdr, const uint8 *payload, uint32 payloadBytes); bool PlaceFragment(const UDPSPacketHeader *hdr, const uint8 *payload, uint32 payloadBytes);
/**
* @brief Reserve a reassembly slot for (@p counter, @p type), reclaiming
* one if none is free.
* @return the slot index (always valid).
*/
uint32 AcquireReassemblySlot(uint32 counter, uint8 type);
/**
* @brief Account one packet abandoned with fragments missing, and report
* it at most once per second.
* @details Fragment loss on a busy stream is chronic, not exceptional: an
* unconditional message per drop buries every other log line.
*/
void NoteDroppedIncomplete(uint32 counter);
void GcReassemblySlots(); void GcReassemblySlots();
void DeliverAssembled(UDPSReassemblySlot &slot); void DeliverAssembled(UDPSReassemblySlot &slot);
/**
* @brief Sequence gate for DATA packets, applied just before delivery.
* @details The producer numbers DATA packets consecutively, so the counter
* reveals both how many packets went missing and whether this one is late.
* A late packet must not be delivered: its samples predate what the
* listener has already stored, so they land behind the current write
* position and collide with data that is already there — which is what a
* consumer sees as two signals occupying the same instant. Reassembly
* completes in arrival order, not counter order, so this ordering is not
* guaranteed upstream.
*
* Also records the number of packets missing immediately before this one,
* for GetLastDataGap().
*
* @param counter The candidate packet's UDPS counter.
* @return true if the packet is newer than the last delivered one.
*/
bool AcceptDataCounter(uint32 counter);
// ------------------------------------------------------------------------- // -------------------------------------------------------------------------
// Configuration // Configuration
@@ -236,6 +307,14 @@ private:
// Reassembly // Reassembly
UDPSReassemblySlot reassemblySlots[UDPS_CLIENT_MAX_REASSEMBLY_SLOTS]; UDPSReassemblySlot reassemblySlots[UDPS_CLIENT_MAX_REASSEMBLY_SLOTS];
uint64 lastGcTicks; ///< Ticks at last GC run uint64 lastGcTicks; ///< Ticks at last GC run
uint64 lastDropWarnTicks;///< Ticks at last incomplete-packet report
uint32 droppedSinceWarn; ///< Incomplete packets since that report
// DATA sequencing (see AcceptDataCounter)
uint32 lastDataCounter; ///< Counter of the last delivered DATA packet
bool lastDataCounterValid; ///< False until the first DATA packet
uint32 lastDataGap; ///< Packets missing before the current one
uint32 staleDataPackets; ///< DATA packets discarded as late
// Receive scratch buffer // Receive scratch buffer
uint8 recvBuf[65535u + UDPS_HEADER_SIZE]; uint8 recvBuf[65535u + UDPS_HEADER_SIZE];
Test/Applications/StreamHub/AccumDtGTest.cpp
+162
View File
@@ -0,0 +1,162 @@
/**
* @file AccumDtGTest.cpp
* @brief Tests UDPSEstimateAccumDt, the per-sample period estimator used for
* accumulated scalars that carry no SamplingRate.
*
* The estimator exists because the natural formula — sender-clock gap divided
* by the previous packet's sample count — is only correct while no packet is
* lost. When one is, the gap covers cycles that count never saw and the period
* comes out too large, which spreads the packet's samples past their real end
* and into the range the next packet claims. The loss count comes from the
* producer's packet counter rather than being inferred from the gap itself, so
* these tests pin both sides: the estimate must not move when packets go
* missing, and it must still follow a genuine rate change — a cycle count
* inferred from the estimate's own period would lock onto the old one.
*
* @copyright Copyright 2015 F4E | European Joint Undertaking for ITER and
* the Development of Fusion Energy ('Fusion for Energy').
* Licensed under the EUPL, Version 1.1 or - as soon they will be approved
* by the European Commission - subsequent versions of the EUPL (the "Licence")
* You may not use this work except in compliance with the Licence.
* You may obtain a copy of the Licence at: http://ec.europa.eu/idabc/eupl
*
* @warning Unless required by applicable law or agreed to in writing,
* software distributed under the Licence is distributed on an "AS IS"
* basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
* or implied. See the Licence permissions and limitations under the Licence.
*/
#include <gtest/gtest.h>
#include "UDPSourceSession.h"
using MARTe::float64;
using MARTe::uint32;
using StreamHub::UDPSEstimateAccumDt;
namespace {
/** A producer emitting batches of BATCH cycles at a period of DT seconds. */
const float64 kDt = 1.0e-3;
const uint32 kBatch = 10u;
const float64 kGap = kDt * static_cast<float64>(kBatch);
/** Feeds n clean packets and returns the settled estimate. */
float64 Warmup(uint32 n, float64 &dtEMA, bool &dtValid) {
float64 dt = 0.0;
for (uint32 i = 0u; i < n; i++) {
dt = UDPSEstimateAccumDt(kGap, kBatch, 0u, dtEMA, dtValid);
}
return dt;
}
} // namespace
/* The first packet has nothing to go on but the previous sample count, so it
* must fall back to gap/prevN rather than to some fixed default. */
TEST(AccumDtGTest, BootstrapsFromPreviousSampleCount) {
float64 dtEMA = 0.0;
bool dtValid = false;
const float64 dt = UDPSEstimateAccumDt(kGap, kBatch, 0u, dtEMA, dtValid);
EXPECT_TRUE(dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-12);
}
/* A clean stream must hold the period steady, not drift. */
TEST(AccumDtGTest, SteadyStreamStaysOnPeriod) {
float64 dtEMA = 0.0;
bool dtValid = false;
const float64 dt = Warmup(50u, dtEMA, dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-9);
}
/* The regression this whole estimator is for: one packet is lost, so the gap
* doubles while prevN does not. Dividing by prevN would report 2x the true
* period — enough to walk a 10-sample batch a full batch past its own end. */
TEST(AccumDtGTest, LostPacketDoesNotInflatePeriod) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
const float64 dt = UDPSEstimateAccumDt(2.0 * kGap, kBatch, 1u, dtEMA, dtValid);
/* What the naive formula would have produced. */
const float64 naive = (2.0 * kGap) / static_cast<float64>(kBatch);
EXPECT_NEAR(2.0 * kDt, naive, 1.0e-12);
EXPECT_NEAR(kDt, dt, 1.0e-6);
}
/* Several consecutive losses are the same situation, just wider. */
TEST(AccumDtGTest, MultiplePacketLossDoesNotInflatePeriod) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
for (uint32 missing = 1u; missing <= 5u; missing++) {
const float64 span = static_cast<float64>(missing + 1u) * kGap;
const float64 dt = UDPSEstimateAccumDt(span, kBatch, missing, dtEMA,
dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-6) << "after " << missing << " lost packet(s)";
}
}
/* Loss must not leave the estimator poisoned for the packets that follow. */
TEST(AccumDtGTest, RecoversToCleanStreamAfterLoss) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
(void) UDPSEstimateAccumDt(3.0 * kGap, kBatch, 2u, dtEMA, dtValid);
const float64 dt = Warmup(20u, dtEMA, dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-6);
}
/* A real, sustained rate change must still be followed — the estimator is a
* smoother, not a latch. Half the period is exactly on the rejection boundary,
* so use a change that lands inside the accepted band. */
TEST(AccumDtGTest, FollowsSustainedRateChange) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
const float64 newDt = kDt * 0.75;
const float64 newGap = newDt * static_cast<float64>(kBatch);
float64 dt = 0.0;
for (uint32 i = 0u; i < 400u; i++) {
dt = UDPSEstimateAccumDt(newGap, kBatch, 0u, dtEMA, dtValid);
}
EXPECT_NEAR(newDt, dt, 1.0e-6);
}
/* A batch that carries fewer cycles than usual (a time-triggered flush) is not
* loss: the gap shrinks with it, so the period must not shrink too. */
TEST(AccumDtGTest, ShortBatchDoesNotDeflatePeriod) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
const uint32 shortBatch = 3u;
const float64 dt = UDPSEstimateAccumDt(
kDt * static_cast<float64>(shortBatch), shortBatch, 0u, dtEMA, dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-6);
}
/* A gap shorter than one period cannot mean zero cycles; the divisor is
* clamped so the estimate can never be driven to infinity. */
TEST(AccumDtGTest, SubPeriodGapDoesNotExplode) {
float64 dtEMA = 0.0;
bool dtValid = false;
(void) Warmup(50u, dtEMA, dtValid);
const float64 dt = UDPSEstimateAccumDt(kDt * 1.0e-3, 1u, 0u, dtEMA, dtValid);
EXPECT_NEAR(kDt, dt, 1.0e-6);
}
Test/Applications/StreamHub/ConfigHrtFreqGTest.cpp
+151
View File
@@ -0,0 +1,151 @@
/**
* @file ConfigHrtFreqGTest.cpp
* @brief Tests UDPSConfigHrtFrequency, which picks the tick rate used to turn
* a producer's DATA timestamps into seconds.
*
* DATA packets carry the raw value of the producer's high-resolution counter.
* Until the rate was published in CONFIG the hub divided by its own timer's
* frequency, which is only right while the producer runs on the same host —
* on x86 that number is the TSC frequency and differs from model to model, so
* off-box every accumulated batch was laid out over the wrong span of time.
*
* The field is a trailer, so these tests pin both directions: a payload that
* carries it must be believed, and one that stops early — an older producer —
* must still decode against the local fallback rather than against zero.
*
* @copyright Copyright 2015 F4E | European Joint Undertaking for ITER and
* the Development of Fusion Energy ('Fusion for Energy').
* Licensed under the EUPL, Version 1.1 or - as soon they will be approved
* by the European Commission - subsequent versions of the EUPL (the "Licence")
* You may not use this work except in compliance with the Licence.
* You may obtain a copy of the Licence at: http://ec.europa.eu/idabc/eupl
*
* @warning Unless required by applicable law or agreed to in writing,
* software distributed under the Licence is distributed on an "AS IS"
* basis, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
* or implied. See the Licence permissions and limitations under the Licence.
*/
#include <gtest/gtest.h>
#include <string.h>
#include "UDPSourceSession.h"
using MARTe::uint8;
using MARTe::uint32;
using MARTe::uint64;
using MARTe::float64;
using MARTe::UDPS_SIGNAL_DESC_SIZE;
using StreamHub::UDPSConfigHrtFrequency;
namespace {
/** This hub's own timer rate, i.e. what the code must fall back to. */
const float64 kLocalFreq = 1.0e9;
/** A plausible producer rate that is deliberately not kLocalFreq. */
const uint64 kWireFreq = 2400000000ULL;
const uint32 kNumSigs = 2u;
/** Offset of the CONFIG trailer that follows the publish-mode byte. */
uint32 TrailerOffset(uint32 numSigs) {
return 4u + (numSigs * UDPS_SIGNAL_DESC_SIZE) + 1u;
}
/**
* Builds a CONFIG payload for kNumSigs signals.
* @param withFreq Append the 8-byte HRT frequency trailer.
* @param freq Value to append when @p withFreq.
* @param[out] size Bytes written.
*/
const uint8 *BuildConfig(bool withFreq, uint64 freq, uint32 &size) {
static uint8 buf[4u + (kNumSigs * UDPS_SIGNAL_DESC_SIZE) + 1u + 8u];
(void) memset(buf, 0, sizeof(buf));
(void) memcpy(buf, &kNumSigs, 4u);
size = TrailerOffset(kNumSigs);
if (withFreq) {
(void) memcpy(buf + size, &freq, 8u);
size += 8u;
}
return buf;
}
} // namespace
/* The whole point of the field: a producer that publishes its rate is believed
* even when the hub's own timer runs at a different one. */
TEST(ConfigHrtFreqGTest, AdoptsThePublishedRate) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(true, kWireFreq, size);
EXPECT_DOUBLE_EQ(static_cast<float64>(kWireFreq),
UDPSConfigHrtFrequency(cfg, size, kNumSigs, kLocalFreq));
}
/* A producer older than the field stops after the publish-mode byte. Reading
* past it would take whatever follows in the receive buffer as a frequency. */
TEST(ConfigHrtFreqGTest, FallsBackWhenTheTrailerIsAbsent) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(false, 0u, size);
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(cfg, size, kNumSigs, kLocalFreq));
}
/* A trailer cut short mid-field is not a frequency either; taking the bytes
* that are there would assemble one out of whatever the rest of the buffer
* holds. */
TEST(ConfigHrtFreqGTest, FallsBackOnATruncatedTrailer) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(true, kWireFreq, size);
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(cfg, size - 1u, kNumSigs,
kLocalFreq));
}
/* Zero is the protocol's "I do not know my own rate". Dividing by it yields
* infinities that propagate into every timestamp. */
TEST(ConfigHrtFreqGTest, FallsBackOnTheUnknownSentinel) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(true, MARTe::UDPS_HRT_FREQUENCY_UNKNOWN,
size);
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(cfg, size, kNumSigs, kLocalFreq));
}
/* No high-resolution timer ticks slower than 1 kHz, so a value that low means
* the payload was misread. Adopting it would stretch a millisecond batch
* across whole seconds. */
TEST(ConfigHrtFreqGTest, FallsBackOnAnImplausiblyLowRate) {
uint32 size = 0u;
const uint8 *cfg = BuildConfig(true, 999u, size);
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(cfg, size, kNumSigs, kLocalFreq));
}
/* The trailer sits after the descriptors, so its offset moves with the signal
* count; a fixed offset would read descriptor bytes on any other config. */
TEST(ConfigHrtFreqGTest, LocatesTheTrailerAfterTheDescriptors) {
const uint32 numSigs = 7u;
const uint32 size = TrailerOffset(numSigs) + 8u;
uint8 buf[4u + (7u * UDPS_SIGNAL_DESC_SIZE) + 1u + 8u];
/* Fill the descriptor area with a byte pattern that would decode as a
* plausible frequency if the offset were wrong. */
(void) memset(buf, 0x11, sizeof(buf));
(void) memcpy(buf, &numSigs, 4u);
(void) memcpy(buf + TrailerOffset(numSigs), &kWireFreq, 8u);
EXPECT_DOUBLE_EQ(static_cast<float64>(kWireFreq),
UDPSConfigHrtFrequency(buf, size, numSigs, kLocalFreq));
}
/* A null payload must not be dereferenced: CONFIG arrives from the network. */
TEST(ConfigHrtFreqGTest, FallsBackOnANullPayload) {
EXPECT_DOUBLE_EQ(kLocalFreq,
UDPSConfigHrtFrequency(NULL_PTR(const uint8 *), 1024u,
kNumSigs, kLocalFreq));
}
Test/Applications/StreamHub/Makefile.inc
+1 -1
View File
@@ -22,7 +22,7 @@
# #
############################################################# #############################################################
OBJSX = TriggerEngineSrc.x BinaryRecorderSrc.x SignalRingBufferGTest.x TriggerEngineGTest.x LTTBGTest.x BinaryRecorderGTest.x BoundsCheckTest.x WSServerBufferTest.x OBJSX = TriggerEngineSrc.x BinaryRecorderSrc.x SignalRingBufferGTest.x TriggerEngineGTest.x LTTBGTest.x BinaryRecorderGTest.x BoundsCheckTest.x WSServerBufferTest.x AccumDtGTest.x ConfigHrtFreqGTest.x
PACKAGE=Applications PACKAGE=Applications
ROOT_DIR=../../.. ROOT_DIR=../../..
Test/Applications/StreamHub/TriggerEngineGTest.cpp
+177
View File
@@ -196,6 +196,183 @@ TEST(TriggerEngineGTest, TestRearmResetsEdgeDetection) {
EXPECT_DOUBLE_EQ(4.0, tt); EXPECT_DOUBLE_EQ(4.0, tt);
} }
/* An edge that arrives while a capture is still being collected, or while it is
* being handed out, used to be dropped on the floor: CheckSample returned early
* in every state but ARMED, and the automatic rearm then waited for a FRESH
* edge. The engine is therefore deaf from its own trigger point until the
* capture has been harvested — a post-window — and then for the holdoff on top.
*
* On a sparse pulse train that rounds the capture spacing up to a whole pulse
* period: at the default 1 s window the blind stretch is 1 s, so a 1 Hz train
* was caught at 0.5 Hz and a wider window lost whole multiples. Remembering the
* edge costs nothing, because the capture is built from the edge's own
* timestamp out of a ring that still holds everything around it. */
TEST(TriggerEngineGTest, TestEdgeDuringCaptureFiresOnRearm) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0)); /* post = 0.8 */
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
/* A second pulse, clear of the capture in flight (1.1 + 0.8 = 1.9). */
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0);
eng.MarkTriggered();
eng.Rearm();
EXPECT_EQ(kTrigCollecting, eng.GetState());
float64 tt, pre, post;
ASSERT_TRUE(eng.GetFiredWindow(tt, pre, post));
EXPECT_DOUBLE_EQ(2.5, tt); /* the remembered edge, not the rearm instant */
}
/* The remembered edge must not be one the capture in flight already covers, nor
* one inside the holdoff — that guard exists to stop the ringing of a single
* event re-triggering on itself, and it is measured from the trigger point, so
* the two overlap rather than add. */
TEST(TriggerEngineGTest, TestEdgeInsideOwnCaptureIsNotRemembered) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0)); /* post = 0.8 */
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
/* Inside 1.1 + max(0.8, 0.2 holdoff) = 1.9: the capture owns this stretch. */
eng.CheckSample(1.4, 0.0);
eng.CheckSample(1.5, 1.0);
eng.MarkTriggered();
eng.Rearm();
EXPECT_EQ(kTrigArmed, eng.GetState());
}
/* A holdoff longer than the post-window is what decides the guard interval. */
TEST(TriggerEngineGTest, TestHoldoffOutlastingPostWindowGovernsRearm) {
TriggerEngine eng;
TriggerConfig cfg = MakeConfig(kEdgeRising, 0.5, 1.0, 80.0); /* post = 0.2 */
cfg.holdoffSec = 2.0;
eng.SetConfig(cfg);
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
/* Past the post-window but inside the holdoff (1.1 + 2.0 = 3.1): ignored. */
eng.CheckSample(1.9, 0.0);
eng.CheckSample(2.0, 1.0);
/* Clear of it: remembered. */
eng.CheckSample(3.4, 0.0);
eng.CheckSample(3.5, 1.0);
eng.MarkTriggered();
eng.Rearm();
ASSERT_EQ(kTrigCollecting, eng.GetState());
float64 tt, pre, post;
ASSERT_TRUE(eng.GetFiredWindow(tt, pre, post));
EXPECT_DOUBLE_EQ(3.5, tt);
}
/* Only the first qualifying edge is worth keeping; a later one would deliver
* the same capture a pulse further on and skip the one in between. */
TEST(TriggerEngineGTest, TestFirstQualifyingEdgeWins) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0); /* first past 1.9 */
eng.CheckSample(3.4, 0.0);
eng.CheckSample(3.5, 1.0); /* later, must not displace it */
eng.MarkTriggered();
eng.Rearm();
float64 tt, pre, post;
ASSERT_TRUE(eng.GetFiredWindow(tt, pre, post));
EXPECT_DOUBLE_EQ(2.5, tt);
}
/* Arm() is the user's own arm: it asks for the next event, not for one that has
* already been and gone, so it drops anything remembered. */
TEST(TriggerEngineGTest, TestUserArmDiscardsRememberedEdge) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0);
eng.MarkTriggered();
eng.Arm();
EXPECT_EQ(kTrigArmed, eng.GetState());
}
/* Reconfiguring drops it too: the edge would be latched against a window it was
* never judged against. */
TEST(TriggerEngineGTest, TestSetConfigDiscardsRememberedEdge) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0);
eng.MarkTriggered();
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 2.0, 20.0));
eng.Rearm();
EXPECT_EQ(kTrigArmed, eng.GetState());
}
/* The comparator keeps running through the dead time, so the first sample after
* an automatic rearm is measured against its real predecessor rather than being
* spent seeding one. A rearm landing mid-pulse would otherwise miss that
* pulse's edge as well as the ones it slept through. */
TEST(TriggerEngineGTest, TestRearmKeepsTrackingTheLevel) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
ASSERT_EQ(kTrigCollecting, eng.GetState());
/* Falls back low during the capture: no rising edge, nothing remembered,
* but the level is now known to be 0. */
eng.CheckSample(2.0, 0.0);
eng.MarkTriggered();
eng.Rearm();
ASSERT_EQ(kTrigArmed, eng.GetState());
eng.CheckSample(2.1, 1.0); /* 0.0 → 1.0 across 0.5, on the very first sample */
EXPECT_EQ(kTrigCollecting, eng.GetState());
float64 tt, pre, post;
ASSERT_TRUE(eng.GetFiredWindow(tt, pre, post));
EXPECT_DOUBLE_EQ(2.1, tt);
}
/* Idle is genuinely deaf: nothing is tracked and nothing is remembered, so a
* disarmed hub cannot fire the moment it is armed again. */
TEST(TriggerEngineGTest, TestDisarmDiscardsRememberedEdge) {
TriggerEngine eng;
eng.SetConfig(MakeConfig(kEdgeRising, 0.5, 1.0, 20.0));
eng.Arm();
eng.CheckSample(1.0, 0.0);
eng.CheckSample(1.1, 1.0);
eng.CheckSample(2.4, 0.0);
eng.CheckSample(2.5, 1.0);
eng.MarkTriggered();
eng.Disarm();
eng.Rearm();
EXPECT_EQ(kTrigArmed, eng.GetState());
}
TEST(TriggerEngineGTest, TestStoppedFlag) { TEST(TriggerEngineGTest, TestStoppedFlag) {
TriggerEngine eng; TriggerEngine eng;
EXPECT_FALSE(eng.GetStopped()); EXPECT_FALSE(eng.GetStopped());
Test/Components/DataSources/UDPStreamer/UDPStreamerGTest.cpp
+5
View File
@@ -225,3 +225,8 @@ TEST(UDPStreamerGTest, TestExecute_MulticastConnectDataDisconnect) {
UDPStreamerTest test; UDPStreamerTest test;
ASSERT_TRUE(test.TestExecute_MulticastConnectDataDisconnect()); ASSERT_TRUE(test.TestExecute_MulticastConnectDataDisconnect());
} }
TEST(UDPStreamerGTest, TestAccumulate_EveryPublishedCycleReachesTheWire) {
UDPStreamerTest test;
ASSERT_TRUE(test.TestAccumulate_EveryPublishedCycleReachesTheWire());
}
Test/Components/DataSources/UDPStreamer/UDPStreamerTest.cpp
+314
View File
@@ -36,11 +36,13 @@
#include "ConfigurationDatabase.h" #include "ConfigurationDatabase.h"
#include "GAM.h" #include "GAM.h"
#include "GAMScheduler.h" #include "GAMScheduler.h"
#include "HighResolutionTimer.h"
#include "MemoryOperationsHelper.h" #include "MemoryOperationsHelper.h"
#include "ObjectRegistryDatabase.h" #include "ObjectRegistryDatabase.h"
#include "RealTimeApplication.h" #include "RealTimeApplication.h"
#include "Sleep.h" #include "Sleep.h"
#include "StandardParser.h" #include "StandardParser.h"
#include "UDPSClient.h"
#include "UDPStreamer.h" #include "UDPStreamer.h"
#include "UDPStreamerTest.h" #include "UDPStreamerTest.h"
@@ -906,6 +908,23 @@ bool UDPStreamerTest::TestExecute_ConnectDataDisconnect() {
reinterpret_cast<const UDPSPacketHeader *>(recvBuf); reinterpret_cast<const UDPSPacketHeader *>(recvBuf);
ok &= (hdr->magic == UDPS_MAGIC); ok &= (hdr->magic == UDPS_MAGIC);
ok &= (hdr->type == UDPS_TYPE_CONFIG); ok &= (hdr->type == UDPS_TYPE_CONFIG);
/* The CONFIG trailer must carry this host's HRT tick rate: DATA
* packets timestamp with the raw counter, so a receiver on another
* machine has nothing to convert it with otherwise. */
const uint8 *payload = recvBuf + UDPS_HEADER_SIZE;
uint32 numSigs = 0u;
if (ok && (hdr->payloadBytes >= 4u)) {
(void) memcpy(&numSigs, payload, 4u);
}
const uint32 freqOff =
4u + (numSigs * UDPS_SIGNAL_DESC_SIZE) + 1u;
ok &= (hdr->payloadBytes >= (freqOff + 8u));
if (ok) {
uint64 wireFreq = 0u;
(void) memcpy(&wireFreq, payload + freqOff, 8u);
ok &= (wireFreq == HighResolutionTimer::Frequency());
}
} }
} }
@@ -1845,3 +1864,298 @@ bool UDPStreamerTest::TestExecute_MulticastConnectDataDisconnect() {
ObjectRegistryDatabase::Instance()->Purge(); ObjectRegistryDatabase::Instance()->Purge();
return ok; return ok;
} }
/*---------------------------------------------------------------------------*/
/* Accumulate publication continuity */
/*---------------------------------------------------------------------------*/
/* Four float64 scalars, no quantisation: 32 wire bytes per RT cycle.
* With MaxPayloadSize = 60 the accumulate header (8 B HRT + 4 B count) leaves
* room for exactly one cycle, so the size condition flushes on every single
* Synchronise() — the maximum number of hand-offs to the sender thread, each
* one a chance for a promoted batch to be skipped. */
#define ACC_FUNCTIONS_BLOCK \
" +Functions = {\n" \
" Class = ReferenceContainer\n" \
" +Writer = {\n" \
" Class = UDPStreamerTestOutputGAM\n" \
" OutputSignals = {\n" \
" A = {\n" \
" DataSource = Streamer\n" \
" Type = float64\n" \
" }\n" \
" B = {\n" \
" DataSource = Streamer\n" \
" Type = float64\n" \
" }\n" \
" C = {\n" \
" DataSource = Streamer\n" \
" Type = float64\n" \
" }\n" \
" D = {\n" \
" DataSource = Streamer\n" \
" Type = float64\n" \
" }\n" \
" }\n" \
" }\n" \
" }\n"
static const MARTe::char8 *const ACC_CFG_CONTINUITY =
"+Test = {\n"
" Class = RealTimeApplication\n"
ACC_FUNCTIONS_BLOCK
" +Data = {\n"
" Class = ReferenceContainer\n"
" +Streamer = {\n"
" Class = UDPStreamer\n"
" Port = 44680\n"
" MaxPayloadSize = 60\n"
" PublishingMode = Accumulate\n"
" MinRefreshRate = 1000.0\n"
" Signals = {\n"
" A = {\n"
" Type = float64\n"
" }\n"
" B = {\n"
" Type = float64\n"
" }\n"
" C = {\n"
" Type = float64\n"
" }\n"
" D = {\n"
" Type = float64\n"
" }\n"
" }\n"
" }\n"
HF_TAIL_BLOCK;
namespace {
/** Cycles driven by TestAccumulate_EveryPublishedCycleReachesTheWire. */
static const MARTe::uint32 ACC_CONTINUITY_CYCLES = 3000u;
/**
* @brief Records which RT cycles reached the wire, and how often.
*
* The test stamps signal A with the cycle index before every Synchronise(),
* and the config is sized so each Accumulate batch carries exactly one cycle.
* The payload is [8 B HRT][4 B numSamples][A][B][C][D], so A of the single
* slot sits at offset 12 and identifies the cycle unambiguously.
*
* Counting distinct cycles (rather than summing numSamples) is what makes this
* able to tell a lost publication from a re-sent one: a sender that never
* consumes its ready buffer emits the right *number* of packets while
* repeating a stale batch, which shows up here as duplicates plus missing
* cycles instead of a clean tally.
*/
class AccumRampRecorder: public MARTe::UDPSClientListener {
public:
AccumRampRecorder() :
packets(0u), duplicates(0u), malformed(0u) {
mux.Create();
for (MARTe::uint32 i = 0u; i < ACC_CONTINUITY_CYCLES; i++) {
seen[i] = false;
}
}
virtual void OnUDPSData(const MARTe::uint8 *payload, MARTe::uint32 payloadSize) {
MARTe::uint32 n = 0u;
MARTe::float64 v = 0.0;
if (payloadSize >= 20u) {
(void) MARTe::MemoryOperationsHelper::Copy(&n, &payload[8], 4u);
(void) MARTe::MemoryOperationsHelper::Copy(&v, &payload[12], 8u);
}
(void) mux.FastLock();
packets++;
if ((payloadSize < 20u) || (n != 1u)) {
malformed++;
}
else {
MARTe::uint32 idx = static_cast<MARTe::uint32>(v);
if ((static_cast<MARTe::float64>(idx) != v) || (idx >= ACC_CONTINUITY_CYCLES)) {
malformed++;
}
else if (seen[idx]) {
duplicates++;
}
else {
seen[idx] = true;
}
}
mux.FastUnLock();
}
MARTe::uint32 DistinctCycles() {
(void) mux.FastLock();
MARTe::uint32 n = 0u;
for (MARTe::uint32 i = 0u; i < ACC_CONTINUITY_CYCLES; i++) {
if (seen[i]) {
n++;
}
}
mux.FastUnLock();
return n;
}
MARTe::uint32 Packets() {
(void) mux.FastLock();
MARTe::uint32 n = packets;
mux.FastUnLock();
return n;
}
MARTe::uint32 Duplicates() {
(void) mux.FastLock();
MARTe::uint32 n = duplicates;
mux.FastUnLock();
return n;
}
MARTe::uint32 Malformed() {
(void) mux.FastLock();
MARTe::uint32 n = malformed;
mux.FastUnLock();
return n;
}
private:
MARTe::FastPollingMutexSem mux;
bool seen[ACC_CONTINUITY_CYCLES];
MARTe::uint32 packets;
MARTe::uint32 duplicates;
MARTe::uint32 malformed;
};
} // namespace
bool UDPStreamerTest::TestAccumulate_EveryPublishedCycleReachesTheWire() {
using namespace MARTe;
/* One-cycle batches every 200 us: ~5000 small packets/s, which the sender
* thread handles comfortably. The period has to be this short because a
* wake-up can only be swallowed while the sender is mid-send; at 1 ms the
* sender is always back in its wait before the next Synchronise() and the
* defect never fires at all. */
const uint32 CYCLES = ACC_CONTINUITY_CYCLES;
static const float64 CYCLE_SEC = 200e-6;
/* Tolerance, as a fraction of CYCLES, for cycles that never reach the wire.
* It is not zero: this is an ordinary userspace thread on a general-purpose
* kernel, so it can occasionally be descheduled past a 200 us slot, and the
* last batch may still be in the accumulation buffer when the loop ends.
* It is small because the defect this guards against is not marginal — a
* sender that decides what to send from the semaphore edge fails to consume
* essentially every batch (~100% here), so a 1% ceiling separates the two
* regimes with three orders of magnitude to spare. */
const uint32 MAX_LOST = CYCLES / 100u;
ReferenceT<RealTimeApplication> app = LoadApplication(ACC_CFG_CONTINUITY);
bool ok = app.IsValid();
if (ok) {
ok = (app->PrepareNextState("State1") == ErrorManagement::NoError);
}
Sleep::MSec(50u);
AccumRampRecorder counter;
UDPSClient client;
ReferenceT<UDPStreamer> ds;
if (ok) {
ConfigurationDatabase clientCfg;
ok = clientCfg.Write("ServerAddr", "127.0.0.1");
ok = ok && clientCfg.Write("Port", 44680u);
ok = ok && clientCfg.Write("SilenceTimeout", 0.0f);
ok = ok && clientCfg.Write("KeepAliveInterval", 0u);
client.SetListener(&counter);
ok = ok && client.Initialise(clientCfg);
ok = ok && client.Start();
}
/* Wait for the CONNECT to register on the streamer side. */
if (ok) {
ds = ObjectRegistryDatabase::Instance()->Find("Test.Data.Streamer");
ok = ds.IsValid();
}
if (ok) {
uint32 waited = 0u;
while ((waited < 3000u) && !ds->IsClientConnected()) {
Sleep::MSec(20u);
waited += 20u;
}
ok = ds->IsClientConnected();
}
/* Signal A carries the cycle index, so every packet identifies exactly
* which RT cycle produced it. Synchronise() snapshots the DataSource
* memory, so writing straight into it is equivalent to a GAM having
* produced the value. */
float64 *sigA = NULL_PTR(float64 *);
if (ok) {
void *addr = NULL_PTR(void *);
ok = ds->GetSignalMemoryBuffer(0u, 0u, addr);
sigA = reinterpret_cast<float64 *>(addr);
ok = ok && (sigA != NULL_PTR(float64 *));
}
/* Drive the RT cycles. */
if (ok) {
for (uint32 i = 0u; (i < CYCLES) && ok; i++) {
*sigA = static_cast<float64>(i);
ok = ds->Synchronise();
Sleep::Sec(CYCLE_SEC);
}
}
/* Let the last packets drain. */
Sleep::MSec(300u);
uint32 distinct = counter.DistinctCycles();
uint32 packets = counter.Packets();
uint32 duplicates = counter.Duplicates();
uint32 malformed = counter.Malformed();
uint32 dropped = (ds.IsValid()) ? ds->GetDroppedPublications() : 0u;
if (ok) {
ok = (malformed == 0u);
if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::FatalError,
"%u of %u DATA packets did not carry exactly one "
"decodable cycle index.", malformed, packets);
}
}
if (ok) {
/* A cycle that never arrives is a hole in the consumer's time series. */
ok = (distinct + MAX_LOST) >= CYCLES;
if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::FatalError,
"Accumulate lost cycles: %u of %u reached the wire "
"in %u packets (%u duplicates, %u publications "
"overwritten before being sent).",
distinct, CYCLES, packets, duplicates, dropped);
}
}
if (ok) {
/* A cycle that arrives twice means the sender re-sent a ready buffer it
* had already transmitted, which lands the same samples on the receiver
* under two different time bases. */
ok = (duplicates == 0u);
if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::FatalError,
"%u of %u DATA packets repeated a cycle already sent.",
duplicates, packets);
}
}
if (ok) {
/* Same ceiling from the producer's side: it sees the overwrite directly
* and does not depend on the packet reaching the loopback socket. */
ok = (dropped <= MAX_LOST);
if (!ok) {
REPORT_ERROR_STATIC(ErrorManagement::FatalError,
"%u of %u publications were overwritten before the "
"sender thread took them.", dropped, CYCLES);
}
}
(void) client.Stop();
ObjectRegistryDatabase::Instance()->Purge();
return ok;
}
Test/Components/DataSources/UDPStreamer/UDPStreamerTest.h
+10
View File
@@ -234,6 +234,16 @@ public:
* @brief Tests full TCP CONNECT → CONFIG → DATA via multicast → DISCONNECT on loopback. * @brief Tests full TCP CONNECT → CONFIG → DATA via multicast → DISCONNECT on loopback.
*/ */
bool TestExecute_MulticastConnectDataDisconnect(); bool TestExecute_MulticastConnectDataDisconnect();
/**
* @brief Tests that Accumulate publishes every RT cycle it batches.
* @details Drives 600 cycles at a rate the sender thread trivially keeps up
* with, and sums the numSamples field of every DATA packet that arrives.
* A batch promoted to the ready buffer but never sent — because the wake-up
* announcing it was swallowed — shows up here as missing cycles, which a
* consumer sees as a hole in the time series.
*/
bool TestAccumulate_EveryPublishedCycleReachesTheWire();
}; };
#endif /* UDPSTREAMERTEST_H_ */ #endif /* UDPSTREAMERTEST_H_ */
Test/GTest/UDPSClientGTest.cpp
+416
View File
@@ -41,6 +41,7 @@
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
#include "BasicUDPSocket.h" #include "BasicUDPSocket.h"
#include "ConfigurationDatabase.h" #include "ConfigurationDatabase.h"
#include "FastPollingMutexSem.h"
#include "InternetHost.h" #include "InternetHost.h"
#include "Sleep.h" #include "Sleep.h"
#include "UDPSClient.h" #include "UDPSClient.h"
@@ -131,6 +132,147 @@ bool WaitForClient(UDPSServer &server, uint32 timeoutMs) {
return false; return false;
} }
/*---------------------------------------------------------------------------*/
/* Fragment-reassembly test harness */
/*---------------------------------------------------------------------------*/
/** Largest reassembled payload the recording listener keeps a copy of. */
const uint32 kMaxRecordedBytes = 8192u;
/** How many reassembled payloads the recording listener keeps. */
const uint32 kMaxRecorded = 16u;
/**
* @brief Listener that records every reassembled DATA/CONFIG payload.
*
* Callbacks run on the UDPSClient receive thread; the test thread reads the
* records after a settle sleep, so both sides take the same lock.
*/
class RecordingListener: public UDPSClientListener {
public:
RecordingListener() :
dataCount(0u), configCount(0u) {
mux.Create();
}
virtual void OnUDPSData(const uint8 *payload, uint32 payloadSize) {
Record(dataPayloads, dataSizes, dataCount, payload, payloadSize);
}
virtual void OnUDPSConfig(const uint8 *payload, uint32 payloadSize) {
Record(configPayloads, configSizes, configCount, payload, payloadSize);
}
uint32 DataCount() {
(void) mux.FastLock();
uint32 n = dataCount;
mux.FastUnLock();
return n;
}
uint32 ConfigCount() {
(void) mux.FastLock();
uint32 n = configCount;
mux.FastUnLock();
return n;
}
/** @return true iff record @p idx matches @p expected byte for byte. */
bool DataMatches(uint32 idx, const uint8 *expected, uint32 expectedSize) {
return Matches(dataPayloads, dataSizes, dataCount, idx, expected,
expectedSize);
}
bool ConfigMatches(uint32 idx, const uint8 *expected, uint32 expectedSize) {
return Matches(configPayloads, configSizes, configCount, idx, expected,
expectedSize);
}
uint32 DataSize(uint32 idx) {
(void) mux.FastLock();
uint32 n = (idx < dataCount) ? dataSizes[idx] : 0u;
mux.FastUnLock();
return n;
}
private:
void Record(uint8 (&dst)[kMaxRecorded][kMaxRecordedBytes],
uint32 (&sizes)[kMaxRecorded], uint32 &count,
const uint8 *payload, uint32 payloadSize) {
(void) mux.FastLock();
if (count < kMaxRecorded) {
sizes[count] = payloadSize;
uint32 n = (payloadSize < kMaxRecordedBytes) ? payloadSize
: kMaxRecordedBytes;
memcpy(dst[count], payload, n);
count++;
}
mux.FastUnLock();
}
bool Matches(uint8 (&src)[kMaxRecorded][kMaxRecordedBytes],
uint32 (&sizes)[kMaxRecorded], uint32 &count, uint32 idx,
const uint8 *expected, uint32 expectedSize) {
(void) mux.FastLock();
bool ok = (idx < count) && (sizes[idx] == expectedSize) &&
(expectedSize <= kMaxRecordedBytes) &&
(memcmp(src[idx], expected, expectedSize) == 0);
mux.FastUnLock();
return ok;
}
FastPollingMutexSem mux;
uint8 dataPayloads[kMaxRecorded][kMaxRecordedBytes];
uint32 dataSizes[kMaxRecorded];
uint32 dataCount;
uint8 configPayloads[kMaxRecorded][kMaxRecordedBytes];
uint32 configSizes[kMaxRecorded];
uint32 configCount;
};
/** Fill @p buf with a position-dependent pattern so misplacement is visible. */
void FillPattern(uint8 *buf, uint32 n, uint8 seed) {
for (uint32 i = 0u; i < n; i++) {
buf[i] = static_cast<uint8>((i * 7u) + seed);
}
}
/** Send one UDPS fragment datagram to 127.0.0.1:@p dstPort. */
bool SendFragment(BasicUDPSocket &sock, uint16 dstPort, uint8 type,
uint32 counter, uint16 fragIdx, uint16 totalFrags,
const uint8 *payload, uint32 payloadBytes) {
uint8 buf[UDPS_HEADER_SIZE + 2048u];
if (payloadBytes > 2048u) {
return false;
}
UDPSBuildHeader(buf, type, counter, fragIdx, totalFrags, payloadBytes);
memcpy(&buf[UDPS_HEADER_SIZE], payload, payloadBytes);
InternetHost dst(dstPort, "127.0.0.1");
(void) sock.SetDestination(dst);
uint32 n = UDPS_HEADER_SIZE + payloadBytes;
return sock.Write(reinterpret_cast<const char8 *>(buf), n);
}
/**
* @brief Bring up a UDPSClient pointed at @p server and learn the ephemeral
* port it receives DATA on (the source port of its CONNECT).
*
* Silence timeout and keepalive are disabled so the session never churns
* underneath the fragments the test injects.
*/
bool StartClientAndLearnPort(UDPSClient &client, ConfigurationDatabase &cfg,
BasicUDPSocket &server, uint16 serverPort,
uint16 &clientPort) {
if (!cfg.Write("ServerAddr", "127.0.0.1")) { return false; }
if (!cfg.Write("Port", static_cast<uint32>(serverPort))) { return false; }
if (!cfg.Write("SilenceTimeout", 0.0f)) { return false; }
if (!cfg.Write("KeepAliveInterval", 0u)) { return false; }
if (!client.Initialise(cfg)) { return false; }
if (!client.Start()) { return false; }
uint8 type = 0xFFu;
if (!WaitDatagram(server, 3000, type, clientPort)) { return false; }
return (type == UDPS_TYPE_CONNECT) && (clientPort != 0u);
}
} // namespace } // namespace
/*---------------------------------------------------------------------------*/ /*---------------------------------------------------------------------------*/
@@ -346,3 +488,277 @@ TEST(UDPSClientGTest, TestSilenceTimeoutSubSecondTriggersReconnect) {
client.Stop(); client.Stop();
server.Close(); server.Close();
} }
TEST(UDPSClientGTest, TestReorderedFragmentsAreReassembled) {
/* UDP gives no ordering guarantee: the fragments of one packet may arrive
* in any order, with nothing lost. Reassembly must not depend on fragment
* 0 arriving first — if it does, an out-of-order burst destroys a packet
* whose bytes all arrived, and leaves a slot occupied until the 2 s GC,
* which is how four slots end up permanently full. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
/* 20-byte payload over three 8-byte chunks: the last one is short, which
* is exactly why chunk size has to be learnt from a non-last fragment. */
uint8 expected[20];
FillPattern(expected, sizeof(expected), 3u);
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 7u, 1u, 3u,
&expected[8], 8u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 7u, 2u, 3u,
&expected[16], 4u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 7u, 0u, 3u,
&expected[0], 8u));
Sleep::MSec(400u);
ASSERT_EQ(listener.DataCount(), 1u)
<< "no fragment was lost, yet the packet was not delivered";
EXPECT_EQ(listener.DataSize(0u), 20u);
EXPECT_TRUE(listener.DataMatches(0u, expected, sizeof(expected)));
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestDataAndConfigWithSameCounterDoNotCollide) {
/* DATA and CONFIG carry independent counter sequences, so the same counter
* value legitimately appears on both. A reassembly slot keyed on the
* counter alone merges the two streams: one payload is delivered under the
* wrong type and the other is silently dropped. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
uint8 dataPayload[16];
uint8 cfgPayload[16];
FillPattern(dataPayload, sizeof(dataPayload), 11u);
FillPattern(cfgPayload, sizeof(cfgPayload), 200u);
/* Same counter (42), interleaved, two fragments each. */
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_CONFIG, 42u, 0u, 2u,
&cfgPayload[0], 8u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 42u, 0u, 2u,
&dataPayload[0], 8u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_CONFIG, 42u, 1u, 2u,
&cfgPayload[8], 8u));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 42u, 1u, 2u,
&dataPayload[8], 8u));
Sleep::MSec(400u);
EXPECT_EQ(listener.ConfigCount(), 1u);
EXPECT_TRUE(listener.ConfigMatches(0u, cfgPayload, sizeof(cfgPayload)));
ASSERT_EQ(listener.DataCount(), 1u)
<< "the DATA packet was swallowed by the CONFIG slot sharing its counter";
EXPECT_TRUE(listener.DataMatches(0u, dataPayload, sizeof(dataPayload)));
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestDuplicateHighIndexFragmentDoesNotFakeCompletion) {
/* Completion is decided by counting fragments, with a received-bitmask to
* reject duplicates. If the mask is narrower than the fragment count the
* client accepts, a duplicated high-index fragment is counted twice and
* the packet is delivered while a fragment is still missing — a payload
* with a hole of stale bytes, reported as valid. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
/* 300 fragments — past the 256 a 32-byte mask covers, but well inside the
* 512 the client's own sanity check permits. */
const uint16 kTotalFrags = 300u;
const uint32 kChunk = 8u;
const uint32 kLastChunk = 4u;
const uint32 kTotalBytes = ((kTotalFrags - 1u) * kChunk) + kLastChunk;
uint8 expected[((kTotalFrags - 1u) * kChunk) + kLastChunk];
FillPattern(expected, kTotalBytes, 5u);
/* Everything except the final fragment, plus one duplicate above 255. */
for (uint16 f = 0u; f < (kTotalFrags - 1u); f++) {
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 9u, f,
kTotalFrags, &expected[f * kChunk], kChunk));
}
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 9u, 260u,
kTotalFrags, &expected[260u * kChunk], kChunk));
Sleep::MSec(500u);
ASSERT_EQ(listener.DataCount(), 0u)
<< "delivered with a fragment still missing (a duplicate was counted "
"as a new fragment)";
/* The genuinely missing fragment completes it, with the right bytes. */
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 9u,
kTotalFrags - 1u, kTotalFrags,
&expected[(kTotalFrags - 1u) * kChunk],
kLastChunk));
Sleep::MSec(400u);
ASSERT_EQ(listener.DataCount(), 1u);
EXPECT_EQ(listener.DataSize(0u), kTotalBytes);
EXPECT_TRUE(listener.DataMatches(0u, expected, kTotalBytes));
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestStaleDataPacketIsNotDelivered) {
/* A DATA packet that arrives after a newer one has already been delivered
* carries an older time base. Delivering it makes the consumer place its
* samples behind the ones it has: they collide with what is already
* plotted, and the range they should have occupied stays empty. The
* counter is the only thing that tells the two apart, so the client must
* drop anything that does not advance it. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
uint8 pkt[8];
FillPattern(pkt, sizeof(pkt), 1u);
/* 10 and 11 advance the counter; 9 and the repeat of 11 do not. */
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 10u, 0u, 1u,
pkt, sizeof(pkt)));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 11u, 0u, 1u,
pkt, sizeof(pkt)));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 9u, 0u, 1u,
pkt, sizeof(pkt)));
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 11u, 0u, 1u,
pkt, sizeof(pkt)));
Sleep::MSec(400u);
EXPECT_EQ(listener.DataCount(), 2u)
<< "a packet older than one already delivered reached the listener";
EXPECT_EQ(client.GetStaleDataPackets(), 2u);
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestCounterGapIsReported) {
/* Consumers that infer a sample period from the sender-clock gap need to
* know how many packets that gap spans; without it a single loss reads as
* a halved rate. The gap comes from the counter, and must exclude the
* packet being delivered. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
uint8 pkt[8];
FillPattern(pkt, sizeof(pkt), 2u);
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 100u, 0u, 1u,
pkt, sizeof(pkt)));
Sleep::MSec(200u);
EXPECT_EQ(client.GetLastDataGap(), 0u) << "the first packet lost nothing";
/* 101, 102 and 103 never arrive. */
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 104u, 0u, 1u,
pkt, sizeof(pkt)));
Sleep::MSec(200u);
EXPECT_EQ(client.GetLastDataGap(), 3u);
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA, 105u, 0u, 1u,
pkt, sizeof(pkt)));
Sleep::MSec(200u);
EXPECT_EQ(client.GetLastDataGap(), 0u) << "the gap must not persist";
EXPECT_EQ(listener.DataCount(), 3u);
EXPECT_EQ(client.GetStaleDataPackets(), 0u);
client.Stop();
server.Close();
}
TEST(UDPSClientGTest, TestCounterWraparoundDoesNotRejectStream) {
/* The counter is a uint32 that wraps. Ordering it by plain comparison
* would call every packet after the wrap older than 0xFFFFFFFF and reject
* the stream permanently, so the ordering has to be done on the signed
* difference. */
BasicUDPSocket server;
ASSERT_TRUE(server.Open());
ASSERT_TRUE(server.Listen(0u));
uint16 serverPort = GetBoundPort(server);
ASSERT_NE(serverPort, 0u);
RecordingListener listener;
UDPSClient client;
client.SetListener(&listener);
ConfigurationDatabase cfg;
uint16 clientPort = 0u;
ASSERT_TRUE(StartClientAndLearnPort(client, cfg, server, serverPort,
clientPort));
uint8 pkt[8];
FillPattern(pkt, sizeof(pkt), 4u);
const uint32 counters[4] = { 0xFFFFFFFEu, 0xFFFFFFFFu, 0u, 1u };
for (uint32 i = 0u; i < 4u; i++) {
ASSERT_TRUE(SendFragment(server, clientPort, UDPS_TYPE_DATA,
counters[i], 0u, 1u, pkt, sizeof(pkt)));
Sleep::MSec(150u);
}
EXPECT_EQ(listener.DataCount(), 4u)
<< "the stream was rejected across the counter wrap";
EXPECT_EQ(client.GetStaleDataPackets(), 0u);
EXPECT_EQ(client.GetLastDataGap(), 0u);
client.Stop();
server.Close();
}
docs/superpowers/plans/2026-08-27-udpscope.md
+17 -139
View File
@@ -128,23 +128,6 @@ TEST(MinMaxDecimate, PreservesTheExtremesOfEveryBucket) {
// A ring whose timestamps are not monotonic breaks any later binary search by // A ring whose timestamps are not monotonic breaks any later binary search by
// time, so the pair emitted per bucket must be ordered by time, not by value. // time, so the pair emitted per bucket must be ordered by time, not by value.
TEST(MinMaxDecimate, EmitsPointsInTimeOrder) { TEST(MinMaxDecimate, EmitsPointsInTimeOrder) {
// Two buckets of four. In the first the minimum comes before the maximum,
// in the second the order is reversed. An implementation that emitted
// (min, max) by value rather than by time passes on bucket 0 and fails on
// bucket 1, so this data exercises the swap that a monotonically growing
// ramp never triggers.
const double st[8] = {0, 1, 2, 3, 4, 5, 6, 7};
const double sv[8] = {-5, 0, 0, 9, 9, 0, 0, -5};
Series pair;
MinMaxDecimate(st, sv, 8, 4, pair);
ASSERT_EQ(pair.size(), 4u);
const double wantT[4] = {0, 3, 4, 7};
const double wantV[4] = {-5, 9, 9, -5};
for (size_t i = 0; i < 4; i++) {
EXPECT_EQ(pair.t[i], wantT[i]) << "time at " << i;
EXPECT_EQ(pair.v[i], wantV[i]) << "value at " << i;
}
std::vector<double> t(400), v(400); std::vector<double> t(400), v(400);
for (size_t i = 0; i < t.size(); i++) { for (size_t i = 0; i < t.size(); i++) {
t[i] = static_cast<double>(i); t[i] = static_cast<double>(i);
@@ -379,11 +362,7 @@ else()
message(WARNING "Bundled Font Awesome missing — using ASCII icon fallbacks") message(WARNING "Bundled Font Awesome missing — using ASCII icon fallbacks")
endif() endif()
# Guarded: file(COPY) is a hard configure error on a missing source, which
# would defeat the fallback the block above just chose.
if(EXISTS ${FONT_DIR})
file(COPY ${FONT_DIR} DESTINATION ${CMAKE_BINARY_DIR}/resources) file(COPY ${FONT_DIR} DESTINATION ${CMAKE_BINARY_DIR}/resources)
endif()
# ── Core library: everything except main.cpp, so tests can link it ──────────── # ── Core library: everything except main.cpp, so tests can link it ────────────
set(CORE_SOURCES set(CORE_SOURCES
@@ -1043,24 +1022,8 @@ TEST(ClockOffset, HoldsTheOffsetThroughSmallArrivalJitter) {
ClockOffset off; ClockOffset off;
off.map(10.0, 1000.0); // offset = 990 off.map(10.0, 1000.0); // offset = 990
// Arrival wanders either side of the prediction. wallSec is a local receive EXPECT_DOUBLE_EQ(off.map(11.0, 1001.02), 1001.0);
// timestamp, so it only ever advances — jitter shows up as the gap growing EXPECT_DOUBLE_EQ(off.map(12.0, 1000.97), 1002.0);
// and shrinking, never as the clock going backwards.
EXPECT_DOUBLE_EQ(off.map(11.0, 1001.02), 1001.0); // +0.02 late
EXPECT_DOUBLE_EQ(off.map(12.0, 1001.97), 1002.0); // -0.03 early
}
// The threshold has to be symmetric. A producer whose clock steps FORWARD (an
// NTP correction on the producer's host, say) puts the prediction permanently
// ahead of the wall clock — a one-sided "recalibrate only when wall is ahead"
// test never fires for it, and the trace sits in the future for the rest of the
// run.
TEST(ClockOffset, RecalibratesWhenTheProducerClockJumpsForward) {
ClockOffset off;
off.map(10.0, 1000.0); // offset = 990
// Producer leaps 100 s ahead while only 1 s of wall time passes.
EXPECT_DOUBLE_EQ(off.map(111.0, 1001.0), 1001.0);
} }
TEST(ClockOffset, RecalibratesWhenDriftExceedsTheThreshold) { TEST(ClockOffset, RecalibratesWhenDriftExceedsTheThreshold) {
@@ -1609,42 +1572,6 @@ TEST(FrameDecoder, AccumulatedScalarSurvivesBurstyDelivery) {
} }
} }
// ADDED in Task 4 review. The counterweight to the test above: suppressing
// arrival jitter by chaining bursts is only safe while the chain is CHECKED. On
// UDP packets are lost, and an unchecked chain closes the hole silently and
// dates every later sample early for the rest of the run.
TEST(FrameDecoder, AccumulatedScalarResynchronisesAfterLostPackets) {
FrameDecoder dec;
SignalMeta m;
m.name = "Acc";
m.typeCode = 9;
m.numRows = 1;
m.samplingRate = 1000.0; /* 10 samples = 10 ms per packet */
dec.setSignals({m});
std::vector<double> ts;
for (int p = 0; p < 10; p++) {
FrameBuilder fb;
fb.addSignal(std::vector<double>(10, 1.0));
const FrameView& f = fb.build(0, 500.0 + p * 0.010, 10);
dec.beginFrame(f);
ASSERT_TRUE(dec.timestamps(f, 0, ts));
}
EXPECT_NEAR(ts[9], 500.090, 1e-9);
/* A full second of packets never arrives. The next one lands at 501.100. */
FrameBuilder fb;
fb.addSignal(std::vector<double>(10, 1.0));
const FrameView& f = fb.build(0, 501.100, 10);
dec.beginFrame(f);
ASSERT_TRUE(dec.timestamps(f, 0, ts));
/* Chaining blindly would put this burst at 500.091..500.100, as though no
* data were missing. */
EXPECT_NEAR(ts[0], 501.091, 1e-9);
EXPECT_NEAR(ts[9], 501.100, 1e-9);
}
TEST(FrameDecoder, AccumulatedScalarDerivesDtFromTheHrtGapWhenNoRateIsDeclared) { TEST(FrameDecoder, AccumulatedScalarDerivesDtFromTheHrtGapWhenNoRateIsDeclared) {
FrameDecoder dec; FrameDecoder dec;
SignalMeta m; SignalMeta m;
@@ -1918,44 +1845,8 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
return true; return true;
} }
/* Rule 3: accumulated scalar. /* Rule 3: accumulated scalar, based on the producer's own hrt. */
*
* AMENDED after Task 4 review. The version below originally sent EVERY
* accumulated scalar through the hrt fit, falling back to packetBurst until
* the fit was ready. That cannot work when a declared samplingRate is
* present: HrtRateFit needs 32 packets, bursty delivery can begin before
* that, and packetBurst then crams a 10 ms burst into a 50 us arrival gap —
* exactly the sawtooth this rule exists to prevent. Worse, HrtRateFit fits
* hrt against ARRIVAL time, so a burst episode corrupts the very rate the
* fallback is waiting on.
*
* With a declared rate none of that is needed: the intra-packet step is
* exact, and bursts are contiguous, so the next burst is PREDICTED at
* lastEmittedEnd + dt. The prediction must be checked, not trusted — a pure
* chain silently closes the hole left by a lost datagram and dates every
* later sample early for the rest of the run. So each packet compares the
* prediction against the arrival anchor and abandons it beyond
* kBurstResyncThresholdS. The hrt path below remains for samplingRate == 0. */
if (d.numElements() == 1u && nElems > 1u) { if (d.numElements() == 1u && nElems > 1u) {
const double dtDeclared = (d.samplingRate > 0.0) ? (1.0 / d.samplingRate) : 0.0;
if (d.samplingRate > 0.0) {
const double arrivalAnchor =
wallNow - static_cast<double>(nElems - 1u) * dtDeclared;
double base = arrivalAnchor;
if (st.lastEmittedValid) {
const double predicted = st.lastEmittedEnd + dtDeclared;
if (std::fabs(predicted - arrivalAnchor) <= kBurstResyncThresholdS) {
base = predicted;
}
}
tsOut.resize(nElems);
for (uint32_t e = 0; e < nElems; e++) {
tsOut[e] = base + static_cast<double>(e) * dtDeclared;
}
st.lastEmittedEnd = tsOut[nElems - 1u];
st.lastEmittedValid = true;
return true;
}
if (!hrtFit_.ready()) { if (!hrtFit_.ready()) {
return packetBurst(idx, nElems, wallNow, tsOut); return packetBurst(idx, nElems, wallNow, tsOut);
} }
@@ -1963,7 +1854,9 @@ bool FrameDecoder::timestamps(const FrameView& f, uint32_t idx,
const double base = st.offset.map(hrtSec, wallNow); const double base = st.offset.map(hrtSec, wallNow);
double dt; double dt;
if (st.lastAccValid && st.prevAccCount > 0u && if (d.samplingRate > 0.0) {
dt = 1.0 / d.samplingRate;
} else if (st.lastAccValid && st.prevAccCount > 0u &&
hrtSec > st.lastAccHrtSec) { hrtSec > st.lastAccHrtSec) {
/* The flushes carry contiguous RT cycles, so the gap divided by the /* The flushes carry contiguous RT cycles, so the gap divided by the
* previous packet's sample count is exactly one cycle period. */ * previous packet's sample count is exactly one cycle period. */
@@ -2015,7 +1908,7 @@ cd Client/udpscope && cmake --build build -j && ./build/udpscope_tests --gtest_f
Expected: PASS, 9 tests. Expected: PASS, 9 tests.
If `AccumulatedScalarSurvivesBurstyDelivery` fails, do NOT reach for the hrt fit: with a declared `samplingRate` rule 3 never consults it, precisely because the fit is not ready for the first 32 packets and is itself corrupted by bursty arrivals. Check instead that `lastEmittedEnd`/`lastEmittedValid` are being updated on every emitted burst. The only test that may legitimately fall through to rule 4 early is `AccumulatedScalarDerivesDtFromTheHrtGapWhenNoRateIsDeclared`, whose arrivals are uniform, so `packetBurst` is accurate there. If `AccumulatedScalarSurvivesBurstyDelivery` fails on the first few samples, check that `beginFrame()` is being called before `timestamps()` — the hrt fit needs 32 packets before rule 3 engages, and the packets before that legitimately go through rule 4.
- [ ] **Step 8: Commit** - [ ] **Step 8: Commit**
@@ -3006,19 +2899,7 @@ TEST(SignalStore, ConcurrentPushAndReadDoNotCrash) {
} }
stop.store(true); stop.store(true);
writer.join(); writer.join();
SUCCEED();
/* Not just "it did not crash": the store must still be coherent after the
race. The window is 0.05 s at 1 MHz, so the ring spans at most
0.05 * kRingMargin seconds, and readLast was capped at 4096 points. */
double oldest = 0.0, newest = 0.0;
ASSERT_TRUE(s.span("a", oldest, newest));
EXPECT_GE(newest, oldest);
EXPECT_LE(newest - oldest, 0.05 * SignalStore::kRingMargin * 1.5);
EXPECT_LE(out.size(), 4096u);
EXPECT_EQ(out.t.size(), out.v.size());
for (size_t i = 1; i < out.size(); ++i) {
EXPECT_GE(out.t[i], out.t[i - 1]) << "timestamps went backwards at " << i;
}
} }
``` ```
@@ -4257,7 +4138,7 @@ TEST(ReceiverLink, StopIsSafeWhenNeverStarted) {
Receiver rx(store); Receiver rx(store);
rx.stop(); rx.stop();
EXPECT_FALSE(rx.running()); EXPECT_FALSE(rx.running());
EXPECT_FALSE(rx.link().running); SUCCEED();
} }
``` ```
@@ -5257,6 +5138,10 @@ void App::drawMenuBar() {
} }
ImGui::EndMenu(); ImGui::EndMenu();
} }
if (ImGui::BeginMenu("View")) {
ImGui::MenuItem("(cursors land in Task 13)", nullptr, false, false);
ImGui::EndMenu();
}
if (ImGui::BeginMenu("Help")) { if (ImGui::BeginMenu("Help")) {
ImGui::MenuItem("UDPScope — direct UDPS oscilloscope", nullptr, false, false); ImGui::MenuItem("UDPScope — direct UDPS oscilloscope", nullptr, false, false);
ImGui::EndMenu(); ImGui::EndMenu();
@@ -6089,9 +5974,8 @@ In `Client/udpscope/App.h`, add `#include "PaneView.h"` and the members:
double xSpanSec_ = 1.0; /**< live window width, Task 12 makes it settable */ double xSpanSec_ = 1.0; /**< live window width, Task 12 makes it settable */
``` ```
In `Client/udpscope/SignalList.cpp` — that is where Task 9 put In `Client/udpscope/App.cpp`, make the signal list a drag source by replacing
`App::drawSignalList()`, not `App.cpp` — make the signal list a drag source by the `ImGui::Selectable(m.name.c_str());` line with:
replacing the `ImGui::Selectable(m.name.c_str());` line with:
```cpp ```cpp
ImGui::Selectable(m.name.c_str()); ImGui::Selectable(m.name.c_str());
@@ -6151,14 +6035,10 @@ set(CORE_SOURCES
set(APP_SOURCES set(APP_SOURCES
main.cpp main.cpp
App.cpp App.cpp
SignalList.cpp
PaneView.cpp PaneView.cpp
) )
``` ```
`SignalList.cpp` stays in the list — it holds `App::drawSignalList()` and
dropping it is a link error, not a warning.
`PaneView.cpp` needs ImGui headers, so it belongs to the executable, not the `PaneView.cpp` needs ImGui headers, so it belongs to the executable, not the
core library — that is what keeps `udpscope_tests` free of a GUI dependency. core library — that is what keeps `udpscope_tests` free of a GUI dependency.
@@ -7097,8 +6977,7 @@ and after `paneView_.drawTree(...)`:
} }
``` ```
There is no View menu yet — Task 9 built only File and Help. Add one to Add the live control to the View menu in `drawMenuBar()`:
`drawMenuBar()`, between the File and Help blocks:
```cpp ```cpp
if (ImGui::BeginMenu("View")) { if (ImGui::BeginMenu("View")) {
@@ -9290,8 +9169,7 @@ TEST(PaneTree, CloneIsADeepCopy) {
- [ ] **Step 8: Add the File menu and save on exit** - [ ] **Step 8: Add the File menu and save on exit**
In `App::drawMenuBar()`, **replace** the File menu block Task 9 wrote (the one In `App::drawMenuBar()`, before the View menu:
whose only item is Quit) — do not add a second one:
```cpp ```cpp
if (ImGui::BeginMenu("File")) { if (ImGui::BeginMenu("File")) {