fix(StreamHub): correct UTF-8 tail repair to run only after truncation

The previous walk-back in SetCalibrationEntry was unconditional, corrupting short valid units ending in multi-byte characters (e.g. Omega, mu, degree). Also failed to drop an orphaned lead byte left after stripping continuation bytes. Restructured to use a 256-byte staging buffer so truncation can be detected, then repair runs only in the truncation branch. Algorithm now matches Go CalConfig.Normalise() exactly: scan back over continuation bytes (up to 3), find the lead byte, derive expected sequence length, cut if incomplete. Covers all cases: orphaned continuation, orphaned lead, cut on lead byte. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-08-17 00:02:33 +02:00
parent 2f9b135c62
commit 93e00d0c21
2 changed files with 120 additions and 22 deletions
@@ -132,3 +132,60 @@ Step 9 (source file with whitespace-padded source/signal and `[0]` suffix):
 [StreamHub][Information]: StreamHub: initialised with 1 session(s), WSPort=8099, MaxPoints=20000, PushRate=30 Hz.
 ```
 Entry loaded correctly (trimmed to `wave`/`Sine`, `[0]` stripped).
+
+## Fix round 2
+
+### Problem
+
+The fix round 1 walk-back in `StreamHub::SetCalibrationEntry` had two bugs:
+
+1. It ran unconditionally, not only after truncation. A valid short unit ending in a multi-byte character (e.g. `"Ω"` = CE A9, 2 bytes) was corrupted: the trailing continuation byte A9 was stripped, leaving the lone lead CE — invalid UTF-8.
+2. It only stripped continuation bytes, never an orphaned lead byte. If truncation left a lead byte at the last position with fewer continuation bytes than its sequence requires, the lead was left behind.
+
+### Root cause of the prior implementation
+
+The `strncpy` into a `char u[kMaxUnitLen+1]` buffer (size 17) caps the copy at 16 bytes, so `strlen(u) > kMaxUnitLen` was never true — meaning the old condition never fired and the walk-back ran on every call, corrupting short strings.
+
+### Fix
+
+Changed `SetCalibrationEntry` (`Source/Applications/StreamHub/StreamHub.cpp`) to:
+
+1. Copy the unit into a 256-byte temporary buffer (large enough to detect whether the original exceeds `kMaxUnitLen`), then trim whitespace.
+2. If the trimmed length is `<= kMaxUnitLen`: copy verbatim, no repair. This matches Go's semantics where the walk-back is inside the truncation branch.
+3. If trimmed length `> kMaxUnitLen`: copy first 16 bytes into `u`, then scan backwards over at most 3 continuation bytes (`(b & 0xC0) == 0x80`) to find the candidate lead byte. Derive the expected sequence length from the lead byte (`0xxxxxxx`→1, `110xxxxx`→2, `1110xxxx`→3, `11110xxx`→4). If bytes present (`cont + 1`) is fewer than expected, cut at the lead byte. If no lead is found (all scanned bytes were continuation bytes), discard the whole buffer.
+
+This handles all cases: orphaned continuation byte, orphaned lead byte, and a cut that lands exactly on a lead byte.
+
+### Build output
+
+```
+make -f Makefile.gcc apps
+```
+Compiled cleanly with `-std=c++98 -Wall -Werror`, no warnings.
+
+### GTest output
+
+```
+[==========] 132 tests from 12 test cases ran.
+[  PASSED  ] 128 tests.
+[  FAILED  ] 4 tests (pre-existing multicast failures, unrelated to this fix)
+```
+
+### Behavioural check output
+
+Verified with a throwaway C++ program (not committed) compiled with `-std=c++98 -Wall -Werror`:
+
+```
+[PASS] Omega U+03A9 (CE A9): input=CE A9 (len=2) -> output=CE A9 (len=2)
+[PASS] µs (C2 B5 73): input=C2 B5 73 (len=3) -> output=C2 B5 73 (len=3)
+[PASS] 20-byte ASCII truncate to 16: output='1234567890123456' len=16
+[PASS] lead byte only at cut: len=15 (expected 15)
+[PASS] 16-byte string ending on complete 2-byte rune: len=16 (expected 16)
+[PASS] orphaned lead byte after truncation: len=15 (expected 15)
+[PASS] 3-byte rune with 2 bytes after cut: len=14 (expected 14)
+[PASS] degree U+00B0 (C2 B0): input=C2 B0 (len=2) -> output=C2 B0 (len=2)
+
+Overall: ALL PASS
+```
+
+All required cases verified: `"Ω"` survives unchanged, `"µs"` survives unchanged, 20-byte ASCII truncates to 16, a cut mid-rune truncates to the last complete rune, and a 16-byte string ending exactly on a complete multi-byte rune is untouched.