FIrs

2026-05-15 17:42:14 +02:00
commit e3389f932b
40 changed files with 7622 additions and 0 deletions
@@ -0,0 +1,359 @@
+# Tutorial: Streaming Signals with UDPStreamer
+
+This tutorial walks you through:
+
+1. Setting up the environment
+2. Running the demo application
+3. Visualising signals in the browser
+4. Adding UDPStreamer to your own MARTe2 application
+5. Streaming high-frequency packed signals
+
+**Prerequisites:** MARTe2 and MARTe2-components must already be built.
+See the [MARTe2 installation guide](https://vcis.f4e.europa.eu/marte2-docs/) if needed.
+
+---
+
+## 1. Environment Setup
+
+Edit `marte_env.sh` in the repository root to point at your MARTe2 installations:
+
+```bash
+# marte_env.sh (key variables)
+export MARTe2_DIR="$HOME/workspace/MARTe2"
+export MARTe2_Components_DIR="$HOME/workspace/MARTe2-components"
+```
+
+Then source it in your shell:
+
+```bash
+cd /path/to/MARTe_IO_components
+source marte_env.sh
+```
+
+Verify the environment is correct:
+
+```bash
+echo $MARTe2_DIR
+ls $MARTe2_DIR/Build/x86-linux/App/MARTeApp.ex    # should exist
+```
+
+---
+
+## 2. Running the Demo Application
+
+The demo is in `Test/MARTeApp/`. It runs a 10 kHz MARTe2 application that streams:
+
+- **Counter** and **Time** — scalar counters from the Linux timer
+- **Sine1** — 1 Hz sine wave (float32, amplitude 10, quantized to uint16 on wire)
+- **Sine2** — 0.3 Hz sine wave (float32, amplitude 5, raw float32 on wire)
+- **Ch1**, **Ch2** — 1 kHz sine bursts packed as 1000 samples/packet (10 MSps)
+
+Start everything with one command:
+
+```bash
+cd Test/MARTeApp
+./run.sh --webui
+```
+
+The script will:
+1. Build the UDPStreamer shared library.
+2. Build the Go WebUI binary (first run only).
+3. Start the WebUI relay on `http://localhost:8080`.
+4. Launch the MARTe2 application.
+
+Press `Ctrl+C` to stop both processes.
+
+---
+
+## 3. Visualising Signals in the Browser
+
+Open `http://localhost:8080` in any modern browser.
+
+### Add your first plot
+
+1. Click **+ Add Plot** in the toolbar.
+2. A blank plot panel appears with a "Drop signals here" hint.
+
+### Plot a signal
+
+1. In the left sidebar find **Sine1** (listed as `Sine1 · f32`).
+2. Click and drag it onto the plot panel.
+3. The sine wave appears immediately.
+
+### Overlay multiple signals
+
+Drag **Sine2** onto the same plot — it is added as a second trace.
+
+### Adjust the time window
+
+Use the **Window** dropdown in the top bar to change the rolling display window
+(1 s, 5 s, 10 s, 30 s, 60 s).
+
+### Plot layout
+
+Use the layout buttons (`1×1`, `2×1`, `2×2`, …) to split the screen into multiple
+plot panels. Each panel is independent — drag different signals onto each.
+
+### High-frequency signals
+
+Drag **Ch1** (shown as `Ch1 · [1000] f32`) onto a plot. Each UDP packet carries
+1000 samples at 10 MSps; the WebUI reconstructs per-sample timestamps and displays
+the continuous waveform.
+
+### Export data
+
+Click **⬇** on any plot to download the visible window as a CSV file.
+
+---
+
+## 4. Adding UDPStreamer to Your Own Application
+
+### Step 1 — Declare the DataSource
+
+Add UDPStreamer to the `+Data` section of your MARTe2 configuration:
+
+```
+Data = {
+    Class = ReferenceContainer
+    DefaultDataSource = DDB
+
+    +DDB = { Class = GAMDataSource }
+
+    +Streamer = {
+        Class          = UDPStreamer
+        Port           = 44500
+        MaxPayloadSize = 1400
+
+        Signals = {
+            Voltage = {
+                Type     = float32
+                Unit     = "V"
+                RangeMin = -10.0
+                RangeMax = 10.0
+                QuantizedType = uint16      // 16-bit quantized on wire
+            }
+            Current = {
+                Type = float32
+                Unit = "A"
+            }
+        }
+    }
+
+    +Timings = { Class = TimingDataSource }
+}
+```
+
+### Step 2 — Route signals with IOGAM
+
+Use IOGAM to copy signals from your inter-GAM DDB into the Streamer:
+
+```
+StreamerGAM = {
+    Class = IOGAM
+    InputSignals = {
+        Voltage = { DataSource = DDB; Type = float32 }
+        Current = { DataSource = DDB; Type = float32 }
+    }
+    OutputSignals = {
+        Voltage = { DataSource = Streamer; Type = float32 }
+        Current = { DataSource = Streamer; Type = float32 }
+    }
+}
+```
+
+Add `StreamerGAM` at the **end** of the thread's `Functions` list so it runs after
+your control GAMs have written their outputs.
+
+### Step 3 — Add the library to LD_LIBRARY_PATH
+
+In your run script, add the UDPStreamer build directory:
+
+```bash
+export LD_LIBRARY_PATH="/path/to/Build/x86-linux/Components/DataSources/UDPStreamer:$LD_LIBRARY_PATH"
+```
+
+### Step 4 — Start the WebUI and connect
+
+```bash
+# From the Client/WebUI directory:
+./udpstreamer-webui --streamer 127.0.0.1:44500 --listen :8080 --clientport 44900
+```
+
+Open `http://localhost:8080`, drag your signals onto a plot, and you're done.
+
+---
+
+## 5. Streaming High-Frequency Packed Signals
+
+This section shows how to stream 1000 samples per RT cycle at 1 MSps.
+
+### Overview
+
+At 1 kHz RT rate with 1000 samples per cycle the effective sample rate is 1 MSps.
+Each UDP packet carries a burst of 1000 samples; the client reconstructs timestamps
+using the anchor timestamp and `SamplingRate`.
+
+### Step 1 — Generate burst data with SineArrayGAM
+
+`SineArrayGAM` (bundled in `UDPStreamer.so`) produces a continuous float32 array:
+
+```
+Ch1GAM = {
+    Class        = SineArrayGAM
+    Frequency    = 1000.0       // 1 kHz signal
+    Amplitude    = 1.0
+    Phase        = 0.0
+    SamplingRate = 1000000.0    // must match Streamer config below
+    OutputSignals = {
+        Ch1 = {
+            DataSource         = DDB
+            Type               = float32
+            NumberOfDimensions = 1
+            NumberOfElements   = 1000
+        }
+    }
+}
+```
+
+### Step 2 — Add a time reference signal
+
+Add a scalar time signal that will anchor the first sample's timestamp:
+
+```
+TimerGAM = {
+    Class = IOGAM
+    InputSignals = {
+        Time = { DataSource = Timer; Type = uint32; Frequency = 1000 }
+    }
+    OutputSignals = {
+        Time = { DataSource = DDB; Type = uint32 }
+    }
+}
+```
+
+### Step 3 — Configure UDPStreamer for packed signals
+
+```
+Streamer = {
+    Class          = UDPStreamer
+    Port           = 44500
+    MaxPayloadSize = 1400
+
+    Signals = {
+        Time = { Type = uint32; Unit = "us" }   // time reference (scalar)
+
+        Ch1 = {
+            Type               = float32
+            NumberOfDimensions = 1
+            NumberOfElements   = 1000
+            Unit               = "V"
+            TimeMode           = FirstSample    // Time = timestamp of first sample
+            TimeSignal         = Time
+            SamplingRate       = 1000000.0      // Hz
+        }
+    }
+}
+```
+
+### Step 4 — Wire everything in the thread
+
+```
+Thread1 = {
+    Class     = RealTimeThread
+    Functions = { TimerGAM Ch1GAM StreamerGAM }
+}
+```
+
+Where `StreamerGAM` is the IOGAM that copies `Time` and `Ch1` from DDB to Streamer.
+
+### Step 5 — Fragmentation note
+
+A single 1000-element float32 channel plus a uint32 time signal produces:
+
+```
+payload = 8 B (HRT) + 4 B (Time/uint32) + 4000 B (float32×1000) = 4012 B
+```
+
+With `MaxPayloadSize = 1400`:
+
+```
+fragments = ceil(4012 / 1383) = 3 datagrams per cycle
+```
+
+At 1 kHz that is 3000 UDP datagrams/second per channel — well within typical LAN capacity.
+
+### Step 6 — Create the SineArrayGAM symlink
+
+MARTe2 tries to `dlopen("SineArrayGAM.so")` the first time it encounters the class.
+Create the symlink in your build directory:
+
+```bash
+UDPSTREAMER_LIB=/path/to/Build/x86-linux/Components/DataSources/UDPStreamer
+ln -sf "${UDPSTREAMER_LIB}/UDPStreamer.so" "${UDPSTREAMER_LIB}/SineArrayGAM.so"
+```
+
+---
+
+## 6. Writing a Custom UDP Client
+
+A minimal Python client that receives and prints signal data:
+
+```python
+import socket, struct
+
+MAGIC   = 0x53504455
+HDR     = struct.Struct('<IBHHI')   # 17 bytes: magic, type, counter, fragIdx, total, payloadBytes
+SERVER  = ('127.0.0.1', 44500)
+MY_PORT = 44900
+
+sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+sock.bind(('', MY_PORT))
+sock.settimeout(5.0)
+
+# Send CONNECT
+sock.sendto(HDR.pack(MAGIC, 3, 0, 0, 1, 0), SERVER)
+print("CONNECT sent")
+
+reassembly = {}
+
+while True:
+    data, _ = sock.recvfrom(65536)
+    if len(data) < 17:
+        continue
+    magic, ptype, counter, frag_idx, total_frags, payload_bytes = HDR.unpack_from(data)
+    if magic != MAGIC:
+        continue
+    payload = data[17:17 + payload_bytes]
+
+    # Accumulate fragments
+    bucket = reassembly.setdefault((ptype, counter), {})
+    bucket[frag_idx] = payload
+    if len(bucket) < total_frags:
+        continue
+    full = b''.join(bucket[i] for i in range(total_frags))
+    del reassembly[(ptype, counter)]
+
+    if ptype == 1:   # CONFIG
+        num_sigs = struct.unpack_from('<I', full)[0]
+        print(f"CONFIG: {num_sigs} signals")
+    elif ptype == 0: # DATA
+        hrt = struct.unpack_from('<Q', full)[0]
+        print(f"DATA counter={counter} hrt={hrt} payload={len(full)}B")
+```
+
+For a full-featured client with CONFIG parsing and dequantization see
+[`Client/WebUI/protocol.go`](../Client/WebUI/protocol.go) (Go)
+and the [Protocol reference](Protocol.md).
+
+---
+
+## Troubleshooting
+
+| Symptom | Cause | Fix |
+|---------|-------|-----|
+| `Failed dlopen(): UDPStreamer.so` | Library not in `LD_LIBRARY_PATH` | Add build dir to `LD_LIBRARY_PATH` |
+| `Failed dlopen(): WaveformSin.so` | Symlink missing | Run `run.sh`; it creates the symlinks automatically |
+| `Failed dlopen(): SineArrayGAM.so` | Symlink missing | Create `SineArrayGAM.so → UDPStreamer.so` in the build dir |
+| WebUI shows "No data for Xs" | UDPStreamer not running / wrong port | Check port numbers; check MARTe2 logs |
+| Plots only show ~167 ms of HF data | Browser buffer too small | Use `TEMPORAL_CAP` in JS (already 600 000 in current WebUI) |
+| Fragmentation error / missing data | MTU too small | Reduce `MaxPayloadSize` to 1200 or smaller |