Martino Ferrari
360 lines
9.5 KiB
Markdown
360 lines
9.5 KiB
Markdown
# Tutorial: Streaming Signals with UDPStreamer
|
||
|
||
This tutorial walks you through:
|
||
|
||
1. Setting up the environment
|
||
2. Running the demo application
|
||
3. Visualising signals in the browser
|
||
4. Adding UDPStreamer to your own MARTe2 application
|
||
5. Streaming high-frequency packed signals
|
||
|
||
**Prerequisites:** MARTe2 and MARTe2-components must already be built.
|
||
See the [MARTe2 installation guide](https://vcis.f4e.europa.eu/marte2-docs/) if needed.
|
||
|
||
---
|
||
|
||
## 1. Environment Setup
|
||
|
||
Edit `marte_env.sh` in the repository root to point at your MARTe2 installations:
|
||
|
||
```bash
|
||
# marte_env.sh (key variables)
|
||
export MARTe2_DIR="$HOME/workspace/MARTe2"
|
||
export MARTe2_Components_DIR="$HOME/workspace/MARTe2-components"
|
||
```
|
||
|
||
Then source it in your shell:
|
||
|
||
```bash
|
||
cd /path/to/MARTe_IO_components
|
||
source marte_env.sh
|
||
```
|
||
|
||
Verify the environment is correct:
|
||
|
||
```bash
|
||
echo $MARTe2_DIR
|
||
ls $MARTe2_DIR/Build/x86-linux/App/MARTeApp.ex # should exist
|
||
```
|
||
|
||
---
|
||
|
||
## 2. Running the Demo Application
|
||
|
||
The demo is in `Test/MARTeApp/`. It runs a 10 kHz MARTe2 application that streams:
|
||
|
||
- **Counter** and **Time** — scalar counters from the Linux timer
|
||
- **Sine1** — 1 Hz sine wave (float32, amplitude 10, quantized to uint16 on wire)
|
||
- **Sine2** — 0.3 Hz sine wave (float32, amplitude 5, raw float32 on wire)
|
||
- **Ch1**, **Ch2** — 1 kHz sine bursts packed as 1000 samples/packet (10 MSps)
|
||
|
||
Start everything with one command:
|
||
|
||
```bash
|
||
cd Test/MARTeApp
|
||
./run.sh --webui
|
||
```
|
||
|
||
The script will:
|
||
1. Build the UDPStreamer shared library.
|
||
2. Build the Go WebUI binary (first run only).
|
||
3. Start the WebUI relay on `http://localhost:8080`.
|
||
4. Launch the MARTe2 application.
|
||
|
||
Press `Ctrl+C` to stop both processes.
|
||
|
||
---
|
||
|
||
## 3. Visualising Signals in the Browser
|
||
|
||
Open `http://localhost:8080` in any modern browser.
|
||
|
||
### Add your first plot
|
||
|
||
1. Click **+ Add Plot** in the toolbar.
|
||
2. A blank plot panel appears with a "Drop signals here" hint.
|
||
|
||
### Plot a signal
|
||
|
||
1. In the left sidebar find **Sine1** (listed as `Sine1 · f32`).
|
||
2. Click and drag it onto the plot panel.
|
||
3. The sine wave appears immediately.
|
||
|
||
### Overlay multiple signals
|
||
|
||
Drag **Sine2** onto the same plot — it is added as a second trace.
|
||
|
||
### Adjust the time window
|
||
|
||
Use the **Window** dropdown in the top bar to change the rolling display window
|
||
(1 s, 5 s, 10 s, 30 s, 60 s).
|
||
|
||
### Plot layout
|
||
|
||
Use the layout buttons (`1×1`, `2×1`, `2×2`, …) to split the screen into multiple
|
||
plot panels. Each panel is independent — drag different signals onto each.
|
||
|
||
### High-frequency signals
|
||
|
||
Drag **Ch1** (shown as `Ch1 · [1000] f32`) onto a plot. Each UDP packet carries
|
||
1000 samples at 10 MSps; the WebUI reconstructs per-sample timestamps and displays
|
||
the continuous waveform.
|
||
|
||
### Export data
|
||
|
||
Click **⬇** on any plot to download the visible window as a CSV file.
|
||
|
||
---
|
||
|
||
## 4. Adding UDPStreamer to Your Own Application
|
||
|
||
### Step 1 — Declare the DataSource
|
||
|
||
Add UDPStreamer to the `+Data` section of your MARTe2 configuration:
|
||
|
||
```
|
||
+Data = {
|
||
Class = ReferenceContainer
|
||
DefaultDataSource = DDB
|
||
|
||
+DDB = { Class = GAMDataSource }
|
||
|
||
+Streamer = {
|
||
Class = UDPStreamer
|
||
Port = 44500
|
||
MaxPayloadSize = 1400
|
||
|
||
Signals = {
|
||
Voltage = {
|
||
Type = float32
|
||
Unit = "V"
|
||
RangeMin = -10.0
|
||
RangeMax = 10.0
|
||
QuantizedType = uint16 // 16-bit quantized on wire
|
||
}
|
||
Current = {
|
||
Type = float32
|
||
Unit = "A"
|
||
}
|
||
}
|
||
}
|
||
|
||
+Timings = { Class = TimingDataSource }
|
||
}
|
||
```
|
||
|
||
### Step 2 — Route signals with IOGAM
|
||
|
||
Use IOGAM to copy signals from your inter-GAM DDB into the Streamer:
|
||
|
||
```
|
||
+StreamerGAM = {
|
||
Class = IOGAM
|
||
InputSignals = {
|
||
Voltage = { DataSource = DDB; Type = float32 }
|
||
Current = { DataSource = DDB; Type = float32 }
|
||
}
|
||
OutputSignals = {
|
||
Voltage = { DataSource = Streamer; Type = float32 }
|
||
Current = { DataSource = Streamer; Type = float32 }
|
||
}
|
||
}
|
||
```
|
||
|
||
Add `StreamerGAM` at the **end** of the thread's `Functions` list so it runs after
|
||
your control GAMs have written their outputs.
|
||
|
||
### Step 3 — Add the library to LD_LIBRARY_PATH
|
||
|
||
In your run script, add the UDPStreamer build directory:
|
||
|
||
```bash
|
||
export LD_LIBRARY_PATH="/path/to/Build/x86-linux/Components/DataSources/UDPStreamer:$LD_LIBRARY_PATH"
|
||
```
|
||
|
||
### Step 4 — Start the WebUI and connect
|
||
|
||
```bash
|
||
# From the Client/WebUI directory:
|
||
./udpstreamer-webui --streamer 127.0.0.1:44500 --listen :8080 --clientport 44900
|
||
```
|
||
|
||
Open `http://localhost:8080`, drag your signals onto a plot, and you're done.
|
||
|
||
---
|
||
|
||
## 5. Streaming High-Frequency Packed Signals
|
||
|
||
This section shows how to stream 1000 samples per RT cycle at 1 MSps.
|
||
|
||
### Overview
|
||
|
||
At 1 kHz RT rate with 1000 samples per cycle the effective sample rate is 1 MSps.
|
||
Each UDP packet carries a burst of 1000 samples; the client reconstructs timestamps
|
||
using the anchor timestamp and `SamplingRate`.
|
||
|
||
### Step 1 — Generate burst data with SineArrayGAM
|
||
|
||
`SineArrayGAM` (bundled in `UDPStreamer.so`) produces a continuous float32 array:
|
||
|
||
```
|
||
+Ch1GAM = {
|
||
Class = SineArrayGAM
|
||
Frequency = 1000.0 // 1 kHz signal
|
||
Amplitude = 1.0
|
||
Phase = 0.0
|
||
SamplingRate = 1000000.0 // must match Streamer config below
|
||
OutputSignals = {
|
||
Ch1 = {
|
||
DataSource = DDB
|
||
Type = float32
|
||
NumberOfDimensions = 1
|
||
NumberOfElements = 1000
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
### Step 2 — Add a time reference signal
|
||
|
||
Add a scalar time signal that will anchor the first sample's timestamp:
|
||
|
||
```
|
||
+TimerGAM = {
|
||
Class = IOGAM
|
||
InputSignals = {
|
||
Time = { DataSource = Timer; Type = uint32; Frequency = 1000 }
|
||
}
|
||
OutputSignals = {
|
||
Time = { DataSource = DDB; Type = uint32 }
|
||
}
|
||
}
|
||
```
|
||
|
||
### Step 3 — Configure UDPStreamer for packed signals
|
||
|
||
```
|
||
+Streamer = {
|
||
Class = UDPStreamer
|
||
Port = 44500
|
||
MaxPayloadSize = 1400
|
||
|
||
Signals = {
|
||
Time = { Type = uint32; Unit = "us" } // time reference (scalar)
|
||
|
||
Ch1 = {
|
||
Type = float32
|
||
NumberOfDimensions = 1
|
||
NumberOfElements = 1000
|
||
Unit = "V"
|
||
TimeMode = FirstSample // Time = timestamp of first sample
|
||
TimeSignal = Time
|
||
SamplingRate = 1000000.0 // Hz
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
### Step 4 — Wire everything in the thread
|
||
|
||
```
|
||
+Thread1 = {
|
||
Class = RealTimeThread
|
||
Functions = { TimerGAM Ch1GAM StreamerGAM }
|
||
}
|
||
```
|
||
|
||
Where `StreamerGAM` is the IOGAM that copies `Time` and `Ch1` from DDB to Streamer.
|
||
|
||
### Step 5 — Fragmentation note
|
||
|
||
A single 1000-element float32 channel plus a uint32 time signal produces:
|
||
|
||
```
|
||
payload = 8 B (HRT) + 4 B (Time/uint32) + 4000 B (float32×1000) = 4012 B
|
||
```
|
||
|
||
With `MaxPayloadSize = 1400`:
|
||
|
||
```
|
||
fragments = ceil(4012 / 1383) = 3 datagrams per cycle
|
||
```
|
||
|
||
At 1 kHz that is 3000 UDP datagrams/second per channel — well within typical LAN capacity.
|
||
|
||
### Step 6 — Create the SineArrayGAM symlink
|
||
|
||
MARTe2 tries to `dlopen("SineArrayGAM.so")` the first time it encounters the class.
|
||
Create the symlink in your build directory:
|
||
|
||
```bash
|
||
UDPSTREAMER_LIB=/path/to/Build/x86-linux/Components/DataSources/UDPStreamer
|
||
ln -sf "${UDPSTREAMER_LIB}/UDPStreamer.so" "${UDPSTREAMER_LIB}/SineArrayGAM.so"
|
||
```
|
||
|
||
---
|
||
|
||
## 6. Writing a Custom UDP Client
|
||
|
||
A minimal Python client that receives and prints signal data:
|
||
|
||
```python
|
||
import socket, struct
|
||
|
||
MAGIC = 0x53504455
|
||
HDR = struct.Struct('<IBHHI') # 17 bytes: magic, type, counter, fragIdx, total, payloadBytes
|
||
SERVER = ('127.0.0.1', 44500)
|
||
MY_PORT = 44900
|
||
|
||
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||
sock.bind(('', MY_PORT))
|
||
sock.settimeout(5.0)
|
||
|
||
# Send CONNECT
|
||
sock.sendto(HDR.pack(MAGIC, 3, 0, 0, 1, 0), SERVER)
|
||
print("CONNECT sent")
|
||
|
||
reassembly = {}
|
||
|
||
while True:
|
||
data, _ = sock.recvfrom(65536)
|
||
if len(data) < 17:
|
||
continue
|
||
magic, ptype, counter, frag_idx, total_frags, payload_bytes = HDR.unpack_from(data)
|
||
if magic != MAGIC:
|
||
continue
|
||
payload = data[17:17 + payload_bytes]
|
||
|
||
# Accumulate fragments
|
||
bucket = reassembly.setdefault((ptype, counter), {})
|
||
bucket[frag_idx] = payload
|
||
if len(bucket) < total_frags:
|
||
continue
|
||
full = b''.join(bucket[i] for i in range(total_frags))
|
||
del reassembly[(ptype, counter)]
|
||
|
||
if ptype == 1: # CONFIG
|
||
num_sigs = struct.unpack_from('<I', full)[0]
|
||
print(f"CONFIG: {num_sigs} signals")
|
||
elif ptype == 0: # DATA
|
||
hrt = struct.unpack_from('<Q', full)[0]
|
||
print(f"DATA counter={counter} hrt={hrt} payload={len(full)}B")
|
||
```
|
||
|
||
For a full-featured client with CONFIG parsing and dequantization see
|
||
[`Client/WebUI/protocol.go`](../Client/WebUI/protocol.go) (Go)
|
||
and the [Protocol reference](Protocol.md).
|
||
|
||
---
|
||
|
||
## Troubleshooting
|
||
|
||
| Symptom | Cause | Fix |
|
||
|---------|-------|-----|
|
||
| `Failed dlopen(): UDPStreamer.so` | Library not in `LD_LIBRARY_PATH` | Add build dir to `LD_LIBRARY_PATH` |
|
||
| `Failed dlopen(): WaveformSin.so` | Symlink missing | Run `run.sh`; it creates the symlinks automatically |
|
||
| `Failed dlopen(): SineArrayGAM.so` | Symlink missing | Create `SineArrayGAM.so → UDPStreamer.so` in the build dir |
|
||
| WebUI shows "No data for Xs" | UDPStreamer not running / wrong port | Check port numbers; check MARTe2 logs |
|
||
| Plots only show ~167 ms of HF data | Browser buffer too small | Use `TEMPORAL_CAP` in JS (already 600 000 in current WebUI) |
|
||
| Fragmentation error / missing data | MTU too small | Reduce `MaxPayloadSize` to 1200 or smaller |
|