Implemented history writer
This commit is contained in:
@@ -98,7 +98,78 @@ client and uses full-rate ring data (not the decimated live stream).
|
||||
On capture, the trigger view shows all plotted signals relative to the trigger
|
||||
instant (marker at t = 0) over `[-pre, +post]`.
|
||||
|
||||
## 5. Troubleshooting
|
||||
## 5. History (disk-backed storage)
|
||||
|
||||
StreamHub can store signal data to disk, allowing you to browse hours of past
|
||||
data even after the in-memory ring buffers have wrapped.
|
||||
|
||||
### Enabling history
|
||||
|
||||
Add a `+History` block to the hub configuration:
|
||||
|
||||
```
|
||||
Hub = {
|
||||
WSPort = 8090
|
||||
+History = {
|
||||
Directory = "/data/streamhub_history"
|
||||
DurationHours = 1
|
||||
Decimation = 10
|
||||
FlushIntervalSec = 5
|
||||
MinDiskFreeMB = 200
|
||||
}
|
||||
Sources = { … }
|
||||
}
|
||||
```
|
||||
|
||||
| Key | Default | Description |
|
||||
|-----|---------|-------------|
|
||||
| `Directory` | *(required)* | Root directory for `.shist` files. Created automatically. Each source gets a subdirectory; each signal a separate file. |
|
||||
| `DurationHours` | 1 | How many hours of data to keep per signal. The file size is fixed at creation: `duration × samplingRate / decimation × 16 bytes`. |
|
||||
| `Decimation` | 1 | Write every N-th sample (1 = no decimation). Useful for high-rate signals where full-rate disk writes are impractical. |
|
||||
| `FlushIntervalSec` | 5 | How often file headers are flushed to disk (data is written immediately via `pwrite`). |
|
||||
| `MinDiskFreeMB` | 500 | Writing pauses when free disk space drops below this threshold and resumes when space is available again. |
|
||||
|
||||
### Disk usage
|
||||
|
||||
Each sample on disk is 16 bytes (8 bytes timestamp + 8 bytes value). Example:
|
||||
|
||||
- 10 signals at 1 ksps, decimation 10, duration 1 hour:
|
||||
`10 × (1000/10) × 3600 × 16 bytes = 57.6 MB`
|
||||
- 10 signals at 1 Msps, decimation 1000, duration 1 hour:
|
||||
`10 × (1e6/1000) × 3600 × 16 bytes = 576 MB`
|
||||
|
||||
Files are pre-allocated at creation and never grow. The directory structure is:
|
||||
|
||||
```
|
||||
/data/streamhub_history/
|
||||
├── scalar/
|
||||
│ ├── Sine1.shist
|
||||
│ └── Sine2.shist
|
||||
├── med/
|
||||
│ ├── Ch1.shist
|
||||
│ └── Ch2.shist
|
||||
└── fast/
|
||||
├── Ch3.shist
|
||||
└── Ch4.shist
|
||||
```
|
||||
|
||||
### Using history in clients
|
||||
|
||||
When history is enabled, both the web SPA and ImGui client show a **HIST**
|
||||
indicator. History data is fetched transparently:
|
||||
|
||||
- **Zoom**: when you zoom or pan into a time range that extends beyond the
|
||||
in-memory ring buffer, the client issues a `historyZoom` request in parallel
|
||||
with the regular `zoom` request and merges the results — history data fills
|
||||
the older part of the window, ring data the recent part.
|
||||
- **Status**: the web SPA displays a history badge in the status bar showing
|
||||
the configured duration and decimation factor. The ImGui client shows
|
||||
"HIST" in the plot header when history data is being displayed.
|
||||
|
||||
History files survive hub restarts. If a `.shist` file exists with a matching
|
||||
capacity, the hub reopens it and continues writing where it left off.
|
||||
|
||||
## 6. Troubleshooting
|
||||
|
||||
| Symptom | Check |
|
||||
|---------|-------|
|
||||
@@ -111,7 +182,7 @@ instant (marker at t = 0) over `[-pre, +post]`.
|
||||
Hub-side logs go to stdout; the demo scripts write
|
||||
`/tmp/streamhub_e2e_{marte,hub}.log`.
|
||||
|
||||
## 6. Further reading
|
||||
## 7. Further reading
|
||||
|
||||
- [StreamHub-API.md](StreamHub-API.md) — WebSocket protocol (write your own client)
|
||||
- [StreamHub-Developer.md](StreamHub-Developer.md) — internals, build, tests
|
||||
|
||||
Reference in New Issue
Block a user