Martino Ferrari
454 lines
20 KiB
Markdown
454 lines
20 KiB
Markdown
# Technical Specification — uopi
|
||
|
||
## 1. Technology Choices
|
||
|
||
### 1.1 Backend — Go
|
||
|
||
**Rationale:**
|
||
|
||
- Compiles to a single static binary with no runtime dependencies, trivially portable to old Linux targets.
|
||
- `//go:embed` packs the compiled frontend assets into the binary at build time.
|
||
- Goroutine-per-connection model maps naturally onto the fan-out data broker pattern.
|
||
- CGo bindings to EPICS `libca` / `libCom` are straightforward.
|
||
- `gopher-lua` provides an embedded Lua 5.1-compatible interpreter for synthetic signals with zero additional dependencies.
|
||
- Strong standard library: `net/http`, `encoding/xml`, `encoding/json`.
|
||
|
||
**Go version:** 1.22+
|
||
|
||
**Key dependencies:**
|
||
|
||
| Package | Purpose |
|
||
| ---------------------------- | ------------------------------------------------------ |
|
||
| `nhooyr.io/websocket` | WebSocket server (no CGo, more ergonomic than gorilla) |
|
||
| CGo wrapper to `libca` | EPICS Channel Access |
|
||
| `yuin/gopher-lua` | Lua 5.1 runtime for synthetic signals |
|
||
| `BurntSushi/toml` | TOML config parsing |
|
||
| `encoding/xml` (stdlib) | Interface file serialisation |
|
||
| `net/http` (stdlib) | HTTP server and static file serving |
|
||
|
||
### 1.2 Frontend — Preact + TypeScript
|
||
|
||
**Rationale:**
|
||
|
||
- Preact is a 3 kB React-compatible virtual DOM library — small bundle, fast diffing, no extra framework overhead.
|
||
- esbuild (invoked via its Go API) bundles the TypeScript/TSX source in milliseconds with no Node.js or npm dependency at build time.
|
||
- TypeScript catches signal subscription and widget property type errors at build time.
|
||
- All vendor JS/CSS is checked into `web/vendor/` so the repo builds without internet access.
|
||
|
||
**Key dependencies (vendored):**
|
||
|
||
| Package | Purpose |
|
||
| ---------------- | --------------------------------------------------------------- |
|
||
| `preact` 10 | Virtual DOM UI framework |
|
||
| `uPlot` | Extremely fast time-series/line plot (canvas-based, < 40 kB) |
|
||
| `Apache ECharts` | FFT, waterfall, histogram, bar, logic analyser plots |
|
||
| `uplot.css` | uPlot default stylesheet |
|
||
|
||
**Intentionally excluded:** React, Vue, Svelte, Konva, WebGPU, jQuery, npm at runtime.
|
||
|
||
---
|
||
|
||
## 2. Repository Layout
|
||
|
||
```
|
||
uopi/
|
||
├── cmd/uopi/ # main package — CLI flags, wiring
|
||
├── internal/
|
||
│ ├── server/ # HTTP + WebSocket handlers
|
||
│ ├── broker/ # signal fan-out to clients
|
||
│ ├── datasource/
|
||
│ │ ├── iface.go # DataSource interface
|
||
│ │ ├── epics/ # EPICS CA implementation (CGo)
|
||
│ │ └── synthetic/ # synthetic signal engine + DSP bridge
|
||
│ ├── dsp/ # DSP node implementations (lowpass, MA, etc.)
|
||
│ ├── storage/ # interface XML read/write
|
||
│ └── api/ # REST handler functions
|
||
├── web/
|
||
│ ├── embed.go # //go:embed dist — exports FS to Go
|
||
│ ├── src/ # TypeScript/TSX source (Preact)
|
||
│ │ ├── lib/
|
||
│ │ │ ├── ws.ts # WebSocket client + subscription manager
|
||
│ │ │ ├── stores.ts # signal value + metadata stores
|
||
│ │ │ ├── types.ts # shared TypeScript interfaces
|
||
│ │ │ ├── xml.ts # interface XML parse/serialize
|
||
│ │ │ └── format.ts # value formatting helpers
|
||
│ │ ├── widgets/ # one .tsx file per widget type
|
||
│ │ ├── App.tsx # top-level component, mode routing
|
||
│ │ ├── ViewMode.tsx # view mode layout + tabs
|
||
│ │ ├── EditMode.tsx # edit mode layout + toolbar
|
||
│ │ ├── Canvas.tsx # live HMI canvas (view mode)
|
||
│ │ ├── EditCanvas.tsx # free-form widget editor canvas
|
||
│ │ ├── PlotPanel.tsx # live plot side-panel (Plot tab)
|
||
│ │ ├── InfoPanel.tsx # signal info side-panel
|
||
│ │ ├── ZoomControl.tsx # UI zoom A-/A+ control
|
||
│ │ ├── SyntheticWizard.tsx # new synthetic signal dialog
|
||
│ │ ├── SyntheticEditor.tsx # edit existing synthetic signal
|
||
│ │ ├── LuaEditor.tsx # Lua code editor with syntax highlight
|
||
│ │ └── styles.css # all component styles
|
||
│ ├── vendor/ # vendored JS/CSS (preact, uplot, echarts)
|
||
│ └── dist/ # built frontend — generated, not committed
|
||
├── tools/buildfrontend/ # esbuild Go API bundler (go generate)
|
||
├── docs/ # specs, work plan
|
||
├── CLAUDE.md
|
||
└── README.md
|
||
```
|
||
|
||
The embed package lives at `web/embed.go` (not in `cmd/`) because `//go:embed` paths cannot use `..`.
|
||
|
||
---
|
||
|
||
## 3. Backend Architecture
|
||
|
||
### 3.1 DataSource Interface
|
||
|
||
```go
|
||
type Value struct {
|
||
Timestamp time.Time
|
||
Data any // float64 | []float64 | string | int64 | bool
|
||
Quality Quality // Good | Bad | Uncertain
|
||
}
|
||
|
||
type Metadata struct {
|
||
Name string
|
||
Type DataType
|
||
Unit string
|
||
DisplayLow float64
|
||
DisplayHigh float64
|
||
DriveHigh float64
|
||
DriveLow float64
|
||
EnumStrings []string
|
||
Writable bool
|
||
}
|
||
|
||
type DataSource interface {
|
||
Name() string
|
||
Connect(ctx context.Context) error
|
||
ListSignals(ctx context.Context) ([]Metadata, error)
|
||
GetMetadata(ctx context.Context, signal string) (Metadata, error)
|
||
Subscribe(ctx context.Context, signal string, ch chan<- Value) (CancelFunc, error)
|
||
Write(ctx context.Context, signal string, value any) error
|
||
History(ctx context.Context, signal string, start, end time.Time, maxPoints int) ([]Value, error)
|
||
}
|
||
```
|
||
|
||
New data sources are registered at startup via `datasource.Register(name string, ds DataSource)`.
|
||
|
||
### 3.2 Signal Broker
|
||
|
||
The broker is the central fan-out component:
|
||
|
||
```
|
||
DataSource ──subscribe──► rawCh ──► Broker ──► [clientCh1, clientCh2, ...]
|
||
```
|
||
|
||
- One goroutine per active signal subscription to the underlying data source.
|
||
- Per-signal subscriber list protected by a sync.RWMutex.
|
||
- When the last client unsubscribes, the broker cancels the upstream subscription.
|
||
- No data is buffered in the broker; clients receive the latest value at the moment they subscribe and all subsequent updates.
|
||
|
||
### 3.3 WebSocket Protocol
|
||
|
||
Framing: JSON messages over a single persistent WebSocket connection per client.
|
||
|
||
**Client → Server messages:**
|
||
|
||
```jsonc
|
||
// Subscribe to one or more signals
|
||
{ "type": "subscribe", "signals": ["EPICS:PV1", "synth:mySignal"] }
|
||
|
||
// Unsubscribe
|
||
{ "type": "unsubscribe", "signals": ["EPICS:PV1"] }
|
||
|
||
// Write a value
|
||
{ "type": "write", "signal": "EPICS:PV1", "value": 3.14 }
|
||
|
||
// Request historical data
|
||
{ "type": "history", "signal": "EPICS:PV1", "start": "2026-01-01T00:00:00Z", "end": "2026-01-02T00:00:00Z", "maxPoints": 5000 }
|
||
```
|
||
|
||
**Server → Client messages:**
|
||
|
||
```jsonc
|
||
// Live value update
|
||
{ "type": "update", "signal": "EPICS:PV1", "ts": "2026-04-24T12:00:00.123Z", "value": 42.7, "quality": "good" }
|
||
|
||
// Metadata (sent once on first subscribe)
|
||
{ "type": "meta", "signal": "EPICS:PV1", "meta": { "unit": "A", "displayLow": 0, "displayHigh": 100, ... } }
|
||
|
||
// Historical data response
|
||
{ "type": "history", "signal": "EPICS:PV1", "points": [ { "ts": "...", "value": 1.2 }, ... ] }
|
||
|
||
// Error
|
||
{ "type": "error", "code": "NOT_FOUND", "message": "Signal not found" }
|
||
```
|
||
|
||
### 3.4 REST API
|
||
|
||
Base path: `/api/v1`
|
||
|
||
| Method | Path | Description |
|
||
| ------ | ------------------------- | -------------------------------------------- |
|
||
| GET | `/datasources` | List connected data sources and their status |
|
||
| GET | `/signals?ds=epics` | List signals for a data source |
|
||
| GET | `/signals/:ds/:name/meta` | Get full metadata for a signal |
|
||
| GET | `/interfaces` | List saved interfaces |
|
||
| POST | `/interfaces` | Create a new interface (body: XML) |
|
||
| GET | `/interfaces/:id` | Download interface XML |
|
||
| PUT | `/interfaces/:id` | Update interface XML |
|
||
| DELETE | `/interfaces/:id` | Delete interface |
|
||
| POST | `/interfaces/:id/clone` | Clone an interface |
|
||
|
||
### 3.5 EPICS Data Source
|
||
|
||
- Uses CGo bindings to EPICS Base `libca` (Channel Access).
|
||
- Channel connections are lazy: connected on first Subscribe, disconnected when the broker releases it.
|
||
- On connect, a `ca_get` retrieves full DBR_CTRL metadata (units, limits, enum strings).
|
||
- `ca_add_event` sets up the monitor. Update callbacks push into the broker's raw channel.
|
||
- Multiple PV subscriptions share one CA context per data source instance (thread-safe with `ca_attach_context`).
|
||
- EPICS Archive Appliance is queried via its JSON HTTP API for history requests.
|
||
|
||
### 3.6 Synthetic Data Source
|
||
|
||
- Each synthetic signal is defined as a directed acyclic graph (DAG) of processing nodes.
|
||
- Processing nodes are re-evaluated whenever any upstream signal emits a new value.
|
||
- Definitions are stored in a configurable JSON/TOML file alongside server configuration.
|
||
- The `dsp_bridge.go` file maps node type names to `dsp.Node` implementations.
|
||
|
||
**Built-in node types:**
|
||
|
||
| Node type | Parameters | Description |
|
||
| ------------ | ----------------------------------- | ------------------------------------------------ |
|
||
| `source` | `ds`, `name` | Reads a signal from any data source |
|
||
| `gain` | `factor` | Multiplies by a constant |
|
||
| `offset` | `value` | Adds a constant |
|
||
| `moving_avg` | `window` (samples) | Rolling mean |
|
||
| `lowpass` | `freq` (Hz), `order` (1–8) | Cascaded IIR Butterworth-style low-pass filter |
|
||
| `formula` | `expr` | Inline math expression (variables: `a`, `b`, …) |
|
||
| `lua` | `script` | Arbitrary Lua 5.1 code with persistent state |
|
||
|
||
**Low-pass filter implementation:** Cascaded first-order IIR sections. Each stage computes `y = y_prev + α·(x − y_prev)` where `α = dt / (RC + dt)` and `RC = 1/(2π·fc)`. `dt` is computed per sample from source timestamps so the filter is correct for event-driven (non-uniform) data.
|
||
|
||
**Lua node:** Receives `inputs` table (indexed by signal name) and a persistent `state` table across calls. The `os`, `io`, `package`, and `debug` libraries are disabled.
|
||
|
||
### 3.7 Interface Storage
|
||
|
||
Interfaces are stored as XML files in a configurable directory on the server.
|
||
|
||
```xml
|
||
<interface name="My Panel" version="1" created="2026-04-24T12:00:00Z">
|
||
<widget id="w1" type="plot" x="100" y="200" w="600" h="300">
|
||
<signal ds="epics" name="EPICS:CURRENT" color="#ff0000"/>
|
||
<signal ds="epics" name="EPICS:VOLTAGE" color="#0000ff"/>
|
||
<option key="plotType" value="timeseries"/>
|
||
<option key="yMin" value="auto"/>
|
||
<option key="yMax" value="auto"/>
|
||
<option key="timeWindow" value="60"/>
|
||
<option key="legend" value="bottom"/>
|
||
</widget>
|
||
<widget id="w2" type="led" x="50" y="50" w="80" h="80">
|
||
<signal ds="epics" name="EPICS:STATUS"/>
|
||
<option key="condition" value="value > 0"/>
|
||
<option key="colorTrue" value="#00ff00"/>
|
||
<option key="colorFalse" value="#ff0000"/>
|
||
<option key="label" value="OK"/>
|
||
</widget>
|
||
</interface>
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Frontend Architecture
|
||
|
||
### 4.1 WebSocket Client (`lib/ws.ts`)
|
||
|
||
- Singleton WebSocket connection, reconnects with exponential back-off.
|
||
- Subscription reference counting: multiple widgets subscribing to the same signal result in one server subscription message.
|
||
- Incoming updates are dispatched to per-signal stores.
|
||
- `wsClient.history(sig, start, end, maxPoints)` returns a Promise resolving to timestamped point arrays.
|
||
|
||
### 4.2 Signal Stores (`lib/stores.ts`)
|
||
|
||
```typescript
|
||
// One nanostores atom per subscribed signal
|
||
const signalStores = new Map<string, SignalStore>();
|
||
|
||
export function getSignalStore(ref: SignalRef): SignalStore { ... }
|
||
export function getMetaStore(ref: SignalRef): MetaStore { ... }
|
||
```
|
||
|
||
Widgets subscribe to stores directly; store updates trigger re-renders only in the consuming component.
|
||
|
||
### 4.3 Edit Mode Canvas (`EditCanvas.tsx`)
|
||
|
||
The edit canvas is a free-form HTML div with absolutely positioned widget components:
|
||
|
||
- Each widget renders as an absolutely positioned `<div>` at `(x, y)` with `(w, h)` dimensions.
|
||
- Selection shows a CSS-outlined bounding box with 8 resize handles rendered as small squares.
|
||
- Drag-and-drop from the signal tree uses the HTML Drag-and-Drop API; on drop, the canvas coordinate is computed from the drop event offset.
|
||
- Undo/redo uses an array of past interface snapshots (max depth 50).
|
||
- Align/distribute operations compute target positions geometrically and generate a single undo entry.
|
||
- Multi-select via Ctrl+click or rubber-band area select.
|
||
|
||
### 4.4 Widget Rendering in View Mode (`Canvas.tsx`)
|
||
|
||
View mode renders widgets as absolutely positioned Preact components on a scrollable canvas div:
|
||
|
||
- Each widget subscribes to its signal store(s) in a `useEffect` and re-renders only when values change.
|
||
- uPlot (time series) and ECharts (histogram, bar, FFT, waterfall, logic analyser) manage their own canvas elements inside their widget component.
|
||
- Plot widgets maintain a rolling ring buffer of 200,000 samples per signal for smooth long-window display.
|
||
- Step-hold interpolation: when multiple signals at different update rates share a plot, the most recent value is carried forward to fill the shared time axis correctly.
|
||
|
||
### 4.5 Plot Panel (`PlotPanel.tsx`)
|
||
|
||
A live multi-signal plot panel accessible from the "Plot" tab in View mode:
|
||
|
||
- Signals are added by right-clicking any widget and choosing "Plot".
|
||
- Supports configurable time window (10s to 1h).
|
||
- Per-signal style editor: color picker, line width (0 = hidden), line dash (solid/dashed/dotted), marker size (none/S/M/L).
|
||
- Statistics table per signal: last, min, max, mean over the current time window.
|
||
- Uses a `requestAnimationFrame` loop limited to ≤1 redraw/second when data is not changing.
|
||
- Chart fills its container; `ResizeObserver` keeps the uPlot canvas sized correctly.
|
||
|
||
### 4.6 Resizable Panels
|
||
|
||
Both edit and view modes support mouse-drag panel resizing:
|
||
|
||
- **View mode**: drag handle between the interface list pane and the main content area.
|
||
- **Edit mode**: drag handles on both sides of the central canvas (signal tree ↔ canvas, canvas ↔ properties pane).
|
||
- Handle width: 5 px, cursor changes to `ew-resize` on hover.
|
||
- Minimum panel widths enforced to prevent collapse below usable size.
|
||
|
||
### 4.7 HiDPI / Zoom Support
|
||
|
||
- `html { font-size: clamp(13px, 1.5vh, 18px); }` — base font scales with viewport height, making the UI naturally larger on 4K screens where the browser zoom level is 100%.
|
||
- Key structural heights (toolbar, panel headers, tab bar, plot toolbar) are expressed in `rem` so they scale with the base font.
|
||
- **ZoomControl** (A− / % / A+) in the toolbar lets users manually override the zoom level in 11 steps from 50% to 250%. The preference is persisted in `localStorage` (`uopi:ui-zoom`) and applied by setting `document.documentElement.style.fontSize` on load.
|
||
- Canvas pixel rendering (uPlot, ECharts) reads `window.devicePixelRatio` and sizes canvases accordingly.
|
||
|
||
### 4.8 Lua Editor (`LuaEditor.tsx`)
|
||
|
||
A syntax-highlighted code editor for Lua scripts in the Synthetic signal wizard:
|
||
|
||
- Implemented as a `<textarea>` overlaid on a `<pre>` element; the textarea has `color: transparent; caret-color: #e2e8f0` so only the caret is visible — the `<pre>` provides the coloured text behind it.
|
||
- Tokeniser handles: `--` line comments, `"..."` / `'...'` string literals, `[[...]]` long strings, hex and float numeric literals, and all Lua 5.1 keywords.
|
||
- Scroll position is synchronised between textarea and pre on every scroll event.
|
||
|
||
---
|
||
|
||
## 5. Build System
|
||
|
||
### 5.1 Backend
|
||
|
||
```bash
|
||
# Full build (frontend then backend)
|
||
make all
|
||
|
||
# Backend only (frontend must already be built)
|
||
make backend
|
||
|
||
# All tests
|
||
make test
|
||
|
||
# Single Go test
|
||
go test ./internal/broker/... -run TestFanOut
|
||
|
||
# Go vet
|
||
go vet ./...
|
||
```
|
||
|
||
EPICS `libca.a` is statically linked via `CGO_LDFLAGS` in `internal/datasource/epics/cgo.go`.
|
||
|
||
### 5.2 Frontend
|
||
|
||
The frontend is built by a Go tool in `tools/buildfrontend/` that invokes the esbuild Go API:
|
||
|
||
```bash
|
||
make frontend
|
||
# or equivalently:
|
||
go generate ./web/...
|
||
```
|
||
|
||
No Node.js, npm, or any JS build tool is required on the host. The bundler:
|
||
- Reads entry point `web/src/main.tsx`.
|
||
- Resolves `preact`, `uplot`, and `echarts` imports from `web/vendor/`.
|
||
- Outputs `web/dist/main.js` and `web/dist/main.css`.
|
||
- Copies `web/dist/index.html`, vendor CSS, and other static assets.
|
||
|
||
### 5.3 Combined Build
|
||
|
||
```makefile
|
||
.PHONY: all frontend backend clean
|
||
|
||
all: frontend backend
|
||
|
||
frontend:
|
||
go run ./tools/buildfrontend
|
||
|
||
backend:
|
||
CGO_ENABLED=1 go build -ldflags="-s -w" -o dist/uopi ./cmd/uopi
|
||
|
||
test:
|
||
go test ./...
|
||
|
||
clean:
|
||
rm -rf dist/ web/dist/
|
||
```
|
||
|
||
The backend's `//go:embed dist` directive in `web/embed.go` picks up the built frontend automatically. `main.go` does `fs.Sub(web.FS, "dist")` to serve a clean root.
|
||
|
||
---
|
||
|
||
## 6. Configuration
|
||
|
||
Server is configured via a TOML file (default: `uopi.toml`, overridable via `--config` flag):
|
||
|
||
```toml
|
||
[server]
|
||
listen = ":8080"
|
||
storage_dir = "./interfaces"
|
||
|
||
[datasource.epics]
|
||
enabled = true
|
||
ca_addr_list = "" # EPICS_CA_ADDR_LIST override
|
||
archive_url = "" # EPICS Archive Appliance URL
|
||
|
||
[datasource.synthetic]
|
||
enabled = true
|
||
definitions_file = "./synthetic.toml"
|
||
```
|
||
|
||
All settings can also be overridden with `UOPI_*` environment variables (e.g. `UOPI_SERVER_LISTEN`, `UOPI_EPICS_CA_ADDR_LIST`).
|
||
|
||
---
|
||
|
||
## 7. Testing Strategy
|
||
|
||
| Layer | Approach |
|
||
| ------------------ | --------------------------------------------------------------------------------- |
|
||
| Broker | Unit tests with mock data source; verify fan-out, subscribe/unsubscribe lifecycle |
|
||
| Synthetic DSP | Table-driven unit tests against known signal inputs/outputs |
|
||
| Low-pass filter | Unit tests: step response, frequency attenuation vs. analytical expectation |
|
||
| Lua sandbox | Unit tests for sandbox isolation and API surface |
|
||
| REST API | `httptest` integration tests |
|
||
| WebSocket protocol | Integration tests with a test client |
|
||
| EPICS data source | Integration tests against a local SoftIOC (optional, CI-gated) |
|
||
|
||
---
|
||
|
||
## 8. Security Considerations
|
||
|
||
- Lua sandbox: disable `os`, `io`, `package`, `debug` libraries; restrict `math` and `string` to safe subsets.
|
||
- WebSocket write operations: validate that the target signal is writable before forwarding to the data source.
|
||
- Interface XML parsing: use strict schema validation to prevent XXE.
|
||
- No authentication in v1; intended for trusted LAN / SSH-tunnel deployment.
|
||
|
||
---
|
||
|
||
## 9. Non-goals (v1)
|
||
|
||
- User authentication and authorisation.
|
||
- TLS termination (expected to be handled by SSH tunnel or a reverse proxy).
|
||
- Windows or macOS server binary.
|
||
- Mobile-optimised frontend layout.
|
||
- Remote plugin loading (plugins compiled in at build time only).
|