Files
uopi/docs/TECHNICAL_SPEC.md
Martino Ferrari afefba3184 Add control logic engine, panel logic dialogs, logic-edit restriction
Introduce server-side control-logic flow graphs (cron/alarm triggers,
Lua blocks) with CRUD endpoints, panel-logic lifecycle triggers and
user-interaction dialog nodes, and a synthetic node-graph editor.

Add an optional logic-editor allowlist (server.logic_editors) gating who
may add/edit panel logic and control logic, surfaced via /api/v1/me and
enforced in the API; hide logic affordances in the UI accordingly.

Update README, example config, and functional/technical specs to cover
all current features (plot panels, panel/control logic, local variables,
access control) and refresh the in-app manual and contextual help.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-19 07:27:35 +02:00

535 lines
26 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Technical Specification — uopi
## 1. Technology Choices
### 1.1 Backend — Go
**Rationale:**
- Compiles to a single static binary with no runtime dependencies, trivially portable to old Linux targets.
- `//go:embed` packs the compiled frontend assets into the binary at build time.
- Goroutine-per-connection model maps naturally onto the fan-out data broker pattern.
- CGo bindings to EPICS `libca` / `libCom` are straightforward.
- `gopher-lua` provides an embedded Lua 5.1-compatible interpreter for synthetic signals with zero additional dependencies.
- Strong standard library: `net/http`, `encoding/xml`, `encoding/json`.
**Go version:** 1.22+
**Key dependencies:**
| Package | Purpose |
| ---------------------------- | ------------------------------------------------------ |
| `nhooyr.io/websocket` | WebSocket server (no CGo, more ergonomic than gorilla) |
| CGo wrapper to `libca` | EPICS Channel Access |
| `yuin/gopher-lua` | Lua 5.1 runtime for synthetic signals |
| `BurntSushi/toml` | TOML config parsing |
| `encoding/xml` (stdlib) | Interface file serialisation |
| `net/http` (stdlib) | HTTP server and static file serving |
### 1.2 Frontend — Preact + TypeScript
**Rationale:**
- Preact is a 3 kB React-compatible virtual DOM library — small bundle, fast diffing, no extra framework overhead.
- esbuild (invoked via its Go API) bundles the TypeScript/TSX source in milliseconds with no Node.js or npm dependency at build time.
- TypeScript catches signal subscription and widget property type errors at build time.
- All vendor JS/CSS is checked into `web/vendor/` so the repo builds without internet access.
**Key dependencies (vendored):**
| Package | Purpose |
| ---------------- | --------------------------------------------------------------- |
| `preact` 10 | Virtual DOM UI framework |
| `uPlot` | Extremely fast time-series/line plot (canvas-based, < 40 kB) |
| `Apache ECharts` | FFT, waterfall, histogram, bar, logic analyser plots |
| `uplot.css` | uPlot default stylesheet |
**Intentionally excluded:** React, Vue, Svelte, Konva, WebGPU, jQuery, npm at runtime.
---
## 2. Repository Layout
```
uopi/
├── cmd/uopi/ # main package — CLI flags, wiring
├── internal/
│ ├── server/ # HTTP + WebSocket handlers
│ ├── broker/ # signal fan-out to clients
│ ├── datasource/
│ │ ├── iface.go # DataSource interface
│ │ ├── epics/ # EPICS CA implementation (CGo)
│ │ └── synthetic/ # synthetic signal engine + DSP bridge
│ ├── dsp/ # DSP node implementations (lowpass, MA, etc.)
│ ├── storage/ # interface XML read/write
│ └── api/ # REST handler functions
├── web/
│ ├── embed.go # //go:embed dist — exports FS to Go
│ ├── src/ # TypeScript/TSX source (Preact)
│ │ ├── lib/
│ │ │ ├── ws.ts # WebSocket client + subscription manager
│ │ │ ├── stores.ts # signal value + metadata stores
│ │ │ ├── types.ts # shared TypeScript interfaces
│ │ │ ├── xml.ts # interface XML parse/serialize
│ │ │ └── format.ts # value formatting helpers
│ │ ├── widgets/ # one .tsx file per widget type
│ │ ├── App.tsx # top-level component, mode routing
│ │ ├── ViewMode.tsx # view mode layout + tabs
│ │ ├── EditMode.tsx # edit mode layout + toolbar
│ │ ├── Canvas.tsx # live HMI canvas (view mode)
│ │ ├── EditCanvas.tsx # free-form widget editor canvas
│ │ ├── PlotPanel.tsx # live plot side-panel (Plot tab)
│ │ ├── InfoPanel.tsx # signal info side-panel
│ │ ├── ZoomControl.tsx # UI zoom A-/A+ control
│ │ ├── SyntheticWizard.tsx # new synthetic signal dialog
│ │ ├── SyntheticEditor.tsx # edit existing synthetic signal
│ │ ├── LuaEditor.tsx # Lua code editor with syntax highlight
│ │ └── styles.css # all component styles
│ ├── vendor/ # vendored JS/CSS (preact, uplot, echarts)
│ └── dist/ # built frontend — generated, not committed
├── tools/buildfrontend/ # esbuild Go API bundler (go generate)
├── docs/ # specs, work plan
├── CLAUDE.md
└── README.md
```
The embed package lives at `web/embed.go` (not in `cmd/`) because `//go:embed` paths cannot use `..`.
---
## 3. Backend Architecture
### 3.1 DataSource Interface
```go
type Value struct {
Timestamp time.Time
Data any // float64 | []float64 | string | int64 | bool
Quality Quality // Good | Bad | Uncertain
}
type Metadata struct {
Name string
Type DataType
Unit string
DisplayLow float64
DisplayHigh float64
DriveHigh float64
DriveLow float64
EnumStrings []string
Writable bool
}
type DataSource interface {
Name() string
Connect(ctx context.Context) error
ListSignals(ctx context.Context) ([]Metadata, error)
GetMetadata(ctx context.Context, signal string) (Metadata, error)
Subscribe(ctx context.Context, signal string, ch chan<- Value) (CancelFunc, error)
Write(ctx context.Context, signal string, value any) error
History(ctx context.Context, signal string, start, end time.Time, maxPoints int) ([]Value, error)
}
```
New data sources are registered at startup via `datasource.Register(name string, ds DataSource)`.
### 3.2 Signal Broker
The broker is the central fan-out component:
```
DataSource ──subscribe──► rawCh ──► Broker ──► [clientCh1, clientCh2, ...]
```
- One goroutine per active signal subscription to the underlying data source.
- Per-signal subscriber list protected by a sync.RWMutex.
- When the last client unsubscribes, the broker cancels the upstream subscription.
- No data is buffered in the broker; clients receive the latest value at the moment they subscribe and all subsequent updates.
### 3.3 WebSocket Protocol
Framing: JSON messages over a single persistent WebSocket connection per client.
**Client → Server messages:**
```jsonc
// Subscribe to one or more signals
{ "type": "subscribe", "signals": ["EPICS:PV1", "synth:mySignal"] }
// Unsubscribe
{ "type": "unsubscribe", "signals": ["EPICS:PV1"] }
// Write a value
{ "type": "write", "signal": "EPICS:PV1", "value": 3.14 }
// Request historical data
{ "type": "history", "signal": "EPICS:PV1", "start": "2026-01-01T00:00:00Z", "end": "2026-01-02T00:00:00Z", "maxPoints": 5000 }
```
**Server → Client messages:**
```jsonc
// Live value update
{ "type": "update", "signal": "EPICS:PV1", "ts": "2026-04-24T12:00:00.123Z", "value": 42.7, "quality": "good" }
// Metadata (sent once on first subscribe)
{ "type": "meta", "signal": "EPICS:PV1", "meta": { "unit": "A", "displayLow": 0, "displayHigh": 100, ... } }
// Historical data response
{ "type": "history", "signal": "EPICS:PV1", "points": [ { "ts": "...", "value": 1.2 }, ... ] }
// Error
{ "type": "error", "code": "NOT_FOUND", "message": "Signal not found" }
```
### 3.4 REST API
Base path: `/api/v1`
| Method | Path | Description |
| --------------- | ------------------------------------- | ------------------------------------------------- |
| GET | `/me` | Caller identity, global level, groups, `canEditLogic` |
| GET | `/datasources` | List connected data sources and their status |
| GET | `/signals?ds=epics` | List signals for a data source |
| GET | `/signals/search?q=` | Search signals across all sources |
| GET | `/channel-finder?q=` | Proxy to EPICS Channel Finder |
| GET | `/archiver/search?q=` | Search the EPICS archiver for PV names |
| GET, POST | `/interfaces` | List saved interfaces / create one (body: XML) |
| POST | `/interfaces/reorder` | Reorder panels / move between folders |
| GET, PUT, DELETE| `/interfaces/{id}` | Download, update, or delete an interface |
| POST | `/interfaces/{id}/clone` | Clone an interface |
| GET | `/interfaces/{id}/versions` | List saved versions; `…/{version}` to fetch one |
| PUT | `/interfaces/{id}/versions/{v}/tag` | Tag a version |
| POST | `/interfaces/{id}/versions/{v}/promote` | Promote a version to current |
| POST | `/interfaces/{id}/versions/{v}/fork` | Fork a version into a new panel |
| GET, PUT | `/interfaces/{id}/acl` | Read or set a panel's sharing rules |
| GET, POST | `/folders` | List or create panel folders |
| PUT, DELETE | `/folders/{id}` | Rename/reparent or delete a folder |
| GET | `/usergroups` | List configured users and groups (for sharing) |
| GET, PUT | `/groups` | Read or set group definitions |
| GET, POST | `/synthetic` | List or create synthetic signal definitions |
| GET, PUT, DELETE| `/synthetic/{name}` | Read, update, or delete a synthetic definition |
| GET, POST | `/controllogic` | List or create server-side control-logic graphs |
| GET, PUT, DELETE| `/controllogic/{id}` | Read, update, or delete a control-logic graph |
Mutating requests are gated by the access middleware (§8): global level for writes,
per-panel ACL for interface endpoints, and the logic-editor allowlist for control-logic
endpoints and for any change to a panel's `<logic>` block.
### 3.5 EPICS Data Source
- Uses CGo bindings to EPICS Base `libca` (Channel Access).
- Channel connections are lazy: connected on first Subscribe, disconnected when the broker releases it.
- On connect, a `ca_get` retrieves full DBR_CTRL metadata (units, limits, enum strings).
- `ca_add_event` sets up the monitor. Update callbacks push into the broker's raw channel.
- Multiple PV subscriptions share one CA context per data source instance (thread-safe with `ca_attach_context`).
- EPICS Archive Appliance is queried via its JSON HTTP API for history requests.
### 3.6 Synthetic Data Source
- Each synthetic signal is defined as a directed acyclic graph (DAG) of processing nodes.
- Processing nodes are re-evaluated whenever any upstream signal emits a new value.
- Definitions are stored in a configurable JSON/TOML file alongside server configuration.
- The `dsp_bridge.go` file maps node type names to `dsp.Node` implementations.
**Built-in node types:**
| Node type | Parameters | Description |
| ------------ | ----------------------------------- | ------------------------------------------------ |
| `source` | `ds`, `name` | Reads a signal from any data source |
| `gain` | `factor` | Multiplies by a constant |
| `offset` | `value` | Adds a constant |
| `moving_avg` | `window` (samples) | Rolling mean |
| `lowpass` | `freq` (Hz), `order` (18) | Cascaded IIR Butterworth-style low-pass filter |
| `formula` | `expr` | Inline math expression (variables: `a`, `b`, …) |
| `lua` | `script` | Arbitrary Lua 5.1 code with persistent state |
**Low-pass filter implementation:** Cascaded first-order IIR sections. Each stage computes `y = y_prev + α·(x y_prev)` where `α = dt / (RC + dt)` and `RC = 1/(2π·fc)`. `dt` is computed per sample from source timestamps so the filter is correct for event-driven (non-uniform) data.
**Lua node:** Receives `inputs` table (indexed by signal name) and a persistent `state` table across calls. The `os`, `io`, `package`, and `debug` libraries are disabled.
### 3.7 Interface Storage
Interfaces are stored as XML files in a configurable directory on the server. The
`<interface>` element carries an optional `kind` attribute (`panel` default, or `plot`);
plot panels add a nested `<layout>` split tree referencing plot widgets by id. Beyond
widgets, an interface may also contain panel-local variables and a `<logic>` flow graph
(`<node>` + `<param>` + `<wire>`), all round-tripping through the same XML.
Per-panel access rules and folder placement are kept in a sidecar `acl.json` in the same
directory (not in the interface XML), and prior versions of each panel are retained for the
version history endpoints.
```xml
<interface name="My Panel" version="1" created="2026-04-24T12:00:00Z">
<widget id="w1" type="plot" x="100" y="200" w="600" h="300">
<signal ds="epics" name="EPICS:CURRENT" color="#ff0000"/>
<signal ds="epics" name="EPICS:VOLTAGE" color="#0000ff"/>
<option key="plotType" value="timeseries"/>
<option key="yMin" value="auto"/>
<option key="yMax" value="auto"/>
<option key="timeWindow" value="60"/>
<option key="legend" value="bottom"/>
</widget>
<widget id="w2" type="led" x="50" y="50" w="80" h="80">
<signal ds="epics" name="EPICS:STATUS"/>
<option key="condition" value="value &gt; 0"/>
<option key="colorTrue" value="#00ff00"/>
<option key="colorFalse" value="#ff0000"/>
<option key="label" value="OK"/>
</widget>
</interface>
```
### 3.8 Panel Logic Engine
Panel logic is a client-side flow engine (`web/src/lib/logic.ts`, singleton `logicEngine`).
The graph is built/edited in the panel editor's Logic tab (`LogicEditor.tsx`) and serialized
into the interface XML. In View mode, `Canvas` calls `logicEngine.load(iface.logic)` on
mount and `clear()` on unmount. The engine subscribes to every referenced signal into a live
cache, then on each trigger activation walks the wired graph (gates, if/loop, actions),
evaluating expression fields via a small safe recursive-descent evaluator (no `eval`).
Triggers include button/threshold/change/timer/loop and On-open/On-close lifecycle; actions
include signal writes, delay, log, in-memory data arrays (accumulate/export-CSV/clear) and
user dialogs (info/error/set-point). The engine runs entirely in the browser — no backend
component.
### 3.9 Control Logic Engine
Control logic is server-side (`internal/controllogic`): always-on flow graphs that run under
the root context, independent of any client. The engine subscribes to the broker, fires on
cron schedules and signal alarm/threshold conditions, executes a Lua block for custom logic,
and writes results back to signals. Graphs are persisted by a store and managed via
`/api/v1/controllogic`; each mutation calls `Engine.Reload()` to apply changes live. Graphs
can be individually enabled/disabled.
### 3.10 Access Control
Identity and global policy live in `internal/access` (`Policy`, `Level`). The user identity
is read per request from `server.trusted_user_header` (with a `default_user` fallback) and
stored on the request context. `accessMiddleware` gates mutating HTTP methods by the user's
global level. Per-panel ownership and ACL evaluation (with folder inheritance) live in
`internal/panelacl`, backed by the `acl.json` sidecar. `Policy.CanEditLogic(user)` enforces
the optional `server.logic_editors` allowlist over control-logic endpoints and over any
change to a panel's `<logic>` block; it is surfaced to the frontend through `/api/v1/me`
(`canEditLogic`) so the UI can hide logic-editing affordances.
---
## 4. Frontend Architecture
### 4.1 WebSocket Client (`lib/ws.ts`)
- Singleton WebSocket connection, reconnects with exponential back-off.
- Subscription reference counting: multiple widgets subscribing to the same signal result in one server subscription message.
- Incoming updates are dispatched to per-signal stores.
- `wsClient.history(sig, start, end, maxPoints)` returns a Promise resolving to timestamped point arrays.
### 4.2 Signal Stores (`lib/stores.ts`)
```typescript
// One nanostores atom per subscribed signal
const signalStores = new Map<string, SignalStore>();
export function getSignalStore(ref: SignalRef): SignalStore { ... }
export function getMetaStore(ref: SignalRef): MetaStore { ... }
```
Widgets subscribe to stores directly; store updates trigger re-renders only in the consuming component.
### 4.3 Edit Mode Canvas (`EditCanvas.tsx`)
The edit canvas is a free-form HTML div with absolutely positioned widget components:
- Each widget renders as an absolutely positioned `<div>` at `(x, y)` with `(w, h)` dimensions.
- Selection shows a CSS-outlined bounding box with 8 resize handles rendered as small squares.
- Drag-and-drop from the signal tree uses the HTML Drag-and-Drop API; on drop, the canvas coordinate is computed from the drop event offset.
- Undo/redo uses an array of past interface snapshots (max depth 50).
- Align/distribute operations compute target positions geometrically and generate a single undo entry.
- Multi-select via Ctrl+click or rubber-band area select.
### 4.4 Widget Rendering in View Mode (`Canvas.tsx`)
View mode renders widgets as absolutely positioned Preact components on a scrollable canvas div:
- Each widget subscribes to its signal store(s) in a `useEffect` and re-renders only when values change.
- uPlot (time series) and ECharts (histogram, bar, FFT, waterfall, logic analyser) manage their own canvas elements inside their widget component.
- Plot widgets maintain a rolling ring buffer of 200,000 samples per signal for smooth long-window display.
- Step-hold interpolation: when multiple signals at different update rates share a plot, the most recent value is carried forward to fill the shared time axis correctly.
### 4.5 Plot Panels (`SplitLayout.tsx`, `PlotPanelCanvas.tsx`)
Plot panels are interfaces of `kind === 'plot'` whose plots fill the viewport in a recursive
split layout (`lib/plotLayout.ts` holds the pure tree helpers; `SplitLayout.tsx` is the
shared presentational renderer with draggable dividers):
- In View mode, `Canvas` renders the saved layout read-only with live `PlotWidget`s.
- In Edit mode, `PlotPanelCanvas.tsx` adds per-pane overlays: split (⬌/⬍), close (✕), click
to select, and signal-drop to add a signal to that pane's plot.
- Each pane reuses the standard `PlotWidget`, so all plot sub-types and options apply.
- Layout edits (split/close/resize) participate in the editor's undo/redo.
### 4.6 Resizable Panels
Both edit and view modes support mouse-drag panel resizing:
- **View mode**: drag handle between the interface list pane and the main content area.
- **Edit mode**: drag handles on both sides of the central canvas (signal tree ↔ canvas, canvas ↔ properties pane).
- Handle width: 5 px, cursor changes to `ew-resize` on hover.
- Minimum panel widths enforced to prevent collapse below usable size.
### 4.7 HiDPI / Zoom Support
- `html { font-size: clamp(13px, 1.5vh, 18px); }` — base font scales with viewport height, making the UI naturally larger on 4K screens where the browser zoom level is 100%.
- Key structural heights (toolbar, panel headers, tab bar, plot toolbar) are expressed in `rem` so they scale with the base font.
- **ZoomControl** (A / % / A+) in the toolbar lets users manually override the zoom level in 11 steps from 50% to 250%. The preference is persisted in `localStorage` (`uopi:ui-zoom`) and applied by setting `document.documentElement.style.fontSize` on load.
- Canvas pixel rendering (uPlot, ECharts) reads `window.devicePixelRatio` and sizes canvases accordingly.
### 4.8 Lua Editor (`LuaEditor.tsx`)
A syntax-highlighted code editor for Lua scripts in the Synthetic signal wizard:
- Implemented as a `<textarea>` overlaid on a `<pre>` element; the textarea has `color: transparent; caret-color: #e2e8f0` so only the caret is visible — the `<pre>` provides the coloured text behind it.
- Tokeniser handles: `--` line comments, `"..."` / `'...'` string literals, `[[...]]` long strings, hex and float numeric literals, and all Lua 5.1 keywords.
- Scroll position is synchronised between textarea and pre on every scroll event.
---
## 5. Build System
### 5.1 Backend
```bash
# Full build (frontend then backend)
make all
# Backend only (frontend must already be built)
make backend
# All tests
make test
# Single Go test
go test ./internal/broker/... -run TestFanOut
# Go vet
go vet ./...
```
EPICS `libca.a` is statically linked via `CGO_LDFLAGS` in `internal/datasource/epics/cgo.go`.
### 5.2 Frontend
The frontend is built by a Go tool in `tools/buildfrontend/` that invokes the esbuild Go API:
```bash
make frontend
# or equivalently:
go generate ./web/...
```
No Node.js, npm, or any JS build tool is required on the host. The bundler:
- Reads entry point `web/src/main.tsx`.
- Resolves `preact`, `uplot`, and `echarts` imports from `web/vendor/`.
- Outputs `web/dist/main.js` and `web/dist/main.css`.
- Copies `web/dist/index.html`, vendor CSS, and other static assets.
### 5.3 Combined Build
```makefile
.PHONY: all frontend backend clean
all: frontend backend
frontend:
go run ./tools/buildfrontend
backend:
CGO_ENABLED=1 go build -ldflags="-s -w" -o dist/uopi ./cmd/uopi
test:
go test ./...
clean:
rm -rf dist/ web/dist/
```
The backend's `//go:embed dist` directive in `web/embed.go` picks up the built frontend automatically. `main.go` does `fs.Sub(web.FS, "dist")` to serve a clean root.
---
## 6. Configuration
Server is configured via a TOML file (default: `uopi.toml`, overridable via `--config` flag):
```toml
[server]
listen = ":8080"
storage_dir = "./interfaces"
# Access control (all optional)
trusted_user_header = "" # header carrying the proxy-authenticated user
default_user = "" # identity when the header is absent (LAN/dev)
logic_editors = [] # users/groups allowed to edit panel & control logic
# [[server.blacklist]] # downgrade users: level = "readonly" | "noaccess"
# user = "guest"
# level = "readonly"
# [[groups]] # named user sets for per-panel sharing
# name = "operators"
# members = ["alice", "bob"]
[datasource.epics]
enabled = true
ca_addr_list = "" # EPICS_CA_ADDR_LIST override
archive_url = "" # EPICS Archive Appliance URL
channel_finder_url = "" # EPICS Channel Finder URL
[datasource.synthetic]
enabled = true
```
All settings can also be overridden with `UOPI_*` environment variables (e.g.
`UOPI_SERVER_LISTEN`, `UOPI_SERVER_LOGIC_EDITORS`, `UOPI_EPICS_CA_ADDR_LIST`).
---
## 7. Testing Strategy
| Layer | Approach |
| ------------------ | --------------------------------------------------------------------------------- |
| Broker | Unit tests with mock data source; verify fan-out, subscribe/unsubscribe lifecycle |
| Synthetic DSP | Table-driven unit tests against known signal inputs/outputs |
| Low-pass filter | Unit tests: step response, frequency attenuation vs. analytical expectation |
| Lua sandbox | Unit tests for sandbox isolation and API surface |
| REST API | `httptest` integration tests |
| WebSocket protocol | Integration tests with a test client |
| EPICS data source | Integration tests against a local SoftIOC (optional, CI-gated) |
---
## 8. Security Considerations
- Lua sandbox: disable `os`, `io`, `package`, `debug` libraries; restrict `math` and `string` to safe subsets.
- WebSocket write operations: validate that the target signal is writable before forwarding to the data source.
- Interface XML parsing: use strict schema validation to prevent XXE.
- **Identity & access control:** the end-user identity is taken from a header set by a
trusted authenticating reverse proxy (`trusted_user_header`), never from client-supplied
values — the proxy MUST strip any inbound copy of that header or it can be spoofed. A
global blacklist downgrades users (read-only/no-access); per-panel ACLs and the optional
`logic_editors` allowlist provide finer control. An unidentified caller (no header, no
`default_user`) is treated as a trusted-LAN user with full write access, preserving the
unproxied/SSH-tunnel deployment model.
---
## 9. Non-goals (v1)
- Built-in user authentication (a login page / credential store) — identity is delegated to
the front-end reverse proxy; authorisation (levels, ACLs, logic allowlist) is implemented.
- TLS termination (expected to be handled by SSH tunnel or a reverse proxy).
- Windows or macOS server binary.
- Mobile-optimised frontend layout.
- Remote plugin loading (plugins compiled in at build time only).