uopi/docs/superpowers/specs/2026-06-24-local-arrays-design.md

# Design: Local array values for the node-editor flow engines

**Date:** 2026-06-24
**Status:** Approved (design phase)
**TODO refs:** "Logic editor → add full support to local array values: dynamic,
dynamic but capped max, fixed size etc; array functions should work with new
local array" and "Control loop → add full support to server side array values".

## 1. Goal & scope

Add first-class **array-valued local variables** to both flow engines:

- **Panel logic** (client TS): `web/src/lib/logic.ts`, `LogicEditor.tsx`,
  `web/src/lib/types.ts`, `web/src/lib/expr.ts`, `web/src/lib/localstate.ts`,
  `web/src/lib/xml.ts`.
- **Control logic** (server Go): `internal/controllogic/` (`model.go`,
  `engine.go`, `expr.go`), editor `web/src/ControlLogicEditor.tsx`.

**Build order:** design both together; implement **panel logic (TS) first**,
then port the same design to control logic (Go). The panel `StateVar` schema is
the richer reference and doing TS first de-risks the expression-language change
before the Go port.

**In scope for v1:** declaration + sizing policies, an array-aware expression
language (shared by both engines), array mutation nodes, persistence, **and
widget binding** (multi-LED/bitset, table, plot read array locals).

## 2. Data model — array local declaration

A new array kind on the existing `StateVar` (TS `types.ts`; mirrored as a Go
struct in control logic, which gains state-var declarations for the first time):

```
StateVar {
  name: string
  type: 'number' | 'bool' | 'string' | 'array'   // 'array' is new
  initial: string            // arrays: JSON literal e.g. "[0,0,0]" / "[[1,2],[3,4]]", or "" = empty/zero-fill
  unit?, low?, high?         // existing scalar fields

  // present only when type === 'array':
  elem?: 'number' | 'bool' | 'array'  // element type; 'array' ⇒ nested (recursive, arbitrary depth, jagged allowed)
  sizing?: 'dynamic' | 'capped' | 'fixed'   // default 'dynamic'
  capacity?: number          // required for capped/fixed
}
```

Semantics:

- **dynamic** — unbounded, guarded by a global safety cap (1e6 elements) to
  prevent runaway growth.
- **capped** — `capacity` max; pushing past full **drops the oldest** element
  (ring / FIFO).
- **fixed** — exactly `capacity` slots. Initialised from the `initial` literal
  (truncated / zero-padded to `capacity`), else **zero-filled**. `push` is a
  no-op (write via index/set); `clear` zero-refills rather than emptying.
- **initial** — non-empty JSON literal is parsed and used; empty ⇒ dynamic and
  capped start `[]`, fixed starts zero-filled.
- `elem: 'bool'` is **display metadata only**. Runtime leaves are numeric
  `1`/`0` (consistent with the engine's existing "booleans are 1/0" rule);
  widgets use the declaration to render on/off.

**Declaration-time validation:** capped/fixed require `capacity >= 1`;
`initial`, if non-empty, must parse as JSON and match the declared element
type / nesting. Errors surface inline in the editor.

## 3. Expression engine (`expr.ts` + `expr.go`)

**Value model (tagged union — "Approach 1"):**

- TS: `type EvalValue = number | EvalValue[]`. The evaluator returns
  `EvalValue`; functions/indexing type-check at runtime and throw on misuse
  (caught → node error badge).
- Go: a boxed `value{ num float64; arr []value; isArr bool }`. `Resolver`
  returns `value`; every `*Node.eval` returns `value` (a contained, mechanical
  refactor of `expr.go`, which today returns `float64`). The `float64` leaf
  stays the fast path.

**New syntax (both parsers):**

- **Array literal:** `[a, b, c]`, nested `[[1,2],[3,4]]`.
- **Indexing:** postfix `expr[expr]`, chainable `a[i][j]`. Index rounds to int;
  **negative index counts from the end** (`a[-1]` = last); out-of-range → node
  error.

**Functions** — added to the existing `abs/min/max/sqrt/...` table. All the
read/transform functions are **pure** (return new values; never mutate a local):

| Function | Result | Meaning / notes |
|---|---|---|
| `len(a)` | number | element count (top level) |
| `sum(a)`, `mean(a)`, `min(a)`, `max(a)` | number | over a 1-D numeric array; error if elements are arrays |
| `slice(a, s, e)` | array | subrange, `e` exclusive, negative indices allowed |
| `concat(a, b)` | array | join |
| `reverse(a)` | array | |
| `sort(a)` | array | ascending numeric |
| `scale(a, k)` | array | element-wise `a[i]*k` |
| `add(a, b)`, `sub(a, b)` | array | element-wise pairwise; length = min(len a, len b) |
| `push(a, v)` | array | copy with `v` appended |
| `set(a, i, v)` | array | copy with element `i` replaced (negative `i` ok) |
| `insert(a, i, v)` | array | copy with `v` inserted at `i` |
| `remove(a, i)` | array | copy without element `i` |
| `pop(a)` | array | copy without the last element (read it with `a[-1]`) |
| `shift(a)` | array | copy without the first element (read with `a[0]`) |
| `indexOf(a, v)` | number | first index of `v`, else `-1` |
| `contains(a, v)` | number | `1`/`0` |
| `fill(n, v)` | array | new length-`n` array of `v` |

- `min`/`max` keep their existing **scalar variadic** form (`min(x,y,z)`) and
  gain a 1-arg **array** form (`min(a)`) — dispatch on arg count + type.
- **Resolution:** a bare identifier naming an array local returns the whole
  array value; `{ds:sig}` waveform signals (EPICS `float64[]`) become first-class
  array values usable by every function above.

**Purity & persistence interplay:** the mutator-named functions
(`push/set/insert/remove/pop/shift`) are **immutable transforms** — they return a
new array and do not touch the local. You persist a result by writing it back;
the **sizing policy is enforced at store time** in `writeLocalState` (TS) /
`setLocal` (Go) whenever the write target is a typed array local (ring-drop for
capped, clamp / no-op for fixed). The mutation nodes (§4) are convenient, visible
sugar for "store with policy".

**Errors** (e.g. `sum` of nested array, indexing a scalar, `add` of non-arrays)
throw in the evaluator and surface as the node's error reason via the existing
`checkExpr` validation + runtime-catch / badge path.

**Ref-collection** (`collectRefs` / `CollectRefs`) walks the new literal/index
AST so subscriptions still discover every `{ds:sig}` inside array expressions.

## 4. Mutation nodes + accumulate/export unification

**New action nodes** (panel `LogicNodeKind`, mirrored in Go control-logic kinds):

- `action.array.push{array, expr}` — append `eval(expr)`; sizing-policy aware
  (ring-drop if capped; no-op if fixed).
- `action.array.set{array, index, expr}` — store at `eval(index)`. Supports
  **nested targets via an index path**: `index = "i, j"` ⇒ `a[i][j]`. Negative
  indices allowed; out-of-range → node error. (This is the imperative
  path-assignment style.)
- `action.array.remove{array, index}` — remove element at index (in place).
- `action.array.pop{array}` — remove last element (in place).
- `action.array.clear{array}` — empty (dynamic/capped) or zero-refill (fixed).

These are the sizing-policy-aware, in-place counterparts to the pure expression
functions.

**Unification of the existing `{t,v}` array system (decision: unify):**

- `action.accumulate{array, expr}` → reframed as `action.array.push` (append,
  policy-enforced). Old kind **kept as a compile alias** so saved panels run.
- `action.clear{array}` → `action.array.clear` (alias retained).
- `action.export{columns, align, filename}` → serializes **array locals** by
  column. Array locals are plain numeric (no per-sample `t`), so **time-based
  alignment (`common`/`any`/`interpolate`) is dropped**; columns are emitted
  **side-by-side by index** (ragged columns padded blank). The `align` param is
  ignored and hidden in the inspector. To keep a timestamp column, push
  `{sys:time}` into a parallel array local and add it as a column.
- **Custom column names:** each export column keeps its `label`, surfaced as an
  **editable header name** in the inspector (default = array-local name); the CSV
  header row uses the chosen names.

**Migration (non-destructive, at engine `load()`):** any
`accumulate`/`clear`/`export` node referencing an array name with **no** matching
`StateVar` declaration triggers an **auto-declared dynamic numeric array local**
of that name. No file rewrite.

## 5. Persistence

**Panel logic (XML, `xml.ts`):** `<statevar>` gains optional array attributes,
written only for arrays:

```xml
<statevar name="hist" type="array" elem="number" sizing="capped" capacity="100" initial=""/>
<statevar name="grid" type="array" elem="array" sizing="fixed" capacity="4" initial="[[0,0],[0,0]]"/>
```

Round-trips through the existing verbatim-body store (no Go change for panels).
New nodes serialize via the existing `<node><param/></node>` mechanism.

**Control logic (Go, `model.go`):** control logic has **no** state-var
declarations today (`locals` is an untyped `map[string]float64`). Additions:

- `Graph` gains `StateVars []StateVar` (Go struct mirroring the TS shape:
  `Name, Type, Elem, Sizing string; Capacity int; Initial, Unit string; Low,
  High float64`), serialized in `controllogic.json`.
- `compiledGraph.locals` changes from `map[string]float64` to
  `map[string]value`, initialised from `StateVars` (applying sizing/initial) at
  compile time.
- `getLocal`/`setLocal` operate on `value`; `setLocal` enforces sizing policy.
  Config-apply / snapshot paths that read/write locals as `float64` box/unbox.

**Versioning:** control-logic graphs are already git-style versioned; the new
`StateVars` field rides along in each revision (no diff-engine change — just more
JSON).

## 6. Editor UI + debug/live badges

**Panel `LogicEditor.tsx`:** the `LocalVars` palette subcomponent gains an array
declaration form (type=array reveals element-type / sizing / capacity / `initial`
JSON with inline validation). New array action nodes added to the Actions palette
group with inspectors (array name + expr/index fields, reusing `ExprField` with
array-aware `checkExpr`).

**Control `ControlLogicEditor.tsx`:** control logic has **no** local-var
declaration UI today. Add a `LocalVars` panel mirroring the panel editor (same
component, driven by `Graph.StateVars`) plus the array nodes in its palette. This
brings control-logic locals to parity — scalars *and* arrays become declarable
there for the first time.

**Debug/live badges (`flowDebug.ts` + Go `DebugObserver` / synthetic trace):**
array node values render as a truncated literal, e.g. `[1, 2, 3, …](n=100)`. The
Go debug event payload (`debugNode`) and the synthetic trace already serialize a
value — extended to carry array JSON; the badge formatter stringifies arrays
compactly.

## 7. Widgets reading array locals

The `ds:'local'` plumbing already routes through `stores.ts`/`ws.ts`; the change
is that a local's `SignalValue.value` can be an array (number / nested), held and
initialised per panel instance by `localstate.ts`. No new widget types — new
source modes on three existing widgets:

- **Plot** — a 1-D numeric array local binds as a **waveform sample** (same path
  EPICS `float64[]` waveforms already use for multidimensional/FFT/waterfall).
  Each engine tick that rewrites the array updates the trace. Nested arrays show
  "unsupported shape".
- **Table widget** — gains an **array source mode**: bound to one array local,
  renders **one row per element** (index + value, per-signal value-format applied).
  For an `elem:'array'` (2-D) local, rows are indices and the configured columns
  map to inner-array positions. Falls back to multi-signal mode for scalars.
- **Multi-LED / bitset** — gains an **array source mode**: one LED per element,
  lit per element truthiness; when `elem:'bool'`, on/off labels come from the
  declaration. Existing integer-bitset mode unchanged.

`getLocalMetaStore` carries `elem`/`sizing`/`capacity` so widgets self-configure
(e.g. multi-LED LED count = array length, growing/shrinking live for dynamic
arrays).

## 8. Testing

- **TS unit (expr):** literals; indexing (negative, nested, out-of-range error);
  every new function incl. type-error cases; sizing-policy enforcement on store
  (ring drop-oldest, fixed no-op / zero-refill); accumulate→push migration; CSV
  export by-index with custom headers.
- **Go unit (`internal/controllogic`):** port of the expr suite (boxed `value`,
  all functions/indexing/errors); `StateVars` init from declarations; `setLocal`
  policy enforcement; JSON round-trip of `StateVars`; config-apply/snapshot with
  boxed locals.
- **Cross-engine parity:** a `(expr, expected)` table asserted identical in both
  `expr.ts` and `expr.go` to keep them in lockstep.
- **Widgets:** light component tests for the new array source modes if widget
  tests exist; else manual verification.
- **Gates:** `gofmt`, `go vet`, `go test ./... -race`, frontend typecheck/build.

## 9. Risks

- **Go `expr.go` refactor (`float64` → boxed `value`):** touches every node's
  `eval` and the `Resolver`. Mechanical but broad; the parity test guards
  behavioral drift from `expr.ts`.
- **Backward compatibility of `accumulate`/`export`:** aliasing + auto-declared
  locals keep old panels running, but the **dropped time-alignment in export** is
  a behavioral change for any panel relying on `interpolate`/`common` align. Call
  this out in release notes.
- **Two engines staying in lockstep:** the function set and semantics must match
  exactly across TS and Go; the cross-engine parity fixture is the safeguard.
- **Hot-path purity:** array expression functions are evaluated repeatedly; they
  must remain allocation-light and side-effect-free (mutation only at store time).