he11lib/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this is

`he11lib` reconstructs the Laguerre-Gauss (LG) modal content ("mode purity") of a
free-space-propagating gyrotron RF beam from a set of thermal (flux) images taken at
different distances from the output window. It accounts for camera geometry (unknown
pixel scale / oblique viewing angle), sensor noise, target thermal-diffusion blur, and
unknown beam center/pointing.

The full design rationale lives in
`docs/superpowers/specs/2026-07-02-gyrotron-mode-purity-design.md` — read it before
making architectural changes. `docs/api.md` is the API reference for the implemented
public interface; keep it in sync when changing public signatures.

## Commands

```bash
pip install -e ".[dev]"   # editable install with test dependencies (numpy, scipy, matplotlib, pytest)
pytest                     # run the full test suite (discovers tests/, per pyproject.toml)
pytest tests/test_modes.py                          # run one test file
pytest tests/test_modes.py::test_field_at_waist      # run one test
python examples/full_pipeline_example.py             # runnable end-to-end demo
```

There is a project `.venv`; activate it or otherwise ensure the editable install's
dependencies are available before running tests.

`tests/conftest.py` forces the `Agg` matplotlib backend so plotting tests run headless.

This project is not (yet) a git repository.

## Architecture

Data flows through the pipeline as a list of `MeasurementPlane` (one per imaging
distance `z`), each holding a raw 2D `flux` array plus a nominal `z` and `z_tolerance`.
Camera pose/intrinsics are a single shared `CameraModel`/`CameraModelTolerance` for the
whole reconstruction, not per-plane. Everything downstream is keyed off `LGBasis`, which
defines the mode basis relative to a known waist `w0`/`z0`/`wavelength`.

Module responsibilities (`he11lib/`):

- **`data.py`** — `MeasurementPlane` (flux, nominal `z`, `z_tolerance`), `ReconstructionResult`
  (the shared input/output types) and `validate_planes` (>=3 planes, matching shapes,
  distinct `z`).
- **`modes.py`** — `LGBasis`: closed-form paraxial LG fields, beam radius `w(z)`,
  Gouy phase, inverse radius of curvature, and projection of a measured field onto a
  candidate mode set. This is the analytic ground truth all fitting is checked against.
- **`geometry.py`** — `CameraModel`/`CameraModelTolerance` (a nominal pinhole camera
  pose/intrinsics and its paired per-field refinement bound) and `GeometryCalibration`:
  resolves pixel<->physical coordinates via true pinhole forward/inverse projection
  (ray-plane intersection), producing genuine keystoning for tilted/off-axis poses
  rather than a uniform affine correction. A tolerance of `0` on a `CameraModel` field
  means it's trusted exactly; `>0` means it's refined within `[nominal-tolerance,
  nominal+tolerance]` by `ModalFitter` — the same mechanism applies to
  `MeasurementPlane.z`/`z_tolerance`.
- **`noise.py`** — `NoiseEstimator`: automatic per-image noise-std estimation
  (Laplacian method) and per-pixel weights for noise-weighted least squares.
- **`deconvolution.py`** — `DiffusionDeconvolver`: optional forward blur / Wiener
  deconvolution for thermal-diffusion blur in the absorbing target. The blur kernel is
  isotropic in pixel space; callers use `GeometryCalibration.effective_pixel_scale`
  (a finite-difference approximation at the frame center) as a single figure, so it's
  only exact for an on-axis, untilted camera — an accepted approximation, not a bug.
- **`synthetic.py`** — `SyntheticBeamGenerator`: forward model that produces
  `MeasurementPlane`s from a known ground-truth `CameraModel`, coefficients, center,
  and two pointing angles, rendering each plane at its own true `z` (which may
  deliberately differ from the plane's nominal `z`, via `nominal_z_offsets`). Used
  throughout the test suite and examples to validate the pipeline end-to-end.
- **`fitting.py`** — `ModalFitter` (`fit`, `fit_auto`) and `generate_mode_shells`: the
  core joint nonlinear least-squares fit via `scipy.optimize.least_squares`. Complex LG
  coefficients, per-plane beam center, and the two pointing angles (horizontal/vertical)
  are always free; each `CameraModel` field and each plane's `z` is additionally free
  (bounded by its tolerance) only when its paired tolerance is nonzero, otherwise held
  fixed as a constant. `fit_auto` grows the candidate mode set shell-by-shell (by order
  `2p + |l|`), stopping via a BIC improvement threshold, capped at `max_order` (emits
  `UserWarning`, doesn't raise, if still improving at the cap, or if the free
  camera+`z` parameter count is large relative to the number of planes — see the
  degeneracy pitfall below).
- **`phase_retrieval.py`** — `propagate_angular_spectrum` (FFT-based paraxial
  free-space propagation) and `PhaseRetriever` (multi-plane Gerchberg-Saxton), the
  fallback reconstruction path for when a finite mode basis doesn't fit well. Takes a
  shared `CameraModel` (not per-plane pixel scale) to derive its common physical grid.
- **`reconstruct.py`** — `BeamReconstructor`: the orchestrator, now constructed with a
  required `camera`/`camera_tolerance`. Pipeline order: validate planes → optional
  deconvolution (using `GeometryCalibration(camera).effective_pixel_scale`) →
  `ModalFitter.fit_auto` → optional `PhaseRetriever` fallback (forced via
  `force_phase_retrieval`, or triggered automatically when the noise-weighted RMS
  residual exceeds `phase_retrieval_residual_threshold`). The fallback path projects
  the recovered field onto all modes up to `max_order` and produces a
  `ReconstructionResult` with `used_phase_retrieval=True`, empty `residuals`, empty
  `geometry`, NaN pointing angles, and NaN `coefficient_uncertainty` (no fit covariance
  available from phase retrieval).
- **`plotting.py`** — diagnostic figures (`plot_mode_purity`, `plot_center_trace`,
  `plot_residuals`); each returns a `Figure` rather than calling `plt.show()`.

Everything above is re-exported from the top-level `he11lib` package (see
`he11lib/__init__.py`); import from there rather than submodules.

## Known physics/fitting pitfalls (read before writing new tests or examples)

These aren't library bugs — they're consequences of realistic optics parameters and of
automatic order-selection being genuinely data-driven — but they've caused most of the
debugging time in this project's history:

1. **Rayleigh-range / frame clipping.** `w(z)` grows with `|z - z0|` relative to
   `zR = pi*w0**2/wavelength`. With typical test parameters (`w0=5e-3`,
   `wavelength=1.76e-3`), `zR` is only ~4.46 cm, so z-distances spanning tens of cm put
   the beam many Rayleigh ranges out, where `w(z)` can exceed a small test frame —
   clipping the beam and corrupting fits, or introducing FFT wraparound artifacts in
   `propagate_angular_spectrum`. Keep z-distances within roughly ±1-2 Rayleigh ranges of
   `z0`, or enlarge the frame/pixel_scale accordingly.
2. **Automatic mode-set growth can overfit deconvolution artifacts.** Wiener-deconvolved
   data always has some residual imperfection; `fit_auto`'s BIC-driven growth will try
   to "explain" it with spurious higher-order modes, degrading fitted beam
   center/pointing via parameter degeneracy (observed: pointing angle off by 4-6x at
   `max_order=3` vs. matching ground truth almost exactly at `max_order=1`, for the same
   2-mode ground truth). When demonstrating growth with deconvolution or noise, set
   `max_order` close to the true expected mode content rather than generously high,
   unless the test specifically targets growth behavior itself.
3. **Shared camera/`z` geometry can be underdetermined with few planes.** With only
   3-10 planes, adding the ~7-9 shared `CameraModel` unknowns (whichever have nonzero
   `CameraModelTolerance`) plus one `z` correction per plane (for nonzero
   `z_tolerance`) can be practically underdetermined even though each plane
   contributes many pixels of data, because those unknowns are *global* and only
   weakly constrained by subtle keystone differences between planes. `fit_auto`/
   `BeamReconstructor` emit a `UserWarning` (not an error) when the free-parameter
   count is large relative to the number of planes — if you see it, tighten
   `CameraModelTolerance`/`z_tolerance` toward values you actually trust rather than
   leaving them generously wide.