From 4d3ca49fbae4160026ca3977f35b52627a5146a3 Mon Sep 17 00:00:00 2001 From: arsen Date: Sun, 17 May 2026 10:46:36 +0700 Subject: [PATCH] =?UTF-8?q?docs:=20ADR-101=20/=20ADR-102=20/=20ADR-103=20?= =?UTF-8?q?=E2=80=94=20full=20session=20record?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * ADR-101 raw-amplitude presence/motion classifier — per-node and cross-node fusion logic, hysteresis, per-node UI surface (`PerNodeFeatureInfo.classification` override). * ADR-102 server-side NBVI subcarrier selection — formula, dead-zone gate, ESPectre Step-1 quiet-window finder, why we split FULL vs NBVI-subset broadband. * ADR-103 persistent baseline + universal threshold normalization — JSON schema v2 at `v2/data/baseline.json`, FULL-broadband over NBVI for cross-restart stability, `norm_cv = cv / baseline_cv` with universal 3×/6× gates, recording script workflow. * Updated espectre-techniques.md to reflect the DONE items (Steps 1+2+4 of NBVI, baseline persistence, universal threshold) and the remaining open items in priority order. Each ADR ≤ 200 lines per the operator's docs convention; deep detail lives in `docs/references/espectre-techniques.md` (also ≤ 200) which the ADRs link to. README.md and CLAUDE.md unchanged (no extra content added; existing >200-line state pre-dates this session). --- docs/adr/ADR-101-raw-amplitude-classifier.md | 145 ++++++++++++++ docs/adr/ADR-102-nbvi-subcarrier-selection.md | 138 +++++++++++++ docs/adr/ADR-103-persistent-baseline.md | 185 ++++++++++++++++++ docs/references/espectre-techniques.md | 106 +++++----- 4 files changed, 514 insertions(+), 60 deletions(-) create mode 100644 docs/adr/ADR-101-raw-amplitude-classifier.md create mode 100644 docs/adr/ADR-102-nbvi-subcarrier-selection.md create mode 100644 docs/adr/ADR-103-persistent-baseline.md diff --git a/docs/adr/ADR-101-raw-amplitude-classifier.md b/docs/adr/ADR-101-raw-amplitude-classifier.md new file mode 100644 index 00000000..28857296 --- /dev/null +++ b/docs/adr/ADR-101-raw-amplitude-classifier.md @@ -0,0 +1,145 @@ +# ADR-101 — Raw-Amplitude Presence/Motion Classifier + +**Status**: Accepted +**Date**: 2026-05-17 +**Scope**: `v2/crates/wifi-densepose-sensing-server/src/main.rs` +(`amp_presence_override`, `amp_classify_from_latest`, +`amp_node_level`, `amp_node_snapshot`). + +## Context + +After ADR-100 the AGC drift is gone and the broadband baseline is clean. +Before this ADR the live `classification.motion_level` was being driven +by the legacy DSP (variance + motion_band_power thresholds) plus an +RSSI MAD-Δ override from ADR-099. Both failed on the operator's +deployment: variance overlaps empty/sit/walk within noise, and RSSI +MAD-Δ overlaps within ±0.03 of 0.49 across all three states. The +operator could lie still in the path between AP and sensor and the +detector would silently report `absent`. + +The 30 sec × 3 controlled captures done on 2026-05-17 (lying between +TP-Link AP and sensor 1, see ADR-100 *Verified Acceptance*) showed +that **the broadband CV of mean amplitude separates the three states +by 3-6× on this geometry**. EMPTY = 2.7-5 %, STILL = 3.7-5 %, +WALK = 12.5-29.7 %. EMPTY vs STILL are best separated by the +**mean-amplitude drop** (37 → 22 on the active sensor, -40 %). + +This ADR replaces the RSSI MAD-Δ classifier with a pure-amplitude one +that uses both signals: CV for motion, baseline drop for static body. + +## Decisions + +### D1 — `amp_presence_override` per-node classifier + +For each frame received on the raw-CSI path: + +1. Push current full amplitude vector into the NBVI ranking buffer + (`nbvi_history`, capacity 600 frames ≈ 30 s). +2. Periodically (every `NBVI_REFRESH_TICKS=200` calls, ~5 s) rank + subcarriers by NBVI (see ADR-102) and pick the top-12. +3. Compute **broadband_mean** as the average of NBVI-selected + subcarriers. Falls back to all non-zero subcarriers during warmup. +4. Push to two rolling windows: + - `short` (90 samples ≈ 4.5 s) — for CV. + - `long` (1200 samples ≈ 60 s) — for the rolling-fallback 95 %ile + baseline. +5. Compute `cv = std(short) / mean(short)`. +6. Compute `baseline` — see ADR-103 for the persistent-override path. +7. Stash `(cv, mean_short, baseline)` per node in `AMP_LATEST` for + cross-node fusion. +8. Run `amp_classify_from_latest` (D2 below) to produce the global + `(level, presence, confidence)`. + +Returns `None` until the short window is full so the very first +seconds after boot don't emit garbage. + +### D2 — Cross-node fusion in `amp_classify_from_latest` + +The deployment has two sensors with very different SNR (node 1 mean +amplitude ~22, node 2 mean ~9 on the operator's TP-Link). A single +bursty node should not flip the whole detector. We use: + +* **MAX CV** across nodes for the motion gate. Any node seeing + movement is enough — body modulates only the line-of-sight path + it crosses, the other node may stay clean. +* **ANY baseline drop** → `present_still`. One well-placed node + seeing the body is enough. + +Decision (universal-threshold normalized — see ADR-103 D3): + +``` +norm_max_cv = max_cv / baseline_cv (when calibration loaded) +gates: fallback when no calibration: + norm ≥ 6.0 → "active" max_cv ≥ 0.22 + norm ≥ 3.0 → "present_moving" max_cv ≥ 0.10 + any drop → "present_still" (same) + otherwise → "absent" (same) +``` + +### D3 — Sticky 3-second motion hysteresis + +After each fusion pass, a global `AMP_HOLD` counter is reset to +`AMP_MOTION_HOLD_TICKS = 120` whenever the candidate is `moving` / +`active`. Each subsequent quiet tick decrements the counter; the +prior motion label is kept until it expires (≈ 3 s at the ~40 +combined classifier ticks/s). This bridges the brief CV dips between +walking steps so the GLOBAL doesn't flicker between `moving` and +`absent`. + +### D4 — `amp_classify_from_latest` read-only entry point + +The server has multiple `SensingUpdate` producers — the raw-CSI path +runs the full pipeline above, but the feature_state path (0xC5110006) +arrives without raw amplitudes. We expose a parallel read-only +classifier that pulls the latest stashed per-node `(cv, mean, baseline)` +from `AMP_LATEST` and runs the same fusion. The feature_state path +calls it so its emitted `classification` agrees with the raw-CSI +path's — no flicker between the two SensingUpdate sources. + +### D5 — Per-node labels in `build_node_features` + +`PerNodeFeatureInfo.classification` is overridden via +`amp_node_snapshot(node_id)`, which runs the same per-node +classifier (without cross-node fusion or hysteresis) against the +stashed `(cv, mean, baseline)` for that node alone. UI consumers +(raw.html badges) see each sensor's independent decision plus the +global fused one — useful for finding sensor placement without +moving them. + +## Files Touched + +``` +v2/crates/wifi-densepose-sensing-server/src/main.rs # ~230 lines added +v2/crates/wifi-densepose-sensing-server/static/raw.html # per-node badges +``` + +## Verified Acceptance + +| State | GLOBAL | CV | Per-node detail | +|---|---|---|---| +| EMPTY | `absent` | 4-6 % | both nodes baseline mean, low CV | +| STILL (lying, in node 1 path) | `present_still` | 3-8 % | node 1 mean drops 70 %, RSSI -20 dB | +| WALK | `active` | 12-36 % | node 2 CV explodes, RSSI swings ±5 dB | + +Cross-state separation ratio = 3.4× on node 1 broadband mean, 5.9× +on node 2 CV, compared to ±0.02 inside ±0.10 noise with the old +RSSI MAD-Δ classifier from ADR-099. + +## Open Items + +* ADR-104 will add per-node baseline-drop detection on per-subcarrier + L2 distance — currently the CV signal saturates above ~30 % so we + lose granularity on heavy movement. +* `present_still` requires the body to actually attenuate the path. + Off-axis sit doesn't trigger. Future: bring in per-subcarrier delta + for off-path presence sensing. + +## References + +* ADR-099 — first RSSI MAD-Δ attempt (superseded for `motion_level` / + `presence` / `confidence`; helper kept as `#[allow(dead_code)]`). +* ADR-100 — gain lock that makes this classifier possible. +* ADR-102 — NBVI subcarrier selection that drives the CV computation. +* ADR-103 — persistent baseline + universal threshold normalization. +* [`docs/references/espectre-techniques.md`](../references/espectre-techniques.md) + — full RuView ↔ ESPectre comparison. diff --git a/docs/adr/ADR-102-nbvi-subcarrier-selection.md b/docs/adr/ADR-102-nbvi-subcarrier-selection.md new file mode 100644 index 00000000..1c07dc72 --- /dev/null +++ b/docs/adr/ADR-102-nbvi-subcarrier-selection.md @@ -0,0 +1,138 @@ +# ADR-102 — NBVI Subcarrier Selection (server-side) + +**Status**: Accepted +**Date**: 2026-05-17 +**Scope**: `v2/crates/wifi-densepose-sensing-server/src/main.rs` +(`AmpState.nbvi_*`, `nbvi_select_top_k`). + +## Context + +Each ESP32-S3 CSI frame carries 56 active subcarriers on the HT20 +20 MHz channel. The amplitudes per subcarrier have very different +SNR depending on frequency-selective fading: in the operator's +deployment subcarriers `k=6..11` and `k=22..26` sit at CV ≈ 6 % when +the room is empty, while subcarriers `k=38..43` (middle of the band, +near the LTF nulls) sit at CV ≈ 11 % — pure channel noise, no +information about the room. + +ADR-101's classifier computes broadband-mean CV. Averaging over all +56 subcarriers means the noisy ones drag the baseline CV up to +5-7 %. That blunted the motion gates and we had to push them up to +10-22 %, losing sensitivity to subtle motion. + +## Decisions + +### D1 — Port Francesco Pace's NBVI to the server (not the FW) + +Formula (ESPectre, GPLv3): + +``` +NBVI(k) = α · (σ_k / μ_k²) + (1 - α) · (σ_k / μ_k), α = 0.5 +``` + +* `σ_k / μ_k²` — penalises weak subcarriers (a quiet bin with mean ≈ 0 + gets `∞` and is filtered out). +* `σ_k / μ_k` — standard coefficient of variation; rewards stability. +* `α = 0.5` — empirically balanced (per Pace's α-sweep tests). + +**Where**: in the server, not in FW. Pros: trivial to retune per +deployment, no flash cycle, single source of truth across two FW +variants we ship (`runbot_csi_node` and `esp32s3_csi_capture`). Cons: +we lose the ability to *only emit* selected subcarriers (would save +UDP bandwidth) — but at ~25 fps × 56 × 2 bytes = 2.8 KB/s per node, +bandwidth isn't a concern. + +### D2 — Top-K with K = 12 + +Selected at server boot once `nbvi_history` has 90+ samples; then +re-selected every `NBVI_REFRESH_TICKS = 200` calls (~5 s of combined +classifier ticks). The selected indices live in +`AmpState.nbvi_selected`. + +K=12 matches ESPectre's default. Smaller K = less averaging +smoothing; larger K = drags in worse subcarriers. + +### D3 — Dead-zone gate at 25 % of median mean + +Before NBVI scoring, drop any subcarrier whose mean amplitude is +below `0.25 × median(all subcarrier means)`. Guard tones (FW reports +amp[0] = 0 for DC), edge bins, and dead frequencies are excluded so +they can't "win" with σ/μ² → ∞. + +### D4 — ESPectre Step 1: quiet-window finder + +Naive NBVI ranking over the *entire* history is biased if a body +walked through during the calibration buffer. ADR-102 v2 adds the +quiet-window finder from Pace's Step 1: + +1. Slide an `AMP_SHORT_WIN=90`-sample window across `nbvi_history` + with stride `AMP_SHORT_WIN/3 = 30`. +2. For each window, compute the CV of its per-frame broadband mean. +3. The window with the lowest CV is "quietest". +4. Per-subcarrier mean and std for NBVI scoring use **only that + window**. + +If history is smaller than one window, the whole buffer is used. +Stride 30 (overlap of 60) keeps wall-clock cost trivial for 600 +frames. + +### D5 — `mean_for_baseline` uses FULL broadband, not NBVI + +NBVI top-K re-selects between server restarts (different "quietest" +window may give different ranking). That made the persisted baseline +value incomparable across restarts (see ADR-103 D1). Fix: ADR-101 +classifier keeps a parallel `short_full` ring buffer of FULL +broadband means (all non-zero subcarriers, no NBVI filter). When +ADR-103's persistent override is active, the baseline-drop check +compares full-broadband short window to full-broadband baseline. +NBVI subset is still used for CV (motion sensitivity is what NBVI +shines at — full broadband mean is just the integral level). + +## Files Touched + +``` +v2/crates/wifi-densepose-sensing-server/src/main.rs + - struct AmpState + - nbvi_select_top_k() + - amp_presence_override() (broadband_mean computation) +``` + +## Verified Acceptance (operator's deployment, 2026-05-17) + +Idle empty-room CV, sensing-server with 2 pps housekeeping ping: + +| | Full 56 subc | NBVI top-12 | +|---|---|---| +| node 1 (rssi -53 dBm) | ~5.0 % | **3.1 %** | +| node 2 (rssi -67 dBm) | ~7.0 % | **3.9 %** | + +Reduction 38-44 %. The lower baseline let ADR-101 gates be tightened +from `15 % / 30 %` down to `10 % / 22 %` for moving/active without +raising the false-positive rate — subtler motions like waving while +sitting near a sensor now trigger. + +## Open Items + +* **ADR-102 Step 3 (FP-rate validation)** — Pace's full pipeline + validates each candidate top-K by running the motion detector on + the calibration buffer and picking the K with the lowest FP rate. + We take the raw ranking without validation. Could add later for + defense against NBVI accidentally selecting noise-source overlap. +* **Persist NBVI selection** — `AMP_BASELINE_OVERRIDE` (ADR-103) + persists baseline scalar but not the chosen subcarrier indices. + After server restart NBVI re-ranks from scratch; in deployments + where the channel changes over hours we'd want to re-rank anyway, + so for now this is correct, not an open item. +* **FW boot-time NBVI freeze** — ESPectre's Pace freezes NBVI for + the lifetime of the boot. Trade-off vs our adaptive rolling + refresh. Worth exploring if FP rate is a problem in real homes. + +## References + +* ADR-100 — gain lock (gives NBVI a stable per-subcarrier baseline). +* ADR-101 — classifier that consumes NBVI selection. +* ADR-103 — persistent baseline + universal threshold normalization. +* [Pace's *Part 2*](https://medium.com/@francesco.pace/how-i-turned-my-wi-fi-into-a-motion-sensor-part-2-62038130e530) + + [francescopace/espectre](https://github.com/francescopace/espectre) + on GitHub (GPLv3). +* [`docs/references/espectre-techniques.md`](../references/espectre-techniques.md). diff --git a/docs/adr/ADR-103-persistent-baseline.md b/docs/adr/ADR-103-persistent-baseline.md new file mode 100644 index 00000000..798ba098 --- /dev/null +++ b/docs/adr/ADR-103-persistent-baseline.md @@ -0,0 +1,185 @@ +# ADR-103 — Persistent Empty-Room Baseline + Universal Threshold + +**Status**: Accepted +**Date**: 2026-05-17 +**Scope**: `v2/crates/wifi-densepose-sensing-server/src/main.rs` +(`AMP_BASELINE_OVERRIDE`, `AMP_BASELINE_CV`, `load_baseline_file`, +`amp_node_level`), `v2/data/baseline.json`, `scripts/record-baseline.py`. + +## Context + +ADR-101's classifier relies on a `baseline` value per node — the +mean amplitude the room exhibits when empty. Pre-ADR-103 the baseline +was the rolling 95 %ile of the last 1 200 samples (≈ 60 s) of +broadband mean. That meant every server restart triggered a "step +outside for 60 seconds" ritual before the detector worked, and if +the operator stayed in the room longer than ~4 min the baseline +silently drifted down to the *occupied* amplitude — making +`present_still` under-trigger forever after. + +Additionally, motion gates were hard-coded to the operator's +deployment (10 % / 22 % CV) — wouldn't transfer to a different room +with different noise floor. + +## Decisions + +### D1 — Persistent baseline file at `data/baseline.json` + +JSON schema **v2** (per node): + +```json +{ + "version": 2, + "captured_at": "ISO-8601", + "duration_sec": 90.0, + "trim_head_sec": 15.0, + "trim_tail_sec": 15.0, + "clean_window_sec": 30.0, + "method": "record → trim head/tail → find lowest-CV sub-window → FULL-broadband stats per node", + "nodes": { + "1": { + "full_broadband_mean": 26.11, + "full_broadband_p50": 26.16, + "full_broadband_p95": 27.04, ← used as `baseline` + "full_broadband_std": 0.68, + "full_broadband_cv_pct": 2.62, ← used to normalize gates (D3) + "rssi_dbm": -52.3, + "n_samples": 149, + "per_subcarrier_mean": [..56 floats..] + } + } +} +``` + +Loader (`load_baseline_file`) reads at server startup. Path is +`$RUVIEW_BASELINE_FILE` or `data/baseline.json` by default. Missing +or unparseable file = log warning + fall back to rolling p95 (= old +ADR-101 behaviour, no breaking change). + +Per-node lookup priority: `full_broadband_p95` → `full_broadband_mean` +→ legacy `p95_amp` → legacy `mean_amp`. v1 baselines load but get +warning about NBVI-drift incompatibility. + +### D2 — FULL broadband for baseline comparison, NBVI for CV + +The persisted baseline must be comparable across server restarts. +NBVI top-12 re-selects on each boot (ADR-102 D4), so a NBVI-subset +mean recorded today doesn't match a NBVI-subset mean tomorrow even +in the same empty room. Fix: + +`amp_presence_override` keeps two short windows: + +| Field | Source | Used for | +|---|---|---| +| `short` | NBVI-subset broadband mean | CV (motion sensitivity) | +| `short_full` | **all non-zero subcarriers** mean | baseline drop check | + +The recording script also computes full-broadband statistics from +the captured frames. Both sides of `mean / baseline` ratio are +full-broadband ⇒ stable across NBVI selection. + +### D3 — Universal threshold via baseline-CV normalization + +(Pace's Problem #3.) Hard-coded gates are deployment-tuned. Fix: +normalize the runtime CV by the empty-room CV measured during +calibration: + +``` +norm_cv = current_cv / baseline_cv +gates: norm_cv ≥ 3.0 → present_moving + norm_cv ≥ 6.0 → active +``` + +Both `amp_node_level` (per-node) and `amp_classify_from_latest` +(global) use the same normalization. When no calibration is loaded, +fall back to absolute gates `0.10 / 0.22` (the deployment-tuned +values) — keeps backwards compatibility. + +`AMP_BASELINE_CV` is a separate per-node map loaded alongside +`AMP_BASELINE_OVERRIDE`. The CV value is the FULL-broadband CV % from +the calibration file divided by 100. + +### D4 — Recording script `scripts/record-baseline.py` + +CLI helper (Python 3, requires `pip install websockets`). Connects +to the live `ws://localhost:8765/ws/sensing`, records `duration` (90 +s default), then: + +1. Trim `trim_head_sec` (15 s default) and `trim_tail_sec` (15 s + default) to discard door-open / re-entry transients. +2. Slide a `clean_window_sec` (30 s default) sub-window across the + trimmed buffer, pick the one with the lowest broadband CV. +3. Per node, compute full-broadband mean / median / p95 / std / CV % + and rssi mean over that cleanest window. +4. Also compute per-subcarrier mean across the cleanest window (saved + as diagnostic for future per-subcarrier delta classifier). +5. Write `v2/data/baseline.json` (path overridable via `--out`). + +Operator workflow now: step out, run script, come back, restart +server. One-time per deployment (or after room rearrangement). No +recurring ritual. + +## Files Touched + +``` +v2/crates/wifi-densepose-sensing-server/src/main.rs # ~120 lines added +v2/data/baseline.json # new, gitignored? +scripts/record-baseline.py # new helper +docs/adr/ADR-103-persistent-baseline.md # this ADR +``` + +## Verified Acceptance (operator's deployment, 2026-05-17) + +``` +boot: baseline: loaded 2 node overrides from data/baseline.json + (node1=27.04, node2=14.72; + node1_cv=2.62%, node2_cv=3.65%) +``` + +Empty room, immediately after restart (no warmup wait): + +``` +GLOBAL: absent CV=5.0% + node 1 ratio=0.93, norm_cv=0.80× + node 2 ratio=0.93, norm_cv=0.83× +``` + +Sitting in node 2 path (off-axis from node 1): + +``` +GLOBAL: present_still CV=8.1% + node 1 ratio=1.05, norm_cv=1.2× (not in path, no drop) + node 2 ratio=0.70, norm_cv=1.7× ← drop fires present_still +``` + +Walking: + +``` +GLOBAL: active CV=28-36% + node 1 norm_cv=4-6×, node 2 norm_cv=7-9× ← well above 6× gate +``` + +Universal-threshold gates `3.0 / 6.0` map to the same absolute +12 % / 22 % we hand-tuned earlier — but now any-room-portable. + +## Open Items + +* **REST endpoint POST /api/v1/baseline/calibrate** — would let the + operator press a button in `raw.html` instead of running the + Python script. ~30 min. +* **Per-subcarrier baseline comparison** — `per_subcarrier_mean` is + saved but not yet consumed; future ADR-104 can use L2 distance + per-subcarrier instead of broadband mean ratio for off-axis + presence detection. +* **Auto-recalibrate on long quiet periods** — if the classifier sees + `absent` for 30 minutes with low variance, it could opportunistically + update the baseline. Would eliminate the manual script step + entirely. ~1 h. + +## References + +* ADR-100 — gain lock. +* ADR-101 — classifier consumes the baseline. +* ADR-102 — NBVI selection drift was the root cause of D1/D2. +* [`docs/references/espectre-techniques.md`](../references/espectre-techniques.md) + — Pace's full technique catalogue including Problem #3 normalization. diff --git a/docs/references/espectre-techniques.md b/docs/references/espectre-techniques.md index 2a377159..1ca2b36f 100644 --- a/docs/references/espectre-techniques.md +++ b/docs/references/espectre-techniques.md @@ -69,30 +69,12 @@ Four-step pipeline at boot: | 3 | **Rank + validate** | Sort by NBVI ascending. Run the motion detector on each candidate config, measure false-positive rate, take the config with the lowest FP. | | 4 | **Pick winners** | Top-K by lowest NBVI (typically K = 12 for HT20). | -Memory: O(N) running with on-the-fly mean/variance updates ⇒ ≈ 256 B -for 64 subcarriers. Time: O(N · L) per recompute, milliseconds on a -$10 device. +Memory: O(N) running with on-the-fly mean/variance, ≈ 256 B for 64 +subcarriers. Time: O(N · L) per recompute, ms on a $10 device. -**RuView status — PARTIALLY DONE.** ADR-102 (commit `2f12a223`). -Server-side port in `amp_presence_override` / -`nbvi_select_top_k`. What we have: - -- ✅ NBVI formula with α = 0.5 -- ✅ Top-12 selection -- ✅ Dead-zone gate (`NBVI_DEAD_GATE_PCT = 0.25`) -- ✅ Recompute throttled (`NBVI_REFRESH_TICKS = 200` ≈ every 5 s) - -What we **do not** have: - -- ❌ **Step 1 quiet-window finder** — we use the *whole* history - buffer. If the buffer captures someone moving, ranking is biased. - Pace's percentile-window detector should be added. -- ❌ **Step 3 FP-rate validation** — we accept the raw NBVI ranking - without testing it on the calibration data. -- ❌ **Boot calibration sequence** (FW-side, 7 s post gain-lock). - Ours is server-side rolling, which means selection drifts forever - rather than locking after boot. Trade-off: adapts to room - rearrangement, but never "settles". +**RuView status — DONE (Steps 1, 2, 4).** ADR-102 (commits +`2f12a223` + `f4119924`). Server-side, see ADR-102 for detail. +Missing: ❌ Step 3 FP-rate validation, ❌ FW-side boot freeze. Empirically on the operator's deployment NBVI alone gave a 1.5-2× CV reduction: @@ -118,9 +100,11 @@ Reference 0.25 is what a quiet room typically measures during NBVI calibration. Apply the scale to the live motion score, so the user- facing threshold (`= 1.0`) is universal across rooms. -**RuView status — NOT DONE.** Our `amp_node_level` uses fixed -thresholds tuned to one deployment (CV 10 % moving, CV 22 % active, -mean/baseline < 0.75 still). Other deployments will need re-tuning. +**RuView status — DONE.** ADR-103 D3 (commit `2f4b2d53`). +`amp_node_level` and `amp_classify_from_latest` divide live CV by +`baseline_cv` loaded from `data/baseline.json` and gate at universal +`3×` (moving) / `6×` (active). Falls back to absolute gates +`0.10 / 0.22` when no calibration loaded — backwards compatible. ## 4. Two-phase boot calibration @@ -142,9 +126,12 @@ After NBVI calibration, ESPectre writes the AGC/FFT lock values, the chosen subcarrier set, the baseline variance, and the threshold into NVS so reboots don't need re-calibration. -**RuView status — NOT DONE.** Each server restart triggers a fresh -60-second baseline learn. NBVI also re-ranks from scratch on restart. -Open item: persist `AMP_LATEST.baseline` to disk + load at startup. +**RuView status — DONE for baseline; PARTIAL for the rest.** ADR-103 +(commits `f4119924`, `2f4b2d53`). `data/baseline.json` persists per- +node full-broadband mean/p95/CV + per-subcarrier means, loaded at +server boot via `load_baseline_file`. Gain lock + NBVI selection +still happen fresh on each FW boot / server boot respectively (open +items 4 + 5 below). ## 6. Interactive Web Serial game (`espectre.dev/game`) @@ -176,37 +163,36 @@ for the amplitude classifier or NBVI. | Item | Pace / ESPectre | RuView | |---|---|---| -| Gain lock | FW, 300 pkt median, AGC+FFT, AGC<30 skip | ✅ Same, in `csi_collector.c` | -| NBVI formula | α·σ/μ² + (1-α)·σ/μ, α=0.5, top-12 | ✅ Same, server-side | -| Dead-zone gate | 25th percentile of mean | ✅ `NBVI_DEAD_GATE_PCT=0.25` | -| Quiet-window finder | Percentile-window in calibration buffer | ❌ Use full window verbatim | -| FP-rate validation of NBVI pick | Yes | ❌ Take raw ranking | -| Boot-time NBVI freeze | FW, ~7 s post-lock | ❌ Server-side rolling | -| Baseline variance normalization | `scale = 0.25 / σ²` | ❌ Fixed thresholds per deployment | -| NVS persistence of calibration | Yes | ❌ Fresh learn each restart | -| Universal threshold | One value across rooms | ❌ Re-tune per deployment | -| Calibration UI | Web Serial game | ❌ Read-only raw.html | -| HA integration | ESPHome native | ❌ None | -| Test suite | 500+ tests, 90 % coverage | ❌ ~2 parser tests only | -| Phase / amplitude | Amplitude only (TPR avoidance) | ✅ Same | -| Subcarrier count | 64 (HT20) | 56 (ESP32-S3 reports 56 non-guard) | +| Gain lock | FW, 300 pkt median, AGC+FFT, AGC<30 skip | ✅ ADR-100 | +| NBVI formula α=0.5, top-12, dead-zone gate | ✅ | ✅ ADR-102 | +| Quiet-window finder (Step 1) | ✅ | ✅ ADR-102 v2 | +| FP-rate validation (Step 3) | ✅ | ❌ raw ranking | +| Boot-time NBVI freeze | FW, ~7 s post-lock | ❌ server-side rolling | +| Baseline variance normalization (universal threshold) | ✅ | ✅ ADR-103 D3 | +| Persisted baseline to disk | NVS | ✅ ADR-103 D1 (`data/baseline.json`) | +| NVS persistence of FW calibration | ✅ | ❌ fresh each FW boot | +| Calibration UI | Web Serial game | ❌ read-only `raw.html` | +| HA / ESPHome integration | ✅ | ❌ none | +| Test suite | 500+ tests, 90 % cov | ❌ 2 parser tests | +| Phase / amplitude | amplitude only | ✅ same | ## Open items, ranked by expected impact on RuView -1. **Quiet-window finder for NBVI Step 1** — if the operator restarts - the server while the room is occupied, current NBVI biases its - ranking toward subcarriers stable on the *occupied* state. Bug: - present_still then under-triggers. ~1 h. -2. **Persist `AMP_LATEST.baseline` to disk** — eliminates the - "step outside for 60 s" ritual after every restart. ~30 min. -3. **Baseline variance normalization** — would let us ship one - threshold set for any deployment. ~1 h. -4. **FP-rate validation of NBVI pick** — would catch the case where - the top-12 ranked subcarriers happen to overlap with a noise - source. ~1 h. -5. **Boot-time NBVI freeze** — if we want fully reproducible - behaviour. Trade-off: doesn't adapt to room changes. ~2 h. -6. **HA / ESPHome integration** — depends on whether RuView wants - to be a HA sensor or stay standalone. ~1 day. -7. **Web Serial calibration UI** — nice-to-have, lower priority than - the algorithmic items. ~1 day. +1. **REST `POST /api/v1/baseline/calibrate`** — drives the recording + script from a button in `raw.html` instead of CLI. ~30 min. +2. **FP-rate validation of NBVI pick** — defense against the top-12 + accidentally overlapping a noise source. ~1 h. +3. **Per-subcarrier baseline comparison (ADR-104 draft)** — uses the + already-saved `per_subcarrier_mean` in `baseline.json` for L2 + distance instead of broadband mean ratio. Better off-axis + presence sensing. ~1 h. +4. **Auto-recalibrate on long quiet periods** — if classifier sees + `absent` with low variance for 30 min, refresh baseline in + background. Eliminates manual script step entirely. ~1 h. +5. **FW-side NBVI boot-freeze + NVS persistence** — full + reproducibility, sub-second post-boot ready. Trade-off: doesn't + adapt to room changes. ~2 h. +6. **HA / ESPHome integration** — sensor as HA entity. ~1 day. +7. **Test suite vs fixed 2 000-packet replay** — regression + protection for the classifier + NBVI. ~1 day. +8. **Web Serial calibration game** — nice-to-have. ~1 day.