docs: ADR-101 / ADR-102 / ADR-103 — full session record
* ADR-101 raw-amplitude presence/motion classifier — per-node and cross-node fusion logic, hysteresis, per-node UI surface (`PerNodeFeatureInfo.classification` override). * ADR-102 server-side NBVI subcarrier selection — formula, dead-zone gate, ESPectre Step-1 quiet-window finder, why we split FULL vs NBVI-subset broadband. * ADR-103 persistent baseline + universal threshold normalization — JSON schema v2 at `v2/data/baseline.json`, FULL-broadband over NBVI for cross-restart stability, `norm_cv = cv / baseline_cv` with universal 3×/6× gates, recording script workflow. * Updated espectre-techniques.md to reflect the DONE items (Steps 1+2+4 of NBVI, baseline persistence, universal threshold) and the remaining open items in priority order. Each ADR ≤ 200 lines per the operator's docs convention; deep detail lives in `docs/references/espectre-techniques.md` (also ≤ 200) which the ADRs link to. README.md and CLAUDE.md unchanged (no extra content added; existing >200-line state pre-dates this session).
This commit is contained in:
arsen
parent
2f4b2d5304
commit
4d3ca49fba
|
|
@ -0,0 +1,145 @@
|
|||
# ADR-101 — Raw-Amplitude Presence/Motion Classifier
|
||||
|
||||
**Status**: Accepted
|
||||
**Date**: 2026-05-17
|
||||
**Scope**: `v2/crates/wifi-densepose-sensing-server/src/main.rs`
|
||||
(`amp_presence_override`, `amp_classify_from_latest`,
|
||||
`amp_node_level`, `amp_node_snapshot`).
|
||||
|
||||
## Context
|
||||
|
||||
After ADR-100 the AGC drift is gone and the broadband baseline is clean.
|
||||
Before this ADR the live `classification.motion_level` was being driven
|
||||
by the legacy DSP (variance + motion_band_power thresholds) plus an
|
||||
RSSI MAD-Δ override from ADR-099. Both failed on the operator's
|
||||
deployment: variance overlaps empty/sit/walk within noise, and RSSI
|
||||
MAD-Δ overlaps within ±0.03 of 0.49 across all three states. The
|
||||
operator could lie still in the path between AP and sensor and the
|
||||
detector would silently report `absent`.
|
||||
|
||||
The 30 sec × 3 controlled captures done on 2026-05-17 (lying between
|
||||
TP-Link AP and sensor 1, see ADR-100 *Verified Acceptance*) showed
|
||||
that **the broadband CV of mean amplitude separates the three states
|
||||
by 3-6× on this geometry**. EMPTY = 2.7-5 %, STILL = 3.7-5 %,
|
||||
WALK = 12.5-29.7 %. EMPTY vs STILL are best separated by the
|
||||
**mean-amplitude drop** (37 → 22 on the active sensor, -40 %).
|
||||
|
||||
This ADR replaces the RSSI MAD-Δ classifier with a pure-amplitude one
|
||||
that uses both signals: CV for motion, baseline drop for static body.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1 — `amp_presence_override` per-node classifier
|
||||
|
||||
For each frame received on the raw-CSI path:
|
||||
|
||||
1. Push current full amplitude vector into the NBVI ranking buffer
|
||||
(`nbvi_history`, capacity 600 frames ≈ 30 s).
|
||||
2. Periodically (every `NBVI_REFRESH_TICKS=200` calls, ~5 s) rank
|
||||
subcarriers by NBVI (see ADR-102) and pick the top-12.
|
||||
3. Compute **broadband_mean** as the average of NBVI-selected
|
||||
subcarriers. Falls back to all non-zero subcarriers during warmup.
|
||||
4. Push to two rolling windows:
|
||||
- `short` (90 samples ≈ 4.5 s) — for CV.
|
||||
- `long` (1200 samples ≈ 60 s) — for the rolling-fallback 95 %ile
|
||||
baseline.
|
||||
5. Compute `cv = std(short) / mean(short)`.
|
||||
6. Compute `baseline` — see ADR-103 for the persistent-override path.
|
||||
7. Stash `(cv, mean_short, baseline)` per node in `AMP_LATEST` for
|
||||
cross-node fusion.
|
||||
8. Run `amp_classify_from_latest` (D2 below) to produce the global
|
||||
`(level, presence, confidence)`.
|
||||
|
||||
Returns `None` until the short window is full so the very first
|
||||
seconds after boot don't emit garbage.
|
||||
|
||||
### D2 — Cross-node fusion in `amp_classify_from_latest`
|
||||
|
||||
The deployment has two sensors with very different SNR (node 1 mean
|
||||
amplitude ~22, node 2 mean ~9 on the operator's TP-Link). A single
|
||||
bursty node should not flip the whole detector. We use:
|
||||
|
||||
* **MAX CV** across nodes for the motion gate. Any node seeing
|
||||
movement is enough — body modulates only the line-of-sight path
|
||||
it crosses, the other node may stay clean.
|
||||
* **ANY baseline drop** → `present_still`. One well-placed node
|
||||
seeing the body is enough.
|
||||
|
||||
Decision (universal-threshold normalized — see ADR-103 D3):
|
||||
|
||||
```
|
||||
norm_max_cv = max_cv / baseline_cv (when calibration loaded)
|
||||
gates: fallback when no calibration:
|
||||
norm ≥ 6.0 → "active" max_cv ≥ 0.22
|
||||
norm ≥ 3.0 → "present_moving" max_cv ≥ 0.10
|
||||
any drop → "present_still" (same)
|
||||
otherwise → "absent" (same)
|
||||
```
|
||||
|
||||
### D3 — Sticky 3-second motion hysteresis
|
||||
|
||||
After each fusion pass, a global `AMP_HOLD` counter is reset to
|
||||
`AMP_MOTION_HOLD_TICKS = 120` whenever the candidate is `moving` /
|
||||
`active`. Each subsequent quiet tick decrements the counter; the
|
||||
prior motion label is kept until it expires (≈ 3 s at the ~40
|
||||
combined classifier ticks/s). This bridges the brief CV dips between
|
||||
walking steps so the GLOBAL doesn't flicker between `moving` and
|
||||
`absent`.
|
||||
|
||||
### D4 — `amp_classify_from_latest` read-only entry point
|
||||
|
||||
The server has multiple `SensingUpdate` producers — the raw-CSI path
|
||||
runs the full pipeline above, but the feature_state path (0xC5110006)
|
||||
arrives without raw amplitudes. We expose a parallel read-only
|
||||
classifier that pulls the latest stashed per-node `(cv, mean, baseline)`
|
||||
from `AMP_LATEST` and runs the same fusion. The feature_state path
|
||||
calls it so its emitted `classification` agrees with the raw-CSI
|
||||
path's — no flicker between the two SensingUpdate sources.
|
||||
|
||||
### D5 — Per-node labels in `build_node_features`
|
||||
|
||||
`PerNodeFeatureInfo.classification` is overridden via
|
||||
`amp_node_snapshot(node_id)`, which runs the same per-node
|
||||
classifier (without cross-node fusion or hysteresis) against the
|
||||
stashed `(cv, mean, baseline)` for that node alone. UI consumers
|
||||
(raw.html badges) see each sensor's independent decision plus the
|
||||
global fused one — useful for finding sensor placement without
|
||||
moving them.
|
||||
|
||||
## Files Touched
|
||||
|
||||
```
|
||||
v2/crates/wifi-densepose-sensing-server/src/main.rs # ~230 lines added
|
||||
v2/crates/wifi-densepose-sensing-server/static/raw.html # per-node badges
|
||||
```
|
||||
|
||||
## Verified Acceptance
|
||||
|
||||
| State | GLOBAL | CV | Per-node detail |
|
||||
|---|---|---|---|
|
||||
| EMPTY | `absent` | 4-6 % | both nodes baseline mean, low CV |
|
||||
| STILL (lying, in node 1 path) | `present_still` | 3-8 % | node 1 mean drops 70 %, RSSI -20 dB |
|
||||
| WALK | `active` | 12-36 % | node 2 CV explodes, RSSI swings ±5 dB |
|
||||
|
||||
Cross-state separation ratio = 3.4× on node 1 broadband mean, 5.9×
|
||||
on node 2 CV, compared to ±0.02 inside ±0.10 noise with the old
|
||||
RSSI MAD-Δ classifier from ADR-099.
|
||||
|
||||
## Open Items
|
||||
|
||||
* ADR-104 will add per-node baseline-drop detection on per-subcarrier
|
||||
L2 distance — currently the CV signal saturates above ~30 % so we
|
||||
lose granularity on heavy movement.
|
||||
* `present_still` requires the body to actually attenuate the path.
|
||||
Off-axis sit doesn't trigger. Future: bring in per-subcarrier delta
|
||||
for off-path presence sensing.
|
||||
|
||||
## References
|
||||
|
||||
* ADR-099 — first RSSI MAD-Δ attempt (superseded for `motion_level` /
|
||||
`presence` / `confidence`; helper kept as `#[allow(dead_code)]`).
|
||||
* ADR-100 — gain lock that makes this classifier possible.
|
||||
* ADR-102 — NBVI subcarrier selection that drives the CV computation.
|
||||
* ADR-103 — persistent baseline + universal threshold normalization.
|
||||
* [`docs/references/espectre-techniques.md`](../references/espectre-techniques.md)
|
||||
— full RuView ↔ ESPectre comparison.
|
||||
|
|
@ -0,0 +1,138 @@
|
|||
# ADR-102 — NBVI Subcarrier Selection (server-side)
|
||||
|
||||
**Status**: Accepted
|
||||
**Date**: 2026-05-17
|
||||
**Scope**: `v2/crates/wifi-densepose-sensing-server/src/main.rs`
|
||||
(`AmpState.nbvi_*`, `nbvi_select_top_k`).
|
||||
|
||||
## Context
|
||||
|
||||
Each ESP32-S3 CSI frame carries 56 active subcarriers on the HT20
|
||||
20 MHz channel. The amplitudes per subcarrier have very different
|
||||
SNR depending on frequency-selective fading: in the operator's
|
||||
deployment subcarriers `k=6..11` and `k=22..26` sit at CV ≈ 6 % when
|
||||
the room is empty, while subcarriers `k=38..43` (middle of the band,
|
||||
near the LTF nulls) sit at CV ≈ 11 % — pure channel noise, no
|
||||
information about the room.
|
||||
|
||||
ADR-101's classifier computes broadband-mean CV. Averaging over all
|
||||
56 subcarriers means the noisy ones drag the baseline CV up to
|
||||
5-7 %. That blunted the motion gates and we had to push them up to
|
||||
10-22 %, losing sensitivity to subtle motion.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1 — Port Francesco Pace's NBVI to the server (not the FW)
|
||||
|
||||
Formula (ESPectre, GPLv3):
|
||||
|
||||
```
|
||||
NBVI(k) = α · (σ_k / μ_k²) + (1 - α) · (σ_k / μ_k), α = 0.5
|
||||
```
|
||||
|
||||
* `σ_k / μ_k²` — penalises weak subcarriers (a quiet bin with mean ≈ 0
|
||||
gets `∞` and is filtered out).
|
||||
* `σ_k / μ_k` — standard coefficient of variation; rewards stability.
|
||||
* `α = 0.5` — empirically balanced (per Pace's α-sweep tests).
|
||||
|
||||
**Where**: in the server, not in FW. Pros: trivial to retune per
|
||||
deployment, no flash cycle, single source of truth across two FW
|
||||
variants we ship (`runbot_csi_node` and `esp32s3_csi_capture`). Cons:
|
||||
we lose the ability to *only emit* selected subcarriers (would save
|
||||
UDP bandwidth) — but at ~25 fps × 56 × 2 bytes = 2.8 KB/s per node,
|
||||
bandwidth isn't a concern.
|
||||
|
||||
### D2 — Top-K with K = 12
|
||||
|
||||
Selected at server boot once `nbvi_history` has 90+ samples; then
|
||||
re-selected every `NBVI_REFRESH_TICKS = 200` calls (~5 s of combined
|
||||
classifier ticks). The selected indices live in
|
||||
`AmpState.nbvi_selected`.
|
||||
|
||||
K=12 matches ESPectre's default. Smaller K = less averaging
|
||||
smoothing; larger K = drags in worse subcarriers.
|
||||
|
||||
### D3 — Dead-zone gate at 25 % of median mean
|
||||
|
||||
Before NBVI scoring, drop any subcarrier whose mean amplitude is
|
||||
below `0.25 × median(all subcarrier means)`. Guard tones (FW reports
|
||||
amp[0] = 0 for DC), edge bins, and dead frequencies are excluded so
|
||||
they can't "win" with σ/μ² → ∞.
|
||||
|
||||
### D4 — ESPectre Step 1: quiet-window finder
|
||||
|
||||
Naive NBVI ranking over the *entire* history is biased if a body
|
||||
walked through during the calibration buffer. ADR-102 v2 adds the
|
||||
quiet-window finder from Pace's Step 1:
|
||||
|
||||
1. Slide an `AMP_SHORT_WIN=90`-sample window across `nbvi_history`
|
||||
with stride `AMP_SHORT_WIN/3 = 30`.
|
||||
2. For each window, compute the CV of its per-frame broadband mean.
|
||||
3. The window with the lowest CV is "quietest".
|
||||
4. Per-subcarrier mean and std for NBVI scoring use **only that
|
||||
window**.
|
||||
|
||||
If history is smaller than one window, the whole buffer is used.
|
||||
Stride 30 (overlap of 60) keeps wall-clock cost trivial for 600
|
||||
frames.
|
||||
|
||||
### D5 — `mean_for_baseline` uses FULL broadband, not NBVI
|
||||
|
||||
NBVI top-K re-selects between server restarts (different "quietest"
|
||||
window may give different ranking). That made the persisted baseline
|
||||
value incomparable across restarts (see ADR-103 D1). Fix: ADR-101
|
||||
classifier keeps a parallel `short_full` ring buffer of FULL
|
||||
broadband means (all non-zero subcarriers, no NBVI filter). When
|
||||
ADR-103's persistent override is active, the baseline-drop check
|
||||
compares full-broadband short window to full-broadband baseline.
|
||||
NBVI subset is still used for CV (motion sensitivity is what NBVI
|
||||
shines at — full broadband mean is just the integral level).
|
||||
|
||||
## Files Touched
|
||||
|
||||
```
|
||||
v2/crates/wifi-densepose-sensing-server/src/main.rs
|
||||
- struct AmpState
|
||||
- nbvi_select_top_k()
|
||||
- amp_presence_override() (broadband_mean computation)
|
||||
```
|
||||
|
||||
## Verified Acceptance (operator's deployment, 2026-05-17)
|
||||
|
||||
Idle empty-room CV, sensing-server with 2 pps housekeeping ping:
|
||||
|
||||
| | Full 56 subc | NBVI top-12 |
|
||||
|---|---|---|
|
||||
| node 1 (rssi -53 dBm) | ~5.0 % | **3.1 %** |
|
||||
| node 2 (rssi -67 dBm) | ~7.0 % | **3.9 %** |
|
||||
|
||||
Reduction 38-44 %. The lower baseline let ADR-101 gates be tightened
|
||||
from `15 % / 30 %` down to `10 % / 22 %` for moving/active without
|
||||
raising the false-positive rate — subtler motions like waving while
|
||||
sitting near a sensor now trigger.
|
||||
|
||||
## Open Items
|
||||
|
||||
* **ADR-102 Step 3 (FP-rate validation)** — Pace's full pipeline
|
||||
validates each candidate top-K by running the motion detector on
|
||||
the calibration buffer and picking the K with the lowest FP rate.
|
||||
We take the raw ranking without validation. Could add later for
|
||||
defense against NBVI accidentally selecting noise-source overlap.
|
||||
* **Persist NBVI selection** — `AMP_BASELINE_OVERRIDE` (ADR-103)
|
||||
persists baseline scalar but not the chosen subcarrier indices.
|
||||
After server restart NBVI re-ranks from scratch; in deployments
|
||||
where the channel changes over hours we'd want to re-rank anyway,
|
||||
so for now this is correct, not an open item.
|
||||
* **FW boot-time NBVI freeze** — ESPectre's Pace freezes NBVI for
|
||||
the lifetime of the boot. Trade-off vs our adaptive rolling
|
||||
refresh. Worth exploring if FP rate is a problem in real homes.
|
||||
|
||||
## References
|
||||
|
||||
* ADR-100 — gain lock (gives NBVI a stable per-subcarrier baseline).
|
||||
* ADR-101 — classifier that consumes NBVI selection.
|
||||
* ADR-103 — persistent baseline + universal threshold normalization.
|
||||
* [Pace's *Part 2*](https://medium.com/@francesco.pace/how-i-turned-my-wi-fi-into-a-motion-sensor-part-2-62038130e530)
|
||||
+ [francescopace/espectre](https://github.com/francescopace/espectre)
|
||||
on GitHub (GPLv3).
|
||||
* [`docs/references/espectre-techniques.md`](../references/espectre-techniques.md).
|
||||
|
|
@ -0,0 +1,185 @@
|
|||
# ADR-103 — Persistent Empty-Room Baseline + Universal Threshold
|
||||
|
||||
**Status**: Accepted
|
||||
**Date**: 2026-05-17
|
||||
**Scope**: `v2/crates/wifi-densepose-sensing-server/src/main.rs`
|
||||
(`AMP_BASELINE_OVERRIDE`, `AMP_BASELINE_CV`, `load_baseline_file`,
|
||||
`amp_node_level`), `v2/data/baseline.json`, `scripts/record-baseline.py`.
|
||||
|
||||
## Context
|
||||
|
||||
ADR-101's classifier relies on a `baseline` value per node — the
|
||||
mean amplitude the room exhibits when empty. Pre-ADR-103 the baseline
|
||||
was the rolling 95 %ile of the last 1 200 samples (≈ 60 s) of
|
||||
broadband mean. That meant every server restart triggered a "step
|
||||
outside for 60 seconds" ritual before the detector worked, and if
|
||||
the operator stayed in the room longer than ~4 min the baseline
|
||||
silently drifted down to the *occupied* amplitude — making
|
||||
`present_still` under-trigger forever after.
|
||||
|
||||
Additionally, motion gates were hard-coded to the operator's
|
||||
deployment (10 % / 22 % CV) — wouldn't transfer to a different room
|
||||
with different noise floor.
|
||||
|
||||
## Decisions
|
||||
|
||||
### D1 — Persistent baseline file at `data/baseline.json`
|
||||
|
||||
JSON schema **v2** (per node):
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 2,
|
||||
"captured_at": "ISO-8601",
|
||||
"duration_sec": 90.0,
|
||||
"trim_head_sec": 15.0,
|
||||
"trim_tail_sec": 15.0,
|
||||
"clean_window_sec": 30.0,
|
||||
"method": "record → trim head/tail → find lowest-CV sub-window → FULL-broadband stats per node",
|
||||
"nodes": {
|
||||
"1": {
|
||||
"full_broadband_mean": 26.11,
|
||||
"full_broadband_p50": 26.16,
|
||||
"full_broadband_p95": 27.04, ← used as `baseline`
|
||||
"full_broadband_std": 0.68,
|
||||
"full_broadband_cv_pct": 2.62, ← used to normalize gates (D3)
|
||||
"rssi_dbm": -52.3,
|
||||
"n_samples": 149,
|
||||
"per_subcarrier_mean": [..56 floats..]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Loader (`load_baseline_file`) reads at server startup. Path is
|
||||
`$RUVIEW_BASELINE_FILE` or `data/baseline.json` by default. Missing
|
||||
or unparseable file = log warning + fall back to rolling p95 (= old
|
||||
ADR-101 behaviour, no breaking change).
|
||||
|
||||
Per-node lookup priority: `full_broadband_p95` → `full_broadband_mean`
|
||||
→ legacy `p95_amp` → legacy `mean_amp`. v1 baselines load but get
|
||||
warning about NBVI-drift incompatibility.
|
||||
|
||||
### D2 — FULL broadband for baseline comparison, NBVI for CV
|
||||
|
||||
The persisted baseline must be comparable across server restarts.
|
||||
NBVI top-12 re-selects on each boot (ADR-102 D4), so a NBVI-subset
|
||||
mean recorded today doesn't match a NBVI-subset mean tomorrow even
|
||||
in the same empty room. Fix:
|
||||
|
||||
`amp_presence_override` keeps two short windows:
|
||||
|
||||
| Field | Source | Used for |
|
||||
|---|---|---|
|
||||
| `short` | NBVI-subset broadband mean | CV (motion sensitivity) |
|
||||
| `short_full` | **all non-zero subcarriers** mean | baseline drop check |
|
||||
|
||||
The recording script also computes full-broadband statistics from
|
||||
the captured frames. Both sides of `mean / baseline` ratio are
|
||||
full-broadband ⇒ stable across NBVI selection.
|
||||
|
||||
### D3 — Universal threshold via baseline-CV normalization
|
||||
|
||||
(Pace's Problem #3.) Hard-coded gates are deployment-tuned. Fix:
|
||||
normalize the runtime CV by the empty-room CV measured during
|
||||
calibration:
|
||||
|
||||
```
|
||||
norm_cv = current_cv / baseline_cv
|
||||
gates: norm_cv ≥ 3.0 → present_moving
|
||||
norm_cv ≥ 6.0 → active
|
||||
```
|
||||
|
||||
Both `amp_node_level` (per-node) and `amp_classify_from_latest`
|
||||
(global) use the same normalization. When no calibration is loaded,
|
||||
fall back to absolute gates `0.10 / 0.22` (the deployment-tuned
|
||||
values) — keeps backwards compatibility.
|
||||
|
||||
`AMP_BASELINE_CV` is a separate per-node map loaded alongside
|
||||
`AMP_BASELINE_OVERRIDE`. The CV value is the FULL-broadband CV % from
|
||||
the calibration file divided by 100.
|
||||
|
||||
### D4 — Recording script `scripts/record-baseline.py`
|
||||
|
||||
CLI helper (Python 3, requires `pip install websockets`). Connects
|
||||
to the live `ws://localhost:8765/ws/sensing`, records `duration` (90
|
||||
s default), then:
|
||||
|
||||
1. Trim `trim_head_sec` (15 s default) and `trim_tail_sec` (15 s
|
||||
default) to discard door-open / re-entry transients.
|
||||
2. Slide a `clean_window_sec` (30 s default) sub-window across the
|
||||
trimmed buffer, pick the one with the lowest broadband CV.
|
||||
3. Per node, compute full-broadband mean / median / p95 / std / CV %
|
||||
and rssi mean over that cleanest window.
|
||||
4. Also compute per-subcarrier mean across the cleanest window (saved
|
||||
as diagnostic for future per-subcarrier delta classifier).
|
||||
5. Write `v2/data/baseline.json` (path overridable via `--out`).
|
||||
|
||||
Operator workflow now: step out, run script, come back, restart
|
||||
server. One-time per deployment (or after room rearrangement). No
|
||||
recurring ritual.
|
||||
|
||||
## Files Touched
|
||||
|
||||
```
|
||||
v2/crates/wifi-densepose-sensing-server/src/main.rs # ~120 lines added
|
||||
v2/data/baseline.json # new, gitignored?
|
||||
scripts/record-baseline.py # new helper
|
||||
docs/adr/ADR-103-persistent-baseline.md # this ADR
|
||||
```
|
||||
|
||||
## Verified Acceptance (operator's deployment, 2026-05-17)
|
||||
|
||||
```
|
||||
boot: baseline: loaded 2 node overrides from data/baseline.json
|
||||
(node1=27.04, node2=14.72;
|
||||
node1_cv=2.62%, node2_cv=3.65%)
|
||||
```
|
||||
|
||||
Empty room, immediately after restart (no warmup wait):
|
||||
|
||||
```
|
||||
GLOBAL: absent CV=5.0%
|
||||
node 1 ratio=0.93, norm_cv=0.80×
|
||||
node 2 ratio=0.93, norm_cv=0.83×
|
||||
```
|
||||
|
||||
Sitting in node 2 path (off-axis from node 1):
|
||||
|
||||
```
|
||||
GLOBAL: present_still CV=8.1%
|
||||
node 1 ratio=1.05, norm_cv=1.2× (not in path, no drop)
|
||||
node 2 ratio=0.70, norm_cv=1.7× ← drop fires present_still
|
||||
```
|
||||
|
||||
Walking:
|
||||
|
||||
```
|
||||
GLOBAL: active CV=28-36%
|
||||
node 1 norm_cv=4-6×, node 2 norm_cv=7-9× ← well above 6× gate
|
||||
```
|
||||
|
||||
Universal-threshold gates `3.0 / 6.0` map to the same absolute
|
||||
12 % / 22 % we hand-tuned earlier — but now any-room-portable.
|
||||
|
||||
## Open Items
|
||||
|
||||
* **REST endpoint POST /api/v1/baseline/calibrate** — would let the
|
||||
operator press a button in `raw.html` instead of running the
|
||||
Python script. ~30 min.
|
||||
* **Per-subcarrier baseline comparison** — `per_subcarrier_mean` is
|
||||
saved but not yet consumed; future ADR-104 can use L2 distance
|
||||
per-subcarrier instead of broadband mean ratio for off-axis
|
||||
presence detection.
|
||||
* **Auto-recalibrate on long quiet periods** — if the classifier sees
|
||||
`absent` for 30 minutes with low variance, it could opportunistically
|
||||
update the baseline. Would eliminate the manual script step
|
||||
entirely. ~1 h.
|
||||
|
||||
## References
|
||||
|
||||
* ADR-100 — gain lock.
|
||||
* ADR-101 — classifier consumes the baseline.
|
||||
* ADR-102 — NBVI selection drift was the root cause of D1/D2.
|
||||
* [`docs/references/espectre-techniques.md`](../references/espectre-techniques.md)
|
||||
— Pace's full technique catalogue including Problem #3 normalization.
|
||||
|
|
@ -69,30 +69,12 @@ Four-step pipeline at boot:
|
|||
| 3 | **Rank + validate** | Sort by NBVI ascending. Run the motion detector on each candidate config, measure false-positive rate, take the config with the lowest FP. |
|
||||
| 4 | **Pick winners** | Top-K by lowest NBVI (typically K = 12 for HT20). |
|
||||
|
||||
Memory: O(N) running with on-the-fly mean/variance updates ⇒ ≈ 256 B
|
||||
for 64 subcarriers. Time: O(N · L) per recompute, milliseconds on a
|
||||
$10 device.
|
||||
Memory: O(N) running with on-the-fly mean/variance, ≈ 256 B for 64
|
||||
subcarriers. Time: O(N · L) per recompute, ms on a $10 device.
|
||||
|
||||
**RuView status — PARTIALLY DONE.** ADR-102 (commit `2f12a223`).
|
||||
Server-side port in `amp_presence_override` /
|
||||
`nbvi_select_top_k`. What we have:
|
||||
|
||||
- ✅ NBVI formula with α = 0.5
|
||||
- ✅ Top-12 selection
|
||||
- ✅ Dead-zone gate (`NBVI_DEAD_GATE_PCT = 0.25`)
|
||||
- ✅ Recompute throttled (`NBVI_REFRESH_TICKS = 200` ≈ every 5 s)
|
||||
|
||||
What we **do not** have:
|
||||
|
||||
- ❌ **Step 1 quiet-window finder** — we use the *whole* history
|
||||
buffer. If the buffer captures someone moving, ranking is biased.
|
||||
Pace's percentile-window detector should be added.
|
||||
- ❌ **Step 3 FP-rate validation** — we accept the raw NBVI ranking
|
||||
without testing it on the calibration data.
|
||||
- ❌ **Boot calibration sequence** (FW-side, 7 s post gain-lock).
|
||||
Ours is server-side rolling, which means selection drifts forever
|
||||
rather than locking after boot. Trade-off: adapts to room
|
||||
rearrangement, but never "settles".
|
||||
**RuView status — DONE (Steps 1, 2, 4).** ADR-102 (commits
|
||||
`2f12a223` + `f4119924`). Server-side, see ADR-102 for detail.
|
||||
Missing: ❌ Step 3 FP-rate validation, ❌ FW-side boot freeze.
|
||||
|
||||
Empirically on the operator's deployment NBVI alone gave a 1.5-2× CV
|
||||
reduction:
|
||||
|
|
@ -118,9 +100,11 @@ Reference 0.25 is what a quiet room typically measures during NBVI
|
|||
calibration. Apply the scale to the live motion score, so the user-
|
||||
facing threshold (`= 1.0`) is universal across rooms.
|
||||
|
||||
**RuView status — NOT DONE.** Our `amp_node_level` uses fixed
|
||||
thresholds tuned to one deployment (CV 10 % moving, CV 22 % active,
|
||||
mean/baseline < 0.75 still). Other deployments will need re-tuning.
|
||||
**RuView status — DONE.** ADR-103 D3 (commit `2f4b2d53`).
|
||||
`amp_node_level` and `amp_classify_from_latest` divide live CV by
|
||||
`baseline_cv` loaded from `data/baseline.json` and gate at universal
|
||||
`3×` (moving) / `6×` (active). Falls back to absolute gates
|
||||
`0.10 / 0.22` when no calibration loaded — backwards compatible.
|
||||
|
||||
## 4. Two-phase boot calibration
|
||||
|
||||
|
|
@ -142,9 +126,12 @@ After NBVI calibration, ESPectre writes the AGC/FFT lock values, the
|
|||
chosen subcarrier set, the baseline variance, and the threshold into
|
||||
NVS so reboots don't need re-calibration.
|
||||
|
||||
**RuView status — NOT DONE.** Each server restart triggers a fresh
|
||||
60-second baseline learn. NBVI also re-ranks from scratch on restart.
|
||||
Open item: persist `AMP_LATEST.baseline` to disk + load at startup.
|
||||
**RuView status — DONE for baseline; PARTIAL for the rest.** ADR-103
|
||||
(commits `f4119924`, `2f4b2d53`). `data/baseline.json` persists per-
|
||||
node full-broadband mean/p95/CV + per-subcarrier means, loaded at
|
||||
server boot via `load_baseline_file`. Gain lock + NBVI selection
|
||||
still happen fresh on each FW boot / server boot respectively (open
|
||||
items 4 + 5 below).
|
||||
|
||||
## 6. Interactive Web Serial game (`espectre.dev/game`)
|
||||
|
||||
|
|
@ -176,37 +163,36 @@ for the amplitude classifier or NBVI.
|
|||
|
||||
| Item | Pace / ESPectre | RuView |
|
||||
|---|---|---|
|
||||
| Gain lock | FW, 300 pkt median, AGC+FFT, AGC<30 skip | ✅ Same, in `csi_collector.c` |
|
||||
| NBVI formula | α·σ/μ² + (1-α)·σ/μ, α=0.5, top-12 | ✅ Same, server-side |
|
||||
| Dead-zone gate | 25th percentile of mean | ✅ `NBVI_DEAD_GATE_PCT=0.25` |
|
||||
| Quiet-window finder | Percentile-window in calibration buffer | ❌ Use full window verbatim |
|
||||
| FP-rate validation of NBVI pick | Yes | ❌ Take raw ranking |
|
||||
| Boot-time NBVI freeze | FW, ~7 s post-lock | ❌ Server-side rolling |
|
||||
| Baseline variance normalization | `scale = 0.25 / σ²` | ❌ Fixed thresholds per deployment |
|
||||
| NVS persistence of calibration | Yes | ❌ Fresh learn each restart |
|
||||
| Universal threshold | One value across rooms | ❌ Re-tune per deployment |
|
||||
| Calibration UI | Web Serial game | ❌ Read-only raw.html |
|
||||
| HA integration | ESPHome native | ❌ None |
|
||||
| Test suite | 500+ tests, 90 % coverage | ❌ ~2 parser tests only |
|
||||
| Phase / amplitude | Amplitude only (TPR avoidance) | ✅ Same |
|
||||
| Subcarrier count | 64 (HT20) | 56 (ESP32-S3 reports 56 non-guard) |
|
||||
| Gain lock | FW, 300 pkt median, AGC+FFT, AGC<30 skip | ✅ ADR-100 |
|
||||
| NBVI formula α=0.5, top-12, dead-zone gate | ✅ | ✅ ADR-102 |
|
||||
| Quiet-window finder (Step 1) | ✅ | ✅ ADR-102 v2 |
|
||||
| FP-rate validation (Step 3) | ✅ | ❌ raw ranking |
|
||||
| Boot-time NBVI freeze | FW, ~7 s post-lock | ❌ server-side rolling |
|
||||
| Baseline variance normalization (universal threshold) | ✅ | ✅ ADR-103 D3 |
|
||||
| Persisted baseline to disk | NVS | ✅ ADR-103 D1 (`data/baseline.json`) |
|
||||
| NVS persistence of FW calibration | ✅ | ❌ fresh each FW boot |
|
||||
| Calibration UI | Web Serial game | ❌ read-only `raw.html` |
|
||||
| HA / ESPHome integration | ✅ | ❌ none |
|
||||
| Test suite | 500+ tests, 90 % cov | ❌ 2 parser tests |
|
||||
| Phase / amplitude | amplitude only | ✅ same |
|
||||
|
||||
## Open items, ranked by expected impact on RuView
|
||||
|
||||
1. **Quiet-window finder for NBVI Step 1** — if the operator restarts
|
||||
the server while the room is occupied, current NBVI biases its
|
||||
ranking toward subcarriers stable on the *occupied* state. Bug:
|
||||
present_still then under-triggers. ~1 h.
|
||||
2. **Persist `AMP_LATEST.baseline` to disk** — eliminates the
|
||||
"step outside for 60 s" ritual after every restart. ~30 min.
|
||||
3. **Baseline variance normalization** — would let us ship one
|
||||
threshold set for any deployment. ~1 h.
|
||||
4. **FP-rate validation of NBVI pick** — would catch the case where
|
||||
the top-12 ranked subcarriers happen to overlap with a noise
|
||||
source. ~1 h.
|
||||
5. **Boot-time NBVI freeze** — if we want fully reproducible
|
||||
behaviour. Trade-off: doesn't adapt to room changes. ~2 h.
|
||||
6. **HA / ESPHome integration** — depends on whether RuView wants
|
||||
to be a HA sensor or stay standalone. ~1 day.
|
||||
7. **Web Serial calibration UI** — nice-to-have, lower priority than
|
||||
the algorithmic items. ~1 day.
|
||||
1. **REST `POST /api/v1/baseline/calibrate`** — drives the recording
|
||||
script from a button in `raw.html` instead of CLI. ~30 min.
|
||||
2. **FP-rate validation of NBVI pick** — defense against the top-12
|
||||
accidentally overlapping a noise source. ~1 h.
|
||||
3. **Per-subcarrier baseline comparison (ADR-104 draft)** — uses the
|
||||
already-saved `per_subcarrier_mean` in `baseline.json` for L2
|
||||
distance instead of broadband mean ratio. Better off-axis
|
||||
presence sensing. ~1 h.
|
||||
4. **Auto-recalibrate on long quiet periods** — if classifier sees
|
||||
`absent` with low variance for 30 min, refresh baseline in
|
||||
background. Eliminates manual script step entirely. ~1 h.
|
||||
5. **FW-side NBVI boot-freeze + NVS persistence** — full
|
||||
reproducibility, sub-second post-boot ready. Trade-off: doesn't
|
||||
adapt to room changes. ~2 h.
|
||||
6. **HA / ESPHome integration** — sensor as HA entity. ~1 day.
|
||||
7. **Test suite vs fixed 2 000-packet replay** — regression
|
||||
protection for the classifier + NBVI. ~1 day.
|
||||
8. **Web Serial calibration game** — nice-to-have. ~1 day.
|
||||
|
|
|
|||
Loading…
Reference in New Issue