From e4204595b0829f6972abc59d286d042480fcfc73 Mon Sep 17 00:00:00 2001 From: arsen Date: Sun, 17 May 2026 13:51:08 +0700 Subject: [PATCH] =?UTF-8?q?docs:=20actualization=20sweep=20=E2=80=94=20clo?= =?UTF-8?q?se=20items=20shipped=20this=20session?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-referenced every ADR Open Items section + both reference docs against the actual implementation state on the branch. Closed items the session shipped, kept stale "will be done in ADR-X" forward-refs honest: ADR-100 ✅ NBVI port (ADR-102), RSSI parse fix (3393c1e8), idle- channel keepalive (ADR-106). ⏳ Tailscale-target still open. ADR-101 ✅ per-sub baseline-drop / off-axis sit (both via ADR-104). ⏳ CV saturation above ~30 % still open. ADR-102 ✅ Step 3 FP-rate validation (ADR-104 D4). ADR-103 ✅ all three open items closed (REST endpoint via ADR-107, per-sub comparison via ADR-104, auto-recalibrate via ADR-107). ADR-106 ✅ FW-side µs timestamp via OTA (b787f40a). espectre-techniques.md: - NBVI: now "DONE (all 4 NBVI steps)" instead of "missing Step 3". - Persisted calibration: split into "server (ADR-103) + FW NVS (ADR-108)" with intentional design note for NBVI staying server-side. espectre-gap-analysis.md: - NBVI Step 3, gain-lock NVS, baseline persistence, threshold persistence all flipped to ✅ in the per-section comparison tables. - Priority list restructured into "✅ Done in this session" (10 items) + "⏳ Still open by impact" (14 items) with reality-checked estimates. Top 3 open: HA via MQTT, 2 000-packet test suite, per-sub delta sparkline in raw.html. Verbatim Pace Part-2 article still informs the gap structure; nothing was removed from his pipeline, only RuView's column updated. --- ...DR-100-gain-lock-baseline-stabilization.md | 30 +++---- docs/adr/ADR-101-raw-amplitude-classifier.md | 14 ++-- docs/adr/ADR-102-nbvi-subcarrier-selection.md | 8 +- docs/adr/ADR-103-persistent-baseline.md | 17 ++-- .../adr/ADR-106-full-complex-csi-keepalive.md | 9 ++- docs/references/espectre-gap-analysis.md | 78 +++++++++++++------ docs/references/espectre-techniques.md | 35 +++++---- 7 files changed, 106 insertions(+), 85 deletions(-) diff --git a/docs/adr/ADR-100-gain-lock-baseline-stabilization.md b/docs/adr/ADR-100-gain-lock-baseline-stabilization.md index a9d3e996..f4d3fdc2 100644 --- a/docs/adr/ADR-100-gain-lock-baseline-stabilization.md +++ b/docs/adr/ADR-100-gain-lock-baseline-stabilization.md @@ -129,25 +129,17 @@ docs/adr/ADR-100-gain-lock-baseline-stabilization.md # this ADR ## Open Items -* **NBVI subcarrier selection** is the next ESPectre technique to - port. With gain-lock alone we see 0–44 subcarriers below CV 5 % per - state — NBVI would automatically select the top-K stable ones at - boot and let the DSP compute motion variance only on those. - Expected to lift the SNR another factor of 2–3×. -* **Server-side RSSI parsing** is currently broken for the new frame - shape: `mean_rssi` returns 0 in the WS payload even though the - raw CSI frame carries a valid int8. Cosmetic; doesn't affect amplitude. -* **NVS target_ip is hardcoded** to one of Mac's two possible IPs - (192.168.0.103 on TP-Link side). When the operator switches Mac WiFi - the CSI stream stops. Long-term fix: provision sensors to send to - the Mac's Tailscale IP, which is stable across networks. Optional - short-term: a static DHCP lease on TP-Link admin so 192.168.0.103 - is reserved for the Mac. -* **Calibration latency on an idle channel.** If no host traffic - exists when the sensor boots, gain-lock collects samples at the - beacon-only rate (~0.3 fps) and takes ~17 min to converge. In - practice the host always sends something. If not — `ping -i 0.1 - 192.168.0.10x` for 30 s right after boot is enough. +* ✅ **NBVI subcarrier selection** — closed in ADR-102 (server-side + port with quiet-window finder). +* ✅ **Server-side RSSI parsing** — fixed by parallel agent in commit + `3393c1e8` (parse_esp32_frame offset realignment + carrying RSSI + through feature_state packets). +* ✅ **Calibration latency on an idle channel** — closed in ADR-106 + by the built-in managed-`ping` keepalive (drives sensor RX at + 25 pkt/s/node out of the box). +* ⏳ **NVS target_ip is hardcoded** — still open. Tailscale-target + option not implemented; sensors still send to the Mac's TP-Link- + side IP (192.168.0.103). Mac roaming still breaks the CSI stream. ## References diff --git a/docs/adr/ADR-101-raw-amplitude-classifier.md b/docs/adr/ADR-101-raw-amplitude-classifier.md index 28857296..44d27261 100644 --- a/docs/adr/ADR-101-raw-amplitude-classifier.md +++ b/docs/adr/ADR-101-raw-amplitude-classifier.md @@ -127,12 +127,14 @@ RSSI MAD-Δ classifier from ADR-099. ## Open Items -* ADR-104 will add per-node baseline-drop detection on per-subcarrier - L2 distance — currently the CV signal saturates above ~30 % so we - lose granularity on heavy movement. -* `present_still` requires the body to actually attenuate the path. - Off-axis sit doesn't trigger. Future: bring in per-subcarrier delta - for off-path presence sensing. +* ✅ **Per-subcarrier baseline-drop** — closed in ADR-104 (per-sub + drift channel with 10 % gate, triggers `present_still` even when + broadband doesn't move). +* ✅ **Off-axis sit doesn't trigger** — closed in ADR-104 (drift + channel catches off-line-of-sight body presence). +* ⏳ **CV saturates above ~30 %** — still open. Heavy-motion granularity + (run vs jog vs jump) lost above the `active` gate. Would need a + log-CV or rank-based metric to extend the dynamic range. ## References diff --git a/docs/adr/ADR-102-nbvi-subcarrier-selection.md b/docs/adr/ADR-102-nbvi-subcarrier-selection.md index 1c07dc72..85593ee8 100644 --- a/docs/adr/ADR-102-nbvi-subcarrier-selection.md +++ b/docs/adr/ADR-102-nbvi-subcarrier-selection.md @@ -113,11 +113,9 @@ sitting near a sensor now trigger. ## Open Items -* **ADR-102 Step 3 (FP-rate validation)** — Pace's full pipeline - validates each candidate top-K by running the motion detector on - the calibration buffer and picking the K with the lowest FP rate. - We take the raw ranking without validation. Could add later for - defense against NBVI accidentally selecting noise-source overlap. +* ✅ **Step 3 FP-rate validation** — closed in ADR-104 D4 (commit + `6212b17e`). K ∈ {6,8,10,12,16,20} sweep, smallest-FP wins; ties + broken by smallest total-NBVI score. * **Persist NBVI selection** — `AMP_BASELINE_OVERRIDE` (ADR-103) persists baseline scalar but not the chosen subcarrier indices. After server restart NBVI re-ranks from scratch; in deployments diff --git a/docs/adr/ADR-103-persistent-baseline.md b/docs/adr/ADR-103-persistent-baseline.md index 798ba098..8e6cd3a7 100644 --- a/docs/adr/ADR-103-persistent-baseline.md +++ b/docs/adr/ADR-103-persistent-baseline.md @@ -164,17 +164,12 @@ Universal-threshold gates `3.0 / 6.0` map to the same absolute ## Open Items -* **REST endpoint POST /api/v1/baseline/calibrate** — would let the - operator press a button in `raw.html` instead of running the - Python script. ~30 min. -* **Per-subcarrier baseline comparison** — `per_subcarrier_mean` is - saved but not yet consumed; future ADR-104 can use L2 distance - per-subcarrier instead of broadband mean ratio for off-axis - presence detection. -* **Auto-recalibrate on long quiet periods** — if the classifier sees - `absent` for 30 minutes with low variance, it could opportunistically - update the baseline. Would eliminate the manual script step - entirely. ~1 h. +* ✅ **REST endpoint POST /api/v1/baseline/calibrate** — closed in + ADR-107 D3 + UI button D6. +* ✅ **Per-subcarrier baseline comparison** — closed in ADR-104 + (per-sub drift channel consumes `per_subcarrier_mean`). +* ✅ **Auto-recalibrate on long quiet periods** — closed in ADR-107 D5 + (30-min quiet + 1-h cooldown defaults). ## References diff --git a/docs/adr/ADR-106-full-complex-csi-keepalive.md b/docs/adr/ADR-106-full-complex-csi-keepalive.md index 14183d6e..bc9c2868 100644 --- a/docs/adr/ADR-106-full-complex-csi-keepalive.md +++ b/docs/adr/ADR-106-full-complex-csi-keepalive.md @@ -133,10 +133,11 @@ which is more sensitive to chest-wall micrometric motion. ## Out of scope / open -* **FW-side µs timestamp** — `info->rx_ctrl.timestamp` (u32, µs) is - in `wifi_pkt_rx_ctrl_t` per ESP-IDF docs. Not yet propagated through - the 0xC511_0001 binary header (reserved bytes 18..19 are available - but unused). Future ADR-107 will reshape the header to include it. +* ✅ **FW-side µs timestamp** — closed in commit `b787f40a`. FW now + appends `info->rx_ctrl.timestamp` (u32 LE) as 4 trailing bytes + after I/Q data; server parses opportunistically (None for older + FW). NodeInfo.timestamp_us now carries sensor monotonic µs when + available, falls back to server SystemTime otherwise. * **Per-frame antenna selection** when ESP32-S3 reports >1 antenna — current FW hard-codes `n_antennas=1` in `csi_collector.c`. Single- antenna deployments are unaffected. diff --git a/docs/references/espectre-gap-analysis.md b/docs/references/espectre-gap-analysis.md index 01ec7d8b..129c4e97 100644 --- a/docs/references/espectre-gap-analysis.md +++ b/docs/references/espectre-gap-analysis.md @@ -12,12 +12,13 @@ missing** breakdown, structured exactly along the sections of Pace's | Formula `α·σ/μ² + (1-α)·σ/μ`, α = 0.5 | ✅ ADR-102 | | Step 1: quiet-window finder | ✅ ADR-102 v2 | | Step 2: 25 %-percentile dead-zone gate | ✅ ADR-102 | -| **Step 3: rank + validate (run motion detector on the calibration buffer, pick K with lowest FP rate)** | ❌ raw ranking accepted | +| **Step 3: rank + validate** | ✅ ADR-104 D4 (commit `6212b17e`) — K ∈ {6,8,10,12,16,20} sweep, smallest-FP wins, ties by smallest total-NBVI | | Step 4: pick top-K (K=12) | ✅ ADR-102 | | Amplitude only (no phase) | ✅ same | -Step 3 absence means a noisy WiFi neighbour with energy concentrated -on our top-12 subcarriers would still get picked. Defence: validate. +All four NBVI steps shipped. If a noisy neighbour energy-overlaps the +top-K, the validator counts FPs over the quiet window and a tighter +(or different) K wins. ## Problem #2: Gain Lock (AGC + FFT) @@ -43,12 +44,14 @@ PHASE 2 (7 s) rank subcarriers with gain locked → save top-K to NVS | Phase | Status in RuView | |---|---| | Phase 1 in FW | ✅ ADR-100 (`csi_collector.c::rv_gain_lock_process`) | -| **Phase 2 in FW after Phase 1** | ❌ NBVI lives in the server as a rolling refresh, not a boot-time freeze | -| **NVS save of both lock + selection** | ❌ each FW boot re-calibrates gain; NBVI re-ranks every server boot | +| **Phase 2 in FW after Phase 1** | ⏳ NBVI intentionally in server as rolling refresh (adapts to slow channel drift). Not planned in FW. | +| **NVS save of gain-lock** | ✅ ADR-108 (commit `3779bb76`) — `csi_cfg/gl_agc` + `gl_fft` | +| **NVS save of NBVI selection** | ⏳ NBVI lives server-side, doesn't apply | -Doing Phase 2 in FW would mean reboot → ready in 0.5 s instead of -~10 s. Trade-off: doesn't adapt to room changes without explicit -re-calibration. +After ADR-108 the FW boots → CSI ready in ~0.5 s (NVS restore) instead +of ~10 s (full 300-packet calibration). Adapting to room changes +without recalibration is now a "clear NVS keys" operation — open item +ADR-108 #1 will surface that as a REST endpoint. ## Persisted calibration (NVS on the sensor) @@ -65,13 +68,14 @@ second: | Item | Status in RuView | |---|---| | WiFi creds + collector IP in NVS | ✅ `csi_cfg` namespace | -| **Gain lock NVS persistence** | ❌ recomputed on every FW boot | -| **NBVI selection NVS persistence** | ❌ recomputed on every server boot | -| **Baseline NVS persistence** | partial — server persists to disk (ADR-103), not on the sensor | -| **Threshold NVS persistence** | ❌ universal threshold loaded from `data/baseline.json` server-side | +| **Gain lock NVS persistence** | ✅ ADR-108 (`csi_cfg/gl_agc` + `gl_fft`) | +| **NBVI selection NVS persistence** | ⏳ server-side rolling, intentional | +| **Baseline NVS persistence** | ✅ on host disk via ADR-103 (`data/baseline.json`); not on sensor — server is required | +| **Threshold NVS persistence** | ✅ derives from baseline_cv loaded by ADR-103 | If we ever ship to operators who don't run the Rust server (pure FW -+ HA), all of these become required. ++ HA), the server-side bits (NBVI / baseline / threshold) would have +to migrate to the sensor's NVS. Not on the current roadmap. ## The Game (Web Serial calibration UI) @@ -123,26 +127,54 @@ ESPHome component, an MQTT bridge, or a custom HA integration. * Fall detection * Person vs. pet classification -## Priority for RuView, ranked by expected impact +## Priority for RuView — current state + +### ✅ Done in this session + +| Item | Where | +|---|---| +| NVS persistence of gain-lock | ADR-108 (`3779bb76`) | +| FP-rate validation of NBVI (Step 3) | ADR-104 D4 (`6212b17e`) | +| `POST /api/v1/baseline/calibrate` + UI button | ADR-107 (`0f373467`, `45c1464c`) | +| Auto-recalibrate on long-quiet periods | ADR-107 (`0f373467`) | +| Per-subcarrier baseline comparison | ADR-104 (`6212b17e`) | +| Full complex CSI in WS (amp+phase+meta) | ADR-106 (`4daa2c9b`) | +| Sensor µs timestamp from FW | ADR-106 (`b787f40a`) | +| Managed-ping CSI keepalive (no ручной ping) | ADR-106 (`8489efe9`) | +| No synthetic data in production runtime | ADR-105 (`9aa027e9`, `30244d27`) | +| OTA flash via WiFi (8032 port) | `ota-pipeline.md` (`274984d3`) | + +### ⏳ Still open, by impact | # | Item | Net benefit | Estimate | |---|---|---|---| -| 1 | NVS persistence + boot-time NBVI freeze in FW | reboot → ready in 0.5 s instead of ~10 s; survives server outage | 3-4 h | -| 2 | FP-rate validation of NBVI Step 3 | defence against noise-source subcarrier overlap | 1 h | -| 3 | `POST /api/v1/baseline/calibrate` + button in `raw.html` | calibrate from browser instead of CLI script | 30 min | -| 4 | Auto-recalibrate on long-quiet periods | drops the manual step entirely | 1-2 h | -| 5 | HA via MQTT (lighter than full ESPHome rewrite) | sensor as HA entity | 1 day | -| 6 | Fixed-replay test suite (2 000 packets) | regression protection | 1 day | -| 7 | Per-subcarrier baseline comparison (ADR-104 draft) | off-axis presence detection | 1 h | -| 8 | Web Serial calibration game | nice-to-have | 1 day | -| 9 | ESPHome native component (instead of MQTT bridge) | tighter HA integration | 2-3 days | +| 1 | **HA via MQTT** | sensor as HA entity, ecosystem reach | 1 day | +| 2 | **Fixed-replay test suite (2 000 packets)** | regression protection over the classifier + NBVI | 1 day | +| 3 | **Per-sub delta sparkline in `raw.html`** | operator sees off-axis drift channel firing in real time | 30 min | +| 4 | **`POST /ota/recalibrate` (clear NVS gain-lock)** | reset gain-lock without USB after AP swap or relocation | 30 min FW + flash | +| 5 | **Track AP MAC in NVS alongside AGC/FFT** | auto-invalidate stale gain-lock on AP change | 1 h FW + flash | +| 6 | **Multi-AP signal_field via `MultistaticFuser`** | physically real spatial map (today zero-filled per ADR-105 D6) | 2-3 h | +| 7 | **Per-subcarrier baseline AGE check** | flag for re-calibration when channel slowly drifts | 1 h | +| 8 | **Phase-domain drift (vs amplitude-only today)** | sub-mm chest-wall motion detection for vitals | 1 h script + 30 min server | +| 9 | **Tailscale-target in NVS** | sensor stream keeps working when Mac roams networks | 30 min provision + reflash | +| 10 | **ESPHome native component (instead of MQTT bridge)** | tighter HA integration than #1 | 2-3 days | +| 11 | **Web Serial calibration game** | playful threshold tuning | 1 day | +| 12 | **Boot-time NBVI freeze in FW** | trade-off vs adaptive: don't adopt unless we see FP issues in real homes | 2 h | +| 13 | **Per-channel NVS cache for gain-lock** | only needed if channel hopping (ADR-029) re-activated | 1 h | +| 14 | **DensePose model train + load** | unlock pose estimation; depends on MM-Fi / Wi-Pose dataset access | 1-3 days | ## References * [`espectre-techniques.md`](espectre-techniques.md) — technique catalogue +* [`ota-pipeline.md`](ota-pipeline.md) — WiFi-OTA recipe (port 8032) * [ADR-100](../adr/ADR-100-gain-lock-baseline-stabilization.md) — gain lock * [ADR-101](../adr/ADR-101-raw-amplitude-classifier.md) — classifier * [ADR-102](../adr/ADR-102-nbvi-subcarrier-selection.md) — NBVI * [ADR-103](../adr/ADR-103-persistent-baseline.md) — baseline persistence +* [ADR-104](../adr/ADR-104-per-subcarrier-drift-presence.md) — per-sub drift + NBVI FP-validation +* [ADR-105](../adr/ADR-105-no-synthetic-data-in-production-runtime.md) — no synthetic data +* [ADR-106](../adr/ADR-106-full-complex-csi-keepalive.md) — full complex CSI + keepalive +* [ADR-107](../adr/ADR-107-auto-recalibrate-and-rest-baseline.md) — REST + auto-recalibrate +* [ADR-108](../adr/ADR-108-fw-nvs-persist-gain-lock.md) — FW NVS persist gain-lock * Pace, *How I Turned My Wi-Fi Into a Motion Sensor — Part 2*, Dec 2025 * `francescopace/espectre` on GitHub (GPLv3) diff --git a/docs/references/espectre-techniques.md b/docs/references/espectre-techniques.md index 21e86588..da5c2f9e 100644 --- a/docs/references/espectre-techniques.md +++ b/docs/references/espectre-techniques.md @@ -1,13 +1,9 @@ # ESPectre (Francesco Pace) — Technique Reference -Source: *How I Turned My Wi-Fi Into a Motion Sensor — Part 2* -(Dec 2025), Medium / [francescopace/espectre](https://github.com/francescopace/espectre) -on GitHub, GPLv3. - -Captures the three core techniques and the support tooling Pace -shipped. RuView has adopted some, partially adopted others, and not -adopted the rest. This doc is a living checklist — update when items -move. +Source: Pace's *Part 2* (Dec 2025) + +[francescopace/espectre](https://github.com/francescopace/espectre) +(GPLv3). Living checklist of techniques + RuView adoption status; +update when items move. ## 1. Gain Lock (AGC + FFT scale) @@ -72,9 +68,11 @@ Four-step pipeline at boot: Memory: O(N) running with on-the-fly mean/variance, ≈ 256 B for 64 subcarriers. Time: O(N · L) per recompute, ms on a $10 device. -**RuView status — DONE (Steps 1, 2, 4).** ADR-102 (commits -`2f12a223` + `f4119924`). Server-side, see ADR-102 for detail. -Missing: ❌ Step 3 FP-rate validation, ❌ FW-side boot freeze. +**RuView status — DONE (all 4 NBVI steps).** Server-side: ADR-102 +(`2f12a223`, `f4119924`) covers Steps 1+2+4; ADR-104 D4 (`6212b17e`) +closes Step 3 (K ∈ {6,8,10,12,16,20} sweep, smallest-FP wins). FW- +side boot freeze remains intentionally absent — server-side rolling +refresh adapts to slow channel drift (ADR-102 D6). Empirically on the operator's deployment NBVI alone gave a 1.5-2× CV reduction: @@ -126,12 +124,15 @@ After NBVI calibration, ESPectre writes the AGC/FFT lock values, the chosen subcarrier set, the baseline variance, and the threshold into NVS so reboots don't need re-calibration. -**RuView status — DONE for baseline; PARTIAL for the rest.** ADR-103 -(commits `f4119924`, `2f4b2d53`). `data/baseline.json` persists per- -node full-broadband mean/p95/CV + per-subcarrier means, loaded at -server boot via `load_baseline_file`. Gain lock + NBVI selection -still happen fresh on each FW boot / server boot respectively (open -items 4 + 5 below). +**RuView status — DONE.** Two-layer persistence: +* **Server side (ADR-103, commits `f4119924`, `2f4b2d53`)**: + `data/baseline.json` keeps per-node full-broadband mean/p95/CV + + per-subcarrier means, loaded on server boot via `load_baseline_file`. +* **FW side (ADR-108, commit `3779bb76`)**: gain-lock AGC + FFT + saved to NVS namespace `csi_cfg` keys `gl_agc`/`gl_fft` after the + first calibration; subsequent boots restore instantly (skip the + 300-packet sampler). NBVI selection is **intentionally** server- + side rolling, not persisted — design choice, not a gap. ## 6. Interactive Web Serial game (`espectre.dev/game`)