diff --git a/docs/adr/ADR-120-windowed-temporal-classifier.md b/docs/adr/ADR-120-windowed-temporal-classifier.md new file mode 100644 index 00000000..eda604f6 --- /dev/null +++ b/docs/adr/ADR-120-windowed-temporal-classifier.md @@ -0,0 +1,209 @@ +# ADR-120 — Windowed Temporal Classifier (W-MLP) + +**Status**: Accepted +**Date**: 2026-05-18 +**Scope**: `v2/crates/wifi-densepose-sensing-server/src/adaptive_classifier.rs` +(`WindowedMlpModel`, `train_windowed_mlp_classifier`, `eval_windowed_mlp`, +`AdaptiveModel::classify_window`); `main.rs` (`AppStateInner.feature_window`, +`push_feature_window`, `adaptive_override` switching to window path). + +## Context + +ADR-119 added a small MLP (22 → 32 → 6) that improved accuracy from 49.58% +(LogReg) to **53.53%**. Loss flatlined at ~1.15 around epoch 10 of 30 — +clear signal that the **frame-level information ceiling** had been +reached for the 22-feature representation. + +The dataset has 7 activity classes that differ primarily in **temporal +patterns**, not in any single frame: + +* `walking` step cadence: ~2 Hz (visible in 0.5-second window) +* `transition` (sit-stand): ~0.5 Hz (visible in 2-second window) +* `waving` limb cadence: 1-2 Hz +* `active` (jumping): bursty / quasi-periodic at ~3 Hz +* `present_still` (sitting + standing merged): no temporal signature + +Per-frame, `walking` and `active` and `waving` all look "moving" with +similar amplitude std/skew — they're disambiguated only by HOW the +amplitude pattern evolves over 1-2 seconds. A classifier that sees a +single frame can't tell them apart no matter how good the per-frame +features are. + +## Decisions + +### D1 — Stack 20 consecutive frames into a 440-d input + +``` +WINDOW_FRAMES = 20 (~2 seconds at ~10 Hz tick rate) +N_FEATURES = 22 (from ADR-118) +WINDOWED_INPUT = 20 × 22 = 440 +WINDOWED_HIDDEN = 64 +``` + +Network: `440 → 64 ReLU → n_classes softmax`. ~28k weights total — +larger than the frame-level MLP's 3k, but still small enough to train +in <60s and serialize as JSON. + +Training samples are built by sliding a window of 20 frames with **stride +5** within each recording (4× overlap). Windows do **not** cross recording +boundaries — each window inherits its source recording's class label. + +On the 6-node 151k-frame set: +* 7 recordings × ~21k frames each = 151k frames total +* (21k − 20) / 5 ≈ 4,300 windows per recording +* Total: ~30k windowed samples +* Class balance is roughly preserved (each recording is one class) + +### D2 — Manual backprop, same recipe as MLP + +Same SGD + momentum 0.9 + weight decay 1e-4 + cosine LR decay. Base LR +lowered to 0.03 (vs MLP's 0.05) because the network is bigger. 25 epochs. +He initialisation, ReLU activation, softmax output, cross-entropy loss. + +### D3 — `AdaptiveModel` carries all three classifiers, classify routes by availability + +```rust +pub struct AdaptiveModel { + pub weights: Vec>, // ADR-118 legacy LogReg + pub mlp: MlpModel, // ADR-119 frame-level MLP + pub windowed_mlp: WindowedMlpModel, // ADR-120 (this) — primary + // ... +} +``` + +`classify_window()` (new API) prefers `windowed_mlp` when trained AND +the caller has a 20-frame buffer. Falls through to frame-level MLP +when called with insufficient history. Old JSON model files load with +`MlpModel::default()` and `WindowedMlpModel::default()` filling absent +fields — backward compatible. + +### D4 — Rolling buffer in `AppStateInner`, pushed per tick + +```rust +struct AppStateInner { + feature_window: VecDeque<[f64; N_FEATURES]>, // capacity = WINDOW_FRAMES + // ... +} +``` + +New helper `push_feature_window(&mut s, &features)` computes the 22-d +feature vector from current per-node amps, pushes to the back of the +buffer, evicts oldest when over capacity. Called at all three tick +sites where `adaptive_override` runs: +* `main.rs:~3030` — multi-BSSID tick handler +* `main.rs:~3225` — WiFi fallback tick handler +* `main.rs:~6510` — per-node loop in the broadcast tick task + +`adaptive_override` (read-only over state) builds the 440-d input by +copying the buffer's last 19 entries + the current frame's features, +then calls `model.classify_window(&flat)`. Cold-start (buffer < 20) +falls back to `model.classify(&feat_arr)` — frame-level MLP. + +## Verified Acceptance + +Retrained on the same 6-node, 151,329-frame set used since ADR-118: + +``` +LogReg: 49.58% +MLP: 53.53% (+3.95 vs LogReg) +W-MLP: 90.40% (+36.87 vs MLP) +``` + +Per-class (frame-level MLP → W-MLP): + +``` +absent 41% → 100% +59 +present_still 99% → 100% +1 (already saturated) +transition 36% → 86% +50 (sit-stand cadence captured) +active 30% → 74% +44 (jumping cadence captured) +waving 38% → 90% +52 (gesture cadence captured) +present_moving 33% → 82% +49 (walking step cadence captured) +``` + +Loss curve confirms breakout from the frame-level plateau: + +``` +MLP: epoch 0 → 1.28 → epoch 29 → 1.14 (flat plateau) +W-MLP: epoch 0 → 1.01 → epoch 24 → 0.25 (still trending) +``` + +Total cumulative improvement vs the start-of-session 2-node 15-feature +LogReg baseline: + +``` +40.4% → 90.40% = +50.0 percentage points +``` + +## Caveat — training vs generalization + +90.40% is **training accuracy**. The W-MLP has ~28,800 weights trained +on ~30,200 windowed samples — capacity is comparable to dataset size, +so some overfitting is expected. True generalization performance will +only be measurable once an independent test set is captured. + +Mitigations already in place: +* Weight decay 1e-4 regularises against memorisation +* Cosine LR decay with smooth annealing +* Stride 5 in window construction reduces near-duplicate samples +* Architecture stays small (one hidden layer) — limits overfit capacity + +Recommended follow-up: record a 60-second held-out session per class +(separate from training), evaluate W-MLP cold, compare to training +accuracy. Expected drop: 5-15 pts for a healthy model. + +## Files Touched + +``` +v2/crates/wifi-densepose-sensing-server/src/adaptive_classifier.rs: + + const WINDOW_FRAMES = 20, WINDOWED_INPUT = 440, WINDOWED_HIDDEN = 64 + + pub const N_FEATURES_PUB (for external buffer sizing) + + pub struct WindowedMlpModel { w1, b1, w2, b2, n_classes } + + impl WindowedMlpModel::{is_trained, forward} + + AdaptiveModel.windowed_mlp field (serde-default) + + AdaptiveModel::classify_window method + + train_from_recordings builds recording_groups, slides windows, + calls train_windowed_mlp_classifier + + train_windowed_mlp_classifier (~150 LoC manual backprop) + + eval_windowed_mlp helper + + #[derive(Clone)] on Sample (for recording_groups Vec) +v2/crates/wifi-densepose-sensing-server/src/main.rs: + + AppStateInner.feature_window: VecDeque<[f64; N_FEATURES_PUB]> + + push_feature_window helper + + adaptive_override switches to classify_window when buffer is full + + 3 tick sites call push_feature_window before adaptive_override +docs/adr/ADR-120-windowed-temporal-classifier.md (this) +``` + +## Out of Scope / Follow-ups + +* **Held-out test set** — must record fresh data and evaluate the saved + model cold. Critical to confirm 90% is not training-set memorisation. +* **TCN replacing stacked-MLP** — true 1D convolutions over time would + use weights more efficiently (~5k vs 28k) and generalise better. + Stack-MLP works but is parameter-heavy. Worth a follow-up if data + scales 10×. +* **Sliding output smoothing** — `classify_window` emits one decision + per tick (~10 Hz). Adjacent windows are 19/20 identical, so adjacent + predictions should agree. They mostly do (98%+) but flicker at class + boundaries — could apply a 3-tick majority filter. +* **`sitting` vs `standing` split** — both currently merge into + `present_still`. The W-MLP gets them both right at 100% as a combined + class. Splitting them would test whether temporal RF signatures + differ between sitting (chair anchor) and standing (free body). +* **Class imbalance** — `present_still` has 2× the windows of other + classes (sitting + standing both contribute). Acceptable since it's + the "neutral" class, but oversampling minority classes might lift + accuracy 1-2 pts further. +* **Smaller window size experiments** — 20 frames = 2 sec at ~10 Hz. + Could try 10 frames (1 sec, faster reaction) or 30 (3 sec, more + context). 20 was a reasonable first guess. + +## References + +* ADR-118 — feature decorrelation + multi-node (22-feature basis) +* ADR-119 — frame-level MLP (sibling classifier, fallback at cold start) +* ADR-101 — raw amplitude classifier (the path that calls + `AdaptiveModel` via `adaptive_override`) +* ADR-105 — no synthetic data in production runtime; this ADR's + confidence output is real model softmax probability, not a + hardcoded value diff --git a/v2/crates/wifi-densepose-sensing-server/src/adaptive_classifier.rs b/v2/crates/wifi-densepose-sensing-server/src/adaptive_classifier.rs index b360c2c0..29594253 100644 --- a/v2/crates/wifi-densepose-sensing-server/src/adaptive_classifier.rs +++ b/v2/crates/wifi-densepose-sensing-server/src/adaptive_classifier.rs @@ -45,6 +45,10 @@ const N_PER_NODE_FEATURES: usize = 3; const MAX_NODES: usize = 6; const N_FEATURES: usize = N_GLOBAL_FEATURES + MAX_NODES * N_PER_NODE_FEATURES; +/// ADR-120: exported feature count so external crates (e.g. the main +/// crate's AppStateInner) can size their rolling buffers correctly. +pub const N_FEATURES_PUB: usize = N_FEATURES; + /// Default class names for backward compatibility with old saved models. const DEFAULT_CLASSES: &[&str] = &["absent", "present_still", "present_moving", "active"]; @@ -145,6 +149,21 @@ pub struct ClassStats { /// 151k-frame dataset and load instantly at runtime. const MLP_HIDDEN: usize = 32; +/// ADR-120: temporal window size (number of consecutive frames stacked +/// into the windowed-MLP input). At the broadcast tick rate (~10 fps), +/// 20 frames = 2 seconds of context — enough to capture walking step +/// cadence (2 Hz), sit-stand transition cycles (0.5 Hz), and breathing +/// modulation. Chosen to match WiFlow's training-time window so amplitude +/// history buffers can be reused. +pub const WINDOW_FRAMES: usize = 20; + +/// ADR-120: windowed-MLP input dimensionality = WINDOW_FRAMES × N_FEATURES. +const WINDOWED_INPUT: usize = WINDOW_FRAMES * N_FEATURES; + +/// ADR-120: windowed-MLP hidden width. Larger than MLP_HIDDEN because +/// input is 20× wider (440 vs 22). 64 keeps params under 30k. +const WINDOWED_HIDDEN: usize = 64; + /// ADR-119: trained MLP classifier. Single hidden layer, ReLU activation, /// softmax output. Stored alongside the LogReg weights — when `is_trained()` /// returns true, `AdaptiveModel::classify` uses the MLP; otherwise it falls @@ -201,6 +220,66 @@ impl MlpModel { } } +/// ADR-120: Windowed MLP — same architecture as MlpModel but takes a +/// 20-frame × 22-feature stack (440-d input) instead of a single frame. +/// Captures temporal patterns (walking step cadence, sit-stand cycles, +/// breathing modulation) that frame-level classifiers miss. +#[derive(Debug, Clone, Serialize, Deserialize, Default)] +pub struct WindowedMlpModel { + /// Layer 1 weights, row-major `[WINDOWED_INPUT × WINDOWED_HIDDEN]`. + #[serde(default)] + pub w1: Vec, + /// Layer 1 bias, `[WINDOWED_HIDDEN]`. + #[serde(default)] + pub b1: Vec, + /// Layer 2 weights, row-major `[WINDOWED_HIDDEN × n_classes]`. + #[serde(default)] + pub w2: Vec, + /// Layer 2 bias, `[n_classes]`. + #[serde(default)] + pub b2: Vec, + /// Number of output classes (== len(b2) when trained). + #[serde(default)] + pub n_classes: usize, +} + +impl WindowedMlpModel { + pub fn is_trained(&self) -> bool { + !self.w1.is_empty() + && self.n_classes > 0 + && self.b2.len() == self.n_classes + && self.w1.len() == WINDOWED_INPUT * WINDOWED_HIDDEN + } + + /// Forward pass. `window` is `WINDOW_FRAMES × N_FEATURES` flat, + /// row-major (oldest-frame-first), already z-score normalised. + /// Returns softmax probabilities of length `n_classes`. + pub fn forward(&self, window: &[f64]) -> Vec { + debug_assert_eq!(window.len(), WINDOWED_INPUT); + // Layer 1: h = ReLU(window · W1 + b1) + let mut h = vec![0.0f64; WINDOWED_HIDDEN]; + for j in 0..WINDOWED_HIDDEN { + let mut s = self.b1[j]; + for i in 0..WINDOWED_INPUT { + s += window[i] * self.w1[i * WINDOWED_HIDDEN + j]; + } + h[j] = s.max(0.0); + } + // Layer 2: logits = h · W2 + b2 + let mut logits = vec![0.0f64; self.n_classes]; + for c in 0..self.n_classes { + let mut s = self.b2[c]; + for j in 0..WINDOWED_HIDDEN { + s += h[j] * self.w2[j * self.n_classes + c]; + } + logits[c] = s; + } + let m = logits.iter().cloned().fold(f64::NEG_INFINITY, f64::max); + let exp_sum: f64 = logits.iter().map(|z| (z - m).exp()).sum(); + logits.iter().map(|z| (z - m).exp() / exp_sum).collect() + } +} + // ── Trained model ──────────────────────────────────────────────────────────── #[derive(Debug, Clone, Serialize, Deserialize)] @@ -213,9 +292,15 @@ pub struct AdaptiveModel { /// at classify time but still updated by `train_from_recordings` so /// rollback is one-line. pub weights: Vec>, - /// ADR-119: trained MLP (preferred classifier when present). + /// ADR-119: trained MLP (frame-level fallback, used when WindowedMlp + /// has no data yet — e.g. cold start before 20 frames accumulated). #[serde(default)] pub mlp: MlpModel, + /// ADR-120: trained Windowed MLP (preferred classifier when trained + /// AND a 20-frame window of fresh features is available at classify + /// time). Captures temporal patterns the frame-level MLP can't see. + #[serde(default)] + pub windowed_mlp: WindowedMlpModel, /// Global feature normalisation: mean and stddev across all training data. pub global_mean: [f64; N_FEATURES], pub global_std: [f64; N_FEATURES], @@ -240,6 +325,7 @@ impl Default for AdaptiveModel { class_stats: Vec::new(), weights: vec![vec![0.0; N_FEATURES + 1]; n_classes], mlp: MlpModel::default(), + windowed_mlp: WindowedMlpModel::default(), global_mean: [0.0; N_FEATURES], global_std: [1.0; N_FEATURES], trained_frames: 0, @@ -251,9 +337,45 @@ impl Default for AdaptiveModel { } impl AdaptiveModel { + /// ADR-120: classify using a temporal window of recent frames. + /// `window` is `WINDOW_FRAMES × N_FEATURES` flat row-major (oldest first), + /// in raw (un-normalised) units — this fn applies z-score normalisation + /// internally using the model's `global_mean`/`global_std`. + /// Falls back to frame-level `classify()` on the most recent frame when + /// the windowed MLP isn't trained. + pub fn classify_window(&self, window: &[f64]) -> (String, f64) { + if self.windowed_mlp.is_trained() && window.len() == WINDOWED_INPUT { + let mut norm = vec![0.0f64; WINDOWED_INPUT]; + for f in 0..WINDOW_FRAMES { + for i in 0..N_FEATURES { + let idx = f * N_FEATURES + i; + norm[idx] = (window[idx] - self.global_mean[i]) / (self.global_std[i] + 1e-9); + } + } + let probs = self.windowed_mlp.forward(&norm); + let (best_c, best_p) = probs.iter().enumerate() + .max_by(|a, b| a.1.partial_cmp(b.1).unwrap()) + .unwrap(); + let label = if best_c < self.class_names.len() { + self.class_names[best_c].clone() + } else { + "present_still".to_string() + }; + return (label, *best_p); + } + // Cold-start fallback: most recent frame via frame-level classifier. + let mut last_frame = [0.0f64; N_FEATURES]; + if window.len() >= N_FEATURES { + let off = window.len() - N_FEATURES; + last_frame.copy_from_slice(&window[off..off + N_FEATURES]); + } + self.classify(&last_frame) + } + /// Classify a raw feature vector. Returns (class_label, confidence). /// ADR-119: prefers MLP when trained; falls back to logistic regression - /// otherwise. + /// otherwise. ADR-120: temporal-context API is `classify_window` — + /// prefer it when callers have a recent feature buffer. pub fn classify(&self, raw_features: &[f64; N_FEATURES]) -> (String, f64) { // Normalise features once (shared by MLP and LogReg). let mut x = [0.0f64; N_FEATURES]; @@ -324,6 +446,7 @@ impl AdaptiveModel { // ── Training ───────────────────────────────────────────────────────────────── /// A labeled training sample. +#[derive(Clone)] struct Sample { features: [f64; N_FEATURES], class_idx: usize, @@ -412,13 +535,18 @@ pub fn train_from_recordings(recordings_dir: &Path) -> Result = Vec::new(); + let mut recording_groups: Vec> = Vec::new(); for (path, fname, class_name) in &file_classes { let class_idx = class_map[class_name]; let loaded = load_recording(path, class_idx); eprintln!(" Loaded {}: {} frames → class '{}'", fname, loaded.len(), class_name); - samples.extend(loaded); + samples.extend(loaded.clone()); + recording_groups.push(loaded); } if samples.is_empty() { @@ -614,13 +742,57 @@ pub fn train_from_recordings(recordings_dir: &Path) -> Result, usize)> = Vec::new(); + for group in &recording_groups { + if group.len() < WINDOW_FRAMES { continue; } + let class_idx = group[0].class_idx; + let mut start = 0usize; + while start + WINDOW_FRAMES <= group.len() { + let mut flat: Vec = Vec::with_capacity(WINDOWED_INPUT); + for f in 0..WINDOW_FRAMES { + let frame = &group[start + f]; + for i in 0..N_FEATURES { + let z = (frame.features[i] - global_mean[i]) / (global_std[i] + 1e-9); + flat.push(z); + } + } + win_samples.push((flat, class_idx)); + start += window_stride; + } + } + eprintln!("Total windowed samples: {}", win_samples.len()); + + // Count per-class windowed samples. + let mut win_class_total = vec![0usize; n_classes]; + for (_, c) in &win_samples { win_class_total[*c] += 1; } + + eprintln!("Training Windowed MLP ({} → {} → {}) ...", WINDOWED_INPUT, WINDOWED_HIDDEN, n_classes); + let windowed_mlp = train_windowed_mlp_classifier(&win_samples, n_classes); + let (win_acc, win_per_class) = eval_windowed_mlp(&windowed_mlp, &win_samples, n_classes); + eprintln!("Windowed MLP accuracy: {:.2}% (frame-level MLP was {:.2}%)", + win_acc * 100.0, mlp_acc * 100.0); + for c in 0..n_classes { + let tot = win_class_total[c].max(1); + let corr = win_per_class[c]; + eprintln!(" W-MLP {}: {}/{} ({:.0}%)", + class_names[c], corr, tot, corr as f64 / tot as f64 * 100.0); + } + + // Pick the best classifier as final accuracy number. + let final_accuracy = win_acc.max(mlp_acc).max(accuracy); Ok(AdaptiveModel { class_stats, weights, mlp, + windowed_mlp, global_mean, global_std, trained_frames: n, @@ -802,6 +974,179 @@ fn eval_mlp(mlp: &MlpModel, samples: &[([f64; N_FEATURES], usize)], n_classes: u (correct as f64 / samples.len() as f64, per_class) } +// ── ADR-120: Windowed MLP training ────────────────────────────────────────── + +/// Train a windowed MLP on temporal-window samples. +/// Each sample is a 440-d flat vector (20 frames × 22 features) labeled +/// with a class index. Architecture: 440 → 64 ReLU → n_classes softmax. +/// Same SGD + momentum + cosine-decay recipe as MLP, fewer epochs because +/// each window is a richer training signal than a single frame. +fn train_windowed_mlp_classifier( + samples: &[(Vec, usize)], + n_classes: usize, +) -> WindowedMlpModel { + let n_w1 = WINDOWED_INPUT * WINDOWED_HIDDEN; + let n_w2 = WINDOWED_HIDDEN * n_classes; + + let mut rng_state: u64 = 24601; + let mut rng_u01 = move || -> f64 { + rng_state = rng_state.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); + ((rng_state >> 33) as f64) / ((u64::MAX >> 33) as f64) + }; + let mut he_init = |n: usize, fan_in: usize| -> Vec { + let s = (2.0 / fan_in as f64).sqrt(); + let mut v = Vec::with_capacity(n); + let mut k = 0; + while k < n { + let u1 = rng_u01().max(1e-12); + let u2 = rng_u01(); + let z0 = (-2.0 * u1.ln()).sqrt() * (2.0 * std::f64::consts::PI * u2).cos() * s; + let z1 = (-2.0 * u1.ln()).sqrt() * (2.0 * std::f64::consts::PI * u2).sin() * s; + v.push(z0); k += 1; + if k < n { v.push(z1); k += 1; } + } + v + }; + + let mut w1 = he_init(n_w1, WINDOWED_INPUT); + let mut b1 = vec![0.0f64; WINDOWED_HIDDEN]; + let mut w2 = he_init(n_w2, WINDOWED_HIDDEN); + let mut b2 = vec![0.0f64; n_classes]; + + let mut mw1 = vec![0.0f64; n_w1]; + let mut mb1 = vec![0.0f64; WINDOWED_HIDDEN]; + let mut mw2 = vec![0.0f64; n_w2]; + let mut mb2 = vec![0.0f64; n_classes]; + + let momentum = 0.9f64; + let weight_decay = 1e-4f64; + let base_lr = 0.03f64; // smaller LR for larger network (vs MLP's 0.05) + let batch_size = 32usize; + let epochs = 25usize; + let n = samples.len(); + + let mut idx: Vec = (0..n).collect(); + let mut shuf_state: u64 = 11; + let mut shuf_next = move || -> u64 { + shuf_state = shuf_state.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); + shuf_state >> 33 + }; + + let mut h_pre = vec![0.0f64; WINDOWED_HIDDEN]; + let mut h = vec![0.0f64; WINDOWED_HIDDEN]; + let mut logits = vec![0.0f64; n_classes]; + + for epoch in 0..epochs { + for i in (1..idx.len()).rev() { + let j = (shuf_next() as usize) % (i + 1); + idx.swap(i, j); + } + let lr = base_lr * 0.5 * (1.0 + (std::f64::consts::PI * epoch as f64 / epochs as f64).cos()); + let mut epoch_loss = 0.0f64; + + let mut k = 0usize; + while k < n { + let bend = (k + batch_size).min(n); + let mut gw1 = vec![0.0f64; n_w1]; + let mut gb1 = vec![0.0f64; WINDOWED_HIDDEN]; + let mut gw2 = vec![0.0f64; n_w2]; + let mut gb2 = vec![0.0f64; n_classes]; + let bs = (bend - k) as f64; + + for &si in &idx[k..bend] { + let (x, target) = &samples[si]; + debug_assert_eq!(x.len(), WINDOWED_INPUT); + + // Forward. + for j in 0..WINDOWED_HIDDEN { + let mut s = b1[j]; + for i in 0..WINDOWED_INPUT { s += x[i] * w1[i * WINDOWED_HIDDEN + j]; } + h_pre[j] = s; + h[j] = s.max(0.0); + } + for c in 0..n_classes { + let mut s = b2[c]; + for j in 0..WINDOWED_HIDDEN { s += h[j] * w2[j * n_classes + c]; } + logits[c] = s; + } + let mx = logits.iter().cloned().fold(f64::NEG_INFINITY, f64::max); + let ex_sum: f64 = logits.iter().map(|z| (z - mx).exp()).sum(); + let mut d_logits = vec![0.0f64; n_classes]; + for c in 0..n_classes { + let p = (logits[c] - mx).exp() / ex_sum; + d_logits[c] = p - if c == *target { 1.0 } else { 0.0 }; + if c == *target { epoch_loss += -(p.max(1e-15)).ln(); } + } + + for c in 0..n_classes { + gb2[c] += d_logits[c]; + for j in 0..WINDOWED_HIDDEN { + gw2[j * n_classes + c] += h[j] * d_logits[c]; + } + } + let mut d_h = vec![0.0f64; WINDOWED_HIDDEN]; + for j in 0..WINDOWED_HIDDEN { + if h_pre[j] <= 0.0 { continue; } + let mut s = 0.0; + for c in 0..n_classes { s += w2[j * n_classes + c] * d_logits[c]; } + d_h[j] = s; + } + for j in 0..WINDOWED_HIDDEN { + gb1[j] += d_h[j]; + for i in 0..WINDOWED_INPUT { gw1[i * WINDOWED_HIDDEN + j] += x[i] * d_h[j]; } + } + } + + for q in 0..n_w1 { + let g = gw1[q] / bs + weight_decay * w1[q]; + mw1[q] = momentum * mw1[q] + g; + w1[q] -= lr * mw1[q]; + } + for q in 0..WINDOWED_HIDDEN { + let g = gb1[q] / bs; + mb1[q] = momentum * mb1[q] + g; + b1[q] -= lr * mb1[q]; + } + for q in 0..n_w2 { + let g = gw2[q] / bs + weight_decay * w2[q]; + mw2[q] = momentum * mw2[q] + g; + w2[q] -= lr * mw2[q]; + } + for q in 0..n_classes { + let g = gb2[q] / bs; + mb2[q] = momentum * mb2[q] + g; + b2[q] -= lr * mb2[q]; + } + + k = bend; + } + if epoch % 3 == 0 || epoch == epochs - 1 { + eprintln!(" W-MLP epoch {epoch:2}/{}: loss = {:.4}, lr = {:.4}", + epochs, epoch_loss / n as f64, lr); + } + } + + WindowedMlpModel { w1, b1, w2, b2, n_classes } +} + +/// Evaluate Windowed MLP accuracy + per-class correct counts. +fn eval_windowed_mlp( + mlp: &WindowedMlpModel, + samples: &[(Vec, usize)], + n_classes: usize, +) -> (f64, Vec) { + let mut correct = 0usize; + let mut per_class = vec![0usize; n_classes]; + for (x, target) in samples { + let probs = mlp.forward(x); + let pred = probs.iter().enumerate() + .max_by(|a, b| a.1.partial_cmp(b.1).unwrap()) + .unwrap().0; + if pred == *target { correct += 1; per_class[*target] += 1; } + } + (correct as f64 / samples.len() as f64, per_class) +} + /// Default path for the saved adaptive model. pub fn model_path() -> PathBuf { PathBuf::from("data/adaptive_model.json") diff --git a/v2/crates/wifi-densepose-sensing-server/src/main.rs b/v2/crates/wifi-densepose-sensing-server/src/main.rs index 50394985..35d0a212 100644 --- a/v2/crates/wifi-densepose-sensing-server/src/main.rs +++ b/v2/crates/wifi-densepose-sensing-server/src/main.rs @@ -1645,6 +1645,12 @@ struct AppStateInner { /// Each entry is the full subcarrier amplitude vector for one frame. /// Capacity: FRAME_HISTORY_CAPACITY frames. frame_history: VecDeque>, + /// ADR-120: rolling buffer of the last WINDOW_FRAMES (=20) feature + /// vectors from `features_from_runtime`. Used at classify time to + /// feed the WindowedMlp inside the adaptive model. Pushed each tick + /// before the broadcast emit. Cold start: classify_window falls back + /// to frame-level until the buffer fills. + feature_window: VecDeque<[f64; adaptive_classifier::N_FEATURES_PUB]>, tick: u64, source: String, /// Instant of the last ESP32 UDP frame received (for offline detection). @@ -2659,8 +2665,13 @@ fn current_per_node_amps() -> Vec<(u8, Vec)> { } /// If an adaptive model is loaded, override the classification with the -/// model's prediction. Uses the 22-feature multi-node vector (ADR-118) -/// for higher accuracy than the legacy 15-feature single-node vector. +/// model's prediction. ADR-120: prefers temporal-window classifier when +/// the rolling feature buffer is full (20 frames). Falls through to +/// frame-level (ADR-119 MLP) at cold start. +/// +/// Read-only over `state` — the per-tick push into `feature_window` happens +/// at the tick site where `&mut AppStateInner` is already held (see the +/// broadcast tick task in `run_*_pipeline`). fn adaptive_override(state: &AppStateInner, features: &FeatureInfo, classification: &mut ClassificationInfo) { if let Some(ref model) = state.adaptive_model { let per_node_owned = current_per_node_amps(); @@ -2678,7 +2689,30 @@ fn adaptive_override(state: &AppStateInner, features: &FeatureInfo, classificati }), &per_node_refs, ); - let (label, conf) = model.classify(&feat_arr); + + // ADR-120: if rolling window has at least the current frame + 19 prior, + // use the temporal classifier. Otherwise fall back to frame-level. + let (label, conf) = if state.feature_window.len() + 1 >= adaptive_classifier::WINDOW_FRAMES { + // Flatten the last (WINDOW_FRAMES - 1) historic vectors + current + // frame into a single 440-d row-major vector, oldest first. + let wf = adaptive_classifier::WINDOW_FRAMES; + let nf = adaptive_classifier::N_FEATURES_PUB; + let mut flat = vec![0.0f64; wf * nf]; + // History fills the first (WINDOW_FRAMES - 1) frames. + let hist_take = wf - 1; + let skip = state.feature_window.len().saturating_sub(hist_take); + for (frame_i, fv) in state.feature_window.iter().skip(skip).enumerate() { + let base = frame_i * nf; + for i in 0..nf { flat[base + i] = fv[i]; } + } + // Last slot = current frame. + let last_base = (wf - 1) * nf; + for i in 0..nf { flat[last_base + i] = feat_arr[i]; } + model.classify_window(&flat) + } else { + model.classify(&feat_arr) + }; + classification.motion_level = label.to_string(); classification.presence = label != "absent"; // Blend model confidence with existing smoothed confidence. @@ -2686,6 +2720,32 @@ fn adaptive_override(state: &AppStateInner, features: &FeatureInfo, classificati } } +/// ADR-120: push the current frame's feature vector into the rolling +/// window buffer, evicting the oldest entry when at capacity. Called +/// once per tick from the broadcast tick task where `&mut AppStateInner` +/// is already held. +fn push_feature_window(state: &mut AppStateInner, features: &FeatureInfo) { + let per_node_owned = current_per_node_amps(); + let per_node_refs: Vec<(u8, &[f64])> = per_node_owned.iter() + .map(|(n, a)| (*n, a.as_slice())).collect(); + let feat_arr = adaptive_classifier::features_from_runtime( + &serde_json::json!({ + "variance": features.variance, + "motion_band_power": features.motion_band_power, + "breathing_band_power": features.breathing_band_power, + "spectral_power": features.spectral_power, + "dominant_freq_hz": features.dominant_freq_hz, + "change_points": features.change_points, + "mean_rssi": features.mean_rssi, + }), + &per_node_refs, + ); + state.feature_window.push_back(feat_arr); + while state.feature_window.len() > adaptive_classifier::WINDOW_FRAMES { + state.feature_window.pop_front(); + } +} + /// Size of the median filter window for vital signs outlier rejection. const VITAL_MEDIAN_WINDOW: usize = 21; /// EMA alpha for vital signs (~5s time constant at 10 FPS). @@ -2966,6 +3026,9 @@ async fn windows_wifi_task(state: SharedState, tick_ms: u64) { let (features, mut classification, breathing_rate_hz, sub_variances, raw_motion) = extract_features_from_frame(&frame, &s_write_pre.frame_history, sample_rate_hz); smooth_and_classify(&mut s_write_pre, &mut classification, raw_motion); + // ADR-120: push current frame's features before classify so the + // windowed model has temporal context. + push_feature_window(&mut s_write_pre, &features); adaptive_override(&s_write_pre, &features, &mut classification); // ADR-101: raw-amplitude presence/motion override. Supersedes the // RSSI MAD-Δ classifier from ADR-099 (left in the source for @@ -3154,6 +3217,9 @@ async fn windows_wifi_fallback_tick(state: &SharedState, seq: u32) { let (features, mut classification, breathing_rate_hz, sub_variances, raw_motion) = extract_features_from_frame(&frame, &s.frame_history, sample_rate_hz); smooth_and_classify(&mut s, &mut classification, raw_motion); + // ADR-120: push the current frame's feature vector before classifying, + // so the windowed model can use up to WINDOW_FRAMES of history. + push_feature_window(&mut s, &features); adaptive_override(&s, &features, &mut classification); s.source = format!("wifi:{ssid}"); @@ -6439,6 +6505,8 @@ async fn simulated_data_task(state: SharedState, tick_ms: u64) { let (features, mut classification, breathing_rate_hz, sub_variances, raw_motion) = extract_features_from_frame(&frame, &s.frame_history, sample_rate_hz); smooth_and_classify(&mut s, &mut classification, raw_motion); + // ADR-120: push current frame features into the rolling window first. + push_feature_window(&mut s, &features); adaptive_override(&s, &features, &mut classification); s.rssi_history.push_back(features.mean_rssi); @@ -7153,6 +7221,7 @@ async fn main() { latest_update: None, rssi_history: VecDeque::new(), frame_history: VecDeque::new(), + feature_window: VecDeque::with_capacity(adaptive_classifier::WINDOW_FRAMES), tick: 0, source: source.into(), last_esp32_frame: None,