Commit Graph

6 Commits

Author SHA1 Message Date
rUv d2089c342a
fix(engine security): close witness domain-separation collision in governed-trust cycle + prove privacy monotonicity (#1074)
* fix(engine): length-prefix witness fields to close domain-separation collision

The BLAKE3 trust witness concatenated model_version, calibration_version,
and privacy_decision boundary-to-boundary, with the variable-length evidence
list lacking an explicit count. A string straddling a field boundary (e.g. a
per-room adapter id absorbing the leading bytes of the calibration epoch, or a
model_version absorbing a trailing evidence ref) collided with a different
trust decision — silently un-distinguishing two distinct privacy-relevant
inputs and defeating the ADR-137 tamper/drift audit guarantee. model_version
is operator-influenceable via the adapter id (ADR-150 §3.4), so the ambiguity
was reachable.

Fix: domain-tag the hash and length-prefix every field (8-byte LE length),
plus an explicit evidence count. Pinned by two fails-on-old tests:
witness_distinguishes_model_calibration_boundary and
witness_distinguishes_evidence_model_boundary.

Co-Authored-By: claude-flow <ruv@ruv.net>

* test(engine): pin privacy monotonicity, fail-closed boundaries; de-magic constants

Review hardening for the governed-trust cycle (no behavior change):

- forced_contradiction_never_relaxes_class: property test over all 5 privacy
  modes proving a forced contradiction only ever raises the emitted class byte
  (more restrictive) and a clean cycle emits exactly the base class — the
  ADR-141/120 information-only-removed invariant.
- empty_cycle_fails_closed: a zero-frame cycle errors (fusion NoFrames),
  emits no SemanticState, and does not advance the cycle counter.
- single_node_cycle_is_well_formed: characterizes the n=1 boundary (no mesh,
  no directional, base class, witness still emitted) — documents single-node
  sensing as a valid non-demoting mode, not a bypass.
- De-magicked the engine-construction literals (coherence accept gate, ADR-143
  SLAM discovery + static-anchor thresholds) into named documented consts,
  value-identical, pinned by engine_constants_match_prior_values.

Co-Authored-By: claude-flow <ruv@ruv.net>

* docs(engine-review): record witness domain-separation fix + monotonicity clean bill

CHANGELOG [Unreleased] Security entry and review notes appended to ADR-137
(witness domain-separation fix) and ADR-141 (privacy monotonicity confirmed
clean over all 5 modes, fail-closed boundaries pinned).

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-06-14 15:32:24 -04:00
ruv fd1430e46f test(engine): update contradiction_demotes_privacy for #1031 guard thresholds
The streaming-engine privacy-demotion test fed a 2 ms timestamp spread, which
demoted under the old 1 ms soft guard. #1031 raised the default soft guard to
20 ms (to accommodate the real TDM slot offset), so 2 ms now fuses cleanly with
no demotion. Bump the test spread to 25 ms (above the 20 ms soft guard, within
the 60 ms hard guard) so it still proves the ADR-137 -> ADR-141 demotion wiring.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-06-13 12:14:11 -04:00
rUv 29de574e63
Beyond-SOTA engine/signal/train improvements: mesh partition guard, FFT CIR solver, canonical frame decoder, falsifiable occupancy benchmark, governed streaming, adapter provenance (#1018)
* docs(research): add RuView beyond-SOTA system review (00)

First document of the beyond-SOTA research series: capability audit of
the current RuView engine with role-to-crate maturity matrix, ruvsense
module inventory, gap analysis, and risk register.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* docs(research): add beyond-SOTA architecture design (02, in progress)

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* docs(research): finalize beyond-SOTA architecture (02)

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* docs(research): add benchmark/validation methodology snapshot (03)

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* docs(research): add beyond-SOTA series index with validation results; changelog

README index ties the 5 research docs together with the session's
measured validation evidence: 2,797 workspace tests / 0 failed, Python
proof PASS (bit-exact), and paired pre/post criterion CIR benchmarks.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* perf(signal): precompute CIR warm-start system; hoist tomography solver allocs

Exact, determinism-safe optimizations (bit-identical float results):

- cir.rs: diag(PhiH Phi)+lambda*I and its CSR matrix depend only on Phi
  and lambda (fixed at CirEstimator::new) but were rebuilt every frame
  (O(K*G) pass + CSR allocation). Now built once in new() via
  build_warm_start_system; summation order unchanged.
- tomography.rs: ISTA gradient buffer hoisted out of the 100-iteration
  loop (fill(0.0) reset) and the Frobenius Lipschitz bound moved from
  per-reconstruct to construction.

Verified: signal 456 tests green; engine 11/11 green including
cycle_is_deterministic and witness-stability tests. Criterion paired
pre/post: cir_estimate/he40 -3.9% (p<0.01), multiband -1.2/-1.4%.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* fix(worldgraph): bound SemanticState growth with deterministic retention

StreamingEngine::process_cycle appended one SemanticState belief per cycle
with no eviction — ~1.7M nodes/day at 20 Hz (beyond-SOTA roadmap finding #6).

Add WorldGraph::prune_semantic_states(max): deterministic eviction of the
oldest beliefs by (valid_from_unix_ms, id); structural nodes (rooms, zones,
sensors, anchors, tracks, events) are never eligible. Wire it into the
engine after each belief append (DEFAULT_SEMANTIC_RETENTION = 7,200, ~6 min
at 20 Hz; set_semantic_retention to tune). The WorldGraph holds current
beliefs; durable history is the recorder's job, so no audit data is lost.

3 new tests: end-to-end bounded growth, oldest-only eviction, deterministic
equal-timestamp tie-break. Workspace gate: 2,865 passed, 0 failed.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* feat(sensing-server): route live frames through the governed StreamingEngine

Closes the live-trust-path gap (ADR-136 section 8, beyond-SOTA system review):
the running server fused live CSI with the bare MultistaticFuser, while the
privacy/provenance/witness control plane (ADR-135..146) only ever ran on
synthetic in-test frames. The privacy control plane was therefore bypassable
on the real path.

New engine_bridge module drives StreamingEngine::process_cycle from the
server's live NodeState map, reusing the existing NodeState -> MultiBandCsiFrame
conversion. It lazily wires each contributing node as a WorldGraph sensor
(idempotent), bounds belief growth via the retention cap, and forwards explicit
timestamps/calibration ids so the path stays deterministic and replayable.

Wired additively into both live ESP32/WiFi fusion sites in main.rs via a
split-borrow off the write guard, so person-count behavior is unchanged; the
latest BLAKE3 witness is stored on AppState. Every published belief now carries
evidence + model + calibration + privacy decision and a deterministic witness.

Adds wifi-densepose-engine/-worldgraph/-bfld/-geo deps. 6 new bridge tests
(witnessed belief with full provenance, cross-run determinism, idempotent node
registration, retention bound, privacy-mode propagation). sensing-server suite
430+128 green; workspace gate 2,904 passed / 0 failed.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* feat(train): falsifiable occupancy benchmark with anti-overfitting gate

Makes the presence/person-count "beyond SOTA" claim falsifiable in code
instead of aspirational (the unfalsifiability gap from the beyond-SOTA system
review). occupancy_bench grades predictions vs ground truth and gates a SOTA
claim behind one claim_allowed invariant requiring ALL of:

- DataProvenance::Measured — synthetic/mock data is scorable for regression
  but never claimable (anti-mock-contamination; the CLAUDE.md Kconfig-bug
  lesson made structural).
- A leak-free EvalSplit — validate() refuses any split where a subject OR
  environment id appears in both train and test (subject leakage /
  per-environment overfitting).
- n_test >= min_test_samples (small-N guard).
- Presence F1 whose bootstrap-CI lower bound (deterministic seeded splitmix64)
  clears the threshold — not the point estimate.
- Count MAE within threshold.

The claim string is unreadable except through the gate (NO_CLAIM otherwise),
same discipline as the ruview-gamma acceptance gate. What remains is data, not
method: a frozen, SHA-pinned, subject/environment-disjoint measured replay set
turns the claim into a passing/failing test.

Lives in wifi-densepose-train (the eval bounded context, alongside ablation/
eval/metrics). 10 tests cover each refusal path; warning-clean under the
crate's missing_docs lint. Workspace gate 2,914 passed / 0 failed. Doc 03
updated.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* feat(engine): per-room adapter provenance + drift-to-recalibration advisor

Closes the trust-chain gap where an ~11 KB per-room LoRA adapter (ADR-150
section 3.4) could silently change inference without the witness noticing:
provenance carried only "rfenc-v<N>" with no notion of adapter identity.

- StreamingEngine::set_room_adapter(AdapterInfo): pins the adapter's
  content-derived id into provenance model_version
  ("rfenc-v1+adapter:<id>") — and therefore into the BLAKE3 witness — so
  swapping or clearing adapter weights always shifts the witness. Engine test
  proves base -> adapter -> other-adapter -> cleared all witness differently
  and cleared == base.
- RecalibrationAdvisor: recommends re-running the ADR-135 empty-room baseline
  / refitting the room adapter on sustained low fusion coherence (streak
  threshold, default 60 cycles ~ 3 s at 20 Hz) or an ADR-142 change-point.
  Surfaced as TrustedOutput::recalibration_recommended, stored on the
  sensing-server AppState alongside the witness at both live fusion sites.
- Bridge plumbing: EngineBridge::{set_room_adapter, clear_room_adapter} +
  live-path test that the adapter id flows into the live witness.

Scope note (honest): this is the deployable provenance/trigger half of the
"retrained model" roadmap item. Fitting the adapter itself runs in the
existing external calibration service (aether-arena/calibration/); a trained
RF-encoder checkpoint still does not exist in-tree.

Engine 15 tests, bridge 7 tests. Workspace gate: 2,918 passed / 0 failed.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* fix(mat): gate api module behind its feature — standalone no-default-features builds

pub mod api was unconditional while its only dependency, serde, is optional
behind the 'api' feature, so any build without default features failed with
101 unresolved-serde errors (masked in --workspace runs by feature
unification). The api module and its create_router/AppState re-export are now
cfg(feature = "api")-gated with docsrs annotations.

All combos compile: bare --no-default-features (was 101 errors, now 0),
--no-default-features --features api, and full default (177 tests pass).
Workspace gate: 2,918 passed / 0 failed.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* perf(signal): opt-in FFT operator for the CIR ISTA solver (8-14x measured)

Phi is a sub-DFT, so each ISTA mat-vec can run as one length-G FFT
(O(G log G)) instead of a dense O(K*G) product — the dominant-latency-hazard
finding from the beyond-SOTA optimization roadmap.

New CirConfig::fft_operator, default FALSE: the dense path stays the
bit-exact witness default. The FFT evaluates the same sums in a different
order, so enabling it shifts float results in the last bits and requires
regenerating any pinned witness — strictly opt-in per deployment.

FftOperator (rustfft, planned once at CirEstimator::new, scratch buffers
reused across the ISTA loop) dispatches inside ista_solve:
  Phi x   = scale * forward-FFT(x) sampled at bins (k_idx mod G)
  Phi^H v = scale * unnormalised inverse-FFT of v scattered into those bins
Warm-start and Lipschitz estimation stay dense at construction.

Measured (criterion, same run, same machine):
  ht20: 2.22 ms -> 265 us  (8.4x)
  ht40: 10.26 ms -> 717 us (14.3x)
The real HE40 grid (K=484, G=1452) scales further per the O(K*G)/O(G log G)
ratio.

3 new tests: FFT<->dense matvec equivalence to float tolerance on ht20 and
he40 grids; end-to-end dominant-tap agreement on a single-path frame; all
default configs keep FFT off. New cir_estimate_fft bench group.

Workspace gate: 2,921 passed / 0 failed (default path bit-exact, witnesses
unchanged).

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* feat(core): canonical frame decoder — capture-to-claim replay (ADR-136)

The encode half of the ADR-136 frame contract existed (ComplexSample,
to_canonical_bytes, witness_hash) but there was no decoder: a captured
canonical frame could be witnessed but never reconstructed, blocking
replay-from-capture.

CsiFrame::from_canonical_bytes is the exact inverse: same id, metadata,
complex payload, and witness hash (tested as the round-trip law AC7 — the
replayed frame re-encodes byte-identically). Amplitude/phase are recomputed
from the payload (projections, not independent state). Every malformed-input
class fails closed (AC8): header truncation -> Truncated, payload truncation
-> PayloadMismatch, unknown discriminants, non-UTF-8 device id, trailing
bytes. Nil calibration uuid decodes as None per the documented encoding.

Core: 36 tests pass. Workspace gate: 2,937 passed / 0 failed.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* feat(engine): dynamic min-cut mesh partition guard (ruvector-mincut)

Maintains an exact min-cut over the live mesh coupling graph — nodes are
sensing nodes, coupling is the product of fusion attention weights — and
surfaces per cycle, as TrustedOutput::mesh:

- cut value: the global "how close is the array to partitioning" number,
  a structural measure per-node heuristics miss;
- weak side: which specific nodes would split off (failure/jamming triage,
  feeds ADR-032 posture);
- at-risk flag: counts as a structural event for the drift->recalibration
  advisor (alongside ADR-142 change-points).

Degenerate cases fail toward risk: a node with zero coupling is reported as
already partitioned (cut 0, that node as the weak side).

Measured cost policy (criterion, 12-node mesh — the honest part):
- weights quantized (1/64) + change-gated: steady-state cycles do ZERO graph
  work and reuse the cached cut (~7.3 us, ~23x cheaper than building);
- on any real change a full exact rebuild (~171 us) is used, because ONE
  DynamicMinCut delete+insert measured ~240 us — the subpolynomial machinery
  amortizes on much larger graphs, so rebuild-on-change is the measured
  optimum at mesh scale (one-edge case -28% after switching policy);
- full process_cycle with the guard: ~33 us for 4 nodes vs the 50 ms budget.

9 mesh_guard tests (weak-node detection, steady-state zero updates,
sub-quantum gating, join/drop rebuild, determinism, disconnection) + an
engine-level wiring test (down-weighted node -> weak side -> recalibration).
Engine 24 tests; workspace gate 2,946 passed / 0 failed.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* feat(engine): mesh partition risk demotes privacy + enters the witness (ADR-032)

Completes the mesh-guard integration: its at_risk signal was advisory-only
(fed the recalibration advisor). It now also contributes to the ADR-141
privacy demotion alongside fusion- and array-level contradictions — a mesh
close to partitioning makes the fused belief less trustworthy, so the cycle
emits at a more restricted class (monotonic; information only removed).

Because effective_class feeds the BLAKE3 witness, a fragmenting array now
shifts the witness: partition risk is auditable, not just logged. The mesh
computation moved ahead of the demotion step in process_cycle; mesh_guard_mut
exposes risk-threshold tuning.

Test: a forced-risk 3-node cycle demotes PrivateHome Anonymous->Restricted
and shifts the witness vs a clean baseline. Engine 25 tests; workspace gate
2,947 passed / 0 failed.

https://claude.ai/code/session_01MjBucx95K4BuUxZi8NWwRH

* fix: public-PR review findings — privacy-path honesty, gate holes, mesh-guard cliff

- sensing-server: engine errors logged+counted (no silent swallow), trust
  state exposed via status surface, privacy-demotion claims aligned with
  the actual parallel-audit-path behavior
- occupancy_bench: vacuous-F1 hole closed (degenerate test sets fail with
  their own criterion); CI-lower-bound test made probative
- mesh_guard: quantization scaled to observed coupling range — >=65-node
  balanced meshes no longer permanently at_risk (regression test)
- engine: both wiring tests made probative (same-topology witness compare,
  deterministic risk-crossing fixture)
- mat: axum/tokio optional behind api; real serde feature (api enables it)
- core: canonical decoder strict (non-zero reserved bytes and nil UUID
  rejected — injective on accepted domain, forged-bytes tests)
- CHANGELOG: un-spliced the FFT/adapter bullet mangle

Co-Authored-By: claude-flow <ruv@ruv.net>

* chore: strip private-track references for public PR

Reword the occupancy-benchmark changelog bullet to drop a cross-reference
to the private research track, and restore the WorldGraph retention bullet
header that was glued onto the preceding MAT bullet.

Co-Authored-By: claude-flow <ruv@ruv.net>

* chore: lockfile refresh for cherry-picked feature set

Co-Authored-By: claude-flow <ruv@ruv.net>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-11 16:08:54 -04:00
ruv 5878868060 feat(signal,engine): ADR-137 calibration-mismatch contradiction + trust witness
- signal: MultistaticFuser::fuse_scored_calibrated() threads per-node
  CalibrationId; agreeing epochs → calibration_id set + CalibrationApplied
  evidence; disagreeing → calibration_id None + CalibrationIdMismatch flag
  (forces demotion). +2 tests.
- engine: process_cycle_calibrated() per-node calibration path; process_cycle
  delegates with a uniform epoch. TrustedOutput gains a deterministic BLAKE3
  witness over (provenance || class). calibration_version='cal:none' on mismatch.
- ADR-137 acceptance test: two frames + mismatched calibration -> QualityScore
  contradiction -> Restricted -> calibration_id None -> witness stable. +happy path.
- 11 engine tests, signal 411+ lib tests; workspace 0 errors.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-29 08:35:40 -04:00
ruv 2517a16d88 feat(engine): compose ADR-138/142/143 + ADR-139 live loop
- ADR-138: process_cycle runs ArrayCoordinator when node geometry is registered;
  array contradictions (CoherenceDrop/GeometryInsufficient) fold into the
  privacy demotion; DirectionalEvidence surfaced in TrustedOutput
- ADR-142: per-node mean-amplitude → EvolutionTracker; cross-link change-point
  recorded as a WorldGraph Event node
- ADR-143: ingest_reflectors() runs Rf-SLAM discovery, writes stable
  Wall/Furniture reflectors as ObjectAnchor nodes
- ADR-139 live loop: update_person_track(), apply_active_privacy_mode()
  (PrivacyRollup suppresses person_track under identity-strict modes),
  snapshot_json()
- Acceptance test live_frame_to_reload_same_contents: full path
  fusion->worldgraph->privacy_rollup->persist->reload->same contents, no raw RF
- 9 engine tests; workspace 0 errors

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-29 08:31:05 -04:00
ruv 2eada40e3b feat(engine): integrate ADR-135..141 into an end-to-end trust pipeline
- signal/calibration.rs: BaselineCalibration gains calibration_id()/
  calibration_uuid()/apply() — the ADR-135->136 link that stamps
  FrameMeta.calibration_id (deterministic id, no serialization change). +1 test.
- NEW crate wifi-densepose-engine: StreamingEngine::process_cycle() composes
  fuse_scored (137) -> calibration provenance (135/136) -> privacy demotion on
  contradiction (141) -> WorldGraph SemanticState with mandatory provenance +
  DerivedFrom edge (139). Returns TrustedOutput (the trust chain made concrete).
- Validates the throughline: every output names evidence + model + calibration
  + privacy decision; calibration_id flows input->QualityScore->provenance;
  contradiction demotes class; deterministic; privacy mode attested.
- 4 integration tests; workspace 0 errors; signal 410 lib tests pass.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-05-29 08:21:48 -04:00