Iter 26 — closes the ABI gap between the Python and Rust SyncPacket
decoders. Before this, Python could decode the wire but had no helpers
to apply offsets or recover per-frame mesh time; any Python-side tooling
(host scripts, replay analysers, notebooks) would have to re-implement
the math from scratch and could drift from Rust silently.
New methods on the Python SyncPacket dataclass:
local_minus_epoch_us() -> int
Signed local-vs-mesh offset. Matches Rust byte-for-byte.
apply_to_local(local_at_frame_us: int) -> int
offset = epoch_us - local_us
return local_at_frame_us + offset
Identity at local_at_frame_us == self.local_us returns epoch_us.
mesh_aligned_us_for_sequence(frame_seq: int, fps_hz: float) -> int
Sequence-based interpolation matching Rust's identical method.
Includes u32 wraparound handling via masked-subtract — verified
against Rust's iter 17 `mesh_aligned_for_sequence_handles_seq_wraparound`.
3 new Python tests (10 total in TestSyncPacketParser, all green in 0.24s):
test_apply_to_local_recovers_epoch_at_sync_point
Identity at the sync point. Also verifies local_minus_epoch_us()
matches §A0.10's measured 1,163,565 µs bench number.
test_apply_to_local_preserves_inter_frame_delta
Frame arriving 5 s after the sync on the follower's local clock
produces mesh time exactly 5 s after sync.epoch_us.
test_mesh_aligned_us_for_sequence_matches_rust
Cross-language parity with Rust's
`end_to_end_sync_decode_then_frame_mesh_recovery` (iter 20):
100 frames after sync.sequence at 20 fps = sync.epoch_us + 5 s.
Cross-checks via apply_to_local — both paths must agree.
Test count after iter 26:
Python TestSyncPacketParser: 10/10 (was 7/7)
Rust sync_packet::tests: 15/15
Combined: 25 unit tests defending the SyncPacket contract across
the two host language stacks.
Co-Authored-By: claude-flow <ruv@ruv.net>
Iter 21 — ultra-opt for protocol correctness across the two production
decoders. Pin the same 32-byte canonical hex in both Python and Rust
tests; if either decoder drifts from the wire, ONE of the tests starts
failing — and it's clear which side moved.
Canonical packet: COM9 sync-pkt #1 from §A0.12 live capture, expressed
as exact little-endian bytes:
10a111c5 09 01 06 00 magic + node + ver + flags + rsvd
f26db70100000000 local_us = 28_798_450
c5aca50100000000 epoch_us = 27_634_885
1400000000000000 sequence = 20 + reserved
Python test:
archive/v1/tests/unit/test_esp32_binary_parser.py::TestSyncPacketParser
::test_canonical_wire_bytes_match_rust_decoder
— decodes the pinned hex, asserts every field including the §A0.10
1,163,565 µs offset.
Rust test:
v2/crates/wifi-densepose-hardware/src/sync_packet.rs::tests
::canonical_wire_bytes_match_python_decoder
— decodes the same bytes, asserts the same fields, then re-encodes
via to_bytes() and asserts the round-trip produces the EXACT same
32 bytes. So this also catches drift in the Rust encoder.
Test counts after this iter:
Rust sync_packet: 15/15 green (was 14)
Python SyncPacketParser: 7/7 green (was 6)
Branch contract: if a future PR changes the firmware wire format, BOTH
tests must be updated atomically with the new canonical hex. CI will
gate this naturally.
Co-Authored-By: claude-flow <ruv@ruv.net>
Python ESP32BinaryParser was using struct format '<IBBHIIBB2x' — the
'2x' skipped bytes 18-19 as reserved. After the Rust-side decoder was
extended to surface PPDU type + flags, the Python pipeline (which
archive/v1 still uses for testing + the proof verifier) needs the same
update so its consumers see the HE metadata too.
csi_extractor.py:
- HEADER_FMT now '<IBBHIIBBBB' (captures bytes 18-19)
- New metadata fields: ppdu_type ('ht_legacy'|'he_su'|'he_mu'|'he_tb'|'unknown'),
ppdu_type_raw, he_capable, bw40, stbc, ldpc, ieee802154_sync_valid,
adr018_flags_raw
- Class constants PPDU_HT_LEGACY..PPDU_UNKNOWN mirror the firmware
test_esp32_binary_parser.py:
- build_binary_frame() takes optional ppdu_byte + flags_byte (default 0)
- New TestAdr110ByteEncoding class with 5 tests:
- Pre-ADR-110 zeros decode as 'ht_legacy' + all-flags-false
- HE-SU / HE-MU / HE-TB decode correctly
- 0xFF decodes as 'unknown'
- All-flags-set round-trip (0x1D)
11/11 parser tests pass (6 existing + 5 new). Backwards compat verified.
Pairs with the Rust-side decoder in commit 3959fabf3. Both pipelines now
read the same wire format produced by the C6 firmware's
CONFIG_CSI_FRAME_HE_TAGGING path.
Ref: ruvnet/RuView#762, draft PR #764
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(verify): quantize features before SHA-256 for cross-platform hash stability (#560)
## The bug
archive/v1/data/proof/verify.py:172 claimed the hash was "platform-
independent for IEEE 754 compliant systems". That claim is empirically
false. scipy.fft's pocketfft uses SIMD vector kernels — AVX2/AVX-512 on
x86_64, NEON on Apple Silicon — that reorder vectorized FP operations
differently per build. IEEE 754 guarantees per-operation determinism,
not associativity under reordering, so two correct platforms produce
values that differ at ULP precision (~1e-14 at our magnitudes of 1-100).
The SHA-256 of features_to_bytes() then explodes that ULP-level
divergence into a totally different hash, which is what bug report #560
caught on macOS arm64:
| Platform | numpy/scipy | sha256 (legacy) |
|----------|-------------|-----------------|
| Windows (Intel AVX-512) | 2.4.2 / 1.17.1 | 78b3fb… |
| ruvultra (Linux x86_64) | 1.26.4 / 1.14.1 | 41dc56… |
| ruv-mac-mini (Apple Silicon NEON) | 2.4.4 / 1.17.1 | 9b5e19… |
## The fix
features_to_bytes() now np.round(.., HASH_QUANTIZATION_DECIMALS=9)s each
array before packing as little-endian f64. That snaps the float bytes
to a single canonical representation across SIMD backends.
The 9-decimal precision is:
- ~5 orders of magnitude above the worst-case ULP drift observed in
probe-fft-platform.py measurements
- Many orders of magnitude below any meaningful signal change (CSI
phase precision is ~1e-3 rad; PSD bins differ by orders of magnitude)
- Conservative — could tighten to 11-12 decimals if needed, but 9
leaves comfortable headroom for future scipy SIMD changes
## Probe-side verification
scripts/probe-fft-platform.py now emits BOTH sha256_raw (unrounded,
legacy) and sha256_quantized (new platform-invariant hash). Running it
on Windows here produced:
sha256_raw = 78b3fb4acb8cc18c3e870f92e29ee98143c7cac4767f2f71b0fc384a82b92f6e
sha256_quantized = a587792c050cf697366b9bef4611050f9dc3af56624915ab2452c3c11362e79a
quantization_decimals = 9
On Linux and macOS arm64 the maintainer should observe the SAME
sha256_quantized value (and a different sha256_raw) — that's the
fix working.
## What this PR does NOT do
The published archive/v1/data/proof/expected_features.sha256
(8c0680d7d285739ea9597715e84959d9c356c87ee3ad35b5f1e69a4ca41151c6) is
not regenerated by this commit. That step needs to run on a canonical
CI platform (likely the Linux x86_64 host used for releases) AFTER this
fix lands. The regeneration command is:
python archive/v1/data/proof/verify.py --generate-hash
After regeneration, every platform running ./verify will produce the
same hash and the proof replay will be honestly cross-platform — which
is what the ADR-028 trust-kill-switch promised.
## Files
- archive/v1/data/proof/verify.py — add HASH_QUANTIZATION_DECIMALS=9
constant, quantize in features_to_bytes(), correct the misleading
"platform-independent" claim in the docstring
- scripts/probe-fft-platform.py — emit both raw and quantized hashes
- scripts/fix-markers.json — RuView#560 marker prevents removing the
np.round() call without explicit intent
- CHANGELOG.md — Fixed entry under [Unreleased] documenting the change
and flagging the expected_features.sha256 regeneration as a follow-up
Co-Authored-By: claude-flow <ruv@ruv.net>
* ci: fix verify-pipeline.yml working-directory from v1/ to archive/v1/
The verify-pipeline workflow's "Run pipeline verification" and "Run
verification twice to confirm determinism" steps use
`working-directory: v1` but `v1/` was archived to `archive/v1/` long
ago. The workflow fails before verify.py even runs:
##[error]An error occurred trying to start process '/usr/bin/bash'
with working directory '/home/runner/work/RuView/RuView/v1'.
No such file or directory
Same v1 → archive/v1 path correction that already shipped for the
./verify wrapper (RuView#559 / PR #590) and the other lint workflows
(RuView#489).
Required to make the determinism check actually run on PR #609 (the
quantize-before-hash work) — the canonical Linux hash needed for
expected_features.sha256 will fall out of the next CI log once this
fix lands.
* fix(proof): regenerate expected_features.sha256 with the quantized canonical hash
The hash on the previous line was the legacy pre-quantization value
(8c0680d7d28573…), which by definition cannot match the quantized
output that this branch's verify.py now produces. Replaced with the
canonical Linux x86_64 hash captured from the CI run on this branch:
d9985569b3ab833c74b7c9254df568bbb144879e2222edb0bcf2605bfd4c155b
Source of truth: run 26005976495 / "Verify Pipeline Determinism (3.11)"
on Ubuntu 24.04, Python 3.11.15, exercising the full verify.py pipeline
on the 100 reference frames in archive/v1/data/proof/sample_csi_data.json.
Reproducibility expectation now changes:
- Linux x86_64 (canonical platform): sha256 = d9985569… ✓ this commit
- macOS arm64 / Apple Silicon NEON: sha256 = d9985569… should match
after quantization
- Windows AMD64 (with pydantic-clean .env): sha256 = d9985569… should match
after quantization
If macOS arm64 still mismatches after this, the quantization decimals
need to be tightened from 9 to 11 or 12 (HASH_QUANTIZATION_DECIMALS
in verify.py); the headroom analysis in the original commit suggests
9 is safe but 9-decimal SIMD drift hasn't been measured in the
full-pipeline output yet (only in the probe).
Closes the maintainer-action-required item on PR #609.
* fix(proof): bump quantization to 6 decimals (9 wasn't enough across Azure CI microarchs)
Two back-to-back Ubuntu 24.04 / Python 3.11 / scipy 1.17 CI runs on
PR #609 landed on different Azure VM microarchitectures and produced
two different SHA-256s even after np.round(.., 9):
Run 1: d9985569b3ab833c74b7c9254df568bbb144879e2222edb0bcf2605bfd4c155b
Run 2: 37c49a1f6b87207fa9fc67f2d6a85c4417dd4a536573605fd175510d1dce7cbe
Same JSON input, same byte count hashed (294,400), same Python version,
same scipy version. The only variable is the underlying CPU pocketfft
SIMD kernel.
The full DSP pipeline (preprocess → biquad bandpass → FFT → PSD →
variance accumulation) amplifies the ~1e-14 raw FFT divergence by
several orders of magnitude — the actual drift at features_to_bytes()
input can reach 1e-7 or worse, which is well within the 1e-9 quantization
window I originally picked.
Bumping to 6 decimals = parts per million. ~6 orders of magnitude
headroom over observed pipeline-amplified ULP drift. Still far below
any meaningful signal change (CSI phase precision ~1e-3 rad). Kept the
probe constant in sync.
Will trigger CI on this branch immediately after push; the new
expected_features.sha256 will be regenerated from whichever microarch
the next CI run lands on, but should be stable across all subsequent
runs at 6-decimal quantization.
* chore(probe): keep HASH_QUANTIZATION_DECIMALS in sync with verify.py (now 6)
* fix(proof): regenerate expected_features.sha256 for 6-decimal quantization
* ci: pin thread count to 1 for proof verification (scipy.fft threading non-determinism)
Reported by @bannned-bit. archive/v1/src/services/pose_service.py:223:
sanitized_phase = self.phase_sanitizer.sanitize(phase_data)
PhaseSanitizer exposes the full-pipeline entry point as `sanitize_phase`
(unwrap_phase + remove_outliers + smooth_phase), not `sanitize`. The
shorter name doesn't exist on the class, so any path that reaches this
branch raises AttributeError mid-frame and crashes the pose service.
archive/v1/src/core/phase_sanitizer.py:266 is the canonical name:
def sanitize_phase(self, phase_data: np.ndarray) -> np.ndarray:
"""Sanitize phase data through complete pipeline."""
One-line rename. No other call sites use the wrong name; verified with
grep -rn 'phase_sanitizer\.sanitize\b' archive/v1/src/.
This is v1 archived code, but the proof verify path still exercises it
(./verify reaches into archive/v1/src/), so the bug was a latent
regression risk for the trust-kill-switch flow.
The Rust port at v2/ has been the primary codebase since the rename
in #427. The Python implementation at v1/ is no longer the active
target; the only load-bearing path is the deterministic proof bundle
at v1/data/proof/ (per ADR-011 / ADR-028 witness verification).
Move the whole Python tree into archive/v1/ and document the policy
in archive/README.md: no new features, bug fixes only when they affect
a still-load-bearing path (currently just the proof), CI continues to
verify the proof on every push and PR.
Path references updated in 26 files via path-pattern sed (only
matches v1/<known-child> patterns, never bare v1 or API URLs like
/api/v1/). Two double-prefix typos (archive/archive/v1/) caught and
hand-fixed in verify-pipeline.yml and ADR-011.
Validated:
- Python proof verify.py imports cleanly at archive/v1/data/proof/
(numpy/scipy still required; CI installs requirements-lock.txt
from archive/v1/ now)
- cargo test --workspace --no-default-features → 1,539 passed,
0 failed, 8 ignored (unaffected by Python tree relocation)
- ESP32-S3 on COM7 untouched (no firmware paths changed)
After-merge: contributors should re-run any local `python v1/...`
commands as `python archive/v1/...` (CLAUDE.md and CHANGELOG already
updated).
ruv
DavidKrame