Production-ready implementation of InvisPose - a revolutionary WiFi-based dense human pose estimation system that enables real-time full-body tracking through walls using commodity mesh routers
0bffe27288
* docs(adr-117): seed branch — ADR-117 pip-modernization spec + soul-signature research bundle
Two artifacts landing together on this new branch as the prerequisite
documentation for the v2.0.0 Python wheel modernization work:
1. **docs/adr/ADR-117-pip-wifi-densepose-modernization.md** (644 lines)
— Plan to bring the 2025-published `wifi-densepose` PyPI package
(last release v1.1.0, 2025-06-07, 11.5 months out of sync) up to
the current Rust v2/ workspace SOTA. Recommends PyO3 + maturin
with abi3-py310 (one binary covers Python 3.10–3.13 per OS/arch),
first-wheel scope = core + vitals + signal crates (~5 MB), v1.99.0
tombstone + 90-day un-yank window for v1.1.0, v2.0.0 hard break.
Open questions catalogued; phases P1–P6+ laid out with concrete
acceptance criteria.
2. **docs/research/soul/** (5 files, ~1,450 lines) — Soul Signature
research spec: 7-channel electromagnetic biometric fingerprint
(AETHER 128-dim + cardiac HR/HRV + cardiac waveform morphology +
respiratory pattern + gait timing + skeletal proportions +
subcarrier reflection profile), fused into one RVF graph file.
Includes 60s scanning protocol, 5-layer security model,
threat-model + mitigations, references to existing ADRs (014,
021, 024, 027, 030, 039, 079, 106, 108, 109, 110, 115). Marked
"Research Specification (Pre-Implementation)". Explicit "what
this is NOT" disclaimers preempt pseudoscience drift; every
discriminative-power claim either cites a measurement or is
marked "open research; baseline TBD".
Branch off main at HEAD; ready for /loop 10m implementation
iterations.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117/p1): scaffold python/ workspace — PyO3 + maturin + smoke tests (refs #785)
ADR-117 P1 — the python/ directory is now a working maturin-buildable
crate that produces the v2.x replacement for the legacy pure-Python
wifi-densepose==1.1.0 PyPI wheel.
## What lands
- `python/Cargo.toml` — PyO3 0.22 with `extension-module` + `abi3-py310`
(one binary covers Python 3.10–3.13 per OS/arch — keeps the
cibuildwheel matrix to 5 wheels per release, not 20). Depends on
`wifi-densepose-core` from the existing v2/ workspace via relative
path.
- `python/pyproject.toml` — maturin>=1.7 build backend with
`python-source = "python"` and `module-name = "wifi_densepose._native"`
so the compiled module loads as an internal underscore-private
submodule of the user-facing `wifi_densepose` package. PEP 621
metadata + classifiers + project URLs. Optional-deps:
`wifi-densepose[client]` for the P4 WS/MQTT pure-Python layer,
`wifi-densepose[dev]` for the test toolchain (pytest, ruff, mypy).
- `python/src/lib.rs` — minimal `#[pymodule] wifi_densepose_native`
exporting `__rust_version__`, `__rust_build_tag__`,
`__build_features__`, and a `hello()` smoke function. P2 will land
the core type bindings here.
- `python/wifi_densepose/__init__.py` — pure-Python facade re-exporting
the compiled module's symbols under their stable user-facing names.
Docstring teaches the v1→v2 migration story up-front.
- `python/wifi_densepose/py.typed` — PEP 561 marker so `mypy --strict`
in user code treats the wheel as fully typed (real stubs land in P2).
- `python/tests/test_smoke.py` — 6 P1 acceptance tests:
1. package imports without error
2. version string is PEP 440-compliant
3. `__rust_version__` is reachable from Python (the diagnostic
surface ADR-117 §5.2 promised)
4. `__build_features__` lists `p1-scaffold` marker
5. `wifi_densepose.hello()` returns "ok" (FFI round-trip)
6. `wifi_densepose._native` is reachable but the leading underscore
conveys "private; users should import the parent package"
- `python/README.md` — phase ledger, local build instructions
(`maturin develop`), layout diagram.
## What's deferred to P2+
- Core type bindings (`CsiFrame`, `Keypoint`, `PoseEstimate`) — P2
- Vitals + signal DSP bindings + witness v2 — P3
- Pure-Python WS/MQTT client layer (`wifi_densepose[client]`) — P4
- cibuildwheel + PyPI publish — P5
- v1.99.0 tombstone — concurrent with P5
The new `python/` crate is intentionally OUTSIDE the v2/ Cargo
workspace — it has its own Cargo.toml with `[package]` not
`[workspace.package]` inheritance — to keep maturin's `python-source`
+ `module-name` config self-contained and to avoid forcing every
`cargo test --workspace` invocation in v2/ to compile pyo3.
Refs ADR-117 §5 (Detailed design) and §6 (Phased migration).
Refs #785 (tracking issue).
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(adr-117/p1): standalone Cargo.toml + python-source=. + #[pyo3(name=_native)] (P1 GREEN)
Three fixes to make maturin develop actually work locally:
1. `python/Cargo.toml` removed `*.workspace = true` inheritance —
the python/ crate is intentionally outside the v2/ workspace
(ADR-117 §5.2) so it needs every `[package]` field local.
2. `python/pyproject.toml` `python-source = "python"` was wrong
because pyproject.toml lives at python/ — maturin was looking for
python/python/. Changed to `python-source = "."` so the
`wifi_densepose/` package directory sibling-to-pyproject is found.
3. `python/src/lib.rs` `#[pymodule] fn wifi_densepose_native` →
`#[pymodule] #[pyo3(name = "_native")] fn wifi_densepose_native`.
PyO3 generates `PyInit__native` from the pyo3-name attribute, which
must match the `module-name` in pyproject.toml's [tool.maturin]
block ("wifi_densepose._native"). Without this attribute the wheel
builds but `import wifi_densepose._native` fails with
ModuleNotFoundError.
## Local validation (P1 acceptance gate)
```
$ python -m venv .venv && .venv/Scripts/python -m pip install maturin pytest
$ VIRTUAL_ENV=… maturin develop --release
…
Finished `release` profile [optimized] target(s)
📦 Built wheel for abi3 Python ≥ 3.10
🛠 Installed wifi-densepose-2.0.0a1
$ .venv/Scripts/python -c 'import wifi_densepose; print(wifi_densepose.__version__, wifi_densepose.__rust_version__, wifi_densepose.hello())'
2.0.0a1 2.0.0-alpha.1 ok
$ .venv/Scripts/python -m pytest tests/ -v
tests/test_smoke.py::test_package_imports PASSED
tests/test_smoke.py::test_version_string_well_formed PASSED
tests/test_smoke.py::test_rust_version_surfaced PASSED
tests/test_smoke.py::test_build_features_listed PASSED
tests/test_smoke.py::test_hello_returns_ok PASSED
tests/test_smoke.py::test_native_module_private PASSED
======================== 6 passed in 0.05s =========================
```
P1 closed. Moving to P2 (core type bindings).
Refs #785, ADR-117 §6.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117/p2): Keypoint + KeypointType bindings — 23 new tests (29/29 GREEN)
Lands the first chunk of P2: PyO3 bindings for `Keypoint` and
`KeypointType` from `wifi_densepose_core`. Bound types surface to
Python as `wifi_densepose.Keypoint` / `wifi_densepose.KeypointType`.
## Design choices that affect the API surface
1. **`Confidence` is NOT bound as a separate class.** Users hate
wrapping a float in a constructor. Python-side, confidence is just
a `float in [0.0, 1.0]`; the binding validates on construction
(`ValueError` for out-of-range, matching the Rust core error).
2. **`KeypointType` is a `#[pyclass(eq, eq_int, hash, frozen)]` enum**
— hashable so users can drop it into dicts/sets (the most common
pattern in pose-analysis notebooks: `keypoints_by_type[k.type] = k`).
3. **`Keypoint.__init__` keyword-only `z`** so 2D users don't have to
write `None` and 3D users get a clear named arg:
`Keypoint(KeypointType.LeftWrist, 0.2, 0.4, 0.8, z=0.1)`.
4. **`Keypoint` is `#[pyclass(frozen)]`** — no in-place mutation. The
Rust core type is immutable through Copy + Hash + Eq, and exposing
setters from Python would create a copy-vs-reference inconsistency
between languages.
## Files
- `python/src/bindings/keypoint.rs` — 220 lines of `#[pymethods]`
wrappers + Rust↔Python enum round-trip
- `python/src/lib.rs` — `mod bindings { pub mod keypoint; }` +
`bindings::keypoint::register(m)?` call from `#[pymodule]`
- `python/wifi_densepose/__init__.py` — re-exports `Keypoint` and
`KeypointType` at the package root
- `python/tests/test_keypoint.py` — 23 tests covering:
- 17-element COCO ordering of `KeypointType.all()`
- index→type mapping for every variant
- snake_name matches COCO spec
- `is_face()` / `is_upper_body()` predicates
- hashability (the bug I caught when I added the set-based face
test — fixed by adding `hash` to the `#[pyclass]` attribute)
- 2D + 3D constructor variants
- position_2d / position_3d tuples
- is_visible threshold
- confidence validation (Err on out-of-range)
- distance_to (2D Euclidean, 3D Euclidean, fallback when one is 2D
and the other is 3D)
- __repr__ + __eq__
- the new `p2-keypoint-bindings` feature marker landed
## Local validation
\`\`\`
$ cd python && .venv/Scripts/python -m pytest tests/ -v
tests/test_smoke.py::test_package_imports PASSED
tests/test_smoke.py::test_version_string_well_formed PASSED
tests/test_smoke.py::test_rust_version_surfaced PASSED
tests/test_smoke.py::test_build_features_listed PASSED
tests/test_smoke.py::test_hello_returns_ok PASSED
tests/test_smoke.py::test_native_module_private PASSED
tests/test_keypoint.py::test_keypoint_type_all_returns_17 PASSED
…
======================== 29 passed in 0.06s =========================
\`\`\`
Wheel size after both bindings: still well under the 5 MB ADR §5.4
budget (release build with --strip on Windows: ~340 KB).
Also adds `python/.gitignore` to prevent the `.venv/` + `target/` +
`_native.abi3.pyd` artifacts from getting committed.
## What's left in P2
CsiFrame + PoseEstimate bindings land in the next iteration. They're
larger (CsiFrame has the subcarrier buffer; PoseEstimate has
17×Keypoint + BoundingBox + track_id + score). Pattern is now proven
so they go faster.
Refs #785, ADR-117 §6.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117/p2): BoundingBox + PersonPose + PoseEstimate — P2 COMPLETE (57/57 tests GREEN)
Lands the second + third chunks of P2: PyO3 bindings for `BoundingBox`,
`PersonPose`, `PoseEstimate` from `wifi_densepose_core`. Combined with
the prior Keypoint + KeypointType bindings (
|
||
|---|---|---|
| .claude | ||
| .claude-flow | ||
| .claude-plugin | ||
| .github | ||
| .swarm | ||
| .vscode | ||
| archive | ||
| assets | ||
| dashboard | ||
| data | ||
| docker | ||
| docs | ||
| examples | ||
| firmware | ||
| logging | ||
| monitoring | ||
| plans | ||
| plugins/ruview | ||
| python | ||
| references | ||
| releases/desktop | ||
| scripts | ||
| tests | ||
| tools | ||
| ui | ||
| v2 | ||
| vendor | ||
| wifi_densepose | ||
| .dockerignore | ||
| .gitignore | ||
| .gitmodules | ||
| .mcp.json | ||
| CHANGELOG.md | ||
| CLAUDE.md | ||
| LICENSE | ||
| Makefile | ||
| README.md | ||
| benchmark_baseline.json | ||
| deploy.sh | ||
| example.env | ||
| install.sh | ||
| pyproject.toml | ||
| requirements-dev.txt | ||
| requirements.txt | ||
| verify | ||
README.md
π RuView
Beta Software — Under active development. APIs and firmware may change. Known limitations:
- ESP32-C3 and original ESP32 are not supported (single-core, insufficient for CSI DSP)
- Single ESP32 deployments have limited spatial resolution — use 2+ nodes or add a Cognitum Seed for best results
- Camera-free pose accuracy is limited (PCK@20 ≈ 2.5% with proxy labels) — camera ground-truth training targets 35%+ PCK@20; the pipeline is implemented, but the data-collection and evaluation phases (ADR-079 P7–P9) are still pending.
Contributions and bug reports welcome at Issues.
See through walls with WiFi
Turn ordinary WiFi into a spatial intelligence / sensing system. Detect people, measure breathing and heart rate, track movement, and monitor rooms — through walls, in the dark, with no cameras or wearables. Just physics.
Drop into any Home Assistant install with one
--mqttflag. Or pair into Apple Home / Google Home / Alexa / SmartThings as a Matter Bridge. Ships 21 entities per node (11 raw signals + 10 inferred semantic states: someone-sleeping, possible-distress, room-active, elderly-inactivity-anomaly, meeting-in-progress, bathroom-occupied, fall-risk-elevated, bed-exit, no-movement, multi-room-transition) plus 3 starter HA Blueprints. Seedocs/integrations/home-assistant.md· ADR-115.
π RuView is a WiFi sensing platform that turns radio signals into spatial intelligence.
Every WiFi router already fills your space with radio waves. When people move, breathe, or even sit still, they disturb those waves in measurable ways. RuView captures these disturbances using Channel State Information (CSI) from low-cost ESP32 sensors and turns them into actionable data: who's there, what they're doing, and whether they're okay.
What it senses:
- Presence and occupancy — detect people through walls, count them, track entries and exits
- Vital signs — breathing rate and heart rate, contactless, while sleeping or sitting
- Activity recognition — walking, sitting, gestures, falls — from temporal CSI patterns
- Environment mapping — RF fingerprinting identifies rooms, detects moved furniture, spots new objects
- Sleep quality — overnight monitoring with sleep stage classification and apnea screening
Built on RuVector and Cognitum Seed, RuView runs entirely on edge hardware — an ESP32 mesh (as low as $9 per node) paired with a Cognitum Seed for persistent memory, cryptographic attestation, and AI integration. No cloud, no cameras, no internet required.
The system learns each environment locally using spiking neural networks that adapt in under 30 seconds, with multi-frequency mesh scanning across 6 WiFi channels that uses your neighbors' routers as free radar illuminators. Every measurement is cryptographically attested via an Ed25519 witness chain.
RuView turns ordinary WiFi into a contactless sensor. A $9 ESP32 board reads the radio reflections off the people in a room, and a small pretrained model — published on Hugging Face at ruvnet/wifi-densepose-pretrained — tells you who's there, how they're breathing, and how their heart rate is trending. The model fits in 8 KB (4-bit quantized), runs in microseconds on a Raspberry Pi, and reports 100% presence accuracy on the validation set. No cameras, no wearables, no app on the user's phone.
Built for low-power edge applications
Edge modules are small programs that run directly on the ESP32 sensor — no internet needed, no cloud fees, instant response.
What How Speed / scale 🫁 Breathing rate Bandpass 0.1–0.5 Hz on wrapped phase, circular variance, zero-crossing BPM (#593) 6–30 BPM, real-time 💓 Heart rate Bandpass 0.8–2.0 Hz, zero-crossing BPM 40–120 BPM, real-time 👤 Presence detection Trained head on Hugging Face ( ruvnet/wifi-densepose-pretrained, 100% validation accuracy) + a phase-variance fallback that needs no model< 1 ms, ~30 s ambient calibration 🧬 CSI embeddings 128-dim contrastive encoder shipped on Hugging Face, 4-bit quantised variant fits in 8 KB 164,183 emb/s on M4 Pro 🦴 17-keypoint pose estimation cog-pose-estimationCog v0.0.1 — signed aarch64 + x86_64 binaries on GCS, loadspose_v1.safetensorsvia Candle. Train your own from paired data in 2.1 s on an RTX 5080 (ADR-101, benchmarks)8.4 ms cold-start on a Pi 5 🚶 Motion / activity Motion-band power + phase acceleration Real-time 🤸 Fall detection Phase-acceleration threshold + 3-frame debounce + 5 s cooldown (#263) < 200 ms 🧮 Multi-person count Adaptive P95 normalisation + runtime-tunable dedup factor ( /api/v1/config/dedup-factor, #491). Six specialised learned counters available as Cogs:occupancy-zones,elevator-count,queue-length,customer-flow,clean-room,person-matchingReal-time, self-calibrating 🧱 Through-wall sensing Fresnel-zone geometry + multipath modeling Up to ~5 m, signal-dependent 🧠 Edge intelligence 105-cog catalog (ADR-102) live from app-registry.json— health, security, building, retail, industrial, research, AI, swarm, signal, network, and developer modules. Optional Cognitum Seed adds persistent vector store + kNN + witness chain$140 total BOM 🎯 Camera-free pre-training Self-supervised contrastive encoder, 12.2M training steps on 60K frames, shipped on Hugging Face 84 s/epoch retrain on M4 Pro 📷 Camera-supervised fine-tune MediaPipe + ESP32 CSI paired training, end-to-end Candle pipeline on RTX 5080 (ADR-079) 2.1 s for 400 epochs (~5 ms/epoch) 📡 Multi-frequency mesh Channel hopping across 6 bands, TDM slot scheduling (ADR-029) 3× sensing bandwidth 🌐 3D point cloud fusion Camera depth (MiDaS) + WiFi CSI + mmWave radar → unified spatial model 22 ms pipeline · 19K+ points/frame Browse the full 105-module catalog (with practical descriptions, sizes, and difficulty) below in 🧩 Edge Module Catalog, or visit seed.cognitum.one/store.
🤗 Pretrained weights: download from
ruvnet/wifi-densepose-pretrained— see Loading the pretrained model below for one-command setup.
# Option 1: Docker (simulated data, no hardware needed)
docker pull ruvnet/wifi-densepose:latest
docker run -p 3000:3000 ruvnet/wifi-densepose:latest
# Open http://localhost:3000
# Option 2a: Live sensing with ESP32-S3 hardware ($9)
# Flash firmware, provision WiFi, and start sensing:
python -m esptool --chip esp32s3 --port COM9 --baud 460800 \
write_flash 0x0 bootloader.bin 0x8000 partition-table.bin \
0xf000 ota_data_initial.bin 0x20000 esp32-csi-node.bin
python firmware/esp32-csi-node/provision.py --port COM9 \
--ssid "YourWiFi" --password "secret" --target-ip 192.168.1.20
# Option 2b: WiFi 6 + 802.15.4 research sensing with ESP32-C6 ($6-10, ADR-110)
# Same csi-node firmware compiled for the C6 target — picks up the C6
# overlay (sdkconfig.defaults.esp32c6) automatically.
cd firmware/esp32-csi-node
idf.py set-target esp32c6 && idf.py build
idf.py -p COM6 flash
# C6 boot extras (vs S3): HE-LTF subcarrier tagging in ADR-018 bytes 18-19,
# 802.15.4 mesh time-sync on channel 15, TWT setup when the AP supports it,
# opt-in LP-core wake-on-motion for ~5 µA battery seed nodes.
# v0.6.7 adds: real LP-core RISC-V motion-gate program (debounce + motion
# counter) and a Wi-Fi 6 soft-AP with TWT Responder so two C6 boards can
# benchmark real iTWT without buying an 11ax router. Both default off,
# flip CONFIG_C6_{LP_CORE,SOFTAP_HE}_ENABLE to turn them on.
# Option 3: Full system with Cognitum Seed ($140)
# ESP32 streams CSI → bridge forwards to Seed for persistent storage + kNN + witness chain
node scripts/rf-scan.js --port 5006 # Live RF room scan
node scripts/snn-csi-processor.js --port 5006 # SNN real-time learning
node scripts/mincut-person-counter.js --port 5006 # Correct person counting
# Option 4: Python — talk to a RuView node from your own code (ADR-117)
pip install "wifi-densepose[client]" # ~250 KB compiled wheel, abi3-py310
# from wifi_densepose import BreathingExtractor, HeartRateExtractor
# from wifi_densepose.client import SensingClient, RuViewMqttClient
[!NOTE] CSI-capable hardware recommended. Presence, vital signs, through-wall sensing, and all advanced capabilities require Channel State Information (CSI) from an ESP32-S3 ($9) or research NIC. The Docker image runs with simulated data for evaluation. Consumer WiFi laptops provide RSSI-only presence detection.
Hardware options for live CSI capture:
Option Hardware Cost Full CSI Capabilities ESP32 + Cognitum Seed (recommended) ESP32-S3 + Cognitum Seed ~$140 Yes Presence, motion, breathing, heart rate, fall detection, multi-person counting, 17-keypoint pose (signed Cog binary), 105-cog catalog, persistent vector store, kNN search, witness chain, MCP proxy ESP32 Mesh 3-6× ESP32-S3 + WiFi router ~$54 Yes Same capabilities as above without the persistent-memory features ESP32-C6 research node (ADR-110, witness, reviewer guide, firmware v0.7.0) ESP32-C6-DevKit ($6–10) ~$10 Yes (Wi-Fi 6 capable) Same CSI pipeline as S3 with the dual-target firmware. Firmware-side ADR-110 substrate now closed (v0.7.0): ESP-NOW cross-board mesh quantified at 99.56 % match / 104 µs smoothed offset stdev / 3.95× EMA suppression over a 5-min two-board soak (witness §A0.10), 32-byte UDP sync packet with operator-tunable cadence (§A0.12), ADR-018 byte 19 bit 4 wire-fix sourced from the working ESP-NOW path (§A0.13). Wire format ready for HE-LTF PPDU tagging in ADR-018 bytes 18-19 (firmware encoder + Rust + Python decoders verified end-to-end across 23 unit tests). LP-core motion-gate RISC-V program and Wi-Fi 6 soft-AP with TWT Responder both ship as opt-in code paths (default off). Hardware-gated for measurement: HE-LTF live subcarrier capture needs an 11ax AP (IDF v5.4 doesn't expose AP-side HE config — §A0.6); ~5 µA LP-core hibernation needs an INA meter to capture; 802.15.4 raw RX is broken in IDF v5.4 (workaround: ESP-NOW transport, shipped + measured). See witness log for the empirical / claimed split. Research NIC Intel 5300 / Atheros AR9580 ~$50-100 Yes Full CSI with 3x3 MIMO Any WiFi Windows, macOS, or Linux laptop $0 No RSSI-only: coarse presence and motion (see tutorial #36) No hardware? Verify the signal processing pipeline with the deterministic reference signal:
python archive/v1/data/proof/verify.py
Real-time pose skeleton from WiFi CSI signals — no cameras, no wearables
▶ Live Observatory Demo | ▶ Dual-Modal Pose Fusion Demo | ▶ Live 3D Point Cloud | ▶ three.js Demos (5)
The server is optional for visualization and aggregation — the ESP32 runs independently for presence detection, vital signs, and fall alerts.
Live ESP32 pipeline: Connect an ESP32-S3 node → run the sensing server → open the pose fusion demo for real-time dual-modal pose estimation (webcam + WiFi CSI). See ADR-059.
three.js scene gallery at
/three.js/— five progressively richer ADR-097 demos: helpers, cinematic, GLTF skinned, FBX skinned, and a live MediaPipe→Mixamo retargeting feed driven by ESP32 CSI. Demos 04 and 05 require a local MixamoX Bot.fbx(license boundary — not redistributed).
🤗 Pretrained model on Hugging Face
Pretrained CSI weights live at ruvnet/wifi-densepose-pretrained — 12.2M training steps on 60K frames / 610K contrastive triplets, 100% presence accuracy on the validation set, 4-bit quantized variant fits in 8 KB. The release includes a contrastive CSI encoder producing 128-dim embeddings (164,183 emb/s on M4 Pro) and a presence-detection head. Per-node LoRA adapters are included for environment-specific fine-tuning.
# Download the model bundle
pip install huggingface_hub
huggingface-cli download ruvnet/wifi-densepose-pretrained --local-dir models/wifi-densepose-pretrained
What works today vs. what's pending wiring:
| Consumer | Format used | Status |
|---|---|---|
| Python training / evaluation / embedding extraction | model.safetensors |
✅ Works — load with safetensors.torch.load_file |
| Inspect / re-export the bundle | model.rvf.jsonl (line-by-line JSON) |
✅ Works — plain JSONL |
Sensing-server --model <PATH> flag |
binary RVF (RVFS magic) |
⚠️ Loader does not yet accept the JSONL container |
Known gap: the HF model ships in JSONL RVF format, but v2/crates/wifi-densepose-sensing-server/src/rvf_container.rs only parses the binary RVF segment format. Pointing --model at model.rvf.jsonl currently errors with invalid magic at offset 0: expected 0x52564653, got 0x7974227B and the live pipeline degrades to null output rather than falling back to heuristic mode — so for the live sensing-server, run without --model until a JSONL adapter lands (or the model is re-published as binary RVF). Use the weights from Python / training in the meantime.
Quantization choices (all in the HF repo): model-q2.bin (4 KB) · model-q4.bin ⭐ recommended (8 KB) · model-q8.bin (16 KB) · model.safetensors full (48 KB)
The separate 17-keypoint pose-estimation model is not in this release — pipeline is implemented but keypoint weights are still pending. Tracked in #509; see ADR-079 phases P7–P9.
🧩 Edge Module Catalog
🧩 105 edge modules ready to install on a Cognitum appliance — live catalog from app-registry.json v2.1.0 (updated 2026-05-13). Browse + install at seed.cognitum.one/store or your local appliance http://<appliance>:9000/cogs.
Each module is a small signed binary (~400 KB) that runs alongside the WiFi-DensePose sensing stack on a Cognitum-V0 appliance. The catalog updates over the air — your appliance fetches it via GET /api/v1/edge/registry (ADR-102) and verifies each binary against an Ed25519 signature (ADR-100) before install.
🫀 Health — 14 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
air-quality-index |
Track indoor air quality with CO2 and particle sensors | 8 KB | Easy |
baby-cry |
Sustained mid-band energy detector for nursery / infant monitoring. Audio-only, no camera. | 451 KB | Easy |
breathing-sync |
Detects when two people breathe in sync | 10 KB | Hard |
cardiac-arrhythmia |
Spots irregular heartbeats and abnormal heart rhythms | 8 KB | Hard |
cough-detect |
Acoustic transient + spectral cough detector with 30s cluster aggregation. Early-warning signal for respiratory illness. | 451 KB | Easy |
dream-stage |
Tracks your sleep stages — light, deep, and dreaming | 14 KB | Hard |
fall-detect |
Two-stage impact + stillness fall detector over ambient feature stream (ESP32 motion / mic). Optional ruview-mode for CSI-based pose reinforcement. | 402 KB | Easy |
gait-analysis |
Detects walking problems and scores fall risk | 12 KB | Hard |
health-monitor |
Contactless heart rate, breathing, sleep, and fall alerts | 30 KB | Med |
respiratory-distress |
Alerts when breathing becomes labored or dangerously fast | 10 KB | Hard |
seizure-detect |
Recognizes seizures and sends immediate alerts | 10 KB | Hard |
sleep-apnea |
Detects when someone stops breathing during sleep | 4 KB | Easy |
snore-monitor |
Periodic low-band energy tracker for sleep-quality / apnea-risk trending. Companion to sleep-apnea cog. | 451 KB | Easy |
vital-trend |
Tracks breathing and heart rate trends over weeks | 6 KB | Med |
🔒 Security — 14 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
audit-logger |
Record every action for compliance — tamper-proof log | 8 KB | Easy |
behavioral-profiler |
Learns normal behavior and flags anything unusual | 12 KB | Hard |
fleet-auth |
Manage device certificates and access across all seeds | 12 KB | Med |
glass-break |
Two-phase bang + shatter acoustic detector. Distinguishes glass break from ordinary impulse noise. | 451 KB | Easy |
gunshot-detect |
Saturating peak + exponential decay acoustic detector with optional ruview CSI motion-drop reinforcement. | 451 KB | Easy |
intrusion |
Alerts when an unauthorized person enters a room | 6 KB | Med |
intrusion-detect-ml |
Detect network attacks using machine learning | 14 KB | Hard |
loitering |
Alerts when someone lingers too long in one spot | 3 KB | Easy |
network-firewall |
Block unauthorized network access per cog | 6 KB | Easy |
panic-motion |
Detects sudden panicked or erratic movement | 6 KB | Med |
perimeter-breach |
Guards multiple zones and shows entry direction | 10 KB | Med |
prompt-shield |
Blocks signal replay and injection attacks on the seed | 10 KB | Med |
tailgating |
Catches when someone sneaks in behind a badge holder | 6 KB | Med |
weapon-detect |
Detects concealed metal objects on a person | 8 KB | Hard |
🏢 Building — 11 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
beehive-monitor |
Acoustic hive state classifier. Detects healthy / chaotic / queenless / swarming / robbing via hum-band energy + chaos + piping autocorr. | 451 KB | Easy |
elevator-count |
Counts how many people are in an elevator | 8 KB | Med |
energy-audit |
Learns your schedule and cuts wasted energy | 6 KB | Med |
frost-warning |
Predicts frost 6 hours ahead via temperature trend + dewpoint-depression gate. Field/orchard agriculture. | 451 KB | Easy |
hvac-presence |
Turns heating and cooling on when you arrive | 3 KB | Easy |
lighting-zones |
Turns lights on and off as people move between rooms | 4 KB | Easy |
meeting-room |
Shows if a meeting room is free or occupied | 5 KB | Easy |
occupancy-zones |
Counts people in each room through walls | 8 KB | Med |
predictive-maintenance |
Vibration harmonic analyzer for rotating equipment. Tracks F1 / 2×F1 / high-order / sideband energy to score degradation severity. | 451 KB | Easy |
smoke-fire |
Multi-signal smoke and fire detector. Fuses acoustic crackle, thermal drift proxy, and optional ruview CSI plume signature. Not a UL-listed replacement for code-required smoke alarms. | 451 KB | Easy |
water-leak |
Persistent low-amplitude hiss + periodic drip acoustic detector with multi-minute persistence gate. Two-stage likely → confirmed. | 451 KB | Easy |
🛍️ Retail — 7 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
customer-flow |
Counts foot traffic in and out of each entrance | 8 KB | Med |
dwell-heatmap |
Shows where customers spend the most time | 6 KB | Med |
package-detect |
Sustained CSI-shift detector for porch / loading bay package arrivals and departures. Requires ESP32 CSI ruview input. | 451 KB | Easy |
parking-occupancy |
Per-zone parking occupancy via ESP32 CSI subcarrier-amplitude shift. Tracks utilization and churn-per-hour. Requires ruview. | 451 KB | Easy |
queue-length |
Estimates line length and wait time | 6 KB | Med |
shelf-engagement |
Detects when customers interact with products | 6 KB | Med |
table-turnover |
Tracks which restaurant tables are free or occupied | 4 KB | Easy |
🏭 Industrial — 7 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
clean-room |
Enforces max headcount in controlled environments | 4 KB | Easy |
confined-space |
Monitors workers in tight spaces for safety | 5 KB | Med |
forklift-proximity |
Warns if a forklift gets too close to workers | 10 KB | Hard |
livestock-monitor |
Monitors animals for distress, escape, or illness | 6 KB | Med |
ppe-compliance |
Cog-composition layer: alerts when ruview-densepose detects presence in a restricted zone without an accompanying PPE-camera-cog confirmation vector. | 387 KB | Easy |
slip-fall-zone |
Pre-fall risk detector. Fires when motion-variance drop, splash audio, and optional cautious-gait CSI all signal elevated slip risk. | 451 KB | Easy |
structural-vibration |
Detects dangerous vibrations in buildings or machines | 8 KB | Hard |
🔬 Research — 12 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
emotion-detect |
Reads stress and calm from body language and breathing | 10 KB | Hard |
energy-harvester |
Optimize solar and battery for off-grid seed deployment | 6 KB | Med |
gesture-language |
Recognizes sign language gestures in real time | 12 KB | Hard |
ghost-hunter |
Finds unexplained environmental anomalies — for fun | 10 KB | Hard |
happiness-score |
Estimates well-being from movement and mood signals | 8 KB | Med |
hyperbolic-space |
Maps data into curved space for tree-like structures | 12 KB | Hard |
music-conductor |
Reads a conductor's gestures for tempo and dynamics | 12 KB | Hard |
plant-growth |
Tracks plant growth rate and day/night cycles | 8 KB | Med |
rain-detect |
Detects when rain starts, stops, and how heavy it is | 6 KB | Med |
ruview-densepose |
Full body pose tracking from WiFi — no cameras needed | 50 KB | Hard |
sound-classifier |
Identify sounds like glass break, alarm, or baby cry | 16 KB | Hard |
time-crystal |
Experiments with repeating time-pattern symmetry | 12 KB | Hard |
🤖 Ai — 15 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
anomaly-attractor |
Learns what's normal and catches anything weird | 10 KB | Hard |
cognitive-pipeline |
FastGRNN anomaly gate + SmolLM2 sparse-LLM inference for on-device Pi Zero 2W cognitive events | 320 KB | Hard |
dtw-gesture-learn |
Teach custom hand gestures by showing examples | 14 KB | Med |
ewc-lifelong |
Learns new things without forgetting old lessons | 8 KB | Hard |
federated-learning |
Train AI across seeds without sharing raw data | 18 KB | Hard |
goap-autonomy |
Plans and executes goals on its own | 14 KB | Hard |
meta-adapt |
Automatically tunes itself for best performance | 10 KB | Hard |
micro-hnsw |
Fast on-device fingerprinting and classification | 12 KB | Med |
neural-trader |
Spot market patterns and trends from live data | 20 KB | Hard |
pagerank-influence |
Finds the most influential person in a group | 12 KB | Med |
pattern-sequence |
Detects daily routines and repeated habits | 10 KB | Med |
rag-local |
Search your documents using AI — runs on the seed | 14 KB | Med |
spiking-tracker |
Brain-inspired tracker that runs on tiny hardware | 16 KB | Hard |
temporal-logic |
Enforces safety rules on live event streams | 12 KB | Hard |
time-series-forecast |
Predict sensor trends using historical patterns | 12 KB | Med |
🐝 Swarm — 11 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
swarm-backup-restore |
Auto-backup data to other seeds — one-click restore | 8 KB | Easy |
swarm-cluster-monitor |
Live dashboard of every seed's health and status | 6 KB | Easy |
swarm-consensus |
Seeds vote before making critical changes together | 16 KB | Hard |
swarm-delta-sync |
Auto-sync data between seeds — only sends changes | 8 KB | Med |
swarm-deploy |
Install or remove cogs on all seeds at once | 10 KB | Med |
swarm-distributed-store |
Spread data across seeds and search them all at once | 14 KB | Hard |
swarm-edge-orchestrator |
Manage all ESP32 sensor nodes from one place | 14 KB | Hard |
swarm-load-balancer |
Spread queries across seeds so no single one overloads | 10 KB | Med |
swarm-mesh-manager |
Find, connect, and monitor all seeds on your network | 12 KB | Easy |
swarm-mqtt-bridge |
Share events between seeds over MQTT messaging | 6 KB | Easy |
swarm-witness-federation |
Share tamper-proof audit trails across seeds | 12 KB | Hard |
📡 Signal — 6 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
coherence-gate |
Filters out noisy signals and keeps clean ones | 8 KB | Med |
flash-attention |
Focuses sensing on specific areas for better accuracy | 12 KB | Med |
optimal-transport |
Measures motion using shape-aware signal comparison | 12 KB | Hard |
person-matching |
Tells apart multiple people in the same room | 18 KB | Hard |
sparse-recovery |
Recovers missing signal data from partial readings | 16 KB | Hard |
temporal-compress |
Shrinks old data to save memory without losing meaning | 14 KB | Med |
🌐 Network — 1 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
tailscale |
Reach the seed from anywhere via a private WireGuard mesh (Tailscale). Userspace mode — no root. | 700 KB | Med |
🛠️ Developer — 7 modules
| ID | What it does | Size | Difficulty |
|---|---|---|---|
adversarial |
Detects tampered or spoofed sensor signals | 4 KB | Easy |
coherence |
Monitors signal quality across multiple channels | 4 KB | Easy |
gesture |
Core gesture recognition building block for cogs | 6 KB | Med |
interference-search |
Searches many possibilities at once for fast answers | 14 KB | Hard |
psycho-symbolic |
Reasons over knowledge graphs with multiple styles | 16 KB | Hard |
quantum-coherence |
Quantum-inspired model for advanced signal states | 16 KB | Hard |
self-healing-mesh |
Keeps sensor mesh running even when nodes drop out | 14 KB | Hard |
ℹ️ Build your own cog: see ADR-100 for the packaging spec. The first cog this repo ships into the catalog lives in v2/crates/cog-pose-estimation/ (17-keypoint WiFi pose, ADR-101).
🔬 How It Works
WiFi routers flood every room with radio waves. When a person moves — or even breathes — those waves scatter differently. WiFi DensePose reads that scattering pattern and reconstructs what happened:
WiFi Router → radio waves pass through room → hit human body → scatter
↓
ESP32 mesh (4-6 nodes) captures CSI on channels 1/6/11 via TDM protocol
↓
Multi-Band Fusion: 3 channels × 56 subcarriers = 168 virtual subcarriers per link
↓
Multistatic Fusion: N×(N-1) links → attention-weighted cross-viewpoint embedding
↓
Coherence Gate: accept/reject measurements → stable for days without tuning
↓
Signal Processing: Hampel, SpotFi, Fresnel, BVP, spectrogram → clean features
↓
AI Backbone (RuVector): attention, graph algorithms, compression, field model
↓
Signal-Line Protocol (CRV): 6-stage gestalt → sensory → topology → coherence → search → model
↓
Neural Network: processed signals → 17 body keypoints + vital signs + room model
↓
Output: real-time pose, breathing, heart rate, room fingerprint, drift alerts
No training cameras required — the Self-Learning system (ADR-024) bootstraps from raw WiFi data alone. MERIDIAN (ADR-027) ensures the model works in any room, not just the one it trained in.
🏢 Use Cases & Applications
WiFi sensing works anywhere WiFi exists. No new hardware in most cases — just software on existing access points or a $8 ESP32 add-on. Because there are no cameras, deployments avoid privacy regulations (GDPR video, HIPAA imaging) by design.
Scaling: Each AP distinguishes ~3-5 people (56 subcarriers). Multi-AP multiplies linearly — a 4-AP retail mesh covers ~15-20 occupants. No hard software limit; the practical ceiling is signal physics.
| Why WiFi sensing wins | Traditional alternative | |
|---|---|---|
| 🔒 | No video, no GDPR/HIPAA imaging rules | Cameras require consent, signage, data retention policies |
| 🧱 | Works through walls, shelving, debris | Cameras need line-of-sight per room |
| 🌙 | Works in total darkness | Cameras need IR or visible light |
| 💰 | $0-$8 per zone (existing WiFi or ESP32) | Camera systems: $200-$2,000 per zone |
| 🔌 | WiFi already deployed everywhere | PIR/radar sensors require new wiring per room |
🏥 Everyday — Healthcare, retail, office, hospitality (commodity WiFi)
| Use Case | What It Does | Hardware | Key Metric | Edge Module |
|---|---|---|---|---|
| Elderly care / assisted living | Fall detection, nighttime activity monitoring, breathing rate during sleep — no wearable compliance needed | 1 ESP32-S3 per room ($8) | Fall alert <2s | Sleep Apnea, Gait Analysis |
| Hospital patient monitoring | Continuous breathing + heart rate for non-critical beds without wired sensors; nurse alert on anomaly | 1-2 APs per ward | Breathing: 6-30 BPM | Respiratory Distress, Cardiac Arrhythmia |
| Emergency room triage | Automated occupancy count + wait-time estimation; detect patient distress (abnormal breathing) in waiting areas | Existing hospital WiFi | Occupancy accuracy >95% | Queue Length, Panic Motion |
| Retail occupancy & flow | Real-time foot traffic, dwell time by zone, queue length — no cameras, no opt-in, GDPR-friendly | Existing store WiFi + 1 ESP32 | Dwell resolution ~1m | Customer Flow, Dwell Heatmap |
| Office space utilization | Which desks/rooms are actually occupied, meeting room no-shows, HVAC optimization based on real presence | Existing enterprise WiFi | Presence latency <1s | Meeting Room, HVAC Presence |
| Hotel & hospitality | Room occupancy without door sensors, minibar/bathroom usage patterns, energy savings on empty rooms | Existing hotel WiFi | 15-30% HVAC savings | Energy Audit, Lighting Zones |
| Restaurants & food service | Table turnover tracking, kitchen staff presence, restroom occupancy displays — no cameras in dining areas | Existing WiFi | Queue wait ±30s | Table Turnover, Queue Length |
| Parking garages | Pedestrian presence in stairwells and elevators where cameras have blind spots; security alert if someone lingers | Existing WiFi | Through-concrete walls | Loitering, Elevator Count |
🏟️ Specialized — Events, fitness, education, civic (CSI-capable hardware)
| Use Case | What It Does | Hardware | Key Metric | Edge Module |
|---|---|---|---|---|
| Smart home automation | Room-level presence triggers (lights, HVAC, music) that work through walls — no dead zones, no motion-sensor timeouts | 2-3 ESP32-S3 nodes ($24) | Through-wall range ~5m | HVAC Presence, Lighting Zones |
| Fitness & sports | Rep counting, posture correction, breathing cadence during exercise — no wearable, no camera in locker rooms | 3+ ESP32-S3 mesh | Pose: 17 keypoints | Breathing Sync, Gait Analysis |
| Childcare & schools | Naptime breathing monitoring, playground headcount, restricted-area alerts — privacy-safe for minors | 2-4 ESP32-S3 per zone | Breathing: ±1 BPM | Sleep Apnea, Perimeter Breach |
| Event venues & concerts | Crowd density mapping, crush-risk detection via breathing compression, emergency evacuation flow tracking | Multi-AP mesh (4-8 APs) | Density per m² | Customer Flow, Panic Motion |
| Stadiums & arenas | Section-level occupancy for dynamic pricing, concession staffing, emergency egress flow modeling | Enterprise AP grid | 15-20 per AP mesh | Dwell Heatmap, Queue Length |
| Houses of worship | Attendance counting without facial recognition — privacy-sensitive congregations, multi-room campus tracking | Existing WiFi | Zone-level accuracy | Elevator Count, Energy Audit |
| Warehouse & logistics | Worker safety zones, forklift proximity alerts, occupancy in hazardous areas — works through shelving and pallets | Industrial AP mesh | Alert latency <500ms | Forklift Proximity, Confined Space |
| Civic infrastructure | Public restroom occupancy (no cameras possible), subway platform crowding, shelter headcount during emergencies | Municipal WiFi + ESP32 | Real-time headcount | Customer Flow, Loitering |
| Museums & galleries | Visitor flow heatmaps, exhibit dwell time, crowd bottleneck alerts — no cameras near artwork (flash/theft risk) | Existing WiFi | Zone dwell ±5s | Dwell Heatmap, Shelf Engagement |
🤖 Robotics & Industrial — Autonomous systems, manufacturing, android spatial awareness
WiFi sensing gives robots and autonomous systems a spatial awareness layer that works where LIDAR and cameras fail — through dust, smoke, fog, and around corners. The CSI signal field acts as a "sixth sense" for detecting humans in the environment without requiring line-of-sight.
| Use Case | What It Does | Hardware | Key Metric | Edge Module |
|---|---|---|---|---|
| Cobot safety zones | Detect human presence near collaborative robots — auto-slow or stop before contact, even behind obstructions | 2-3 ESP32-S3 per cell | Presence latency <100ms | Forklift Proximity, Perimeter Breach |
| Warehouse AMR navigation | Autonomous mobile robots sense humans around blind corners, through shelving racks — no LIDAR occlusion | ESP32 mesh along aisles | Through-shelf detection | Forklift Proximity, Loitering |
| Android / humanoid spatial awareness | Ambient human pose sensing for social robots — detect gestures, approach direction, and personal space without cameras always on | Onboard ESP32-S3 module | 17-keypoint pose | Gesture Language, Emotion Detection |
| Manufacturing line monitoring | Worker presence at each station, ergonomic posture alerts, headcount for shift compliance — works through equipment | Industrial AP per zone | Pose + breathing | Confined Space, Gait Analysis |
| Construction site safety | Exclusion zone enforcement around heavy machinery, fall detection from scaffolding, personnel headcount | Ruggedized ESP32 mesh | Alert <2s, through-dust | Panic Motion, Structural Vibration |
| Agricultural robotics | Detect farm workers near autonomous harvesters in dusty/foggy field conditions where cameras are unreliable | Weatherproof ESP32 nodes | Range ~10m open field | Forklift Proximity, Rain Detection |
| Drone landing zones | Verify landing area is clear of humans — WiFi sensing works in rain, dust, and low light where downward cameras fail | Ground ESP32 nodes | Presence: >95% accuracy | Perimeter Breach, Tailgating |
| Clean room monitoring | Personnel tracking without cameras (particle contamination risk from camera fans) — gown compliance via pose | Existing cleanroom WiFi | No particulate emission | Clean Room, Livestock Monitor |
🔥 Extreme — Through-wall, disaster, defense, underground
These scenarios exploit WiFi's ability to penetrate solid materials — concrete, rubble, earth — where no optical or infrared sensor can reach. The WiFi-Mat disaster module (ADR-001) is specifically designed for this tier.
| Use Case | What It Does | Hardware | Key Metric | Edge Module |
|---|---|---|---|---|
| Search & rescue (WiFi-Mat) | Detect survivors through rubble/debris via breathing signature, START triage color classification, 3D localization | Portable ESP32 mesh + laptop | Through 30cm concrete | Respiratory Distress, Seizure Detection |
| Firefighting | Locate occupants through smoke and walls before entry; breathing detection confirms life signs remotely | Portable mesh on truck | Works in zero visibility | Sleep Apnea, Panic Motion |
| Prison & secure facilities | Cell occupancy verification, distress detection (abnormal vitals), perimeter sensing — no camera blind spots | Dedicated AP infrastructure | 24/7 vital signs | Cardiac Arrhythmia, Loitering |
| Military / tactical | Through-wall personnel detection, room clearing confirmation, hostage vital signs at standoff distance | Directional WiFi + custom FW | Range: 5m through wall | Perimeter Breach, Weapon Detection |
| Border & perimeter security | Detect human presence in tunnels, behind fences, in vehicles — passive sensing, no active illumination to reveal position | Concealed ESP32 mesh | Passive / covert | Perimeter Breach, Tailgating |
| Mining & underground | Worker presence in tunnels where GPS/cameras fail, breathing detection after collapse, headcount at safety points | Ruggedized ESP32 mesh | Through rock/earth | Confined Space, Respiratory Distress |
| Maritime & naval | Below-deck personnel tracking through steel bulkheads (limited range, requires tuning), man-overboard detection | Ship WiFi + ESP32 | Through 1-2 bulkheads | Structural Vibration, Panic Motion |
| Wildlife research | Non-invasive animal activity monitoring in enclosures or dens — no light pollution, no visual disturbance | Weatherproof ESP32 nodes | Zero light emission | Livestock Monitor, Dream Stage |
🧠 Self-Learning WiFi AI (ADR-024) — Adaptive recognition, self-optimization, and intelligent anomaly detection
Every WiFi signal that passes through a room creates a unique fingerprint of that space. WiFi-DensePose already reads these fingerprints to track people, but until now it threw away the internal "understanding" after each reading. The Self-Learning WiFi AI captures and preserves that understanding as compact, reusable vectors — and continuously optimizes itself for each new environment.
What it does in plain terms:
- Turns any WiFi signal into a 128-number "fingerprint" that uniquely describes what's happening in a room
- Learns entirely on its own from raw WiFi data — no cameras, no labeling, no human supervision needed
- Recognizes rooms, detects intruders, identifies people, and classifies activities using only WiFi
- Runs on an $8 ESP32 chip (the entire model fits in 55 KB of memory)
- Produces both body pose tracking AND environment fingerprints in a single computation
Key Capabilities
| What | How it works | Why it matters |
|---|---|---|
| Self-supervised learning | The model watches WiFi signals and teaches itself what "similar" and "different" look like, without any human-labeled data | Deploy anywhere — just plug in a WiFi sensor and wait 10 minutes |
| Room identification | Each room produces a distinct WiFi fingerprint pattern | Know which room someone is in without GPS or beacons |
| Anomaly detection | An unexpected person or event creates a fingerprint that doesn't match anything seen before | Automatic intrusion and fall detection as a free byproduct |
| Person re-identification | Each person disturbs WiFi in a slightly different way, creating a personal signature | Track individuals across sessions without cameras |
| Environment adaptation | MicroLoRA adapters (1,792 parameters per room) fine-tune the model for each new space | Adapts to a new room with minimal data — 93% less than retraining from scratch |
| Memory preservation | EWC++ regularization remembers what was learned during pretraining | Switching to a new task doesn't erase prior knowledge |
| Hard-negative mining | Training focuses on the most confusing examples to learn faster | Better accuracy with the same amount of training data |
Architecture
WiFi Signal [56 channels] → Transformer + Graph Neural Network
├→ 128-dim environment fingerprint (for search + identification)
└→ 17-joint body pose (for human tracking)
Quick Start
# Step 1: Learn from raw WiFi data (no labels needed)
cargo run -p wifi-densepose-sensing-server -- --pretrain --dataset data/csi/ --pretrain-epochs 50
# Step 2: Fine-tune with pose labels for full capability
cargo run -p wifi-densepose-sensing-server -- --train --dataset data/mmfi/ --epochs 100 --save-rvf model.rvf
# Step 3: Use the model — extract fingerprints from live WiFi
cargo run -p wifi-densepose-sensing-server -- --model model.rvf --embed
# Step 4: Search — find similar environments or detect anomalies
cargo run -p wifi-densepose-sensing-server -- --model model.rvf --build-index env
Training Modes
| Mode | What you need | What you get |
|---|---|---|
| Self-Supervised | Just raw WiFi data | A model that understands WiFi signal structure |
| Supervised | WiFi data + body pose labels | Full pose tracking + environment fingerprints |
| Cross-Modal | WiFi data + camera footage | Fingerprints aligned with visual understanding |
Fingerprint Index Types
| Index | What it stores | Real-world use |
|---|---|---|
env_fingerprint |
Average room fingerprint | "Is this the kitchen or the bedroom?" |
activity_pattern |
Activity boundaries | "Is someone cooking, sleeping, or exercising?" |
temporal_baseline |
Normal conditions | "Something unusual just happened in this room" |
person_track |
Individual movement signatures | "Person A just entered the living room" |
Model Size
| Component | Parameters | Memory (on ESP32) |
|---|---|---|
| Transformer backbone | ~28,000 | 28 KB |
| Embedding projection head | ~25,000 | 25 KB |
| Per-room MicroLoRA adapter | ~1,800 | 2 KB |
| Total | ~55,000 | 55 KB (of 520 KB available) |
The self-learning system builds on the AI Backbone (RuVector) signal-processing layer — attention, graph algorithms, and compression — adding contrastive learning on top.
See docs/adr/ADR-024-contrastive-csi-embedding-model.md for full architectural details.
🧩 Claude Code & Codex Plugin
RuView ships a Claude Code plugin (and Codex prompt mirror) that wraps the whole workflow — onboarding, ESP32 setup, configuration, sensing apps, model training, advanced multistatic sensing, CLI/API/WASM, mmWave radar, and witness verification — as 9 skills, 7 /ruview-* commands, and 3 agents. It lives in plugins/ruview/; the marketplace manifest is .claude-plugin/marketplace.json at the repo root.
# In Claude Code — add this repo as a plugin marketplace, then install:
/plugin marketplace add ruvnet/RuView
/plugin install ruview@ruview
# Or try it for one session without installing (from a local clone of the repo):
claude --plugin-dir ./plugins/ruview
# Then, in Claude Code:
# /ruview-start → onboarding (Docker demo / repo build / live ESP32)
# /ruview-flash → build + flash ESP32 firmware
# /ruview-provision → provision WiFi creds, sink IP, channel/MAC, mesh slots
# /ruview-app → run a sensing application (presence / vitals / pose / sleep / MAT / point cloud)
# /ruview-train → train / evaluate / publish a model (incl. GPU on GCloud)
# /ruview-advanced → multistatic / tomography / cross-viewpoint / mesh-security
# /ruview-verify → tests + deterministic proof + witness bundle
Codex (OpenAI CLI): cp plugins/ruview/codex/prompts/*.md ~/.codex/prompts/ — the seven /ruview-* commands are mirrored as Codex prompts; plugins/ruview/codex/AGENTS.md carries the project rules. See plugins/ruview/codex/README.md.
Verify the plugin structure: bash plugins/ruview/scripts/smoke.sh. Full details: plugins/ruview/README.md.
📖 Documentation
| Document | Description |
|---|---|
| User Guide | Step-by-step guide: installation, first run, API usage, hardware setup, training |
| Build Guide | Building from source (Rust and Python) |
| Home Assistant + Matter Integration | Works with Home Assistant via MQTT auto-discovery + Works with Matter (Apple Home / Google Home / Alexa / SmartThings) — full entity catalog, 3 starter blueprints, Lovelace dashboards, privacy mode, threshold tuning (ADR-115). |
| Semantic Primitives — Precision/Recall | Per-primitive F1 on the held-out paired-capture set: someone-sleeping, possible-distress, room-active, elderly-inactivity-anomaly, meeting, bathroom, fall-risk, bed-exit, no-movement, multi-room. |
| Claude Code / Codex Plugin | The ruview plugin + marketplace — skills, /ruview-* commands, agents, and the Codex prompt mirror |
| Architecture Decisions | 96 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
| Domain Models | 8 DDD models (RuvSense, Signal Processing, Training Pipeline, Hardware Platform, Sensing Server, WiFi-Mat, CHCI, rvCSI) — bounded contexts, aggregates, domain events, and ubiquitous language |
| rvCSI — edge RF sensing runtime | Rust-first / TypeScript-accessible / hardware-abstracted CSI runtime: multi-source ingestion (incl. real nexmon_csi .pcap from a Raspberry Pi 5 / Pi 4 / Pi 3B+ — CYW43455 / BCM43455c0) → validation → DSP → typed events → RuVector RF memory (ADR-095, ADR-096, domain model). Now its own repo — ruvnet/rvcsi — vendored here under vendor/rvcsi; 9 rvcsi-* crates on crates.io, @ruv/rvcsi on npm, plus a Claude Code plugin. |
| Desktop App | WIP — Tauri v2 desktop app for node management, OTA updates, WASM deployment, and mesh visualization |
| Medical Examples | Contactless blood pressure, heart rate, breathing rate via 60 GHz mmWave radar — $15 hardware, no wearable |
| Extended Documentation | Latest additions, key features, installation, quick start, signal processing, training, CLI, testing, deployment, and changelog |
📄 License
MIT License — see LICENSE for details.
🤝 Creator Affiliate Program
For TikTok · Instagram · YouTube creators — earn 25% on every Cognitum sale you refer. The RuFlo, RuView, and RuVector videos you're already making have done millions of views; get paid for the orders they drive. Click-tracking activates instantly; commissions activate after a quick manual review (usually under 24 hours).
Apply now → cognitum.one/affiliate
📞 Support
GitHub Issues | Discussions | PyPI
WiFi DensePose — Privacy-preserving human pose estimation through WiFi signals.