docs: add ADR-059 live ESP32 CSI pipeline + update README with demo links
- ADR-059: Documents end-to-end ESP32 → sensing server → browser pipeline - README: Add dual-modal pose fusion demo link, update ADR count to 49 - References issue #245 Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
ruv
parent
2f5e7ffb41
commit
0223ef6d2e
|
|
@ -75,7 +75,7 @@ docker run -p 3000:3000 ruvnet/wifi-densepose:latest
|
|||
|----------|-------------|
|
||||
| [User Guide](docs/user-guide.md) | Step-by-step guide: installation, first run, API usage, hardware setup, training |
|
||||
| [Build Guide](docs/build-guide.md) | Building from source (Rust and Python) |
|
||||
| [Architecture Decisions](docs/adr/README.md) | 48 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
|
||||
| [Architecture Decisions](docs/adr/README.md) | 49 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
|
||||
| [Domain Models](docs/ddd/README.md) | 7 DDD models (RuvSense, Signal Processing, Training Pipeline, Hardware Platform, Sensing Server, WiFi-Mat, CHCI) — bounded contexts, aggregates, domain events, and ubiquitous language |
|
||||
| [Desktop App](rust-port/wifi-densepose-rs/crates/wifi-densepose-desktop/README.md) | **WIP** — Tauri v2 desktop app for node management, OTA updates, WASM deployment, and mesh visualization |
|
||||
|
||||
|
|
@ -89,8 +89,12 @@ docker run -p 3000:3000 ruvnet/wifi-densepose:latest
|
|||
<em>Real-time pose skeleton from WiFi CSI signals — no cameras, no wearables</em>
|
||||
<br>
|
||||
<a href="https://ruvnet.github.io/RuView/"><strong>▶ Live Observatory Demo</strong></a>
|
||||
|
|
||||
<a href="https://ruvnet.github.io/RuView/pose-fusion.html"><strong>▶ Dual-Modal Pose Fusion Demo</strong></a>
|
||||
|
||||
> The [server](#-quick-start) is optional for visualization and aggregation — the ESP32 [runs independently](#esp32-s3-hardware-pipeline) for presence detection, vital signs, and fall alerts.
|
||||
>
|
||||
> **Live ESP32 pipeline**: Connect an ESP32-S3 node → run the [sensing server](#sensing-server) → open the [pose fusion demo](https://ruvnet.github.io/RuView/pose-fusion.html) for real-time dual-modal pose estimation (webcam + WiFi CSI). See [ADR-059](docs/adr/ADR-059-live-esp32-csi-pipeline.md).
|
||||
|
||||
|
||||
## 🚀 Key Features
|
||||
|
|
|
|||
|
|
@ -0,0 +1,83 @@
|
|||
# ADR-059: Live ESP32 CSI Pipeline Integration
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Date
|
||||
|
||||
2026-03-12
|
||||
|
||||
## Context
|
||||
|
||||
ADR-058 established a dual-modal browser demo combining webcam video and WiFi CSI for pose estimation. However, it used simulated CSI data. To demonstrate real-world capability, we need an end-to-end pipeline from physical ESP32 hardware through to the browser visualization.
|
||||
|
||||
The ESP32-S3 firmware (`firmware/esp32-csi-node/`) already supports CSI collection and UDP streaming (ADR-018). The sensing server (`wifi-densepose-sensing-server`) already supports UDP ingestion and WebSocket bridging. The missing piece was connecting these components and enabling the browser demo to consume live data.
|
||||
|
||||
## Decision
|
||||
|
||||
Implement a complete live CSI pipeline:
|
||||
|
||||
```
|
||||
ESP32-S3 (CSI capture) → UDP:5005 → sensing-server (Rust/Axum) → WS:8765 → browser demo
|
||||
```
|
||||
|
||||
### Components
|
||||
|
||||
1. **ESP32 Firmware** — Rebuilt with native Windows ESP-IDF v5.4.0 toolchain (no Docker). Configured for target network and PC IP via `sdkconfig`. Helper scripts added:
|
||||
- `build_firmware.ps1` — Sets up IDF environment, cleans, builds, and flashes
|
||||
- `read_serial.ps1` — Serial monitor with DTR/RTS reset capability
|
||||
|
||||
2. **Sensing Server** — `wifi-densepose-sensing-server` started with:
|
||||
- `--source esp32` — Expect real ESP32 UDP frames
|
||||
- `--bind-addr 0.0.0.0` — Accept connections from any interface
|
||||
- `--ui-path <path>` — Serve the demo UI via HTTP
|
||||
|
||||
3. **Browser Demo** — `main.js` updated to auto-connect to `ws://localhost:8765/ws/sensing` on page load. Falls back to simulated CSI if the WebSocket is unavailable (GitHub Pages).
|
||||
|
||||
### Network Configuration
|
||||
|
||||
The ESP32 sends UDP packets to a configured target IP. If the PC's IP doesn't match the firmware's compiled target, a secondary IP alias can be added:
|
||||
|
||||
```powershell
|
||||
# PowerShell (Admin)
|
||||
New-NetIPAddress -IPAddress 192.168.1.100 -PrefixLength 24 -InterfaceAlias "Wi-Fi"
|
||||
```
|
||||
|
||||
### Data Flow
|
||||
|
||||
| Stage | Protocol | Format | Rate |
|
||||
|-------|----------|--------|------|
|
||||
| ESP32 → Server | UDP | ADR-018 binary frame (magic `0xC5110001`, I/Q pairs) | ~100 Hz |
|
||||
| Server → Browser | WebSocket | ADR-018 binary frame (forwarded) | ~10 Hz (tick-ms=100) |
|
||||
| Browser decode | JavaScript | Float32 amplitude/phase arrays | Per frame |
|
||||
|
||||
### Build Environment (Windows)
|
||||
|
||||
ESP-IDF v5.4.0 on Windows requires:
|
||||
- IDF_PATH pointing to the ESP-IDF framework
|
||||
- IDF_TOOLS_PATH pointing to toolchain binaries
|
||||
- MSYS/MinGW environment variables removed (ESP-IDF rejects them)
|
||||
- Python venv from ESP-IDF tools for `idf.py` execution
|
||||
|
||||
The `build_firmware.ps1` script handles all of this automatically.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- First end-to-end demonstration of real WiFi CSI → pose estimation in a browser
|
||||
- No Docker required for firmware builds on Windows
|
||||
- Demo gracefully degrades to simulated CSI when no server is available
|
||||
- Same demo works on GitHub Pages (simulated) and locally (live ESP32)
|
||||
|
||||
### Negative
|
||||
- ESP32 target IP is compiled into firmware; changing it requires a rebuild or NVS override
|
||||
- Windows firewall may block UDP:5005; user must allow it
|
||||
- Mixed content restrictions prevent HTTPS pages from connecting to ws:// (local only)
|
||||
|
||||
## Related
|
||||
|
||||
- [ADR-018](ADR-018-esp32-dev-implementation.md) — ESP32 CSI frame format and UDP streaming
|
||||
- [ADR-058](ADR-058-ruvector-wasm-browser-pose-example.md) — Dual-modal WASM browser pose demo
|
||||
- [ADR-039](ADR-039-edge-intelligence-framework.md) — Edge intelligence on ESP32
|
||||
- Issue [#245](https://github.com/ruvnet/RuView/issues/245) — Tracking issue
|
||||
Loading…
Reference in New Issue