diff --git a/ui/README.md b/ui/README.md index 541b4c11..e337ad5a 100644 --- a/ui/README.md +++ b/ui/README.md @@ -1,46 +1,73 @@ # WiFi DensePose UI -A modular, modern web interface for the WiFi DensePose human tracking system. This UI provides real-time monitoring, configuration, and visualization of WiFi-based pose estimation. +A modular, modern web interface for the WiFi DensePose human tracking system. Provides real-time monitoring, WiFi sensing visualization, and pose estimation from CSI (Channel State Information). -## ๐๏ธ Architecture +## Architecture The UI follows a modular architecture with clear separation of concerns: ``` ui/ -โโโ app.js # Main application entry point -โโโ index.html # Updated HTML with modular structure -โโโ style.css # Complete CSS with additional styles -โโโ config/ # Configuration modules -โ โโโ api.config.js # API endpoints and configuration -โโโ services/ # Service layer for API communication -โ โโโ api.service.js # HTTP API client -โ โโโ websocket.service.js # WebSocket client -โ โโโ pose.service.js # Pose estimation API wrapper -โ โโโ health.service.js # Health monitoring API wrapper -โ โโโ stream.service.js # Streaming API wrapper -โโโ components/ # UI components -โ โโโ TabManager.js # Tab navigation component -โ โโโ DashboardTab.js # Dashboard component with live data -โ โโโ HardwareTab.js # Hardware configuration component -โ โโโ LiveDemoTab.js # Live demo with streaming -โโโ utils/ # Utility functions and helpers -โ โโโ mock-server.js # Mock server for testing -โโโ tests/ # Comprehensive test suite - โโโ test-runner.html # Test runner UI - โโโ test-runner.js # Test framework and cases - โโโ integration-test.html # Integration testing page +โโโ app.js # Main application entry point +โโโ index.html # HTML shell with tab structure +โโโ style.css # Complete CSS design system +โโโ config/ +โ โโโ api.config.js # API endpoints and configuration +โโโ services/ +โ โโโ api.service.js # HTTP API client +โ โโโ websocket.service.js # WebSocket connection manager +โ โโโ websocket-client.js # Low-level WebSocket client +โ โโโ pose.service.js # Pose estimation API wrapper +โ โโโ sensing.service.js # WiFi sensing data service (live + simulation fallback) +โ โโโ health.service.js # Health monitoring API wrapper +โ โโโ stream.service.js # Streaming API wrapper +โ โโโ data-processor.js # Signal data processing utilities +โโโ components/ +โ โโโ TabManager.js # Tab navigation component +โ โโโ DashboardTab.js # Dashboard with live system metrics +โ โโโ SensingTab.js # WiFi sensing visualization (3D signal field, metrics) +โ โโโ LiveDemoTab.js # Live pose detection with setup guide +โ โโโ HardwareTab.js # Hardware configuration +โ โโโ SettingsPanel.js # Settings panel +โ โโโ PoseDetectionCanvas.js # Canvas-based pose skeleton renderer +โ โโโ gaussian-splats.js # 3D Gaussian splat signal field renderer (Three.js) +โ โโโ body-model.js # 3D body model +โ โโโ scene.js # Three.js scene management +โ โโโ signal-viz.js # Signal visualization utilities +โ โโโ environment.js # Environment/room visualization +โ โโโ dashboard-hud.js # Dashboard heads-up display +โโโ utils/ +โ โโโ backend-detector.js # Auto-detect backend availability +โ โโโ mock-server.js # Mock server for testing +โ โโโ pose-renderer.js # Pose rendering utilities +โโโ tests/ + โโโ test-runner.html # Test runner UI + โโโ test-runner.js # Test framework and cases + โโโ integration-test.html # Integration testing page ``` -## ๐ Features +## Features -### Smart Backend Detection -- **Automatic Detection**: Automatically detects if your FastAPI backend is running -- **Real Backend Priority**: Always uses the real backend when available -- **Mock Fallback**: Falls back to mock server only when backend is unavailable -- **Testing Mode**: Can force mock mode for testing and development +### WiFi Sensing Tab +- 3D Gaussian-splat signal field visualization (Three.js) +- Real-time RSSI, variance, motion band, breathing band metrics +- Presence/motion classification with confidence scores +- **Data source banner**: green "LIVE - ESP32", yellow "RECONNECTING...", or red "SIMULATED DATA" +- Sparkline RSSI history graph +- "About This Data" card explaining CSI capabilities per sensor count -### Real-time Dashboard +### Live Demo Tab +- WebSocket-based real-time pose skeleton rendering +- **Estimation Mode badge**: green "Signal-Derived" or blue "Model Inference" +- **Setup Guide panel** showing what each ESP32 count provides: + - 1 ESP32: presence, breathing, gross motion + - 2-3 ESP32s: body localization, motion direction + - 4+ ESP32s + trained model: individual limb tracking, full pose +- Debug mode with log export +- Zone selection and force-reconnect controls +- Performance metrics sidebar (frames, uptime, errors) + +### Dashboard - Live system health monitoring - Real-time pose detection statistics - Zone occupancy tracking @@ -53,284 +80,118 @@ ui/ - Configuration panels - Hardware status monitoring -### Live Demo -- WebSocket-based real-time streaming -- Signal visualization -- Pose detection visualization -- Interactive controls +## Data Sources -### API Integration -- Complete REST API coverage -- WebSocket streaming support -- Authentication handling -- Error management -- Request/response interceptors +The sensing service (`sensing.service.js`) supports three connection states: -## ๐ API Coverage +| State | Banner Color | Description | +|-------|-------------|-------------| +| **LIVE - ESP32** | Green | Connected to the Rust sensing server receiving real CSI data | +| **RECONNECTING** | Yellow (pulsing) | WebSocket disconnected, retrying (up to 20 attempts) | +| **SIMULATED DATA** | Red | Fallback to client-side simulation after 5+ failed reconnects | -The UI integrates with all WiFi DensePose API endpoints: +Simulated frames include a `_simulated: true` marker so code can detect synthetic data. -### Health Endpoints -- `GET /health/health` - System health check -- `GET /health/ready` - Readiness check -- `GET /health/live` - Liveness check -- `GET /health/metrics` - System metrics -- `GET /health/version` - Version information +## Backends -### Pose Estimation -- `GET /api/v1/pose/current` - Current pose data -- `POST /api/v1/pose/analyze` - Trigger analysis -- `GET /api/v1/pose/zones/{zone_id}/occupancy` - Zone occupancy -- `GET /api/v1/pose/zones/summary` - All zones summary -- `POST /api/v1/pose/historical` - Historical data -- `GET /api/v1/pose/activities` - Recent activities -- `POST /api/v1/pose/calibrate` - System calibration -- `GET /api/v1/pose/stats` - Statistics +### Rust Sensing Server (primary) +The Rust-based `wifi-densepose-sensing-server` serves the UI and provides: +- `GET /health` โ server health +- `GET /api/v1/sensing/latest` โ latest sensing features +- `GET /api/v1/vital-signs` โ vital sign estimates (HR/RR) +- `GET /api/v1/model/info` โ RVF model container info +- `WS /ws/sensing` โ real-time sensing data stream +- `WS /api/v1/stream/pose` โ real-time pose keypoint stream -### Streaming -- `WS /api/v1/stream/pose` - Real-time pose stream -- `WS /api/v1/stream/events` - Event stream -- `GET /api/v1/stream/status` - Stream status -- `POST /api/v1/stream/start` - Start streaming -- `POST /api/v1/stream/stop` - Stop streaming -- `GET /api/v1/stream/clients` - Connected clients -- `DELETE /api/v1/stream/clients/{client_id}` - Disconnect client +### Python FastAPI (legacy) +The original Python backend on port 8000 is still supported. The UI auto-detects which backend is available via `backend-detector.js`. -## ๐งช Testing - -### Test Runner -Open `tests/test-runner.html` to run the complete test suite: +## Quick Start +### With Docker (recommended) ```bash -# Serve the UI directory on port 3000 (to avoid conflicts with FastAPI on 8000) -cd /workspaces/wifi-densepose/ui +cd docker/ + +# Default: auto-detects ESP32 on UDP 5005, falls back to simulation +docker-compose up + +# Force real ESP32 data +CSI_SOURCE=esp32 docker-compose up + +# Force simulation (no hardware needed) +CSI_SOURCE=simulated docker-compose up +``` +Open http://localhost:3000/ui/index.html + +### With local Rust binary +```bash +cd rust-port/wifi-densepose-rs +cargo build -p wifi-densepose-sensing-server --no-default-features + +# Run with simulated data +../../target/debug/sensing-server --source simulated --tick-ms 100 --ui-path ../../ui --http-port 3000 + +# Run with real ESP32 +../../target/debug/sensing-server --source esp32 --tick-ms 100 --ui-path ../../ui --http-port 3000 +``` +Open http://localhost:3000/ui/index.html + +### With Python HTTP server (legacy) +```bash +# Start FastAPI backend on port 8000 +wifi-densepose start + +# Serve the UI on port 3000 +cd ui/ python -m http.server 3000 -# Open http://localhost:3000/tests/test-runner.html ``` +Open http://localhost:3000 -### Test Categories -- **API Configuration Tests** - Configuration and URL building -- **API Service Tests** - HTTP client functionality -- **WebSocket Service Tests** - WebSocket connection management -- **Pose Service Tests** - Pose estimation API wrapper -- **Health Service Tests** - Health monitoring functionality -- **UI Component Tests** - Component behavior and interaction -- **Integration Tests** - End-to-end functionality +## Pose Estimation Modes -### Integration Testing -Use `tests/integration-test.html` for visual integration testing: +| Mode | Badge | Requirements | Accuracy | +|------|-------|-------------|----------| +| **Signal-Derived** | Green | 1+ ESP32, no model needed | Presence, breathing, gross motion | +| **Model Inference** | Blue | 4+ ESP32s + trained `.rvf` model | Full 17-keypoint COCO pose | +To use model inference, start the server with a trained model: ```bash -# Open http://localhost:3000/tests/integration-test.html +sensing-server --source esp32 --model path/to/model.rvf --ui-path ./ui ``` -Features: -- Mock server with realistic API responses -- Visual testing of all components -- Real-time data simulation -- Error scenario testing -- WebSocket stream testing - -## ๐ ๏ธ Usage - -### Basic Setup -```html - - -
- - - -