From 9dd61bdbfa4928d05053ef37a679d7e7b6ef2ff7 Mon Sep 17 00:00:00 2001 From: ruv Date: Mon, 2 Mar 2026 10:57:56 -0500 Subject: [PATCH] docs: update UI README with sensing tab, Rust backend, data sources Reflects current state: Rust sensing server as primary backend, sensing tab with 3D signal field, data source indicators, estimation mode badge, setup guide, Docker deployment with CSI_SOURCE, and updated file tree with all components/services. Co-Authored-By: claude-flow --- ui/README.md | 403 +++++++++++++++++---------------------------------- 1 file changed, 132 insertions(+), 271 deletions(-) diff --git a/ui/README.md b/ui/README.md index 541b4c11..e337ad5a 100644 --- a/ui/README.md +++ b/ui/README.md @@ -1,46 +1,73 @@ # WiFi DensePose UI -A modular, modern web interface for the WiFi DensePose human tracking system. This UI provides real-time monitoring, configuration, and visualization of WiFi-based pose estimation. +A modular, modern web interface for the WiFi DensePose human tracking system. Provides real-time monitoring, WiFi sensing visualization, and pose estimation from CSI (Channel State Information). -## ๐Ÿ—๏ธ Architecture +## Architecture The UI follows a modular architecture with clear separation of concerns: ``` ui/ -โ”œโ”€โ”€ app.js # Main application entry point -โ”œโ”€โ”€ index.html # Updated HTML with modular structure -โ”œโ”€โ”€ style.css # Complete CSS with additional styles -โ”œโ”€โ”€ config/ # Configuration modules -โ”‚ โ””โ”€โ”€ api.config.js # API endpoints and configuration -โ”œโ”€โ”€ services/ # Service layer for API communication -โ”‚ โ”œโ”€โ”€ api.service.js # HTTP API client -โ”‚ โ”œโ”€โ”€ websocket.service.js # WebSocket client -โ”‚ โ”œโ”€โ”€ pose.service.js # Pose estimation API wrapper -โ”‚ โ”œโ”€โ”€ health.service.js # Health monitoring API wrapper -โ”‚ โ””โ”€โ”€ stream.service.js # Streaming API wrapper -โ”œโ”€โ”€ components/ # UI components -โ”‚ โ”œโ”€โ”€ TabManager.js # Tab navigation component -โ”‚ โ”œโ”€โ”€ DashboardTab.js # Dashboard component with live data -โ”‚ โ”œโ”€โ”€ HardwareTab.js # Hardware configuration component -โ”‚ โ””โ”€โ”€ LiveDemoTab.js # Live demo with streaming -โ”œโ”€โ”€ utils/ # Utility functions and helpers -โ”‚ โ””โ”€โ”€ mock-server.js # Mock server for testing -โ””โ”€โ”€ tests/ # Comprehensive test suite - โ”œโ”€โ”€ test-runner.html # Test runner UI - โ”œโ”€โ”€ test-runner.js # Test framework and cases - โ””โ”€โ”€ integration-test.html # Integration testing page +โ”œโ”€โ”€ app.js # Main application entry point +โ”œโ”€โ”€ index.html # HTML shell with tab structure +โ”œโ”€โ”€ style.css # Complete CSS design system +โ”œโ”€โ”€ config/ +โ”‚ โ””โ”€โ”€ api.config.js # API endpoints and configuration +โ”œโ”€โ”€ services/ +โ”‚ โ”œโ”€โ”€ api.service.js # HTTP API client +โ”‚ โ”œโ”€โ”€ websocket.service.js # WebSocket connection manager +โ”‚ โ”œโ”€โ”€ websocket-client.js # Low-level WebSocket client +โ”‚ โ”œโ”€โ”€ pose.service.js # Pose estimation API wrapper +โ”‚ โ”œโ”€โ”€ sensing.service.js # WiFi sensing data service (live + simulation fallback) +โ”‚ โ”œโ”€โ”€ health.service.js # Health monitoring API wrapper +โ”‚ โ”œโ”€โ”€ stream.service.js # Streaming API wrapper +โ”‚ โ””โ”€โ”€ data-processor.js # Signal data processing utilities +โ”œโ”€โ”€ components/ +โ”‚ โ”œโ”€โ”€ TabManager.js # Tab navigation component +โ”‚ โ”œโ”€โ”€ DashboardTab.js # Dashboard with live system metrics +โ”‚ โ”œโ”€โ”€ SensingTab.js # WiFi sensing visualization (3D signal field, metrics) +โ”‚ โ”œโ”€โ”€ LiveDemoTab.js # Live pose detection with setup guide +โ”‚ โ”œโ”€โ”€ HardwareTab.js # Hardware configuration +โ”‚ โ”œโ”€โ”€ SettingsPanel.js # Settings panel +โ”‚ โ”œโ”€โ”€ PoseDetectionCanvas.js # Canvas-based pose skeleton renderer +โ”‚ โ”œโ”€โ”€ gaussian-splats.js # 3D Gaussian splat signal field renderer (Three.js) +โ”‚ โ”œโ”€โ”€ body-model.js # 3D body model +โ”‚ โ”œโ”€โ”€ scene.js # Three.js scene management +โ”‚ โ”œโ”€โ”€ signal-viz.js # Signal visualization utilities +โ”‚ โ”œโ”€โ”€ environment.js # Environment/room visualization +โ”‚ โ””โ”€โ”€ dashboard-hud.js # Dashboard heads-up display +โ”œโ”€โ”€ utils/ +โ”‚ โ”œโ”€โ”€ backend-detector.js # Auto-detect backend availability +โ”‚ โ”œโ”€โ”€ mock-server.js # Mock server for testing +โ”‚ โ””โ”€โ”€ pose-renderer.js # Pose rendering utilities +โ””โ”€โ”€ tests/ + โ”œโ”€โ”€ test-runner.html # Test runner UI + โ”œโ”€โ”€ test-runner.js # Test framework and cases + โ””โ”€โ”€ integration-test.html # Integration testing page ``` -## ๐Ÿš€ Features +## Features -### Smart Backend Detection -- **Automatic Detection**: Automatically detects if your FastAPI backend is running -- **Real Backend Priority**: Always uses the real backend when available -- **Mock Fallback**: Falls back to mock server only when backend is unavailable -- **Testing Mode**: Can force mock mode for testing and development +### WiFi Sensing Tab +- 3D Gaussian-splat signal field visualization (Three.js) +- Real-time RSSI, variance, motion band, breathing band metrics +- Presence/motion classification with confidence scores +- **Data source banner**: green "LIVE - ESP32", yellow "RECONNECTING...", or red "SIMULATED DATA" +- Sparkline RSSI history graph +- "About This Data" card explaining CSI capabilities per sensor count -### Real-time Dashboard +### Live Demo Tab +- WebSocket-based real-time pose skeleton rendering +- **Estimation Mode badge**: green "Signal-Derived" or blue "Model Inference" +- **Setup Guide panel** showing what each ESP32 count provides: + - 1 ESP32: presence, breathing, gross motion + - 2-3 ESP32s: body localization, motion direction + - 4+ ESP32s + trained model: individual limb tracking, full pose +- Debug mode with log export +- Zone selection and force-reconnect controls +- Performance metrics sidebar (frames, uptime, errors) + +### Dashboard - Live system health monitoring - Real-time pose detection statistics - Zone occupancy tracking @@ -53,284 +80,118 @@ ui/ - Configuration panels - Hardware status monitoring -### Live Demo -- WebSocket-based real-time streaming -- Signal visualization -- Pose detection visualization -- Interactive controls +## Data Sources -### API Integration -- Complete REST API coverage -- WebSocket streaming support -- Authentication handling -- Error management -- Request/response interceptors +The sensing service (`sensing.service.js`) supports three connection states: -## ๐Ÿ“‹ API Coverage +| State | Banner Color | Description | +|-------|-------------|-------------| +| **LIVE - ESP32** | Green | Connected to the Rust sensing server receiving real CSI data | +| **RECONNECTING** | Yellow (pulsing) | WebSocket disconnected, retrying (up to 20 attempts) | +| **SIMULATED DATA** | Red | Fallback to client-side simulation after 5+ failed reconnects | -The UI integrates with all WiFi DensePose API endpoints: +Simulated frames include a `_simulated: true` marker so code can detect synthetic data. -### Health Endpoints -- `GET /health/health` - System health check -- `GET /health/ready` - Readiness check -- `GET /health/live` - Liveness check -- `GET /health/metrics` - System metrics -- `GET /health/version` - Version information +## Backends -### Pose Estimation -- `GET /api/v1/pose/current` - Current pose data -- `POST /api/v1/pose/analyze` - Trigger analysis -- `GET /api/v1/pose/zones/{zone_id}/occupancy` - Zone occupancy -- `GET /api/v1/pose/zones/summary` - All zones summary -- `POST /api/v1/pose/historical` - Historical data -- `GET /api/v1/pose/activities` - Recent activities -- `POST /api/v1/pose/calibrate` - System calibration -- `GET /api/v1/pose/stats` - Statistics +### Rust Sensing Server (primary) +The Rust-based `wifi-densepose-sensing-server` serves the UI and provides: +- `GET /health` โ€” server health +- `GET /api/v1/sensing/latest` โ€” latest sensing features +- `GET /api/v1/vital-signs` โ€” vital sign estimates (HR/RR) +- `GET /api/v1/model/info` โ€” RVF model container info +- `WS /ws/sensing` โ€” real-time sensing data stream +- `WS /api/v1/stream/pose` โ€” real-time pose keypoint stream -### Streaming -- `WS /api/v1/stream/pose` - Real-time pose stream -- `WS /api/v1/stream/events` - Event stream -- `GET /api/v1/stream/status` - Stream status -- `POST /api/v1/stream/start` - Start streaming -- `POST /api/v1/stream/stop` - Stop streaming -- `GET /api/v1/stream/clients` - Connected clients -- `DELETE /api/v1/stream/clients/{client_id}` - Disconnect client +### Python FastAPI (legacy) +The original Python backend on port 8000 is still supported. The UI auto-detects which backend is available via `backend-detector.js`. -## ๐Ÿงช Testing - -### Test Runner -Open `tests/test-runner.html` to run the complete test suite: +## Quick Start +### With Docker (recommended) ```bash -# Serve the UI directory on port 3000 (to avoid conflicts with FastAPI on 8000) -cd /workspaces/wifi-densepose/ui +cd docker/ + +# Default: auto-detects ESP32 on UDP 5005, falls back to simulation +docker-compose up + +# Force real ESP32 data +CSI_SOURCE=esp32 docker-compose up + +# Force simulation (no hardware needed) +CSI_SOURCE=simulated docker-compose up +``` +Open http://localhost:3000/ui/index.html + +### With local Rust binary +```bash +cd rust-port/wifi-densepose-rs +cargo build -p wifi-densepose-sensing-server --no-default-features + +# Run with simulated data +../../target/debug/sensing-server --source simulated --tick-ms 100 --ui-path ../../ui --http-port 3000 + +# Run with real ESP32 +../../target/debug/sensing-server --source esp32 --tick-ms 100 --ui-path ../../ui --http-port 3000 +``` +Open http://localhost:3000/ui/index.html + +### With Python HTTP server (legacy) +```bash +# Start FastAPI backend on port 8000 +wifi-densepose start + +# Serve the UI on port 3000 +cd ui/ python -m http.server 3000 -# Open http://localhost:3000/tests/test-runner.html ``` +Open http://localhost:3000 -### Test Categories -- **API Configuration Tests** - Configuration and URL building -- **API Service Tests** - HTTP client functionality -- **WebSocket Service Tests** - WebSocket connection management -- **Pose Service Tests** - Pose estimation API wrapper -- **Health Service Tests** - Health monitoring functionality -- **UI Component Tests** - Component behavior and interaction -- **Integration Tests** - End-to-end functionality +## Pose Estimation Modes -### Integration Testing -Use `tests/integration-test.html` for visual integration testing: +| Mode | Badge | Requirements | Accuracy | +|------|-------|-------------|----------| +| **Signal-Derived** | Green | 1+ ESP32, no model needed | Presence, breathing, gross motion | +| **Model Inference** | Blue | 4+ ESP32s + trained `.rvf` model | Full 17-keypoint COCO pose | +To use model inference, start the server with a trained model: ```bash -# Open http://localhost:3000/tests/integration-test.html +sensing-server --source esp32 --model path/to/model.rvf --ui-path ./ui ``` -Features: -- Mock server with realistic API responses -- Visual testing of all components -- Real-time data simulation -- Error scenario testing -- WebSocket stream testing - -## ๐Ÿ› ๏ธ Usage - -### Basic Setup -```html - - - - - - -
- -
- - - -``` - -### Using Services -```javascript -import { poseService } from './services/pose.service.js'; -import { healthService } from './services/health.service.js'; - -// Get current pose data -const poseData = await poseService.getCurrentPose(); - -// Subscribe to health updates -healthService.subscribeToHealth(health => { - console.log('Health status:', health.status); -}); - -// Start pose streaming -poseService.startPoseStream({ - minConfidence: 0.7, - maxFps: 30 -}); - -poseService.subscribeToPoseUpdates(update => { - if (update.type === 'pose_update') { - console.log('New pose data:', update.data); - } -}); -``` - -### Using Components -```javascript -import { TabManager } from './components/TabManager.js'; -import { DashboardTab } from './components/DashboardTab.js'; - -// Initialize tab manager -const container = document.querySelector('.container'); -const tabManager = new TabManager(container); -tabManager.init(); - -// Initialize dashboard -const dashboardContainer = document.getElementById('dashboard'); -const dashboard = new DashboardTab(dashboardContainer); -await dashboard.init(); -``` - -## ๐Ÿ”ง Configuration +## Configuration ### API Configuration -Edit `config/api.config.js` to modify API settings: +Edit `config/api.config.js`: ```javascript export const API_CONFIG = { BASE_URL: window.location.origin, API_VERSION: '/api/v1', - - // Rate limiting - RATE_LIMITS: { - REQUESTS_PER_MINUTE: 60, - BURST_LIMIT: 10 - }, - - // WebSocket configuration WS_CONFIG: { RECONNECT_DELAY: 5000, - MAX_RECONNECT_ATTEMPTS: 5, + MAX_RECONNECT_ATTEMPTS: 20, PING_INTERVAL: 30000 } }; ``` -### Authentication -```javascript -import { apiService } from './services/api.service.js'; +## Testing -// Set authentication token -apiService.setAuthToken('your-jwt-token'); +Open `tests/test-runner.html` to run the test suite: -// Add request interceptor for auth -apiService.addRequestInterceptor((url, options) => { - // Modify request before sending - return { url, options }; -}); -``` - -## ๐ŸŽจ Styling - -The UI uses a comprehensive CSS design system with: - -- CSS Custom Properties for theming -- Dark/light mode support -- Responsive design -- Component-based styling -- Smooth animations and transitions - -### Key CSS Variables -```css -:root { - --color-primary: rgba(33, 128, 141, 1); - --color-background: rgba(252, 252, 249, 1); - --color-surface: rgba(255, 255, 253, 1); - --color-text: rgba(19, 52, 59, 1); - --space-16: 16px; - --radius-lg: 12px; -} -``` - -## ๐Ÿ” Monitoring & Debugging - -### Health Monitoring -```javascript -import { healthService } from './services/health.service.js'; - -// Start automatic health checks -healthService.startHealthMonitoring(30000); // Every 30 seconds - -// Check if system is healthy -const isHealthy = healthService.isSystemHealthy(); - -// Get specific component status -const apiStatus = healthService.getComponentStatus('api'); -``` - -### Error Handling -```javascript -// Global error handling -window.addEventListener('error', (event) => { - console.error('Global error:', event.error); -}); - -// API error handling -apiService.addResponseInterceptor(async (response, url) => { - if (!response.ok) { - console.error(`API error: ${response.status} for ${url}`); - } - return response; -}); -``` - -## ๐Ÿš€ Deployment - -### Development - -**Option 1: Use the startup script** ```bash -cd /workspaces/wifi-densepose/ui -./start-ui.sh -``` - -**Option 2: Manual setup** -```bash -# First, start your FastAPI backend (runs on port 8000) -wifi-densepose start -# or from the main project directory: -python -m wifi_densepose.main - -# Then, start the UI server on a different port to avoid conflicts -cd /workspaces/wifi-densepose/ui +cd ui/ python -m http.server 3000 -# or -npx http-server . -p 3000 - -# Open the UI at http://localhost:3000 -# The UI will automatically detect and connect to your backend +# Open http://localhost:3000/tests/test-runner.html ``` -### Backend Detection Behavior -- **Real Backend Available**: UI connects to `http://localhost:8000` and shows โœ… "Connected to real backend" -- **Backend Unavailable**: UI automatically uses mock server and shows โš ๏ธ "Mock server active - testing mode" -- **Force Mock Mode**: Set `API_CONFIG.MOCK_SERVER.ENABLED = true` for testing +Test categories: API configuration, API service, WebSocket, pose service, health service, UI components, integration. -### Production -1. Configure `API_CONFIG.BASE_URL` for your backend -2. Set up HTTPS for WebSocket connections -3. Configure authentication if required -4. Optimize assets (minify CSS/JS) -5. Set up monitoring and logging +## Styling -## ๐Ÿค Contributing +Uses a CSS design system with custom properties, dark/light mode, responsive layout, and component-based styling. Key variables in `:root` of `style.css`. -1. Follow the modular architecture -2. Add tests for new functionality -3. Update documentation -4. Ensure TypeScript compatibility -5. Test with mock server +## License -## ๐Ÿ“„ License - -This project is part of the WiFi-DensePose system. See the main project LICENSE file for details. \ No newline at end of file +Part of the WiFi-DensePose system. See the main project LICENSE file.