berkus
/
wifi-densepose
mirror of https://github.com/ruvnet/wifi-densepose.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs: update README with ADR-045–048, Observatory, adaptive classifier, AMOLED display 

- Update ADR count from 44 to 48
- Add adaptive classifier (ADR-048) to Intelligence features
- Add Observatory visualization (ADR-047) and AMOLED display (ADR-045) to Deployment features
- Update screenshot to v2-screen.png
- Add ADR-045 (AMOLED), ADR-046 (Android TV), ADR-047 (Observatory), DDD deployment model
- Add AMOLED display firmware (display_hal, display_task, display_ui, LVGL config)
- Add Observatory UI (13 Three.js modules, CSS, HTML entry point)
- Add trained adaptive model JSON
- Update .gitignore for managed_components, recordings, .swarm

Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
ruv 2026-03-05 10:20:48 -05:00
parent 5fa61ba7ea
commit 8b57a6f64c
35 changed files with 8674 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
.gitignore vendored
View File

@ -8,6 +8,16 @@ firmware/esp32-csi-node/sdkconfig.defaults
firmware/esp32-csi-node/sdkconfig.old
# Downloaded WASM3 source (fetched at configure time)
firmware/esp32-csi-node/components/wasm3/wasm3-src/
# ESP-IDF managed components (downloaded at build time)
firmware/esp32-csi-node/managed_components/
firmware/esp32-csi-node/dependencies.lock
firmware/esp32-csi-node/sdkconfig.defaults.bak
# Claude Flow swarm runtime state
.swarm/
# CSI recordings (local training data, machine-specific)
rust-port/wifi-densepose-rs/data/recordings/
# NVS partition images and CSVs (contain WiFi credentials)
nvs.bin

7
README.md
View File

@ -57,13 +57,13 @@ docker run -p 3000:3000 ruvnet/wifi-densepose:latest
|----------|-------------|
| [User Guide](docs/user-guide.md) | Step-by-step guide: installation, first run, API usage, hardware setup, training |
| [Build Guide](docs/build-guide.md) | Building from source (Rust and Python) |
| [Architecture Decisions](docs/adr/README.md) | 44 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
| [Architecture Decisions](docs/adr/README.md) | 48 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
| [Domain Models](docs/ddd/README.md) | 7 DDD models (RuvSense, Signal Processing, Training Pipeline, Hardware Platform, Sensing Server, WiFi-Mat, CHCI) — bounded contexts, aggregates, domain events, and ubiquitous language |
---
<img src="assets/screen.png" alt="WiFi DensePose — Live pose detection with setup guide" width="800">
<img src="assets/v2-screen.png" alt="WiFi DensePose — Live pose detection with setup guide" width="800">
<br>
<em>Real-time pose skeleton from WiFi CSI signals — no cameras, no wearables</em>
@ -98,6 +98,7 @@ The system learns on its own and gets smarter over time — no hand-tuning, no l
| 👁️ | **Cross-Viewpoint Fusion** | AI combines what each sensor sees from its own angle — fills in blind spots and depth ambiguity that no single viewpoint can resolve on its own ([ADR-031](docs/adr/ADR-031-ruview-sensing-first-rf-mode.md)) |
| 🔮 | **Signal-Line Protocol** | A 6-stage processing pipeline transforms raw WiFi signals into structured body representations — from signal cleanup through graph-based spatial reasoning to final pose output ([ADR-033](docs/adr/ADR-033-crv-signal-line-sensing-integration.md)) |
| 🔒 | **QUIC Mesh Security** | All sensor-to-sensor communication is encrypted end-to-end with tamper detection, replay protection, and seamless reconnection if a node moves or drops offline ([ADR-032](docs/adr/ADR-032-multistatic-mesh-security-hardening.md)) |
| 🎯 | **Adaptive Classifier** | Records labeled CSI sessions, trains a 15-feature logistic regression model in pure Rust, and learns your room's unique signal characteristics — replaces hand-tuned thresholds with data-driven classification ([ADR-048](docs/adr/ADR-048-adaptive-csi-classifier.md)) |
### Performance & Deployment
@ -110,6 +111,8 @@ Fast enough for real-time use, small enough for edge devices, simple enough for
| 🐳 | **One-Command Setup** | `docker pull ruvnet/wifi-densepose:latest` — live sensing in 30 seconds, no toolchain needed (amd64 + arm64 / Apple Silicon) |
| 📡 | **Fully Local** | Runs completely on a $9 ESP32 — no internet connection, no cloud account, no recurring fees. Detects presence, vital signs, and falls on-device with instant response |
| 📦 | **Portable Models** | Trained models package into a single `.rvf` file — runs on edge, cloud, or browser (WASM) |
| 🔭 | **Observatory Visualization** | Cinematic Three.js dashboard with 5 holographic panels — subcarrier manifold, vital signs oracle, presence heatmap, phase constellation, convergence engine — all driven by live or demo CSI data ([ADR-047](docs/adr/ADR-047-psychohistory-observatory-visualization.md)) |
| 📟 | **AMOLED Display** | ESP32-S3 boards with built-in AMOLED screens show real-time presence, vital signs, and room status directly on the sensor — no phone or PC needed ([ADR-045](docs/adr/ADR-045-amoled-display-support.md)) |
---

BIN
assets/v2-screen.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.0 MiB

110
docs/adr/ADR-045-amoled-display-support.md Normal file
View File

@ -0,0 +1,110 @@
# ADR-045: AMOLED Display Support for ESP32-S3 CSI Node
## Status
Proposed
## Context
The ESP32-S3 board (LilyGO T-Display-S3 AMOLED) has an integrated RM67162 QSPI AMOLED display (536x240) and 8MB octal PSRAM that were unused by the CSI firmware. Users want real-time on-device visualization of CSI statistics, vital signs, and system health without relying on an external server.
### Constraints
- Binary was 947 KB in a 1 MB partition — needed 8MB flash + custom partition table
- SPIRAM was disabled in sdkconfig despite hardware having 8MB PSRAM
- Core 1 is pinned to DSP (edge processing) — display must use Core 0
- Existing CSI pipeline must not be affected
### Available APIs
Thread-safe edge APIs already exist (`edge_get_vitals()`, `edge_get_multi_person()`) — the display task only reads from these, no new synchronization needed.
## Decision
Add optional AMOLED display support with the following architecture:
### Hardware Abstraction Layer
- `display_hal.c/h`: RM67162 QSPI panel driver + CST816S capacitive touch via I2C
- Auto-detect at boot: probe RM67162 and check SPIRAM; log warning and skip if absent
### UI Layer
- `display_ui.c/h`: LVGL 8.3 with 4 swipeable views via tileview widget
- Dark theme (#0a0a0f) with cyan (#00d4ff) accent for three.js-like aesthetic
- Views: Dashboard (CSI amplitude chart + stats), Vitals (breathing + HR line graphs), Presence (4x4 occupancy grid), System (CPU, heap, PSRAM, WiFi, uptime, FPS)
### Task Layer
- `display_task.c/h`: FreeRTOS task on Core 0, priority 1 (lowest)
- LVGL pump loop at configurable FPS (default 30)
- Double-buffered draw buffers allocated in SPIRAM
### Compile-Time Control
- `CONFIG_DISPLAY_ENABLE=y` (default): compiles display code, auto-detects hardware at boot
- `CONFIG_DISPLAY_ENABLE=n`: zero-cost — no display code compiled
- `CONFIG_SPIRAM_IGNORE_NOTFOUND=y`: boots fine on boards without PSRAM
### Flash Layout
8MB partition table (`partitions_display.csv`):
- Dual OTA partitions: 2 x 2MB (supports larger binaries with LVGL)
- SPIFFS: 1.9MB (for future font/asset storage)
- NVS + otadata + phy: standard sizes
### Core/Task Layout
| Task | Core | Priority | Impact |
|------|------|----------|--------|
| WiFi/LwIP | 0 | 18-23 | unchanged |
| OTA httpd | 0 | 5 | unchanged |
| **display_task** | **0** | **1** | **NEW — lowest priority** |
| edge_task (DSP) | 1 | 5 | unchanged |
### Dependencies
- LVGL ~8.3 (via ESP-IDF managed components)
- espressif/esp_lcd_touch_cst816s ^1.0
- espressif/esp_lcd_touch ^1.0
## Consequences
### Positive
- Real-time on-device stats without network dependency
- Zero impact on CSI pipeline (display reads thread-safe APIs, runs at lowest priority)
- Graceful degradation: works on boards without display or PSRAM
- SPIRAM enabled for all boards (benefits WASM runtime too)
- 8MB flash + dual OTA 2MB partitions give headroom for future features
### Negative
- Binary size increase (~200-300 KB with LVGL)
- SPIRAM + 8MB flash config is specific to T-Display-S3 AMOLED boards
- Boards with only 4MB flash need `CONFIG_DISPLAY_ENABLE=n` and the old partition table
### Risks
- RM67162 init sequence is board-specific; other AMOLED panels may need different commands
- QSPI bus conflicts if other peripherals use SPI2_HOST (currently unused)
## New Files
| File | Purpose |
|------|---------|
| `main/display_hal.c/h` | RM67162 QSPI + CST816S touch HAL |
| `main/display_ui.c/h` | LVGL 4-view UI |
| `main/display_task.c/h` | FreeRTOS task, LVGL pump |
| `main/lv_conf.h` | LVGL compile config |
| `partitions_display.csv` | 8MB partition table |
| `idf_component.yml` | Managed component deps |
## Modified Files
| File | Change |
|------|--------|
| `sdkconfig.defaults` | 8MB flash, SPIRAM, custom partitions |
| `main/CMakeLists.txt` | Conditional display sources + deps |
| `main/main.c` | +1 include, +5 lines guarded init |
| `main/Kconfig.projbuild` | "AMOLED Display" menu |

263
docs/adr/ADR-046-android-tv-box-armbian-deployment.md Normal file
View File

@ -0,0 +1,263 @@
# ADR-046: Android TV Box / Armbian Deployment Target
## Status
Proposed
## Context
Issue [#138](https://github.com/ruvnet/wifi-densepose/issues/138) requests ESP8266 and mobile device support. The ESP8266 lacks CSI capability and sufficient resources, but the discussion revealed a compelling deployment target: **Android TV boxes** (Amlogic/Allwinner/Rockchip SoCs) running **Armbian** (Debian for ARM).
These devices cost $15–35, are always-on mains-powered, include 802.11ac WiFi, 2–4 GB RAM, quad-core ARM Cortex-A53/A55 CPUs, and HDMI output. They are widely available as consumer "IPTV boxes" (T95, H96 Max, X96, MXQ Pro, etc.) and can boot Armbian from SD card without modifying the factory Android installation.
### Current deployment model
```
[ESP32-S3 nodes] --UDP CSI--> [Laptop/PC running sensing-server] --browser--> [UI]
```
This requires a general-purpose computer ($300+) to run the Rust sensing server, NN inference, and web dashboard. For permanent installations (elder care, smart home, security), dedicating a laptop is impractical.
### Proposed deployment model
```
[ESP32-S3 nodes] --UDP CSI--> [TV Box running Armbian + sensing-server] --HDMI--> [Display]
$25, always-on, fanless
```
### Future: custom WiFi firmware for standalone operation
Many TV box WiFi chipsets (Realtek RTL8822CS, MediaTek MT7661, Broadcom BCM43455) can potentially be patched for CSI extraction when running under Linux with custom drivers. This would eliminate the ESP32 dependency entirely for basic sensing:
```
[TV Box with patched WiFi driver] --CSI extraction--> [sensing-server on same box] --HDMI--> [Display]
$25 total, single device
```
This ADR covers Phase 1 (TV box as aggregator) and Phase 2 (custom WiFi firmware for CSI). Phase 2 is speculative and requires per-chipset R&D.
## Decision
### Phase 1: TV Box as Aggregator (Armbian)
1. **Cross-compile the sensing server** for `aarch64-unknown-linux-gnu` using `cross` or Docker-based cross-compilation.
2. **Create an Armbian deployment package** containing:
- Pre-built `wifi-densepose-sensing-server` binary (aarch64)
- systemd service file for auto-start on boot
- Kiosk-mode Chromium configuration for HDMI dashboard display
- Network configuration for ESP32 UDP reception (port 5005)
- Optional: `hostapd` config to create a dedicated WiFi AP for the ESP32 mesh
3. **Define minimum hardware requirements:**
| Component | Minimum | Recommended |
|-----------|---------|-------------|
| SoC | Amlogic S905W (A53 quad) | Amlogic S905X3 (A55 quad) |
| RAM | 2 GB | 4 GB |
| Storage | 8 GB eMMC + 8 GB SD | 16 GB eMMC + 16 GB SD |
| WiFi | 802.11n 2.4 GHz | 802.11ac dual-band |
| Ethernet | 100 Mbps | Gigabit |
| USB | 1x USB 2.0 | 2x USB 3.0 |
| HDMI | 1.4 | 2.0 |
4. **Tested reference devices** (initial target list):
| Device | SoC | WiFi Chip | Price | Armbian Support |
|--------|-----|-----------|-------|-----------------|
| T95 Max+ | S905X3 | RTL8822CS | ~$30 | Good (meson-sm1) |
| H96 Max X3 | S905X3 | RTL8822CS | ~$35 | Good (meson-sm1) |
| X96 Max+ | S905X3 | RTL8822CS | ~$28 | Good (meson-sm1) |
| Tanix TX6S | H616 | MT7668 | ~$25 | Moderate (sun50i-h616) |
5. **New Rust compilation target** in workspace CI:
- Add `aarch64-unknown-linux-gnu` to cross-compilation matrix
- Binary size target: <15 MB stripped (fits easily in SD card)
- No GPU dependency — CPU-only inference using `candle` or ONNX Runtime for ARM
### Phase 2: Custom WiFi Firmware for CSI Extraction (Future)
1. **CSI extraction feasibility by chipset:**
| Chipset | Driver | CSI Support | Monitor Mode | Effort |
|---------|--------|-------------|--------------|--------|
| Broadcom BCM43455 | brcmfmac | **Proven** (Nexmon CSI) | Yes | Low — patches exist |
| Realtek RTL8822CS | rtw88 | **Moderate** — driver is open-source, CSI hooks need adding | Yes (patched) | Medium |
| MediaTek MT7661 | mt76 | **Unknown** — MediaTek has released CSI tools for some chips | Yes | Medium-High |
2. **CSI extraction architecture** (Linux kernel driver modification):
```
[WiFi chipset firmware] → [Modified kernel driver] → [Netlink/procfs CSI export]
↓
[userspace CSI reader]
↓
[sensing-server UDP input]
```
The CSI data would be reformatted into the existing ESP32 binary protocol (ADR-018 header, magic `0xC5100001`) so the sensing server treats it identically to ESP32 frames. This means zero changes to the ingestion context.
3. **Hybrid mode**: When the TV box has both patched WiFi CSI and ESP32 UDP input, the sensing server's multi-node architecture (already supporting multiple `node_id` values) handles both sources transparently. The TV box's own WiFi becomes an additional viewpoint in the multistatic array.
### Phase 3: Android Companion App (Optional)
For users who want mobile monitoring without Armbian:
1. **PWA (Progressive Web App)**: The sensing server already serves a web UI. Adding a PWA manifest with offline caching makes it installable on any Android device. No native app needed.
2. **Native Android app** (future): Only if PWA proves insufficient. Would use Kotlin + Jetpack Compose, consuming the existing REST API and WebSocket endpoints.
## Deployment Architecture
### Single-Room Deployment (Phase 1)
```
┌──────────────────────────────────────────────────────────────┐
│ Room │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ ESP32-S3 │ │ ESP32-S3 │ │ ESP32-S3 │ CSI sensor mesh │
│ │ Node 1 │ │ Node 2 │ │ Node 3 │ ($10 each) │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │
│ └──────────────┼──────────────┘ │
│ │ UDP port 5005 │
│ ▼ │
│ ┌──────────────────────────────────────┐ │
│ │ Android TV Box (Armbian) │ │
│ │ │ │
│ │ ┌──────────────────────────────┐ │ │
│ │ │ wifi-densepose-sensing- │ │ │
│ │ │ server (aarch64 binary) │ │ │
│ │ │ │ │ │
│ │ │ • CSI ingestion (UDP) │ │ │
│ │ │ • Feature extraction │ │ │
│ │ │ • NN inference (CPU) │ │ │
│ │ │ • WebSocket streaming │ │ │
│ │ │ • REST API │ │ │
│ │ │ • Web UI (:3000) │ │ │
│ │ └──────────────────────────────┘ │ │
│ │ │ │
│ │ ┌──────────────────────────────┐ │ │
│ │ │ Chromium Kiosk Mode │───│──→ HDMI out │
│ │ │ (localhost:3000) │ │ to display │
│ │ └──────────────────────────────┘ │ │
│ │ │ │
│ │ Cost: $25-35 │ │
│ │ Power: 5-10W (USB-C or barrel) │ │
│ │ Form: fits behind TV/monitor │ │
│ └──────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────┘
Total system cost: $55-65 (3 ESP32 nodes + 1 TV box)
```
### Multi-Room Deployment
```
┌──────────────┐
│ Router │
│ (WiFi AP) │
└──────┬───────┘
│ LAN
┌──────────────┼──────────────┐
│ │ │
┌───────▼───────┐ ┌───▼────────┐ ┌──▼──────────┐
│ Room A │ │ Room B │ │ Room C │
│ TV Box + │ │ TV Box + │ │ TV Box + │
│ 3x ESP32 │ │ 3x ESP32 │ │ 3x ESP32 │
│ HDMI display │ │ HDMI │ │ HDMI │
└───────────────┘ └────────────┘ └─────────────┘
Each room: self-contained sensing + display
Central dashboard: aggregate all rooms via REST API
```
### Standalone Mode (Phase 2 — Custom WiFi FW)
```
┌──────────────────────────────────────┐
│ Android TV Box (Armbian) │
│ │
│ ┌────────────────────┐ │
│ │ Patched WiFi │ │
│ │ Driver │ │
│ │ (CSI extraction) │ │
│ └─────────┬──────────┘ │
│ │ CSI frames │
│ ▼ │
│ ┌────────────────────┐ │
│ │ sensing-server │──→ HDMI out │
│ │ (inference + │ │
│ │ dashboard) │ │
│ └────────────────────┘ │
│ │
│ Single device: $25 │
│ No ESP32 nodes needed │
└──────────────────────────────────────┘
```
## Consequences
### Positive
- **10x cost reduction** for aggregator: $25 TV box vs $300+ laptop/PC
- **Always-on deployment**: Mains-powered, fanless, designed for 24/7 operation
- **HDMI output**: Direct connection to TV/monitor for wall-mounted dashboards
- **Familiar hardware**: Available globally, no specialized ordering required
- **Armbian ecosystem**: Mature Debian-based distro with package management, systemd, SSH
- **Path to standalone**: Custom WiFi firmware could eliminate ESP32 dependency entirely
- **PWA for mobile**: No native app development needed for mobile monitoring
- **Multi-room scaling**: One TV box per room, each self-contained
### Negative
- **ARM cross-compilation**: Adds CI complexity; `candle`/ONNX Runtime ARM builds need testing
- **Armbian compatibility**: Not all TV boxes are well-supported; need a tested device list
- **Performance uncertainty**: ARM A53 cores are ~3-5x slower than x86 for NN inference; may need model quantization (INT8) for real-time operation
- **Phase 2 risk**: Custom WiFi firmware is chipset-specific, may require kernel patches per driver version, and CSI quality varies by chipset
- **Support burden**: Different hardware = more configurations to support
- **No GPU**: TV boxes lack discrete GPU; inference is CPU-only (but our models are small enough)
### Neutral
- **No changes to existing ESP32 firmware** — TV box receives the same UDP frames
- **No changes to sensing server protocol** — Phase 2 CSI output uses same binary format
- **Existing web UI works as-is** — Chromium kiosk mode or any browser on the LAN
## Implementation Plan
### Phase 1 (2-3 weeks)
1. Add `aarch64-unknown-linux-gnu` cross-compilation target using `cross`
2. Build and test sensing-server binary on reference TV box (T95 Max+ / S905X3)
3. Create systemd service + Armbian deployment script
4. Benchmark: measure inference latency, memory usage, thermal throttling
5. Create `docs/deployment/armbian-tv-box.md` setup guide
6. Add HDMI kiosk mode configuration (Chromium autostart)
### Phase 2 (4-8 weeks, R&D)
1. Acquire TV box with BCM43455 (proven Nexmon CSI support)
2. Build Armbian with Nexmon CSI patches for BCM43455
3. Write userspace CSI reader → ESP32 binary protocol converter
4. Test CSI quality comparison: ESP32 vs BCM43455
5. If viable: add RTL8822CS CSI extraction via rtw88 driver modification
### Phase 3 (1 week)
1. Add PWA manifest to sensing server web UI
2. Test on Android Chrome, iOS Safari
3. Add service worker for offline dashboard caching
## References
- [Nexmon CSI](https://github.com/seemoo-lab/nexmon_csi) — Broadcom WiFi CSI extraction (BCM43455, BCM4339, BCM4358)
- [Armbian](https://www.armbian.com/) — Debian/Ubuntu for ARM SBCs and TV boxes
- [rtw88 driver](https://github.com/torvalds/linux/tree/master/drivers/net/wireless/realtek/rtw88) — Mainline Linux driver for Realtek 802.11ac chips
- [mt76 driver](https://github.com/torvalds/linux/tree/master/drivers/net/wireless/mediatek/mt76) — Mainline Linux driver for MediaTek WiFi chips
- [cross](https://github.com/cross-rs/cross) — Zero-setup Rust cross-compilation
- [ADR-018: ESP32 CSI Binary Protocol](ADR-018-dev-implementation.md) — Binary frame format reused for Phase 2 CSI extraction
- [ADR-039: Edge Intelligence](ADR-039-esp32-edge-intelligence.md) — On-device processing tiers
- [ADR-043: Sensing Server](ADR-043-sensing-server-ui-api-completion.md) — Single-binary deployment target

152
docs/adr/ADR-047-psychohistory-observatory-visualization.md Normal file
View File

@ -0,0 +1,152 @@
# ADR-047: RuView Observatory — Immersive Three.js WiFi Sensing Visualization
## Status
Accepted (Implemented)
## Date
2026-03-04
## Context
The project has a functional tabbed dashboard UI (`ui/index.html`) with existing Three.js components (body model, gaussian splats, signal visualization, environment). While effective for monitoring, it lacks a cinematic, immersive visualization suitable for demonstrations and stakeholder presentations.
We need an immersive Three.js room-based visualization with practical WiFi sensing data overlays — human wireframe pose, dot-matrix body mass, vital signs HUD, signal field heatmap — powered by ESP32 CSI data (demo mode with live WebSocket path).
## Decision
### Standalone Page Architecture
`ui/observatory.html` is a standalone full-screen entry point, separate from the tabbed dashboard. Linked via "Observatory" nav tab in `ui/index.html`. No build step — vanilla JS modules with Three.js r160 via CDN importmap.
### Room-Based Visualization
Instead of abstract holographic panels, the observatory renders a practical room scene with:
| Element | Implementation | Data Source |
|---------|---------------|-------------|
| Human wireframe | COCO 17-keypoint skeleton, CylinderGeometry tube bones, SphereGeometry joints with glow halos | `persons[].position`, `vital_signs.breathing_rate_bpm` |
| Dot-matrix mist | 800 Points with per-particle alpha ShaderMaterial, body-shaped distribution | `persons[].position`, `persons[].motion_score` |
| Particle trail | 200 Points with age-based fade, emitted from moving person | `persons[].position`, `persons[].motion_score` |
| Signal field | 400 floor-level Points with green→amber color ramp | `signal_field.values` (20×20 grid) |
| WiFi waves | 5 wireframe SphereGeometry shells, AdditiveBlending, pulsing outward | Always-on animation from router position |
| Router | BoxGeometry body, 3 CylinderGeometry antennas, pulsing LED, PointLight | Static scene element |
| Room | GridHelper floor, BoxGeometry wireframe boundary, reflective MeshStandardMaterial floor, furniture (table, bed) | Static scene element |
### HUD Overlay
Glass-morphism HTML panels overlaid on the 3D canvas:
- **Left panel (Vital Signs):** Heart rate (BPM), respiration (RPM), confidence (%) with animated bars
- **Right panel (WiFi Signal):** RSSI, variance, motion power, person count, 2D RSSI sparkline, presence state badge, fall alert
- **Top-right:** Data source badge (DEMO/LIVE), scenario badge, FPS counter, settings gear
- **Bottom:** Capability bar (Pose Estimation, Vital Monitoring, Presence Detection)
- **Bottom-right:** Keyboard shortcut hints
### Settings Dialog (4 Tabs)
Full customization with localStorage persistence and JSON export:
| Tab | Controls |
|-----|----------|
| **Rendering** | Bloom strength/radius/threshold, exposure, vignette, film grain, chromatic aberration |
| **Wireframe** | Bone thickness, joint size, glow intensity, particle trail, wireframe color, joint color, aura opacity |
| **Scene** | Signal field opacity, WiFi wave intensity, room brightness, floor reflection, FOV, orbit speed, grid toggle, room boundary toggle |
| **Data** | Scenario selector (auto-cycle or fixed), cycle speed, data source (demo/WebSocket), WS URL, reset camera, export settings |
### Demo-First with Live Data Path
Four auto-cycling scenarios (30s default, configurable) with 2s cosine crossfade:
| Scenario | Description |
|----------|-------------|
| `empty_room` | Low variance, no presence, flat amplitude, stable RSSI -45dBm |
| `single_breathing` | 1 person, breathing 16 BPM, HR 72 BPM, sinusoidal subcarrier modulation |
| `two_walking` | 2 persons, high motion, Doppler-like shifts, moving signal field peaks |
| `fall_event` | 2s variance spike at t=5s, then stillness, fall flag, confidence drop |
Data contract matches `SensingUpdate` struct from the Rust sensing server. Live WebSocket connection configurable in settings dialog.
### Post-Processing Pipeline
EffectComposer chain: RenderPass → UnrealBloomPass → custom VignetteShader
- **UnrealBloom:** strength 1.0, radius 0.5, threshold 0.25 (configurable)
- **VignetteShader:** warm shadow shift, edge chromatic aberration, film grain
- **Adaptive quality:** Auto-degrades when FPS < 25, restores when FPS > 55
### RuView Foundation Color Palette
| Role | Color | Hex |
|------|-------|-----|
| Background | Deep dark | `#080c14` |
| Primary wireframe | Green glow | `#00d878` |
| Warm accent | Amber | `#ffb020` |
| Signal | Blue | `#2090ff` |
| Heart / joints | Red | `#ff4060` |
| Alert | Crimson | `#ff3040` |
### Technology Choices
| Decision | Rationale |
|----------|-----------|
| Standalone page vs tab | Full-screen immersion, independent loading |
| Room-based vs abstract panels | Practical spatial context for WiFi sensing data |
| Vanilla JS + CDN, no build step | Matches existing `ui/` pattern, served as static files by Axum |
| Custom ShaderMaterial for mist | Per-particle alpha, body-shaped distribution, AdditiveBlending |
| CylinderGeometry tube bones | Visible at any zoom vs thin Line geometry |
| COCO 17-keypoint skeleton | Standard pose format, 16 bone connections |
| localStorage settings | Persistent customization without server round-trip |
| Adaptive quality | 3 levels, auto-switches based on FPS measurement |
### Keyboard Shortcuts
| Key | Action |
|-----|--------|
| `A` | Toggle autopilot orbit |
| `D` | Cycle demo scenario |
| `F` | Toggle FPS counter |
| `S` | Open/close settings |
| `Space` | Pause/resume data |
## Files
| File | Purpose |
|------|---------|
| `ui/observatory.html` | Full-screen entry point with HUD overlay + settings dialog |
| `ui/observatory/js/main.js` | Scene orchestrator (~1,100 lines): room, wireframe, mist, trails, settings, HUD, animation loop |
| `ui/observatory/js/demo-data.js` | 4 scenarios with cosine crossfade, setScenario/setCycleDuration API |
| `ui/observatory/js/nebula-background.js` | Procedural fBM nebula + star field background sphere |
| `ui/observatory/js/post-processing.js` | EffectComposer: UnrealBloom + VignetteShader (chromatic, grain, warmth) |
| `ui/observatory/css/observatory.css` | Foundation color scheme, glass-morphism panels, settings dialog, responsive |
| `ui/index.html` | Modified: added Observatory nav link |
## Consequences
### Positive
- Standalone page does not affect existing dashboard stability
- Demo-first allows offline presentations without hardware
- Same `SensingUpdate` contract enables seamless live WebSocket switch
- Room-based visualization provides intuitive spatial context for WiFi sensing
- Dot-matrix mist gives visual body mass without occluding wireframe
- Full settings customization without code changes (localStorage + JSON export)
- Adaptive quality ensures usability on weaker hardware
- ~20 draw calls keeps performance well within budget
### Negative
- Additional static files served by Axum (minimal overhead)
- Three.js r160 loaded from CDN (no build step, matches existing pattern)
- Settings persistence is per-browser (localStorage, not synced)
### Risks
- CDN dependency for Three.js (mitigated: can vendor locally if needed)
- Post-processing may not work on very old GPUs (mitigated: adaptive quality disables bloom)
## References
- ADR-045: AMOLED display support
- ADR-046: Android TV / Armbian deployment
- Existing `ui/components/scene.js` — Three.js scene pattern
- Existing `ui/components/gaussian-splats.js` — ShaderMaterial pattern
- Existing `ui/services/sensing.service.js` — WebSocket data contract

648
docs/ddd/deployment-platform-domain-model.md Normal file
View File

@ -0,0 +1,648 @@
# Deployment Platform Domain Model
The Deployment Platform domain covers everything from cross-compiling the sensing server for ARM targets to managing TV box appliances running Armbian: provisioning devices, deploying binaries, configuring kiosk displays, and coordinating multi-room installations. It bridges the gap between the Sensing Server domain (which produces the binary) and the physical hardware it runs on.
This document defines the system using [Domain-Driven Design](https://martinfowler.com/bliki/DomainDrivenDesign.html) (DDD): bounded contexts that own their data and rules, aggregate roots that enforce invariants, value objects that carry meaning, and domain events that connect everything.
**Bounded Contexts:**
| # | Context | Responsibility | Key ADRs | Code |
|---|---------|----------------|----------|------|
| 1 | [Appliance Management](#1-appliance-management-context) | Device inventory, provisioning, health monitoring, OTA updates for TV box deployments | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md) | `scripts/deploy/`, `config/armbian/` |
| 2 | [Cross-Compilation](#2-cross-compilation-context) | Build pipeline for aarch64, binary packaging, CI/CD release artifacts | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md) | `.github/workflows/`, `Cross.toml` |
| 3 | [Display Kiosk](#3-display-kiosk-context) | HDMI output management, Chromium kiosk mode, screen rotation, auto-start | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md) | `config/armbian/kiosk/` |
| 4 | [WiFi CSI Bridge](#4-wifi-csi-bridge-context) | Custom WiFi driver CSI extraction, protocol translation to ESP32 binary format | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md) | `tools/csi-bridge/` |
| 5 | [Network Topology](#5-network-topology-context) | ESP32 mesh ↔ TV box connectivity, dedicated AP mode, multi-room routing | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md), [ADR-012](../adr/ADR-012-esp32-csi-sensor-mesh.md) | `config/armbian/network/` |
---
## Domain-Driven Design Specification
### Ubiquitous Language
| Term | Definition |
|------|------------|
| **Appliance** | A TV box running Armbian with the sensing server deployed, treated as a managed device in the fleet |
| **Fleet** | The set of all appliances across a multi-room or multi-site installation |
| **Deployment Package** | A self-contained archive containing the sensing-server binary, systemd unit, configuration, and setup script for a target architecture |
| **Kiosk Mode** | Chromium running in full-screen, no-UI mode pointing at `localhost:3000`, auto-started by systemd on HDMI-connected appliances |
| **CSI Bridge** | A userspace daemon that reads CSI data from a patched WiFi driver and re-encodes it as ESP32-compatible UDP frames for the sensing server |
| **Dedicated AP** | An optional `hostapd`-managed WiFi access point on the TV box that creates an isolated network for ESP32 nodes |
| **OTA Update** | Over-the-air binary replacement: download new sensing-server binary, validate checksum, swap via atomic rename, restart service |
| **Reference Device** | A TV box model that has been tested and validated for Armbian + sensing-server deployment (e.g., T95 Max+ / S905X3) |
| **Provisioning** | First-time setup of an appliance: flash Armbian to SD, deploy package, configure WiFi, start services |
| **Health Beacon** | Periodic JSON payload sent by each appliance to a central coordinator (if multi-room) containing uptime, CPU temp, memory usage, inference latency, connected ESP32 count |
---
## Bounded Contexts
### 1. Appliance Management Context
**Responsibility:** Track deployed TV box appliances, provision new devices, monitor health, and coordinate OTA updates across the fleet.
```
+------------------------------------------------------------+
| Appliance Management Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | Device | | Provisioning | |
| | Registry | | Service | |
| | (fleet state) | | (first-time | |
| | | | setup) | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Health Monitor | |
| | (beacon receiver,| |
| | thermal alerts, | |
| | connectivity) | |
| +--------+----------+ |
| v |
| +-------------------+ |
| | OTA Updater | |
| | (binary swap, | |
| | rollback, | |
| | checksum verify)| |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Aggregates:**
```rust
/// Aggregate Root: A managed TV box appliance in the fleet.
/// Identified by MAC address of the primary Ethernet interface.
pub struct Appliance {
/// Unique device identifier (Ethernet MAC address).
pub device_id: DeviceId,
/// Human-readable name (e.g., "living-room", "bedroom-1").
pub name: String,
/// Hardware model (e.g., "T95 Max+ S905X3").
pub hardware_model: HardwareModel,
/// Current deployment state.
pub state: ApplianceState,
/// Installed sensing-server version.
pub server_version: SemanticVersion,
/// Network configuration.
pub network: NetworkConfig,
/// Last received health beacon.
pub last_health: Option<HealthBeacon>,
/// Provisioning timestamp.
pub provisioned_at: DateTime<Utc>,
/// Connected ESP32 node IDs (from last beacon).
pub connected_nodes: Vec<u8>,
}
/// Lifecycle states for an appliance.
pub enum ApplianceState {
/// SD card prepared, not yet booted.
Provisioned,
/// Booted and running, health beacons received.
Online,
/// No health beacon for >5 minutes.
Unreachable,
/// OTA update in progress.
Updating,
/// Manual maintenance / stopped.
Offline,
/// Thermal throttling or hardware issue detected.
Degraded,
}
```
**Value Objects:**
```rust
/// Hardware model specification for a TV box.
pub struct HardwareModel {
/// Marketing name (e.g., "T95 Max+").
pub name: String,
/// SoC identifier (e.g., "Amlogic S905X3").
pub soc: String,
/// WiFi chipset (e.g., "RTL8822CS").
pub wifi_chipset: String,
/// Total RAM in MB.
pub ram_mb: u32,
/// eMMC storage in GB.
pub emmc_gb: u32,
/// Whether CSI bridge is supported for this WiFi chipset.
pub csi_bridge_supported: bool,
/// Armbian device tree name (e.g., "meson-sm1-sei610").
pub armbian_dtb: String,
}
/// Periodic health report from an appliance.
pub struct HealthBeacon {
pub device_id: DeviceId,
pub timestamp: DateTime<Utc>,
pub uptime_secs: u64,
pub cpu_temp_celsius: f32,
pub cpu_usage_percent: f32,
pub memory_used_mb: u32,
pub memory_total_mb: u32,
pub disk_used_percent: f32,
pub inference_latency_ms: f32,
pub connected_esp32_nodes: Vec<u8>,
pub server_version: SemanticVersion,
pub csi_frames_per_sec: f32,
pub websocket_clients: u32,
}
/// Network configuration for an appliance.
pub struct NetworkConfig {
/// Primary IP address (Ethernet or WiFi client).
pub ip_address: IpAddr,
/// Whether the appliance runs a dedicated AP for ESP32 nodes.
pub dedicated_ap: Option<DedicatedApConfig>,
/// UDP port for ESP32 CSI reception.
pub csi_udp_port: u16, // default: 5005
/// HTTP port for sensing server.
pub http_port: u16, // default: 3000
}
/// Configuration for a dedicated WiFi AP hosted by the appliance.
pub struct DedicatedApConfig {
/// SSID for the ESP32 mesh network.
pub ssid: String,
/// WPA2 passphrase.
pub passphrase: String,
/// Channel (1-11 for 2.4 GHz).
pub channel: u8,
/// DHCP range for connected ESP32 nodes.
pub dhcp_range: (IpAddr, IpAddr),
}
/// Unique device identifier (Ethernet MAC).
pub struct DeviceId(pub [u8; 6]);
/// Semantic version for tracking installed software.
pub struct SemanticVersion {
pub major: u16,
pub minor: u16,
pub patch: u16,
pub pre: Option<String>,
}
```
**Domain Services:**
- `ProvisioningService` — Generates Armbian SD card image with pre-configured deployment package, WiFi credentials, and systemd units
- `HealthMonitorService` — Listens for UDP health beacons from fleet appliances, triggers alerts on thermal throttling (>80°C), unreachable (>5 min), or high memory usage (>90%)
- `OtaUpdateService` — Downloads new binary from release URL, verifies SHA-256 checksum, performs atomic swap (`rename(new, current)`), restarts systemd service, rolls back if health beacon fails within 60s
**Invariants:**
- Device ID (MAC address) is immutable after provisioning
- OTA update refuses to proceed if current CPU temperature >75°C (thermal headroom)
- Rollback is automatic if no healthy beacon is received within 60 seconds of restart
- Dedicated AP SSID must not match the upstream WiFi SSID
---
### 2. Cross-Compilation Context
**Responsibility:** Build the sensing-server binary for ARM64 targets, package deployment archives, and manage CI/CD release artifacts.
```
+------------------------------------------------------------+
| Cross-Compilation Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | Cross.toml | | GitHub Actions| |
| | (target cfg) | | CI Matrix | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Build Pipeline | |
| | (cross build | |
| | --target | |
| | aarch64-unknown-| |
| | linux-gnu) | |
| +--------+----------+ |
| v |
| +-------------------+ |
| | Binary Packager | |
| | (strip, compress,|---> .tar.gz artifact |
| | bundle assets, | |
| | systemd units) | |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Value Objects:**
```rust
/// A packaged deployment archive for a target platform.
pub struct DeploymentPackage {
/// Target triple (e.g., "aarch64-unknown-linux-gnu").
pub target: String,
/// Sensing server binary (stripped).
pub binary: PathBuf,
/// Binary size in bytes.
pub binary_size: u64,
/// SHA-256 checksum of the binary.
pub checksum: String,
/// Systemd service unit file.
pub service_unit: String,
/// Static web UI assets directory.
pub ui_assets: PathBuf,
/// Armbian configuration files (kiosk, network, etc.).
pub config_files: Vec<PathBuf>,
/// Setup script (runs on first boot).
pub setup_script: PathBuf,
/// Version being packaged.
pub version: SemanticVersion,
}
/// Build target specification.
pub struct BuildTarget {
/// Rust target triple.
pub triple: String,
/// CPU architecture description.
pub arch: String,
/// Whether NEON SIMD is available.
pub has_neon: bool,
/// Cross-compilation Docker image.
pub cross_image: String,
/// Binary size limit in bytes.
pub size_limit: u64,
}
```
**Supported Targets:**
| Target Triple | Architecture | Use Case | Size Limit |
|---------------|-------------|----------|------------|
| `x86_64-unknown-linux-gnu` | x86-64 | PC/laptop (existing) | 30 MB |
| `aarch64-unknown-linux-gnu` | ARM64 | TV box (Armbian) | 15 MB |
| `armv7-unknown-linux-gnueabihf` | ARMv7 | Older TV boxes (32-bit) | 12 MB |
| `x86_64-pc-windows-msvc` | x86-64 | Windows (existing) | 30 MB |
**Invariants:**
- Stripped binary must be under size limit for target
- SHA-256 checksum is computed and included in every deployment package
- UI assets are embedded in binary via `include_dir!` or bundled alongside
- No native GPU dependencies — CPU-only inference (candle or ONNX Runtime)
---
### 3. Display Kiosk Context
**Responsibility:** Manage HDMI output on TV box appliances, running Chromium in kiosk mode to display the sensing dashboard full-screen on boot.
```
+------------------------------------------------------------+
| Display Kiosk Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | systemd | | Chromium | |
| | autologin + | | Kiosk Launch | |
| | X11/Wayland | | (full-screen, | |
| | session | | no-UI bars) | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Display Manager | |
| | (resolution, | |
| | rotation, | |
| | overscan, | |
| | sleep/wake) | |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Value Objects:**
```rust
/// Display configuration for kiosk mode.
pub struct KioskConfig {
/// URL to display (default: "http://localhost:3000").
pub url: String,
/// Screen rotation in degrees (0, 90, 180, 270).
pub rotation: u16,
/// Whether to hide the mouse cursor.
pub hide_cursor: bool,
/// Auto-refresh interval in seconds (0 = disabled).
pub auto_refresh_secs: u32,
/// Display sleep schedule (e.g., off 23:00-06:00).
pub sleep_schedule: Option<SleepSchedule>,
/// Overscan compensation percentage (0-10).
pub overscan_percent: u8,
}
/// Sleep schedule for display power management.
pub struct SleepSchedule {
/// Time to turn display off (HH:MM local time).
pub sleep_time: String,
/// Time to turn display on (HH:MM local time).
pub wake_time: String,
}
```
**Invariants:**
- Chromium kiosk starts only after sensing-server systemd unit is `active`
- If Chromium crashes, systemd restarts it within 5 seconds (`Restart=always`)
- Display sleep/wake uses CEC commands (HDMI-CEC) to control TV power when available
- No browser UI elements are visible (address bar, scrollbars, etc.)
---
### 4. WiFi CSI Bridge Context
**Responsibility:** Extract CSI data from patched WiFi drivers on the TV box and translate it into ESP32-compatible binary frames for the sensing server. This is the Phase 2 custom firmware path.
```
+------------------------------------------------------------+
| WiFi CSI Bridge Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | Patched WiFi | | CSI Reader | |
| | Driver | | (Netlink / | |
| | (kernel space)| | procfs / | |
| | CSI hooks | | UDP socket) | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Protocol | |
| | Translator | |
| | (chipset CSI → | |
| | ESP32 binary | |
| | 0xC5100001) | |
| +--------+----------+ |
| v |
| +-------------------+ |
| | UDP Sender | |
| | (localhost:5005) |---> sensing-server |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Value Objects:**
```rust
/// Raw CSI extraction from a WiFi chipset.
pub struct ChipsetCsiFrame {
/// Source chipset type.
pub chipset: WifiChipset,
/// Timestamp of extraction (kernel monotonic clock).
pub timestamp_us: u64,
/// Number of subcarriers (varies by chipset and bandwidth).
pub n_subcarriers: u16,
/// Number of spatial streams / antennas.
pub n_streams: u8,
/// Channel frequency in MHz.
pub freq_mhz: u16,
/// Bandwidth (20/40/80/160 MHz).
pub bandwidth_mhz: u16,
/// RSSI in dBm.
pub rssi_dbm: i8,
/// Noise floor estimate in dBm.
pub noise_floor_dbm: i8,
/// Complex CSI values (I/Q pairs) per subcarrier per stream.
pub csi_matrix: Vec<Complex<f32>>,
/// Source MAC address (BSSID of the AP being measured).
pub source_mac: [u8; 6],
}
/// Supported WiFi chipsets for CSI extraction.
pub enum WifiChipset {
/// Broadcom BCM43455 via Nexmon CSI patches.
BroadcomBcm43455,
/// Realtek RTL8822CS via modified rtw88 driver.
RealtekRtl8822cs,
/// MediaTek MT7661 via mt76 driver modification.
MediatekMt7661,
}
/// Translated frame in ESP32 binary protocol (ADR-018).
pub struct Esp32CompatFrame {
/// Magic: 0xC5100001
pub magic: u32,
/// Virtual node ID assigned to this WiFi interface.
pub node_id: u8,
/// Number of antennas / spatial streams.
pub n_antennas: u8,
/// Number of subcarriers (resampled to match ESP32 format).
pub n_subcarriers: u8,
/// Frequency in MHz.
pub freq_mhz: u16,
/// Sequence number (monotonic counter).
pub sequence: u32,
/// RSSI in dBm.
pub rssi: i8,
/// Noise floor in dBm.
pub noise_floor: i8,
/// Amplitude values (extracted from complex CSI).
pub amplitudes: Vec<f32>,
/// Phase values (extracted from complex CSI).
pub phases: Vec<f32>,
}
```
**Domain Services:**
- `CsiExtractionService` — Reads raw CSI from patched driver via Netlink socket (BCM43455), procfs (RTL8822CS), or UDP (MT7661)
- `SubcarrierResamplerService` — Resamples chipset-specific subcarrier counts to match ESP32 format (e.g., 256 → 128 via decimation or interpolation)
- `ProtocolTranslatorService` — Converts `ChipsetCsiFrame` to `Esp32CompatFrame` with ADR-018 binary encoding
- `CalibrationService` — Compensates for chipset-specific phase offsets, antenna spacing, and gain differences relative to ESP32 CSI
**Invariants:**
- Bridge assigns virtual `node_id` in range 200-254 (reserved for non-ESP32 sources) to avoid collision with physical ESP32 node IDs (1-199)
- Subcarrier resampling preserves frequency ordering (lowest to highest)
- Phase values are unwrapped before encoding (continuous, not wrapped to ±π)
- Bridge daemon starts only if a compatible patched driver is detected at boot
---
### 5. Network Topology Context
**Responsibility:** Manage network connectivity between ESP32 sensor nodes and TV box appliances, including optional dedicated AP mode and multi-room routing.
```
+------------------------------------------------------------+
| Network Topology Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | hostapd | | DHCP Server | |
| | (dedicated AP | | (dnsmasq for | |
| | for ESP32 | | ESP32 nodes) | |
| | mesh) | | | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Topology Manager | |
| | (node discovery, | |
| | IP assignment, | |
| | route config) | |
| +--------+----------+ |
| v |
| +-------------------+ |
| | Firewall Rules | |
| | (iptables/nft: | |
| | allow UDP 5005, | |
| | block external | |
| | access to ESP32 | |
| | subnet) | |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Value Objects:**
```rust
/// Network topology for a single-room deployment.
pub struct RoomTopology {
/// Appliance acting as the aggregator.
pub appliance: DeviceId,
/// Whether the appliance runs a dedicated AP.
pub dedicated_ap: bool,
/// Connected ESP32 nodes with their assigned IPs.
pub nodes: Vec<EspNodeConnection>,
/// Upstream network interface (Ethernet or WiFi client).
pub uplink_interface: String,
/// Sensing network interface (dedicated AP or same as uplink).
pub sensing_interface: String,
}
/// An ESP32 node's network connection to the appliance.
pub struct EspNodeConnection {
/// ESP32 node ID (from firmware NVS).
pub node_id: u8,
/// MAC address of the ESP32.
pub mac: [u8; 6],
/// Assigned IP address (via DHCP or static).
pub ip: IpAddr,
/// Last CSI frame received timestamp.
pub last_seen: DateTime<Utc>,
/// Average CSI frames per second from this node.
pub fps: f32,
}
```
**Domain Services:**
- `DedicatedApService` — Configures `hostapd` to create a WPA2 AP on the TV box's WiFi interface, assigns DHCP range via `dnsmasq`, sets up IP forwarding
- `NodeDiscoveryService` — Monitors UDP port 5005 for new ESP32 node IDs, registers them in the topology, alerts on node departure (no frames for >30s)
- `FirewallService` — Configures `nftables`/`iptables` to isolate the ESP32 subnet from the upstream LAN, allowing only UDP 5005 inbound and HTTP 3000 outbound
**Invariants:**
- Dedicated AP uses a separate WiFi interface or virtual interface (not the uplink)
- ESP32 subnet is isolated from upstream LAN by default (firewall rules)
- If dedicated AP is disabled, ESP32 nodes must be on the same LAN subnet as the appliance
- Node discovery does not require mDNS or any discovery protocol — ESP32 nodes are configured with the appliance's IP via NVS provisioning (ADR-044)
---
## Domain Events
| Event | Published By | Consumed By | Payload |
|-------|-------------|-------------|---------|
| `ApplianceProvisioned` | Appliance Mgmt | Fleet Dashboard | `{ device_id, name, hardware_model, ip }` |
| `ApplianceOnline` | Appliance Mgmt | Fleet Dashboard | `{ device_id, server_version, uptime }` |
| `ApplianceUnreachable` | Appliance Mgmt | Fleet Dashboard, Alerting | `{ device_id, last_seen, reason }` |
| `ApplianceDegraded` | Appliance Mgmt | Fleet Dashboard, Alerting | `{ device_id, cpu_temp, reason }` |
| `OtaUpdateStarted` | Appliance Mgmt | Fleet Dashboard | `{ device_id, from_version, to_version }` |
| `OtaUpdateCompleted` | Appliance Mgmt | Fleet Dashboard | `{ device_id, new_version, duration_secs }` |
| `OtaUpdateRolledBack` | Appliance Mgmt | Fleet Dashboard, Alerting | `{ device_id, attempted_version, rollback_version, reason }` |
| `BinaryBuilt` | Cross-Compilation | Release Pipeline | `{ target, version, binary_size, checksum }` |
| `DeploymentPackageCreated` | Cross-Compilation | Appliance Mgmt | `{ target, version, package_url }` |
| `KioskStarted` | Display Kiosk | Appliance Mgmt | `{ device_id, url, resolution }` |
| `KioskCrashed` | Display Kiosk | Appliance Mgmt | `{ device_id, exit_code, restart_count }` |
| `CsiBridgeStarted` | WiFi CSI Bridge | Appliance Mgmt, Sensing Server | `{ device_id, chipset, virtual_node_id }` |
| `CsiBridgeFailed` | WiFi CSI Bridge | Appliance Mgmt | `{ device_id, chipset, error }` |
| `EspNodeDiscovered` | Network Topology | Appliance Mgmt | `{ appliance_id, node_id, mac, ip }` |
| `EspNodeLost` | Network Topology | Appliance Mgmt, Alerting | `{ appliance_id, node_id, last_seen }` |
| `DedicatedApStarted` | Network Topology | Appliance Mgmt | `{ appliance_id, ssid, channel }` |
---
## Context Map
```
+-------------------+ +---------------------+
| Appliance |--------->| Fleet Dashboard |
| Management | events | (external UI for |
| (fleet state) | -------> | multi-room mgmt) |
+--------+----------+ +---------------------+
|
| provisions, monitors
v
+-------------------+ +---------------------+
| Cross-Compilation |--------->| GitHub Releases |
| (build pipeline) | uploads | (binary artifacts) |
+-------------------+ +---------------------+
|
| provides binary
v
+-------------------+ +---------------------+
| Display Kiosk |--------->| Sensing Server |
| (Chromium on | loads | (upstream domain, |
| HDMI output) | UI from | produces web UI) |
+-------------------+ +----------+----------+
^
+-------------------+ |
| WiFi CSI Bridge |-----UDP 5005------>|
| (patched driver) | ESP32 compat |
+-------------------+ frames |
|
+-------------------+ |
| Network Topology |-----UDP 5005------>|
| (ESP32 mesh | ESP32 frames |
| connectivity) | |
+-------------------+ |
```
**Relationships:**
| Upstream | Downstream | Relationship | Mechanism |
|----------|-----------|--------------|-----------|
| Cross-Compilation | Appliance Mgmt | Supplier-Consumer | Build produces binary; Appliance Mgmt deploys it |
| Appliance Mgmt | Display Kiosk | Customer-Supplier | Appliance Mgmt starts kiosk after server is healthy |
| WiFi CSI Bridge | Sensing Server (external) | Conformist | Bridge adapts its output to match ESP32 binary protocol (ADR-018) |
| Network Topology | Sensing Server (external) | Shared Kernel | Both depend on UDP port 5005 and ESP32 node ID scheme |
| Appliance Mgmt | Network Topology | Customer-Supplier | Appliance config determines whether dedicated AP is enabled |
---
## Anti-Corruption Layers
### ESP32 Protocol ACL (CSI Bridge)
The WiFi CSI Bridge translates chipset-specific CSI formats (Nexmon, rtw88, mt76) into the ESP32 binary protocol (ADR-018). The sensing server never knows whether frames came from a real ESP32 or a TV box WiFi chipset. Virtual node IDs (200-254) prevent collision with physical ESP32 IDs but are otherwise treated identically by the ingestion context.
### Armbian Platform ACL
Appliance Management abstracts over Armbian specifics (device tree names, boot configuration, dtb overlays) through the `HardwareModel` value object. Higher-level contexts (Cross-Compilation, Display Kiosk) depend only on the target triple (`aarch64-unknown-linux-gnu`) and systemd service interface, not on Amlogic/Allwinner/Rockchip kernel specifics.
### Fleet Coordination ACL
For multi-room deployments, each appliance is self-contained (runs its own sensing server, display, and network). The fleet dashboard reads health beacons but never controls individual appliances directly. OTA updates are pulled by each appliance (not pushed), maintaining the appliance as the authority over its own state.
---
## Related
- [ADR-046: Android TV Box / Armbian Deployment](../adr/ADR-046-android-tv-box-armbian-deployment.md) — Primary architectural decision
- [ADR-012: ESP32 CSI Sensor Mesh](../adr/ADR-012-esp32-csi-sensor-mesh.md) — ESP32 mesh network design
- [ADR-018: Dev Implementation](../adr/ADR-018-dev-implementation.md) — ESP32 binary CSI protocol
- [ADR-039: Edge Intelligence](../adr/ADR-039-esp32-edge-intelligence.md) — On-device processing tiers
- [ADR-044: Provisioning Tool](../adr/ADR-044-provisioning-tool-enhancements.md) — NVS provisioning for ESP32 nodes
- [Hardware Platform Domain Model](hardware-platform-domain-model.md) — Upstream domain (ESP32 hardware)
- [Sensing Server Domain Model](sensing-server-domain-model.md) — Upstream domain (server software)

23
firmware/esp32-csi-node/main/CMakeLists.txt
View File

@ -1,6 +1,19 @@
idf_component_register(
SRCS "main.c" "csi_collector.c" "stream_sender.c" "nvs_config.c"
"edge_processing.c" "ota_update.c" "power_mgmt.c"
"wasm_runtime.c" "wasm_upload.c" "rvf_parser.c"
INCLUDE_DIRS "."
set(SRCS
"main.c" "csi_collector.c" "stream_sender.c" "nvs_config.c"
"edge_processing.c" "ota_update.c" "power_mgmt.c"
"wasm_runtime.c" "wasm_upload.c" "rvf_parser.c"
)
set(REQUIRES "")
# ADR-045: AMOLED display support (compile-time optional)
if(CONFIG_DISPLAY_ENABLE)
list(APPEND SRCS "display_hal.c" "display_ui.c" "display_task.c")
set(REQUIRES esp_lcd esp_lcd_touch lvgl)
endif()
idf_component_register(
SRCS ${SRCS}
INCLUDE_DIRS "."
REQUIRES ${REQUIRES}
)

81
firmware/esp32-csi-node/main/Kconfig.projbuild
View File

@ -85,6 +85,87 @@ menu "Edge Intelligence (ADR-039)"
endmenu
menu "AMOLED Display (ADR-045)"
config DISPLAY_ENABLE
bool "Enable AMOLED display support"
default y
help
Enable RM67162 QSPI AMOLED display and LVGL UI.
Auto-detects at boot; gracefully skips if no display hardware.
Requires SPIRAM for frame buffers.
config DISPLAY_FPS_LIMIT
int "Display refresh rate limit (FPS)"
default 30
range 10 60
depends on DISPLAY_ENABLE
help
Maximum display refresh rate. Lower values save CPU.
config DISPLAY_BRIGHTNESS
int "Default backlight brightness (%)"
default 80
range 0 100
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_CS
int "QSPI CS GPIO"
default 6
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_CLK
int "QSPI CLK GPIO"
default 47
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_D0
int "QSPI D0 GPIO"
default 18
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_D1
int "QSPI D1 GPIO"
default 7
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_D2
int "QSPI D2 GPIO"
default 48
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_D3
int "QSPI D3 GPIO"
default 5
depends on DISPLAY_ENABLE
config DISPLAY_TOUCH_SDA
int "Touch I2C SDA GPIO"
default 3
depends on DISPLAY_ENABLE
config DISPLAY_TOUCH_SCL
int "Touch I2C SCL GPIO"
default 2
depends on DISPLAY_ENABLE
config DISPLAY_TOUCH_INT
int "Touch INT GPIO"
default 21
depends on DISPLAY_ENABLE
config DISPLAY_TOUCH_RST
int "Touch RST GPIO"
default 17
depends on DISPLAY_ENABLE
config DISPLAY_BL_PIN
int "Backlight PWM GPIO"
default 38
depends on DISPLAY_ENABLE
endmenu
menu "WASM Programmable Sensing (ADR-040)"
config WASM_ENABLE

382
firmware/esp32-csi-node/main/display_hal.c Normal file
View File

@ -0,0 +1,382 @@
/**
* @file display_hal.c
* @brief ADR-045: SH8601 QSPI AMOLED HAL for Waveshare ESP32-S3-Touch-AMOLED-1.8.
*
* Uses ESP-IDF esp_lcd_panel_io_spi in QSPI mode (quad_mode=true, lcd_cmd_bits=32).
* The panel_io layer handles the 0x02/0x32 QSPI command encoding.
*
* Hardware: SH8601 368x448, FT3168 touch, TCA9554 I/O expander for power/reset.
*
* Pin assignments (Waveshare ESP32-S3-Touch-AMOLED-1.8):
* QSPI: CS=12, CLK=11, D0=4, D1=5, D2=6, D3=7
* I2C: SDA=15, SCL=14 (shared: touch FT3168 + TCA9554 expander)
* Touch INT=21
*/
#include "display_hal.h"
#include "sdkconfig.h"
#if CONFIG_DISPLAY_ENABLE
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_lcd_panel_io.h"
#include "driver/spi_master.h"
#include "driver/gpio.h"
#include "driver/i2c.h"
#include "esp_heap_caps.h"
static const char *TAG = "disp_hal";
/* ---- QSPI Pin Definitions (Waveshare board) ---- */
#define DISP_QSPI_CS 12
#define DISP_QSPI_CLK 11
#define DISP_QSPI_D0 4
#define DISP_QSPI_D1 5
#define DISP_QSPI_D2 6
#define DISP_QSPI_D3 7
/* ---- I2C (shared: touch + TCA9554 expander) ---- */
#define I2C_SDA 15
#define I2C_SCL 14
#define TOUCH_INT_PIN 21
#define I2C_MASTER_NUM I2C_NUM_0
#define I2C_MASTER_FREQ_HZ 400000
/* ---- TCA9554 I/O expander ---- */
#define TCA9554_ADDR 0x20
#define TCA9554_REG_OUTPUT 0x01
#define TCA9554_REG_CONFIG 0x03
/* ---- FT3168 touch controller ---- */
#define FT3168_ADDR 0x38
/* ---- Display dimensions ---- */
#define DISP_H_RES 368
#define DISP_V_RES 448
/* ---- QSPI opcodes (packed into lcd_cmd bits [31:24]) ---- */
#define LCD_OPCODE_WRITE_CMD 0x02
#define LCD_OPCODE_WRITE_COLOR 0x32
/* ---- State ---- */
static esp_lcd_panel_io_handle_t s_io_handle = NULL;
static bool s_i2c_initialized = false;
static bool s_touch_initialized = false;
/* ---- I2C helpers ---- */
static esp_err_t i2c_write_reg(uint8_t dev_addr, uint8_t reg, const uint8_t *data, size_t len)
{
i2c_cmd_handle_t cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (dev_addr << 1) | I2C_MASTER_WRITE, true);
i2c_master_write_byte(cmd, reg, true);
if (data && len > 0) {
i2c_master_write(cmd, data, len, true);
}
i2c_master_stop(cmd);
esp_err_t ret = i2c_master_cmd_begin(I2C_MASTER_NUM, cmd, pdMS_TO_TICKS(100));
i2c_cmd_link_delete(cmd);
return ret;
}
static esp_err_t i2c_read_reg(uint8_t dev_addr, uint8_t reg, uint8_t *data, size_t len)
{
i2c_cmd_handle_t cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (dev_addr << 1) | I2C_MASTER_WRITE, true);
i2c_master_write_byte(cmd, reg, true);
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (dev_addr << 1) | I2C_MASTER_READ, true);
i2c_master_read(cmd, data, len, I2C_MASTER_LAST_NACK);
i2c_master_stop(cmd);
esp_err_t ret = i2c_master_cmd_begin(I2C_MASTER_NUM, cmd, pdMS_TO_TICKS(100));
i2c_cmd_link_delete(cmd);
return ret;
}
static esp_err_t init_i2c_bus(void)
{
if (s_i2c_initialized) return ESP_OK;
i2c_config_t i2c_cfg = {
.mode = I2C_MODE_MASTER,
.sda_io_num = I2C_SDA,
.scl_io_num = I2C_SCL,
.sda_pullup_en = GPIO_PULLUP_ENABLE,
.scl_pullup_en = GPIO_PULLUP_ENABLE,
.master.clk_speed = I2C_MASTER_FREQ_HZ,
};
esp_err_t ret = i2c_param_config(I2C_MASTER_NUM, &i2c_cfg);
if (ret != ESP_OK) return ret;
ret = i2c_driver_install(I2C_MASTER_NUM, I2C_MODE_MASTER, 0, 0, 0);
if (ret != ESP_OK) return ret;
s_i2c_initialized = true;
ESP_LOGI(TAG, "I2C bus init OK (SDA=%d, SCL=%d)", I2C_SDA, I2C_SCL);
return ESP_OK;
}
/* ---- TCA9554 I/O expander: toggle pins for display power/reset ---- */
static esp_err_t tca9554_init_display_power(void)
{
/* Set pins 0, 1, 2 as outputs */
uint8_t cfg = 0xF8;
esp_err_t ret = i2c_write_reg(TCA9554_ADDR, TCA9554_REG_CONFIG, &cfg, 1);
if (ret != ESP_OK) {
ESP_LOGW(TAG, "TCA9554 not found at 0x%02X: %s", TCA9554_ADDR, esp_err_to_name(ret));
return ret;
}
/* Set pins 0,1,2 LOW (reset state) */
uint8_t out = 0x00;
i2c_write_reg(TCA9554_ADDR, TCA9554_REG_OUTPUT, &out, 1);
vTaskDelay(pdMS_TO_TICKS(200));
/* Set pins 0,1,2 HIGH (power on + release reset) */
out = 0x07;
i2c_write_reg(TCA9554_ADDR, TCA9554_REG_OUTPUT, &out, 1);
vTaskDelay(pdMS_TO_TICKS(200));
ESP_LOGI(TAG, "TCA9554 display power/reset toggled");
return ESP_OK;
}
/* ---- Panel IO helpers: send commands via esp_lcd QSPI panel IO ---- */
static esp_err_t panel_write_cmd(uint8_t dcs_cmd, const void *data, size_t data_len)
{
/* Pack as 32-bit lcd_cmd: [31:24]=opcode, [23:8]=dcs_cmd, [7:0]=0 */
uint32_t lcd_cmd = ((uint32_t)LCD_OPCODE_WRITE_CMD << 24) | ((uint32_t)dcs_cmd << 8);
return esp_lcd_panel_io_tx_param(s_io_handle, (int)lcd_cmd, data, data_len);
}
static esp_err_t panel_write_color(const void *color_data, size_t data_len)
{
/* RAMWR (0x2C) packed as 32-bit lcd_cmd with quad opcode */
uint32_t lcd_cmd = ((uint32_t)LCD_OPCODE_WRITE_COLOR << 24) | (0x2C << 8);
return esp_lcd_panel_io_tx_color(s_io_handle, (int)lcd_cmd, color_data, data_len);
}
/* ---- SH8601 init sequence (from Waveshare reference) ---- */
typedef struct {
uint8_t cmd;
uint8_t data[4];
uint8_t data_len;
uint16_t delay_ms;
} sh8601_init_cmd_t;
static const sh8601_init_cmd_t sh8601_init_cmds[] = {
{0x11, {0x00}, 0, 120}, /* Sleep Out + 120ms */
{0x44, {0x01, 0xD1}, 2, 0}, /* Partial area */
{0x35, {0x00}, 1, 0}, /* Tearing Effect ON */
{0x53, {0x20}, 1, 10}, /* Write CTRL Display */
{0x2A, {0x00, 0x00, 0x01, 0x6F}, 4, 0}, /* CASET: 0-367 */
{0x2B, {0x00, 0x00, 0x01, 0xBF}, 4, 0}, /* RASET: 0-447 */
{0x51, {0x00}, 1, 10}, /* Brightness: 0 */
{0x29, {0x00}, 0, 10}, /* Display ON */
{0x51, {0xFF}, 1, 0}, /* Brightness: max */
{0x00, {0x00}, 0xFF, 0}, /* End sentinel */
};
static esp_err_t send_init_sequence(void)
{
for (int i = 0; sh8601_init_cmds[i].data_len != 0xFF; i++) {
const sh8601_init_cmd_t *cmd = &sh8601_init_cmds[i];
esp_err_t ret = panel_write_cmd(
cmd->cmd,
cmd->data_len > 0 ? cmd->data : NULL,
cmd->data_len);
if (ret != ESP_OK) {
ESP_LOGE(TAG, "CMD 0x%02X failed: %s", cmd->cmd, esp_err_to_name(ret));
return ret;
}
if (cmd->delay_ms > 0) {
vTaskDelay(pdMS_TO_TICKS(cmd->delay_ms));
}
}
return ESP_OK;
}
/* ---- Public API ---- */
esp_err_t display_hal_init_panel(void)
{
ESP_LOGI(TAG, "Initializing Waveshare AMOLED 1.8\" (SH8601 368x448)...");
/* Step 1: Init I2C bus */
esp_err_t ret = init_i2c_bus();
if (ret != ESP_OK) {
ESP_LOGW(TAG, "I2C bus init failed");
return ESP_ERR_NOT_FOUND;
}
/* Step 2: TCA9554 display power/reset (optional — only present on Waveshare board) */
ret = tca9554_init_display_power();
if (ret != ESP_OK) {
ESP_LOGW(TAG, "TCA9554 not found — assuming display power is always-on (direct wiring)");
/* Continue without TCA9554 — the display may be powered directly */
}
/* Step 3: Initialize SPI bus */
spi_bus_config_t bus_cfg = {
.sclk_io_num = DISP_QSPI_CLK,
.data0_io_num = DISP_QSPI_D0,
.data1_io_num = DISP_QSPI_D1,
.data2_io_num = DISP_QSPI_D2,
.data3_io_num = DISP_QSPI_D3,
.max_transfer_sz = DISP_H_RES * DISP_V_RES * 2,
};
ret = spi_bus_initialize(SPI2_HOST, &bus_cfg, SPI_DMA_CH_AUTO);
if (ret != ESP_OK) {
ESP_LOGW(TAG, "SPI bus init failed: %s", esp_err_to_name(ret));
return ESP_ERR_NOT_FOUND;
}
/* Step 4: Create panel IO with QSPI mode */
esp_lcd_panel_io_spi_config_t io_config = {
.dc_gpio_num = -1, /* No DC pin in QSPI mode */
.cs_gpio_num = DISP_QSPI_CS,
.pclk_hz = 40 * 1000 * 1000,
.lcd_cmd_bits = 32, /* 32-bit command: [opcode|dcs_cmd|0x00] */
.lcd_param_bits = 8,
.spi_mode = 0,
.trans_queue_depth = 10,
.flags = {
.quad_mode = true,
},
};
ret = esp_lcd_new_panel_io_spi((esp_lcd_spi_bus_handle_t)SPI2_HOST, &io_config, &s_io_handle);
if (ret != ESP_OK) {
ESP_LOGE(TAG, "Panel IO init failed: %s", esp_err_to_name(ret));
spi_bus_free(SPI2_HOST);
return ESP_ERR_NOT_FOUND;
}
ESP_LOGI(TAG, "QSPI panel IO created (40MHz, quad mode)");
/* Step 5: Send SH8601 init sequence */
ret = send_init_sequence();
if (ret != ESP_OK) {
ESP_LOGW(TAG, "SH8601 init sequence failed");
esp_lcd_panel_io_del(s_io_handle);
spi_bus_free(SPI2_HOST);
s_io_handle = NULL;
return ESP_ERR_NOT_FOUND;
}
/* Step 6: Draw test pattern — cyan bar at top */
ESP_LOGI(TAG, "Drawing test pattern...");
uint16_t *line_buf = heap_caps_malloc(DISP_H_RES * 2, MALLOC_CAP_DMA);
if (line_buf) {
uint8_t caset[4] = {0, 0, (DISP_H_RES - 1) >> 8, (DISP_H_RES - 1) & 0xFF};
uint8_t raset[4] = {0, 0, (DISP_V_RES - 1) >> 8, (DISP_V_RES - 1) & 0xFF};
panel_write_cmd(0x2A, caset, 4);
panel_write_cmd(0x2B, raset, 4);
for (int y = 0; y < DISP_V_RES; y++) {
uint16_t color = (y < 30) ? 0x07FF : 0x0841;
for (int x = 0; x < DISP_H_RES; x++) {
line_buf[x] = color;
}
panel_write_color(line_buf, DISP_H_RES * 2);
}
free(line_buf);
ESP_LOGI(TAG, "Test pattern drawn");
}
ESP_LOGI(TAG, "SH8601 panel init OK (%dx%d)", DISP_H_RES, DISP_V_RES);
return ESP_OK;
}
void display_hal_draw(int x_start, int y_start, int x_end, int y_end,
const void *color_data)
{
if (!s_io_handle) return;
/* SH8601 requires coordinates divisible by 2 */
x_start &= ~1;
y_start &= ~1;
if (x_end & 1) x_end++;
if (y_end & 1) y_end++;
if (x_end > DISP_H_RES) x_end = DISP_H_RES;
if (y_end > DISP_V_RES) y_end = DISP_V_RES;
uint8_t caset[4] = {
(x_start >> 8) & 0xFF, x_start & 0xFF,
((x_end - 1) >> 8) & 0xFF, (x_end - 1) & 0xFF,
};
panel_write_cmd(0x2A, caset, 4);
uint8_t raset[4] = {
(y_start >> 8) & 0xFF, y_start & 0xFF,
((y_end - 1) >> 8) & 0xFF, (y_end - 1) & 0xFF,
};
panel_write_cmd(0x2B, raset, 4);
size_t len = (x_end - x_start) * (y_end - y_start) * 2;
panel_write_color(color_data, len);
}
esp_err_t display_hal_init_touch(void)
{
ESP_LOGI(TAG, "Probing FT3168 touch controller...");
if (!s_i2c_initialized) {
esp_err_t ret = init_i2c_bus();
if (ret != ESP_OK) return ESP_ERR_NOT_FOUND;
}
gpio_config_t int_cfg = {
.pin_bit_mask = (1ULL << TOUCH_INT_PIN),
.mode = GPIO_MODE_INPUT,
.pull_up_en = GPIO_PULLUP_ENABLE,
.intr_type = GPIO_INTR_DISABLE,
};
gpio_config(&int_cfg);
uint8_t chip_id = 0;
esp_err_t ret = i2c_read_reg(FT3168_ADDR, 0xA8, &chip_id, 1);
if (ret != ESP_OK || chip_id == 0x00 || chip_id == 0xFF) {
ESP_LOGW(TAG, "FT3168 not found (ret=%s, id=0x%02X)", esp_err_to_name(ret), chip_id);
return ESP_ERR_NOT_FOUND;
}
s_touch_initialized = true;
ESP_LOGI(TAG, "FT3168 touch init OK (chip_id=0x%02X)", chip_id);
return ESP_OK;
}
bool display_hal_touch_read(uint16_t *x, uint16_t *y)
{
if (!s_touch_initialized) return false;
uint8_t buf[7] = {0};
esp_err_t ret = i2c_read_reg(FT3168_ADDR, 0x01, buf, 7);
if (ret != ESP_OK) return false;
uint8_t num_points = buf[1];
if (num_points == 0 || num_points > 2) return false;
*x = ((buf[2] & 0x0F) << 8) | buf[3];
*y = ((buf[4] & 0x0F) << 8) | buf[5];
return true;
}
void display_hal_set_brightness(uint8_t percent)
{
if (!s_io_handle) return;
if (percent > 100) percent = 100;
uint8_t val = (uint8_t)((uint32_t)percent * 255 / 100);
panel_write_cmd(0x51, &val, 1);
}
#endif /* CONFIG_DISPLAY_ENABLE */

71
firmware/esp32-csi-node/main/display_hal.h Normal file
View File

@ -0,0 +1,71 @@
/**
* @file display_hal.h
* @brief ADR-045: RM67162 QSPI AMOLED + CST816S touch HAL.
*
* Hardware abstraction for the LilyGO T-Display-S3 AMOLED panel.
* Probes hardware at boot; returns ESP_ERR_NOT_FOUND if absent.
*/
#ifndef DISPLAY_HAL_H
#define DISPLAY_HAL_H
#include <stdbool.h>
#include <stdint.h>
#include "esp_err.h"
#ifdef __cplusplus
extern "C" {
#endif
/**
* Probe and initialize the RM67162 QSPI AMOLED panel.
*
* Configures QSPI bus, sends panel init sequence, and fills
* the screen with dark background to confirm it works.
* Returns ESP_ERR_NOT_FOUND if the panel does not respond.
*
* @return ESP_OK on success, ESP_ERR_NOT_FOUND if no display detected.
*/
esp_err_t display_hal_init_panel(void);
/**
* Draw a rectangle of pixels to the AMOLED.
* Sends CASET + RASET + RAMWR directly via QSPI.
*
* @param x_start Left column (inclusive).
* @param y_start Top row (inclusive).
* @param x_end Right column (exclusive).
* @param y_end Bottom row (exclusive).
* @param color_data RGB565 pixel data, (x_end-x_start)*(y_end-y_start) pixels.
*/
void display_hal_draw(int x_start, int y_start, int x_end, int y_end,
const void *color_data);
/**
* Probe and initialize the CST816S capacitive touch controller.
*
* @return ESP_OK on success, ESP_ERR_NOT_FOUND if no touch IC detected.
*/
esp_err_t display_hal_init_touch(void);
/**
* Read touch point (non-blocking).
*
* @param[out] x Touch X coordinate (0..535).
* @param[out] y Touch Y coordinate (0..239).
* @return true if touch is active, false if released.
*/
bool display_hal_touch_read(uint16_t *x, uint16_t *y);
/**
* Set AMOLED brightness via MIPI DCS command.
*
* @param percent Brightness 0-100.
*/
void display_hal_set_brightness(uint8_t percent);
#ifdef __cplusplus
}
#endif
#endif /* DISPLAY_HAL_H */

169
firmware/esp32-csi-node/main/display_task.c Normal file
View File

@ -0,0 +1,169 @@
/**
* @file display_task.c
* @brief ADR-045: FreeRTOS display task — LVGL pump on Core 0, priority 1.
*
* Gracefully skips if RM67162 panel or SPIRAM is absent.
* Reads from edge_get_vitals() / edge_get_multi_person() (thread-safe).
*/
#include "display_task.h"
#include "sdkconfig.h"
#if CONFIG_DISPLAY_ENABLE
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_heap_caps.h"
#include "lvgl.h"
#include "display_hal.h"
#include "display_ui.h"
#define DISP_H_RES 368
#define DISP_V_RES 448
static const char *TAG = "disp_task";
/* ---- Config ---- */
#ifdef CONFIG_DISPLAY_FPS_LIMIT
#define DISP_FPS_LIMIT CONFIG_DISPLAY_FPS_LIMIT
#else
#define DISP_FPS_LIMIT 30
#endif
#define DISP_TASK_STACK (8 * 1024)
#define DISP_TASK_PRIORITY 1
#define DISP_TASK_CORE 0
#define DISP_BUF_LINES 40
/* ---- LVGL flush callback — calls display_hal_draw directly ---- */
static void lvgl_flush_cb(lv_disp_drv_t *drv, const lv_area_t *area, lv_color_t *color_p)
{
display_hal_draw(area->x1, area->y1, area->x2 + 1, area->y2 + 1, color_p);
lv_disp_flush_ready(drv);
}
/* ---- LVGL touch input callback ---- */
static void lvgl_touch_cb(lv_indev_drv_t *drv, lv_indev_data_t *data)
{
uint16_t x, y;
if (display_hal_touch_read(&x, &y)) {
data->point.x = x;
data->point.y = y;
data->state = LV_INDEV_STATE_PRESSED;
} else {
data->state = LV_INDEV_STATE_RELEASED;
}
}
/* ---- Display task ---- */
static void display_task(void *arg)
{
const TickType_t frame_period = pdMS_TO_TICKS(1000 / DISP_FPS_LIMIT);
ESP_LOGI(TAG, "Display task running on Core %d, %d fps limit",
xPortGetCoreID(), DISP_FPS_LIMIT);
display_ui_create(lv_scr_act());
TickType_t last_wake = xTaskGetTickCount();
while (1) {
display_ui_update();
lv_timer_handler();
vTaskDelayUntil(&last_wake, frame_period);
}
}
/* ---- Public API ---- */
esp_err_t display_task_start(void)
{
ESP_LOGI(TAG, "Initializing display subsystem...");
bool use_psram = false;
#if CONFIG_SPIRAM
size_t psram_free = heap_caps_get_free_size(MALLOC_CAP_SPIRAM);
if (psram_free >= 64 * 1024) {
use_psram = true;
ESP_LOGI(TAG, "PSRAM available: %u KB — using PSRAM buffers", (unsigned)(psram_free / 1024));
} else {
ESP_LOGW(TAG, "PSRAM too small (%u bytes) — falling back to internal DMA memory", (unsigned)psram_free);
}
#else
ESP_LOGW(TAG, "SPIRAM not enabled — using internal DMA memory (smaller buffers)");
#endif
/* Probe display hardware */
esp_err_t ret = display_hal_init_panel();
if (ret != ESP_OK) {
ESP_LOGW(TAG, "Display not available — running headless");
return ESP_OK;
}
/* Init touch (optional) */
esp_err_t touch_ret = display_hal_init_touch();
/* Initialize LVGL */
lv_init();
/* Double-buffered draw buffers — prefer PSRAM, fall back to internal DMA */
size_t buf_lines = use_psram ? DISP_BUF_LINES : 10; /* Smaller buffers without PSRAM */
size_t buf_size = DISP_H_RES * buf_lines * sizeof(lv_color_t);
uint32_t alloc_caps = use_psram ? MALLOC_CAP_SPIRAM : (MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL);
lv_color_t *buf1 = heap_caps_malloc(buf_size, alloc_caps);
lv_color_t *buf2 = heap_caps_malloc(buf_size, alloc_caps);
if (!buf1 || !buf2) {
ESP_LOGE(TAG, "Failed to allocate LVGL buffers (%u bytes, caps=0x%lx)",
(unsigned)buf_size, (unsigned long)alloc_caps);
if (buf1) free(buf1);
if (buf2) free(buf2);
return ESP_OK;
}
ESP_LOGI(TAG, "LVGL buffers: 2x %u bytes (%u lines, %s)",
(unsigned)buf_size, (unsigned)buf_lines, use_psram ? "PSRAM" : "internal DMA");
static lv_disp_draw_buf_t draw_buf;
lv_disp_draw_buf_init(&draw_buf, buf1, buf2, DISP_H_RES * buf_lines);
static lv_disp_drv_t disp_drv;
lv_disp_drv_init(&disp_drv);
disp_drv.hor_res = DISP_H_RES;
disp_drv.ver_res = DISP_V_RES;
disp_drv.flush_cb = lvgl_flush_cb;
disp_drv.draw_buf = &draw_buf;
lv_disp_drv_register(&disp_drv);
if (touch_ret == ESP_OK) {
static lv_indev_drv_t indev_drv;
lv_indev_drv_init(&indev_drv);
indev_drv.type = LV_INDEV_TYPE_POINTER;
indev_drv.read_cb = lvgl_touch_cb;
lv_indev_drv_register(&indev_drv);
ESP_LOGI(TAG, "Touch input registered");
}
BaseType_t xret = xTaskCreatePinnedToCore(
display_task, "display", DISP_TASK_STACK,
NULL, DISP_TASK_PRIORITY, NULL, DISP_TASK_CORE);
if (xret != pdPASS) {
ESP_LOGE(TAG, "Failed to create display task");
return ESP_OK;
}
ESP_LOGI(TAG, "Display task started (Core %d, priority %d, %d fps)",
DISP_TASK_CORE, DISP_TASK_PRIORITY, DISP_FPS_LIMIT);
return ESP_OK;
}
#else /* !CONFIG_DISPLAY_ENABLE */
esp_err_t display_task_start(void)
{
return ESP_OK;
}
#endif /* CONFIG_DISPLAY_ENABLE */

29
firmware/esp32-csi-node/main/display_task.h Normal file
View File

@ -0,0 +1,29 @@
/**
* @file display_task.h
* @brief ADR-045: FreeRTOS display task — LVGL pump on Core 0.
*/
#ifndef DISPLAY_TASK_H
#define DISPLAY_TASK_H
#include "esp_err.h"
#ifdef __cplusplus
extern "C" {
#endif
/**
* Start the display task on Core 0, priority 1.
*
* Probes for RM67162 panel and SPIRAM. If either is absent,
* logs a warning and returns ESP_OK (graceful skip).
*
* @return ESP_OK always (display is optional).
*/
esp_err_t display_task_start(void);
#ifdef __cplusplus
}
#endif
#endif /* DISPLAY_TASK_H */

387
firmware/esp32-csi-node/main/display_ui.c Normal file
View File

@ -0,0 +1,387 @@
/**
* @file display_ui.c
* @brief ADR-045: LVGL 4-view swipeable UI — Dashboard | Vitals | Presence | System.
*
* Dark theme (#0a0a0f background) with cyan (#00d4ff) accent.
* Glowing line effects via layered semi-transparent chart series.
*/
#include "display_ui.h"
#include "sdkconfig.h"
#if CONFIG_DISPLAY_ENABLE
#include <stdio.h>
#include <string.h>
#include "esp_log.h"
#include "esp_system.h"
#include "esp_timer.h"
#include "esp_heap_caps.h"
#include "edge_processing.h"
static const char *TAG = "disp_ui";
/* ---- Theme colors ---- */
#define COLOR_BG lv_color_make(0x0A, 0x0A, 0x0F)
#define COLOR_CYAN lv_color_make(0x00, 0xD4, 0xFF)
#define COLOR_AMBER lv_color_make(0xFF, 0xB0, 0x00)
#define COLOR_GREEN lv_color_make(0x00, 0xFF, 0x80)
#define COLOR_RED lv_color_make(0xFF, 0x40, 0x40)
#define COLOR_DIM lv_color_make(0x30, 0x30, 0x40)
#define COLOR_TEXT lv_color_make(0xCC, 0xCC, 0xDD)
#define COLOR_TEXT_DIM lv_color_make(0x66, 0x66, 0x77)
/* ---- Chart data points ---- */
#define CHART_POINTS 60
/* ---- View handles ---- */
static lv_obj_t *s_tileview = NULL;
/* Dashboard */
static lv_obj_t *s_dash_chart = NULL;
static lv_chart_series_t *s_csi_series = NULL;
static lv_obj_t *s_dash_persons = NULL;
static lv_obj_t *s_dash_rssi = NULL;
static lv_obj_t *s_dash_motion = NULL;
/* Vitals */
static lv_obj_t *s_vital_chart = NULL;
static lv_chart_series_t *s_breath_series = NULL;
static lv_chart_series_t *s_hr_series = NULL;
static lv_obj_t *s_vital_bpm_br = NULL;
static lv_obj_t *s_vital_bpm_hr = NULL;
/* Presence */
#define GRID_COLS 4
#define GRID_ROWS 4
static lv_obj_t *s_grid_cells[GRID_COLS * GRID_ROWS];
static lv_obj_t *s_presence_label = NULL;
/* System */
static lv_obj_t *s_sys_cpu = NULL;
static lv_obj_t *s_sys_heap = NULL;
static lv_obj_t *s_sys_psram = NULL;
static lv_obj_t *s_sys_rssi = NULL;
static lv_obj_t *s_sys_uptime = NULL;
static lv_obj_t *s_sys_fps = NULL;
static lv_obj_t *s_sys_node = NULL;
/* ---- Style helpers ---- */
static lv_style_t s_style_bg;
static lv_style_t s_style_label;
static lv_style_t s_style_label_big;
static bool s_styles_inited = false;
static void init_styles(void)
{
if (s_styles_inited) return;
s_styles_inited = true;
lv_style_init(&s_style_bg);
lv_style_set_bg_color(&s_style_bg, COLOR_BG);
lv_style_set_bg_opa(&s_style_bg, LV_OPA_COVER);
lv_style_set_border_width(&s_style_bg, 0);
lv_style_set_pad_all(&s_style_bg, 4);
lv_style_init(&s_style_label);
lv_style_set_text_color(&s_style_label, COLOR_TEXT);
lv_style_set_text_font(&s_style_label, &lv_font_montserrat_14);
lv_style_init(&s_style_label_big);
lv_style_set_text_color(&s_style_label_big, COLOR_CYAN);
lv_style_set_text_font(&s_style_label_big, &lv_font_montserrat_20);
}
static lv_obj_t *make_label(lv_obj_t *parent, const char *text, const lv_style_t *style)
{
lv_obj_t *lbl = lv_label_create(parent);
lv_label_set_text(lbl, text);
if (style) lv_obj_add_style(lbl, (lv_style_t *)style, 0);
return lbl;
}
static lv_obj_t *make_tile(lv_obj_t *tv, uint8_t col, uint8_t row)
{
lv_obj_t *tile = lv_tileview_add_tile(tv, col, row, LV_DIR_HOR);
lv_obj_add_style(tile, &s_style_bg, 0);
return tile;
}
/* ---- View 0: Dashboard ---- */
static void create_dashboard(lv_obj_t *tile)
{
make_label(tile, "CSI Dashboard", &s_style_label);
/* CSI amplitude chart */
s_dash_chart = lv_chart_create(tile);
lv_obj_set_size(s_dash_chart, 400, 130);
lv_obj_align(s_dash_chart, LV_ALIGN_TOP_LEFT, 0, 24);
lv_chart_set_type(s_dash_chart, LV_CHART_TYPE_LINE);
lv_chart_set_point_count(s_dash_chart, CHART_POINTS);
lv_chart_set_range(s_dash_chart, LV_CHART_AXIS_PRIMARY_Y, 0, 100);
lv_obj_set_style_bg_color(s_dash_chart, COLOR_BG, 0);
lv_obj_set_style_border_color(s_dash_chart, COLOR_DIM, 0);
lv_obj_set_style_line_width(s_dash_chart, 0, LV_PART_TICKS);
s_csi_series = lv_chart_add_series(s_dash_chart, COLOR_CYAN, LV_CHART_AXIS_PRIMARY_Y);
/* Stats panel on the right */
lv_obj_t *panel = lv_obj_create(tile);
lv_obj_set_size(panel, 120, 130);
lv_obj_align(panel, LV_ALIGN_TOP_RIGHT, 0, 24);
lv_obj_set_style_bg_color(panel, lv_color_make(0x12, 0x12, 0x1A), 0);
lv_obj_set_style_border_width(panel, 1, 0);
lv_obj_set_style_border_color(panel, COLOR_DIM, 0);
lv_obj_set_style_pad_all(panel, 8, 0);
lv_obj_set_flex_flow(panel, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(panel, LV_FLEX_ALIGN_SPACE_EVENLY, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START);
make_label(panel, "Persons", &s_style_label);
s_dash_persons = make_label(panel, "0", &s_style_label_big);
s_dash_rssi = make_label(panel, "RSSI: --", &s_style_label);
s_dash_motion = make_label(panel, "Motion: 0.0", &s_style_label);
}
/* ---- View 1: Vitals ---- */
static void create_vitals(lv_obj_t *tile)
{
make_label(tile, "Vital Signs", &s_style_label);
s_vital_chart = lv_chart_create(tile);
lv_obj_set_size(s_vital_chart, 480, 150);
lv_obj_align(s_vital_chart, LV_ALIGN_TOP_LEFT, 0, 24);
lv_chart_set_type(s_vital_chart, LV_CHART_TYPE_LINE);
lv_chart_set_point_count(s_vital_chart, CHART_POINTS);
lv_chart_set_range(s_vital_chart, LV_CHART_AXIS_PRIMARY_Y, 0, 120);
lv_obj_set_style_bg_color(s_vital_chart, COLOR_BG, 0);
lv_obj_set_style_border_color(s_vital_chart, COLOR_DIM, 0);
lv_obj_set_style_line_width(s_vital_chart, 0, LV_PART_TICKS);
/* Breathing series (cyan) */
s_breath_series = lv_chart_add_series(s_vital_chart, COLOR_CYAN, LV_CHART_AXIS_PRIMARY_Y);
/* Heart rate series (amber) */
s_hr_series = lv_chart_add_series(s_vital_chart, COLOR_AMBER, LV_CHART_AXIS_PRIMARY_Y);
/* BPM readouts */
s_vital_bpm_br = make_label(tile, "Breathing: -- BPM", &s_style_label);
lv_obj_align(s_vital_bpm_br, LV_ALIGN_BOTTOM_LEFT, 4, -8);
lv_obj_set_style_text_color(s_vital_bpm_br, COLOR_CYAN, 0);
s_vital_bpm_hr = make_label(tile, "Heart Rate: -- BPM", &s_style_label);
lv_obj_align(s_vital_bpm_hr, LV_ALIGN_BOTTOM_RIGHT, -4, -8);
lv_obj_set_style_text_color(s_vital_bpm_hr, COLOR_AMBER, 0);
}
/* ---- View 2: Presence Grid ---- */
static void create_presence(lv_obj_t *tile)
{
make_label(tile, "Occupancy Map", &s_style_label);
int cell_w = 50;
int cell_h = 45;
int x_off = (368 - GRID_COLS * (cell_w + 4)) / 2;
int y_off = 30;
for (int r = 0; r < GRID_ROWS; r++) {
for (int c = 0; c < GRID_COLS; c++) {
lv_obj_t *cell = lv_obj_create(tile);
lv_obj_set_size(cell, cell_w, cell_h);
lv_obj_set_pos(cell, x_off + c * (cell_w + 4), y_off + r * (cell_h + 4));
lv_obj_set_style_bg_color(cell, COLOR_DIM, 0);
lv_obj_set_style_bg_opa(cell, LV_OPA_COVER, 0);
lv_obj_set_style_border_color(cell, COLOR_DIM, 0);
lv_obj_set_style_border_width(cell, 1, 0);
lv_obj_set_style_radius(cell, 4, 0);
s_grid_cells[r * GRID_COLS + c] = cell;
}
}
s_presence_label = make_label(tile, "Persons: 0", &s_style_label);
lv_obj_align(s_presence_label, LV_ALIGN_BOTTOM_MID, 0, -8);
}
/* ---- View 3: System ---- */
static void create_system(lv_obj_t *tile)
{
make_label(tile, "System Info", &s_style_label);
lv_obj_t *panel = lv_obj_create(tile);
lv_obj_set_size(panel, 500, 180);
lv_obj_align(panel, LV_ALIGN_TOP_LEFT, 0, 24);
lv_obj_set_style_bg_color(panel, lv_color_make(0x12, 0x12, 0x1A), 0);
lv_obj_set_style_border_width(panel, 1, 0);
lv_obj_set_style_border_color(panel, COLOR_DIM, 0);
lv_obj_set_style_pad_all(panel, 10, 0);
lv_obj_set_flex_flow(panel, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(panel, LV_FLEX_ALIGN_SPACE_EVENLY, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START);
s_sys_node = make_label(panel, "Node: --", &s_style_label);
s_sys_cpu = make_label(panel, "CPU: --%", &s_style_label);
s_sys_heap = make_label(panel, "Heap: -- KB free", &s_style_label);
s_sys_psram = make_label(panel, "PSRAM: -- KB free",&s_style_label);
s_sys_rssi = make_label(panel, "WiFi RSSI: --", &s_style_label);
s_sys_uptime = make_label(panel, "Uptime: --", &s_style_label);
s_sys_fps = make_label(panel, "FPS: --", &s_style_label);
}
/* ---- Public API ---- */
void display_ui_create(lv_obj_t *parent)
{
init_styles();
s_tileview = lv_tileview_create(parent);
lv_obj_add_style(s_tileview, &s_style_bg, 0);
lv_obj_set_style_bg_color(s_tileview, COLOR_BG, 0);
lv_obj_t *t0 = make_tile(s_tileview, 0, 0);
lv_obj_t *t1 = make_tile(s_tileview, 1, 0);
lv_obj_t *t2 = make_tile(s_tileview, 2, 0);
lv_obj_t *t3 = make_tile(s_tileview, 3, 0);
create_dashboard(t0);
create_vitals(t1);
create_presence(t2);
create_system(t3);
ESP_LOGI(TAG, "UI created: 4 views (Dashboard|Vitals|Presence|System)");
}
/* ---- FPS tracking ---- */
static uint32_t s_frame_count = 0;
static uint32_t s_last_fps_time = 0;
static uint32_t s_current_fps = 0;
void display_ui_update(void)
{
/* FPS counter */
s_frame_count++;
uint32_t now_ms = (uint32_t)(esp_timer_get_time() / 1000);
if (now_ms - s_last_fps_time >= 1000) {
s_current_fps = s_frame_count;
s_frame_count = 0;
s_last_fps_time = now_ms;
}
/* Read edge data (thread-safe) */
edge_vitals_pkt_t vitals;
bool has_vitals = edge_get_vitals(&vitals);
edge_person_vitals_t persons[EDGE_MAX_PERSONS];
uint8_t n_active = 0;
edge_get_multi_person(persons, &n_active);
/* ---- Dashboard update ---- */
if (s_dash_chart && has_vitals) {
/* Push motion energy as amplitude proxy (scaled 0-100) */
int val = (int)(vitals.motion_energy * 10.0f);
if (val > 100) val = 100;
if (val < 0) val = 0;
lv_chart_set_next_value(s_dash_chart, s_csi_series, val);
}
if (s_dash_persons) {
char buf[8];
snprintf(buf, sizeof(buf), "%u", has_vitals ? vitals.n_persons : 0);
lv_label_set_text(s_dash_persons, buf);
}
if (s_dash_rssi && has_vitals) {
char buf[16];
snprintf(buf, sizeof(buf), "RSSI: %d", vitals.rssi);
lv_label_set_text(s_dash_rssi, buf);
}
if (s_dash_motion && has_vitals) {
char buf[24];
snprintf(buf, sizeof(buf), "Motion: %.1f", (double)vitals.motion_energy);
lv_label_set_text(s_dash_motion, buf);
}
/* ---- Vitals update ---- */
if (s_vital_chart && has_vitals) {
int br = (int)(vitals.breathing_rate / 100); /* Fixed-point to int BPM */
int hr = (int)(vitals.heartrate / 10000);
if (br > 120) br = 120;
if (hr > 120) hr = 120;
lv_chart_set_next_value(s_vital_chart, s_breath_series, br);
lv_chart_set_next_value(s_vital_chart, s_hr_series, hr);
char buf[32];
snprintf(buf, sizeof(buf), "Breathing: %d BPM", br);
lv_label_set_text(s_vital_bpm_br, buf);
snprintf(buf, sizeof(buf), "Heart Rate: %d BPM", hr);
lv_label_set_text(s_vital_bpm_hr, buf);
}
/* ---- Presence grid update ---- */
if (has_vitals) {
/* Simple visualization: color cells based on motion energy distribution */
float energy = vitals.motion_energy;
uint8_t active_cells = (uint8_t)(energy * 2); /* Scale for visibility */
if (active_cells > GRID_COLS * GRID_ROWS) active_cells = GRID_COLS * GRID_ROWS;
for (int i = 0; i < GRID_COLS * GRID_ROWS; i++) {
if (i < active_cells) {
/* Color gradient: green → amber → red based on intensity */
if (energy > 5.0f) {
lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_RED, 0);
} else if (energy > 2.0f) {
lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_AMBER, 0);
} else {
lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_GREEN, 0);
}
} else {
lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_DIM, 0);
}
}
char buf[20];
snprintf(buf, sizeof(buf), "Persons: %u", vitals.n_persons);
lv_label_set_text(s_presence_label, buf);
}
/* ---- System info update ---- */
{
char buf[48];
#ifdef CONFIG_CSI_NODE_ID
snprintf(buf, sizeof(buf), "Node: %d", CONFIG_CSI_NODE_ID);
#else
snprintf(buf, sizeof(buf), "Node: --");
#endif
lv_label_set_text(s_sys_node, buf);
snprintf(buf, sizeof(buf), "Heap: %lu KB free",
(unsigned long)(esp_get_free_heap_size() / 1024));
lv_label_set_text(s_sys_heap, buf);
#if CONFIG_SPIRAM
snprintf(buf, sizeof(buf), "PSRAM: %lu KB free",
(unsigned long)(heap_caps_get_free_size(MALLOC_CAP_SPIRAM) / 1024));
#else
snprintf(buf, sizeof(buf), "PSRAM: N/A");
#endif
lv_label_set_text(s_sys_psram, buf);
if (has_vitals) {
snprintf(buf, sizeof(buf), "WiFi RSSI: %d dBm", vitals.rssi);
lv_label_set_text(s_sys_rssi, buf);
}
uint32_t uptime_s = (uint32_t)(esp_timer_get_time() / 1000000);
uint32_t h = uptime_s / 3600;
uint32_t m = (uptime_s % 3600) / 60;
uint32_t s = uptime_s % 60;
snprintf(buf, sizeof(buf), "Uptime: %luh %02lum %02lus",
(unsigned long)h, (unsigned long)m, (unsigned long)s);
lv_label_set_text(s_sys_uptime, buf);
snprintf(buf, sizeof(buf), "FPS: %lu", (unsigned long)s_current_fps);
lv_label_set_text(s_sys_fps, buf);
}
}
#endif /* CONFIG_DISPLAY_ENABLE */

31
firmware/esp32-csi-node/main/display_ui.h Normal file
View File

@ -0,0 +1,31 @@
/**
* @file display_ui.h
* @brief ADR-045: LVGL 4-view swipeable UI for CSI node stats.
*
* Views: Dashboard | Vitals | Presence | System
* Dark theme with cyan (#00d4ff) accent.
*/
#ifndef DISPLAY_UI_H
#define DISPLAY_UI_H
#include "lvgl.h"
#ifdef __cplusplus
extern "C" {
#endif
/** Create all LVGL views on the given tileview parent. */
void display_ui_create(lv_obj_t *parent);
/**
* Update all views with latest data. Called every display refresh cycle.
* Reads from edge_get_vitals() and edge_get_multi_person() internally.
*/
void display_ui_update(void);
#ifdef __cplusplus
}
#endif
#endif /* DISPLAY_UI_H */

10
firmware/esp32-csi-node/main/idf_component.yml Normal file
View File

@ -0,0 +1,10 @@
## ESP-IDF Managed Component Dependencies (ADR-045)
dependencies:
## LVGL graphics library
lvgl/lvgl: "~8.3"
## CST816S capacitive touch driver
espressif/esp_lcd_touch_cst816s: "^1.0"
## LCD touch abstraction
espressif/esp_lcd_touch: "^1.0"

94
firmware/esp32-csi-node/main/lv_conf.h Normal file
View File

@ -0,0 +1,94 @@
/**
* @file lv_conf.h
* @brief LVGL compile-time configuration for ESP32-S3 AMOLED display (ADR-045).
*
* Tuned for RM67162 536x240 QSPI AMOLED with 8MB PSRAM.
* Color depth: RGB565 (16-bit) for QSPI bandwidth.
* Double-buffered in SPIRAM, 30fps target.
*/
#ifndef LV_CONF_H
#define LV_CONF_H
#include <stdint.h>
/* ---- Core ---- */
#define LV_COLOR_DEPTH 16
#define LV_COLOR_16_SWAP 1 /* Byte-swap for SPI/QSPI displays */
#define LV_MEM_CUSTOM 1 /* Use ESP-IDF heap instead of LVGL's internal allocator */
#define LV_MEM_CUSTOM_INCLUDE <stdlib.h>
#define LV_MEM_CUSTOM_ALLOC malloc
#define LV_MEM_CUSTOM_FREE free
#define LV_MEM_CUSTOM_REALLOC realloc
/* ---- Display ---- */
#define LV_HOR_RES_MAX 368
#define LV_VER_RES_MAX 448
#define LV_DPI_DEF 200
/* ---- Tick (provided by esp_timer in display_task.c) ---- */
#define LV_TICK_CUSTOM 1
#define LV_TICK_CUSTOM_INCLUDE "esp_timer.h"
#define LV_TICK_CUSTOM_SYS_TIME_EXPR ((uint32_t)(esp_timer_get_time() / 1000))
/* ---- Drawing ---- */
#define LV_DRAW_COMPLEX 1
#define LV_SHADOW_CACHE_SIZE 0
#define LV_CIRCLE_CACHE_SIZE 4
#define LV_IMG_CACHE_DEF_SIZE 0
/* ---- Fonts ---- */
#define LV_FONT_MONTSERRAT_14 1
#define LV_FONT_MONTSERRAT_20 1
#define LV_FONT_DEFAULT &lv_font_montserrat_14
/* ---- Widgets ---- */
#define LV_USE_ARC 1
#define LV_USE_BAR 1
#define LV_USE_BTN 0
#define LV_USE_BTNMATRIX 0
#define LV_USE_CANVAS 0
#define LV_USE_CHECKBOX 0
#define LV_USE_DROPDOWN 0
#define LV_USE_IMG 0
#define LV_USE_LABEL 1
#define LV_USE_LINE 1
#define LV_USE_ROLLER 0
#define LV_USE_SLIDER 0
#define LV_USE_SWITCH 0
#define LV_USE_TEXTAREA 0
#define LV_USE_TABLE 0
/* ---- Extra widgets ---- */
#define LV_USE_CHART 1
#define LV_CHART_AXIS_TICK_LABEL_MAX_LEN 32
#define LV_USE_METER 0
#define LV_USE_SPINBOX 0
#define LV_USE_SPAN 0
#define LV_USE_TILEVIEW 1 /* Used for swipeable page navigation */
#define LV_USE_TABVIEW 0
#define LV_USE_WIN 0
/* ---- Themes ---- */
#define LV_USE_THEME_DEFAULT 1
#define LV_THEME_DEFAULT_DARK 1
/* ---- Logging ---- */
#define LV_USE_LOG 0
#define LV_USE_ASSERT_NULL 1
#define LV_USE_ASSERT_MALLOC 1
/* ---- GPU / render ---- */
#define LV_USE_GPU_ESP32_S3 0 /* No parallel LCD interface — we use QSPI */
/* ---- Animation ---- */
#define LV_USE_ANIM 1
#define LV_ANIM_DEF_TIME 200
/* ---- Misc ---- */
#define LV_USE_GROUP 1 /* For touch/input device routing */
#define LV_USE_PERF_MONITOR 0
#define LV_USE_MEM_MONITOR 0
#define LV_SPRINTF_CUSTOM 0
#endif /* LV_CONF_H */

7
firmware/esp32-csi-node/main/main.c
View File

@ -26,6 +26,7 @@
#include "power_mgmt.h"
#include "wasm_runtime.h"
#include "wasm_upload.h"
#include "display_task.h"
#include "esp_timer.h"
@ -203,6 +204,12 @@ void app_main(void)
/* Initialize power management. */
power_mgmt_init(g_nvs_config.power_duty);
/* ADR-045: Start AMOLED display task (gracefully skips if no display). */
esp_err_t disp_ret = display_task_start();
if (disp_ret != ESP_OK) {
ESP_LOGW(TAG, "Display init returned: %s", esp_err_to_name(disp_ret));
}
ESP_LOGI(TAG, "CSI streaming active → %s:%d (edge_tier=%u, OTA=%s, WASM=%s)",
g_nvs_config.target_ip, g_nvs_config.target_port,
g_nvs_config.edge_tier,

8
firmware/esp32-csi-node/partitions_display.csv Normal file
View File

@ -0,0 +1,8 @@
# ESP32-S3 CSI Node — 8MB flash partition table (ADR-045)
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x6000,
otadata, data, ota, 0xf000, 0x2000,
phy_init, data, phy, 0x11000, 0x1000,
ota_0, app, ota_0, 0x20000, 0x200000,
ota_1, app, ota_1, 0x220000, 0x200000,
spiffs, data, spiffs, 0x420000, 0x1E0000,
1 # ESP32-S3 CSI Node — 8MB flash partition table (ADR-045)
2 # Name, Type, SubType, Offset, Size, Flags
3 nvs, data, nvs, 0x9000, 0x6000,
4 otadata, data, ota, 0xf000, 0x2000,
5 phy_init, data, phy, 0x11000, 0x1000,
6 ota_0, app, ota_0, 0x20000, 0x200000,
7 ota_1, app, ota_1, 0x220000, 0x200000,
8 spiffs, data, spiffs, 0x420000, 0x1E0000,

267
rust-port/wifi-densepose-rs/data/adaptive_model.json Normal file
View File

@ -0,0 +1,267 @@
{
"class_stats": [
{
"label": "absent",
"count": 862,
"mean": [
66.68196972264862,
67.23973219951662,
65.0340640002779,
205.65861248066514,
1.2587006960556917,
8.192575406032482,
0.0,
9.823395623712905,
6.970045450727901,
-0.04488812678641681,
-0.9594767860850162,
10.78889030301701,
0.8330000846014487,
22.47189099978742,
22.47189099978742
],
"stddev": [
64.0493846652119,
90.27545165651007,
40.157907144682206,
161.60550836256004,
1.3807130815029451,
3.2814660018571113,
0.0,
2.219723108446689,
1.6521309619598676,
0.342852106459665,
0.30620004291079783,
3.529722483499124,
0.17574148506941875,
5.519861526721805,
5.519861526721805
]
},
{
"label": "present_still",
"count": 852,
"mean": [
66.39259262094396,
64.42298266818027,
68.34546366405283,
203.34049479166666,
1.1900821596244182,
8.200704225352112,
0.0,
10.032339700775715,
7.234479413048846,
0.027056637948278107,
-0.9161490234231624,
10.991429347401095,
0.8298622589530178,
23.588978503428145,
23.588978503428145
],
"stddev": [
59.144593976065984,
82.61098004853669,
40.08306971525127,
152.89405234329087,
1.2031203046363153,
3.0571012493320526,
0.0,
2.22294769203091,
1.6508044238677446,
0.3315329147240876,
0.29437997092330526,
3.3214071045026303,
0.17096813624285292,
5.622953396738593,
5.622953396738593
]
},
{
"label": "present_moving",
"count": 808,
"mean": [
65.17005228763453,
66.55424930761484,
63.785855267654334,
208.73719832920793,
1.3400990099009942,
7.570544554455446,
0.0,
10.069915394050431,
6.923405617584522,
-0.1440461642917184,
-1.0022460352626226,
10.664608744841848,
0.8384559212414682,
21.798331033369895,
21.798331033369895
],
"stddev": [
66.1800697503931,
93.22042148141067,
42.07226450730718,
164.93282045618218,
1.3706144246607475,
3.1453995481213224,
0.0,
2.431170975696439,
1.672707406405861,
0.35643090355922863,
0.30897080072710387,
3.325911716352165,
0.1806597020966414,
5.418714527442832,
5.418714527442832
]
},
{
"label": "active",
"count": 794,
"mean": [
61.85289600233076,
61.12723986655727,
62.468831971775344,
193.2018524349286,
1.2329974811083138,
8.083123425692696,
0.0,
9.747035051350043,
7.009904234422278,
0.007176072447431498,
-0.9950501087764124,
11.015545839210892,
0.8278984910895401,
22.445656559614797,
22.445656559614797
],
"stddev": [
50.44687370766278,
74.07914900524236,
31.558067649516538,
121.0762294406304,
1.2507304998955402,
3.4503520526220344,
0.0,
2.2730029390882156,
1.6768264387667406,
0.3214256392367928,
0.31003127617615406,
3.1187829194728285,
0.1772099351197549,
5.595050695741912,
5.595050695741912
]
}
],
"weights": [
[
0.9923736589617821,
-0.4600422332552322,
-0.3922101552522972,
-0.1686954616947851,
-0.08471937018349271,
0.033940973559074515,
0.0,
-1.116294981490482,
-0.213861080404439,
-0.41727297566573723,
0.08025552056009382,
0.20864577739519874,
0.36814779033318357,
0.46242679535538855,
0.46242679535538855,
0.09475205040199337
],
[
0.04661470129518883,
0.7974124099989739,
0.3953040913806362,
-1.2708868935843511,
0.10073070355913086,
0.0735810797517633,
0.0,
-0.3957608057630568,
0.22091779039114648,
-0.43105406953304665,
0.24907697332262252,
-0.17604200203759515,
-0.5059663705836186,
0.5740861193153091,
0.5740861193153091,
0.020569218347928304
],
[
-0.5295363836864718,
0.14729609046092632,
0.16131671233151712,
0.15039859740752318,
0.08189110214725194,
-0.1429062024394049,
0.0,
2.459247211223509,
-0.162133339181718,
0.6345474095048843,
0.16626892477248892,
0.2710091094981082,
-0.08197569509399917,
-1.2007197895193034,
-1.2007197895193034,
-0.10027402587742726
],
[
-0.5094519765704947,
-0.48466626720467487,
-0.1644106484598614,
1.2891837578716183,
-0.0979024355228887,
0.0353841491285671,
0.0,
-0.9471914239699604,
0.15507662919500606,
0.2137796356938993,
-0.49560141865520463,
-0.30361288485571664,
0.21979427534444013,
0.16420687484859928,
0.16420687484859928,
-0.015047242872495047
]
],
"global_mean": [
65.08291570815048,
64.88537161757283,
64.96650236787292,
202.8304440905207,
1.25474969843183,
8.016887816646562,
0.0,
9.918865477040464,
7.036167472733628,
-0.038097952045357715,
-0.9672836370393502,
10.86491812646321,
0.8323017200972911,
22.58850497890069,
22.58850497890069
],
"global_std": [
60.376895354908775,
85.49291935872783,
38.814475392686795,
151.54766198012683,
1.3049002582695195,
3.2446975526483737,
1e-9,
2.2904371592847603,
1.667114434239705,
0.34470363318292857,
0.3067332188136679,
3.334427501751985,
0.17614366955910027,
5.577838072123601,
5.577838072123601
],
"trained_frames": 3316,
"training_accuracy": 0.4149577804583836,
"version": 1
}

1
ui/index.html
View File

@ -29,6 +29,7 @@
<button class="nav-tab" data-tab="applications">Applications</button>
<button class="nav-tab" data-tab="sensing">Sensing</button>
<button class="nav-tab" data-tab="training">Training</button>
<a href="observatory.html" class="nav-tab" style="text-decoration:none">Observatory</a>
</nav>
<!-- Dashboard Tab -->

340
ui/observatory.html Normal file
View File

@ -0,0 +1,340 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>RuView Observatory — WiFi DensePose</title>
<link rel="stylesheet" href="observatory/css/observatory.css">
</head>
<body>
<canvas id="observatory-canvas"></canvas>
<!-- ======= HUD Overlay ======= -->
<div id="hud">
<!-- Top-left: Branding -->
<div id="brand">
<div id="brand-logo"><span class="pi">&pi;</span> RuView</div>
<div id="brand-tagline">WiFi DensePose Sensing Observatory</div>
</div>
<!-- Top-right: Connection + status + gear -->
<div id="status-bar">
<div id="data-source-badge">
<span class="dot dot--demo"></span>
<span id="data-source-label">DEMO</span>
</div>
<div id="scenario-area">
<span id="autoplay-icon" title="Auto-cycling">&#9654;</span>
<select id="scenario-quick-select" title="Change scenario">
<option value="auto">Auto-Cycle</option>
<option value="empty_room">Empty Room</option>
<option value="single_breathing">Vital Signs</option>
<option value="two_walking">Multi-Person</option>
<option value="fall_event">Fall Detect</option>
<option value="sleep_monitoring">Sleep Monitor</option>
<option value="intrusion_detect">Intrusion</option>
<option value="gesture_control">Gesture Ctrl</option>
<option value="crowd_occupancy">Crowd (4 ppl)</option>
<option value="search_rescue">Search Rescue</option>
<option value="elderly_care">Elderly Care</option>
<option value="fitness_tracking">Fitness</option>
<option value="security_patrol">Security Patrol</option>
</select>
</div>
<div id="scenario-description"></div>
<div id="fps-counter" style="display:none">60 FPS</div>
<button id="settings-btn" title="Settings">&#9881;</button>
</div>
<!-- Left panel: Vital Signs -->
<div id="panel-vitals" class="data-panel">
<div class="panel-header">Vital Signs</div>
<div class="vital-row">
<div class="vital-icon">&#9825;</div>
<div class="vital-data">
<div class="vital-label">Heart Rate</div>
<div class="vital-value"><span id="hr-value">--</span> <span class="vital-unit">BPM</span></div>
<div class="vital-bar"><div id="hr-bar" class="vital-bar-fill vital-bar--hr"></div></div>
</div>
</div>
<div class="vital-row">
<div class="vital-icon">&#9788;</div>
<div class="vital-data">
<div class="vital-label">Respiration</div>
<div class="vital-value"><span id="br-value">--</span> <span class="vital-unit">RPM</span></div>
<div class="vital-bar"><div id="br-bar" class="vital-bar-fill vital-bar--br"></div></div>
</div>
</div>
<div class="vital-row">
<div class="vital-icon">&#9878;</div>
<div class="vital-data">
<div class="vital-label">Confidence</div>
<div class="vital-value"><span id="conf-value">--</span><span class="vital-unit">%</span></div>
<div class="vital-bar"><div id="conf-bar" class="vital-bar-fill vital-bar--conf"></div></div>
</div>
</div>
</div>
<!-- Right panel: Signal & Presence -->
<div id="panel-signal" class="data-panel">
<div class="panel-header">WiFi Signal</div>
<div class="signal-row">
<span class="signal-label">RSSI</span>
<span class="signal-value" id="rssi-value">-- dBm</span>
</div>
<div class="signal-row">
<span class="signal-label">Variance</span>
<span class="signal-value" id="var-value">--</span>
</div>
<div class="signal-row">
<span class="signal-label">Motion</span>
<span class="signal-value" id="motion-value">--</span>
</div>
<div class="signal-row">
<span class="signal-label">Persons</span>
<span class="signal-value" id="persons-value">0</span>
<span id="persons-dots" class="persons-dots"></span>
</div>
<canvas id="rssi-sparkline" width="200" height="48"></canvas>
<div class="panel-header" style="margin-top:12px">Presence</div>
<div id="presence-indicator" class="presence-state presence--absent">
<span id="presence-label">ABSENT</span>
</div>
<div id="fall-alert" class="fall-alert" style="display:none">FALL DETECTED</div>
</div>
<!-- Edge module badges (populated dynamically by HudController) -->
<div id="edge-modules-bar"></div>
<!-- Bottom bar: capabilities -->
<div id="capabilities-bar">
<div class="cap-item"><span class="cap-icon">&#9898;</span><span>Human Pose Estimation</span></div>
<div class="cap-divider"></div>
<div class="cap-item"><span class="cap-icon">&#9829;</span><span>Vital Sign Monitoring</span></div>
<div class="cap-divider"></div>
<div class="cap-item"><span class="cap-icon">&#9784;</span><span>Presence Detection</span></div>
</div>
<!-- Bottom-right: keyboard hints -->
<div id="key-hints">
<span class="key-hint">[A] Orbit</span>
<span class="key-hint">[D] Scenario</span>
<span class="key-hint">[F] FPS</span>
<span class="key-hint">[S] Settings</span>
<span class="key-hint">[Space] Pause</span>
</div>
</div>
<!-- ======= Settings Dialog ======= -->
<div id="settings-overlay" class="settings-overlay" style="display:none">
<div class="settings-dialog">
<div class="settings-header">
<span>Settings</span>
<button id="settings-close">&times;</button>
</div>
<div class="settings-tabs">
<button class="stab active" data-stab="rendering">Rendering</button>
<button class="stab" data-stab="wireframe">Wireframe</button>
<button class="stab" data-stab="scene">Scene</button>
<button class="stab" data-stab="data">Data</button>
</div>
<!-- Rendering tab -->
<div class="stab-content active" id="stab-rendering">
<label class="setting-row">
<span>Bloom Strength</span>
<input type="range" id="opt-bloom" min="0" max="3" step="0.1" value="1.0">
<span class="range-val" id="opt-bloom-val">1.0</span>
</label>
<label class="setting-row">
<span>Bloom Radius</span>
<input type="range" id="opt-bloom-radius" min="0" max="1" step="0.05" value="0.5">
<span class="range-val" id="opt-bloom-radius-val">0.5</span>
</label>
<label class="setting-row">
<span>Bloom Threshold</span>
<input type="range" id="opt-bloom-thresh" min="0" max="1" step="0.05" value="0.25">
<span class="range-val" id="opt-bloom-thresh-val">0.25</span>
</label>
<label class="setting-row">
<span>Exposure</span>
<input type="range" id="opt-exposure" min="0.3" max="2" step="0.05" value="0.9">
<span class="range-val" id="opt-exposure-val">0.9</span>
</label>
<label class="setting-row">
<span>Vignette</span>
<input type="range" id="opt-vignette" min="0" max="1" step="0.05" value="0.5">
<span class="range-val" id="opt-vignette-val">0.5</span>
</label>
<label class="setting-row">
<span>Film Grain</span>
<input type="range" id="opt-grain" min="0" max="0.15" step="0.005" value="0.03">
<span class="range-val" id="opt-grain-val">0.03</span>
</label>
<label class="setting-row">
<span>Chromatic Aberration</span>
<input type="range" id="opt-chromatic" min="0" max="0.008" step="0.0005" value="0.0015">
<span class="range-val" id="opt-chromatic-val">0.0015</span>
</label>
</div>
<!-- Wireframe tab -->
<div class="stab-content" id="stab-wireframe">
<label class="setting-row">
<span>Bone Thickness</span>
<input type="range" id="opt-bone-thick" min="0.005" max="0.06" step="0.002" value="0.02">
<span class="range-val" id="opt-bone-thick-val">0.02</span>
</label>
<label class="setting-row">
<span>Joint Size</span>
<input type="range" id="opt-joint-size" min="0.02" max="0.12" step="0.005" value="0.05">
<span class="range-val" id="opt-joint-size-val">0.05</span>
</label>
<label class="setting-row">
<span>Glow Intensity</span>
<input type="range" id="opt-glow" min="0" max="2" step="0.1" value="0.8">
<span class="range-val" id="opt-glow-val">0.8</span>
</label>
<label class="setting-row">
<span>Particle Trail</span>
<input type="range" id="opt-trail" min="0" max="1" step="0.05" value="0.6">
<span class="range-val" id="opt-trail-val">0.6</span>
</label>
<label class="setting-row">
<span>Wireframe Color</span>
<input type="color" id="opt-wire-color" value="#00d878">
</label>
<label class="setting-row">
<span>Joint Color</span>
<input type="color" id="opt-joint-color" value="#ff4060">
</label>
<label class="setting-row">
<span>Aura Opacity</span>
<input type="range" id="opt-aura" min="0" max="0.2" step="0.01" value="0.06">
<span class="range-val" id="opt-aura-val">0.06</span>
</label>
</div>
<!-- Scene tab -->
<div class="stab-content" id="stab-scene">
<label class="setting-row">
<span>Signal Field</span>
<input type="range" id="opt-field" min="0" max="1" step="0.05" value="0.5">
<span class="range-val" id="opt-field-val">0.5</span>
</label>
<label class="setting-row">
<span>WiFi Waves</span>
<input type="range" id="opt-waves" min="0" max="1" step="0.05" value="0.6">
<span class="range-val" id="opt-waves-val">0.6</span>
</label>
<label class="setting-row">
<span>Room Brightness</span>
<input type="range" id="opt-ambient" min="0" max="1" step="0.05" value="0.4">
<span class="range-val" id="opt-ambient-val">0.4</span>
</label>
<label class="setting-row">
<span>Floor Reflection</span>
<input type="range" id="opt-reflect" min="0" max="1" step="0.05" value="0.3">
<span class="range-val" id="opt-reflect-val">0.3</span>
</label>
<label class="setting-row">
<span>FOV</span>
<input type="range" id="opt-fov" min="30" max="90" step="1" value="50">
<span class="range-val" id="opt-fov-val">50</span>
</label>
<label class="setting-row">
<span>Orbit Speed</span>
<input type="range" id="opt-orbit-speed" min="0.02" max="0.5" step="0.02" value="0.15">
<span class="range-val" id="opt-orbit-speed-val">0.15</span>
</label>
<label class="setting-row check-row">
<span>Show Grid</span>
<input type="checkbox" id="opt-grid" checked>
</label>
<label class="setting-row check-row">
<span>Show Room Boundary</span>
<input type="checkbox" id="opt-room" checked>
</label>
</div>
<!-- Data tab -->
<div class="stab-content" id="stab-data">
<label class="setting-row">
<span>Scenario</span>
<select id="opt-scenario">
<option value="auto">Auto-Cycle (30s)</option>
<optgroup label="Core Sensing">
<option value="empty_room">Empty Room</option>
<option value="single_breathing">Vital Signs (Breathing)</option>
<option value="two_walking">Multi-Person Tracking</option>
<option value="fall_event">Fall Detection</option>
</optgroup>
<optgroup label="Medical / Health">
<option value="sleep_monitoring">Sleep Monitoring (Apnea)</option>
<option value="elderly_care">Elderly Care (Gait)</option>
<option value="fitness_tracking">Fitness Tracking</option>
</optgroup>
<optgroup label="Security">
<option value="intrusion_detect">Intrusion Detection</option>
<option value="security_patrol">Security Patrol</option>
</optgroup>
<optgroup label="Building / Retail">
<option value="crowd_occupancy">Crowd Occupancy (4 ppl)</option>
<option value="gesture_control">Gesture Control (DTW)</option>
</optgroup>
<optgroup label="Disaster / Tactical">
<option value="search_rescue">Search &amp; Rescue (WiFi-Mat)</option>
</optgroup>
</select>
</label>
<label class="setting-row">
<span>Cycle Speed (s)</span>
<input type="range" id="opt-cycle" min="10" max="120" step="5" value="30">
<span class="range-val" id="opt-cycle-val">30</span>
</label>
<label class="setting-row">
<span>Style Preset</span>
<select id="opt-preset">
<option value="custom">Custom</option>
<option value="foundation">Foundation (Default)</option>
<option value="cinematic">Cinematic</option>
<option value="minimal">Minimal / Clean</option>
<option value="neon">Neon Glow</option>
<option value="tactical">Tactical / Military</option>
<option value="medical">Medical Monitor</option>
</select>
</label>
<label class="setting-row">
<span>Data Source</span>
<select id="opt-data-source">
<option value="demo" selected>Demo Generator</option>
<option value="ws">Live WebSocket</option>
</select>
</label>
<label class="setting-row" id="ws-url-row" style="display:none">
<span>WS URL</span>
<input type="text" id="opt-ws-url" value="" placeholder="ws://localhost:3000/ws/sensing">
</label>
<button id="btn-reset-camera" class="settings-btn">Reset Camera</button>
<button id="btn-reset-settings" class="settings-btn">Reset to Defaults</button>
<button id="btn-export-settings" class="settings-btn">Export Settings</button>
</div>
</div>
</div>
<!-- Three.js r160 + addons from CDN -->
<script type="importmap">
{
"imports": {
"three": "https://unpkg.com/three@0.160.0/build/three.module.js",
"three/addons/": "https://unpkg.com/three@0.160.0/examples/jsm/"
}
}
</script>
<script type="module" src="observatory/js/main.js"></script>
</body>
</html>

698
ui/observatory/css/observatory.css Normal file
View File

@ -0,0 +1,698 @@
/* ============================================================
RuView Observatory — Foundation Color Scheme
Warm dark background, electric green wireframe, amber data
============================================================ */
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@300;400;600;700&family=JetBrains+Mono:wght@400;600&display=swap');
:root {
--bg-deep: #080c14;
--bg-panel: rgba(8, 16, 28, 0.85);
--bg-panel-border: rgba(0, 210, 120, 0.2);
--green-glow: #00d878;
--green-bright:#3eff8a;
--green-dim: #0a6b3a;
--amber: #ffb020;
--amber-dim: #a06800;
--blue-signal: #2090ff;
--blue-dim: #0a3060;
--red-alert: #ff3040;
--red-heart: #ff4060;
--text-primary: #e8ece0;
--text-secondary: rgba(232,236,224, 0.55);
--text-label: rgba(232,236,224, 0.4);
}
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
background: var(--bg-deep);
overflow: hidden;
font-family: 'Inter', -apple-system, sans-serif;
color: var(--text-primary);
-webkit-font-smoothing: antialiased;
}
#observatory-canvas {
position: fixed;
top: 0; left: 0;
width: 100vw; height: 100vh;
}
/* ---- HUD Overlay ---- */
#hud {
position: fixed;
top: 0; left: 0;
width: 100%; height: 100%;
pointer-events: none;
z-index: 10;
}
/* ---- Brand ---- */
#brand {
position: absolute;
top: 24px; left: 28px;
}
#brand-logo {
font-family: 'Inter', sans-serif;
font-weight: 700;
font-size: 32px;
color: var(--text-primary);
letter-spacing: -0.5px;
text-shadow: 0 0 30px rgba(0, 216, 120, 0.3);
}
.pi {
color: var(--green-glow);
font-style: italic;
margin-right: 2px;
}
#brand-tagline {
font-size: 11px;
color: var(--text-secondary);
letter-spacing: 1.5px;
text-transform: uppercase;
margin-top: 2px;
}
/* ---- Status bar (top right) ---- */
#status-bar {
position: absolute;
top: 24px; right: 28px;
display: flex;
align-items: center;
gap: 12px;
}
#data-source-badge {
display: flex;
align-items: center;
gap: 6px;
padding: 5px 12px;
border-radius: 20px;
background: rgba(0, 216, 120, 0.1);
border: 1px solid rgba(0, 216, 120, 0.25);
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
letter-spacing: 1px;
color: var(--green-glow);
}
.dot {
width: 7px; height: 7px;
border-radius: 50%;
display: inline-block;
}
.dot--demo { background: var(--amber); box-shadow: 0 0 6px var(--amber); }
.dot--live { background: var(--green-glow); box-shadow: 0 0 6px var(--green-glow); animation: pulse-dot 2s infinite; }
@keyframes pulse-dot {
0%, 100% { opacity: 1; }
50% { opacity: 0.4; }
}
#scenario-area {
display: flex;
align-items: center;
gap: 8px;
padding: 5px 14px;
border-radius: 20px;
background: rgba(255, 176, 32, 0.1);
border: 1px solid rgba(255, 176, 32, 0.25);
pointer-events: auto;
}
#autoplay-icon {
font-size: 10px;
color: var(--green-glow);
animation: pulse-dot 2s infinite;
}
#autoplay-icon.hidden { display: none; }
#scenario-quick-select {
background: none;
border: none;
padding: 0;
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
letter-spacing: 0.5px;
color: var(--amber);
cursor: pointer;
outline: none;
}
#scenario-quick-select:hover,
#scenario-quick-select:focus { color: var(--green-glow); }
#scenario-quick-select option {
background: #0c1420;
color: var(--text-primary);
font-family: 'JetBrains Mono', monospace;
font-size: 12px;
padding: 4px 8px;
}
#fps-counter {
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
color: var(--text-secondary);
}
/* ---- Data Panels ---- */
.data-panel {
position: absolute;
width: 220px;
background: var(--bg-panel);
border: 1px solid var(--bg-panel-border);
border-radius: 12px;
padding: 16px;
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
pointer-events: auto;
}
.panel-header {
font-family: 'JetBrains Mono', monospace;
font-size: 10px;
font-weight: 600;
letter-spacing: 2px;
text-transform: uppercase;
color: var(--text-label);
margin-bottom: 14px;
padding-bottom: 8px;
border-bottom: 1px solid rgba(255,255,255,0.06);
}
/* ---- Vitals Panel (left) ---- */
#panel-vitals {
left: 28px;
top: 50%;
transform: translateY(-50%);
}
.vital-row {
display: flex;
align-items: flex-start;
gap: 12px;
margin-bottom: 18px;
}
.vital-row:last-child { margin-bottom: 0; }
.vital-icon {
font-size: 20px;
line-height: 1;
margin-top: 2px;
width: 24px;
text-align: center;
}
.vital-row:nth-child(2) .vital-icon { color: var(--red-heart); }
.vital-row:nth-child(3) .vital-icon { color: var(--green-glow); }
.vital-row:nth-child(4) .vital-icon { color: var(--amber); }
.vital-data { flex: 1; }
.vital-label {
font-size: 10px;
color: var(--text-label);
letter-spacing: 1px;
text-transform: uppercase;
margin-bottom: 3px;
}
.vital-value {
font-family: 'JetBrains Mono', monospace;
font-size: 26px;
font-weight: 600;
line-height: 1.1;
}
.vital-unit {
font-size: 12px;
font-weight: 400;
color: var(--text-secondary);
}
.vital-bar {
height: 3px;
background: rgba(255,255,255,0.06);
border-radius: 2px;
margin-top: 6px;
overflow: hidden;
}
.vital-bar-fill {
height: 100%;
border-radius: 2px;
transition: width 0.5s ease;
}
.vital-bar--hr { background: var(--red-heart); width: 0%; }
.vital-bar--br { background: var(--green-glow); width: 0%; }
.vital-bar--conf { background: var(--amber); width: 0%; }
/* ---- Signal Panel (right) ---- */
#panel-signal {
right: 28px;
top: 50%;
transform: translateY(-50%);
}
.signal-row {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 8px;
}
.signal-label {
font-size: 11px;
color: var(--text-label);
letter-spacing: 0.5px;
}
.signal-value {
font-family: 'JetBrains Mono', monospace;
font-size: 13px;
font-weight: 600;
color: var(--blue-signal);
}
#rssi-sparkline {
width: 100%;
height: 48px;
margin-top: 8px;
border-radius: 6px;
background: rgba(0,0,0,0.3);
}
/* Presence */
.presence-state {
text-align: center;
padding: 8px;
border-radius: 8px;
font-family: 'JetBrains Mono', monospace;
font-size: 14px;
font-weight: 600;
letter-spacing: 2px;
transition: all 0.5s ease;
}
.presence--absent {
background: rgba(255,255,255,0.03);
color: var(--text-label);
border: 1px solid rgba(255,255,255,0.05);
}
.presence--present {
background: rgba(0, 216, 120, 0.1);
color: var(--green-glow);
border: 1px solid rgba(0, 216, 120, 0.3);
box-shadow: 0 0 20px rgba(0, 216, 120, 0.1);
}
.presence--active {
background: rgba(255, 176, 32, 0.1);
color: var(--amber);
border: 1px solid rgba(255, 176, 32, 0.3);
box-shadow: 0 0 20px rgba(255, 176, 32, 0.1);
}
.fall-alert {
margin-top: 10px;
text-align: center;
padding: 8px;
border-radius: 8px;
font-family: 'JetBrains Mono', monospace;
font-size: 12px;
font-weight: 700;
letter-spacing: 2px;
background: rgba(255, 48, 64, 0.15);
color: var(--red-alert);
border: 1px solid rgba(255, 48, 64, 0.4);
animation: pulse-alert 0.8s infinite;
}
@keyframes pulse-alert {
0%, 100% { opacity: 1; }
50% { opacity: 0.5; }
}
/* ---- Capabilities Bar (bottom center) ---- */
#capabilities-bar {
position: absolute;
bottom: 20px;
left: 50%;
transform: translateX(-50%);
display: flex;
align-items: center;
gap: 0;
background: var(--bg-panel);
border: 1px solid var(--bg-panel-border);
border-radius: 30px;
padding: 8px 24px;
backdrop-filter: blur(12px);
}
.cap-item {
display: flex;
align-items: center;
gap: 8px;
font-size: 12px;
font-weight: 500;
color: var(--text-secondary);
padding: 0 16px;
}
.cap-icon {
font-size: 16px;
color: var(--green-glow);
}
.cap-item:nth-child(3) .cap-icon { color: var(--red-heart); }
.cap-item:nth-child(5) .cap-icon { color: var(--blue-signal); }
.cap-divider {
width: 1px;
height: 20px;
background: rgba(255,255,255,0.1);
}
/* ---- Key hints ---- */
#key-hints {
position: absolute;
bottom: 24px;
right: 28px;
display: flex;
gap: 8px;
}
.key-hint {
font-family: 'JetBrains Mono', monospace;
font-size: 10px;
color: rgba(255,255,255,0.2);
letter-spacing: 0.5px;
padding: 3px 8px;
border-radius: 4px;
background: rgba(255,255,255,0.03);
border: 1px solid rgba(255,255,255,0.05);
}
/* ---- Settings button ---- */
#settings-btn {
pointer-events: auto;
background: rgba(255,255,255,0.06);
border: 1px solid rgba(255,255,255,0.1);
color: var(--text-secondary);
font-size: 18px;
width: 34px; height: 34px;
border-radius: 50%;
cursor: pointer;
transition: all 0.2s;
display: flex; align-items: center; justify-content: center;
padding: 0;
}
#settings-btn:hover {
background: rgba(0, 216, 120, 0.15);
border-color: var(--green-glow);
color: var(--green-glow);
}
/* ---- Settings Dialog ---- */
.settings-overlay {
position: fixed;
top: 0; left: 0;
width: 100%; height: 100%;
z-index: 100;
background: rgba(0,0,0,0.5);
backdrop-filter: blur(4px);
display: flex;
align-items: center;
justify-content: center;
pointer-events: auto;
}
.settings-dialog {
background: rgba(10, 16, 28, 0.96);
border: 1px solid rgba(0, 216, 120, 0.2);
border-radius: 16px;
width: 440px;
max-height: 80vh;
overflow-y: auto;
padding: 0;
box-shadow: 0 20px 60px rgba(0,0,0,0.6), 0 0 40px rgba(0,216,120,0.05);
}
.settings-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 16px 20px;
border-bottom: 1px solid rgba(255,255,255,0.06);
font-family: 'JetBrains Mono', monospace;
font-size: 13px;
font-weight: 600;
letter-spacing: 1px;
text-transform: uppercase;
color: var(--text-primary);
}
.settings-header button {
background: none;
border: none;
color: var(--text-secondary);
font-size: 22px;
cursor: pointer;
padding: 0 4px;
line-height: 1;
}
.settings-header button:hover { color: var(--red-alert); }
.settings-tabs {
display: flex;
border-bottom: 1px solid rgba(255,255,255,0.06);
padding: 0 12px;
}
.stab {
background: none;
border: none;
color: var(--text-label);
font-family: 'JetBrains Mono', monospace;
font-size: 10px;
letter-spacing: 1px;
text-transform: uppercase;
padding: 10px 14px;
cursor: pointer;
border-bottom: 2px solid transparent;
transition: all 0.2s;
}
.stab:hover { color: var(--text-secondary); }
.stab.active {
color: var(--green-glow);
border-bottom-color: var(--green-glow);
}
.stab-content {
display: none;
padding: 16px 20px;
}
.stab-content.active { display: block; }
.setting-row {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
margin-bottom: 14px;
font-size: 12px;
color: var(--text-secondary);
}
.setting-row span:first-child {
min-width: 120px;
flex-shrink: 0;
}
.setting-row input[type="range"] {
flex: 1;
height: 4px;
-webkit-appearance: none;
appearance: none;
background: rgba(255,255,255,0.08);
border-radius: 2px;
outline: none;
}
.setting-row input[type="range"]::-webkit-slider-thumb {
-webkit-appearance: none;
width: 14px; height: 14px;
border-radius: 50%;
background: var(--green-glow);
cursor: pointer;
box-shadow: 0 0 6px rgba(0,216,120,0.4);
}
.setting-row input[type="color"] {
-webkit-appearance: none;
width: 36px; height: 24px;
border: 1px solid rgba(255,255,255,0.15);
border-radius: 4px;
background: none;
cursor: pointer;
padding: 0;
}
.setting-row input[type="color"]::-webkit-color-swatch-wrapper { padding: 2px; }
.setting-row input[type="color"]::-webkit-color-swatch { border-radius: 2px; border: none; }
.setting-row select,
.setting-row input[type="text"] {
flex: 1;
background: #0c1420;
border: 1px solid rgba(255,255,255,0.1);
color: var(--text-primary);
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
padding: 6px 10px;
border-radius: 6px;
outline: none;
}
.setting-row select:focus,
.setting-row input[type="text"]:focus {
border-color: var(--green-glow);
}
.setting-row select option {
background: #0c1420;
color: var(--text-primary);
padding: 6px 10px;
}
.setting-row select optgroup {
background: #0a1018;
color: var(--green-glow);
font-style: normal;
font-weight: 600;
padding: 4px 0;
}
.setting-row input[type="checkbox"] {
width: 18px; height: 18px;
accent-color: var(--green-glow);
cursor: pointer;
}
.check-row {
flex-direction: row;
}
.range-val {
font-family: 'JetBrains Mono', monospace;
font-size: 10px;
color: var(--green-glow);
min-width: 44px;
text-align: right;
}
.settings-btn {
width: 100%;
padding: 8px;
margin-top: 6px;
background: rgba(0, 216, 120, 0.08);
border: 1px solid rgba(0, 216, 120, 0.2);
color: var(--green-glow);
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
letter-spacing: 1px;
border-radius: 6px;
cursor: pointer;
transition: all 0.2s;
}
.settings-btn:hover {
background: rgba(0, 216, 120, 0.15);
border-color: var(--green-glow);
}
/* ---- Scenario Description ---- */
#scenario-description {
position: absolute;
top: 60px;
right: 28px;
max-width: 340px;
font-size: 11px;
color: var(--text-secondary);
font-style: italic;
letter-spacing: 0.3px;
line-height: 1.4;
pointer-events: none;
opacity: 0.7;
transition: opacity 0.5s ease;
}
/* ---- Edge Module Badges ---- */
#edge-modules-bar {
position: absolute;
bottom: 58px;
left: 50%;
transform: translateX(-50%);
display: flex;
align-items: center;
gap: 6px;
pointer-events: none;
}
.edge-badge {
display: inline-block;
padding: 2px 8px;
border-radius: 10px;
font-family: 'JetBrains Mono', monospace;
font-size: 9px;
font-weight: 600;
letter-spacing: 1px;
color: var(--badge-color, var(--text-secondary));
background: rgba(255,255,255,0.04);
border: 1px solid var(--badge-color, rgba(255,255,255,0.1));
box-shadow: 0 0 6px color-mix(in srgb, var(--badge-color, transparent) 30%, transparent);
}
/* ---- Person Count Dots ---- */
.persons-dots {
display: inline-flex;
align-items: center;
gap: 3px;
margin-left: 6px;
vertical-align: middle;
}
.person-dot {
width: 6px;
height: 6px;
border-radius: 50%;
display: inline-block;
background: rgba(255,255,255,0.08);
border: 1px solid rgba(255,255,255,0.1);
transition: background 0.4s ease, border-color 0.4s ease, box-shadow 0.4s ease;
}
.person-dot--active {
background: var(--green-glow);
border-color: var(--green-glow);
box-shadow: 0 0 4px rgba(0, 216, 120, 0.4);
}
/* ---- Vital Value Color Transitions ---- */
.vital-value span:first-child {
transition: color 0.6s ease;
}
.vital-bar-fill {
transition: width 0.5s ease, background 0.6s ease;
}
/* ---- Responsive ---- */
@media (max-width: 1200px) {
.data-panel { width: 190px; padding: 12px; }
.vital-value { font-size: 22px; }
#capabilities-bar { display: none; }
}
@media (max-width: 800px) {
.data-panel { display: none; }
#key-hints { display: none; }
.settings-dialog { width: 95vw; }
}

221
ui/observatory/js/convergence-engine.js Normal file
View File

@ -0,0 +1,221 @@
/**
* Module E — "Statistical Convergence Engine"
* RSSI waveform, person orbs, classification, fall alert, metric bars
*/
import * as THREE from 'three';
const WAVEFORM_POINTS = 120;
export class ConvergenceEngine {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
// --- RSSI Waveform (scrolling line) ---
this._rssiHistory = new Float32Array(WAVEFORM_POINTS);
const waveGeo = new THREE.BufferGeometry();
this._wavePositions = new Float32Array(WAVEFORM_POINTS * 3);
for (let i = 0; i < WAVEFORM_POINTS; i++) {
this._wavePositions[i * 3] = (i / WAVEFORM_POINTS) * 6 - 3; // x: -3 to 3
this._wavePositions[i * 3 + 1] = 0;
this._wavePositions[i * 3 + 2] = 0;
}
waveGeo.setAttribute('position', new THREE.BufferAttribute(this._wavePositions, 3));
const waveMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.8,
blending: THREE.AdditiveBlending,
});
this._waveform = new THREE.Line(waveGeo, waveMat);
this._waveform.position.y = 1.5;
this.group.add(this._waveform);
// Waveform glow (thicker, dimmer duplicate)
const glowMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.2,
linewidth: 2,
blending: THREE.AdditiveBlending,
});
this._waveGlow = new THREE.Line(waveGeo.clone(), glowMat);
this._waveGlow.position.y = 1.5;
this._waveGlow.scale.set(1, 1.3, 1);
this.group.add(this._waveGlow);
// --- Person orbs (up to 4) ---
this._personOrbs = [];
for (let i = 0; i < 4; i++) {
const orbGeo = new THREE.SphereGeometry(0.2, 16, 16);
const orbMat = new THREE.MeshBasicMaterial({
color: 0xff8800,
transparent: true,
opacity: 0,
blending: THREE.AdditiveBlending,
});
const orb = new THREE.Mesh(orbGeo, orbMat);
orb.position.set(-2 + i * 1.2, -0.5, 0);
this.group.add(orb);
const light = new THREE.PointLight(0xff8800, 0, 3);
orb.add(light);
this._personOrbs.push({ mesh: orb, light, mat: orbMat });
}
// --- Classification text sprite ---
this._classCanvas = document.createElement('canvas');
this._classCanvas.width = 256;
this._classCanvas.height = 48;
this._classCtx = this._classCanvas.getContext('2d');
this._classTex = new THREE.CanvasTexture(this._classCanvas);
const classMat = new THREE.SpriteMaterial({
map: this._classTex,
transparent: true,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._classSprite = new THREE.Sprite(classMat);
this._classSprite.scale.set(3, 0.6, 1);
this._classSprite.position.y = 0.3;
this.group.add(this._classSprite);
// --- Fall alert ring ---
const alertGeo = new THREE.TorusGeometry(2.5, 0.05, 8, 48);
this._alertMat = new THREE.MeshBasicMaterial({
color: 0xff2244,
transparent: true,
opacity: 0,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._alertRing = new THREE.Mesh(alertGeo, this._alertMat);
this._alertRing.rotation.x = Math.PI / 2;
this._alertRing.position.y = -1;
this.group.add(this._alertRing);
// --- Metric bars (3: frame rate, confidence, variance) ---
this._metricBars = [];
const barLabels = ['CONF', 'VAR', 'SPEC'];
for (let i = 0; i < 3; i++) {
const barGeo = new THREE.PlaneGeometry(0.15, 1.5);
const barMat = new THREE.MeshBasicMaterial({
color: [0x00d4ff, 0x8844ff, 0xff8800][i],
transparent: true,
opacity: 0.5,
blending: THREE.AdditiveBlending,
depthWrite: false,
side: THREE.DoubleSide,
});
const bar = new THREE.Mesh(barGeo, barMat);
bar.position.set(2 + i * 0.4, -1.2, 0);
this.group.add(bar);
this._metricBars.push({ mesh: bar, mat: barMat });
}
this._rssiHead = 0;
this._lastClassification = '';
}
update(dt, elapsed, data) {
const features = data?.features || {};
const classification = data?.classification || {};
const persons = data?.persons || [];
const estPersons = data?.estimated_persons || 0;
// --- Update RSSI waveform ---
const rssi = features.mean_rssi || -50;
this._rssiHistory[this._rssiHead] = rssi;
this._rssiHead = (this._rssiHead + 1) % WAVEFORM_POINTS;
for (let i = 0; i < WAVEFORM_POINTS; i++) {
const histIdx = (this._rssiHead + i) % WAVEFORM_POINTS;
const val = this._rssiHistory[histIdx];
// Normalize RSSI (-80 to -20 range) to -1.5 to 1.5
this._wavePositions[i * 3 + 1] = ((val + 50) / 30) * 1.5;
}
this._waveform.geometry.attributes.position.needsUpdate = true;
// Copy to glow
const glowPos = this._waveGlow.geometry.attributes.position;
glowPos.array.set(this._wavePositions);
glowPos.needsUpdate = true;
// --- Person orbs ---
for (let i = 0; i < this._personOrbs.length; i++) {
const { mesh, light, mat } = this._personOrbs[i];
if (i < estPersons) {
mat.opacity = 0.7;
light.intensity = 1.0 + Math.sin(elapsed * 3 + i * 1.5) * 0.5;
const pulse = 1.0 + Math.sin(elapsed * 2 + i) * 0.15;
mesh.scale.set(pulse, pulse, pulse);
} else {
mat.opacity = 0.05;
light.intensity = 0;
mesh.scale.set(0.5, 0.5, 0.5);
}
}
// --- Classification text ---
const motionLevel = classification.motion_level || 'absent';
const label = motionLevel.toUpperCase().replace('_', ' ');
if (label !== this._lastClassification) {
this._lastClassification = label;
const ctx = this._classCtx;
ctx.clearRect(0, 0, 256, 48);
ctx.font = '600 24px "Courier New", monospace';
ctx.textAlign = 'center';
if (motionLevel === 'active') ctx.fillStyle = '#ff8800';
else if (motionLevel.includes('present')) ctx.fillStyle = '#00d4ff';
else ctx.fillStyle = '#445566';
ctx.fillText(label, 128, 32);
this._classTex.needsUpdate = true;
}
// --- Fall alert ---
const fallDetected = classification.fall_detected || false;
if (fallDetected) {
this._alertMat.opacity = 0.3 + Math.abs(Math.sin(elapsed * 6)) * 0.5;
const scale = 1.0 + Math.sin(elapsed * 4) * 0.1;
this._alertRing.scale.set(scale, scale, 1);
} else {
this._alertMat.opacity = 0;
}
// --- Metric bars ---
const confidence = classification.confidence || 0;
const variance = Math.min(1, (features.variance || 0) / 5);
const spectral = Math.min(1, (features.spectral_power || 0) / 0.5);
const values = [confidence, variance, spectral];
for (let i = 0; i < 3; i++) {
const bar = this._metricBars[i];
const v = values[i];
bar.mesh.scale.y = Math.max(0.05, v);
bar.mesh.position.y = -1.2 + v * 0.75;
bar.mat.opacity = 0.3 + v * 0.4;
}
}
dispose() {
this._waveform.geometry.dispose();
this._waveform.material.dispose();
this._waveGlow.geometry.dispose();
this._waveGlow.material.dispose();
this._alertRing.geometry.dispose();
this._alertMat.dispose();
this._classTex.dispose();
for (const { mesh, mat } of this._personOrbs) {
mesh.geometry.dispose();
mat.dispose();
}
for (const { mesh, mat } of this._metricBars) {
mesh.geometry.dispose();
mat.dispose();
}
}
}

1794
ui/observatory/js/demo-data.js Normal file
View File

File diff suppressed because it is too large Load Diff

513
ui/observatory/js/figure-pool.js Normal file
View File

@ -0,0 +1,513 @@
/**
* FigurePool — Manages a pool of wireframe human figures for multi-person rendering.
*
* Extracted from main.js Observatory class. Owns the lifecycle of up to MAX_FIGURES
* Three.js figure groups, each containing joints, bones, body segments, and aura.
*
* Improvements over the original inline implementation:
* - Smooth joint interpolation (lerp toward target instead of snapping)
* - Joint pulsation synced with breathing
* - Natural bone thickness taper (thicker at shoulder/hip, thinner at extremities)
* - Secondary motion with slight delay/overshoot for organic feel
* - Pose-adaptive aura shape (wider for exercise, narrower for crouching)
*/
import * as THREE from 'three';
// 17-keypoint COCO skeleton connectivity
export const SKELETON_PAIRS = [
[0, 1], [0, 2], [1, 3], [2, 4],
[5, 6], [5, 7], [7, 9], [6, 8], [8, 10],
[5, 11], [6, 12], [11, 12],
[11, 13], [13, 15], [12, 14], [14, 16],
];
// Body segment cylinders that give volume to the wireframe
export const BODY_SEGMENT_DEFS = [
{ joints: [5, 11], radius: 0.12 }, // left torso
{ joints: [6, 12], radius: 0.12 }, // right torso
{ joints: [5, 6], radius: 0.1 }, // shoulder bar
{ joints: [11, 12], radius: 0.1 }, // hip bar
{ joints: [5, 7], radius: 0.05 }, // left upper arm
{ joints: [6, 8], radius: 0.05 }, // right upper arm
{ joints: [7, 9], radius: 0.04 }, // left forearm
{ joints: [8, 10], radius: 0.04 }, // right forearm
{ joints: [11, 13], radius: 0.07 }, // left thigh
{ joints: [12, 14], radius: 0.07 }, // right thigh
{ joints: [13, 15], radius: 0.05 }, // left shin
{ joints: [14, 16], radius: 0.05 }, // right shin
{ joints: [0, 0], radius: 0.1, isHead: true },
];
// Bone thickness multipliers — thicker at torso, thinner at extremities
const BONE_TAPER = (() => {
const tapers = new Map();
// Torso and shoulder/hip connections are thickest
tapers.set('5-6', 1.4); // shoulder bar
tapers.set('11-12', 1.3); // hip bar
tapers.set('5-11', 1.3); // left torso
tapers.set('6-12', 1.3); // right torso
// Upper limbs
tapers.set('5-7', 1.0); // left upper arm
tapers.set('6-8', 1.0); // right upper arm
tapers.set('11-13', 1.1); // left thigh
tapers.set('12-14', 1.1); // right thigh
// Lower limbs / extremities — thinnest
tapers.set('7-9', 0.7); // left forearm
tapers.set('8-10', 0.7); // right forearm
tapers.set('13-15', 0.8); // left shin
tapers.set('14-16', 0.8); // right shin
// Head connections
tapers.set('0-1', 0.5);
tapers.set('0-2', 0.5);
tapers.set('1-3', 0.4);
tapers.set('2-4', 0.4);
return tapers;
})();
// Secondary motion delay factors per joint — extremities lag more
const SECONDARY_DELAY = [
0.12, // 0 nose
0.10, // 1 left eye
0.10, // 2 right eye
0.08, // 3 left ear
0.08, // 4 right ear
0.18, // 5 left shoulder
0.18, // 6 right shoulder
0.14, // 7 left elbow
0.14, // 8 right elbow
0.10, // 9 left wrist (most lag)
0.10, // 10 right wrist
0.20, // 11 left hip (anchored, fast follow)
0.20, // 12 right hip
0.15, // 13 left knee
0.15, // 14 right knee
0.10, // 15 left ankle
0.10, // 16 right ankle
];
// Overshoot factors — extremities overshoot more for organic feel
const OVERSHOOT = [
0.02, // 0 nose
0.01, // 1 left eye
0.01, // 2 right eye
0.01, // 3 left ear
0.01, // 4 right ear
0.03, // 5 left shoulder
0.03, // 6 right shoulder
0.05, // 7 left elbow
0.05, // 8 right elbow
0.08, // 9 left wrist
0.08, // 10 right wrist
0.02, // 11 left hip
0.02, // 12 right hip
0.04, // 13 left knee
0.04, // 14 right knee
0.06, // 15 left ankle
0.06, // 16 right ankle
];
const MAX_FIGURES = 4;
// Reusable vectors to avoid per-frame allocation
const _vecFrom = new THREE.Vector3();
const _vecTo = new THREE.Vector3();
const _vecTarget = new THREE.Vector3();
export class FigurePool {
/**
* @param {THREE.Scene} scene - The Three.js scene to add figures to
* @param {object} settings - Shared settings object (boneThick, jointSize, glow, etc.)
* @param {object} poseSystem - PoseSystem instance with generateKeypoints(person, elapsed, breathPulse)
*/
constructor(scene, settings, poseSystem) {
this._scene = scene;
this._settings = settings;
this._poseSystem = poseSystem;
this._figures = [];
this._maxFigures = MAX_FIGURES;
this._build();
}
/** @returns {Array} The array of figure objects */
get figures() { return this._figures; }
// ---- Construction ----
_build() {
for (let f = 0; f < this._maxFigures; f++) {
this._figures.push(this._createFigure());
}
}
_createFigure() {
const group = new THREE.Group();
this._scene.add(group);
const wireColor = new THREE.Color(this._settings.wireColor);
const jointColor = new THREE.Color(this._settings.jointColor);
// Joints (17 COCO keypoints)
const joints = [];
for (let i = 0; i < 17; i++) {
const isNose = i === 0;
const size = isNose ? this._settings.jointSize * 0.7 : this._settings.jointSize;
const geo = new THREE.SphereGeometry(size, 12, 12);
const mat = new THREE.MeshStandardMaterial({
color: isNose ? wireColor : jointColor,
emissive: isNose ? wireColor : jointColor,
emissiveIntensity: 0.35,
transparent: true, opacity: 0,
roughness: 0.3, metalness: 0.2,
});
const sphere = new THREE.Mesh(geo, mat);
sphere.castShadow = true;
group.add(sphere);
joints.push(sphere);
// Halo glow on key joints
if ([5, 6, 9, 10, 11, 12, 15, 16].includes(i)) {
const haloGeo = new THREE.SphereGeometry(size * 1.3, 8, 8);
const haloMat = new THREE.MeshBasicMaterial({
color: jointColor,
transparent: true, opacity: 0,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
const halo = new THREE.Mesh(haloGeo, haloMat);
sphere.add(halo);
sphere._halo = halo;
sphere._haloMat = haloMat;
const glow = new THREE.PointLight(jointColor, 0, 0.8);
sphere.add(glow);
sphere._glow = glow;
}
}
// Bones — tapered thickness
const bones = [];
for (const [a, b] of SKELETON_PAIRS) {
const taperKey = `${Math.min(a, b)}-${Math.max(a, b)}`;
const taper = BONE_TAPER.get(taperKey) || 1.0;
const thick = this._settings.boneThick * taper;
// Top radius thicker than bottom for natural taper along bone length
const topRadius = thick;
const botRadius = thick * 0.65;
const geo = new THREE.CylinderGeometry(topRadius, botRadius, 1, 8, 1);
geo.translate(0, 0.5, 0);
geo.rotateX(Math.PI / 2);
const mat = new THREE.MeshStandardMaterial({
color: wireColor, emissive: wireColor, emissiveIntensity: 0.3,
transparent: true, opacity: 0, roughness: 0.4, metalness: 0.1,
});
const mesh = new THREE.Mesh(geo, mat);
mesh.castShadow = true;
group.add(mesh);
bones.push({ mesh, a, b, taper });
}
// Body segments (volume cylinders and head sphere)
const bodySegments = [];
for (const seg of BODY_SEGMENT_DEFS) {
const geo = seg.isHead
? new THREE.SphereGeometry(seg.radius, 12, 12)
: new THREE.CylinderGeometry(seg.radius, seg.radius * 0.85, 1, 8, 1);
if (!seg.isHead) {
geo.translate(0, 0.5, 0);
geo.rotateX(Math.PI / 2);
}
const mat = new THREE.MeshStandardMaterial({
color: wireColor, emissive: wireColor, emissiveIntensity: 0.12,
transparent: true, opacity: 0, roughness: 0.5, metalness: 0.1,
side: THREE.DoubleSide,
});
const mesh = new THREE.Mesh(geo, mat);
group.add(mesh);
bodySegments.push({ mesh, mat, a: seg.joints[0], b: seg.joints[1], isHead: seg.isHead });
}
// Aura cylinder
const auraGeo = new THREE.CylinderGeometry(0.4, 0.3, 1.7, 16, 1, true);
const auraMat = new THREE.MeshBasicMaterial({
color: wireColor, transparent: true, opacity: 0,
side: THREE.DoubleSide, blending: THREE.AdditiveBlending, depthWrite: false,
});
const aura = new THREE.Mesh(auraGeo, auraMat);
aura.position.y = 1;
group.add(aura);
// Per-figure point light
const personLight = new THREE.PointLight(wireColor, 0, 6);
personLight.position.y = 1;
group.add(personLight);
// Interpolation state: previous positions for smooth lerp and secondary motion
const prevPositions = [];
const velocities = [];
for (let i = 0; i < 17; i++) {
prevPositions.push(new THREE.Vector3(0, 0, 0));
velocities.push(new THREE.Vector3(0, 0, 0));
}
return {
group, joints, bones, bodySegments, aura, auraMat, personLight,
visible: false,
prevPositions,
velocities,
_initialized: false,
_lastPose: null,
};
}
// ---- Per-frame update ----
/**
* Update all figures based on current data frame.
* @param {object} data - Current sensing data with persons[], vital_signs, classification
* @param {number} elapsed - Elapsed time in seconds
*/
update(data, elapsed) {
const persons = data?.persons || [];
const vs = data?.vital_signs || {};
const isPresent = data?.classification?.presence || false;
const breathBpm = vs.breathing_rate_bpm || 0;
const breathPulse = breathBpm > 0
? Math.sin(elapsed * Math.PI * 2 * (breathBpm / 60)) * 0.012
: 0;
for (let f = 0; f < this._figures.length; f++) {
const fig = this._figures[f];
if (f < persons.length && isPresent) {
const p = persons[f];
const kps = this._poseSystem.generateKeypoints(p, elapsed, breathPulse);
this.applyKeypoints(fig, kps, breathPulse, p.position || [0, 0, 0], elapsed, p.pose);
fig.visible = true;
} else {
if (fig.visible) {
this.hide(fig);
fig.visible = false;
}
}
}
}
/**
* Apply keypoints to a figure with smooth interpolation, pulsation, and secondary motion.
* @param {object} fig - Figure object from the pool
* @param {Array} kps - 17-element array of [x,y,z] keypoint positions
* @param {number} breathPulse - Current breathing pulse value
* @param {Array} pos - Person world position [x,y,z]
* @param {number} elapsed - Elapsed time for pulsation effects
* @param {string} pose - Current pose name for aura adaptation
*/
applyKeypoints(fig, kps, breathPulse, pos, elapsed = 0, pose = 'standing') {
const lerpFactor = fig._initialized ? 0.18 : 1.0;
// Joints with smooth interpolation and secondary motion
for (let i = 0; i < 17 && i < kps.length; i++) {
const j = fig.joints[i];
_vecTarget.set(kps[i][0], kps[i][1], kps[i][2]);
if (fig._initialized) {
// Compute velocity for overshoot
const prev = fig.prevPositions[i];
const vel = fig.velocities[i];
// Smooth lerp with per-joint delay
const delay = SECONDARY_DELAY[i];
const jointLerp = lerpFactor + delay;
j.position.lerp(_vecTarget, Math.min(jointLerp, 0.95));
// Apply subtle overshoot based on velocity change
const overshoot = OVERSHOOT[i];
vel.subVectors(j.position, prev).multiplyScalar(overshoot);
j.position.add(vel);
prev.copy(j.position);
} else {
// First frame: snap to position
j.position.copy(_vecTarget);
fig.prevPositions[i].copy(_vecTarget);
fig.velocities[i].set(0, 0, 0);
}
j.material.opacity = 0.95;
// Joint pulsation synced with breathing
const pulseFactor = 1.0 + Math.abs(breathPulse) * 8.0;
j.material.emissiveIntensity = 0.35 * pulseFactor;
const baseScale = this._settings.jointSize / 0.04;
// Subtle size pulsation on breathing
const pulseScale = baseScale * (1.0 + Math.abs(breathPulse) * 3.0);
j.scale.setScalar(pulseScale);
if (j._haloMat) {
j._haloMat.opacity = 0.04 * this._settings.glow * pulseFactor;
}
if (j._glow) {
j._glow.intensity = this._settings.glow * 0.12 * pulseFactor;
}
}
fig._initialized = true;
// Bones with tapered thickness
for (const bone of fig.bones) {
const pA = kps[bone.a], pB = kps[bone.b];
if (pA && pB) {
_vecFrom.set(pA[0], pA[1], pA[2]);
_vecTo.set(pB[0], pB[1], pB[2]);
const len = _vecFrom.distanceTo(_vecTo);
// Use interpolated joint positions for smooth bone movement
if (fig._initialized) {
const jA = fig.joints[bone.a];
const jB = fig.joints[bone.b];
bone.mesh.position.copy(jA.position);
bone.mesh.scale.set(1, 1, jA.position.distanceTo(jB.position));
bone.mesh.lookAt(jB.position);
} else {
bone.mesh.position.copy(_vecFrom);
bone.mesh.scale.set(1, 1, len);
bone.mesh.lookAt(_vecTo);
}
bone.mesh.material.opacity = 0.85;
bone.mesh.material.emissiveIntensity = 0.3 + Math.abs(breathPulse) * 2.0;
}
}
// Body segments
for (const seg of fig.bodySegments) {
if (seg.isHead) {
const headJoint = fig.joints[seg.a];
seg.mesh.position.set(headJoint.position.x, headJoint.position.y + 0.05, headJoint.position.z);
seg.mat.opacity = 0.15;
} else {
const jA = fig.joints[seg.a];
const jB = fig.joints[seg.b];
if (jA && jB) {
const len = jA.position.distanceTo(jB.position);
seg.mesh.position.copy(jA.position);
seg.mesh.scale.set(1, 1, len);
seg.mesh.lookAt(jB.position);
seg.mat.opacity = 0.12;
}
}
seg.mat.emissiveIntensity = 0.1 + Math.abs(breathPulse) * 0.4;
}
// Aura — adapt shape to pose
const hipY = (fig.joints[11].position.y + fig.joints[12].position.y) / 2;
const cx = (fig.joints[11].position.x + fig.joints[12].position.x) / 2;
const cz = (fig.joints[11].position.z + fig.joints[12].position.z) / 2;
fig.aura.position.set(cx, hipY, cz);
fig.auraMat.opacity = this._settings.aura + Math.abs(breathPulse) * 0.8;
// Pose-adaptive aura: compute from actual keypoint spread
const auraShape = this._computeAuraShape(fig, pose, breathPulse);
fig.aura.scale.set(auraShape.scaleX, auraShape.scaleY, auraShape.scaleZ);
// Person light
fig.personLight.position.set(pos[0], 1.2, pos[2]);
fig.personLight.intensity = this._settings.glow * 0.4;
fig._lastPose = pose;
}
/**
* Compute pose-adaptive aura shape based on actual keypoint spread.
* Wider for exercise/spread poses, narrower for crouching/compact poses.
*/
_computeAuraShape(fig, pose, breathPulse) {
// Measure horizontal spread from shoulders and hips
const lShoulder = fig.joints[5].position;
const rShoulder = fig.joints[6].position;
const lHip = fig.joints[11].position;
const rHip = fig.joints[12].position;
const nose = fig.joints[0].position;
const lAnkle = fig.joints[15].position;
const rAnkle = fig.joints[16].position;
// Horizontal spread (X-Z plane)
const shoulderWidth = Math.sqrt(
(rShoulder.x - lShoulder.x) ** 2 +
(rShoulder.z - lShoulder.z) ** 2
);
const ankleWidth = Math.sqrt(
(rAnkle.x - lAnkle.x) ** 2 +
(rAnkle.z - lAnkle.z) ** 2
);
const maxWidth = Math.max(shoulderWidth, ankleWidth);
// Vertical extent
const headY = nose.y;
const footY = Math.min(lAnkle.y, rAnkle.y);
const height = headY - footY;
// Normalize to base aura dimensions
const baseWidth = 0.44; // default shoulder width
const baseHeight = 1.7; // default standing height
const widthRatio = Math.max(0.6, Math.min(2.0, maxWidth / baseWidth));
const heightRatio = Math.max(0.4, Math.min(1.3, height / baseHeight));
// Breathing modulation
const breathMod = 1 + breathPulse * 2;
return {
scaleX: widthRatio * breathMod,
scaleY: heightRatio * breathMod,
scaleZ: widthRatio * breathMod,
};
}
/**
* Hide a figure by fading all materials to invisible.
* @param {object} fig - Figure object to hide
*/
hide(fig) {
for (const j of fig.joints) {
j.material.opacity = 0;
if (j._haloMat) j._haloMat.opacity = 0;
if (j._glow) j._glow.intensity = 0;
}
for (const b of fig.bones) b.mesh.material.opacity = 0;
for (const seg of fig.bodySegments) seg.mat.opacity = 0;
fig.auraMat.opacity = 0;
fig.personLight.intensity = 0;
fig._initialized = false;
}
/**
* Apply wire and joint colors to all figures in the pool.
* @param {THREE.Color} wireColor
* @param {THREE.Color} jointColor
*/
applyColors(wireColor, jointColor) {
for (const fig of this._figures) {
for (let i = 0; i < fig.joints.length; i++) {
const j = fig.joints[i];
if (i === 0) {
j.material.color.copy(wireColor);
j.material.emissive.copy(wireColor);
} else {
j.material.color.copy(jointColor);
j.material.emissive.copy(jointColor);
}
if (j._haloMat) j._haloMat.color.copy(jointColor);
if (j._glow) j._glow.color.copy(jointColor);
}
for (const b of fig.bones) {
b.mesh.material.color.copy(wireColor);
b.mesh.material.emissive.copy(wireColor);
}
for (const seg of fig.bodySegments) {
seg.mat.color.copy(wireColor);
seg.mat.emissive.copy(wireColor);
}
fig.auraMat.color.copy(wireColor);
fig.personLight.color.copy(wireColor);
}
}
}

121
ui/observatory/js/holographic-panel.js Normal file
View File

@ -0,0 +1,121 @@
/**
* Holographic Panel — Reusable frame with border shader, scan line, title
*/
import * as THREE from 'three';
const BORDER_VERTEX = `
varying vec2 vUv;
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
`;
const BORDER_FRAGMENT = `
uniform float uTime;
uniform vec3 uColor;
varying vec2 vUv;
void main() {
// Thin border
float bx = step(vUv.x, 0.015) + step(1.0 - 0.015, vUv.x);
float by = step(vUv.y, 0.02) + step(1.0 - 0.02, vUv.y);
float border = clamp(bx + by, 0.0, 1.0);
// Scan line moving upward
float scan = smoothstep(0.0, 0.02, abs(vUv.y - fract(uTime * 0.15))) ;
scan = 1.0 - (1.0 - scan) * 0.4;
// Corner accents
float corner = 0.0;
float cx = min(vUv.x, 1.0 - vUv.x);
float cy = min(vUv.y, 1.0 - vUv.y);
if (cx < 0.06 && cy < 0.08) corner = 0.6;
// Subtle fill
float fill = 0.03 + corner * 0.05;
float alpha = max(border * 0.7, fill) * scan;
gl_FragColor = vec4(uColor, alpha);
}
`;
export class HolographicPanel {
/**
* @param {Object} opts
* @param {number[]} opts.position - [x, y, z]
* @param {number} opts.width
* @param {number} opts.height
* @param {string} opts.title
* @param {number} [opts.color=0x00d4ff]
*/
constructor(opts) {
this.group = new THREE.Group();
this.group.position.set(...opts.position);
const color = new THREE.Color(opts.color || 0x00d4ff);
// Border plane
this._uniforms = {
uTime: { value: 0 },
uColor: { value: color },
};
const borderGeo = new THREE.PlaneGeometry(opts.width, opts.height);
const borderMat = new THREE.ShaderMaterial({
vertexShader: BORDER_VERTEX,
fragmentShader: BORDER_FRAGMENT,
uniforms: this._uniforms,
transparent: true,
side: THREE.DoubleSide,
depthWrite: false,
blending: THREE.AdditiveBlending,
});
this._border = new THREE.Mesh(borderGeo, borderMat);
this.group.add(this._border);
// Title sprite
if (opts.title) {
const canvas = document.createElement('canvas');
canvas.width = 512;
canvas.height = 64;
const ctx = canvas.getContext('2d');
ctx.fillStyle = 'transparent';
ctx.fillRect(0, 0, 512, 64);
ctx.font = '600 28px "Courier New", monospace';
ctx.fillStyle = `#${color.getHexString()}`;
ctx.textAlign = 'center';
ctx.fillText(opts.title.toUpperCase(), 256, 42);
const tex = new THREE.CanvasTexture(canvas);
const spriteMat = new THREE.SpriteMaterial({
map: tex,
transparent: true,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
const sprite = new THREE.Sprite(spriteMat);
sprite.scale.set(opts.width * 0.8, opts.width * 0.1, 1);
sprite.position.y = opts.height / 2 + 0.3;
this.group.add(sprite);
this._titleSprite = sprite;
this._titleTex = tex;
}
}
update(dt, elapsed) {
this._uniforms.uTime.value = elapsed;
}
/** Make panel face camera */
lookAt(cameraPos) {
this.group.lookAt(cameraPos);
}
dispose() {
this._border.geometry.dispose();
this._border.material.dispose();
if (this._titleTex) this._titleTex.dispose();
if (this._titleSprite) this._titleSprite.material.dispose();
}
}

115
ui/observatory/js/nebula-background.js Normal file
View File

@ -0,0 +1,115 @@
/**
* Room Atmosphere Background — Warm dark gradient with subtle particles
* Matches RuView Foundation aesthetic: deep blue-black with warm undertones
*/
import * as THREE from 'three';
const BG_VERTEX = `
varying vec3 vWorldPos;
void main() {
vWorldPos = (modelMatrix * vec4(position, 1.0)).xyz;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
`;
const BG_FRAGMENT = `
uniform float uTime;
uniform float uOctaves;
varying vec3 vWorldPos;
vec3 hash33(vec3 p) {
p = fract(p * vec3(443.8975, 397.2973, 491.1871));
p += dot(p, p.yxz + 19.19);
return fract(vec3(p.x * p.y, p.y * p.z, p.z * p.x));
}
float noise3d(vec3 p) {
vec3 i = floor(p);
vec3 f = fract(p);
f = f * f * (3.0 - 2.0 * f);
float n = mix(
mix(mix(dot(hash33(i), f), dot(hash33(i + vec3(1,0,0)), f - vec3(1,0,0)), f.x),
mix(dot(hash33(i + vec3(0,1,0)), f - vec3(0,1,0)), dot(hash33(i + vec3(1,1,0)), f - vec3(1,1,0)), f.x), f.y),
mix(mix(dot(hash33(i + vec3(0,0,1)), f - vec3(0,0,1)), dot(hash33(i + vec3(1,0,1)), f - vec3(1,0,1)), f.x),
mix(dot(hash33(i + vec3(0,1,1)), f - vec3(0,1,1)), dot(hash33(i + vec3(1,1,1)), f - vec3(1,1,1)), f.x), f.y),
f.z);
return n * 0.5 + 0.5;
}
float fbm(vec3 p, float octaves) {
float v = 0.0, a = 0.5;
for (float i = 0.0; i < 5.0; i++) {
if (i >= octaves) break;
v += a * noise3d(p);
p *= 2.0;
a *= 0.5;
}
return v;
}
void main() {
vec3 dir = normalize(vWorldPos);
// Warm dark atmosphere with subtle color variation
float n1 = fbm(dir * 2.5 + uTime * 0.008, uOctaves);
float n2 = fbm(dir * 4.0 - uTime * 0.005, max(1.0, uOctaves - 1.0));
// Foundation palette: deep blue-black with warm undertones
vec3 deepBlack = vec3(0.03, 0.04, 0.06);
vec3 warmNavy = vec3(0.04, 0.05, 0.10);
vec3 greenTint = vec3(0.01, 0.06, 0.04);
vec3 bg = mix(deepBlack, warmNavy, n1 * 0.5);
bg = mix(bg, greenTint, n2 * 0.15);
// Subtle top-down gradient (lighter ceiling)
float upFactor = max(0.0, dir.y) * 0.08;
bg += vec3(0.02, 0.03, 0.05) * upFactor;
// Very subtle dim stars (distant)
vec3 c = floor(dir * 200.0);
vec3 h = hash33(c);
float star = step(0.998, h.x) * h.y * 0.15;
star *= 0.7 + 0.3 * sin(uTime * 1.5 + h.z * 80.0);
bg += vec3(0.6, 0.7, 0.8) * star;
gl_FragColor = vec4(bg, 1.0);
}
`;
export class NebulaBackground {
constructor(scene) {
this._octaves = 4;
this.uniforms = {
uTime: { value: 0 },
uOctaves: { value: this._octaves },
};
const geo = new THREE.SphereGeometry(150, 32, 32);
const mat = new THREE.ShaderMaterial({
vertexShader: BG_VERTEX,
fragmentShader: BG_FRAGMENT,
uniforms: this.uniforms,
side: THREE.BackSide,
depthWrite: false,
});
this.mesh = new THREE.Mesh(geo, mat);
scene.add(this.mesh);
}
update(dt, elapsed) {
this.uniforms.uTime.value = elapsed;
}
setQuality(level) {
this._octaves = [2, 3, 4][level] || 4;
this.uniforms.uOctaves.value = this._octaves;
}
dispose() {
this.mesh.geometry.dispose();
this.mesh.material.dispose();
}
}

170
ui/observatory/js/phase-constellation.js Normal file
View File

@ -0,0 +1,170 @@
/**
* Module D — "The Phase Constellation"
* I/Q star map with constellation lines and rotating temporal view
*/
import * as THREE from 'three';
const NUM_SUBCARRIERS = 64;
export class PhaseConstellation {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
// Star points (current frame)
const starGeo = new THREE.BufferGeometry();
this._positions = new Float32Array(NUM_SUBCARRIERS * 3);
this._colors = new Float32Array(NUM_SUBCARRIERS * 3);
this._sizes = new Float32Array(NUM_SUBCARRIERS);
starGeo.setAttribute('position', new THREE.BufferAttribute(this._positions, 3));
starGeo.setAttribute('color', new THREE.BufferAttribute(this._colors, 3));
starGeo.setAttribute('size', new THREE.BufferAttribute(this._sizes, 1));
const starMat = new THREE.PointsMaterial({
size: 0.12,
vertexColors: true,
transparent: true,
opacity: 0.9,
blending: THREE.AdditiveBlending,
depthWrite: false,
sizeAttenuation: true,
});
this._stars = new THREE.Points(starGeo, starMat);
this.group.add(this._stars);
// Ghost layer (previous frame)
const ghostGeo = new THREE.BufferGeometry();
this._ghostPos = new Float32Array(NUM_SUBCARRIERS * 3);
ghostGeo.setAttribute('position', new THREE.BufferAttribute(this._ghostPos, 3));
const ghostMat = new THREE.PointsMaterial({
color: 0x00d4ff,
size: 0.06,
transparent: true,
opacity: 0.2,
blending: THREE.AdditiveBlending,
depthWrite: false,
sizeAttenuation: true,
});
this._ghosts = new THREE.Points(ghostGeo, ghostMat);
this.group.add(this._ghosts);
// Constellation lines (connecting adjacent subcarriers)
const lineGeo = new THREE.BufferGeometry();
this._linePos = new Float32Array(NUM_SUBCARRIERS * 2 * 3); // pairs
lineGeo.setAttribute('position', new THREE.BufferAttribute(this._linePos, 3));
const lineMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.15,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._lines = new THREE.LineSegments(lineGeo, lineMat);
this.group.add(this._lines);
// Axes
this._addAxes();
this._prevIQ = null;
}
_addAxes() {
const axesMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.1,
});
// I axis
const iGeo = new THREE.BufferGeometry().setFromPoints([
new THREE.Vector3(-2.5, 0, 0),
new THREE.Vector3(2.5, 0, 0),
]);
this.group.add(new THREE.Line(iGeo, axesMat));
// Q axis
const qGeo = new THREE.BufferGeometry().setFromPoints([
new THREE.Vector3(0, -2.5, 0),
new THREE.Vector3(0, 2.5, 0),
]);
this.group.add(new THREE.Line(qGeo, axesMat));
}
update(dt, elapsed, data) {
const iq = data?._observatory?.subcarrier_iq;
const variance = data?._observatory?.per_subcarrier_variance;
const amplitude = data?.nodes?.[0]?.amplitude;
// Slow Y rotation for temporal evolution
this.group.rotation.y = elapsed * 0.05;
if (!iq || iq.length < NUM_SUBCARRIERS) return;
// Copy current to ghost
this._ghostPos.set(this._positions);
this._ghosts.geometry.attributes.position.needsUpdate = true;
// Update current positions from I/Q
for (let s = 0; s < NUM_SUBCARRIERS; s++) {
const i3 = s * 3;
const iVal = (iq[s]?.i || 0) * 4; // scale for visibility
const qVal = (iq[s]?.q || 0) * 4;
this._positions[i3] = iVal;
this._positions[i3 + 1] = qVal;
this._positions[i3 + 2] = 0;
// Size from amplitude
const amp = amplitude ? (amplitude[s % amplitude.length] || 0.1) : 0.1;
this._sizes[s] = 0.06 + amp * 0.15;
// Color from variance: blue(low) -> amber(high)
const v = variance ? Math.min(1, (variance[s] || 0) * 2) : 0;
this._colors[i3] = v * 1.0; // R
this._colors[i3 + 1] = 0.5 + v * 0.3; // G
this._colors[i3 + 2] = 1.0 - v * 0.7; // B
}
this._stars.geometry.attributes.position.needsUpdate = true;
this._stars.geometry.attributes.color.needsUpdate = true;
this._stars.geometry.attributes.size.needsUpdate = true;
// Update constellation lines
for (let s = 0; s < NUM_SUBCARRIERS - 1; s++) {
const li = s * 6;
const i3a = s * 3;
const i3b = (s + 1) * 3;
this._linePos[li] = this._positions[i3a];
this._linePos[li + 1] = this._positions[i3a + 1];
this._linePos[li + 2] = this._positions[i3a + 2];
this._linePos[li + 3] = this._positions[i3b];
this._linePos[li + 4] = this._positions[i3b + 1];
this._linePos[li + 5] = this._positions[i3b + 2];
}
// Last pair: wrap around
const lastLi = (NUM_SUBCARRIERS - 1) * 6;
const lastI3 = (NUM_SUBCARRIERS - 1) * 3;
this._linePos[lastLi] = this._positions[lastI3];
this._linePos[lastLi + 1] = this._positions[lastI3 + 1];
this._linePos[lastLi + 2] = this._positions[lastI3 + 2];
this._linePos[lastLi + 3] = this._positions[0];
this._linePos[lastLi + 4] = this._positions[1];
this._linePos[lastLi + 5] = this._positions[2];
this._lines.geometry.attributes.position.needsUpdate = true;
}
dispose() {
this._stars.geometry.dispose();
this._stars.material.dispose();
this._ghosts.geometry.dispose();
this._ghosts.material.dispose();
this._lines.geometry.dispose();
this._lines.material.dispose();
}
}

567
ui/observatory/js/pose-system.js Normal file
View File

@ -0,0 +1,567 @@
/**
* PoseSystem -- Stateless pose keypoint generator for COCO 17-keypoint format.
*
* Keypoint indices:
* 0:nose 1:left_eye 2:right_eye 3:left_ear 4:right_ear
* 5:left_shoulder 6:right_shoulder 7:left_elbow 8:right_elbow
* 9:left_wrist 10:right_wrist 11:left_hip 12:right_hip
* 13:left_knee 14:right_knee 15:left_ankle 16:right_ankle
*
* Every public method is a pure function: parameters in, keypoint array out.
*/
export class PoseSystem {
// ---- Entry point -------------------------------------------------------
generateKeypoints(person, elapsed, breathPulse) {
const pose = person.pose || 'standing';
const pos = person.position || [0, 0, 0];
const facing = person.facing || 0;
const px = pos[0], pz = pos[2];
const ms = person.motion_score || 0;
const bp = breathPulse;
let kps;
switch (pose) {
case 'lying': kps = this.poseLying(px, pos[1] || 0, pz, elapsed, bp); break;
case 'sitting': kps = this.poseSitting(px, pz, elapsed, bp); break;
case 'fallen': kps = this.poseFallen(px, pz, elapsed); break;
case 'falling': kps = this.poseFalling(px, pz, elapsed, person.fallProgress || 0); break;
case 'exercising': kps = this.poseExercising(px, pz, elapsed, person.exerciseType, person.exerciseTime); break;
case 'gesturing': kps = this.poseGesturing(px, pz, elapsed, person.gestureType, person.gestureIntensity || 0); break;
case 'crouching': kps = this.poseCrouching(px, pz, elapsed, bp); break;
case 'walking': kps = this.poseWalking(px, pz, elapsed, ms, bp); break;
case 'standing':
default: kps = this.poseStanding(px, pz, elapsed, ms, bp); break;
}
// Apply facing rotation
if (Math.abs(facing) > 0.01) {
this.rotateKps(kps, px, pz, facing);
}
return kps;
}
// ---- Rotation utility --------------------------------------------------
rotateKps(kps, cx, cz, angle) {
const cos = Math.cos(angle), sin = Math.sin(angle);
for (const kp of kps) {
const dx = kp[0] - cx, dz = kp[2] - cz;
kp[0] = cx + dx * cos - dz * sin;
kp[2] = cz + dx * sin + dz * cos;
}
}
// ---- Standing ----------------------------------------------------------
// Weight shift between feet, idle head look-around, breathing
poseStanding(px, pz, elapsed, ms, bp) {
// Slow weight shift side to side
const weightShift = Math.sin(elapsed * 0.6) * 0.012;
// Idle head look around
const headTurn = Math.sin(elapsed * 0.3) * 0.015;
const headTilt = Math.cos(elapsed * 0.25) * 0.008;
// Slight sway from micro-balance adjustments
const sway = Math.sin(elapsed * 0.8) * 0.005 + weightShift;
// Knee bend alternation with weight shift
const leftKneeBend = Math.max(0, Math.sin(elapsed * 0.6)) * 0.015;
const rightKneeBend = Math.max(0, -Math.sin(elapsed * 0.6)) * 0.015;
return [
[px + sway + headTurn, 1.72 + bp + headTilt, pz], // 0 nose
[px - 0.03 + sway + headTurn, 1.74 + bp + headTilt, pz - 0.02], // 1 left eye
[px + 0.03 + sway + headTurn, 1.74 + bp + headTilt, pz - 0.02], // 2 right eye
[px - 0.07 + headTurn * 0.5, 1.72 + bp, pz], // 3 left ear
[px + 0.07 + headTurn * 0.5, 1.72 + bp, pz], // 4 right ear
[px - 0.22 + weightShift * 0.3, 1.48 + bp, pz], // 5 left shoulder
[px + 0.22 + weightShift * 0.3, 1.48 + bp, pz], // 6 right shoulder
[px - 0.24 + weightShift * 0.2, 1.18 + bp, pz + 0.02], // 7 left elbow
[px + 0.24 + weightShift * 0.2, 1.18 + bp, pz - 0.02], // 8 right elbow
[px - 0.22 + weightShift * 0.15, 0.92 + bp, pz + 0.05], // 9 left wrist
[px + 0.22 + weightShift * 0.15, 0.92 + bp, pz - 0.05], // 10 right wrist
[px - 0.11 + weightShift * 0.5, 0.98 + bp, pz], // 11 left hip
[px + 0.11 + weightShift * 0.5, 0.98 + bp, pz], // 12 right hip
[px - 0.12 + weightShift * 0.3, 0.52 + leftKneeBend, pz], // 13 left knee
[px + 0.12 + weightShift * 0.3, 0.52 + rightKneeBend, pz], // 14 right knee
[px - 0.12 + weightShift * 0.4, 0.04, pz], // 15 left ankle
[px + 0.12 + weightShift * 0.4, 0.04, pz], // 16 right ankle
];
}
// ---- Walking -----------------------------------------------------------
// Torso rotation, head bob, natural arm pendulum with elbow bend
poseWalking(px, pz, elapsed, ms, bp) {
const speed = Math.min(ms / 100, 2.5);
const wp = elapsed * speed * 1.8;
const sFactor = Math.min(speed, 1);
// Leg stride
const legStride = Math.sin(wp) * 0.25 * sFactor;
const legBack = Math.sin(wp + Math.PI) * 0.25 * sFactor;
const kneeAmt = Math.abs(Math.sin(wp)) * 0.08;
// Natural arm pendulum -- opposite to legs, with elbow bend
const armPhase = Math.sin(wp);
const armSwingL = -armPhase * 0.3 * sFactor; // left arm opposite right leg
const armSwingR = armPhase * 0.3 * sFactor;
const elbowBendL = Math.max(0, -armPhase) * 0.12 * sFactor; // bend on backswing
const elbowBendR = Math.max(0, armPhase) * 0.12 * sFactor;
// Torso twist (shoulders rotate opposite to hips)
const torsoTwist = Math.sin(wp) * 0.03 * sFactor;
// Vertical bob (double frequency -- peak at mid-stance)
const bob = Math.abs(Math.sin(wp)) * 0.025;
// Head bob -- slight lag behind body
const headBob = Math.abs(Math.sin(wp - 0.2)) * 0.015;
const headLean = Math.sin(wp) * 0.008;
return [
[px + headLean, 1.72 + bp + bob + headBob, pz], // 0 nose
[px - 0.03 + headLean, 1.74 + bp + bob + headBob, pz - 0.02], // 1 left eye
[px + 0.03 + headLean, 1.74 + bp + bob + headBob, pz - 0.02], // 2 right eye
[px - 0.07, 1.72 + bp + bob + headBob, pz], // 3 left ear
[px + 0.07, 1.72 + bp + bob + headBob, pz], // 4 right ear
[px - 0.22 - torsoTwist, 1.48 + bp + bob, pz], // 5 left shoulder (twist)
[px + 0.22 - torsoTwist, 1.48 + bp + bob, pz], // 6 right shoulder
[px - 0.28 + armSwingL * 0.3, 1.18 + bp + bob - elbowBendL, pz + armSwingL * 0.3], // 7 left elbow
[px + 0.28 + armSwingR * 0.3, 1.18 + bp + bob - elbowBendR, pz + armSwingR * 0.3], // 8 right elbow
[px - 0.26 + armSwingL * 0.6, 0.92 + bp + bob - elbowBendL * 1.5, pz + armSwingL * 0.5], // 9 left wrist
[px + 0.26 + armSwingR * 0.6, 0.92 + bp + bob - elbowBendR * 1.5, pz + armSwingR * 0.5], // 10 right wrist
[px - 0.11 + torsoTwist * 0.5, 0.98 + bp + bob, pz], // 11 left hip (counter-twist)
[px + 0.11 + torsoTwist * 0.5, 0.98 + bp + bob, pz], // 12 right hip
[px - 0.12 + legStride * 0.3, 0.52 + kneeAmt, pz + legStride], // 13 left knee
[px + 0.12 + legBack * 0.3, 0.52 + kneeAmt, pz + legBack], // 14 right knee
[px - 0.12 + legStride * 0.6, 0.04, pz + legStride * 1.5], // 15 left ankle
[px + 0.12 + legBack * 0.6, 0.04, pz + legBack * 1.5], // 16 right ankle
];
}
// ---- Lying -------------------------------------------------------------
// Subtle micro-movements, differentiate supine vs side-lying via elapsed hash
poseLying(px, surfaceY, pz, elapsed, bp) {
const y = (surfaceY || 0) + 0.2;
const chest = bp * 0.015;
// Micro-movements -- tiny random-feeling shifts (deterministic from elapsed)
const microX = Math.sin(elapsed * 0.17) * 0.004;
const microZ = Math.cos(elapsed * 0.13) * 0.003;
const fingerTwitch = Math.sin(elapsed * 0.7) * 0.008;
// Determine supine vs side-lying from a slow oscillation (stays one way for ~20s)
const lyingMode = Math.sin(elapsed * 0.05);
if (lyingMode > 0.3) {
// Side-lying (on left side)
const curl = Math.sin(elapsed * 0.1) * 0.02; // slight fetal curl
return [
[px - 0.72 + microX, y + 0.12, pz - 0.08], // 0 nose (turned)
[px - 0.70, y + 0.14, pz - 0.10], // 1 left eye
[px - 0.70, y + 0.16, pz - 0.06], // 2 right eye (up)
[px - 0.76, y + 0.11, pz - 0.12], // 3 left ear (down)
[px - 0.76, y + 0.14, pz - 0.04], // 4 right ear
[px - 0.45, y + chest + 0.05, pz - 0.12], // 5 left shoulder (down)
[px - 0.45, y + chest + 0.2, pz + 0.04], // 6 right shoulder (up)
[px - 0.38, y + 0.02, pz - 0.28 + curl], // 7 left elbow
[px - 0.35, y + 0.18, pz + 0.15 + fingerTwitch], // 8 right elbow
[px - 0.20, y - 0.01, pz - 0.30 + curl], // 9 left wrist
[px - 0.18, y + 0.12, pz + 0.25 + fingerTwitch], // 10 right wrist
[px + 0.05 + microX, y + chest * 0.4 + 0.03, pz - 0.08], // 11 left hip
[px + 0.05 + microX, y + chest * 0.4 + 0.12, pz + 0.06], // 12 right hip
[px + 0.40 + curl * 2, y + 0.02, pz - 0.14 + curl], // 13 left knee
[px + 0.38 + curl * 2, y + 0.10, pz + 0.10 + curl], // 14 right knee
[px + 0.75, y - 0.01, pz - 0.12], // 15 left ankle
[px + 0.72, y + 0.04, pz + 0.08], // 16 right ankle
];
}
// Supine (face up) -- default
return [
[px - 0.75 + microX, y + 0.08, pz + microZ], // 0 nose
[px - 0.72, y + 0.1, pz - 0.02 + microZ], // 1 left eye
[px - 0.72, y + 0.1, pz + 0.02 + microZ], // 2 right eye
[px - 0.78, y + 0.08, pz - 0.05], // 3 left ear
[px - 0.78, y + 0.08, pz + 0.05], // 4 right ear
[px - 0.45, y + chest, pz - 0.18], // 5 left shoulder
[px - 0.45, y + chest, pz + 0.18], // 6 right shoulder
[px - 0.42, y, pz - 0.35 + fingerTwitch], // 7 left elbow
[px - 0.42, y, pz + 0.35 - fingerTwitch], // 8 right elbow
[px - 0.2, y - 0.02, pz - 0.38 + fingerTwitch], // 9 left wrist
[px - 0.2, y - 0.02, pz + 0.38 - fingerTwitch], // 10 right wrist
[px + 0.05 + microX, y + chest * 0.5, pz - 0.1], // 11 left hip
[px + 0.05 + microX, y + chest * 0.5, pz + 0.1], // 12 right hip
[px + 0.45, y, pz - 0.11], // 13 left knee
[px + 0.45, y, pz + 0.11], // 14 right knee
[px + 0.82, y - 0.02, pz - 0.1], // 15 left ankle
[px + 0.82, y - 0.02, pz + 0.1], // 16 right ankle
];
}
// ---- Sitting -----------------------------------------------------------
// Occasional fidget, breathing chest expansion, weight shift
poseSitting(px, pz, elapsed, bp) {
const sway = Math.sin(elapsed * 0.5) * 0.003;
// Fidget: occasional hand movement (every ~6s a small gesture)
const fidgetCycle = elapsed % 6.0;
const fidgetActive = fidgetCycle > 5.2 && fidgetCycle < 5.8;
const fidgetAmt = fidgetActive ? Math.sin((fidgetCycle - 5.2) * Math.PI / 0.6) * 0.06 : 0;
// Weight shift side to side (slow)
const weightShift = Math.sin(elapsed * 0.25) * 0.008;
// Chest expansion from breathing
const chestExpand = bp * 0.008;
return [
[px + sway + weightShift, 1.15 + bp, pz], // 0 nose
[px - 0.03 + sway + weightShift, 1.17 + bp, pz - 0.02], // 1 left eye
[px + 0.03 + sway + weightShift, 1.17 + bp, pz - 0.02], // 2 right eye
[px - 0.07 + weightShift, 1.15 + bp, pz], // 3 left ear
[px + 0.07 + weightShift, 1.15 + bp, pz], // 4 right ear
[px - 0.20 - chestExpand + weightShift, 0.95 + bp, pz], // 5 left shoulder
[px + 0.20 + chestExpand + weightShift, 0.95 + bp, pz], // 6 right shoulder
[px - 0.25 + weightShift, 0.72 + bp, pz + 0.08], // 7 left elbow
[px + 0.25 + weightShift, 0.72 + bp, pz + 0.08], // 8 right elbow
[px - 0.18 + fidgetAmt, 0.55 + fidgetAmt * 0.3, pz + 0.15], // 9 left wrist (fidgets)
[px + 0.18, 0.55, pz + 0.15], // 10 right wrist
[px - 0.11 + weightShift * 0.5, 0.48, pz + 0.02], // 11 left hip
[px + 0.11 + weightShift * 0.5, 0.48, pz + 0.02], // 12 right hip
[px - 0.12, 0.48, pz + 0.4], // 13 left knee
[px + 0.12, 0.48, pz + 0.4], // 14 right knee
[px - 0.12, 0.04, pz + 0.4], // 15 left ankle
[px + 0.12, 0.04, pz + 0.4], // 16 right ankle
];
}
// ---- Fallen ------------------------------------------------------------
// Occasional twitch/attempt to move, asymmetric breathing
poseFallen(px, pz, elapsed) {
// Irregular twitch -- sharper, less periodic
const twitchArm = Math.sin(elapsed * 0.3) * 0.003 +
Math.sin(elapsed * 1.7) * 0.008 * Math.max(0, Math.sin(elapsed * 0.15));
const twitchLeg = Math.cos(elapsed * 0.4) * 0.005 *
Math.max(0, Math.sin(elapsed * 0.2 + 1.0));
// Asymmetric breathing (one side of chest rises more)
const breathL = Math.sin(elapsed * 0.8) * 0.006;
const breathR = Math.sin(elapsed * 0.8 + 0.3) * 0.004;
// Attempt to move (slow reach every ~10s)
const attemptCycle = elapsed % 10.0;
const attempting = attemptCycle > 8.0 && attemptCycle < 9.5;
const attemptAmt = attempting ? Math.sin((attemptCycle - 8.0) * Math.PI / 1.5) * 0.05 : 0;
return [
[px + 0.35, 0.12, pz + 0.15 + twitchArm], // 0 nose
[px + 0.33, 0.14, pz + 0.13], // 1 left eye
[px + 0.37, 0.14, pz + 0.17], // 2 right eye
[px + 0.38, 0.11, pz + 0.1], // 3 left ear
[px + 0.38, 0.11, pz + 0.2], // 4 right ear
[px + 0.15, 0.15 + breathL, pz - 0.1], // 5 left shoulder
[px + 0.15, 0.2 + breathR, pz + 0.25], // 6 right shoulder
[px - 0.05, 0.08, pz - 0.25 + twitchArm], // 7 left elbow
[px + 0.3, 0.22 + attemptAmt * 0.5, pz + 0.45 + attemptAmt], // 8 right elbow (reaching)
[px - 0.15, 0.05, pz - 0.3 + twitchArm * 1.5], // 9 left wrist
[px + 0.4, 0.15 + attemptAmt, pz + 0.5 + attemptAmt * 1.5], // 10 right wrist (reaching)
[px - 0.05, 0.12, pz - 0.05], // 11 left hip
[px - 0.05, 0.12, pz + 0.15], // 12 right hip
[px - 0.2, 0.08 + twitchLeg, pz - 0.3], // 13 left knee
[px - 0.15, 0.15, pz + 0.35 + twitchLeg], // 14 right knee
[px - 0.35, 0.04, pz - 0.2], // 15 left ankle
[px - 0.3, 0.04, pz + 0.5], // 16 right ankle
];
}
// ---- Falling -----------------------------------------------------------
// Flailing arms, head snap, non-linear easing (cubic ease-in)
poseFalling(px, pz, elapsed, progress) {
const standing = this.poseStanding(px, pz, elapsed, 0, 0);
const fallen = this.poseFallen(px, pz, elapsed);
// Cubic ease-in for realistic acceleration
const t = progress * progress * progress;
// Arm flailing -- sinusoidal perturbation that peaks mid-fall then diminishes
const flailIntensity = Math.sin(progress * Math.PI) * 0.15;
const flailL = Math.sin(elapsed * 8 + progress * 5) * flailIntensity;
const flailR = Math.cos(elapsed * 8 + progress * 5) * flailIntensity;
// Head snaps back early in the fall
const headSnap = progress < 0.4 ? Math.sin(progress * Math.PI / 0.4) * 0.06 : 0;
const kps = [];
for (let i = 0; i < 17; i++) {
kps.push([
standing[i][0] * (1 - t) + fallen[i][0] * t,
standing[i][1] * (1 - t) + fallen[i][1] * t,
standing[i][2] * (1 - t) + fallen[i][2] * t,
]);
}
// Apply head snap (tilt backward)
kps[0][1] += headSnap;
kps[1][1] += headSnap * 0.9;
kps[2][1] += headSnap * 0.9;
// Apply arm flailing
kps[7][0] += flailL; kps[7][2] += flailL * 0.5; // left elbow
kps[8][0] += flailR; kps[8][2] -= flailR * 0.5; // right elbow
kps[9][0] += flailL * 1.5; kps[9][2] += flailL; // left wrist
kps[10][0] += flailR * 1.5; kps[10][2] -= flailR; // right wrist
return kps;
}
// ---- Exercising --------------------------------------------------------
poseExercising(px, pz, elapsed, exerciseType, exerciseTime) {
const et = exerciseTime || elapsed;
if (exerciseType === 'squats') {
return this._poseSquats(px, pz, et);
}
return this._poseJumpingJacks(px, pz, et);
}
// Squats: forward lean, hip hinge, arm counterbalance, depth variation
_poseSquats(px, pz, et) {
const rawPhase = (Math.sin(et * 2.5) + 1) / 2; // 0=up, 1=down
// Depth variation -- every other rep is shallower
const repIndex = Math.floor(et * 2.5 / Math.PI);
const depthMod = (repIndex % 2 === 0) ? 1.0 : 0.7;
const phase = rawPhase * depthMod;
const squat = phase * 0.5;
const armFwd = phase * 0.4;
// Forward lean increases with squat depth
const forwardLean = phase * 0.08;
// Hip hinge -- hips push back
const hipBack = phase * 0.12;
return [
[px + forwardLean * 0.3, 1.72 - squat, pz + forwardLean], // 0 nose
[px - 0.03 + forwardLean * 0.3, 1.74 - squat, pz - 0.02 + forwardLean], // 1 left eye
[px + 0.03 + forwardLean * 0.3, 1.74 - squat, pz - 0.02 + forwardLean], // 2 right eye
[px - 0.07, 1.72 - squat, pz + forwardLean * 0.8], // 3 left ear
[px + 0.07, 1.72 - squat, pz + forwardLean * 0.8], // 4 right ear
[px - 0.22, 1.48 - squat + forwardLean * 0.2, pz + forwardLean * 0.5], // 5 left shoulder
[px + 0.22, 1.48 - squat + forwardLean * 0.2, pz + forwardLean * 0.5], // 6 right shoulder
[px - 0.22, 1.25 - squat * 0.7, pz + armFwd], // 7 left elbow
[px + 0.22, 1.25 - squat * 0.7, pz + armFwd], // 8 right elbow
[px - 0.22, 1.05 - squat * 0.5, pz + armFwd * 1.5], // 9 left wrist (counterbalance)
[px + 0.22, 1.05 - squat * 0.5, pz + armFwd * 1.5], // 10 right wrist
[px - 0.11, 0.98 - squat * 0.3, pz - hipBack], // 11 left hip (pushed back)
[px + 0.11, 0.98 - squat * 0.3, pz - hipBack], // 12 right hip
[px - 0.15, 0.52 - squat * 0.1, pz + squat * 0.3], // 13 left knee
[px + 0.15, 0.52 - squat * 0.1, pz + squat * 0.3], // 14 right knee
[px - 0.13, 0.04, pz + 0.05], // 15 left ankle
[px + 0.13, 0.04, pz + 0.05], // 16 right ankle
];
}
// Jumping jacks: full arm arc, hip sway, landing impact
_poseJumpingJacks(px, pz, et) {
const rawPhase = (Math.sin(et * 3) + 1) / 2; // 0=closed, 1=open
const phase = rawPhase;
// Full arm arc -- from sides to overhead in a smooth arc
const armAngle = phase * Math.PI * 0.85; // 0 to ~153 degrees
const armX = Math.sin(armAngle) * 0.55; // lateral spread
const armY = Math.cos(armAngle) * 0.55; // vertical component
const legSpread = phase * 0.25;
// Landing impact -- brief compression at bottom of cycle
const impact = Math.max(0, -Math.sin(et * 3)) * 0.03;
const jump = Math.max(0, Math.sin(et * 3)) * 0.06;
// Hip sway at apex
const hipSway = Math.sin(et * 3) * 0.015;
return [
[px, 1.72 + jump - impact, pz], // 0 nose
[px - 0.03, 1.74 + jump - impact, pz - 0.02], // 1 left eye
[px + 0.03, 1.74 + jump - impact, pz - 0.02], // 2 right eye
[px - 0.07, 1.72 + jump - impact, pz], // 3 left ear
[px + 0.07, 1.72 + jump - impact, pz], // 4 right ear
[px - 0.22, 1.48 + jump - impact, pz], // 5 left shoulder
[px + 0.22, 1.48 + jump - impact, pz], // 6 right shoulder
[px - 0.22 - armX * 0.6, 1.48 - armY * 0.3 + jump, pz], // 7 left elbow (arc)
[px + 0.22 + armX * 0.6, 1.48 - armY * 0.3 + jump, pz], // 8 right elbow
[px - 0.22 - armX, 1.48 - armY + 0.55 + jump, pz], // 9 left wrist (arc)
[px + 0.22 + armX, 1.48 - armY + 0.55 + jump, pz], // 10 right wrist
[px - 0.11 + hipSway, 0.98 + jump - impact, pz], // 11 left hip
[px + 0.11 + hipSway, 0.98 + jump - impact, pz], // 12 right hip
[px - 0.12 - legSpread, 0.52 + jump * 0.5 - impact * 0.5, pz], // 13 left knee
[px + 0.12 + legSpread, 0.52 + jump * 0.5 - impact * 0.5, pz], // 14 right knee
[px - 0.13 - legSpread * 1.3, 0.04 - impact * 0.3, pz], // 15 left ankle
[px + 0.13 + legSpread * 1.3, 0.04 - impact * 0.3, pz], // 16 right ankle
];
}
// ---- Gesturing ---------------------------------------------------------
poseGesturing(px, pz, elapsed, gestureType, intensity) {
const base = this.poseStanding(px, pz, elapsed, 0, 0);
if (intensity <= 0) return base;
const gt = elapsed;
switch (gestureType) {
case 'wave':
return this._gestureWave(base, px, pz, gt, intensity);
case 'swipe_left':
return this._gestureSwipe(base, px, pz, gt, intensity);
case 'circle':
return this._gestureCircle(base, px, pz, gt, intensity);
case 'point':
return this._gesturePoint(base, px, pz, gt, intensity);
default:
return base;
}
}
// Wave: fluid hand oscillation, elbow pivot, slight shoulder raise
_gestureWave(base, px, pz, gt, intensity) {
const wave = Math.sin(gt * 6) * 0.15 * intensity;
const waveSmooth = Math.sin(gt * 6 + 0.3) * 0.08 * intensity; // secondary harmonic
const shoulderRaise = 0.04 * intensity;
const elbowPivot = Math.sin(gt * 3) * 0.03 * intensity;
// Shoulder rises slightly during wave
base[6][1] += shoulderRaise;
// Elbow raised and pivoting
base[8] = [
px + 0.32 + elbowPivot,
1.55 * intensity + 1.18 * (1 - intensity) + shoulderRaise,
pz + 0.05,
];
// Wrist oscillates fluidly
base[10] = [
px + 0.32 + wave + waveSmooth * 0.3,
1.7 * intensity + 0.92 * (1 - intensity) + shoulderRaise,
pz + 0.08 + waveSmooth,
];
// Slight body lean away from waving arm
base[0][0] -= 0.01 * intensity;
base[5][0] -= 0.008 * intensity;
return base;
}
// Swipe: full body rotation follow-through, arm extension
_gestureSwipe(base, px, pz, gt, intensity) {
const sweep = Math.sin(gt * 2) * intensity;
// Body rotation follows the arm
const bodyRotation = sweep * 0.04;
const shoulderTwist = sweep * 0.025;
// Upper body rotates
for (let i = 0; i <= 4; i++) base[i][0] += bodyRotation * 0.5;
base[5][0] -= shoulderTwist;
base[6][0] += shoulderTwist;
// Arm extends fully during swipe
base[8] = [px + 0.15 + sweep * 0.4, 1.3, pz + 0.3];
base[10] = [px - 0.1 + sweep * 0.6, 1.3, pz + 0.55];
// Hip counter-rotation
base[11][0] += bodyRotation * -0.2;
base[12][0] += bodyRotation * -0.2;
return base;
}
// Circle: smooth circular motion with forearm rotation
_gestureCircle(base, px, pz, gt, intensity) {
const angle = gt * 2.5;
const radius = 0.25 * intensity;
const cx = Math.cos(angle) * radius;
const cy = Math.sin(angle) * radius;
// Forearm rotation -- wrist traces a smaller secondary circle
const forearmAngle = angle * 1.5;
const forearmR = 0.06 * intensity;
base[8] = [
px + 0.3 + cx * 0.5,
1.3 + cy * 0.5,
pz + 0.2 + Math.sin(angle) * 0.05,
];
base[10] = [
px + 0.3 + cx + Math.cos(forearmAngle) * forearmR,
1.3 + cy + Math.sin(forearmAngle) * forearmR,
pz + 0.35 + Math.sin(angle) * 0.08,
];
// Slight shoulder movement following arm
base[6][0] += cx * 0.08;
base[6][1] += cy * 0.04;
return base;
}
// Point: extended index finger simulation with arm sway
_gesturePoint(base, px, pz, gt, intensity) {
const point = intensity;
// Slight arm sway -- breathing/holding still
const sway = Math.sin(gt * 1.5) * 0.01 * intensity;
const vertSway = Math.cos(gt * 1.2) * 0.008 * intensity;
base[8] = [px + 0.15 + sway, 1.35 + vertSway, pz + 0.35 * point];
base[10] = [px + 0.08 + sway * 0.5, 1.38 + vertSway * 0.5, pz + 0.70 * point];
// Lean slightly toward point direction
base[0][2] += 0.02 * point;
base[5][2] += 0.01 * point;
base[6][2] += 0.01 * point;
return base;
}
// ---- Crouching ---------------------------------------------------------
// Stealth-crawl option, weight transfer between legs
poseCrouching(px, pz, elapsed, bp) {
const sway = Math.sin(elapsed * 1.5) * 0.005;
// Weight transfer between legs (slow rocking)
const weightTransfer = Math.sin(elapsed * 0.8) * 0.025;
const leftDown = Math.max(0, weightTransfer) * 0.03;
const rightDown = Math.max(0, -weightTransfer) * 0.03;
// Stealth-crawl micro-movement (slow forward creep every ~4s)
const crawlCycle = elapsed % 4.0;
const crawlActive = crawlCycle > 3.0;
const crawlAmt = crawlActive ? Math.sin((crawlCycle - 3.0) * Math.PI) * 0.02 : 0;
// Arms adjust for balance during weight transfer
const armBalance = weightTransfer * 0.3;
return [
[px + sway, 1.05 + bp, pz + 0.15 + crawlAmt], // 0 nose
[px - 0.03, 1.07 + bp, pz + 0.13 + crawlAmt], // 1 left eye
[px + 0.03, 1.07 + bp, pz + 0.13 + crawlAmt], // 2 right eye
[px - 0.07, 1.05 + bp, pz + 0.12 + crawlAmt], // 3 left ear
[px + 0.07, 1.05 + bp, pz + 0.12 + crawlAmt], // 4 right ear
[px - 0.22, 0.88 + bp, pz + 0.05], // 5 left shoulder
[px + 0.22, 0.88 + bp, pz + 0.05], // 6 right shoulder
[px - 0.28 - armBalance, 0.65 + bp, pz + 0.15 + crawlAmt * 0.5], // 7 left elbow
[px + 0.28 + armBalance, 0.65 + bp, pz + 0.15 + crawlAmt * 0.5], // 8 right elbow
[px - 0.22 - armBalance * 0.5, 0.48, pz + 0.2 + crawlAmt], // 9 left wrist
[px + 0.22 + armBalance * 0.5, 0.48, pz + 0.2 + crawlAmt], // 10 right wrist
[px - 0.12 + weightTransfer, 0.42, pz - 0.05], // 11 left hip
[px + 0.12 + weightTransfer, 0.42, pz - 0.05], // 12 right hip
[px - 0.15 + weightTransfer * 0.5, 0.35 - leftDown, pz + 0.25], // 13 left knee
[px + 0.15 + weightTransfer * 0.5, 0.35 - rightDown, pz + 0.25], // 14 right knee
[px - 0.13, 0.04, pz + 0.1], // 15 left ankle
[px + 0.13, 0.04, pz + 0.1], // 16 right ankle
];
}
}

125
ui/observatory/js/post-processing.js Normal file
View File

@ -0,0 +1,125 @@
/**
* Post-Processing — Subtle bloom for green glow wireframe,
* warm vignette, minimal grain. Foundation-style.
*/
import * as THREE from 'three';
import { EffectComposer } from 'three/addons/postprocessing/EffectComposer.js';
import { RenderPass } from 'three/addons/postprocessing/RenderPass.js';
import { UnrealBloomPass } from 'three/addons/postprocessing/UnrealBloomPass.js';
import { ShaderPass } from 'three/addons/postprocessing/ShaderPass.js';
const VignetteShader = {
uniforms: {
tDiffuse: { value: null },
uTime: { value: 0 },
uVignetteStrength: { value: 0.5 },
uChromaticStrength: { value: 0.0015 },
uGrainStrength: { value: 0.03 },
uWarmth: { value: 0.08 },
},
vertexShader: `
varying vec2 vUv;
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
`,
fragmentShader: `
uniform sampler2D tDiffuse;
uniform float uTime;
uniform float uVignetteStrength;
uniform float uChromaticStrength;
uniform float uGrainStrength;
uniform float uWarmth;
varying vec2 vUv;
float rand(vec2 co) {
return fract(sin(dot(co, vec2(12.9898, 78.233))) * 43758.5453);
}
void main() {
vec2 uv = vUv;
vec2 center = uv - 0.5;
float dist = length(center);
// Subtle chromatic aberration at edges only
vec2 offset = center * dist * uChromaticStrength;
float r = texture2D(tDiffuse, uv + offset).r;
float g = texture2D(tDiffuse, uv).g;
float b = texture2D(tDiffuse, uv - offset * 0.5).b;
vec3 color = vec3(r, g, b);
// Warm vignette
float vignette = 1.0 - dist * dist * uVignetteStrength * 1.8;
color *= vignette;
// Very subtle warm shift in shadows
float luma = dot(color, vec3(0.299, 0.587, 0.114));
color.r += (1.0 - luma) * uWarmth * 0.5;
color.g += (1.0 - luma) * uWarmth * 0.2;
// Minimal grain
float grain = (rand(uv * uTime * 0.01) - 0.5) * uGrainStrength;
color += grain;
gl_FragColor = vec4(color, 1.0);
}
`,
};
export class PostProcessing {
constructor(renderer, scene, camera) {
const size = renderer.getSize(new THREE.Vector2());
this.composer = new EffectComposer(renderer);
this.composer.addPass(new RenderPass(scene, camera));
// Bloom — tuned for green wireframe glow
this._bloomPass = new UnrealBloomPass(
new THREE.Vector2(size.x, size.y),
1.0, // strength (less aggressive than before)
0.5, // radius
0.25 // threshold
);
this.composer.addPass(this._bloomPass);
// Vignette + warmth
this._vignettePass = new ShaderPass(VignetteShader);
this.composer.addPass(this._vignettePass);
this._bloomEnabled = true;
}
update(elapsed) {
this._vignettePass.uniforms.uTime.value = elapsed;
}
render() {
this.composer.render();
}
resize(width, height) {
this.composer.setSize(width, height);
this._bloomPass.resolution.set(width, height);
}
setQuality(level) {
if (level === 0) {
this._bloomPass.strength = 0;
this._vignettePass.uniforms.uChromaticStrength.value = 0;
this._vignettePass.uniforms.uGrainStrength.value = 0;
} else if (level === 1) {
this._bloomPass.strength = 0.6;
this._vignettePass.uniforms.uChromaticStrength.value = 0.001;
this._vignettePass.uniforms.uGrainStrength.value = 0.02;
} else {
this._bloomPass.strength = 1.0;
this._vignettePass.uniforms.uChromaticStrength.value = 0.0015;
this._vignettePass.uniforms.uGrainStrength.value = 0.03;
}
}
dispose() {
this.composer.dispose();
}
}

178
ui/observatory/js/presence-cartography.js Normal file
View File

@ -0,0 +1,178 @@
/**
* Module C — "Presence Cartography"
* InstancedMesh 20x4x20 voxel heatmap with person lights
*/
import * as THREE from 'three';
const GRID_X = 20;
const GRID_Y = 4;
const GRID_Z = 20;
const TOTAL_VOXELS = GRID_X * GRID_Y * GRID_Z;
const VOXEL_SIZE = 0.22;
export class PresenceCartography {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
// Instanced cubes
const cubeGeo = new THREE.BoxGeometry(VOXEL_SIZE, VOXEL_SIZE, VOXEL_SIZE);
const cubeMat = new THREE.MeshBasicMaterial({
color: 0xffffff,
transparent: true,
opacity: 1,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._mesh = new THREE.InstancedMesh(cubeGeo, cubeMat, TOTAL_VOXELS);
this._mesh.instanceMatrix.setUsage(THREE.DynamicDrawUsage);
// Color attribute
this._colors = new Float32Array(TOTAL_VOXELS * 3);
this._mesh.instanceColor = new THREE.InstancedBufferAttribute(this._colors, 3);
// Initialize positions
const dummy = new THREE.Object3D();
const halfX = (GRID_X * VOXEL_SIZE * 1.1) / 2;
const halfZ = (GRID_Z * VOXEL_SIZE * 1.1) / 2;
for (let y = 0; y < GRID_Y; y++) {
for (let z = 0; z < GRID_Z; z++) {
for (let x = 0; x < GRID_X; x++) {
const idx = y * GRID_Z * GRID_X + z * GRID_X + x;
dummy.position.set(
x * VOXEL_SIZE * 1.1 - halfX,
y * VOXEL_SIZE * 1.1,
z * VOXEL_SIZE * 1.1 - halfZ
);
dummy.scale.set(0.01, 0.01, 0.01); // start invisible
dummy.updateMatrix();
this._mesh.setMatrixAt(idx, dummy.matrix);
this._colors[idx * 3] = 0;
this._colors[idx * 3 + 1] = 0.2;
this._colors[idx * 3 + 2] = 0.4;
}
}
}
this._mesh.instanceMatrix.needsUpdate = true;
this._mesh.instanceColor.needsUpdate = true;
this.group.add(this._mesh);
// Room wireframe
const roomW = GRID_X * VOXEL_SIZE * 1.1;
const roomH = GRID_Y * VOXEL_SIZE * 1.1;
const roomD = GRID_Z * VOXEL_SIZE * 1.1;
const boxGeo = new THREE.BoxGeometry(roomW, roomH, roomD);
const edges = new THREE.EdgesGeometry(boxGeo);
const lineMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.15,
});
const wireframe = new THREE.LineSegments(edges, lineMat);
wireframe.position.y = roomH / 2;
this.group.add(wireframe);
// Person lights (up to 4)
this._personLights = [];
for (let i = 0; i < 4; i++) {
const light = new THREE.PointLight(0xff8800, 0, 3);
this.group.add(light);
this._personLights.push(light);
}
this._dummy = new THREE.Object3D();
this._halfX = halfX;
this._halfZ = halfZ;
}
update(dt, elapsed, data) {
const field = data?.signal_field?.values;
const persons = data?.persons || [];
const dummy = this._dummy;
if (field && field.length >= GRID_X * GRID_Z) {
for (let y = 0; y < GRID_Y; y++) {
for (let z = 0; z < GRID_Z; z++) {
for (let x = 0; x < GRID_X; x++) {
const idx = y * GRID_Z * GRID_X + z * GRID_X + x;
const fieldIdx = z * GRID_X + x;
const val = field[fieldIdx] || 0;
// Extrude vertically: layer 0 = full val, higher layers diminish
const layerFactor = Math.max(0, 1 - y / GRID_Y);
const v = val * layerFactor;
// Scale voxel by value
const s = v > 0.05 ? 0.3 + v * 0.7 : 0.01;
dummy.position.set(
x * VOXEL_SIZE * 1.1 - this._halfX,
y * VOXEL_SIZE * 1.1,
z * VOXEL_SIZE * 1.1 - this._halfZ
);
dummy.scale.set(s, s, s);
dummy.updateMatrix();
this._mesh.setMatrixAt(idx, dummy.matrix);
// Color: blue(low) -> cyan(mid) -> amber(high)
let r, g, b;
if (v < 0.3) {
const t = v / 0.3;
r = 0.02;
g = 0.06 + t * 0.6;
b = 0.2 + t * 0.6;
} else if (v < 0.6) {
const t = (v - 0.3) / 0.3;
r = t * 0.8;
g = 0.66 + t * 0.2;
b = 0.8 - t * 0.5;
} else {
const t = (v - 0.6) / 0.4;
r = 0.8 + t * 0.2;
g = 0.86 - t * 0.5;
b = 0.3 - t * 0.3;
}
this._colors[idx * 3] = r;
this._colors[idx * 3 + 1] = g;
this._colors[idx * 3 + 2] = b;
}
}
}
this._mesh.instanceMatrix.needsUpdate = true;
this._mesh.instanceColor.needsUpdate = true;
}
// Person lights
for (let i = 0; i < this._personLights.length; i++) {
const light = this._personLights[i];
if (i < persons.length) {
const p = persons[i].position || [0, 0, 0];
light.position.set(p[0] * 2, 1.5, p[2] * 2);
light.intensity = 1.5 + Math.sin(elapsed * 3 + i) * 0.5;
light.color.setHex(0xff8800);
} else {
light.intensity = 0;
}
}
}
/** Reduce voxel count for performance */
setQuality(level) {
// For now just toggle visibility of upper layers
// level 0 = show only ground, 2 = show all
this._mesh.count = level === 0
? GRID_X * GRID_Z
: level === 1
? GRID_X * GRID_Z * 2
: TOTAL_VOXELS;
}
dispose() {
this._mesh.geometry.dispose();
this._mesh.material.dispose();
}
}

739
ui/observatory/js/scenario-props.js Normal file
View File

@ -0,0 +1,739 @@
/**
* ScenarioProps — Scenario-specific room furniture and props
*
* Extracted from main.js. Builds and manages visibility of all physical
* objects that appear/disappear based on the active scenario: bed, chair,
* exercise mat, door, rubble wall, screen/TV, desks, security cameras,
* and the alert light system.
*/
import * as THREE from 'three';
// Scenario-to-prop-name mapping
const SCENARIO_PROPS = {
empty_room: [],
single_breathing: [],
two_walking: [],
fall_event: [],
sleep_monitoring: ['bed'],
intrusion_detect: ['door'],
gesture_control: ['screen'],
crowd_occupancy: ['desk', 'desk2'],
search_rescue: ['rubbleWall'],
elderly_care: ['chair'],
fitness_tracking: ['exerciseMat'],
security_patrol: ['camera1', 'camera2'],
};
export class ScenarioProps {
constructor(scene) {
this._scene = scene;
this._props = {};
this._currentScenario = null;
this._alertLight = null;
this._alertIntensity = 0;
// Animatable references
this._screenGlow = null;
this._camera1Group = null;
this._camera2Group = null;
this._cam1Cone = null;
this._cam2Cone = null;
this._cam1Led = null;
this._cam2Led = null;
this._dustParticles = null;
this._doorSpotlight = null;
this._alarmHousing = null;
this._powerLed = null;
this._build();
}
// ---- helper: positioned box with shadow ----
_box(x, y, z, w, h, d, mat) {
const m = new THREE.Mesh(new THREE.BoxGeometry(w, h, d), mat);
m.position.set(x, y, z);
m.castShadow = true;
m.receiveShadow = true;
return m;
}
// ---- helper: positioned cylinder with shadow ----
_cyl(x, y, z, rTop, rBot, h, segs, mat) {
const m = new THREE.Mesh(new THREE.CylinderGeometry(rTop, rBot, h, segs), mat);
m.position.set(x, y, z);
m.castShadow = true;
m.receiveShadow = true;
return m;
}
// ========================================
// BUILD ALL PROPS
// ========================================
_build() {
const darkMat = new THREE.MeshStandardMaterial({ color: 0x6b5840, roughness: 0.6, emissive: 0x1a1408, emissiveIntensity: 0.25 });
const metalMat = new THREE.MeshStandardMaterial({ color: 0x808088, roughness: 0.3, metalness: 0.7, emissive: 0x1a1a20, emissiveIntensity: 0.2 });
const accentMat = new THREE.MeshStandardMaterial({ color: 0x606070, roughness: 0.4, metalness: 0.4, emissive: 0x101018, emissiveIntensity: 0.15 });
this._buildBed(darkMat);
this._buildChair(darkMat, accentMat);
this._buildExerciseMat();
this._buildDoor();
this._buildRubbleWall();
this._buildScreen(metalMat);
this._buildDesks(darkMat, metalMat, accentMat);
this._buildCameras(metalMat);
this._buildAlertSystem();
}
// ---- BED (sleep monitoring) ----
_buildBed(darkMat) {
const bedGroup = new THREE.Group();
// Bed frame with legs
const frameMat = new THREE.MeshStandardMaterial({ color: 0x7a6448, roughness: 0.55, metalness: 0.25, emissive: 0x181008, emissiveIntensity: 0.25 });
const bedFrame = new THREE.Mesh(new THREE.BoxGeometry(2.2, 0.12, 1.2), frameMat);
bedFrame.position.set(3.5, 0.32, -3.5);
bedFrame.castShadow = true;
bedGroup.add(bedFrame);
// Frame legs (4 short posts)
for (const [lx, lz] of [[2.5, -4.0], [4.5, -4.0], [2.5, -3.0], [4.5, -3.0]]) {
bedGroup.add(this._cyl(lx, 0.13, lz, 0.04, 0.04, 0.26, 6, frameMat));
}
// Headboard — tall panel at head of bed
const headboardMat = new THREE.MeshStandardMaterial({ color: 0x6a5440, roughness: 0.65, emissive: 0x140e08, emissiveIntensity: 0.2 });
const headboard = new THREE.Mesh(new THREE.BoxGeometry(0.08, 0.7, 1.2), headboardMat);
headboard.position.set(2.38, 0.65, -3.5);
headboard.castShadow = true;
bedGroup.add(headboard);
// Mattress
const mattressMat = new THREE.MeshStandardMaterial({ color: 0x484860, roughness: 0.75, emissive: 0x0c0c1a, emissiveIntensity: 0.2 });
const mattress = new THREE.Mesh(new THREE.BoxGeometry(2.0, 0.15, 1.1), mattressMat);
mattress.position.set(3.5, 0.455, -3.5);
mattress.castShadow = true;
bedGroup.add(mattress);
// Wrinkled sheet — wave-displaced plane
const sheetGeo = new THREE.PlaneGeometry(1.4, 1.0, 20, 20);
const posAttr = sheetGeo.getAttribute('position');
for (let i = 0; i < posAttr.count; i++) {
const px = posAttr.getX(i);
const py = posAttr.getY(i);
posAttr.setZ(i, Math.sin(px * 4) * 0.015 + Math.cos(py * 5) * 0.01 + Math.sin(px * py * 3) * 0.008);
}
posAttr.needsUpdate = true;
sheetGeo.computeVertexNormals();
const sheetMat = new THREE.MeshStandardMaterial({
color: 0x506880, roughness: 0.75, side: THREE.DoubleSide, emissive: 0x0c1018, emissiveIntensity: 0.2,
});
const sheet = new THREE.Mesh(sheetGeo, sheetMat);
sheet.rotation.x = -Math.PI / 2;
sheet.position.set(3.7, 0.54, -3.5);
sheet.castShadow = true;
bedGroup.add(sheet);
// Pillow — soft shape using scaled sphere
const pillowGeo = new THREE.SphereGeometry(0.18, 12, 8);
pillowGeo.scale(1, 0.35, 1.4);
const pillowMat = new THREE.MeshStandardMaterial({ color: 0x706868, roughness: 0.7, emissive: 0x141010, emissiveIntensity: 0.2 });
const pillow = new THREE.Mesh(pillowGeo, pillowMat);
pillow.position.set(2.65, 0.52, -3.5);
pillow.castShadow = true;
bedGroup.add(pillow);
// Bedside lamp — small cylinder + sphere shade on a tiny table
const lampBaseMat = new THREE.MeshStandardMaterial({ color: 0x686870, roughness: 0.3, metalness: 0.7, emissive: 0x101018, emissiveIntensity: 0.15 });
// Nightstand
bedGroup.add(this._box(2.15, 0.25, -3.5, 0.35, 0.5, 0.35, darkMat));
// Lamp base
bedGroup.add(this._cyl(2.15, 0.55, -3.5, 0.04, 0.05, 0.1, 8, lampBaseMat));
// Lamp stem
bedGroup.add(this._cyl(2.15, 0.68, -3.5, 0.015, 0.015, 0.2, 6, lampBaseMat));
// Lamp shade (emissive warm glow)
const shadeMat = new THREE.MeshStandardMaterial({
color: 0x705830, emissive: 0x604018, emissiveIntensity: 1.0, roughness: 0.6,
side: THREE.DoubleSide, transparent: true, opacity: 0.9,
});
const shade = new THREE.Mesh(new THREE.ConeGeometry(0.08, 0.1, 8, 1, true), shadeMat);
shade.position.set(2.15, 0.78, -3.5);
shade.rotation.x = Math.PI;
bedGroup.add(shade);
// Warm lamp light
const lampLight = new THREE.PointLight(0xffcc88, 2.0, 6, 1.2);
lampLight.position.set(2.15, 0.78, -3.5);
bedGroup.add(lampLight);
this._props.bed = bedGroup;
bedGroup.visible = false;
this._scene.add(bedGroup);
}
// ---- CHAIR (elderly care) ----
_buildChair(darkMat, accentMat) {
const chairGroup = new THREE.Group();
chairGroup.position.set(1, 0, -1.5);
const cushionMat = new THREE.MeshStandardMaterial({ color: 0x5a5078, roughness: 0.7, emissive: 0x10101a, emissiveIntensity: 0.2 });
// Seat
chairGroup.add(this._box(0, 0.45, 0, 0.5, 0.04, 0.45, darkMat));
// Seat cushion — slightly puffy
const cushionGeo = new THREE.BoxGeometry(0.46, 0.06, 0.42);
// Gentle puff on top vertices
const cPos = cushionGeo.getAttribute('position');
for (let i = 0; i < cPos.count; i++) {
if (cPos.getY(i) > 0) {
const dx = cPos.getX(i) / 0.23;
const dz = cPos.getZ(i) / 0.21;
cPos.setY(i, cPos.getY(i) + 0.015 * (1 - dx * dx) * (1 - dz * dz));
}
}
cPos.needsUpdate = true;
cushionGeo.computeVertexNormals();
const cushion = new THREE.Mesh(cushionGeo, cushionMat);
cushion.position.set(0, 0.50, 0);
cushion.castShadow = true;
chairGroup.add(cushion);
// Back
chairGroup.add(this._box(0, 0.72, -0.22, 0.5, 0.5, 0.04, darkMat));
// Legs
for (const [lx, lz] of [[-0.22, -0.2], [0.22, -0.2], [-0.22, 0.2], [0.22, 0.2]]) {
chairGroup.add(this._box(lx, 0.22, lz, 0.04, 0.44, 0.04, darkMat));
}
// Armrests
chairGroup.add(this._box(-0.28, 0.6, 0, 0.04, 0.04, 0.4, accentMat));
chairGroup.add(this._box(0.28, 0.6, 0, 0.04, 0.04, 0.4, accentMat));
// Armrest supports
chairGroup.add(this._box(-0.28, 0.52, -0.18, 0.04, 0.12, 0.04, accentMat));
chairGroup.add(this._box(0.28, 0.52, -0.18, 0.04, 0.12, 0.04, accentMat));
// Small side table
const tableMat = new THREE.MeshStandardMaterial({ color: 0x685840, roughness: 0.55, emissive: 0x14100a, emissiveIntensity: 0.2 });
chairGroup.add(this._box(0.65, 0.3, 0, 0.35, 0.03, 0.35, tableMat));
// Table legs
for (const [tx, tz] of [[0.5, -0.14], [0.8, -0.14], [0.5, 0.14], [0.8, 0.14]]) {
chairGroup.add(this._cyl(tx, 0.15, tz, 0.015, 0.015, 0.28, 6, tableMat));
}
this._props.chair = chairGroup;
chairGroup.visible = false;
this._scene.add(chairGroup);
}
// ---- EXERCISE MAT (fitness tracking) ----
_buildExerciseMat() {
const matGroup = new THREE.Group();
const matMat = new THREE.MeshStandardMaterial({ color: 0x408858, roughness: 0.75, emissive: 0x0c2010, emissiveIntensity: 0.25 });
// Mat body
const exerciseMat = new THREE.Mesh(new THREE.BoxGeometry(1.8, 0.015, 0.8), matMat);
exerciseMat.position.set(0, 0.008, 0);
exerciseMat.receiveShadow = true;
matGroup.add(exerciseMat);
// Boundary lines on the mat (thin strips)
const lineMat = new THREE.MeshStandardMaterial({ color: 0x50a068, roughness: 0.7, emissive: 0x102818, emissiveIntensity: 0.3 });
// Longitudinal borders
matGroup.add(this._box(0, 0.017, -0.37, 1.7, 0.003, 0.02, lineMat));
matGroup.add(this._box(0, 0.017, 0.37, 1.7, 0.003, 0.02, lineMat));
// Cross lines (exercise area markers)
for (const xOff of [-0.6, 0, 0.6]) {
matGroup.add(this._box(xOff, 0.017, 0, 0.02, 0.003, 0.74, lineMat));
}
// Water bottle (cylinder body + hemisphere cap)
const bottleMat = new THREE.MeshStandardMaterial({ color: 0x4878a8, roughness: 0.2, metalness: 0.7, emissive: 0x0c1828, emissiveIntensity: 0.25 });
const bottleBody = new THREE.Mesh(new THREE.CylinderGeometry(0.035, 0.035, 0.18, 10), bottleMat);
bottleBody.position.set(1.1, 0.09, 0.25);
bottleBody.castShadow = true;
matGroup.add(bottleBody);
const bottleCap = new THREE.Mesh(new THREE.SphereGeometry(0.035, 8, 6, 0, Math.PI * 2, 0, Math.PI / 2), bottleMat);
bottleCap.position.set(1.1, 0.18, 0.25);
matGroup.add(bottleCap);
// Bottle neck
const neckMat = new THREE.MeshStandardMaterial({ color: 0x587088, roughness: 0.3, metalness: 0.6, emissive: 0x0c1420, emissiveIntensity: 0.2 });
matGroup.add(this._cyl(1.1, 0.21, 0.25, 0.018, 0.025, 0.04, 8, neckMat));
// Small towel (flat draped box)
const towelMat = new THREE.MeshStandardMaterial({ color: 0x686890, roughness: 0.75, emissive: 0x101020, emissiveIntensity: 0.2 });
const towel = this._box(1.1, 0.01, -0.25, 0.3, 0.008, 0.15, towelMat);
towel.rotation.y = 0.15;
matGroup.add(towel);
this._props.exerciseMat = matGroup;
matGroup.visible = false;
this._scene.add(matGroup);
}
// ---- DOOR (intrusion detection) ----
_buildDoor() {
const doorGroup = new THREE.Group();
doorGroup.position.set(-5.5, 0, -1);
const doorMat = new THREE.MeshStandardMaterial({ color: 0x7a6040, roughness: 0.5, emissive: 0x18140a, emissiveIntensity: 0.25 });
const hingeMat = new THREE.MeshStandardMaterial({ color: 0x909098, roughness: 0.2, metalness: 0.85, emissive: 0x181820, emissiveIntensity: 0.15 });
// Left jamb
doorGroup.add(this._box(-0.45, 1.1, 0, 0.08, 2.2, 0.15, doorMat));
// Right jamb
doorGroup.add(this._box(0.45, 1.1, 0, 0.08, 2.2, 0.15, doorMat));
// Top
doorGroup.add(this._box(0, 2.2, 0, 0.98, 0.08, 0.15, doorMat));
// Door panel (partially open)
const doorPanel = new THREE.Mesh(new THREE.BoxGeometry(0.85, 2.1, 0.04), doorMat);
doorPanel.position.set(0.2, 1.05, -0.2);
doorPanel.rotation.y = -0.7;
doorPanel.castShadow = true;
doorGroup.add(doorPanel);
// Door handle (torus)
const handleMat = new THREE.MeshStandardMaterial({ color: 0xaaaaB0, roughness: 0.1, metalness: 0.9, emissive: 0x1a1a20, emissiveIntensity: 0.2 });
const handle = new THREE.Mesh(new THREE.TorusGeometry(0.035, 0.008, 6, 12), handleMat);
// Position on the door panel (relative to panel pivot)
handle.position.set(0.48, 1.05, -0.22);
handle.rotation.y = -0.7;
handle.rotation.x = Math.PI / 2;
doorGroup.add(handle);
// Hinge details — small cylinders at jamb
for (const hy of [0.4, 1.1, 1.8]) {
const hinge = new THREE.Mesh(new THREE.CylinderGeometry(0.015, 0.015, 0.06, 6), hingeMat);
hinge.position.set(-0.42, hy, 0.06);
doorGroup.add(hinge);
}
// Light spill through the gap — spotlight from outside
const doorSpot = new THREE.SpotLight(0x88aacc, 3.0, 10, Math.PI / 4, 0.3, 0.6);
doorSpot.position.set(-0.8, 1.2, -0.5);
doorSpot.target.position.set(0.5, 0, 0.5);
doorGroup.add(doorSpot);
doorGroup.add(doorSpot.target);
this._doorSpotlight = doorSpot;
// Window next to door — simple frame with translucent pane
const windowFrame = new THREE.MeshStandardMaterial({ color: 0x686878, roughness: 0.35, metalness: 0.6, emissive: 0x101018, emissiveIntensity: 0.15 });
// Frame
doorGroup.add(this._box(1.2, 1.5, 0, 0.04, 0.8, 0.06, windowFrame));
doorGroup.add(this._box(1.2, 1.5, 0, 0.6, 0.04, 0.06, windowFrame));
doorGroup.add(this._box(0.92, 1.5, 0, 0.04, 0.8, 0.06, windowFrame));
doorGroup.add(this._box(1.48, 1.5, 0, 0.04, 0.8, 0.06, windowFrame));
doorGroup.add(this._box(1.2, 1.1, 0, 0.6, 0.04, 0.06, windowFrame));
doorGroup.add(this._box(1.2, 1.9, 0, 0.6, 0.04, 0.06, windowFrame));
// Glass pane
const glassMat = new THREE.MeshStandardMaterial({
color: 0x305880, transparent: true, opacity: 0.4, roughness: 0.05, metalness: 0.3, emissive: 0x0c1830, emissiveIntensity: 0.35,
});
const glass = new THREE.Mesh(new THREE.BoxGeometry(0.52, 0.72, 0.01), glassMat);
glass.position.set(1.2, 1.5, 0);
doorGroup.add(glass);
this._props.door = doorGroup;
doorGroup.visible = false;
this._scene.add(doorGroup);
}
// ---- RUBBLE WALL (search & rescue) ----
_buildRubbleWall() {
const rubbleGroup = new THREE.Group();
const rubbleMat = new THREE.MeshStandardMaterial({ color: 0x807868, roughness: 0.75, emissive: 0x181610, emissiveIntensity: 0.25 });
const rebarMat = new THREE.MeshStandardMaterial({ color: 0x8a7858, roughness: 0.4, metalness: 0.7, emissive: 0x1a1408, emissiveIntensity: 0.2 });
// Broken wall — main slab
rubbleGroup.add(this._box(2, 1, 0, 0.4, 2, 3, rubbleMat));
// Wall crack lines (thin dark boxes embedded in wall surface)
const crackMat = new THREE.MeshStandardMaterial({ color: 0x403828, roughness: 0.9 });
const cracks = [
[1.82, 1.4, -0.3, 0.01, 0.6, 0.02, 0.3],
[1.82, 0.8, 0.5, 0.01, 0.5, 0.02, -0.2],
[1.82, 1.6, 0.8, 0.01, 0.4, 0.02, 0.15],
[1.82, 0.5, -0.7, 0.01, 0.35, 0.02, -0.25],
];
for (const [cx, cy, cz, cw, ch, cd, rot] of cracks) {
const crack = this._box(cx, cy, cz, cw, ch, cd, crackMat);
crack.rotation.z = rot;
rubbleGroup.add(crack);
}
// Rebar — thin metal cylinders protruding from the wall
for (const [rx, ry, rz, rLen, rRot] of [
[1.6, 1.7, -0.4, 0.8, 0.3],
[1.5, 1.2, 0.6, 0.6, -0.2],
[1.7, 0.9, -0.8, 0.5, 0.5],
[1.55, 1.5, 1.0, 0.7, -0.4],
]) {
const rebar = new THREE.Mesh(new THREE.CylinderGeometry(0.012, 0.012, rLen, 6), rebarMat);
rebar.position.set(rx, ry, rz);
rebar.rotation.z = Math.PI / 2 + rRot;
rebar.rotation.y = rRot * 0.5;
rebar.castShadow = true;
rubbleGroup.add(rebar);
}
// Rubble pieces — more varied with random rotations
const rubbleColors = [0x807868, 0x706860, 0x908878, 0x686058];
for (let i = 0; i < 10; i++) {
const s = 0.12 + Math.random() * 0.3;
const rMat = new THREE.MeshStandardMaterial({
color: rubbleColors[i % rubbleColors.length], roughness: 0.7 + Math.random() * 0.15,
emissive: 0x141210, emissiveIntensity: 0.2,
});
const piece = this._box(
1.3 + Math.random() * 1.4, s / 2, -1.5 + Math.random() * 3,
s, s * (0.4 + Math.random() * 0.5), s * (0.6 + Math.random() * 0.4), rMat
);
piece.rotation.x = (Math.random() - 0.5) * 0.6;
piece.rotation.y = (Math.random() - 0.5) * 1.2;
piece.rotation.z = (Math.random() - 0.5) * 0.4;
rubbleGroup.add(piece);
}
// Dust particles near rubble
const dustCount = 60;
const dustGeo = new THREE.BufferGeometry();
const dustPositions = new Float32Array(dustCount * 3);
for (let i = 0; i < dustCount; i++) {
dustPositions[i * 3] = 1.0 + Math.random() * 2.0;
dustPositions[i * 3 + 1] = Math.random() * 2.5;
dustPositions[i * 3 + 2] = -1.5 + Math.random() * 3.0;
}
dustGeo.setAttribute('position', new THREE.BufferAttribute(dustPositions, 3));
const dustMaterial = new THREE.PointsMaterial({
color: 0xaa9988, size: 0.03, transparent: true, opacity: 0.5,
blending: THREE.AdditiveBlending, depthWrite: false,
});
this._dustParticles = new THREE.Points(dustGeo, dustMaterial);
rubbleGroup.add(this._dustParticles);
this._props.rubbleWall = rubbleGroup;
rubbleGroup.visible = false;
this._scene.add(rubbleGroup);
}
// ---- SCREEN / TV (gesture control) ----
_buildScreen(metalMat) {
const screenGroup = new THREE.Group();
const screenFrame = new THREE.MeshStandardMaterial({ color: 0x484850, roughness: 0.2, metalness: 0.7, emissive: 0x0c0c14, emissiveIntensity: 0.15 });
// Frame
screenGroup.add(this._box(0, 1.5, -4.7, 1.8, 1.1, 0.06, screenFrame));
// Screen surface (emissive, color shifts in update())
const screenSurfMat = new THREE.MeshStandardMaterial({
color: 0x1a3868, emissive: 0x1a3868, emissiveIntensity: 1.2, roughness: 0.1,
});
const screenSurf = new THREE.Mesh(new THREE.BoxGeometry(1.6, 0.9, 0.02), screenSurfMat);
screenSurf.position.set(0, 1.5, -4.66);
screenGroup.add(screenSurf);
this._screenGlow = screenSurfMat;
// Stand / mount — neck + base
screenGroup.add(this._box(0, 0.88, -4.7, 0.08, 0.16, 0.08, screenFrame));
screenGroup.add(this._box(0, 0.78, -4.7, 0.4, 0.03, 0.2, metalMat));
// Power LED indicator
const ledMat = new THREE.MeshStandardMaterial({
color: 0x00ff40, emissive: 0x00ff40, emissiveIntensity: 1.0,
});
const powerLed = new THREE.Mesh(new THREE.SphereGeometry(0.012, 6, 4), ledMat);
powerLed.position.set(0.82, 0.96, -4.66);
screenGroup.add(powerLed);
this._powerLed = ledMat;
// Subtle screen glow (point light)
const screenLight = new THREE.PointLight(0x4080e0, 1.5, 6);
screenLight.position.set(0, 1.5, -4.5);
screenGroup.add(screenLight);
// Media console below the screen
const consoleMat = new THREE.MeshStandardMaterial({ color: 0x484858, roughness: 0.45, metalness: 0.5, emissive: 0x0c0c14, emissiveIntensity: 0.15 });
screenGroup.add(this._box(0, 0.55, -4.7, 1.2, 0.35, 0.35, consoleMat));
// Console shelf divider
screenGroup.add(this._box(0, 0.55, -4.54, 1.1, 0.02, 0.01, metalMat));
this._props.screen = screenGroup;
screenGroup.visible = false;
this._scene.add(screenGroup);
}
// ---- DESKS (crowd / office) ----
_buildDesks(darkMat, metalMat, accentMat) {
// Desk 1 (left)
const deskGroup = new THREE.Group();
deskGroup.add(this._box(-2, 0.38, -1, 1.2, 0.04, 0.6, darkMat));
for (const [lx, lz] of [[-2.55, -1.25], [-1.45, -1.25], [-2.55, -0.75], [-1.45, -0.75]]) {
deskGroup.add(this._box(lx, 0.19, lz, 0.04, 0.38, 0.04, darkMat));
}
// Monitor on desk 1
const monitorMat = new THREE.MeshStandardMaterial({ color: 0x484850, roughness: 0.2, metalness: 0.7, emissive: 0x0c0c14, emissiveIntensity: 0.15 });
const monScreenMat = new THREE.MeshStandardMaterial({
color: 0x183858, emissive: 0x183858, emissiveIntensity: 1.0, roughness: 0.1,
});
deskGroup.add(this._box(-2, 0.62, -1.15, 0.5, 0.35, 0.03, monitorMat));
deskGroup.add(this._box(-2, 0.62, -1.13, 0.44, 0.29, 0.01, monScreenMat));
deskGroup.add(this._box(-2, 0.42, -1.1, 0.06, 0.04, 0.06, metalMat)); // stand neck
deskGroup.add(this._box(-2, 0.40, -1.05, 0.18, 0.01, 0.12, metalMat)); // stand base
// Keyboard outline
deskGroup.add(this._box(-2, 0.405, -0.85, 0.35, 0.008, 0.12, accentMat));
// Office chair at desk 1
this._buildOfficeChair(deskGroup, -2, -0.55, darkMat, metalMat);
// Monitor glow light
const monLight = new THREE.PointLight(0x4080e0, 1.2, 4);
monLight.position.set(-2, 0.7, -1.0);
deskGroup.add(monLight);
this._props.desk = deskGroup;
deskGroup.visible = false;
this._scene.add(deskGroup);
// Desk 2 (right)
const desk2Group = new THREE.Group();
desk2Group.add(this._box(2, 0.38, 1, 1.0, 0.04, 0.6, darkMat));
for (const [lx, lz] of [[1.45, 0.75], [2.55, 0.75], [1.45, 1.25], [2.55, 1.25]]) {
desk2Group.add(this._box(lx, 0.19, lz, 0.04, 0.38, 0.04, darkMat));
}
// Monitor on desk 2
desk2Group.add(this._box(2, 0.62, 1.15, 0.5, 0.35, 0.03, monitorMat));
desk2Group.add(this._box(2, 0.62, 1.17, 0.44, 0.29, 0.01, monScreenMat));
desk2Group.add(this._box(2, 0.42, 1.1, 0.06, 0.04, 0.06, metalMat));
desk2Group.add(this._box(2, 0.40, 1.05, 0.18, 0.01, 0.12, metalMat));
// Keyboard
desk2Group.add(this._box(2, 0.405, 0.85, 0.35, 0.008, 0.12, accentMat));
// Office chair at desk 2
this._buildOfficeChair(desk2Group, 2, 0.55, darkMat, metalMat);
// Water cooler / plant between desks area
const plantMat = new THREE.MeshStandardMaterial({ color: 0x2a7838, roughness: 0.7, emissive: 0x0c2810, emissiveIntensity: 0.3 });
const potMat = new THREE.MeshStandardMaterial({ color: 0x706858, roughness: 0.6, emissive: 0x14120c, emissiveIntensity: 0.15 });
desk2Group.add(this._cyl(3.2, 0.15, 0, 0.12, 0.1, 0.3, 8, potMat));
// Foliage — cluster of small spheres
for (const [fx, fy, fz] of [[3.2, 0.45, 0], [3.15, 0.4, 0.06], [3.25, 0.42, -0.05]]) {
const leaf = new THREE.Mesh(new THREE.SphereGeometry(0.08, 6, 5), plantMat);
leaf.position.set(fx, fy, fz);
desk2Group.add(leaf);
}
// Monitor glow light
const monLight2 = new THREE.PointLight(0x4080e0, 1.2, 4);
monLight2.position.set(2, 0.7, 1.0);
desk2Group.add(monLight2);
this._props.desk2 = desk2Group;
desk2Group.visible = false;
this._scene.add(desk2Group);
}
// Helper: small office chair
_buildOfficeChair(parent, x, z, darkMat, metalMat) {
// Seat
parent.add(this._box(x, 0.38, z, 0.35, 0.03, 0.35, darkMat));
// Backrest
parent.add(this._box(x, 0.55, z - 0.16, 0.32, 0.3, 0.03, darkMat));
// Central post
parent.add(this._cyl(x, 0.22, z, 0.025, 0.025, 0.28, 6, metalMat));
// Base star (5 legs)
for (let i = 0; i < 5; i++) {
const angle = (i / 5) * Math.PI * 2;
const legLen = 0.16;
const leg = this._box(
x + Math.cos(angle) * legLen * 0.5, 0.04, z + Math.sin(angle) * legLen * 0.5,
legLen, 0.015, 0.025, metalMat
);
leg.rotation.y = -angle;
parent.add(leg);
}
}
// ---- SECURITY CAMERAS (patrol) ----
_buildCameras(metalMat) {
const camData = [
['camera1', [5, 3.5, -4.5]],
['camera2', [-5, 3.5, 4.5]],
];
for (const [name, pos] of camData) {
const camGroup = new THREE.Group();
camGroup.position.set(...pos);
// Camera body
camGroup.add(this._box(0, 0, 0, 0.15, 0.1, 0.2, metalMat));
// Lens
const lens = new THREE.Mesh(new THREE.CylinderGeometry(0.04, 0.04, 0.08, 8), metalMat);
lens.rotation.x = Math.PI / 2;
lens.position.z = 0.14;
camGroup.add(lens);
// Bracket / mount arm
camGroup.add(this._box(0, 0.1, -0.08, 0.04, 0.2, 0.04, metalMat));
// Rotating motor housing (visible joint)
const motorMat = new THREE.MeshStandardMaterial({ color: 0x686870, roughness: 0.35, metalness: 0.8, emissive: 0x141418, emissiveIntensity: 0.15 });
const motor = new THREE.Mesh(new THREE.CylinderGeometry(0.03, 0.03, 0.04, 8), motorMat);
motor.position.set(0, 0.05, -0.08);
camGroup.add(motor);
// FOV cone (semi-transparent)
const coneMat = new THREE.MeshStandardMaterial({
color: 0xff3040, transparent: true, opacity: 0.15,
side: THREE.DoubleSide, depthWrite: false,
emissive: 0xff2020, emissiveIntensity: 0.3,
});
const cone = new THREE.Mesh(new THREE.ConeGeometry(1.5, 3, 16, 1, true), coneMat);
cone.rotation.x = Math.PI / 2;
cone.position.z = 1.7;
camGroup.add(cone);
// Status LED (blinks in update)
const ledMat = new THREE.MeshStandardMaterial({
color: 0xff2020, emissive: 0xff2020, emissiveIntensity: 1.0,
});
const led = new THREE.Mesh(new THREE.SphereGeometry(0.015, 6, 4), ledMat);
led.position.set(0.08, 0.04, 0.08);
camGroup.add(led);
this._props[name] = camGroup;
camGroup.visible = false;
this._scene.add(camGroup);
// Store references for animation
if (name === 'camera1') {
this._camera1Group = camGroup;
this._cam1Cone = cone;
this._cam1Led = ledMat;
} else {
this._camera2Group = camGroup;
this._cam2Cone = cone;
this._cam2Led = ledMat;
}
}
}
// ---- ALERT SYSTEM ----
_buildAlertSystem() {
// Main alert point light
this._alertLight = new THREE.PointLight(0xff3040, 0, 10);
this._alertLight.position.set(0, 3.5, 0);
this._scene.add(this._alertLight);
// Ceiling-mounted alarm housing
const housingMat = new THREE.MeshStandardMaterial({ color: 0x686878, roughness: 0.35, metalness: 0.6, emissive: 0x101018, emissiveIntensity: 0.15 });
const housing = new THREE.Group();
// Base plate
housing.add(this._box(0, 3.95, 0, 0.2, 0.02, 0.2, housingMat));
// Housing body
housing.add(this._cyl(0, 3.85, 0, 0.08, 0.1, 0.16, 8, housingMat));
// Alarm lens (red when active, dark when inactive)
const lensMat = new THREE.MeshStandardMaterial({
color: 0x330808, emissive: 0x000000, emissiveIntensity: 0, roughness: 0.2,
transparent: true, opacity: 0.8,
});
const alarmLens = new THREE.Mesh(new THREE.SphereGeometry(0.06, 10, 8, 0, Math.PI * 2, 0, Math.PI / 2), lensMat);
alarmLens.position.set(0, 3.76, 0);
alarmLens.rotation.x = Math.PI;
housing.add(alarmLens);
this._alarmHousing = housing;
this._alarmLensMat = lensMat;
this._scene.add(housing);
}
// ========================================
// UPDATE (called every frame)
// ========================================
update(data, currentScenario) {
const scenario = data?.scenario || currentScenario;
const elapsed = Date.now() * 0.001;
// Switch visible props when scenario changes
if (scenario !== this._currentScenario) {
this._currentScenario = scenario;
for (const prop of Object.values(this._props)) prop.visible = false;
const propsToShow = SCENARIO_PROPS[scenario] || [];
for (const name of propsToShow) {
if (this._props[name]) this._props[name].visible = true;
}
}
// --- Alert light (fall / intrusion) ---
const cls = data?.classification || {};
if (cls.fall_detected || cls.intrusion) {
this._alertIntensity = Math.min(2, this._alertIntensity + 0.1);
} else {
this._alertIntensity = Math.max(0, this._alertIntensity - 0.05);
}
// Sawtooth pattern for urgency instead of smooth sine
const alertPhase = (elapsed * 3) % 1.0;
const sawtooth = alertPhase < 0.5 ? alertPhase * 2 : 2 - alertPhase * 2;
this._alertLight.intensity = this._alertIntensity * sawtooth;
// Alarm housing lens glow tracks alert
if (this._alarmLensMat) {
const alertFrac = Math.min(this._alertIntensity / 2, 1);
this._alarmLensMat.emissive.setHex(alertFrac > 0.05 ? 0xff2020 : 0x000000);
this._alarmLensMat.emissiveIntensity = alertFrac * sawtooth;
}
// Subtle ambient color shift during alerts
if (this._alertIntensity > 0.1 && this._alertLight) {
const r = 0.08 + 0.04 * sawtooth * this._alertIntensity;
const g = 0.05 - 0.02 * this._alertIntensity;
const b = 0.10 - 0.04 * this._alertIntensity;
// Shift the alert light color slightly over time
this._alertLight.color.setRGB(
Math.max(0, Math.min(1, 1.0)),
Math.max(0, Math.min(1, 0.15 - 0.1 * sawtooth)),
Math.max(0, Math.min(1, 0.2 - 0.15 * sawtooth))
);
} else if (this._alertLight) {
this._alertLight.color.setHex(0xff3040);
}
// --- Camera rotation animation ---
if (this._camera1Group && this._camera1Group.visible) {
this._camera1Group.rotation.y = Math.sin(elapsed * 0.4) * 0.5;
}
if (this._camera2Group && this._camera2Group.visible) {
this._camera2Group.rotation.y = Math.sin(elapsed * 0.4 + Math.PI) * 0.5;
}
// Camera LED blink
if (this._cam1Led && this._camera1Group?.visible) {
this._cam1Led.emissiveIntensity = (Math.sin(elapsed * 4) > 0.3) ? 1.0 : 0.1;
}
if (this._cam2Led && this._camera2Group?.visible) {
this._cam2Led.emissiveIntensity = (Math.sin(elapsed * 4 + 1) > 0.3) ? 1.0 : 0.1;
}
// --- Screen glow color shift ---
if (this._screenGlow && this._props.screen?.visible) {
const hue = (elapsed * 0.03) % 1;
const r = 0.10 + 0.06 * Math.sin(hue * Math.PI * 2);
const g = 0.16 + 0.08 * Math.sin(hue * Math.PI * 2 + 2.1);
const b = 0.28 + 0.12 * Math.sin(hue * Math.PI * 2 + 4.2);
this._screenGlow.emissive.setRGB(r, g, b);
}
// Power LED gentle pulse
if (this._powerLed && this._props.screen?.visible) {
this._powerLed.emissiveIntensity = 0.5 + 0.5 * Math.sin(elapsed * 2);
}
// --- Dust particle drift near rubble ---
if (this._dustParticles && this._props.rubbleWall?.visible) {
const dPos = this._dustParticles.geometry.getAttribute('position');
for (let i = 0; i < dPos.count; i++) {
let y = dPos.getY(i) + 0.002 * Math.sin(elapsed + i);
if (y > 2.5) y = 0;
dPos.setY(i, y);
dPos.setX(i, dPos.getX(i) + Math.sin(elapsed * 0.5 + i * 0.3) * 0.0005);
}
dPos.needsUpdate = true;
}
}
}

163
ui/observatory/js/subcarrier-manifold.js Normal file
View File

@ -0,0 +1,163 @@
/**
* Module A — "The Subcarrier Manifold"
* 3D scrolling surface: 64 subcarriers x 60 time slots
*/
import * as THREE from 'three';
const MANIFOLD_VERTEX = `
attribute float aHeight;
attribute float aAge; // 0 = newest, 1 = oldest
varying float vHeight;
varying float vAge;
void main() {
vec3 pos = position;
pos.y += aHeight * 2.0;
vHeight = aHeight;
vAge = aAge;
gl_Position = projectionMatrix * modelViewMatrix * vec4(pos, 1.0);
}
`;
const MANIFOLD_FRAGMENT = `
uniform float uTime;
varying float vHeight;
varying float vAge;
void main() {
// Color map: low=deep blue, mid=cyan, high=amber
vec3 lo = vec3(0.02, 0.06, 0.2);
vec3 mid = vec3(0.0, 0.83, 1.0);
vec3 hi = vec3(1.0, 0.53, 0.0);
float h = clamp(vHeight, 0.0, 1.0);
vec3 col = h < 0.5
? mix(lo, mid, h * 2.0)
: mix(mid, hi, (h - 0.5) * 2.0);
// Fade older rows
float alpha = 0.3 + 0.7 * (1.0 - vAge);
gl_FragColor = vec4(col, alpha);
}
`;
const SUBS = 64;
const TIME_SLOTS = 60;
export class SubcarrierManifold {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
this._history = []; // ring buffer of Float32Array[64]
for (let i = 0; i < TIME_SLOTS; i++) {
this._history.push(new Float32Array(SUBS));
}
this._head = 0;
// Build surface geometry
const geo = new THREE.PlaneGeometry(8, 5, SUBS - 1, TIME_SLOTS - 1);
const vertCount = SUBS * TIME_SLOTS;
this._heights = new Float32Array(vertCount);
this._ages = new Float32Array(vertCount);
for (let t = 0; t < TIME_SLOTS; t++) {
for (let s = 0; s < SUBS; s++) {
this._ages[t * SUBS + s] = t / TIME_SLOTS;
}
}
geo.setAttribute('aHeight', new THREE.BufferAttribute(this._heights, 1));
geo.setAttribute('aAge', new THREE.BufferAttribute(this._ages, 1));
// Solid surface
const mat = new THREE.ShaderMaterial({
vertexShader: MANIFOLD_VERTEX,
fragmentShader: MANIFOLD_FRAGMENT,
uniforms: { uTime: { value: 0 } },
transparent: true,
side: THREE.DoubleSide,
depthWrite: false,
blending: THREE.AdditiveBlending,
});
this._mesh = new THREE.Mesh(geo, mat);
this._mesh.rotation.x = -Math.PI * 0.35;
this.group.add(this._mesh);
// Wireframe overlay
const wireGeo = geo.clone();
wireGeo.setAttribute('aHeight', new THREE.BufferAttribute(this._heights, 1));
wireGeo.setAttribute('aAge', new THREE.BufferAttribute(this._ages, 1));
const wireMat = new THREE.ShaderMaterial({
vertexShader: MANIFOLD_VERTEX,
fragmentShader: `
varying float vHeight;
varying float vAge;
void main() {
float alpha = 0.15 * (1.0 - vAge);
gl_FragColor = vec4(0.0, 0.83, 1.0, alpha);
}
`,
uniforms: { uTime: { value: 0 } },
transparent: true,
wireframe: true,
depthWrite: false,
blending: THREE.AdditiveBlending,
});
this._wire = new THREE.Mesh(wireGeo, wireMat);
this._wire.rotation.x = -Math.PI * 0.35;
this.group.add(this._wire);
this._frameAccum = 0;
this._pushInterval = 1 / 15; // push ~15 rows/sec
}
update(dt, elapsed, data) {
this._mesh.material.uniforms.uTime.value = elapsed;
// Push new amplitude data at regular intervals
this._frameAccum += dt;
if (this._frameAccum >= this._pushInterval && data) {
this._frameAccum = 0;
const amp = data.nodes?.[0]?.amplitude;
const row = new Float32Array(SUBS);
if (amp && amp.length > 0) {
for (let i = 0; i < SUBS; i++) {
row[i] = amp[i % amp.length] || 0;
}
}
this._history[this._head] = row;
this._head = (this._head + 1) % TIME_SLOTS;
this._rebuildHeights();
}
}
_rebuildHeights() {
for (let t = 0; t < TIME_SLOTS; t++) {
const histIdx = (this._head + t) % TIME_SLOTS;
const row = this._history[histIdx];
for (let s = 0; s < SUBS; s++) {
const idx = t * SUBS + s;
this._heights[idx] = row[s];
this._ages[idx] = t / TIME_SLOTS;
}
}
const geo = this._mesh.geometry;
geo.attributes.aHeight.needsUpdate = true;
geo.attributes.aAge.needsUpdate = true;
const wGeo = this._wire.geometry;
wGeo.attributes.aHeight.needsUpdate = true;
wGeo.attributes.aAge.needsUpdate = true;
}
dispose() {
this._mesh.geometry.dispose();
this._mesh.material.dispose();
this._wire.geometry.dispose();
this._wire.material.dispose();
}
}

187
ui/observatory/js/vitals-oracle.js Normal file
View File

@ -0,0 +1,187 @@
/**
* Module B — "Vital Signs Oracle"
* Breathing/HR as orbital torus rings with beat markers + trail particles
*/
import * as THREE from 'three';
export class VitalsOracle {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
// Outer torus — breathing (violet)
const breathGeo = new THREE.TorusGeometry(1.8, 0.06, 16, 64);
this._breathMat = new THREE.MeshBasicMaterial({
color: 0x8844ff,
transparent: true,
opacity: 0.7,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._breathRing = new THREE.Mesh(breathGeo, this._breathMat);
this._breathRing.rotation.x = Math.PI * 0.4;
this.group.add(this._breathRing);
// Inner torus — heart rate (crimson)
const hrGeo = new THREE.TorusGeometry(1.2, 0.04, 16, 64);
this._hrMat = new THREE.MeshBasicMaterial({
color: 0xff2244,
transparent: true,
opacity: 0.6,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._hrRing = new THREE.Mesh(hrGeo, this._hrMat);
this._hrRing.rotation.x = Math.PI * 0.5;
this._hrRing.rotation.z = Math.PI * 0.15;
this.group.add(this._hrRing);
// Center orb
const orbGeo = new THREE.SphereGeometry(0.35, 24, 24);
this._orbMat = new THREE.MeshBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.5,
blending: THREE.AdditiveBlending,
});
this._orb = new THREE.Mesh(orbGeo, this._orbMat);
this.group.add(this._orb);
// Bloom point light
this._light = new THREE.PointLight(0x00d4ff, 1.5, 8);
this.group.add(this._light);
// Trail particles along breathing ring
const trailCount = 120;
const trailGeo = new THREE.BufferGeometry();
const trailPos = new Float32Array(trailCount * 3);
const trailSizes = new Float32Array(trailCount);
for (let i = 0; i < trailCount; i++) {
const angle = (i / trailCount) * Math.PI * 2;
trailPos[i * 3] = Math.cos(angle) * 1.8;
trailPos[i * 3 + 1] = 0;
trailPos[i * 3 + 2] = Math.sin(angle) * 1.8;
trailSizes[i] = 3;
}
trailGeo.setAttribute('position', new THREE.BufferAttribute(trailPos, 3));
trailGeo.setAttribute('size', new THREE.BufferAttribute(trailSizes, 1));
const trailMat = new THREE.PointsMaterial({
color: 0x8844ff,
size: 0.08,
transparent: true,
opacity: 0.4,
blending: THREE.AdditiveBlending,
depthWrite: false,
sizeAttenuation: true,
});
this._trails = new THREE.Points(trailGeo, trailMat);
this._trails.rotation.x = Math.PI * 0.4;
this.group.add(this._trails);
// Beat flash sprites
this._beatFlash = this._createBeatSprite(0xff2244);
this.group.add(this._beatFlash);
this._beatTimer = 0;
this._lastBeatTime = 0;
// State
this._breathBpm = 0;
this._hrBpm = 0;
this._breathConf = 0;
this._hrConf = 0;
}
_createBeatSprite(color) {
const canvas = document.createElement('canvas');
canvas.width = 64;
canvas.height = 64;
const ctx = canvas.getContext('2d');
const gradient = ctx.createRadialGradient(32, 32, 0, 32, 32, 32);
gradient.addColorStop(0, `rgba(255, 34, 68, 1)`);
gradient.addColorStop(0.3, `rgba(255, 34, 68, 0.5)`);
gradient.addColorStop(1, `rgba(255, 34, 68, 0)`);
ctx.fillStyle = gradient;
ctx.fillRect(0, 0, 64, 64);
const tex = new THREE.CanvasTexture(canvas);
const mat = new THREE.SpriteMaterial({
map: tex,
transparent: true,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
const sprite = new THREE.Sprite(mat);
sprite.scale.set(0, 0, 0);
return sprite;
}
update(dt, elapsed, data) {
const vs = data?.vital_signs || {};
this._breathBpm = vs.breathing_rate_bpm || 0;
this._hrBpm = vs.heart_rate_bpm || 0;
this._breathConf = vs.breathing_confidence || 0;
this._hrConf = vs.heart_rate_confidence || 0;
// Breathing ring pulsation
const breathFreq = this._breathBpm / 60;
const breathPulse = breathFreq > 0 ? Math.sin(elapsed * Math.PI * 2 * breathFreq) : 0;
const breathScale = 1.0 + breathPulse * 0.08 * this._breathConf;
this._breathRing.scale.set(breathScale, breathScale, 1);
this._breathMat.opacity = 0.3 + this._breathConf * 0.5;
// HR ring pulsation (faster)
const hrFreq = this._hrBpm / 60;
const hrPulse = hrFreq > 0 ? Math.sin(elapsed * Math.PI * 2 * hrFreq) : 0;
const hrScale = 1.0 + hrPulse * 0.06 * this._hrConf;
this._hrRing.scale.set(hrScale, hrScale, 1);
this._hrMat.opacity = 0.2 + this._hrConf * 0.5;
// Slow rotation
this._breathRing.rotation.z = elapsed * 0.1;
this._hrRing.rotation.z = -elapsed * 0.15;
this._trails.rotation.z = elapsed * 0.1;
// Center orb pulse
const orbPulse = 1.0 + breathPulse * 0.1;
this._orb.scale.set(orbPulse, orbPulse, orbPulse);
this._light.intensity = 0.8 + Math.abs(breathPulse) * 1.0;
// Beat flash on HR cycle
if (hrFreq > 0) {
this._beatTimer += dt;
const beatInterval = 1 / hrFreq;
if (this._beatTimer >= beatInterval) {
this._beatTimer -= beatInterval;
this._lastBeatTime = elapsed;
}
const beatAge = elapsed - this._lastBeatTime;
const flashSize = Math.max(0, 1.2 - beatAge * 4) * this._hrConf;
this._beatFlash.scale.set(flashSize, flashSize, 1);
} else {
this._beatFlash.scale.set(0, 0, 0);
}
// Update trail particle sizes based on breathing
const sizes = this._trails.geometry.attributes.size;
if (sizes) {
for (let i = 0; i < sizes.count; i++) {
const phase = (i / sizes.count) * Math.PI * 2 + elapsed * breathFreq * Math.PI * 2;
sizes.array[i] = 0.04 + Math.abs(Math.sin(phase)) * 0.06 * this._breathConf;
}
sizes.needsUpdate = true;
}
}
dispose() {
this._breathRing.geometry.dispose();
this._breathMat.dispose();
this._hrRing.geometry.dispose();
this._hrMat.dispose();
this._orb.geometry.dispose();
this._orbMat.dispose();
this._trails.geometry.dispose();
this._trails.material.dispose();
}
}