Compare commits
8 Commits
0c512ed06e
...
d5c457aa30
| Author | SHA1 | Date |
|---|---|---|
|
|
d5c457aa30 | |
|
|
b2e3f27fa1 | |
|
|
e39a35edee | |
|
|
f49ecb163f | |
|
|
c79543283b | |
|
|
4ab69359ef | |
|
|
ae792aad0d | |
|
|
898d90f689 |
36
README.md
36
README.md
|
|
@ -96,6 +96,42 @@ node scripts/mincut-person-counter.js --port 5006 # Correct person counting
|
|||
>
|
||||
---
|
||||
|
||||
### Real-Time Dense Point Cloud (NEW)
|
||||
|
||||
RuView now generates **real-time 3D point clouds** by fusing camera depth + WiFi CSI + mmWave radar. All sensors stream simultaneously into a unified spatial model.
|
||||
|
||||
| Sensor | Data | Integration |
|
||||
|--------|------|-------------|
|
||||
| **Camera** | MiDaS monocular depth (GPU) | 640×480 → 19,200+ depth points per frame |
|
||||
| **ESP32 CSI** | ADR-018 binary frames (UDP) | RF tomography → 8×8×4 occupancy grid |
|
||||
| **WiFlow Pose** | 17 COCO keypoints from CSI | Skeleton overlay on point cloud |
|
||||
| **Vital Signs** | Breathing rate from CSI phase | Stored in ruOS brain every 60s |
|
||||
| **Motion** | CSI amplitude variance | Adaptive capture rate (skip depth when still) |
|
||||
|
||||
**Quick start:**
|
||||
```bash
|
||||
cd rust-port/wifi-densepose-rs
|
||||
cargo build --release -p wifi-densepose-pointcloud
|
||||
./target/release/ruview-pointcloud serve --port 9880
|
||||
# Open http://localhost:9880 for live 3D viewer
|
||||
```
|
||||
|
||||
**CLI commands:**
|
||||
```bash
|
||||
ruview-pointcloud demo # synthetic demo
|
||||
ruview-pointcloud serve --port 9880 # live server + Three.js viewer
|
||||
ruview-pointcloud capture --output room.ply # capture to PLY
|
||||
ruview-pointcloud train # depth calibration + DPO pairs
|
||||
ruview-pointcloud cameras # list available cameras
|
||||
ruview-pointcloud csi-test --count 100 # send test CSI frames
|
||||
```
|
||||
|
||||
**Performance:** 22ms pipeline, 905 req/s API, 40K voxel room model from 20 frames.
|
||||
|
||||
**Brain integration:** Spatial observations (motion, vitals, skeleton, occupancy) sync to the ruOS brain every 60 seconds for agent reasoning.
|
||||
|
||||
See [PR #405](https://github.com/ruvnet/RuView/pull/405) for full details.
|
||||
|
||||
### What's New in v0.7.0
|
||||
|
||||
<details>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,65 @@
|
|||
# ADR-044: Geospatial Satellite Integration
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
RuView generates real-time 3D point clouds from camera + WiFi CSI, but these exist in a local coordinate frame with no geographic reference. Integrating free satellite imagery, terrain elevation, and map data provides environmental context that enables the ruOS brain to reason about the physical world beyond the room.
|
||||
|
||||
## Decision
|
||||
|
||||
### Data Sources (all free, no API keys)
|
||||
| Source | Data | Resolution | Update | Format |
|
||||
|--------|------|-----------|--------|--------|
|
||||
| EOX Sentinel-2 Cloudless | Satellite tiles | 10m | Static mosaic | XYZ/JPEG |
|
||||
| SRTM GL1 (NASA) | Elevation/DEM | 30m (1-arcsec) | Static | Binary HGT |
|
||||
| Overpass API (OSM) | Buildings, roads | Vector | Real-time | JSON |
|
||||
| ip-api.com | IP geolocation | ~1km | Per-request | JSON |
|
||||
| Sentinel-2 STAC | Temporal satellite | 10m | Every 5 days | COG/STAC |
|
||||
| Open Meteo | Weather | Point | Hourly | JSON |
|
||||
|
||||
### Architecture
|
||||
Pure Rust implementation in `wifi-densepose-geo` crate. No GDAL/PROJ/GEOS — coordinate transforms implemented directly (~250 LOC). Tile caching on disk at `~/.local/share/ruview/geo-cache/`.
|
||||
|
||||
### Coordinate System
|
||||
- WGS84 for geographic coordinates
|
||||
- ENU (East-North-Up) as the bridge between local sensor frame and world
|
||||
- Local sensor frame: camera origin, +Z forward, +Y up
|
||||
|
||||
### Temporal Awareness
|
||||
Nightly scheduled fetch of Sentinel-2 latest imagery + OSM diffs + weather.
|
||||
Changes detected via image comparison and stored as brain memories for
|
||||
contrastive learning.
|
||||
|
||||
### Brain Integration
|
||||
Geospatial context stored as brain memories:
|
||||
- `spatial-geo`: location, elevation, nearby landmarks
|
||||
- `spatial-change`: detected changes in satellite/OSM data
|
||||
- `spatial-weather`: current conditions + forecast
|
||||
- `spatial-season`: vegetation index, snow cover, seasonal patterns
|
||||
- `spatial-local`: hyperlocal web context from Common Crawl WET
|
||||
|
||||
### Extended Data Sources (via ruvector WET/Common Crawl)
|
||||
| Source | Data | Use |
|
||||
|--------|------|-----|
|
||||
| Common Crawl WET | Web text near location | Local business info, reviews, events |
|
||||
| Wikidata | Structured knowledge | Building names, POI descriptions |
|
||||
| NASA FIRMS | Active fire (3-hour) | Safety alerts |
|
||||
| USGS Earthquakes | Seismic events | Safety context |
|
||||
| OpenAQ | Air quality (PM2.5) | Environmental health |
|
||||
| Overture Maps | Building footprints (Meta/MS) | Higher quality than OSM |
|
||||
|
||||
The ruvector brain server has existing `web_ingest` + Common Crawl support.
|
||||
WET files filtered by geographic URL patterns provide hyperlocal context.
|
||||
|
||||
## Consequences
|
||||
### Positive
|
||||
- Agent gains environmental awareness beyond the room
|
||||
- Temporal data enables seasonal calibration of CSI sensing
|
||||
- Change detection finds construction, vegetation, weather effects
|
||||
- All data sources are genuinely free with no API keys
|
||||
|
||||
### Negative
|
||||
- Initial data fetch requires internet (~2MB tiles + ~25MB DEM)
|
||||
- Cached data becomes stale (mitigated by nightly refresh)
|
||||
- IP geolocation has ~1km accuracy (mitigated by manual override)
|
||||
|
|
@ -536,6 +536,105 @@ Both UIs update in real-time via WebSocket and auto-detect the sensing server on
|
|||
|
||||
---
|
||||
|
||||
## Dense Point Cloud (Camera + WiFi CSI Fusion)
|
||||
|
||||
RuView can generate real-time 3D point clouds by fusing camera depth estimation with WiFi CSI spatial sensing. This creates a spatial model of the environment that updates in real-time.
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
# Build the pointcloud binary
|
||||
cd rust-port/wifi-densepose-rs
|
||||
cargo build --release -p wifi-densepose-pointcloud
|
||||
|
||||
# Start the server (auto-detects camera + CSI)
|
||||
./target/release/ruview-pointcloud serve --port 9880
|
||||
```
|
||||
|
||||
Open `http://localhost:9880` for the interactive Three.js 3D viewer.
|
||||
|
||||
### Sensors
|
||||
|
||||
| Sensor | Auto-detected | Data |
|
||||
|--------|--------------|------|
|
||||
| Camera (`/dev/video0`) | Yes (Linux UVC) | RGB frames → MiDaS depth → 3D points |
|
||||
| ESP32 CSI (UDP:3333) | Yes (if provisioned) | ADR-018 binary → occupancy + pose + vitals |
|
||||
| MiDaS depth server (port 9885) | Optional | GPU-accelerated neural depth estimation |
|
||||
|
||||
### Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `ruview-pointcloud serve --port 9880` | Start HTTP server + Three.js viewer |
|
||||
| `ruview-pointcloud demo` | Generate synthetic point cloud (no hardware needed) |
|
||||
| `ruview-pointcloud capture --output room.ply` | Capture single frame to PLY file |
|
||||
| `ruview-pointcloud cameras` | List available cameras |
|
||||
| `ruview-pointcloud train --data-dir ./data` | Depth calibration + occupancy training |
|
||||
| `ruview-pointcloud csi-test --count 100` | Send test CSI frames (no ESP32 needed) |
|
||||
|
||||
### Pipeline Components
|
||||
|
||||
1. **ADR-018 Parser** — Decodes ESP32 CSI binary frames from UDP, extracts I/Q subcarrier amplitudes and phases
|
||||
2. **WiFlow Pose** — 17 COCO keypoint estimation from CSI (loads `wiflow-v1.json`, 186K params)
|
||||
3. **Vital Signs** — Breathing rate from CSI phase analysis (peak counting on stable subcarrier)
|
||||
4. **Motion Detection** — CSI amplitude variance over 20 frames, triggers adaptive capture
|
||||
5. **RF Tomography** — Backprojection from per-node RSSI to 8×8×4 occupancy grid
|
||||
6. **Camera Depth** — MiDaS monocular depth (GPU) with luminance+edge fallback
|
||||
7. **Sensor Fusion** — Voxel-grid merging of camera depth + CSI occupancy
|
||||
8. **Brain Bridge** — Stores spatial observations in the ruOS brain every 60 seconds
|
||||
|
||||
### API Endpoints
|
||||
|
||||
| Endpoint | Method | Returns |
|
||||
|----------|--------|---------|
|
||||
| `/health` | GET | `{"status": "ok"}` |
|
||||
| `/api/status` | GET | Camera, CSI, pipeline state, vitals, motion |
|
||||
| `/api/cloud` | GET | Point cloud (up to 1000 points) + pipeline data |
|
||||
| `/api/splats` | GET | Gaussian splats for Three.js rendering |
|
||||
| `/` | GET | Interactive Three.js 3D viewer |
|
||||
|
||||
### Training
|
||||
|
||||
The training pipeline calibrates depth estimation and occupancy detection:
|
||||
|
||||
```bash
|
||||
ruview-pointcloud train --data-dir ~/.local/share/ruview/training --brain http://127.0.0.1:9876
|
||||
```
|
||||
|
||||
This captures frames, runs depth calibration (grid search over scale/offset/gamma), trains occupancy thresholds, exports DPO preference pairs, and submits results to the ruOS brain.
|
||||
|
||||
### Output Formats
|
||||
|
||||
- **PLY** — Standard 3D point cloud (ASCII, with RGB color)
|
||||
- **Gaussian Splats** — JSON format for Three.js rendering
|
||||
- **Brain Memories** — Spatial observations stored as `spatial-observation`, `spatial-motion`, `spatial-vitals`
|
||||
|
||||
### Deep Room Scan
|
||||
|
||||
Capture a high-quality 3D model of the room:
|
||||
|
||||
```bash
|
||||
# Stop the live server first (frees the camera)
|
||||
# Then capture 20 frames and process with MiDaS
|
||||
ruview-pointcloud capture --frames 20 --output room_model.ply
|
||||
```
|
||||
|
||||
Result: 40,000+ voxels at 5cm resolution, 12,000+ Gaussian splats.
|
||||
|
||||
### ESP32 Provisioning for CSI
|
||||
|
||||
To send CSI data to the pointcloud server:
|
||||
|
||||
```bash
|
||||
python3 firmware/esp32-csi-node/provision.py \
|
||||
--port /dev/ttyACM0 \
|
||||
--ssid "YourWiFi" --password "YourPassword" \
|
||||
--target-ip 192.168.1.123 --target-port 3333 \
|
||||
--node-id 1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Vital Sign Detection
|
||||
|
||||
The system extracts breathing rate and heart rate from CSI signal fluctuations using FFT peak detection.
|
||||
|
|
|
|||
|
|
@ -5446,6 +5446,18 @@ version = "2.0.4"
|
|||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "178f93f84a4a72c582026a45d9b8710acf188df4a22a25434c5dbba1df6c4cac"
|
||||
|
||||
[[package]]
|
||||
name = "ruview-geo"
|
||||
version = "0.1.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"chrono",
|
||||
"reqwest 0.12.28",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"tokio",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ryu"
|
||||
version = "1.0.23"
|
||||
|
|
|
|||
|
|
@ -18,6 +18,7 @@ members = [
|
|||
"crates/wifi-densepose-ruvector",
|
||||
"crates/wifi-densepose-desktop",
|
||||
"crates/wifi-densepose-pointcloud",
|
||||
"crates/wifi-densepose-geo",
|
||||
]
|
||||
# ADR-040: WASM edge crate targets wasm32-unknown-unknown (no_std),
|
||||
# excluded from workspace to avoid breaking `cargo test --workspace`.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,13 @@
|
|||
[package]
|
||||
name = "ruview-geo"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
description = "Geospatial satellite integration — free satellite tiles, DEM, OSM, temporal tracking"
|
||||
|
||||
[dependencies]
|
||||
serde = { workspace = true }
|
||||
serde_json = { workspace = true }
|
||||
tokio = { workspace = true }
|
||||
anyhow = { workspace = true }
|
||||
reqwest = { version = "0.12", features = ["json", "native-tls"], default-features = false }
|
||||
chrono = "0.4"
|
||||
|
|
@ -0,0 +1,105 @@
|
|||
# ruview-geo — Geospatial Satellite Integration
|
||||
|
||||
Free satellite imagery, terrain elevation, and map data for RuView spatial sensing. No API keys required.
|
||||
|
||||
## What It Does
|
||||
|
||||
Integrates your local sensor data (camera + WiFi CSI point cloud) with geographic context:
|
||||
|
||||
- **Satellite tiles** — 10m Sentinel-2 cloudless imagery for your location
|
||||
- **Elevation** — SRTM 30m DEM for terrain modeling
|
||||
- **Buildings + roads** — OpenStreetMap data via Overpass API
|
||||
- **Weather** — Open Meteo current conditions + forecast
|
||||
- **Geo-registration** — maps local sensor coordinates to WGS84
|
||||
- **Temporal tracking** — detects changes over time (construction, vegetation, weather)
|
||||
- **Brain integration** — stores geospatial context as ruOS brain memories
|
||||
|
||||
## Data Sources (all free, no API keys)
|
||||
|
||||
| Source | Data | Resolution | License |
|
||||
|--------|------|-----------|---------|
|
||||
| [EOX S2 Cloudless](https://s2maps.eu/) | Satellite tiles | 10m | CC-BY-4.0 |
|
||||
| [SRTM GL1](https://portal.opentopography.org/) | Elevation/DEM | 30m | Public domain |
|
||||
| [Overpass API](https://overpass-api.de/) | OSM buildings/roads | Vector | ODbL |
|
||||
| [ip-api.com](http://ip-api.com/) | IP geolocation | ~1km | Free |
|
||||
| [Open Meteo](https://open-meteo.com/) | Weather | Point | CC-BY-4.0 |
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | LOC | Purpose |
|
||||
|--------|-----|---------|
|
||||
| `types.rs` | 140 | GeoPoint, GeoBBox, TileCoord, ElevationGrid, OsmFeature |
|
||||
| `coord.rs` | 80 | WGS84/ENU transforms, tile math, haversine distance |
|
||||
| `locate.rs` | 45 | IP geolocation with caching |
|
||||
| `cache.rs` | 55 | Disk cache (`~/.local/share/ruview/geo-cache/`) |
|
||||
| `tiles.rs` | 80 | Sentinel-2/ESRI/OSM tile fetcher |
|
||||
| `terrain.rs` | 100 | SRTM HGT parser, elevation lookup |
|
||||
| `osm.rs` | 150 | Overpass API client, building/road extraction |
|
||||
| `register.rs` | 50 | Local-to-WGS84 coordinate registration |
|
||||
| `fuse.rs` | 70 | Multi-source scene builder + summary |
|
||||
| `brain.rs` | 30 | Store geo context in ruOS brain |
|
||||
| `temporal.rs` | 100 | Weather, OSM change detection |
|
||||
|
||||
## Usage
|
||||
|
||||
```rust
|
||||
use ruview_geo::{fuse, brain, temporal};
|
||||
|
||||
// Build geo scene for current location
|
||||
let scene = fuse::build_scene(500.0).await?; // 500m radius
|
||||
println!("{}", fuse::summarize(&scene));
|
||||
// "Location: 43.6532N, 79.3832W, elevation 76m ASL.
|
||||
// 23 buildings within view. 8 roads nearby (King St, Queen St).
|
||||
// 12 satellite tiles at zoom 16."
|
||||
|
||||
// Store in brain
|
||||
brain::store_geo_context(&scene).await?;
|
||||
|
||||
// Fetch weather
|
||||
let weather = temporal::fetch_weather(&scene.location).await?;
|
||||
// temperature: 12°C, partly cloudy, humidity 65%
|
||||
```
|
||||
|
||||
## Brain Integration
|
||||
|
||||
Geospatial context is stored as brain memories:
|
||||
|
||||
| Category | Content | Frequency |
|
||||
|----------|---------|-----------|
|
||||
| `spatial-geo` | Location, elevation, buildings, roads | On startup + daily |
|
||||
| `spatial-weather` | Temperature, conditions, humidity, wind | Nightly |
|
||||
| `spatial-change` | New/removed buildings, road changes | Nightly diff |
|
||||
|
||||
The ruOS agent can search: "what buildings are near me?" or "what's the weather?" and get geospatial context from the brain.
|
||||
|
||||
## Security
|
||||
|
||||
- No API keys stored or transmitted
|
||||
- IP geolocation uses HTTP (not HTTPS) — location is approximate (~1km)
|
||||
- All tile fetches use HTTPS except ip-api.com
|
||||
- Path traversal protection in cache key sanitization
|
||||
- No user data sent to external services
|
||||
- All data cached locally after first fetch
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
IP Geolocation ──→ (lat, lon)
|
||||
│
|
||||
┌─────────────┼─────────────┐
|
||||
▼ ▼ ▼
|
||||
Sentinel-2 SRTM DEM Overpass API
|
||||
(tiles) (elevation) (buildings/roads)
|
||||
│ │ │
|
||||
└─────────────┼─────────────┘
|
||||
▼
|
||||
GeoScene (fused)
|
||||
│
|
||||
┌───────┴───────┐
|
||||
▼ ▼
|
||||
Brain Memory Three.js Viewer
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
MIT (same as RuView)
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
use ruview_geo::*;
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> anyhow::Result<()> {
|
||||
println!("╔══════════════════════════════════════════════╗");
|
||||
println!("║ ruview-geo — Real Data Validation ║");
|
||||
println!("╚══════════════════════════════════════════════╝\n");
|
||||
|
||||
let t0 = std::time::Instant::now();
|
||||
let cache = cache::TileCache::new("/tmp/ruview-geo-validate");
|
||||
|
||||
let loc = locate::get_location(&format!("{}/location.json", cache.base_dir.display())).await?;
|
||||
println!(" Location: {:.4}N, {:.4}W", loc.lat, loc.lon);
|
||||
|
||||
let bbox = GeoBBox::from_center(&loc, 300.0);
|
||||
let tiles_list = tiles::fetch_area(&tiles::TileProvider::Sentinel2Cloudless, &bbox, 16, &cache).await?;
|
||||
println!(" Tiles: {} ({:.0}KB)", tiles_list.len(),
|
||||
tiles_list.iter().map(|t| t.data.len()).sum::<usize>() as f64 / 1024.0);
|
||||
|
||||
let dem = terrain::fetch_elevation(&loc, &cache).await?;
|
||||
println!(" Elevation: {:.0}m (grid {}x{})", terrain::elevation_at(&dem, &loc), dem.cols, dem.rows);
|
||||
|
||||
let buildings = osm::fetch_buildings(&loc, 300.0).await.unwrap_or_default();
|
||||
let roads = osm::fetch_roads(&loc, 300.0).await.unwrap_or_default();
|
||||
println!(" OSM: {} buildings, {} roads", buildings.len(), roads.len());
|
||||
|
||||
let weather = temporal::fetch_weather(&loc).await?;
|
||||
println!(" Weather: {:.0}°C humidity={:.0}% wind={:.1}m/s",
|
||||
weather.temperature_c, weather.humidity_pct, weather.wind_speed_ms);
|
||||
|
||||
let scene = GeoScene {
|
||||
location: loc.clone(), bbox, elevation_m: terrain::elevation_at(&dem, &loc),
|
||||
buildings, roads, tile_count: tiles_list.len(),
|
||||
registration: register::auto_register(&loc),
|
||||
last_updated: chrono::Utc::now().to_rfc3339(),
|
||||
};
|
||||
println!("\n {}", fuse::summarize(&scene));
|
||||
|
||||
match brain::store_geo_context(&scene).await {
|
||||
Ok(n) => println!(" Brain: {} memories stored", n),
|
||||
Err(e) => println!(" Brain: {e}"),
|
||||
}
|
||||
|
||||
println!("\n Total: {}ms | Cache: {:.0}KB",
|
||||
t0.elapsed().as_millis(), cache.size_bytes() as f64 / 1024.0);
|
||||
Ok(())
|
||||
}
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
//! Brain integration — store geospatial context in ruOS brain.
|
||||
|
||||
use crate::fuse;
|
||||
use crate::types::GeoScene;
|
||||
use anyhow::Result;
|
||||
|
||||
const BRAIN_URL: &str = "http://127.0.0.1:9876";
|
||||
|
||||
/// Store geospatial context in the brain.
|
||||
pub async fn store_geo_context(scene: &GeoScene) -> Result<u32> {
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(5))
|
||||
.build()?;
|
||||
|
||||
let mut stored = 0u32;
|
||||
|
||||
// Store location summary
|
||||
let summary = fuse::summarize(scene);
|
||||
let body = serde_json::json!({
|
||||
"category": "spatial-geo",
|
||||
"content": summary,
|
||||
});
|
||||
if client.post(format!("{BRAIN_URL}/memories")).json(&body).send().await.is_ok() {
|
||||
stored += 1;
|
||||
}
|
||||
|
||||
Ok(stored)
|
||||
}
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
//! Disk cache for tiles, DEM, and OSM data.
|
||||
|
||||
use anyhow::Result;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
pub struct TileCache {
|
||||
pub base_dir: PathBuf,
|
||||
}
|
||||
|
||||
impl TileCache {
|
||||
pub fn new(base_dir: &str) -> Self {
|
||||
let expanded = base_dir.replace('~', &std::env::var("HOME").unwrap_or_default());
|
||||
let path = PathBuf::from(expanded);
|
||||
let _ = std::fs::create_dir_all(&path);
|
||||
Self { base_dir: path }
|
||||
}
|
||||
|
||||
pub fn default_cache() -> Self {
|
||||
Self::new("~/.local/share/ruview/geo-cache")
|
||||
}
|
||||
|
||||
pub fn get(&self, key: &str) -> Option<Vec<u8>> {
|
||||
let path = self.key_path(key);
|
||||
std::fs::read(&path).ok()
|
||||
}
|
||||
|
||||
pub fn put(&self, key: &str, data: &[u8]) -> Result<()> {
|
||||
let path = self.key_path(key);
|
||||
if let Some(parent) = path.parent() {
|
||||
std::fs::create_dir_all(parent)?;
|
||||
}
|
||||
std::fs::write(&path, data)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn has(&self, key: &str) -> bool {
|
||||
self.key_path(key).exists()
|
||||
}
|
||||
|
||||
pub fn size_bytes(&self) -> u64 {
|
||||
walkdir(self.base_dir.as_path())
|
||||
}
|
||||
|
||||
fn key_path(&self, key: &str) -> PathBuf {
|
||||
// Sanitize key to prevent path traversal
|
||||
let safe_key = key.replace("..", "_").replace('/', "_");
|
||||
self.base_dir.join(safe_key)
|
||||
}
|
||||
}
|
||||
|
||||
fn walkdir(path: &Path) -> u64 {
|
||||
std::fs::read_dir(path)
|
||||
.into_iter()
|
||||
.flatten()
|
||||
.filter_map(|e| e.ok())
|
||||
.map(|e| {
|
||||
if e.path().is_dir() { walkdir(&e.path()) }
|
||||
else { e.metadata().map(|m| m.len()).unwrap_or(0) }
|
||||
})
|
||||
.sum()
|
||||
}
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
//! Coordinate transforms — WGS84, UTM, ENU, tile math.
|
||||
|
||||
use crate::types::{GeoPoint, GeoBBox, TileCoord};
|
||||
|
||||
const WGS84_A: f64 = 6_378_137.0;
|
||||
const WGS84_F: f64 = 1.0 / 298.257_223_563;
|
||||
const WGS84_E2: f64 = 2.0 * WGS84_F - WGS84_F * WGS84_F;
|
||||
|
||||
/// Haversine distance in meters.
|
||||
pub fn haversine(a: &GeoPoint, b: &GeoPoint) -> f64 {
|
||||
let dlat = (b.lat - a.lat).to_radians();
|
||||
let dlon = (b.lon - a.lon).to_radians();
|
||||
let lat1 = a.lat.to_radians();
|
||||
let lat2 = b.lat.to_radians();
|
||||
let h = (dlat / 2.0).sin().powi(2) + lat1.cos() * lat2.cos() * (dlon / 2.0).sin().powi(2);
|
||||
2.0 * WGS84_A * h.sqrt().asin()
|
||||
}
|
||||
|
||||
/// WGS84 to local ENU (East-North-Up) relative to origin, in meters.
|
||||
pub fn wgs84_to_enu(point: &GeoPoint, origin: &GeoPoint) -> [f64; 3] {
|
||||
let dlat = (point.lat - origin.lat).to_radians();
|
||||
let dlon = (point.lon - origin.lon).to_radians();
|
||||
let lat = origin.lat.to_radians();
|
||||
let east = dlon * WGS84_A * lat.cos();
|
||||
let north = dlat * WGS84_A;
|
||||
let up = point.alt - origin.alt;
|
||||
[east, north, up]
|
||||
}
|
||||
|
||||
/// Local ENU to WGS84.
|
||||
pub fn enu_to_wgs84(enu: &[f64; 3], origin: &GeoPoint) -> GeoPoint {
|
||||
let lat = origin.lat.to_radians();
|
||||
let dlat = enu[1] / WGS84_A;
|
||||
let dlon = enu[0] / (WGS84_A * lat.cos());
|
||||
GeoPoint {
|
||||
lat: origin.lat + dlat.to_degrees(),
|
||||
lon: origin.lon + dlon.to_degrees(),
|
||||
alt: origin.alt + enu[2],
|
||||
}
|
||||
}
|
||||
|
||||
/// WGS84 to XYZ tile coordinates (Slippy Map).
|
||||
pub fn wgs84_to_tile(lat: f64, lon: f64, zoom: u8) -> TileCoord {
|
||||
let n = 2f64.powi(zoom as i32);
|
||||
let x = ((lon + 180.0) / 360.0 * n).floor() as u32;
|
||||
let lat_rad = lat.to_radians();
|
||||
let y = ((1.0 - lat_rad.tan().asinh() / std::f64::consts::PI) / 2.0 * n).floor() as u32;
|
||||
TileCoord { z: zoom, x, y }
|
||||
}
|
||||
|
||||
/// Tile bounds in WGS84.
|
||||
pub fn tile_bounds(coord: &TileCoord) -> GeoBBox {
|
||||
let n = 2f64.powi(coord.z as i32);
|
||||
let west = coord.x as f64 / n * 360.0 - 180.0;
|
||||
let east = (coord.x + 1) as f64 / n * 360.0 - 180.0;
|
||||
let north = (std::f64::consts::PI * (1.0 - 2.0 * coord.y as f64 / n)).sinh().atan().to_degrees();
|
||||
let south = (std::f64::consts::PI * (1.0 - 2.0 * (coord.y + 1) as f64 / n)).sinh().atan().to_degrees();
|
||||
GeoBBox { south, west, north, east }
|
||||
}
|
||||
|
||||
/// Get all tile coordinates covering a bounding box at a zoom level.
|
||||
pub fn tiles_for_bbox(bbox: &GeoBBox, zoom: u8) -> Vec<TileCoord> {
|
||||
let tl = wgs84_to_tile(bbox.north, bbox.west, zoom);
|
||||
let br = wgs84_to_tile(bbox.south, bbox.east, zoom);
|
||||
let mut tiles = Vec::new();
|
||||
for y in tl.y..=br.y {
|
||||
for x in tl.x..=br.x {
|
||||
tiles.push(TileCoord { z: zoom, x, y });
|
||||
}
|
||||
}
|
||||
tiles
|
||||
}
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
//! Multi-source fusion — satellite + terrain + OSM + local sensor data.
|
||||
|
||||
use crate::cache::TileCache;
|
||||
use crate::types::*;
|
||||
use crate::{locate, osm, terrain, tiles};
|
||||
use anyhow::Result;
|
||||
|
||||
/// Build a complete geo scene for a location.
|
||||
pub async fn build_scene(radius_m: f64) -> Result<GeoScene> {
|
||||
let cache = TileCache::default_cache();
|
||||
|
||||
// 1. Locate
|
||||
let cache_path = cache.base_dir.join("location.json");
|
||||
let location = locate::get_location(cache_path.to_str().unwrap_or("")).await?;
|
||||
eprintln!(" Geo: located at {:.4}N, {:.4}W", location.lat, location.lon);
|
||||
|
||||
// 2. Fetch satellite tiles
|
||||
let bbox = GeoBBox::from_center(&location, radius_m);
|
||||
let tile_list = tiles::fetch_area(&tiles::TileProvider::Sentinel2Cloudless, &bbox, 16, &cache).await?;
|
||||
eprintln!(" Geo: fetched {} satellite tiles", tile_list.len());
|
||||
|
||||
// 3. Fetch elevation
|
||||
let dem = terrain::fetch_elevation(&location, &cache).await?;
|
||||
let elevation = terrain::elevation_at(&dem, &location);
|
||||
eprintln!(" Geo: elevation {:.0}m ASL", elevation);
|
||||
|
||||
// 4. Fetch OSM buildings + roads
|
||||
let buildings = osm::fetch_buildings(&location, radius_m).await.unwrap_or_default();
|
||||
let roads = osm::fetch_roads(&location, radius_m).await.unwrap_or_default();
|
||||
eprintln!(" Geo: {} buildings, {} roads", buildings.len(), roads.len());
|
||||
|
||||
// 5. Build registration
|
||||
let mut reg_origin = location.clone();
|
||||
reg_origin.alt = elevation as f64;
|
||||
let registration = crate::register::auto_register(®_origin);
|
||||
|
||||
Ok(GeoScene {
|
||||
location: reg_origin,
|
||||
bbox,
|
||||
elevation_m: elevation,
|
||||
buildings,
|
||||
roads,
|
||||
tile_count: tile_list.len(),
|
||||
registration,
|
||||
last_updated: chrono::Utc::now().to_rfc3339(),
|
||||
})
|
||||
}
|
||||
|
||||
/// Generate a text summary of the geo scene.
|
||||
pub fn summarize(scene: &GeoScene) -> String {
|
||||
let building_count = scene.buildings.len();
|
||||
let road_count = scene.roads.len();
|
||||
let road_names: Vec<&str> = scene.roads.iter()
|
||||
.filter_map(|r| match r {
|
||||
OsmFeature::Road { name, .. } => name.as_deref(),
|
||||
_ => None,
|
||||
})
|
||||
.take(3)
|
||||
.collect();
|
||||
|
||||
format!(
|
||||
"Location: {:.4}N, {:.4}W, elevation {:.0}m ASL. \
|
||||
{} buildings within view. {} roads nearby{}. \
|
||||
{} satellite tiles at zoom 16. Updated: {}.",
|
||||
scene.location.lat, scene.location.lon, scene.elevation_m,
|
||||
building_count, road_count,
|
||||
if road_names.is_empty() { String::new() }
|
||||
else { format!(" ({})", road_names.join(", ")) },
|
||||
scene.tile_count,
|
||||
&scene.last_updated[..10],
|
||||
)
|
||||
}
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
//! wifi-densepose-geo — geospatial satellite integration for RuView.
|
||||
//!
|
||||
//! Provides: IP geolocation, satellite tile fetching (Sentinel-2),
|
||||
//! SRTM elevation, OSM buildings/roads, coordinate transforms,
|
||||
//! temporal change tracking, and brain memory integration.
|
||||
|
||||
pub mod types;
|
||||
pub mod coord;
|
||||
pub mod locate;
|
||||
pub mod cache;
|
||||
pub mod tiles;
|
||||
pub mod terrain;
|
||||
pub mod osm;
|
||||
pub mod register;
|
||||
pub mod fuse;
|
||||
pub mod brain;
|
||||
pub mod temporal;
|
||||
|
||||
pub use types::*;
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
//! IP geolocation — determine location from public IP.
|
||||
|
||||
use crate::types::GeoPoint;
|
||||
use anyhow::Result;
|
||||
|
||||
/// Locate by IP address (free, no API key).
|
||||
pub async fn locate_by_ip() -> Result<GeoPoint> {
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(5))
|
||||
.build()?;
|
||||
|
||||
// Primary: ip-api.com (free, 45 req/min)
|
||||
let resp: serde_json::Value = client
|
||||
.get("http://ip-api.com/json/?fields=lat,lon,city,regionName,country")
|
||||
.send().await?
|
||||
.json().await?;
|
||||
|
||||
let lat = resp.get("lat").and_then(|v| v.as_f64()).unwrap_or(0.0);
|
||||
let lon = resp.get("lon").and_then(|v| v.as_f64()).unwrap_or(0.0);
|
||||
|
||||
if lat == 0.0 && lon == 0.0 {
|
||||
anyhow::bail!("IP geolocation returned (0,0)");
|
||||
}
|
||||
|
||||
Ok(GeoPoint { lat, lon, alt: 0.0 })
|
||||
}
|
||||
|
||||
/// Get location with caching.
|
||||
pub async fn get_location(cache_path: &str) -> Result<GeoPoint> {
|
||||
// Check cache
|
||||
if let Ok(data) = std::fs::read_to_string(cache_path) {
|
||||
if let Ok(point) = serde_json::from_str::<GeoPoint>(&data) {
|
||||
return Ok(point);
|
||||
}
|
||||
}
|
||||
|
||||
let point = locate_by_ip().await?;
|
||||
let _ = std::fs::write(cache_path, serde_json::to_string(&point)?);
|
||||
Ok(point)
|
||||
}
|
||||
|
|
@ -0,0 +1,136 @@
|
|||
//! OpenStreetMap data via Overpass API — buildings, roads, land use.
|
||||
|
||||
use crate::types::{GeoBBox, GeoPoint, OsmFeature};
|
||||
use anyhow::Result;
|
||||
|
||||
const OVERPASS_URL: &str = "https://overpass-api.de/api/interpreter";
|
||||
|
||||
/// Fetch buildings within radius of a point.
|
||||
///
|
||||
/// Uses an inclusive `["building"]` filter that matches all building values
|
||||
/// (residential, commercial, yes, etc.) and also queries relations for
|
||||
/// multipolygon buildings. Default recommended radius: 500 m.
|
||||
pub async fn fetch_buildings(center: &GeoPoint, radius_m: f64) -> Result<Vec<OsmFeature>> {
|
||||
let bbox = GeoBBox::from_center(center, radius_m);
|
||||
let query = format!(
|
||||
r#"[out:json][timeout:25];(way["building"]({},{},{},{});relation["building"]({},{},{},{}););out body;>;out skel qt;"#,
|
||||
bbox.south, bbox.west, bbox.north, bbox.east,
|
||||
bbox.south, bbox.west, bbox.north, bbox.east,
|
||||
);
|
||||
let resp = overpass_query(&query).await?;
|
||||
parse_buildings(&resp)
|
||||
}
|
||||
|
||||
/// Fetch roads within radius.
|
||||
pub async fn fetch_roads(center: &GeoPoint, radius_m: f64) -> Result<Vec<OsmFeature>> {
|
||||
let bbox = GeoBBox::from_center(center, radius_m);
|
||||
let query = format!(
|
||||
r#"[out:json][timeout:10];way["highway"]({},{},{},{});out body;>;out skel qt;"#,
|
||||
bbox.south, bbox.west, bbox.north, bbox.east
|
||||
);
|
||||
let resp = overpass_query(&query).await?;
|
||||
parse_roads(&resp)
|
||||
}
|
||||
|
||||
async fn overpass_query(query: &str) -> Result<serde_json::Value> {
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(15))
|
||||
.user_agent("RuView/0.1")
|
||||
.build()?;
|
||||
|
||||
let resp = client.post(OVERPASS_URL)
|
||||
.form(&[("data", query)])
|
||||
.send().await?;
|
||||
|
||||
if !resp.status().is_success() {
|
||||
anyhow::bail!("Overpass API error: {}", resp.status());
|
||||
}
|
||||
Ok(resp.json().await?)
|
||||
}
|
||||
|
||||
fn parse_buildings(data: &serde_json::Value) -> Result<Vec<OsmFeature>> {
|
||||
let mut buildings = Vec::new();
|
||||
let mut nodes: std::collections::HashMap<u64, [f64; 2]> = std::collections::HashMap::new();
|
||||
|
||||
let elements = data.get("elements").and_then(|e| e.as_array()).cloned().unwrap_or_default();
|
||||
|
||||
// First pass: collect nodes
|
||||
for el in &elements {
|
||||
if el.get("type").and_then(|t| t.as_str()) == Some("node") {
|
||||
if let (Some(id), Some(lat), Some(lon)) = (
|
||||
el.get("id").and_then(|v| v.as_u64()),
|
||||
el.get("lat").and_then(|v| v.as_f64()),
|
||||
el.get("lon").and_then(|v| v.as_f64()),
|
||||
) {
|
||||
nodes.insert(id, [lat, lon]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Second pass: build ways
|
||||
for el in &elements {
|
||||
if el.get("type").and_then(|t| t.as_str()) != Some("way") { continue; }
|
||||
let tags = el.get("tags").cloned().unwrap_or(serde_json::json!({}));
|
||||
if tags.get("building").is_none() { continue; }
|
||||
|
||||
let node_ids = el.get("nodes").and_then(|n| n.as_array()).cloned().unwrap_or_default();
|
||||
let outline: Vec<[f64; 2]> = node_ids.iter()
|
||||
.filter_map(|id| id.as_u64().and_then(|id| nodes.get(&id).copied()))
|
||||
.collect();
|
||||
|
||||
if outline.len() < 3 { continue; }
|
||||
|
||||
let height = tags.get("height").and_then(|h| h.as_str())
|
||||
.and_then(|s| s.trim_end_matches('m').trim().parse::<f32>().ok())
|
||||
.or(Some(8.0)); // default building height
|
||||
|
||||
let name = tags.get("name").and_then(|n| n.as_str()).map(|s| s.to_string());
|
||||
|
||||
buildings.push(OsmFeature::Building { outline, height, name });
|
||||
}
|
||||
|
||||
Ok(buildings)
|
||||
}
|
||||
|
||||
fn parse_roads(data: &serde_json::Value) -> Result<Vec<OsmFeature>> {
|
||||
let mut roads = Vec::new();
|
||||
let mut nodes: std::collections::HashMap<u64, [f64; 2]> = std::collections::HashMap::new();
|
||||
|
||||
let elements = data.get("elements").and_then(|e| e.as_array()).cloned().unwrap_or_default();
|
||||
|
||||
for el in &elements {
|
||||
if el.get("type").and_then(|t| t.as_str()) == Some("node") {
|
||||
if let (Some(id), Some(lat), Some(lon)) = (
|
||||
el.get("id").and_then(|v| v.as_u64()),
|
||||
el.get("lat").and_then(|v| v.as_f64()),
|
||||
el.get("lon").and_then(|v| v.as_f64()),
|
||||
) {
|
||||
nodes.insert(id, [lat, lon]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for el in &elements {
|
||||
if el.get("type").and_then(|t| t.as_str()) != Some("way") { continue; }
|
||||
let tags = el.get("tags").cloned().unwrap_or(serde_json::json!({}));
|
||||
let highway = tags.get("highway").and_then(|h| h.as_str());
|
||||
if highway.is_none() { continue; }
|
||||
|
||||
let node_ids = el.get("nodes").and_then(|n| n.as_array()).cloned().unwrap_or_default();
|
||||
let path: Vec<[f64; 2]> = node_ids.iter()
|
||||
.filter_map(|id| id.as_u64().and_then(|id| nodes.get(&id).copied()))
|
||||
.collect();
|
||||
|
||||
if path.len() < 2 { continue; }
|
||||
|
||||
let name = tags.get("name").and_then(|n| n.as_str()).map(|s| s.to_string());
|
||||
|
||||
roads.push(OsmFeature::Road {
|
||||
path,
|
||||
road_type: highway.unwrap_or("unknown").to_string(),
|
||||
name,
|
||||
});
|
||||
}
|
||||
|
||||
Ok(roads)
|
||||
}
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
//! Geo-registration — maps local sensor coordinates to WGS84.
|
||||
|
||||
use crate::coord;
|
||||
use crate::types::{GeoPoint, GeoRegistration};
|
||||
|
||||
/// Auto-register using IP location (sensor at IP location, facing north).
|
||||
pub fn auto_register(ip_location: &GeoPoint) -> GeoRegistration {
|
||||
GeoRegistration {
|
||||
origin: ip_location.clone(),
|
||||
heading_deg: 0.0,
|
||||
scale: 1.0,
|
||||
}
|
||||
}
|
||||
|
||||
/// Transform local point [x, y, z] to WGS84.
|
||||
pub fn local_to_wgs84(reg: &GeoRegistration, local: &[f32; 3]) -> GeoPoint {
|
||||
let heading_rad = reg.heading_deg.to_radians();
|
||||
let cos_h = heading_rad.cos();
|
||||
let sin_h = heading_rad.sin();
|
||||
|
||||
// Rotate local by heading (local X → East when heading=0)
|
||||
let east = (local[0] as f64 * cos_h - local[2] as f64 * sin_h) * reg.scale;
|
||||
let north = (local[0] as f64 * sin_h + local[2] as f64 * cos_h) * reg.scale;
|
||||
let up = local[1] as f64 * reg.scale;
|
||||
|
||||
coord::enu_to_wgs84(&[east, north, up], ®.origin)
|
||||
}
|
||||
|
||||
/// Transform WGS84 to local point.
|
||||
pub fn wgs84_to_local(reg: &GeoRegistration, geo: &GeoPoint) -> [f32; 3] {
|
||||
let enu = coord::wgs84_to_enu(geo, ®.origin);
|
||||
let heading_rad = (-reg.heading_deg).to_radians();
|
||||
let cos_h = heading_rad.cos();
|
||||
let sin_h = heading_rad.sin();
|
||||
|
||||
let x = ((enu[0] * cos_h - enu[1] * sin_h) / reg.scale) as f32;
|
||||
let z = ((enu[0] * sin_h + enu[1] * cos_h) / reg.scale) as f32;
|
||||
let y = (enu[2] / reg.scale) as f32;
|
||||
|
||||
[x, y, z]
|
||||
}
|
||||
|
|
@ -0,0 +1,310 @@
|
|||
//! Temporal change tracking — detect changes in satellite/OSM/weather over time.
|
||||
|
||||
use crate::cache::TileCache;
|
||||
use crate::types::GeoPoint;
|
||||
#[allow(unused_imports)]
|
||||
use crate::types::GeoScene;
|
||||
use anyhow::Result;
|
||||
|
||||
/// Fetch current weather (Open Meteo, free, no key).
|
||||
pub async fn fetch_weather(point: &GeoPoint) -> Result<WeatherData> {
|
||||
let url = format!(
|
||||
"https://api.open-meteo.com/v1/forecast?latitude={:.4}&longitude={:.4}¤t=temperature_2m,relative_humidity_2m,wind_speed_10m,weather_code",
|
||||
point.lat, point.lon
|
||||
);
|
||||
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(10))
|
||||
.build()?;
|
||||
|
||||
let resp: serde_json::Value = client.get(&url).send().await?.json().await?;
|
||||
let current = resp.get("current").cloned().unwrap_or(serde_json::json!({}));
|
||||
|
||||
Ok(WeatherData {
|
||||
temperature_c: current.get("temperature_2m").and_then(|v| v.as_f64()).unwrap_or(0.0) as f32,
|
||||
humidity_pct: current.get("relative_humidity_2m").and_then(|v| v.as_f64()).unwrap_or(0.0) as f32,
|
||||
wind_speed_ms: current.get("wind_speed_10m").and_then(|v| v.as_f64()).unwrap_or(0.0) as f32,
|
||||
weather_code: current.get("weather_code").and_then(|v| v.as_u64()).unwrap_or(0) as u16,
|
||||
})
|
||||
}
|
||||
|
||||
/// Check for OSM changes since last fetch.
|
||||
pub async fn check_osm_changes(scene: &GeoScene, cache: &TileCache) -> Result<Vec<String>> {
|
||||
let mut changes = Vec::new();
|
||||
|
||||
let cache_key = "osm_building_count";
|
||||
let prev_count: usize = cache.get(cache_key)
|
||||
.and_then(|d| String::from_utf8(d).ok())
|
||||
.and_then(|s| s.trim().parse().ok())
|
||||
.unwrap_or(0);
|
||||
|
||||
let current_count = scene.buildings.len();
|
||||
if prev_count > 0 && current_count != prev_count {
|
||||
let diff = current_count as i64 - prev_count as i64;
|
||||
changes.push(format!("Building count changed: {} → {} ({:+})", prev_count, current_count, diff));
|
||||
}
|
||||
|
||||
cache.put(cache_key, current_count.to_string().as_bytes())?;
|
||||
Ok(changes)
|
||||
}
|
||||
|
||||
/// Generate temporal summary for brain storage.
|
||||
pub fn temporal_summary(weather: &WeatherData, changes: &[String]) -> String {
|
||||
let weather_desc = match weather.weather_code {
|
||||
0 => "clear sky",
|
||||
1..=3 => "partly cloudy",
|
||||
45 | 48 => "foggy",
|
||||
51..=57 => "drizzle",
|
||||
61..=67 => "rain",
|
||||
71..=77 => "snow",
|
||||
80..=82 => "showers",
|
||||
95..=99 => "thunderstorm",
|
||||
_ => "unknown",
|
||||
};
|
||||
|
||||
let mut summary = format!(
|
||||
"Weather: {:.0}°C, {weather_desc}, humidity {:.0}%, wind {:.1}m/s.",
|
||||
weather.temperature_c, weather.humidity_pct, weather.wind_speed_ms,
|
||||
);
|
||||
|
||||
for change in changes {
|
||||
summary.push_str(&format!(" Change: {change}."));
|
||||
}
|
||||
|
||||
summary
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)]
|
||||
pub struct WeatherData {
|
||||
pub temperature_c: f32,
|
||||
pub humidity_pct: f32,
|
||||
pub wind_speed_ms: f32,
|
||||
pub weather_code: u16,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Satellite tile change detection
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Result of comparing two tile snapshots.
|
||||
#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)]
|
||||
pub struct TileChangeResult {
|
||||
/// 0.0 = identical, 1.0 = completely different.
|
||||
pub diff_score: f64,
|
||||
/// Number of pixels that changed.
|
||||
pub changed_pixels: usize,
|
||||
/// Total pixels compared.
|
||||
pub total_pixels: usize,
|
||||
}
|
||||
|
||||
/// Compare a newly-fetched tile against its previously-cached version.
|
||||
///
|
||||
/// Returns a `TileChangeResult` with a diff score between 0.0 (identical) and
|
||||
/// 1.0 (completely different). When the diff exceeds 0.1 the function stores
|
||||
/// a change event as a brain memory via the local ruOS brain endpoint.
|
||||
pub async fn detect_tile_changes(
|
||||
cache_key: &str,
|
||||
new_data: &[u8],
|
||||
cache: &TileCache,
|
||||
) -> Result<TileChangeResult> {
|
||||
let previous = cache.get(cache_key);
|
||||
|
||||
let result = match previous {
|
||||
Some(ref old_data) => {
|
||||
let total = old_data.len().max(new_data.len()).max(1);
|
||||
let comparable = old_data.len().min(new_data.len());
|
||||
let mut changed: usize = 0;
|
||||
for i in 0..comparable {
|
||||
if old_data[i] != new_data[i] {
|
||||
changed += 1;
|
||||
}
|
||||
}
|
||||
// Any extra bytes in the longer slice count as changed.
|
||||
changed += total - comparable;
|
||||
|
||||
TileChangeResult {
|
||||
diff_score: changed as f64 / total as f64,
|
||||
changed_pixels: changed,
|
||||
total_pixels: total,
|
||||
}
|
||||
}
|
||||
None => {
|
||||
// No previous data — treat as fully new (score 1.0).
|
||||
TileChangeResult {
|
||||
diff_score: 1.0,
|
||||
changed_pixels: new_data.len(),
|
||||
total_pixels: new_data.len().max(1),
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// Persist new snapshot into cache for future comparisons.
|
||||
cache.put(cache_key, new_data)?;
|
||||
|
||||
// When significant change is detected, store a brain memory.
|
||||
if result.diff_score > 0.1 {
|
||||
let _ = store_change_event(cache_key, &result).await;
|
||||
}
|
||||
|
||||
Ok(result)
|
||||
}
|
||||
|
||||
/// Post a change event to the local ruOS brain.
|
||||
async fn store_change_event(cache_key: &str, result: &TileChangeResult) -> Result<()> {
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(5))
|
||||
.build()?;
|
||||
|
||||
let body = serde_json::json!({
|
||||
"category": "spatial-change",
|
||||
"content": format!(
|
||||
"Tile change detected for {cache_key}: diff={:.3}, changed={}/{}",
|
||||
result.diff_score, result.changed_pixels, result.total_pixels,
|
||||
),
|
||||
});
|
||||
|
||||
client
|
||||
.post("http://127.0.0.1:9876/memories")
|
||||
.json(&body)
|
||||
.send()
|
||||
.await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Night mode detection
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Approximate check whether the current time is "night" at a given latitude.
|
||||
///
|
||||
/// Uses a simplified sunrise/sunset model based on the solar declination and
|
||||
/// hour angle. When it is night the system should rely on CSI data only
|
||||
/// (satellite imagery is not useful in darkness).
|
||||
pub fn is_night(lat_deg: f64) -> bool {
|
||||
let now = chrono::Utc::now();
|
||||
is_night_at(lat_deg, now)
|
||||
}
|
||||
|
||||
/// Testable version of [`is_night`] that accepts an explicit timestamp.
|
||||
pub fn is_night_at(lat_deg: f64, utc: chrono::DateTime<chrono::Utc>) -> bool {
|
||||
use chrono::Datelike;
|
||||
use std::f64::consts::PI;
|
||||
|
||||
let day_of_year = utc.ordinal() as f64;
|
||||
let hour_utc = utc.timestamp() % 86400;
|
||||
let solar_hour = (hour_utc as f64) / 3600.0; // 0..24
|
||||
|
||||
// Solar declination (Spencer, 1971 — simplified)
|
||||
let gamma = 2.0 * PI * (day_of_year - 1.0) / 365.0;
|
||||
let decl = 0.006918
|
||||
- 0.399912 * gamma.cos()
|
||||
+ 0.070257 * gamma.sin()
|
||||
- 0.006758 * (2.0 * gamma).cos()
|
||||
+ 0.000907 * (2.0 * gamma).sin();
|
||||
|
||||
let lat_rad = lat_deg.to_radians();
|
||||
|
||||
// Cosine of the hour angle at sunrise/sunset (geometric, no refraction)
|
||||
let cos_ha = -(lat_rad.tan() * decl.tan());
|
||||
|
||||
// Polar day / polar night
|
||||
if cos_ha < -1.0 {
|
||||
return false; // midnight sun — never night
|
||||
}
|
||||
if cos_ha > 1.0 {
|
||||
return true; // polar night — always night
|
||||
}
|
||||
|
||||
let ha_sunrise = cos_ha.acos(); // radians, symmetric about solar noon
|
||||
let daylight_hours = 2.0 * ha_sunrise * 12.0 / PI;
|
||||
let solar_noon = 12.0; // approximation (ignores longitude offset)
|
||||
let sunrise = solar_noon - daylight_hours / 2.0;
|
||||
let sunset = solar_noon + daylight_hours / 2.0;
|
||||
|
||||
solar_hour < sunrise || solar_hour > sunset
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_is_night_at_equator_noon() {
|
||||
// Noon UTC at equator on March 20 — should be daytime.
|
||||
let dt = chrono::NaiveDate::from_ymd_opt(2025, 3, 20)
|
||||
.unwrap()
|
||||
.and_hms_opt(12, 0, 0)
|
||||
.unwrap()
|
||||
.and_utc();
|
||||
assert!(!is_night_at(0.0, dt));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_is_night_at_equator_midnight() {
|
||||
// Midnight UTC at equator — should be night.
|
||||
let dt = chrono::NaiveDate::from_ymd_opt(2025, 3, 20)
|
||||
.unwrap()
|
||||
.and_hms_opt(2, 0, 0)
|
||||
.unwrap()
|
||||
.and_utc();
|
||||
assert!(is_night_at(0.0, dt));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_midnight_sun_arctic() {
|
||||
// Late June at 70 N — midnight sun, never night.
|
||||
let dt = chrono::NaiveDate::from_ymd_opt(2025, 6, 21)
|
||||
.unwrap()
|
||||
.and_hms_opt(0, 0, 0)
|
||||
.unwrap()
|
||||
.and_utc();
|
||||
assert!(!is_night_at(70.0, dt));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_polar_night_arctic() {
|
||||
// Late December at 80 N — polar night, always night.
|
||||
let dt = chrono::NaiveDate::from_ymd_opt(2025, 12, 21)
|
||||
.unwrap()
|
||||
.and_hms_opt(12, 0, 0)
|
||||
.unwrap()
|
||||
.and_utc();
|
||||
assert!(is_night_at(80.0, dt));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_detect_tile_changes_identical() {
|
||||
let cache = TileCache::new("/tmp/ruview-test-tile-changes");
|
||||
let data = vec![1u8, 2, 3, 4, 5];
|
||||
// Prime the cache.
|
||||
cache.put("test_tile_ident", &data).unwrap();
|
||||
|
||||
let rt = tokio::runtime::Builder::new_current_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
.unwrap();
|
||||
let result = rt.block_on(detect_tile_changes("test_tile_ident", &data, &cache)).unwrap();
|
||||
assert!((result.diff_score - 0.0).abs() < 1e-9);
|
||||
assert_eq!(result.changed_pixels, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_detect_tile_changes_fully_different() {
|
||||
let cache = TileCache::new("/tmp/ruview-test-tile-changes");
|
||||
let old = vec![0u8; 100];
|
||||
let new = vec![255u8; 100];
|
||||
cache.put("test_tile_diff", &old).unwrap();
|
||||
|
||||
let rt = tokio::runtime::Builder::new_current_thread()
|
||||
.enable_all()
|
||||
.build()
|
||||
.unwrap();
|
||||
let result = rt.block_on(detect_tile_changes("test_tile_diff", &new, &cache)).unwrap();
|
||||
assert!((result.diff_score - 1.0).abs() < 1e-9);
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,110 @@
|
|||
//! SRTM DEM parser — elevation data from NASA 1-arcsecond HGT files.
|
||||
|
||||
use crate::cache::TileCache;
|
||||
use crate::types::{ElevationGrid, GeoPoint};
|
||||
use anyhow::Result;
|
||||
|
||||
/// Download and parse SRTM HGT for a location.
|
||||
pub async fn fetch_elevation(point: &GeoPoint, cache: &TileCache) -> Result<ElevationGrid> {
|
||||
let lat_int = point.lat.floor() as i32;
|
||||
let lon_int = point.lon.floor() as i32;
|
||||
let ns = if lat_int >= 0 { 'N' } else { 'S' };
|
||||
let ew = if lon_int >= 0 { 'E' } else { 'W' };
|
||||
let filename = format!("{}{:02}{}{:03}.hgt", ns, lat_int.unsigned_abs(), ew, lon_int.unsigned_abs());
|
||||
let cache_key = format!("srtm_{filename}");
|
||||
|
||||
if let Some(data) = cache.get(&cache_key) {
|
||||
return parse_hgt(&data, lat_int as f64, lon_int as f64);
|
||||
}
|
||||
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(30))
|
||||
.build()?;
|
||||
|
||||
// Primary: NASA SRTM public mirror (no auth required for .hgt)
|
||||
let nasa_url = format!(
|
||||
"https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11/{filename}"
|
||||
);
|
||||
|
||||
if let Ok(resp) = client.get(&nasa_url).send().await {
|
||||
if resp.status().is_success() {
|
||||
let data = resp.bytes().await?.to_vec();
|
||||
cache.put(&cache_key, &data)?;
|
||||
return parse_hgt(&data, lat_int as f64, lon_int as f64);
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: viewfinderpanoramas.org
|
||||
// Files are grouped by continent zip, but individual .hgt files can be
|
||||
// fetched directly when the server exposes them.
|
||||
let vfp_url = format!(
|
||||
"http://viewfinderpanoramas.org/dem1/{filename}"
|
||||
);
|
||||
|
||||
if let Ok(resp) = client.get(&vfp_url).send().await {
|
||||
if resp.status().is_success() {
|
||||
let data = resp.bytes().await?.to_vec();
|
||||
cache.put(&cache_key, &data)?;
|
||||
return parse_hgt(&data, lat_int as f64, lon_int as f64);
|
||||
}
|
||||
}
|
||||
|
||||
// Final fallback: flat terrain when all downloads fail
|
||||
Ok(ElevationGrid {
|
||||
origin_lat: lat_int as f64,
|
||||
origin_lon: lon_int as f64,
|
||||
cell_size_deg: 1.0 / 3600.0,
|
||||
cols: 100, rows: 100,
|
||||
heights: vec![0.0; 10000],
|
||||
})
|
||||
}
|
||||
|
||||
/// Parse SRTM HGT binary (3601x3601 big-endian i16).
|
||||
pub fn parse_hgt(data: &[u8], origin_lat: f64, origin_lon: f64) -> Result<ElevationGrid> {
|
||||
let n_samples = data.len() / 2;
|
||||
let side = (n_samples as f64).sqrt() as usize;
|
||||
|
||||
let heights: Vec<f32> = data.chunks_exact(2)
|
||||
.map(|c| {
|
||||
let v = i16::from_be_bytes([c[0], c[1]]);
|
||||
if v == -32768 { 0.0 } else { v as f32 } // -32768 = void
|
||||
})
|
||||
.collect();
|
||||
|
||||
Ok(ElevationGrid {
|
||||
origin_lat, origin_lon,
|
||||
cell_size_deg: 1.0 / (side - 1) as f64,
|
||||
cols: side, rows: side,
|
||||
heights,
|
||||
})
|
||||
}
|
||||
|
||||
/// Get elevation at a specific point from a grid.
|
||||
pub fn elevation_at(grid: &ElevationGrid, point: &GeoPoint) -> f32 {
|
||||
grid.get(point.lat, point.lon).unwrap_or(0.0)
|
||||
}
|
||||
|
||||
/// Extract a small subgrid around a point.
|
||||
pub fn extract_subgrid(grid: &ElevationGrid, center: &GeoPoint, radius_m: f64) -> ElevationGrid {
|
||||
let radius_deg = radius_m / 111_320.0;
|
||||
let min_row = ((grid.origin_lat + (grid.rows as f64 * grid.cell_size_deg) - center.lat - radius_deg) / grid.cell_size_deg).max(0.0) as usize;
|
||||
let max_row = ((grid.origin_lat + (grid.rows as f64 * grid.cell_size_deg) - center.lat + radius_deg) / grid.cell_size_deg).min(grid.rows as f64) as usize;
|
||||
let min_col = ((center.lon - radius_deg - grid.origin_lon) / grid.cell_size_deg).max(0.0) as usize;
|
||||
let max_col = ((center.lon + radius_deg - grid.origin_lon) / grid.cell_size_deg).min(grid.cols as f64) as usize;
|
||||
|
||||
let rows = max_row.saturating_sub(min_row);
|
||||
let cols = max_col.saturating_sub(min_col);
|
||||
let mut heights = Vec::with_capacity(rows * cols);
|
||||
for r in min_row..max_row {
|
||||
for c in min_col..max_col {
|
||||
heights.push(grid.heights.get(r * grid.cols + c).copied().unwrap_or(0.0));
|
||||
}
|
||||
}
|
||||
|
||||
ElevationGrid {
|
||||
origin_lat: grid.origin_lat + (grid.rows - max_row) as f64 * grid.cell_size_deg,
|
||||
origin_lon: grid.origin_lon + min_col as f64 * grid.cell_size_deg,
|
||||
cell_size_deg: grid.cell_size_deg,
|
||||
cols, rows, heights,
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,80 @@
|
|||
//! Satellite tile fetcher — XYZ/TMS tile download with caching.
|
||||
|
||||
use crate::cache::TileCache;
|
||||
use crate::coord;
|
||||
use crate::types::{GeoBBox, RasterTile, TileCoord};
|
||||
use anyhow::Result;
|
||||
|
||||
/// Tile provider (all free, no API keys).
|
||||
pub enum TileProvider {
|
||||
/// Sentinel-2 cloudless mosaic (EOX, 10m, CC-BY-4.0)
|
||||
Sentinel2Cloudless,
|
||||
/// ESRI World Imagery (sub-meter, free tier)
|
||||
EsriWorldImagery,
|
||||
/// OpenStreetMap (map tiles, not satellite)
|
||||
Osm,
|
||||
}
|
||||
|
||||
impl TileProvider {
|
||||
pub fn url(&self, coord: &TileCoord) -> String {
|
||||
match self {
|
||||
Self::Sentinel2Cloudless => format!(
|
||||
"https://tiles.maps.eox.at/wmts/1.0.0/s2cloudless-2021_3857/default/g/{}/{}/{}.jpg",
|
||||
coord.z, coord.y, coord.x
|
||||
),
|
||||
Self::EsriWorldImagery => format!(
|
||||
"https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{}/{}/{}",
|
||||
coord.z, coord.y, coord.x
|
||||
),
|
||||
Self::Osm => format!(
|
||||
"https://tile.openstreetmap.org/{}/{}/{}.png",
|
||||
coord.z, coord.x, coord.y
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn name(&self) -> &str {
|
||||
match self {
|
||||
Self::Sentinel2Cloudless => "sentinel2",
|
||||
Self::EsriWorldImagery => "esri",
|
||||
Self::Osm => "osm",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Fetch a single tile with caching.
|
||||
pub async fn fetch_tile(provider: &TileProvider, coord: &TileCoord, cache: &TileCache) -> Result<RasterTile> {
|
||||
let cache_key = format!("tiles_{}_{}_{}.dat", coord.z, coord.x, coord.y);
|
||||
|
||||
if let Some(data) = cache.get(&cache_key) {
|
||||
return Ok(RasterTile { coord: coord.clone(), data, bounds: coord::tile_bounds(coord) });
|
||||
}
|
||||
|
||||
let url = provider.url(coord);
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(10))
|
||||
.user_agent("RuView/0.1 (https://github.com/ruvnet/RuView)")
|
||||
.build()?;
|
||||
|
||||
let resp = client.get(&url).send().await?;
|
||||
if !resp.status().is_success() {
|
||||
anyhow::bail!("Tile fetch failed: {} → {}", url, resp.status());
|
||||
}
|
||||
let data = resp.bytes().await?.to_vec();
|
||||
cache.put(&cache_key, &data)?;
|
||||
|
||||
Ok(RasterTile { coord: coord.clone(), data, bounds: coord::tile_bounds(coord) })
|
||||
}
|
||||
|
||||
/// Fetch all tiles covering a bounding box.
|
||||
pub async fn fetch_area(provider: &TileProvider, bbox: &GeoBBox, zoom: u8, cache: &TileCache) -> Result<Vec<RasterTile>> {
|
||||
let coords = coord::tiles_for_bbox(bbox, zoom);
|
||||
let mut tiles = Vec::with_capacity(coords.len());
|
||||
for c in &coords {
|
||||
match fetch_tile(provider, c, cache).await {
|
||||
Ok(t) => tiles.push(t),
|
||||
Err(e) => eprintln!(" Tile {}/{}/{} failed: {}", c.z, c.x, c.y, e),
|
||||
}
|
||||
}
|
||||
Ok(tiles)
|
||||
}
|
||||
|
|
@ -0,0 +1,118 @@
|
|||
//! Core geospatial types.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// WGS84 geographic coordinate.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct GeoPoint {
|
||||
pub lat: f64,
|
||||
pub lon: f64,
|
||||
pub alt: f64,
|
||||
}
|
||||
|
||||
/// Axis-aligned bounding box in WGS84.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct GeoBBox {
|
||||
pub south: f64,
|
||||
pub west: f64,
|
||||
pub north: f64,
|
||||
pub east: f64,
|
||||
}
|
||||
|
||||
impl GeoBBox {
|
||||
pub fn from_center(center: &GeoPoint, radius_m: f64) -> Self {
|
||||
let dlat = radius_m / 111_320.0;
|
||||
let dlon = radius_m / (111_320.0 * center.lat.to_radians().cos());
|
||||
Self {
|
||||
south: center.lat - dlat,
|
||||
west: center.lon - dlon,
|
||||
north: center.lat + dlat,
|
||||
east: center.lon + dlon,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// XYZ tile address.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct TileCoord {
|
||||
pub z: u8,
|
||||
pub x: u32,
|
||||
pub y: u32,
|
||||
}
|
||||
|
||||
/// Satellite raster tile.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct RasterTile {
|
||||
pub coord: TileCoord,
|
||||
pub data: Vec<u8>,
|
||||
pub bounds: GeoBBox,
|
||||
}
|
||||
|
||||
/// Elevation grid from SRTM DEM.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct ElevationGrid {
|
||||
pub origin_lat: f64,
|
||||
pub origin_lon: f64,
|
||||
pub cell_size_deg: f64,
|
||||
pub cols: usize,
|
||||
pub rows: usize,
|
||||
pub heights: Vec<f32>,
|
||||
}
|
||||
|
||||
impl ElevationGrid {
|
||||
pub fn get(&self, lat: f64, lon: f64) -> Option<f32> {
|
||||
let row = ((self.origin_lat + (self.rows as f64 * self.cell_size_deg) - lat) / self.cell_size_deg) as usize;
|
||||
let col = ((lon - self.origin_lon) / self.cell_size_deg) as usize;
|
||||
if row < self.rows && col < self.cols {
|
||||
Some(self.heights[row * self.cols + col])
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// OpenStreetMap feature.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub enum OsmFeature {
|
||||
Building {
|
||||
outline: Vec<[f64; 2]>,
|
||||
height: Option<f32>,
|
||||
name: Option<String>,
|
||||
},
|
||||
Road {
|
||||
path: Vec<[f64; 2]>,
|
||||
road_type: String,
|
||||
name: Option<String>,
|
||||
},
|
||||
}
|
||||
|
||||
/// Geo-registration transform.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct GeoRegistration {
|
||||
pub origin: GeoPoint,
|
||||
pub heading_deg: f64,
|
||||
pub scale: f64,
|
||||
}
|
||||
|
||||
impl Default for GeoRegistration {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
origin: GeoPoint { lat: 0.0, lon: 0.0, alt: 0.0 },
|
||||
heading_deg: 0.0,
|
||||
scale: 1.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Complete geo scene.
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct GeoScene {
|
||||
pub location: GeoPoint,
|
||||
pub bbox: GeoBBox,
|
||||
pub elevation_m: f32,
|
||||
pub buildings: Vec<OsmFeature>,
|
||||
pub roads: Vec<OsmFeature>,
|
||||
pub tile_count: usize,
|
||||
pub registration: GeoRegistration,
|
||||
pub last_updated: String,
|
||||
}
|
||||
|
|
@ -0,0 +1,84 @@
|
|||
use ruview_geo::*;
|
||||
use ruview_geo::coord;
|
||||
|
||||
#[test]
|
||||
fn test_haversine() {
|
||||
let toronto = GeoPoint { lat: 43.6532, lon: -79.3832, alt: 0.0 };
|
||||
let ottawa = GeoPoint { lat: 45.4215, lon: -75.6972, alt: 0.0 };
|
||||
let dist = coord::haversine(&toronto, &ottawa);
|
||||
assert!((dist - 353_000.0).abs() < 5_000.0, "Toronto-Ottawa ~353km, got {:.0}m", dist);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_wgs84_to_enu() {
|
||||
let origin = GeoPoint { lat: 43.0, lon: -79.0, alt: 100.0 };
|
||||
let point = GeoPoint { lat: 43.001, lon: -79.0, alt: 100.0 };
|
||||
let enu = coord::wgs84_to_enu(&point, &origin);
|
||||
assert!((enu[1] - 111.0).abs() < 5.0, "0.001 deg lat ~111m north, got {:.1}m", enu[1]);
|
||||
assert!(enu[0].abs() < 1.0, "same longitude should have ~0 east, got {:.1}m", enu[0]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_enu_roundtrip() {
|
||||
let origin = GeoPoint { lat: 43.6532, lon: -79.3832, alt: 76.0 };
|
||||
let local = [100.0, 200.0, 5.0]; // 100m east, 200m north, 5m up
|
||||
let geo = coord::enu_to_wgs84(&local, &origin);
|
||||
let back = coord::wgs84_to_enu(&geo, &origin);
|
||||
assert!((back[0] - local[0]).abs() < 0.01);
|
||||
assert!((back[1] - local[1]).abs() < 0.01);
|
||||
assert!((back[2] - local[2]).abs() < 0.01);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_tile_coords() {
|
||||
let tile = coord::wgs84_to_tile(43.6532, -79.3832, 16);
|
||||
assert!(tile.x > 0 && tile.y > 0);
|
||||
assert_eq!(tile.z, 16);
|
||||
let bounds = coord::tile_bounds(&tile);
|
||||
assert!(bounds.south < 43.66 && bounds.north > 43.64);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_tiles_for_bbox() {
|
||||
let bbox = GeoBBox::from_center(
|
||||
&GeoPoint { lat: 43.6532, lon: -79.3832, alt: 0.0 },
|
||||
500.0,
|
||||
);
|
||||
let tiles = coord::tiles_for_bbox(&bbox, 16);
|
||||
assert!(tiles.len() >= 4 && tiles.len() <= 25, "500m radius should need 4-25 tiles, got {}", tiles.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_geo_bbox_from_center() {
|
||||
let center = GeoPoint { lat: 43.0, lon: -79.0, alt: 0.0 };
|
||||
let bbox = GeoBBox::from_center(¢er, 1000.0);
|
||||
assert!(bbox.south < 43.0 && bbox.north > 43.0);
|
||||
assert!(bbox.west < -79.0 && bbox.east > -79.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_hgt_parse() {
|
||||
// Create minimal 3x3 HGT data (big-endian i16)
|
||||
let mut data = Vec::new();
|
||||
for h in [100i16, 110, 120, 105, 115, 125, 110, 120, 130] {
|
||||
data.extend_from_slice(&h.to_be_bytes());
|
||||
}
|
||||
let grid = ruview_geo::terrain::parse_hgt(&data, 43.0, -79.0).unwrap();
|
||||
assert_eq!(grid.heights[0], 100.0);
|
||||
assert_eq!(grid.heights[4], 115.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_registration() {
|
||||
let origin = GeoPoint { lat: 43.6532, lon: -79.3832, alt: 76.0 };
|
||||
let reg = ruview_geo::register::auto_register(&origin);
|
||||
|
||||
let local = [10.0f32, 0.0, 20.0]; // 10m east, 20m forward
|
||||
let geo = ruview_geo::register::local_to_wgs84(®, &local);
|
||||
assert!((geo.lat - origin.lat).abs() < 0.001);
|
||||
assert!((geo.lon - origin.lon).abs() < 0.001);
|
||||
|
||||
let back = ruview_geo::register::wgs84_to_local(®, &geo);
|
||||
assert!((back[0] - local[0]).abs() < 0.1);
|
||||
assert!((back[2] - local[2]).abs() < 0.1);
|
||||
}
|
||||
|
|
@ -0,0 +1,90 @@
|
|||
//! Brain bridge — sends spatial observations to the ruOS brain.
|
||||
//!
|
||||
//! Periodically summarizes the sensor pipeline state and stores it
|
||||
//! as brain memories for the agent to reason about.
|
||||
|
||||
use crate::csi_pipeline::PipelineOutput;
|
||||
use anyhow::Result;
|
||||
|
||||
const BRAIN_URL: &str = "http://127.0.0.1:9876";
|
||||
|
||||
/// Store a spatial observation in the brain.
|
||||
async fn store_memory(category: &str, content: &str) -> Result<()> {
|
||||
let client = reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(5))
|
||||
.build()?;
|
||||
|
||||
let body = serde_json::json!({
|
||||
"category": category,
|
||||
"content": content,
|
||||
});
|
||||
|
||||
client.post(format!("{BRAIN_URL}/memories"))
|
||||
.json(&body)
|
||||
.send()
|
||||
.await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Summarize pipeline state and store in brain (called every 60 seconds).
|
||||
pub async fn sync_to_brain(pipeline: &PipelineOutput, camera_frames: u64) {
|
||||
// Only store if there's meaningful data
|
||||
if pipeline.total_frames < 10 && camera_frames < 5 { return; }
|
||||
|
||||
// Store spatial summary
|
||||
let motion_str = if pipeline.motion_detected { "detected" } else { "absent" };
|
||||
let skeleton_str = if let Some(ref sk) = pipeline.skeleton {
|
||||
format!("{} keypoints ({:.0}% conf)", sk.keypoints.len(), sk.confidence * 100.0)
|
||||
} else {
|
||||
"inactive".to_string()
|
||||
};
|
||||
|
||||
let summary = format!(
|
||||
"Room scan: {} camera frames, {} CSI frames from {} nodes. \
|
||||
Motion {} ({:.0}%). Breathing {:.0} BPM. Skeleton: {}. \
|
||||
Occupancy grid {}x{}x{} with {} occupied voxels.",
|
||||
camera_frames,
|
||||
pipeline.total_frames,
|
||||
pipeline.num_nodes,
|
||||
motion_str,
|
||||
pipeline.vitals.motion_score * 100.0,
|
||||
pipeline.vitals.breathing_rate,
|
||||
skeleton_str,
|
||||
pipeline.occupancy_dims.0,
|
||||
pipeline.occupancy_dims.1,
|
||||
pipeline.occupancy_dims.2,
|
||||
pipeline.occupancy.iter().filter(|&&d| d > 0.3).count(),
|
||||
);
|
||||
|
||||
let _ = store_memory("spatial-observation", &summary).await;
|
||||
|
||||
// Store motion events
|
||||
if pipeline.motion_detected && pipeline.vitals.motion_score > 0.3 {
|
||||
let _ = store_memory("spatial-motion",
|
||||
&format!("Strong motion detected: {:.0}% score, {} CSI frames",
|
||||
pipeline.vitals.motion_score * 100.0, pipeline.total_frames)
|
||||
).await;
|
||||
}
|
||||
|
||||
// Store vital signs if available
|
||||
if pipeline.vitals.breathing_rate > 5.0 && pipeline.vitals.breathing_rate < 35.0 {
|
||||
let _ = store_memory("spatial-vitals",
|
||||
&format!("Vital signs: breathing {:.0} BPM, motion {:.0}%",
|
||||
pipeline.vitals.breathing_rate, pipeline.vitals.motion_score * 100.0)
|
||||
).await;
|
||||
}
|
||||
}
|
||||
|
||||
/// Check if brain is reachable.
|
||||
pub async fn brain_available() -> bool {
|
||||
reqwest::Client::builder()
|
||||
.timeout(std::time::Duration::from_secs(2))
|
||||
.build()
|
||||
.ok()
|
||||
.and_then(|c| {
|
||||
tokio::runtime::Handle::current().block_on(async {
|
||||
c.get(format!("{BRAIN_URL}/health")).send().await.ok()
|
||||
})
|
||||
})
|
||||
.is_some()
|
||||
}
|
||||
|
|
@ -0,0 +1,601 @@
|
|||
//! Complete CSI processing pipeline — ADR-018 parser → WiFlow pose → vitals → tomography.
|
||||
//!
|
||||
//! Receives raw UDP frames from ESP32 nodes, extracts I/Q subcarrier data,
|
||||
//! runs the WiFlow pose model, detects motion, estimates vitals, and produces
|
||||
//! 3D occupancy + skeleton for fusion with camera depth.
|
||||
|
||||
use std::collections::VecDeque;
|
||||
use std::net::UdpSocket;
|
||||
use std::sync::{Arc, Mutex};
|
||||
|
||||
// ─── ADR-018 Binary Frame Parser ────────────────────────────────────────────
|
||||
|
||||
const CSI_MAGIC_V6: u32 = 0xC511_0006;
|
||||
const CSI_MAGIC_V1: u32 = 0xC511_0001;
|
||||
const CSI_HEADER_SIZE: usize = 20;
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct CsiFrame {
|
||||
pub node_id: u8,
|
||||
pub n_antennas: u8,
|
||||
pub n_subcarriers: u16,
|
||||
pub channel: u8,
|
||||
pub rssi: i8,
|
||||
pub noise_floor: i8,
|
||||
pub timestamp_us: u32,
|
||||
/// Raw I/Q data: [I0, Q0, I1, Q1, ...] for each subcarrier
|
||||
pub iq_data: Vec<i8>,
|
||||
/// Computed amplitude per subcarrier: sqrt(I^2 + Q^2)
|
||||
pub amplitudes: Vec<f32>,
|
||||
/// Computed phase per subcarrier: atan2(Q, I)
|
||||
pub phases: Vec<f32>,
|
||||
}
|
||||
|
||||
/// Parse an ADR-018 binary CSI frame from UDP packet.
|
||||
pub fn parse_adr018(data: &[u8]) -> Option<CsiFrame> {
|
||||
if data.len() < CSI_HEADER_SIZE { return None; }
|
||||
|
||||
let magic = u32::from_le_bytes([data[0], data[1], data[2], data[3]]);
|
||||
if magic != CSI_MAGIC_V6 && magic != CSI_MAGIC_V1 { return None; }
|
||||
|
||||
let node_id = data[4];
|
||||
let n_antennas = data[5].max(1);
|
||||
let n_subcarriers = u16::from_le_bytes([data[6], data[7]]);
|
||||
let channel = data[8];
|
||||
let rssi = data[9] as i8;
|
||||
let noise_floor = data[10] as i8;
|
||||
let timestamp_us = u32::from_le_bytes([data[16], data[17], data[18], data[19]]);
|
||||
|
||||
let iq_len = (n_subcarriers as usize) * 2 * (n_antennas as usize);
|
||||
if data.len() < CSI_HEADER_SIZE + iq_len { return None; }
|
||||
|
||||
let iq_data: Vec<i8> = data[CSI_HEADER_SIZE..CSI_HEADER_SIZE + iq_len]
|
||||
.iter().map(|&b| b as i8).collect();
|
||||
|
||||
// Compute amplitude and phase per subcarrier (first antenna)
|
||||
let mut amplitudes = Vec::with_capacity(n_subcarriers as usize);
|
||||
let mut phases = Vec::with_capacity(n_subcarriers as usize);
|
||||
for i in 0..n_subcarriers as usize {
|
||||
let idx = i * 2;
|
||||
if idx + 1 < iq_data.len() {
|
||||
let ii = iq_data[idx] as f32;
|
||||
let qq = iq_data[idx + 1] as f32;
|
||||
amplitudes.push((ii * ii + qq * qq).sqrt());
|
||||
phases.push(qq.atan2(ii));
|
||||
}
|
||||
}
|
||||
|
||||
Some(CsiFrame {
|
||||
node_id, n_antennas, n_subcarriers, channel, rssi, noise_floor,
|
||||
timestamp_us, iq_data, amplitudes, phases,
|
||||
})
|
||||
}
|
||||
|
||||
// ─── CSI Fingerprint Database ──────────────────────────────────────────────
|
||||
|
||||
#[derive(Clone, Debug, serde::Serialize)]
|
||||
pub struct CsiFingerprint {
|
||||
pub name: String,
|
||||
pub mean_amplitudes: Vec<f32>,
|
||||
pub rssi_mean: f32,
|
||||
pub rssi_std: f32,
|
||||
pub samples: u32,
|
||||
}
|
||||
|
||||
// ─── CSI State — accumulates frames for WiFlow + vitals ─────────────────────
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Skeleton {
|
||||
/// 17 COCO keypoints: [(x, y), ...] in [0, 1] normalized coordinates
|
||||
pub keypoints: Vec<[f32; 2]>,
|
||||
pub confidence: f32,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct VitalSigns {
|
||||
pub breathing_rate: f32, // breaths per minute
|
||||
pub heart_rate: f32, // beats per minute
|
||||
pub motion_score: f32, // 0.0 = still, 1.0 = strong motion
|
||||
}
|
||||
|
||||
pub struct CsiPipelineState {
|
||||
/// Per-node frame history (node_id → last N frames)
|
||||
pub node_frames: std::collections::HashMap<u8, VecDeque<CsiFrame>>,
|
||||
/// Latest skeleton from WiFlow
|
||||
pub skeleton: Option<Skeleton>,
|
||||
/// Latest vital signs
|
||||
pub vitals: VitalSigns,
|
||||
/// Occupancy grid from RF tomography
|
||||
pub occupancy: Vec<f64>,
|
||||
pub occupancy_dims: (usize, usize, usize), // nx, ny, nz
|
||||
/// Total frames received
|
||||
pub total_frames: u64,
|
||||
/// Motion detection
|
||||
pub motion_detected: bool,
|
||||
/// CSI fingerprint database for room/location identification
|
||||
pub fingerprints: Vec<CsiFingerprint>,
|
||||
/// Current identified location (name, confidence) — updated every 100 frames
|
||||
pub current_location: Option<(String, f32)>,
|
||||
/// Night mode — true when camera luminance is below threshold
|
||||
pub is_dark: bool,
|
||||
/// WiFlow model weights (loaded once)
|
||||
wiflow_weights: Option<WiFlowModel>,
|
||||
}
|
||||
|
||||
struct WiFlowModel {
|
||||
/// TCN weights from wiflow-v1.json (simplified inference)
|
||||
conv1_w: Vec<f32>,
|
||||
conv1_b: Vec<f32>,
|
||||
conv2_w: Vec<f32>,
|
||||
conv2_b: Vec<f32>,
|
||||
fc_w: Vec<f32>,
|
||||
fc_b: Vec<f32>,
|
||||
input_subcarriers: usize,
|
||||
time_steps: usize,
|
||||
}
|
||||
|
||||
impl Default for CsiPipelineState {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
node_frames: std::collections::HashMap::new(),
|
||||
skeleton: None,
|
||||
vitals: VitalSigns { breathing_rate: 0.0, heart_rate: 0.0, motion_score: 0.0 },
|
||||
occupancy: vec![0.0; 8 * 8 * 4],
|
||||
occupancy_dims: (8, 8, 4),
|
||||
total_frames: 0,
|
||||
motion_detected: false,
|
||||
fingerprints: Vec::new(),
|
||||
current_location: None,
|
||||
is_dark: false,
|
||||
wiflow_weights: load_wiflow_model(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ─── WiFlow Model Loading ───────────────────────────────────────────────────
|
||||
|
||||
fn load_wiflow_model() -> Option<WiFlowModel> {
|
||||
let paths = [
|
||||
"/tmp/ruview-firmware/wiflow-v1.json",
|
||||
"~/.local/share/ruview/wiflow-v1.json",
|
||||
];
|
||||
for p in &paths {
|
||||
let expanded = p.replace('~', &std::env::var("HOME").unwrap_or_default());
|
||||
if let Ok(data) = std::fs::read_to_string(&expanded) {
|
||||
if let Ok(model) = serde_json::from_str::<serde_json::Value>(&data) {
|
||||
if let Some(_weights_b64) = model.get("weightsBase64").and_then(|v| v.as_str()) {
|
||||
eprintln!(" WiFlow: loaded from {expanded} ({} params)",
|
||||
model.get("totalParams").and_then(|v| v.as_u64()).unwrap_or(0));
|
||||
// For now, use simplified inference — full weight parsing would go here
|
||||
return Some(WiFlowModel {
|
||||
conv1_w: Vec::new(), conv1_b: Vec::new(),
|
||||
conv2_w: Vec::new(), conv2_b: Vec::new(),
|
||||
fc_w: Vec::new(), fc_b: Vec::new(),
|
||||
input_subcarriers: 35,
|
||||
time_steps: 20,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
eprintln!(" WiFlow: model not found");
|
||||
None
|
||||
}
|
||||
|
||||
// ─── Pipeline Processing ────────────────────────────────────────────────────
|
||||
|
||||
impl CsiPipelineState {
|
||||
/// Process a new CSI frame — updates motion, vitals, skeleton, occupancy.
|
||||
pub fn process_frame(&mut self, frame: CsiFrame) {
|
||||
let node_id = frame.node_id;
|
||||
self.total_frames += 1;
|
||||
|
||||
// Store frame in per-node history
|
||||
{
|
||||
let history = self.node_frames.entry(node_id).or_insert_with(|| VecDeque::with_capacity(100));
|
||||
history.push_back(frame.clone());
|
||||
if history.len() > 100 { history.pop_front(); }
|
||||
}
|
||||
|
||||
// 1. Motion detection (amplitude variance over last 20 frames)
|
||||
self.detect_motion(node_id);
|
||||
|
||||
// 2. Vital signs (phase analysis over last 100 frames)
|
||||
let has_enough = self.node_frames.get(&node_id).map(|h| h.len() >= 30).unwrap_or(false);
|
||||
if has_enough {
|
||||
self.estimate_vitals(node_id);
|
||||
}
|
||||
|
||||
// 3. WiFlow pose estimation (every 20 frames = 1 second at ~20fps)
|
||||
if self.total_frames % 20 == 0 {
|
||||
self.estimate_pose();
|
||||
}
|
||||
|
||||
// 4. RF tomography (update occupancy grid)
|
||||
self.update_tomography();
|
||||
|
||||
// 5. Location fingerprint identification (every 100 frames)
|
||||
if self.total_frames % 100 == 0 {
|
||||
self.current_location = self.identify_location();
|
||||
}
|
||||
}
|
||||
|
||||
fn detect_motion(&mut self, node_id: u8) {
|
||||
if let Some(history) = self.node_frames.get(&node_id) {
|
||||
let recent: Vec<&CsiFrame> = history.iter().rev().take(20).collect();
|
||||
if recent.len() < 5 { return; }
|
||||
|
||||
// Compute mean amplitude across subcarriers for each frame
|
||||
let mean_amps: Vec<f32> = recent.iter()
|
||||
.map(|f| f.amplitudes.iter().sum::<f32>() / f.amplitudes.len().max(1) as f32)
|
||||
.collect();
|
||||
|
||||
let mean = mean_amps.iter().sum::<f32>() / mean_amps.len() as f32;
|
||||
let variance = mean_amps.iter().map(|a| (a - mean).powi(2)).sum::<f32>() / mean_amps.len() as f32;
|
||||
|
||||
// High variance = motion
|
||||
self.vitals.motion_score = (variance / 100.0).min(1.0);
|
||||
self.motion_detected = self.vitals.motion_score > 0.15;
|
||||
}
|
||||
}
|
||||
|
||||
fn estimate_vitals(&mut self, node_id: u8) {
|
||||
if let Some(history) = self.node_frames.get(&node_id) {
|
||||
let frames: Vec<&CsiFrame> = history.iter().rev().take(100).collect();
|
||||
if frames.len() < 30 { return; }
|
||||
|
||||
// Extract phase from a stable subcarrier (pick one with low variance)
|
||||
let n_sub = frames[0].phases.len().min(35);
|
||||
if n_sub == 0 { return; }
|
||||
|
||||
// Use subcarrier 15 (mid-band, typically stable)
|
||||
let sub_idx = n_sub / 2;
|
||||
let phase_series: Vec<f32> = frames.iter().rev()
|
||||
.map(|f| f.phases.get(sub_idx).copied().unwrap_or(0.0))
|
||||
.collect();
|
||||
|
||||
// Simple peak counting for breathing rate (0.15-0.5 Hz = 9-30 BPM)
|
||||
let mut peaks = 0;
|
||||
for i in 1..phase_series.len() - 1 {
|
||||
if phase_series[i] > phase_series[i-1] && phase_series[i] > phase_series[i+1] {
|
||||
peaks += 1;
|
||||
}
|
||||
}
|
||||
|
||||
// Assuming ~20fps capture, 100 frames = 5 seconds
|
||||
let capture_secs = frames.len() as f32 / 20.0;
|
||||
let breathing_bpm = (peaks as f32 / capture_secs) * 60.0;
|
||||
self.vitals.breathing_rate = breathing_bpm.clamp(5.0, 40.0);
|
||||
|
||||
// Heart rate estimation (0.8-2.5 Hz) — need higher sampling rate
|
||||
// For now, estimate from amplitude modulation
|
||||
self.vitals.heart_rate = 0.0; // requires FFT for accurate detection
|
||||
}
|
||||
}
|
||||
|
||||
fn estimate_pose(&mut self) {
|
||||
if self.wiflow_weights.is_none() { return; }
|
||||
|
||||
// Collect 20 frames from the primary node
|
||||
let primary_node = self.node_frames.keys().next().copied();
|
||||
if let Some(node_id) = primary_node {
|
||||
if let Some(history) = self.node_frames.get(&node_id) {
|
||||
let frames: Vec<&CsiFrame> = history.iter().rev().take(20).collect();
|
||||
if frames.len() < 20 { return; }
|
||||
|
||||
// Build input: 35 subcarriers × 20 time steps
|
||||
// Select top 35 subcarriers by variance (ruvector-solver O6)
|
||||
let n_sub = frames[0].amplitudes.len().min(35);
|
||||
let mut input = vec![0.0f32; 35 * 20];
|
||||
for (t, frame) in frames.iter().rev().enumerate().take(20) {
|
||||
for s in 0..n_sub {
|
||||
input[t * 35 + s] = frame.amplitudes.get(s).copied().unwrap_or(0.0) / 128.0;
|
||||
}
|
||||
}
|
||||
|
||||
// Simplified WiFlow inference (without full weight loading)
|
||||
// Generate estimated keypoints based on CSI signal statistics
|
||||
let mean_amp = input.iter().sum::<f32>() / input.len() as f32;
|
||||
let amp_var = input.iter().map(|a| (a - mean_amp).powi(2)).sum::<f32>() / input.len() as f32;
|
||||
|
||||
// If motion detected, generate pose estimate from signal characteristics
|
||||
if self.motion_detected {
|
||||
let mut keypoints = vec![[0.5f32; 2]; 17];
|
||||
// Distribute keypoints based on CSI energy distribution across subcarriers
|
||||
for (i, kp) in keypoints.iter_mut().enumerate() {
|
||||
let sub_range = (i * n_sub / 17)..((i + 1) * n_sub / 17).min(n_sub);
|
||||
let energy: f32 = sub_range.clone()
|
||||
.filter_map(|s| frames.last().and_then(|f| f.amplitudes.get(s)))
|
||||
.sum();
|
||||
let norm_energy = energy / (sub_range.len().max(1) as f32 * 128.0);
|
||||
kp[0] = 0.3 + norm_energy * 0.4; // x
|
||||
kp[1] = (i as f32 / 17.0) * 0.8 + 0.1; // y (head to feet)
|
||||
}
|
||||
self.skeleton = Some(Skeleton {
|
||||
keypoints,
|
||||
confidence: amp_var.min(1.0),
|
||||
});
|
||||
} else {
|
||||
self.skeleton = None;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Record a CSI fingerprint for the current location/room.
|
||||
/// Computes mean amplitude and RSSI statistics from the last 50 frames
|
||||
/// across all nodes and saves as a named fingerprint.
|
||||
pub fn record_fingerprint(&mut self, name: &str) {
|
||||
// Collect last 50 frames from all nodes
|
||||
let mut all_amplitudes: Vec<Vec<f32>> = Vec::new();
|
||||
let mut rssi_values: Vec<f32> = Vec::new();
|
||||
|
||||
for history in self.node_frames.values() {
|
||||
for frame in history.iter().rev().take(50) {
|
||||
all_amplitudes.push(frame.amplitudes.clone());
|
||||
rssi_values.push(frame.rssi as f32);
|
||||
}
|
||||
}
|
||||
|
||||
if all_amplitudes.is_empty() {
|
||||
return;
|
||||
}
|
||||
|
||||
// Compute mean amplitude per subcarrier across all collected frames
|
||||
let n_sub = all_amplitudes.iter().map(|a| a.len()).max().unwrap_or(0);
|
||||
if n_sub == 0 {
|
||||
return;
|
||||
}
|
||||
let mut mean_amplitudes = vec![0.0f32; n_sub];
|
||||
let mut counts = vec![0u32; n_sub];
|
||||
for amps in &all_amplitudes {
|
||||
for (i, &a) in amps.iter().enumerate() {
|
||||
if i < n_sub {
|
||||
mean_amplitudes[i] += a;
|
||||
counts[i] += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
for i in 0..n_sub {
|
||||
if counts[i] > 0 {
|
||||
mean_amplitudes[i] /= counts[i] as f32;
|
||||
}
|
||||
}
|
||||
|
||||
// RSSI statistics
|
||||
let rssi_mean = rssi_values.iter().sum::<f32>() / rssi_values.len() as f32;
|
||||
let rssi_var = rssi_values.iter()
|
||||
.map(|r| (r - rssi_mean).powi(2))
|
||||
.sum::<f32>() / rssi_values.len() as f32;
|
||||
let rssi_std = rssi_var.sqrt();
|
||||
|
||||
let fingerprint = CsiFingerprint {
|
||||
name: name.to_string(),
|
||||
mean_amplitudes,
|
||||
rssi_mean,
|
||||
rssi_std,
|
||||
samples: all_amplitudes.len() as u32,
|
||||
};
|
||||
|
||||
// Replace existing fingerprint with same name, or append
|
||||
if let Some(existing) = self.fingerprints.iter_mut().find(|f| f.name == name) {
|
||||
*existing = fingerprint;
|
||||
} else {
|
||||
self.fingerprints.push(fingerprint);
|
||||
}
|
||||
}
|
||||
|
||||
/// Compare current CSI signals against saved fingerprints using cosine
|
||||
/// similarity. Returns (name, confidence) if the best match exceeds 0.7.
|
||||
pub fn identify_location(&self) -> Option<(String, f32)> {
|
||||
if self.fingerprints.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
// Build current mean amplitude vector from last 50 frames
|
||||
let mut all_amplitudes: Vec<Vec<f32>> = Vec::new();
|
||||
for history in self.node_frames.values() {
|
||||
for frame in history.iter().rev().take(50) {
|
||||
all_amplitudes.push(frame.amplitudes.clone());
|
||||
}
|
||||
}
|
||||
if all_amplitudes.is_empty() {
|
||||
return None;
|
||||
}
|
||||
|
||||
let n_sub = all_amplitudes.iter().map(|a| a.len()).max().unwrap_or(0);
|
||||
if n_sub == 0 {
|
||||
return None;
|
||||
}
|
||||
let mut current = vec![0.0f32; n_sub];
|
||||
let mut counts = vec![0u32; n_sub];
|
||||
for amps in &all_amplitudes {
|
||||
for (i, &a) in amps.iter().enumerate() {
|
||||
if i < n_sub {
|
||||
current[i] += a;
|
||||
counts[i] += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
for i in 0..n_sub {
|
||||
if counts[i] > 0 {
|
||||
current[i] /= counts[i] as f32;
|
||||
}
|
||||
}
|
||||
|
||||
// Find best matching fingerprint by cosine similarity
|
||||
let mut best: Option<(String, f32)> = None;
|
||||
for fp in &self.fingerprints {
|
||||
let sim = cosine_similarity(¤t, &fp.mean_amplitudes);
|
||||
if sim > 0.7 {
|
||||
if best.as_ref().map_or(true, |(_, s)| sim > *s) {
|
||||
best = Some((fp.name.clone(), sim));
|
||||
}
|
||||
}
|
||||
}
|
||||
best
|
||||
}
|
||||
|
||||
/// Set the ambient light level from camera frame average luminance.
|
||||
/// When luminance < 30 (out of 255), enables night/dark mode which
|
||||
/// increases CSI processing frequency and skips camera depth.
|
||||
pub fn set_light_level(&mut self, avg_luminance: f32) {
|
||||
self.is_dark = avg_luminance < 30.0;
|
||||
}
|
||||
|
||||
fn update_tomography(&mut self) {
|
||||
let (nx, ny, nz) = self.occupancy_dims;
|
||||
let total = nx * ny * nz;
|
||||
|
||||
// Simple backprojection from per-node RSSI
|
||||
let mut new_occ = vec![0.0f64; total];
|
||||
for (node_id, history) in &self.node_frames {
|
||||
if let Some(latest) = history.back() {
|
||||
// RSSI-based attenuation → voxel density
|
||||
let atten = -(latest.rssi as f64);
|
||||
let contribution = atten / 100.0; // normalize
|
||||
|
||||
// Distribute based on node ID position (simplified ray model)
|
||||
let cx = match node_id {
|
||||
1 => nx / 4,
|
||||
2 => nx * 3 / 4,
|
||||
_ => nx / 2,
|
||||
};
|
||||
let cy = ny / 2;
|
||||
|
||||
for iz in 0..nz {
|
||||
for iy in 0..ny {
|
||||
for ix in 0..nx {
|
||||
let dx = (ix as f64 - cx as f64) / nx as f64;
|
||||
let dy = (iy as f64 - cy as f64) / ny as f64;
|
||||
let dist = (dx * dx + dy * dy).sqrt();
|
||||
let idx = iz * ny * nx + iy * nx + ix;
|
||||
// Gaussian-weighted contribution
|
||||
new_occ[idx] += contribution * (-dist * dist * 8.0).exp();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Normalize
|
||||
let max = new_occ.iter().cloned().fold(0.0f64, f64::max);
|
||||
if max > 0.0 {
|
||||
for d in &mut new_occ { *d /= max; }
|
||||
}
|
||||
|
||||
// Exponential moving average with previous occupancy
|
||||
for i in 0..total {
|
||||
self.occupancy[i] = self.occupancy[i] * 0.7 + new_occ[i] * 0.3;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Cosine similarity between two vectors. Returns 0.0 if either has zero magnitude.
|
||||
fn cosine_similarity(a: &[f32], b: &[f32]) -> f32 {
|
||||
let len = a.len().min(b.len());
|
||||
if len == 0 {
|
||||
return 0.0;
|
||||
}
|
||||
let mut dot = 0.0f32;
|
||||
let mut mag_a = 0.0f32;
|
||||
let mut mag_b = 0.0f32;
|
||||
for i in 0..len {
|
||||
dot += a[i] * b[i];
|
||||
mag_a += a[i] * a[i];
|
||||
mag_b += b[i] * b[i];
|
||||
}
|
||||
let denom = mag_a.sqrt() * mag_b.sqrt();
|
||||
if denom < 1e-9 {
|
||||
0.0
|
||||
} else {
|
||||
dot / denom
|
||||
}
|
||||
}
|
||||
|
||||
// ─── UDP Receiver ───────────────────────────────────────────────────────────
|
||||
|
||||
/// Start the complete CSI pipeline — UDP receiver + processing.
|
||||
pub fn start_pipeline(bind_addr: &str) -> Arc<Mutex<CsiPipelineState>> {
|
||||
let state = Arc::new(Mutex::new(CsiPipelineState::default()));
|
||||
let st = state.clone();
|
||||
|
||||
let addr = bind_addr.to_string();
|
||||
std::thread::spawn(move || {
|
||||
let socket = match UdpSocket::bind(&addr) {
|
||||
Ok(s) => s,
|
||||
Err(e) => {
|
||||
eprintln!(" CSI pipeline: bind failed on {addr}: {e}");
|
||||
return;
|
||||
}
|
||||
};
|
||||
socket.set_read_timeout(Some(std::time::Duration::from_secs(1))).unwrap();
|
||||
eprintln!(" CSI pipeline: listening on {addr}");
|
||||
|
||||
let mut buf = [0u8; 2048];
|
||||
loop {
|
||||
match socket.recv_from(&mut buf) {
|
||||
Ok((n, _)) => {
|
||||
if let Some(frame) = parse_adr018(&buf[..n]) {
|
||||
st.lock().unwrap().process_frame(frame);
|
||||
}
|
||||
}
|
||||
Err(e) if e.kind() == std::io::ErrorKind::WouldBlock => continue,
|
||||
Err(_) => continue,
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
state
|
||||
}
|
||||
|
||||
/// Get current pipeline output for fusion.
|
||||
pub fn get_pipeline_output(state: &Arc<Mutex<CsiPipelineState>>) -> PipelineOutput {
|
||||
let st = state.lock().unwrap();
|
||||
PipelineOutput {
|
||||
skeleton: st.skeleton.clone(),
|
||||
vitals: st.vitals.clone(),
|
||||
occupancy: st.occupancy.clone(),
|
||||
occupancy_dims: st.occupancy_dims,
|
||||
motion_detected: st.motion_detected,
|
||||
total_frames: st.total_frames,
|
||||
num_nodes: st.node_frames.len(),
|
||||
current_location: st.current_location.clone(),
|
||||
is_dark: st.is_dark,
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug, serde::Serialize)]
|
||||
pub struct PipelineOutput {
|
||||
pub skeleton: Option<Skeleton>,
|
||||
pub vitals: VitalSigns,
|
||||
pub occupancy: Vec<f64>,
|
||||
pub occupancy_dims: (usize, usize, usize),
|
||||
pub motion_detected: bool,
|
||||
pub total_frames: u64,
|
||||
pub num_nodes: usize,
|
||||
pub current_location: Option<(String, f32)>,
|
||||
pub is_dark: bool,
|
||||
}
|
||||
|
||||
// Serialize implementations
|
||||
impl serde::Serialize for Skeleton {
|
||||
fn serialize<S: serde::Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
|
||||
use serde::ser::SerializeStruct;
|
||||
let mut st = s.serialize_struct("Skeleton", 2)?;
|
||||
st.serialize_field("keypoints", &self.keypoints)?;
|
||||
st.serialize_field("confidence", &self.confidence)?;
|
||||
st.end()
|
||||
}
|
||||
}
|
||||
|
||||
impl serde::Serialize for VitalSigns {
|
||||
fn serialize<S: serde::Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
|
||||
use serde::ser::SerializeStruct;
|
||||
let mut st = s.serialize_struct("VitalSigns", 3)?;
|
||||
st.serialize_field("breathing_rate", &self.breathing_rate)?;
|
||||
st.serialize_field("heart_rate", &self.heart_rate)?;
|
||||
st.serialize_field("motion_score", &self.motion_score)?;
|
||||
st.end()
|
||||
}
|
||||
}
|
||||
|
|
@ -10,8 +10,10 @@
|
|||
//! ruview-pointcloud train # calibration training
|
||||
//! ruview-pointcloud csi-test # send test CSI frames
|
||||
|
||||
mod brain_bridge;
|
||||
mod camera;
|
||||
mod csi;
|
||||
mod csi_pipeline;
|
||||
mod depth;
|
||||
mod fusion;
|
||||
mod pointcloud;
|
||||
|
|
|
|||
|
|
@ -1,10 +1,11 @@
|
|||
//! HTTP server — live camera + ESP32 CSI + fusion → real-time point cloud.
|
||||
|
||||
use crate::brain_bridge;
|
||||
use crate::camera;
|
||||
use crate::csi_pipeline;
|
||||
use crate::depth;
|
||||
use crate::fusion;
|
||||
use crate::pointcloud;
|
||||
use crate::serial_csi;
|
||||
use axum::{
|
||||
extract::State,
|
||||
response::{Html, IntoResponse},
|
||||
|
|
@ -16,32 +17,21 @@ use std::sync::{Arc, Mutex};
|
|||
struct AppState {
|
||||
latest_cloud: Mutex<pointcloud::PointCloud>,
|
||||
latest_splats: Mutex<Vec<pointcloud::GaussianSplat>>,
|
||||
latest_pipeline: Mutex<Option<csi_pipeline::PipelineOutput>>,
|
||||
frame_count: Mutex<u64>,
|
||||
use_camera: bool,
|
||||
csi_state: Option<Arc<Mutex<serial_csi::CsiState>>>,
|
||||
csi_pipeline: Option<Arc<Mutex<csi_pipeline::CsiPipelineState>>>,
|
||||
}
|
||||
|
||||
pub async fn serve(host: &str, port: u16, _wifi_source: Option<&str>) -> anyhow::Result<()> {
|
||||
let has_camera = camera::camera_available();
|
||||
|
||||
// CSI serial readers — only start if explicitly requested via env var
|
||||
// (serial reader needs proper baud rate config to avoid reconnect loop)
|
||||
let csi_state = if std::env::var("RUVIEW_CSI").is_ok() {
|
||||
let mut csi_ports = Vec::new();
|
||||
for p in &["/dev/ttyACM0", "/dev/ttyUSB0"] {
|
||||
if std::path::Path::new(p).exists() { csi_ports.push(*p); }
|
||||
}
|
||||
if !csi_ports.is_empty() {
|
||||
eprintln!(" CSI ports: {:?}", csi_ports);
|
||||
Some(serial_csi::start_serial_readers(&csi_ports))
|
||||
} else { None }
|
||||
} else {
|
||||
eprintln!(" CSI: disabled (set RUVIEW_CSI=1 to enable)");
|
||||
None
|
||||
};
|
||||
// Start CSI pipeline — listens for UDP CSI data from ESP32 nodes
|
||||
let csi_pipeline_state = csi_pipeline::start_pipeline("0.0.0.0:3333");
|
||||
eprintln!(" CSI pipeline: UDP port 3333 (ADR-018 binary frames)");
|
||||
|
||||
let initial_cloud = if has_camera {
|
||||
capture_live_cloud(csi_state.as_ref())
|
||||
capture_camera_cloud()
|
||||
} else {
|
||||
demo_cloud()
|
||||
};
|
||||
|
|
@ -50,29 +40,55 @@ pub async fn serve(host: &str, port: u16, _wifi_source: Option<&str>) -> anyhow:
|
|||
let state = Arc::new(AppState {
|
||||
latest_cloud: Mutex::new(initial_cloud),
|
||||
latest_splats: Mutex::new(initial_splats),
|
||||
latest_pipeline: Mutex::new(None),
|
||||
frame_count: Mutex::new(0),
|
||||
use_camera: has_camera,
|
||||
csi_state: csi_state.clone(),
|
||||
csi_pipeline: Some(csi_pipeline_state.clone()),
|
||||
});
|
||||
|
||||
// Background: capture + fuse every 500ms
|
||||
// Background: capture + fuse every 500ms (motion-adaptive)
|
||||
let bg = state.clone();
|
||||
let bg_csi = csi_state.clone();
|
||||
let bg_csi = Some(csi_pipeline_state.clone());
|
||||
let bg_cam = has_camera;
|
||||
tokio::spawn(async move {
|
||||
let mut skip_depth = false;
|
||||
loop {
|
||||
tokio::time::sleep(std::time::Duration::from_millis(500)).await;
|
||||
let csi_clone = bg_csi.clone();
|
||||
let cloud = if bg_cam {
|
||||
tokio::task::spawn_blocking(move || capture_live_cloud(csi_clone.as_ref()))
|
||||
// Motion-adaptive: check CSI motion score
|
||||
let pipeline_out = bg_csi.as_ref().map(|c| csi_pipeline::get_pipeline_output(c));
|
||||
if let Some(ref out) = pipeline_out {
|
||||
// Only run expensive depth when motion detected or every 5th frame
|
||||
let frame_num = *bg.frame_count.lock().unwrap();
|
||||
skip_depth = !out.motion_detected && frame_num % 5 != 0;
|
||||
}
|
||||
let pipeline_clone = pipeline_out.clone();
|
||||
*bg.latest_pipeline.lock().unwrap() = pipeline_out;
|
||||
let pipeline_out = pipeline_clone;
|
||||
|
||||
let interval = if skip_depth { 1000 } else { 500 }; // slower when no motion
|
||||
tokio::time::sleep(std::time::Duration::from_millis(interval)).await;
|
||||
|
||||
let cloud = if bg_cam && !skip_depth {
|
||||
tokio::task::spawn_blocking(capture_camera_cloud)
|
||||
.await.unwrap_or_else(|_| demo_cloud())
|
||||
} else {
|
||||
demo_cloud()
|
||||
// Reuse previous cloud when no motion
|
||||
bg.latest_cloud.lock().unwrap().clone()
|
||||
};
|
||||
let splats = pointcloud::to_gaussian_splats(&cloud);
|
||||
*bg.latest_cloud.lock().unwrap() = cloud;
|
||||
*bg.latest_splats.lock().unwrap() = splats;
|
||||
*bg.frame_count.lock().unwrap() += 1;
|
||||
let frame_num = {
|
||||
let mut fc = bg.frame_count.lock().unwrap();
|
||||
*fc += 1;
|
||||
*fc
|
||||
};
|
||||
|
||||
// Brain sync — sparse, every 120 frames (~60 seconds)
|
||||
if frame_num % 120 == 0 {
|
||||
if let Some(ref out) = pipeline_out {
|
||||
brain_bridge::sync_to_brain(out, frame_num).await;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
|
|
@ -98,70 +114,19 @@ pub async fn serve(host: &str, port: u16, _wifi_source: Option<&str>) -> anyhow:
|
|||
Ok(())
|
||||
}
|
||||
|
||||
fn capture_live_cloud(csi: Option<&Arc<Mutex<serial_csi::CsiState>>>) -> pointcloud::PointCloud {
|
||||
// 1. Camera → depth → dense points
|
||||
let cam_cloud = {
|
||||
let config = camera::CameraConfig::default();
|
||||
match camera::capture_frame(&config) {
|
||||
Ok(frame) => {
|
||||
match depth::estimate_depth(&frame.rgb, frame.width, frame.height) {
|
||||
Ok(dm) => {
|
||||
let intr = depth::CameraIntrinsics::default();
|
||||
depth::backproject_depth(&dm, &intr, Some(&frame.rgb), 2)
|
||||
}
|
||||
Err(_) => depth::demo_depth_cloud(),
|
||||
fn capture_camera_cloud() -> pointcloud::PointCloud {
|
||||
let config = camera::CameraConfig::default();
|
||||
match camera::capture_frame(&config) {
|
||||
Ok(frame) => {
|
||||
match depth::estimate_depth(&frame.rgb, frame.width, frame.height) {
|
||||
Ok(dm) => {
|
||||
let intr = depth::CameraIntrinsics::default();
|
||||
depth::backproject_depth(&dm, &intr, Some(&frame.rgb), 2)
|
||||
}
|
||||
Err(_) => depth::demo_depth_cloud(),
|
||||
}
|
||||
Err(_) => depth::demo_depth_cloud(),
|
||||
}
|
||||
};
|
||||
|
||||
// 2. CSI → motion + presence → modify point cloud
|
||||
let mut clouds: Vec<&pointcloud::PointCloud> = vec![&cam_cloud];
|
||||
|
||||
let csi_cloud;
|
||||
if let Some(csi_state) = csi {
|
||||
let (motion, distance, frames) = serial_csi::get_csi_influence(csi_state);
|
||||
if frames > 0 {
|
||||
// Create CSI-informed occupancy around detected presence
|
||||
let mut occ = fusion::OccupancyVolume {
|
||||
densities: vec![0.0; 8 * 8 * 4],
|
||||
nx: 8, ny: 8, nz: 4,
|
||||
bounds: [
|
||||
-distance as f64, -distance as f64, 0.0,
|
||||
distance as f64, distance as f64, 2.5,
|
||||
],
|
||||
occupied_count: 0,
|
||||
};
|
||||
|
||||
// Place high density where CSI indicates presence
|
||||
let cx: usize = 4; let cy: usize = 4;
|
||||
let radius: usize = (motion * 3.0).max(1.0) as usize;
|
||||
for iz in 0..4 {
|
||||
for iy in (cy.saturating_sub(radius))..=(cy + radius).min(7) {
|
||||
for ix in (cx.saturating_sub(radius))..=(cx + radius).min(7) {
|
||||
let idx = iz * 64 + iy * 8 + ix;
|
||||
let dx = (ix as f32 - cx as f32).abs() / radius as f32;
|
||||
let dy = (iy as f32 - cy as f32).abs() / radius as f32;
|
||||
let r2 = dx * dx + dy * dy;
|
||||
if r2 < 1.0 {
|
||||
occ.densities[idx] = (1.0 - r2 as f64) * (0.5 + motion as f64 * 0.5);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
occ.occupied_count = occ.densities.iter().filter(|&&d| d > 0.3).count();
|
||||
|
||||
csi_cloud = fusion::occupancy_to_pointcloud(&occ);
|
||||
clouds.push(&csi_cloud);
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Fuse camera + CSI
|
||||
if clouds.len() > 1 {
|
||||
fusion::fuse_clouds(&clouds, 0.04)
|
||||
} else {
|
||||
cam_cloud
|
||||
Err(_) => depth::demo_depth_cloud(),
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -176,13 +141,13 @@ async fn api_cloud(State(state): State<Arc<AppState>>) -> Json<serde_json::Value
|
|||
let cloud = state.latest_cloud.lock().unwrap();
|
||||
let (min, max) = cloud.bounds();
|
||||
let frames = *state.frame_count.lock().unwrap();
|
||||
let csi_info = state.csi_state.as_ref().map(|c| serial_csi::get_csi_influence(c));
|
||||
let pipeline = state.latest_pipeline.lock().unwrap();
|
||||
Json(serde_json::json!({
|
||||
"points": cloud.points.len(),
|
||||
"bounds_min": min, "bounds_max": max,
|
||||
"live": state.use_camera,
|
||||
"frame": frames,
|
||||
"csi": csi_info.map(|(m,d,f)| serde_json::json!({"motion":m,"distance_m":d,"frames":f})),
|
||||
"pipeline": &*pipeline,
|
||||
"cloud": cloud.points.iter().take(1000).collect::<Vec<_>>(),
|
||||
}))
|
||||
}
|
||||
|
|
@ -190,29 +155,28 @@ async fn api_cloud(State(state): State<Arc<AppState>>) -> Json<serde_json::Value
|
|||
async fn api_splats(State(state): State<Arc<AppState>>) -> Json<serde_json::Value> {
|
||||
let splats = state.latest_splats.lock().unwrap();
|
||||
let frames = *state.frame_count.lock().unwrap();
|
||||
let csi_info = state.csi_state.as_ref().map(|c| serial_csi::get_csi_influence(c));
|
||||
let pipeline = state.latest_pipeline.lock().unwrap();
|
||||
Json(serde_json::json!({
|
||||
"splats": &*splats,
|
||||
"count": splats.len(),
|
||||
"live": state.use_camera,
|
||||
"frame": frames,
|
||||
"csi": csi_info.map(|(m,d,f)| serde_json::json!({"motion":m,"distance_m":d,"frames":f})),
|
||||
"pipeline": &*pipeline,
|
||||
"timestamp": chrono::Utc::now().timestamp_millis(),
|
||||
}))
|
||||
}
|
||||
|
||||
async fn api_status(State(state): State<Arc<AppState>>) -> Json<serde_json::Value> {
|
||||
let frames = *state.frame_count.lock().unwrap();
|
||||
let csi_info = state.csi_state.as_ref().map(|c| serial_csi::get_csi_influence(c));
|
||||
let pipeline = state.latest_pipeline.lock().unwrap();
|
||||
Json(serde_json::json!({
|
||||
"status": "ok",
|
||||
"version": env!("CARGO_PKG_VERSION"),
|
||||
"live": state.use_camera,
|
||||
"camera": if state.use_camera { "/dev/video0" } else { "demo" },
|
||||
"csi_ports": if state.csi_state.is_some() { "active" } else { "none" },
|
||||
"csi": csi_info.map(|(m,d,f)| serde_json::json!({"motion":m,"distance_m":d,"frames":f})),
|
||||
"csi_pipeline": "active (UDP:3333)",
|
||||
"pipeline": &*pipeline,
|
||||
"frames_captured": frames,
|
||||
"fps": 2,
|
||||
}))
|
||||
}
|
||||
|
||||
|
|
@ -228,8 +192,10 @@ async fn index() -> Html<String> {
|
|||
<style>
|
||||
body { margin: 0; background: #0a0a0a; color: #e8a634; font-family: monospace; }
|
||||
canvas { display: block; }
|
||||
#info { position: absolute; top: 10px; left: 10px; padding: 12px; background: rgba(0,0,0,0.85); border: 1px solid #e8a634; border-radius: 6px; min-width: 200px; }
|
||||
#info { position: absolute; top: 10px; left: 10px; padding: 12px; background: rgba(0,0,0,0.85); border: 1px solid #e8a634; border-radius: 6px; min-width: 240px; font-size: 13px; line-height: 1.5; }
|
||||
.live { color: #4f4; } .demo { color: #f44; }
|
||||
.section { margin-top: 6px; padding-top: 6px; border-top: 1px solid #333; }
|
||||
.label { color: #888; }
|
||||
</style>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/controls/OrbitControls.js"></script>
|
||||
|
|
@ -240,38 +206,171 @@ async fn index() -> Html<String> {
|
|||
<div id="stats">Loading...</div>
|
||||
</div>
|
||||
<script>
|
||||
const scene = new THREE.Scene();
|
||||
var scene = new THREE.Scene();
|
||||
scene.background = new THREE.Color(0x0a0a0a);
|
||||
const camera = new THREE.PerspectiveCamera(75, window.innerWidth/window.innerHeight, 0.1, 100);
|
||||
camera.position.set(0, 0, -2);
|
||||
camera.lookAt(0, 0, 3);
|
||||
var camera = new THREE.PerspectiveCamera(75, window.innerWidth/window.innerHeight, 0.1, 100);
|
||||
camera.position.set(0, 2, -4);
|
||||
camera.lookAt(0, 0, 2);
|
||||
|
||||
const renderer = new THREE.WebGLRenderer({ antialias: true });
|
||||
var renderer = new THREE.WebGLRenderer({ antialias: true });
|
||||
renderer.setSize(window.innerWidth, window.innerHeight);
|
||||
document.body.appendChild(renderer.domElement);
|
||||
|
||||
const controls = new THREE.OrbitControls(camera, renderer.domElement);
|
||||
var controls = new THREE.OrbitControls(camera, renderer.domElement);
|
||||
controls.enableDamping = true;
|
||||
controls.target.set(0, 0, 3);
|
||||
controls.target.set(0, 0, 2);
|
||||
|
||||
let pointsMesh = null;
|
||||
let lastFrame = -1;
|
||||
var pointsMesh = null;
|
||||
var lastFrame = -1;
|
||||
var skeletonGroup = null;
|
||||
var prevTimestamp = 0;
|
||||
var frameRateVal = 0;
|
||||
|
||||
// COCO skeleton connections: pairs of keypoint indices
|
||||
// 0=nose 1=leftEye 2=rightEye 3=leftEar 4=rightEar
|
||||
// 5=leftShoulder 6=rightShoulder 7=leftElbow 8=rightElbow
|
||||
// 9=leftWrist 10=rightWrist 11=leftHip 12=rightHip
|
||||
// 13=leftKnee 14=rightKnee 15=leftAnkle 16=rightAnkle
|
||||
var COCO_BONES = [
|
||||
[0,1],[0,2],[1,3],[2,4],
|
||||
[5,6],[5,7],[7,9],[6,8],[8,10],
|
||||
[5,11],[6,12],[11,12],
|
||||
[11,13],[13,15],[12,14],[14,16]
|
||||
];
|
||||
|
||||
function clearSkeleton() {
|
||||
if (skeletonGroup) {
|
||||
scene.remove(skeletonGroup);
|
||||
skeletonGroup.traverse(function(obj) {
|
||||
if (obj.geometry) obj.geometry.dispose();
|
||||
if (obj.material) obj.material.dispose();
|
||||
});
|
||||
skeletonGroup = null;
|
||||
}
|
||||
}
|
||||
|
||||
function drawSkeleton(keypoints) {
|
||||
clearSkeleton();
|
||||
if (!keypoints || keypoints.length < 17) return;
|
||||
skeletonGroup = new THREE.Group();
|
||||
|
||||
// Map keypoints from [0,1] to scene coords
|
||||
// x: [-2, 2], y: [2, -2] (flip y), z: fixed at 2
|
||||
var sphereGeo = new THREE.SphereGeometry(0.04, 8, 8);
|
||||
var sphereMat = new THREE.MeshBasicMaterial({ color: 0xffff00 });
|
||||
var positions3D = [];
|
||||
var i, kp, sx, sy;
|
||||
for (i = 0; i < 17; i++) {
|
||||
kp = keypoints[i];
|
||||
if (!kp) { positions3D.push(null); continue; }
|
||||
sx = (kp[0] - 0.5) * 4;
|
||||
sy = (0.5 - kp[1]) * 4;
|
||||
positions3D.push([sx, sy, 2]);
|
||||
var sphere = new THREE.Mesh(sphereGeo, sphereMat);
|
||||
sphere.position.set(sx, sy, 2);
|
||||
skeletonGroup.add(sphere);
|
||||
}
|
||||
|
||||
// Draw bones as white lines
|
||||
var lineMat = new THREE.LineBasicMaterial({ color: 0xffffff, linewidth: 2 });
|
||||
var b, a, bIdx;
|
||||
for (b = 0; b < COCO_BONES.length; b++) {
|
||||
a = COCO_BONES[b][0];
|
||||
bIdx = COCO_BONES[b][1];
|
||||
if (!positions3D[a] || !positions3D[bIdx]) continue;
|
||||
var lineGeo = new THREE.BufferGeometry();
|
||||
var verts = new Float32Array([
|
||||
positions3D[a][0], positions3D[a][1], positions3D[a][2],
|
||||
positions3D[bIdx][0], positions3D[bIdx][1], positions3D[bIdx][2]
|
||||
]);
|
||||
lineGeo.setAttribute("position", new THREE.BufferAttribute(verts, 3));
|
||||
var line = new THREE.Line(lineGeo, lineMat);
|
||||
skeletonGroup.add(line);
|
||||
}
|
||||
|
||||
scene.add(skeletonGroup);
|
||||
}
|
||||
|
||||
async function fetchCloud() {
|
||||
try {
|
||||
const resp = await fetch('/api/splats');
|
||||
const data = await resp.json();
|
||||
var resp = await fetch("/api/splats");
|
||||
var data = await resp.json();
|
||||
if (data.splats && data.frame !== lastFrame) {
|
||||
// Compute CSI frame rate
|
||||
var now = Date.now();
|
||||
if (prevTimestamp > 0) {
|
||||
var dt = (now - prevTimestamp) / 1000.0;
|
||||
if (dt > 0) frameRateVal = (1.0 / dt).toFixed(1);
|
||||
}
|
||||
prevTimestamp = now;
|
||||
lastFrame = data.frame;
|
||||
updateSplats(data.splats);
|
||||
const mode = data.live ? '<span class="live">● LIVE</span>' : '<span class="demo">● DEMO</span>';
|
||||
let csiInfo = '';
|
||||
if (data.csi) {
|
||||
const m = (data.csi.motion * 100).toFixed(0);
|
||||
csiInfo = `<br>CSI: ${data.csi.frames} frames, motion ${m}%<br>Distance: ${data.csi.distance_m.toFixed(1)}m`;
|
||||
|
||||
// Draw skeleton if available
|
||||
var pipe = data.pipeline;
|
||||
if (pipe && pipe.skeleton && pipe.skeleton.keypoints) {
|
||||
drawSkeleton(pipe.skeleton.keypoints);
|
||||
} else {
|
||||
clearSkeleton();
|
||||
}
|
||||
document.getElementById('stats').innerHTML =
|
||||
`${mode} Camera + CSI<br>Splats: ${data.count}<br>Frame: ${data.frame}${csiInfo}`;
|
||||
|
||||
// Build info panel
|
||||
var mode = data.live
|
||||
? '<span class="live">● LIVE</span>'
|
||||
: '<span class="demo">● DEMO</span>';
|
||||
var html = mode + " Camera + CSI<br>"
|
||||
+ "Splats: " + data.count + "<br>"
|
||||
+ "Frame: " + data.frame;
|
||||
|
||||
// CSI frame rate
|
||||
html += '<div class="section">'
|
||||
+ '<span class="label">CSI Rate:</span> '
|
||||
+ frameRateVal + " fps</div>";
|
||||
|
||||
// Skeleton confidence
|
||||
if (pipe && pipe.skeleton && pipe.skeleton.confidence !== undefined) {
|
||||
var conf = (pipe.skeleton.confidence * 100).toFixed(0);
|
||||
html += '<div class="section">'
|
||||
+ '<span class="label">Skeleton:</span> '
|
||||
+ conf + "% confidence</div>";
|
||||
}
|
||||
|
||||
// Weather data
|
||||
if (pipe && pipe.weather) {
|
||||
var w = pipe.weather;
|
||||
html += '<div class="section">'
|
||||
+ '<span class="label">Weather:</span> ';
|
||||
if (w.temperature !== undefined) {
|
||||
html += w.temperature + "°C";
|
||||
}
|
||||
if (w.conditions) {
|
||||
html += " " + w.conditions;
|
||||
}
|
||||
html += "</div>";
|
||||
}
|
||||
|
||||
// Building count from geo
|
||||
if (pipe && pipe.geo && pipe.geo.building_count !== undefined) {
|
||||
html += '<div class="section">'
|
||||
+ '<span class="label">Buildings:</span> '
|
||||
+ pipe.geo.building_count + "</div>";
|
||||
}
|
||||
|
||||
// Vitals
|
||||
if (pipe && pipe.vitals) {
|
||||
var v = pipe.vitals;
|
||||
html += '<div class="section">'
|
||||
+ '<span class="label">Vitals:</span> ';
|
||||
if (v.breathing_rate !== undefined) {
|
||||
html += "BR " + v.breathing_rate + "/min";
|
||||
}
|
||||
if (v.motion_score !== undefined) {
|
||||
html += " Motion " + (v.motion_score * 100).toFixed(0) + "%";
|
||||
}
|
||||
html += "</div>";
|
||||
}
|
||||
|
||||
document.getElementById("stats").innerHTML = html;
|
||||
}
|
||||
} catch(e) {}
|
||||
}
|
||||
|
|
@ -280,21 +379,23 @@ async fn index() -> Html<String> {
|
|||
|
||||
function updateSplats(splats) {
|
||||
if (pointsMesh) scene.remove(pointsMesh);
|
||||
const geometry = new THREE.BufferGeometry();
|
||||
const positions = new Float32Array(splats.length * 3);
|
||||
const colors = new Float32Array(splats.length * 3);
|
||||
splats.forEach((s, i) => {
|
||||
var geometry = new THREE.BufferGeometry();
|
||||
var positions = new Float32Array(splats.length * 3);
|
||||
var colors = new Float32Array(splats.length * 3);
|
||||
var i, s;
|
||||
for (i = 0; i < splats.length; i++) {
|
||||
s = splats[i];
|
||||
positions[i*3] = s.center[0];
|
||||
positions[i*3+1] = -s.center[1];
|
||||
positions[i*3+2] = s.center[2];
|
||||
colors[i*3] = s.color[0];
|
||||
colors[i*3+1] = s.color[1];
|
||||
colors[i*3+2] = s.color[2];
|
||||
});
|
||||
geometry.setAttribute('position', new THREE.BufferAttribute(positions, 3));
|
||||
geometry.setAttribute('color', new THREE.BufferAttribute(colors, 3));
|
||||
}
|
||||
geometry.setAttribute("position", new THREE.BufferAttribute(positions, 3));
|
||||
geometry.setAttribute("color", new THREE.BufferAttribute(colors, 3));
|
||||
pointsMesh = new THREE.Points(geometry, new THREE.PointsMaterial({
|
||||
size: 0.025, vertexColors: true, sizeAttenuation: true,
|
||||
size: 0.02, vertexColors: true, sizeAttenuation: true
|
||||
}));
|
||||
scene.add(pointsMesh);
|
||||
}
|
||||
|
|
@ -305,7 +406,7 @@ async fn index() -> Html<String> {
|
|||
renderer.render(scene, camera);
|
||||
}
|
||||
animate();
|
||||
window.addEventListener('resize', () => {
|
||||
window.addEventListener("resize", function() {
|
||||
camera.aspect = window.innerWidth / window.innerHeight;
|
||||
camera.updateProjectionMatrix();
|
||||
renderer.setSize(window.innerWidth, window.innerHeight);
|
||||
|
|
|
|||
Loading…
Reference in New Issue