aa3a6725a6
Each fix ships a test that would have caught the bug: - ruview_metrics OKS: derive scale from GT extent (no s=1.0 fake-Gold), reject s<=0, bound the loop to array extents (no panic on short/adversarial input). - config.validate(): UPPER bounds on window_frames/subcarriers/backbone_channels/ heatmap_size/keypoints/body_parts/batch_size + reject negative gpu_device_id (closes the config-OOM class); defaults+presets still validate. - subcarrier.rs: graceful fallback instead of panic on non-contiguous input. - ablation.rs latency_percentiles: total_cmp + NaN guard (no partial_cmp unwrap). - tensor.rs softmax(axis): normalize per-lane along the given axis (was whole- tensor), out-of-range axis -> NnError; fixes densepose per-pixel probs. - translator.rs apply_attention: real scaled-dot-product attention (was a uniform 1/seq_len stub that made any "with attention" ablation == without); mis-shaped checkpoint projections rejected. Co-Authored-By: claude-flow <ruv@ruv.net> |
||
|---|---|---|
| .. | ||
| benches | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
README.md
wifi-densepose-nn
Multi-backend neural network inference for WiFi-based DensePose estimation.
Overview
wifi-densepose-nn provides the inference engine that maps processed WiFi CSI features to
DensePose body surface predictions. It supports three backends -- ONNX Runtime (default),
PyTorch via tch-rs, and Candle -- so models can run on CPU, CUDA GPU, or TensorRT depending
on the deployment target.
The crate implements two key neural components:
- DensePose Head -- Predicts 24 body part segmentation masks and per-part UV coordinate regression.
- Modality Translator -- Translates CSI feature embeddings into visual feature space, bridging the domain gap between WiFi signals and image-based pose estimation.
Features
- ONNX Runtime backend (default) -- Load and run
.onnxmodels with CPU or GPU execution providers. - PyTorch backend (
tch-backend) -- Native PyTorch inference via libtorch FFI. - Candle backend (
candle-backend) -- Pure-Rust inference withcandle-coreandcandle-nn. - CUDA acceleration (
cuda) -- GPU execution for supported backends. - TensorRT optimization (
tensorrt) -- INT8/FP16 optimized inference via ONNX Runtime. - Batched inference -- Process multiple CSI frames in a single forward pass.
- Model caching -- Memory-mapped model weights via
memmap2.
Feature flags
| Flag | Default | Description |
|---|---|---|
onnx |
yes | ONNX Runtime backend |
tch-backend |
no | PyTorch (tch-rs) backend |
candle-backend |
no | Candle pure-Rust backend |
cuda |
no | CUDA GPU acceleration |
tensorrt |
no | TensorRT via ONNX Runtime |
all-backends |
no | Enable onnx + tch + candle together |
Quick Start
use wifi_densepose_nn::{InferenceEngine, DensePoseConfig, OnnxBackend};
// Create inference engine with ONNX backend
let config = DensePoseConfig::default();
let backend = OnnxBackend::from_file("model.onnx")?;
let engine = InferenceEngine::new(backend, config)?;
// Run inference on a CSI feature tensor
let input = ndarray::Array4::zeros((1, 256, 64, 64));
let output = engine.infer(&input)?;
println!("Body parts: {}", output.body_parts.shape()[1]); // 24
Architecture
wifi-densepose-nn/src/
lib.rs -- Re-exports, constants (NUM_BODY_PARTS=24), prelude
densepose.rs -- DensePoseHead, DensePoseConfig, DensePoseOutput
inference.rs -- Backend trait, InferenceEngine, InferenceOptions
onnx.rs -- OnnxBackend, OnnxSession (feature-gated)
tensor.rs -- Tensor, TensorShape utilities
translator.rs -- ModalityTranslator (CSI -> visual space)
error.rs -- NnError, NnResult
Related Crates
| Crate | Role |
|---|---|
wifi-densepose-core |
Foundation types and NeuralInference trait |
wifi-densepose-signal |
Produces CSI features consumed by inference |
wifi-densepose-train |
Trains the models this crate loads |
ort |
ONNX Runtime Rust bindings |
tch |
PyTorch Rust bindings |
candle-core |
Hugging Face pure-Rust ML framework |
License
MIT OR Apache-2.0