277 lines
11 KiB
Markdown
277 lines
11 KiB
Markdown
# ADR-071: ruvllm Training Pipeline for CSI Sensing Models
|
|
|
|
- **Status**: Proposed
|
|
- **Date**: 2026-04-02
|
|
- **Deciders**: ruv
|
|
- **Relates to**: ADR-069 (Cognitum Seed CSI Pipeline), ADR-070 (Self-Supervised Pretraining), ADR-024 (Contrastive CSI Embedding / AETHER), ADR-016 (RuVector Training Pipeline)
|
|
|
|
## Context
|
|
|
|
The WiFi-DensePose project needs a training pipeline to convert collected CSI data
|
|
(`.csi.jsonl` frames from ESP32 nodes) into deployable models for presence detection,
|
|
activity classification, and vital sign estimation.
|
|
|
|
Previous ADRs established the data collection protocol (ADR-070) and Cognitum Seed
|
|
inference target (ADR-069). What was missing was the actual training, refinement,
|
|
quantization, and export pipeline connecting raw CSI recordings to deployable models.
|
|
|
|
### Why ruvllm instead of PyTorch
|
|
|
|
| Criterion | ruvllm | PyTorch | ONNX Runtime |
|
|
|-----------|--------|---------|--------------|
|
|
| Runtime dependency | Node.js only | Python + CUDA + pip | C++ runtime |
|
|
| Install size | ~5 MB (npm) | ~2 GB (torch+cuda) | ~50 MB |
|
|
| SONA adaptation | <1ms native | N/A | N/A |
|
|
| Quantization | 2/4/8-bit TurboQuant | INT8/FP16 (separate tool) | INT8 only |
|
|
| LoRA fine-tuning | Built-in LoraAdapter | Requires PEFT library | N/A |
|
|
| EWC protection | Built-in EwcManager | Manual implementation | N/A |
|
|
| SafeTensors export | Native SafeTensorsWriter | Via safetensors library | N/A |
|
|
| Contrastive training | Built-in ContrastiveTrainer | Manual triplet loss | N/A |
|
|
| Edge deployment | ESP32, Pi Zero, browser | GPU servers only | ARM (limited) |
|
|
| M4 Pro performance | 88-135 tok/s native | ~30 tok/s (MPS) | ~50 tok/s |
|
|
| Ecosystem integration | RuVector, Cognitum Seed | Standalone | Standalone |
|
|
|
|
The ruvllm package (`@ruvector/ruvllm` v2.5.4) provides the complete training
|
|
lifecycle in a single dependency: contrastive pretraining, task head training,
|
|
LoRA refinement, EWC consolidation, quantization, and SafeTensors/RVF export.
|
|
No Python dependency means the entire pipeline runs on the same Node.js runtime
|
|
as the Cognitum Seed inference engine.
|
|
|
|
## Decision
|
|
|
|
Use ruvllm's `ContrastiveTrainer`, `TrainingPipeline`, `LoraAdapter`, `EwcManager`,
|
|
`SafeTensorsWriter`, and `ModelExporter` for the complete CSI model training lifecycle.
|
|
|
|
### Training Phases
|
|
|
|
The pipeline executes five sequential phases:
|
|
|
|
#### Phase 1: Contrastive Pretraining
|
|
|
|
Learns an embedding space where temporally and spatially similar CSI states are close
|
|
and dissimilar states are far apart.
|
|
|
|
- **Encoder architecture**: 8-dim CSI feature vector -> 64-dim hidden (ReLU) -> 128-dim embedding (L2-normalized)
|
|
- **Loss functions**: Triplet loss (margin=0.3) + InfoNCE (temperature=0.07)
|
|
- **Triplet strategies**:
|
|
- Temporal positive: frames within 1 second (same environment state)
|
|
- Temporal negative: frames >30 seconds apart (different state)
|
|
- Cross-node positive: same timestamp from different ESP32 nodes (same person, different viewpoint)
|
|
- Cross-node negative: different timestamp + different node
|
|
- Hard negatives: frames near motion energy transition boundaries
|
|
- **Hyperparameters**: 20 epochs, batch size 32, hard negative ratio 0.7
|
|
- **Implementation**: `ContrastiveTrainer.addTriplet()` + `.train()`
|
|
|
|
#### Phase 2: Task Head Training
|
|
|
|
Trains supervised heads on top of the frozen embedding for specific sensing tasks.
|
|
|
|
- **Presence head**: 128 -> 1 (sigmoid), threshold at presence_score > 0.3
|
|
- **Activity head**: 128 -> 3 (softmax: still/moving/empty), derived from motion_energy thresholds
|
|
- **Vitals head**: 128 -> 2 (linear: breathing BPM, heart rate BPM), normalized targets
|
|
- **Implementation**: `TrainingPipeline.addData()` + `.train()` with cosine LR scheduler,
|
|
early stopping (patience=5), and quality-weighted MSE loss
|
|
|
|
#### Phase 3: LoRA Refinement
|
|
|
|
Per-node LoRA adapters for room-specific adaptation without forgetting the base model.
|
|
|
|
- **Configuration**: rank=4, alpha=8, dropout=0.1
|
|
- **Per-node training**: Each ESP32 node gets its own LoRA adapter trained on
|
|
node-specific data with reduced learning rate (0.5x base)
|
|
- **Implementation**: `LoraManager.create()` for each node, `TrainingPipeline` with
|
|
`LoraAdapter` passed to constructor
|
|
|
|
#### Phase 4: Quantization (TurboQuant)
|
|
|
|
Reduces model size for edge deployment with minimal quality loss.
|
|
|
|
| Bit Width | Compression | Typical RMSE | Target Device |
|
|
|-----------|-------------|-------------|---------------|
|
|
| 8-bit | 4x | <0.001 | Cognitum Seed (Pi Zero) |
|
|
| 4-bit | 8x | <0.01 | Standard edge inference |
|
|
| 2-bit | 16x | <0.05 | ESP32-S3 feature extraction |
|
|
|
|
- **Method**: Uniform affine quantization with scale/zero-point per tensor
|
|
- **Quality validation**: RMSE between original fp32 and dequantized weights
|
|
|
|
#### Phase 5: EWC Consolidation
|
|
|
|
Elastic Weight Consolidation prevents catastrophic forgetting when the model
|
|
is later fine-tuned on new room data or updated CSI conditions.
|
|
|
|
- **Fisher information**: Computed from training data gradients
|
|
- **Lambda**: 2000 (base), 3000 (per-node)
|
|
- **Tasks registered**: Base pretraining + one per ESP32 node
|
|
- **Implementation**: `EwcManager.registerTask()` for each training phase
|
|
|
|
### Data Pipeline
|
|
|
|
```
|
|
.csi.jsonl files
|
|
|
|
|
v
|
|
Parse frames: feature (8-dim), vitals, raw CSI
|
|
|
|
|
v
|
|
Generate contrastive triplets (temporal, cross-node, hard negatives)
|
|
|
|
|
v
|
|
Encode through CsiEncoder (8 -> 64 -> 128)
|
|
|
|
|
v
|
|
Phase 1: ContrastiveTrainer (triplet + InfoNCE loss)
|
|
|
|
|
v
|
|
Phase 2: TrainingPipeline (presence + activity + vitals heads)
|
|
|
|
|
v
|
|
Phase 3: LoRA per-node refinement
|
|
|
|
|
v
|
|
Phase 4: TurboQuant (2/4/8-bit quantization)
|
|
|
|
|
v
|
|
Phase 5: EWC consolidation
|
|
|
|
|
v
|
|
Export: SafeTensors, JSON config, RVF manifest, per-node LoRA adapters
|
|
```
|
|
|
|
### Export Formats
|
|
|
|
| Format | File | Consumer |
|
|
|--------|------|----------|
|
|
| SafeTensors | `model.safetensors` | HuggingFace ecosystem, general inference |
|
|
| JSON config | `config.json` | Model loading metadata |
|
|
| JSON model | `model.json` | Full model state for Node.js loading |
|
|
| Quantized binaries | `quantized/model-q{2,4,8}.bin` | Edge deployment |
|
|
| Per-node LoRA | `lora/node-{id}.json` | Room-specific adaptation |
|
|
| RVF manifest | `model.rvf.jsonl` | Cognitum Seed ingest (ADR-069) |
|
|
| Training metrics | `training-metrics.json` | Dashboards, CI validation |
|
|
|
|
### Hardware Targets
|
|
|
|
| Device | Role | Quantization | Expected Latency |
|
|
|--------|------|-------------|-----------------|
|
|
| Mac Mini M4 Pro | Training (primary) | fp32 | <5 min total |
|
|
| Cognitum Seed Pi Zero | Inference | 4-bit / 8-bit | <10 ms per frame |
|
|
| ESP32-S3 | Feature extraction only | 2-bit (encoder weights) | <5 ms per frame |
|
|
| Browser (WASM) | Visualization | 4-bit | <20 ms per frame |
|
|
|
|
### Performance Targets
|
|
|
|
| Metric | Target | Measured |
|
|
|--------|--------|----------|
|
|
| Training time (5,783 frames, M4 Pro) | <5 min | TBD |
|
|
| Inference latency (M4 Pro) | <1 ms | TBD |
|
|
| Inference latency (Pi Zero) | <10 ms | TBD |
|
|
| SONA adaptation | <1 ms | <0.05 ms (ruvllm spec) |
|
|
| Presence detection accuracy | >85% | TBD |
|
|
| 4-bit quality loss (RMSE) | <0.01 | TBD |
|
|
| 2-bit quality loss (RMSE) | <0.05 | TBD |
|
|
|
|
## Consequences
|
|
|
|
### Positive
|
|
|
|
- **Zero Python dependency**: The entire training and inference pipeline runs on
|
|
Node.js, eliminating Python/CUDA/pip dependency management on training and
|
|
deployment targets.
|
|
- **Integrated lifecycle**: Contrastive pretraining, task heads, LoRA refinement,
|
|
EWC consolidation, and quantization in a single script using one library.
|
|
- **Edge-first**: 2-bit quantization enables running the encoder on ESP32-S3.
|
|
4-bit quantization fits comfortably on Cognitum Seed Pi Zero.
|
|
- **Continual learning**: EWC protection means the model can be updated with new
|
|
room data without losing previously learned patterns.
|
|
- **Per-node adaptation**: LoRA adapters allow room-specific fine-tuning with
|
|
minimal storage overhead (rank-4 adapter ~2KB per node).
|
|
- **HuggingFace compatibility**: SafeTensors export enables sharing models on the
|
|
HuggingFace Hub and loading in other frameworks.
|
|
- **Reproducibility**: Seeded encoder initialization and deterministic data pipeline
|
|
ensure reproducible training runs.
|
|
|
|
### Negative
|
|
|
|
- **No GPU acceleration**: ruvllm's JS training loop does not use GPU compute.
|
|
For the small model sizes in CSI sensing (8->64->128), this is acceptable
|
|
(~seconds on M4 Pro), but would not scale to large vision models.
|
|
- **Simplified backpropagation**: The LoRA backward pass and contrastive training
|
|
use approximate gradient updates rather than full automatic differentiation.
|
|
Sufficient for the target model sizes but not equivalent to PyTorch autograd.
|
|
- **Quantization is post-training only**: No quantization-aware training (QAT).
|
|
For 4-bit and 8-bit this produces acceptable quality loss; 2-bit may need
|
|
QAT in future if quality degrades.
|
|
|
|
### Risks
|
|
|
|
- **Quality ceiling**: The simplified training may produce lower accuracy than a
|
|
PyTorch-trained equivalent. Mitigated by: (a) the model is small enough that
|
|
the training loop converges quickly, (b) SONA adaptation can compensate at
|
|
inference time, (c) we can switch to PyTorch for training only if needed
|
|
while keeping ruvllm for inference.
|
|
- **ruvllm API stability**: The library is at v2.5.4 with active development.
|
|
Mitigated by vendoring the package in `vendor/ruvector/npm/packages/ruvllm/`.
|
|
|
|
## Implementation
|
|
|
|
### Scripts
|
|
|
|
| Script | Purpose |
|
|
|--------|---------|
|
|
| `scripts/train-ruvllm.js` | Full 5-phase training pipeline |
|
|
| `scripts/benchmark-ruvllm.js` | Model benchmarking (latency, quality, accuracy) |
|
|
|
|
### Usage
|
|
|
|
```bash
|
|
# Train on collected CSI data
|
|
node scripts/train-ruvllm.js \
|
|
--data data/recordings/pretrain-1775182186.csi.jsonl \
|
|
--output models/csi-v1 \
|
|
--epochs 20
|
|
|
|
# Train with benchmark
|
|
node scripts/train-ruvllm.js \
|
|
--data data/recordings/pretrain-*.csi.jsonl \
|
|
--output models/csi-v1 \
|
|
--benchmark
|
|
|
|
# Standalone benchmark
|
|
node scripts/benchmark-ruvllm.js \
|
|
--model models/csi-v1 \
|
|
--data data/recordings/pretrain-*.csi.jsonl \
|
|
--samples 5000 \
|
|
--json
|
|
```
|
|
|
|
### Output Structure
|
|
|
|
```
|
|
models/csi-v1/
|
|
model.safetensors # SafeTensors (HuggingFace compatible)
|
|
config.json # Model configuration
|
|
model.json # Full JSON model state
|
|
model.rvf.jsonl # RVF manifest for Cognitum Seed
|
|
training-metrics.json # Training loss curves, timing, config
|
|
contrastive/
|
|
triplets.jsonl # Contrastive training pairs
|
|
triplets.csv # CSV format for analysis
|
|
embeddings.json # Embedding matrices
|
|
quantized/
|
|
model-q2.bin # 2-bit quantized (ESP32 edge)
|
|
model-q4.bin # 4-bit quantized (Pi Zero default)
|
|
model-q8.bin # 8-bit quantized (high quality)
|
|
lora/
|
|
node-1.json # LoRA adapter for ESP32 node 1
|
|
node-2.json # LoRA adapter for ESP32 node 2
|
|
```
|
|
|
|
## References
|
|
|
|
- [ruvllm source](vendor/ruvector/npm/packages/ruvllm/) — v2.5.4
|
|
- [ADR-069](ADR-069-cognitum-seed-csi-pipeline.md) — Cognitum Seed CSI Pipeline
|
|
- [ADR-070](ADR-070-self-supervised-pretraining.md) — Self-Supervised Pretraining Protocol
|
|
- [ADR-024](ADR-024-contrastive-csi-embedding.md) — Contrastive CSI Embedding / AETHER
|
|
- [ADR-016](ADR-016-ruvector-training-pipeline.md) — RuVector Training Pipeline Integration
|