# ADR-170: yoga-mode — pose detection, classification, and scoring for the three.js realtime demo | Field | Value | |-------|-------| | **Status** | Proposed | | **Date** | 2026-06-02 | | **Deciders** | ruv | | **Codename** | **yoga-mode** | | **Scope** | `examples/three.js/demos/05-skinned-realtime.html` (primary); new `examples/three.js/demos/06-yoga-mode.html` (secondary, slimmed-down) | | **Relates to** | ADR-169 (adam-mode light theme), ADR-019 (sensing-only UI), ADR-035 (live sensing UI accuracy) | | **Tracking issue** | none yet | --- ## 1. Context `examples/three.js/demos/05-skinned-realtime.html` already runs the full MediaPipe Pose Heavy pipeline at ~30 Hz: 33 BlazePose landmarks flow through a one-euro-filter bank into joint-angle extraction and then into a Mixamo X Bot IK retarget. The `#pose-panel` HUD shows landmark count, visibility, and pose FPS. The `#helpers` panel (ADR-097) has adam-mode (ADR-169) and eight visualisation toggles. This infrastructure is complete. Every frame, per-joint angles are already computable from the existing `liveKp` world-space landmark array. What does not yet exist is any layer that interprets those angles as a known yoga pose, scores the user's alignment against a target shape, and guides the user through a structured sequence. ### 1.1 Why yoga-mode in this demo Three concrete use-cases drive this: 1. **Developer self-test for the retargeting pipeline.** Cycling through a Sun Salutation A is a systematic, reproducible way to exercise every major joint (shoulder, elbow, hip, knee, spine). A pose-scoring overlay makes regression immediately visible — if a code change breaks elbow retargeting, the yoga classifier will output a depressed alignment score on Chaturanga even before a visual inspection. 2. **Public demonstration value.** The demo is served at `http://127.0.0.1:8765/examples/three.js/demos/05-skinned-realtime.html` and shown to evaluators. A guided instructional mode that scores real-time body alignment against Tadasana or Downward Dog is immediately intelligible to a non-technical audience in a way that raw CSI amplitude bars are not. 3. **Future bridge to the Rust host.** The Rust-side `wifi-densepose-signal/src/ruvsense/pose_tracker.rs` maintains a 17-keypoint Kalman tracker in COCO convention. yoga-mode in the demo operates on the 33-landmark MediaPipe convention. These are not the same: MediaPipe indices 0–32 (BlazePose) map non-trivially to COCO 0–16. Deciding the mapping now — even in a pure-JS context — canonicalises it for the eventual Rust integration. ### 1.2 What this ADR is *not* - Not a backend service. No WebSocket endpoint, no session record, no cloud upload. Pure client-side HTML. - Not a fitness-app competitor. The scope is Sun Salutation A (8 poses). The full 84-asana classical corpus is out of scope. - Not an integration with the Rust `pose_tracker.rs`. That bridge is documented here as a future consequence, not an immediate deliverable. - Not a redesign of demo 05. Panel layout, three.js scene geometry, and the CSI overlay are unchanged. - Not a new design system. yoga-mode inherits every existing CSS custom property. ### 1.3 COCO-17 ↔ BlazePose-33 mapping note The Rust tracker uses COCO 17-keypoint indices (0=nose, 5=left-shoulder, 6=right-shoulder, 7=left-elbow, 8=right-elbow, 9=left-wrist, 10=right-wrist, 11=left-hip, 12=right-hip, 13=left-knee, 14=right-knee, 15=left-ankle, 16=right-ankle). MediaPipe BlazePose-33 uses a different, denser scheme where shoulders are at 11–12, elbows at 13–14, wrists at 15–16, hips at 23–24, knees at 25–26, ankles at 27–28. The mapping for the 13 joints used in yoga-mode angle computation is: | Joint role | COCO idx | BlazePose idx | |---|---|---| | nose | 0 | 0 | | left shoulder | 5 | 11 | | right shoulder | 6 | 12 | | left elbow | 7 | 13 | | right elbow | 8 | 14 | | left wrist | 9 | 15 | | right wrist | 10 | 16 | | left hip | 11 | 23 | | right hip | 12 | 24 | | left knee | 13 | 25 | | right knee | 14 | 26 | | left ankle | 15 | 27 | | right ankle | 16 | 28 | When the Rust host integration is implemented, the joint-angle features extracted by yoga-mode in JS and by `pose_tracker.rs` in Rust will be computed from the same physical joints via this table. No translation layer is needed at runtime — yoga-mode always uses BlazePose indices; `pose_tracker.rs` always uses COCO indices. ### 1.4 Biomechanical basis for joint-angle targets The joint-angle targets in this ADR are grounded in peer-reviewed measurements. Perez-Testor et al. (2019, PMC6521759) captured 10 trained practitioners performing Surya Namaskar A on a 12-camera Vicon system at 100 Hz, reporting sagittal-plane joint angles at each pose transition. Key ranges: elbow 22°–116°, hip 15° extension to 134° flexion, knee 3° hyperextension to 140° flexion, spine 44° extension to 58° flexion, shoulder 56°–183°. These empirical ranges set the upper and lower bounds for the tolerance bands in this ADR's pose templates. Where Perez-Testor does not report a joint (e.g. wrist flexion for Chaturanga arm angle), the Iyengar geometry — "elbows at 90° bent close to the body" — supplies the target value. A 2023 PMC yoga-pose review (PMC10280249) confirming angle-heuristic approaches as the most reliable real-time classification method validates the algorithmic choice. --- ## 2. Decision ### 2.1 Pose taxonomy — Sun Salutation A, 8 poses Sun Salutation A is chosen for the first ship. It satisfies three criteria simultaneously: the poses are geometrically distinct from each other (no two share the same joint-angle signature), they form a complete bilateral sequence (both left and right sides are exercised), and they are among the best-documented asanas in the biomechanics literature. The Sanskrit and English names are unambiguous in the Ashtanga tradition. The 8 poses in sequence order with their one-line joint-angle signatures: | Stage | Sanskrit | English | Joint-angle signature | |---|---|---|---| | 1 | Tāḍāsana | Mountain Pose | All limbs extended: knees 180°, hips 180°, elbows 180°, spine vertical | | 2 | Ūrdhva Hastāsana | Upward Salute | Arms overhead: shoulders ~180° abducted, elbows 180°, torso elongated | | 3 | Uttānāsana | Standing Forward Fold | Hips ~0–30° (full fold), knees 180°, elbows relaxed, spine flexed | | 4 | Ardha Uttānāsana | Half Lift / Flat-Back | Hips ~90° (parallel torso), knees 180°, spine neutral (horizontal) | | 5 | Catvāri (Chaturanga Daṇḍāsana) | Four-Limbed Staff | Hips 180° (plank line), elbows ~90°, shoulders depressed, body horizontal | | 6 | Ūrdhva Mukha Śvānāsana | Upward-Facing Dog | Hips extended ~160°+, shoulders over wrists, spine extended, knees off floor | | 7 | Adho Mukha Śvānāsana | Downward-Facing Dog | Hips ~80–110° (inverted V), knees 180°, shoulders ~180° (arms overhead), spine long | | 8 | Uttānāsana | Standing Forward Fold (return) | Same as stage 3 — mirrors the descent; re-classified as stage 8 for sequence tracking | "All 84 classical asanas" is explicitly rejected. Even the 26-pose Bikram set is rejected — the goal is a complete, self-contained instructional sequence for a 2–3 minute demo session, not exhaustive coverage. Eight poses are the minimum for a meaningful sequence narrative and the maximum that fits a single UI strip without horizontal scrolling on a 1080p screen. ### 2.2 Detection algorithm — joint-angle threshold matching with weighted scoring **Chosen: joint-angle threshold matching.** For each frame, compute the angle at 6–10 named joints (one angle per joint, defined as the interior angle at the vertex formed by three landmarks). Compare each computed angle to the per-pose target. Score by weighted absolute deviation. Classify the argmax. **Why not the alternatives:** | Alternative | Verdict | Reason | |---|---|---| | Skeleton-as-vector cosine similarity | Rejected | Position-sensitive: a person standing 2 m from the camera vs. 1 m produces different vectors. Joint angles are translation- and scale-invariant by construction. | | Small MLP trained on a labelled dataset | Rejected | No labelled dataset exists in this codebase. Training a reliable MLP for 8 poses would require hundreds of labelled examples per class, a train/test split, and a model serialization format — none of which belongs in a single-file demo HTML. Joint-angle matching achieves the same discrimination for 8 geometrically distinct poses with zero training data. | | MediaPipe Tasks PoseClassifier (EfficientNet-based) | Rejected | Requires loading a separate `.task` bundle (~4 MB), adds a network dependency to the demo's offline-capable design, and uses a black-box embedding — undebuggable when a pose is misclassified. Threshold matching is fully inspectable in DevTools. | | DTW template matching on full landmark sequences | Rejected | Appropriate for gesture recognition over time (ADR-014's `gesture.rs`), not static pose classification. Sun Salutation transitions are slow (2–5 seconds per pose); per-frame angle scoring is sufficient. | **Joint angle computation.** For three landmark positions A (proximal), B (vertex), C (distal), the interior angle at B is: ``` angle_B = arccos( dot(A-B, C-B) / (|A-B| * |C-B|) ) in degrees ``` This is computed in world-space from the existing `liveKp` THREE.Vector3 array. The computation is purely arithmetic — no matrix inversion, no DFT. At 30 Hz on any modern laptop it is unmeasurably fast relative to the MediaPipe inference cost. **Named joints used in yoga-mode.** Joint names, their three-landmark triplets (proximal-vertex-distal), and the BlazePose indices: | Joint name | Triplet (P-V-D) | Indices | |---|---|---| | `left_elbow` | shoulder→elbow→wrist | 11→13→15 | | `right_elbow` | shoulder→elbow→wrist | 12→14→16 | | `left_knee` | hip→knee→ankle | 23→25→27 | | `right_knee` | hip→knee→ankle | 24→26→28 | | `left_hip` | shoulder→hip→knee | 11→23→25 | | `right_hip` | shoulder→hip→knee | 12→24→26 | | `left_shoulder` | hip→shoulder→elbow | 23→11→13 | | `right_shoulder` | hip→shoulder→elbow | 24→12→14 | | `torso_lean` | hip-midpoint→shoulder-midpoint→vertical | synthetic | `torso_lean` is the angle between the hip-to-shoulder axis and the world vertical (Y axis). It distinguishes standing-upright (≈0°) from folded-forward (≈90°) from plank-horizontal (≈90° in a different axis pattern). In practice, it is implemented as `acos(dot(hipToShoulder.normalize(), UP_VECTOR))` where `UP_VECTOR = (0,1,0)`. ### 2.3 Pose template format — inline JSON, single-file portable Templates live as a JS object literal inside the `