diff --git a/three.js/README.md b/three.js/README.md new file mode 100644 index 00000000..893d51da --- /dev/null +++ b/three.js/README.md @@ -0,0 +1,77 @@ +# three.js demos + +Five progressively richer browser demos of the ADR-097 sensing-helpers scene, +ending with a live MediaPipe-Pose → Mixamo X Bot retargeting pipeline driven +by a real ESP32 CSI feed. + +## Run them + +```bash +python examples/three.js/server/serve-demo.py +# then open one of the URLs the script prints +``` + +`server/serve-demo.py` is a tiny `ThreadingHTTPServer` with aggressive +no-cache headers — the stdlib `http.server` is single-threaded and times out +on the parallel script + FBX fetches the demos make. + +## Demos + +| # | File | What it shows | +|---|------|---------------| +| 01 | [`demos/01-helpers.html`](demos/01-helpers.html) | Plain ADR-097 helpers in the point-cloud viewer | +| 02 | [`demos/02-cinematic.html`](demos/02-cinematic.html) | Cinematic camera + pseudo-CSI visualization on top of #01 | +| 03 | [`demos/03-skinned.html`](demos/03-skinned.html) | GLTF skinned mesh + additive animation blending | +| 04 | [`demos/04-skinned-fbx.html`](demos/04-skinned-fbx.html) | Mixamo X Bot loaded from FBX in the ADR-097 scene | +| 05 | [`demos/05-skinned-realtime.html`](demos/05-skinned-realtime.html) | Webcam → MediaPipe Pose Heavy → Mixamo IK retarget, live ESP32 CSI overlay | + +| Screenshot | | +|---|---| +| ![01](screenshots/01-helpers.png) | ![02](screenshots/02-cinematic.png) | +| ![03](screenshots/03-skinned.png) | ![04](screenshots/04-skinned-fbx.png) | +| ![05](screenshots/05-skinned-realtime.png) | | + +## Layout + +``` +examples/three.js/ +├── README.md +├── .gitignore +├── demos/ # 5 self-contained HTML demos +│ ├── 01-helpers.html +│ ├── 02-cinematic.html +│ ├── 03-skinned.html +│ ├── 04-skinned-fbx.html +│ └── 05-skinned-realtime.html +├── screenshots/ # one PNG per demo +│ └── 0N-*.png +├── server/ +│ ├── serve-demo.py # local HTTP server with no-cache headers +│ └── ruvultra-csi-bridge.py # ESP32 CSI WebSocket bridge (ruvultra:8766) +└── assets/ + └── X Bot.fbx # gitignored — get your own from mixamo.com + # (FBX Binary, T-Pose, Without Skin) + # used by demos 04 and 05 +``` + +## Mixamo X Bot + +Demos 04 and 05 expect `assets/X Bot.fbx`. It's gitignored (size + license +boundary). Download yours from [mixamo.com](https://mixamo.com): pick the +"X Bot" character, export as **FBX Binary**, **T-Pose**, **Without Skin**, +and drop it into `assets/`. + +## Live ESP32 CSI overlay (demo 05 only) + +`server/ruvultra-csi-bridge.py` is the systemd-deployable bridge that runs on +the `ruvultra` host (over Tailscale). It listens for ESP32-S3 CSI on UDP and +re-broadcasts it as WebSocket frames at `ws://ruvultra:8766/csi`. Demo 05 +auto-connects; if the socket is down, it falls back to the bundled idle clip +plus a synthetic CSI driver. + +## Open issues + +- [#583](https://github.com/ruvnet/RuView/issues/583) — head/face tracking + fidelity in `05-skinned-realtime.html`. Recommended fix: swap MediaPipe + Pose Heavy for MediaPipe Holistic (same API, adds 468-point face mesh + + hand landmarks for proper PnP head pose and finger curl tracking). diff --git a/three.js/assets/README.txt b/three.js/assets/README.txt new file mode 100644 index 00000000..4d3c1d83 --- /dev/null +++ b/three.js/assets/README.txt @@ -0,0 +1,7 @@ +The Mixamo "X Bot.fbx" required by demos 04-skinned-fbx.html and +05-skinned-realtime.html is intentionally not redistributed here. + +Download your own from https://mixamo.com (FBX Binary, T-Pose, +Without Skin) and place it here as "X Bot.fbx" if you want to +run those demos locally. See examples/three.js/README.md in the +repo for context. diff --git a/three.js/demos/01-helpers.html b/three.js/demos/01-helpers.html new file mode 100644 index 00000000..e915fd13 --- /dev/null +++ b/three.js/demos/01-helpers.html @@ -0,0 +1,587 @@ + + + + + + RuView · ADR-097 · three.js helpers in the point cloud viewer + + + + + + +
+

RuView · Helpers Demo

+
ADR-097 · three.js helpers for the point cloud viewer
+
Scene● SYNTHETIC
+
Skeleton17 kpts · COCO
+
Point cloud— pts
+
Sensor nodes4 · multistatic
+
Frame rate— fps
+
Bbox volume— m³
+
+ +
+

Helpers

+ + + + + +
+ +
+

Scene

+
COCO-17 keypoints (yellow)
+
Bones (white lines)
+
Face point cloud (cyan→white)
+
ESP32 sensor nodes
+
+ +
+ ADR-097 · three.js helpers +
+ + + + diff --git a/three.js/demos/02-cinematic.html b/three.js/demos/02-cinematic.html new file mode 100644 index 00000000..0ec1ff82 --- /dev/null +++ b/three.js/demos/02-cinematic.html @@ -0,0 +1,1034 @@ + + + + + + RuView · Cinematic · ADR-097 helpers + pseudo-CSI visualization + + + + + + + + + + + + +
+
+
+ +
+

RuView · Cinematic

+
ADR-097 · pseudo-CSI visualization layer
+
Subject● Tracked
+
Posewalking
+
Heart rate— bpm
+
Breathing— bpm
+
Mesh nodes4 · multistatic
+
Coherence— %
+
Tomographyidle
+
Bbox vol— m³
+
Render— fps
+
+ +
+

Helpers · ADR-097

+ + + + + + + +
+ +
+

Per-node CSI · synthetic

+
N1·BL
—
+
N2·BR
—
+
N3·FL
—
+
N4·FR
—
+
CSI amplitude derived from distance-to-keypoint + Doppler + thermal noise. Drives bone glow, ping coherence, tomography trigger threshold.
+
+ +
+ RuView · Seldon Vault +
multistatic wifi pose · ADR-097
+
+ +
+ ADR-097 · three.js helpers · cinematic +
+ + + + diff --git a/three.js/demos/03-skinned.html b/three.js/demos/03-skinned.html new file mode 100644 index 00000000..2d2ea73b --- /dev/null +++ b/three.js/demos/03-skinned.html @@ -0,0 +1,854 @@ + + + + + + RuView · Skinned · ADR-097 + GLTF skinned mesh + additive animation blending + + + + + + + + + + + + + +
+
+
+ +
▸ Loading skinned subject · Xbot.glb · 2.9 MB
+ +
+

RuView · Skinned

+
ADR-097 · GLTF skinned mesh · additive animation blending
+
Subject● Tracked
+
ModelXbot.glb · 14k tris
+
Base animwalk
+
AdditiveheadShake · 0.40
+
Mesh nodes4 · multistatic
+
Coherence— %
+
Heart rate— bpm
+
Bbox vol— m³
+
Render— fps
+
+ +
+

AnimationMixer

+
+
Base · loops
+ + + +
+
+
Additive · layered
+ + + + +
+
+
+ add weight + + 0.40 +
+
+ time scale + + 1.00 +
+
+
+ +
+

Per-node CSI

+
N1·BL
—
+
N2·BR
—
+
N3·FL
—
+
N4·FR
—
+
+ +
+

ADR-097 helpers

+ + + + + + + +
+ +
+ RuView · Seldon Vault +
skinned · ADR-097 · CCDIKSolver next
+
+ +
+ additive blend + skinning IK +
+ + + + diff --git a/three.js/demos/04-skinned-fbx.html b/three.js/demos/04-skinned-fbx.html new file mode 100644 index 00000000..8353c015 --- /dev/null +++ b/three.js/demos/04-skinned-fbx.html @@ -0,0 +1,961 @@ + + + + + + RuView · Skinned (FBX) · Mixamo X Bot in the ADR-097 helpers scene + + + + + + + + + + + + + + + +
+
+
+
▸ Loading skinned subject · X Bot.fbx
+ +
+

RuView · Skinned (FBX)

+
ADR-097 · Mixamo X Bot · loaded via FBXLoader
+
Subject● Tracked
+
SourceX Bot.fbx
+
FormatFBX 7700 · 1.75 MB
+
Bones—
+
Animation—
+
Mesh nodes4 · multistatic
+
Coherence— %
+
Heart rate— bpm
+
Bbox vol— m³
+
Render— fps
+
+ +
+

AnimationMixer

+
+ clips + +
+
+ time scale + + 1.00 +
+ +
+ +
+

Per-node CSI

+
N1·BL
—
+
N2·BR
—
+
N3·FL
—
+
N4·FR
—
+
+ +
+

ADR-097 helpers

+ + + + + + + + +
+ +
+ RuView · Seldon Vault +
FBXLoader · Mixamo · ADR-097
+
+ + + + diff --git a/three.js/demos/05-skinned-realtime.html b/three.js/demos/05-skinned-realtime.html new file mode 100644 index 00000000..a366096b --- /dev/null +++ b/three.js/demos/05-skinned-realtime.html @@ -0,0 +1,2141 @@ + + + + + + + + + + RuView · Skinned Realtime · MediaPipe Pose → Mixamo IK retargeting + + + + + + + + + + + + + + + + + +
+
+
+
▸ Loading skinned subject · X Bot.fbx
+ +
+

RuView · Skinned Realtime

+
MediaPipe Pose → Mixamo direct-retargeting · live CSI from real keypoints
+
Subject● Idle (no webcam)
+
SourceX Bot.fbx · 1.75 MB
+
Bones—
+
Pose trackeridle
+
Tracking conf— %
+
Retargets0 / 12
+
RSSI / Wrist L— m
+
Yield / Wrist R— m
+
Bbox vol— m³
+
Render— fps
+
+ +
+

MediaPipe Pose

+
Webcam disabled
+
+ + +
+
+
Landmarks0 / 33
+
Visible0 / 33
+
Pose fps— fps
+
+ + + +
build 2026-05-15-fps-tune · default Holistic@Full 20fps · ?cnn=2 ?infer=30 to crank
+
+ +
+

Per-node CSI · LIVE

+
N1·BL
—
+
N2·BR
—
+
N3·FL
—
+
N4·FR
—
+
connecting to ESP32-S3 via ruvultra (Tailscale ws://100.104.125.72:8766)…
+
+ +
+

ADR-097 helpers

+ + + + + + + + +
+ +
+ RuView · Seldon Vault +
Live · MediaPipe Pose · Mixamo retarget
+
+ + + + diff --git a/three.js/index.html b/three.js/index.html new file mode 100644 index 00000000..31567377 --- /dev/null +++ b/three.js/index.html @@ -0,0 +1,168 @@ + + + + + + +RuView · three.js demos · ADR-097 sensing-helpers scene + + + +
+ +

RuView · three.js demos

+

+ Five progressively richer browser demos of the ADR-097 + sensing-helpers scene, ending with a live MediaPipe-Pose → Mixamo X Bot retargeting pipeline driven + by a real ESP32 CSI feed. +

+ +
+ + +

01 · Helpers standalone

+
Plain ADR-097 helpers in the point-cloud viewer. No external assets.
+ 01 screenshot + + + +

02 · Cinematic standalone

+
Cinematic camera + pseudo-CSI visualization on top of #01.
+ 02 screenshot + + + +

03 · Skinned (GLTF) standalone

+
GLTF skinned mesh + additive animation blending in the ADR-097 scene.
+ 03 screenshot + + + +

04 · Skinned FBX needs FBX

+
Mixamo X Bot via FBXLoader. Requires a local assets/X Bot.fbx.
+ 04 screenshot + + + +

05 · Realtime (Pose + CSI) needs FBX

+
Webcam → MediaPipe Pose Heavy → Mixamo IK retarget, live ESP32 CSI overlay.
+ 05 screenshot + + +
+ +
+ Demos 04 and 05 need a Mixamo asset. The Mixamo + X Bot.fbx file is intentionally not redistributed in + this deployment — it's licensed for end-users to download from + mixamo.com directly. + To run these locally: clone the repo, download X Bot.fbx + (FBX Binary, T-Pose, Without Skin) into + examples/three.js/assets/, then run + python examples/three.js/server/serve-demo.py. +
+ +
+ Source: github.com/ruvnet/RuView/tree/main/examples/three.js +  ·  ADR-097 · three.js r128 +
+ +
+ + diff --git a/three.js/screenshots/01-helpers.png b/three.js/screenshots/01-helpers.png new file mode 100644 index 00000000..4755e996 Binary files /dev/null and b/three.js/screenshots/01-helpers.png differ diff --git a/three.js/screenshots/02-cinematic.png b/three.js/screenshots/02-cinematic.png new file mode 100644 index 00000000..c4737196 Binary files /dev/null and b/three.js/screenshots/02-cinematic.png differ diff --git a/three.js/screenshots/03-skinned.png b/three.js/screenshots/03-skinned.png new file mode 100644 index 00000000..713aa593 Binary files /dev/null and b/three.js/screenshots/03-skinned.png differ diff --git a/three.js/screenshots/04-skinned-fbx.png b/three.js/screenshots/04-skinned-fbx.png new file mode 100644 index 00000000..b747f6dc Binary files /dev/null and b/three.js/screenshots/04-skinned-fbx.png differ diff --git a/three.js/screenshots/05-skinned-realtime.png b/three.js/screenshots/05-skinned-realtime.png new file mode 100644 index 00000000..58268eca Binary files /dev/null and b/three.js/screenshots/05-skinned-realtime.png differ