The structural advantage that's the entire point of ADR-096: O(log T)
per new token via decode_step against an accumulated KvCache, vs
O(N²) recompute for dense MHA. This commit lands the API and proves
the numerical equivalence at the last position.
API:
- AetherTemporalHead::step(q_new, k_new, v_new, &mut cache)
Single-token decode. Appends (k_new, v_new) to cache, runs
decode_step(q_new) against the now-updated cache, returns the new
position's output.
- AetherTemporalHead::make_cache(capacity)
Convenience constructor — caller doesn't need to import
ruvllm_sparse_attention to size a cache. Per ADR-096 §8.5 the
natural lifetime is per-PoseTrack (re-ID) or per-session (online
classification); when the track drops, drop the cache.
- KvCache re-exported at the crate root.
Contract:
- q_new/k_new/v_new must each have seq == 1. Multi-token q is the
prefill path (forward), not decode_step.
- Cache lifetime is the caller's. The crate enforces shape via
make_cache so callers can't mismatch kv_heads / head_dim / block_size.
- KvCache fill is the caller's problem. Upstream H2O heavy-hitter
eviction is opt-in; this crate's wrapper doesn't pre-pick a policy.
Tests (18/18 total now passing):
- streaming_step_matches_forward_at_last_position — central claim:
16-token sequence, append k/v one at a time via step(), compare
the streamed last-token output to forward(full Q,K,V)[N-1].
max_abs_err < 1e-3 (currently passes well under that bound for
the 0.1-magnitude activations the test uses).
- step_rejects_multi_token_q — contract enforcement.
- make_cache_returns_kvcache_with_correct_shape — wiring smoke,
confirms (capacity, kv_heads, dim, block_size) ordering is correct
through the make_cache wrapper.
Test config uses MHA shape (q_heads == kv_heads) because the upstream
decode_step is wired to the MHA branch; the GQA decode path is on
upstream's roadmap and lands in a separate ADR-096 follow-up when it
does.
Co-Authored-By: claude-flow <ruv@ruv.net>
49e57efcec