berkus
/
ANE
mirror of https://github.com/maderix/ANE.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
39 Commits 3 Branches 0 Tags 1.1 MiB
Commit Graph

5 Commits

Author SHA1 Message Date
maderix 475348ad14 Add Qwen3-0.6B GQA support and multi-model build system 
Implement Grouped-Query Attention (16q/8kv heads, head_dim=128) for
Qwen3-0.6B (28 layers, 596M params). Model configs moved to
models/*.h headers selected at build time via make MODEL=xxx.

Key changes:
- GQA-aware MIL kernels: sdpaFwd split from woFwd (Q_DIM!=DIM),
  qBwd/kvBwd split from qkvBwd (different IC dimensions)
- K/V tile (KV_HEADS→HEADS) before SDPA backward, reduce after
- 10 kernels total, all model-agnostic via compile-time defines
- Makefile: make MODEL=qwen3_06b (default) or MODEL=stories110m
- Both models verified: Stories110M ~115ms/step, Qwen3 ~412ms/step
 2026-03-06 06:23:15 -08:00
nabbilkhan c04168ee17 Add --data path support for static training pipelines 2026-03-03 19:19:49 +00:00
maderix 4c14ed0e25 CLI fixes + --no-ane-extras flag + README benchmark table 
- Fix positional arg parsing (model_path, steps, lr were silently ignored)
- Add --model, --ckpt flags; forward ckpt_path across exec() restarts
- Add --no-ane-extras to disable ANE classifier/softmax/rmsnorm_bwd
- CPU fallback for softmax/classifier/rmsnorm_bwd when extras disabled
- Update README with 4-way benchmark comparison table (20 steps)
 2026-03-03 04:34:55 -08:00
Vipul ebac5dd73f Python Bridge+Memory leak fix+More functions 2026-03-03 02:04:36 -05:00
maderix 4d67db1bdb stories110M: 12-layer ANE training with dashboard, 107ms/step 
- Scale to full stories110M (109M params, 12 layers) with real TinyStories data
- vDSP-vectorized cross-entropy (110ms→14ms), NEON fp16 IO, async dW
- TUI dashboard: loss curve, ANE/CPU power, CPU/memory graphs, text generation
- Split into modular headers: config, io, mil, cpu_ops
 2026-03-01 03:14:39 -08:00