berkus
/
ANE
mirror of https://github.com/maderix/ANE.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Add tokenizer via git-lfs, fix dashboard tokenizer path 

- Add tokenizer.bin (434KB) to assets/models/ via git-lfs
- Fix dashboard tokenizer path (was one parent too many)
This commit is contained in:
maderix 2026-03-05 07:41:33 -08:00
parent 926f977b40
commit 9595b1a499
3 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitattributes vendored Normal file
View File

@ -0,0 +1,2 @@
assets/models/*.bin filter=lfs diff=lfs merge=lfs -text
assets/data/*.bin filter=lfs diff=lfs merge=lfs -text

3
assets/models/tokenizer.bin Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:50a52ef822ee9e83de5ce9d0be0a025a773d019437f58b5ff9dcafb063ece361
size 433869

2
training/dashboard.py
View File

@ -23,7 +23,7 @@ HD = DIM // HEADS
CKPT_PATH_STATIC = 'ane_stories110M_ckpt.bin'
CKPT_PATH_DYNAMIC = 'training_dynamic/ane_stories110M_dyn_ckpt.bin'
CKPT_PATH = CKPT_PATH_STATIC # set in main() based on --dynamic
TOKENIZER_PATH = str(Path(__file__).resolve().parent.parent.parent / 'assets' / 'models' / 'tokenizer.bin')
TOKENIZER_PATH = str(Path(__file__).resolve().parent.parent / 'assets' / 'models' / 'tokenizer.bin')
class State: