Model Submission Request: CLOD-149M (plz)
#240
by Murzik344 - opened
Repo: https://huggingface.co/Murzik344/CLOD-149M
- 148,898,078 params
- pretrained from scratch by me with the nanochat harness
- open weights
- Apache-2.0
- safetensors on HF.
- Weights are the genuine final training checkpoint (step 38,180), not a merge.
Recipe:
20B tokens - 18.2B ClimbMix-400B (single epoch) + ~10% terminal procedural-arithmetic anneal at LR floor; custom 32k digit-split BPE; one RTX 5050 Laptop GPU, 8 days.
Results (zero-shot, acc_norm; lm-eval-style likelihood scoring; ArithMark-3.0 official dataset):
HellaSwag 34.86
ARC-Easy 45.37
ARC-Challenge 26.88
PIQA 67.08
ArithMark-3 36.00
=> Int Index 19.64 by the published formula.
Reproducibility:
- token_bytes.pt ships in the repo for byte-exact bpb checks
- full nanochat config embedded in config.json.
Cool model, do you mind making a pull request on the leaderboard with the changes added?
Datdanboi25 changed discussion status to closed