Jackson Yeskie

@RakshithFury I'm training a model right now and by 18b tokens it was able to do division, carrying, and even some multiplication that would be difficult in your head. 1000% It just depends on your data.

@jiteshkprasad For your next model you should focus on data, if you are just training on random scrapes, or datasets that are extremely old, you should look to much more quality, newer datasets. At this scale training on large, unfiltered dumps is only worth it if you NEED data at the trillions

1,000× less training data than Qwen2.5-0.5B and accuracy almost equal. Have you benchmaxed it?

Seems so. I tried it out and it couldn't even make a script that combines a + b and returns their outputs or even reverse a string. It also, as the post says, is extremely bad at math. Even on their website the prompt 'Explain in simple words why the sky is blue.' that it tells you just incorrect stuff consistently. Its SFT is sort of good in how it looks with certain prompts but is very bad at instruction following. For example: 'Give me 9 points on a bullet point list about gardening.' produces is only actually 9 points sometimes.

why did you use a repetition penalty of 1?

Honestly for the size not bad

BananaMind-2.1-NanoCoder I'm just realizing this now but, did you do SFT on these models, I only see base training data in the readme. If you do, could you tell me which model to try of yours that does?

Wonder if they would raise the prices if this is actually added

Yeah, and I think spending more time on one model would be better than releasing a bunch of almost unusable ones at once like they've been doing. I just tried their coding model and it cant do anything at all, not even making a script to reverse a string in python or any sort of SUPER basic debugging

Hey, we still got at least a year or two..