🏗️ Building on HF

Dipankar Sarkar PRO

dipankarsarkar

4 714 143

https://www.dipankar.cc

AI & ML interests

Building the AI-native stack. Agents as infrastructure, safety as architecture, performance as plumbing. I publish the receipts: papers, datasets, demos.

Recent Activity

upvoted a paper about 1 hour ago

The Problem Is the Problem: Towards Scalable Mathematical Discovery

reacted to nightmedia's post with 🔥 about 2 hours ago

Qwen3.8-27B metrics It's hard to track all model cards where I post these, so I figured people would get more value out of seeing these in the open. ```brainwaves quant arc arc/e boolq hswag obkqa piqa wino mxfp8 0.591,0.782,0.896,0.746,0.448,0.801,0.711 q8-hi 0.602,0.779,0.896,0.747,0.446,0.793,0.703 q6-hi 0.602,0.775,0.895,0.748,0.448,0.795,0.710 q4-hi 0.604,0.780,0.898,0.744,0.454,0.795,0.708 mxfp4 0.581,0.771,0.889,0.738,0.442,0.798,0.713 Quant Perplexity Peak Memory Tokens/sec mxfp8 6.090 ± 0.054 34.74 GB 138 mxfp4 5.952 ± 0.051 21.30 GB 148 ``` The performance is as measured on a M4 MBP 128GB, speed may vary depending on your platform. These are all instruct metrics, generated by including this line in the jinja template: ``` {%- set enable_thinking = false %} ``` Then run the test suite to generate the metrics: ``` mlx_lm.evaluate --model MODEL --tasks winogrande boolq arc_challenge arc_easy hellaswag openbookqa piqa ``` This will generate the file: ``` eval_MODEL_0.4.9_winogrande_boolq_arc_challenge_arc_easy_hellaswag_openbookqa_piqa ``` This is a JSON containing all gathered metrics; for example the q4-hi: ``` "arc_challenge": { "alias": "arc_challenge", "acc,none": 0.5819112627986348, "acc_stderr,none": 0.014413988396996116, "acc_norm,none": 0.6040955631399317, "acc_norm_stderr,none": 0.01429122839353657 }, ``` I use the value of acc_norm for metrics, rounded to 3 decimals. As I get more quants tested, I will add them here. A complete test run for a single quant takes 7-9 hours depending on quant size, 10-12 hours for BF16 depending on perplexity: this is why you see on my model cards that I usually post the first three, that only take 2-3 hours :) -G

liked a dataset about 3 hours ago

issdandavis/prompt-injection-bit-signatures

View all activity

Organizations

Posts 2

Post

487

Your issue tracker is in the wrong place.

It lives on a server. Your code lives in git. So every time an agent picks up work it makes an API call, burns a token, fights a rate limit, and still cannot see what the other agent just did.

Move the issues into the repo. Append-only event log in git refs. Branches when you branch, merges when you merge, CRDT so two agents never conflict. No server, no database.

The coordination signal that PR-level telemetry misses lives before the pull request. The paper, and a live demo running the real tool:

Before the Pull Request: Mining Multi-Agent Coordination (2606.19616)
https://huggingface.co/spaces/neullabs/grite

If your agents share a repo, where does their shared state actually live right now?