EVERY MODEL I RAN ON MY MAC MINI LAST WEEK 1) @thewebAI's TwIL-8B 4-bit MLX: 53.8 tok/s 2) @AlibabaQwen's…
@pratikg
EVERY MODEL I RAN ON MY MAC MINI LAST WEEK
@thewebAI's TwIL-8B 4-bit MLX: 53.8 tok/s http://x.com/pratikg/status/2118146597884502486
@Alibaba_Qwen's Qwen3.8 27B: 16.85 tok/s (56.43 tok/s with TensorFold!) http://x.com/pratikg/status/2117801754989592866
@NousResearch's Arete-27B (finetune of above): 17.2 tok/s (TensorFold) & 17.7 tok/s (MLX @ 512) http://x.com/pratikg/status/2119251866198610314
@RicursiveAI's Ripple-65B Preview: ~6.9-7.0 tok/s http://x.com/pratikg/status/2118456841659375670
@MacrocosmosAI's IOTA Decide 4B: ~90 ms/decision (short) http://x.com/pratikg/status/2118194907974357370
@NaceAI's Drex Max v1 (Kapitsa): 0.95 s/decision (507 tokens) http://x.com/pratikg/status/2119129766787981743
All models were run locally on my M5 Pro Mac Mini with 64GB of RAM.
I was surprised how well @tensorfold_dev (by @ashxhart) performed on the 27B model (almost 3.4x the speed of the same model without TensorFold) and that the Arete finetune didn’t seem to slow things down at all relative to the base model it was finetuned from.
Excited to try out Qwen 4 as soon as weights drop and Inkling 2 Small when it’s released. And more.
Let me know what you all think I should run!
EVERY MODEL I RAN ON MY MAC MINI LAST WEEK
@thewebAI's TwIL-8B 4-bit MLX: 53.8 tok/s http://x.com/pratikg/status/2118146597884502486
@Alibaba_Qwen's Qwen3.8 27B: 16.85 tok/s (56.43 tok/s with TensorFold!) http://x.com/pratikg/status/2117801754989592866
@NousResearch's Arete-27B (finetune of above): 17.2 tok/s (TensorFold) & 17.7 tok/s (MLX @ 512) http://x.com/pratikg/status/2119251866198610314
@RicursiveAI's Ripple-65B Preview: ~6.9-7.0 tok/s http://x.com/pratikg/status/2118456841659375670
@MacrocosmosAI's IOTA Decide 4B: ~90 ms/decision (short) http://x.com/pratikg/status/2118194907974357370
@NaceAI's Drex Max v1 (Kapitsa): 0.95 s/decision (507 tokens) http://x.com/pratikg/status/2119129766787981743
All models were run locally on my M5 Pro Mac Mini with 64GB of RAM.
I was surprised how well @tensorfold_dev (by @ashxhart) performed on the 27B model (almost 3.4x the speed of the same model without TensorFold) and that the Arete finetune didn’t seem to slow things down at all relative to the base model it was finetuned from.
Excited to try out Qwen 4 as soon as weights drop and Inkling 2 Small when it’s released. And more.
Let me know what you all think I should run!
