# Local LLM

- Page: https://shared.btbytes.com/p/local-llm
- Date: 2023-11-30
- Tags: llm, local-llm

## Local Coding

[How to Setup a Local Coding Agent on macOS - Kyle Howells](https://ikyle.me/blog/2026/how-to-setup-a-local-coding-agent-on-macos)

> running Gemma 4 26B-A4B and Qwen3.6 35B-A3B locally with llama.cpp, MTP speculative decoding, multimodal support, and PI as a coding agent.

## Local LLMs

[GitHub - jamesob/local-llm](https://github.com/jamesob/local-llm): Everything I know about running LLMs locally.

[GitHub - ml-explore/mlx-lm: Run LLMs with MLX · GitHub](https://github.com/ml-explore/mlx-lm/tree/main); May 2026. [full setup](https://x.com/Michaelzsguo/status/2057117647607632081/photo/1) -- `mlx_lm.server` + [Aperture by Tailscale Aperture](https://tailscale.com/docs/aperture) + Tailscale [funnel](https://tailscale.com/docs/features/tailscale-funnel).

[Running local models on an M4 with 24GB memory \| jola.dev](https://jola.dev/posts/running-local-models-on-m4); May 2026.

[Introducing llamafile](https://hacks.mozilla.org/2023/11/introducing-llamafile/) "llamafile lets you turn large language model (LLM) weights into executables."

[HN Discussion of llamafile](https://news.ycombinator.com/item?id=38464057) has some cool application of this given by `simonw` -- [llamafile is the new best way to run a LLM on your own computer](https://simonwillison.net/2023/Nov/29/llamafile/)

[LMStudio + FreeChat](https://x.com/chakkaradeep/status/1730077126282612744)
- [Freechat](https://www.freechat.run) "Local, secure, open source AI chat for macOS"
- [LMStudio](https://lmstudio.ai) "Discover, download, and run local LLMs"


[Jan: Open source ChatGPT-alternative that runs 100% offline](https://jan.ai/); has to see how it compares to ollama; `brew install --cask jan`
- has local api server
- connect to cloud api servers like OpenAI

[Experimenting with local LLMs on macOS | Fatih's Personal Blog](https://blog.6nok.org/experimenting-with-local-llms-on-macos/); Sep 2025 -  covers some of the above stuff, but also enumerates different decision points like: model size, runtime, quantization, vision models, reasoning, tool use, agents ...

[Setting up local LLMs for R and Python](https://posit.co/blog/setting-up-local-llms-for-r-and-python/) by Posit; Aug 2025. This is an introductory tutorial for R and Python on how to setup ollama and then use `ellmer` and `chatlas` packages for R, and [python](https://shared.btbytes.com/p/python) respectively to connect to it.

---

Note on [Deepseek v2](https://www.reddit.com/r/LocalLLaMA/comments/1enb0p2/shout_out_to_deepseek_v2/); Aug 8, 2024


---

[Sep 26, 2025](https://x.com/0x_Sero/status/1971291593056682105)

1. GLM-4.5-air: best agentic/coding model that runs on consumer hardware at very decent speeds. Rivals Claude 4-sonnet.
2. 2. Nousresearch/hermes-70B: the only model that will do whatever you ask, and tell you whatever you want to know. Literally critical to have.
3. 3. GPT-OSS-120B: very intelligent it’s like having 4o at home, great context window, great agent
4. 4. Qwen3-coder-30B-3A-instruct: very good coding agent, excellent workhorse, incredibly fast
5. 5. Mistral-magistral-small: very fast, excellent agent, great coder, multimodal, punches way, way, above its size.

[Sep 26, 2025](https://x.com/awnihannun/status/1971603427131351218) about [Cline + LM Studio: the local coding stack with Qwen3 Coder 30B - Cline Blog](https://cline.bot/blog/local-models) "Qwen3 Coder + Cline + LM Studio for high-quality local coding"

Also mentioned: [facebook/cwm · Hugging Face](https://huggingface.co/facebook/cwm) -- llama3 + long-context modifications (alternating attention + large RoPE scaling). MLX should be supported out of the box (verify).

[You Don’t Need to Spend $100/mo on Claude Code: Your Guide to Local Coding Models](https://www.aiforswes.com/p/you-dont-need-to-spend-100mo-on-claude); Dec 2025. [HN](https://news.ycombinator.com/item?id=46348329)