here you goThings I’ve made, read and saved — shared here.

Archive

9,565 entries: 46 from X, 8,923 from Pinboard, saved 1998 to 2026.

Browse by tag, site, author, folder or language

Tags

Sites

Authors

Folders

Languages

Showing 1–50 of 81#opensource ×

October 2026

April 2026

2025

Sky-T1: Train your own O1 preview model within $450

novasky-ai.github.io

Stanford's Sky Lab announced the "Sky-T1" model that performs competitively in reasoning and coding while comparison to OpenAI's o1 model, while using a small compute budget.

They set about to produce this open-weights model because the current leaders like o1 and gemini 2.0 flash thinking while excel in reasoning, showing the ability to solve complex tasks by producing a long internal chain of thought, among other advancements. However, the technical details and model weights are un-accessible.

Summary:

  • They used QwQ to generate training data (with some cleanup using GPT-4o-mini)
  • The training data was then used to FT Qwen2.5-32B-Instruct (non-reasoning model)
  • Result was that Sky-T1 performs slightly worse than QwQ but much better than Qwen2.5 on reasoning tasks.

This is pretty interesting as it shows how you can FT a foundation model to do better at reasoning.


Inference time compute is still underutilized for AI deployments. It is better to distill the reasoning from larger models like R1 for specific tasks. Even better, one can mix in custom thinking instructions for specific sub-problems, so that a fine-tuned model learns o mix of task specific reasoning and custom logic. This can often beat custom prompt iteration. 🔗

This Guide: Train a Reasoning Model | Kiln AI Docs goes into how to distill for thinking models.

DeepSeek

DeepSeek is a AI research company from China that has been doing cutting edge work in the area of Large Language Models, including DeepSeek-R1.

DeepSeek tried to do one thing: transplant its own model to Huawei Ascend 910B chip to run. Through the "dynamic precision adjustment" technology, they only lost 5% of the performance in the same task, but the cost dropped by 70%. via

notes#llm#opensource

2024

llama

llama homepage The latest, as on Jul 2024, is llama 3.1 which has 405B, 70B and 8B models, optimized for "do it all", cost/performance tradeoff, and light-weight+speed.

You can run llama locally on your laptop or pc using the following tools:

  1. ollama + OpenWebUI;
    1. ollama is an open-source project that provides an easy way to download and run local LLMs
    2. OpenWebUI gives you a web-based ChatGPT like user interface to interact with the local LLMs.
  2. LM Studio - is a desktop app
  3. GPT4All is an open-source ecosystem for running LLMs locally on your computer.
  4. MLX; optimized to run LLMs on Apple Silicon on macOS.
llama stack
llama stack APIs
[via](https://x.com/shrihacker/status/1838996780236812638)

notes#llama#llm#opensource

2023

Opensource LLM

GitHub - allenai/awesome-open-source-lms: Friends of OLMo and their links.

Papers

Appears to be an exhaustive survey paper -- (Chen et al. 2023)

(Zhuang et al. 2024)

ChuXin, an entirely open-source language model with a size of 1.6 billion parameters. Unlike the majority of works that only open-sourced the model weights and architecture, we have made everything needed to train a model available, including the training data, the training process, and the evaluation code. Our goal is to empower and strengthen the open research community, fostering transparency and enabling a new wave of innovation in the field of language modeling. Furthermore, we extend the context length to 1M tokens through lightweight continual pretraining and demonstrate strong needle-in-a-haystack retrieval performance. The weights for both models are available at Hugging Face to download and use.

References

notes#llm#opensource

2022

2014

2013

2011

2010

2009

Arduino - HomePage

arduino.cc

Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It's intended for artists, designers, hobbyists, and anyone interested in creating interactive objects or environments.

2008

2007

2006