Archive

Development containers

A development container (or dev container for short) allows you to use a container as a full-featured development environment. It can be used to run an application, to separate tools, libraries, or runtimes needed for working with a codebase, and to aid in continuous integration and testing. Dev containers can be run locally or remotely, in a private or public cloud, in a variety of supporting tools and editors.

"Build modern full-stack applications on AWS". SST example

See also: Nitric Cloud-Native Framework | Get Infrastructure from Code -- "The cloud aware application framework"

Nitric is a multi-language framework for rapidly building cloud applications that uses infrastructure automation to reduce boilerplate, prevent drift and enable applications to run on any cloud provider – such as AWS, Google Cloud or Microsoft Azure. Nitric streamlines cloud development from local dev iteration through to testing and deployment.

Claude Code Docs, Guides & Best Practices | ClaudeLog via @rickasaurus

Some good examples of Claude Artifacts to automate common actions

anthropics/anthropic-cookbook: A collection of notebooks/recipes showcasing some fun and effective ways of using Claude. -- via simonw.

Writing a good CLAUDE.md | HumanLayer Blog

• CLAUDE.md goes into every single session with Claude code. use it wisely.
• shorter is better
• less than 300 lines
• use progressive disclosure -- "keeping task-specific instructions in separate markdown files with self-descriptive names somewhere in your project."
• don't autogenerate claude.md. Write it youself, keep it short, put only the most important instructions into it.

Claude Design

See HTML

C3 programming language. Full C ABI compatibility, Module system, Generic modules, Design by contract, Zero overhead errors, Semantic macro system, First-class SIMD vector types, Struct subtyping, Safe array access using slices, Easy to use inline assembly, Cross-platform standard library which includes dynamic containers and strings, LLVM backend.

C3 Tutorial; has online playground.

C3 Blog by the creator, Christoffer Lernö

Also look at dlang's betterC - Better C - D Programming Language

llama homepage The latest, as on Jul 2024, is llama 3.1 which has 405B, 70B and 8B models, optimized for "do it all", cost/performance tradeoff, and light-weight+speed.

You can run llama locally on your laptop or pc using the following tools:

1. ollama + OpenWebUI;
   1. ollama is an open-source project that provides an easy way to download and run local LLMs
   2. OpenWebUI gives you a web-based ChatGPT like user interface to interact with the local LLMs.
2. LM Studio - is a desktop app
3. GPT4All is an open-source ecosystem for running LLMs locally on your computer.
4. MLX; optimized to run LLMs on Apple Silicon on macOS.

[via](https://x.com/shrihacker/status/1838996780236812638)

[@10.14778/3611479.3611527]; PDF

We argue that the disruptive influence that LLMs will have on data management will come from two angles. (1) A number of hard database problems, namely, entity resolution, schema matching, data discovery, and query synthesis, hit a ceiling of automation because the system does not fully understand the semantics of the underlying data. Based on large training corpora of natural language, structured data, and code, LLMs have an unprece- dented ability to ground database tuples, schemas, and queries in real-world concepts. We will provide examples of how LLMs may completely change our approaches to these problems. (2) LLMs blur the line between predictive models and information retrieval systems with their ability to answer questions. We will present examples showing how large databases and information retrieval systems have complementary functionality.

(Shi et al. 2024)

writing SQL queries requires specialized knowledge, which poses a challenge for non-professional users trying to access and query databases. Text-to-SQL parsing solves this issue by converting natural language queries into SQL queries, thus making database access more accessible for non-expert users. To take advantage of the recent developments in Large Language Models (LLMs), a range of new methods have emerged, with a primary focus on prompt engineering and fine-tuning. This survey provides a comprehensive overview of LLMs in text-to-SQL tasks, discussing benchmark datasets, prompt engineering, fine-tuning methods, and future research directions.

References

• Shi, Tang, Yang (2024). A Survey on Employing Large Language Models for Text-to-SQL Tasks.

Apache Pinot™ -- "The real-time analytics open source platform for lightning-fast insights, effortless scaling, and cost-effective data-driven decisions."

What is Apache Pinot? | Apache Pinot 101 - YouTube by Tim Berglund. latency, concurrency, freshness.

15 Years of Realtime OLAP (Part 1) - by Chris Riccomini

Pinot used columnar storage and in-memory buffering. But they also included a new feature: star-tree indexes. Star-trees give operators a knob to tune, trading off space for query latency. Though data was stored in columnar files as with Druid, for very fast (single-digit millisecond) reads, data could be partially pre-materialized. Pinot became widely adopted within LinkedIn and Uber for internal business intelligence, user-facing analytics features, log analytics, metrics, and more.

Keynote: Kishore Gopalakrishna (CEO, StarTree) Mission Critical Data Can't Wait | RTA Summit '24 - YouTube

Wes McKinney - The Road to Composable Data Systems: Thoughts on the Last 15 Years and the Future; (Pedreira et al. 2023) PDF

The stack proposed by this article:

• apache-arrow -- language-independent compute and data interchange layer and supporting systems infrastructure across important programming languages
• ibis and dplyr --  backend-agnostic data frame interfaces
• RAPIDS -- GPU-accelerated libraries for data analytics and machine learning
• duckdb and velox --  embeddable systems providing fast columnar query processing
• Substrait -- - language-independent intermediate representation (IR) middleware for analytical computing to assist in decoupling user interfaces from compute engines

References

• Pedreira, Erling, Karanasos, Schneider, McKinney, Valluri, Zait, Nadeau (2023). The Composable Data Management System Manifesto.

Homepage: Quarto.

If you are using pandoc to publish stuff, quarto might be a better wrapper for "books". My interest was re-piqued because now they also support typst. In addition:

Quarto includes native support for Observable JS, a set of JavaScript enhancements created by Mike Bostock (the author of D3). Observable JS uses a reactive execution model, and is especially well suited for interactive data exploration and analysis.

You can email me at pradeep🌀btbytes.com.

---

More often than not, I choose btbytes as the username on any web services I use.

• github
• linkedin
• twitter
• exercism (coding exercises)
• filtra
• hacker rank
• ORCID is 0000-0002-3906-1041

generative art is made by a process that has some level of autonomy. The role of the artist is to design or influence this process to some degree. This process can be any number of things, and as such it is not limited to the digital realm. It can just as easily be a system driven by a chemical reaction. Or a set of rules for coloring in a checkered sheet of paper. viano

• Schotter - Georg Nees
• Anifusion.ai -- infinite canvas UI for generative AI art.
• Generative art by Anders Hoff · inconvergent

Pretty much all major languages compile[^1] to Javascript .. so this is no longer an exclusive category. This page is an attempt to capture slightly offbeat/interesting languages that do. You may also be intereste din wasm, javascript.

--

Wasp "Rails-like framework for React, Node.js and Prisma. Build your app in a day and deploy it with a single CLI command."

   

The special syntax of Wasp reminds me of ballerina programming language. See cover letter gpt for code examples.

[^1]: I'll never say "transpile"

Computers

• Macbook M2 Pro 14-inch, 2023 with 16GB RAM.
• Custom built PC for Linux and Windows.
  • NixOS 24.05
  • Windows 11 dual boot

Software

• Editors: zed, sublime, and IntelliJ Idea
• Terminal: iTerm2
• Browser: Safari (default), and Edge (Work)
• RSS: reeder

Gear

• Kindle
• iPhone 11

How free software hijacked Philip Hazel's life [LWN.net]

Philip Hazel was 51 when he began the Exim message transfer agent (MTA) project in 1995, which led to the Perl-Compatible Regular Expressions (PCRE) project in 1998. At 80, he's maintained PCRE, and its successor PCRE2, for more than 27 years.

"Discipline is doing what you hate to do, but do it like you love it." -- Mike Tyson

"People of very great ability will as a rule get on better with people of very limited ability than they will with people do ordinary ability, for the same reason as the despot and the plebeian, the grandparents and the grandchildren are natural" -- Schopenhauer invented the midwit meme 200 years ago from "Essays and aphorisms".

"Anxiety is what happens when your brains are bigger than your balls. You're smart enough to know what you should be doing but lack the courage to do it. This imbalance is what triggers anxiety. The only way to demolish anxiety is to choke it out with ruthless action. This is why you never feel anxious when you're in motion, even if that motion leads to temporary failure. Anxiety has nothing to do with failure and everything to do with a lack of courage." -- x/waronweakness

Data applications (usually), web application frameworks that make it easy to publish data as dynamic reports that have an element of interactivity to them that is not possible in static reports, they are often purpose-built to tell a story or present findings backed by data.

Frameworks

evidence is a lightweight framework for building data apps using SQL. It's open source and free to get started.

observable is another example -- that uses javascript

Briefer | The collaborative data platform -- "jupyter notebooks meet Notion-like documents. Briefer is the multiplayer notebook that let's you run SQL and Python, and turn notebooks into dashboards and Data Apps.

Mercury – Build Data Web Apps in Jupyter Notebook You don't need to know frontend frameworks, HTML, CSS or JavaScript. All in pure Python without leaving Jupyter Notebook.

my mercury account

davecheney/httpstat: It's like curl -v, with colours. "It is a single file🌟 Python script that has no dependency👏 and is compatible with Python 3🍻"

--

Hurl - Run and Test HTTP Requests

Hurl is a command line tool that runs HTTP requests defined in a simple plain text format.

It can chain requests, capture values and evaluate queries on headers and body response. Hurl is very versatile: it can be used for both fetching data and testing HTTP sessions.

Hurl makes it easy to work with HTML content, REST / SOAP / GraphQL APIs, or any other XML / JSON based APIs.

• Dokku -- "The smallest PaaS implementation you've ever seen"
• piku -- "The tiniest PaaS you've ever seen. Piku allows you to do git push deployments to your own servers."
• coolify.io -- "Self-hosting with superpowers." Coolify uses nixpacks as build pack by default. nixpacks detect what kind of application are you trying to deploy and builds it accordingly.
• KubeSail -- "Run a server in your home or office that runs websites & apps, saves money, and respects your privacy." They also offer pibox a custom low-cost cluster, pre-installed with Kubernetes and the KubeSail agent. The design is based on the Raspberry Pi 4 Compute Module, includes 2 expandable SATA SSD ports, and costs $299 (June 2024). PiBox is ideal for home and small office use, and makes it easy to take advantage of KubeSail templates and backups.

--

FlightControl -  "is a PaaS that deploys to your AWS." is a "self service" alternative for render, railway, flyio, and vercel.

Here are 10 thoughts from Charlie Munger

1. Why ignoring experts is smart: "You must have the confidence to override people with more credentials whose cognition is impaired by incentive-caused bias.”

2. Intensity wins: "Passion is more important than brainpower."

3. The Roman System: "An example of a really responsible system is the system the Romans used when they built an arch. The guy who created the arch stood under it as the scaffolding was removed. It’s like packing your own parachute."

4. Just because you’ve had one misfortune doesn’t mean you should let it compound into two: "You should never, when faced with one unbelievable tragedy, let one tragedy increase into two or three because of a failure of will."

5. A forest that does not clear its deadwood with small fires will, one day, suffer a giant uncontrollable fire. Brains must clear their deadwood too. Munger: "Rapid destruction of your ideas when the time is right is one of the most valuable qualities you can acquire."

6. Your gadgets are more deadweight than they are your engines: "Look at this generation, with all of its electronic devices and multitasking. I will confidently predict less success than Warren, who just focused on reading. If you want wisdom, you’ll get it sitting on your ass. That’s the way it comes."

7. Pure thinking (not interrupted by meetings and other "tasks") is a productivity hack the world is sleeping on: "We insist on a lot of time being available almost every day to just sit and think. That is very uncommon in American business. We read and think."

8. There's a strange taboo against reading. Elon Musk has said in the new Walter Isaacson book that his parents were always trying to get him to read less. Charlie Munger said his kids laugh at his reading habit, calling him a book with two legs sticking out.

9. This mockery is misplaced. Munger: "In my whole life, I have known no wise people who didn't read all the time - none, zero." Without books, travel, and aggressive independent thinking, there is no way for you to leave the intellectual context you were born in.

10. Only high-trust tribes build civilization: "The highest form that civilization can reach is a seamless web of deserved trust—not much procedure, just totally reliable people correctly trusting one another."

Mungerisms

via X/aschwags3

1. Write your obituary and live it backwards.
2. The really big opportunities in life will be very few. So when you find a lollapalooza, and you can see it, don't hang back like a timid rabbit. Step boldly and grab it. And don't do it small.
3. Learn the big ideas in all the disciplines. Pay no heed to sectoral boundaries. And cross-apply like crazy.
4. Whenever you think something or some person is ruining your life, it's you. A victimization mentality is so debilitating
5. A lot of success in life and business comes from knowing what you want to avoid: early death, a bad marriage
6. Go to sleep each night a little less stupid than you were when you woke up.
7. Never, ever, think about something else when you should be thinking about the power of incentives
8. Assume life will be really tough, and then ask if you can handle it. If the answer is yes, you've won.
9. Do what you like and are good at. Three rules for a career: Don't sell anything you wouldn't buy yourself; Don't work for anyone you don't respect and admire; Work only with people you enjoy
10. It is remarkable how much long-term advantage people like us have gotten by trying to be consistently not stupid.

Course notes from Building Multimodal Search and RAG about Multimodal Learning.

• Instructor: Sebastian Witalec, head of developer relations at Weaviate,
• human's learn multi-modally
• human's foundational knowledge is built using multi-modal interaction with the world and not via language.
• multi-modal embedding models produce a joint embedding space that understand all your modalities (e.g.: text, video, audio, picture).
• Training:
  • start with specialist models
  • unify these models
  • the vectors that they generate are going to be similar
  • Models are unified using a process called contrastive representative learning.
    • CRL is a general purpose process that can be used to train any embedding model.
    • create one unified vector space representation for multiple modalities
      • by giving positive and negative examples, we train the models to pull closer vectors together for similar examples. and push away dissimilar examples
      • Anchor point
      • The pushing pulling process is achieved with the contrastive loss function.
        • encode the anchor and the (+ve and -ve ) examples into vector embeddings.
        • calculate the distance between the anchor and the examples.
        • minimize the distance between the positive example and maximize the negative example during the training process.
        • in multimodal learning, the examples can be from different modalities.
        • Minimize the loss... (Girdhar et al. 2023)

   

[via](https://learn.deeplearning.ai/courses/building-multimodal-search-and-rag/lesson/2/overview-of-multimodality)

References

• Girdhar, El-Nouby, Liu, Singh, Alwala, Joulin, Misra (2023). Imagebind: One Embedding Space to Bind Them All.

Discovering Small GitHub Projects for Contributing to FOSS; interesting use of github search.

Tools related to github:

• GitDiagram -- Turn any GitHub repository into an interactive diagram for visualization. This is useful for quickly visualizing projects. You can also replace 'hub' with 'diagram' in any Github URL; created by Ahmed Khaleel. See an example of this on my project pandaku -- gitdiagram/btbytes/pandaku.
• DeepWiki | AI documentation you can talk to, for every repo -- explains the code base. just paste the github url.
• HTML Preview Enter URL of the HTML file to preview.

Stering-PDf - Locally hosted web application that allows you to perform various operations on PDF files. See also PDFPea - Edit PDFs Online Securely and Free | Privacy-First PDF Editor

Dangerzone - Convert potentially dangerous documents into safe PDFs

ExifTool by Phil Harvey -Read, Write and Edit Meta Information!

QPDF - A Content-Preserving PDF Transformation System

Some of these above were found via

---

small script to cleanup pdfs in general (reducing their size as well), via.

pdftops -paper A4 -expand -level3 file.pdf
ps2pdf14 -dEmbedAllFonts=true          \
    -dUseFlateCompression=true           \
    -dOptimize=true                      \
    -dProcessColorModel=/DeviceRGB       \
    -r72                                 \
    -dDownsampleGrayImages=true          \
    -dGrayImageResolution=150            \
    -dAutoFilterGrayImages=false         \
    -dGrayImageDownsampleType=/Bicubic   \
    -dDownsampleMonoImages=true          \
    -dMonoImageResolution=150            \
    -dMonoImageDownsampleType=/Subsample \
    -dDownsampleColorImages=true         \
    -dColorImageResolution=150           \
    -dAutoFilterColorImages=false        \
    -dColorImageDownsampleType=/Bicubic  \
    -dPDFSETTINGS=/ebook                 \
    -dNOSAFER                            \
    -dALLOWPSTRANSPARENCY                \
    -dShowAnnots=false                   \
     file.pdf file.pdf

Open Source Durable Execution | Temporal Technologies

Durable Execution is a development abstraction that preserves complete application state so that upon host or software failure it can seamlessly migrate execution to another machine.

Temporal is an open-source implementation of Durable Execution created by the originators of the abstraction. See examples page

---

Restate a system for workflows-as-code (durable execution). With SDKs in JS | Java | Kotlin and a lightweight runtime built in Rust/Tokio. Is licensed BSL. See HN Threat

• Restate's runtime is a single binary, self-contained, no dependencies aside from a durable disk. It contains basically a lightweight integrated version of a durable log, workflow state machine, state storage, etc. That makes it very compact and easy to run both on a laptop and a server.
• Restate implements durable execution not only for workflows, but the core building block is durable RPC handlers (or event handler). It adds a few concepts on top of durable execution, like virtual objects (turn RPC handlers into virtual actors), durable communication, and durable promises. Here are more details: Programming model.

---

dbos-inc/dbos-transact-py: Ultra-Lightweight Durable Execution in Python

Durable execution means your program is resilient to any failure. If it is ever interrupted or crashes, all your workflows will automatically resume from the last completed step.

Under the hood, DBOS Transact works by storing your program's execution state (which workflows are currently executing and which steps they've completed) in a Postgres database. So all you need to use it is a Postgres database to connect to—there's no need for a "workflow server."

Django from first principles

Static site generator for Django - Django Distill

While making https://thesummarist.net, I came across https://django-distill.com, which is THE BEST way to create static sites, bar none. You just make a standard, dynamic Django site, then you run a command and boom! It's static. I love it. -- via Stravos

Django SQLite Production Config

# yourproject/settings.py
DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.sqlite3",
        "OPTIONS": {
            "transaction_mode": "IMMEDIATE",
            "timeout": 5,  # seconds
            "init_command": """
                PRAGMA journal_mode=WAL;
                PRAGMA synchronous=NORMAL;
                PRAGMA mmap_size = 134217728;
                PRAGMA journal_size_limit = 27103364;
                PRAGMA cache_size=2000;
            """,
        },
    }
}

Blessing the newly weds - "Live a long life acquiring 16 boons"

Via "Pathinarum Petru Peru Vazhvu Vazhga" 16 boons - "thudhi vaani veeram visayam sandhaanam thunivu dhanam madhi dhaaniyam saubaaggiyam bogam - arivu azhagu pudhidhaam perumai aram kulam novagal pun vayadhu"

• thudhi = fame
• vaani = education
• veeram = courage
• visayam = success
• sandhaanam (santānam) = progeny
• thunivu = daring
• dhanam = finance
• madhidhaaniyam = valued grain(abundance of food)
• saubaaggiyam (saubhāgyam) = well-being
• bogam (bhōgam) = enjoyment of pleasures
• arivu = wisdom
• azhagu = beauty

(IAST notes in bracket by me).

Write complete web applications in Python without writing javascript. There are a large class of problems for which this is an excellent solution, without having to dip into the javascript tarpit.

The advantage of something like this over using templates is that you can think of "widgets" vs "templates".

Some frameworks are:

Reflex (formerly known as pynecone). Uses FastAPI + React.

NiceGUI. Uses FastAPI + Vue.js + Quasar, tailwind.

Pydantic/FastUI. Uses React. still early in development.

Gradio -- "Build & Share Delightful Machine Learning Apps"

Via AI Papers Community notes via \@zaph0id; Apr 2024.

• https://aman.ai/papers/
• https://vinija.ai/nlp/GenerativeAI/
• http://vinija.ai/models/LLM/
• https://huyenchip.com/2023/10/10/multimodal.html

--

Ilya's reccommendation to Carmack.. also available as this Arc Folder with links to actual papers and blog posts. See the twitter thread for summaries.

   

via

Gemini is Google's AI/llm model introduced in Dec '23 - Introducing Gemini: Google’s most capable AI model yet

In Apr 2024, they announced Gemini 1.5 Pro - Gemini 1.5 Pro Now Available in 180+ Countries; With Native Audio Understanding, System Instructions, JSON Mode and More - Google for Developers understands audio, uses unlimited files, acts on your commands, and lets devs build things with JSON mode. 1 million token context window. Ask Gemini about ten books of text, 9 hours of audio or a movie worth of video frames.

Devin from Cognitive AI made waves when it came out in March 2024.

There are follow-on developments since then.

• SWE-Agent Agent Computer Interfaces Enable Software Engineering Language Models.SWE-agent turns LMs (e.g. GPT-4) into software engineering agents that can fix bugs and issues in real GitHub repositories. (Yang et al. 2024)
• [2407.16741] OpenDevin: An Open Platform for AI Software Developers as Generalist Agents

References

• Yang, Jimenez, Wettig, Yao, Narasimhan, Press (2024). SWE-agent: Agent Computer Interfaces Enable Software Engineering Language Models.

Winglang

This is still in development, but interesting to keep an eye on, in light of Terraform/OpenTOFU kerfuffle. Written by the author of AWS CDK.

Cloud applications are fundamentally different from applications that run on a single machine - they are distributed systems that rely on cloud infrastructure to achieve their goals.

In order to be able to express both infrastructure and application logic in a safe and unified programming model, Winglang has two execution phases: preflight for infrastructure definitions and inflight for runtime code. Preflight code is executed during compilation and produces the infrastructure configuration for your app (e.g. Terraform, CloudFormation, etc). Inflight code is compiled into JavaScript and executed within cloud compute platforms in Node.js environments.

---

OpenTaco – Terraform automation that feels invisible "Raise a PR and OpenTaco runs terraform plan automatically, posting the output as a comment. Approve and merge, it applies. State is versioned with rollback built in. Drift is detected on a schedule and filed as issues in GitHub. Terraform runs natively in your CI. Your cloud credentials never leave your environment. No third-party compute, no secrets shared with a SaaS."

Why fine tune?

• Greater Accuracy -- By tailoring the model to specific use cases and industries, it can better understand and generate contextually relevant responses.
• Improve Efficiency -- Fine-tuning streamlines performance by reducing token usage and condensing the effectiveness of a larger model into a smaller, more efficient one.

---

Fine-tuning tailors the LLM's weights to specific domains and tasks, improving performance on long-term challenges. via

Tutorials

Fine-Tuning LLMs: A Guide With Examples ; and How to Fine Tune GPT 3.5

Notes

some good experience shared here: https://x.com/MaximeRivest/status/1895315348544483633 in response to https://x.com/simonw/status/1895301139819860202

gemma-3-270m might be a good model to finetune; should be possible to do it on my MBP.

Cohere -- an AI SaaS platform. It has - Command Embed and Rerank models. Cohere also has Fine Tuning.

Command R+

Announced on Apr 4, 2024, where Command R+ is an enterprise-grade Large Language Model (LLM) optimized for Retrieval Augmented Generation (RAG) and tool use. This multilingual model, which excels in 10 key languages, is available on Cohere's Playground and Azure, with plans for expansion to other platforms. Command R+ is designed to be more accurate and relevant for specific uses, leveraging a 128k-token context window.

How to use Command R+, a thread on twitter.

Adaptive RAG with Cohere's Command R+ by. LangChainAI.

Building adaptive RAG from scratch with Command-R - YouTube

---

Cohere North Mini Code 1.0

Use it with pi.dev

uv run mlx_vlm.server --model CohereLabs/North-Mini-Code-1.0
pi --provider mlx-lm --model CohereLabs/North-Mini-Code-1.0

See also: llm

via ➡️ Text embedding models represent natural language as dense vectors, positioning semantically similar text near each other within the embedding space. Or in simple terms, text embedding models are like translators for computers. They take text and convert it into numbers in a way the computer can understand.

The numerical representations, also known as embeddings, capture semantic information about the words or sentences in the text. By allowing computers to process natural language, these embeddings are used to carry out a wide range of downstream tasks including document retrieval, sentence similarity, classification, and clustering.

An intuitive introduction to text embeddings - Stack Overflow blog

We can frame a lot of useful tasks in terms of text similarity.

• Search: How similar is a query to a document in your database?
• Spam filtering: How close is an email to examples of spam?
• Content moderation: How close is a social media message to known examples of abuse?
• Conversational agent: Which examples of known intents are closest to the user’s message?

Effective and fast text embedding methods transform textual input into a numeric form, which allows models such as GPT-4 to process immense volumes of data and show a remarkable level of natural language understanding.

A deep, intuitive understanding of text embeddings can help you follow the advances of these models, letting you effectively incorporate them into your own systems without combing through the technical specs of each new improvement as it emerges.

---

The Beginner’s Guide to Text Embeddings | deepset

The foundational technology for modern-day NLP is the text embedding, also known as a “vector”: without word and sentence vectors, there would be no cutting-edge translation apps, no automated summaries, and no semantic search engines as we know them. What’s more, embeddings can be used to represent other data types as well, like images and audio files.

Embeddings 101: The foundation of large language models

Embeddings are underrated

What embeddings offer to technical writers is the ability to discover connections between texts at previously impossible scales.

Gecko

Gecko is an text embedding model from Google, it's 768-dim vectors being competitive with models that have 7x more parameters and 5x larger embeddings. (Lee et al. 2024). Results: Gecko is best-in-class for models with BERT-sized embeddings, and due to its support for Matryoshka Learning, it outperforms all such models with just 256 dimensions. Furthermore, when using all 768 dimensions, it competes directly with titans such as E5-Mistral and GritLM, which have 7B parameters (vs Gecko's 1.2B) and 4096 embedding dimensions. Impressive! An interesting finding that gives credit to the use of synthetic data can be seen in the last row, where the model trained exclusively on LLM-generated data (instead of the typical mixture that includes MTEB datasets) remains quite performant for its size.

NOTE: likely not usable right now since the model weights are not available.

References

• Lee, Dai, Ren, Chen, Cer, Cole, Hui, Boratko, Kapadia, Ding, Luan, Duddu, Abrego, Shi, Gupta, Kusupati, Jain, Jonnalagadda, Chang, Naim (2024). Gecko: Versatile Text Embeddings Distilled from Large Language Models.

Status: Ideating

Apple Notes ships on all MacOS and iOS devices, and is a quick way to take notes; I have notes that I took 10+ years ago, that have survived many device upgrades.

Notes has many features that makes it a lightweight knowledge management tool like ability to attach pictures directly from the phone, HTML tables, and most recently - Linking to other notes.

This has me wondering if Notes can be used a lightweight authoring tool for a website. However, notes supports only exporting single note to PDF, and nothing much else.

The first step to this is to export the content into a more web friendly format like HTML (or markdown, whence it can be converted to HTML easily).

The first clue down this road is apple-notes-to-sqlite tool written by @simonw.

The exported [notes] table contains HTML in the body column:

CREATE TABLE [notes] (
   [id] TEXT PRIMARY KEY,
   [created] TEXT,
   [updated] TEXT,
   [title] TEXT,
   [body] TEXT
);

If one can build a MacOS native app that can directly publish the notes as webpages to a hosting service, then we can see the classic Desktop based web publishing make a come back in a small way.

At the least this NotesPublisher should have the following options:

1. Select which folders to export
2. Select "Home page"
3. Some kind of navigation.

---

See also: yakshaves, sqlite, publishing

(Moritz et al. 2018) PDF

Ray unifies task-parallel and actor programming models in a single dynamic task graph and employs a scalable architecture enabled by the global control store and a bottom-up distributed scheduler. The programming flexibility, high throughput, and low latencies simultaneously achieved by this architecture is particularly important for emerging artificial intelligence work- loads, which produce tasks diverse in their resource requirements, duration, and functionality.

Learning Ray by Max Pumperla, Edward Oakes, and Richard Liaw. ( PDF ).

In short, Ray sets up and manages clusters of computers so that you can run distributed tasks on them. A Ray Cluster consists of nodes that are connected to each other via a network. You program against the so-called driver, the program root, which lives on the head node. The driver can run jobs, a collection of tasks, that are run on the nodes in the cluster. Specifically, the individual tasks of a job are run on worker processes on worker nodes.

This cluster can also be local, on your laptop. The default number of worker processes is the number of CPUs available on your machine.

Github - https://github.com/ray-project/ray

$ pip install "ray[rllib, serve, tune]==2.10.0"

import ray
ray.init()

Libraries:

• data processing -- apache-arrow
• model training -- TensorFlow, pytorch
• hyperparameter tuning
• model serving

News:

• Amazon’s Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2 | AWS Open Source Blog

References

• Moritz, Nishihara, Wang, Tumanov, Liaw, Liang, Elibol, Yang, Paul, Jordan, Stoica (2018). Ray: A Distributed Framework for Emerging AI Applications.

"I did not recruit extraordinary people. I recruited people who had extraordinary potential—and then I trained them." -- HGR

Hyman G. Rickover was an admiral in the United States Navy. He directed the original development of naval nuclear propulsion and controlled its operations for three decades as director of the U.S. Naval Reactors office.

I made one man responsible for his entire area of equipment—for design, production, maintenance, and contracting. If anything went wrong, I knew exactly at whom to point. -- HGR, via.

In the above thread:

Amazon has a name for it STO (single threaded owner) and STL (single threaded leader). Each initiative/service/project needs to formally have a named person as STO or a STL.

---

Rickover’s Rules | The Snell Group

Rule 1: You must have a rising standard of quality over time, and well beyond what is required by any minimum standard.

Rule 2: People running complex systems should be highly capable.

Rule 3: Supervisors have to face bad news when it comes and take problems to a level high enough to fix those problems.

Rule 4: You must have a healthy respect for the dangers and risks of your particular job.

Rule 5: Training must be constant and rigorous.

Rule 6: All the functions of repair, quality control, and technical support must fit together.

Rule 7: The organization and members thereof must have the ability and willingness to learn from mistakes of the past.

(Lambert et al. 2024)

With RewardBench, we:

1. Release a common framework for evaluating the many different architectures of reward models, along with tools for visualization, training, and other analysis. We also release all data used in the evaluation, composed of text-score pairs for all inputs, to enable further data analysis on the properties of reward models.
2. Illustrate the differences between DPO and classifier-based reward models across a variety of datasets. DPO models, while more plentiful due to the method’s simplicity, fail to generalize to popular preference data test sets and present a higher variance in performance.
3. Chart the landscape of current state-of-the-art reward models. We showcase the scaling laws, the propensity to refuse (or not), the reasoning capabilities, and more for popular RMs.
4. Show the limitations of existing preference data test sets for evaluating these models, showcasing common pitfalls of RMs on subtle, but challenging instruction pairs (e.g. intentionally modified rejected responses, which superficially look high quality but answer the wrong prompt).

---

(Yuan et al. 2024)

   

Current approaches commonly train reward models from human preferences, which may then be bottlenecked by human performance level, and secondly these separate frozen reward models cannot then learn to improve during LLM training. In this work, we study Self-Rewarding Language Models, where the language model itself is used via LLM-as-a-Judge prompting to provide its own rewards during training. We show that during Iterative DPO training that not only does instruction following ability improve, but also the ability to provide high-quality rewards to itself.

Self Play Fine Tuning (SPIN)

According to (Das 2024), (creator of premai, which is also a good overview of RHLF (Reinforcement Learning from Human Feedback), DPO (Direct Preference Optimization), IPO (Identify Preference Optimization), the propose Self-Play Fine-Tuning (SPIN) is a method designed to enhance the capabilities of Large Language Models (LLMs) through a unique self-improvement process that doesn't rely on additional human-annotated data -- (Chen et al. 2024)

References

• Lambert, Pyatkin, Morrison, Miranda, Lin, Chandu, Dziri, Kumar, Zick, Choi, Smith, Hajishirzi (2024). RewardBench: Evaluating Reward Models for Language Modeling.
• Yuan, Pang, Cho, Li, Sukhbaatar, Xu, Weston (2024). Self-Rewarding Language Models.
• Das (2024). Model Alignment Process.
• Chen, Deng, Yuan, Ji, Gu (2024). Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models.

Adaptive Engine,

Roe AI -- "AI for Risk, Compliance and Trust. Create AI workers to automate your risk operations." ^e9eac9

Stack-AI.com -- The No-Code Generative AI Platform. Deploy custom AI Assistants, Chatbots, and Workflow Automations to make your company 10x more efficient.

Ray.IO - Productionizing and scaling Python ML workload -- "Ray is an open-source unified compute framework that makes it easy to scale AI and Python workloads — from reinforcement learning to deep learning to tuning, and model serving." See presentation Flexible Distributed Python for Machine Learning. See also Ray

Defog.ai - Fine-tuned AI models for enterprise SQL Deploy fine-tuned LLMs for enterprise analytics. Speed up data analyses in SQL, Python and R with AI assistants and agents tailored for your business - without sharing your data; twitter; Cofounder: Rishab

Workhack.ai -- Turn your Forms into Great AI Conversations; twitter; Siddish

Cohere -- Optimized for enterprise generative AI,  search and discovery, and advanced retrieval.

modal labs -- Run generative AI models, large-scale batch jobs, job queues, and much more. Eg: host your own mistral model etc. "Bring your own code — we run the infrastructure."

LlamaIndex, Data Framework for LLM Applications -- Connect your data with the industry-leading framework for LLMs:

• loading
• indexing
• querying
• evaluating

With applications like - Q&A, Structured Extraction, Chat, Semantic Search, and Agents.

Taylor - API for Text Classification -- Proprietary text classifiers designed for speed. Classify all your text data in real time: every user SMS, every support ticket, and every document.

crewAI - Multi AI Agents systems. -- "AI Agents reimagined for engineers". Docs website. Core Concepts: Agentic Systems, Tasks, Tools, Processes and Crews.

Agenthub -- "Automate any workflow with AI. Drag, drop, and deploy custom tools your business needs."

Hercules -- "Multi-Agent AI solutions that automate complex workflows. Transform complex workflows with Generative AI solutions built on the Hercules Platform."

Get Coral -- one more 'chat with your PDF' app.

Sedai: AI-Powered Cloud Cost Optimization and Performance Tuning

CommandBar | AI-powered user assistance via Dwarkesh's podcast

Magick - Cutting-edge tools for AI creators - "Artificial Intelligence Development Environment (AIDE) is similar to existing tools you know, as well as platform agnostic, and open-source, allowing full control to always be in your hands."

Gumloop | AI Automation Framework "Drag, drop, and deploy custom tools your business needs. No AI expertise needed." ( previously AgentHub)

LLM API companies

• Together AI -- offers Inference, fine-tuning, custom-models, and GPU Clusters. eg: $1.20/million tokens of Mixtral-8x22B (Apr 11, 2024) (login:github )

• Deep Infra -- "Fast ML Inference, Simple API. Run the top AI models using a simple API, pay per use. Low cost, scalable and production ready infrastructure." $0.65/Million tokens

• Fireworks.AI "The production AI platform built for developers"; $0.9/M

• OpenPipe "Fine-Tuning for Developers", "Software engineers and data scientists use OpenPipe’s intuitive fine-tuning and monitoring services to decrease the cost and latency of their LLM operations. You can use OpenPipe to collect and analyze LLM logs, create fine-tuned models, and compare output from multiple models given the same input."

@OpenPipeAI makes fine-tuning a model a 1 hour project instead of months. And when a new better model drops it's a 5 minute project to fine-tune it and switch.

• From prompt to model: fine tuning when you've already deployed LLMs in prod by corbtt

Platforms

Restack — Restack

lumentis - AI powered one-click comprehensive docs from transcripts and text

Generate beautiful docs from your transcripts and unstructured information with a single command. A simple way to generate comprehensive, easy-to-skim docs from your meeting transcripts and large documents.

See examples: Self-Discover in DSPy with Chris Dossman - an interesting conversation between Chris Dossman and Weviate about DSPy and structured reasoning, one of the core concepts behind the framework.

This an LLM OpenSource Library

Large language models (LLMs) often generate content that contains factual errors when responding to fact-seeking prompts on open-ended topics.

This paper by Google Deepmind -- (Wei et al. 2024) -- proposes a method which they call Search-Augmented Factuality Evaluator (SAFE). SAFE utilizes an LLM to break down a long-form response into a set of individual facts and to evaluate the accuracy of each fact using a multi-step reasoning process comprising sending search queries to Google Search and determining whether a fact is supported by the search results. Furthermore, they propose extending F1 score as an aggregated metric for long-form factuality. To do so, they balance the percentage of supported facts in a response (precision) with the percentage of provided facts relative to a hyper-parameter representing a user's preferred response length (recall).

---

See also: TruLens for LLMs

Create credible and powerful LLM apps, faster. TruLens is a software tool that helps you to objectively measure the quality and effectiveness of your LLM-based applications using feedback functions. Feedback functions help to programmatically evaluate the quality of inputs, outputs, and intermediate results, so that you can expedite and scale up experiment evaluation. Use it for a wide variety of use cases including question answering, summarization, retrieval-augmented generation, and agent-based applications.

References

• Wei, Yang, Song, Lu, Hu, Tran, Peng, Liu, Huang, Du, Le (2024). Long-Form Factuality in Large Language Models.

MoscaicML was a startup acquired by Databricks in July 6, 2023 for $1.3B. In March 2024, they were behind the announcement of Databricks's new Foundation Model - DBRX

Some libraries by MosaicML are:

• LLM Foundary  code for training, finetuning, evaluating, and deploying LLMs for inference with Composerand the MosaicML platform.

• Composer is an  open-source deep learning training library. Built on top of pytorch, this library makes it easier to implement distributed training workflows on large-scale clusters.  Use Composer to speedup experimentation workflow if training neural networks of any size, including:

  • Large Language Models (llms)
  • Diffusion models
  • Embedding models (e.g. BERT)
  • Transformer-based models
  • Convolutional Neural Networks (CNNs)

• Streaming

• and more

huggingface -- mosaicml (Mosaic ML, Inc.)

Why Semantic HTML Still Matters - Jono Alderson

Semantic HTML5 Guide

Mora

Sora is the first large-scale generalist video generation model that garnered significant attention across society. Since its launch by OpenAI in February 2024, no other video generation models have paralleled Sora's performance or its capacity to support a broad spectrum of video generation tasks. Additionally, there are only a few fully published video generation models, with the majority being closed-source. To address this gap, this paper proposes a new multi-agent framework Mora, which incorporates several advanced visual AI agents to replicate generalist video generation demonstrated by Sora.

(Yuan et al. 2024) -- lichao-sun/Mora: Mora: More like Sora for Generalist Video Generation

Text To Video

Pika

• text to video
• image to video
• video to video

Notes on Multi Agents

From LossFunk's X Page:

"Don't build multi-agents"

References

• Yuan, Chen, Li, Jia, He, Wang, Sun (2024). Mora: Enabling Generalist Video Generation via a Multi-Agent Framework.

Agentic systems are a category of advanced AI systems focused on autonomy and goal-directed behavior. They possess the following characteristics:

--

Agent Skills by Anthropic -- Agent Skills are folders of instructions, scripts, and resources that agents can discover and use to do things more accurately and efficiently.

--

(Tao et al. 2024)

 MAGIS: a novel LLM-based Multi-Agent framework for GitHub Issue reSolution, MAGIS, consisting of four kinds of agents customized for the software evolution: Manager, Repository Custodian, Developer, and Quality Assurance Engineer agents. This framework leverages the collaboration of various agents in the planning and coding process to unlock the potential of LLMs to resolve GitHub issues. In experiments, we employ the SWE-bench benchmark to compare MAGIS with popular LLMs, including GPT-3.5, GPT-4, and Claude-2.

(Qian et al. 2023)

Note: This paper was mentioned in Crewai's multi ai agent system course on deeplearning.ai

 In this paper, we present an innovative paradigm that leverages large language models (LLMs) throughout the entire software development process, streamlining and unifying key processes through natural language communication, thereby eliminating the need for specialized models at each phase. At the core of this paradigm lies ChatDev, a virtual chat-powered software development company that mirrors the established waterfall model, meticulously dividing the development process into four distinct chronological stages: designing, coding, testing, and documenting. Each stage engages a team of "software agents", such as programmers, code reviewers, and test engineers, fostering collaborative dialogue and facilitating a seamless workflow. The chat chain acts as a facilitator, breaking down each stage into atomic subtasks. This enables dual roles, allowing for proposing and validating solutions through context-aware communication, leading to efficient resolution of specific subtasks. The instrumental analysis of ChatDev highlights its remarkable efficacy in software generation, enabling the completion of the entire software development process in under seven minutes at a cost of less than one dollar. It not only identifies and alleviates potential vulnerabilities but also rectifies potential hallucinations while maintaining commendable efficiency and cost-effectiveness.

 Github repo

Microsoft Autogen -- (Wu et al. 2023)

---

To read

• <https://x.com/svpino/status/1775946787783692435)
• LLM Powered Autonomous Agents | Lil'Log; Jun 2023.

References

• Tao, Zhou, Zhang, Cheng (2024). MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution.
• Qian, Cong, Liu, Yang, Chen, Su, Dang, Li, Xu, Li, Liu, Sun (2023). Communicative Agents for Software Development.
• Wu, Bansal, Zhang, Wu, Zhang, Zhu, Li, Jiang, Zhang, Wang (2023). AutoGen: Enabling next-Gen LLM Applications via Multi-Agent Conversation.

Alternatives to Redis

Garnet from Microsoft is a redis protocol (RESP) compliant remote cache-store. Garnet is thread-scalable within a single node. It also supports sharded cluster execution, with replication, checkpointing, failover, and transactions. It can operate over main memory as well as tiered storage (such as SSD and Azure Storage). Garnet supports a rich API surface and a powerful extensibility model. one can use Garnet with unmodified Redis clientsavailable in most programming languages. Blog post - A brief history of Garnet

Dragonfly - The Fastest In-Memory Data Store "is a drop-in Redis replacement, designed to meet the performance and efficiency requirements of modern cloud-based applications. Organizations that switch to Dragonfly require less hardware and achieve dramatically improved data performance." - licensed under BSL 1.1

valkey is a fork of redis after redis' license change.

DiceDB -- an in-memory, real-time, and reactive database with Redis and SQL support optimized for modern hardware and building real-time applications by Arpit Bhayani.

Cloud

Hosted Redis - Upstash: Serverless Data for Redis® and Kafka®

Learning

• Redis Interview Questions for DevOps and Platform Engineers

OODA method was given by John Boyd.

Boyd’s OODA Loop by Chet Richards.

The concepts of the late US Air Force Colonel John R. Boyd have influenced military thought in profound ways, from the design of modern fighter aircraft to the tactics used by the US Marine Corps in both Gulf Wars. This paper describes the best known aspect of his strategic thought, the OODA “loop,” and how practitioners in war and business can use the loop to implement a framework that has proven successful since the time of Sun Tzu.

--

“OODA loop as a way of making luck work for you”

Boyd argued that the loop applies to companies, too. If you can "get inside the OODA loop" of your competitor, you have the advantage. By the time they even notice a market is there, you have identified an opportunity, gathered data, tried an approach, and learned from mistakes.

[via](https://x.com/abesorock/status/1769797163888832898); text taken from `vgr`.

There are many ways to exploit others' habits, but wrangling luck is perhaps the most powerful way. Wrangling luck effectively creates a sort of "competitive climate" comprising patterns of serendipity and its opposite, zemblanity.

Competitive Climates

You're in a world where luck is the scarce resource, because there are more ways for things to go wrong than right. If you can direct more luck towards your- self and more misfortune towards your adversary, you've amplified the effectiveness of just about everything else you do.

In open competitive environments, luck is not just about die-rolls or card shuffles. It is a control variable. The control variable. In the overall game of life, your big goal is to drive serendipity towards yourself and its opposite, zemblanity (there, you learned a new word) towards your adversaries.

When luck is being consciously wrangled in a real-life game, winning feels like things miraculously going your way, things falling into place for you just right, unreasonably out of proportion to your actions. This is serendipity.

By contrast, losing feels like an overpowering sense of doom and snowballing misfortunes out of proportion to your sins and stupidity. This is zemblanity.

I got Dspy certificate on Deeplearning.ai - on Dec 31, 2025

With DSPy -- you can build your system using predefined modules, replacing intricate prompting techniques with straightforward, effective solutions. (Khattab et al. 2023)

Whether you're working with powerhouse models like GPT-3.5 or GPT-4, or local models such as T5-base or Llama2-13b, DSPy seamlessly integrates and enhances their performance in your system.

[via](https://x.com/lateinteraction/status/1772758620511310112) See also: [Virtual Machinations: Leveraging the Linguistic Bytecode of Large Language Models to Emulate Programming Language VMs](https://myconf.io/news/myconf-2024/speaker-announcement-erik-meijer/)

---

Cheatsheet

A gentle intro to DSPY, 25 min video.

Why are prompt optimizers still so underrated? - YouTube; Nov 20, 2025.

---

Let the LLM Write the Prompts: An Intro to DSPy in Compound AI Pipelines - YouTube

Just write code, not textual prompts.

• This talk shows a breakdown of what very successful prompts (system prompts of Claude etc) look like and what components each of them have
  1. the task definition
  2. chain-of-thought instructions
  3. detailed context and instructions
  4. tool definitions
  5. formatting instructions (this might be the secret sauce for many models improving benchmarks)
  6. other
• DSPY allows you to decouple your tasks from the LLM. switching models will be easy.
• dspy allows you to define your tasks as a contract and managing prompting strategies
• Components
  • Signatures -- input, output
  • Modules -- strategies for executing signatures. Runners that generate and run prompts.

---

Autoblocks Blog - Collaboratively Test & Evaluate your DSPy App

DSPy on Databricks | Databricks

Pedram Navid | I finally tried DSPy and now I get why everyone won't shut up about it

Learning DSPy (1): The power of good abstractions • The Data Quarry

Why DSpy

x.com/lateinteraction/ (Omar Khattab):

DSPy's biggest strength is also the reason it can admittedly be hard to wrap your head around it.

It's basically say: LLMs & their methods will continue to improve but not equally in every axis, so:

• What's the smallest set of fundamental abstractions that allow you to build downstream AI software that is "future-proof" and rides the tide of progress?

• Equivalently, what are the right algorithmic problems that researchers should focus on to enable as much progress as possible for AI software?

  But this is necessarily complex, in the sense that the answer has to be composed of a few things, not one concept only. (Though if you had to understand one concept only, the fundamental glue is DSPy Signatures.)

  It's actually only a handful of bets, though, not too many. I've been tweeting them non-stop since late 2022, but I've never collected them in one place.

  All of these have proven beyond a doubt to have been the right bets so far for 2.5 years, and I think they'll stay the right bets for the next 3 years at least.

  1. Information Flow is the single most key aspect of good AI software.

As foundation models improve, the bottleneck becomes basically whether you can actually (1) ask them the right question and (2) provide them with all the necessary context to address it.

Since 2022, DSPy addressed this in two directions: (i) free-form control flow ("Compound AI Systems" / LM programs) and (ii) Signatures.

Prompts have been a massive distraction here, with people thinking they need to find the magical keyword to talk to LLMs. From 2022, DSPy put the focus on Signatures (back then called Templates) which force you to break down LM interactions into structured and named input fields and structured and named output fields.

Getting simply those fields right was (and has been) a lot more important than "engineering" the "right prompt". That's the point of Signatures. (We know it's hard for people to force them to define their signatures so carefully, but if you can't do that, your system is going to be bad.)

2. Interactions with LLMs should be Functional and Structured. Again, prompts are bad. People are misled from their chat interaction with LLMs to think that LLMs should take "strings", hence the magical status of "prompts". But actually, you should define a functional contract. What are the things you will give to the function? What is the function supposed to do with them? What is it then supposed to give you back? This is again Signatures. It's (i) structured inputs, (ii) structured outputs, and (iii) instructions. You've got to decouple these three things, which until DSP (2022) and really until very recently with mainstream structured outputs, were just meshed together into "prompts". This bears repeating: your programmatic LLM interactions need to be functions, not strings. Why? Because there are many concerns that are actually not part of the LLM behavior that you'd otherwise need to handle ad-hoc when working with strings: - How do you format the inputs to your LLM into a string? - How do you separate instructions and inputs (data)? - How do you specify the output format (string) that your LLM should produce so you can parse it? - How do you layer on top of this the inference strategy, like CoT or ReAct, without entirely rewriting your prompt? Signatures solve this. They ask you to just specify the input fields, output fields, and task instruction. The rest are the job of Modules and Optimizers, which instantiate Signatures. 3) Inference Strategies should be Polymorphic Modules. This sounds scary but the point is that all the cool general-purpose prompting techniques or inference-scaling strategies should be Modules, like the layers in DNN frameworks like PyTorch. Modules are generic functions, which in this case take any Signature, and instantiate its behavior generically into a well-defined strategy. This means that we can talk about "CoT" or "ReAct" without actually committing at all to the specific task (Signature) you want to apply them to. This is a huge deal, which again only exists in DSPy. One key thing that Modules do is that they define parameters. What part(s) of the Module are fixed and which parts can be learned? For example, in CoT, the specific string that asks the model to think step by step could be learned. Or the few-shot examples of thinking step by step should be learnable. In ReAct, demonstrations of good trajectories should be learnable. 4) Specification of your AI software behavior should be decoupled from learning paradigms. Before DSPy, every time a new ML paradigm came by, we re-wrote our AI software. Oh, we moved from LSTMs to Transformers? Or we moved from fine-tuning BERT to ICL with GPT-3? Entirely new system. DSPy says: if you write signatures and instantiate Modules, the Modules actually know exactly what about them can be optimized: the LM underneath, the instructions in the prompt, the demonstrations, etc. The learning paradigms (RL, prompt optimization, program transformations that respect the signature) should be layered on top, with the same frontend / language for expressing the programmatic behavior. This means that the same programs you wrote in 2023 in DSPy can now be optimized with dspy.GRPO, the way they could be optimized with dspy.MIPROv2, the way they were optimized with dspy.BootstrapFS before that. The second half of this piece is Downstream Alignment or compile-time scaling. Basically, no matter how good LLMs get, they might not perfectly align with your downstream task, especially when your information flow requires multiple modules and multiple LLM interactions. You need to "compile" towards a metric "late", i.e. after the system is fully defined, no matter how RLHF'ed your models are. 5) Natural Language Optimization is a powerful paradigm of learning. We've said this for years, like with the BetterTogether optimizer paper, but you need both fine-tuning and coarse-tuning at a higher level in natural language. The analogy I use all the time is riding a bike: it's very hard to learn to ride a bike without practice (fine-tuning), but it's extremely inefficient to learn avoiding to ride the bike on the side walk from rewards, you want to understand and learn this rule in natural language to adhere ASAP. This is the source of DSPy's focus on prompt optimizers as a foundational piece here; it's often far superior in sample efficiency to doing policy gradient RL if your problem has the right information flow structure. That's it. That's the set of core bets DSPy has made since 2022/2023 until today. Compiling Declarative AI Functions into LM Calls, with Signatures, Modules, and Optimizers.

3. Information Flow is the single most key aspect of good AI software.

4. Interactions with LLMs should be Functional and Structured.

5. Inference Strategies should be Polymorphic Modules.

6. Specification of your AI software behavior should be decoupled from learning paradigms.

7. Natural Language Optimization is a powerful paradigm of learning.

---

I used to think that frameworks that make prompt engineering easier, are the way to go. But then I just began converting some existing prompts into DSPy signatures, and soon realized that modules in DSPy are infinitely composable. You can build almost anything with signatures and modules. There are quite a lot of misconceptions out there about DSPy: that it's mainly about prompt optimization, that you need tons of training data. What it's really about instead, is just finding that better abstraction to translate human intent to LMs. The more time you spend with DSPy, the more you realize that PROMPTS AREN'T A GOOD ABSTRACTION. Signatures and modules are a far better interface, and it so happens that there's this thing called optimizers that can help discover better prompts. But it all starts with signatures and modules. I recommend just translating a couple of use cases from other frameworks to DSPy and seeing it for yourself. It doesn't take all that much time. -- x.com/tech_optimist

Good enough prompt, portable across models and tasks/evals, faster. This “faster” is doing the heavy lifting. When you’re faster at the prompt layer, you have much more time to iterate on what actually matters: programming (aka context engineering) via error analysis. -- X/DSPyOSS

Uses of Dspy

GitHub - evalops/founder-email-optimizer: DSPy-powered email optimization for startup founders: drop in your 3 best emails, get optimized outreach for new leads

How we optimized Dash's relevance judge with DSPy - Dropbox; Mar 2026

References

• Khattab, Singhvi, Maheshwari, Zhang, Santhanam, Vardhamanan, Haq, Sharma, Joshi, Moazam, Miller, Zaharia, Potts (2023). DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines.

Webcraft by Joodaloop. Personal home page design.

Things I care about: Typography, delightful minimalism, semantic HTML, information architecture, performance, progressive enhancement, silly details.

--

Simon Repp -- why should not be the front page a collage?

---

Devtools design of a faux-macOS window is nice. ^cb2493

---

A Website To End All Websites | Henry From Online

-- See also websites.

"Above all, don't lie to yourself. The man who lies to himself and listens to his own lie comes to such a pass that he cannot distinguish the truth within him, or around him, and so loses all respect for himself and for others. And having no respect he ceases to love, and in order to occupy and distract himself without love he gives way to passions and coarse pleasures, and sinks to bestiality in his vices, all from continual lying to other men and to himself. The man who lies to himself can be more easily offended than any one. You know it is sometimes very pleasant to take offense, isn't it? A man may know that nobody has insulted him, but that he has invented the insult for himself, has lied and exaggerated to make it picturesque, has caught at a word and made a mountain out of a molehill—he knows that himself, yet he will be the first to take offense, and will revel in his resentment till he feels great pleasure in it, and so pass to genuine vindictiveness."

• Features of your font you had no idea about · OlegWock
• Modern Font Stacks System font stack CSS organized by typeface classification for every modern OS. The fastest fonts available. No downloading, no layout shifts, no flashes — just instant renders.

Fonts

See my Font Matter Page.

• ET Book is a Bembo-like font for the computer designed by Dmitry Krasny, Bonnie Scranton, and Edward Tufte. It is free and open-source. Used by svs.
• Ink & Switch uses Merriweather and Merriweather Sans for very nice and pleasant reading experience.
• llama.ttf  is a font file which is also a large language model and an inference engine for that model.
• Server Mono -- a monspace font.
• Inter; made popular by next.js
• Tetris Font by Erik Demaine and Martin Demaine
• Recursive Sans & Mono "A typographic palette for vibrant code & UI", via ✩ Bending Zed to render my beautiful fonts with Nix

Web Fonts

• Time to Say Goodbye to Google Fonts: Cache Performance

Google Fonts resources will be redownloaded for every website, regardless it being cached on the CDN. Self-host your fonts for better performance. The old performance argument is not valid anymore.

Font Designers

• DAVID JONATHAN ROSS