here you goThings I’ve made, read and saved — shared here.

Archive

9,565 entries: 46 from X, 8,923 from Pinboard, saved 1998 to 2026.

Browse by tag, site, author, folder or language

Tags

Sites

Authors

Folders

Languages

11 entriesgithub.com ×2024 ×Clear all

2024

paveldedik/ludic: Lightweight framework for building HTML pages in pure Python.

github.com

Ludic is a lightweight framework for building HTML pages with component approach similar to React. It is built to be used together with htmx.org so that developers don't need to write almost any JavaScript to create dynamic web services. It's potential can be leveraged together with its web framework which is a wrapper around powerful Starlette framework.

mfarragher/obsidiantools: Obsidian tools - a Python package for analysing an Obsidian.md vault

github.com

obsidiantools is a Python package for getting structured metadata about your Obsidian.md notes and analysing your vault. Complement your Obsidian workflows by getting metrics and detail about all your notes in one place through the widely-used Python data stack.

It's incredibly easy to explore structured data on your vault through this fluent interface. This is all the code you need to generate a vault object that stores all the data:

stanfordnlp/dspy: DSPy: The framework for programming—not prompting—foundation models

github.com

DSPy is a framework for algorithmically optimizing LM prompts and weights, especially when LMs are used one or more times within a pipeline. To use LMs to build a complex system without DSPy, you generally have to: (1) break the problem down into steps, (2) prompt your LM well until each step works well in isolation, (3) tweak the steps to work well together, (4) generate synthetic examples to tune each step, and (5) use these examples to finetune smaller LMs to cut costs. Currently, this is hard and messy: every time you change your pipeline, your LM, or your data, all prompts (or finetuning steps) may need to change.

To make this more systematic and much more powerful, DSPy does two things. First, it separates the flow of your program (modules) from the parameters (LM prompts and weights) of each step. Second, DSPy introduces new optimizers, which are LM-driven algorithms that can tune the prompts and/or the weights of your LM calls, given a metric you want to maximize.

DSPy can routinely teach powerful models like GPT-3.5 or GPT-4 and local models like T5-base or Llama2-13b to be much more reliable at tasks, i.e. having higher quality and/or avoiding specific failure patterns. DSPy optimizers will "compile" the same program into different instructions, few-shot prompts, and/or weight updates (finetunes) for each LM. This is a new paradigm in which LMs and their prompts fade into the background as optimizable pieces of a larger system that can learn from data. tldr; less prompting, higher scores, and a more systematic approach to solving hard tasks with LMs.

github.comPinboardPermalink

#llm

mufeedvh/code2prompt: A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.

github.com

You can run this tool on the entire directory and it would generate a well-formatted Markdown prompt detailing the source tree structure, and all the code. You can then upload this document to either GPT or Claude models with higher context windows and ask it to:

Rewrite the code to another language.
Find bugs/security vulnerabilities.
Document the code.
Implement new features.
You can customize the prompt template to achieve any of the desired use cases. It essentially traverses a codebase and creates a prompt with all source files combined. In short, it automates copy-pasting multiple source files into your prompt and formatting them along with letting you know how many tokens your code consumes.

pvolok/mprocs: Run multiple commands in parallel

github.com

mprocs runs multiple commands in parallel and shows output of each command separately.

When you work on a project you very often need the same list of commands to be running. For example: webpack serve, jest --watch, node src/server.js. With mprocs you can list these command in mprocs.yaml and run all of them by running mprocs. Then you can switch between outputs of running commands and interact with them.

It is similar to concurrently1 but mprocs shows output of each command separately and allows to interact with processes (you can even work in vim inside mprocs).