Archive

Winglang This is still in development, but interesting to keep an eye on, in light of Terraform/OpenTOFU kerfuffle. Written by the author of AWS CDK. Cloud applications are fundamentally different from applications that run on a single…

Why fine tune? Greater Accuracy -- By tailoring the model to specific use cases and industries, it can better understand and generate contextually relevant responses. Improve Efficiency -- Fine-tuning streamlines performance by reducing…

Cohere -- an AI SaaS platform. It has - Command Embed and Rerank models. Cohere also has Fine Tuning. Command R+ Announced on Apr 4, 2024, where Command R+ is an enterprise-grade Large Language Model (LLM) optimized for Retrieval Augmented…

See also: llm via ➡️ Text embedding models represent natural language as dense vectors, positioning semantically similar text near each other within the embedding space. Or in simple terms, text embedding models are like translators for…

Copart

Exploring way to export Apples notes as a website

is an open source distributed computing framework to scale compute-intensive workloads in Python

"I did not recruit extraordinary people. I recruited people who had extraordinary potential—and then I trained them." -- HGR Hyman G. Rickover was an admiral in the United States Navy. He directed the original development of naval nuclear…

(Lambert et al. 2024) RewardBench With RewardBench, we: Release a common framework for evaluating the many different architectures of reward models, along with tools for visualization, training, and other analysis. We also release all data…

Adaptive Engine, Roe AI -- "AI for Risk, Compliance and Trust. Create AI workers to automate your risk operations." ^e9eac9 Stack-AI.com -- The No-Code Generative AI Platform. Deploy custom AI Assistants, Chatbots, and Workflow Automations…

Xr0 is a verifier for C. It eliminates many stubborn instances of undefined behaviour, like use-after-frees, double frees, null pointer dereferences and the use of uninitialised memory.

lumentis - AI powered one-click comprehensive docs from transcripts and text Generate beautiful docs from your transcripts and unstructured information with a single command. A simple way to generate comprehensive, easy-to-skim docs from…

How factual are the LLMs when generating answers to open ended questions?

a team of Databricks

Admiral Hyman Rickover (1900-1986), the “Father of the Nuclear Navy,” was one of the most successful—and controversial- public managers of the 20th Century. His accomplishments are the stuff of legend. For example, in three short years, Rickover’s team designed and built the first nuclear submarine—the Nautilus—an amazing feat of engineering given that it involved the development of the first use of a controlled nuclear reactor. The Nautilus not only transformed submarine warfare, but also laid the groundwork for a whole fleet of nuclear aircraft carriers and cruisers (which was also built by Rickover and his team).

The text below is an excerpt from a speech Rickover delivered at Columbia University in 1982, in which he succinctly outlined his management philosophy. His determination, clarity of purpose, emphasis on developing his people, high standards, and willingness to give his people ownership of their work had to have been very inspiring. He had exceptionally high standards and was known to take some of these same strengths to extremes, however, which no doubt led to his reputation in some circles as being difficult to work for. On that cautionary note, GovLeaders.org is pleased to present Rickover’s own description of his management style.

blog series called Remotely India where I use satellite images to explore India's geology.

See my latest post on complex evolution of sedimentary basins inferred from
@isro
Cartosat.

As described in American Nations, our true regional fissures can be traced back to the contrasting ideals of the distinct European colonial cultures that first took root on the eastern and southern rims of what is now the United States, and then spread across much of the continent in mutually exclusive settlement bands, laying down the institutions, symbols and cultural norms later arrivals would encounter and, by and large, assimilate into. (Academically, this work is built on the foundations of the late cultural geographer Wilbur Zelinsky’s Doctrine of First Effective Settlement.)

The most precious, creative and innovative period in your life is the 10-year period around the age of 32. Plan your career path to use this precious 10-year period wisely and effectively to produce your greatest achievements in your life.
 
The pace of innovations and technology advances is getting faster and faster and is forcing everybody to compete fiercely at the Internet speed on the information super-highways. The highly productive and highly efficient workplace in USA is a pressure-cooker and a high-speed battleground for highly creative and dynamic young people to compete and to flourish.
 
However, when you get older, you should plan your career path and financial matter so that you can retire comfortably at the age of 55 or earlier to enjoy your long, happy and leisure retirement life into your golden age of 80s and beyond. In retirement, you can still enjoy some fun work of great interest to you and of great values to the society and the community, but at a part-time leisure pace on your own term.
 
On the other hand, if you are not able to get out of the pressure-cooker or the high-speed battleground at the age of 55 and “have” to keep on working very hard until the age of 65 or older before your retirement, then you probably will die within 18 months of retirement. By working very hard in the pressure cooker for 10 more years beyond the age of 55, you give up at least 20 years of your life span on average.

Self-Hosted Remote Development Environment
Open source, self hosted with ease. It uses Devcontainers spec and scales from a single machine to a global fleet of servers.

Tiptap is an open source headless content editor and real-time collaboration framework to craft exactly the content experience you’d like to have – built for developers.

will be useful for having Notion like WYSIWYG editor experience

Why Semantic HTML Still Matters - Jono Alderson Semantic HTML5 Guide

Mora Sora is the first large-scale generalist video generation model that garnered significant attention across society. Since its launch by OpenAI in February 2024, no other video generation models have paralleled Sora's performance or…

Agentic systems are a category of advanced AI systems focused on autonomy and goal-directed behavior. They possess the following characteristics: -- Agent Skills by Anthropic -- Agent Skills are folders of instructions, scripts, and…

Velox, the new C++ vectorized database acceleration library aimed at optimizing query engines and data processing systems.

Ludic is a lightweight framework for building HTML pages with component approach similar to React. It is built to be used together with htmx.org so that developers don't need to write almost any JavaScript to create dynamic web services. It's potential can be leveraged together with its web framework which is a wrapper around powerful Starlette framework.

develop web uis using fastapi and python (uses react in the background).

See also NiceGUI

Vue.JS + FastAPI to provide a single API to write backend + frontend applications

a profile of the people that co-authored "Attention is all you need" paper that was the turning point in the evolution of LLMs.

discussion of various python framework/code to do HTML generation to work with/like HTMX

obsidiantools is a Python package for getting structured metadata about your Obsidian.md notes and analysing your vault. Complement your Obsidian workflows by getting metrics and detail about all your notes in one place through the widely-used Python data stack.

It's incredibly easy to explore structured data on your vault through this fluent interface. This is all the code you need to generate a vault object that stores all the data:

The Great Tables package is all about making it simple to produce nice-looking display tables. Display tables? Well yes, we are trying to distinguish between data tables (i.e., DataFrames) and those tables you’d find in a web page, a journal article, or in a magazine. Such tables can likewise be called presentation tables, summary tables, or just tables really. Here are some examples, ripped straight from the web:

in-memory database

Garnet is a redis protocol (RESP) compliant remote cache-store. Garnet is thread-scalable within a single node. It also supports sharded cluster execution, with replication, checkpointing, failover, and transactions. It can operate over main memory as well as tiered storage (such as SSD and Azure Storage). Garnet supports a rich API surface and a powerful extensibility model. one can use Garnet with unmodified Redis clientsavailable in most programming languages.

Battery-Included, Local-First PostgreSQL Distribution as an Open-Source RDS Alternative.

Observe, Orient, Decide, Act

A community-driven platform for AI enthusiasts.

From Apple

In this work, we discuss building performant Multimodal Large Language Models (MLLMs). In particular, we study the importance of various architecture components and data choices. Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons. For example, we demonstrate that for large-scale multimodal pre-training using a careful mix of image-caption, interleaved image-text, and text-only data is crucial for achieving state-of-the-art (SOTA) few-shot results across multiple benchmarks, compared to other published pre-training results. Further, we show that the image encoder together with image resolution and the image token count has substantial impact, while the vision-language connector design is of comparatively negligible importance. By scaling up the presented recipe, we build MM1, a family of multimodal models up to 30B parameters, consisting of both dense models and mixture-of-experts (MoE) variants, that are SOTA in pre-training metrics and achieve competitive performance after supervised fine-tuning on a range of established multimodal benchmarks. Thanks to large-scale pre-training, MM1 enjoys appealing properties such as enhanced in-context learning, and multi-image reasoning, enabling few-shot chain-of-thought prompting.

Content as structured data

We compile content to syntax trees and syntax trees to content.
We also provide hundreds of packages to work on the trees in between.
You can build on the unified collective to make all kinds of interesting things.

When writing and talking, people sometimes pause to think. Although reasoning-focused works have often framed reasoning as a method of answering questions or completing agentic tasks, reasoning is implicit in almost all written text. For example, this applies to the steps not stated between the lines of a proof or to the theory of mind underlying a conversation. In the Self-Taught Reasoner (STaR, Zelikman et al. 2022), useful thinking is learned by inferring rationales from few-shot examples in question-answering and learning from those that lead to a correct answer. This is a highly constrained setting -- ideally, a language model could instead learn to infer unstated rationales in arbitrary text. We present Quiet-STaR, a generalization of STaR in which LMs learn to generate rationales at each token to explain future text, improving their predictions. We address key challenges, including 1) the computational cost of generating continuations, 2) the fact that the LM does not initially know how to generate or use internal thoughts, and 3) the need to predict beyond individual next tokens. To resolve these, we propose a tokenwise parallel sampling algorithm, using learnable tokens indicating a thought's start and end, and an extended teacher-forcing technique. Encouragingly, generated rationales disproportionately help model difficult-to-predict tokens and improve the LM's ability to directly answer difficult questions. In particular, after continued pretraining of an LM on a corpus of internet text with Quiet-STaR, we find zero-shot improvements on GSM8K (5.9%→10.9%) and CommonsenseQA (36.3%→47.2%) and observe a perplexity improvement of difficult tokens in natural text. Crucially, these improvements require no fine-tuning on these tasks. Quiet-STaR marks a step towards LMs that can learn to reason in a more general and scalable way.

DSPy is a framework for algorithmically optimizing LM prompts and weights, especially when LMs are used one or more times within a pipeline. To use LMs to build a complex system without DSPy, you generally have to: (1) break the problem down into steps, (2) prompt your LM well until each step works well in isolation, (3) tweak the steps to work well together, (4) generate synthetic examples to tune each step, and (5) use these examples to finetune smaller LMs to cut costs. Currently, this is hard and messy: every time you change your pipeline, your LM, or your data, all prompts (or finetuning steps) may need to change.

To make this more systematic and much more powerful, DSPy does two things. First, it separates the flow of your program (modules) from the parameters (LM prompts and weights) of each step. Second, DSPy introduces new optimizers, which are LM-driven algorithms that can tune the prompts and/or the weights of your LM calls, given a metric you want to maximize.

DSPy can routinely teach powerful models like GPT-3.5 or GPT-4 and local models like T5-base or Llama2-13b to be much more reliable at tasks, i.e. having higher quality and/or avoiding specific failure patterns. DSPy optimizers will "compile" the same program into different instructions, few-shot prompts, and/or weight updates (finetunes) for each LM. This is a new paradigm in which LMs and their prompts fade into the background as optimizable pieces of a larger system that can learn from data. tldr; less prompting, higher scores, and a more systematic approach to solving hard tasks with LMs.

You can run this tool on the entire directory and it would generate a well-formatted Markdown prompt detailing the source tree structure, and all the code. You can then upload this document to either GPT or Claude models with higher context windows and ask it to:

Rewrite the code to another language.
Find bugs/security vulnerabilities.
Document the code.
Implement new features.
You can customize the prompt template to achieve any of the desired use cases. It essentially traverses a codebase and creates a prompt with all source files combined. In short, it automates copy-pasting multiple source files into your prompt and formatting them along with letting you know how many tokens your code consumes.

Programming, not prompting, Language Models

The modern replacement for Jupyter Notebooks