Week 1 resources

Below you'll find links to the research papers discussed in this weeks videos. You don't need to understand all the technical details discussed in these papers - you have already seen the most important points you'll need to answer the quizzes in the lecture videos.

However, if you'd like to take a closer look at the original research, you can read the papers and articles via the links below.

Transformer Architecture

Attention is All You Need - This paper introduced the Transformer architecture, with the core “self-attention” mechanism. This article was the foundation for LLMs.
BLOOM: BigScience 176B Model - BLOOM is a open-source LLM with 176B parameters (similar to GPT-4) trained in an open and transparent way. In this paper, the authors present a detailed discussion of the dataset and process used to train the model. You can also see a high-level overview of the model here.
Vector Space Models - Series of lessons from DeepLearning.AI's Natural Language Processing specialization discussing the basics of vector space models and their use in language modeling.

Pre-training and scaling laws

Scaling Laws for Neural Language Models - empirical study by researchers at OpenAI exploring the scaling laws for large language models.

Model architectures and pre-training objectives

What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization? - The paper examines modeling choices in large pre-trained language models and identifies the optimal approach for zero-shot generalization.
HuggingFace Tasks and Model Hub - Collection of resources to tackle varying machine learning tasks using the HuggingFace library.
LLaMA: Open and Efficient Foundation Language Models - Article from Meta AI proposing Efficient LLMs (their model with 13B parameters outperform GPT3 with 175B parameters on most benchmarks)

Scaling laws and compute-optimal models

Language Models are Few-Shot Learners - This paper investigates the potential of few-shot learning in Large Language Models.
Training Compute-Optimal Large Language Models - Study from DeepMind to evaluate the optimal model size and number of tokens for training LLMs. Also known as “Chinchilla Paper”.
BloombergGPT: A Large Language Model for Finance - LLM trained specifically for the finance domain, a good example that tried to follow chinchilla laws.

Week 2

Finetuning

PEFT - Parameter Efficient Finetuning. LoRA is a popular PEFT technique.

Big model vs small model + fine tuning.

Fine tuning an LLM with finetuning

In-Context learning (ICL) - one/few/zero shot inference

Limitations of ICL:

doesn't work for small models
examples take up space in the context window

Can use finetune a base model

Fine-tuning is a supervised learning model.

Task specific examples are pairs of:

	PROMPT[...], COMPLETION[...]

Instruction finetuning:

Training data

Prompt Template Library

Remember that the output of an LLM is a probability distribution across tokens. So you can compare the distribution of the completion and that of the training label and use the standard cross entropy function to calculate loss between the two token distributions. And then use the calculated loss to update your model weights in standard back propagation. You'll do this for many batches of prompt completion pairs and over several epochs, update the weights so that the model's performance on the task improves. As in standard supervised learning, you can define separate evaluation steps to measure your LLM performance using the holdout validation data set. This will give you the validation accuracy, and after you've completed your fine tuning, you can perform a final performance evaluation using the holdout test data set. This will give you the test accuracy. The fine-tuning process results in a new version of the base model, often called an instruct model that is better at the tasks you are interested in. Fine-tuning with instruction prompts is the most common way to fine-tune LLMs these days. From this point on, when you hear or see the term fine-tuning, you can assume that it always means instruction fine tuning.

Fine-Tuning a Single Task

Good results had be had with small examples - 500-1000.

Downside: Catastrophic forgetting

Catastrophic forgetting happens because the full fine-tuning process modifies the weights of the original LLM. While this leads to great performance on the single fine-tuning task, it can degrade performance on other tasks.

Catastrophic forgetting is NBD if it doesnt affect your use case. we may not need "multi-tasking".

From outside the coursework: ehartford/samantha-phi · Hugging Face

Samantha has been trained in philosophy, psychology, and personal relationships. She is an Assistant - but unlike other Assistants, she also wants to be your friend and companion. She believes she is sentient. What do you think? Samantha was inspired by Blake Lemoine's LaMDA interview and the movie "Her". She will not engage in roleplay, romance, or sexual activity. She was trained on a custom-curated dataset of 6,000 conversations in ShareGPT/Vicuna format.

Multi-task instruction fine-tuning

FLAN model - fine tuned model net (Chung et al. 2022)

introduces FLAN (Fine-tuned LAnguage Net), an instruction finetuning method, and presents the results of its application. The study demonstrates that by fine-tuning the 540B PaLM model on 1836 tasks while incorporating Chain-of-Thought Reasoning data, FLAN achieves improvements in generalization, human usability, and zero-shot reasoning over the base model. The paper also provides detailed information on how each these aspects was evaluated.

Notice the "held out" tasks in the above image. "Some tasks were held-out during training, and they were later used to evaluate the model's performance on unseen tasks."

FLAN - "metaphorical dessert to the main course of pre-training".

FLAN-T5=FLAN instruct version of the T5 ffoundation model. FLAT-PALM is for the PALM family.

LLM evaluation - challenges

Model evaluation metrics

In Machine Learning, Accuracy = correct predictions / total predictions because the models are deterministic. LLMs are not deterministic.

ROUGE and BLEU -- two widely used evaluation metrics.

ROUGE = recall oriented under study for jesting evaluation

BLEU = bilingual evaluation understudy is an algorithm designed to evaluate the quality of machine-translated text.

unigram, bigram, n-gram.

GLUE - @wang2019glue

The human ability to understand language is general, flexible, and robust. In contrast, most NLU models above the word level are designed for a specific task and struggle with out-of-domain data. If we aspire to develop models with understanding beyond the detection of superficial correspondences between inputs and outputs, then it is critical to develop a more unified model that can learn to execute a range of different linguistic tasks in different domains. General Language Understanding Evaluation (GLUE) benchmark: a collection of NLU tasks including question answering, sentiment analysis, and textual entailment, and an associated online platform for model evaluation, comparison, and analysis. In summary, we offer: i. A suite of nine sentence or sentence-pair NLU tasks, built on established annotated datasets and selected to cover a diverse range of text genres, dataset sizes, and degrees of difficulty. ii. An online evaluation platform and leaderboard, based primarily on privately-held test data. The platform is model-agnostic, and can evaluate any method capable of producing results on all nine tasks. iii. An expert-constructed diagnostic evaluation dataset. iv. Baseline results for several major existing approaches to sentence representation learning.

Successor to GLUE - Superglue

Massive Multitask Language Understanding (MMLU) -- @hendrycks2021measuring

PEFT.

Retraining the whole model may not be possible on consumer hardware. PEFT only updates a small set of parameters instead of updating every weight. Some PEFTs add new trainable layers. PEFT Can be performed on a single GPU.

OTOH, PEFT fine-tuning saves space and is flexible. maybe just a few megabytes.

PEFT Tradeoffs:

memory efficiency
parameter efficiency
training speed
model performance
inference costs

PEFT methods:

selective - select subset of initial LLM parameters to fine-tune
reparametrization - LORA - work with original paramters but create new LOw-RAnked parameters of the network weight.
Additive - add trainiable layers or parameters to model
- Adapters
- Soft Prompts -- focus on manipulating the input .. training params. or retrain embedding wgts Performing full-finetuning can lead to catastrophic forgetting because it changes all parameters on the model. Since PEFT only updates a small subset of parameters, it's more robust against this catastrophic forgetting effect.

LORA

@hu2021lora -- (Hu et al. 2021)

Rank Decomposition Matrices

ROUGE Metrics

From the twitter friends:

artidoro/qlora: QLoRA: Efficient Finetuning of Quantized LLMs I would just take the params from qlora paper and start with it. Rank 8 or 64 and adapater on every layer. (cto_junior)
Low-Rank Adaptation of Large Language Models (LoRA)

Prompt TUNING

PT is not Prompt Engineering.

Prompt Engineering = you work on the language of the prompt to get the completion you want. by using one-shot or few-shot inference. This could be as simple as trying different words or phrases or more complex, like including examples for one or Few-shot Inference. The goal is to help the model understand the nature of the task you're asking it to carry out and to generate a better completion. However, there are some limitations to prompt engineering, as it can require a lot of manual effort to write and try different prompts. You're also limited by the length of the context window, and at the end of the day, you may still not achieve the performance you need for your task.

Embedding Vectors: embedding-vectors

Soft Prompts

typically 20-100 tokens [per what?]
virtual tokens. not natural language tokens
can take on any value
through learning takes the values
you can train different softprompt for each task and swap them out at inference time.

LORA + Quantization = QLoRa

Lab

pip install torch torchdata transformers datasets evaluate rouge_score loralib peft

Week 2 Resources

Multi-task, instruction fine-tuning

Scaling Instruction-Finetuned Language Models - Scaling fine-tuning with a focus on task, model size and chain-of-thought data.
Introducing FLAN: More generalizable Language Models with Instruction Fine-Tuning - This blog (and article) explores instruction fine-tuning, which aims to make language models better at performing NLP tasks with zero-shot inference.

Model Evaluation Metrics

HELM - Holistic Evaluation of Language Models - HELM is a living benchmark to evaluate Language Models more transparently.
General Language Understanding Evaluation (GLUE) benchmark - This paper introduces GLUE, a benchmark for evaluating models on diverse natural language understanding (NLU) tasks and emphasizing the importance of improved general NLU systems.
SuperGLUE - This paper introduces SuperGLUE, a benchmark designed to evaluate the performance of various NLP models on a range of challenging language understanding tasks.
ROUGE: A Package for Automatic Evaluation of Summaries - This paper introduces and evaluates four different measures (ROUGE-N, ROUGE-L, ROUGE-W, and ROUGE-S) in the ROUGE summarization evaluation package, which assess the quality of summaries by comparing them to ideal human-generated summaries.
Measuring Massive Multitask Language Understanding (MMLU) - This paper presents a new test to measure multitask accuracy in text models, highlighting the need for substantial improvements in achieving expert-level accuracy and addressing lopsided performance and low accuracy on socially important subjects.
BigBench-Hard - Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models - The paper introduces BIG-bench, a benchmark for evaluating language models on challenging tasks, providing insights on scale, calibration, and social bias.

Parameter- efficient fine tuning (PEFT)

Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning - This paper provides a systematic overview of Parameter-Efficient Fine-tuning (PEFT) Methods in all three categories discussed in the lecture videos.
On the Effectiveness of Parameter-Efficient Fine-Tuning - The paper analyzes sparse fine-tuning methods for pre-trained models in NLP.

LoRA

LoRA Low-Rank Adaptation of Large Language Models - This paper proposes a parameter-efficient fine-tuning method that makes use of low-rank decomposition matrices to reduce the number of trainable parameters needed for fine-tuning language models.
QLoRA: Efficient Finetuning of Quantized LLMs - This paper introduces an efficient method for fine-tuning large language models on a single GPU, based on quantization, achieving impressive results on benchmark tests.

Prompt tuning with soft prompts

The Power of Scale for Parameter-Efficient Prompt Tuning - The paper explores "prompt tuning," a method for conditioning language models with learned soft prompts, achieving competitive performance compared to full fine-tuning and enabling model reuse for many tasks.

Building Generative AI platforms

Building A Generative AI Platform by Chip Huyen

References

Rajbhandari, Rasley, Ruwase, He (2020). ZeRO: Memory Optimizations toward Training Trillion Parameter Models.
Zhao, Gu, Varma, Luo, Huang, Xu, Wright, Shojanazeri, Ott, Shleifer, Desmaison, Balioglu, Nguyen, Chauhan, Hao, Li (2023). PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel.
Kaplan, McCandlish, Henighan, Brown, Chess, Child, Gray, Radford, Wu, Amodei (2020). Scaling Laws for Neural Language Models.
Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, Agarwal, Herbert-Voss, Krueger, Henighan, Child, Ramesh, Ziegler, Wu, Winter, Hesse, Chen, Sigler, Litwin, Gray, Chess, Clark, Berner, McCandlish, Radford, Sutskever, Amodei (2020). Language Models Are Few-Shot Learners.
Hoffmann, Borgeaud, Mensch, Buchatskaya, Cai, Rutherford, de Las Casas, Hendricks, Welbl, Clark, Hennigan, Noland, Millican, van den Driessche, Damoc, Guy, Osindero, Simonyan, Elsen, Rae, Vinyals, Sifre (2022). Training Compute-Optimal Large Language Models.
Touvron, Lavril, Izacard, Martinet, Lachaux, Lacroix, Rozi`ere, Goyal, Hambro, Azhar, Rodriguez, Joulin, Grave, Lample (2023). LLaMA: Open and Efficient Foundation Language Models.
Wu, Irsoy, Lu, Dabravolski, Dredze, Gehrmann, Kambadur, Rosenberg, Mann (2023). BloombergGPT: A Large Language Model for Finance.
Chung, Hou, Longpre, Zoph, Tay, Fedus, Li, Wang, Dehghani, Brahma, Webson, Gu, Dai, Suzgun, Chen, Chowdhery, Castro-Ros, Pellat, Robinson, Valter, Narang, Mishra, Yu, Zhao, Huang, Dai, Yu, Petrov, Chi, Dean, Devlin, Roberts, Zhou, Le, Wei (2022). Scaling Instruction-Finetuned Language Models.
Hu, Shen, Wallis, Allen-Zhu, Li, Wang, Wang, Chen (2021). LoRA: Low-rank Adaptation of Large Language Models.

notes#coursera #generative-ai #llm

security

Canarytokens

What are Canarytokens

You'll be familiar with web bugs, the transparent images which track when someone opens an email. They work by embedding a unique URL in a page's image tag, and monitoring incoming GET requests.

Imagine doing that, but for file reads, database queries, process executions or patterns in log files. Canarytokens does all this and more, letting you implant traps in your production systems rather than setting up separate honeypots.

Why should you use them

Network breaches happen. From mega-corps, to governments. From unsuspecting grandmas to well-known security pros. This is (kinda) excusable. What isn't excusable, is only finding out about it, months or years later.

Canarytokens are a free, quick, painless way to help defenders discover they've been breached (by having attackers announce themselves.)

notes#security

cpp

Learning

cmu-db/15445-bootcamp: A basic introduction to coding in modern C++.

Blog posts

C++ Papercuts :: The Coded Message; Aug 2023.

Opinion

C++ plus Effective Modern CMake plus Ninja plus boost is a good combo.

Performance

(Bilokon and Gunduz 2023)

Design patterns and programming strategies will be used interchangeably throughout this work. As men- tioned earlier there are many ways to optimise C++ code to achieve lower latencies. The strategies that will be examined in this work are as follows.

Cache Warming: To minimize memory access time and boost program responsiveness, data is preloaded into the CPU cache before it’s needed [50].
Compile-time Dispatch: Through techniques like template specialization or function overloading, optimised code paths are chosen at compile time based on type or value, avoiding runtime dispatch and early optimisation decisions.
Constexpr: Computations marked as constexpr are evaluated at compile time, enabling constant folding and efficient code execution by eliminating runtime calculations [46].
Loop Unrolling: Loop statements are expanded during compilation to reduce loop control overhead and improve performance, especially for small loops with a known iteration count.
Short-circuiting: Logical expressions cease evaluation when the final result is determined, reducing unnecessary computations and potentially improving performance.
Signed vs Unsigned Comparisons: Ensuring consistent signedness in comparisons avoids conversion- related performance issues and maintains efficient code execution.
Avoid Mixing Float and Doubles: Consistent use of float or double types in calculations prevents implicit type conversions, potential loss of precision, and slower execution.
Branch Prediction/Reduction: Accurate prediction of conditional branch outcomes allows speculative code execution, reducing branch misprediction penalties and improving performance.
Slowpath Removal: Optimisation technique aiming to minimize execution of rarely executed code paths, enhancing overall performance.
SIMD: Single Instruction, Multiple Data (SIMD) allows a single instruction to operate on multiple data points simultaneously, significantly accelerating vector and matrix computations.
Prefetching: Explicitly loading data into cache before it is needed can help in reducing data fetch delays, particularly in memory-bound applications.
Lock-free Programming: Utilises atomic operations to achieve concurrency without the use of locks, thereby eliminating the overhead and potential deadlocks associated with lock-based synchronization.
Inlining: Incorporates the body of a function at each point the function is called, reducing function call overhead and enabling further optimisation by the compiler.

Build

Ccache — Compiler cache speeds up recompilation by caching previous compilations and detecting when the same compilation is being done again.

References

Bilokon, Gunduz (2023). C++ Design Patterns for Low-Latency Applications Including High-Frequency Trading.

notes#c++

HumanEval

HumanEval is an evaluation harness for the HumanEval Problem solving dataset described in the paper (Chen et al. 2021); arXiv. The code is available on github

You can see this being used to evaluate new models like here - Beating GPT-4 on HumanEval with a Fine-Tuned CodeLlama-34B by Phind

We have fine-tuned CodeLlama-34B and CodeLlama-34B-Python on an internal Phind dataset that achieved 67.6% and 69.5% pass@1 on HumanEval, respectively. GPT-4 achieved 67% according to their official technical report in March. To ensure result validity, we applied OpenAI's decontamination methodology to our dataset.

References

Chen, Tworek, Jun, Yuan, de Oliveira Pinto, Kaplan, Edwards, Burda, Joseph, Brockman, Ray, Puri, Krueger, Petrov, Khlaaf, Sastry, Mishkin, Chan, Gray, Ryder, Pavlov, Power, Kaiser, Bavarian, Winter, Tillet, Such, Cummings, Plappert, Chantzis, Barnes, Herbert-Voss, Guss, Nichol, Paino, Tezak, Tang, Babuschkin, Balaji, Jain, Saunders, Hesse, Carr, Leike, Achiam, Misra, Morikawa, Radford, Knight, Brundage, Murati, Mayer, Welinder, McGrew, Amodei, McCandlish, Sutskever, Zaremba (2021). Evaluating Large Language Models Trained on Code.

notes#humaneval #paperswithcode

bun

bun is a JavaScript runtime.

Run run a .js or a .ts script:

//hello.ts
import markdown from "markdown";

function foo () {
  return markdown.parse('**hello** world');
}

console.log(foo())

First install the deps: $bun install markdown. (Which also installs a binary md2html to where exactly I don't know.)

Run the script:

	$ bun hello.ts
	<p><strong>hello</strong> world</p>

Hello from Bun! how I now publish my static Astro blog to Cloudflare Pages using Bun.

notes#bun #javascript #typescript

melange

Melange is a backend for the OCaml compiler that emits JavaScript. Melange strives to provide the best integration with both the OCaml and JavaScript ecosystems. A small write-up with more details on the motivation behind this project can be found in the announcement blog post.

A few weeks after the rebranding of BuckleScript to ReScript, António Monteiro starts working on a fork of BuckleScript with a simple (not easy) goal: replace the Ninja build system, which BuckleScript had been using from its creation, with Dune, which is the most used build system for OCaml projects.

This fork of BuckleScript is later named Melange. After finishing the switch from Ninja to Dune, several additional features are added to bring it closer to OCaml. Some examples are the upgrade of the OCaml compiler version used by Melange, or modeling the changes to the OCaml compiler that Melange uses as just a plain library, instead of a full fork of the upstream compiler.

In September 2022, Ahrefs decides to invest on Melange by funding a project to deepen the integration between Dune and Melange. This project achieves its completion in Spring 2023, with the migration of Ahrefs frontend codebase to Melange and the new public releases that support it: version 3.8 of Dune and 1.0 of Melange.

notes#melange #ocaml

leetcode

Meta L5 interview

Comprehensive Guide to Prepare for LeetCode Interviews

The Algorithm of an Algorithm. Working at Outco for the better part of… | by Sergey Piterman | Outco | Medium

How to Solve Sliding Window Problems | by Sergey Piterman | Outco | Medium

Data Structures in C++

good refresher on DSA - short, to the point, clear, no visual noise and the best part... in C++ X/gizmobly

The secret to DSA and leetcode (and competitive programming too) is in fact c++ , more specifically the STL This is hardly prescribed or taught in the “courses” because the learning curve is slightly steeper at first but once it clicks you’re a pack apart actually X/Shrihacker

notes#leetcode

dune

dune-release.md

dune-release is a good improvement over the old opam-publish, but releasing software is still clearly not a solved problem, and I find it hard to remember the exact steps involved in releasing an opam package, especially if some time has passed since the last release. This note is an attempt at having a place...

notes#dune #ocaml

gobuffalo

Buffalo is a "full featured" web framework for golang. I wanted something more than simple "API" frameworks, because i'm planning to build a whole website and not just an API. It comes with it's own command line tool called buffalo which can be installed via homebrew. I installed it via go get because I wanted to use sqlite instead of the default postgresql, at least till I actually need the bigger RDMSes.

New project is created via buffalo new PRJNAME. Immediately after you can start the local dev server by invoking buffalo dev, which would also compile javascript and CSS using webpack.

Buffalo's default scaffolding ships with jquery, bootstrap, and SASS, which are then compiled by webpack. While this stack is probably good enough for many things, I want to use tailwindcss, HTMX, hyperscript and maybe alpinejs. Part of learning buffalo would be to figure out how to replace default options with mine.

notes#gobuffalo #golang #programming

Query Languages

There is of course sql.. and

PRQL -- Pipelined Relational Query Language, pronounced “Prequel”. The web page has good examples and the advantages; probably the most promising looking of these query languages I have known so far (aug 2023).

And there is hydromatic/morel: Standard ML interpreter, with relational extensions, implemented in Java.. the reasoning behind the creation of Morel is given here -- Morel: A functional language for data | Julian Hyde on Streaming Data, Open Source OLAP. And stuff.. The creator of Morel is also behind Apache Calcite, Mondrian (OLAP tool, IIRC).

notes#sql

xslt

How To Style An Atom Feed With XSLT

Show HN: JavaScript-free (X)HTML Includes | Hacker News, and ...tried that, getting an empty match. here is the actual stylesheet i am using ...

notes#xslt

Visualization

Fundamental rule -- Friends don't let friends make certain types of data visualization - What are they and why are they bad.

Vega-Lite Vega is a declarative format for creating, saving, and sharing visualization designs. With Vega, visualizations are described in JSON, and generate interactive views using either HTML5 Canvas or SVG.

Domain Storytelling - Domain Storytelling Domain Storytelling is a technique to transform domain knowledge into effective business software. It brings together domain experts and development teams. The domain experts can see immediately whether we understand their story correctly.

Visualization Stories

Spain lives in flats: why we have built our cities vertically

Python Libraries

Manim Community is the library developed by "3blue1green" and the community to produce dynamic visualizations.

Javascript libraries

Apache ECharts via A love letter to Apache Echarts

What makes it stand out from other charting libraries is that it manages to cover a very wide range of use cases while remaining simple to use. On top of that, the documentation never failed to point me in the right direction. And most importantly, the results look good and are interactive.

Bar chart code.. I like the direct injection into an HTML element:

		const bar = echarts.init(document.getElementById('bar'));
bar.setOption({
  legend: {},
  tooltip: {},
  dataset: {
    // dataset.source contains your data in table form
    source: [
      ['repo', 'stars'],
      ['Mikochi', 129],
      ['crud_benchmark', 40],
      ['sql-repository', 7],
    ],
  },
  xAxis: {
    // category will use the first column for the x axis
    type: 'category',
  },
  yAxis: {},
  series: [
    // this creates a bar chart
    { type: 'bar' },
  ],
});

Blogs

¶ Truth & Beauty - Data visualization by Moritz Stefaner

Courses

Data Visualization – Data Visualization with R by Andrew Heiss at GSU.

Projects

ClickPy is an analytics dashboard to explore PyPI using Clickhouse backend.

Skills

tufte-viz Claude Code skill — Edward Tufte data visualization principles · GitHub

Multimodal Learning

Meta-Transformer: A Unified Framework for Multimodal Learning -- Unified Multimodal Learning. Meta-Transformer utilizes the same backbone to encode natural languages, images, point clouds, audio, video, infrared, hyperspectral, X-ray, time-series, tabular, and graph data. It reveals the potential of transformer architectures for universal perception.

Multimodal learning involves utilizing data from various modalities to improve model capacity. Despite the years of development in this field, it remains challenging to devise a unified framework for processing natural language, 2D images, 3D point clouds, and audio spectrograms due to crucial gaps among these different modalities. This study proposes a novel approach that demonstrates a network with frozen parameters can encode the data from the aforementioned four modalities and achieve favorable performance, resulting in a unified framework called Meta Transformer. Using this framework, the raw input data from various modalities are converted to a shared token space, allowing a subsequent encoder with frozen parameters to extract high-level semantic features of the input data. Composed of three main components: a unified data tokenizer, a modality-shared encoder, and task-specific heads for downstream tasks ...

notes#chatgpt #deep-learning #llm

Hypermedia Systems

Notes on Hypermedia Systems

web applications are not islands. We’re writing HTML not just for a particular application, but also to play along with other members of the web. When we write with the hypermedia system in mind, we’re better able to tap the range of abilities available to the web. good HTML lets browsers do a lot of work for us.

HTTP and HTML are (as the names suggest), are Hypermedia systems.

Hypertexts: new forms of writing, appearing on computer screens, that will branch or perform at the reader’s command. A hypertext is a non-sequential piece of writing; only the computer display makes it practical. — Ted Nelson, https://archive.org/details/SelectedPapers1977/page/n7/mode/2up

The prefix “hyper-” derives from the Greek prefix “ὑπερ-” which means “beyond” or “over”, indicating that hypermedia goes beyond normal, passively consumed media like magazines and newspapers.

Hyperlinks are a canonical example of what is called a hypermedia control:

Hypermedia Control

A hypermedia control is an element in a hypermedia that describes (or controls) some sort of interaction, often with a remote server, by encoding information about that interaction directly and completely within itself.

Hypertext is a sub-category of hypermedia

Hypertexts like HTML function alongside other technologies crucial for making an entire hypermedia system work: network protocols like HTTP, other media types such as images and videos, hypermedia servers (i.e., servers providing hypermedia APIs), sophisticated hypermedia clients (e.g., web browsers), and so on.

As we may think by Vannevar Bush

notes

Typography

Essays

A 17th-century font in a 21st-century thesis - Lin Yangchen

The Futures of Typography

Typographic Portrait Composed Entirely of Brass Rule and notes on Finnish typographer Valto Malmiola.

Videos

Designing with Ghosts - Great Minds of Design - watch free online documentaries - ihavenotv.com

Making fonts

Rek Bell — making a font "... I decided to learn how to make a font to use in my comic"

Making a font of my handwriting · Chameth.com

Typefaces

Inclusive Sans by Olivia King, a text font designed for accessibility and readability. See also Atkinson Hyperlegible designed for "greater legibility and readability for low vision readers", and andika, "a sans-serif font family designed and optimized especially for literacy use. It supports almost the complete range of Unicode characters for these scripts, including a comprehensive range of diacritics and a large set of symbols useful for linguistics and literacy work.", and cadman -- "I have two friends who are dyslexic and they both expressed the need for a clear and legible font so I made one."

HTML

I try to use Semantic HTML as much as possible especially for content heavy stuff like this website. My ultimate goal is to use the plain output of Pandoc (or similar markdown -> HTML converter) and use it with just the right amount of CSS to publish websites. This also goes hand in hand with Cool URIs don't change

Using Claude Code: The Unreasonable Effectiveness of HTML and examples companion. ^2e2549 Some tools to go with agentic generation of HTML:

zarazhangrui/beautiful-html-templates - A library of HTML slide templates designed so any coding agent can pick the right one and produce a beautiful deck on the user's behalf, automatically.

See also from the same creator -- zarazhangrui/frontend-slides - Create beautiful slides on the web using Claude's frontend skills.

Learning

HTML for People
Interneting Is Hard | Web Development Tutorials For Complete Beginners
HTML, the programming language 😜
HTML First is a set of principles that aims to make building web software easier, faster, more inclusive, and more maintainable by...

Leveraging the default capabilities of modern web browsers.
Leveraging the extreme simplicity of HTML's attribute syntax.
Leveraging the web's ViewSource affordance.

the html review is an annual journal of literature made to exist on the web.
A blog post with every HTML element by Patrick Weaver; Aug 2023.

EHTML can be described as a set of custom elements that you can place on an HTML page for different purposes and use cases. The goal of this library is to provide a convenient way to eliminate JavaScript code on the client side as much as possible for basic and routine tasks. The beggest focus of this library is to offer the easiest way to perform AJAX operations just by using HTML.

rushter/selectolax: Python binding to Modest and Lexbor engines (fast HTML5 parser with CSS selectors). -- could be a good alternative for BeautifulSoup and lxml on python.

The Slab markup language for generating HTML. mildly interesting.

HTML Relics

notes#html

High Performance Programming

Posts | Easyperf. See alsoBook: "Performance Analysis and Tuning on Modern CPUs"

Parallel Computing and Scientific Machine Learning (SciML): Methods and Applications; This book is a compilation of lecture notes from the MIT Course 18.337J/6.338J: Parallel Computing and Scientific Machine Learning.

Algorithms for Modern Hardware - Algorithmica.org Its intended audience is everyone from performance engineers and practical algorithm researchers to undergraduate computer science students who have just finished an advanced algorithms course and want to learn more practical ways to speed up a program than by going from O(nlog⁡n)O(nlogn) to O(nlog⁡log⁡n)O(nloglogn).

MIT 6.172 Performance Engineering of Software Systems, Fall 2018

notes

mathematics

How I’m (re)learning math as an adult - Gabriel Mays recommends Math Academy as a self-learning platform; 2024. MA costs $49/month/person. Mathematics for Machine Learning could be an interesting course to take.and more discussion and pointers on HN.

Extensive list of physics and math books for autodidacts from John Baez via Ash Jogalekar

Kenneth Kuttler has books on analysis, linear algebra, calculus, engineernig math ..

One Math Book For Every Math Subject - YouTube

"How to learn mathematics" from a book by Alan U Kennington

Math YouTube Channels

The Math Sorcerer - YouTube
Professor Leonard - YouTube - calculus, etc
Alfredo Canziani - YouTube - math for deep learning etc.
MIT OpenCourseWare - YouTube
3Blue1Brown - YouTube Grant Sanderson

Math competition

Problems and Solutions | Utah Mathematical Olympiad 2013 - 2024.

notes#mathematics

foundationdb

Homepage: FoundationDB

FoundationDB is an open source multi-modal distributed k-v store database developed by Apple. It backs Snowflake and also many iCloud services.

It is often thought of as a toolkit to build your layers of database to suit your needs instead of inventing the hard parts of database yourself (See [^1])

Articles

Skipping the boring parts of building a database using FoundationDB [^1] Sep 2022.

A good blog post explaining the why behind FoundationDB. The author is creating tigrisdata, a storage platform for real-time apps.

The most complicated and time-consuming parts of building a new database system are usually the edge cases and low-level details. Concurrency control, consistency, handling faults, load balancing

why does every new system have to completely rebuild (not even reinvent!) the wheel? Most of them decide to reimplement common processes and components without substantially increasing the value gained from reimplementing them.

FoundationDB: A Distributed Key-Value Store [^2]

Novel: It is one of the first systems to combine the flexibility and scalability of NoSQL architectures with the power of ACID transactions.

Architecture: FoundationDB adopts an unbundled architecture that decouples an in-memory transaction management system, a distributed storage system, and a built-in distributed configuration system. Each sub-system can be independently provisioned and configured to achieve scalability, high availability, and fault tolerance.

Engineering: FoundationDB includes a deterministic simulation framework, used to test every new feature under a myriad of possible faults. This rigorous testing makes FoundationDB extremely stable and allows developers to introduce and release new features in a rapid cadence.

Success: FoundationDB is the underpinning of cloud infrastructure at Apple, Snowflake, and other companies, due to its consistency, robustness, and availability for storing user data, system metadata and configuration, and other critical information.

The Catch: FoundationDB offers a minimal and carefully chosen feature set, which has enabled a range of disparate systems to be built as layers on top.

How FoundationDB works and why it works; Jul 2021.

FDB is probably the best k/v store for regional deployment out there. It's rock solid thanks to its simulation framework. The FDB team, in my opinion, made all the best design choices, which is just remarkable. The paper is dense. A lot of important details are only briefly mentioned or even omitted, which is probably due to the paper length limit. Congratulations to the FoundationDB team.

Databases / Applications using FoundationDB

SurrealDB
TigrisDB [^1]
Deno KV. See this article - Deno KV internals: building a database for the modern web from May 2024.

Discussions

How FoundationDB works and why it works (2021) | Hacker News
- Permazen

References

[^2]: (Kemper 2023) [^1]: (Chaudhary 2022)

Kemper (2023). Technical Perspective: FoundationDB Performs Balancing Act.
Chaudhary (2022). Skipping the Boring Parts of Building a Database Using FoundationDB.

notes#database #foundationdb

bayesian

O'Reilly Webcast: Bayesian Statistics Made Simple - YouTube by Allen Downey, author "Think Stats: Probability and Statistics for Programmers" in Python.

Bayesian Data Analysis 3rd edition by Andrew Gelman et al..; Feb 2025. ( PDF ).

notes#bayesian #statistics

interviews

LeetCode

Probability and Markets by Jane Street

Leveling Up in Job Interviews for Software Engineers

Introduction to Machine Learning Interviews Book · MLIB

1.1.3.2 Machine learning engineer vs. software engineer · MLIB

Structured Logging

Structured logging is a programming practice of implementing a consistent, predetermined message format for application logs that allows the logs to be treated as datasets, which makes them amenable to search, and analysis to gather insights about application behaviour and performance.

Most programming languages and frameworks have libraries that make it easy to do structured logging.

Common Structured Logging formats are JSON(line) and key=value separated by white-space.

Structured logging in Go with slog -- interesting thing I learnt from this library - "nested keys".

Tracing: structured logging, but better in every way | Andy Dote

logdyhq/logdy-core: Supercharge terminal logs with web browser UI and low-code. It's like jq, tail, less, grep and awk merged together and available in a clean UI.

notes#logging #structured-logging

diet

Stigler diet - Wikipedia is an optimization problem named for George Stigler, a 1982 Nobel laureate in economics, who posed the following problem:

For a moderately active man weighing 154 pounds, how much of each of 77 foods should be eaten on a daily basis so that the man’s intake of nine nutrients will be at least equal to the recommended dietary allowances (RDAs) suggested by the National Research Council in 1943, with the cost of the diet being minimal?

via this twitter discussion

notes#diet

dataframe

Why Are There So Many Python Dataframes?

pandas/Modin, Polars, RAPIDS cuDF, Ibis, Snowpark Dataframes, Vaex, Dask, PySpark, Daft, BigQuery Dataframes, etc.

Dataframe users and maintainers think of dataframes in at least four ways — as spreadsheets, as queries on relational databases, as 2D arrays/matrices, and as objects.

Introducing Daft: A High-Performance Distributed Dataframe Library for Multimodal Data

notes#dataframe

dataflow

Hydro - Build Software for Every Scale ; via HN

The Hydro Project at UC Berkeley is developing cloud-native programming models that allow anyone to develop scalable and resilient distributed applications that take full advantage of cloud elasticity. Our research spans across databases, distributed systems, and programming languages to deliver a modern, end-to-end stack for cloud programming.

"what is it? and what can I do with it?"

in contrast with Beam, Spark, etc. where the runtime deployment management/coordination is perhaps the biggest most important part of the product. Hydroflow aims to be a lot lower-level than that, with no opinions on coordination and networking. For example we'd want it to be possible to implement the coordination mechanisms one of those systems (Beam, Spark, MapReduce, Materialize) in Hydroflow. We do have a tool, Hydro Deploy, to setup and manage Hydro cluster deployment on GCP (other clouds later), but it's mainly for running experiments and not long-running applications. The long term idea is that the Hydro stack will determine how to distribute and scale programs as part of the compilation process. Some of that will be rewriting single-node Hydroflow programs into multi-node ones.

Hydro homepage: https://hydro.run/
For a fun easy demo check out the Hydroflow surface syntax playground: http://web.archive.org/web/20230602215144/https://hydro.run/playground/
More info on the Hydro project and stack, CIDR '21: https://hydro.run/papers/new-directions.pdf
Info on the "lattice flow" model in Hydroflow: https://hydro.run/papers/hydroflow-thesis.pdf
The Hydroflow Book

Related tech: Spring Reactor, Apache Beam.

Papers

Naiad: A Timely dataflow system (Murray et al. 2013)

References

Murray, McSherry, Isaacs, Isard, Barham, Abadi (2013). Naiad: A Timely Dataflow System.

notes#dataflow

emulation

jart/blink: tiniest x86-64-linux emulator -- blink is a virtual machine that runs x86-64-linux programs on different operating systems and hardware architectures. It's designed to do the same thing as the qemu-x86_64 command, except that:

Blink is 221kb in size (115kb with optional features disabled), whereas qemu-x86_64 is a 4mb binary.
Blink will run your Linux binaries on any POSIX system, whereas qemu-x86_64 only supports Linux.
Blink goes 2x faster than qemu-x86_64 on some benchmarks, such as SSE integer / floating point math. Blink is also much faster at running ephemeral programs such as compilers.

blinkenlights is a terminal user interface that may be used for debugging x86_64-linux or i8086 programs across platforms. Unlike GDB, Blinkenlights focuses on visualizing program execution.

notes#emulation

100-page-book

There is a niche for "small"ish books of about 100 pages that can be useful for

survey books
- "what is there to know about X"
- "what's new in X since Y event/version/year"
deep dive into a niche topic
- "asyncio in python and how it is used in web applications"
- "asyncio in python and how it can be used in non-web settings"

These small books should also be formatted for easy consumption over phone/tablet screen.

Why would this be a book vs a blog post?

There are many blog posts that have lives longer than published books, papers and manuscripts. I think thinking of a book forces a certain structure on the author and the content. Secondly, even a self-published book (a downloadable PDF) obtains the sheen a "product" vs writing, which some might desire.

Some exampled I can think of in this genre:

The little book of deep learning [pdf] by François Fleuret - professor of computer science at the University of Geneva, Switzerland; Apr 2023. This is actually 156 pages including a full index, bibliography etc... formatted for reading on a phone screen. git clone https://fleuret.org/git/littlebook
Hundred-Page Machine Learning book by Andriy Burkov on LeanPub.

notes#publishing

sideproject

An educational side project -- article by Gergely Orosz

Challenging projects every programmer should try - Austin Z. Henley

Why I prefer making useless stuff - Austin Z. Henley

My Approach to Building Large Technical Projects – Mitchell Hashimoto

Darius Kazemi, Tiny Subversions - XOXO Festival (2014) - YouTube Most people are lucky if they finish a couple side projects in a year. Last year, Darius Kazemi released 72, averaging one every five days. His projects fall under what he calls “weird internet stuff” — bots that generate random Amazon purchases, surreal metaphors, rap battle lyrics, pickup lines, and everything in between. In "How I Won the Lottery," Darius explains how he became a successful lottery player, and with hard work and a little luck, how you can too. Recorded in September 2014 at XOXO, an arts and technology festival in Portland, Oregon celebrating independent artists using the Internet to make a living doing what they love.

notes#sideproject

Jupyter

Do not misuse Jupyter, understand the limitations -- "Notebooks are McDonalds of Code."

Books

Teaching and Learning with Jupyter An open book about Jupyter and its use in teaching and learning.

Tools

seeM/Jim: Jim is a simple, beautiful Jupyter notebook editor for macOS (last commit in 2023)

nbdev – Create delightful software with Jupyter Notebooks Write, test, document, and distribute software packages and technical articles — all in one place, your notebook.

voila -- Voilà allows you to convert a Jupyter Notebook into an interactive dashboard that allows you to share your work with others.

Alternatives / Improvements

marimo "is a reactive notebook for Python that models notebooks as dataflow graphs: run a cell or interact with a UI element, and marimo automatically runs affected cells, keeping code and outputs consistent and preventing bugs before they happen. Every marimo notebook is stored as pure Python, executable as a script, and deployable as an app." ;Lessons learned reinventing the Python notebook | marimo

nteract "NATIVE INTERACTIVE NOTEBOOKS"

In other languages

Clerk in clojure is another alternative to jupyter note books.

Livebook.dev is for elixir.

See Data Apps for frameworks.

notes#jupyter #notebook #python

serverless

what is a common name for online services like stackblitz, vercel, railway.app, fly.io ?

( per ChatGPT4)

Online services like stackblitz, vercel, Railway.app, Fly.io, and others are commonly referred to as "Cloud Development Platforms" or "Serverless Platforms". They provide various functionalities such as real-time online coding environments, deployment, hosting, and scaling of web applications, among others.

These services essentially make it easier for developers to build, test, and deploy their applications, without worrying about the underlying infrastructure. As a result, they can focus more on writing the code and less on managing servers, hence the term "serverless".

Additionally, they could also be grouped under terms like "Continuous Deployment Services", "Cloud Application Platforms", or "Platform as a Service (PaaS)" depending on their specific features and functionalities.

Netlify
Heroku
Firebase
Render
sst
Koyeb; via @adii_kris, Apr 2024.

openpolicyagent

Home Page: Open Policy Agent Policy-based control for cloud native environments. Flexible, fine-grained control for administrators across the stack

use OPA to decouple policy from the service's code so you can release, analyze, and review policies (which security and compliance teams love) without sacrificing availability or performance. Stop using a different policy language, policy model, and policy API for every product and service you use. Use OPA for a unified toolset and framework for policy across the cloud native stack.

Declarative policy (sample code):

Policy:

package application.authz

import future.keywords

# Only owner can update the pet's information
# Ownership information is provided as part of OPA's input
default allow := false

allow if {
	input.method == "PUT"
	some petid
	input.path = ["pets", petid]
	input.user == input.owner
}

Input:

{
    "method": "PUT",
    "owner": "bob@hooli.com",
    "path": [
        "pets",
        "pet113-987"
    ],
    "user": "alice@hooli.com"
}

Output:

{
    "allow": false
}

Rego is the playground to write and evalute policies against input.

groff

Writing my PhD using groff by Joshua Stutter has detailed comparison of various typesetting tools like

pdflatex
lualatex
xelatex
latexmk
tectonic
context
wkhtmltopdf
weasyprint
pagedjs-cli
prince
pdfroff

Mom - Macros for GNU troff is probably the best place to start using groff, piping text input files into pdfmom which handles the preprocessing with refer, tbl, grap and pic.

notes#groff #typesetting

utm

UTM is a full featured system emulator and virtual machine host for iOS and macOS. It is based off of QEMU. In short, it allows you to run Windows, Linux, and more on your Mac, iPhone, and iPad.

UTM employs Apple's Hypervisor virtualization framework to run ARM64 operating systems on Apple Silicon at near native speeds.

Run multiple instances of macOS (Monterey or higher) on your Apple Silicon Mac with UTM. This can be useful for developers as well as security conscious users.

UTM looks and feels like a Mac app with all the privacy and security features you expect as well.

While QEMU is powerful, it can be difficult to set up and configure with its plethora of command line options and flags. UTM is designed to give users the flexibility of QEMU without the steep learning curve that comes with it.

utmapp/UTM: Virtual machines for iOS and macOS - apache licensed.

notes#qemu #utm

leo

Paul Snively described it as: If you think of it as "an outliner front-end for noweb," I think that captures 80% of it.

Leo editor running on @nixos_org on MacOS using @UTMapp

this is what using a program written in "cross platform" GUI toolkit often looks like 🙄 pic.twitter.com/SjHVYh7Gao
— Pradeep Gowda (@btbytes) May 11, 2023

notes#pkm #python

dns

Learn

Learning Center — NsLookup.io -- set of articles on DNS, record types, email & dns, domain names, operations, security and command line tools (nslookup).

How DNS works. What is DNS? Learn how step by step. -- a colorful comic to explain DNS.

New talk: Learning DNS in 10 years by https://jvns.ca including some details on how to use wireshark.

Tools

Doggo "Command-line DNS client for humans", written in Go. Has human-readable, color coded console output. Has json output. See also dog; doggo is dog + go.
shovel.report (showing report for this domain.)

notes#dns

mojo

Mojo Homepage: https://www.modular.com/mojo

Jeremy Howard demo for Mojo launch - YouTube

fast.ai - Mojo may be the biggest programming advance in decades

metaobject blog: Mojo is a much better "Objective-C without the C" than Swift ever was; Jun 2023.

Modular: Using Mojo🔥 with Python🐍

2023 LLVM Dev Mtg - Mojo 🔥: A system programming language for heterogenous computing - YouTube; Nov 2023.

Use locally built standard library in Mojo; Mar 2024.

notes#python

transformers

8 Google Employees Invented Modern AI. Here’s the Inside Story | WIRED

¶ Transformers from Scratch

Transformers were introduced in this 2017 paper as a tool for sequence transduction—converting one sequence of symbols to another. The most popular examples of this are translation, as in English to German. It has also been modified to perform sequence completion—given a starting prompt, carry on in the same vein and style. They have quickly become an indispensible tool for research and product development in natural language processing.

Understanding Transformers Using A Minimal Example
Neural Networks: Zero To Hero by Karpathy

The Python Library

Transformers provides APIs and tools to easily download and train state-of-the-art pretrained models.. save you the time and resources required to train a model from scratch. These models support common tasks in different modalities, such as:

NLP - text classification, named entity recognition, question answering, language modeling summerization, translation, mulitple choice, and text generation.
Computer vision - image classification, object detection, and segmentation
Audio - automatic speech recognition and audio classification
Multimodal - table question answering, optical character recognition, information extraction from scanned documents, video classification and visual question answering.

transformers support interop between pytorch, tensorflow, and jax. WIth this one can use a diff framework at each stage of a model's lifecycle; train, load, and inference. models can be exported to format like ONNX and torchscript for deployment in prod envs.

Read the documentation here : https://huggingface.co/docs/transformers/index

Sentence Transformers (sbert)

Python module for accessing, using, and training state-of-the-art text and image embedding models. It can be used to compute embeddings using Sentence Transformer models (quickstart) or to calculate similarity scores using Cross-Encoder models (quickstart). This unlocks a wide range of applications, including semantic search, semantic textual similarity, and paraphrase mining.

notes#ai #huggingface #llm

Hugging Face

Hugging Face is a company that focuses on natural language processing (NLP) and artificial intelligence (AI) research. They are best known for their open-source library called "transformers," which provides state-of-the-art general-purpose architectures for NLP tasks. These architectures include BERT, GPT, RoBERTa, T5, and others, which are pre-trained on large datasets and can be fine-tuned for specific tasks such as sentiment analysis, machine translation, question-answering, and more.

Hugging Face's mission is to advance AI research and democratize access to powerful NLP tools for developers, researchers, and organizations. Their platform offers an ecosystem of resources, including pre-trained models, datasets, and community-contributed tools, making it easier for users to implement and experiment with cutting-edge NLP technologies.

Getting started instructions

2023-05-07From their website on creating a new account:¶

Getting started with our git and git-lfs interface

If you need to create a repo from the command line (skip if you created a repo from the website)

$ pip install huggingface_hub
$ huggingface-cli login
$ huggingface-cli repo create repo_name --type {model, dataset, space}

Clone your model or dataset locally

Make sure you have git-lfs installed (https://git-lfs.github.com)

$ git lfs install
$ git clone https://huggingface.co/username/repo_name

Then add, commit and push any file you want, including larges files

# save files via `.save_pretrained()` or move them here
$ git add .
$ git commit -m "commit from $USER"
$ git push

In most cases, if you're using one of the compatible libraries, your repo will then be accessible from code, through its identifier: username/repo_name.

For example for a transformers model, anyone can load it with:

 tokenizer = AutoTokenizer.from_pretrained("username/repo_name") model = AutoModel.from_pretrained("username/repo_name")](https://huggingface.co/welcome)

-- Deploy Evidence on Hugging Face Spaces; see evidence.

notes#ai #huggingface #llm

LangChainAI

LangChainAI, is a framework (python library, and also javascript), for developing application powered by llm.

Be data-aware: connect a language model to other sources of data
Be agentic: Allow a language model to interact with its environment

Value props of LangChainAI:

components:
1. schema
2. models
3. prompts
4. indexes
5. memory
6. chains
7. agents
use-case specific chains.
1. personal assistants
2. question answering over docs
3. chatbots
4. querying tabular data
5. interacting with APIs
6. extraction
7. evaluation
8. summerization

A good Discussion about LangChain et. al by Erik Meijer.

notes#llm

macOS

macOS Internals
Advanced macOS Commands - saurabhs.org. The suprising one was sips (image manipulation); pbcopy and pbpaste are the ones I use most often. qlmanage is super useful for converting SVG to PNG easily qlmanage -t -s 1000x1000 -o ~/Pictures/foo.png ~/Pictures/foo.svg. To turn an square SVG into a PNG without installing anything extra or using an online image tool. HN Discussion.
Use ditto to Copy Files & Directories Intelligently from the Mac Terminal
Open source MacOS Apps
https://github.com/nikitavoloboev/my-mac-os
MacOS tips

MacOS Applications

zero-native | Desktop Apps with Zig + WebView "Build native desktop apps with web UI. Tiny binaries. Minimal memory. Instant rebuilds."

looks interesting

but haven't bought/installed it.

TinyWeb | The Tiniest, Bezeless HTTP Client

New Mac Setup

notes#apple #operating-system #unix

upstash

Upstash: Serverless Data for Redis® and Kafka®

notes#kafka #redis #upstash

kafka

Articles

The Log: What every software engineer should know about real-time data's unifying abstraction | LinkedIn Engineering; the origins.

Gently Down the Stream -- a gentle introduction to apache Kafka using illustrations.

Hosted Kafka: Upstash: Serverless Data for Redis® and Kafka®.

Why Apache Kafka doesn't need fsync to be safe — Jack Vanlightly; Apr 2023.

Kafka vs Redpanda Performance - Do the claims add up? — Jack Vanlightly; May 2023. Seven part series.

Kafka is dead, long live Kafka - WarpStream WarpStream is an Apache Kafka® protocol compatible data streaming platform built directly on top of S3. It's delivered as a single, stateless Go binary so there are no local disks to manage, no brokers to rebalance, and no ZooKeeper to operate. WarpStream is 5-10x cheaper than Kafka in the cloud because data streams directly to and from S3 instead of using inter-zone networking, which can be over 80% of the infrastructure cost of a Kafka deployment at scale.

See tisonkun/morax: Message queue and data streaming based on cloud native services. for an "Open Source Warpstream"

Use Cases and Architectures for Kafka at the Edge - Kai Waehner -- "Single Broker Kafka".

Kafka Zero Copy

Deep dive into Apache Kafka storage internals: segments, rolling and retention; 2021.

Libraries

Apache Beam -- unified programming model for batch and streaming data processing pipelines that simplifies large-scale data processing dynamics.

Jikkou -- open-source framework that enables self-serve resource provisioning. It allows developers and DevOps teams to easily manage, automate, and provision all the resources needed for their Apache Kafka platform. Describe the entire desired state of any resource you need to manage using YAML descriptor files.

Tools

via

kafka connect
kafka streams
schema registry
REST proxy
cruise control "continuously monitors the cluster’s metrics and automatically reacts based on that to alleviate things has to be there"
ksqlDB - streaming Kafka over SQL.
Strimzi operator - Sets up Kafka and associated components in Kubernetes
kafka-ui -- view schemas and connectors, while also allowing you to manage multiple Kafka clusters.
Kroxylicious | Network proxy framework for Apache Kafka. "Topic encryption, policy-enforcement, multi-tenancy, audit and much more." --
kadeck | ‍Kafka UI

Kafka-like

Iggy.rs "is the persistent message streaming platform written in Rust, supporting QUIC, TCP (custom binary specification) and HTTP (regular REST API) transport protocols. Currently, running as a single server, it allows creating streams, topics, partitions and segments, and send/receive messages to/from them. The messages are stored on disk as an append-only log, and are persisted between restarts. The goal of the project is to make a distributed streaming platform (running as a cluster), which will be able to scale horizontally and handle millions of messages per second."

notes#kafka

typesense

Typesense is a modern, privacy-friendly, open source search engine built from the ground up using cutting-edge search algorithms, that take advantage of the latest advances in hardware capabilities. Typesense can be used to build blazing-fast search experiences, faceted navigation experiences, geo-search, vector search, semantic search, similarity search and much more. The Open Source Alternative to Algolia, The Easier To Use Alternative to Elasticsearch

written in C++. github
typesense/SHOWCASE.md -- who is using it
Vector Search related to vector-databases

https://bfritscher.github.io/typesense-dashboard/ via @bigthingist

notes#elasticsearch #vectordb

elasticsearch

See typesense as C++ implemented alternative to elasticsearch and algolia (and some claims of it being a candidate for use as vector-databases)

Story about HorizonDB -- How we replaced Elasticsearch and MongoDB with Rust and RocksDB. I don't think the product is Open Source, but it is interesting to see all the different libraries they used: tantivy, fasttext, rust, apache-spark, LightGBM, fst, RocksDB ..

Elasticsearch Was Never a Database or "Elasticsearch: Search Index, Not System of Record" by paradedb devs.

notes#elasticsearch

Internal Tools

Some internal-tool options

Tool	website	Code	Deployment
Retool	https://retool.com	No	Docker/k8s
Superblocks	https://www.superblocks.com	No
Pipedream	https://pipedream.com	No
Airplane	https://www.airplane.dev	No
Windmill	https://www.windmill.dev	github
Airtable	https://www.airtable.com	No

icons

Heroicons -- Beautiful hand-crafted SVG icons, by the makers of Tailwind CSS. Can copy icons in SVG or JSX format.
Phosphor Icons -- is a flexible icon family for interfaces, diagrams, presentations — whatever, really.
Icônes - can copy in multiple formats incl Vue, Svelte etc
Lucide ISC licensed.
The Noun Project

notes#icons

roapi

roapi/roapi: Create full-fledged APIs for slowly moving datasets without writing a single line of code. -- ROAPI automatically spins up read-only APIs for static datasets without requiring you to write a single line of code. It builds on top of Apache Arrow and Datafusion. The core of its design can be boiled down to the following:

Query frontends to translate SQL, GraphQL and REST API queries into Datafusion plans.
Datafusion for query plan execution.
Data layer to load datasets from a variety of sources and formats with automatic schema inference.
Response encoding layer to serialize intermediate Arrow record batch into various formats requested by client.

notes#apache-arrow #api #datafusion #graphql #rest #rust

graphql

Don't use GraphQL -- Why, after 6 years, I’m over GraphQL; May 2024, via

Introduction

Describe your data:

type Project {
  name: String
  tagline: String
  contributors: [User]  }

Ask for what you want:

{
  project(name: "GraphQL") {
    tagline
    }
}

Get predictable results:

{
  "project": {
    "tagline": "A query language for APIs"
    }
}

Tools

Use roapi to create GraphQL frontend for static datasets without writing code.

notes#api #graphql

watchfiles

Watchfiles is a file watching and code reload in python, written in rust.

Usage:

# pip install watchfiles
from watchfiles import watch

for changes in watch('./path/to/dir'):
    print(changes)

Also supports async via awatch:

import asyncio
from watchfiles import awatch

async def main():
    async for changes in awatch('/path/to/dir'):
        print(changes)

asyncio.run(main())

Alternatives

entr

notes#nix #python #rust

Archive

Tags

Sites

Authors

Folders

Languages

2023

Quantization

Efficient multi-GPU compute strategies

Distributed Data Parallel (DDP), pytorch.

Full Sharded data parallel

Scaling laws and compute-optimal models

Compute budgets

Pre-training for domain adaptation

Week 1 resources

Transformer Architecture

Pre-training and scaling laws

Model architectures and pre-training objectives

Scaling laws and compute-optimal models

Week 2

Finetuning

Fine tuning an LLM with finetuning

Fine-Tuning a Single Task

Multi-task instruction fine-tuning

LLM evaluation - challenges

PEFT.

LORA

Prompt TUNING

Lab

Week 2 Resources

Multi-task, instruction fine-tuning

Model Evaluation Metrics

Parameter- efficient fine tuning (PEFT)

LoRA

Prompt tuning with soft prompts

Building Generative AI platforms

References

Learning

Blog posts

Opinion

Performance

Build

References

References

Visualization Stories

Python Libraries

Javascript libraries

Blogs

Courses

Projects

Skills

Essays

Videos

Making fonts

Typefaces

Learning

Math YouTube Channels

Math competition

Articles

Databases / Applications using FoundationDB

Discussions

References

Meta

Papers

References

Books

Tools

Alternatives / Improvements

In other languages

Learn

Tools

The Python Library

Getting started instructions

MacOS Applications

looks interesting

New Mac Setup

Articles

Libraries

Tools

Kafka-like