Why you procrastinate even when it feels bad

(below is the transcript of the above YouTube video with my highlights)

It’s 5 p.m. and you’ve just realized that report you’ve been putting off is due tomorrow. It’s time to buckle down, open your computer... and check your phone. Maybe catch up on your favorite YouTube channel? Actually, you should probably make dinner first. You usually like cooking, though it’s hard to enjoy with this work hanging over your head, and oh— it’s actually pretty late! Maybe you should just try again in the morning? This is the cycle of procrastination, and I promise you, we have all been there. But why do we keep procrastinating even when we know it’s bad for us? To be clear, putting something off isn’t always procrastinating. Responsible time management requires deciding which tasks are important and which ones can wait.

Procrastination is when we avoid a task we said we would do, for no good reason, despite expecting our behavior to bring negative consequences.&

Obviously, it’s irrational to do something you expect to harm you.

But ironically, procrastination is the result of our bodies trying to protect us, specifically by avoiding a task we see as threatening.

When you realize you need to write that report, your brain responds like it would to any incoming threat. Your amygdala, a set of neurons involved in emotional processing and threat identification, releases hormones including adrenaline that kick off a fear response. This stress-induced panic can overpower the impulses from your prefrontal cortex, which typically help you think long term and regulate your emotions. And it’s in the midst of this fight, flight, or freeze response that you decide to handle the threat by avoiding it in favor of some less stressful task.

This response might seem extreme— after all, it’s just a deadline, not a bear attack. But we’re most likely to procrastinate tasks that evoke negative feelings, were more likely to put off tasks they perceived as stressful or challenging. In one experiment, students were given reminders to study throughout the day. While they were studying, most reported that it wasn’t so bad. But when they were procrastinating, Because procrastination is motivated by our negative feelings, and those who struggle with low self-esteem are much more likely to procrastinate, regardless of how good they are at time management.

However, it's a common misconception that all procrastinators are lazy. In the body and brain, laziness is marked by no energy and general apathy. When you’re feeling lazy, you’re more likely to sit around doing nothing than distract yourself with unimportant tasks. In fact, many people procrastinate because they care too much.

Procrastinators often report a high fear of failure, putting things off because they’re afraid their work won’t live up to their high standards.

Whatever the reason for procrastination, the results are often the same. Frequent procrastinators are likely to suffer from anxiety and depression, ongoing feelings of shame, higher stress levels and physical ailments associated with high stress. Worst of all, while procrastination hurts us in the long run, it does temporarily reduce our stress level, reinforcing it as a bodily response for coping with stressful tasks. So, how can we break the cycle of procrastination? Traditionally, people thought procrastinators needed to cultivate discipline and practice strict time management. But today, many researchers feel the exact opposite.

Being too hard on yourself can layer additional bad emotions onto a task, making the threat even more intense.

To short-circuit this stress response, we need to address and reduce these negative emotions. Some simple strategies include

• breaking a task into smaller elements
• or journaling about why it's stressing you out and addressing those underlying concerns.
• try removing nearby distractions that make it easy to impulsively procrastinate.
• and more than anything, it helps to cultivate an attitude of self-compassion, forgiving yourself, and making a plan to do better next time.

Because a culture that perpetuates this cycle of stress and procrastination hurts all of us in the long term.

Introductory Materials

Generative AI exists because of the transformer -- A visual story from Financial Times; Sept 2023.

Large language models, explained with a minimum of math and jargon

A High-level Overview of Large Language Models - Borealis AI

Tiny LLM - LLM Serving in a Week -- uses MLX (on MacOS) to  ... implement an LLM serving project from scratch -- with only matrix manipulations APIs, so that I can understand what it takes to load those LLM model parameters and do the math magic to generate text.

Books

• Language Models in Plain English by Austin Eovito and Marina Danilevsky published by IBM on O'Reilly.
• Transformers & Large Language Models | Super Study Guide

Study Guides

-Mastering LLMs workshop. See Takeaways from Mastering LLMs Course from SwaroopCH.

• The Novice's LLM Training Guide; copy
• Normcore LLM Reads by Vicki Boykis
• A Hackers' Guide to Language Models - YouTube by jeremy-howard
• Transformer Math 101 | EleutherAI Blog
• Lil'Log; lilian weng's blog
• The maths you need to start understanding LLMs :: Giles' blog

Blogs and websites

• Lil'Log Hi, this is Lilian. I’m documenting my learning notes in this blog. Other than writing a ML blog, I’m leading Applied Research at OpenAI on the side.
• Finbarr Timbers -- eg: Five years of GPT progress
• Vespa Blog
• Lycee.AI blog
• Notes on AI; someone's obsidian notes on AI.

Papers

• Collection of LLM papers: GitHub - horseee/Awesome-Efficient-LLM: A curated list for Efficient Large Language Models by Xinyin Ma.
• Microsoft LMOps -- a research initiative on fundamental research and technology for building AI products w/ foundation models, especially on the general technology for enabling AI capabilities w/ LLMs and Generative AI models.
• Sparks of Artificial General Intelligence: Early experiments with GPT-4. (2023) PDF -- (Bubeck et al. 2023)
• SeamlessM4T—Massively Multilingual & Multimodal Machine Translation | Meta AI Research -- (Barrault et al. 2023)
• (Brown et al. 2020)

Videos

An observation on Generalization - YouTube by Ilya Sutskever (OpenAI); Aug 14, 2023.

• Supervised Learning - precise mathematical condition under which learning should succeed, which is - Low training error + more training data than "degrees of freedom" = low test error

Prompt Engineering

• ChatGPT Prompt Engineering for Developers - DeepLearning.AI
• Prompt Engineering | Lil'Log -- "This post only focuses on prompt engineering for autoregressive language models, so nothing with Cloze tests, image generation or multimodality models."
• Controllable Neural Text Generation | Lil'Log -- "How to steer a powerful unconditioned language model? In this post, we will delve into several approaches for controlled content generation with an unconditioned langage model. For example, if we plan to use LM to generate reading materials for kids, we would like to guide the output stories to be safe, educational and easily understood by children."
• Replacing my best friends with an LLM trained on 500,000 group chat messages
• microsoft/guidance: A guidance language for controlling large language models. Guidance programs allow you to interleave generation, prompting, and logical control into a single continuous flow matching how the language model actually processes the text.

Models

See this page - Models Table – Dr Alan D. Thompson – Life Architect for a visual representation of models, and a table of various attributes of models.

Stuff you can run on your computer

Everything I've learned so far about running local LLMs; Nov 2024.

smol-ai/developer: with 100k context windows on the way, it's now feasible for every dev to have their own smol developer

How is LLaMa.cpp possible? how can we run llama.cpp on local machines when the expectation is that large models need expensive GPUS (eg: A100) to run

Introducing Code Llama, a state-of-the-art large language model for coding Code Llama is a code-specialized version of Llama 2 that was created by further training Llama 2 on its code-specific datasets, sampling more data from that same dataset for longer. Essentially, Code Llama features enhanced coding capabilities, built on top of Llama 2. It can generate code, and natural language about code, from both code and natural language prompts (e.g., “Write me a function that outputs the fibonacci sequence.”) It can also be used for code completion and debugging. It supports many of the most popular languages being used today, including Python, C++, Java, PHP, Typescript (Javascript), C#, and Bash.

Ask HN: Cheapest way to run local LLMs? | Hacker News

See also Perplexity Labs where they have multiple models to try from

   

• Simon Willison's simonw/language-models-on-the-command-line: Handout for a talk I gave about LLM and CLI tools

• gptscript-ai/gptscript: Build AI assistants that interact with your systems "GPTScript is a framework that allows Large Language Models (LLMs) to operate and interact with various systems. These systems can range from local executables to complex applications with OpenAPI schemas, SDK libraries, or any RAG-based solutions. GPTScript is designed to easily integrate any system, whether local or remote, with your LLM using just a few lines of prompts."

LLMs in your language

All languages are NOT created (tokenized) equal

Small Language Models

• (Eldan and Li 2023)

Building LLMs

• naklecha/llama3-from-scratch - llama3 implementation one matrix multiplication at a time

Using LLMs

• A Simple but Powerful Method to Analyze Online PDFs with Bing Chat - AI Demos
• What's new in Llama 2 & how to run it locally - AGI Sphere; Aug 2023.
• Fine-tune your own Llama 2 to replace GPT-3.5/4 | Hacker News

Fine-tuning has one huge advantage though: it is far more effective at guiding a model's behavior than prompting, so you can often get away with a much smaller model. That gets you faster responses and lower inference costs. A fine-tuned Llama 7B model is 50x cheaper than GPT-3.5 on a per-token basis, and for many use cases can produce results that are as good or better! For example, classifying the 2M recipes at https://huggingface.co/datasets/corbt/all-recipes with GPT-4 would cost $23k. Even with GPT-3.5 it would cost over $1k. The model we fine-tuned performs similarly to GPT-4 and costs just $19 to run over the entire dataset. OpenBMB/ToolBench: An open platform for training, serving, and evaluating large language model for tool learning.

Frameworks

LlamaIndex 🦙 0.8.13

Haystack | Haystack Open-source LLM framework to build production-ready applications.

Use the latest LLMs: hosted models by OpenAI or Cohere, open-source LLMs, or other pre-trained models All tooling in one place: preprocessing, pipelines, agents & tools, prompts, evaluation and finetuning Choose your favorite database: Elasticsearch, OpenSearch, Weaviate, Pinecone, Qdrant, Milvus and more Scale to millions of documents: use Haystack's proven retrieval architecture Compare it to LangChainAI

GPT4All A free-to-use, locally running, privacy-aware chatbot. No GPU or internet required.

AI Proxy

an AI proxy that lets you use a variety of providers (OpenAI, Anthropic, LLaMa2, Mistral, and others) behind a single interface w/ caching & API key management.

MLC LLM Machine Learning Compilation for Large Language Models (MLC LLM) is a high-performance universal deployment solution that allows native deployment of any large language models with native APIs with compiler acceleration. The mission of this project is to enable everyone to develop, optimize and deploy AI models natively on everyone’s devices with ML compilation techniques.

Project Overview of MLC LLM

Multimodal Learning

todo

Security

OWASP | Top 10 for Large Language Models

Abliteration

Uncensor any LLM with abliteration

Operational Issues

• Why won’t Llama13B fit on my 4090_.pptx - Google Slides by Mark Saroufim

GGUF

GGUF and GGML are file formats used for storing models for inference, especially in the context of language models like GPT

• GGUF, the long way around by Vicki Boykis

Benchmarking LLMs

LLM Benchmark Report for: NousResearch/Redmond-Puffin-13B

Linkdump

2023

• Shoggoth  is a peer-to-peer, anonymous network for publishing and distributing open-source code, Machine Learning models, datasets, and research papers.
• What We Know About LLMs (Primer) #primer
• A comprehensive guide to running Llama 2 locally - Replicate – Replicate; Jul 22, 2023.
• PaLM2 Technical Report #pdf #google
• dalai -- Run LLaMA and Alpaca on your computer. ??
• ggerganov/llama.cpp: Port of Facebook's LLaMA model in C/C++
• LlamaIndex - (GPT Index) is a project that provides a central interface to connect your LLM's with external data. see <https://x.com/gpt_index)
• LLM Introduction: Learn Language Models
• Announcing OpenFlamingo: An open-source framework for training vision-language models with in-context learning | LAION
• become a 1000x engineer or die tryin'
• Simon Willison: LLMs on personal devices Series.
• How You Can Install A ChatGPT-like Personal AI On Your Own Computer And Run It With No Internet.
• How to run your own LLM (GPT); Apr 2023.
• modal-labs/quillman A complete chat app that transcribes audio in real-time, streams back a response from a language model, and synthesizes this response as natural-sounding speech. This repo is meant to serve as a starting point for your own language model-based apps, as well as a playground for experimentation.
• A brief history of LLaMA models - AGI Sphere
• See LangChainAI
• ray-project/llm-numbers: Numbers every LLM developer should know; via HN
• Beginner's guide to Llama models - AGI Sphere; Aug 2023.
• The Mathematics of Training LLMs — with Quentin Anthony of Eleuther AI
• Google Gemini Eats The World – Gemini Smashes GPT-4 By 5X, The GPU-Poors; Aug 2023.
• fast.ai - Can LLMs learn from a single example?

2024

• Stuff we figured out about AI in 2023

• Large language models use a surprisingly simple mechanism to retrieve some stored knowledge | MIT News | Massachusetts Institute of Technology

• Survey of Open source repos -- What I learned from looking at 900 most popular open source AI tools (via)

• (Allen-Zhu and Li 2024)

• (You et al. 2024)

• About BERT -- (Geiping and Goldstein 2022) -- via

• (Ma et al. 2024); github repo (the repo link on paper wasn't working as of 2024-04-17).

• "phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3.5 (e.g., phi-3-mini achieves 69% on MMLU and 8.38 on MT-bench), despite being small enough to be deployed on a phone." (Abdin et al. 2024) (No code, or model was announced with the paper.)

• (@liu2024kan)

• (Lin et al. 2024) "Unstructured data formats account for over 80% of the data currently stored, and extracting value from such formats remains a considerable challenge.... ZenDB efficiently extracts semantic hierarchical structures from such templatized documents, and introduces a novel query engine that leverages these structures for accurate and cost-effective query execution. Users can impose a schema on their documents, and query it, all via SQL. Extensive experiments on three real-world document collections demonstrate ZenDB's benefits, achieving up to 30% cost savings compared to LLM-based baselines, while maintaining or improving accuracy, and surpassing RAG-based baselines by up to 61% in precision and 80% in recall, at a marginally higher cost."

• Call to Build Open Multi-Modal Models for Personal Assistants | LAION

• What We Learned from a Year of Building with LLMs (Part I) – O’Reilly

• What We Learned from a Year of Building with LLMs (Part II) – O’Reilly

• ARC Prize

  • (Chollet 2019) by François Chollet.

  • "To make deliberate progress towards more intelligent and more human-like artificial systems, we need to be following an appropriate feedback signal: we need to be able to define and evaluate intelligence in a way that enables comparisons between two systems, as well as comparisons with humans."

  • "We then articulate a new formal definition of intelligence based on Algorithmic Information Theory, describing intelligence as skill-acquisition efficiency and highlighting the concepts of scope, generalization difficulty, priors, and experience, as critical pieces to be accounted for in characterizing intelligent systems."

• (Ren et al. 2024)

2026

References

• Meta Reference - letsbuild.ai
• Keeping up with AGI reading list by cto_junior.
• LLM Course by mlabonne

---

See also: Multimodal Learning, Multi Agent Frameworks, AI Agent Framework, generative-ai, smol-llm, transformer-math, LlamaIndex, RAG, local-llm, llm-embedding, AI SaaS, LLM Training, Reward Models, AI Code Assistants, Building LLM Based Systems, LLMs in Data Management

• Bubeck, Chandrasekaran, Eldan, Gehrke, Horvitz, Kamar, Lee, Lee, Li, Lundberg, Nori, Palangi, Ribeiro, Zhang (2023). Sparks of Artificial General Intelligence: Early Experiments with GPT-4.
• Barrault, Chung, Meglioli, Dale, Dong, Duquenne, Elsahar, Gong, Heffernan, Hoffman, others (2023). SeamlessM4T-Massively Multilingual & Multimodal Machine Translation.
• Brown, Mann, Ryder, Subbiah, Kaplan, Dhariwal, Neelakantan, Shyam, Sastry, Askell, Agarwal, Herbert-Voss, Krueger, Henighan, Child, Ramesh, Ziegler, Wu, Winter, Hesse, Chen, Sigler, Litwin, Gray, Chess, Clark, Berner, McCandlish, Radford, Sutskever, Amodei (2020). Language Models Are Few-Shot Learners.
• Eldan, Li (2023). TinyStories: How Small Can Language Models Be and Still Speak Coherent English?.
• Allen-Zhu, Li (2024). Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws.
• You, Zhang, Schoop, Weers, Swearngin, Nichols, Yang, Gan (2024). Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs.
• Geiping, Goldstein (2022). Cramming: Training a Language Model on a Single GPU in One Day.
• Ma, Yang, Xiong, Chen, Yu, Zhang, May, Zettlemoyer, Levy, Zhou (2024). Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length.
• Abdin, Jacobs, Awan, Aneja, Awadallah, Awadalla, Bach, Bahree, Bakhtiari, Behl, Benhaim, Bilenko, Bjorck, Bubeck, Cai, Mendes, Chen, Chaudhary, Chopra, Giorno, de Rosa, Dixon, Eldan, Iter, Goswami, Gunasekar, Haider, Hao, Hewett, Huynh, Javaheripi, Jin, Kauffmann, Karampatziakis, Kim, Khademi, Kurilenko, Lee, Lee, Li, Liang, Liu, Lin, Lin, Madan, Mitra, Modi, Nguyen, Norick, Patra, Perez-Becker, Portet, Pryzant, Qin, Radmilac, Rosset, Roy, Saarikivi, Saied, Salim, Santacroce, Shah, Shang, Sharma, Song, Ruwase, Wang, Ward, Wang, Witte, Wyatt, Xu, Xu, Yadav, Yang, Yang, Yu, Zhang, Zhang, Zhang, Zhang, Zhang, Zhang, Zhou (2024). Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone.
• Lin, Hulsebos, Ma, Shankar, Zeigham, Parameswaran, Wu (2024). Towards Accurate and Efficient Document Analytics with Large Language Models.
• Chollet (2019). On the Measure of Intelligence.
• Ren, Liu, Lu, Shen, Liang, Chen (2024). Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling.

Modern Java - an online book; a 2026 approach to learning Java in bite-sized lessons. Don't have to start in an IDE for one.

Internals

• Overhead of Returning Optional Values in Java and Rust | Piotr Kołaczkowski
• Writing a Profiler in 240 Lines of Pure Java | Mostly nerdless

Articles

• Use Java for Everything (2014) and related Java in the small (2024).
• Java Best Practices | Jonathan Giles

Interesting libraries

Permazen Permazen is a ground-up rethink of Java persistence. It has been in production use in commercial software since 2015.

 Instead of starting from the storage technology side, it starts from the programming language side, asking the simple question, "What are the issues that are inherent to persistence programming, regardless of programming language or database storage technology, and how can they be addressed at the language level in the simplest, most correct, and most language-natural way?"

JAX is a Python library which augments numpy and Python code with function transformations which make it trivial to perform operations common in machine learning programs. Concretely, this makes it simple to write standard Python/numpy code and immediately be able to

Compute the derivative of a function via a successor to autograd Just-in-time compile a function to run efficiently on an accelerator via XLA Automagically vectorize a function, so that e.g. you can process a “batch” of data in parallel. learn more ... "You don't know Jax" (2019)

Jax's website - google/jax: Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

JAX and Equinox: What are they and why should I bother?

Notes from "Making it Work"

Six Horizons of Focus

---

Other GTD links

Getting things done (in small increments), tactics:

• Make progress every day.
• Always know what the next thing to do is
• Pay attention to feedback loops
• For bigger things, make a plan and keep a log
• Block distractions

Mac

• I Use Hammerspoon with ShiftIT
• Magnet – Window manager for Mac
• Window Manager for Mac OS - VEEER

Windows

• LGUG2Z/komorebi: A tiling window manager for Windows written in Rust

Linux

There are plenty of window managers for Linux; Some iI have tried

• xmonad
• i3.

widgets

• elkowar/eww: ElKowars wacky widgets
• EWW Powermenu - dead airspace

Learning Git

Why Git is hard Confusing git terminology; Nov 2023 by Julia Evans.

Things I wish everyone knew about Git ) and (Part II) by Mark Dominus ( plover.com )

Inside Git by jvns.

eficode-academy/git-katas, A set of exercises for deliberate Git Practice

Oh My Git, learn git through playing games.

Artisanal Handcrafted Git Repositories | drew's dev blog -- "I’m going to talk about how to handmake your git repositories without using these silly git commands. You might also learn a bit more about how git works under the hood during the process, or whatever."

So You Think You Know Git - FOSDEM 2024 - YouTube by Scott Chacon, now creator of Git Butler

USEFUL CONFIG STUFF

$ git config --global alias.staash 'stash --all'
$ git clean

https://gist.gihub.com/schacon

$ git config --global alias.bb !better-branch.sh

Conditional Configs
[includeif "gitdir:~/projects/work/"]
    path = ~/projects/work/.gitconfig
[includeif "gitdir:~/projects/oss/"]
    path = ~/projects/oss/.gitconfig

OLDIES BUT GOODIES

$ git blame -L
git blame a portion of the file

$ git log -L 15,26:path/to/file
works the same way with git log

$ git blame -w
ignore whitespace

$ git blame -w -C
.. and detect lines or copies in the same commit

$ git blame -w -C -C
.. or the commit that created the file

$ git blame -w -C -C -C
.. or any commit at all

$ git log -S files_watcher -p
the pickaxe

$ git reflog

$ git diff --word-diff

$ git config --global rerere.enabled true
REuse REcorded REsolution

NEW STUFF

$ git branch --column

$ git config --global column.ui auto
$ git config --global branch.sort -committerdate

$ git push --force-with-lease
This one is better than using --force by default

signing commits with ssh
$ git config gpg.format ssh

$ git push --signed
github and gitlab do not support it.

$ git maintenance

gc(disabled),
commit-graph and prefetch (hourly),
loose-objects incremental-repack (daily)
pack-refs (none)

BIG REPO STUFF

Half a million files in working directory

windows rpeo - 300gb, 3.5 million files
linux kernel - 4.5gb in comparison

prefetching
$ git for-each-ref | grep prefe

commit-graph
$ git config --global fetch.writeCommitGraph true

filesystem monitor
$ git config core.untrackedcache true
$ git config core.fsmonitor true

partial cloning
$ git clone --filter=blob:none
$ git clone --filter=tree:0

GITHUB STUFF

auto merge

merge queue

$ git ls-remote

"all mypull requests are actual refs"

$ git config remote.origin.fetch '+refs/pull/*:refs/remotes/origin/pull/*'

$ git fetch


GITBUTLER

virtual branches

Tools

jesseduffield/lazygit: simple terminal UI for git commands

https://gitless.com -- A Git-compatible VCS that does away with the staging area (and stashes), and instead has separate working dirs for every branch (branch = automatically stashed). They have a detailed analysis of Git that explains their design: What's wrong with Git? A conceptual design analysis #pdf

chshersh/zbg: ✨ Zero bullshit git, porcelain for git in ocaml

Git man page generator; funny.

Jujutsu/jj - A Git-compatible DVCS that is both simple and powerful -- It combines features from Git (data model, speed), Mercurial (anonymous branching, simple CLI free from "the index", revsets, powerful history-rewriting), and Pijul/Darcs (first-class conflicts), with features not found in most of them (working-copy-as-a-commit, undo functionality, automatic rebase, safe replication via rsync, Dropbox, or distributed file system). Basic jj Workflows

GitHub - jnsahaj/lumen: Beautiful git diff, Instant AI Git Commit message, Git changes summary from the CLI (no API key required) -- A command-line tool that uses AI to streamline your git workflow - generate commit messages, view diff and explain changes, all without requiring an API key.

Git workflows

• Why some of us like "interdiff" code review systems (not GitHub); via lobste.rs

Hosted git

Cloudflare Artifacts --

• Elixir for Humans Who Know Python · Hibox
• The Soul of Erlang and Elixir • Sasa Juric • GOTO 2019 - YouTube
• Understanding the Elixir Machine Learning Ecosystem
• Elixir Saves Pinterest $2 Million a Year In Server Costs
• From $erverless to Elixir. I recently rewrote a service that was… | by Cory O'Daniel | CoryODaniel | Medium
• Elixir and Phoenix can do it all! · The Phoenix Files -How we build forms in LiveView + LiveSvelte

GPT = Generative Pre-trained Transformer

• GPT in 60 Lines of NumPy | Jay Mody
• How GPT3 Works - Visualizations and Animations – Jay Alammar – Visualizing machine learning one concept at a time.
• Transformers from Scratch
• The GPT-3 Architecture, on a Napkin; HN
• GPT-3 - Wikipedia
• A GPT in 60 Lines of NumPy
• Let's build GPT: from scratch, in code, spelled out. - YouTube by Andrej Karpathy.
• mlx-gpt2: gpt-2 from scratch in mlx code from GPT from scratch with MLX
• karpathy/minGPT: A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training
• TheR1D/shell_gpt: A command-line interface (CLI) productivity tool powered by OpenAI's text-davinci-003 model, will help you accomplish your tasks faster and more efficiently.; Get API keys here.
• Simply explained: how does GPT work? | Confused bit
• GPT4 should be part of your toolkit • Buttondown
• imartinez/privateGPT: Interact privately with your documents using the power of GPT, 100% privately, no data leaks -- a really promising
• Understanding GPT tokenizers
• [2106.06981] Thinking Like Transformers (Weiss et al. 2021) and explanation blog post - Thinking Like Transformers by Alexander Rush

See: AutoGPT

References

• Weiss, Goldberg, Yahav (2021). Thinking Like Transformers.

The Missing Semester of Your CS Education -- You should do this self taught course from MIT whether if you have a degree in CS or not. This is my top recommendation for learning the craft of programming/development.

Classes teach you all about advanced topics within CS, from operating systems to machine learning, but there’s one critical subject that’s rarely covered, and is instead left to students to figure out on their own: proficiency with their tools. We’ll teach you how to master the command-line, use a powerful text editor, use fancy features of version control systems, and much more!

Open Source Society University - computer-science: Path to a free self-taught education in Computer Science! -- links to various MOOCs for pretty much all the topics covered in a CS undergraduate curriculum.

A Functional Introduction To Computer Science (Part I) and Part II

CS Primer Courses (This is a paid course) -- The TL;DR is:

• Start wherever looks interesting
• Focus on solving problems
• Use as much or as little of the problem walkthroughs and explainer videos as you need to keep making progress and having fun
• Keep going!

Some links from Archive.org

• Panchatantra in Karnataka Sculptures by Channabasappa Patil S.
• Lilavati's daughers, the women scientists of India.
• The Kautilya Arthashastra - part I by R P Kangle in sanskrit.
• Kautilya arthashasra - part II by R P Kangle (PDF)
• Lord Siva And His Worship : Sri Swami Sivananda : Free Download, Borrow, and Streaming : Internet Archive
• ದುರ್ಗಸಿಂಹ ಕವಿಯ ಕರ್ನಾಟಕ ಪಂಚತಂತ್ರಂ : Gundmi Chandrashekara Aithal : Free Download, Borrow, and Streaming : Internet Archive
• Gramadevatas A Study Of The Feminine Divine In Bangalore Rural District 16th To 19th A D : Gomathi, Manohar (2012) from the Internet Archive.

Books

• Mathematical Engineering of Deep Learning (online book). (Liquet et al. 2024)
• Deep Learning with Python by François Chollet (Google; Keras, TensorFlow, ARC Challenge)
• The little book of deep learning [pdf] by François Fleuret - professor of computer science at the University of Geneva, Switzerland; Apr 2023. #phoneformatted. See also 100-page-book
• Book | Alfredo Canziani (in progress...)
• udlbook/Understanding Deep Learning by Simon J.D. Prince. Has 68 Jupyter Notebooks, which can be downloaded from github. Plots are drawn with -- Inkscape with the texText plugin to do the equations. Free and highly recommended. I draw the axes and ticks by hand because it looks nicer than anything from PyPlot etc. 2023-11-15 ¶
• Deep Learning: Foundations and Concepts by Chris Bishop of PRML book fame.
• [2106.10165] The Principles of Deep Learning Theory

Papers

• Browse the State-of-the-Art in Machine Learning | Papers With Code

Articles

• From Deep to Long Learning? · Hazy Research
• A Completely Non-Technical Explanation of AI and Deep Learning - Parand

Learning Paths

• Learning From Data - Online Course (MOOC)
• MIT Deep Learning 6.S191
• AI for Beginners from Microsoft
• Data Science for Beginners from Microsoft
• Machine Learning for Beginners from Microsoft
• Stanford University CS231n: Deep Learning for Computer Vision
• Stanford CS229: Machine Learning I Spring 2022 - YouTube
• UNIGE 14x050 – Deep Learning course by François Fleuret, who also authored "The Little book of deep learning"
• Practical Deep Learning for Coders from fast.ai, the book - Deep Learning for Coders with fastai and PyTorch: AI Applications Without a PhD
  • fast.ai – Alternatives to a Degree to Prove Yourself in Deep Learning

Courses

Sargur Shrihari at Uni Buffalo

• Deep Learning prerequisites
  • Introduction to Machine Learning: Course Materials
  • Probabilistic Graphical Models: Course Slides

Videos

• Practical Deep Learning for Coders - YouTube by Jeremy Howard
• after the above watch this this from 2019 Deep Learning from the Foundations - YouTube
• Endless Experimentation - building models in the real world by Nikolas Laskaris and Douglas Blank from CometAI at MIT intro to deep learning

Blogs

• kipply's blog
• Lilian Weng's blog archive

People

• Demis Hassabis - Wikipedia CEO of deepmind.
• Alec Radford, researcher at OpenAI; google scholar page

References

• Liquet, Moka, Nazarathy (2024). Mathematical Engineering of Deep Learning.

Attention Conservation Notice

This specifically refers to software systems. Perhaps even more narrowly to Distributed Systems that involve Data, Web etc., Very unlikely hardware/embedded etc., because I do not deal in those.

Everything I know about good system design by sean goedecke. This article links to a few others he has writte on the topic.

software design is how you assemble lines of code, system design is how you assemble services. The primitives of software design are variables, functions, classes, and so on. The primitives of system design are app servers, databases, caches, queues, event buses, proxies, and so on.

---

Popular Service Level Indicators (SLIs):

• Correctness
• Availability
• Throughput
• Response Time ( p50, p99, etc)
• Functional requirements describe the SLI for correctness.
• Non-functional requirements describe the SLIs for availability, throughput, response time etc.

Service Level Objective (SLO):

• A target value (or range) for an SLI. (e.g.: SLI <= target).
• non-functional requirements for scaling, like - data size, bulky service time, geolocation, and data hotspots, to satisfy SLOs.

Service Level Agreement (SLA) is a contract with users on what the SLOs are, including the consequences of meeting/missing the SLOs. (e.g.: penalty/rebate).

---

videos on system design

Distributed Systems in One Lesson by Tim Berglund

Google Systems Design Interview With An Ex-Googler

My System Design Template - LeetCode Discuss

ByteByteGo | Ace Your Next System Design Interview

How to crawl a quarter billion webpages in 40 hours – DDI

CS 329S - Machine Learning Systems Design; "an iterative framework for developing real-world machine learning systems that are deployable, reliable, and scalable."

MLOps Requirements

---

visenger/awesome-mlops: A curated list of references for MLOps

MLOps References

Papers

Operationalizing machine learning: An interview study (Shankar et al. 2022)

The process of operationalizing ML, or MLOps, consists of a continual loop of (i) data collection and labeling, (ii) experimentation to improve ML performance, (iii) evaluation throughout a multi-staged deployment process, and (iv) monitoring of performance drops in production. When considered together, these responsibilities seem staggering -- how does anyone do MLOps, what are the unaddressed challenges, and what are the implications for tool builders? We conducted semi-structured ethnographic interviews with 18 MLEs working across many applications, including chatbots, autonomous vehicles, and finance. Our interviews expose three variables that govern success for a production ML deployment: Velocity, Validation, and Versioning. We summarize common practices for successful ML experimentation, deployment, and sustaining production performance.

Unsolved Problems in MLOps - ACM Queue by Niall Murphy and Todd Underwood

Platforms

• Ray Distributed Computing - Anyscale

Frameworks

Chronon  is an open source end-to-end feature platform that allows Machine Learning (ML) teams to easily build, deploy, manage and monitor data pipelines for machine learning. It’s currently used to power all major ML applications within Airbnb, as well as major use cases at Stripe. Airbnb and Stripe jointly manage and maintain the project.

[via ](https://docs.google.com/document/d/10o75QTLfkaHCiiT2BftBld8ImuTojefZ03AYm__L9yw/edit) / [HN](https://news.ycombinator.com/item?id=41709547)

Blog posts

• MLOps is Mostly Data Engineering. • Kostas Heaven on Net

References

• Shankar, Garcia, Hellerstein, Parameswaran (2022). Operationalizing Machine Learning: An Interview Study.

Why Tracebit is written in C#; Jan 2025. "Tracebit deploys and maintains tailored security canaries, proactively detecting intrusions across your organization.  From cloud infrastructure, to identity and endpoints."

---

bflat - C# as you know it but with Go-inspired tooling (small, selfcontained, and native executables)

• Comes with two standard libraries: one (DotNet) that is compatible with .NET and supports everything from console to JSON, and another (Zero) that is stripped to bare minimum and doesn't even have a GC.
• Run C# with no operating system, on bare metal
• Build native code or .NET bytecode
• Based on parts of the .NET 7 SDK
• Invoke bflat directly from terminal, integrate with a shell script, Makefile, or even MSBuild.

Deno — A modern runtime for JavaScript and TypeScript. Learn it -- Deno by Example

How Deno works especially the bits about Extensions, web workers, and building your own runtime.

Why Deno

• <3 Deno by matklad

Applications written in Typescript

• SilverBullet

Libraries

• Deno KV - a global database for global apps -- uses sqlite locally and foundationdb on the hosted service using the same interface.
  • Deno KV | Simon Willison’s TILs has some interesting implementation details behind the local k/v store backed by sqlite. (also found out how to use the activity monitor on macOS).
• Ultra: The Quest for Zero-Legacy Ultra is an all ESM React/Deno framework that is built for Suspense Server Side Rendering.  Write ESM, ship ESM, simplify your life. ESM stands for ES modules.

Tooling

• Velociraptor | The script runner for Deno

Frameworks

• Lume, I especially like the ebook theme for publishing online books like 37signals' ONCE — Writebook

Show HN: SinglePage – Quickly and anonymously publish a page to the web | Hacker News; see some interesting options to publish single pages on the web -- telegra.ph, carrd.co, Write.as, Neocities, nekoweb, mmm.page — Dead Simple, Drag & Drop Websites. Of these carrd, and mmm* have visual tools, and the rest are simple text publishing tools.

I recommend Neocities if all you want to do is upload HTML files (whether handwritten or produced by an editor).

If you are someone familiar with using git, you also have the option of using github pages, and codeberg pages, of these github will give you CI/CD system in case you want to build a website from source files (eg: markdown -> HTML)

Cloudflare pages also has the CI option, in addition giving lot more features in dealing with DNS, caching, routing etc.. This is what I would recommend to a technical savvy person.

None of the above systems support (afaik), any kind of dynamic server side programming (eg: PHP, Python, Node), nor do they have traditional databases like MySQL support.

---

pgs.sh -- A zero-install static site hosting service for hackers

A note on various podcast episodes I have heard (partially or fully). Starting in Aug 2025, I will try to make an attempt to note what I have heard.

• 2025-08-02 Adam Van Buskirk (@Empty_Ameri… - Subversive w/Alex Kaschuta - Apple Podcasts ^61a2aa

Recommendations to check out

Episodes Recommended by others from the CoRecursive Podcast:

• The Untold Story of SQLite With Richard Hipp - CoRecursive Podcast
• From 486 to Vue.js Evan You's Full-Time Gamble on Open Source - CoRecursive Podcast
• Software World Tour With Son Luong Ngoc - CoRecursive Podcast

Corecursive episodes by topic: Episodes organized by theme. - CoRecursive Podcast

Podcast Tech

• Transistor - podcast hosting for creatives, brands, professionals; about $190/yr. Found via Oxide and Friends
• 2025-08-02I recently got curious about how popular podcasters host thier shows, specifically after listening to Naval Ravikant's short 3-5 minute episodes. I noticed that he hosted the podcast on Libsyn. It does appear to be quite affordable for $5/mo for the "bare essentials".