Archive

Cog: A Code Generation Tool Written in Python; this is apparently an old tool (blog date 2004), that I had never heard about before. But, the idea of code generation is a useful one, and something Lisps do really well, and is a core part of their identity "code is data".

nim among other languages also have this "metaprogramming" ability.

Do not misuse Jupyter, understand the limitations -- "Notebooks are McDonalds of Code."

Books

Teaching and Learning with Jupyter An open book about Jupyter and its use in teaching and learning.

Tools

seeM/Jim: Jim is a simple, beautiful Jupyter notebook editor for macOS (last commit in 2023)

nbdev – Create delightful software with Jupyter Notebooks Write, test, document, and distribute software packages and technical articles — all in one place, your notebook.

voila -- Voilà allows you to convert a Jupyter Notebook into an interactive dashboard that allows you to share your work with others.

Alternatives / Improvements

marimo "is a reactive notebook for Python that models notebooks as dataflow graphs: run a cell or interact with a UI element, and marimo automatically runs affected cells, keeping code and outputs consistent and preventing bugs before they happen. Every marimo notebook is stored as pure Python, executable as a script, and deployable as an app." ;Lessons learned reinventing the Python notebook | marimo

nteract "NATIVE INTERACTIVE NOTEBOOKS"

In other languages

See also: srcbookdev/srcbook: JS/TS notebook for typescript.

Clerk in clojure is another alternative to jupyter note books.

Livebook.dev is for elixir.

See Data Apps for frameworks.

Paul Snively described it as: If you think of it as "an outliner front-end for noweb," I think that captures 80% of it.

Leo editor running on @nixos_org on MacOS using @UTMapp

this is what using a program written in "cross platform" GUI toolkit often looks like 🙄 pic.twitter.com/SjHVYh7Gao

— Pradeep Gowda (@btbytes) May 11, 2023

Mojo Homepage: https://www.modular.com/mojo

Jeremy Howard demo for Mojo launch - YouTube

fast.ai - Mojo may be the biggest programming advance in decades

metaobject blog: Mojo is a much better "Objective-C without the C" than Swift ever was; Jun 2023.

Modular: Using Mojo🔥 with Python🐍

2023 LLVM Dev Mtg - Mojo 🔥: A system programming language for heterogenous computing - YouTube; Nov 2023.

Use locally built standard library in Mojo; Mar 2024.

Watchfiles is a file watching and code reload in python, written in rust.

Usage:

# pip install watchfiles
from watchfiles import watch

for changes in watch('./path/to/dir'):
    print(changes)

Also supports async via awatch:

import asyncio
from watchfiles import awatch

async def main():
    async for changes in awatch('/path/to/dir'):
        print(changes)

asyncio.run(main())

Alternatives

• entr

Top level notes

• Pandas Documentation Homepage
• Installing (2.0) -- pip install pandas==2.0
  • installing pandas will also install numpy
• Pandas helps in working with tabular data (databases, spreadsheets)
• Helps = explore, clean and proess
• Anything you can do with SQL you can also do with Pandas.
  • Group by operations
• Summary Statistics. eg., mean, median, std..
• The data table is called a DataFrame
• pandas can read (read_ methods) from various file formats like csv, xls, parquet, hdf5, json, sql (db) and generate output (to_ methods) in the above formats.
• matplotlib is used to generated plots from data tables.
• split-apply-combine approach. Hmm.. map-reduce?
• melt() to convert from wide -> long/tidy form and pivot() to convert from long to wide format.
• combine multiple tables using concat() function for column-wise or row-wise joining
• for database like joining/merging, use the merge function. pd.merge(tbone, tbtwo, how="left", left_on="col1", right_on="col2"). also merge_asof
• pandas supports inner, outer, right and left joins.
• use shape() to find the rowxcolumn shape.
• TODO: how vectorization is done in pandas
• TODO: where numpy is faster than pandas
• TODO: Pyspark with pandas

Quick intro

import numpy as np
import pandas as pd

# create new Series
s = pd.Seres([1,2,3, pd.nan, 8, 121])
# pd.nan = not a number

# pd.date_range("<YYYYMMDD", periods=n) produces a
# a date range starting at YYYYMMDD for the period `n`

# dataframes are created with

df = pd.DataFrame(nparray, index=<indexvec>, columns=list(names))
# the nparray is a 2-d numpy array of mxn dimensions

# the df can also be created by passing a dict with keys as the
# column names and the value as the series.

Articles

• Practical SQL for Data Analysis | Haki Benita in comparison to pandas.

wilsonzlin/minify-html : Extremely fast and smart HTML + JS + CSS minifier, available for Rust, Deno, Java, Node.js, Python, Ruby, and WASM

The Python library on pypi.

Using the Python library:

# code/example_minifyhtml.py
#!/usr/bin/env python

import minify_html

with open("output/index.html") as f:
    t = f.read()
    print(t)
    o = minify_html.minify(
        t,
        do_not_minify_doctype=True,
        keep_spaces_between_attributes=True,
        ensure_spec_compliant_unquoted_attribute_values=True,
    )
    print(o)

JAX is a Python library which augments numpy and Python code with function transformations which make it trivial to perform operations common in machine learning programs. Concretely, this makes it simple to write standard Python/numpy code and immediately be able to

Compute the derivative of a function via a successor to autograd Just-in-time compile a function to run efficiently on an accelerator via XLA Automagically vectorize a function, so that e.g. you can process a “batch” of data in parallel. learn more ... "You don't know Jax" (2019)

Jax's website - google/jax: Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

JAX and Equinox: What are they and why should I bother?