here you goThings I’ve made, read and saved — shared here.

Archive

9,631 entries: 60 from X, 8,925 from Pinboard, saved 1998 to 2026.

Browse by tag, site, author, folder or language

Tags

Sites

Authors

Folders

Languages

Tags and folders

Showing 451–500 of 547Folder notes ×

2020

Clojure programming language

Clojure is a Lisp that runs on the Java Virtual Machine (JVM).

Learning resources

Clojure as a Lisp

Courseware

Presentations

Clojure like languages

IDE and infrastructure

  • Lightmod -- Lightmod is a tightly integrated tool for making full stack Clojure and ClojureScript web apps. It is fully self-contained -- no need to install the JDK or any build tool. It hot reloads both client and server code automatically with zero setup.

  • Nightlight is an editor you run inside your Clojure projects. Instead of being a separate application like traditional editors and IDEs, it's just a library that you add to your project. It spins up a web server and provides a completely browser-based interface to edit your code.

  • CIDER is the successor to nrepl.

  • Cursive Clojure IDE -- is an IDE built on IntelliJ

  • Parinfer

REPL

online playgrounds

Getting started with Lighttable

Via

Basic instructions. I use cmd for the shortcuts below, substitute ctrl on non-mac platforms.

  1. Install Leiningen http://leiningen.org/ you do not need to install clojure, lein will grab clojure for you.
  2. lein new playground from the command line to create a 'playground' project
  3. Open Light Table
  4. ctrl-space woto for "Toggle Workspace tree"
  5. Add the playground project folder you just created
  6. cmd-o core.clj to open the main clojure file for the project
  7. On an empty line type (+ 1 1) and hit cmd-enter

Things will grind away for a while as all the dependencies are downloaded and the JVM starts up. Get a cup of coffee or something. Eventually you'll see a green 2 show up.

You can then type anything in the tutorial and hit cmd-enter to evaluate it or cmd-shift-enter to evaluate the entire file. Think of the file as a shell with an editable history instead of something you'd want to commit to source control.

Documentation is searchable via ctrl-shift-d or moving your cursor on top of a function name and hitting ctrl-d. Autocomplete is available on tab, you can jump to definition with cmd-. and back with cmd-,

TODO: Explore CIDER + AC(autocomplete) + ritz.

Applications

Libraries

Spec

the biggest win from spec has been with parsing. This completely changes how you'd write a library that takes a data structure and parses it into something meaningful (for example, what hiccup does for html or what honeysql does for sql). In the past, this required a lot of very ugly parsing code and manual error-checking. With spec, you write specs and call s/conform. If it failed, you get a nice error, especially if you pair it with expound. If it succeeded, you get a destructured value that is really easy to pull data out of. I've done this in a half dozen different libraries and i'm pretty sure i wouldn't have even written them without spec.

Mallimalli and specmonstahspecmonstah are my favourite Clojure(Script) libraries built on top of Clojure Spec.

Clojure books

The first chapter, Names, explains why names define the structure of our software, and how to judge whether a name is any good.

The second chapter, Idioms, provides specific, syntactic advice for writing Clojure which is clean and readable.

The third chapter, Indirection, looks at how code can made simpler and more robust through separation.

The final chapter, Composition, explores how the constituent pieces of our code can be combined into an effective whole.

Clojurescript books

Experience reports and discussions

Articles

Talks and presentations

Tips and tricks

Cool projects / Read Code

Visualization / Notebook

Web development

Big data

Blogs

ClojureScript

Graal

  • This nextjournal notebook has a recipe for building Babashka "executing Clojure snippets or scripts in the same space where you would use Bash."

Web development

PCP

alekcz/pcp: PCP: Clojure Processor -- Like drugs but better

Just use it like PHP on a webserver (say nginx):

    #new pcp config => change .php to .clj
    index index.clj index.html index.htm;
    #new pcp config => change .php to .clj
    location ~ \.clj$ {
        #default nginx config on digital ocean lemp image
        #include snippets/fastcgi-php.conf;
        #fastcgi_pass unix:/run/php/php7.0-fpm.sock;

        include scgi_params;
        scgi_intercept_errors on;
        scgi_pass   127.0.0.1:9000;
    }

PCP has following libraries built in:

(require  '[cheshire.core :as json]
      '[clostache.parser :as clo]
      '[selmer.parser :as parser]
      '[selmer.filters :as filters]
      '[clj-http.lite.client :as client]
      '[next.jdbc :as jdbc]
      '[honeysql.core :as sql]
      '[honeysql.helpers :as helpers]
      '[postal.core :as email]
      '[clojurewerkz.scrypt.core :as sc])

Learning Journal

  • this is a good demo of how to use Clojure's -Sdeps at the command line to execute a script (even one hosted remotely on git):
clojure -Sdeps '{:deps
    {hello-clojure
        {:git/url "https://gist.github.com/athos/b68b15b08efedffaf14d8c020b125202"
        :sha "1c9a05106171f97f9b8e8ac8f58c7bd8dbe021f9"}}}'
    -m hello-clojure

See also: lisp, scheme.

notes#clojure#jvm#lisp#programming-language

Javascript

News

Learning

WAT

JS Alternatives

--

Reading

Functional programming

Tooling

Libraries

Visualization

  • Fabric.js Javascript Canvas Library -- via
  • Two.js -- two-dimensional drawing api geared towards modern web browsers. It is renderer agnostic enabling the same api to draw in multiple contexts: svg, canvas, and webgl. Jul 2015.

Frameworks

It is similar to JavaScript frameworks such as React and Vue, which share a goal of making it easy to build slick interactive user interfaces. But there's a crucial difference: Svelte converts your app into ideal JavaScript at build time, rather than interpreting your application code at run time. This means you don't pay the performance cost of the framework's abstractions, and you don't incur a penalty when your app first loads. You can build your entire app with Svelte, or you can add it incrementally to an existing codebase. You can also ship components as standalone packages that work anywhere, without the overhead of a dependency on a conventional framework.

The Bandolier: A Learn JavaScript the Hard Way Project by Zed Shaw. A fresh approach, and since it was built to teach JS programming, it should be easy to grok.

Links

Apps

Books

Talks

See also: webpack

notes#javascript#programming-language

Org Mode

In some users view, org-mode is itself reason enough to learn to use emacs, even if you don't use it for anything else (programming).

Zettelkasten

GitHub - nobiot/Zero-to-Emacs-and-Org-roam: Step by step guide from zero to installing and setting up Emacs and Org-roam on Windows 10

Org mode for writing web documents

While Org-mode is used by many people as an organizer, TODO list tracker, journal etc., I'm interested in it as primarily as a markup language that can be extended. This is is primary area where I want to become good at vis-a-vis orgmode (Jan 2020).

web document friendly CSS themes

See my Org-simple CSS theme page.

Blogging with Org mode

Links

Org Libraries

My notes

  • the metadata tags at the top of the document start with #+TAG: value format. That is "hash plus tag colon value" format.
  • use the - character for un numbered lists
  • what are the keyboard shortcuts to type blocks like EXAMPLE and CODE. Typing C-c C-, brings up a menu using which i could select an src block or an example block.

notes#emacs#programming#tools#writing

Emacs Notes

This page is a journal of my experiments with emacs. In November of 2008, I decided to use emacs for all my text editing purposes, after long been convinced of Emacs's power and utility in editing programs and texts. At this time, I consider myself to be an adequate user of vim, emacs and TextMate. My goal is to teach myself emacs in an auto-didactic manner and make a note of the questions I had and the solutions I found for them. I hope this serves as a reminder for myself on a later date.

Note: All the instructions on this page, unless otherwise noted, are tested on Mac OS X systems and Aquamacs.

Python + Emacs

Install rope, ropemode, pep8 and pyflakes.

$ pip install ropemode pep8 pyflakes

Install yasnippet-bundle and pymacs in emacs:

M-x package-install yasnippet-bundle

References:

Emacs Client on Mac OS X

Here is a nice AppleScript created by Brian McCallister that provides proper emacsclient behaviour. Update: this script launches a "terminal" window, which is an annoyance.

Navigation

Go to Shortcut

Goto line M-gg. End of the buffer M-x > End of the line C-e Beginning of the line C-a

Editing

Do what Shortcut

zap-to-char (say e) M-z e merge multiple lines M-^ (alt-shift-6)

Customising emacs

Customizing emacs is done through editing ~/.emacs file. Though, if you are using emacs-starter-kit you will be putting files under emacs.d.

Adding a new key binding.

You can bind any operation that you can do in emacs to a set of key strokes. For example: the emacs-git integration -- magit can be invoked by typing M-x magit-status. But, this gets tiring after a while, so you may want to assign a short cut. Short cuts in Emacs start with 'C-c'. You have to decide what comes after C-c. In this case, I chose magit-status.

Add the following lines to your .emacs file:

;; C-c ms Magit-status
(global-set-key (kbd "C-c ms") 'magit-status)

The first line is the comment (a good practice) and the second line tells emacs to call magit-status when you press C-c ms.

After editing a custom function in .emacs file how do I reload it?

Select the block of code that you want to reload and hit C-x C-e (evaluate). This can also be achieved by positioning the cursor (point in emacs lingo) after the last closing parentheses of the function and typing C-x C-e.

Editing ConTEXt files with Emacs

Install ConTEXt. Install AUCTEX using port (on mac).

$ sudo port install auctex

Active AucTEX by putting this (load "auctex.el" nil t t) in .emacs.

Auto completion.

Install YASnippet to get auto-completion on emacs. To use YASnippet, type in the trigger word and press TAB. For example, typing def and pressing TAB will give you

    def name():
        """
        """

The next tab press allows you replace the default function name with your own, with subsequent tabs taking you to the parameter list, docstring and finally the body of the function

What programming font?

This page has a nice collection of free programming fonts optimized for editing code over long durations of code. The rule of thumb to remember while selecting a programming font is: it should be a fixed width font. This is especially true in languages like "Python":Python where indentation is essential in understanding the code flow.

I currently use Monaco, a font that is available on Mac OS X set to size 10.

Emacs Muse Notes

Muse tutorial.

Instead of editing the muse-latex.el and muse-book.el files, just use customize on the muse-latex-header and muse-book-latex-header variables. Upgrades to new versions of muse will be much smoother.

Emacs + Git integration

Main entry point M-x magit-status (bound to C-c ms). Go to the magit buffer *magit: PRJNAME* and press tab on the line containing the filenames. The modified text will expand underneath. Press TAB again collapse.

Press g in the magit buffer to refresh git status. or execute M-x magit-status from one of the repository files.

Removing trailing white spaces

The emacs manual defines tells us that setting buffer variable show-trailing-whitespace to t will show trailing white spaces.

(setq show-trailing-whitespace 't)

Paste the above line into the current emacs buffer and evaluate the statement by pressing C-x C-e. Delete the white spaces with M-x delete-trailing-whitespace

Check to see if your emacs configuration is correct.

Source

emacs -q --eval '(condition-case err (progn (load "~/.emacs") (kill-emacs 0)) (error (kill-emacs 1)))'

Shut down emacs gracefully from the command line

emacsclient -n -e '(kill-emacs)'

How to ? Command

Create a Directory from within emacs M-x make-direct ory

[Spell M-x ispell check](http://www.gnu.org/software/emacs/manual/html_no de/emacs/Spelling.html)

Undo C-x u or C-_

Describe key C-h k

Describe bindings C-h b

Describe functions C-h f

Emacs Server/Client

See Running emacs as daemon.

Setting up SLIME

Movie showing how to setup slime, swank, sbcl and emacs

Some notes:

check out slime and put it somewhere convenient, ie., $HOME/elisp/slime.

Put the following lines in .sbclrc

(require :asdf)
(push "/Users/pradeep/elisp/slime/" asdf:*central-registry*)

Load swank in sbcl:

$ sbcl
(asdf:oos 'asdf:load-op :swank)
(setf swank:*use-dedicated-output-stream* nil)
(setf swank:*communication-style* :fd-handler)
(swank:create-server :dont-close t)

Installing emacs24

What's new in emacs 24?

git clone git://git.savannah.gnu.org/emacs.git
cd emacs
sudo apt-get install libxpm-dev
sudo apt-get install libgif-dev
sudo apt-get install libtiff-dev
./configure
make
sudo make install

My initial impression of emacs24 is that it feels much faster compared to the stock emacs-snapshot available on ubuntu maverick.

Understanding Emacs

Essays on Emacs

Interesting modes

Help

Interesting reads

Dotemacs

Distributions

Packages

Information well / emacs people

TODO

Is it possible to show the keybindings in the modeline?

Use case: I have configured 'Fn' keys for mode specific actions, but do not remember them, or I want emacs to prompt me to use them.

Starting point: http://www.emacswiki.org/emacs/ModeLineConfiguration

Random tips

  • Use M-x new-frame create a new Emacs window associated with the current Emacs instance. Great for organising, or when running Emacs on a multi monitor setup!

CIDER

Journal

Fri, January 3, 2020 : Trying out Swaroop's Rangoli-emacs

Sat, Jan 4, 2020 : if you see emacs ssl certificate problem error when installing a new dependency (zencoding-mode in my case), try to launch emacs from the terminal emacs -nw instead of the graphical mode. This finds the root certificates.

Thu, Jan 16, 2020 : settings up mac os x and iterm2 for emacs. what I want is to be able to:

1. on the laptop: use the `command` key as `option` and vice versa, so that it is similar to the "regular keyboard" (I use the Das Keyboards at work and home)
2. in ther iterm2 window, be able to use left-alt key as `meta`. so that when I type `left-alt-m` I get `Meta-m`.

notes#emacs#lisp#text-editor

jq

"A post announcing the 1.0 release of pq, my Rust command-line protobuf deserializer"

asciicast

Associated programs

JSONata and gnata

  • Lightweight query and transformation language for JSON data
  • Inspired by the location path semantics of XPath 3.1
  • Sophisticated query expressions with minimal syntax
  • Built in operators and functions for manipulating and combining data
  • Create user-defined functions
  • Format query results into any JSON output structure

Read gnata blog post (Mar 2026),

gron

gron - "make JSON greppable". brew install gron

▶ **gron** "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | fgrep "commit.author"
json[0].commit.author = {};
json[0].commit.author.date = "2016-07-02T10:51:21Z";
json[0].commit.author.email = "mail@tomnomnom.com";
json[0].commit.author.name = "Tom Hudson";

Jo

See also: jo, a tool to write JSON in the shell. Example usage:

$ jo -p name=jo n=17 parser=false
{
    "name": "jo",
    "n": 17,
    "parser": false
}

jc

2020-06-01: https://github.com/kellyjonbrazil/jc This tool converts the output of popular command-line tools and file-types to JSON. This allows piping of output to tools like jq.

Example:

$ ls -l /usr/bin | jc --ls | jq '.[] | select(.size > 50000000)'
{
  "filename": "docker",
  "flags": "-rwxr-xr-x",
  "links": 1,
  "owner": "root",
  "group": "root",
  "size": 68677120,
  "date": "Aug 14 19:41"
}

jtbl

2020-06-01: https://github.com/kellyjonbrazil/jtbl -- A simple cli tool to print JSON and JSON Lines data as a table in the terminal

Example:

$  cat cities.json | jtbl
  LatD    LatM    LatS  NS      LonD    LonM    LonS  EW    City               State
------  ------  ------  ----  ------  ------  ------  ----  -----------------  -------
    41       5      59  N         80      39       0  W     Youngstown         OH
    42      52      48  N         97      23      23  W     Yankton            SD
    46      35      59  N        120      30      36  W     Yakima             WA
    42      16      12  N         71      48       0  W     Worcester          MA

jello - Filter JSON and JSON Lines data with Python syntax

2020-06-01: jello

jid - json incremental digger

2020-06-05: simeji/jid

jj - JSON Stream editor

2023-05-26: tidwall/jj)

fq - jq for binary formats

2023-06-03: wader/fq: jq for binary formats - tool, language and decoders for working with binary and text formats It was originally designed to query, inspect and debug media codecs and containers like mp4, flac, mp3, jpeg. But has since then been extended to support a variety of formats like executables, packet captures (with TCP reassembly) and serialization formats like JSON, YAML, XML, ASN1 BER, Avro, CBOR, protobuf. In addition it also has functions to work with URL:s, convert to/from hex, number bases, search for things etc.

jaq

jaq aims to support a large subset of jq's syntax and operations.

Goals:

  1. Correctness
  2. Performance
  3. Simplicity

faq

jzelinskie/faq: Format Agnostic jQ

faq is a tool intended to be a more flexible "jq", supporting additional formats. The additional formats are converted into JSON and processed with libjq. Supported formats: BSON, Bencode, JSON, TOML, XML, YAML.

jf

2024-04-28 say4n/jf: flatten them json

$ jf -f foo.json
{"foo.bar.0":"baz"}

$ jf --filename foo.json
{"foo.bar.0":"baz"}

$ echo '{"foo": {"bar": ["baz"]}}' | jf
{"foo.bar.0":"baz"}

See also

cube2222/jql: Easy JSON Query Processor with a Lispy syntax in Go

glomRestructuring data, the Python way.

from glom import glom

target = {'a': {'b': {'c': 'd'}}}
glom(target, 'a.b.c')  # returns 'd'

mq – jq-like tool for Markdown processing

mq '.h' demo.md
... lists all the h elements ..

The Zed Project | Zed

Ecosystem of jq adjacent tools.

notes#json

D3.js

See also: Visualisation

notes

Cascading Style Sheets

My CSS Hacks

Learn

Learning CSS by playing

Frameworks

There are lightweight alternatives to commonly-used libraries. Some do away with compatibility with old browsers, some are less bloated, and some just do less.

Functional CSS

Programming history leading resulting in "functional" CSS -- CSS Utility Classes and "Separation of Concerns"

In defense of Functional CSS | Mike Crittenden's Blog

Functional CSS frameworks

Javascript

StyleX is a JavaScript syntax and compiler for styling web apps.

StyleX combines the strengths and avoids the weaknesses of both inline styles and static CSS. Defining and using styles requires only local knowledge within a component, and avoids specificity issues while retaining features like Media Queries. StyleX builds optimized styles using collision-free atomic CSS which is superior to what could be authored and maintained by hand.

Tools

CSS Variables

Colors

OKLCH Color Picker & Converter; OKLCH in CSS: why we moved from RGB and HSL

Typography

use Modern Font Stacks -- use the System font stack "The fastest fonts available. No downloading, no layout shifts, no flashes — just instant renders."

Markdown

ReMarkdown — HTML that looks like Markdown text

Flexbox

Semantic css

My favorites in particular order are: water.css, pico.css, mvp, though matcha looks excellent (May 2024), and I'm going to try it going forward.

  • matcha.css | Drop-in semantic styling library in pure CSS; the site also has a "preview website" feature - which, given a URL, will apply the semantic CSS and let you preview the site. It also has a custom build feature that allows you to select exactly which elements you want to apply these styles + customize colors. Excellent library and website!
  • missing.css -- The goal of missing.css is to reduce needed intervention in HTML. It lets authors
    • start with a good out-of-the-box experience,
    • customize it easily, even create multiple themes,
    • build common components using plain, semantic HTML,
    • create unique sites by applying utility classes as needed.
  • Carpetsmoker/hello-css: A CSS template focused on readability
  • Water.css -- is a just-add-css collection of styles to make simple websites like this just a little bit nicer..

Minimal CSS

Javascript styling system

StyleX "JavaScript syntax and compiler for styling web apps." by Meta.

StyleX combines the strengths and avoids the weaknesses of both inline styles and static CSS. Defining and using styles requires only local knowledge within a component, and avoids specificity issues while retaining features like Media Queries. StyleX builds optimized styles using collision-free atomic CSS which is superior to what could be authored and maintained by hand.

Cool stuff / experiments

Classic CSS

Good CSS examples

Tips and tricks

Typographic CSS

Introducing TODS – a typographic and OpenType default stylesheet | Clagnut by Richard Rutter

notes#css#web

ZettelKasten

ZettelKasten(ZK) method is a method of optimizing workflow of learning and producing knowledge.

Zettelkasten — How One German Scholar Was So Freakishly Productive is a good introduction to the method.

Here is twitter therad by the creator of roam research. See also these two essays by Niklas Luhmann who was the originator of this method .

A paper on Zettelkasten - Project MUSE - Life without Toothache: Hans Blumenberg's Zettelkasten and History of Science as Theoretical Attitude. Hans Blumenberg's use of ZK system (a historian and philosopher of science and technology).

The Zettelkasten Method: How to (Really) Take Smart Notes

Niklas Luhmann’s Card Index: Thinking Tool, Communication Partner, Publication Machin ( PDF )

The Wikipedia page on Zettelkasten is rich and informative.

Talk How to take smart notes by Sönke Ahrens.

  • Feynman -- "writing is thinking". Notes are not records of our thinking. we think in our notes.

Niklas Luhmann (1927-1998)

  • publications ~600, 60+ books.
  • Luhmann wrote a thesis, rehabilitation and got a degree all in one year when he was offered a professor's job and he did not have any of these pre-requisites.

I'm not doing that on my own, i'm doing it with my zettlekasten. I never force myself to do anything that I don't feel like.

  • Zettelkasten = slip-box. 90,000 notes, index with ~1250 entries. about 600 notes as manuscript outlines. one extra box for literature notes with ~15,000 notes.
  • 6 notes per workday (excluding weekends).
  • day starts with reading
  • simple notes on a sheet paper of the form "on page X see Y"
  • at the end of the day write the notes on a card
  • every note would get an index, starting at 1.

  • nothing like a topic structure. everything builds from bottom-up.
  • "entries into a line of thought"

Specifics / core features:

  • what kind of workflow this kind of note taking enforces?

  • note sequences for developing ideas (instead of storing)

  • links and indeces are helpful, but not central features

  • the workflow is streamlined towards writing. Turn notes into writing. The time consuming part is not the writing, but finding the right order

  • three main types of notes:

    • Fleeting -- daily, reminders of what you have in your head. you chuck it after a day
    • Project Related
    • Permanent notes -- main notes. Understandable even if you forget where they come from

  • one idea per note / all notes have the same format.

Status quo of higher education / writing:

  1. Find topic / research question (brainstorming)
  2. research / find literature
  3. read and take notes
  4. draw conclusions / outline text
  5. write
  6. hit or miss your deadline
  7. start all over with the next project.

"the above makes sense intuitively" (subtext: but does not work very well in practice)

ed: invert, always invert!

Status quo: note taking

  1. collect notes for each course in one place, in sep notebook or section of a notebook... this only arranges the material in chronological order.

The difference between the methods makes the difference:

If you can't explain it your own words, you have not understood it -- Richard Feynman.

What happens when you use the ZK system:

  • thinking, connecting and understanding become concrete actions... people tend to want to "do things", ZK helps you to convert vague notions into concrete things.
  • focus on the process -- and not on the outcome.. making connection happens on the paper, not in the brain.
  • writing broken down into reasonable steps (one note at a time). "Write 3/4 notes a day. because that's the outcome of reading something, thought about it and made some connections".
  • the value of each idea compounds -- the more notes the better
  • clear distinction between permanent and temporary notes

Mentions "Inbox" by DavidAllen's GTD. difference is, this is not goal oriented, but "insights".

What works What doesn't
Spacing Cramming
Interleaving One thing at a tie
Connecting Compartmentalizaton
Self-testing Underlining and cpying
Elaboration Re-reading
  • spacing is the technique of "spaced learning".. problem: they dont come with context.
  • "self-testing" can be frustrating because you feel the gaps in your knowledge. many students go back to re-reading because it feels like progress.

Recommended books:

  • "make it stick" by peter c brown, et al
  • "how to take smart notes" by sonke ahrens (speaker).

--

"real experts dont make plans, they become experts in making informed decisions in the context they are in".

Articles

5 main features of Luhmann’s slip-box:

Category-free notes
Atomized ideas
Note Sequences
Links
And an index

Videos

How to Take Smart Notes | Zettelkasten Method in Roam Research - YouTube:

Org-Roam

GitHub - nobiot/Zero-to-Emacs-and-Org-roam: Step by step guide from zero to installing and setting up Emacs and Org-roam on Windows 10

My ZettelKasten setup

(NOTE: not doing this anymore; using voodoopad instead)

How I would like to use zettelkasten:

  • use "daily notes" to make note of things I read on twitter/books/blogposts, thoughts, questions, "nice to know/do" etc.,
  • remember something "SURPRISING" as Claude Shannon said (something along the lines of the information content in a message is the "surprisingness" in it).

Software

Some comments by lobste.rs user MordecaiMalignatus need further thinking on how to use ZK:

Seeing these Zettelkasten-management systems pop up recently has been very interesting to me. Being German, the Zettelkasten was known to me earlier than for most Americans, so I’ve been stewing on the idea for quite some time (and wrote my own software to support it, as one does)

Most of the now-available Zettelkasten have the following things I see as flaws these days:

1.Titles for cards. These, to me, are an error and lead to ideas being discarded that have value in context. Often it’s much easier to have an idea in context and to write it down. Forcing titling of ideas leads to ideas being discarded that would otherwise (in context) be valuable. 2. The use of meaningless numbers/dates/hashes for file names. If you look into how Luhmann used his Kasten, he made heavy use of ad-hoc sequences and branched ideas based on ancestors, hence his tree-like notation for card IDs. This is a completely ignored aspects in many of the modern implementations. My own Kasten started out this way, too, until I found that it encouraged bigger individual notes - every note tends to be an island, and groups don’t actually form. 3. The treatment of the Kasten as a graph, rather than a tree. The Kasten is not meant to be an arbitrary graph - it’s a tree, where later nodes relate to their ancestors and build on them. Cross references are possible and encouraged, but need to be explicit. > This leads to many people treating a Kasten like a local-hosted wiki with fancier/more integrated interface - everything that was already possible with a directory full of markdown files and some editor support. In writing my own emacs-based solution for this, I’ve fallen into all of the above pitfalls, repeatedly re-writing my kasten into something that I will actually use and converse with.

For further reading, the Kasten is partially transcribed/digitized by the university of Bielefeld. The project can be found here (in German), and it might give some insight into how the person that invented it used it. The common literature that introduced the idea was How To Take Smart Notes, which was targeted at academic writing. Since then it seems the idea has taken off since Roam became a thing. I’m curious as to whether or not my personal conclusions will be replicated in other people’s experiences, or if I’m just a weirdo here.

Examples of Zettelkastens / Evergreen Notes

notes#note-taking#pkm#writing#zettelkasten

A Programming Language (APL)

kdb, Q etc.,

APL learning resources

APL like languages

notes#apl#programming-language

Rust Programming Langauge

A modern alternative to C and C++ for system programming.

Why?

How

Articles / Talks

Books

The Little Book of Rust Books

Print books

Online books

Experience reports

Papers / reports

Videos / Talks

Blogs

Tutorials

Courseware

Source code to study

Links

Libraries and tools

WASM

  • Seed -- A Rust framework for creating web apps

Applications

To study code, library usage etc.,

  • mcux/kylyp: A new Forum. The project uses rust, rocket, diesel, postgresql, and responsive design to build a new Forum.

Command Line Tools in Rust: See also: Common *nix commands written in Rust – gcollazo

ekzhang/rustpad: Efficient and minimal collaborative code editor, self-hosted, no database required alternative to codepad.

Editor setup

Performance

Memory Management

Safety-critical programming

What is Ferrocene?

Ferrocene is the main Rust compiler - rustc - but quality managed and qualified for use in automotive and industrial environments (currently by ISO 26262 and IEC 61508) by Ferrous Systems. It operates as a downstream to the Rust project, further increasing its testing and quality on specific platforms.

References and borrowing

Lifetimes

Async / Await

Web

Embedded

This book is an "introductory course" on microcontroller-based "embedded systems" that uses Rust as the teaching language rather than the usual C/C++.

Comparisons (Rust vs X)

FFI

// rustimport:pyo3
use pyo3::prelude::*;

#[pyfunction]  // ← expose the function to Python
fn fibonacci(number: u64) -> u64 {
...

import rustimport.import_hook
from rustfib import fibonacci

Discussions

Cargo

Installing a package that is not on crates.io: cargo install -f --git https://github.com/cjbassi/ytop ytop

  • Cargo Dist distributes your binaries. and in combination with oranda it also generates websites.

Best Practices / Tips

Hardware / bare metal

Rust vs Python

Rust and Python

Rust vs Java for rest services

-- via reddit

I gave a talk (video) today about "Rust for Java programmers" at my local java users group. Part of the talk was comparing the implementation of a minimal rest service in a container for the following 4 approaches:

  1. Payara-micro (JEE/JAXRS)
  2. Wildfly-swarm (another JEE/JAXRS micro deploy)
  3. Javaspark (non JEE "lightweight" framework)
  4. Rust-rocket

The comparisons are striking, viewed in the same 1/2/3/4 sequence above:

  • Container Image Size: 173MB / 151MB / 104MB / 2MB

  • Container Startup Time: 22.2s / 17.6s / 1.6s / 0.6s

  • Memory Usage at Startup: 630MB / 401MB / 34MB / 1.6MB

  • Memory Usage under Stress: 647MB / 522MB / 74MB / 1.7MB

  • CPU usage and throughput show a similar progression as above.

Nice job, Rocket! (results are unscientific. fight me) -- Source code

Success Stories

We had a similar challenge supporting sending this many notifications at OneSignal, which we solved using Rust. We recently hit a peak of 850 Million notifications per second, and 5 billion notifications per day. Here's a blog post on how we do it. Written back when we were at "only" 2 billion notifications per week: https://onesignal.com/blog/rust-at-onesignal/

Rolling log of things

2017-09-08:

There is a new book -- Programming Rust by Jim Blandy, and Jason Orendorff. Jim Blandy has also written a concise short book Why Rust [PDF]

Rust is currently unable to call directly into a C++ library; so that way Rust is probably behind D.

Rust, by design, makes certain programming patterns more painful than others. This is a GOOD thing! It turns out that, for game development, the patterns that end up being the easiest to deal with in Rust closely mirror the patterns that are easiest for game development in any language. Rust highly rewards data-oriented design with simple, understandable ownership semantics, and this is great news because this is a really good fit for game development.

See also: rust2018

References

notes#programming-language#rust

wiki

I use obsidian which is a personal, desktop wiki software to take notes as well publish some of those pages as this website

Why use a wiki? -- "Use a Wiki to document everything you know" by Nikita Voloboev.

Wiki Software

notes

Frasier

Frasier was an American TV sitcom (1993-2004).

French phrases in Frasier

  • contretemps (ಕಾನ್ ತ್ರೆ ತೋ (ನ್)) - a minor dispute or disagreement. "she had occasional contretemps with her staff".

  • bête noire (ಬೆಟ್ ನುಆ) - a person or thing that someone dislikes very much.

  • depeché-toi (ದೂಪಶೆ ತುಆ) - Hurry up!

  • chef-d'oeu·vre ಶೆವ್ ಡುವಿ ಏ ಹ್ಯು - master piece

  • insouciance ಐನ್ ಸುಸ್ ಯಾನ್ಸ್ - recklessness

  • merde ಮೆರ್ದ್ shit

  • "Tally me banana"

notes#television

Interesting Websites

Trying to collect websites that are intersting, free-standing, with a bit (or a lot) of personality. Preference given to "hand-made" websites. And to do so, all you may need is HTML.

The art of building "personal" websites is a thing of the 90s, it appears.. (See also the idea of IndieWeb).

Why have a personal website?

Meta

Related

Portals

See also:

notes#web

FUNctional programming

See also: <functional-programming.html>

Blog-posts and articles

People

  • Hudak
  • Pierce
  • Milner
  • Harper
  • Peyton-Jones
  • Don Stewart
  • The Racket people - [Brown, IU, NEU]
  • The OCaml people - [INRIA]
  • The Haskell people - [MSR]

See Also: My pinboard bookmarks tagged functional-programming

notes#programming

Go Programming Language

Go is the only programming language that has been designed from the ground down. -- vzverovich

Go advocates for making thread creation easy and light-weight and for using message passing over shared memory for inter-thread communication. Indeed, we saw more gor-outines created in Go programs than traditional threads andthere are significant usages of Go channel and other message passing mechanisms. However, our study show that if not used correctly, these two programming practices canpotentially cause concurrency bugs.

Shared memory vs. message passing: Our study found that message passing does not necessarily make multi-threaded programs less error-prone than shared memory.In fact, message passing is the main cause of blocking bugs.To make it worse, when combined with traditional synchro-nization primitives or with other new language featuresand libraries, message passing can cause blocking bugs thatare very hard to detect. Message passing causes less non-blocking bugs than shared memory synchronization and sur-prisingly, was even used to fix bugs that are caused by wrongshared memory synchronization. We believe that messagepassing offers a clean form of inter-thread communicationand can be useful in passing data and signals. But they areonly useful if used correctly, which requires programmersto not only understand message passing mechanisms wellbut also other synchronization mechanisms of Go.

Project structure

A very common question Go beginners have is "how do I organize my code?". Some of the things folks are wondering about are:

  • How does my repository structure reflect the way users import my code?
  • How do I distribute commands (command-line programs that users can install) in addition to code?
  • How do modules change the way I organize my code?
  • How do multiple packages coexist in a single module?

Moving to Go

History

Learning

The Go Programming Language by Donovan and Kernighan.

Training

Reading

Development practices

The Go team has published official guidelines for organizing / structuring a Go project: https://go.dev/doc/modules/layout This includes recommended project layouts for different kinds of projects: a simple package, a command-line tool, a mix of the two, etc. via

Deployment practices

Performance monitoring

User Groups

Presentations

Libraries

Read Code

List of Lists on Go

  • Awesome Go -- A curated list of awesome Go frameworks, libraries and software.

Editors

FFI

Garbage Collection

Why Discord is switching from Go to Rust - Discord Blog

In Go, on cache key eviction, memory is not immediately freed. Instead, the garbage collector runs every so often to find any memory that has no references and then frees it. In other words, instead of freeing immediately after the memory is out of use, memory hangs out for a bit until the garbage collector can determine if it’s truly out of use. During garbage collection, Go has to do a lot of work to determine what memory is free, which can slow the program down. These latency spikes definitely smelled like garbage collection performance impact, but we had written the Go code very efficiently and had very few allocations. We were not creating a lot of garbage. After digging through the Go source code, we learned that Go will force a garbage collection run every 2 minutes at minimum. In other words, if garbage collection has not run for 2 minutes, regardless of heap growth, go will still force a garbage collection.

garbage collection - How fast is the go 1.5 gc with terabytes of RAM? - Stack Overflow

3-4ms pauses under 1.6, with about an 8GB heap and 150M allocations/minute.

Concurrency

Programming notes

Debugging notes

Copying core dumps

Build notes

Cross compiling a binary for Linux on Mac.

git clone git@github.com:xyproto/algernon.git
cd algernon
env GOOS=linux GOARCH=amd64 go build

Tools

Package management

Web

Frameworks

  • Go Blueprint Hello -- "Choose your desired features, and we'll generate the command you can use to create your project."

  • blackfyre/wga: Jumping forward ~30 years -- This repositry contains the code for the Web Gallery of Art project. The project is a web application that allows users to browse through a collection of paintings, sculptures and other forms of Art. This project is inteded to shave off the 3 decades of tech debt on the original website and provide a modern, responsive and user friendly experience with the same content. -- study this project and the choices it has made -- pocketbase, htmx, bulma, golang, sass, and goreleaser

  • anthdm/superkit still in development.

Templating

templ -- "A HTML templating language for Go that has great developer tooling." github

package main

templ Hello(name string) {
<div>Hello, { name }</div>
}

templ Greeting(person Person) {
<div class="greeting">
@Hello(person.Name)
</div>
}

notes#golang#programming-language

Software tools

List of programs and shell scripts that I find useful.

Meta (lists made by other people)

Chrome extensions

Safari extensions

  • Ghostery -- block intrusive javascript.

Pandoc

pandoc is a fantastic program to convert text to various output formats.

Typcial usage: To convert a markdown formatted text file to html

$ pandoc somefile.txt -f markdown -t html --standalone -o somefile.html

See the detailed pandoc page.

Command line tools

abcde

Command line tool to rip CDs to .mp3.

Installation: $ sudo apt-get install abcde id3v2

Insert Disc #1 of a 3-cd audio book $ abcde -o mp3 -p -W 1.

bcat

bcat pipe to browser utility.

Typical usage: To see the git log.

$ git log -p --color | bcat

fd

sharkdp/fd: A simple, fast and user-friendly alternative to find.

gist

gist pastes code, text to [gist.github.com] from command line.

$ gist one.txt

partimage

partimage is a Linux disk imaging software. It has a curses based interface to guide you through the options.

Typical usage: make a backup of an entire HDD(say SSD/Flash of a prototype) to your desktop.

Make Image:

$ partimage -z1 save /dev/sdg1 $HOME/backup/prototype-hdd-20100707.partimg.gz

Restore:

$ partimage restore /dev/sdg1 $HOME/backup/prototype-hdd-20100707.partimg.gz

qpdf

qpdf -- is a command-line program that does structural, content-preserving transformations on PDF files.

http-prompt

pip install http-prompt

fzf

junegunn/fzf: A command-line fuzzy finder written in Go

peco

q

entr

entr(1) -- Run arbitrary commands when files change

ripgrep

ripgrep recursively searches directories for a regex pattern. See rga, like rg, but also search in PDFs, E-Books, Office documents, zip, tar.gz, etc.

sd

sd is an intuitive find & replace CLI.

rargs

rargs xargs + awk with pattern matching support. ls *.bak | rargs -p '(.*)\.bak' mv {0} {1}

rak

lizmat/App-Rak: 21st century grep / find / ack / ag / rg on steroids

Code tools

Database tools

  • dbeaver -- crossplatform database manager for all popular sql dbs and some nosql dbs.

Web development tools

Data Cleaning

Interesting ~/bins on github

Some more linked on this page: 15 Greatest Open Source Terminal Applications Of 2012

Interesting

  • ronn - the opposite of roff.

Lorem ipsum

OCR

ASCII Banner

Use Figlet and Toilet to generate ascii text banners in linux terminal.

notes#macos#tools

OCaml programming language

OCaml for the impatient . The web's quickest and dirtiest OCaml getting started guide This is the most up to date setup using Opam, jbuilder, Merlin, VSCode etc.,(. August 2017). (ed note, 2023-10-10: Since then the OCaml system has made huge strides in developer productivity.)

xvw.lol - Why I chose OCaml as my primary language, an opinion piece by a programmer that uses OCaml professionally at tarides. Is a good intro the ecosystem in 2025; via

Language

Concurrency

Learning OCaml

Courseware

Tutorials

Best practices

  • Our current OCaml best practices. -- part 1 and part 2 by Issuu Engineering; Dec 2018.

Books

Textbooks

Articles

Blogs

Resources

Papers

Package managers

OCaml Languages

Reason: a syntax for OCaml, so native development or melange development to compile to JS

ReScript: a syntax similar to Reason, but the entire toolchain. Closer to JavaScript, can't compile to native.

Talks

Yaron Minsky:

Miscellaneous

OCaml as a replacement language

Writing languages using OCaml

Tools, ecosystem and libraries

Js_of_Ocaml

  • js-skeleton -- sample project showing how to build a Javascript web-app using js_of_ocaml, TyXML and React
  • H.261 Video Decoding in OCaml. -- This notebook will demonstrate a simple H.261 Video Decoder written in OCaml and compiled to javascript with js_of_ocaml.

OCaml project tools

Build tools:

Some opinions on the merits of using these tools:

Code bits

Explicit Typing of a polymorphic variant

type download_result =
  [ `aborted_by_user
  | `network_failure of string
  | `success of filepath ]

via

Information

QQ

  • What is good about OCaml's module system?
  • What good is First class Modules?
  • "Using an error monad in a language like ocaml is superior to exceptions in almost every way." via

Sample code / Read Source

Multicore OCaml

Object oriened OCaml

Hindley-Milner type system

Semantics

Web programming with OCaml

PPX

Comparison

min-caml

esumii/min-caml: a complete optimizing native-code compiler for a reasonably usable subset of OCaml, in 2000 lines of OCaml. Paper.

Running log

PSA: Tired of inscrutable type errors from OCaml? If you use jbuilder/dune, just add this to your jbuild build stanza: (flags (:standard -short-paths)) and your type errors and inferred types from Merlin will magically improve!

NixOS

Data Science and AI

OCaml Garbage Collection

My notes

emacs + utop + merlin + typerex = Excellent IDE.

2014-11-25: Run ocamlbrowser in a shell to see a tk based documentaiton browser.

OCamlEditor

Fortunately, you can install it using opam install ocamleditor... unfortunately you are dealing with various idiosyncrasies of GTK on mac... ugh.. To keep it short, lablgtk installation by opam might fail with pkg-config errors. The fix is:

https://github.com/Homebrew/homebrew/issues/14123#issuecomment-37896791

... before running opam install ocamleditor. Solution via

Note: looks butt-ugly on mac, as expected.

opam allows you create "sandboxes" and install packages in them. See the -A --alias-of option.

See Setting up an OCaml environment for Programming. Use ocamlfind, ocamlbuild and make.

Apr 2016. See ogen -- a tool for creating new OCaml projects with OPAM, Oasis, and Merlin.

Compiling an ocaml file:

  • That has no dependecies

    ocamlopt foo.ml -o foo

  • that has a dependency (say csv)

    ocamlfind ocamlopt -o test -linkpkg -package csv test.ml

ocamlfind

  • ocamlfind is a program that finds all the ocaml packages that are installed on your system (or opam instance, for that matter).
  • ocamlfind -h to see list of commands available.
  • ocamlfind printconf shows the path configs etc.,
  • ocamlfind list will list all the packages installed.

ocamldep

typical usage: ocamldep *.ml, which will dumps dependency lines.

** Different file types **

  • .cmo --
  • .cmi --
  • .mli --

--

OCaml has good study materials -- textbooks, lecture notes. There is also the advantage of being able to roll-over to F# for "enterprise" stuff.

notes#ocaml#programming-language

Interesting Programming Languages

An opinionated collection of programming languages. This is a list of what I consider interesting. Requests to add to this list are welcome, but there is no guarantee I'll include them. As you can see many mainstream programming languages are missing from this list.

Of these, I try to maintain interest in -- ATS, Chapel, Zig, Nim, Flix, Crystal, Vale, and Roc, outside the mainstream languages.

  • Alore is a language that explores optional typing in a Python-like programming language.
  • Alpaca is a statically typed, strict/eagerly evaluated, functional programming language for the Erlang virtual machine (BEAM). At present it relies on type inference but does provide a way to add type specifications to top-level function and value bindings. It was formerly known as ML-flavoured Erlang (MLFE).
  • Amber -- "The Programming Language compiled to bash. Write your scripts in a modern type-safe and runtime-safe programming language that handles many bugs and mistakes during compilation process."
  • appcypher/astro: A multi-paradigm programming language that compiles to WebAssembly and native code. with Python syntax.
  • Arturo programming language; looks like REBOL? and claims "Want to ship your scripts anywhere? For any operating system? Without weird dependencies and without asking the user to install an endless list of modules/gems/packages? That's exactly what Arturo was designed for.", and batteries included.
  • Asymptote is a descriptive vector graphics language that provides a natural coordinate-based framework for technical drawing. Labels and equations are typeset with LaTeX, for high-quality PostScript output.
  • ATS is a statically typed programming language that unifies implementation with formal specification. It is equipped with a highly expressive type system rooted in the framework Applied Type System, which gives the language its name. In particular, both dependent types and linear types are available in ATS.
  • Ballerina "Ballerina is a compiled, transactional, statically and strongly typed programming language with textual and graphical syntaxes. Ballerina incorporates fundamental concepts of distributed system integration into the language and offers a type safe, concurrent environment to implement microservices with distributed transactions, reliable messaging, stream processing, and workflows." See ballerina.
  • Boomerang is a language for writing lenses—well-behaved bidirectional transformations—that operate on ad-hoc, textual data formats. Every lens program, when read from left to right, describes a function that maps an input to an output; when read from right to left, the very same program describes a "backwards" function that maps a modified output, together with the original input, back to a modified input.
  • Biscuit is simple imperative programming language using LLVM backend implemented in C. See Tine editor
  • Blorp; Blorp is meant to feel direct at the surface while keeping the compiler's safety and runtime choices explicit.. Static Types, Hindley-Milner-Style inference, Value Semantics, Perceus Ownership, Native Output. Blorp compiles to C, so idiomatic Blorp code is intended to run within range of hand-written C.
  • C2 an evolution of C, using LLVM.
  • C3 - Full C ABI compatibility, Module system, Generic modules, Design by contract, Zero overhead errors, Semantic macro system, First-class SIMD vector types, Struct subtyping, Safe array access using slices, Easy to use inline assembly, Cross-platform standard library which includes dynamic containers and strings, LLVM backend
  • Cell offers, among other things, a very flexible, entirely structural type system; deterministic, repeatable execution; support for reactive programming; the ability to use relations to store data; transactions and propagation of undefined values for error handling; and ortogonal persistence. "Cell is designed to complement and integrate with your primary language, not to replace it. Once a component of an application has been implement in Cell, the compiler will generate from it an easy-to-use Java, C++ or C# class that can be integrated into an existing project. Support for other languages will be added in the future."
  • Chapel is a modern programming language that is parallel, productive, portable, scalable and open-source by Cray.
  • Ciao is a modern prolog implementation and a general-purpose programming language which supports logic, constraint, functional, higher-order, and object-oriented programming styles. Its main design objectives are high expressive power, extensibility, safety, reliability, and efficient execution.
  • Circle compiler enables you to metaprogram C++ in C++. It encourages a seamless mixing of compile-time and runtime capabilities that upends decades of language design.
  • Cito automatically translates the Ć programming language to C, Java, C#, JavaScript, ActionScript, Perl and D. Ć is a new language, aimed at crafting portable programming libraries, with syntax akin to C#.
  • Clean is a general purpose, state-of-the-art, pure and lazy functional programming language designed for making real-world applications. Some of its most notable language features are uniqueness typing, dynamic typing, and generic functions. quick look.
  • Cool is a small language designed for use in an undergraduate compiler course project. While small enough for a one term project, Cool still has many of the features of modern programming languages, including objects, automatic memory management, and strong static typing. Used for Stanford CS1 Compiler class.
  • Crystal -- is a ruby syntax inspired, statically type checked language that compiles to binary. Mint is a language for writing Single page applications. Mint is written in Crystal.
  • Cuneiform is a functional programming language for large-scale data analysis workflows. Cuneiform uses distributed Erlang to scalably run in cluster and cloud environments. The flexible foreign function interface lets you integrate tools and libraries from many different sources. This allows you to drive a heterogeneous software collection through a uniform interface (Bash, Erlang, Python, Java, R, MATLAB, JS). Static type checking.
  • Curry -- Haskell with Logic Extensions
  • Cyber -- is a scripting language for fast, efficient, and concurrent scripting, written in Zig -- Dynamic and static types. Concurrency with fibers. Memory safe. FFI and Embeddable.Fast VM and JIT. Cross platform.
  • Dafny: A Language and Program Verifier for Functional Correctness - Microsoft Research
  • Dao is a lightweight and optionally typed programming language; Optional typing with type inference and static type checking; Object-Oriented Programming (OOP) with class and mixin class; Abstract interface and concrete interface to supplement class-based for OOP; Code section methods as a better alternative to functional methods; Support closures, anonymous functions; Deferred block and exception handling by defer-recover; Native support for concurrent programming; Concurrent garbage collection;
  • Derw -- is a programming language aimed at making complex apps in a simple way. It's built on top of TypeScript and Node, so it's easy to have interop with TypeScript. See also Gren
  • dg — it's a Python! No, it's a Haskell! -- "Slow. Stupid. Absolutely adorable.". Python with Haskell syntax.
  • Duet delta is an educational dialect of Haskell aimed at interactivity.
  • Eff Programming Language is a functional programming language based on algebraic effect handlers. This means that Eff provides handlers of not only exceptions, but of any computational effect, allowing you to redirect output, wrap state modifications in transactions, schedule asynchronous threads, and much much more… Try it out in the browser
  • The Egison Programming Language
  • Ela combines strict and lazy evaluation, dynamic typing and features, which are normally adopted by statically typed languages, such as algebraic data types andHaskell style type classes. Compiles to .NET
  • Euphoria -- the language that brought me back from the C++ induced panic :D
  • rxi/fe -- A tiny, embeddable language implemented in ANSI C. Used to make grid based game framework cel7
  • DRAKON, is a visual language for specifications. Discussions
  • Eyg
  • Elixir is a functional meta-programming aware language built on top of the Erlang VM.
  • Fay is a proper subset of Haskell that compiles to JavaScript
  • Fennel -- Lisp in Lua. "Full Lua compatibilty, Zero Overhead, Compile-time macros, Embeddable".
  • Felix claims to have its own type system based on a combination of Ocaml and Haskell. Like Ocaml it provides strong support for functional programming, whilst also supporting imperative programming. The type system is strict.
  • Flapjax is a new programming language designed around the demands of modern, client-based Web applications. Its principal features include: Event-driven, reactive evaluation; An event-stream abstraction for communicating with web services; and Interfaces to external web services. Flapjax is just a JavaScript framework. Flapjax is built entirely atop JavaScript, it runs on traditional Web browsers without the need for plug-ins or other downloads. It integrates seamlessly with existing JavaScript code and other frameworks.
  • Flask staged Functional Programming for Sensor Networks
  • Flex group at YC Research has a few interesting lanauges like: Ohm, Seymour, Purple, Trainee, and transcript.
  • Flix -- "takes inspiration from F#, Go, OCaml, Haskell, Rust, and Scala. Flix visually resembles Scala, but its type system is closer to that of OCaml and Haskell. Its concurrency model is inspired by Go-style processes and channels. Flix compiles to JVM bytecode and runs on the Java Virtual Machine. Flix supports full tail call elimination which means that tail calls (even to other functions) never overflow the stack."
  • Futhark - A High Performance Functional Array Language. "a small programming language designed to be compiled to efficient parallel code. It is a statically typed, data-parallel, and purely functional array language in the ML family, and comes with a heavily optimising ahead-of-time compiler that presently generates GPU code via OpenCL, although the language itself is hardware-agnostic."
  • Gleam is a statically typed functional programming language for building scalable concurrent systems. It compiles to Erlang and has straightforward interop with other BEAM languages such as Erlang, Elixir and LFE. Interview with the creator of Gleam
  • Gren is a functional programming language with carefully managed side-effects and a strong static type system. is a "fork" of Elm with a node.js target.
  • Gush: A stack based language eventually for genetic programming
  • Hazel, a live functional programming environment featuring typed holes.
  • hobbes -- a programming language, JIT compiler, and database system for low-latency, high-volume trading applications (built at Morgan Stanley).
  • Idris is is a general purpose pure functional programming language with dependent types.
  • Imba is a programming language for the web that compiles to performant and readable JavaScript. It has language level support for defining, extending, subclassing, instantiating and rendering dom nodes.
  • Inko " is a gradually-typed, safe, object-oriented programming language for writing concurrent programs. By using lightweight isolated processes, data race conditions can not occur. The syntax is easy to learn and remember, and thanks to its error handling model you will never have to worry about unexpected runtime errors."

"Inko is an object-oriented programming language, drawing heavy inspiration from Smalltalk, Self, and Ruby. There are no statements used for conditionals and loops, instead Inko uses message passing for (almost) everything." Processes don't share their memory, and communicate by passing messages, which are deep copied. This removes the need for explicit synchronisation, and makes data race conditions impossible. Inko uses a high performance parallel garbage collector, based on Immix. Each process is garbage collected independently, removing the need for a global stop-the-world phase. Inko is an interpreted programming language, with a bytecode virtual machine written in Rust. Inko is gradually typed, with static typing being the default. This gives you the safety of a statically typed language, and the option to exchange this for the flexibility of a dynamically typed language. Related reading: * Joe Duffy - The Error Model

  • ISPC -- Intel® SPMD Program Compiler is a compiler for a variant of the C programming language, with extensions for "single program, multiple data" (SPMD) programming. Under the SPMD model, the programmer writes a program that generally appears to be a regular serial program, though the execution model is actually that a number of program instances execute in parallel on the hardware. The story of ispc: origins (part 1)
  • Janet is a functional and imperative programming language. Janet makes a good system scripting language, or a language to embed in other programs. Think Lua or Guile. Janet also can be used for rapid prototying, dynamic systems, and other domains where lisp shines.
  • Jolie is a service oriented proramming language that compiles to the JVM. I'm reminded of Ballerina. Jolie crystallises the programming concepts of service-oriented computing as linguistic constructs. The basic building blocks of software are not objects or functions, but rather services that can be relocated and replicated as needed. A composition of services is a service.
  • Julia is a high-level, high-performance dynamic programming language for technical computing.
  • Kit is a “Compiles-to-C” language. It is strong, and statically typed with many modern language features like type inference, algebraic data types, pattern matching, generics, implicits, while still having low level features like raw pointers, and manual memory management. Kit is written in Haskell. kit notes.
  • Koka -- is a function-oriented programming language that seperates pure values from side-effecting computations, where the effect of every function is automatically inferred. The language is strict (as in ML), but seperates pure values from side effecting computations (as in Haskell). Through the effect types, there is also a strong connection to its denotational semantics, where the effect type of each function has a direct translation to the type signature of the denotational function. more Recently, Koka fully supports user-defined effects through algebraic effect handlers.
  • Koto -- language for adding scripting to applications developed in Rust. Koto's runtime is fast enough for many applications, with performance comparable to similar embedded scripting languages for Rust like Rhai and Dyon
  • lambdaway :: _ -- S-expression based markup, styling & scripting
  • The Lemon Programming Language -- python with curly braces. embeddable.
  • Lily -- an interpreted, statically typed programming language focusing on expressiveness and safety.
  • The Little Language is a statically typed, C-like scripting language. "... might be useful for you if you are still using Tcl (or Tcl/Tk) but wish it had a more C-like syntax. ". L forum on bitkeeper
  • Lobster is a game programming language. Unlike other game making systems that focus on an engine/editor that happens to be able to call out to a scripting language, Lobster is a general purpose stand-alone programming language that comes with a built-in library suitable for making games and other graphical things. Looks like python, but is statically typed.
  • Logtalk is an object-oriented logic programming language that can use most Prolog implementations as a back-end compiler.
  • Luna "is a data processing and visualization environment built on a principle that people need an immediate connection to what they are building. It provides an ever-growing library of highly tailored, domain specific components and an extensible framework for building new ones. Luna targets domains where data processing is the primary focus, such as data science, IoT, bioinformatics, graphic design and architecture."
  • LuxLang/lux: The Lux Programming Language "It's meant to be a functional, statically-typed Lisp that will run on several platforms, such as the Java Virtual Machine and JavaScript interpreters." Written in Clojure. tutorial.
  • Mercury is a logic/functional programming language, which combines the clarity and expressiveness of declarative programming with advanced static analysis and error detection features.
  • Myrddin is a programming language that runs close to the metal, giving the programmer predictable and transparent behavior and mental model. It also does strong type checking, generics, type inference, closures, and traits.
  • Mythryl is a rewrite of the SML/NJ compiler
  • Nim -- see my more exhaustive set of notes here.
  • Nitlanguage.org
  • Objective-Smalltalk is an evolution of Smalltalk based on the Objective-C runtime.
  • odinlang - Odin is a general-purpose programming language with distinct typing built for high performance, modern systems and data-oriented programming. Odin is the C alternative for the Joy of Programming. ed: seems to have gained much popularity among game developers.
  • Onyx -- A data-oriented, expressive, and modern programming language, targeted at WASM.
  • Opis reliable distributed systems in Ocaml (paper, 2009).
  • P: Safe Asynchronous Event-Driven Programming by Microsoft Research
  • P P provides a high-level state machine based programming language to formally model and specify distributed systems. See Youtube talk
  • Picat is a simple, and yet powerful, logic-based multi-paradigm programming language aimed for general-purpose applications. Picat is a rule-based language, in which predicates, functions, and actors are defined with pattern-matching rules. Picat incorporates many declarative language features for better productivity of software development, including explicit non-determinism, explicit unification, functions, list comprehensions, constraints, and tabling. Picat also provides imperative language constructs, such as assignments and loops, for programming everyday things. The Picat implementation, which is based on a well-designed virtual machine and incorporates a memory manager that garbage-collects and expands the stacks and data areas when needed, is efficient and scalable. Picat can be used for not only symbolic computations, which is a traditional application domain of declarative languages, but also for scripting and modeling tasks.
  • Pike - a dynamic programming language with a syntax similar to Java and C. "the World Wide Web servers Roxen WebServer and Caudium are both written in Pike. Pike’s advanced data types and built-in support for sockets makes it ideal for use in Internet applications."
  • Piplin is a language that lets you write Clojure to simulate and program FPGAs.
  • Ponylang -- is an object-oriented, actor-model, capabilities-secure, high-performance programming language.
  • Play -- a statically typed stack language (aka Forth)
  • Propane Language a language for simple, correct-by-construction network configuration
  • Pure -- is a modern-style functional programming language based on term rewriting. It offers equational definitions with pattern matching, full symbolic rewriting capabilities, dynamic typing, eager and lazy evaluation, lexical closures, built-in list and matrix support and an easy-to-use C interface. The interpreter uses LLVM as a backend to JIT-compile Pure programs to fast native code.
  • Pyret -- python like language (syntactially) with FP ideas. inbuilt testing emphasis with where: keyword. structured data like OCaml etc., Runs on top of Racket.
  • The QED programming language -- "[for]... designing complex graphical user interfaces the simple way. Its parallel GUI/business logic (GUI/BL - or guible) coding paradigm offers unequaled binding power, which considerably reduces the lines of code required to develop a GUI application. QED runs in both native and web environments, without changing a single line of code."
  • RacketScript is an experimental lightweight Racket to JavaScript (ECMAScript 6) compiler. RacketScript aims to leverage both JavaScript and Racket's ecosystem, and make interoperability between them clean and smooth. RacketScript takes in Racket source files, uses Racket's macro expander to produce Fully Expanded Programs, and then compile these fully expanded programs to JavaScript.
  • Rascal -- is a metaprogramming language that integrates source code analysis, transformation, and generation primitives on the language level
  • Red ; FAQ
  • Reva -- a small, cross-platform Forth. Recommended by @deech in the context of "read the source code of the programming language to learn how things work".
  • Roc's goal is to be a fast, friendly, functional language. tutorial; via
  • Rosie Pattern Language -- a better Regex. See MWPLS 2019 slides (PDF). Has a REPL, JSON output, Can use with C, Python, Haskell, Go.
  • Roy is a programming language that tries to meld JavaScript semantics with some features common in static functional languages and targets JavaScript.
  • Rye is a high level, homoiconic dynamic programming language based on ideas from Rebol, flavored by factor, Linux shell and golang. For Interactive use, backend, servers, information exploration and processing. It's written in Go and could also be seen as Go's scripting companion as Go's libraries are very easy to integrate, and Rye can be embedded into Go programs as a scripting or a config language.
  • Quirrel is a purely declarative query language designed for performing analytics and statistics on large-scale, multi-structured data sets.
  • Quorum -- an OOPs lang for teaching that compiles to Java byte-code.
  • scrapscript  solves the software sharability problem. Modern software breaks at boundaries. APIs diverge, packages crumble, configs ossify, serialization corrupts, git tangles, dependencies break, documentation dies, vulnerabilities surface, etc. To make software safe and sharable, scrapscript combines existing wisdom in new ways: 
    • all expressions are content-addressible “scraps” 
    • all programs are data 
    • all programs are “platformed”
  • Self is a prototype-based dynamic object-oriented programming language, environment, and virtual machine centered around the principles of simplicity, uniformity, concreteness, and liveness.
  • SharpScript, a scripting language for .NET Core and .NET Apps that utilizes a familiar expressive Syntax to enable dynamic scripting of .NET Apps via controlled access to pluggable methods and arguments within a sandbox environment. looks good for exploratory and literate programming.
  • Shen is a statically (optional) typed Lisp variant targeting Common Lisp, JavaScript, the JVM and possibly more targets
  • SparForte "... allows you to write small scripts (think bash) and when they (inevitably) grow, you can transform them piecewise into Ada. The approach is similar to starting with Python and extending it with Rust where necessary. The advantage is the language is explicitly designed to support that use case." -- via
  • syn·di·cate -- a language for interactive programs. Syndicate is an Actor-based programming language offering. Pub/sub pattern-based message routing, for mapping events to actors. Dataspaces, stores for semi-structured data, for managing shared state. State change notifications for keeping actors informed of changes in dataspaces. Integrated techniques for registering and discovering services and for cleaning up after both graceful and unexpected actor failures. Recursive layering of groups of actors, each group with a private dataspace of its own, for organising larger programs
  • Tab is a kind of general-purpose query language for text files.
  • Terra is a new low-level system programming language that is designed to interoperate seamlessly with the Lua programming language. Like C, Terra is a simple, statically-typed, compiled language with manual memory management. But unlike C, it is designed from the beginning to interoperate with Lua. Terra functions are first-class Lua values created using the terra keyword. When needed they are JIT-compiled to machine code.
  • Tree Language Tree Notation is a simpler kind of code. Tree Notation is an error-free base notation like binary. Check out the "dumbdown" (markdown) designer demo.
  • TXL is a unique programming language specifically designed to support computer software analysis and source transformation tasks.
  • Uiua - A stack-based array programming language (think FORTH + APL)
  • Vale -- "a fast, safe, and easy programming language." AOT compiled to LLVM, statically-typed, and uses the new generational references technique for memory safety with speed and flexibility. Read comparisons to JS, C++ and Rust.
  • Verse -- language by Epic Games, designed by SPJ et al
  • virgil - A Fast and Lightweight Systems Programming Language by Ben L. Titzer of CMU; WebAssembly co-founder, former V8 engineer.
  • wing - programming language for the cloud ☁️ A unified programming model, combining infrastructure and runtime code into one language ⚡
  • Wuffs - Wrangling Untrusted File Formats Safely - is a memory-safe programming language (and a standard library written in that language) for wrangling untrusted file formats safely. Wrangling includes parsing, decoding and encoding. Example file formats include images, audio, video, fonts and compressed archives.
  • Wren is a small, fast, class-based concurrent scripting language # Think Smalltalk in a Lua-sized package with a dash of Erlang and wrapped up in a familiar, modern syntax. Used in luxe game engine.
  • Yaksha - A manual memory managed, compiled to C99, programming language with Python-like syntax.
  • Yeti is ML style functional programming language, that runs on the JVM.
  • Zig Programming Language is a system programming language which prioritizes optimality, safety, and readability. Ambition to replace C discussion on lobste.rs. Iterative Replacement of C with Zig
  • ZL is a C compatible and C++ like programming language that focuses on extensiblilty and giving the programmer control over how high-level constructs (such as classes) are implemented. ZL archives the first goal by means of a customizable grammar and a powerful Scheme-like macro system. ZL archives the second goal by using the macro system to define high-level constructs from a C-like core language in a similar in spirit to Scheme.

Other Programming Langauge Related Lists

Blogs etc.,

Courseware

Related Reading

Tools

Comby is a tool for matching and rewriting code. You start by writing a simple template to match syntax. Look at this Go function:

func main() {
    fmt.Println("hello world")
}

We can match the arguments to fmt.Println with this match template:

fmt.Println(:[arguments])

Learning Programming

Fun

Implementing programming languages

Blog posts

Books

Benchmarks

Linkbacks to this page

See also: D lang, Rust, OCaml, C, C++, Python

notes#programming-language

Advaita

Heard about "nityaanityavivekachaara" via Arvind Iyer. Some more reading here - (Bhaja Govindam verse 30). (Feb 12, 2020).

Nityanitya vivekavicaram is the discrimination or viveka between nitya and anitya. Let my intellect be engaged in discrimination. Let me become an enquiring, thinking or contemplative person. Let me not take things for granted. There are so many notions and assumptions in our life, which we have never stopped to analyze. Therefore, let me consider what is nitya and what is anitya, or what is permanent and what is impermanent.

SADHANA CHATUSTAYA (BASIC QUALIFICATIONS) | Advaitam and Science

The sadhana catustaya are:

  1. Viveka or differentiation

  2. Vairagya or dispassion

  3. Samadi-satka-sampatti or the six diciplines (or virtues) of the mind

  4. Mumuksutvam or the desire for liberation.

notes

Lisp

What makes lisp special?

Starting to code in Common Lisp

CL-References

Other Lisps

T

Urn

Urn is a Lisp that compiles to Lua. Its tutorial is one of the friendliest to learn Lisp itslef.

Picolisp

uLisp

  • uLisp -- Lisp for Arduino, Adafruit M4, Micro Bit, ESP8266/ESP32, and MSP430 boards.

Notes

To add readline support to the SBCL REPL, use rlwrap

Use quicklisp to discover, install and manager Lisp libraries

Starting SBCL with QuickLisp pre-loaded.

$ rlwrap sbcl --load ~/quicklisp/setup.lisp

Quotes

In C, you are writing assembly constructs disguised as a high level language. In lisp, you are writing compiler constructs (abstract syntax trees) disguised as a high level language. Assembly language targets hardware-oriented geeks. Lisp targets compiler-oriented geeks. Since the two don't (generally) understand the other's viewpoint, each hates the other's favorite language.

-- gvb at i hate lisp.

Implement your own Lisp

Links

Blogs

Videos

Algorithms

Lisp Machines

Books

IDEs

The de-facto IDE for most lisps in Emacs. Howver Portacle - A Portable Common Lisp Development Environment is a good, ready-to-go, purpose-built distribution of Eamcs for use on Windows, Mac and Linux.

Talks

People

Implementations

Distributions

Interesting programs / libraries

Tips

lisp tip: run visual programs with (uiop:run-program '("sudo" "htop") :output :interactive :input :interactive) works for sudo, vim, htop, ncdu… see also (cmd:cmd "sudo htop" :<> :interactive)

Discussions/ Comparisons

Understanding the words "package", "system", "project" .. which have different meanings than commonly understood ones.. As explained by sjl on lobste.rs:

Common Lisp’s terminology is confusing because it’s old and uses a lot of words that we use now (like “package”) to mean different things than people mean today. Things get easier once you internalize what Common Lisp means by the terms.

A package in Common Lisp is a container for symbols. They’re a way to group related names (symbols) together so you don’t have to do the miserable prefixing of every name with mylibrary-... like you do in Emacs Lisp or C to avoid name clashes.

Unlike other languages like Python/Clojure/etc, packages and files on the hard disk are not tied strongly together in Common Lisp. You can have several files that all work with the same package, a single file that switches back and forth between packages, or even write code that creates/mutates packages at runtime. Files and packages are orthogonal in Common Lisp, which gives you maximum flexibility to work however you want.

A system in Common Lisp is a collection of code (mostly) and a description of how to load that code (and some metadata like author, license, version, etc). For example: my directed graph library cl-digraph contains a system called cl-digraph. That system has a description of how to load the code, and that description lives in the cl-digraph.asd file. It creates a Common Lisp package (called digraph).

The Common Lisp language itself has no knowledge of systems. ASDF is a Common Lisp library bundled with most (all?) modern implementations that handles defining/loading systems. (The name stands for Another System Definition Facility, so as you might guess there have been several other such libraries. ASDF is the one everyone uses today.)

Systems and packages are orthogonal in Common Lisp. Some systems (like small libraries) will define exactly one package. Some systems will define multiple packages. Rarely a system might not define any new packages, but will use or add to an existing one.

A project in Common Lisp is not an official term defined anywhere that I know of, but is generally used to mean something like a library, a framework, an application, etc. It will usually define at least one system, because systems are where you define how to load the code, and if it didn’t define a system how would you know how to load the code? My cl-digraph library mentioned above is a project that defines three systems:

  • cl-digraph which contains the actual data structure and interface.
  • cl-digraph.test which contains the unit tests. This is a separate system because it lets users load the main code without also having to load the unit test framework if they’re not going to be running the tests.
  • cl-digraph.dot which contains code for drawing the directed graphs with Graphviz. This is a separate system because it lets users load the main code without also having to load the cl-dot system (the graphviz bindings) if they don’t care about drawing.

If I were writing this today I’d use ASDF’s foo/bar system naming notation instead of separating the names with a dot, because there’s some nice extra support for that. I just didn’t know about it at the time, and don’t want to break backwards compatibility now.

We saw how Common Lisp has no concept of a system — that comes from ASDF. Similarly, ASDF has no concept of the internet, or reaching out to somewhere to download things. It assumes you have somehow acquired the systems you want to load, maybe by sending a check to an address and receiving a copy of the code on floppy disk, as many of my old Lisp books offer in the back pages.

Quicklisp is another library that works on top of ASDF to provide the “download projects from the internet automatically if necessary” functionality that people want in our modern world. So when you say (ql:quickload :cl-digraph) you’re asking Quicklisp to download cl-digraph (and any dependencies) if necessary, and then hand it off to ASDF to actually load the code of the cl-digraph system. Unlike ASDF, Quicklisp is relatively new in the Common Lisp world (it’s only about eight years old) bundled with any modern Lisp implementations (that I know of) (yet?), which is why you need to install it separately.

So to summarize:

  • Files are files on your hard drive.
  • Packages are containers of symbols. They are orthogonal to files.
  • Systems are collections of code, plus instructions on how to load this code. They are orthogonal to packages.
  • Projects are high-level collections of… “stuff” (code, documentation, maybe some image assets, etc). They are (mostly) orthogonal to systems (seeing a trend here?).
  • Common Lisp knows about files and packages. ASDF adds systems. Quicklisp adds the internet.

This is a pretty high-level view that glosses over a lot, but maybe it’ll clear things up and help you dive into the details later.

notes#lisp#programming-language

Meta list of book recommendations by people

Book lists

Medical books for non-medicine people by Michael Sanger, MD

  • Learn Physiology: John E. Hall PhD, Guyton and Hall Textbook of Medical Physiology (Guyton Physiology)
  • Systems Medicine. Editors: Schmitz, Ulf, Wolkenhauer, Olaf (Eds.) Includes cutting-edge methods and protocols.
  • Didier Sornette. Critical Phenomena in Natural Sciences: Chaos, Fractals, Selforganization and Disorder: Concepts and Tools

Hackernews

Lobsters

Probabilistic Computing

ML and AI

Business and economics

Programming and software development

Book reviews by people:

Book lists

Other

Blog posts

Websites

See also: Twitter Book Recommendations, lists, 100-page-book

notes#books#reading#software

Nim Programming Language

NOTE: nim was previously known as nimrod.

A quick look at nim programming language

This is a short intro to a fun, productive language called nim.

nim is a new programming language developed by Andreas Rumph.

nim is a statically typed, compiled langauge.

What kind of language is it?

There are few things more irriting than a programming language or library webpage that doesn't show how the syntax/typical usage looks like. So, let's see how nim code looks like:

# compute average line length
var count = 0
var sum = 0

for line in stdin.lines:
    count += 1
    sum += line.len

echo "Average line length: ",
if count > 0: sum / count else: 0

The var keyword is familiar if you have used C#, which essentially means that the variable count has an implicit type. That is, you don't have to explicitly mention the type, in this case an integer.

The for loop line reads naturally for the Python programmer, including the : starting a new block.

nim is inspired by many popular languages old and new. It is quite easy to read and understand what is going on.

According to Araq, the creator of Nim:

Macro system inspired by Lisp.
Export marker taken from Oberon.
Argument passing semantics taken from Ada.
distinct types inspired by Ada.
Syntax also heavily influenced by Python.
Generics inspired by C++.
The 3 pointer like types ptr, ref, var taken from Modula 3.
async / await stolen from C#.
let taken from ML.
...

One could also argue that Nim is a statically typed Lisp with an
infix syntax extended by essential features necessary for
systems programming. And dumbed down to support efficient
compilation to C (where are my continuations?!)..

How to install it?

Checkout the code from github. (Note these steps are outdated since the language was renamed from nimrod to nim).

$ git clone git://github.com/Araq/Nimrod.git
$ cd Nimrod
$ git clone --depth 1 git://github.com/nimrod-code/csources
$ cd csources && ./build.sh
$ cd ..
$ bin/nimrod c koch
$ ./koch boot -d:release
$ ./kock install $HOME/nimrod

This installs nimrod under $HOME/nimrod. Add $HOME/nimrod/bin to the $PATH.

What is the performance like?

Nim compiles to C, so you can expect C-like performance (hard performance numbers are a game of benchmarks and nobody ain't got no time for that...). To go with Nim's authors' own word "it is performant for 'soft real time' systems like games". Take it as you may.

For me, Nim is a vast upgrade over dynamic languages like Python and Ruby without sacrificing a whole lot of expressiveness.

What are some of the quirks

By looking at examples and reading applications/libs written in Nim you would not guess it, but nim is case insensitive. That is var hello and var HELLO and var HeLlo are treated as the same by the compiler. Where nim takes it one step ahead is it's treatment of underscores, which means var H_E_l_lo is same as var hello. This might appear crazy, but this removes the entire bikeshed argument about which style of variable naming is best.

The nim programming community has it's own best practices for variable names, for instance, new types are declared as TFoo an inspiration from Pascal.

A quick tour of the command line

To compile a nim file foo.nim, you type

nim c foo.nim

which will give you a foo executable in the same directory, assuming there were no compilation errors. You will also see a nimcache directory.

This directory contains the intermediary C files generated.

What are the resources to learn more

The website: http://www.nim-lang.org The forum: http://forum.nim-lang.org -- this is probably the best place to get your questions answered.

How can I use nim?

For a young language like nim has not managed to corner itself into any one niche.

nim has been used to write: (Update, Aug 2024: See Awesome Nim and nimble.directory for a list of libraries for nim - a lot more diverse than you think!)

Personally, I have been using nim to replace a few of command line python applications that process large amounts of data.

Standard libraries

Nim's standard library has XML, CSV, Web, String, Regular expressions, OS, Database drivers, Network programming and more.... It also has wrappers to popular C libraries like X, GTK, pcre, cairo, sdl, OpenGL etc.,. nim libraries have you covered for day-to-day programming needs.

Web development

Editors and IDEs

Emacs has a nim-mode. A nim specific IDE called Aporia built using the GTK toolkit. I haven't used it much beyond installing it because I prefer using the Emacs nim-mode.

Aug 2024: I use zed, and it uses LSP.

Packaging and reusing libraries

nim libraries can be installed using the nimble package manager. The nim After successful compilation, copy the babel executable to a directory in your $PATH. I put it under $HOME/nim/bin along with other nimrod executables.

Distributing applications written in nim

A .nimble file is required at the root of your application source directory. Take for example the jester.nimble file that is part of the Jester web framework distribution" --

[Package]
name          = "jester"
version       = "0.1.0"
author        = "Dominik Picheta"
description   = "A sinatra-like web framework for Nimrod."
license       = "MIT"

SkipFiles = "todo.markdown"

The file is self explanatory. Once you have the nimble file in the directory, running nimble build will build binaries.

Cross Compilation

From SO

I was having the same issue getting nim to compile executables for Windows, from a GNU/Linux machine, so I made a bash script. It takes the path to the directory containing *.nim source files and the name of the executable file to output. I'm sure you could swap out the GCC compiler (MinGW in this case) and change the --os: switch as appropriate:

#!/usr/bin/env bash
# Nim must generate C sources only, to be fed to MingW
nim c --cpu:amd64 --os:windows --opt:speed --embedsrc --threads:on --checks:on -c -d:release $1/*.nim
# Copy nimbase.h so MingW32 can find it during compilation and linking
cp /opt/Nim/lib/nimbase.h $1/nimcache/nimbase.h
mkdir -p $1/bin
cd $1/nimcache && x86_64-w64-mingw32-gcc -save-temps $1/nimcache/*.c -o $1/bin/$2.exe
rm $1/nimcache/*.{i,s} # only care about *.o objects
ls -lAhF $1/nimcache
ls -lAhF $1/bin

Concepts

Memory Model

Related

Links

Books

Tutorials

Interop (FFI)

Nim+D

Testing

Interesting projects in Nim

Compile time

Libraries

Web development

Dev tools

Dev Notes

Upgrade nim to the latest version using choosenim:

$ choosenim update stable
# downloaded, built and installed 2.0.8 (Aug 2, 2024)
$ which nim
/Users/pradeep/.nimble/bin/nim
$ nim -v
Nim Compiler Version 2.0.8 [MacOSX: amd64]
...

Random notes

Nim's GC is per thread: You can easily have threads which are hard realtime and don't use the GC and happily use the GC in the other threads... And the type and effect system ensures you don't mix these different per thread heaps. -- Araq

[Nim has] C++-like features (generic types, operator overloading, function overloading, inline functions, optional O-O, optional data hiding, C pointers, bitwise-compatibility with C, your choice of manual memory management or GC) and C++-like run-time speed, combined with Python-like syntax and compile-time Lisp-like macros. -- HN comment

GC free Nim programming

The no GC future is --gc:stack which replaces the GC with memory regions. So any library that uses the GC really uses a memory region. The entire situation is ok. Note that the GC is always thread local anyway, so you can also just run realtime threads with other threads that use the stdlib and thus the GC. -- Araq

use these: alloc, dealloc, allocShared, deallocShared. You can cast the pointer from and to ptr types and then you have your manually managed heap memory. (This is a general answer to how to use non-GC'd memory and has nothing to do with memory regions) -- flyx

A good example of using FFI in nim in comparison to D and rust.

Discussion in /r/python about a nim article.

From a reddit thread: What problem does Nim solve?

Low friction, high performance development.

It feels like a scripting language in terms of writing code rapidly. Deferred GC means no need to worry about memory (although easy to manually manage it). So, a sort of statically typed, compiled Python with C-level speed and tiny, self contained executables.

High level AST metaprogramming that lets you hygienically extend the language, has static typing with inference and uses structural typing for generics which often means you don't even need object hierarchies, extensive compile time evaluation (to the point you can process files, create hash maps or pull web data by simply defining your desired variables as 'const').

It compiles to C, C++, LLVM (experimental), and Javascript. Excellent FFI so you can use all the C, C++ and JS libraries as well as Nim ones. Very fast compilation that's going to get faster from what I've read about future plans.

This may be subjective but I also like how clean it looks. Few $!;@ symbols and easy to read.

Personally, it's brought the fun back into development for me.

Stack vs GC vs Manual memory management

  • type Foo = object -> Value type, on the stack
  • type Foo = ref object -> Ref type, managed by GC
  • type Foo = ptr object -> Pointer type, manual memory management (malloc, free, memcopy ...)
  • Pointer arithmetic can be achieved using casts. via

Stack vs Heap from a Reddit thread:

By PMunch:

First order of business heap vs. stack: Whenever you call a function it creates a stack frame. Stacks are simply said first in last out, so when a function call is done it pops it's frame from the stack, and you return to the previous frame. When you declare a variable inside a scope in is typically allocated on the stack. This means that when the function returns that part of memory is not in use anymore and the value is gone (technically it's still there, but we shouldn't try to access it). The heap on the other hand is quite a bit different. Items on the heap are allocated and live there until you deallocate them. In C this is done manually with calls like malloc and free. In Nim however, and C# for that matter, we have the garbage collector (or GC), this nifty thing reads through our memory and checks if anything still points to things allocated on the heap and if nothing is pointing to that which was previously allocated it gets free'd. This means that we won't leak memory as easily as you would in C as things that we've lost every reference to get's automatically cleaned. Okay, now that we know where our stuff lives in memory, let's look at how Nim represents those things. Basic types are things like integers, floats, bools, and characters. Their size is known and static. Nim also calls strings a basic type, but those are not quite like the rest since they can change in size. Since our stack is first in last out it means that it makes sense to store everything in order. But storing things in order isn't possible if you want to change the size of something (for example appending to a string). So when we create numbers in our Nim code they will be stored on the stack, this is why you never need to free your numbers and don't have to set them to nil when you're done with them. So what are types? In Nim you can create aliases for types like type age = int this is just a way to say that age is an integer, and it will be treated like one for all intents and purposes. If we want to create collections of types to represent something particular we can create objects, don't think of these quite like objects in an object oriented language, think of them more like structs in C. Such objects are simply a collection of values. So in Nim when we create an object it will live with us on the stack. If we return an object it will be copied to our caller, and if we insert it into a data structure it will also be copied. While this might be practical in many cases (even offering a speed benefit if done right) we often don't want to copy our large objects around. This is when allocating on the heap comes into play. If we define our type as a ref object it means that the type is actually a reference to an object. I'll come back to the difference between a reference and a pointer later, but for now just remember that a reference is the memory location of an object on the heap. This means that if we return a ref object it means that we're only returning the memory address of that object, not copying the object itself. This also means that if we insert it into a data structure only the reference to the object is inserted. Whenever you see new SomeObject it means that it allocates memory for that object on the heap, and gives us a reference to this object. If we had simply done var myObject: SomeObject and SomeObject was defined as a ref object we would only have a reference on our stack, so trying to access it would crash saying we had an "Illegal storage access". This is because Nim defaults our value to nil, and no-one is allowed to access memory area 0. So imagine we had an object that contained the height, the weight, and the age of a person. That could be represented by three integers. If we wanted to return this object it would mean copying all those three values to our caller. If we defined it as a reference, we would only pass one integer, the position in memory (on the heap) where we stored the three others. Conveniently this also means that if one functions modifies a value in a referenced object, that change would be visible for all other functions using the same reference (since they all point to the same place in memory). This is practical for example if you want to make one list sorted by age, one by height, and the third by weight. Instead of copying our person three times, we could just use three references, one in each list. So now that we know where and how are values are stored we can look at the difference between a pointer and a reference. In pure Nim code you would typically only use references, these are what every call to new creates and what goes on behind the scenes most of the time when working with strings. A reference is also called a managed pointer, it simply means that Nim manages this area of memory, and that it will be automatically free'd for us by the garbage collector when Nim sees that we're not using it any longer. Pointers on the other hand are unmanaged meaning that Nim doesn't try to do anything with the memory behind that pointer, most of the time you won't even know what is there. The reason there are pointers in Nim is mostly to interface with C. In C every time you want objects on the heap you need to manually malloc and free them. Many C libraries work by passing around a pointer to a structure containing some state and all good libraries have some way of dealing with this memory. Typically you do it by calling some initialisation function when you begin and then some cleanup function when you are done. In Nim we might want to use a C library such as that, but since we might loose the reference in our own code while the library still keeps a reference somewhere which Nim doesn't know about we can't have a reference to it as Nim would garbage collect it. So instead we have pointers. I hope this made it somewhat clearer, if you want more examples or if anything was unclear, feel free to ask.

Benchmarks

Talks

Editors

Visual studio Code

Nimlime

Nimlime is a plugin for Sublime Text the provides Nim compatibility. This is my custom settings overriding the default settings:

{
    "nim.executable": "/Users/pgowda/src/Nim/bin/nim",
    "nimble.executable": "/Users/pgowda/.nimble/bin/nimble",
    "nimsuggest.executable": "/Users/pgowda/.nimble/bin/nimsuggest",
    "provide_immediate_nim_completions": true,
    "output.send": false
}

The output.send option set to false is an important one; withtout this, the plugin would create a new compiler output buffer everytime the file is built, which is everytime I saved the file!.

Blogs

Code

Code to read etc.,

Eventually I decided, that being able to do transformations on the AST (abstract syntax tree) of the programming language would greatly help to develop this toolbox. I tried out Rust, but I quickly found out that their macros are too weak to even implement println, and I had to look somewhere else. (At this point in time Rust also has procedural macros, but they still don't appear to be powerful enough to do the things I need to do for this project). Eventually with Nim I finally found a language that suited my needs. It has AST based macros, arbitrary code execution at compile time, static typing, reflection and a lot more.

News

See also: Crystal, Interesting Programming Languages, D, Python, Rust.

notes#nim#programming-language

GNU Guix

Guix is a package manager.

NixOS:Nix::GuixSD:Guix

Links

Talks

notes

Unikernels

Talks

Blogs

Articles

Projects

HermitCore

LightVM

LightVM - Systems and Machine Learning

We achieve lightweight VMs by using unikernels for specialized applications and with Tinyx, a tool that enables creating tailor-made, trimmed-down Linux virtual machines. LightVM can boot a VM in 2.3ms which comparable to fork/exec on Linux (1ms), and two orders of magnitude faster than Docker. On a somewhat slower server with 64 cores, LightVM can pack thousands of VMs with memory and CPU usage comparable to that of processes

Hermitux

HermiTux | A binary-compatible unikernel; Apr 2019.

HermiTux is binary-compatible with Linux: it can run native Linux executables.

Although being a proof-of-concept, HermiTux supports multiple compiled (C, C++, Fortran) and interpreted (Python, LUA) languages. It provides binary analysis and rewriting techniques to optimize system call latency and modularize a kernel in the presence of unmodified binaries. It supports statically and dynamically linked programs, different compilers and optimization levels. HermiTux also provides basic support for multithreading, debugging and profiling.

Unik

solo-io/unik: The Unikernel Compilation and Deployment Platform

UniK (pronounced you-neek) is a tool for compiling application sources into unikernels (lightweight bootable disk images) rather than binaries. UniK runs and manages instances of compiled images across a variety of cloud providers as well as locally on Virtualbox. UniK utilizes a simple docker-like command line interface, making building unikernels as easy as building containers.

Unik supports the following unikernel types:

  • Rumpkernels
  • OSv
  • IncludeOS
  • MirageOS

And supports the following providers:

  • Virtualbox
  • AWS
  • Google Cloud
  • vSphere
  • Xen etc.,

More info on Unik:

Unikraft

Related

Links

... the symmetric multiprocessing story - Unikernels don't have such a story. They are inherently single core. ... I just couldn't ignore was that Linux let you configure it into very small and secure configurations with full SMP support AND you could run ANY existing Linux languages or applications on those tiny systems. Unikernels generally seemed to need special builds and toolchains and configuration systems and recompilations to get them to run applications. AUTHORS's takeaway -- "I had come to see that it was possible to build very very small Linux systems, compatible with all Linux software, with all the security advantages of Unikernels, but with the benefit of symmetric multiprocessing and all the Linux driver support and community support."

Papers

News

Tutorials

dysinger/restack: Full Stack ReasonML Tutorials

This is a tutorial series on using the ReasonML language to build MirageOS unikernels and user-facing applications with tools like Reason-React. Using the same language for the entire stack is appealing. You’ll likely be able to stay in one editor, become very familiar with the syntax, and share common code between the layers of your application.

notes

Github Gists

Gist -- http://gist.github.com/ is a small code hosting service provided by github. It is a convenient place to store:

  • one-off scripts
  • snippets of code that is reusable.
  • Short HOW-TO notes that can save you time the next time around
  • Funny, memorable, poignant texts that you got as a forward etc.,

See my Gist Index and this is a better index (gist.aviperl.me), to see rendered files.

By others -- vsouza/awesome-gists: A collection of amazing gists

gist hostby simon willison is a tool that lets you see a browser-rendered version of an HTML page that you have saved to a Gist.

Sublime text

I use sublimetext's gist package to paste, edit and manage the gists directly from inside the sublime text editor. This is a very nifty thing I found out in Dec 2018.

Databases

Shell

Python

C

Web

Gist related projects

Others gists

notes#github#web

Lua Programming Language

Learn

Lua associated languages

Pallene + Lua. Paper .Found via janet lang's gitter channel. It is a "Friendly Fork" of Titan Language. Pallene is a moon of saturn?

Terra -- low level lua like language.

teal; The compiler for Teal, a typed dialect of Lua

Luau is a fast, small, safe, gradually typed embeddable scripting language derived from Lua.

Links and references

Cool Lua stuff

2020-06-04 oasislinux/oasis: a small statically-linked linux system; uses lua for setup, building, package spec etc.,

Mako Server This tiny, ready-to-run application server packs a punch with its super compact and efficient Lua web framework and its non-blocking asynchronous sockets. Plus, it comes with everything you need built-in, including an integrated database, SMTP, HTTP client/server, and even IoT protocols like MQTT and industrial protocols like OPC-UA and Modbus.

People

  • Calvin Rose -- bakpakin.com (creator of Fennel lang, moonmint web framework, etc.,)

Compiling Lua source files

Let's say you have a lua program -- hello.lua.

print("Hello Lua!")

The source code can be compiled into a binary file using luac

$ luac -o hello.luac hello.lua

The file hello.luac can then be used in place of the source file. Using the binary file has some advantages like: faster loading, simple source hiding etc.,

To run a .luac file (this is my own convention, the Lua docs do not seem to prefer any particular suffix for the compiled file), you still need to call the lua interpreter,

$ lua hello.luac

Creating standalone executables for lua

A Lua source file can be converted into a executable using different methods. srlua appears to be the popular one.

Download srlua from here and build it using make. Copy glue and srlua files into a convenient location, say ~/bin.

You can now create standalone executable from the Lua source file --hello.lua thus:

$ ~/bin/glue ~/bin/srlua hello.lua hello
$ chmod +x hello
$ ./hello
Hello Lua!

Installing new software packages

Luarocks is the deployment and package management system for Lua. Think CPAN for Perl and PyPI for Python.

I have installed Lua in my $HOME directory. That is, lua interpreter resides in $HOME/bin, include files in $HOME/include etc.,

So, to install Luarocks:

$ export PREFIX=$HOME
$ ./configure --prefix=$PREFIX --sysconfdir=$PREFIX/luarocks --force-config

Installing new lua packages is as simple as:

$ cd
luarocks install luasocket

where luasocket is a new package that is fetched from the luarocks mirrors and installed.

To make this package available to the Lua interpreter export LUA_PATH thusly:

export LUA_PATH="/opt/local/share/lua/5.1//?.lua;/opt/local/lib/luarocks/?.lua;;"

Another LUA_PATH that makes more sense:

?;?.lua;/usr/local/lua/?/?.lua

Also, to explicitly add paths to a script:

# cat t_paths.lua
package.path = package.path.. ";/opt/local/share/lua/5.1/?.lua"
package.cpath = package.cpath.. ";/opt/local/lib/lua/5.1/?.so"
local s = require "socket"
print(s._VERSION)

Incidentally, on the day I was learning aobut luarocks, luaforge.net, the main lua modules repository was down. Friendly folks on the #lua IRC channel told me to add mirrors to the luarocks configuration.

$ vim ~/luarocks/config.lua

I replaced the luaforge.net line with

http://www.luarocks.org/repositories/rocks/",

Misc. Notes

Setting up LUA_PATH and LUA_CPATH is always more complicated than it needs to be. Remember not to have any spaces after the path seperator ;.

LuaDist appears promising to set up virtualenv like sandboxed environments (and more..) for lua.

Lua application packaging

luastatic README tells us that it can package lua applications into executables for windows, macos & linux. A few applications that use luastatic to do their packaging are given. It will be nice to find build setups which can compile to all the three OSes using github CI pipeline.

--

See also: Luarocks.

notes#lua#programming-language

Shell programming

notes#programming-language

Janet

Janet Programming Language is a Clojure like Lisp scripting language also suitable for embedding.

Why

Why Janet?; Apr 2023. Book "Janet for Mortals" by Ian Henry.

Learn

Notes

  • Project definitions go into a project.janet file.
  • Third party dependencies are defined like this: {:dependencies [{:repo "https://github.com/joy-framework/moondown" :tag "0.1.0"}]}

The structure of a project.janet file is: (See more here)

(declare-project
  :name "mylib" # required
  :description "a library that does things" # some example metadata.

  # Optional urls to git repositories that contain required artifacts.
  :dependencies ["https://github.com/janet-lang.json.git"])
  • See littleserver on how to build an executable from Janet source.

Cool projects in Janet

  • Bauble studio -- making art online with lisp(janet) and math.

  • Mendoza -- static website generator with a markdown language inspired by Scribble (of racket).

  • Tester -- testing framework. Readme file has an example of how to use fswatch to run the tests when files change.

  • Joy Framework -- a web framework for Janet.

    • jpm install joy.
    • has an ORM?

  • esv a web application built using Janet, (and circlet library).

Janet Shell

janetsh "A powerful new shell that uses the janet programming language for both the implementation and repl.". This is pretty cool when you want a regular shell that can do mundane things like ls -lart, but also use janet as a REPL and scripting language when you want to more advaned things without resolving to bash scripting. See this introductory article by the creator -- Andrew Chambers.

In this article, -- JSON in Janet Shell you can see how to use the ~/.janetsh.rc file to load additional modules (in this case ability to process JSON) to extend the capability of janetsh.

# Tell janet it can load janet and native modules from ~/janet-modules
(array/insert module/paths 0
   [(string (os/getenv "HOME") "/janet-modules/:all:.janet") :source]
   [(string (os/getenv "HOME") "/janet-modules/:all:.:native:") :native])

# copy combined dynamic library to janet-modules dir
cp build/json.so ~/janetsh-modules

# use it
$ (import json)
$ (var astronaut-in-space
    (json/decode ($$ curl http://api.open-notify.org/astros.json)))
# note: $$ appears to be the function call to invoke programs

# Print each one and the space craft they are in.
$ (each astronaut (astronauts-in-space "people")
    (print
      "Name: " (astronaut "name")
      ", Craft: " (astronaut "craft")))
...
Name: Oleg Kononenko, Craft: ISS
Name: David Saint-Jacques, Craft: ISS
Name: Anne McClain, Craft: ISS

This is a very nice capability (being able to extend the shell programming language).

notes#clojure#lisp#programming-language

Fennel programming language

Fennel is another clojure-like language that compiles down to lua created by Phil HagelBerg "technomancy".

Anywhere you can run Lua code, you can run Fennel code.

Phil Hagelberg/Technomancy has three.blog.posts about using Fennel to write a new game EXO_encounter that is hosted on itch.io (which itself is written by Lua-u* leafo).

Here is an introductory post on Fennel by Technomancy.

Software written using Fennel

*What's the demonym for a Lua programmer? (like Pythonista for Python programmers) -- Lua'r, Lua-oo?

See also: janet, lua

notes#clojure#lisp#lua#programming-language

C Programming Langauge

Why C?

"The Endurance of an Unmanageable Language" - Some Were Meant for C; via ATS As a Systems Programming Language.

Learning

Tooling

Why not C?

C Is Not a Low-level Language - ACM Queue

Books

People and blogs

Important

  • cdecl; C gibberish ↔ English
  • ~sircmpwn/cstyle -- style guidelines on the lines of "SHOULD" and "SHOULD NOT"s.

C libraries

Memory Management

C programming practices

Cool stuff

Programming Language Theory

we argue that the C standard does not allow Turing complete implementations, and that its evaluation semantics does not preserve typing. Finally, we claim that no strictly conforming programs exist. That is, there is no C program for which the standard can guarantee that it will not crash.

Discussions

Safe C

A Tour of Xr0 - Xr0

Xr0 is a new verifier for the C programming language that eliminates many classes of undefined behaviour, such as use-after-frees (of which double frees are a special case), dangling and null pointer dereferences, memory leaks and the use of uninitialised memory. Most importantly, we believe Xr0 is on a clear trajectory towards enabling programming in C with no undefined behaviour whatever.

notes#c#programming-language

Functional programming

  • (Hughes 1984); link.

Papers

Learning Functional Programming

Book reviews and suggestions

Videos

Misc

-- Xavier Leroy via

References

  • Hughes (1984). Why Functional Programming Matters.
  • Okasaki (1998). Purely Functional Data Structures.

notes#functional-programming#haskell#ocaml

Probability Library

Dr Harry Crane asked for recommendations for a "Probability Library".

These are some of the recommendations:

Harry Crane:

  1. Technical: An Introduction to Probability Theory and Its Applications, Vol. 1, 3rd Edition
  2. Classic: Foundations of the Theory of Probability: Second English Edition (Dover Books on Mathematics) by Kolmogorov
  3. Real World: Antifragile. NNTaleb recommended Statistical Consequences of Fat Tails (book in the works) instead over Antifragile.
  4. Application: Beyond Counting : Exploiting Casino Games from Blackjack to Video Poker by James Grosjean.
  5. Biography: A man for all ages by Ed Thorpe.

Nassim Taleb:

  1. Probability Theory I, 4th Edition and Vol 2 by Loeve
  2. Methods of Mathematical Finance by Karatzas and Shreve
  3. Elements of Information Theory 2nd Edition by Cover and Thomas
  4. Probability and measure, 3rd edition by Patrick Billingsley
  5. Chance and Stability, Stable Distributions and Their Applications by Uchaikin, Vladimir V. / Zolotarev, Vladimir M. PDF
  6. Stable Non-Gaussian Random Processes: Stochastic Models with Infinite Variance by Gennady Samorodnitsky, M.S. Taqqu
  7. Stochastic Processes by J. L. Doob
  8. Limit distributions for sums of independent random variables by B.V.Gnedenko and A.N.Kolmogorov.
  9. Extreme Value Theory by Laurens de Haan and Ana Ferreira
  10. Extreme Value Theory (PDF) by Embrechts et. al.

Misc:

via <https://x.com/BristOliver)

  • Feller vol 1
  • Kolmogorov/Gnedenko
  • Petrov Sums of Independent RVs
  • Grimmett Percolation
  • David Williams Martingales

notes

Python Programming Language

My Python Stuff

Meta

Advanced Python

dabeaz-course/python-mastery: Advanced Python Mastery (course by David Beazley)

Books

  • Fundamentals of Python programming (PDF) -- "... introduction to computer programming textbooks are available in PDF form for free download. University and high school students and self learners around the world have found them helpful. See the terms of use in the preface of each book."

  • Intermediate Python -- "Fluent python" by Luciano Ramalho.

Learning sources

"This repository contains the text for the Tiny Python 3.6 Notebook. it is a notebook containing curated examples for Python 3 as well as the new features found in Python 3.6."

Internals

Developer Experience (DevEx)

Make Python DevEx -- this is very close to how I choose to do Python development. good read. I don't have such an extensive Makefile setup.

Debugging

Profiling

Web

Multiprocessing

Async

Sublime Text

Alternate Python implementations

To Install Zippy:

hg clone https://bitbucket.org/ssllab/zippy
cd zippy
TBD

Libraries

Data science libraries

Conferences

Python3

Pypy

Package management

python setup.py sdist
python setup.py bdist_wheel
twine upload --repository-url https://test.pypi.org/legacy/ dist/*

Project Managers

Do not use requirements.txt - quanttype -- instead choose Poetry, Hatch or pdm.

Poetry

I have been using poetry for a while.

Poetry uses the pyproject.toml standard, but it does not follow the standard specifying how metadata should be represented in a pyproject.toml file (PEP 621), instead using a custom [tool.poetry] table. This is partly because Poetry came out before PEP 621. -- from pdm's documentation

Hatch

pdm

Unlike Poetry and Hatch, PDM is not limited to a specific build backend; users have the freedom to choose any build backend they prefer.

Links

2019

Via My Python Development Environment, 2020 Edition | Jacob Kaplan-Moss:

notes#programming-language#python

Futhark

Futhark is a small programming language designed to be compiled to efficient parallel code. It is a statically typed, data-parallel, and purely functional array language in the ML family, and comes with a heavily optimising ahead-of-time compiler that presently generates GPU code via CUDA and OpenCL, although the language itself is hardware-agnostic and can also run on multicore CPUs. As a simple example, this function computes the average of an array of 64-bit floating-point numbers:

let average (xs: []f64) = reduce (+) 0.0 xs / r64 (length xs)

Futhark is not designed for graphics programming, but instead uses the compute power of the GPU to accelerate data-parallel array computations.

Code in Futhark

notes#programming-language

Algorithms

Algorithm Visualizer

Success stories

Course Pages with Lecture notes

Duke University's CPS130 --Introduction to Design and analysis of Algorithms is an undergraduate course. The lecture notes are good for revising basic concepts.

Jeff Erickson's Algorithm course materials --this is a huge collection of lecture notes, homeworks and exams all in one big document (800+) pages. The tone is conversational, which helps to get a feel of the langauge of the subject. This is a graduate level course.

MIT's Mathematics for Computer Science -- 6.042J, has exhaustive notes on Discrete Mathematics -- 2010

Stanford CS 168: The Modern Algorithmic Toolbox, Spring 2023 by Gregory Valiant; see Tim Roughgarden's Lecture Notes

Cambridge - Algorithms – Course materials

Video Lectures

Articles

Interviews

Books

Free

  • Algorithms by Vazirani et. al
  • Lisp, the Universe and Everything: "Programming Algorithms" Book Work in progress (as of Jan 2020). All the algorithms and data structures are explained using Lisp.
  • Algorithms and Data Structures - The Basic Toolbox by Kurt Mehlhorn and Peter Sanders. [PDF]
  • Algorithms for Modern Hardware - Algorithmica -- Its intended audience is everyone from performance engineers and practical algorithm researchers to undergraduate computer science students who have just finished an advanced algorithms course and want to learn more practical ways to speed up a program than by going from O(nlog⁡n)O(nlogn) to O(nlog⁡log⁡n)O(nloglogn).
  • (Tuzhilin and Zhang 2023) PDF This book is result of Sino-Russian collaboration. It collects the lectures which were given to students of mathematics departments of Moscow State University and Peking University about graph theory and its applications. The book’s narrative is sequential: earlier theorems and definitions are used in later material.

Non-Free

  • Introduction to Algorithms aka CLRS (affl.) -- the standard textbook for this course in large percentage of undergraduate and graduate classes. The book weighs a ton and looks intimidating, but is quite approachable in practice.

Topics

Preliminaries

Probability and Randomness

Dynamic Programming

Divide and conquer algorithms partition the problem into disjoint subproblems, sovle the subproblems recurisvely and then combine their solutions to solve the original problem.

Dynamic programming algorithms apply when the subproblems overlap --ie., when subproblems share subproblems. Dynamic program solves each subproblem just once and saves the answer, thereby avoiding the recomputing the answer.

DP is typically applied to optimisation problems.

Four steps of DP algorithm:

  1. Characterise the structure of an optimal solution.
  2. Recursively define the value of an optimal solution
  3. COmpute the value of the optimal solution, typicall bottom-up.
  4. Construct an optimal solution from computed info.

//Characterise the structure. Define the Value. Compute the Value. Construct.

To Read

2013-11-01: Towards a discipline of experimental algorithms by Bernard Moret.

2013-11-01: O-notation considered harmful (use Analytic Combinatorics instead)

References

notes#algorithms#books#computer-science#data-structures

Ergodicity

Lecture notes – Ergodicity Economics

These notes are our attempt to re-develop economic theory from scratch, namely starting with the axiom that individuals optimize what happens to them over time, not what happens to them on average in a collection of parallel worlds. The latter, surprisingly, is the starting point of the currently dominant form of economic theory. -- Ole Peters

Everything We’ve Learned About Modern Economic Theory Is Wrong - Bloomberg article -- Dec 11, 2020.

(1) How you will go bust on a favorable bet. (Kelly/Shannon/Thorp) - YouTube -- Taleb's video about the

notes

2019

Markdown

Why Use plain text (like markdown) to write things? Dr Kieran Healy has a nice website explaining how

The Markdown Guide for the Perplexed

Desktop Tools

I MarkEdit as the default app to open .md files on MacOS. Otherwise it is painful to wait for XCode to open it. MarkEdit is quite fast and nimble. I have also used Typora before for the same thing before.

  • Glow terminal based markdown reader.
  • markdown-service-tools are a collection of OS X Services designed to make writing Markdown text that much easier.
  • PanWriter Markdown editor for macOS, Windows and Linux.
  • MarkdownDown -- "Convert any webpage to a clean markdown w/ images downloaded."

Web tools

uselink — Turn any doc into a living, shareable link

Wrappers

Foliant

Foliant is a all-in-one documentation authoring tool. It lets you produce standalone documents in pdf and docx, as well as websites, from single Markdown source. Foliant is a higher order tool, which means it uses other programs to do its job. For pdf and docx, it uses Pandoc, for websites it uses MkDocs. Foliant preprocessors let you include parts of documents in other documents, show and hide content with flags, render diagrams from text, and much more.

Who Is Foliant for?

You'll love Foliant if you:

  • need to ship documentation as pdf, docx, and website forms
  • want to use Markdown with consistent extension system instead of custome syntax for every new bit of functionality
  • like reStructuredText's extensibility and Asciidoc's flexibility, but actually would rather use Markdown
  • want a tool that you can actually write custom extensions for without dealing with something as overengineered as Sphinx
  • have documentation spread across many repos and want to reuse parts between documents

pp - A generic Preprocessor (with Pandoc in mind)

PP is a text preprocessor designed for Pandoc (and more generally Markdown and reStructuredText).

pp now implements:

  • macros
  • literate programming
  • GraphViz, PlantUML and ditaa diagrams
  • Asymptote and R figures
  • Bash, Cmd, PowerShell, Python, Lua, Haskell and R scripts
  • Mustache

GitHub - drasimwagan/mdv: MDV — a Markdown superset for documents, dashboards, and slides with embedded data and visualizations. HTML + PDF export, live preview, VS Code extension. · GitHub

Markdown to PDF conversion

HTML to Markdown

Show HN: HTML-to-Markdown – convert entire websites to Markdown with Golang/CLI | Hacker News -- a few tools mentioned, like jina.ai

markdown.new - Markdown for Agents | Cloudflare-Powered URL to Markdown

Markdown Styling

ReMarkdown — HTML that looks like Markdown text

Command line tools

Python One liner:

python3 -m markdown -x markdown.extensions.fenced_code -x markdown.extensions.codehilite index.md > index2.html

mq '.h' demo.md
... lists all the h elements ..

Browser tools

Python Markdown

This is intersting to me because i use Python Markdown to render this site, and use many built-in and third party extensions.

Libraries

Mistune for Python

: * claims to be fastest markdown renderer for Python

  • mistune.markdown('I am using **mistune markdown parser**')
  • options are configurable
  • contribs

lowdown — simple markdown translator is a Markdown translator producing HTML5 and roff documents in the ms and man formats. Written in C. "Beyond the basic Markdown syntax support, lowdown supports the following Markdown features and extensions: autolinking, fenced code, tables, superscripts, footnotes, disabled inline HTML, “smartypants”, metadata, commonmark (in progress). You can control which parser features are used by using the -d (disable feature) and -e (enable feature) flags."

Gottox/smu: simple markup - markdown like syntax

syntax-tree/mdast: Markdown Abstract Syntax Tree format; see also syntax-tree/unist: Universal Syntax Tree used by unifiedjs

lkesteloot/teamten's markdown parser in Java is very easy to understand how the parsing works. It is a good read!

wernsey/d.awk: An Awk script to generate documentation from Markdown comments in C, C++, Java, JavaScript, C# source code.

Markdown for the component era | MDX MDX allows you to use JSX in your markdown content. You can import components, such as interactive charts or alerts, and embed them within your content. This makes writing long-form content with components a blast. 🚀

Competing mark-up languages

notes#markdown

Email

Email Driven Apps

"Email As API" #concept

Third party hosted email services

After considering different alternatives, migadu appears to be a good solution to host email on; Nov 2019.

Moving off Google

Using gmail

Self Hosting email

notmuch -- "If you want the convenience of fast, reliable search of all your email, but don't want to give a 3rd-party access to your email... Then notmuch may be exactly what you've been looking for."

How To Run Your Own Mail Server

Synergetic Labs | Email Alchemy; HN

Having recently had a small side project which required email capabilities whilst simultaneously wishing to avoid adopting any specific ecosystem, I set about building a self-hosted email solution.

Mail is not hard via [HN] where I found this interesting comment by user skrause.

I think the biggest pain in hosting your own mail server is getting your outbound mail delivered into the mailboxes of the large providers without being marked as spam. Especially if you don't actually send a lot of mail, so you can never really build up a good IP reputation.

That's why I generally recommend a hybrid setup: Host inbound mail completely by yourself so that you have full control, but ship off outbound mail to a trusted relay of a privacy oriented provider. For example https://posteo.de/en doesn't filter any outgoing messages by sender, so you can send mail from your own domains. Your local Postfix can DKIM sign your mail before sending them to Posteo and if you add include:posteo.de to your SPF record all your mail will be DKIM signed, SPF authenticated and coming from a reputable IP, so all your deliverability issues will be gone.

How to run your own mail server (2017) | Hacker News; some new discussion in Nov 2019.

vedetta-com/caesonia: OpenBSD Email Service

Stalwart Mail Server | Stalwart Labs Ltd.

maddy is an email server designed for transactional email, with web hooks on different events. It replaces Postfix, Dovecot, OpenDKIM, OpenSPF, OpenDMARC and more with one daemon with uniform configuration and minimal maintenance cost. Maddy Mail Server implements all functionality required to run a e-mail server. It can send messages via SMTP (works as MTA), accept messages via SMTP (works as MX) and store messages while providing access to them via IMAP. In addition to that it implements auxiliary protocols that are mandatory to keep email reasonably secure (DKIM, SPF, DMARC, DANE, MTA-STS).

  • Learn and Test DMARC Here, you'll get a visual breakdown of how email servers communicate, giving you a better understanding of SPF, DKIM, and DMARC and how they work together.

Self-Hosted email is the hardest it's ever been, but also the easiest.

Calendars

Searching Email

Elasticsearch For Beginners: Indexing your Gmail Inbox | Hacker News; see links within.

Email testing

  • MailCatcher -- Catches mail and serves it through a dream.

MailCatcher runs a super simple SMTP server which catches any message sent to it to display in a web interface. Run mailcatcher, set your favourite app to deliver to smtp://127.0.0.1:1025 instead of your default SMTP server, then check out http://127.0.0.1:1080 to see the mail that's arrived so far.

last release of mailcatcher was in 2016.

Mailhog: How to Use It and Why | Mailtrap Blog

"Mailhog alternatives MailCatcher, SMTP Bucket, FakeSMTP, and LunaticSMTP are good replacements for MailHog."

In contrast to these tools, Mailtrap is a purely online service. You don’t have to tinker with any sort of installation or setup. All you need to do is to configure your SMTP server according to the provided credentials. After that, you can get on with testing.

Securing email

Sending Email

  • Brevo API for commercial uses (newsletters, transactional, customer service etc)
  • mjml-The Responsive Email Framework MJML is a markup language designed to reduce the pain of coding a responsive email. Its semantic syntax makes it easy and straightforward and its rich standard components library speeds up your development time and lightens your email codebase. MJML’s open-source engine generates high quality responsive HTML compliant with best practices. MJML rolls up all of what Mailjet has learned about HTML email design over the past few years and abstracts the whole layer of complexity related to responsive email design.
  • A Deep Dive into Email Deliverability in 2024

notes#email

Datalog

notes#programming-language

Presentation

Presentation tools

Installing metropolis beamer theme on Mac

metropolis is a modern beamer theme that uses Mozilla's Fira Sans typeface.

git clone git@github.com:matze/mtheme.git
cd mtheme
make
mkdir -p `kpsewhich -var-value TEXMFHOME`/tex/latex/beamer/base/themes/theme
cp *.sty `kpsewhich -var-value TEXMFHOME`/tex/latex/beamer/base/themes/theme

See also: TeXLive

notes

Object Oriented Programming

Object-oriented programming: some history, and challenges for the next fifty years

This article is based on a lecture given on 22ndAugust 2011, on theoccasion of the scientific opening of the Ole-Johan Dahl hus at the Universityof Oslo. It looks at the foundational ideas fromSimulathat stand behindobject-orientation, how those ideas have evolved to become the dominantprogramming paradigm, and what they have to offer as we approach thechallenges of the next fifty years of informatics.

notes

Erlang Programming Language

See also: elixir and gleam

Concepts

Scalability

Books / Learning Resouces

Papers / videos

Emacs

John Daily's collection of Erlang links.

Instruction

Documentation

Community

Mailing lists

Advocacy

Experience Reports

Style

Libraries

NIFs

Tools

Projects

Exercises

General functional programming exercises

Alternative languages targeting the Erlang VM

Scheduling

Misc

Blogs

notes

Ansible

To look into:

notes

JSON

About JSON

Fast JSON parsers

Query Tools

First see jq.

ObjectPath - The agile query language

In weather readings find cities where temperature is higher than 25°C and the sky is clear.

$..*[@..temp > 25 and @.clouds.all is 0].name

Gron -- "Make JSON greppable"

▶ gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | fgrep "commit.author"
json[0].commit.author = {};
json[0].commit.author.date = "2016-07-02T10:51:21Z";
json[0].commit.author.email = "mail@tomnomnom.com";
json[0].commit.author.name = "Tom Hudson";

Fastgron

fastgron transforms JSON into discrete assignments to make it easier to grep for what you want and see the absolute 'path' to it. It eases the exploration of APIs that return large blobs of JSON but have terrible documentation.

fastgron is a high-performance JSON to GRON converter, developed in C++20, utilizing simdjson library. It's 50x faster than gron on big files (400MB/s input / 1.8GB/s output on M1 Macbook Pro), so it makes big JSON files greppable.

antonmedv/fx: Command-line tool and terminal JSON viewer 🔥

glomRestructuring data, the Python way.

from glom import glom

target = {'a': {'b': {'c': 'd'}}}
glom(target, 'a.b.c')  # returns 'd'

Charting tools

JSON Templating

JSON Schema

JSON-LD

Being on The Semantic Web is easy, and, frankly, well worth the bother JSON-LD = (JSON "for Linked Data") is the leading format for encoding Sematic Web metadata. It's just JSON, really. There are schemas that lay out the "types" you can use to describe the stuff on your page.

JSON Databases

Softmotions/ejdb: EJDB2 — Embeddable JSON Database engine C library. Simple XPath like query language (JQL). Websockets / Android / iOS / React Native / Flutter / Java / Dart / Node.js bindings.

EJDB query language (JQL) syntax inspired by ideas behind XPath and Unix shell pipes. It designed for easy querying and updating sets of JSON documents. See ejdb.

See also: BBEdit JSON format

notes#json

Iron-Clad Java

These are the notes from the book - 'Iron-Clad Java: Building Secure Web Applications' by Jim Manico, August Detlefsen and others, published by Oracle Press.

I picked up the book from our local library on a whim and decided to give it a go. The book was published in 2015, and so hopefully it is not too out of date.

Web application security basics

As expected in a book from Oracle, the book appears to focus on Java and J2EE technologies from the get go.

Untrusted data: any data that enters your web application from an outside source, even databases and other applications within your organization, should be treated as untrusted. Untrusted data can be any part of the HTTP request . Do not trust any aspect of it. Attacked can and will modify all aspects of a HTTP request to exploit your web application.

Intercepting proxies are important tools in intercepting and modifying requests from users (human and machines). Attackers can use these proxy tools to:

  • modify user-entered input
  • add/modify/delete headers, cookies, and HTTP verbs(GET, POST, DELETE)

Some well known intercepting proxies are:

  1. Burp suite
  2. OWASP ZAP
  3. OWASP WebScarab
  4. Firefox plug-in Tamper Data

Safe handling of dat means properly validating all sources of data that enter your web application and also properly encoding data when it the application to a web page, database or other location.

Security is often lumped into non-functional business requirement like performance. Security requirements have to be fluid in order to adapt to the ever-evolving threat landscape (and businesss requirements -- ed).

Security Quality -- business rules to ensure that the data is consistent with business business requirements.

HTTP security considerations

HTTP is a stateless, request and response protocol that drives the vast majority of web traffic.

HTTPS allows for:

  1. confidentiality -- protect your data from being seen
  2. integrity -- protect your data from being changed
  3. authenticity -- assure that the domain you are visiting is really the real version of that domain

Cipher suits are negotiated between browser and server to determine the cryptographic strength of the session transport. This meanss the cryptographic strength of an HTTPS connection can vary widely between diffetnt HTTPS servers.

HTTP/S GET request

components of a HTTP request

  • HTTP verb: GET
  • Resource: https://www.btbytes.com/
  • Version: HTTP/1.1
  • Request Headers:
    • User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:2232.0)
    • Accept: text/html
    • Accept-Language: en-US, en; q=0.5
    • Connection: keep-alive
    • Host: www.btbytes.com

Questions:

  1. what are the valid values for these request headers?
  2. how much are they validated by servers?
  3. are they used as attack vectors?
  4. what are the well known exploits around Request Headers.

notes#java#security-engineering

Kannada

See: kannada-poetry

Learning

For beginners: Kannada Kalike

From a by Stelo who is learning Kannada on his own:

  1. An Intensive Course in Kannada by Lingadevaru Halemane (CIIL, Mysore)
  2. भारतीय भाषा ज्योति कन्नड़ (CIIL, Mysore)
  3. कन्नड़ स्वयं बोधिनि (कन्नड़ अभिवृद्धि प्राधिकार)
  4. A Practical Key to the Kanarese Language

For those learning to learn more advanced Kannada, recommended by Stelo

  1. Zydenbos' A Manual of Modern Kannada
  2. Spencer's Kannada Grammar
  3. Halemane's An Intensive Course in Kannada
  4. Jensen's Grammatik der kanaresischen Schriftsprache
  5. Dr. Kalegoda's भारतीय भाषा ज्योति कन्नड़
  6. Halemane's Kannada A Self-Instructional Course (in 5+ languages)
  7. Andronov's Язык каннада (this booklet is more of an overview of Kannada grammar than a full-fledged course).

One can refer Kittle dictionary which is available on online. Old and good one. I also use Champu nudigannadi for few words which were from Old Kannada literature. via

Grammar

Websites

Blogs

Videos

Books

Fonts

notes#kannada