Archive

casey/just: 🤖 Just a command runner; manual.

Interesting use of Just -- Create an Internal CLI using Just

This is my justfilethat I used to generate this file. I have almost deprecated the use of taskfile.

# Default recipe (similar to 'all' in Makefile)
default: full scrapbook calendar codefiles opml papers

here0:
    ./bin/mkhtmlindex ./content/tweetstorms > ./content/tweetstorms/here0.html

scrapbook:
    bin/scraps.py

calendar:
    bin/calendarview.py

c:
    ./sitebuild

full:
    ./sitebuild --full

sync:
    /usr/bin/rsync -avz -d --delete-after -e ssh ./output/ myweb@surya.btbytes.com:/data/web/www.btbytes.com/www/

push:
    git push origin master

serve:
    cd output && python3 -m http.server 8004

setup:
    python3 -m venv venv
    venv/bin/pip install --upgrade pip setuptools wheel virtualenv
    venv/bin/pip install -r requirements.txt
    mkdir -p .cache

weekly:
    bin/python3 weekly.py

pb:
    bin/pinboard-to-sqlite posts backups/pinboard.db

codefiles:
    pygmentize -O full,style=emacs,linenos=1,lineanchors=1 -f html -l python -o content/sitebuild.html sitebuild
    pygmentize -O full,style=emacs,linenos=1,lineanchors=1 -f html -l lua -o content/filters.html pandoc/filters.lua
    pygmentize -O full,style=emacs,linenos=1,lineanchors=1 -f html -l yaml -o content/Taskfile.yml.html Taskfile.yml
    pygmentize -O full,style=emacs,linenos=1,lineanchors=1 -f html -l python -o content/opml_to_html.html bin/opml_to_html.py
    pygmentize -O full,style=emacs,linenos=1  -l python -o content/papers/papersindex.html bin/papersindex.py
    pygmentize -O full,style=emacs,linenos=1  -l python -o content/papers/papersindex.html bin/papersindex.py

opml:
    bin/opml_to_html.py

papers: nbrefs
    python bin/papersindex.py

nbrefs:
    bibtex2html  -o content/bib/nbrefs -css https://www.btbytes.com/css/semantic.css \
        --title References \
        --header "<nav><p><a href="/">Home (www.btbytes.com)</a>, <a href="nbrefs_bib.html">View .bib file</a>, <a href="nbrefs.bib">Download .bib file</a>.</p></nav>" \
        --footer "<footer><p>Generated on <code>$(shell date)</code></p></footer>" \
        content/bib/nbrefs.bib

soupault:
    soupault

clean:
    rm -rf output

roapi/roapi: Create full-fledged APIs for slowly moving datasets without writing a single line of code. -- ROAPI automatically spins up read-only APIs for static datasets without requiring you to write a single line of code. It builds on top of Apache Arrow and Datafusion. The core of its design can be boiled down to the following:

• Query frontends to translate SQL, GraphQL and REST API queries into Datafusion plans.
• Datafusion for query plan execution.
• Data layer to load datasets from a variety of sources and formats with automatic schema inference.
• Response encoding layer to serialize intermediate Arrow record batch into various formats requested by client.

Watchfiles is a file watching and code reload in python, written in rust.

Usage:

# pip install watchfiles
from watchfiles import watch

for changes in watch('./path/to/dir'):
    print(changes)

Also supports async via awatch:

import asyncio
from watchfiles import awatch

async def main():
    async for changes in awatch('/path/to/dir'):
        print(changes)

asyncio.run(main())

Alternatives

• entr

A modern alternative to C and C++ for system programming.

Why?

• Why Rust?, a mini book by Jim Blandy of Mozilla. [Sep 2015]
• Rust for Clojurists [updated Sep 2015].
• A Swift guide to Rust · faq.sa
• Oxidation Plan by Mercury SCM gives good reasons to replace Python code with Rust (and not C).
• Why is Rust difficult?
• (Blanco-Cuaresma and Bolmont 2017) arXiv

How

• How to Learn Rust in 2024: A Complete Beginner’s Guide to Mastering Rust Programming | The RustRover Blog

Articles / Talks

• Visualising Rust's type system; [Jul 2015]
• OMG Iterators by Alexis Beingessner of Carleton University + Mozilla Research
• Packet Capturing MySQL with Rust -- talks about Spyglass a network traffic tool aimed at MySQL data packets.
• Rust Means Never Having to Close a Socket - good discussion of the borrow checker etc.,
• What Are Sum, Product, and Pi Types? by Manish.
• Math with distances in Rust: safety and correctness across units
• Rust for Serious Developers by Armin Ronacher.
• Karol Kuczmarski's Blog – Rust as a gateway drug to Haskell
• Learning systems programming with Rust - Julia Evans Sept 2016.
• 5 Tips for Writing Small CLI Tools in Rust - Pascal’s Scribbles
• What Elm and Rust Teach us About the Future
• Elegant Library APIs in Rust - Pascal’s Scribbles
• Josef “Jeff” Sipek » Rust Pointers for C Programmers
• 21-security-perf.pdf by Pierre C, a contributor to Suricata.
• Learning rust programming by AndreaPessino
• From Bash to Rust Sept 2018.
• Rust after the honeymoon Oct 2020 by BryanCantrill.
• Rust Anthology Master List

Books

The Little Book of Rust Books

Print books

• PacktPublishing/Hands-On-Concurrency-with-Rust: Hands-On Concurrency with Rust, published by Packt
• PacktPublishing/Network-Programming-with-Rust: Network Programming with Rust, published by Packt

Online books

• Introduction - The Rust Performance Book
• Rust Design Patterns
• Learn Rust 101 | A guide to aid you in your journey of becoming a Rustacean (Rust developer). at https://rust-lang.guide
• Effective Rust
• Data Engineering With Rust

Experience reports

• Packet Capturing MySQL with Rust
• From Mordor, with love: Rust: First impressions from a C++ developer
• Falling in love with Rust by Bryan Cantrill.
• From Julia to Rust Aug 2024.

Papers / reports

• You can't spell trust without rust -- master's thesis of Alexis Beingessner. PDF

Videos / Talks

• into_rust(): screencasts for learning Rust
• Rust Camp 2015
• Zen and the art of convincing your company to use Rust
• Steve Klabnik on Why Rust [Jul 2015]
• RustCamp 2015
• Gökberk Yaltıraklı videos about "port scanner in Rust, Dictionary server with Rust".
• C++Now 2017: Niko Matsakis "Rust: Hack Without Fear!" - YouTube

Blogs

• Llogiq's blog -- rust "learner's" blog.
• Three months of Rust; [Jun 2015] -- appears to be a decent blog to read in general.
• Herman Radtke's blog
• The blog of Martin Wernstål
• Design is refactoring
• Burntsushi's blog
• Ironic blog -- Explore the ownership system in Rust.
• Sevag H -- creator of pq.
• Introducing the Rust crash course by Snoyman (Haskell/Yesod)
• Sabrina Jewson; this entry talks about using Rust to build her blog; src.

Tutorials

• Learning Rust by quinedot.
• rust-learning
• mre/idiomatic-rust: A peer-reviewed collection of articles/talks/repos which teach idiomatic Rust.
• Rust Language Cheat Sheet
• Breakdown Notes; Jan 2019
• Rust Quiz ; med-hard.
• Welcome - 100 Exercises To Learn Rust, created by the author of Zero to Production in Rust.
• Welcome to Comprehensive Rust 🦀 - Comprehensive Rust 🦀
• Introduction - Roguelike Tutorial - In Rust
• amos - A half-hour to learn Rust
• Guaranteeing memory safety in Rust | Air Mozilla
• Notes on Rust by rbranson
• Rust: A Type System You Didn't Know You Wanted by Felix Klock of Mozilla. July 2015/ Curry On conf.
• Arcaders a tutorial series in building a game with Rust.
• rust-101 -- 16 part tutorial.
• The Rustonomicon -- The Dark Arts of Advanced and Unsafe Rust Programming
• 24 Days of Rust
• nrc/r4cppp: Rust for C++ programmers -- this is a pretty good collection of articles.
• Hello, Rust! -- This site will collect resources, guides, articles and links around Rust & WebAssembly.
• rochacbruno/py2rs: A quick reference guide for the Pythonista in the process of becoming a Rustacean
• A gentle introduction to Rust by Steve Donovan.
• The Periodic Table of Rust Types

Courseware

• The Adventures of OS; Making a RISC-V Operating System using Rust by Stephen Marz.
• EECS 3/495: Concurrent Programming in Rust -2017; 2016.
• Introduction - The Rust Programming Language ; the second edition of the Rust book, WIP.
• CIS 198: Rust Programming; Spring 2016.
• Rust class on Operating Systems at Virginia University.
• CIS 198: Rust Programming -- Fall 2016 at UPenn.

Source code to study

• LukeMathWalker/zero-to-production: Code for "Zero To Production In Rust", a book on API development using Rust.

Links

• Rust Playground
• Rust rosetta code problems
• This week in Rust

Libraries and tools

• Lib.rs — home for Rust crates "Index of 97k+ Rust libraries and applications. Fast, lightweight, opinionated, unofficial alternative to crates.io"
• Blessed.rs "which crates they ought to use and which crates they ought to trust."
• BurntSushi/xsv A fast CSV toolkit written in Rust. xsv is a command line program for indexing, slicing, analyzing, splitting and joining CSV files.
• rustdoc and sliderust to convert this into this
• Multirust -- tool for managing multiple installations of the Rust toolchain. It replaces the standard Rust toolchain with components that dynamically choose between alternate implementations based on context.
• .gdbinit for rust
• coio-rs -- Coroutine scheduling with work-stealing algorithm; Non-blocking I/O; Asynchronous computing APIs.
• Nickel.rs -- web application framework for Rust.
• leptos-rs/leptos: Build fast web applications with Rust.
• poem-web/poem: A full-featured and easy-to-use web framework with the Rust programming language.

WASM

• Seed -- A Rust framework for creating web apps

Applications

To study code, library usage etc.,

• mcux/kylyp: A new Forum. The project uses rust, rocket, diesel, postgresql, and responsive design to build a new Forum.

Command Line Tools in Rust: See also: Common *nix commands written in Rust – gcollazo

ekzhang/rustpad: Efficient and minimal collaborative code editor, self-hosted, no database required alternative to codepad.

Editor setup

• Emacs and Rust. The best setup guide for emacs to date. [Aug 2015]
• Visual Studio + Rust's tasks.json setup.

Performance

• Profiling Rust Applications on Linux using valgrind, oprofile.

Memory Management

• Memory Management in Rust and Swift – Ryan Levick – Medium
• Custom allocators in Rust -- "making it possible for different parts of a program to chose their memory allocation strategies."

Safety-critical programming

What is Ferrocene?

Ferrocene is the main Rust compiler - rustc - but quality managed and qualified for use in automotive and industrial environments (currently by ISO 26262 and IEC 61508) by Ferrous Systems. It operates as a downstream to the Rust project, further increasing its testing and quality on specific platforms.

References and borrowing

• Rust: The Hard Parts - Part One · naftuli.wtf -- references and borrowing.

Lifetimes

• Rust Lifetimes for the uninitialised by Florian Gilcher.
• Understanding Rust Lifetimes; Jan 2019.
• The Power of Lifetimes
• common-rust-lifetime-misconceptions · pretzelhammer/rust-blog

Async / Await

• Introduction - Async programming in Rust with async-std

Web

• Introduction · The Rusty Web
• EpicZenGarden webassembly demo by unreal engine.
• Gotham | A flexible web framework created in Rust

Embedded

• Discover the world of microcontrollers through Rust!; hn

This book is an "introductory course" on microcontroller-based "embedded systems" that uses Rust as the teaching language rather than the usual C/C++.

Comparisons (Rust vs X)

• Comparing Rust and Java — Llogiq on stuff
• Rust for professionals -- "... intended for developers that already know another language. In the examples, Rust is compared with TypeScript, JavaScript or Java, sometimes with C++ or Kotlin."

• How Swift Achieved Dynamic Linking Where Rust Couldn't; Nov 2019.

FFI

• rustgo: calling Rust from Go with near-zero overhead
• Calling Rust From Python | bheisler.github.io
• The easiest way to speed up Python with Rust; Jul 2023.

// rustimport:pyo3
use pyo3::prelude::*;

#[pyfunction]  // ← expose the function to Python
fn fibonacci(number: u64) -> u64 {
...

import rustimport.import_hook
from rustfib import fibonacci

• Extending C with Rust – 101 – Spejss Blog
• Making Python 100x faster with less than 100 lines of Rust; Mar 2023. Good example with profiling. Uses pyo3
• PyO3/maturin: Build and publish crates with pyo3, rust-cpython and cffi bindings as well as rust binaries as python packages; Apr 2023. Some projects using maturin are ruff, polars, orjson, watchfiles and a few apache-arrow projects.

Discussions

• A discussion around "Advanced Rust Programming Book Draft"
• Yield/async/continuations etc.,
• What does Rust have instead of garbage collector? - SO
• A decent discussion of Rust vs other PLs on r/rust [Oct 2015].
• What has rust borrowed from Haskell
• rust - How can this instance seemingly outlive its own parameter lifetime? - Stack Overflow

Cargo

• Make your own make -- using Cargo as a general purpose task automation solution.

Installing a package that is not on crates.io: cargo install -f --git https://github.com/cjbassi/ytop ytop

• Cargo Dist distributes your binaries. and in combination with oranda it also generates websites.

Best Practices / Tips

• Perfect Rust Packaging
• spacejam/elements-of-rust: rust style and philosophy

Hardware / bare metal

• Rust bare metal on ARM microcontroller

Rust vs Python

• rochacbruno/py2rs: A quick reference guide for the Pythonista in the process of becoming a Rustacean

Rust and Python

• Making Python 100x faster with less than 100 lines of Rust uses py-spy for profiling; uses maturin to create the project, and pyo3 to create the bindings.

Rust vs Java for rest services

-- via reddit

I gave a talk (video) today about "Rust for Java programmers" at my local java users group. Part of the talk was comparing the implementation of a minimal rest service in a container for the following 4 approaches:

1. Payara-micro (JEE/JAXRS)
2. Wildfly-swarm (another JEE/JAXRS micro deploy)
3. Javaspark (non JEE "lightweight" framework)
4. Rust-rocket

The comparisons are striking, viewed in the same 1/2/3/4 sequence above:

• Container Image Size: 173MB / 151MB / 104MB / 2MB

• Container Startup Time: 22.2s / 17.6s / 1.6s / 0.6s

• Memory Usage at Startup: 630MB / 401MB / 34MB / 1.6MB

• Memory Usage under Stress: 647MB / 522MB / 74MB / 1.7MB

• CPU usage and throughput show a similar progression as above.

Nice job, Rocket! (results are unscientific. fight me) -- Source code

Success Stories

• Rust at OneSignal via

We had a similar challenge supporting sending this many notifications at OneSignal, which we solved using Rust. We recently hit a peak of 850 Million notifications per second, and 5 billion notifications per day. Here's a blog post on how we do it. Written back when we were at "only" 2 billion notifications per week: https://onesignal.com/blog/rust-at-onesignal/

---

Rolling log of things

2017-09-08:

There is a new book -- Programming Rust by Jim Blandy, and Jason Orendorff. Jim Blandy has also written a concise short book Why Rust [PDF]

Rust is currently unable to call directly into a C++ library; so that way Rust is probably behind D.

• simple event hooks in Rust
• Multithreading in Rust with MPSC (Multi-Producer, Single Consumer) channels
• A web application completely in Rust – Sascha Grunert – Medium
• Rust + actixweb success story July 2018
• My RustConf 2018 Closing Keynote Sept 2018.

Rust, by design, makes certain programming patterns more painful than others. This is a GOOD thing! It turns out that, for game development, the patterns that end up being the easiest to deal with in Rust closely mirror the patterns that are easiest for game development in any language. Rust highly rewards data-oriented design with simple, understandable ownership semantics, and this is great news because this is a really good fit for game development.

---

See also: rust2018

References

• Blanco-Cuaresma, Bolmont (2017). What Can the Programming Language Rust Do for Astrophysics?.