Archive

davecheney/httpstat: It's like curl -v, with colours. "It is a single file🌟 Python script that has no dependency👏 and is compatible with Python 3🍻"

--

Hurl - Run and Test HTTP Requests

Hurl is a command line tool that runs HTTP requests defined in a simple plain text format.

It can chain requests, capture values and evaluate queries on headers and body response. Hurl is very versatile: it can be used for both fetching data and testing HTTP sessions.

Hurl makes it easy to work with HTML content, REST / SOAP / GraphQL APIs, or any other XML / JSON based APIs.

In some users view, org-mode is itself reason enough to learn to use emacs, even if you don't use it for anything else (programming).

Zettelkasten

GitHub - nobiot/Zero-to-Emacs-and-Org-roam: Step by step guide from zero to installing and setting up Emacs and Org-roam on Windows 10

Org mode for writing web documents

While Org-mode is used by many people as an organizer, TODO list tracker, journal etc., I'm interested in it as primarily as a markup language that can be extended. This is is primary area where I want to become good at vis-a-vis orgmode (Jan 2020).

web document friendly CSS themes

See my Org-simple CSS theme page.

• dakrone demo
• rtags doc
• This page (eden.tuxee.net/org/tree) is a good example of writing a standalone "document" using Org that is great for reading.
• CSS for Org-exported HTML

Blogging with Org mode

• bastibe/org-static-blog: A static site generator using org-mode
• A blog in pure Org/Lisp
• Two wrongs, a blog published using org.

Links

• 200ok-ch/organice: An implementation of Org mode without the dependency of Emacs - built for mobile and desktop browsers -- very nice!
  • Introduction to "organice" - Using Org mode from the browser and smartphone - YouTube

• LaTeX, org-mode and animations - Edgar Aroutiounian -- auto insert header when you open a new .org file.
• Convert org files to html using a bash script
• Getting organized with Org Mode -- 2014-11-27
• Org-bullets.el -- show org-mode bullets as UTF-8 characters.
• Org mode quickref

Org Libraries

• jkitchin/org-ref: org-mode modules for citations, cross-references, bibliographies in org-mode and useful bibtex tools to go with it.

My notes

• the metadata tags at the top of the document start with #+TAG: value format. That is "hash plus tag colon value" format.
• use the - character for un numbered lists
• what are the keyboard shortcuts to type blocks like EXAMPLE and CODE. Typing C-c C-, brings up a menu using which i could select an src block or an example block.

List of programs and shell scripts that I find useful.

• 2014-11-26: Conkeror
• 2015-05-06: ccat -- colourised cat. there are other ways to do it, including using pygmentize
• 2016-07-22: joeyespo/grip: Preview GitHub Markdown files like Readme locally before committing them.
• 2018-02-19: java_launcher: Helper script to treat Java programs as scripts.
• 2019-12-05: imgproxy -- imgproxy does one thing — resizing remote images — and does it well. It works great when you need to resize multiple images on the fly to make them match your application design without preparing a ton of cached resized images or re-doing it every time the design changes. imgproxy is a Go application, ready to be installed and used in any Unix environment
• 2020-02-25: antonmedv/fx: Command-line tool and terminal JSON viewer 🔥
• 2020-02-25: antonmedv/red: Terminal log analysis tools

Meta (lists made by other people)

• cjbarber/ToolsOfTheTrade: Tools of The Trade, from Hacker News.
• An Illustrated Guide to Useful Command Line Tools - WezM.net by Wesley Moore

Chrome extensions

• PriceBlink --useful for comparison shopping. I've saved quite a bit of money using this extension.
• Personal Blocklist -- filter out content farms from your searches. -Adblock Plus -- A must have for all. -Send this link with Gmail -- very useful to share the web links via email. Works even Google Apps domains. -MeasureIt! -JSONView -- "an extension that helps you to parse and view JSON documents". Highly recommended if you do web development -Screen Capture -- screen capture, highlighting and annotation.

Safari extensions

• Ghostery -- block intrusive javascript.

Pandoc

pandoc is a fantastic program to convert text to various output formats.

Typcial usage: To convert a markdown formatted text file to html

$ pandoc somefile.txt -f markdown -t html --standalone -o somefile.html

See the detailed pandoc page.

Command line tools

abcde

Command line tool to rip CDs to .mp3.

Installation: $ sudo apt-get install abcde id3v2

Insert Disc #1 of a 3-cd audio book $ abcde -o mp3 -p -W 1.

bcat

bcat pipe to browser utility.

Typical usage: To see the git log.

$ git log -p --color | bcat

fd

sharkdp/fd: A simple, fast and user-friendly alternative to find.

gist

gist pastes code, text to [gist.github.com] from command line.

$ gist one.txt

partimage

partimage is a Linux disk imaging software. It has a curses based interface to guide you through the options.

Typical usage: make a backup of an entire HDD(say SSD/Flash of a prototype) to your desktop.

Make Image:

$ partimage -z1 save /dev/sdg1 $HOME/backup/prototype-hdd-20100707.partimg.gz

Restore:

$ partimage restore /dev/sdg1 $HOME/backup/prototype-hdd-20100707.partimg.gz

qpdf

qpdf -- is a command-line program that does structural, content-preserving transformations on PDF files.

http-prompt

• HTTP Prompt - An interactive command-line HTTP client

pip install http-prompt

fzf

junegunn/fzf: A command-line fuzzy finder written in Go

peco

• peco/peco: Simplistic interactive filtering tool

q

• q is a command line tool that allows direct execution of SQL-like queries on CSVs/TSVs (and any other tabular text files).

entr

entr(1) -- Run arbitrary commands when files change

ripgrep

ripgrep recursively searches directories for a regex pattern. See rga, like rg, but also search in PDFs, E-Books, Office documents, zip, tar.gz, etc.

sd

sd is an intuitive find & replace CLI.

rargs

rargs xargs + awk with pattern matching support. ls *.bak | rargs -p '(.*)\.bak' mv {0} {1}

rak

lizmat/App-Rak: 21st century grep / find / ack / ag / rg on steroids

Code tools

• etsy/hound: Lightning fast code searching made easy

Database tools

• dbeaver -- crossplatform database manager for all popular sql dbs and some nosql dbs.

Web development tools

• HTTPie + prompt_toolkit = an interactive command-line HTTP client featuring autocomplete and syntax highlighting

• TheWaWaR/simple-http-server: SimpleHTTPServer in Rust

• Yaak is a cross-platform desktop app for interacting with REST, GraphQL, and gRPC

Data Cleaning

• jq - command line json processor
• HTML-XML-utils -- makes scraping tasks that much easier.
• pup - Parsing HTML at the command line. written in go.
• xidel - HTML/XML data extraction tool which is written in Pascal; also by the same author - texstudio

Interesting ~/bins on github

• https://github.com/sartak/bindir
• https://github.com/dustin/bindir

Some more linked on this page: 15 Greatest Open Source Terminal Applications Of 2012

Interesting

• ronn - the opposite of roff.

Lorem ipsum

• Jeffsum | A little text placeholder generator of Jeff Goldblum awesome... obviously.

OCR

• Screenotate Take screenshots you can search.

ASCII Banner

Use Figlet and Toilet to generate ascii text banners in linux terminal.

NOTE: When you can, prefer ASV over CSV -- Text File formats – ASCII Delimited Text – Not CSV or TAB delimited text, especially  when you aren’t writing the files by hand. (Caveat: "UTF-8 won’t trip this since it is ASCII compatible. That is, any ASCII byte in valid UTF-8 encoded bytes corresponds to its ASCII codepoint, regardless of context. (All continuation bytes are at least 0x80 and all leading bytes for codepoints greater than U+007F are at least 0xC0, which makes the 0x1E and 0x1Fseparators safe.") UTF-16 or UTF-32 on the other hand...)

Convert CSV to ASV: xsv fmt -t$'\x1f' one.csv > one.asv

Command line tools

• GitHub - dinedal/textql: Execute SQL against structured text like CSV or TSV
• GitHub - BurntSushi/xsv: A fast CSV command line toolkit written in Rust.; a fork called qsv has added more features to the original xsv.
• csvkit 1.0.3 — csvkit 1.0.3 documentation
• Miller -- "Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON. You get to work with your data using named fields, without needing to count positional column indices."
• interesting thread
• tab -- A modern text/number processing language for the shell.

MacOS

• Table Tool is a fast, open source, mac native application to open csv files. I have set this as my default CSV opener.

Web tools

csvbase, a simple web database, faq

• View, add, edit and delete rows from a browser
• Export to CSV, XLSX, Parquet, etc
• Each table has a single URL, RESTfully accessed by HTTP verb
• has an API with API key, that you get when you register
• works out of the box with R, pandas, duckdb, apache-spark
• paste cells directly from Microsoft Excel
• code is open source on github under AGPL license, written in python.

Apache Metamodel

Apache Metamodel provides a common interface for discovery, exploration of metadata and querying of different types of data sources. Uniform connector and query API for CSV & data formats, relational databases, noSQL databases.

With MetaModel you use a type-safe SQL-like API for querying any datastore:

DataContext dataContext = DataContextFactory.create[TypeOfDatastore](...);
DataSet dataSet = dataContext.query()
    .from("libraries")
    .select("name")
    .where("language").eq("Java")
    .and("enhances_data_access").eq(true)
    .execute();

The MetaModel query API allows you to use the power of SQL, even on data formats such as CSV files, Excel spreadsheets, NoSQL databases and more.