here you goThings I’ve made, read and saved — shared here.

Archive

9,692 entries: 92 from X, 8,929 from Pinboard, saved 1998 to 2026.

Browse by tag, site, author, folder or language

Tags

Sites

Authors

Folders

Languages

Showing 4,151–4,200 of 9,692

2019

JSON

About JSON

Fast JSON parsers

Query Tools

First see jq.

ObjectPath - The agile query language

In weather readings find cities where temperature is higher than 25°C and the sky is clear.

$..*[@..temp > 25 and @.clouds.all is 0].name

Gron -- "Make JSON greppable"

▶ gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" | fgrep "commit.author"
json[0].commit.author = {};
json[0].commit.author.date = "2016-07-02T10:51:21Z";
json[0].commit.author.email = "mail@tomnomnom.com";
json[0].commit.author.name = "Tom Hudson";

Fastgron

fastgron transforms JSON into discrete assignments to make it easier to grep for what you want and see the absolute 'path' to it. It eases the exploration of APIs that return large blobs of JSON but have terrible documentation.

fastgron is a high-performance JSON to GRON converter, developed in C++20, utilizing simdjson library. It's 50x faster than gron on big files (400MB/s input / 1.8GB/s output on M1 Macbook Pro), so it makes big JSON files greppable.


antonmedv/fx: Command-line tool and terminal JSON viewer 🔥


glomRestructuring data, the Python way.

from glom import glom

target = {'a': {'b': {'c': 'd'}}}
glom(target, 'a.b.c')  # returns 'd'

Charting tools

JSON Templating

JSON Schema

JSON-LD

Being on The Semantic Web is easy, and, frankly, well worth the bother JSON-LD = (JSON "for Linked Data") is the leading format for encoding Sematic Web metadata. It's just JSON, really. There are schemas that lay out the "types" you can use to describe the stuff on your page.

JSON Databases

Softmotions/ejdb: EJDB2 — Embeddable JSON Database engine C library. Simple XPath like query language (JQL). Websockets / Android / iOS / React Native / Flutter / Java / Dart / Node.js bindings.

EJDB query language (JQL) syntax inspired by ideas behind XPath and Unix shell pipes. It designed for easy querying and updating sets of JSON documents. See ejdb.


See also: BBEdit JSON format

notes#json

Iron-Clad Java

These are the notes from the book - 'Iron-Clad Java: Building Secure Web Applications' by Jim Manico, August Detlefsen and others, published by Oracle Press.

I picked up the book from our local library on a whim and decided to give it a go. The book was published in 2015, and so hopefully it is not too out of date.

Web application security basics

As expected in a book from Oracle, the book appears to focus on Java and J2EE technologies from the get go.

Untrusted data: any data that enters your web application from an outside source, even databases and other applications within your organization, should be treated as untrusted. Untrusted data can be any part of the HTTP request . Do not trust any aspect of it. Attacked can and will modify all aspects of a HTTP request to exploit your web application.

Intercepting proxies are important tools in intercepting and modifying requests from users (human and machines). Attackers can use these proxy tools to:

  • modify user-entered input
  • add/modify/delete headers, cookies, and HTTP verbs(GET, POST, DELETE)

Some well known intercepting proxies are:

  1. Burp suite
  2. OWASP ZAP
  3. OWASP WebScarab
  4. Firefox plug-in Tamper Data

Safe handling of dat means properly validating all sources of data that enter your web application and also properly encoding data when it the application to a web page, database or other location.

Security is often lumped into non-functional business requirement like performance. Security requirements have to be fluid in order to adapt to the ever-evolving threat landscape (and businesss requirements -- ed).

Security Quality -- business rules to ensure that the data is consistent with business business requirements.

HTTP security considerations

HTTP is a stateless, request and response protocol that drives the vast majority of web traffic.

HTTPS allows for:

  1. confidentiality -- protect your data from being seen
  2. integrity -- protect your data from being changed
  3. authenticity -- assure that the domain you are visiting is really the real version of that domain

Cipher suits are negotiated between browser and server to determine the cryptographic strength of the session transport. This meanss the cryptographic strength of an HTTPS connection can vary widely between diffetnt HTTPS servers.

HTTP/S GET request

components of a HTTP request

  • HTTP verb: GET
  • Resource: https://www.btbytes.com/
  • Version: HTTP/1.1
  • Request Headers:
    • User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:2232.0)
    • Accept: text/html
    • Accept-Language: en-US, en; q=0.5
    • Connection: keep-alive
    • Host: www.btbytes.com

Questions:

  1. what are the valid values for these request headers?
  2. how much are they validated by servers?
  3. are they used as attack vectors?
  4. what are the well known exploits around Request Headers.

notes#java#security-engineering

Kannada

See: kannada-poetry

Learning

For beginners: Kannada Kalike

From a by Stelo who is learning Kannada on his own:

  1. An Intensive Course in Kannada by Lingadevaru Halemane (CIIL, Mysore)
  2. भारतीय भाषा ज्योति कन्नड़ (CIIL, Mysore)
  3. कन्नड़ स्वयं बोधिनि (कन्नड़ अभिवृद्धि प्राधिकार)
  4. A Practical Key to the Kanarese Language

For those learning to learn more advanced Kannada, recommended by Stelo

  1. Zydenbos' A Manual of Modern Kannada
  2. Spencer's Kannada Grammar
  3. Halemane's An Intensive Course in Kannada
  4. Jensen's Grammatik der kanaresischen Schriftsprache
  5. Dr. Kalegoda's भारतीय भाषा ज्योति कन्नड़
  6. Halemane's Kannada A Self-Instructional Course (in 5+ languages)
  7. Andronov's Язык каннада (this booklet is more of an overview of Kannada grammar than a full-fledged course).

One can refer Kittle dictionary which is available on online. Old and good one. I also use Champu nudigannadi for few words which were from Old Kannada literature. via

Grammar

Websites

Blogs


Videos


Books

Fonts

notes#kannada

Ballerina programming language

"Ballerina" Could Become the Programming Language of Integration - By dhanushka

Ballerina is a static typed language.

Another special type of variable is that “anydata” data type. This type is a union of the ()|boolean|int|float|decimal|string|(anydata|error)[]|map<anydata|error>|xml|table data types. anydata variables can be used in places where you expect pure values.

Since Ballerina specifically designed to construct apps operating on networks, it natively supports JSON and XML.

json user = {
         fname: "Peter",
         lname: "Stallone",
         "age": age,
         address: {
             line: "20 Palm Grove",
             city: "Colombo 03",
             country: "Sri Lanka"
         }
    };
io:println(user.address.country);
xml x1 = xml `<book>The Lost World</book>`;
io:println(x1);

News

Articles

James Clark's blog:

notes#ballerina#jvm#programming-language

Scala programming language

See Also: Scala for Datascience, ScalaJS, coursier, select-scala

July, 2015 -- Scala book reviews Feb, 2018 -- Recursion and Trampolines in Scala

Random Notes

scala-cli—power repl -S 2.13 -A —ammonite-ver 2.5.11

Notes from Programming in Scala

Chapter 7 - built-in control structures.

  • has very few control structures
  • if while for try match and function calls.
  • scala accumulates control structs in libraries.
  • use val instead of var wherever possible. makes it easy to refactor. "equational reasoning".
  • no = sign while defining a function => returns a Unit value ()
  • while and do-while are NOT expressions. they return Unit.
  • prefer NOT to use while.
  • for
  • "generator" syntax -- file <- files. works on all collections.
  • file is a val.
  • Ranges -- 1 to 4 (yields 1,2,3,4)
  • 1 until 4 -- exlcude the upper bound (yields 1,2,3)
  • generators can also include filters -- for (f <- files if f.getName.endsWith('.py'))
  • nested iteration.
  • mid-stream variable bindings
  • new collection:
def pyFiles =
  for {
    file <- files
    if file.getName.endsWith('.py')
  } yield file
  • IMP: don't put yield inside the curlies.

Exceptions:

try {


} catch {
  case ex: FileNotFoundException => //handle
  case ex: IOException => //handle
}
  • like other control structures, try-catch-finally also results in a value. (this is unlike Java) //imp

Match:


val fruit =
    someArg match {
      case "foo" => "bar"
      case "banana" => "pepper"
      case _ => "what?"
}

Chapter 8- Functions and Closures

First class functions

(x: Int) => x+1

named function:

var increase = (x: Int) => x + 1
  • target typing: someNumbersList.filter((x) => x > 0). the type is inferred from the the list of integers.
  • alternatively: someNumbersList.filter(x => x > 0)

Placeholder syntax:

someNumbersList.filter( _ > 0)
val f = (_: Int) + (_: Int)

Chapter 12 -- Traits

Traits are like java interfaces with concrete methods + declare fields and maintain state.

Traits can do everything classes can do, except --

  1. trait cannot have any clas parameters (ie., those passed to primary constructors).
  2. super calls are not statically bound.

Unlike Java, adding a concrete method to scala trait is an one-time effort.

Chapter 15 -- Case classes and pattern matching


Notes from "Scala in Action"

Chapter 2 -- Getting started

// working with arrays and Lists
@ val nList = 1 :: 2 :: 3 :: 4 :: Nil
nList: List[Int] = List(1, 2, 3, 4)
@ nList.fil
filter     filterNot
@ nList.filter( _ > 2)
res17: List[Int] = List(3, 4)

@ // for comprehensions
@ oldList
res19: List[Int] = List(1, 2)
@ newList
res20: List[Int] = List(1, 2, 3)
@ for { a <- oldList; b <- newList } println (a+b)
2
3
4
3
4
5
@ // above is imperative form
@ // The functional form for comprehension is:
@ // also called the sequence composition
@ for { a <- oldList; b <- newList } yield a+b
res22: List[Int] = List(2, 3, 4, 3, 4, 5)

@ // Pattern Matching
@ val x = 3
x: Int = 3
@ def ordinal(number: Int):  Unit = number match {
    case 0 => println("One")
    case 1 => println("Two")
    case 2 => println("Three")
    case 3 => println("Four")
    case _ => println("what is this?")
  }
defined function ordinal
@ ordinal(x)
Four

@ // Pattern Matching Types
@ dBef printType(obj: AnyRef) = obj match {
    case s: String => println("This is a string")
    case i: java.util.Date => println("This is a Date")
    case l: List[_] => println("An array eh?")
    case _ => println("dunno who ye are")
  }
defined function printType
@ printType(List(1,1,2))
An array eh?


// Guards
@ def numCatcher(num: Int): Unit = num match {
    case x if x < 10 => println("Number less than 10")
    case y if (y%2 == 0) => println("Is an even number")
  }
defined function numCatcher
@ numCatcher(5)
Number less than 10

@ numCatcher(12)
Is an even number

Miscellaneous notes

sbt plays an important role in the day to day programming of scala. I believe 'sbt' stood for "scala build tool" at one time, but is no longer.

To start working on a scala program, you can start with a simple text editor, however using something like IntelliJ IDEA is probably far more productive.

Creating a "project structure" can be automated. Essentially this boilBs down to creating the build.sbt file, project directory and optional plugins.sbt file inside it, a scala file inside the proper src/main/scala/com/example/projectname/ directory etc., There is little reason to do all this manually, except when you are trying to understand the reason behind such a recommended directory layout.

giter8 ie., the g8 is a tool that can create a project structure based on pre-defined templates. Think of this as paster tool for Python.

When you have g8 installed, creating a new project (say, a basic scala project using this template ), all you have to do is:

$ g8 fayimora/basic-scala-project.g8

A basic scala project

name [Basic Project]: basic-scala
organization [com.example]: com.btbytes
version [0.0.1]:

Template applied in ./basic-scala

$ cd basic-scala
$ ls
build-sbt src

$ sbt run will download the dependencies defined in the build.sbt file,B compile source and run the program.

sbt can also create the files necessary to make this an "IntelliJ IDEA project". For this, we add a dependency to the plugins.sbt file:

#project/plugins.sbt -- add this line
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.6.0")

Running sbt gen-idea will generate the IDEA project files (.idea, some .iml).

Launch up IDEA and "open project" and open the root directory of the project. You are now set to work on the code.

Packaging the binary to a .jar:

$ sbt packageBin

Will produce target/scala-2.10/basic-scala_2.10-0.0.1.jar (where basic-scala is the name of the project).

Why not Scala

Concepts

Covariance, Contravariance and Invariance

Reading:

In general, covariant type parameter can be used as immutable field type, method return type and also as method argument type if the method argument type has a lower bound. Because of those restrictions, covariance is most commonly used in producers (types that return something) and immutable types.

Adhoc polymorphism

In this article we discussed the motivation for introducing ad hoc polymorphism and implementing it using the Type Class Pattern in Scala. It allows us to build abstractions which are completely decoupled from the underlying data types on which they operate. Hence we can implement polymorphic functions which operate on types we have no control over, such as types defined in the standard library or some other third party library, without sacrificing the static type safety.

Concurrency (Akka etc.,)

StBreams

Libraries

Tools

  • lihaoyi/Ammonite: Scala Scripting. Ammonite enables shell-like scripting in the Scala programming language.
  • Conscript -- is a tool for installing and updating Scala software programs.
  • giter8 -- is a command line tool to generate files and directories from templates published on github or any other git repository.

SBT

Ensime -- Enhanced Scala

Interaction Mode for Emacs.

Talks

Articles

Blogs

Libraries

  • kantan.csv has good documentation/tutorials.
  • Quill Quoted Domain Specific Language (QDSL) to express queries in Scala and execute them in a target language. The library’s core is designed to support multiple target languages, currently featuring specializations for Structured Query Language (SQL) and Cassandra Query Language (CQL).

REPL

Ammonite REPL is much more featureful than the default REPL and comes with a lot of ergonomic improvements and configurability that may be familiar to people coming from IDEs or other REPLs such as IPython or Zsh.

Links

Scalajs

Scala Native

Scala CLI

With scala-cli, it is quite easy to package a scala program into a standalone program:

scala-cli --power package --native-image Echo.scala -o echo -- --no-fallback

where Echo.scala is:

object Echo {
def main(args: Array[String]): Unit =
	println(args.mkString(" "))
}
$ ls -alrt echo
-rwxr-xr-x@ 1 pradeep  staff  14120984 Aug 23 17:15 echo
$ du -sh echo
 13M    echo

Books

Tutorials

Indyscala notes

Feb 6, 2017 -- Resource management in Scala

Presenter -- Ross A. Baker

May 1, 2017

notes#jvm#programming-language#scala

Mercury Programming Language

Mercury language -- Home page

Mercury combines (predicate) logic programming and one of the more rigorous type systems to give a programming language that is declarative, efficient and consistent.

-- Good explanation of Mercury syntax by Markus Triska, a prolog programmer; Apr 2019.

notes#programming-language