1.1--- a/20230730.org Tue Aug 27 21:35:44 2024 -0400
1.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
1.3@@ -1,42 +0,0 @@
1.4-* VC infrastructure
1.5-:PROPERTIES:
1.6-:ID: 2d40eec1-62f4-4f3d-9be7-110400808e06
1.7-:END:
1.8-In heptapod we have a root group named =comp=, containg a variety of
1.9-subgroups. Some of these groups should be public, while others are
1.10-internal to comp members exclusively. Within each subgroup, we should
1.11-have the root group members automatically granted privileged access to
1.12-projects. This is relevant for the =startup= subgroup in particular,
1.13-where each project is potentially maintained by multiple non-root
1.14-contributors.
1.15-
1.16-We also need to consider how we will manage subrepos across the
1.17-organization. It is about time we start integrating HG bundles and
1.18-potentially mirrors. For our core VC pipeline we should have no
1.19-reliance on Git, but this may be difficult. It depends on the behavior
1.20-of HG bundles.
1.21-
1.22-Bookmarks/tags should be used for milestones in the root group and are
1.23-infrequent. They are more frequent in projects with a regular release
1.24-life-cycle.
1.25-* Approaching Webapps
1.26-:PROPERTIES:
1.27-:ID: 0a52be58-990a-47da-ba5a-a7da7192361c
1.28-:END:
1.29-I started poking around in the webapp space again so that I can launch
1.30-a landing page for NAS-T quickly. The Rust situation has improved
1.31-somewhat on the frontend side, and the axum backend stack is nice.
1.32-
1.33-This might seem like a lot of Rust and not a lot of Lisp, which it is,
1.34-but there's still room for Lisp wherever we need it. It mostly plays a
1.35-role in the backend, servicing the database and responding to requests
1.36-from the Rust edges. All of the important tests for the web APIs are
1.37-also written in Lisp. We will almost certainly use Lisp for all static
1.38-processing and HTML generation at compile-time.
1.39-
1.40-This I believe, is the appropriate way to integrate Lisp into a
1.41-cutting-edge web-app. You get the good parts of Lisp where you need
1.42-them (interactive debugging, dynamic language, REPL) and avoid the bad
1.43-parts (OOB optimization, RPS performance) in areas where the customer
1.44-would be impacted. In this domain, Lisp takes the form of a glue
1.45-rather than the bricks and mortar it sometimes appears to us as.
2.1--- a/20231024.org Tue Aug 27 21:35:44 2024 -0400
2.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
2.3@@ -1,128 +0,0 @@
2.4-* virt
2.5-:PROPERTIES:
2.6-:ID: a51b54d9-5663-411e-b0ba-d1405622eb88
2.7-:END:
2.8-** QEMU
2.9-:PROPERTIES:
2.10-:ID: 34dc9d12-d651-4204-9abb-a6a9a6aeb07e
2.11-:END:
2.12-** KVM
2.13-:PROPERTIES:
2.14-:ID: 909e1127-14f5-4783-97af-65aa25439226
2.15-:END:
2.16-** Hyper-V
2.17-:PROPERTIES:
2.18-:ID: d89dc59d-25d9-4304-bcb8-379dd43cc3f7
2.19-:END:
2.20-** Firecracker
2.21-:PROPERTIES:
2.22-:ID: 7c4f08b7-2bc0-4a85-a6ca-2decdce8dc91
2.23-:END:
2.24-** Docker
2.25-:PROPERTIES:
2.26-:ID: 42ff77c2-69c9-4f89-a297-88efd592502e
2.27-:END:
2.28-** Vagrant
2.29-:PROPERTIES:
2.30-:ID: c53c4d87-b0fd-40de-8db7-b92c22dc946c
2.31-:END:
2.32-** LXC
2.33-:PROPERTIES:
2.34-:ID: 85c5f471-cf87-49b7-a6ce-5c1526df4080
2.35-:END:
2.36-** LXD
2.37-:PROPERTIES:
2.38-:ID: f92d34dd-7b24-498c-a5c8-29fd3b528092
2.39-:END:
2.40-** containerd
2.41-:PROPERTIES:
2.42-:ID: a967645f-5df3-4bed-9e7b-a1adaed3796c
2.43-:END:
2.44-** systemd-nspawn
2.45-:PROPERTIES:
2.46-:ID: 6329c6f9-c54c-4eb0-99e7-4772399161e7
2.47-:END:
2.48-** VirtualBox
2.49-:PROPERTIES:
2.50-:ID: 638aa81a-8926-480a-854f-0434e8119426
2.51-:END:
2.52-
2.53-* Concatenative
2.54-:PROPERTIES:
2.55-:ID: 964c6bac-486a-4459-825f-6e7e9351c9ab
2.56-:END:
2.57-** Factor :factor:
2.58-:PROPERTIES:
2.59-:ID: 52994125-5689-402a-ac61-680aa690bf24
2.60-:END:
2.61-- [2023-07-04 Tue]
2.62- Factor is a cool concatenative lang but unfortunately the C interface
2.63- (vm/master.h) no longer exists on the master branch.
2.64-** Joy :joy:
2.65-:PROPERTIES:
2.66-:ID: 60ca3000-2ff5-4372-93fb-d5c311fb6409
2.67-:END:
2.68-
2.69-*** https://hypercubed.github.io/joy/html/j02maf.html
2.70-:PROPERTIES:
2.71-:ID: 56e64d52-4950-4fec-b933-73d1cd8048d1
2.72-:END:
2.73-
2.74-*** [[https://builds.openlogicproject.org/content/incompleteness/arithmetization-syntax/arithmetization-syntax.pdf][arithmetization of syntax]]
2.75-:PROPERTIES:
2.76-:ID: 59979091-395d-4067-9c3d-e557fc5287ee
2.77-:END:
2.78-* Lisp :lisp:
2.79-:PROPERTIES:
2.80-:ID: 4b842734-135e-4c86-9337-3841c1241d3b
2.81-:END:
2.82-These notes pertain to Lisp. More specifically, ANSI Common Lisp in
2.83-most places.
2.84-
2.85-- https://github.com/lispnik/iup/ - doesn't support MacOS yet, looks
2.86- cool though
2.87- - what we really need is wasm compiler.. TBD
2.88-* Rust
2.89-:PROPERTIES:
2.90-:ID: f92ce2ba-1b66-42f4-b5e2-f1586b14760b
2.91-:END:
2.92-** Serde
2.93-:PROPERTIES:
2.94-:ID: 31512714-9c30-4144-9673-327808a18767
2.95-:END:
2.96-- [2023-07-05 Wed] \\
2.97- important part of the Rust ecosystem, another dtolnay
2.98- contribution. If you want to program a /data/ format in the Rust
2.99- ecosystem, this is how you do it.
2.100-
2.101- The way it works is that you define some special structs, a
2.102- Serializer and a Deserializer which implement the Serialize and
2.103- Deserialize traits provided by serde, respectively.
2.104-
2.105- You can use these structs to provide your public API. The
2.106- conventional choice is public top-level functions like from-str
2.107- and to-string. That's it, your serialization library can now read and
2.108- write your data format as Rust data types.
2.109-
2.110- [[https://serde.rs/enum-representations.html][enum-representations]]
2.111- - the default behavior is an externally tagged representation (verbose)
2.112-
2.113- The docs use strings as core IO when implementing a custom format,
2.114- but the convention is to implement for T where T is bound by std::io
2.115- Read or Write trait. Then you can provide a more robust public API
2.116- (from_bytes, from_writer, etc).
2.117-* C
2.118-:PROPERTIES:
2.119-:ID: 060042a0-d581-411a-9091-d16d11860d89
2.120-:END:
2.121-* CPP
2.122-:PROPERTIES:
2.123-:ID: f09d7294-a997-4715-bceb-ffe6af7f3978
2.124-:END:
2.125-* Nu
2.126-:PROPERTIES:
2.127-:ID: 765ad403-54e1-49cf-b4e6-918de9f910c2
2.128-:END:
2.129-[[https://www.nushell.sh/][~]]
2.130-[[https://www.nushell.sh/cookbook/][cookbook]]
2.131-[[https://github.com/nushell/nu_scripts][nu_scripts]]
3.1--- a/20231101.org Tue Aug 27 21:35:44 2024 -0400
3.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
3.3@@ -1,21 +0,0 @@
3.4-* AWS usage
3.5-:PROPERTIES:
3.6-:ID: b2f8d58f-a486-4fe9-895b-504987c0f63d
3.7-:END:
3.8-We're leveraging AWS for some of our public web servers for now. It's
3.9-really not realistic to expect that my home desktop and spotty Comcast
3.10-internet can serve any production workflow. What it /is/ capable of is
3.11-a private VPN, which can communicate with AWS and other cloud VPN
3.12-depots via WireGuard ([[https://dev.to/gabrieltetzner/setting-up-a-vpn-with-wireguard-server-on-aws-ec2-4a49][article]]).
3.13-
3.14-I currently use Google Domains for nas-t.net, otom8.dev, and
3.15-rwest.io - but that business is now owned by squarespace, so I would
3.16-rather move it to Route53.
3.17-
3.18-We have archlinux ec2 image builds [[https://wiki.archlinux.org/title/Arch_Linux_AMIs_for_Amazon_Web_Services][here]] and [[https://gitlab.com/anemos-io/archlinux-ec2][here]] - only half work and not
3.19-maintained, but it's a start. I'm not even sure if I should stick with
3.20-arch or cave and use Ubuntu or AWS Linux. We can serve the static
3.21-services with little cost, the only big spender will be the heptapod
3.22-instance which requires a larger instance and some workers.
3.23-
3.24-We'll try to keep the cost at or around $30/month.
4.1--- a/20231102.org Tue Aug 27 21:35:44 2024 -0400
4.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
4.3@@ -1,139 +0,0 @@
4.4-* IDEAS
4.5-:PROPERTIES:
4.6-:ID: 836b5ca7-6c6b-4d57-8b44-3bb4a8f38a38
4.7-:END:
4.8-** shed
4.9-:PROPERTIES:
4.10-:ID: fc9a94e1-91c5-4915-90b8-73218fa3b8bc
4.11-:END:
4.12-:LOGBOOK:
4.13-- State "TODO" from [2023-04-07 Fri 23:24]
4.14-:END:
4.15-rlib
4.16-> ulib
4.17-> ulib
4.18-> ulib
4.19-> ulib
4.20-
4.21-*** sh* tools
4.22-:PROPERTIES:
4.23-:ID: c0613a13-7ccb-4af9-b47e-e14a41c782c2
4.24-:END:
4.25-:LOGBOOK:
4.26-- State "TODO" from "TODO" [2023-04-07 Fri 23:22]
4.27-:END:
4.28-shc,shx,etc
4.29-** packy
4.30-:PROPERTIES:
4.31-:ID: 6bd29c37-4b23-4a05-b3a4-72b6ee712f27
4.32-:END:
4.33-:LOGBOOK:
4.34-- State "TODO" from [2023-04-07 Fri 23:33]
4.35-:END:
4.36-*** rust
4.37-:PROPERTIES:
4.38-:ID: 3bfa733e-d777-4e82-b108-93ba74c4dc08
4.39-:END:
4.40-*** common-lisp
4.41-:PROPERTIES:
4.42-:ID: 0f3b6182-1dd1-4e65-8b1c-4630a46fff4c
4.43-:END:
4.44-*** emacs-lisp
4.45-:PROPERTIES:
4.46-:ID: e9fc19d5-daea-40a5-a0f2-882b925e58f2
4.47-:END:
4.48-*** python
4.49-:PROPERTIES:
4.50-:ID: c2e62af0-0397-4560-a283-b33d12fbd966
4.51-:END:
4.52-*** julia
4.53-:PROPERTIES:
4.54-:ID: d2466880-1721-4386-b385-ded06d5b15cf
4.55-:END:
4.56-*** C
4.57-:PROPERTIES:
4.58-:ID: fb722462-9e11-40c1-a5d7-30698dba7e20
4.59-:END:
4.60-*** C++
4.61-:PROPERTIES:
4.62-:ID: 663cd894-0a5f-4352-acf1-4528b9efeeaf
4.63-:END:
4.64-** tenex
4.65-:PROPERTIES:
4.66-:ID: b4264fa2-6e96-4679-bd17-2428ba3489ee
4.67-:END:
4.68-:LOGBOOK:
4.69-- State "TODO" from [2023-04-07 Fri 23:52]
4.70-:END:
4.71-** mpk
4.72-:PROPERTIES:
4.73-:ID: d9661e4a-0328-4864-878d-096f76ddaf55
4.74-:END:
4.75-:LOGBOOK:
4.76-- State "TODO" from [2023-04-07 Fri 23:52]
4.77-:END:
4.78-** cfg
4.79-:PROPERTIES:
4.80-:ID: 501593d4-702a-40cb-8a8f-19edfe2f9f29
4.81-:END:
4.82-:LOGBOOK:
4.83-- State "TODO" from [2023-04-07 Fri 23:34]
4.84-:END:
4.85-** obj
4.86-:PROPERTIES:
4.87-:ID: 24700a26-b238-48dd-a24f-5539984b4fa6
4.88-:END:
4.89-:LOGBOOK:
4.90-- State "TODO" from [2023-04-07 Fri 23:51]
4.91-:END:
4.92-split out from rlib to separate package
4.93-- a purely OOP class library
4.94-** lab
4.95-:PROPERTIES:
4.96-:ID: fa1c9563-338b-4a56-bfc3-6f4983f2a732
4.97-:END:
4.98-:LOGBOOK:
4.99-- State "TODO" from [2023-04-07 Fri 23:34]
4.100-:END:
4.101-** source categories
4.102-:PROPERTIES:
4.103-:ID: c43fdb5d-4f52-478e-9384-ec0e6e89cf09
4.104-:END:
4.105-- need a way of extracting metadata from a repo
4.106-- need ability to search and query libs/packages
4.107-- separate modules based on where they belong in our stack?
4.108- - app
4.109- - lib
4.110- - script?
4.111- - dist
4.112- - software distros
4.113-** generic query language
4.114-:PROPERTIES:
4.115-:ID: 8647801a-d5ad-446d-8ae1-fd896bce129a
4.116-:END:
4.117-from obj protocol?
4.118-sql compatibility?
4.119-
4.120-/check out kdb/
4.121-** bbdb
4.122-:PROPERTIES:
4.123-:ID: ff7c2876-e471-480b-b532-9921dbd3bc89
4.124-:END:
4.125-:LOGBOOK:
4.126-- Note taken on [2023-10-24 Tue 22:16] \\
4.127- graph database, build on rocksdb
4.128-:END:
4.129-insidious Big Brother database.
4.130-- an application built with obj
4.131-- sql
4.132-
4.133-** NAS-TV :nas:t:
4.134-:PROPERTIES:
4.135-:ID: 00e5fe6c-80b4-4172-a9cb-ce8adc40d3dc
4.136-:END:
4.137-:LOGBOOK:
4.138-- State "NOTE" from "TODO" [2024-08-18 Sun 18:46]
4.139-:END:
4.140-- media streaming
4.141-- gstreamer backend
4.142-- audio/video
5.1--- a/20231105.org Tue Aug 27 21:35:44 2024 -0400
5.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
5.3@@ -1,187 +0,0 @@
5.4-* DRAFT dylib-skel-1
5.5-:PROPERTIES:
5.6-:ID: b4d1bc91-f344-45fd-becc-cb20f00a3a61
5.7-:END:
5.8-- State "DRAFT" from [2023-11-05 Sun 22:23]
5.9-** Overview
5.10-:PROPERTIES:
5.11-:ID: 2e490c4b-344e-4790-9184-1c05ba675f15
5.12-:END:
5.13-Our core languages are [[https://www.rust-lang.org/][Rust]] and [[https://lisp-lang.org/][Lisp]] - this is the killer combo which will allow NAS-T
5.14-to rapidly develop high-quality software. As such, it's crucial that these two very
5.15-different languages (i.e. compilers) are able to interoperate seamlessly.
5.16-
5.17-Some interop methods are easy to accomodate via the OS - such as IPC or data sharing,
5.18-but others are a bit more difficult.
5.19-
5.20-In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something
5.21-that /can/ be difficult, due to some complications with Rust and because this is not the
5.22-most popular software stack (yet ;). This is an experiment and may not make it to our
5.23-code-base, but it's definitely something worth adding to the toolbox in case we need it.
5.24-
5.25-** FFI
5.26-:PROPERTIES:
5.27-:ID: 985019fc-612a-44ab-b726-b9067432ad87
5.28-:END:
5.29-The level of interop we're after in this case is [[https://en.wikipedia.org/wiki/Foreign_function_interface][FFI]].
5.30-
5.31-Basically, calling Rust code from Lisp and vice-versa. There's an article about calling
5.32-Rust from Common Lisp [[https://dev.to/veer66/calling-rust-from-common-lisp-45c5][here]] which shows the basics and serves as a great starting point
5.33-for those interested.
5.34-*** Rust != C
5.35-:PROPERTIES:
5.36-:ID: 2f71a3c1-0b14-46a6-9d8d-f6ec697729cc
5.37-:END:
5.38-The complication(s) with Rust I mentioned early is really just that /it is not C/. =C=
5.39-is old, i.e. well-supported with a stable ABI, making the process of creating bindings
5.40-for a C library a breeze in many languages.
5.41-
5.42-For a Rust library we need to first appease the compiler, as explained in [[https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c][this section]]
5.43-of the Rustonomicon. Among other things it involves changing the calling-convention of
5.44-functions with a type signature and editing the Cargo.toml file to produce a
5.45-C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used
5.46-like the C ABI can.
5.47-
5.48-*** Overhead
5.49-:PROPERTIES:
5.50-:ID: 4ea79f68-55ec-4da3-a184-8343d49532b6
5.51-:END:
5.52-Using FFI involves some overhead. Check [[https://github.com/dyu/ffi-overhead][here]] for an example benchmark across a few
5.53-languages. While building the NAS-T core, I'm very much aware of this, and will need a
5.54-few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular,
5.55-I'm concerned about crossing multiple language barriers (Rust<->C<->Lisp).
5.56-
5.57-** Rust -> C -> Lisp
5.58-:PROPERTIES:
5.59-:ID: a498276c-8525-4a43-aa40-4b05f76a29a9
5.60-:END:
5.61-*** Setup
5.62-:PROPERTIES:
5.63-:ID: 19f96ef7-af92-496e-9d42-70c4d4c85051
5.64-:END:
5.65-For starters, I'm going to assume we all have Rust (via =rustup=) and Lisp (=sbcl= only)
5.66-installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
5.67-this post).
5.68-**** Cargo
5.69-:PROPERTIES:
5.70-:ID: c929e0b6-b6f2-4383-9412-1610329ab28c
5.71-:END:
5.72-Create a new library crate. For this example we're focusing on a 'skeleton' for
5.73-/dynamic/ libraries only, so our experiment will be called =dylib-skel= or *dysk* for
5.74-short.
5.75-src_sh[:exports code]{cargo init dysk --lib && cd dysk}
5.76-
5.77-A =src/lib.rs= will be generated for you. Go ahead and delete that. We're going to be
5.78-making our own =lib.rs= file directly in the root directory (just to be cool).
5.79-
5.80-The next step is to edit your =Cargo.toml= file. Add these lines after the =[package]=
5.81-section and before =[dependencies]=:
5.82-#+begin_src conf-toml
5.83-[lib]
5.84-crate-type = ["cdylib","rlib"]
5.85-path = "lib.rs"
5.86-[[bin]]
5.87-name="dysk-test"
5.88-path="test.rs"
5.89-#+end_src
5.90-
5.91-This tells Rust to generate a shared C-compatible object with a =.so= extension which we
5.92-can open using [[https://man.archlinux.org/man/dlopen.3.en][dlopen]].
5.93-**** cbindgen
5.94-:PROPERTIES:
5.95-:ID: 256ac288-c5a0-473a-ab65-2d6503bd423c
5.96-:END:
5.97-***** install
5.98-:PROPERTIES:
5.99-:ID: fc476f64-6b68-417a-8540-ca23ce27fa25
5.100-:END:
5.101-Next, we want the =cbindgen= program which we'll use to generate header files for
5.102-C/C++. This step isn't necessary at all, we just want it for further experimentation.
5.103-
5.104-src_sh[:exports code]{cargo install --force cbindgen}
5.105-
5.106-We append the =cbindgen= crate as a /build dependency/ to our =Cargo.toml= like so:
5.107-#+begin_src conf-toml
5.108-[build-dependencies]
5.109-cbindgen = "0.24"
5.110-#+end_src
5.111-***** cbindgen.toml
5.112-:PROPERTIES:
5.113-:ID: 111e27f7-0b9c-4eef-9117-f7c8ba3f511c
5.114-:END:
5.115-#+begin_src conf-toml :tangle cbindgen.toml
5.116-language = "C"
5.117-autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
5.118-include_version = true
5.119-namespace = "dysk"
5.120-cpp_compat = true
5.121-after_includes = "#define DYSK_VERSION \"0.1.0\""
5.122-line_length = 88
5.123-tab_width = 2
5.124-documentation = true
5.125-documentation_style = "c99"
5.126-usize_is_size_t = true
5.127-[cython]
5.128-header = '"dysk.h"'
5.129-#+end_src
5.130-***** build.rs
5.131-:PROPERTIES:
5.132-:ID: 9fc271b2-9acb-4f4b-aa61-82d60d2ddb9e
5.133-:END:
5.134-#+begin_src rust :tangle build.rs
5.135-fn main() -> Result<(), cbindgen::Error> {
5.136- if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
5.137- b.write_to_file("dysk.h"); Ok(())}
5.138- else { panic!("failed to generate dysk.h from cbindgen.toml") } }
5.139-#+end_src
5.140-*** lib.rs
5.141-:PROPERTIES:
5.142-:ID: 6b524921-2ae0-43f0-bb85-d9955b0e689c
5.143-:END:
5.144-#+begin_src rust :tangle lib.rs
5.145-//! lib.rs --- dysk library
5.146-use std::ffi::{c_char, c_int, CString};
5.147-#[no_mangle]
5.148-pub extern "C" fn dysk_hello() -> *const c_char {
5.149- CString::new("hello from rust").unwrap().into_raw()}
5.150-#[no_mangle]
5.151-pub extern "C" fn dysk_plus(a:c_int,b:c_int) -> c_int {a+b}
5.152-#[no_mangle]
5.153-pub extern "C" fn dysk_plus1(n:c_int) -> c_int {n+1}
5.154-#+end_src
5.155-*** test.rs
5.156-:PROPERTIES:
5.157-:ID: cc7c6538-33a6-40c6-94ef-2a9c259c975a
5.158-:END:
5.159-#+begin_src rust :tangle test.rs
5.160-//! test.rs --- dysk test
5.161-fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::dysk_plus1(2 as core::ffi::c_int);}}
5.162-#+end_src
5.163-*** compile
5.164-:PROPERTIES:
5.165-:ID: 337a24d1-f305-4e1a-9052-47a53591cb2f
5.166-:END:
5.167-#+begin_src sh
5.168-cargo build --release
5.169-#+end_src
5.170-*** load from SBCL
5.171-:PROPERTIES:
5.172-:ID: a4813269-92fb-4f52-aef0-3a36dce3cf69
5.173-:END:
5.174-#+begin_src lisp :tangle dysk.lisp
5.175-(load-shared-object #P"target/release/libdysk.so")
5.176-(define-alien-routine dysk-hello c-string)
5.177-(define-alien-routine dysk-plus int (a int) (b int))
5.178-(define-alien-routine dysk-plus1 int (n int))
5.179-(dysk-hello) ;; => "hello from rust"
5.180-#+end_src
5.181-*** benchmark
5.182-:PROPERTIES:
5.183-:ID: 1a8ca441-f290-46c7-b979-1e7e0d1d063b
5.184-:END:
5.185-#+begin_src shell
5.186-time target/release/dysk-test
5.187-#+end_src
5.188-#+begin_src lisp :tangle test.lisp
5.189-(time (dotimes (_ 500000000) (dysk-plus1 2)))
5.190-#+end_src
6.1--- a/20231124.org Tue Aug 27 21:35:44 2024 -0400
6.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
6.3@@ -1,36 +0,0 @@
6.4-* cl-dot examples
6.5-:PROPERTIES:
6.6-:ID: c5f4f25c-071b-4a2d-85bc-08676eacce5d
6.7-:END:
6.8-#+begin_src lisp
6.9-(defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object cons))
6.10- (make-instance 'cl-dot:node
6.11- :attributes '(:label "cell \\N"
6.12- :shape :box)))
6.13-(defmethod cl-dot:graph-object-points-to ((graph (eql 'example)) (object cons))
6.14- (list (car object)
6.15- (make-instance 'cl-dot:attributed
6.16- :object (cdr object)
6.17- :attributes '(:weight 3))))
6.18-;; Symbols
6.19-(defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object symbol))
6.20- (make-instance 'cl-dot:node
6.21- :attributes `(:label ,object
6.22- :shape :hexagon
6.23- :style :filled
6.24- :color :black
6.25- :fillcolor "#ccccff")))
6.26-(let* ((data '(a b c #1=(b z) c d #1#))
6.27- (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
6.28- '(:rankdir "LR" :layout "twopi" :labelloc "t"))))
6.29- (cl-dot:dot-graph dgraph "test-lr.svg" :format #+nil :x11 :svg))
6.30-#+end_src
6.31-
6.32-#+RESULTS:
6.33-
6.34-#+begin_src lisp
6.35-(let* ((data '(a b))
6.36- (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
6.37- '(:rankdir "LR"))))
6.38- (cl-dot:print-graph dgraph))
6.39-#+end_src
7.1--- a/20231205.org Tue Aug 27 21:35:44 2024 -0400
7.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
7.3@@ -1,8 +0,0 @@
7.4-* global refs
7.5-:PROPERTIES:
7.6-:ID: 7fb40057-05aa-445b-bc75-0b1b6ca2c994
7.7-:END:
7.8-need a way of indexing, referring to, and annotating objects such as
7.9-URLs, docs, articles, source files, etc.
7.10-
7.11-What is the best way to get this done?
8.1--- a/20231209.org Tue Aug 27 21:35:44 2024 -0400
8.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
8.3@@ -1,7 +0,0 @@
8.4-* doc best practices
8.5-:PROPERTIES:
8.6-:ID: ba4a3074-f29d-4f81-b4ba-d6a523b3f291
8.7-:END:
8.8-https://rust-lang.github.io/api-guidelines/documentation.html
8.9-
8.10-also: https://lisp-lang.org/style-guide/
9.1--- a/20231212.org Tue Aug 27 21:35:44 2024 -0400
9.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
9.3@@ -1,101 +0,0 @@
9.4-* On Computers
9.5-:PROPERTIES:
9.6-:ID: 7845b0fb-b5d3-4488-bdba-4876d9c8816e
9.7-:END:
9.8-If you've met me in the past decade, you probably know that I am
9.9-extremely passionate about computers. Let me first explain why.
9.10-
9.11-On the most basic level computers are little (or big) machines that
9.12-can be programmed to do things, or /compute/ if we're being
9.13-technical.[fn:1]
9.14-
9.15-They host and provide access to the Internet, which is a pretty big
9.16-thing, but they do little things too like unlock your car door and
9.17-tell your microwave to beep at you. They solve problems. Big or small.
9.18-
9.19-They're also /everywhere/ - which can be scary to think about, but
9.20-ultimately helps propel us into the future.
9.21-
9.22-There's something pretty cool about that - when you look at the
9.23-essence of computation. There are endless quantities of these machines
9.24-which follow the same basic rules and can be used to solve /real/
9.25-problems.
9.26-
9.27-** The Programmer
9.28-:PROPERTIES:
9.29-:ID: b17c853e-1728-4bd3-94c5-82e5b33d7398
9.30-:END:
9.31-Now, let us consider the /programmer/. They have power. /real/
9.32-power. They understand the language of computers, can whisper to them
9.33-in various dialects. It can be intimidating to witness until you
9.34-realize how often the programmer says the wrong thing - a bug.
9.35-
9.36-In reality, the programmer has a symbiotic relationship with
9.37-computers. Good programmers understand this relationship well.
9.38-
9.39-#+begin_annecdote
9.40-One day after I got my first job at a software company, I remember
9.41-being on an all-hands meeting due to a client service outage. We had
9.42-some management, our lead devs, product team, and one curious looking
9.43-man who happened to be our lead IT consultant who had just joined. He
9.44-was sitting up on a hotel bed, shirtless, vaping an e-cig, typing
9.45-away in what I can only imagine was a shell prompt.
9.46-
9.47-After several minutes he took a swig from a bottle of Coke and said
9.48-"Node 6 is sick." then a few seconds later our services were
9.49-restored. For the next hour on the call he explained what happened and
9.50-why, but that particular phrase always stuck with me. He didn't say
9.51-Node 6 was down, or had an expired cert - his diagnosis was that /it/
9.52-was /sick/.
9.53-#+end_annecdote
9.54-
9.55-The more you work closely with computers, the more you start to think
9.56-of them this way. You don't start screaming when the computer does the
9.57-wrong thing, you figure out what's wrong and learn from it. With
9.58-experience, you start to understand the different behaviors of the
9.59-machines you work with. I like to call this /Machine Empathy/.
9.60-
9.61-** Programs
9.62-:PROPERTIES:
9.63-:ID: a297e5f9-a875-4512-b126-9a2b3e75c1d8
9.64-:END:
9.65-I already mentioned bugs - I write plenty of those, but usually I try
9.66-to write /programs/. Programs to me are like poetry. I like to think
9.67-they are for the computer too.
9.68-
9.69-Just like computers, /computer programs/ come in different shapes and
9.70-sizes but in basic terms they are sets of instructions used to control
9.71-a computer.
9.72-
9.73-You can write programs to do anything - when I first started, my
9.74-programs made music. The program was a means to an end. Over time, I
9.75-started to see the program as something much more. I saw it as the
9.76-music itself.
9.77-
9.78-[fn:1] ... perform computations
9.79-
9.80-
9.81-* On Infra
9.82-:PROPERTIES:
9.83-:ID: 80d800de-d305-4b95-a28c-a6f19d6f7a11
9.84-:END:
9.85-Something that is missing from many organizations big or large, is an
9.86-effective way to store and access information, even about their own
9.87-org.
9.88-
9.89-It can be difficult problem to solve - usually there's the official
9.90-one, say Microsoft Sharepoint and then the list of unofficial sources
9.91-which becomes tribal corporate hacker knowledge. Maybe the unofficial
9.92-ones are more current, or are annotated nicely, but their very
9.93-existence breaks the system. There's no longer a single source of
9.94-truth.
9.95-
9.96-My priority in this department is writing services which process and
9.97-store information from a variety of sources in a distributed knowledge
9.98-graph. The graph can later be queried to access information on-demand.
9.99-
9.100-My idea of infrastructure is in fact to build my own Cloud. Needless
9.101-to say I don't have an O365 subscription, and wherever possible I'll
9.102-be relying on hardware I have physical access to. I'm not opposed to
9.103-cloud services at large but based on principle I like to think we
9.104-shouldn't be built on them.
10.1--- a/20231223.org Tue Aug 27 21:35:44 2024 -0400
10.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
10.3@@ -1,6 +0,0 @@
10.4-* https://cal-coop.gitlab.io/utena/utena-specification/main.pdf
10.5-:PROPERTIES:
10.6-:ID: 5c3aac3a-67e1-4867-9ac4-b58a3247f9e0
10.7-:END:
10.8-from the author of cl-decentralise2. draft specification of a
10.9-/Maximalist/ Computing System.
11.1--- a/20231224.org Tue Aug 27 21:35:44 2024 -0400
11.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
11.3@@ -1,7 +0,0 @@
11.4-* public datasets
11.5-:PROPERTIES:
11.6-:ID: 28738250-c34e-45e2-8793-f8245472d3a3
11.7-:END:
11.8-https://github.com/awesomedata/awesome-public-datasets
11.9-https://docs.openml.org/Datasets/
11.10-https://wiki.pathmind.com/open-datasets
12.1--- a/20231228.org Tue Aug 27 21:35:44 2024 -0400
12.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
12.3@@ -1,43 +0,0 @@
12.4-* useful internals
12.5-:PROPERTIES:
12.6-:ID: 28581a8e-ae05-45de-9744-24f447668418
12.7-:END:
12.8-#+begin_src lisp
12.9- sb-sys:*runtime-dlhandle*
12.10- sb-fasl:+fasl-file-version+
12.11- sb-fasl:+backend-fasl-file-implementation+
12.12- sb-debug:print-backtrace
12.13- sb-debug:map-backtrace
12.14- sb-pretty:pprint-dispatch-table
12.15- sb-lockless:
12.16- sb-ext:simd-pack
12.17- sb-walker:define-walker-template
12.18- sb-walker:macroexpand-all
12.19- sb-walker:walk-form
12.20- sb-kernel:empty-type
12.21- sb-kernel:*eval-calls*
12.22- sb-kernel:*gc-pin-code-pages*
12.23- sb-kernel:*restart-clusters*
12.24- sb-kernel:*save-lisp-clobbered-globals*
12.25- sb-kernel:*top-level-form-p*
12.26- sb-kernel:*universal-fun-type*
12.27- sb-kernel:*universal-type*
12.28- sb-kernel:*wild-type*
12.29- sb-kernel:+simd-pack-element-types+
12.30- (sb-vm:memory-usage)
12.31- (sb-vm:boxed-context-register)
12.32- (sb-vm:c-find-heap->arena)
12.33- (sb-vm:copy-number-to-heap)
12.34- (sb-vm:dump-arena-objects)
12.35- (sb-vm:fixnumize)
12.36- (sb-vm:rewind-arena)
12.37- (sb-vm:show-heap->arena)
12.38- (sb-vm:with/without-arena)
12.39- (sb-cltl2:{augment-environment,compiler-let,define-declaration,parse-macro})
12.40- (sb-cltl2:{declaration-information, variable-information, function-information})
12.41- sb-di:
12.42- sb-assem:
12.43- sb-md5:
12.44- sb-regalloc:
12.45- sb-disassem:
12.46-#+end_src
13.1--- a/20240103.org Tue Aug 27 21:35:44 2024 -0400
13.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
13.3@@ -1,39 +0,0 @@
13.4-* [[https://github.com/sigmf/SigMF][SigMF]]
13.5-:PROPERTIES:
13.6-:ID: f21a6477-6a3f-48f7-8a9e-43f5cf65286c
13.7-:END:
13.8-#+begin_quote
13.9-Sharing sets of recorded signal data is an important part of science
13.10-and engineering. It enables multiple parties to collaborate, is often
13.11-a necessary part of reproducing scientific results (a requirement of
13.12-scientific rigor), and enables sharing data with those who do not have
13.13-direct access to the equipment required to capture it.
13.14-
13.15-Unfortunately, these datasets have historically not been very
13.16-portable, and there is not an agreed upon method of sharing metadata
13.17-descriptions of the recorded data itself. This is the problem that
13.18-SigMF solves.
13.19-
13.20-By providing a standard way to describe data recordings, SigMF
13.21-facilitates the sharing of data, prevents the "bitrot" of datasets
13.22-wherein details of the capture are lost over time, and makes it
13.23-possible for different tools to operate on the same dataset, thus
13.24-enabling data portability between tools and workflows.
13.25-#+end_quote
13.26-
13.27-the-spec: https://github.com/sigmf/SigMF/blob/sigmf-v1.x/sigmf-spec.md
13.28-* [[https://www.libvolk.org/][LibVOLK]]
13.29-:PROPERTIES:
13.30-:ID: ae34135c-488e-4aff-b795-a4d0d558f22b
13.31-:END:
13.32-Vector-Optimized Library of Kernels (simd)
13.33-* [[https://docs.kernel.org/fb/framebuffer.html][/dev/fb*]]
13.34-:PROPERTIES:
13.35-:ID: 7819a970-1f37-40a6-985f-905b25a08a4a
13.36-:END:
13.37-framebuffers, used by fbgrab/fbcat program
13.38-* [[https://docs.kernel.org/block/ublk.html][ublk]]
13.39-:PROPERTIES:
13.40-:ID: 5ccaed39-7055-4a51-ab62-769a73425270
13.41-:END:
13.42-https://github.com/ming1/ubdsrv
14.1--- a/20240104.org Tue Aug 27 21:35:44 2024 -0400
14.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
14.3@@ -1,6 +0,0 @@
14.4-goals:
14.5-make problems smaller.
14.6-
14.7-sections:
14.8-why lisp?
14.9-- doesn't need mentioning more and more
15.1--- a/20240120.org Tue Aug 27 21:35:44 2024 -0400
15.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
15.3@@ -1,60 +0,0 @@
15.4-* TODO taobench demo
15.5-:PROPERTIES:
15.6-:ID: 81ac81c3-f183-4345-aaf3-ef7ef0434947
15.7-:END:
15.8-:LOGBOOK:
15.9-- State "TODO" from [2024-01-21 Sun 00:32]
15.10-:END:
15.11-https://github.com/audreyccheng/taobench - shouldn't have missed this :)
15.12-obviously we need to implement this using core -- in demo/bench/tao?
15.13-* TODO clap completion for nushell
15.14-:PROPERTIES:
15.15-:ID: a4b35281-83f7-4244-93e9-20e57ba31eb3
15.16-:END:
15.17-:LOGBOOK:
15.18-- State "TODO" from [2024-01-20 Sat 23:23]
15.19-:END:
15.20-https://github.com/clap-rs/clap/tree/master/clap_complete_nushell
15.21-* Dataframe scripting
15.22-:PROPERTIES:
15.23-:ID: 8dc7339e-71c6-4a60-b285-c0d56d1cbc24
15.24-:END:
15.25-https://studioterabyte.nl/en/blog/polars-vs-pandas
15.26-nushell supports DFs, polars underneath?
15.27-https://www.nushell.sh/book/cheat_sheet.html
15.28-
15.29-#+begin_src nushell
15.30-
15.31-#+end_src
15.32-* Cloud Squatting
15.33-:PROPERTIES:
15.34-:ID: d5131c54-9082-4424-91dd-70f17561bbf7
15.35-:END:
15.36-** Google
15.37-:PROPERTIES:
15.38-:ID: 2b0f67de-1217-48a6-b5b7-a85d3ca15926
15.39-:END:
15.40-- [[https://cloud.google.com/free/docs/free-cloud-features][Free Cloud Features]]
15.41- + 90-day $300 credits
15.42- + e2-micro - free hours worth 1 instance/month
15.43- +
15.44-** Amazon
15.45-:PROPERTIES:
15.46-:ID: 0fd0dedd-59b0-4b1a-9241-f65699e20c46
15.47-:END:
15.48-- AWS Free Tier
15.49-** Akamai
15.50-:PROPERTIES:
15.51-:ID: 8315a33f-dd22-4374-979a-9cccd5b5b6a9
15.52-:END:
15.53-- Linode Free Trial
15.54-** Oracle
15.55-:PROPERTIES:
15.56-:ID: 3f1e1116-c123-4fdf-a336-0d9b50c3dcc7
15.57-:END:
15.58-- [[https://www.oracle.com/cloud/free/?intcmp=ohp052322ocift][OCI Free Tier]]
15.59- + always free: 2 x oracle autonomous DB
15.60- + 2 x AMD Compute VMs
15.61- + up to 4 x ARM Ampere A1 with 3k/cpu/hr and 18k/gb/h per month
15.62- + block/object/archive storage
15.63- + 30-day $300 credits
16.1--- a/20240129.org Tue Aug 27 21:35:44 2024 -0400
16.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
16.3@@ -1,73 +0,0 @@
16.4-* trash as block device
16.5-:PROPERTIES:
16.6-:ID: 4452b84f-f3a5-44f1-b60a-4efa6413cbb9
16.7-:END:
16.8-:LOGBOOK:
16.9-- State "NOTE" from [2024-01-29 Mon 20:53]
16.10-- State "NOTE" from [2024-01-29 Mon 20:53]
16.11-:END:
16.12-in nushell there is option for rm command to always use 'trash' -
16.13-AFAIK the current approach is via a service (trashd).
16.14-
16.15-An interesting experiment would be to designate a block device as
16.16-'trash' - may be possible to remove reliance on a service
16.17-
16.18-may be an opportunity for ublk driver to shine - instead of /dev/null
16.19-piping we need a driver for streaming a file to /dev/trash
16.20-* compute power
16.21-:PROPERTIES:
16.22-:ID: aa10a1d7-f1e7-4b92-b91a-56c234d52204
16.23-:END:
16.24-:LOGBOOK:
16.25-- State "NOTE" from [2024-01-29 Mon 16:28]
16.26-:END:
16.27-- mostly x86_64 machines - currently 2 AWS EC2 instances, some podman containers, and our home beowulf server:
16.28-- beowulf:
16.29- - Zor
16.30- - mid-size tower enclosed (Linux/Windows)
16.31- - CPU
16.32- - Intel Core i7-6700K
16.33- - 4 @ 4.0
16.34- - GPU
16.35- - NVIDIA GeForce GTX 1060
16.36- - 6GB
16.37- - Storage
16.38- - Samsung SSD 850: 232.9GB
16.39- - Samsung SSD 850: 465.76GB
16.40- - ST2000DM001-1ER1: 1.82TB
16.41- - WDC WD80EAZZ-00B: 7.28TB
16.42- - PSSD T7 Shield: 3.64TB
16.43- - My Passport 0820: 1.36TB
16.44- - RAM
16.45- - 16GB (2*8) [64GB max]
16.46- - DDR4
16.47- - Jekyll
16.48- - MacBook Pro 2019 (MacOS/Darwin)
16.49- - CPU
16.50- - Intel
16.51- - 8 @
16.52- - RAM
16.53- - 32G DDR4
16.54- - Hyde
16.55- - Thinkpad
16.56- - CPU
16.57- - Intel
16.58- - 4 @
16.59- - RAM
16.60- - 24G DDR3
16.61- - Boris
16.62- - Pinephone Pro
16.63- - CPU
16.64- - 64-bit 6-core 4x ARM Cortex A53 + 2x ARM Cortex A72
16.65- - GPU
16.66- - Mali T860MP4
16.67- - RAM
16.68- - 4GB LPDDR4
16.69- - pi
16.70- - Raspberry Pi 4 Model B
16.71- - CPU
16.72- - Cortex-A72 (ARM v8) 64-bit SoC
16.73- - 4 @ 1.8GHz
16.74- - RAM
16.75- - 8 GB
16.76- - DDR4 4200
17.1--- a/20240210.org Tue Aug 27 21:35:44 2024 -0400
17.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
17.3@@ -1,16 +0,0 @@
17.4-* BigBenches
17.5-:PROPERTIES:
17.6-:ID: 87d0f5d4-c6fe-4e21-911e-2639a24e89f9
17.7-:END:
17.8-#+name: 1trc-nu
17.9-#+begin_src nushell
17.10- let ms = '1trc/measurements-0.parquet'
17.11- dfr open $ms
17.12- | dfr group-by station
17.13- | dfr agg [
17.14- (dfr col measure | dfr min | dfr as "min")
17.15- (dfr col measure | dfr max | dfr as "max")
17.16- (dfr col measure | dfr sum | dfr as "sum")
17.17- (dfr col measure | dfr count | dfr as "count")
17.18- ]
17.19-#+end_src
18.1--- a/20240218.org Tue Aug 27 21:35:44 2024 -0400
18.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
18.3@@ -1,78 +0,0 @@
18.4-* WL vs X
18.5-:PROPERTIES:
18.6-:ID: b4b29989-28f2-48db-ad84-cd8d08eeefdf
18.7-:END:
18.8-:LOGBOOK:
18.9-- State "NOTE" from [2024-02-18 Sun 11:55]
18.10-:END:
18.11-In the past few months there has been drama regarding Wayland vs X. It
18.12-seems to be on everyone's minds after Artem's freakout issue and the
18.13-follow up YT vids/comments.
18.14-
18.15-I admit that it made me reconsider the fitness of WL as a whole -
18.16-there was a github gist that made some scathing arguments against it.
18.17-
18.18-It's an odd debate though. I think there are many misunderstandings.
18.19-
18.20-So first off, if we look at the homepage
18.21-https://wayland.freedesktop.org/, Wayland claims it is a replacement
18.22-for X11. It now has /manifest destiny/, which in my opinion is a great
18.23-shame.
18.24-
18.25-X-pros seem to agree that Wayland has /manifest destiny/ - like if you
18.26-are building softwares that look remotely like a window system, it's a
18.27-successor to X. That's the model of doing things and there's no way
18.28-around it.
18.29-
18.30-The disagreement starts with how this destiny - of an X2 - should be
18.31-fulfilled. X-pros want a fork of X, but it's too late for
18.32-that. WL-pros want X to run on top of Wayland compositor:
18.33-https://wayland.freedesktop.org/xserver.html.
18.34-
18.35-Xwayland is a problem for me. From the project description: 'if we're
18.36-migrating away from X, it makes sense to have a good backwards
18.37-compatibility story.' Full disclosure: I have never done significant
18.38-work on Xwayland, so perhaps my opinion is unwarranted. But I have no
18.39-intention of attempting to maintain a computer system that uses
18.40-Wayland and X clients at the same time.
18.41-
18.42-To me, X is ol' reliable. Every distro has first-class X support, and
18.43-it runs on most systems with very little user intervention. Where it
18.44-doesn't, there is 20+ years of dev history and battle-tested
18.45-workarounds for you to find your solution in.
18.46-
18.47-Wayland is the new kid on the block, born just in 2008. It's a fresh
18.48-start to one of the most difficult challenges in software - window
18.49-systems. A re-write would be pointless though, and so the real
18.50-value-add is in design. Wayland is designed as a protocol and
18.51-collection of libraries which are implemented in your own
18.52-compositor. Coming from Lisp - with ANSI Common Lisp and SRFIs, this
18.53-feels right even if the implementation is something very different
18.54-(compositor vs compiler).
18.55-
18.56-With X, it is assumed to be much harder to write an equivalent
18.57-'compositor'. Here's the thing though - with a significantly complex X
18.58-client implementation, it is /impossible/ to replicate in WL. This is
18.59-really the crux of Artemi's argument in his issue. He asked for a 1:1
18.60-equivalent X/WL comparison when no such thing exists, and in my
18.61-opinion it is a waste of time.
18.62-
18.63-The WL core team is fully aware of this dichotomy, but also that this
18.64-is in no way a problem or weakness in either system. It means they're
18.65-different systems, goddammit.
18.66-
18.67-If it was up to me, Xwayland wouldn't exist. I understand why it does,
18.68-and that it does make things easier for developers who need to support
18.69-both, and users who have multiple apps with multiple windowing
18.70-requirements. It's a bandaid though, and one that is particularly
18.71-dangerous because it re-enforces the idea that Wayland is just X2 and
18.72-that they're fully compatible.
18.73-
18.74-What interests me in the Wayland world right now is the idea of a
18.75-small, modular, full-stack Wayland compositor API. There are several
18.76-'kiosk' based compositors for single applications (cage), but these
18.77-aren't complete solutions. It is possible to get much closer to the
18.78-metal, and that's where I want to be so that I can build my own APIs
18.79-on top - I don't want to live on top of X, and I certainly don't want
18.80-to live on top of X on top of WL. I want a /pure/ solution that hides
18.81-as little as possible, exposing the interesting bits.
19.1--- a/20240301.org Tue Aug 27 21:35:44 2024 -0400
19.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
19.3@@ -1,9 +0,0 @@
19.4-* TODO collect more data
19.5-:PROPERTIES:
19.6-:ID: 69ce5703-d3fe-4df7-abf4-dc93a08ebc12
19.7-:END:
19.8-:LOGBOOK:
19.9-- State "TODO" from [2024-03-01 Fri 15:27]
19.10-:END:
19.11-https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
19.12-weather - music - etc
20.1--- a/20240302.org Tue Aug 27 21:35:44 2024 -0400
20.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
20.3@@ -1,126 +0,0 @@
20.4-* On blocks and devices
20.5-:PROPERTIES:
20.6-:ID: c8304ae3-415d-4ccf-8f4c-1f3f6a555e5b
20.7-:END:
20.8-:LOGBOOK:
20.9-- State "NOTE" from [2024-03-02 Sat 21:30]
20.10-:END:
20.11-[[https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/dev.html][/dev]]
20.12-In Linux, everything is a file.
20.13-
20.14-/dev/ contains special device files - usually block or character
20.15-device.
20.16-
20.17-major, minor = category, device
20.18-0, 5
20.19-
20.20-mknod - create special device files
20.21-
20.22-[[https://www.redhat.com/en/blog/linux-block-devices-hints-debugging-and-new-developments][redhat hints]]
20.23-
20.24-#+begin_src shell
20.25- dd if=/dev/zero of=myfile bs=1M count=32
20.26- losetup --show -f myfile
20.27- ls -al /dev/loop0
20.28- losetup -d /dev/loop0 #teardown
20.29-#+end_src
20.30-
20.31-#+begin_src shell
20.32- echo "sup dude" > /dev/loop0
20.33- dd if=/dev/loop0 -bs=1
20.34- dd if=/dev/nvme0 of=/dev/null progress=true
20.35- #pacman -S hdparm
20.36- hdparm -T /dev/nvme0
20.37-#+end_src
20.38-
20.39-#+begin_src shell
20.40-modprobe scsi_debug add_host=5 max_luns=10 num_tgts=2 dev_size_mb=16
20.41-#+end_src
20.42-
20.43-sparsefiles: create with C, dd, or truncate
20.44-
20.45-#+begin_src shell :results output
20.46-truncate --help
20.47-#+end_src
20.48-
20.49-#+RESULTS:
20.50-#+begin_example
20.51-Usage: truncate OPTION... FILE...
20.52-Shrink or extend the size of each FILE to the specified size
20.53-
20.54-A FILE argument that does not exist is created.
20.55-
20.56-If a FILE is larger than the specified size, the extra data is lost.
20.57-If a FILE is shorter, it is extended and the sparse extended part (hole)
20.58-reads as zero bytes.
20.59-
20.60-Mandatory arguments to long options are mandatory for short options too.
20.61- -c, --no-create do not create any files
20.62- -o, --io-blocks treat SIZE as number of IO blocks instead of bytes
20.63- -r, --reference=RFILE base size on RFILE
20.64- -s, --size=SIZE set or adjust the file size by SIZE bytes
20.65- --help display this help and exit
20.66- --version output version information and exit
20.67-
20.68-The SIZE argument is an integer and optional unit (example: 10K is 10*1024).
20.69-Units are K,M,G,T,P,E,Z,Y,R,Q (powers of 1024) or KB,MB,... (powers of 1000).
20.70-Binary prefixes can be used, too: KiB=K, MiB=M, and so on.
20.71-
20.72-SIZE may also be prefixed by one of the following modifying characters:
20.73-'+' extend by, '-' reduce by, '<' at most, '>' at least,
20.74-'/' round down to multiple of, '%' round up to multiple of.
20.75-
20.76-GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
20.77-Full documentation <https://www.gnu.org/software/coreutils/truncate>
20.78-or available locally via: info '(coreutils) truncate invocation'
20.79-#+end_example
20.80-
20.81-test mkfs.btrfs on 10T dummy block device
20.82-#+begin_src shell
20.83- dd if=/dev/zero of=/tmp/bb1 bs=1 count=1 seek=10T
20.84- du -sh /tmp/bb1
20.85- losetup --show -f /tmp/bb1
20.86- mkfs.btrfs /dev/loop0
20.87-#+end_src
20.88-
20.89-diagnostics
20.90-#+begin_src shell
20.91- iostat # pacman -S sysstat
20.92- blktrace # paru -S blktrace
20.93- iotop # pacman -S iotop
20.94-#+end_src
20.95-
20.96-bcc/ trace: Who/which process is executing specific functions against
20.97-block devices?
20.98-
20.99-bcc/biosnoop: Which process is accessing the block device, how many
20.100-bytes are accessed, which latency for answering the requests?
20.101-
20.102-at the kernel level besides BPF we got kmods and DKMS,
20.103-
20.104-compression/de-duplication can be done via VDO kernel mod
20.105-
20.106-https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support
20.107-
20.108-* save-lisp-and-respawn
20.109-:PROPERTIES:
20.110-:ID: e58a96e7-7b88-4e4a-b958-3000603d8baa
20.111-:END:
20.112-:LOGBOOK:
20.113-- State "NOTE" from [2024-03-02 Sat 22:57]
20.114-:END:
20.115-#+begin_src lisp
20.116- sb-ext:*save-hooks*
20.117-#+end_src
20.118-
20.119-* syslog for log
20.120-:PROPERTIES:
20.121-:ID: 2efa5e4a-f063-49bb-9e61-6917eb115c1a
20.122-:END:
20.123-:LOGBOOK:
20.124-- State "NOTE" from [2024-03-03 Sun 16:35]
20.125-:END:
20.126-sb-posix:
20.127-- openlog syslog closelog
20.128-- levels: emerg alert crit err warning notice info debug
20.129-- setlogmask
21.1--- a/20240313.org Tue Aug 27 21:35:44 2024 -0400
21.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
21.3@@ -1,15 +0,0 @@
21.4-* RESEARCH [[https://github.com/guicho271828/sbcl-wiki/wiki/][sbcl-wiki]]
21.5-:PROPERTIES:
21.6-:ID: 33bc81c1-c5fa-4518-9456-58fd67f301ff
21.7-:END:
21.8-:LOGBOOK:
21.9-- State "RESEARCH" from [2024-03-13 Wed 21:49]
21.10-:END:
21.11-** IR1
21.12-:PROPERTIES:
21.13-:ID: 2c012572-ea16-408c-b741-d1f6cd0c7961
21.14-:END:
21.15-** IR2
21.16-:PROPERTIES:
21.17-:ID: 8d015e5e-7ad3-44c3-88cd-81e293a887e0
21.18-:END:
22.1--- a/20240317.org Tue Aug 27 21:35:44 2024 -0400
22.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
22.3@@ -1,130 +0,0 @@
22.4-* DB Benchmarking
22.5-:PROPERTIES:
22.6-:ID: d30aec42-a9a1-425b-87ca-1e40db0e69de
22.7-:END:
22.8-:LOGBOOK:
22.9-- State "NOTE" from [2024-02-04 Sun 20:40]
22.10-:END:
22.11-[[https://github.com/facebook/rocksdb/wiki/Benchmarking-tools][RocksDB benchmarking tools]]
22.12-
22.13-* packy design
22.14-:PROPERTIES:
22.15-:ID: 76ae24f5-46e8-4b91-8991-41245383d337
22.16-:END:
22.17-:LOGBOOK:
22.18-- State "NOTE" from [2024-01-25 Thu 22:39]
22.19-:END:
22.20-- API root: https://packy.compiler.company
22.21-- source packs: https://vc.compiler.company/packy
22.22-** Lib
22.23-:PROPERTIES:
22.24-:ID: 2d6aa6fb-a34c-4ccd-99d3-4f98d33bb2f9
22.25-:END:
22.26-*** Types
22.27-:PROPERTIES:
22.28-:ID: 8017b6c7-3a04-4e52-b3d6-afc68cbce139
22.29-:END:
22.30-**** Pack
22.31-:PROPERTIES:
22.32-:ID: 2722b7dc-9546-4d94-82b0-abfc02647b4d
22.33-:END:
22.34-Primary data type of the library - typically represents a compressed
22.35-archive, metadata, and ops.
22.36-**** Bundle
22.37-:PROPERTIES:
22.38-:ID: 9ec5d682-da8e-4d02-b2fc-a56a8f209c22
22.39-:END:
22.40-Collection data type, usually contains a set of packs with metadata.
22.41-**** PackyEndpoint
22.42-:PROPERTIES:
22.43-:ID: 8cf2658a-92d5-41ab-8630-4f4e6ddb140d
22.44-:END:
22.45-Represents a Packy instance bound to a UDP socket
22.46-**** PackyEndpointConfig
22.47-:PROPERTIES:
22.48-:ID: 421b8aaa-66e1-4883-937d-fc6705146998
22.49-:END:
22.50-Global endpoint configuration object
22.51-**** PackyClientConfig
22.52-:PROPERTIES:
22.53-:ID: c1899710-034e-4ed6-959a-1eb2df72e5f9
22.54-:END:
22.55-Configuration for outgoing packy connections on an endpoint
22.56-**** PackyServerConfig
22.57-:PROPERTIES:
22.58-:ID: 955e8ce9-ff7e-4f42-9d25-3aeb9846528b
22.59-:END:
22.60-Configuration for incoming packy connection son an endpoint
22.61-**** PackyConnection
22.62-:PROPERTIES:
22.63-:ID: 7075e0c8-65b3-4fbf-9a6b-f27017276a72
22.64-:END:
22.65-Packy connection object
22.66-*** Traits
22.67-:PROPERTIES:
22.68-:ID: ea92ad07-d12c-4ee8-a3d4-0955f0af3921
22.69-:END:
22.70-**** PackyClient
22.71-:PROPERTIES:
22.72-:ID: 6c0fdd7e-bc0a-4217-b315-72fe7766e4df
22.73-:END:
22.74-***** query
22.75-:PROPERTIES:
22.76-:ID: 1358e1c5-b215-48eb-9e1b-4116cc32b2ae
22.77-:END:
22.78-***** install
22.79-:PROPERTIES:
22.80-:ID: b8298207-4d92-4628-8687-9f4e20c8cfec
22.81-:END:
22.82-***** update
22.83-:PROPERTIES:
22.84-:ID: a6fb4fb6-b188-4033-8291-107cf1f00e6d
22.85-:END:
22.86-***** login
22.87-:PROPERTIES:
22.88-:ID: 1fd91c8b-ab90-4db0-b74e-1c2b9ae2e440
22.89-:END:
22.90-***** logout
22.91-:PROPERTIES:
22.92-:ID: 75c0db2e-37cf-4f33-95bd-3f5c95b45c49
22.93-:END:
22.94-***** pull
22.95-:PROPERTIES:
22.96-:ID: 60de92f4-b4a6-4cae-95bc-174e2036e57b
22.97-:END:
22.98-***** push
22.99-:PROPERTIES:
22.100-:ID: e379b71e-8a84-4547-a694-90cad3b7952e
22.101-:END:
22.102-**** PackyServer
22.103-:PROPERTIES:
22.104-:ID: 73ea9478-5251-40c3-8331-cf042d0b9bf1
22.105-:END:
22.106-***** start_packy_server
22.107-:PROPERTIES:
22.108-:ID: c8a9bac3-971a-45a7-988f-92db19ef6601
22.109-:END:
22.110-***** stop_packy_server
22.111-:PROPERTIES:
22.112-:ID: e3122bf1-f770-46a7-a36b-2df9c48bc233
22.113-:END:
22.114-***** start_packy_registry
22.115-:PROPERTIES:
22.116-:ID: b6d214f2-d6de-4d46-b2c2-c51ff0f3b3e6
22.117-:END:
22.118-**** PackyRegistry
22.119-:PROPERTIES:
22.120-:ID: 0e846cbb-5cb4-496f-8100-beae6e66972e
22.121-:END:
22.122-***** register_pack
22.123-:PROPERTIES:
22.124-:ID: 764642ad-a33b-415b-9c32-8a729a7cd62d
22.125-:END:
22.126-***** register_user
22.127-:PROPERTIES:
22.128-:ID: 2b92f584-d769-4266-940e-cfbd466d4e62
22.129-:END:
22.130-***** register_bundle
22.131-:PROPERTIES:
22.132-:ID: 390cb42b-5170-4175-9ccc-b990842356a0
22.133-:END:
23.1--- a/20240325.org Tue Aug 27 21:35:44 2024 -0400
23.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
23.3@@ -1,7 +0,0 @@
23.4-* TBD investigate alieneval for phash opps
23.5-:PROPERTIES:
23.6-:ID: 21ab0a6e-cb66-4c5e-9b10-4ac2340ce75d
23.7-:END:
23.8-:LOGBOOK:
23.9-- State "TBD" from [2024-03-25 Mon 18:56]
23.10-:END:
24.1--- a/20240419.org Tue Aug 27 21:35:44 2024 -0400
24.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
24.3@@ -1,142 +0,0 @@
24.4-* How it works
24.5-:PROPERTIES:
24.6-:ID: 439a4945-de2f-4985-ba6c-fe6421aa9ad9
24.7-:END:
24.8-The backend services are written in Rust and controlled by a simple
24.9-messaging protocol. Services provide common runtime capabilities known
24.10-as the /service protocol/ but are specialized on a unique /service
24.11-type/ which may in turn register their own /custom protocols/ (via
24.12-core).
24.13-
24.14-Services are capable of dispatching data directly to clients, or
24.15-storing data in the /database/ (sqlite, postgres, mysql).
24.16-
24.17-The frontend clients are pre-dominantly written in Common Lisp and
24.18-come in many shapes and sizes. There is a cli-client, web-client
24.19-(CLOG), docker-client (archlinux, stumpwm, McCLIM), and native-client
24.20-which also compiles to WASM (slint-rs).
24.21-
24.22-* Guide
24.23-:PROPERTIES:
24.24-:ID: 65140d2c-5d59-48cd-946c-01c380a37f08
24.25-:END:
24.26-** Build
24.27-:PROPERTIES:
24.28-:ID: ad0652ca-4abf-4d27-8897-a61a13a6e61d
24.29-:END:
24.30-- *install dependencies*
24.31- #+begin_src bash
24.32- ./tools/deps.sh
24.33- #+end_src
24.34-- *make executables* \\
24.35- Simply run =make build=. Read the ~makefile~ and change the options
24.36- as needed.
24.37-- MODE :: Mode (debug, release)
24.38-- LISP :: Lisp (sbcl, cmucl, ccl)
24.39-- CFG :: Config (default.cfg)
24.40-** Run
24.41-:PROPERTIES:
24.42-:ID: b2295442-ccb7-4a5c-8a9e-ea8502aad636
24.43-:END:
24.44-#+begin_src shell
24.45- ./demo -i
24.46-#+end_src
24.47-** Config
24.48-:PROPERTIES:
24.49-:ID: 600be170-d032-4280-a7b5-c46e3c157206
24.50-:END:
24.51-Configs can be specified in JSON, TOML, RON, or of course SEXP. See
24.52-=default.cfg= for an example.
24.53-** Play
24.54-:PROPERTIES:
24.55-:ID: 56a69ed7-2eb3-4490-95a1-072cee985df5
24.56-:END:
24.57-The high-level user interface is presented as a multi-modal GUI
24.58-application which adapts to the specific application /instances/
24.59-below.
24.60-*** Weather
24.61-:PROPERTIES:
24.62-:ID: 804a174b-534c-4a04-966a-b44c0a86fbd4
24.63-:END:
24.64-This backend retrieves weather data using the NWS API.
24.65-*** Stocks
24.66-:PROPERTIES:
24.67-:ID: 4e7339bb-aaf3-4fab-90e4-05e13f24f881
24.68-:END:
24.69-The 'Stocks' backend features a stock ticker with real-time analysis
24.70-capabilities.
24.71-*** Bench
24.72-:PROPERTIES:
24.73-:ID: e3cf0add-6ded-4c05-a05d-7ef463b85b08
24.74-:END:
24.75-This is a benchmark backend for testing the capabilities of our
24.76-demo. It spins up some mock services and allows fine-grained control
24.77-of input/throughput.
24.78-* tasks
24.79-:PROPERTIES:
24.80-:ID: a16a3807-0e06-4c8d-baf6-5dfa8f8f5585
24.81-:END:
24.82-** TODO DSLs
24.83-:PROPERTIES:
24.84-:ID: 3ae4754e-0c0f-4923-9e53-ae962b8638a1
24.85-:END:
24.86-- consider tree-sitter parsing layout, use as a guide for developing a
24.87- single syntax which expands to Rust or C.
24.88-- with-rs
24.89-- with-c
24.90-- with-rs/c
24.91-- with-cargo
24.92-- compile-rs/c
24.93-*** TODO rs-macroexpand
24.94-:PROPERTIES:
24.95-:ID: f612975f-6bcc-4aab-b0d8-65b8ff0a0d0f
24.96-:END:
24.97-- rs-gen-file
24.98-- rs-defmacro
24.99-- rs-macros
24.100-- rs-macroexpand
24.101-- rs-macroexpand-1
24.102-*** TODO c-macroexpand
24.103-:PROPERTIES:
24.104-:ID: 49f5c224-026d-4cdc-a326-1742e64c4279
24.105-:END:
24.106-- c-gen-file h/c
24.107-- c-defmacro
24.108-- c-macros
24.109-- c-macroexpand
24.110-- c-macroexpand-1
24.111-*** TODO slint-macroexpand
24.112-:PROPERTIES:
24.113-:ID: 486beb12-179b-4acc-8507-cdc498edb628
24.114-:END:
24.115-- slint-gen-file
24.116-- slint-defmacro
24.117-- slint-macros
24.118-- slint-macroexpand
24.119-- slint-macroexpand-1
24.120-*** TODO html (using who)
24.121-:PROPERTIES:
24.122-:ID: 2e81a19c-874c-4fd7-be0f-f6cd62c660db
24.123-:END:
24.124-** TODO web templates
24.125-:PROPERTIES:
24.126-:ID: 95f7d6ac-c010-4d1d-a47d-1693bb7d6eab
24.127-:END:
24.128-create a basic static page in CL which will be used to host Slint UIs
24.129-and other WASM doo-dads in a browser.
24.130-** TODO CLI
24.131-:PROPERTIES:
24.132-:ID: fc1d5364-279c-4d5f-baea-ed2c110b63f0
24.133-:END:
24.134-using clingon, decide on generic options and write it up
24.135-** TODO docs
24.136-:PROPERTIES:
24.137-:ID: 467e84e3-e72f-4764-8858-cc15fdc17e8a
24.138-:END:
24.139-work on doc generation -- Rust and CL should be accounted for.
24.140-** TODO tests
24.141-:PROPERTIES:
24.142-:ID: 70aea01c-d01e-4ddf-aaf5-c86183a7e250
24.143-:END:
24.144-We have none! need to make it more comfy - set up testing in all Rust
24.145-crates and for the lisp systems.
25.1--- a/20240425.org Tue Aug 27 21:35:44 2024 -0400
25.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
25.3@@ -1,4 +0,0 @@
25.4-* https://docs.gitlab.com/ee/administration/backup_restore/migrate_to_new_server.html
25.5-:PROPERTIES:
25.6-:ID: 264b81f4-f953-4698-9c82-d8e20b6c31f9
25.7-:END:
26.1--- a/20240731.org Tue Aug 27 21:35:44 2024 -0400
26.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
26.3@@ -1,11 +0,0 @@
26.4-* alpine builders
26.5-:PROPERTIES:
26.6-:ID: 635925f6-e1d8-47bc-a32f-28bca0f19d2d
26.7-:END:
26.8-- make sure to apk add:
26.9- - git, hg
26.10- - clang
26.11- - make
26.12- - linux-headers
26.13- - zstd-dev
26.14- - libc-dev?
27.1--- a/20240804.org Tue Aug 27 21:35:44 2024 -0400
27.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
27.3@@ -1,8 +0,0 @@
27.4-* bookmarks
27.5-:PROPERTIES:
27.6-:ID: 6e9c6bce-0513-4628-b6f9-5c234cccdaa3
27.7-:END:
27.8-- How should such objects be represented within CORE?
27.9-- skel/homer mostly
27.10- - already have alias
27.11-- not sure about obj/otherwise, prob not
28.1--- a/20240808.org Tue Aug 27 21:35:44 2024 -0400
28.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
28.3@@ -1,12 +0,0 @@
28.4-* Intelligent Design in Software
28.5-:PROPERTIES:
28.6-:ID: 1397d8bb-4848-4d9a-9dcc-ee65bed81c9e
28.7-:END:
28.8-- starting from a space where there are no external influences - a biome
28.9-- answer questions regarding the nature of the software and its capabilities
28.10-- incrementally adjust inter-dependencies
28.11-- optimize
28.12-- protect the biome at all costs
28.13-- focus on composition
28.14-- build applications
28.15-- re-integrate lessons learned
29.1--- a/20240816.org Tue Aug 27 21:35:44 2024 -0400
29.2+++ /dev/null Thu Jan 01 00:00:00 1970 +0000
29.3@@ -1,11 +0,0 @@
29.4-* keys.compiler.company
29.5-:PROPERTIES:
29.6-:ID: 86c21790-72b5-4ea1-ba49-f7be3f65a8cc
29.7-:END:
29.8-:LOGBOOK:
29.9-- State "NOTE" from [2024-08-16 Fri 22:00]
29.10-:END:
29.11-- public openpgp server
29.12-- [[https://keys.compiler.company][keys.compiler.company]]
29.13-- https://keys.openpgp.org/
29.14-- [[vc:packy/hagrid/][packy/hagrid]]
30.1--- /dev/null Thu Jan 01 00:00:00 1970 +0000
30.2+++ b/log.org Wed Sep 04 16:58:53 2024 -0400
30.3@@ -0,0 +1,1506 @@
30.4+#+title: log
30.5+#+author: Richard Westhaver
30.6+#+email: richard.westhaver@gmail.com
30.7+#+setupfile: ../clean.theme
30.8+* [2023-07-30 Sun]
30.9+** VC infrastructure
30.10+:PROPERTIES:
30.11+:ID: 2d40eec1-62f4-4f3d-9be7-110400808e06
30.12+:END:
30.13+In heptapod we have a root group named =comp=, containg a variety of
30.14+subgroups. Some of these groups should be public, while others are
30.15+internal to comp members exclusively. Within each subgroup, we should
30.16+have the root group members automatically granted privileged access to
30.17+projects. This is relevant for the =startup= subgroup in particular,
30.18+where each project is potentially maintained by multiple non-root
30.19+contributors.
30.20+
30.21+We also need to consider how we will manage subrepos across the
30.22+organization. It is about time we start integrating HG bundles and
30.23+potentially mirrors. For our core VC pipeline we should have no
30.24+reliance on Git, but this may be difficult. It depends on the behavior
30.25+of HG bundles.
30.26+
30.27+Bookmarks/tags should be used for milestones in the root group and are
30.28+infrequent. They are more frequent in projects with a regular release
30.29+life-cycle.
30.30+** Approaching Webapps
30.31+:PROPERTIES:
30.32+:ID: 0a52be58-990a-47da-ba5a-a7da7192361c
30.33+:END:
30.34+I started poking around in the webapp space again so that I can launch
30.35+a landing page for NAS-T quickly. The Rust situation has improved
30.36+somewhat on the frontend side, and the axum backend stack is nice.
30.37+
30.38+This might seem like a lot of Rust and not a lot of Lisp, which it is,
30.39+but there's still room for Lisp wherever we need it. It mostly plays a
30.40+role in the backend, servicing the database and responding to requests
30.41+from the Rust edges. All of the important tests for the web APIs are
30.42+also written in Lisp. We will almost certainly use Lisp for all static
30.43+processing and HTML generation at compile-time.
30.44+
30.45+This I believe, is the appropriate way to integrate Lisp into a
30.46+cutting-edge web-app. You get the good parts of Lisp where you need
30.47+them (interactive debugging, dynamic language, REPL) and avoid the bad
30.48+parts (OOB optimization, RPS performance) in areas where the customer
30.49+would be impacted. In this domain, Lisp takes the form of a glue
30.50+rather than the bricks and mortar it sometimes appears to us as.
30.51+
30.52+* [2023-10-24 Tue]
30.53+** virt
30.54+:PROPERTIES:
30.55+:ID: a51b54d9-5663-411e-b0ba-d1405622eb88
30.56+:END:
30.57+*** QEMU
30.58+:PROPERTIES:
30.59+:ID: 34dc9d12-d651-4204-9abb-a6a9a6aeb07e
30.60+:END:
30.61+*** KVM
30.62+:PROPERTIES:
30.63+:ID: 909e1127-14f5-4783-97af-65aa25439226
30.64+:END:
30.65+*** Hyper-V
30.66+:PROPERTIES:
30.67+:ID: d89dc59d-25d9-4304-bcb8-379dd43cc3f7
30.68+:END:
30.69+*** Firecracker
30.70+:PROPERTIES:
30.71+:ID: 7c4f08b7-2bc0-4a85-a6ca-2decdce8dc91
30.72+:END:
30.73+*** Docker
30.74+:PROPERTIES:
30.75+:ID: 42ff77c2-69c9-4f89-a297-88efd592502e
30.76+:END:
30.77+*** Vagrant
30.78+:PROPERTIES:
30.79+:ID: c53c4d87-b0fd-40de-8db7-b92c22dc946c
30.80+:END:
30.81+*** LXC
30.82+:PROPERTIES:
30.83+:ID: 85c5f471-cf87-49b7-a6ce-5c1526df4080
30.84+:END:
30.85+*** LXD
30.86+:PROPERTIES:
30.87+:ID: f92d34dd-7b24-498c-a5c8-29fd3b528092
30.88+:END:
30.89+*** containerd
30.90+:PROPERTIES:
30.91+:ID: a967645f-5df3-4bed-9e7b-a1adaed3796c
30.92+:END:
30.93+*** systemd-nspawn
30.94+:PROPERTIES:
30.95+:ID: 6329c6f9-c54c-4eb0-99e7-4772399161e7
30.96+:END:
30.97+*** VirtualBox
30.98+:PROPERTIES:
30.99+:ID: 638aa81a-8926-480a-854f-0434e8119426
30.100+:END:
30.101+
30.102+** Concatenative
30.103+:PROPERTIES:
30.104+:ID: 964c6bac-486a-4459-825f-6e7e9351c9ab
30.105+:END:
30.106+*** Factor :factor:
30.107+:PROPERTIES:
30.108+:ID: 52994125-5689-402a-ac61-680aa690bf24
30.109+:END:
30.110+- [2023-07-04 Tue]
30.111+ Factor is a cool concatenative lang but unfortunately the C interface
30.112+ (vm/master.h) no longer exists on the master branch.
30.113+*** Joy :joy:
30.114+:PROPERTIES:
30.115+:ID: 60ca3000-2ff5-4372-93fb-d5c311fb6409
30.116+:END:
30.117+
30.118+**** https://hypercubed.github.io/joy/html/j02maf.html
30.119+:PROPERTIES:
30.120+:ID: 56e64d52-4950-4fec-b933-73d1cd8048d1
30.121+:END:
30.122+
30.123+**** [[https://builds.openlogicproject.org/content/incompleteness/arithmetization-syntax/arithmetization-syntax.pdf][arithmetization of syntax]]
30.124+:PROPERTIES:
30.125+:ID: 59979091-395d-4067-9c3d-e557fc5287ee
30.126+:END:
30.127+** Lisp :lisp:
30.128+:PROPERTIES:
30.129+:ID: 4b842734-135e-4c86-9337-3841c1241d3b
30.130+:END:
30.131+These notes pertain to Lisp. More specifically, ANSI Common Lisp in
30.132+most places.
30.133+
30.134+- https://github.com/lispnik/iup/ - doesn't support MacOS yet, looks
30.135+ cool though
30.136+ - what we really need is wasm compiler.. TBD
30.137+** Rust
30.138+:PROPERTIES:
30.139+:ID: f92ce2ba-1b66-42f4-b5e2-f1586b14760b
30.140+:END:
30.141+*** Serde
30.142+:PROPERTIES:
30.143+:ID: 31512714-9c30-4144-9673-327808a18767
30.144+:END:
30.145+- [2023-07-05 Wed] \\
30.146+ important part of the Rust ecosystem, another dtolnay
30.147+ contribution. If you want to program a /data/ format in the Rust
30.148+ ecosystem, this is how you do it.
30.149+
30.150+ The way it works is that you define some special structs, a
30.151+ Serializer and a Deserializer which implement the Serialize and
30.152+ Deserialize traits provided by serde, respectively.
30.153+
30.154+ You can use these structs to provide your public API. The
30.155+ conventional choice is public top-level functions like from-str
30.156+ and to-string. That's it, your serialization library can now read and
30.157+ write your data format as Rust data types.
30.158+
30.159+ [[https://serde.rs/enum-representations.html][enum-representations]]
30.160+ - the default behavior is an externally tagged representation (verbose)
30.161+
30.162+ The docs use strings as core IO when implementing a custom format,
30.163+ but the convention is to implement for T where T is bound by std::io
30.164+ Read or Write trait. Then you can provide a more robust public API
30.165+ (from_bytes, from_writer, etc).
30.166+** C
30.167+:PROPERTIES:
30.168+:ID: 060042a0-d581-411a-9091-d16d11860d89
30.169+:END:
30.170+** CPP
30.171+:PROPERTIES:
30.172+:ID: f09d7294-a997-4715-bceb-ffe6af7f3978
30.173+:END:
30.174+** Nu
30.175+:PROPERTIES:
30.176+:ID: 765ad403-54e1-49cf-b4e6-918de9f910c2
30.177+:END:
30.178+[[https://www.nushell.sh/][~]]
30.179+[[https://www.nushell.sh/cookbook/][cookbook]]
30.180+[[https://github.com/nushell/nu_scripts][nu_scripts]]
30.181+* [2023-11-01 Wed]
30.182+** AWS usage
30.183+:PROPERTIES:
30.184+:ID: b2f8d58f-a486-4fe9-895b-504987c0f63d
30.185+:END:
30.186+We're leveraging AWS for some of our public web servers for now. It's
30.187+really not realistic to expect that my home desktop and spotty Comcast
30.188+internet can serve any production workflow. What it /is/ capable of is
30.189+a private VPN, which can communicate with AWS and other cloud VPN
30.190+depots via WireGuard ([[https://dev.to/gabrieltetzner/setting-up-a-vpn-with-wireguard-server-on-aws-ec2-4a49][article]]).
30.191+
30.192+I currently use Google Domains for nas-t.net, otom8.dev, and
30.193+rwest.io - but that business is now owned by squarespace, so I would
30.194+rather move it to Route53.
30.195+
30.196+We have archlinux ec2 image builds [[https://wiki.archlinux.org/title/Arch_Linux_AMIs_for_Amazon_Web_Services][here]] and [[https://gitlab.com/anemos-io/archlinux-ec2][here]] - only half work and not
30.197+maintained, but it's a start. I'm not even sure if I should stick with
30.198+arch or cave and use Ubuntu or AWS Linux. We can serve the static
30.199+services with little cost, the only big spender will be the heptapod
30.200+instance which requires a larger instance and some workers.
30.201+
30.202+We'll try to keep the cost at or around $30/month.
30.203+* [2023-11-02 Thu]
30.204+** IDEAS
30.205+:PROPERTIES:
30.206+:ID: 836b5ca7-6c6b-4d57-8b44-3bb4a8f38a38
30.207+:END:
30.208+*** shed
30.209+:PROPERTIES:
30.210+:ID: fc9a94e1-91c5-4915-90b8-73218fa3b8bc
30.211+:END:
30.212+:LOGBOOK:
30.213+- State "TODO" from [2023-04-07 Fri 23:24]
30.214+:END:
30.215+rlib
30.216+> ulib
30.217+> ulib
30.218+> ulib
30.219+> ulib
30.220+
30.221+**** sh* tools
30.222+:PROPERTIES:
30.223+:ID: c0613a13-7ccb-4af9-b47e-e14a41c782c2
30.224+:END:
30.225+:LOGBOOK:
30.226+- State "TODO" from "TODO" [2023-04-07 Fri 23:22]
30.227+:END:
30.228+shc,shx,etc
30.229+*** packy
30.230+:PROPERTIES:
30.231+:ID: 6bd29c37-4b23-4a05-b3a4-72b6ee712f27
30.232+:END:
30.233+:LOGBOOK:
30.234+- State "TODO" from [2023-04-07 Fri 23:33]
30.235+:END:
30.236+**** rust
30.237+:PROPERTIES:
30.238+:ID: 3bfa733e-d777-4e82-b108-93ba74c4dc08
30.239+:END:
30.240+**** common-lisp
30.241+:PROPERTIES:
30.242+:ID: 0f3b6182-1dd1-4e65-8b1c-4630a46fff4c
30.243+:END:
30.244+**** emacs-lisp
30.245+:PROPERTIES:
30.246+:ID: e9fc19d5-daea-40a5-a0f2-882b925e58f2
30.247+:END:
30.248+**** python
30.249+:PROPERTIES:
30.250+:ID: c2e62af0-0397-4560-a283-b33d12fbd966
30.251+:END:
30.252+**** julia
30.253+:PROPERTIES:
30.254+:ID: d2466880-1721-4386-b385-ded06d5b15cf
30.255+:END:
30.256+**** C
30.257+:PROPERTIES:
30.258+:ID: fb722462-9e11-40c1-a5d7-30698dba7e20
30.259+:END:
30.260+**** C++
30.261+:PROPERTIES:
30.262+:ID: 663cd894-0a5f-4352-acf1-4528b9efeeaf
30.263+:END:
30.264+*** tenex
30.265+:PROPERTIES:
30.266+:ID: b4264fa2-6e96-4679-bd17-2428ba3489ee
30.267+:END:
30.268+:LOGBOOK:
30.269+- State "TODO" from [2023-04-07 Fri 23:52]
30.270+:END:
30.271+*** mpk
30.272+:PROPERTIES:
30.273+:ID: d9661e4a-0328-4864-878d-096f76ddaf55
30.274+:END:
30.275+:LOGBOOK:
30.276+- State "TODO" from [2023-04-07 Fri 23:52]
30.277+:END:
30.278+*** cfg
30.279+:PROPERTIES:
30.280+:ID: 501593d4-702a-40cb-8a8f-19edfe2f9f29
30.281+:END:
30.282+:LOGBOOK:
30.283+- State "TODO" from [2023-04-07 Fri 23:34]
30.284+:END:
30.285+*** obj
30.286+:PROPERTIES:
30.287+:ID: 24700a26-b238-48dd-a24f-5539984b4fa6
30.288+:END:
30.289+:LOGBOOK:
30.290+- State "TODO" from [2023-04-07 Fri 23:51]
30.291+:END:
30.292+split out from rlib to separate package
30.293+- a purely OOP class library
30.294+*** lab
30.295+:PROPERTIES:
30.296+:ID: fa1c9563-338b-4a56-bfc3-6f4983f2a732
30.297+:END:
30.298+:LOGBOOK:
30.299+- State "TODO" from [2023-04-07 Fri 23:34]
30.300+:END:
30.301+*** source categories
30.302+:PROPERTIES:
30.303+:ID: c43fdb5d-4f52-478e-9384-ec0e6e89cf09
30.304+:END:
30.305+- need a way of extracting metadata from a repo
30.306+- need ability to search and query libs/packages
30.307+- separate modules based on where they belong in our stack?
30.308+ - app
30.309+ - lib
30.310+ - script?
30.311+ - dist
30.312+ - software distros
30.313+*** generic query language
30.314+:PROPERTIES:
30.315+:ID: 8647801a-d5ad-446d-8ae1-fd896bce129a
30.316+:END:
30.317+from obj protocol?
30.318+sql compatibility?
30.319+
30.320+/check out kdb/
30.321+*** bbdb
30.322+:PROPERTIES:
30.323+:ID: ff7c2876-e471-480b-b532-9921dbd3bc89
30.324+:END:
30.325+:LOGBOOK:
30.326+- Note taken on [2023-10-24 Tue 22:16] \\
30.327+ graph database, build on rocksdb
30.328+:END:
30.329+insidious Big Brother database.
30.330+- an application built with obj
30.331+- sql
30.332+
30.333+*** NAS-TV :nas:t:
30.334+:PROPERTIES:
30.335+:ID: 00e5fe6c-80b4-4172-a9cb-ce8adc40d3dc
30.336+:END:
30.337+:LOGBOOK:
30.338+- State "NOTE" from "TODO" [2024-08-18 Sun 18:46]
30.339+:END:
30.340+- media streaming
30.341+- gstreamer backend
30.342+- audio/video
30.343+* [2023-11-05 Sun]
30.344+** DRAFT dylib-skel-1
30.345+:PROPERTIES:
30.346+:ID: b4d1bc91-f344-45fd-becc-cb20f00a3a61
30.347+:END:
30.348+- State "DRAFT" from [2023-11-05 Sun 22:23]
30.349+*** Overview
30.350+:PROPERTIES:
30.351+:ID: 2e490c4b-344e-4790-9184-1c05ba675f15
30.352+:END:
30.353+Our core languages are [[https://www.rust-lang.org/][Rust]] and [[https://lisp-lang.org/][Lisp]] - this is the killer combo which will allow NAS-T
30.354+to rapidly develop high-quality software. As such, it's crucial that these two very
30.355+different languages (i.e. compilers) are able to interoperate seamlessly.
30.356+
30.357+Some interop methods are easy to accomodate via the OS - such as IPC or data sharing,
30.358+but others are a bit more difficult.
30.359+
30.360+In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something
30.361+that /can/ be difficult, due to some complications with Rust and because this is not the
30.362+most popular software stack (yet ;). This is an experiment and may not make it to our
30.363+code-base, but it's definitely something worth adding to the toolbox in case we need it.
30.364+
30.365+*** FFI
30.366+:PROPERTIES:
30.367+:ID: 985019fc-612a-44ab-b726-b9067432ad87
30.368+:END:
30.369+The level of interop we're after in this case is [[https://en.wikipedia.org/wiki/Foreign_function_interface][FFI]].
30.370+
30.371+Basically, calling Rust code from Lisp and vice-versa. There's an article about calling
30.372+Rust from Common Lisp [[https://dev.to/veer66/calling-rust-from-common-lisp-45c5][here]] which shows the basics and serves as a great starting point
30.373+for those interested.
30.374+**** Rust != C
30.375+:PROPERTIES:
30.376+:ID: 2f71a3c1-0b14-46a6-9d8d-f6ec697729cc
30.377+:END:
30.378+The complication(s) with Rust I mentioned early is really just that /it is not C/. =C=
30.379+is old, i.e. well-supported with a stable ABI, making the process of creating bindings
30.380+for a C library a breeze in many languages.
30.381+
30.382+For a Rust library we need to first appease the compiler, as explained in [[https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c][this section]]
30.383+of the Rustonomicon. Among other things it involves changing the calling-convention of
30.384+functions with a type signature and editing the Cargo.toml file to produce a
30.385+C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used
30.386+like the C ABI can.
30.387+
30.388+**** Overhead
30.389+:PROPERTIES:
30.390+:ID: 4ea79f68-55ec-4da3-a184-8343d49532b6
30.391+:END:
30.392+Using FFI involves some overhead. Check [[https://github.com/dyu/ffi-overhead][here]] for an example benchmark across a few
30.393+languages. While building the NAS-T core, I'm very much aware of this, and will need a
30.394+few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular,
30.395+I'm concerned about crossing multiple language barriers (Rust<->C<->Lisp).
30.396+
30.397+*** Rust -> C -> Lisp
30.398+:PROPERTIES:
30.399+:ID: a498276c-8525-4a43-aa40-4b05f76a29a9
30.400+:END:
30.401+**** Setup
30.402+:PROPERTIES:
30.403+:ID: 19f96ef7-af92-496e-9d42-70c4d4c85051
30.404+:END:
30.405+For starters, I'm going to assume we all have Rust (via =rustup=) and Lisp (=sbcl= only)
30.406+installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
30.407+this post).
30.408+***** Cargo
30.409+:PROPERTIES:
30.410+:ID: c929e0b6-b6f2-4383-9412-1610329ab28c
30.411+:END:
30.412+Create a new library crate. For this example we're focusing on a 'skeleton' for
30.413+/dynamic/ libraries only, so our experiment will be called =dylib-skel= or *dysk* for
30.414+short.
30.415+src_sh[:exports code]{cargo init dysk --lib && cd dysk}
30.416+
30.417+A =src/lib.rs= will be generated for you. Go ahead and delete that. We're going to be
30.418+making our own =lib.rs= file directly in the root directory (just to be cool).
30.419+
30.420+The next step is to edit your =Cargo.toml= file. Add these lines after the =[package]=
30.421+section and before =[dependencies]=:
30.422+#+begin_src conf-toml
30.423+[lib]
30.424+crate-type = ["cdylib","rlib"]
30.425+path = "lib.rs"
30.426+[[bin]]
30.427+name="dysk-test"
30.428+path="test.rs"
30.429+#+end_src
30.430+
30.431+This tells Rust to generate a shared C-compatible object with a =.so= extension which we
30.432+can open using [[https://man.archlinux.org/man/dlopen.3.en][dlopen]].
30.433+***** cbindgen
30.434+:PROPERTIES:
30.435+:ID: 256ac288-c5a0-473a-ab65-2d6503bd423c
30.436+:END:
30.437+****** install
30.438+:PROPERTIES:
30.439+:ID: fc476f64-6b68-417a-8540-ca23ce27fa25
30.440+:END:
30.441+Next, we want the =cbindgen= program which we'll use to generate header files for
30.442+C/C++. This step isn't necessary at all, we just want it for further experimentation.
30.443+
30.444+src_sh[:exports code]{cargo install --force cbindgen}
30.445+
30.446+We append the =cbindgen= crate as a /build dependency/ to our =Cargo.toml= like so:
30.447+#+begin_src conf-toml
30.448+[build-dependencies]
30.449+cbindgen = "0.24"
30.450+#+end_src
30.451+****** cbindgen.toml
30.452+:PROPERTIES:
30.453+:ID: 111e27f7-0b9c-4eef-9117-f7c8ba3f511c
30.454+:END:
30.455+#+begin_src conf-toml :tangle cbindgen.toml
30.456+language = "C"
30.457+autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
30.458+include_version = true
30.459+namespace = "dysk"
30.460+cpp_compat = true
30.461+after_includes = "#define DYSK_VERSION \"0.1.0\""
30.462+line_length = 88
30.463+tab_width = 2
30.464+documentation = true
30.465+documentation_style = "c99"
30.466+usize_is_size_t = true
30.467+[cython]
30.468+header = '"dysk.h"'
30.469+#+end_src
30.470+****** build.rs
30.471+:PROPERTIES:
30.472+:ID: 9fc271b2-9acb-4f4b-aa61-82d60d2ddb9e
30.473+:END:
30.474+#+begin_src rust :tangle build.rs
30.475+fn main() -> Result<(), cbindgen::Error> {
30.476+ if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
30.477+ b.write_to_file("dysk.h"); Ok(())}
30.478+ else { panic!("failed to generate dysk.h from cbindgen.toml") } }
30.479+#+end_src
30.480+**** lib.rs
30.481+:PROPERTIES:
30.482+:ID: 6b524921-2ae0-43f0-bb85-d9955b0e689c
30.483+:END:
30.484+#+begin_src rust :tangle lib.rs
30.485+//! lib.rs --- dysk library
30.486+use std::ffi::{c_char, c_int, CString};
30.487+#[no_mangle]
30.488+pub extern "C" fn dysk_hello() -> *const c_char {
30.489+ CString::new("hello from rust").unwrap().into_raw()}
30.490+#[no_mangle]
30.491+pub extern "C" fn dysk_plus(a:c_int,b:c_int) -> c_int {a+b}
30.492+#[no_mangle]
30.493+pub extern "C" fn dysk_plus1(n:c_int) -> c_int {n+1}
30.494+#+end_src
30.495+**** test.rs
30.496+:PROPERTIES:
30.497+:ID: cc7c6538-33a6-40c6-94ef-2a9c259c975a
30.498+:END:
30.499+#+begin_src rust :tangle test.rs
30.500+//! test.rs --- dysk test
30.501+fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::dysk_plus1(2 as core::ffi::c_int);}}
30.502+#+end_src
30.503+**** compile
30.504+:PROPERTIES:
30.505+:ID: 337a24d1-f305-4e1a-9052-47a53591cb2f
30.506+:END:
30.507+#+begin_src sh
30.508+cargo build --release
30.509+#+end_src
30.510+**** load from SBCL
30.511+:PROPERTIES:
30.512+:ID: a4813269-92fb-4f52-aef0-3a36dce3cf69
30.513+:END:
30.514+#+begin_src lisp :tangle dysk.lisp
30.515+(load-shared-object #P"target/release/libdysk.so")
30.516+(define-alien-routine dysk-hello c-string)
30.517+(define-alien-routine dysk-plus int (a int) (b int))
30.518+(define-alien-routine dysk-plus1 int (n int))
30.519+(dysk-hello) ;; => "hello from rust"
30.520+#+end_src
30.521+**** benchmark
30.522+:PROPERTIES:
30.523+:ID: 1a8ca441-f290-46c7-b979-1e7e0d1d063b
30.524+:END:
30.525+#+begin_src shell
30.526+time target/release/dysk-test
30.527+#+end_src
30.528+#+begin_src lisp :tangle test.lisp
30.529+(time (dotimes (_ 500000000) (dysk-plus1 2)))
30.530+#+end_src
30.531+* [2023-11-24 Fri]
30.532+** cl-dot examples
30.533+:PROPERTIES:
30.534+:ID: c5f4f25c-071b-4a2d-85bc-08676eacce5d
30.535+:END:
30.536+#+begin_src lisp
30.537+(defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object cons))
30.538+ (make-instance 'cl-dot:node
30.539+ :attributes '(:label "cell \\N"
30.540+ :shape :box)))
30.541+(defmethod cl-dot:graph-object-points-to ((graph (eql 'example)) (object cons))
30.542+ (list (car object)
30.543+ (make-instance 'cl-dot:attributed
30.544+ :object (cdr object)
30.545+ :attributes '(:weight 3))))
30.546+;; Symbols
30.547+(defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object symbol))
30.548+ (make-instance 'cl-dot:node
30.549+ :attributes `(:label ,object
30.550+ :shape :hexagon
30.551+ :style :filled
30.552+ :color :black
30.553+ :fillcolor "#ccccff")))
30.554+(let* ((data '(a b c #1=(b z) c d #1#))
30.555+ (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
30.556+ '(:rankdir "LR" :layout "twopi" :labelloc "t"))))
30.557+ (cl-dot:dot-graph dgraph "test-lr.svg" :format #+nil :x11 :svg))
30.558+#+end_src
30.559+
30.560+#+RESULTS:
30.561+
30.562+#+begin_src lisp
30.563+(let* ((data '(a b))
30.564+ (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
30.565+ '(:rankdir "LR"))))
30.566+ (cl-dot:print-graph dgraph))
30.567+#+end_src
30.568+
30.569+* [2023-12-05 Tue]
30.570+** global refs
30.571+:PROPERTIES:
30.572+:ID: 7fb40057-05aa-445b-bc75-0b1b6ca2c994
30.573+:END:
30.574+need a way of indexing, referring to, and annotating objects such as
30.575+URLs, docs, articles, source files, etc.
30.576+
30.577+What is the best way to get this done?
30.578+* [2023-12-09 Sat]
30.579+** doc best practices
30.580+:PROPERTIES:
30.581+:ID: ba4a3074-f29d-4f81-b4ba-d6a523b3f291
30.582+:END:
30.583+https://rust-lang.github.io/api-guidelines/documentation.html
30.584+
30.585+also: https://lisp-lang.org/style-guide/
30.586+* [2023-12-12 Tue]
30.587+** On Computers
30.588+:PROPERTIES:
30.589+:ID: 7845b0fb-b5d3-4488-bdba-4876d9c8816e
30.590+:END:
30.591+If you've met me in the past decade, you probably know that I am
30.592+extremely passionate about computers. Let me first explain why.
30.593+
30.594+On the most basic level computers are little (or big) machines that
30.595+can be programmed to do things, or /compute/ if we're being
30.596+technical.[fn:1]
30.597+
30.598+They host and provide access to the Internet, which is a pretty big
30.599+thing, but they do little things too like unlock your car door and
30.600+tell your microwave to beep at you. They solve problems. Big or small.
30.601+
30.602+They're also /everywhere/ - which can be scary to think about, but
30.603+ultimately helps propel us into the future.
30.604+
30.605+There's something pretty cool about that - when you look at the
30.606+essence of computation. There are endless quantities of these machines
30.607+which follow the same basic rules and can be used to solve /real/
30.608+problems.
30.609+
30.610+*** The Programmer
30.611+:PROPERTIES:
30.612+:ID: b17c853e-1728-4bd3-94c5-82e5b33d7398
30.613+:END:
30.614+Now, let us consider the /programmer/. They have power. /real/
30.615+power. They understand the language of computers, can whisper to them
30.616+in various dialects. It can be intimidating to witness until you
30.617+realize how often the programmer says the wrong thing - a bug.
30.618+
30.619+In reality, the programmer has a symbiotic relationship with
30.620+computers. Good programmers understand this relationship well.
30.621+
30.622+#+begin_annecdote
30.623+One day after I got my first job at a software company, I remember
30.624+being on an all-hands meeting due to a client service outage. We had
30.625+some management, our lead devs, product team, and one curious looking
30.626+man who happened to be our lead IT consultant who had just joined. He
30.627+was sitting up on a hotel bed, shirtless, vaping an e-cig, typing
30.628+away in what I can only imagine was a shell prompt.
30.629+
30.630+After several minutes he took a swig from a bottle of Coke and said
30.631+"Node 6 is sick." then a few seconds later our services were
30.632+restored. For the next hour on the call he explained what happened and
30.633+why, but that particular phrase always stuck with me. He didn't say
30.634+Node 6 was down, or had an expired cert - his diagnosis was that /it/
30.635+was /sick/.
30.636+#+end_annecdote
30.637+
30.638+The more you work closely with computers, the more you start to think
30.639+of them this way. You don't start screaming when the computer does the
30.640+wrong thing, you figure out what's wrong and learn from it. With
30.641+experience, you start to understand the different behaviors of the
30.642+machines you work with. I like to call this /Machine Empathy/.
30.643+
30.644+*** Programs
30.645+:PROPERTIES:
30.646+:ID: a297e5f9-a875-4512-b126-9a2b3e75c1d8
30.647+:END:
30.648+I already mentioned bugs - I write plenty of those, but usually I try
30.649+to write /programs/. Programs to me are like poetry. I like to think
30.650+they are for the computer too.
30.651+
30.652+Just like computers, /computer programs/ come in different shapes and
30.653+sizes but in basic terms they are sets of instructions used to control
30.654+a computer.
30.655+
30.656+You can write programs to do anything - when I first started, my
30.657+programs made music. The program was a means to an end. Over time, I
30.658+started to see the program as something much more. I saw it as the
30.659+music itself.
30.660+
30.661+[fn:1] ... perform computations
30.662+
30.663+** On Infra
30.664+:PROPERTIES:
30.665+:ID: 80d800de-d305-4b95-a28c-a6f19d6f7a11
30.666+:END:
30.667+Something that is missing from many organizations big or large, is an
30.668+effective way to store and access information, even about their own
30.669+org.
30.670+
30.671+It can be difficult problem to solve - usually there's the official
30.672+one, say Microsoft Sharepoint and then the list of unofficial sources
30.673+which becomes tribal corporate hacker knowledge. Maybe the unofficial
30.674+ones are more current, or are annotated nicely, but their very
30.675+existence breaks the system. There's no longer a single source of
30.676+truth.
30.677+
30.678+My priority in this department is writing services which process and
30.679+store information from a variety of sources in a distributed knowledge
30.680+graph. The graph can later be queried to access information on-demand.
30.681+
30.682+My idea of infrastructure is in fact to build my own Cloud. Needless
30.683+to say I don't have an O365 subscription, and wherever possible I'll
30.684+be relying on hardware I have physical access to. I'm not opposed to
30.685+cloud services at large but based on principle I like to think we
30.686+shouldn't be built on them.
30.687+* [2023-12-23 Sat]
30.688+** https://cal-coop.gitlab.io/utena/utena-specification/main.pdf
30.689+:PROPERTIES:
30.690+:ID: 5c3aac3a-67e1-4867-9ac4-b58a3247f9e0
30.691+:END:
30.692+from the author of cl-decentralise2. draft specification of a
30.693+/Maximalist/ Computing System.
30.694+* [2023-12-24 Sun]
30.695+** public datasets
30.696+:PROPERTIES:
30.697+:ID: 28738250-c34e-45e2-8793-f8245472d3a3
30.698+:END:
30.699+https://github.com/awesomedata/awesome-public-datasets
30.700+https://docs.openml.org/Datasets/
30.701+https://wiki.pathmind.com/open-datasets
30.702+* [2023-12-28 Thu]
30.703+** useful internals
30.704+:PROPERTIES:
30.705+:ID: 28581a8e-ae05-45de-9744-24f447668418
30.706+:END:
30.707+#+begin_src lisp
30.708+ sb-sys:*runtime-dlhandle*
30.709+ sb-fasl:+fasl-file-version+
30.710+ sb-fasl:+backend-fasl-file-implementation+
30.711+ sb-debug:print-backtrace
30.712+ sb-debug:map-backtrace
30.713+ sb-pretty:pprint-dispatch-table
30.714+ sb-lockless:
30.715+ sb-ext:simd-pack
30.716+ sb-walker:define-walker-template
30.717+ sb-walker:macroexpand-all
30.718+ sb-walker:walk-form
30.719+ sb-kernel:empty-type
30.720+ sb-kernel:*eval-calls*
30.721+ sb-kernel:*gc-pin-code-pages*
30.722+ sb-kernel:*restart-clusters*
30.723+ sb-kernel:*save-lisp-clobbered-globals*
30.724+ sb-kernel:*top-level-form-p*
30.725+ sb-kernel:*universal-fun-type*
30.726+ sb-kernel:*universal-type*
30.727+ sb-kernel:*wild-type*
30.728+ sb-kernel:+simd-pack-element-types+
30.729+ (sb-vm:memory-usage)
30.730+ (sb-vm:boxed-context-register)
30.731+ (sb-vm:c-find-heap->arena)
30.732+ (sb-vm:copy-number-to-heap)
30.733+ (sb-vm:dump-arena-objects)
30.734+ (sb-vm:fixnumize)
30.735+ (sb-vm:rewind-arena)
30.736+ (sb-vm:show-heap->arena)
30.737+ (sb-vm:with/without-arena)
30.738+ (sb-cltl2:{augment-environment,compiler-let,define-declaration,parse-macro})
30.739+ (sb-cltl2:{declaration-information, variable-information, function-information})
30.740+ sb-di:
30.741+ sb-assem:
30.742+ sb-md5:
30.743+ sb-regalloc:
30.744+ sb-disassem:
30.745+#+end_src
30.746+* [2024-01-03 Wed]
30.747+** [[https://github.com/sigmf/SigMF][SigMF]]
30.748+:PROPERTIES:
30.749+:ID: f21a6477-6a3f-48f7-8a9e-43f5cf65286c
30.750+:END:
30.751+#+begin_quote
30.752+Sharing sets of recorded signal data is an important part of science
30.753+and engineering. It enables multiple parties to collaborate, is often
30.754+a necessary part of reproducing scientific results (a requirement of
30.755+scientific rigor), and enables sharing data with those who do not have
30.756+direct access to the equipment required to capture it.
30.757+
30.758+Unfortunately, these datasets have historically not been very
30.759+portable, and there is not an agreed upon method of sharing metadata
30.760+descriptions of the recorded data itself. This is the problem that
30.761+SigMF solves.
30.762+
30.763+By providing a standard way to describe data recordings, SigMF
30.764+facilitates the sharing of data, prevents the "bitrot" of datasets
30.765+wherein details of the capture are lost over time, and makes it
30.766+possible for different tools to operate on the same dataset, thus
30.767+enabling data portability between tools and workflows.
30.768+#+end_quote
30.769+
30.770+the-spec: https://github.com/sigmf/SigMF/blob/sigmf-v1.x/sigmf-spec.md
30.771+** [[https://www.libvolk.org/][LibVOLK]]
30.772+:PROPERTIES:
30.773+:ID: ae34135c-488e-4aff-b795-a4d0d558f22b
30.774+:END:
30.775+Vector-Optimized Library of Kernels (simd)
30.776+** [[https://docs.kernel.org/fb/framebuffer.html][/dev/fb*]]
30.777+:PROPERTIES:
30.778+:ID: 7819a970-1f37-40a6-985f-905b25a08a4a
30.779+:END:
30.780+framebuffers, used by fbgrab/fbcat program
30.781+** [[https://docs.kernel.org/block/ublk.html][ublk]]
30.782+:PROPERTIES:
30.783+:ID: 5ccaed39-7055-4a51-ab62-769a73425270
30.784+:END:
30.785+https://github.com/ming1/ubdsrv
30.786+* [2024-01-04 Thu]
30.787+goals:
30.788+make problems smaller.
30.789+
30.790+sections:
30.791+why lisp?
30.792+- doesn't need mentioning more and more
30.793+* [2024-01-20 Sat]
30.794+** TODO taobench demo
30.795+:PROPERTIES:
30.796+:ID: 81ac81c3-f183-4345-aaf3-ef7ef0434947
30.797+:END:
30.798+:LOGBOOK:
30.799+- State "TODO" from [2024-01-21 Sun 00:32]
30.800+:END:
30.801+https://github.com/audreyccheng/taobench - shouldn't have missed this :)
30.802+obviously we need to implement this using core -- in demo/bench/tao?
30.803+** TODO clap completion for nushell
30.804+:PROPERTIES:
30.805+:ID: a4b35281-83f7-4244-93e9-20e57ba31eb3
30.806+:END:
30.807+:LOGBOOK:
30.808+- State "TODO" from [2024-01-20 Sat 23:23]
30.809+:END:
30.810+https://github.com/clap-rs/clap/tree/master/clap_complete_nushell
30.811+** Dataframe scripting
30.812+:PROPERTIES:
30.813+:ID: 8dc7339e-71c6-4a60-b285-c0d56d1cbc24
30.814+:END:
30.815+https://studioterabyte.nl/en/blog/polars-vs-pandas
30.816+nushell supports DFs, polars underneath?
30.817+https://www.nushell.sh/book/cheat_sheet.html
30.818+
30.819+#+begin_src nushell
30.820+
30.821+#+end_src
30.822+** Cloud Squatting
30.823+:PROPERTIES:
30.824+:ID: d5131c54-9082-4424-91dd-70f17561bbf7
30.825+:END:
30.826+*** Google
30.827+:PROPERTIES:
30.828+:ID: 2b0f67de-1217-48a6-b5b7-a85d3ca15926
30.829+:END:
30.830+- [[https://cloud.google.com/free/docs/free-cloud-features][Free Cloud Features]]
30.831+ + 90-day $300 credits
30.832+ + e2-micro - free hours worth 1 instance/month
30.833+ +
30.834+*** Amazon
30.835+:PROPERTIES:
30.836+:ID: 0fd0dedd-59b0-4b1a-9241-f65699e20c46
30.837+:END:
30.838+- AWS Free Tier
30.839+*** Akamai
30.840+:PROPERTIES:
30.841+:ID: 8315a33f-dd22-4374-979a-9cccd5b5b6a9
30.842+:END:
30.843+- Linode Free Trial
30.844+*** Oracle
30.845+:PROPERTIES:
30.846+:ID: 3f1e1116-c123-4fdf-a336-0d9b50c3dcc7
30.847+:END:
30.848+- [[https://www.oracle.com/cloud/free/?intcmp=ohp052322ocift][OCI Free Tier]]
30.849+ + always free: 2 x oracle autonomous DB
30.850+ + 2 x AMD Compute VMs
30.851+ + up to 4 x ARM Ampere A1 with 3k/cpu/hr and 18k/gb/h per month
30.852+ + block/object/archive storage
30.853+ + 30-day $300 credits
30.854+* [2024-01-29 Mon]
30.855+** trash as block device
30.856+:PROPERTIES:
30.857+:ID: 4452b84f-f3a5-44f1-b60a-4efa6413cbb9
30.858+:END:
30.859+:LOGBOOK:
30.860+- State "NOTE" from [2024-01-29 Mon 20:53]
30.861+- State "NOTE" from [2024-01-29 Mon 20:53]
30.862+:END:
30.863+in nushell there is option for rm command to always use 'trash' -
30.864+AFAIK the current approach is via a service (trashd).
30.865+
30.866+An interesting experiment would be to designate a block device as
30.867+'trash' - may be possible to remove reliance on a service
30.868+
30.869+may be an opportunity for ublk driver to shine - instead of /dev/null
30.870+piping we need a driver for streaming a file to /dev/trash
30.871+** compute power
30.872+:PROPERTIES:
30.873+:ID: aa10a1d7-f1e7-4b92-b91a-56c234d52204
30.874+:END:
30.875+:LOGBOOK:
30.876+- State "NOTE" from [2024-01-29 Mon 16:28]
30.877+:END:
30.878+- mostly x86_64 machines - currently 2 AWS EC2 instances, some podman containers, and our home beowulf server:
30.879+- beowulf:
30.880+ - Zor
30.881+ - mid-size tower enclosed (Linux/Windows)
30.882+ - CPU
30.883+ - Intel Core i7-6700K
30.884+ - 4 @ 4.0
30.885+ - GPU
30.886+ - NVIDIA GeForce GTX 1060
30.887+ - 6GB
30.888+ - Storage
30.889+ - Samsung SSD 850: 232.9GB
30.890+ - Samsung SSD 850: 465.76GB
30.891+ - ST2000DM001-1ER1: 1.82TB
30.892+ - WDC WD80EAZZ-00B: 7.28TB
30.893+ - PSSD T7 Shield: 3.64TB
30.894+ - My Passport 0820: 1.36TB
30.895+ - RAM
30.896+ - 16GB (2*8) [64GB max]
30.897+ - DDR4
30.898+ - Jekyll
30.899+ - MacBook Pro 2019 (MacOS/Darwin)
30.900+ - CPU
30.901+ - Intel
30.902+ - 8 @
30.903+ - RAM
30.904+ - 32G DDR4
30.905+ - Hyde
30.906+ - Thinkpad
30.907+ - CPU
30.908+ - Intel
30.909+ - 4 @
30.910+ - RAM
30.911+ - 24G DDR3
30.912+ - Boris
30.913+ - Pinephone Pro
30.914+ - CPU
30.915+ - 64-bit 6-core 4x ARM Cortex A53 + 2x ARM Cortex A72
30.916+ - GPU
30.917+ - Mali T860MP4
30.918+ - RAM
30.919+ - 4GB LPDDR4
30.920+ - pi
30.921+ - Raspberry Pi 4 Model B
30.922+ - CPU
30.923+ - Cortex-A72 (ARM v8) 64-bit SoC
30.924+ - 4 @ 1.8GHz
30.925+ - RAM
30.926+ - 8 GB
30.927+ - DDR4 4200
30.928+* [2024-02-10 Sat]
30.929+** BigBenches
30.930+:PROPERTIES:
30.931+:ID: 87d0f5d4-c6fe-4e21-911e-2639a24e89f9
30.932+:END:
30.933+#+name: 1trc-nu
30.934+#+begin_src nushell
30.935+ let ms = '1trc/measurements-0.parquet'
30.936+ dfr open $ms
30.937+ | dfr group-by station
30.938+ | dfr agg [
30.939+ (dfr col measure | dfr min | dfr as "min")
30.940+ (dfr col measure | dfr max | dfr as "max")
30.941+ (dfr col measure | dfr sum | dfr as "sum")
30.942+ (dfr col measure | dfr count | dfr as "count")
30.943+ ]
30.944+#+end_src
30.945+* [2024-02-18 Sun]
30.946+** WL vs X
30.947+:PROPERTIES:
30.948+:ID: b4b29989-28f2-48db-ad84-cd8d08eeefdf
30.949+:END:
30.950+:LOGBOOK:
30.951+- State "NOTE" from [2024-02-18 Sun 11:55]
30.952+:END:
30.953+In the past few months there has been drama regarding Wayland vs X. It
30.954+seems to be on everyone's minds after Artem's freakout issue and the
30.955+follow up YT vids/comments.
30.956+
30.957+I admit that it made me reconsider the fitness of WL as a whole -
30.958+there was a github gist that made some scathing arguments against it.
30.959+
30.960+It's an odd debate though. I think there are many misunderstandings.
30.961+
30.962+So first off, if we look at the homepage
30.963+https://wayland.freedesktop.org/, Wayland claims it is a replacement
30.964+for X11. It now has /manifest destiny/, which in my opinion is a great
30.965+shame.
30.966+
30.967+X-pros seem to agree that Wayland has /manifest destiny/ - like if you
30.968+are building softwares that look remotely like a window system, it's a
30.969+successor to X. That's the model of doing things and there's no way
30.970+around it.
30.971+
30.972+The disagreement starts with how this destiny - of an X2 - should be
30.973+fulfilled. X-pros want a fork of X, but it's too late for
30.974+that. WL-pros want X to run on top of Wayland compositor:
30.975+https://wayland.freedesktop.org/xserver.html.
30.976+
30.977+Xwayland is a problem for me. From the project description: 'if we're
30.978+migrating away from X, it makes sense to have a good backwards
30.979+compatibility story.' Full disclosure: I have never done significant
30.980+work on Xwayland, so perhaps my opinion is unwarranted. But I have no
30.981+intention of attempting to maintain a computer system that uses
30.982+Wayland and X clients at the same time.
30.983+
30.984+To me, X is ol' reliable. Every distro has first-class X support, and
30.985+it runs on most systems with very little user intervention. Where it
30.986+doesn't, there is 20+ years of dev history and battle-tested
30.987+workarounds for you to find your solution in.
30.988+
30.989+Wayland is the new kid on the block, born just in 2008. It's a fresh
30.990+start to one of the most difficult challenges in software - window
30.991+systems. A re-write would be pointless though, and so the real
30.992+value-add is in design. Wayland is designed as a protocol and
30.993+collection of libraries which are implemented in your own
30.994+compositor. Coming from Lisp - with ANSI Common Lisp and SRFIs, this
30.995+feels right even if the implementation is something very different
30.996+(compositor vs compiler).
30.997+
30.998+With X, it is assumed to be much harder to write an equivalent
30.999+'compositor'. Here's the thing though - with a significantly complex X
30.1000+client implementation, it is /impossible/ to replicate in WL. This is
30.1001+really the crux of Artemi's argument in his issue. He asked for a 1:1
30.1002+equivalent X/WL comparison when no such thing exists, and in my
30.1003+opinion it is a waste of time.
30.1004+
30.1005+The WL core team is fully aware of this dichotomy, but also that this
30.1006+is in no way a problem or weakness in either system. It means they're
30.1007+different systems, goddammit.
30.1008+
30.1009+If it was up to me, Xwayland wouldn't exist. I understand why it does,
30.1010+and that it does make things easier for developers who need to support
30.1011+both, and users who have multiple apps with multiple windowing
30.1012+requirements. It's a bandaid though, and one that is particularly
30.1013+dangerous because it re-enforces the idea that Wayland is just X2 and
30.1014+that they're fully compatible.
30.1015+
30.1016+What interests me in the Wayland world right now is the idea of a
30.1017+small, modular, full-stack Wayland compositor API. There are several
30.1018+'kiosk' based compositors for single applications (cage), but these
30.1019+aren't complete solutions. It is possible to get much closer to the
30.1020+metal, and that's where I want to be so that I can build my own APIs
30.1021+on top - I don't want to live on top of X, and I certainly don't want
30.1022+to live on top of X on top of WL. I want a /pure/ solution that hides
30.1023+as little as possible, exposing the interesting bits.
30.1024+* [2024-03-01 Fri]
30.1025+** TODO collect more data
30.1026+:PROPERTIES:
30.1027+:ID: 69ce5703-d3fe-4df7-abf4-dc93a08ebc12
30.1028+:END:
30.1029+:LOGBOOK:
30.1030+- State "TODO" from [2024-03-01 Fri 15:27]
30.1031+:END:
30.1032+https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
30.1033+weather - music - etc
30.1034+* [2024-03-02 Sat]
30.1035+** On blocks and devices
30.1036+:PROPERTIES:
30.1037+:ID: c8304ae3-415d-4ccf-8f4c-1f3f6a555e5b
30.1038+:END:
30.1039+:LOGBOOK:
30.1040+- State "NOTE" from [2024-03-02 Sat 21:30]
30.1041+:END:
30.1042+[[https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/dev.html][/dev]]
30.1043+In Linux, everything is a file.
30.1044+
30.1045+/dev/ contains special device files - usually block or character
30.1046+device.
30.1047+
30.1048+major, minor = category, device
30.1049+0, 5
30.1050+
30.1051+mknod - create special device files
30.1052+
30.1053+[[https://www.redhat.com/en/blog/linux-block-devices-hints-debugging-and-new-developments][redhat hints]]
30.1054+
30.1055+#+begin_src shell
30.1056+ dd if=/dev/zero of=myfile bs=1M count=32
30.1057+ losetup --show -f myfile
30.1058+ ls -al /dev/loop0
30.1059+ losetup -d /dev/loop0 #teardown
30.1060+#+end_src
30.1061+
30.1062+#+begin_src shell
30.1063+ echo "sup dude" > /dev/loop0
30.1064+ dd if=/dev/loop0 -bs=1
30.1065+ dd if=/dev/nvme0 of=/dev/null progress=true
30.1066+ #pacman -S hdparm
30.1067+ hdparm -T /dev/nvme0
30.1068+#+end_src
30.1069+
30.1070+#+begin_src shell
30.1071+modprobe scsi_debug add_host=5 max_luns=10 num_tgts=2 dev_size_mb=16
30.1072+#+end_src
30.1073+
30.1074+sparsefiles: create with C, dd, or truncate
30.1075+
30.1076+#+begin_src shell :results output
30.1077+truncate --help
30.1078+#+end_src
30.1079+
30.1080+#+RESULTS:
30.1081+#+begin_example
30.1082+Usage: truncate OPTION... FILE...
30.1083+Shrink or extend the size of each FILE to the specified size
30.1084+
30.1085+A FILE argument that does not exist is created.
30.1086+
30.1087+If a FILE is larger than the specified size, the extra data is lost.
30.1088+If a FILE is shorter, it is extended and the sparse extended part (hole)
30.1089+reads as zero bytes.
30.1090+
30.1091+Mandatory arguments to long options are mandatory for short options too.
30.1092+ -c, --no-create do not create any files
30.1093+ -o, --io-blocks treat SIZE as number of IO blocks instead of bytes
30.1094+ -r, --reference=RFILE base size on RFILE
30.1095+ -s, --size=SIZE set or adjust the file size by SIZE bytes
30.1096+ --help display this help and exit
30.1097+ --version output version information and exit
30.1098+
30.1099+The SIZE argument is an integer and optional unit (example: 10K is 10*1024).
30.1100+Units are K,M,G,T,P,E,Z,Y,R,Q (powers of 1024) or KB,MB,... (powers of 1000).
30.1101+Binary prefixes can be used, too: KiB=K, MiB=M, and so on.
30.1102+
30.1103+SIZE may also be prefixed by one of the following modifying characters:
30.1104+'+' extend by, '-' reduce by, '<' at most, '>' at least,
30.1105+'/' round down to multiple of, '%' round up to multiple of.
30.1106+
30.1107+GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
30.1108+Full documentation <https://www.gnu.org/software/coreutils/truncate>
30.1109+or available locally via: info '(coreutils) truncate invocation'
30.1110+#+end_example
30.1111+
30.1112+test mkfs.btrfs on 10T dummy block device
30.1113+#+begin_src shell
30.1114+ dd if=/dev/zero of=/tmp/bb1 bs=1 count=1 seek=10T
30.1115+ du -sh /tmp/bb1
30.1116+ losetup --show -f /tmp/bb1
30.1117+ mkfs.btrfs /dev/loop0
30.1118+#+end_src
30.1119+
30.1120+diagnostics
30.1121+#+begin_src shell
30.1122+ iostat # pacman -S sysstat
30.1123+ blktrace # paru -S blktrace
30.1124+ iotop # pacman -S iotop
30.1125+#+end_src
30.1126+
30.1127+bcc/ trace: Who/which process is executing specific functions against
30.1128+block devices?
30.1129+
30.1130+bcc/biosnoop: Which process is accessing the block device, how many
30.1131+bytes are accessed, which latency for answering the requests?
30.1132+
30.1133+at the kernel level besides BPF we got kmods and DKMS,
30.1134+
30.1135+compression/de-duplication can be done via VDO kernel mod
30.1136+
30.1137+https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support
30.1138+
30.1139+** save-lisp-and-respawn
30.1140+:PROPERTIES:
30.1141+:ID: e58a96e7-7b88-4e4a-b958-3000603d8baa
30.1142+:END:
30.1143+:LOGBOOK:
30.1144+- State "NOTE" from [2024-03-02 Sat 22:57]
30.1145+:END:
30.1146+#+begin_src lisp
30.1147+ sb-ext:*save-hooks*
30.1148+#+end_src
30.1149+
30.1150+** syslog for log
30.1151+:PROPERTIES:
30.1152+:ID: 2efa5e4a-f063-49bb-9e61-6917eb115c1a
30.1153+:END:
30.1154+:LOGBOOK:
30.1155+- State "NOTE" from [2024-03-03 Sun 16:35]
30.1156+:END:
30.1157+sb-posix:
30.1158+- openlog syslog closelog
30.1159+- levels: emerg alert crit err warning notice info debug
30.1160+- setlogmask
30.1161+* [2024-03-13 Wed]
30.1162+** RESEARCH [[https://github.com/guicho271828/sbcl-wiki/wiki/][sbcl-wiki]]
30.1163+:PROPERTIES:
30.1164+:ID: 33bc81c1-c5fa-4518-9456-58fd67f301ff
30.1165+:END:
30.1166+:LOGBOOK:
30.1167+- State "RESEARCH" from [2024-03-13 Wed 21:49]
30.1168+:END:
30.1169+** IR1
30.1170+:PROPERTIES:
30.1171+:ID: 2c012572-ea16-408c-b741-d1f6cd0c7961
30.1172+:END:
30.1173+** IR2
30.1174+:PROPERTIES:
30.1175+:ID: 8d015e5e-7ad3-44c3-88cd-81e293a887e0
30.1176+:END:
30.1177+* [2024-03-17 Sun]
30.1178+** DB Benchmarking
30.1179+:PROPERTIES:
30.1180+:ID: d30aec42-a9a1-425b-87ca-1e40db0e69de
30.1181+:END:
30.1182+:LOGBOOK:
30.1183+- State "NOTE" from [2024-02-04 Sun 20:40]
30.1184+:END:
30.1185+[[https://github.com/facebook/rocksdb/wiki/Benchmarking-tools][RocksDB benchmarking tools]]
30.1186+
30.1187+** packy design
30.1188+:PROPERTIES:
30.1189+:ID: 76ae24f5-46e8-4b91-8991-41245383d337
30.1190+:END:
30.1191+:LOGBOOK:
30.1192+- State "NOTE" from [2024-01-25 Thu 22:39]
30.1193+:END:
30.1194+- API root: https://packy.compiler.company
30.1195+- source packs: https://vc.compiler.company/packy
30.1196+*** Lib
30.1197+:PROPERTIES:
30.1198+:ID: 2d6aa6fb-a34c-4ccd-99d3-4f98d33bb2f9
30.1199+:END:
30.1200+**** Types
30.1201+:PROPERTIES:
30.1202+:ID: 8017b6c7-3a04-4e52-b3d6-afc68cbce139
30.1203+:END:
30.1204+***** Pack
30.1205+:PROPERTIES:
30.1206+:ID: 2722b7dc-9546-4d94-82b0-abfc02647b4d
30.1207+:END:
30.1208+Primary data type of the library - typically represents a compressed
30.1209+archive, metadata, and ops.
30.1210+***** Bundle
30.1211+:PROPERTIES:
30.1212+:ID: 9ec5d682-da8e-4d02-b2fc-a56a8f209c22
30.1213+:END:
30.1214+Collection data type, usually contains a set of packs with metadata.
30.1215+***** PackyEndpoint
30.1216+:PROPERTIES:
30.1217+:ID: 8cf2658a-92d5-41ab-8630-4f4e6ddb140d
30.1218+:END:
30.1219+Represents a Packy instance bound to a UDP socket
30.1220+***** PackyEndpointConfig
30.1221+:PROPERTIES:
30.1222+:ID: 421b8aaa-66e1-4883-937d-fc6705146998
30.1223+:END:
30.1224+Global endpoint configuration object
30.1225+***** PackyClientConfig
30.1226+:PROPERTIES:
30.1227+:ID: c1899710-034e-4ed6-959a-1eb2df72e5f9
30.1228+:END:
30.1229+Configuration for outgoing packy connections on an endpoint
30.1230+***** PackyServerConfig
30.1231+:PROPERTIES:
30.1232+:ID: 955e8ce9-ff7e-4f42-9d25-3aeb9846528b
30.1233+:END:
30.1234+Configuration for incoming packy connection son an endpoint
30.1235+***** PackyConnection
30.1236+:PROPERTIES:
30.1237+:ID: 7075e0c8-65b3-4fbf-9a6b-f27017276a72
30.1238+:END:
30.1239+Packy connection object
30.1240+**** Traits
30.1241+:PROPERTIES:
30.1242+:ID: ea92ad07-d12c-4ee8-a3d4-0955f0af3921
30.1243+:END:
30.1244+***** PackyClient
30.1245+:PROPERTIES:
30.1246+:ID: 6c0fdd7e-bc0a-4217-b315-72fe7766e4df
30.1247+:END:
30.1248+****** query
30.1249+:PROPERTIES:
30.1250+:ID: 1358e1c5-b215-48eb-9e1b-4116cc32b2ae
30.1251+:END:
30.1252+****** install
30.1253+:PROPERTIES:
30.1254+:ID: b8298207-4d92-4628-8687-9f4e20c8cfec
30.1255+:END:
30.1256+****** update
30.1257+:PROPERTIES:
30.1258+:ID: a6fb4fb6-b188-4033-8291-107cf1f00e6d
30.1259+:END:
30.1260+****** login
30.1261+:PROPERTIES:
30.1262+:ID: 1fd91c8b-ab90-4db0-b74e-1c2b9ae2e440
30.1263+:END:
30.1264+****** logout
30.1265+:PROPERTIES:
30.1266+:ID: 75c0db2e-37cf-4f33-95bd-3f5c95b45c49
30.1267+:END:
30.1268+****** pull
30.1269+:PROPERTIES:
30.1270+:ID: 60de92f4-b4a6-4cae-95bc-174e2036e57b
30.1271+:END:
30.1272+****** push
30.1273+:PROPERTIES:
30.1274+:ID: e379b71e-8a84-4547-a694-90cad3b7952e
30.1275+:END:
30.1276+***** PackyServer
30.1277+:PROPERTIES:
30.1278+:ID: 73ea9478-5251-40c3-8331-cf042d0b9bf1
30.1279+:END:
30.1280+****** start_packy_server
30.1281+:PROPERTIES:
30.1282+:ID: c8a9bac3-971a-45a7-988f-92db19ef6601
30.1283+:END:
30.1284+****** stop_packy_server
30.1285+:PROPERTIES:
30.1286+:ID: e3122bf1-f770-46a7-a36b-2df9c48bc233
30.1287+:END:
30.1288+****** start_packy_registry
30.1289+:PROPERTIES:
30.1290+:ID: b6d214f2-d6de-4d46-b2c2-c51ff0f3b3e6
30.1291+:END:
30.1292+***** PackyRegistry
30.1293+:PROPERTIES:
30.1294+:ID: 0e846cbb-5cb4-496f-8100-beae6e66972e
30.1295+:END:
30.1296+****** register_pack
30.1297+:PROPERTIES:
30.1298+:ID: 764642ad-a33b-415b-9c32-8a729a7cd62d
30.1299+:END:
30.1300+****** register_user
30.1301+:PROPERTIES:
30.1302+:ID: 2b92f584-d769-4266-940e-cfbd466d4e62
30.1303+:END:
30.1304+****** register_bundle
30.1305+:PROPERTIES:
30.1306+:ID: 390cb42b-5170-4175-9ccc-b990842356a0
30.1307+:END:
30.1308+* [2024-03-25 Mon]
30.1309+** TBD investigate alieneval for phash opps
30.1310+:PROPERTIES:
30.1311+:ID: 21ab0a6e-cb66-4c5e-9b10-4ac2340ce75d
30.1312+:END:
30.1313+:LOGBOOK:
30.1314+- State "TBD" from [2024-03-25 Mon 18:56]
30.1315+:END:
30.1316+* [2024-04-19 Fri]
30.1317+** How it works
30.1318+:PROPERTIES:
30.1319+:ID: 439a4945-de2f-4985-ba6c-fe6421aa9ad9
30.1320+:END:
30.1321+The backend services are written in Rust and controlled by a simple
30.1322+messaging protocol. Services provide common runtime capabilities known
30.1323+as the /service protocol/ but are specialized on a unique /service
30.1324+type/ which may in turn register their own /custom protocols/ (via
30.1325+core).
30.1326+
30.1327+Services are capable of dispatching data directly to clients, or
30.1328+storing data in the /database/ (sqlite, postgres, mysql).
30.1329+
30.1330+The frontend clients are pre-dominantly written in Common Lisp and
30.1331+come in many shapes and sizes. There is a cli-client, web-client
30.1332+(CLOG), docker-client (archlinux, stumpwm, McCLIM), and native-client
30.1333+which also compiles to WASM (slint-rs).
30.1334+
30.1335+** Guide
30.1336+:PROPERTIES:
30.1337+:ID: 65140d2c-5d59-48cd-946c-01c380a37f08
30.1338+:END:
30.1339+*** Build
30.1340+:PROPERTIES:
30.1341+:ID: ad0652ca-4abf-4d27-8897-a61a13a6e61d
30.1342+:END:
30.1343+- *install dependencies*
30.1344+ #+begin_src bash
30.1345+ ./tools/deps.sh
30.1346+ #+end_src
30.1347+- *make executables* \\
30.1348+ Simply run =make build=. Read the ~makefile~ and change the options
30.1349+ as needed.
30.1350+- MODE :: Mode (debug, release)
30.1351+- LISP :: Lisp (sbcl, cmucl, ccl)
30.1352+- CFG :: Config (default.cfg)
30.1353+*** Run
30.1354+:PROPERTIES:
30.1355+:ID: b2295442-ccb7-4a5c-8a9e-ea8502aad636
30.1356+:END:
30.1357+#+begin_src shell
30.1358+ ./demo -i
30.1359+#+end_src
30.1360+*** Config
30.1361+:PROPERTIES:
30.1362+:ID: 600be170-d032-4280-a7b5-c46e3c157206
30.1363+:END:
30.1364+Configs can be specified in JSON, TOML, RON, or of course SEXP. See
30.1365+=default.cfg= for an example.
30.1366+*** Play
30.1367+:PROPERTIES:
30.1368+:ID: 56a69ed7-2eb3-4490-95a1-072cee985df5
30.1369+:END:
30.1370+The high-level user interface is presented as a multi-modal GUI
30.1371+application which adapts to the specific application /instances/
30.1372+below.
30.1373+**** Weather
30.1374+:PROPERTIES:
30.1375+:ID: 804a174b-534c-4a04-966a-b44c0a86fbd4
30.1376+:END:
30.1377+This backend retrieves weather data using the NWS API.
30.1378+**** Stocks
30.1379+:PROPERTIES:
30.1380+:ID: 4e7339bb-aaf3-4fab-90e4-05e13f24f881
30.1381+:END:
30.1382+The 'Stocks' backend features a stock ticker with real-time analysis
30.1383+capabilities.
30.1384+**** Bench
30.1385+:PROPERTIES:
30.1386+:ID: e3cf0add-6ded-4c05-a05d-7ef463b85b08
30.1387+:END:
30.1388+This is a benchmark backend for testing the capabilities of our
30.1389+demo. It spins up some mock services and allows fine-grained control
30.1390+of input/throughput.
30.1391+** tasks
30.1392+:PROPERTIES:
30.1393+:ID: a16a3807-0e06-4c8d-baf6-5dfa8f8f5585
30.1394+:END:
30.1395+*** TODO DSLs
30.1396+:PROPERTIES:
30.1397+:ID: 3ae4754e-0c0f-4923-9e53-ae962b8638a1
30.1398+:END:
30.1399+- consider tree-sitter parsing layout, use as a guide for developing a
30.1400+ single syntax which expands to Rust or C.
30.1401+- with-rs
30.1402+- with-c
30.1403+- with-rs/c
30.1404+- with-cargo
30.1405+- compile-rs/c
30.1406+**** TODO rs-macroexpand
30.1407+:PROPERTIES:
30.1408+:ID: f612975f-6bcc-4aab-b0d8-65b8ff0a0d0f
30.1409+:END:
30.1410+- rs-gen-file
30.1411+- rs-defmacro
30.1412+- rs-macros
30.1413+- rs-macroexpand
30.1414+- rs-macroexpand-1
30.1415+**** TODO c-macroexpand
30.1416+:PROPERTIES:
30.1417+:ID: 49f5c224-026d-4cdc-a326-1742e64c4279
30.1418+:END:
30.1419+- c-gen-file h/c
30.1420+- c-defmacro
30.1421+- c-macros
30.1422+- c-macroexpand
30.1423+- c-macroexpand-1
30.1424+**** TODO slint-macroexpand
30.1425+:PROPERTIES:
30.1426+:ID: 486beb12-179b-4acc-8507-cdc498edb628
30.1427+:END:
30.1428+- slint-gen-file
30.1429+- slint-defmacro
30.1430+- slint-macros
30.1431+- slint-macroexpand
30.1432+- slint-macroexpand-1
30.1433+**** TODO html (using who)
30.1434+:PROPERTIES:
30.1435+:ID: 2e81a19c-874c-4fd7-be0f-f6cd62c660db
30.1436+:END:
30.1437+*** TODO web templates
30.1438+:PROPERTIES:
30.1439+:ID: 95f7d6ac-c010-4d1d-a47d-1693bb7d6eab
30.1440+:END:
30.1441+create a basic static page in CL which will be used to host Slint UIs
30.1442+and other WASM doo-dads in a browser.
30.1443+*** TODO CLI
30.1444+:PROPERTIES:
30.1445+:ID: fc1d5364-279c-4d5f-baea-ed2c110b63f0
30.1446+:END:
30.1447+using clingon, decide on generic options and write it up
30.1448+*** TODO docs
30.1449+:PROPERTIES:
30.1450+:ID: 467e84e3-e72f-4764-8858-cc15fdc17e8a
30.1451+:END:
30.1452+work on doc generation -- Rust and CL should be accounted for.
30.1453+*** TODO tests
30.1454+:PROPERTIES:
30.1455+:ID: 70aea01c-d01e-4ddf-aaf5-c86183a7e250
30.1456+:END:
30.1457+We have none! need to make it more comfy - set up testing in all Rust
30.1458+crates and for the lisp systems.
30.1459+* [2024-04-25 Thu]
30.1460+** https://docs.gitlab.com/ee/administration/backup_restore/migrate_to_new_server.html
30.1461+:PROPERTIES:
30.1462+:ID: 264b81f4-f953-4698-9c82-d8e20b6c31f9
30.1463+:END:
30.1464+* [2024-07-31 Wed]
30.1465+** alpine builders
30.1466+:PROPERTIES:
30.1467+:ID: 635925f6-e1d8-47bc-a32f-28bca0f19d2d
30.1468+:END:
30.1469+- make sure to apk add:
30.1470+ - git, hg
30.1471+ - clang
30.1472+ - make
30.1473+ - linux-headers
30.1474+ - zstd-dev
30.1475+ - libc-dev?
30.1476+* [2024-08-04 Sun]
30.1477+** bookmarks
30.1478+:PROPERTIES:
30.1479+:ID: 6e9c6bce-0513-4628-b6f9-5c234cccdaa3
30.1480+:END:
30.1481+- How should such objects be represented within CORE?
30.1482+- skel/homer mostly
30.1483+ - already have alias
30.1484+- not sure about obj/otherwise, prob not
30.1485+* [2024-08-08 Thu]
30.1486+** Intelligent Design in Software
30.1487+:PROPERTIES:
30.1488+:ID: 1397d8bb-4848-4d9a-9dcc-ee65bed81c9e
30.1489+:END:
30.1490+- starting from a space where there are no external influences - a biome
30.1491+- answer questions regarding the nature of the software and its capabilities
30.1492+- incrementally adjust inter-dependencies
30.1493+- optimize
30.1494+- protect the biome at all costs
30.1495+- focus on composition
30.1496+- build applications
30.1497+- re-integrate lessons learned
30.1498+* [2024-08-16 Fri]
30.1499+** keys.compiler.company
30.1500+:PROPERTIES:
30.1501+:ID: 86c21790-72b5-4ea1-ba49-f7be3f65a8cc
30.1502+:END:
30.1503+:LOGBOOK:
30.1504+- State "NOTE" from [2024-08-16 Fri 22:00]
30.1505+:END:
30.1506+- public openpgp server
30.1507+- [[https://keys.compiler.company][keys.compiler.company]]
30.1508+- https://keys.openpgp.org/
30.1509+- [[vc:packy/hagrid/][packy/hagrid]]
31.1--- a/readme.org Tue Aug 27 21:35:44 2024 -0400
31.2+++ b/readme.org Wed Sep 04 16:58:53 2024 -0400
31.3@@ -1,44 +1,18 @@
31.4+:PROPERTIES:
31.5+:ID: 751af716-6308-4433-8f2d-78cb4c59c665
31.6+:END:
31.7 #+title: notes
31.8 #+description: CC Notes
31.9 #+author: Richard Westhaver
31.10 #+email:richard.westhaver@gmail.com
31.11 #+setupfile: ../clean.theme
31.12 #+export_file_name: index
31.13-#+options: num:t prop:t timestamp:t toc:t d:t compact-itemx:t
31.14+#+options: num:nil prop:nil timestamp:nil toc:nil html-postamble:nil ^:nil
31.15 #+property: header-args :eval no-export
31.16-#+INCLUDE: 20230730.org
31.17-#+INCLUDE: 20231024.org
31.18-#+INCLUDE: 20231101.org
31.19-#+INCLUDE: 20231102.org
31.20-#+INCLUDE: 20231105.org
31.21-#+INCLUDE: 20231124.org
31.22-#+INCLUDE: 20231205.org
31.23-#+INCLUDE: 20231209.org
31.24-#+INCLUDE: 20231212.org
31.25-#+INCLUDE: 20231223.org
31.26-#+INCLUDE: 20231224.org
31.27-#+INCLUDE: 20231228.org
31.28-#+INCLUDE: 20240103.org
31.29-#+INCLUDE: 20240104.org
31.30-#+INCLUDE: 20240120.org
31.31-#+INCLUDE: 20240129.org
31.32-#+INCLUDE: 20240210.org
31.33-#+INCLUDE: 20240218.org
31.34-#+INCLUDE: 20240301.org
31.35-#+INCLUDE: 20240302.org
31.36-#+INCLUDE: 20240313.org
31.37-#+INCLUDE: 20240317.org
31.38-#+INCLUDE: 20240325.org
31.39-#+INCLUDE: 20240419.org
31.40-#+INCLUDE: 20240425.org
31.41-#+INCLUDE: 20240731.org
31.42-#+INCLUDE: 20240804.org
31.43-#+INCLUDE: 20240808.org
31.44-#+INCLUDE: 20240816.org
31.45-#+INCLUDE: demo-notes.org
31.46-#+INCLUDE: inbox-notes.org
31.47-#+INCLUDE: nas-t.org
31.48-#+INCLUDE: parquet-parsing.org
31.49-#+INCLUDE: query-lang.org
31.50-#+INCLUDE: refs.bib
31.51-#+INCLUDE: skel-readme.org
31.52+* [[file:log.org][log]]
31.53+* [[file:demo-notes.org][demo-notes]]
31.54+* [[file:inbox-notes.org][inbox-notes]]
31.55+* [[file:nas-t.org][nas-t]]
31.56+* [[file:parquet-parsing.org][parquet-parsing]]
31.57+* [[file:query-lang.org][query-lang]]
31.58+* [[file:skel-readme.org][skel-readme]]
32.1--- a/skel-readme.org Tue Aug 27 21:35:44 2024 -0400
32.2+++ b/skel-readme.org Wed Sep 04 16:58:53 2024 -0400
32.3@@ -283,7 +283,7 @@
32.4 - TODO :: CLOS-based core classes
32.5 - TODO :: EIEIO-based wrapper classes
32.6
32.7-#+begin_src dot :file api.svg :exports results
32.8+#+begin_src dot :file /tmp/api.svg :exports results
32.9 digraph { splines=true; label="CLOS API"; labelloc="t"; node [shape=record];
32.10 sk [label="(skel :ID :AST)"]
32.11 methods [label="(sk-compile sk-expand sk-build\nsk-run sk-init sk-new sk-save\nsk-tangle sk-weave sk-call sk-print)"]
32.12@@ -325,4 +325,4 @@
32.13 #+end_src
32.14
32.15 #+RESULTS:
32.16-[[file:api.svg]]
32.17+[[file:/tmp/api.svg]]
