Mercurial > org > notes / log.org

 #+title: log
 #+author: Richard Westhaver
 #+email: richard.westhaver@gmail.com
 #+setupfile: ../clean.theme
 * [2023-07-30 Sun]
 :PROPERTIES:
 :ID:       5a1f17a0-22bb-45bc-a313-ec2cb6c0bfd5
 :END:
 ** VC infrastructure
 :PROPERTIES:
 :ID:       9928339e-14d7-425d-a465-be87e2784590
 :END:
 In heptapod we have a root group named =comp=, containg a variety of
 subgroups. Some of these groups should be public, while others are
 internal to comp members exclusively. Within each subgroup, we should
 have the root group members automatically granted privileged access to
 projects. This is relevant for the =startup= subgroup in particular,
 where each project is potentially maintained by multiple non-root
 contributors.
 
 We also need to consider how we will manage subrepos across the
 organization. It is about time we start integrating HG bundles and
 potentially mirrors. For our core VC pipeline we should have no
 reliance on Git, but this may be difficult. It depends on the behavior
 of HG bundles.
 
 Bookmarks/tags should be used for milestones in the root group and are
 infrequent. They are more frequent in projects with a regular release
 life-cycle.
 ** Approaching Webapps
 :PROPERTIES:
 :ID:       bafaceb9-2798-484d-80b7-7ad5d84fff1a
 :END:
 I started poking around in the webapp space again so that I can launch
 a landing page for NAS-T quickly. The Rust situation has improved
 somewhat on the frontend side, and the axum backend stack is nice.
 
 This might seem like a lot of Rust and not a lot of Lisp, which it is,
 but there's still room for Lisp wherever we need it. It mostly plays a
 role in the backend, servicing the database and responding to requests
 from the Rust edges. All of the important tests for the web APIs are
 also written in Lisp. We will almost certainly use Lisp for all static
 processing and HTML generation at compile-time.
 
 This I believe, is the appropriate way to integrate Lisp into a
 cutting-edge web-app. You get the good parts of Lisp where you need
 them (interactive debugging, dynamic language, REPL) and avoid the bad
 parts (OOB optimization, RPS performance) in areas where the customer
 would be impacted. In this domain, Lisp takes the form of a glue
 rather than the bricks and mortar it sometimes appears to us as.
 
 * [2023-10-24 Tue]
 :PROPERTIES:
 :ID:       7bd213b5-9a4b-4490-b0b8-8a0a9b42a9c6
 :END:
 ** virt
 :PROPERTIES:
 :ID:       07c7016f-c653-4d69-9ca0-366a7584a212
 :END:
 *** QEMU
 :PROPERTIES:
 :ID:       f53afecb-561e-4545-a6d0-dbf6d9504ec4
 :END:
 *** KVM
 :PROPERTIES:
 :ID:       df9e07cd-986f-45a7-b9f7-ea88313c92ee
 :END:
 *** Hyper-V
 :PROPERTIES:
 :ID:       87b019ff-7f05-496d-b7bb-153fea7f8595
 :END:
 *** Firecracker
 :PROPERTIES:
 :ID:       b1c52088-4244-4342-b884-c551ce43cc9c
 :END:
 *** Docker
 :PROPERTIES:
 :ID:       d1969d15-8dc0-4499-bddb-f92fa19aec13
 :END:
 *** Vagrant
 :PROPERTIES:
 :ID:       dd6e8169-fba6-45af-aaef-6acd9dbd7b3f
 :END:
 *** LXC
 :PROPERTIES:
 :ID:       bd393c26-6014-42bb-a425-b9f72e5cee8b
 :END:
 *** LXD
 :PROPERTIES:
 :ID:       800ee282-7b5c-465f-a372-344079f9d875
 :END:
 *** containerd
 :PROPERTIES:
 :ID:       7f34835c-9135-4015-bdbd-99d28d7b40bb
 :END:
 *** systemd-nspawn
 :PROPERTIES:
 :ID:       9fef114a-e2f3-40b4-bddb-9948b47b441f
 :END:
 *** VirtualBox
 :PROPERTIES:
 :ID:       04594cbd-5847-434d-aaf6-a7e23058d980
 :END:
 
 ** Concatenative
 :PROPERTIES:
 :ID:       ad2041de-3915-48f9-950c-03b01968952d
 :END:
 *** Factor                                                           :factor:
 :PROPERTIES:
 :ID:       edb7e4a8-d07d-4953-9b64-decdca1dd76f
 :END:
 - [2023-07-04 Tue]
   Factor is a cool concatenative lang but unfortunately the C interface
   (vm/master.h) no longer exists on the master branch.
 *** Joy                                                                 :joy:
 :PROPERTIES:
 :ID:       257365d1-1b2e-4abd-a2e3-5d1705e41bcf
 :END:
 
 **** https://hypercubed.github.io/joy/html/j02maf.html
 :PROPERTIES:
 :ID:       a685ec2b-fbdc-4724-ab22-12d8a4498f81
 :END:
 
 **** [[https://builds.openlogicproject.org/content/incompleteness/arithmetization-syntax/arithmetization-syntax.pdf][arithmetization of syntax]]
 :PROPERTIES:
 :ID:       895a803a-63d5-4dfb-8ceb-c20627d97b02
 :END:
 ** Lisp                                                                :lisp:
 :PROPERTIES:
 :ID:       8109c42f-ccbb-4a08-ab99-1dced8b22a9f
 :END:
 These notes pertain to Lisp. More specifically, ANSI Common Lisp in
 most places.
 
 - https://github.com/lispnik/iup/ - doesn't support MacOS yet, looks
   cool though
   - what we really need is wasm compiler.. TBD
 ** Rust
 :PROPERTIES:
 :ID:       94a9a2b7-3559-476c-9edf-66b3b0c8cbb0
 :END:
 *** Serde
 :PROPERTIES:
 :ID:       ffb4e87c-e265-4549-91f9-093e714708c0
 :END:
 - [2023-07-05 Wed] \\
   important part of the Rust ecosystem, another dtolnay
   contribution. If you want to program a /data/ format in the Rust
   ecosystem, this is how you do it.
 
   The way it works is that you define some special structs, a
   Serializer and a Deserializer which implement the Serialize and
   Deserialize traits provided by serde, respectively.
 
   You can use these structs to provide your public API. The
   conventional choice is public top-level functions like from-str
   and to-string. That's it, your serialization library can now read and
   write your data format as Rust data types.
 
   [[https://serde.rs/enum-representations.html][enum-representations]]
   - the default behavior is an externally tagged representation (verbose)
 
   The docs use strings as core IO when implementing a custom format,
   but the convention is to implement for T where T is bound by std::io
   Read or Write trait. Then you can provide a more robust public API
   (from_bytes, from_writer, etc).
 ** C
 :PROPERTIES:
 :ID:       05a7192a-4c05-44d9-b859-e2a0cff53d86
 :END:
 ** CPP
 :PROPERTIES:
 :ID:       6800e89d-f49e-4ffe-bbab-ed10fec43174
 :END:
 ** Nu
 :PROPERTIES:
 :ID:       b4048114-ec39-4e53-816f-2f1319d404ee
 :END:
 [[https://www.nushell.sh/][~]]
 [[https://www.nushell.sh/cookbook/][cookbook]]
 [[https://github.com/nushell/nu_scripts][nu_scripts]]
 * [2023-11-01 Wed]
 :PROPERTIES:
 :ID:       6d461ba3-f8cb-4a69-b172-038efcca923a
 :END:
 ** AWS usage
 :PROPERTIES:
 :ID:       a14a7916-6743-45e5-8711-ecded8092131
 :END:
 We're leveraging AWS for some of our public web servers for now. It's
 really not realistic to expect that my home desktop and spotty Comcast
 internet can serve any production workflow. What it /is/ capable of is
 a private VPN, which can communicate with AWS and other cloud VPN
 depots via WireGuard ([[https://dev.to/gabrieltetzner/setting-up-a-vpn-with-wireguard-server-on-aws-ec2-4a49][article]]).
 
 I currently use Google Domains for nas-t.net, otom8.dev, and
 rwest.io - but that business is now owned by squarespace, so I would
 rather move it to Route53.
 
 We have archlinux ec2 image builds [[https://wiki.archlinux.org/title/Arch_Linux_AMIs_for_Amazon_Web_Services][here]] and [[https://gitlab.com/anemos-io/archlinux-ec2][here]] - only half work and not
 maintained, but it's a start. I'm not even sure if I should stick with
 arch or cave and use Ubuntu or AWS Linux. We can serve the static
 services with little cost, the only big spender will be the heptapod
 instance which requires a larger instance and some workers.
 
 We'll try to keep the cost at or around $30/month.
 * [2023-11-02 Thu]
 :PROPERTIES:
 :ID:       b74a60a5-fdc1-4ef2-ac74-7ff5c1395d0c
 :END:
 ** IDEAS
 :PROPERTIES:
 :ID:       a0d84027-06ea-4080-a40c-8fa86ed7ee7a
 :END:
 *** shed
 :PROPERTIES:
 :ID:       9fd5e6e1-bb06-4204-a5ed-01afb5d45379
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2023-04-07 Fri 23:24]
 :END:
 rlib
 > ulib
 > ulib
 > ulib
 > ulib
 
 **** sh* tools
 :PROPERTIES:
 :ID:       3daae7ec-4d0a-4f3e-bd5a-5202707fa614
 :END:
 :LOGBOOK:
 - State "TODO"       from "TODO"       [2023-04-07 Fri 23:22]
 :END:
 shc,shx,etc
 *** packy
 :PROPERTIES:
 :ID:       6b2b99d7-9d45-49fc-bff2-463a82c45f90
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2023-04-07 Fri 23:33]
 :END:
 **** rust
 :PROPERTIES:
 :ID:       0fbbf620-c3b7-4aa8-8a79-b25beca802d0
 :END:
 **** common-lisp
 :PROPERTIES:
 :ID:       7fd63e2e-532b-4742-9d25-6cacbfbf2ca0
 :END:
 **** emacs-lisp
 :PROPERTIES:
 :ID:       2e6a146a-9a81-4b49-9127-646838f5e137
 :END:
 **** python
 :PROPERTIES:
 :ID:       2d634517-84dc-4cf4-a95a-02ae2fe3c2cd
 :END:
 **** julia
 :PROPERTIES:
 :ID:       3d75df13-227c-4aee-a046-1443acdcf619
 :END:
 **** C
 :PROPERTIES:
 :ID:       8d44b6f9-e628-4983-aaa9-fea25117a303
 :END:
 **** C++
 :PROPERTIES:
 :ID:       7c0febaa-468c-4835-83ac-8280faa067b9
 :END:
 *** tenex
 :PROPERTIES:
 :ID:       4e5e71e4-be74-44d0-9221-74de37f11096
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2023-04-07 Fri 23:52]
 :END:
 *** mpk
 :PROPERTIES:
 :ID:       f9200923-f406-4104-9efe-33efab186f90
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2023-04-07 Fri 23:52]
 :END:
 *** cfg
 :PROPERTIES:
 :ID:       57a76ed9-76b9-442e-9a8e-3a7deb63dec4
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2023-04-07 Fri 23:34]
 :END:
 *** obj
 :PROPERTIES:
 :ID:       ce1ef5c1-61e0-4bed-84ff-bdf05d1cb379
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2023-04-07 Fri 23:51]
 :END:
 split out from rlib to separate package
 - a purely OOP class library
 *** lab
 :PROPERTIES:
 :ID:       a9401df7-b77e-4648-bdcf-bc9f8d267faf
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2023-04-07 Fri 23:34]
 :END:
 *** source categories
 :PROPERTIES:
 :ID:       bdfaf877-a8a8-47c0-83b2-7ae3d18fcb8c
 :END:
 - need a way of extracting metadata from a repo
 - need ability to search and query libs/packages
 - separate modules based on where they belong in our stack?
   - app
   - lib
   - script?
   - dist
     - software distros
 *** generic query language
 :PROPERTIES:
 :ID:       4cbbb5bd-6fb0-4249-bce3-c60137a0545d
 :END:
 from obj protocol?
 sql compatibility?
 
 /check out kdb/
 *** bbdb
 :PROPERTIES:
 :ID:       2ef1a2d7-19f5-4d65-a4aa-2f6abf07a202
 :END:
 :LOGBOOK:
 - Note taken on [2023-10-24 Tue 22:16] \\
   graph database, build on rocksdb
 :END:
 insidious Big Brother database.
 - an application built with obj
 - sql
 
 *** NAS-TV                                                            :nas:t:
 :PROPERTIES:
 :ID:       416d5836-fd77-4f7f-ad5c-831f20802a14
 :END:
 :LOGBOOK:
 - State "NOTE"       from "TODO"       [2024-08-18 Sun 18:46]
 :END:
 - media streaming
 - gstreamer backend
 - audio/video
 * [2023-11-05 Sun]
 :PROPERTIES:
 :ID:       195067ad-1c7b-498a-9a90-8c9abba192f5
 :END:
 ** DRAFT dylib-skel-1
 :PROPERTIES:
 :ID:       55fc90c8-4261-4fd2-b991-280e76f0e59c
 :END:
 - State "DRAFT"      from              [2023-11-05 Sun 22:23]
 *** Overview
 :PROPERTIES:
 :ID:       bd3b748b-e792-40b3-abb0-02c274038539
 :END:
 Our core languages are [[https://www.rust-lang.org/][Rust]] and [[https://lisp-lang.org/][Lisp]] - this is the killer combo which will allow NAS-T
 to rapidly develop high-quality software. As such, it's crucial that these two very
 different languages (i.e. compilers) are able to interoperate seamlessly.
 
 Some interop methods are easy to accomodate via the OS - such as IPC or data sharing,
 but others are a bit more difficult.
 
 In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something
 that /can/ be difficult, due to some complications with Rust and because this is not the
 most popular software stack (yet ;). This is an experiment and may not make it to our
 code-base, but it's definitely something worth adding to the toolbox in case we need it.
 
 *** FFI
 :PROPERTIES:
 :ID:       c2ccef54-e93d-406a-af1d-ebb7ad1a935b
 :END:
 The level of interop we're after in this case is [[https://en.wikipedia.org/wiki/Foreign_function_interface][FFI]].
 
 Basically, calling Rust code from Lisp and vice-versa. There's an article about calling
 Rust from Common Lisp [[https://dev.to/veer66/calling-rust-from-common-lisp-45c5][here]] which shows the basics and serves as a great starting point
 for those interested.
 **** Rust != C
 :PROPERTIES:
 :ID:       bb68152d-1794-47d8-86c0-03eb7f9256d7
 :END:
 The complication(s) with Rust I mentioned early is really just that /it is not C/. =C=
 is old, i.e. well-supported with a stable ABI, making the process of creating bindings
 for a C library a breeze in many languages.
 
 For a Rust library we need to first appease the compiler, as explained in [[https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c][this section]]
 of the Rustonomicon. Among other things it involves changing the calling-convention of
 functions with a type signature and editing the Cargo.toml file to produce a
 C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used
 like the C ABI can.
 
 **** Overhead
 :PROPERTIES:
 :ID:       f74790d6-8fa9-4e76-8362-71f074058947
 :END:
 Using FFI involves some overhead. Check [[https://github.com/dyu/ffi-overhead][here]] for an example benchmark across a few
 languages. While building the NAS-T core, I'm very much aware of this, and will need a
 few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular,
 I'm concerned about crossing multiple language barriers (Rust<->C<->Lisp).
 
 *** Rust -> C -> Lisp
 :PROPERTIES:
 :ID:       3d8e8664-337b-4bf1-832d-bc29e9761d56
 :END:
 **** Setup
 :PROPERTIES:
 :ID:       7c1a259a-8bc1-4a42-abb7-8d61d0a1c22d
 :END:
 For starters, I'm going to assume we all have Rust (via =rustup=) and Lisp (=sbcl= only)
 installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
 this post).
 ***** Cargo
 :PROPERTIES:
 :ID:       21b8d6b0-27fd-460d-b584-55765c5f7761
 :END:
 Create a new library crate. For this example we're focusing on a 'skeleton' for
 /dynamic/ libraries only, so our experiment will be called =dylib-skel= or *dysk* for
 short.
 src_sh[:exports code]{cargo init dysk --lib && cd dysk} 
 
 A =src/lib.rs= will be generated for you. Go ahead and delete that. We're going to be
 making our own =lib.rs= file directly in the root directory (just to be cool).
 
 The next step is to edit your =Cargo.toml= file. Add these lines after the =[package]=
 section and before =[dependencies]=:
 #+begin_src conf-toml
 [lib]
 crate-type = ["cdylib","rlib"]
 path = "lib.rs"
 [[bin]]
 name="dysk-test"
 path="test.rs"
 #+end_src
 
 This tells Rust to generate a shared C-compatible object with a =.so= extension which we
 can open using [[https://man.archlinux.org/man/dlopen.3.en][dlopen]].
 ***** cbindgen
 :PROPERTIES:
 :ID:       45c0e1cd-a441-4877-8b64-d30bc0381da2
 :END:
 ****** install
 :PROPERTIES:
 :ID:       d11805bf-1238-4359-ab52-6e6d1b74dc33
 :END:
 Next, we want the =cbindgen= program which we'll use to generate header files for
 C/C++. This step isn't necessary at all, we just want it for further experimentation.
 
 src_sh[:exports code]{cargo install --force cbindgen}
 
 We append the =cbindgen= crate as a /build dependency/ to our =Cargo.toml= like so:
 #+begin_src conf-toml
 [build-dependencies]
 cbindgen = "0.24"
 #+end_src
 ****** cbindgen.toml
 :PROPERTIES:
 :ID:       e33424c1-a328-4e86-b817-86e171584944
 :END:
 #+begin_src conf-toml :tangle cbindgen.toml
 language = "C"
 autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
 include_version = true
 namespace = "dysk"
 cpp_compat = true
 after_includes = "#define DYSK_VERSION \"0.1.0\""
 line_length = 88
 tab_width = 2
 documentation = true
 documentation_style = "c99"
 usize_is_size_t = true
 [cython]
 header = '"dysk.h"'
 #+end_src
 ****** build.rs
 :PROPERTIES:
 :ID:       0e465a4e-6709-4920-bb8a-514189f36f84
 :END:
 #+begin_src rust :tangle build.rs
 fn main() -> Result<(), cbindgen::Error> {
   if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
     b.write_to_file("dysk.h"); Ok(())}
   else { panic!("failed to generate dysk.h from cbindgen.toml") } }
 #+end_src
 **** lib.rs
 :PROPERTIES:
 :ID:       884ae229-0b41-4de3-880b-fb31058dc00a
 :END:
 #+begin_src rust :tangle lib.rs
 //! lib.rs --- dysk library
 use std::ffi::{c_char, c_int, CString};
 #[no_mangle]
 pub extern "C" fn dysk_hello() -> *const c_char {
   CString::new("hello from rust").unwrap().into_raw()}
 #[no_mangle]
 pub extern "C" fn dysk_plus(a:c_int,b:c_int) -> c_int {a+b}
 #[no_mangle]
 pub extern "C" fn dysk_plus1(n:c_int) -> c_int {n+1}
 #+end_src
 **** test.rs
 :PROPERTIES:
 :ID:       b8b27643-3c41-412c-afb7-12e980ffffb3
 :END:
 #+begin_src rust :tangle test.rs
 //! test.rs --- dysk test
 fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::dysk_plus1(2 as core::ffi::c_int);}}
 #+end_src
 **** compile
 :PROPERTIES:
 :ID:       4d429e91-1b3b-40eb-86c0-b4896e277423
 :END:
 #+begin_src sh
 cargo build --release
 #+end_src
 **** load from SBCL
 :PROPERTIES:
 :ID:       9db4d715-421c-4a06-ad25-68aa79396d3a
 :END:
 #+begin_src lisp :tangle dysk.lisp
 (load-shared-object #P"target/release/libdysk.so")
 (define-alien-routine dysk-hello c-string)
 (define-alien-routine dysk-plus int (a int) (b int))
 (define-alien-routine dysk-plus1 int (n int))
 (dysk-hello) ;; => "hello from rust"
 #+end_src
 **** benchmark
 :PROPERTIES:
 :ID:       88cd9e83-64da-4aa4-ab12-125b127bca06
 :END:
 #+begin_src shell
 time target/release/dysk-test
 #+end_src
 #+begin_src lisp :tangle test.lisp
 (time (dotimes (_ 500000000) (dysk-plus1 2)))
 #+end_src
 * [2023-11-24 Fri]
 :PROPERTIES:
 :ID:       509a06ab-1a3d-4d5c-92e1-6fc50e8d1c00
 :END:
 ** cl-dot examples
 :PROPERTIES:
 :ID:       df12c00f-0264-4f08-bff6-a327c7089b4c
 :END:
 #+begin_src lisp
 (defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object cons))
   (make-instance 'cl-dot:node
                  :attributes '(:label "cell \\N"
                                :shape :box)))
 (defmethod cl-dot:graph-object-points-to ((graph (eql 'example)) (object cons))
   (list (car object)
         (make-instance 'cl-dot:attributed
                        :object (cdr object)
                        :attributes '(:weight 3))))
 ;; Symbols
 (defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object symbol))
   (make-instance 'cl-dot:node
                  :attributes `(:label ,object
                                :shape :hexagon
                                :style :filled
                                :color :black
                                :fillcolor "#ccccff")))
 (let* ((data '(a b c #1=(b z) c d #1#))
        (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
                                                  '(:rankdir "LR" :layout "twopi" :labelloc "t"))))
   (cl-dot:dot-graph dgraph "test-lr.svg" :format #+nil :x11 :svg))
 #+end_src
 
 #+RESULTS:
 
 #+begin_src lisp
 (let* ((data '(a b))
        (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
                                                  '(:rankdir "LR"))))
           (cl-dot:print-graph dgraph))
 #+end_src
 
 * [2023-12-05 Tue]
 :PROPERTIES:
 :ID:       4f1d099a-2f9a-40c3-8cf7-57df6e660455
 :END:
 ** global refs
 :PROPERTIES:
 :ID:       a05618ad-ada2-44be-9260-8acc0cf30d4d
 :END:
 need a way of indexing, referring to, and annotating objects such as
 URLs, docs, articles, source files, etc.
 
 What is the best way to get this done?
 * [2023-12-09 Sat]
 :PROPERTIES:
 :ID:       e304a97e-49fe-4fde-a273-7d3f78231033
 :END:
 ** doc best practices
 :PROPERTIES:
 :ID:       ea2f3f8e-1c8c-4c05-b16c-fb197d346409
 :END:
 https://rust-lang.github.io/api-guidelines/documentation.html
 
 also: https://lisp-lang.org/style-guide/
 * [2023-12-12 Tue]
 :PROPERTIES:
 :ID:       eff3b823-6ffd-4355-b289-bdcd7b31ee54
 :END:
 ** On Computers
 :PROPERTIES:
 :ID:       11bd7a7c-4d0c-4d7f-a487-dc59052da2b3
 :END:
 If you've met me in the past decade, you probably know that I am
 extremely passionate about computers. Let me first explain why.
 
 On the most basic level computers are little (or big) machines that
 can be programmed to do things, or /compute/ if we're being
 technical.[fn:1]
 
 They host and provide access to the Internet, which is a pretty big
 thing, but they do little things too like unlock your car door and
 tell your microwave to beep at you. They solve problems. Big or small.
 
 They're also /everywhere/ - which can be scary to think about, but
 ultimately helps propel us into the future.
 
 There's something pretty cool about that - when you look at the
 essence of computation. There are endless quantities of these machines
 which follow the same basic rules and can be used to solve /real/
 problems.
 
 *** The Programmer
 :PROPERTIES:
 :ID:       f75f9b5b-ec2b-4f62-9fee-1ed3cef4ac50
 :END:
 Now, let us consider the /programmer/. They have power. /real/
 power. They understand the language of computers, can whisper to them
 in various dialects. It can be intimidating to witness until you
 realize how often the programmer says the wrong thing - a bug.
 
 In reality, the programmer has a symbiotic relationship with
 computers. Good programmers understand this relationship well.
 
 #+begin_annecdote
 One day after I got my first job at a software company, I remember
 being on an all-hands meeting due to a client service outage. We had
 some management, our lead devs, product team, and one curious looking
 man who happened to be our lead IT consultant who had just joined. He
 was sitting up on a hotel bed, shirtless, vaping an e-cig, typing
 away in what I can only imagine was a shell prompt.
 
 After several minutes he took a swig from a bottle of Coke and said
 "Node 6 is sick." then a few seconds later our services were
 restored. For the next hour on the call he explained what happened and
 why, but that particular phrase always stuck with me. He didn't say
 Node 6 was down, or had an expired cert - his diagnosis was that /it/
 was /sick/. 
 #+end_annecdote
 
 The more you work closely with computers, the more you start to think
 of them this way. You don't start screaming when the computer does the
 wrong thing, you figure out what's wrong and learn from it. With
 experience, you start to understand the different behaviors of the
 machines you work with. I like to call this /Machine Empathy/.
 
 *** Programs
 :PROPERTIES:
 :ID:       b1187fd8-4d19-4566-827c-5594fbf90df7
 :END:
 I already mentioned bugs - I write plenty of those, but usually I try
 to write /programs/. Programs to me are like poetry. I like to think
 they are for the computer too.
 
 Just like computers, /computer programs/ come in different shapes and
 sizes but in basic terms they are sets of instructions used to control
 a computer.
 
 You can write programs to do anything - when I first started, my
 programs made music. The program was a means to an end. Over time, I
 started to see the program as something much more. I saw it as the
 music itself.
 
 [fn:1] ... perform computations
 
 ** On Infra
 :PROPERTIES:
 :ID:       332145f8-50b4-46d1-8368-ddaf097c875f
 :END:
 Something that is missing from many organizations big or large, is an
 effective way to store and access information, even about their own
 org.
 
 It can be difficult problem to solve - usually there's the official
 one, say Microsoft Sharepoint and then the list of unofficial sources
 which becomes tribal corporate hacker knowledge. Maybe the unofficial
 ones are more current, or are annotated nicely, but their very
 existence breaks the system. There's no longer a single source of
 truth.
 
 My priority in this department is writing services which process and
 store information from a variety of sources in a distributed knowledge
 graph. The graph can later be queried to access information on-demand.
 
 My idea of infrastructure is in fact to build my own Cloud. Needless
 to say I don't have an O365 subscription, and wherever possible I'll
 be relying on hardware I have physical access to. I'm not opposed to
 cloud services at large but based on principle I like to think we
 shouldn't be built on them.
 * [2023-12-23 Sat]
 :PROPERTIES:
 :ID:       67b51ca5-909b-495f-b263-5979efe36448
 :END:
 ** https://cal-coop.gitlab.io/utena/utena-specification/main.pdf
 :PROPERTIES:
 :ID:       03b3d70c-5ab6-4be8-b811-a7ba7f84be8f
 :END:
 from the author of cl-decentralise2. draft specification of a
 /Maximalist/ Computing System.
 * [2023-12-24 Sun]
 :PROPERTIES:
 :ID:       bcc178c2-6a96-4ad9-98db-4ddf045e6371
 :END:
 ** public datasets
 :PROPERTIES:
 :ID:       59e3ba53-3f40-47db-929c-acc998f9092d
 :END:
 https://github.com/awesomedata/awesome-public-datasets
 https://docs.openml.org/Datasets/
 https://wiki.pathmind.com/open-datasets
 * [2023-12-28 Thu]
 :PROPERTIES:
 :ID:       db3d353b-8f1c-4645-a4ad-459d36392b22
 :END:
 ** useful internals
 :PROPERTIES:
 :ID:       a8ee911d-6d19-452b-b03a-249a446c1b20
 :END:
 #+begin_src lisp
   sb-sys:*runtime-dlhandle*
   sb-fasl:+fasl-file-version+
   sb-fasl:+backend-fasl-file-implementation+
   sb-debug:print-backtrace
   sb-debug:map-backtrace
   sb-pretty:pprint-dispatch-table
   sb-lockless:
   sb-ext:simd-pack
   sb-walker:define-walker-template
   sb-walker:macroexpand-all
   sb-walker:walk-form
   sb-kernel:empty-type
   sb-kernel:*eval-calls*
   sb-kernel:*gc-pin-code-pages*
   sb-kernel:*restart-clusters*
   sb-kernel:*save-lisp-clobbered-globals*
   sb-kernel:*top-level-form-p*
   sb-kernel:*universal-fun-type*
   sb-kernel:*universal-type*
   sb-kernel:*wild-type*
   sb-kernel:+simd-pack-element-types+
   (sb-vm:memory-usage)
   (sb-vm:boxed-context-register)
   (sb-vm:c-find-heap->arena)
   (sb-vm:copy-number-to-heap)
   (sb-vm:dump-arena-objects)
   (sb-vm:fixnumize)
   (sb-vm:rewind-arena)
   (sb-vm:show-heap->arena)
   (sb-vm:with/without-arena)
   (sb-cltl2:{augment-environment,compiler-let,define-declaration,parse-macro})
   (sb-cltl2:{declaration-information, variable-information, function-information})
   sb-di:
   sb-assem:
   sb-md5:
   sb-regalloc:
   sb-disassem:
 #+end_src
 * [2024-01-03 Wed]
 :PROPERTIES:
 :ID:       b83e3b40-17cc-4fc0-a412-d8b3f8733354
 :END:
 ** [[https://github.com/sigmf/SigMF][SigMF]]
 :PROPERTIES:
 :ID:       82a9833e-39f8-434f-94b4-4f34ce2fef25
 :END:
 #+begin_quote
 Sharing sets of recorded signal data is an important part of science
 and engineering. It enables multiple parties to collaborate, is often
 a necessary part of reproducing scientific results (a requirement of
 scientific rigor), and enables sharing data with those who do not have
 direct access to the equipment required to capture it.
 
 Unfortunately, these datasets have historically not been very
 portable, and there is not an agreed upon method of sharing metadata
 descriptions of the recorded data itself. This is the problem that
 SigMF solves.
 
 By providing a standard way to describe data recordings, SigMF
 facilitates the sharing of data, prevents the "bitrot" of datasets
 wherein details of the capture are lost over time, and makes it
 possible for different tools to operate on the same dataset, thus
 enabling data portability between tools and workflows.
 #+end_quote
 
 the-spec: https://github.com/sigmf/SigMF/blob/sigmf-v1.x/sigmf-spec.md
 ** [[https://www.libvolk.org/][LibVOLK]]
 :PROPERTIES:
 :ID:       af88cd62-2610-4ba7-a5f4-d98c942c81dc
 :END:
 Vector-Optimized Library of Kernels (simd)
 ** [[https://docs.kernel.org/fb/framebuffer.html][/dev/fb*]]
 :PROPERTIES:
 :ID:       b866d034-2d59-4e1e-934f-02903b3de8e7
 :END:
 framebuffers, used by fbgrab/fbcat program
 ** [[https://docs.kernel.org/block/ublk.html][ublk]]
 :PROPERTIES:
 :ID:       d58cb494-ce43-49cc-859b-aa6a518ac314
 :END:
 https://github.com/ming1/ubdsrv
 * [2024-01-04 Thu]
 :PROPERTIES:
 :ID:       86fdf7a7-9d26-4ce5-95a2-484db1843fb3
 :END:
 goals:
 make problems smaller.
 
 sections:
 why lisp?
 - doesn't need mentioning more and more
 * [2024-01-20 Sat]
 :PROPERTIES:
 :ID:       e8015df1-c8f3-4766-9cda-e4bb87c9aa62
 :END:
 ** TODO taobench demo
 :PROPERTIES:
 :ID:       72d7e1cf-79a4-4f30-88da-bb0c13551b1f
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2024-01-21 Sun 00:32]
 :END:
 https://github.com/audreyccheng/taobench - shouldn't have missed this :)
 obviously we need to implement this using core -- in demo/bench/tao?
 ** TODO clap completion for nushell
 :PROPERTIES:
 :ID:       91d3f9ba-92ab-43c7-ac0f-5fab172b7ba1
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2024-01-20 Sat 23:23]
 :END:
 https://github.com/clap-rs/clap/tree/master/clap_complete_nushell
 ** Dataframe scripting
 :PROPERTIES:
 :ID:       23ba507b-8d56-4e1c-bbd6-0d2f5983e792
 :END:
 https://studioterabyte.nl/en/blog/polars-vs-pandas
 nushell supports DFs, polars underneath?
 https://www.nushell.sh/book/cheat_sheet.html
 
 #+begin_src nushell
 
 #+end_src
 ** Cloud Squatting
 :PROPERTIES:
 :ID:       31263e62-7de6-425a-8b3e-0522b4222322
 :END:
 *** Google
 :PROPERTIES:
 :ID:       e9724ccb-f4f6-4f75-816e-34887e2fd776
 :END:
 - [[https://cloud.google.com/free/docs/free-cloud-features][Free Cloud Features]]
   + 90-day $300 credits
   + e2-micro - free hours worth 1 instance/month
   + 
 *** Amazon
 :PROPERTIES:
 :ID:       048a6b82-2a8f-480b-a892-b8c80f97a13c
 :END:
 - AWS Free Tier
 *** Akamai
 :PROPERTIES:
 :ID:       af1dbf25-4b58-4a89-a968-9a8f24499419
 :END:
 - Linode Free Trial
 *** Oracle
 :PROPERTIES:
 :ID:       d2316ff4-4448-4209-87ca-22572def8125
 :END:
 - [[https://www.oracle.com/cloud/free/?intcmp=ohp052322ocift][OCI Free Tier]]
   + always free: 2 x oracle autonomous DB
   + 2 x AMD Compute VMs
   + up to 4 x ARM Ampere A1 with 3k/cpu/hr and 18k/gb/h per month
   + block/object/archive storage
   + 30-day $300 credits
 * [2024-01-29 Mon]
 :PROPERTIES:
 :ID:       9bad5422-eea9-4328-a68e-fd2180da2df1
 :END:
 ** trash as block device
 :PROPERTIES:
 :ID:       cdf84840-46dd-42f3-82df-f5b57c908118
 :END:
 :LOGBOOK:
 - State "NOTE"       from              [2024-01-29 Mon 20:53]
 - State "NOTE"       from              [2024-01-29 Mon 20:53]
 :END:
 in nushell there is option for rm command to always use 'trash' -
 AFAIK the current approach is via a service (trashd).
 
 An interesting experiment would be to designate a block device as
 'trash' - may be possible to remove reliance on a service
 
 may be an opportunity for ublk driver to shine - instead of /dev/null
 piping we need a driver for streaming a file to /dev/trash
 ** compute power
 :PROPERTIES:
 :ID:       fdc5a933-6792-40e0-914e-49629a4d7332
 :END:
 :LOGBOOK:
 - State "NOTE"       from              [2024-01-29 Mon 16:28]
 :END:
 - mostly x86_64 machines - currently 2 AWS EC2 instances, some podman containers, and our home beowulf server:
 - beowulf:
   - Zor
     - mid-size tower enclosed (Linux/Windows)
     - CPU
       - Intel Core i7-6700K
       - 4 @ 4.0
     - GPU
       - NVIDIA GeForce GTX 1060
       - 6GB
     - Storage
       - Samsung SSD 850: 232.9GB
       - Samsung SSD 850: 465.76GB
       - ST2000DM001-1ER1: 1.82TB
       - WDC WD80EAZZ-00B: 7.28TB
       - PSSD T7 Shield: 3.64TB
       - My Passport 0820: 1.36TB
     - RAM
       - 16GB (2*8) [64GB max]
       - DDR4
   - Jekyll
     - MacBook Pro 2019 (MacOS/Darwin)
     - CPU
       - Intel
       - 8 @
     - RAM
       - 32G DDR4
   - Hyde
     - Thinkpad
     - CPU
       - Intel
       - 4 @
     - RAM
       - 24G DDR3
   - Boris
     - Pinephone Pro
     - CPU
       - 64-bit 6-core 4x ARM Cortex A53  + 2x ARM Cortex A72
     - GPU
       - Mali T860MP4
     - RAM
       - 4GB LPDDR4
   - pi
     - Raspberry Pi 4 Model B
     - CPU
       - Cortex-A72 (ARM v8) 64-bit SoC 
       - 4 @ 1.8GHz
     - RAM
       - 8 GB
       - DDR4 4200
 * [2024-02-10 Sat]
 :PROPERTIES:
 :ID:       d05fd1f5-2f85-485f-8963-11a1cd2322fc
 :END:
 ** BigBenches
 :PROPERTIES:
 :ID:       d7e1885d-fb19-448c-beb6-0ed0bd701cec
 :END:
 #+name: 1trc-nu
 #+begin_src nushell
   let ms = '1trc/measurements-0.parquet'
   dfr open $ms
   | dfr group-by  station
   | dfr agg [
     (dfr col measure | dfr min | dfr as "min")
     (dfr col measure | dfr max | dfr as "max")
     (dfr col measure | dfr sum | dfr as "sum")
     (dfr col measure | dfr count | dfr as "count")
   ]
 #+end_src
 * [2024-02-18 Sun]
 :PROPERTIES:
 :ID:       908fd9e6-6307-402c-ba76-813e82839f53
 :END:
 ** WL vs X
 :PROPERTIES:
 :ID:       9a398886-858a-450b-9029-2339cc551bc3
 :END:
 :LOGBOOK:
 - State "NOTE"       from              [2024-02-18 Sun 11:55]
 :END:
 In the past few months there has been drama regarding Wayland vs X. It
 seems to be on everyone's minds after Artem's freakout issue and the
 follow up YT vids/comments.
 
 I admit that it made me reconsider the fitness of WL as a whole -
 there was a github gist that made some scathing arguments against it.
 
 It's an odd debate though. I think there are many misunderstandings.
 
 So first off, if we look at the homepage
 https://wayland.freedesktop.org/, Wayland claims it is a replacement
 for X11. It now has /manifest destiny/, which in my opinion is a great
 shame.
 
 X-pros seem to agree that Wayland has /manifest destiny/ - like if you
 are building softwares that look remotely like a window system, it's a
 successor to X. That's the model of doing things and there's no way
 around it.
 
 The disagreement starts with how this destiny - of an X2 - should be
 fulfilled. X-pros want a fork of X, but it's too late for
 that. WL-pros want X to run on top of Wayland compositor:
 https://wayland.freedesktop.org/xserver.html.
 
 Xwayland is a problem for me. From the project description: 'if we're
 migrating away from X, it makes sense to have a good backwards
 compatibility story.' Full disclosure: I have never done significant
 work on Xwayland, so perhaps my opinion is unwarranted. But I have no
 intention of attempting to maintain a computer system that uses
 Wayland and X clients at the same time.
 
 To me, X is ol' reliable. Every distro has first-class X support, and
 it runs on most systems with very little user intervention. Where it
 doesn't, there is 20+ years of dev history and battle-tested
 workarounds for you to find your solution in.
 
 Wayland is the new kid on the block, born just in 2008. It's a fresh
 start to one of the most difficult challenges in software - window
 systems. A re-write would be pointless though, and so the real
 value-add is in design. Wayland is designed as a protocol and
 collection of libraries which are implemented in your own
 compositor. Coming from Lisp - with ANSI Common Lisp and SRFIs, this
 feels right even if the implementation is something very different
 (compositor vs compiler).
 
 With X, it is assumed to be much harder to write an equivalent
 'compositor'. Here's the thing though - with a significantly complex X
 client implementation, it is /impossible/ to replicate in WL. This is
 really the crux of Artemi's argument in his issue. He asked for a 1:1
 equivalent X/WL comparison when no such thing exists, and in my
 opinion it is a waste of time.
 
 The WL core team is fully aware of this dichotomy, but also that this
 is in no way a problem or weakness in either system. It means they're
 different systems, goddammit.
 
 If it was up to me, Xwayland wouldn't exist. I understand why it does,
 and that it does make things easier for developers who need to support
 both, and users who have multiple apps with multiple windowing
 requirements. It's a bandaid though, and one that is particularly
 dangerous because it re-enforces the idea that Wayland is just X2 and
 that they're fully compatible.
 
 What interests me in the Wayland world right now is the idea of a
 small, modular, full-stack Wayland compositor API. There are several
 'kiosk' based compositors for single applications (cage), but these
 aren't complete solutions. It is possible to get much closer to the
 metal, and that's where I want to be so that I can build my own APIs
 on top - I don't want to live on top of X, and I certainly don't want
 to live on top of X on top of WL. I want a /pure/ solution that hides
 as little as possible, exposing the interesting bits.
 * [2024-03-01 Fri]
 :PROPERTIES:
 :ID:       5d5159dd-5887-4823-81c6-1624f8cc4fd4
 :END:
 ** TODO collect more data
 :PROPERTIES:
 :ID:       9e0339dd-17a8-436c-afd4-245437fe2fea
 :END:
 :LOGBOOK:
 - State "TODO"       from              [2024-03-01 Fri 15:27]
 :END:
 https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
 weather - music - etc
 * [2024-03-02 Sat]
 :PROPERTIES:
 :ID:       148d3b53-be56-4c34-9313-cac26c57300e
 :END:
 ** On blocks and devices
 :PROPERTIES:
 :ID:       242a2380-3d87-403d-b529-0245e77ed1f0
 :END:
 :LOGBOOK:
 - State "NOTE"       from              [2024-03-02 Sat 21:30]
 :END:
 [[https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/dev.html][/dev]]
 In Linux, everything is a file.
 
 /dev/ contains special device files - usually block or character
 device.
 
 major, minor = category, device
 0, 5
 
 mknod - create special device files
 
 [[https://www.redhat.com/en/blog/linux-block-devices-hints-debugging-and-new-developments][redhat hints]]
 
 #+begin_src shell
   dd if=/dev/zero of=myfile bs=1M count=32
   losetup --show -f myfile
   ls -al /dev/loop0
   losetup -d /dev/loop0 #teardown
 #+end_src
 
 #+begin_src shell
   echo "sup dude" > /dev/loop0
   dd if=/dev/loop0 -bs=1
   dd if=/dev/nvme0 of=/dev/null progress=true
   #pacman -S hdparm
   hdparm -T /dev/nvme0
 #+end_src
 
 #+begin_src shell
 modprobe scsi_debug add_host=5 max_luns=10 num_tgts=2 dev_size_mb=16
 #+end_src
 
 sparsefiles: create with C, dd, or truncate
 
 #+begin_src shell :results output
 truncate --help
 #+end_src
 
 #+RESULTS:
 #+begin_example
 Usage: truncate OPTION... FILE...
 Shrink or extend the size of each FILE to the specified size
 
 A FILE argument that does not exist is created.
 
 If a FILE is larger than the specified size, the extra data is lost.
 If a FILE is shorter, it is extended and the sparse extended part (hole)
 reads as zero bytes.
 
 Mandatory arguments to long options are mandatory for short options too.
   -c, --no-create        do not create any files
   -o, --io-blocks        treat SIZE as number of IO blocks instead of bytes
   -r, --reference=RFILE  base size on RFILE
   -s, --size=SIZE        set or adjust the file size by SIZE bytes
       --help        display this help and exit
       --version     output version information and exit
 
 The SIZE argument is an integer and optional unit (example: 10K is 10*1024).
 Units are K,M,G,T,P,E,Z,Y,R,Q (powers of 1024) or KB,MB,... (powers of 1000).
 Binary prefixes can be used, too: KiB=K, MiB=M, and so on.
 
 SIZE may also be prefixed by one of the following modifying characters:
 '+' extend by, '-' reduce by, '<' at most, '>' at least,
 '/' round down to multiple of, '%' round up to multiple of.
 
 GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
 Full documentation <https://www.gnu.org/software/coreutils/truncate>
 or available locally via: info '(coreutils) truncate invocation'
 #+end_example
 
 test mkfs.btrfs on 10T dummy block device
 #+begin_src shell
   dd if=/dev/zero of=/tmp/bb1 bs=1 count=1 seek=10T
   du -sh /tmp/bb1
   losetup --show -f /tmp/bb1
   mkfs.btrfs /dev/loop0
 #+end_src
 
 diagnostics
 #+begin_src shell
   iostat # pacman -S sysstat
   blktrace # paru -S blktrace
   iotop # pacman -S iotop
 #+end_src
 
 bcc/ trace: Who/which process is executing specific functions against
 block devices?
 
 bcc/biosnoop: Which process is accessing the block device, how many
 bytes are accessed, which latency for answering the requests?
 
 at the kernel level besides BPF we got kmods and DKMS,
 
 compression/de-duplication can be done via VDO kernel mod
 
 https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support
 
 ** save-lisp-and-respawn
 :PROPERTIES:
 :ID:       bef3c7e0-c929-434a-9118-cca57fcb5bd3
 :END:
 :LOGBOOK:
 - State "NOTE"       from              [2024-03-02 Sat 22:57]
 :END:
 #+begin_src lisp
   sb-ext:*save-hooks*
 #+end_src
 
 ** syslog for log
 :PROPERTIES:
 :ID:       570a1587-486c-49c0-bca4-9cbcfcd3013e
 :END:
 :LOGBOOK:
 - State "NOTE"       from              [2024-03-03 Sun 16:35]
 :END:
 sb-posix:
 - openlog syslog closelog
 - levels: emerg alert crit err warning notice info debug
 - setlogmask
 * [2024-03-13 Wed]
 :PROPERTIES:
 :ID:       f2293a24-442b-458e-99c8-6488b85cb3c2
 :END:
 ** RESEARCH [[https://github.com/guicho271828/sbcl-wiki/wiki/][sbcl-wiki]]
 :PROPERTIES:
 :ID:       9a507eb4-2153-4abd-9be3-3b38b4989e8c
 :END:
 :LOGBOOK:
 - State "RESEARCH"   from              [2024-03-13 Wed 21:49]
 :END:
 ** IR1
 :PROPERTIES:
 :ID:       ca497ebc-b3b0-419e-aa59-a082c96d61f6
 :END:
 ** IR2
 :PROPERTIES:
 :ID:       c5d40dfe-b059-44f4-8d42-78f61f6e9a95
 :END:
 * [2024-03-17 Sun]
 :PROPERTIES:
 :ID:       b9961009-0a47-4122-a5d7-a6e72e4b5f92
 :END:
 ** DB Benchmarking
 :PROPERTIES:
 :ID:       78f5150b-4ccb-4682-bf9c-60ed1649905d
 :END:
 :LOGBOOK:
 - State "NOTE"       from              [2024-02-04 Sun 20:40]
 :END:
 [[https://github.com/facebook/rocksdb/wiki/Benchmarking-tools][RocksDB benchmarking tools]]
 
 ** packy design
 :PROPERTIES:
 :ID:       1eae39fb-ccb4-419e-857a-cc357e1d3eda
 :END:
 :LOGBOOK:
 - State "NOTE"       from              [2024-01-25 Thu 22:39]
 :END:
 - API root: https://packy.compiler.company
 - source packs: https://vc.compiler.company/packy
 *** Lib
 :PROPERTIES:
 :ID:       3863c9f9-89d9-4233-b6e4-5c5bc0178e02
 :END:
 **** Types
 :PROPERTIES:
 :ID:       010cb138-70c4-4d2d-ba7b-ad423d861690
 :END:
 ***** Pack
 :PROPERTIES:
 :ID:       b2bbc87a-dc55-4df8-81d1-a9de8429dad0
 :END:
 Primary data type of the library - typically represents a compressed
 archive, metadata, and ops.
 ***** Bundle
 :PROPERTIES:
 :ID:       432f7779-a93b-48f1-a095-25482c8d3702
 :END:
 Collection data type, usually contains a set of packs with metadata.
 ***** PackyEndpoint
 :PROPERTIES:
 :ID:       c3c21988-8bb2-448a-a98a-f26cc862c2f3
 :END:
 Represents a Packy instance bound to a UDP socket
 ***** PackyEndpointConfig
 :PROPERTIES:
 :ID:       f732c36f-6e55-4c44-a2fb-6514706ec326
 :END:
 Global endpoint configuration object
 ***** PackyClientConfig
 :PROPERTIES:
 :ID:       794952de-a5c5-4031-a1e1-0b1e0325584a
 :END:
 Configuration for outgoing packy connections on an endpoint
 ***** PackyServerConfig
 :PROPERTIES:
 :ID:       b65636cc-0523-4a1b-992d-6aff66cbab5d
 :END:
 Configuration for incoming packy connection son an endpoint
 ***** PackyConnection
 :PROPERTIES:
 :ID:       74646244-ee39-4ef8-8193-d5484438b81b
 :END:
 Packy connection object
 **** Traits
 :PROPERTIES:
 :ID:       dfe9d569-f951-4485-9b72-055e77404281
 :END:
 ***** PackyClient
 :PROPERTIES:
 :ID:       71d4af37-6e24-48ef-a49e-683ee41ce5d2
 :END:
 ****** query
 :PROPERTIES:
 :ID:       47424475-837c-4996-a1b9-3eb6599105d6
 :END:
 ****** install
 :PROPERTIES:
 :ID:       e80b3402-6afd-4ad7-a800-790994867479
 :END:
 ****** update
 :PROPERTIES:
 :ID:       f4801fb0-bace-48ae-ab63-1a5cac3ea98f
 :END:
 ****** login
 :PROPERTIES:
 :ID:       1f24bd33-1e00-4dc2-bc6c-fac8374e14f0
 :END:
 ****** logout
 :PROPERTIES:
 :ID:       b1f91455-68bc-4cc6-94c7-5fb3544e7539
 :END:
 ****** pull
 :PROPERTIES:
 :ID:       b718721a-3f22-4eb6-80bf-5e7d5def1346
 :END:
 ****** push
 :PROPERTIES:
 :ID:       7781e76b-51a4-42fd-929a-a1ff08ccb95b
 :END:
 ***** PackyServer
 :PROPERTIES:
 :ID:       af8409ec-9865-46d3-a593-df109b45cdff
 :END:
 ****** start_packy_server
 :PROPERTIES:
 :ID:       13e65c38-9591-484d-be1d-3d4b98f95f41
 :END:
 ****** stop_packy_server
 :PROPERTIES:
 :ID:       3cae6660-755c-4242-ac4f-eccb7188c6b5
 :END:
 ****** start_packy_registry
 :PROPERTIES:
 :ID:       6ca2bd7d-9c1a-4790-a3a0-08bda811318b
 :END:
 ***** PackyRegistry
 :PROPERTIES:
 :ID:       1d2ad8dc-6aef-48fa-b079-71d8bb88832f
 :END:
 ****** register_pack
 :PROPERTIES:
 :ID:       43221579-6f71-42aa-8332-96af33531015
 :END:
 ****** register_user
 :PROPERTIES:
 :ID:       3c6034fa-d543-47cd-ae35-bcfd3f2b8150
 :END:
 ****** register_bundle
 :PROPERTIES:
 :ID:       1b041060-7f5e-48fe-8566-e4781a597686
 :END:
 * [2024-03-25 Mon]
 :PROPERTIES:
 :ID:       c6e0fb44-21ba-4a94-bd04-e7528dc46dba
 :END:
 ** TBD investigate alieneval for phash opps
 :PROPERTIES:
 :ID:       18989904-85ad-403b-8195-5ba7f643437f
 :END:
 :LOGBOOK:
 - State "TBD"        from              [2024-03-25 Mon 18:56]
 :END:
 * [2024-04-19 Fri]
 :PROPERTIES:
 :ID:       8ffd1283-5a63-4fe3-868f-57a4f9f09f74
 :END:
 ** How it works
 :PROPERTIES:
 :ID:       1b26fcd9-8d32-4031-a0cc-7ed2e26a75d0
 :END:
 The backend services are written in Rust and controlled by a simple
 messaging protocol. Services provide common runtime capabilities known
 as the /service protocol/ but are specialized on a unique /service
 type/ which may in turn register their own /custom protocols/ (via
 core).
 
 Services are capable of dispatching data directly to clients, or
 storing data in the /database/ (sqlite, postgres, mysql).
 
 The frontend clients are pre-dominantly written in Common Lisp and
 come in many shapes and sizes. There is a cli-client, web-client
 (CLOG), docker-client (archlinux, stumpwm, McCLIM), and native-client
 which also compiles to WASM (slint-rs).
 
 ** Guide
 :PROPERTIES:
 :ID:       90772707-ee13-463a-b0b2-7884745ec640
 :END:
 *** Build
 :PROPERTIES:
 :ID:       a2f7f5f5-0cbd-4aa4-b80e-afa1d86990cd
 :END:
 - *install dependencies*
   #+begin_src bash
     ./tools/deps.sh
   #+end_src
 - *make executables* \\
   Simply run =make build=. Read the ~makefile~ and change the options
   as needed.
 - MODE :: Mode (debug, release)
 - LISP :: Lisp (sbcl, cmucl, ccl)
 - CFG :: Config (default.cfg)
 *** Run
 :PROPERTIES:
 :ID:       572b1b68-f099-4bdb-9a06-9aa2186c17cc
 :END:
 #+begin_src shell
   ./demo -i
 #+end_src
 *** Config
 :PROPERTIES:
 :ID:       cadb26fe-5fc1-4639-9bb1-abeb490e91af
 :END:
 Configs can be specified in JSON, TOML, RON, or of course SEXP. See
 =default.cfg= for an example.
 *** Play
 :PROPERTIES:
 :ID:       a8296224-60bc-46e7-9fa0-9a600e9f869e
 :END:
 The high-level user interface is presented as a multi-modal GUI
 application which adapts to the specific application /instances/
 below.
 **** Weather
 :PROPERTIES:
 :ID:       ec07eb9b-3f83-4ff7-9809-b8190572e821
 :END:
 This backend retrieves weather data using the NWS API.
 **** Stocks
 :PROPERTIES:
 :ID:       a61d9eff-ffc9-42a4-a5ac-16d4a0cbd24e
 :END:
 The 'Stocks' backend features a stock ticker with real-time analysis
 capabilities.
 **** Bench
 :PROPERTIES:
 :ID:       a23c7805-2b48-4910-9231-96a90a0467f2
 :END:
 This is a benchmark backend for testing the capabilities of our
 demo. It spins up some mock services and allows fine-grained control
 of input/throughput.
 ** tasks
 :PROPERTIES:
 :ID:       ba4bdf0d-c29f-4643-af33-09de5e5d1d45
 :END:
 *** TODO DSLs
 :PROPERTIES:
 :ID:       4d7b2ef7-569a-40bf-b37d-d2e1d4dc01e2
 :END:
 - consider tree-sitter parsing layout, use as a guide for developing a
   single syntax which expands to Rust or C.
 - with-rs
 - with-c
 - with-rs/c
 - with-cargo
 - compile-rs/c
 **** TODO rs-macroexpand
 :PROPERTIES:
 :ID:       ab16b2fb-ec87-4c0e-b0be-185a4ae93537
 :END:
 - rs-gen-file
 - rs-defmacro
 - rs-macros
 - rs-macroexpand
 - rs-macroexpand-1
 **** TODO c-macroexpand
 :PROPERTIES:
 :ID:       6e97665d-3e29-4c3a-8040-0a15eeb4bf0a
 :END:
 - c-gen-file h/c
 - c-defmacro
 - c-macros
 - c-macroexpand
 - c-macroexpand-1
 **** TODO slint-macroexpand
 :PROPERTIES:
 :ID:       629303bb-ab04-4840-8264-18f1b43fa748
 :END:
 - slint-gen-file
 - slint-defmacro
 - slint-macros
 - slint-macroexpand
 - slint-macroexpand-1
 **** TODO html (using who)
 :PROPERTIES:
 :ID:       09eb1d3a-4f73-4a29-95cf-92d589f7fec9
 :END:
 *** TODO web templates
 :PROPERTIES:
 :ID:       6c70f44f-4d03-4663-92c8-e4a5b8fedfc1
 :END:
 create a basic static page in CL which will be used to host Slint UIs
 and other WASM doo-dads in a browser.
 *** TODO CLI
 :PROPERTIES:
 :ID:       bd213f5b-1abf-450f-b3f3-6093dd050ea8
 :END:
 using clingon, decide on generic options and write it up
 *** TODO docs
 :PROPERTIES:
 :ID:       d5d980f2-c032-4e15-a1e6-067fee7197fc
 :END:
 work on doc generation -- Rust and CL should be accounted for.
 *** TODO tests
 :PROPERTIES:
 :ID:       92d6ff4d-a643-4f75-89cf-e19b0e2e0d9a
 :END:
 We have none! need to make it more comfy - set up testing in all Rust
 crates and for the lisp systems.
 * [2024-04-25 Thu]
 :PROPERTIES:
 :ID:       ab37ae2a-b168-455e-a4f5-c8c1baada2be
 :END:
 ** https://docs.gitlab.com/ee/administration/backup_restore/migrate_to_new_server.html
 :PROPERTIES:
 :ID:       d18c642a-b8f9-40ff-9f99-df4e9764bbd5
 :END:
 * [2024-07-31 Wed]
 :PROPERTIES:
 :ID:       4b9ab3c7-5ea3-4f1e-9e0e-db4be8548de4
 :END:
 ** alpine builders
 :PROPERTIES:
 :ID:       ab5c8dc6-6941-4b5d-b624-8b232385ebf1
 :END:
 - make sure to apk add:
   - git, hg
   - clang
   - make
   - linux-headers
   - zstd-dev
   - libc-dev?
 * [2024-08-04 Sun]
 :PROPERTIES:
 :ID:       cb3f0071-a1b9-4ef4-897a-c5edd3435a5e
 :END:
 ** bookmarks
 :PROPERTIES:
 :ID:       2a650229-df59-46b9-bd92-f5cdd9eab4d2
 :END:
 - How should such objects be represented within CORE?
 - skel/homer mostly
   - already have alias
 - not sure about obj/otherwise, prob not
 * [2024-08-08 Thu]
 :PROPERTIES:
 :ID:       298b1333-c6f7-45d3-9829-c4c831e84707
 :END:
 ** Intelligent Design in Software
 :PROPERTIES:
 :ID:       c2e682b3-d72a-41b3-b093-16614e7852ff
 :END:
 - starting from a space where there are no external influences - a biome
 - answer questions regarding the nature of the software and its capabilities
 - incrementally adjust inter-dependencies
 - optimize
 - protect the biome at all costs
 - focus on composition
 - build applications
 - re-integrate lessons learned
 * [2024-08-16 Fri]
 :PROPERTIES:
 :ID:       2877ab07-156b-4b00-97fa-9b12952f0ac1
 :END:
 ** keys.compiler.company
 :PROPERTIES:
 :ID:       26922400-2249-4e1d-a2ed-8014df5ad341
 :END:
 :LOGBOOK:
 - State "NOTE"       from              [2024-08-16 Fri 22:00]
 :END:
 - public openpgp server
 - [[https://keys.compiler.company][keys.compiler.company]]
 - https://keys.openpgp.org/
 - [[vc:packy/hagrid/][packy/hagrid]]