2 #+author: Richard Westhaver
3 #+email: richard.westhaver@gmail.com
4 #+setupfile: ../clean.theme
7 :ID: 3a6f861e-f464-4512-8d13-09e59fe6cbc5 11 :ID: 2d40eec1-62f4-4f3d-9be7-110400808e06 13 In heptapod we have a root group named
=comp=, containg a variety of
14 subgroups. Some of these groups should be public, while others are
15 internal to comp members exclusively. Within each subgroup, we should
16 have the root group members automatically granted privileged access to
17 projects. This is relevant for the
=startup= subgroup in particular,
18 where each project is potentially maintained by multiple non-root
21 We also need to consider how we will manage subrepos across the
22 organization. It is about time we start integrating HG bundles and
23 potentially mirrors. For our core VC pipeline we should have no
24 reliance on Git, but this may be difficult. It depends on the behavior
27 Bookmarks/tags should be used for milestones in the root group and are
28 infrequent. They are more frequent in projects with a regular release
30 ** Approaching Webapps 32 :ID: 0a52be58-990a-47da-ba5a-a7da7192361c 34 I started poking around in the webapp space again so that I can launch
35 a landing page for NAS-T quickly. The Rust situation has improved
36 somewhat on the frontend side, and the axum backend stack is nice.
38 This might seem like a lot of Rust and not a lot of Lisp, which it is,
39 but there's still room for Lisp wherever we need it. It mostly plays a
40 role in the backend, servicing the database and responding to requests
41 from the Rust edges. All of the important tests for the web APIs are
42 also written in Lisp. We will almost certainly use Lisp for all static
43 processing and HTML generation at compile-time.
45 This I believe, is the appropriate way to integrate Lisp into a
46 cutting-edge web-app. You get the good parts of Lisp where you need
47 them (interactive debugging, dynamic language, REPL) and avoid the bad
48 parts (OOB optimization, RPS performance) in areas where the customer
49 would be impacted. In this domain, Lisp takes the form of a glue
50 rather than the bricks and mortar it sometimes appears to us as.
54 :ID: dbc29b84-e1ca-4629-a515-b9277e05f2a8 58 :ID: a51b54d9-5663-411e-b0ba-d1405622eb88 62 :ID: 34dc9d12-d651-4204-9abb-a6a9a6aeb07e 66 :ID: 909e1127-14f5-4783-97af-65aa25439226 70 :ID: d89dc59d-25d9-4304-bcb8-379dd43cc3f7 74 :ID: 7c4f08b7-2bc0-4a85-a6ca-2decdce8dc91 78 :ID: 42ff77c2-69c9-4f89-a297-88efd592502e 82 :ID: c53c4d87-b0fd-40de-8db7-b92c22dc946c 86 :ID: 85c5f471-cf87-49b7-a6ce-5c1526df4080 90 :ID: f92d34dd-7b24-498c-a5c8-29fd3b528092 94 :ID: a967645f-5df3-4bed-9e7b-a1adaed3796c 98 :ID: 6329c6f9-c54c-4eb0-99e7-4772399161e7 102 :ID: 638aa81a-8926-480a-854f-0434e8119426 107 :ID: 964c6bac-486a-4459-825f-6e7e9351c9ab 111 :ID: 52994125-5689-402a-ac61-680aa690bf24 114 Factor is a cool concatenative lang but unfortunately the C interface
115 (vm/master.h) no longer exists on the master branch.
118 :ID: 60ca3000-2ff5-4372-93fb-d5c311fb6409 121 **** https://hypercubed.github.io/joy/html/j02maf.html 123 :ID: 56e64d52-4950-4fec-b933-73d1cd8048d1 126 **** [[https://builds.openlogicproject.org/content/incompleteness/arithmetization-syntax/arithmetization-syntax.pdf][arithmetization of syntax]] 128 :ID: 59979091-395d-4067-9c3d-e557fc5287ee 132 :ID: 4b842734-135e-4c86-9337-3841c1241d3b 134 These notes pertain to Lisp. More specifically, ANSI Common Lisp in
137 - https:
//github.com/lispnik/iup/ - doesn't support MacOS yet, looks
139 - what we really need is wasm compiler.. TBD
142 :ID: f92ce2ba-1b66-42f4-b5e2-f1586b14760b 146 :ID: 31512714-9c30-4144-9673-327808a18767 148 - [2023-07-05 Wed]
\\ 149 important part of the Rust ecosystem, another dtolnay
150 contribution. If you want to program a
/data/ format in the Rust
151 ecosystem, this is how you do it.
153 The way it works is that you define some special structs, a
154 Serializer and a Deserializer which implement the Serialize and
155 Deserialize traits provided by serde, respectively.
157 You can use these structs to provide your public API. The
158 conventional choice is public top-level functions like from-str
159 and to-string. That's it, your serialization library can now read and
160 write your data format as Rust data types.
162 [[https://serde.rs/enum-representations.html][enum-representations]] 163 - the default behavior is an externally tagged representation (verbose)
165 The docs use strings as core IO when implementing a custom format,
166 but the convention is to implement for T where T is bound by std::io
167 Read or Write trait. Then you can provide a more robust public API
168 (from_bytes, from_writer, etc).
171 :ID: 060042a0-d581-411a-9091-d16d11860d89 175 :ID: f09d7294-a997-4715-bceb-ffe6af7f3978 179 :ID: 765ad403-54e1-49cf-b4e6-918de9f910c2 181 [[https://www.nushell.sh/][~]] 182 [[https://www.nushell.sh/cookbook/][cookbook]] 183 [[https://github.com/nushell/nu_scripts][nu_scripts]] 186 :ID: 693e0604-1f5f-4f45-ade7-fc67cd278518 190 :ID: b2f8d58f-a486-4fe9-895b-504987c0f63d 192 We're leveraging AWS for some of our public web servers for now. It's
193 really not realistic to expect that my home desktop and spotty Comcast
194 internet can serve any production workflow. What it
/is/ capable of is
195 a private VPN, which can communicate with AWS and other cloud VPN
196 depots via WireGuard (
[[https://dev.to/gabrieltetzner/setting-up-a-vpn-with-wireguard-server-on-aws-ec2-4a49][article]]).
198 I currently use Google Domains for nas-t.net, otom8.dev, and
199 rwest.io - but that business is now owned by squarespace, so I would
200 rather move it to Route53.
202 We have archlinux ec2 image builds
[[https://wiki.archlinux.org/title/Arch_Linux_AMIs_for_Amazon_Web_Services][here]] and
[[https://gitlab.com/anemos-io/archlinux-ec2][here]] - only half work and not
203 maintained, but it's a start. I'm not even sure if I should stick with
204 arch or cave and use Ubuntu or AWS Linux. We can serve the static
205 services with little cost, the only big spender will be the heptapod
206 instance which requires a larger instance and some workers.
208 We'll try to keep the cost at or around $30/month.
211 :ID: c097323d-ad44-43a8-9e1c-dea7ad9298b3 215 :ID: 836b5ca7-6c6b-4d57-8b44-3bb4a8f38a38 219 :ID: fc9a94e1-91c5-4915-90b8-73218fa3b8bc 222 - State "TODO" from [2023-04-07 Fri 23:24] 232 :ID: c0613a13-7ccb-4af9-b47e-e14a41c782c2 235 - State "TODO" from "TODO" [2023-04-07 Fri 23:22] 240 :ID: 6bd29c37-4b23-4a05-b3a4-72b6ee712f27 243 - State "TODO" from [2023-04-07 Fri 23:33] 247 :ID: 3bfa733e-d777-4e82-b108-93ba74c4dc08 251 :ID: 0f3b6182-1dd1-4e65-8b1c-4630a46fff4c 255 :ID: e9fc19d5-daea-40a5-a0f2-882b925e58f2 259 :ID: c2e62af0-0397-4560-a283-b33d12fbd966 263 :ID: d2466880-1721-4386-b385-ded06d5b15cf 267 :ID: fb722462-9e11-40c1-a5d7-30698dba7e20 271 :ID: 663cd894-0a5f-4352-acf1-4528b9efeeaf 275 :ID: b4264fa2-6e96-4679-bd17-2428ba3489ee 278 - State "TODO" from [2023-04-07 Fri 23:52] 282 :ID: d9661e4a-0328-4864-878d-096f76ddaf55 285 - State "TODO" from [2023-04-07 Fri 23:52] 289 :ID: 501593d4-702a-40cb-8a8f-19edfe2f9f29 292 - State "TODO" from [2023-04-07 Fri 23:34] 296 :ID: 24700a26-b238-48dd-a24f-5539984b4fa6 299 - State "TODO" from [2023-04-07 Fri 23:51] 301 split out from rlib to separate package
302 - a purely OOP class library
305 :ID: fa1c9563-338b-4a56-bfc3-6f4983f2a732 308 - State "TODO" from [2023-04-07 Fri 23:34] 310 *** source categories 312 :ID: c43fdb5d-4f52-478e-9384-ec0e6e89cf09 314 - need a way of extracting metadata from a repo
315 - need ability to search and query libs/packages
316 - separate modules based on where they belong in our stack?
322 *** generic query language 324 :ID: 8647801a-d5ad-446d-8ae1-fd896bce129a 332 :ID: ff7c2876-e471-480b-b532-9921dbd3bc89 335 - Note taken on [2023-10-24 Tue 22:16] \\ 336 graph database, build on rocksdb 338 insidious Big Brother database.
339 - an application built with obj
344 :ID: 00e5fe6c-80b4-4172-a9cb-ce8adc40d3dc 347 - State "NOTE" from "TODO" [2024-08-18 Sun 18:46] 354 :ID: 8d2c32db-769f-482d-86b7-b476e21a438f 356 ** DRAFT dylib-skel-1 358 :ID: b4d1bc91-f344-45fd-becc-cb20f00a3a61 360 - State "DRAFT" from [2023-11-05 Sun 22:23]
363 :ID: 2e490c4b-344e-4790-9184-1c05ba675f15 365 Our core languages are
[[https://www.rust-lang.org/][Rust]] and
[[https://lisp-lang.org/][Lisp]] - this is the killer combo which will allow NAS-T
366 to rapidly develop high-quality software. As such, it's crucial that these two very
367 different languages (i.e. compilers) are able to interoperate seamlessly.
369 Some interop methods are easy to accomodate via the OS - such as IPC or data sharing,
370 but others are a bit more difficult.
372 In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something
373 that
/can/ be difficult, due to some complications with Rust and because this is not the
374 most popular software stack (yet ;). This is an experiment and may not make it to our
375 code-base, but it's definitely something worth adding to the toolbox in case we need it.
379 :ID: 985019fc-612a-44ab-b726-b9067432ad87 381 The level of interop we're after in this case is
[[https://en.wikipedia.org/wiki/Foreign_function_interface][FFI]].
383 Basically, calling Rust code from Lisp and vice-versa. There's an article about calling
384 Rust from Common Lisp
[[https://dev.to/veer66/calling-rust-from-common-lisp-45c5][here]] which shows the basics and serves as a great starting point
385 for those interested.
388 :ID: 2f71a3c1-0b14-46a6-9d8d-f6ec697729cc 390 The complication(s) with Rust I mentioned early is really just that
/it is not C/.
=C= 391 is old, i.e. well-supported with a stable ABI, making the process of creating bindings
392 for a C library a breeze in many languages.
394 For a Rust library we need to first appease the compiler, as explained in
[[https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c][this section]] 395 of the Rustonomicon. Among other things it involves changing the calling-convention of
396 functions with a type signature and editing the Cargo.toml file to produce a
397 C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used
402 :ID: 4ea79f68-55ec-4da3-a184-8343d49532b6 404 Using FFI involves some overhead. Check
[[https://github.com/dyu/ffi-overhead][here]] for an example benchmark across a few
405 languages. While building the NAS-T core, I'm very much aware of this, and will need a
406 few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular,
407 I'm concerned about crossing multiple language barriers (Rust
<->C
<->Lisp).
409 *** Rust -> C -> Lisp 411 :ID: a498276c-8525-4a43-aa40-4b05f76a29a9 415 :ID: 19f96ef7-af92-496e-9d42-70c4d4c85051 417 For starters, I'm going to assume we all have Rust (via
=rustup=) and Lisp (
=sbcl= only)
418 installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
422 :ID: c929e0b6-b6f2-4383-9412-1610329ab28c 424 Create a new library crate. For this example we're focusing on a 'skeleton' for
425 /dynamic/ libraries only, so our experiment will be called
=dylib-skel= or
*dysk* for
427 src_sh[:exports code]{cargo init dysk --lib && cd dysk}
429 A
=src/lib.rs= will be generated for you. Go ahead and delete that. We're going to be
430 making our own
=lib.rs= file directly in the root directory (just to be cool).
432 The next step is to edit your
=Cargo.toml= file. Add these lines after the
=[package]= 433 section and before
=[dependencies]=:
434 #+begin_src conf-toml 436 crate-type = ["cdylib","rlib"]
443 This tells Rust to generate a shared C-compatible object with a
=.so= extension which we
444 can open using
[[https://man.archlinux.org/man/dlopen.3.en][dlopen]].
447 :ID: 256ac288-c5a0-473a-ab65-2d6503bd423c 451 :ID: fc476f64-6b68-417a-8540-ca23ce27fa25 453 Next, we want the
=cbindgen= program which we'll use to generate header files for
454 C/C++. This step isn't necessary at all, we just want it for further experimentation.
456 src_sh[:exports code]{cargo install --force cbindgen}
458 We append the
=cbindgen= crate as a
/build dependency/ to our
=Cargo.toml= like so:
459 #+begin_src conf-toml 465 :ID: 111e27f7-0b9c-4eef-9117-f7c8ba3f511c 467 #+begin_src conf-toml :tangle cbindgen.toml 469 autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
470 include_version = true
473 after_includes = "#define DYSK_VERSION \"0.1.0\""
477 documentation_style = "c99"
478 usize_is_size_t = true
484 :ID: 9fc271b2-9acb-4f4b-aa61-82d60d2ddb9e 486 #+begin_src rust :tangle build.rs 487 fn main() -> Result<(), cbindgen::Error> {
488 if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
489 b.write_to_file("dysk.h"); Ok(())}
490 else { panic!("failed to generate dysk.h from cbindgen.toml") } }
494 :ID: 6b524921-2ae0-43f0-bb85-d9955b0e689c 496 #+begin_src rust :tangle lib.rs 497 //! lib.rs --- dysk library
498 use std::ffi::{c_char, c_int, CString};
500 pub extern "C" fn dysk_hello() -> *const c_char {
501 CString::new("hello from rust").unwrap().into_raw()}
503 pub extern "C" fn dysk_plus(a:c_int,b:c_int) -> c_int {a+b}
505 pub extern "C" fn dysk_plus1(n:c_int) -> c_int {n+1}
509 :ID: cc7c6538-33a6-40c6-94ef-2a9c259c975a 511 #+begin_src rust :tangle test.rs 512 //! test.rs --- dysk test
513 fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::dysk_plus1(2 as core::ffi::c_int);}}
517 :ID: 337a24d1-f305-4e1a-9052-47a53591cb2f 520 cargo build --release
524 :ID: a4813269-92fb-4f52-aef0-3a36dce3cf69 526 #+begin_src lisp :tangle dysk.lisp 527 (load-shared-object #P"target/release/libdysk.so")
528 (define-alien-routine dysk-hello c-string)
529 (define-alien-routine dysk-plus int (a int) (b int))
530 (define-alien-routine dysk-plus1 int (n int))
531 (dysk-hello) ;; => "hello from rust"
535 :ID: 1a8ca441-f290-46c7-b979-1e7e0d1d063b 538 time target/release/dysk-test
540 #+begin_src lisp :tangle test.lisp 541 (time (dotimes (_ 500000000) (dysk-plus1 2)))
545 :ID: 63b7f08e-c2df-4a5e-a14f-a8abee2a12c5 549 :ID: c5f4f25c-071b-4a2d-85bc-08676eacce5d 552 (defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object cons))
553 (make-instance 'cl-dot:node
554 :attributes '(:label "cell \\N"
556 (defmethod cl-dot:graph-object-points-to ((graph (eql 'example)) (object cons))
558 (make-instance 'cl-dot:attributed
560 :attributes '(:weight 3))))
562 (defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object symbol))
563 (make-instance 'cl-dot:node
564 :attributes `(:label ,object
568 :fillcolor "#ccccff")))
569 (let* ((data '(a b c #1=(b z) c d #1#))
570 (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
571 '(:rankdir "LR" :layout "twopi" :labelloc "t"))))
572 (cl-dot:dot-graph dgraph "test-lr.svg" :format #+nil :x11 :svg))
579 (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
581 (cl-dot:print-graph dgraph))
586 :ID: 347fbd67-0b04-46a5-a1df-698a7398664d 590 :ID: 7fb40057-05aa-445b-bc75-0b1b6ca2c994 592 need a way of indexing, referring to, and annotating objects such as
593 URLs, docs, articles, source files, etc.
595 What is the best way to get this done?
598 :ID: 92d35872-75a9-448f-acdf-e14daae0c956 600 ** doc best practices 602 :ID: ba4a3074-f29d-4f81-b4ba-d6a523b3f291 604 https://rust-lang.github.io/api-guidelines/documentation.html
606 also: https:
//lisp-lang.org/style-guide/ 609 :ID: 46c2a675-1dcb-4058-9284-55605f523463 613 :ID: 7845b0fb-b5d3-4488-bdba-4876d9c8816e 615 If you've met me in the past decade, you probably know that I am
616 extremely passionate about computers. Let me first explain why.
618 On the most basic level computers are little (or big) machines that
619 can be programmed to do things, or
/compute/ if we're being
622 They host and provide access to the Internet, which is a pretty big
623 thing, but they do little things too like unlock your car door and
624 tell your microwave to beep at you. They solve problems. Big or small.
626 They're also
/everywhere/ - which can be scary to think about, but
627 ultimately helps propel us into the future.
629 There's something pretty cool about that - when you look at the
630 essence of computation. There are endless quantities of these machines
631 which follow the same basic rules and can be used to solve
/real/ 636 :ID: b17c853e-1728-4bd3-94c5-82e5b33d7398 638 Now, let us consider the
/programmer/. They have power.
/real/ 639 power. They understand the language of computers, can whisper to them
640 in various dialects. It can be intimidating to witness until you
641 realize how often the programmer says the wrong thing - a bug.
643 In reality, the programmer has a symbiotic relationship with
644 computers. Good programmers understand this relationship well.
647 One day after I got my first job at a software company, I remember
648 being on an all-hands meeting due to a client service outage. We had
649 some management, our lead devs, product team, and one curious looking
650 man who happened to be our lead IT consultant who had just joined. He
651 was sitting up on a hotel bed, shirtless, vaping an e-cig, typing
652 away in what I can only imagine was a shell prompt.
654 After several minutes he took a swig from a bottle of Coke and said
655 "Node 6 is sick." then a few seconds later our services were
656 restored. For the next hour on the call he explained what happened and
657 why, but that particular phrase always stuck with me. He didn't say
658 Node 6 was down, or had an expired cert - his diagnosis was that /it/
662 The more you work closely with computers, the more you start to think
663 of them this way. You don't start screaming when the computer does the
664 wrong thing, you figure out what's wrong and learn from it. With
665 experience, you start to understand the different behaviors of the
666 machines you work with. I like to call this
/Machine Empathy/.
670 :ID: a297e5f9-a875-4512-b126-9a2b3e75c1d8 672 I already mentioned bugs - I write plenty of those, but usually I try
673 to write
/programs/. Programs to me are like poetry. I like to think
674 they are for the computer too.
676 Just like computers,
/computer programs/ come in different shapes and
677 sizes but in basic terms they are sets of instructions used to control
680 You can write programs to do anything - when I first started, my
681 programs made music. The program was a means to an end. Over time, I
682 started to see the program as something much more. I saw it as the
685 [fn:1] ... perform computations
689 :ID: 80d800de-d305-4b95-a28c-a6f19d6f7a11 691 Something that is missing from many organizations big or large, is an
692 effective way to store and access information, even about their own
695 It can be difficult problem to solve - usually there's the official
696 one, say Microsoft Sharepoint and then the list of unofficial sources
697 which becomes tribal corporate hacker knowledge. Maybe the unofficial
698 ones are more current, or are annotated nicely, but their very
699 existence breaks the system. There's no longer a single source of
702 My priority in this department is writing services which process and
703 store information from a variety of sources in a distributed knowledge
704 graph. The graph can later be queried to access information on-demand.
706 My idea of infrastructure is in fact to build my own Cloud. Needless
707 to say I don't have an O365 subscription, and wherever possible I'll
708 be relying on hardware I have physical access to. I'm not opposed to
709 cloud services at large but based on principle I like to think we
710 shouldn't be built on them.
713 :ID: 12189e52-0b21-4171-8b93-eaa6a82117ce 715 ** https://cal-coop.gitlab.io/utena/utena-specification/main.pdf 717 :ID: 5c3aac3a-67e1-4867-9ac4-b58a3247f9e0 719 from the author of cl-decentralise2. draft specification of a
720 /Maximalist/ Computing System.
723 :ID: b5b9eae5-7d13-435a-9810-68e952b2f887 727 :ID: 28738250-c34e-45e2-8793-f8245472d3a3 729 https:
//github.com/awesomedata/awesome-public-datasets 730 https://docs.openml.org/Datasets/ 731 https:
//wiki.pathmind.com/open-datasets 734 :ID: e2012a31-462a-4f3b-9b92-8a812626ac69 738 :ID: 28581a8e-ae05-45de-9744-24f447668418 741 sb-sys:*runtime-dlhandle* 742 sb-fasl:+fasl-file-version+ 743 sb-fasl:+backend-fasl-file-implementation+ 744 sb-debug:print-backtrace 745 sb-debug:map-backtrace 746 sb-pretty:pprint-dispatch-table 749 sb-walker:define-walker-template 750 sb-walker:macroexpand-all 753 sb-kernel:*eval-calls* 754 sb-kernel:*gc-pin-code-pages* 755 sb-kernel:*restart-clusters* 756 sb-kernel:*save-lisp-clobbered-globals* 757 sb-kernel:*top-level-form-p* 758 sb-kernel:*universal-fun-type* 759 sb-kernel:*universal-type* 760 sb-kernel:*wild-type* 761 sb-kernel:+simd-pack-element-types+ 763 (sb-vm:boxed-context-register) 764 (sb-vm:c-find-heap->arena) 765 (sb-vm:copy-number-to-heap) 766 (sb-vm:dump-arena-objects) 769 (sb-vm:show-heap->arena) 770 (sb-vm:with/without-arena) 771 (sb-cltl2:{augment-environment,compiler-let,define-declaration,parse-macro}) 772 (sb-cltl2:{declaration-information, variable-information, function-information}) 781 :ID: 1c16670e-5470-4650-a07c-45c5884b92b8 783 ** [[https://github.com/sigmf/SigMF][SigMF]]
785 :ID: f21a6477-6a3f-48f7-8a9e-43f5cf65286c 788 Sharing sets of recorded signal data is an important part of science
789 and engineering. It enables multiple parties to collaborate, is often
790 a necessary part of reproducing scientific results (a requirement of
791 scientific rigor), and enables sharing data with those who do not have
792 direct access to the equipment required to capture it.
794 Unfortunately, these datasets have historically not been very
795 portable, and there is not an agreed upon method of sharing metadata
796 descriptions of the recorded data itself. This is the problem that
799 By providing a standard way to describe data recordings, SigMF
800 facilitates the sharing of data, prevents the "bitrot" of datasets
801 wherein details of the capture are lost over time, and makes it
802 possible for different tools to operate on the same dataset, thus
803 enabling data portability between tools and workflows.
806 the-spec: https:
//github.com/sigmf/SigMF/blob/sigmf-v1.x/sigmf-spec.md 807 ** [[https://www.libvolk.org/][LibVOLK]]
809 :ID: ae34135c-488e-4aff-b795-a4d0d558f22b 811 Vector-Optimized Library of Kernels (simd)
812 ** [[https://docs.kernel.org/fb/framebuffer.html][/dev/fb*]] 814 :ID: 7819a970-1f37-40a6-985f-905b25a08a4a 816 framebuffers, used by fbgrab/fbcat program
817 ** [[https://docs.kernel.org/block/ublk.html][ublk]] 819 :ID: 5ccaed39-7055-4a51-ab62-769a73425270 821 https://github.com/ming1/ubdsrv
824 :ID: 002e9449-adb7-4902-ad84-5fbf832eb890 827 make problems smaller.
831 - doesn't need mentioning more and more
834 :ID: 3f3aa409-8b88-45bf-8dbd-6d06e4e053b5 836 ** TODO taobench demo 838 :ID: 81ac81c3-f183-4345-aaf3-ef7ef0434947 841 - State "TODO" from [2024-01-21 Sun 00:32] 843 https:
//github.com/audreyccheng/taobench - shouldn't have missed this :) 844 obviously we need to implement this using core -- in demo/bench/tao? 845 ** TODO clap completion for nushell 847 :ID: a4b35281-83f7-4244-93e9-20e57ba31eb3 850 - State "TODO" from [2024-01-20 Sat 23:23] 852 https://github.com/clap-rs/clap/tree/master/clap_complete_nushell 853 ** Dataframe scripting 855 :ID: 8dc7339e-71c6-4a60-b285-c0d56d1cbc24 857 https://studioterabyte.nl/en/blog/polars-vs-pandas 858 nushell supports DFs, polars underneath? 859 https://www.nushell.sh/book/cheat_sheet.html
866 :ID: d5131c54-9082-4424-91dd-70f17561bbf7 870 :ID: 2b0f67de-1217-48a6-b5b7-a85d3ca15926 872 - [[https://cloud.google.com/free/docs/free-cloud-features][Free Cloud Features]] 873 + 90-day $300 credits
874 + e2-micro - free hours worth 1 instance/month
878 :ID: 0fd0dedd-59b0-4b1a-9241-f65699e20c46 883 :ID: 8315a33f-dd22-4374-979a-9cccd5b5b6a9 888 :ID: 3f1e1116-c123-4fdf-a336-0d9b50c3dcc7 890 - [[https://www.oracle.com/cloud/free/?intcmp=ohp052322ocift][OCI Free Tier]] 891 + always free: 2 x oracle autonomous DB
892 + 2 x AMD Compute VMs
893 + up to 4 x ARM Ampere A1 with 3k/cpu/hr and 18k/gb/h per month
894 + block/object/archive storage
895 + 30-day $300 credits
898 :ID: 0ff31116-18c6-4471-a6fd-01d7b4e456ec 900 ** trash as block device 902 :ID: 4452b84f-f3a5-44f1-b60a-4efa6413cbb9 905 - State "NOTE" from [2024-01-29 Mon 20:53] 906 - State "NOTE" from [2024-01-29 Mon 20:53] 908 in nushell there is option for rm command to always use 'trash' -
909 AFAIK the current approach is via a service (trashd).
911 An interesting experiment would be to designate a block device as
912 'trash' - may be possible to remove reliance on a service
914 may be an opportunity for ublk driver to shine - instead of /dev/null
915 piping we need a driver for streaming a file to /dev/trash
918 :ID: aa10a1d7-f1e7-4b92-b91a-56c234d52204 921 - State "NOTE" from [2024-01-29 Mon 16:28] 923 - mostly x86_64 machines - currently 2 AWS EC2 instances, some podman containers, and our home beowulf server:
926 - mid-size tower enclosed (Linux/Windows)
928 - Intel Core i7-6700K
931 - NVIDIA GeForce GTX 1060
934 - Samsung SSD 850: 232.9GB
935 - Samsung SSD 850: 465.76GB
936 - ST2000DM001-1ER1: 1.82TB
937 - WDC WD80EAZZ-00B: 7.28TB
938 - PSSD T7 Shield: 3.64TB
939 - My Passport 0820: 1.36TB
941 - 16GB (2*8) [64GB max]
944 - MacBook Pro 2019 (MacOS/Darwin)
960 - 64-bit 6-core 4x ARM Cortex A53 + 2x ARM Cortex A72
966 - Raspberry Pi 4 Model B
968 - Cortex-A72 (ARM v8) 64-bit SoC
975 :ID: 8601ee46-5bf7-4a7a-a491-52c50eee619a 979 :ID: 87d0f5d4-c6fe-4e21-911e-2639a24e89f9 983 let ms = '1trc/measurements-0.parquet'
985 | dfr group-by station
987 (dfr col measure | dfr min | dfr as "min")
988 (dfr col measure | dfr max | dfr as "max")
989 (dfr col measure | dfr sum | dfr as "sum")
990 (dfr col measure | dfr count | dfr as "count")
995 :ID: b3af34f4-367d-4a83-9c69-088f9ba4190e 999 :ID: b4b29989-28f2-48db-ad84-cd8d08eeefdf 1002 - State "NOTE" from [2024-02-18 Sun 11:55] 1004 In the past few months there has been drama regarding Wayland vs X. It
1005 seems to be on everyone's minds after Artem's freakout issue and the
1006 follow up YT vids/comments.
1008 I admit that it made me reconsider the fitness of WL as a whole -
1009 there was a github gist that made some scathing arguments against it.
1011 It's an odd debate though. I think there are many misunderstandings.
1013 So first off, if we look at the homepage
1014 https:
//wayland.freedesktop.org/, Wayland claims it is a replacement
1015 for X11. It now has
/manifest destiny/, which in my opinion is a great
1018 X-pros seem to agree that Wayland has
/manifest destiny/ - like if you
1019 are building softwares that look remotely like a window system, it's a
1020 successor to X. That's the model of doing things and there's no way
1023 The disagreement starts with how this destiny - of an X2 - should be
1024 fulfilled. X-pros want a fork of X, but it's too late for
1025 that. WL-pros want X to run on top of Wayland compositor:
1026 https://wayland.freedesktop.org/xserver.html.
1028 Xwayland is a problem for me. From the project description: 'if we're
1029 migrating away from X, it makes sense to have a good backwards
1030 compatibility story.' Full disclosure: I have never done significant
1031 work on Xwayland, so perhaps my opinion is unwarranted. But I have no
1032 intention of attempting to maintain a computer system that uses
1033 Wayland and X clients at the same time.
1035 To me, X is ol' reliable. Every distro has first-class X support, and
1036 it runs on most systems with very little user intervention. Where it
1037 doesn't, there is 20+ years of dev history and battle-tested
1038 workarounds for you to find your solution in.
1040 Wayland is the new kid on the block, born just in 2008. It's a fresh
1041 start to one of the most difficult challenges in software - window
1042 systems. A re-write would be pointless though, and so the real
1043 value-add is in design. Wayland is designed as a protocol and
1044 collection of libraries which are implemented in your own
1045 compositor. Coming from Lisp - with ANSI Common Lisp and SRFIs, this
1046 feels right even if the implementation is something very different
1047 (compositor vs compiler).
1049 With X, it is assumed to be much harder to write an equivalent
1050 'compositor'. Here's the thing though - with a significantly complex X
1051 client implementation, it is
/impossible/ to replicate in WL. This is
1052 really the crux of Artemi's argument in his issue. He asked for a 1:1
1053 equivalent X/WL comparison when no such thing exists, and in my
1054 opinion it is a waste of time.
1056 The WL core team is fully aware of this dichotomy, but also that this
1057 is in no way a problem or weakness in either system. It means they're
1058 different systems, goddammit.
1060 If it was up to me, Xwayland wouldn't exist. I understand why it does,
1061 and that it does make things easier for developers who need to support
1062 both, and users who have multiple apps with multiple windowing
1063 requirements. It's a bandaid though, and one that is particularly
1064 dangerous because it re-enforces the idea that Wayland is just X2 and
1065 that they're fully compatible.
1067 What interests me in the Wayland world right now is the idea of a
1068 small, modular, full-stack Wayland compositor API. There are several
1069 'kiosk' based compositors for single applications (cage), but these
1070 aren't complete solutions. It is possible to get much closer to the
1071 metal, and that's where I want to be so that I can build my own APIs
1072 on top - I don't want to live on top of X, and I certainly don't want
1073 to live on top of X on top of WL. I want a
/pure/ solution that hides
1074 as little as possible, exposing the interesting bits.
1077 :ID: 235cb90a-fd5b-43f0-83fa-d0e9f6b13119 1079 ** TODO collect more data 1081 :ID: 69ce5703-d3fe-4df7-abf4-dc93a08ebc12 1084 - State "TODO" from [2024-03-01 Fri 15:27] 1086 https:
//www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
1087 weather - music - etc
1090 :ID: ecb6ee8a-3bc8-4414-9a43-0ad23b887c3b 1092 ** On blocks and devices 1094 :ID: c8304ae3-415d-4ccf-8f4c-1f3f6a555e5b 1097 - State "NOTE" from [2024-03-02 Sat 21:30] 1099 [[https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/dev.html][/dev]] 1100 In Linux, everything is a file.
1102 /dev/ contains special device files - usually block or character
1105 major, minor = category, device
1108 mknod - create special device files
1110 [[https://www.redhat.com/en/blog/linux-block-devices-hints-debugging-and-new-developments][redhat hints]] 1113 dd if=/dev/zero of=myfile bs=1M count=32
1114 losetup --show -f myfile
1116 losetup -d /dev/loop0 #teardown
1120 echo "sup dude" > /dev/loop0
1121 dd if=/dev/loop0 -bs=1
1122 dd if=/dev/nvme0 of=/dev/null progress=true
1124 hdparm -T /dev/nvme0
1128 modprobe scsi_debug add_host=5 max_luns=10 num_tgts=2 dev_size_mb=16
1131 sparsefiles: create with C, dd, or truncate
1133 #+begin_src shell :results output 1139 Usage: truncate OPTION... FILE...
1140 Shrink or extend the size of each FILE to the specified size
1142 A FILE argument that does not exist is created.
1144 If a FILE is larger than the specified size, the extra data is lost.
1145 If a FILE is shorter, it is extended and the sparse extended part (hole)
1146 reads as zero bytes.
1148 Mandatory arguments to long options are mandatory for short options too.
1149 -c, --no-create do not create any files
1150 -o, --io-blocks treat SIZE as number of IO blocks instead of bytes
1151 -r, --reference=RFILE base size on RFILE
1152 -s, --size=SIZE set or adjust the file size by SIZE bytes
1153 --help display this help and exit
1154 --version output version information and exit
1156 The SIZE argument is an integer and optional unit (example: 10K is 10*1024).
1157 Units are K,M,G,T,P,E,Z,Y,R,Q (powers of 1024) or KB,MB,... (powers of 1000).
1158 Binary prefixes can be used, too: KiB=K, MiB=M, and so on.
1160 SIZE may also be prefixed by one of the following modifying characters:
1161 '+' extend by, '-' reduce by, '<' at most, '>' at least,
1162 '/' round down to multiple of, '%' round up to multiple of.
1164 GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
1165 Full documentation <https://www.gnu.org/software/coreutils/truncate>
1166 or available locally via: info '(coreutils) truncate invocation'
1169 test mkfs.btrfs on 10T dummy block device
1171 dd if=/dev/zero of=/tmp/bb1 bs=1 count=1 seek=10T
1173 losetup --show -f /tmp/bb1
1174 mkfs.btrfs /dev/loop0
1179 iostat # pacman -S sysstat
1180 blktrace # paru -S blktrace
1181 iotop # pacman -S iotop
1184 bcc/ trace: Who/which process is executing specific functions against
1187 bcc/biosnoop: Which process is accessing the block device, how many
1188 bytes are accessed, which latency for answering the requests?
1190 at the kernel level besides BPF we got kmods and DKMS,
1192 compression/de-duplication can be done via VDO kernel mod
1194 https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support
1196 ** save-lisp-and-respawn 1198 :ID: e58a96e7-7b88-4e4a-b958-3000603d8baa 1201 - State "NOTE" from [2024-03-02 Sat 22:57] 1209 :ID: 2efa5e4a-f063-49bb-9e61-6917eb115c1a 1212 - State "NOTE" from [2024-03-03 Sun 16:35] 1215 - openlog syslog closelog
1216 - levels: emerg alert crit err warning notice info debug
1220 :ID: 3536e67a-51f7-45ea-98bb-f01093d9f2ca 1222 ** RESEARCH [[https://github.com/guicho271828/sbcl-wiki/wiki/][sbcl-wiki]] 1224 :ID: 33bc81c1-c5fa-4518-9456-58fd67f301ff 1227 - State "RESEARCH" from [2024-03-13 Wed 21:49] 1231 :ID: 2c012572-ea16-408c-b741-d1f6cd0c7961 1235 :ID: 8d015e5e-7ad3-44c3-88cd-81e293a887e0 1239 :ID: d741d3db-0a1b-4024-8d42-77f0edebf593 1243 :ID: d30aec42-a9a1-425b-87ca-1e40db0e69de 1246 - State "NOTE" from [2024-02-04 Sun 20:40] 1248 [[https://github.com/facebook/rocksdb/wiki/Benchmarking-tools][RocksDB benchmarking tools]] 1252 :ID: 76ae24f5-46e8-4b91-8991-41245383d337 1255 - State "NOTE" from [2024-01-25 Thu 22:39] 1257 - API root: https:
//packy.compiler.company 1258 - source packs: https://vc.compiler.company/packy 1261 :ID: 2d6aa6fb-a34c-4ccd-99d3-4f98d33bb2f9 1265 :ID: 8017b6c7-3a04-4e52-b3d6-afc68cbce139 1269 :ID: 2722b7dc-9546-4d94-82b0-abfc02647b4d 1271 Primary data type of the library - typically represents a compressed 1272 archive, metadata, and ops. 1275 :ID: 9ec5d682-da8e-4d02-b2fc-a56a8f209c22 1277 Collection data type, usually contains a set of packs with metadata. 1280 :ID: 8cf2658a-92d5-41ab-8630-4f4e6ddb140d 1282 Represents a Packy instance bound to a UDP socket 1283 ***** PackyEndpointConfig 1285 :ID: 421b8aaa-66e1-4883-937d-fc6705146998 1287 Global endpoint configuration object 1288 ***** PackyClientConfig 1290 :ID: c1899710-034e-4ed6-959a-1eb2df72e5f9 1292 Configuration for outgoing packy connections on an endpoint 1293 ***** PackyServerConfig 1295 :ID: 955e8ce9-ff7e-4f42-9d25-3aeb9846528b 1297 Configuration for incoming packy connection son an endpoint 1298 ***** PackyConnection 1300 :ID: 7075e0c8-65b3-4fbf-9a6b-f27017276a72 1302 Packy connection object 1305 :ID: ea92ad07-d12c-4ee8-a3d4-0955f0af3921 1309 :ID: 6c0fdd7e-bc0a-4217-b315-72fe7766e4df 1313 :ID: 1358e1c5-b215-48eb-9e1b-4116cc32b2ae 1317 :ID: b8298207-4d92-4628-8687-9f4e20c8cfec 1321 :ID: a6fb4fb6-b188-4033-8291-107cf1f00e6d 1325 :ID: 1fd91c8b-ab90-4db0-b74e-1c2b9ae2e440 1329 :ID: 75c0db2e-37cf-4f33-95bd-3f5c95b45c49 1333 :ID: 60de92f4-b4a6-4cae-95bc-174e2036e57b 1337 :ID: e379b71e-8a84-4547-a694-90cad3b7952e 1341 :ID: 73ea9478-5251-40c3-8331-cf042d0b9bf1 1343 ****** start_packy_server 1345 :ID: c8a9bac3-971a-45a7-988f-92db19ef6601 1347 ****** stop_packy_server 1349 :ID: e3122bf1-f770-46a7-a36b-2df9c48bc233 1351 ****** start_packy_registry 1353 :ID: b6d214f2-d6de-4d46-b2c2-c51ff0f3b3e6 1357 :ID: 0e846cbb-5cb4-496f-8100-beae6e66972e 1359 ****** register_pack 1361 :ID: 764642ad-a33b-415b-9c32-8a729a7cd62d 1363 ****** register_user 1365 :ID: 2b92f584-d769-4266-940e-cfbd466d4e62 1367 ****** register_bundle 1369 :ID: 390cb42b-5170-4175-9ccc-b990842356a0 1373 :ID: b0ee90a7-a7ba-41c1-9864-5617f3bb2abe 1375 ** TBD investigate alieneval for phash opps 1377 :ID: 21ab0a6e-cb66-4c5e-9b10-4ac2340ce75d 1380 - State "TBD" from [2024-03-25 Mon 18:56] 1384 :ID: 0ce61202-62b4-4a18-8a2e-fabf0787970e 1388 :ID: 439a4945-de2f-4985-ba6c-fe6421aa9ad9 1390 The backend services are written in Rust and controlled by a simple 1391 messaging protocol. Services provide common runtime capabilities known 1392 as the /service protocol/ but are specialized on a unique
/service 1393 type/ which may in turn register their own
/custom protocols/ (via
1396 Services are capable of dispatching data directly to clients, or
1397 storing data in the
/database/ (sqlite, postgres, mysql).
1399 The frontend clients are pre-dominantly written in Common Lisp and
1400 come in many shapes and sizes. There is a cli-client, web-client
1401 (CLOG), docker-client (archlinux, stumpwm, McCLIM), and native-client
1402 which also compiles to WASM (slint-rs).
1406 :ID: 65140d2c-5d59-48cd-946c-01c380a37f08 1410 :ID: ad0652ca-4abf-4d27-8897-a61a13a6e61d 1412 - *install dependencies* 1416 - *make executables* \\ 1417 Simply run
=make build=. Read the
~makefile~ and change the options
1419 - MODE :: Mode (debug, release)
1420 - LISP :: Lisp (sbcl, cmucl, ccl)
1421 - CFG :: Config (default.cfg)
1424 :ID: b2295442-ccb7-4a5c-8a9e-ea8502aad636 1431 :ID: 600be170-d032-4280-a7b5-c46e3c157206 1433 Configs can be specified in JSON, TOML, RON, or of course SEXP. See
1434 =default.cfg= for an example.
1437 :ID: 56a69ed7-2eb3-4490-95a1-072cee985df5 1439 The high-level user interface is presented as a multi-modal GUI
1440 application which adapts to the specific application
/instances/ 1444 :ID: 804a174b-534c-4a04-966a-b44c0a86fbd4 1446 This backend retrieves weather data using the NWS API.
1449 :ID: 4e7339bb-aaf3-4fab-90e4-05e13f24f881 1451 The 'Stocks' backend features a stock ticker with real-time analysis
1455 :ID: e3cf0add-6ded-4c05-a05d-7ef463b85b08 1457 This is a benchmark backend for testing the capabilities of our
1458 demo. It spins up some mock services and allows fine-grained control
1459 of input/throughput.
1462 :ID: a16a3807-0e06-4c8d-baf6-5dfa8f8f5585 1466 :ID: 3ae4754e-0c0f-4923-9e53-ae962b8638a1 1468 - consider tree-sitter parsing layout, use as a guide for developing a
1469 single syntax which expands to Rust or C.
1475 **** TODO rs-macroexpand 1477 :ID: f612975f-6bcc-4aab-b0d8-65b8ff0a0d0f 1484 **** TODO c-macroexpand 1486 :ID: 49f5c224-026d-4cdc-a326-1742e64c4279 1493 **** TODO slint-macroexpand 1495 :ID: 486beb12-179b-4acc-8507-cdc498edb628 1501 - slint-macroexpand-1
1502 **** TODO html (using who) 1504 :ID: 2e81a19c-874c-4fd7-be0f-f6cd62c660db 1506 *** TODO web templates 1508 :ID: 95f7d6ac-c010-4d1d-a47d-1693bb7d6eab 1510 create a basic static page in CL which will be used to host Slint UIs
1511 and other WASM doo-dads in a browser.
1514 :ID: fc1d5364-279c-4d5f-baea-ed2c110b63f0 1516 using clingon, decide on generic options and write it up
1519 :ID: 467e84e3-e72f-4764-8858-cc15fdc17e8a 1521 work on doc generation -- Rust and CL should be accounted for.
1524 :ID: 70aea01c-d01e-4ddf-aaf5-c86183a7e250 1526 We have none! need to make it more comfy - set up testing in all Rust
1527 crates and for the lisp systems.
1530 :ID: fe24785e-dffc-41f1-9740-ddf823751e31 1532 ** https://docs.gitlab.com/ee/administration/backup_restore/migrate_to_new_server.html 1534 :ID: 264b81f4-f953-4698-9c82-d8e20b6c31f9 1538 :ID: ff4f24b4-4b9c-4706-aceb-3a09276d2e11 1542 :ID: 635925f6-e1d8-47bc-a32f-28bca0f19d2d 1544 - make sure to apk add:
1553 :ID: bc4078cc-7b47-401e-a75f-19b7ce3f0a66 1557 :ID: 6e9c6bce-0513-4628-b6f9-5c234cccdaa3 1559 - How should such objects be represented within CORE?
1561 - already have alias
1562 - not sure about obj/otherwise, prob not
1565 :ID: 36f63b50-d717-41b1-bda8-4521c3b05072 1567 ** Intelligent Design in Software 1569 :ID: 1397d8bb-4848-4d9a-9dcc-ee65bed81c9e 1571 - starting from a space where there are no external influences - a biome
1572 - answer questions regarding the nature of the software and its capabilities
1573 - incrementally adjust inter-dependencies
1575 - protect the biome at all costs
1576 - focus on composition
1577 - build applications
1578 - re-integrate lessons learned
1581 :ID: ceb25ff7-5ad9-4aa3-b093-260da4f35e56 1583 ** keys.compiler.company 1585 :ID: 86c21790-72b5-4ea1-ba49-f7be3f65a8cc 1588 - State "NOTE" from [2024-08-16 Fri 22:00] 1590 - public openpgp server
1591 - [[https://keys.compiler.company][keys.compiler.company]] 1592 - https:
//keys.openpgp.org/ 1593 - [[vc:packy/hagrid/][packy/hagrid]]