2 #+author: Richard Westhaver
3 #+email: richard.westhaver@gmail.com
4 #+setupfile: ../clean.theme
7 :ID: 5a1f17a0-22bb-45bc-a313-ec2cb6c0bfd5 11 :ID: 9928339e-14d7-425d-a465-be87e2784590 13 In heptapod we have a root group named
=comp=, containg a variety of
14 subgroups. Some of these groups should be public, while others are
15 internal to comp members exclusively. Within each subgroup, we should
16 have the root group members automatically granted privileged access to
17 projects. This is relevant for the
=startup= subgroup in particular,
18 where each project is potentially maintained by multiple non-root
21 We also need to consider how we will manage subrepos across the
22 organization. It is about time we start integrating HG bundles and
23 potentially mirrors. For our core VC pipeline we should have no
24 reliance on Git, but this may be difficult. It depends on the behavior
27 Bookmarks/tags should be used for milestones in the root group and are
28 infrequent. They are more frequent in projects with a regular release
30 ** Approaching Webapps 32 :ID: bafaceb9-2798-484d-80b7-7ad5d84fff1a 34 I started poking around in the webapp space again so that I can launch
35 a landing page for NAS-T quickly. The Rust situation has improved
36 somewhat on the frontend side, and the axum backend stack is nice.
38 This might seem like a lot of Rust and not a lot of Lisp, which it is,
39 but there's still room for Lisp wherever we need it. It mostly plays a
40 role in the backend, servicing the database and responding to requests
41 from the Rust edges. All of the important tests for the web APIs are
42 also written in Lisp. We will almost certainly use Lisp for all static
43 processing and HTML generation at compile-time.
45 This I believe, is the appropriate way to integrate Lisp into a
46 cutting-edge web-app. You get the good parts of Lisp where you need
47 them (interactive debugging, dynamic language, REPL) and avoid the bad
48 parts (OOB optimization, RPS performance) in areas where the customer
49 would be impacted. In this domain, Lisp takes the form of a glue
50 rather than the bricks and mortar it sometimes appears to us as.
54 :ID: 7bd213b5-9a4b-4490-b0b8-8a0a9b42a9c6 58 :ID: 07c7016f-c653-4d69-9ca0-366a7584a212 62 :ID: f53afecb-561e-4545-a6d0-dbf6d9504ec4 66 :ID: df9e07cd-986f-45a7-b9f7-ea88313c92ee 70 :ID: 87b019ff-7f05-496d-b7bb-153fea7f8595 74 :ID: b1c52088-4244-4342-b884-c551ce43cc9c 78 :ID: d1969d15-8dc0-4499-bddb-f92fa19aec13 82 :ID: dd6e8169-fba6-45af-aaef-6acd9dbd7b3f 86 :ID: bd393c26-6014-42bb-a425-b9f72e5cee8b 90 :ID: 800ee282-7b5c-465f-a372-344079f9d875 94 :ID: 7f34835c-9135-4015-bdbd-99d28d7b40bb 98 :ID: 9fef114a-e2f3-40b4-bddb-9948b47b441f 102 :ID: 04594cbd-5847-434d-aaf6-a7e23058d980 107 :ID: ad2041de-3915-48f9-950c-03b01968952d 111 :ID: edb7e4a8-d07d-4953-9b64-decdca1dd76f 114 Factor is a cool concatenative lang but unfortunately the C interface
115 (vm/master.h) no longer exists on the master branch.
118 :ID: 257365d1-1b2e-4abd-a2e3-5d1705e41bcf 121 **** https://hypercubed.github.io/joy/html/j02maf.html 123 :ID: a685ec2b-fbdc-4724-ab22-12d8a4498f81 126 **** [[https://builds.openlogicproject.org/content/incompleteness/arithmetization-syntax/arithmetization-syntax.pdf][arithmetization of syntax]] 128 :ID: 895a803a-63d5-4dfb-8ceb-c20627d97b02 132 :ID: 8109c42f-ccbb-4a08-ab99-1dced8b22a9f 134 These notes pertain to Lisp. More specifically, ANSI Common Lisp in
137 - https:
//github.com/lispnik/iup/ - doesn't support MacOS yet, looks
139 - what we really need is wasm compiler.. TBD
142 :ID: 94a9a2b7-3559-476c-9edf-66b3b0c8cbb0 146 :ID: ffb4e87c-e265-4549-91f9-093e714708c0 148 - [2023-07-05 Wed]
\\ 149 important part of the Rust ecosystem, another dtolnay
150 contribution. If you want to program a
/data/ format in the Rust
151 ecosystem, this is how you do it.
153 The way it works is that you define some special structs, a
154 Serializer and a Deserializer which implement the Serialize and
155 Deserialize traits provided by serde, respectively.
157 You can use these structs to provide your public API. The
158 conventional choice is public top-level functions like from-str
159 and to-string. That's it, your serialization library can now read and
160 write your data format as Rust data types.
162 [[https://serde.rs/enum-representations.html][enum-representations]] 163 - the default behavior is an externally tagged representation (verbose)
165 The docs use strings as core IO when implementing a custom format,
166 but the convention is to implement for T where T is bound by std::io
167 Read or Write trait. Then you can provide a more robust public API
168 (from_bytes, from_writer, etc).
171 :ID: 05a7192a-4c05-44d9-b859-e2a0cff53d86 174 [2024-09-22 Sun 21:07] <- [[id:26922400-2249-4e1d-a2ed-8014df5ad341][keys.compiler.company]] 178 :ID: 6800e89d-f49e-4ffe-bbab-ed10fec43174 182 :ID: b4048114-ec39-4e53-816f-2f1319d404ee 184 [[https://www.nushell.sh/][~]] 185 [[https://www.nushell.sh/cookbook/][cookbook]] 186 [[https://github.com/nushell/nu_scripts][nu_scripts]] 189 :ID: 6d461ba3-f8cb-4a69-b172-038efcca923a 193 :ID: a14a7916-6743-45e5-8711-ecded8092131 195 We're leveraging AWS for some of our public web servers for now. It's
196 really not realistic to expect that my home desktop and spotty Comcast
197 internet can serve any production workflow. What it
/is/ capable of is
198 a private VPN, which can communicate with AWS and other cloud VPN
199 depots via WireGuard (
[[https://dev.to/gabrieltetzner/setting-up-a-vpn-with-wireguard-server-on-aws-ec2-4a49][article]]).
201 I currently use Google Domains for nas-t.net, otom8.dev, and
202 rwest.io - but that business is now owned by squarespace, so I would
203 rather move it to Route53.
205 We have archlinux ec2 image builds
[[https://wiki.archlinux.org/title/Arch_Linux_AMIs_for_Amazon_Web_Services][here]] and
[[https://gitlab.com/anemos-io/archlinux-ec2][here]] - only half work and not
206 maintained, but it's a start. I'm not even sure if I should stick with
207 arch or cave and use Ubuntu or AWS Linux. We can serve the static
208 services with little cost, the only big spender will be the heptapod
209 instance which requires a larger instance and some workers.
211 We'll try to keep the cost at or around $30/month.
214 :ID: b74a60a5-fdc1-4ef2-ac74-7ff5c1395d0c 218 :ID: a0d84027-06ea-4080-a40c-8fa86ed7ee7a 222 :ID: 9fd5e6e1-bb06-4204-a5ed-01afb5d45379 225 - State "TODO" from [2023-04-07 Fri 23:24] 235 :ID: 3daae7ec-4d0a-4f3e-bd5a-5202707fa614 238 - State "TODO" from "TODO" [2023-04-07 Fri 23:22] 243 :ID: 6b2b99d7-9d45-49fc-bff2-463a82c45f90 246 - State "TODO" from [2023-04-07 Fri 23:33] 250 :ID: 0fbbf620-c3b7-4aa8-8a79-b25beca802d0 254 :ID: 7fd63e2e-532b-4742-9d25-6cacbfbf2ca0 258 :ID: 2e6a146a-9a81-4b49-9127-646838f5e137 262 :ID: 2d634517-84dc-4cf4-a95a-02ae2fe3c2cd 266 :ID: 3d75df13-227c-4aee-a046-1443acdcf619 270 :ID: 8d44b6f9-e628-4983-aaa9-fea25117a303 274 :ID: 7c0febaa-468c-4835-83ac-8280faa067b9 278 :ID: 4e5e71e4-be74-44d0-9221-74de37f11096 281 - State "TODO" from [2023-04-07 Fri 23:52] 285 :ID: f9200923-f406-4104-9efe-33efab186f90 288 - State "TODO" from [2023-04-07 Fri 23:52] 292 :ID: 57a76ed9-76b9-442e-9a8e-3a7deb63dec4 295 - State "TODO" from [2023-04-07 Fri 23:34] 299 :ID: ce1ef5c1-61e0-4bed-84ff-bdf05d1cb379 302 - State "TODO" from [2023-04-07 Fri 23:51] 304 split out from rlib to separate package
305 - a purely OOP class library
308 :ID: a9401df7-b77e-4648-bdcf-bc9f8d267faf 311 - State "TODO" from [2023-04-07 Fri 23:34] 313 *** source categories 315 :ID: bdfaf877-a8a8-47c0-83b2-7ae3d18fcb8c 317 - need a way of extracting metadata from a repo
318 - need ability to search and query libs/packages
319 - separate modules based on where they belong in our stack?
325 *** generic query language 327 :ID: 4cbbb5bd-6fb0-4249-bce3-c60137a0545d 335 :ID: 2ef1a2d7-19f5-4d65-a4aa-2f6abf07a202 338 - Note taken on [2023-10-24 Tue 22:16] \\ 339 graph database, build on rocksdb 341 insidious Big Brother database.
342 - an application built with obj
347 :ID: 416d5836-fd77-4f7f-ad5c-831f20802a14 350 - State "NOTE" from "TODO" [2024-08-18 Sun 18:46] 357 :ID: 195067ad-1c7b-498a-9a90-8c9abba192f5 359 ** DRAFT dylib-skel-1 361 :ID: 55fc90c8-4261-4fd2-b991-280e76f0e59c 363 - State "DRAFT" from [2023-11-05 Sun 22:23]
366 :ID: bd3b748b-e792-40b3-abb0-02c274038539 368 Our core languages are
[[https://www.rust-lang.org/][Rust]] and
[[https://lisp-lang.org/][Lisp]] - this is the killer combo which will allow NAS-T
369 to rapidly develop high-quality software. As such, it's crucial that these two very
370 different languages (i.e. compilers) are able to interoperate seamlessly.
372 Some interop methods are easy to accomodate via the OS - such as IPC or data sharing,
373 but others are a bit more difficult.
375 In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something
376 that
/can/ be difficult, due to some complications with Rust and because this is not the
377 most popular software stack (yet ;). This is an experiment and may not make it to our
378 code-base, but it's definitely something worth adding to the toolbox in case we need it.
382 :ID: c2ccef54-e93d-406a-af1d-ebb7ad1a935b 384 The level of interop we're after in this case is
[[https://en.wikipedia.org/wiki/Foreign_function_interface][FFI]].
386 Basically, calling Rust code from Lisp and vice-versa. There's an article about calling
387 Rust from Common Lisp
[[https://dev.to/veer66/calling-rust-from-common-lisp-45c5][here]] which shows the basics and serves as a great starting point
388 for those interested.
391 :ID: bb68152d-1794-47d8-86c0-03eb7f9256d7 393 The complication(s) with Rust I mentioned early is really just that
/it is not C/.
=C= 394 is old, i.e. well-supported with a stable ABI, making the process of creating bindings
395 for a C library a breeze in many languages.
397 For a Rust library we need to first appease the compiler, as explained in
[[https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c][this section]] 398 of the Rustonomicon. Among other things it involves changing the calling-convention of
399 functions with a type signature and editing the Cargo.toml file to produce a
400 C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used
405 :ID: f74790d6-8fa9-4e76-8362-71f074058947 407 Using FFI involves some overhead. Check
[[https://github.com/dyu/ffi-overhead][here]] for an example benchmark across a few
408 languages. While building the NAS-T core, I'm very much aware of this, and will need a
409 few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular,
410 I'm concerned about crossing multiple language barriers (Rust
<->C
<->Lisp).
412 *** Rust -> C -> Lisp 414 :ID: 3d8e8664-337b-4bf1-832d-bc29e9761d56 418 :ID: 7c1a259a-8bc1-4a42-abb7-8d61d0a1c22d 420 For starters, I'm going to assume we all have Rust (via
=rustup=) and Lisp (
=sbcl= only)
421 installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
425 :ID: 21b8d6b0-27fd-460d-b584-55765c5f7761 427 Create a new library crate. For this example we're focusing on a 'skeleton' for
428 /dynamic/ libraries only, so our experiment will be called
=dylib-skel= or
*dysk* for
430 src_sh[:exports code]{cargo init dysk --lib && cd dysk}
432 A
=src/lib.rs= will be generated for you. Go ahead and delete that. We're going to be
433 making our own
=lib.rs= file directly in the root directory (just to be cool).
435 The next step is to edit your
=Cargo.toml= file. Add these lines after the
=[package]= 436 section and before
=[dependencies]=:
437 #+begin_src conf-toml 439 crate-type = ["cdylib","rlib"]
446 This tells Rust to generate a shared C-compatible object with a
=.so= extension which we
447 can open using
[[https://man.archlinux.org/man/dlopen.3.en][dlopen]].
450 :ID: 45c0e1cd-a441-4877-8b64-d30bc0381da2 454 :ID: d11805bf-1238-4359-ab52-6e6d1b74dc33 456 Next, we want the
=cbindgen= program which we'll use to generate header files for
457 C/C++. This step isn't necessary at all, we just want it for further experimentation.
459 src_sh[:exports code]{cargo install --force cbindgen}
461 We append the
=cbindgen= crate as a
/build dependency/ to our
=Cargo.toml= like so:
462 #+begin_src conf-toml 468 :ID: e33424c1-a328-4e86-b817-86e171584944 470 #+begin_src conf-toml :tangle cbindgen.toml 472 autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
473 include_version = true
476 after_includes = "#define DYSK_VERSION \"0.1.0\""
480 documentation_style = "c99"
481 usize_is_size_t = true
487 :ID: 0e465a4e-6709-4920-bb8a-514189f36f84 489 #+begin_src rust :tangle build.rs 490 fn main() -> Result<(), cbindgen::Error> {
491 if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
492 b.write_to_file("dysk.h"); Ok(())}
493 else { panic!("failed to generate dysk.h from cbindgen.toml") } }
497 :ID: 884ae229-0b41-4de3-880b-fb31058dc00a 499 #+begin_src rust :tangle lib.rs 500 //! lib.rs --- dysk library
501 use std::ffi::{c_char, c_int, CString};
503 pub extern "C" fn dysk_hello() -> *const c_char {
504 CString::new("hello from rust").unwrap().into_raw()}
506 pub extern "C" fn dysk_plus(a:c_int,b:c_int) -> c_int {a+b}
508 pub extern "C" fn dysk_plus1(n:c_int) -> c_int {n+1}
512 :ID: b8b27643-3c41-412c-afb7-12e980ffffb3 514 #+begin_src rust :tangle test.rs 515 //! test.rs --- dysk test
516 fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::dysk_plus1(2 as core::ffi::c_int);}}
520 :ID: 4d429e91-1b3b-40eb-86c0-b4896e277423 523 cargo build --release
527 :ID: 9db4d715-421c-4a06-ad25-68aa79396d3a 529 #+begin_src lisp :tangle dysk.lisp 530 (load-shared-object #P"target/release/libdysk.so")
531 (define-alien-routine dysk-hello c-string)
532 (define-alien-routine dysk-plus int (a int) (b int))
533 (define-alien-routine dysk-plus1 int (n int))
534 (dysk-hello) ;; => "hello from rust"
538 :ID: 88cd9e83-64da-4aa4-ab12-125b127bca06 541 time target/release/dysk-test
543 #+begin_src lisp :tangle test.lisp 544 (time (dotimes (_ 500000000) (dysk-plus1 2)))
548 :ID: 509a06ab-1a3d-4d5c-92e1-6fc50e8d1c00 552 :ID: df12c00f-0264-4f08-bff6-a327c7089b4c 555 (defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object cons))
556 (make-instance 'cl-dot:node
557 :attributes '(:label "cell \\N"
559 (defmethod cl-dot:graph-object-points-to ((graph (eql 'example)) (object cons))
561 (make-instance 'cl-dot:attributed
563 :attributes '(:weight 3))))
565 (defmethod cl-dot:graph-object-node ((graph (eql 'example)) (object symbol))
566 (make-instance 'cl-dot:node
567 :attributes `(:label ,object
571 :fillcolor "#ccccff")))
572 (let* ((data '(a b c #1=(b z) c d #1#))
573 (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
574 '(:rankdir "LR" :layout "twopi" :labelloc "t"))))
575 (cl-dot:dot-graph dgraph "test-lr.svg" :format #+nil :x11 :svg))
582 (dgraph (cl-dot:generate-graph-from-roots 'example (list data)
584 (cl-dot:print-graph dgraph))
589 :ID: 4f1d099a-2f9a-40c3-8cf7-57df6e660455 593 :ID: a05618ad-ada2-44be-9260-8acc0cf30d4d 595 need a way of indexing, referring to, and annotating objects such as
596 URLs, docs, articles, source files, etc.
598 What is the best way to get this done?
601 :ID: e304a97e-49fe-4fde-a273-7d3f78231033 603 ** doc best practices 605 :ID: ea2f3f8e-1c8c-4c05-b16c-fb197d346409 607 https://rust-lang.github.io/api-guidelines/documentation.html
609 also: https:
//lisp-lang.org/style-guide/ 612 :ID: eff3b823-6ffd-4355-b289-bdcd7b31ee54 616 :ID: 11bd7a7c-4d0c-4d7f-a487-dc59052da2b3 618 If you've met me in the past decade, you probably know that I am
619 extremely passionate about computers. Let me first explain why.
621 On the most basic level computers are little (or big) machines that
622 can be programmed to do things, or
/compute/ if we're being
625 They host and provide access to the Internet, which is a pretty big
626 thing, but they do little things too like unlock your car door and
627 tell your microwave to beep at you. They solve problems. Big or small.
629 They're also
/everywhere/ - which can be scary to think about, but
630 ultimately helps propel us into the future.
632 There's something pretty cool about that - when you look at the
633 essence of computation. There are endless quantities of these machines
634 which follow the same basic rules and can be used to solve
/real/ 639 :ID: f75f9b5b-ec2b-4f62-9fee-1ed3cef4ac50 641 Now, let us consider the
/programmer/. They have power.
/real/ 642 power. They understand the language of computers, can whisper to them
643 in various dialects. It can be intimidating to witness until you
644 realize how often the programmer says the wrong thing - a bug.
646 In reality, the programmer has a symbiotic relationship with
647 computers. Good programmers understand this relationship well.
650 One day after I got my first job at a software company, I remember
651 being on an all-hands meeting due to a client service outage. We had
652 some management, our lead devs, product team, and one curious looking
653 man who happened to be our lead IT consultant who had just joined. He
654 was sitting up on a hotel bed, shirtless, vaping an e-cig, typing
655 away in what I can only imagine was a shell prompt.
657 After several minutes he took a swig from a bottle of Coke and said
658 "Node 6 is sick." then a few seconds later our services were
659 restored. For the next hour on the call he explained what happened and
660 why, but that particular phrase always stuck with me. He didn't say
661 Node 6 was down, or had an expired cert - his diagnosis was that /it/
665 The more you work closely with computers, the more you start to think
666 of them this way. You don't start screaming when the computer does the
667 wrong thing, you figure out what's wrong and learn from it. With
668 experience, you start to understand the different behaviors of the
669 machines you work with. I like to call this
/Machine Empathy/.
673 :ID: b1187fd8-4d19-4566-827c-5594fbf90df7 675 I already mentioned bugs - I write plenty of those, but usually I try
676 to write
/programs/. Programs to me are like poetry. I like to think
677 they are for the computer too.
679 Just like computers,
/computer programs/ come in different shapes and
680 sizes but in basic terms they are sets of instructions used to control
683 You can write programs to do anything - when I first started, my
684 programs made music. The program was a means to an end. Over time, I
685 started to see the program as something much more. I saw it as the
688 [fn:1] ... perform computations
692 :ID: 332145f8-50b4-46d1-8368-ddaf097c875f 694 Something that is missing from many organizations big or large, is an
695 effective way to store and access information, even about their own
698 It can be difficult problem to solve - usually there's the official
699 one, say Microsoft Sharepoint and then the list of unofficial sources
700 which becomes tribal corporate hacker knowledge. Maybe the unofficial
701 ones are more current, or are annotated nicely, but their very
702 existence breaks the system. There's no longer a single source of
705 My priority in this department is writing services which process and
706 store information from a variety of sources in a distributed knowledge
707 graph. The graph can later be queried to access information on-demand.
709 My idea of infrastructure is in fact to build my own Cloud. Needless
710 to say I don't have an O365 subscription, and wherever possible I'll
711 be relying on hardware I have physical access to. I'm not opposed to
712 cloud services at large but based on principle I like to think we
713 shouldn't be built on them.
716 :ID: 67b51ca5-909b-495f-b263-5979efe36448 718 ** https://cal-coop.gitlab.io/utena/utena-specification/main.pdf 720 :ID: 03b3d70c-5ab6-4be8-b811-a7ba7f84be8f 722 from the author of cl-decentralise2. draft specification of a
723 /Maximalist/ Computing System.
726 :ID: bcc178c2-6a96-4ad9-98db-4ddf045e6371 730 :ID: 59e3ba53-3f40-47db-929c-acc998f9092d 732 https:
//github.com/awesomedata/awesome-public-datasets 733 https://docs.openml.org/Datasets/ 734 https:
//wiki.pathmind.com/open-datasets 737 :ID: db3d353b-8f1c-4645-a4ad-459d36392b22 741 :ID: a8ee911d-6d19-452b-b03a-249a446c1b20 744 sb-sys:*runtime-dlhandle* 745 sb-fasl:+fasl-file-version+ 746 sb-fasl:+backend-fasl-file-implementation+ 747 sb-debug:print-backtrace 748 sb-debug:map-backtrace 749 sb-pretty:pprint-dispatch-table 752 sb-walker:define-walker-template 753 sb-walker:macroexpand-all 756 sb-kernel:*eval-calls* 757 sb-kernel:*gc-pin-code-pages* 758 sb-kernel:*restart-clusters* 759 sb-kernel:*save-lisp-clobbered-globals* 760 sb-kernel:*top-level-form-p* 761 sb-kernel:*universal-fun-type* 762 sb-kernel:*universal-type* 763 sb-kernel:*wild-type* 764 sb-kernel:+simd-pack-element-types+ 766 (sb-vm:boxed-context-register) 767 (sb-vm:c-find-heap->arena) 768 (sb-vm:copy-number-to-heap) 769 (sb-vm:dump-arena-objects) 772 (sb-vm:show-heap->arena) 773 (sb-vm:with/without-arena) 774 (sb-cltl2:{augment-environment,compiler-let,define-declaration,parse-macro}) 775 (sb-cltl2:{declaration-information, variable-information, function-information}) 784 :ID: b83e3b40-17cc-4fc0-a412-d8b3f8733354 786 ** [[https://github.com/sigmf/SigMF][SigMF]]
788 :ID: 82a9833e-39f8-434f-94b4-4f34ce2fef25 791 Sharing sets of recorded signal data is an important part of science
792 and engineering. It enables multiple parties to collaborate, is often
793 a necessary part of reproducing scientific results (a requirement of
794 scientific rigor), and enables sharing data with those who do not have
795 direct access to the equipment required to capture it.
797 Unfortunately, these datasets have historically not been very
798 portable, and there is not an agreed upon method of sharing metadata
799 descriptions of the recorded data itself. This is the problem that
802 By providing a standard way to describe data recordings, SigMF
803 facilitates the sharing of data, prevents the "bitrot" of datasets
804 wherein details of the capture are lost over time, and makes it
805 possible for different tools to operate on the same dataset, thus
806 enabling data portability between tools and workflows.
809 the-spec: https:
//github.com/sigmf/SigMF/blob/sigmf-v1.x/sigmf-spec.md 810 ** [[https://www.libvolk.org/][LibVOLK]]
812 :ID: af88cd62-2610-4ba7-a5f4-d98c942c81dc 814 Vector-Optimized Library of Kernels (simd)
815 ** [[https://docs.kernel.org/fb/framebuffer.html][/dev/fb*]] 817 :ID: b866d034-2d59-4e1e-934f-02903b3de8e7 819 framebuffers, used by fbgrab/fbcat program
820 ** [[https://docs.kernel.org/block/ublk.html][ublk]] 822 :ID: d58cb494-ce43-49cc-859b-aa6a518ac314 824 https://github.com/ming1/ubdsrv
827 :ID: 86fdf7a7-9d26-4ce5-95a2-484db1843fb3 830 make problems smaller.
834 - doesn't need mentioning more and more
837 :ID: e8015df1-c8f3-4766-9cda-e4bb87c9aa62 839 ** TODO taobench demo 841 :ID: 72d7e1cf-79a4-4f30-88da-bb0c13551b1f 844 - State "TODO" from [2024-01-21 Sun 00:32] 846 https:
//github.com/audreyccheng/taobench - shouldn't have missed this :) 847 obviously we need to implement this using core -- in demo/bench/tao? 848 ** TODO clap completion for nushell 850 :ID: 91d3f9ba-92ab-43c7-ac0f-5fab172b7ba1 853 - State "TODO" from [2024-01-20 Sat 23:23] 855 https://github.com/clap-rs/clap/tree/master/clap_complete_nushell 856 ** Dataframe scripting 858 :ID: 23ba507b-8d56-4e1c-bbd6-0d2f5983e792 860 https://studioterabyte.nl/en/blog/polars-vs-pandas 861 nushell supports DFs, polars underneath? 862 https://www.nushell.sh/book/cheat_sheet.html
869 :ID: 31263e62-7de6-425a-8b3e-0522b4222322 873 :ID: e9724ccb-f4f6-4f75-816e-34887e2fd776 875 - [[https://cloud.google.com/free/docs/free-cloud-features][Free Cloud Features]] 876 + 90-day $300 credits
877 + e2-micro - free hours worth 1 instance/month
881 :ID: 048a6b82-2a8f-480b-a892-b8c80f97a13c 886 :ID: af1dbf25-4b58-4a89-a968-9a8f24499419 891 :ID: d2316ff4-4448-4209-87ca-22572def8125 893 - [[https://www.oracle.com/cloud/free/?intcmp=ohp052322ocift][OCI Free Tier]] 894 + always free: 2 x oracle autonomous DB
895 + 2 x AMD Compute VMs
896 + up to 4 x ARM Ampere A1 with 3k/cpu/hr and 18k/gb/h per month
897 + block/object/archive storage
898 + 30-day $300 credits
901 :ID: 9bad5422-eea9-4328-a68e-fd2180da2df1 903 ** trash as block device 905 :ID: cdf84840-46dd-42f3-82df-f5b57c908118 908 - State "NOTE" from [2024-01-29 Mon 20:53] 909 - State "NOTE" from [2024-01-29 Mon 20:53] 911 in nushell there is option for rm command to always use 'trash' -
912 AFAIK the current approach is via a service (trashd).
914 An interesting experiment would be to designate a block device as
915 'trash' - may be possible to remove reliance on a service
917 may be an opportunity for ublk driver to shine - instead of /dev/null
918 piping we need a driver for streaming a file to /dev/trash
921 :ID: fdc5a933-6792-40e0-914e-49629a4d7332 924 - State "NOTE" from [2024-01-29 Mon 16:28] 926 - mostly x86_64 machines - currently 2 AWS EC2 instances, some podman containers, and our home beowulf server:
929 - mid-size tower enclosed (Linux/Windows)
931 - Intel Core i7-6700K
934 - NVIDIA GeForce GTX 1060
937 - Samsung SSD 850: 232.9GB
938 - Samsung SSD 850: 465.76GB
939 - ST2000DM001-1ER1: 1.82TB
940 - WDC WD80EAZZ-00B: 7.28TB
941 - PSSD T7 Shield: 3.64TB
942 - My Passport 0820: 1.36TB
944 - 16GB (2*8) [64GB max]
947 - MacBook Pro 2019 (MacOS/Darwin)
963 - 64-bit 6-core 4x ARM Cortex A53 + 2x ARM Cortex A72
969 - Raspberry Pi 4 Model B
971 - Cortex-A72 (ARM v8) 64-bit SoC
978 :ID: d05fd1f5-2f85-485f-8963-11a1cd2322fc 982 :ID: d7e1885d-fb19-448c-beb6-0ed0bd701cec 986 let ms = '1trc/measurements-0.parquet'
988 | dfr group-by station
990 (dfr col measure | dfr min | dfr as "min")
991 (dfr col measure | dfr max | dfr as "max")
992 (dfr col measure | dfr sum | dfr as "sum")
993 (dfr col measure | dfr count | dfr as "count")
998 :ID: 908fd9e6-6307-402c-ba76-813e82839f53 1002 :ID: 9a398886-858a-450b-9029-2339cc551bc3 1005 - State "NOTE" from [2024-02-18 Sun 11:55] 1007 In the past few months there has been drama regarding Wayland vs X. It
1008 seems to be on everyone's minds after Artem's freakout issue and the
1009 follow up YT vids/comments.
1011 I admit that it made me reconsider the fitness of WL as a whole -
1012 there was a github gist that made some scathing arguments against it.
1014 It's an odd debate though. I think there are many misunderstandings.
1016 So first off, if we look at the homepage
1017 https:
//wayland.freedesktop.org/, Wayland claims it is a replacement
1018 for X11. It now has
/manifest destiny/, which in my opinion is a great
1021 X-pros seem to agree that Wayland has
/manifest destiny/ - like if you
1022 are building softwares that look remotely like a window system, it's a
1023 successor to X. That's the model of doing things and there's no way
1026 The disagreement starts with how this destiny - of an X2 - should be
1027 fulfilled. X-pros want a fork of X, but it's too late for
1028 that. WL-pros want X to run on top of Wayland compositor:
1029 https://wayland.freedesktop.org/xserver.html.
1031 Xwayland is a problem for me. From the project description: 'if we're
1032 migrating away from X, it makes sense to have a good backwards
1033 compatibility story.' Full disclosure: I have never done significant
1034 work on Xwayland, so perhaps my opinion is unwarranted. But I have no
1035 intention of attempting to maintain a computer system that uses
1036 Wayland and X clients at the same time.
1038 To me, X is ol' reliable. Every distro has first-class X support, and
1039 it runs on most systems with very little user intervention. Where it
1040 doesn't, there is 20+ years of dev history and battle-tested
1041 workarounds for you to find your solution in.
1043 Wayland is the new kid on the block, born just in 2008. It's a fresh
1044 start to one of the most difficult challenges in software - window
1045 systems. A re-write would be pointless though, and so the real
1046 value-add is in design. Wayland is designed as a protocol and
1047 collection of libraries which are implemented in your own
1048 compositor. Coming from Lisp - with ANSI Common Lisp and SRFIs, this
1049 feels right even if the implementation is something very different
1050 (compositor vs compiler).
1052 With X, it is assumed to be much harder to write an equivalent
1053 'compositor'. Here's the thing though - with a significantly complex X
1054 client implementation, it is
/impossible/ to replicate in WL. This is
1055 really the crux of Artemi's argument in his issue. He asked for a 1:1
1056 equivalent X/WL comparison when no such thing exists, and in my
1057 opinion it is a waste of time.
1059 The WL core team is fully aware of this dichotomy, but also that this
1060 is in no way a problem or weakness in either system. It means they're
1061 different systems, goddammit.
1063 If it was up to me, Xwayland wouldn't exist. I understand why it does,
1064 and that it does make things easier for developers who need to support
1065 both, and users who have multiple apps with multiple windowing
1066 requirements. It's a bandaid though, and one that is particularly
1067 dangerous because it re-enforces the idea that Wayland is just X2 and
1068 that they're fully compatible.
1070 What interests me in the Wayland world right now is the idea of a
1071 small, modular, full-stack Wayland compositor API. There are several
1072 'kiosk' based compositors for single applications (cage), but these
1073 aren't complete solutions. It is possible to get much closer to the
1074 metal, and that's where I want to be so that I can build my own APIs
1075 on top - I don't want to live on top of X, and I certainly don't want
1076 to live on top of X on top of WL. I want a
/pure/ solution that hides
1077 as little as possible, exposing the interesting bits.
1080 :ID: 5d5159dd-5887-4823-81c6-1624f8cc4fd4 1082 ** TODO collect more data 1084 :ID: 9e0339dd-17a8-436c-afd4-245437fe2fea 1087 - State "TODO" from [2024-03-01 Fri 15:27] 1089 https:
//www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/
1090 weather - music - etc
1093 :ID: 148d3b53-be56-4c34-9313-cac26c57300e 1095 ** On blocks and devices 1097 :ID: 242a2380-3d87-403d-b529-0245e77ed1f0 1100 - State "NOTE" from [2024-03-02 Sat 21:30] 1102 [[https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/dev.html][/dev]] 1103 In Linux, everything is a file.
1105 /dev/ contains special device files - usually block or character
1108 major, minor = category, device
1111 mknod - create special device files
1113 [[https://www.redhat.com/en/blog/linux-block-devices-hints-debugging-and-new-developments][redhat hints]] 1116 dd if=/dev/zero of=myfile bs=1M count=32
1117 losetup --show -f myfile
1119 losetup -d /dev/loop0 #teardown
1123 echo "sup dude" > /dev/loop0
1124 dd if=/dev/loop0 -bs=1
1125 dd if=/dev/nvme0 of=/dev/null progress=true
1127 hdparm -T /dev/nvme0
1131 modprobe scsi_debug add_host=5 max_luns=10 num_tgts=2 dev_size_mb=16
1134 sparsefiles: create with C, dd, or truncate
1136 #+begin_src shell :results output 1142 Usage: truncate OPTION... FILE...
1143 Shrink or extend the size of each FILE to the specified size
1145 A FILE argument that does not exist is created.
1147 If a FILE is larger than the specified size, the extra data is lost.
1148 If a FILE is shorter, it is extended and the sparse extended part (hole)
1149 reads as zero bytes.
1151 Mandatory arguments to long options are mandatory for short options too.
1152 -c, --no-create do not create any files
1153 -o, --io-blocks treat SIZE as number of IO blocks instead of bytes
1154 -r, --reference=RFILE base size on RFILE
1155 -s, --size=SIZE set or adjust the file size by SIZE bytes
1156 --help display this help and exit
1157 --version output version information and exit
1159 The SIZE argument is an integer and optional unit (example: 10K is 10*1024).
1160 Units are K,M,G,T,P,E,Z,Y,R,Q (powers of 1024) or KB,MB,... (powers of 1000).
1161 Binary prefixes can be used, too: KiB=K, MiB=M, and so on.
1163 SIZE may also be prefixed by one of the following modifying characters:
1164 '+' extend by, '-' reduce by, '<' at most, '>' at least,
1165 '/' round down to multiple of, '%' round up to multiple of.
1167 GNU coreutils online help: <https://www.gnu.org/software/coreutils/>
1168 Full documentation <https://www.gnu.org/software/coreutils/truncate>
1169 or available locally via: info '(coreutils) truncate invocation'
1172 test mkfs.btrfs on 10T dummy block device
1174 dd if=/dev/zero of=/tmp/bb1 bs=1 count=1 seek=10T
1176 losetup --show -f /tmp/bb1
1177 mkfs.btrfs /dev/loop0
1182 iostat # pacman -S sysstat
1183 blktrace # paru -S blktrace
1184 iotop # pacman -S iotop
1187 bcc/ trace: Who/which process is executing specific functions against
1190 bcc/biosnoop: Which process is accessing the block device, how many
1191 bytes are accessed, which latency for answering the requests?
1193 at the kernel level besides BPF we got kmods and DKMS,
1195 compression/de-duplication can be done via VDO kernel mod
1197 https://en.wikipedia.org/wiki/Dynamic_Kernel_Module_Support
1199 ** save-lisp-and-respawn 1201 :ID: bef3c7e0-c929-434a-9118-cca57fcb5bd3 1204 - State "NOTE" from [2024-03-02 Sat 22:57] 1212 :ID: 570a1587-486c-49c0-bca4-9cbcfcd3013e 1215 - State "NOTE" from [2024-03-03 Sun 16:35] 1218 - openlog syslog closelog
1219 - levels: emerg alert crit err warning notice info debug
1223 :ID: f2293a24-442b-458e-99c8-6488b85cb3c2 1225 ** RESEARCH [[https://github.com/guicho271828/sbcl-wiki/wiki/][sbcl-wiki]] 1227 :ID: 9a507eb4-2153-4abd-9be3-3b38b4989e8c 1230 - State "RESEARCH" from [2024-03-13 Wed 21:49] 1234 :ID: ca497ebc-b3b0-419e-aa59-a082c96d61f6 1238 :ID: c5d40dfe-b059-44f4-8d42-78f61f6e9a95 1242 :ID: b9961009-0a47-4122-a5d7-a6e72e4b5f92 1246 :ID: 78f5150b-4ccb-4682-bf9c-60ed1649905d 1249 - State "NOTE" from [2024-02-04 Sun 20:40] 1251 [[https://github.com/facebook/rocksdb/wiki/Benchmarking-tools][RocksDB benchmarking tools]] 1255 :ID: 1eae39fb-ccb4-419e-857a-cc357e1d3eda 1258 - State "NOTE" from [2024-01-25 Thu 22:39] 1260 - API root: https:
//packy.compiler.company 1261 - source packs: https://vc.compiler.company/packy 1264 :ID: 3863c9f9-89d9-4233-b6e4-5c5bc0178e02 1268 :ID: 010cb138-70c4-4d2d-ba7b-ad423d861690 1272 :ID: b2bbc87a-dc55-4df8-81d1-a9de8429dad0 1274 Primary data type of the library - typically represents a compressed 1275 archive, metadata, and ops. 1278 :ID: 432f7779-a93b-48f1-a095-25482c8d3702 1280 Collection data type, usually contains a set of packs with metadata. 1283 :ID: c3c21988-8bb2-448a-a98a-f26cc862c2f3 1285 Represents a Packy instance bound to a UDP socket 1286 ***** PackyEndpointConfig 1288 :ID: f732c36f-6e55-4c44-a2fb-6514706ec326 1290 Global endpoint configuration object 1291 ***** PackyClientConfig 1293 :ID: 794952de-a5c5-4031-a1e1-0b1e0325584a 1295 Configuration for outgoing packy connections on an endpoint 1296 ***** PackyServerConfig 1298 :ID: b65636cc-0523-4a1b-992d-6aff66cbab5d 1300 Configuration for incoming packy connection son an endpoint 1301 ***** PackyConnection 1303 :ID: 74646244-ee39-4ef8-8193-d5484438b81b 1305 Packy connection object 1308 :ID: dfe9d569-f951-4485-9b72-055e77404281 1312 :ID: 71d4af37-6e24-48ef-a49e-683ee41ce5d2 1316 :ID: 47424475-837c-4996-a1b9-3eb6599105d6 1320 :ID: e80b3402-6afd-4ad7-a800-790994867479 1324 :ID: f4801fb0-bace-48ae-ab63-1a5cac3ea98f 1328 :ID: 1f24bd33-1e00-4dc2-bc6c-fac8374e14f0 1332 :ID: b1f91455-68bc-4cc6-94c7-5fb3544e7539 1336 :ID: b718721a-3f22-4eb6-80bf-5e7d5def1346 1340 :ID: 7781e76b-51a4-42fd-929a-a1ff08ccb95b 1344 :ID: af8409ec-9865-46d3-a593-df109b45cdff 1346 ****** start_packy_server 1348 :ID: 13e65c38-9591-484d-be1d-3d4b98f95f41 1350 ****** stop_packy_server 1352 :ID: 3cae6660-755c-4242-ac4f-eccb7188c6b5 1354 ****** start_packy_registry 1356 :ID: 6ca2bd7d-9c1a-4790-a3a0-08bda811318b 1360 :ID: 1d2ad8dc-6aef-48fa-b079-71d8bb88832f 1362 ****** register_pack 1364 :ID: 43221579-6f71-42aa-8332-96af33531015 1366 ****** register_user 1368 :ID: 3c6034fa-d543-47cd-ae35-bcfd3f2b8150 1370 ****** register_bundle 1372 :ID: 1b041060-7f5e-48fe-8566-e4781a597686 1376 :ID: c6e0fb44-21ba-4a94-bd04-e7528dc46dba 1378 ** TBD investigate alieneval for phash opps 1380 :ID: 18989904-85ad-403b-8195-5ba7f643437f 1383 - State "TBD" from [2024-03-25 Mon 18:56] 1387 :ID: 8ffd1283-5a63-4fe3-868f-57a4f9f09f74 1391 :ID: 1b26fcd9-8d32-4031-a0cc-7ed2e26a75d0 1393 The backend services are written in Rust and controlled by a simple 1394 messaging protocol. Services provide common runtime capabilities known 1395 as the /service protocol/ but are specialized on a unique
/service 1396 type/ which may in turn register their own
/custom protocols/ (via
1399 Services are capable of dispatching data directly to clients, or
1400 storing data in the
/database/ (sqlite, postgres, mysql).
1402 The frontend clients are pre-dominantly written in Common Lisp and
1403 come in many shapes and sizes. There is a cli-client, web-client
1404 (CLOG), docker-client (archlinux, stumpwm, McCLIM), and native-client
1405 which also compiles to WASM (slint-rs).
1409 :ID: 90772707-ee13-463a-b0b2-7884745ec640 1413 :ID: a2f7f5f5-0cbd-4aa4-b80e-afa1d86990cd 1415 - *install dependencies* 1419 - *make executables* \\ 1420 Simply run
=make build=. Read the
~makefile~ and change the options
1422 - MODE :: Mode (debug, release)
1423 - LISP :: Lisp (sbcl, cmucl, ccl)
1424 - CFG :: Config (default.cfg)
1427 :ID: 572b1b68-f099-4bdb-9a06-9aa2186c17cc 1434 :ID: cadb26fe-5fc1-4639-9bb1-abeb490e91af 1436 Configs can be specified in JSON, TOML, RON, or of course SEXP. See
1437 =default.cfg= for an example.
1440 :ID: a8296224-60bc-46e7-9fa0-9a600e9f869e 1442 The high-level user interface is presented as a multi-modal GUI
1443 application which adapts to the specific application
/instances/ 1447 :ID: ec07eb9b-3f83-4ff7-9809-b8190572e821 1449 This backend retrieves weather data using the NWS API.
1452 :ID: a61d9eff-ffc9-42a4-a5ac-16d4a0cbd24e 1454 The 'Stocks' backend features a stock ticker with real-time analysis
1458 :ID: a23c7805-2b48-4910-9231-96a90a0467f2 1460 This is a benchmark backend for testing the capabilities of our
1461 demo. It spins up some mock services and allows fine-grained control
1462 of input/throughput.
1465 :ID: ba4bdf0d-c29f-4643-af33-09de5e5d1d45 1469 :ID: 4d7b2ef7-569a-40bf-b37d-d2e1d4dc01e2 1471 - consider tree-sitter parsing layout, use as a guide for developing a
1472 single syntax which expands to Rust or C.
1478 **** TODO rs-macroexpand 1480 :ID: ab16b2fb-ec87-4c0e-b0be-185a4ae93537 1487 **** TODO c-macroexpand 1489 :ID: 6e97665d-3e29-4c3a-8040-0a15eeb4bf0a 1496 **** TODO slint-macroexpand 1498 :ID: 629303bb-ab04-4840-8264-18f1b43fa748 1504 - slint-macroexpand-1
1505 **** TODO html (using who) 1507 :ID: 09eb1d3a-4f73-4a29-95cf-92d589f7fec9 1509 *** TODO web templates 1511 :ID: 6c70f44f-4d03-4663-92c8-e4a5b8fedfc1 1513 create a basic static page in CL which will be used to host Slint UIs
1514 and other WASM doo-dads in a browser.
1517 :ID: bd213f5b-1abf-450f-b3f3-6093dd050ea8 1519 using clingon, decide on generic options and write it up
1522 :ID: d5d980f2-c032-4e15-a1e6-067fee7197fc 1524 work on doc generation -- Rust and CL should be accounted for.
1527 :ID: 92d6ff4d-a643-4f75-89cf-e19b0e2e0d9a 1529 We have none! need to make it more comfy - set up testing in all Rust
1530 crates and for the lisp systems.
1533 :ID: ab37ae2a-b168-455e-a4f5-c8c1baada2be 1535 ** https://docs.gitlab.com/ee/administration/backup_restore/migrate_to_new_server.html 1537 :ID: d18c642a-b8f9-40ff-9f99-df4e9764bbd5 1541 :ID: 4b9ab3c7-5ea3-4f1e-9e0e-db4be8548de4 1545 :ID: ab5c8dc6-6941-4b5d-b624-8b232385ebf1 1547 - make sure to apk add:
1556 :ID: cb3f0071-a1b9-4ef4-897a-c5edd3435a5e 1560 :ID: 2a650229-df59-46b9-bd92-f5cdd9eab4d2 1562 - How should such objects be represented within CORE?
1564 - already have alias
1565 - not sure about obj/otherwise, prob not
1568 :ID: 298b1333-c6f7-45d3-9829-c4c831e84707 1570 ** Intelligent Design in Software 1572 :ID: c2e682b3-d72a-41b3-b093-16614e7852ff 1574 - starting from a space where there are no external influences - a biome
1575 - answer questions regarding the nature of the software and its capabilities
1576 - incrementally adjust inter-dependencies
1578 - protect the biome at all costs
1579 - focus on composition
1580 - build applications
1581 - re-integrate lessons learned
1584 :ID: 2877ab07-156b-4b00-97fa-9b12952f0ac1 1586 ** keys.compiler.company 1588 :ID: 26922400-2249-4e1d-a2ed-8014df5ad341 1591 [2024-09-22 Sun 21:07] -> [[id:05a7192a-4c05-44d9-b859-e2a0cff53d86][C]] 1594 - State "NOTE" from [2024-08-16 Fri 22:00] 1596 - public openpgp server
1597 - [[https://keys.compiler.company][keys.compiler.company]] 1598 - https:
//keys.openpgp.org/ 1599 - [[vc:packy/hagrid/][packy/hagrid]]