Mercurial > core / readme.org

 #+title: core
 #+description: The Compiler Company Core
 #+author: Richard Westhaver
 #+email: richard.westhaver@gmail.com
 #+setupfile: https://cdn.compiler.company/org/clean.theme
 #+property: header-args :eval no-export
 - [[https://compiler.company/docs/core][Documentation]]
 
 * Overview
 This repository contains the monolothic core of [[comp:][The Compiler Company]].
 
 To bootstrap the core you will need recent versions of [[https://www.rust-lang.org/][Rust]], [[http://www.sbcl.org/][SBCL]], and
 a C compiler (clang or gcc). Only Unix systems are explicitly
 supported.
 
 Many parts of the core depend on additional libraries which may or may
 not be provided by your system's package manager. See the dependency
 matrix below for details.
 
 In any case, it is always preferred to make use of the [[https://vc.compiler.company/comp/infra][infra]] project
 to reliably provision the host either from source or pre-built
 platform-specific [[https://packy.compiler.company/dist][binary distributions]].
 
 #+NAME: Dependencies
 | dependency  | required by            | src                                           |
 |-------------+------------------------+-----------------------------------------------|
 | Blake3      | ffi/blake3             | https://vc.compiler.company/packy/blake3      |
 | Tree-sitter | ffi/tree-sitter        | https://vc.compiler.company/packy/tree-sitter |
 | Uring       | ffi/uring              | https://vc.compiler.company/packy/uring       |
 | Btrfs       | ffi/btrfs              | https://vc.compiler.company/packy/btrfs       |
 | Ublksrv     | ffi/ublk               | https://vc.compiler.company/packy/ublksrv     |
 | OpenSSL     | lib/net                | [[https://vc.compiler.company/packy/openssl]]     |
 | RocksDB     | ffi/rocksdb            | https://vc.compiler.company/packy/rocksdb     |
 | Git         | lib/vc/git             | https://vc.compiler.company/packy/git         |
 | Hg          | lib/vc/hg              | https://vc.compiler.company/packy/hg          |
 | Zstd        | ffi/zstd               | https://vc.compiler.company/packy/zstd        |
 | Qemu        | lib/box                | https://vc.compiler.company/packy/qemu        |
 | Podman      | lib/pod                | https://vc.compiler.company/packy/podman      |
 | Emacs       | emacs                  | https://vc.compiler.company/packy/emacs       |
 | StumpWM     | lib/gui/wm/x11/stumpwm | https://vc.compiler.company/packy/stumpwm     |
 | Readline    | ffi/readline           | [[https://vc.compiler.company/packy/readline]]    |
 | Keyutils    | ffi/keyutils           | [[https://vc.compiler.company/packy/libkeyutils]] |
 | Mpd         | lib/aud/mpd            | https://vc.compiler.company/packy/mpd         |
 
 * Build
 The Core consists of two major system: the *lisp* system and the
 *rust* system. There is also an auxiliary *emacs* system containing a
 complete Emacs IDE configuration which serves as the base for user
 customizations.
 
 Building the core will produce its output to the =.stash= directory by
 default. You can then test, run, and install the resulting files or
 package them up to be shipped elsewhere.
 
 The core produces a collection of native program binaries, compiled
 lisp modules in FASL format, as well as web distributions containing
 the usual suspects (html, css, js) as well as wasm modules.
 
 ** From Source
 *** Lisp
 The Lisp Core can be found under the =lisp= directory. It is the
 largest system, most actively developed, and is intended to cover the
 complete surface of the user-facing APIs contained in the core.
 
 Starting from a standard Common Lisp environment with at least
 [[https://www.quicklisp.org/beta/][Quicklisp]] installed, you can execute =./x.lisp build skel= to build
 the project compiler and =./x.lisp run skel= to run it. This tool is a
 convenience for building and testing different parts of the core from
 a minimal dependency set.
 
 #+name: x-help
 #+begin_src shell :exports both :results output
 # bootstrap the core build tool
 ./x.lisp --help
 #+end_src
 
 #+RESULTS: x-help
 #+begin_example
 This is SBCL 2.4.7:76bbecb68, an implementation of ANSI Common Lisp.
 More information about SBCL is available at <http://www.sbcl.org/>.
 
 SBCL is free software, provided as is, with absolutely no warranty.
 It is mostly in the public domain; some portions are provided under
 BSD-style licenses.  See the CREDITS and COPYING files in the
 distribution for more information.
 To load "rt":
   Load 1 ASDF system:
     rt
 ; Loading "rt"
 
 x.lisp --- core build tool
 x.lisp [CMD]
 CMDS:
 test
 compile
 build
 make
 test
 run
 save
 install
 #+end_example
 
 To test the lisp standard library:
 #+begin_src shell :results output :exports both
 # test systems
 ./x.lisp test std
 #+end_src
 
 #+RESULTS:
 #+begin_example
 This is SBCL 2.4.7:dc890089a, an implementation of ANSI Common Lisp.
 More information about SBCL is available at <http://www.sbcl.org/>.
 
 SBCL is free software, provided as is, with absolutely no warranty.
 It is mostly in the public domain; some portions are provided under
 BSD-style licenses.  See the CREDITS and COPYING files in the
 distribution for more information.
 To load "rt":
   Load 1 ASDF system:
     rt
 ; Loading "rt"
 
 To load "rt":
   Load 1 ASDF system:
     rt
 ; Loading "rt"
 
 To load "std/tests":
   Load 1 ASDF system:
     std/tests
 ; Loading "std/tests"
 
 in suite STD:
 ; with 15 tests
 #<PASS READTABLES> 
 #<PASS SYM> 
 #<PASS STRING> 
 #<PASS LIST> 
 #<PASS ERR> 
 #<PASS THREADS> 
 #<PASS TIMERS> 
 #<PASS FMT> 
 #<PASS ANA> 
 #<PASS PAN> 
 #<PASS ALIEN> 
 #<PASS CURRY> 
 #<PASS BITS> 
 #<PASS LEB128> 
 #<PASS TASKS> 
 No tests failed.
 #+end_example
 
 To save the prelude to =.stash/prelude.core=:
 #+begin_src shell :results output :noeval t
 # save cores
 ./x save prelude
 #+end_src
 
 And to install binaries (defaults to =/usr/local/bin/=):
 #+begin_src shell :results none :noeval t
 # install binaries
 sudo ./x install
 #+end_src
 
 *** Rust
 The Core Rust system can be found under the =rust= directory.
 
 A workspace is configured such that you can build all components with
 the following command (~NOTE~ - takes a long time):
 #+begin_src shell :exports both :results output
   cd rust && cargo build --release
 #+end_src
 
 *** Emacs
 The core contains a collection of Emacs Lisp libraries under =emacs=
 which may be installed for the current user using the corresponding
 Makefile.
 
 ** TODO Bootstrap from packy                                   :experimental:
 The following can (maybe someday) be used to bootstrap the core without a full
 lisp compiler.
 #+begin_src shell :results output
   curl --proto '=https' \
        --tlsv1.3 \
        -sSf \
        https://packy.compiler.company/dist/x86_64-unknown-linux-gnu/bin/sk \
        --output /tmp/sk
   chmod +x /tmp/sk
   /tmp/sk make skel
 #+end_src
 
 #+RESULTS:
 #+begin_example
 This is SBCL 2.4.5, an implementation of ANSI Common Lisp.
 More information about SBCL is available at <http://www.sbcl.org/>.
 
 SBCL is free software, provided as is, with absolutely no warranty.
 It is mostly in the public domain; some portions are provided under
 BSD-style licenses.  See the CREDITS and COPYING files in the
 distribution for more information.
 To load "rt":
   Load 1 ASDF system:
     rt
 ; Loading "rt"
 
 saving skel to: /home/ellis/comp/core/.stash/skel
 To load "bin/skel":
   Load 1 ASDF system:
     bin/skel
 ; Loading "bin/skel"
 .........
 [undoing binding stack and other enclosing state... done]
 [performing final GC... done]
 [saving current Lisp image into /home/ellis/comp/core/.stash/skel:
 writing 15984 bytes from the static space at 0x50100000
 writing 339476480 bytes from the dynamic space at 0x1000000000
 writing 12498752 bytes from the read-only space at 0x74232dc00000
 writing 0 bytes from the text space at (nil)
 done]
 #+end_example
 
 * Programs
 This section lists all program binaries provided by the core.
 ** skel                                                                :lisp:
 #+begin_src shell :results output :exports both
   skel --help
 #+end_src
 
 #+RESULTS:
 #+begin_example
 skel v0.1.1:ce91ffc6cc7a+ --- A hacker's project compiler.
 
   usage: skel [global] <command> [<arg>]
 
 options:
   -h/--help* :  print this message
   -v/--version* :  print version
   -l/--level* :  set log level (warn,info,debug,trace)
   -c/--config* :  set a custom skel user config
   -i/--input* :  input source
   -o/--output* :  output target
 
 commands:
   init : initialize a skelfile in the current directory
     -n/--name :  project name
   new : make a new skel project
     -n/--name :  project name
   describe : describe a skelfile
   show : show project slots
     -f/--file :  path to skelfile
     -u/--user :  print user configuration
     -s/--system :  print system configuration
   vc : version control
     -r/--root :  repository path
   id : print the project id
   inspect : inspect the project skelfile
     -f/--file :  path to skelfile
   make : build project targets
     -t/--target :  target to build
   run : run a script or command
   compile : compile source code
   build : build programs and libraries
   dist : distribute build artifacts
   install : install stuff
   pack : pack stuff
   unpack : unpack stuff
   bundle : bundle source code
   unbundle : unbundle source code
   clean : clean up the project
   test : run tests
   bench : run benchmark
   status : print the vc status
   push : push the current project upstream
   pull : pull the current project from remote
   clone : clone a remote project
   commit : commit changes to the project vc
   edit : edit a project file in emacs.
   shell : open the sk-shell interpreter
 
 #+end_example
 ** organ                                                               :lisp:
 #+begin_src shell :results output :exports both
 organ --help
 #+end_src
 
 #+RESULTS:
 #+begin_example
 organ v0.0.1 --- org-mode toolbox
 
   usage: organ [global] <command> [<arg>]
 
 options:
   -l/--level* :  set the log level
   -h/--help* :  print help
   -v/--version* :  print version
 
 commands:
   inspect : inspect an org file
   show : display local org info
   describe : describe local org info
   parse 
 #+end_example
 
 ** packy                                                               :lisp:
 #+begin_src shell :results output :exports both
 packy --help
 #+end_src
 
 #+RESULTS:
 #+begin_example
 packy v0.1.0 --- Universal Package Manager
 
   usage: packy [global] <command> [<arg>]
 
 options:
   -l/--level* :  set the log level
   -h/--help* :  print help
   -v/--version* :  print version
 
 commands:
   show 
     -n/--name* 
     -t/--target 
     -t/--thunk* 
     -p/--pk-target* 
 #+end_example
 
 ** rdb                                                                 :lisp:
 #+begin_src shell :results output :exports both
 rdb --help
 #+end_src
 
 #+RESULTS:
 
 ** homer                                                               :lisp:
 #+begin_src shell :results output :exports both
 homer --help
 #+end_src
 
 #+RESULTS:
 #+begin_example
 homer v0.1.0 --- user home manager
 
   usage: homer [global] <command> [<arg>]
 
 options:
   -l/--level* :  set the log level
   -h/--help* :  print help
   -v/--version* :  print version
   -f/--force* :  use force
 
 commands:
   show 
   check 
   push 
   pull 
   install 
 #+end_example
 
 ** COMMENT alik                                                        :rust:
 #+begin_src shell :results output :exports both
 alik --help
 #+end_src
 
 #+RESULTS:
 
 ** COMMENT krypt                                                       :lisp:
 #+begin_src shell :results output :exports both
 krypt --help
 #+end_src
 
 #+RESULTS:
 #+begin_example
 Krypt CLI
 
 Usage: krypt [OPTIONS] [COMMAND]
 
 Commands:
   check   check service providers and config
   show    Show Krypt info
   search  Query the Krypt
   help    Print this message or the help of the given subcommand(s)
 
 Options:
   -c, --cfg <CFG>    Set the default config file [env: KRYPT_CONFIG_FILE=]
   -u, --user <USER>  Set a user for this command [env: USER=ellis]
   -l, --level...     Set log level
   -h, --help         Print help
   -V, --version      Print version
 #+end_example
 
 ** mailman                                                             :rust:
 #+begin_src shell :results output :exports both
 mailman --help
 #+end_src
 
 #+RESULTS:
 #+begin_example
 Mail client util
 
 Usage: mailman [OPTIONS] [COMMAND]
 
 Commands:
   ping    Ping the server
   search  Search for items
   import  Import an account
   export  Export an account
   help    Print this message or the help of the given subcommand(s)
 
 Options:
   -c, --cfg <CFG>    Set the default config file [env: MAILMAN_CONFIG_FILE=]
   -u, --user <USER>  Set a user for this command [env: USER=ellis]
   -l, --level...     Set log level
   -h, --help         Print help
   -V, --version      Print version
 #+end_example