changelog shortlog graph tags branches changeset files file revisions raw help

Mercurial > org > docs / annotate infra/quickstart.org

changeset 34: bc53e27c0163
parent: aefeb7103fb8
child: b925807b50a6
author: Richard Westhaver <ellis@rwest.io>
date: Sat, 22 Jun 2024 23:54:39 -0400
permissions: -rw-r--r--
description: bump
28
a0017112db77 style update
Richard Westhaver <ellis@rwest.io>
parents: 23
diff changeset
1
 #+title: infra/quickstart
a0017112db77 style update
Richard Westhaver <ellis@rwest.io>
parents: 23
diff changeset
2
 #+description: Infra Quickstart Guide
22
889970442a12 docs bump
Richard Westhaver <ellis@rwest.io>
parents: 20
diff changeset
3
 #+property: header-args :dir /home/ellis/comp/infra/
34
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
4
 #+property: header-args:dot :dir /tmp/ :file bootstrap.svg
28
a0017112db77 style update
Richard Westhaver <ellis@rwest.io>
parents: 23
diff changeset
5
 #+setupfile: ../../clean.theme
34
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
6
 #+infojs_opt: toc:nil home:https://compiler.company up:./ view:overview
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
7
 
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
8
 * Overview
20
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
9
 - *Host Requirements*
19
Richard Westhaver <ellis@rwest.io>
parents: 18
diff changeset
10
   - x86_64 4 core CPU
Richard Westhaver <ellis@rwest.io>
parents: 18
diff changeset
11
   - 8G RAM
Richard Westhaver <ellis@rwest.io>
parents: 18
diff changeset
12
   - 32G Disk Space
20
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
13
   - Linux kernel
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
14
 - *Minimum Dependencies*
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
15
   - mercurial
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
16
   - git
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
17
   - zstd
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
18
 
34
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
19
 
33
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
20
 * bootstrap.sh
34
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
21
 #+begin_src lisp :results none :exports none
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
22
     (ql:quickload '(:dat :cli))
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
23
     (in-package :dat/dot)
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
24
     (let ((g (make-instance 'directed-graph)))
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
25
       (add-node g :check)
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
26
       (add-node g :bootstrap)
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
27
       (add-node g :autogen)
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
28
       (add-node g "profile.sxp")
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
29
       (add-node g "host.sxp")
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
30
       (add-edge g '(:check "host.sxp"))
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
31
       (add-edge g '(:bootstrap :check))
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
32
       (add-edge g '(:bootstrap :autogen))
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
33
       (add-edge g '("profile.sxp" :autogen))
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
34
       (add-edge g '("host.sxp" :autogen))
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
35
       (to-dot-file g #p"/tmp/bootstrap.dot")
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
36
       ;; (sb-ext:run-program (cli:find-exe "dot") (list "-Tsvg" "/tmp/bootstrap.dot" "-o" "/tmp/bootstrap.svg"))
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
37
       )
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
38
 #+end_src
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
39
 
Richard Westhaver <ellis@rwest.io>
parents: 33
diff changeset
40
 [[cdn:media/img/bootstrap.svg]]
19
Richard Westhaver <ellis@rwest.io>
parents: 18
diff changeset
41
 
33
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
42
 The easiest way to get started is to run the top-level script
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
43
 =bootstrap.sh=. Running the script will:
18
b6889b8a2f83 core/notes
Richard Westhaver <ellis@rwest.io>
parents: 17
diff changeset
44
 
33
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
45
 - run =check.sh=
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
46
 - install the latest cores and lisp runtime from
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
47
   https://packy.compiler.company to the stash
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
48
 - load =autogen.lisp= and call the function =infra/autogen:autogen=
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
49
   - check and apply the host config and profile
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
50
   - build all features in parallel
20
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
51
 
33
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
52
 [[cdn:media/clip/infra-bootstrap.gif]]
20
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
53
 
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
54
 Upon completion a local =stash= directory is provisioned. At this
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
55
 point we no longer need to depend on shell scripting as we have a
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
56
 fully interactive Lisp environment to hack on our builds with.
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
57
 
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
58
 From here, you may run =skel inspect= to inspect the infra =skelfile=
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
59
 and select a target to build on your machine or in a container. Use
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
60
 =skel --help= for a list of other commands.
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
61
 
33
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
62
 ** check.sh
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
63
 This script checks that the current host is qualified. It needs to be
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
64
 ran once before the =autogen.lisp= file is loaded, usually via
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
65
 =bootstrap.sh=.
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
66
 
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
67
 This will interrogate the system, checking for minimum
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
68
 requirements. If any requirement isn't met an error is signaled. On
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
69
 completion, the name of an SXP file is printed which contains the host
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
70
 config.
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
71
 
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
72
 The minimum software dependencies above should be installed via your
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
73
 system's package manager.
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
74
 
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
75
 #+begin_src shell
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
76
 ./check.sh
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
77
 #+end_src
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
78
 
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
79
 #+RESULTS:
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
80
 : zor.sxp
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
81
 
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
82
 ** autogen.lisp
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
83
 Once a core and lisp runtime are available, we load =autogen.lisp= and
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
84
 call =infra/autogen:autogen=. This will process the host configuration
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
85
 (=zor.sxp= in the example above) and the profile (defaults to
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
86
 =default.sxp=), update environment variables for the current shell and
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
87
 take any additional actions needed.
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
88
 
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
89
 The order of operations are loosely as follows:
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
90
 - initialize profile
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
91
 - update host environment variables
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
92
 - check features and ensure host dependencies
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
93
 - initialize host
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
94
 - initialize infra =skelfile=
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
95
 - process all features
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
96
 - quit
aefeb7103fb8 update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 32
diff changeset
97
 
20
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
98
 * TODO deploy.lisp
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
99
 Deploy =infra= builds to a remote.
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
100
 
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
101
 Deployment involves 'packing' a target found in =stash=, archiving it,
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
102
 compressing it, and sending it to a remote location possibly with a
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
103
 signature, checksum, or other metadata, and possibly encrypted. A
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
104
 'package' may also be a raw directory of plain-text files. It's a very
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
105
 loose term. Where the package ends up and how it is packed depends on
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
106
 the build-config and skelfile slots.
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
107
 
6932edcf60ec update infra/quickstart
Richard Westhaver <ellis@rwest.io>
parents: 19
diff changeset
108
 Production deployments end up at https://packy.compiler.company.
23
411eeaebc6d2 doc updates
Richard Westhaver <ellis@rwest.io>
parents: 22
diff changeset
109
 
411eeaebc6d2 doc updates
Richard Westhaver <ellis@rwest.io>
parents: 22
diff changeset
110
 * TODO install.lisp
411eeaebc6d2 doc updates
Richard Westhaver <ellis@rwest.io>
parents: 22
diff changeset
111
 Install core packages locally.