1 #+title: Shared Library Skeletons
2 #+setupfile: ../../clean.theme
3 #
+header-args: :eval never 6 :ID: 748ba6e4-60db-4ff7-9d78-12c3f67644d8 8 + CODE ::
[[https://lab.rwest.io/packy/stash/dysk][packy/stash/dysk]] 9 Our core languages are
[[https://www.rust-lang.org/][Rust]] and
[[https://lisp-lang.org/][Lisp]] - this is the killer combo which will allow NAS-T
10 to rapidly develop high-quality software. As such, it's crucial that these two very
11 different languages (i.e. compilers) are able to interoperate seamlessly.
13 Some interop methods are easy to accomodate via the OS - such as IPC or data sharing,
14 but others are a bit more difficult.
16 In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something
17 that
/can/ be difficult, due to some complications with Rust and because this is not the
18 most popular software stack (yet ;). This is an experiment and may not make it to our
19 code-base, but it's definitely something worth adding to the toolbox in case we need it.
22 :ID: 0e490b3b-0c15-4963-9d43-7d2a689ec371 24 The level of interop we're after in this case is
[[https://en.wikipedia.org/wiki/Foreign_function_interface][FFI]].
26 Basically, calling Rust code from Lisp and vice-versa. There's an article about calling
27 Rust from Common Lisp
[[https://dev.to/veer66/calling-rust-from-common-lisp-45c5][here]] which shows the basics and serves as a great starting point
31 :ID: 6c24dc95-eea6-44fb-8c7f-bb49227ca7bf 33 The complication(s) with Rust I mentioned early is really just that
/it is not C/.
=C= 34 is old, i.e. well-supported with a stable ABI, making the process of creating bindings
35 for a C library a breeze in many languages.
37 For a Rust library we need to first appease the compiler, as explained in
[[https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c][this section]] 38 of the Rustonomicon. Among other things it involves changing the calling-convention of
39 functions with a type signature and editing the Cargo.toml file to produce a
40 C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used
43 [[https://github.com/rodrimati1992/abi_stable_crates][abi_stable_crates]] is a project which addresses some of the ABI concerns, presenting a
44 sort of ABI-API as a Rust library. Perhaps this is the direction the ecosystem will go
45 with in order to maintain an unstable ABI, but for now there is no 'clear' pathway for a
46 friction-less FFI development experience in Rust.
50 :ID: faf8fbcb-3403-4e2f-b881-ef1404ad9780 52 Using FFI involves some overhead. Check
[[https://github.com/dyu/ffi-overhead][here]] for an example benchmark across a few
53 languages. While building the NAS-T core, I'm very much aware of this, and will need a
54 few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular,
55 I'm concerned about crossing multiple language barriers (Rust
<->C
<->Lisp).
59 :ID: ab04fff7-51c9-4f1c-b61b-fa4739932343 63 :ID: 08420b18-3856-4b62-9523-98fb3398afca 65 For starters, I'm going to assume we all have Rust (via
[[https://rustup.rs/][rustup]]) and Lisp (
[[https://www.sbcl.org/][sbcl]] only)
66 installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
70 :ID: cfeb1299-67c5-4a64-863e-f214e195e176 72 Create a new library crate. For this example we're focusing on a 'skeleton' for
73 /dynamic/ libraries only, so our experiment will be called
=dylib-skel= or
*dysk* for
75 src_sh[:exports code]{cargo init dysk --lib && cd dysk}
77 A
=src/lib.rs= will be generated for you. Go ahead and delete that. We're going to be
78 making our own
=lib.rs= file directly in the root directory (just to be cool).
80 The next step is to edit your
=Cargo.toml= file. Add these lines after the
=[package]= 81 section and before
=[dependencies]=:
84 crate-type = ["cdylib","rlib"]
91 This tells Rust to generate a shared C-compatible object with a
=.so= extension which we
92 can open using
[[https://man.archlinux.org/man/dlopen.3.en][dlopen]].
95 :ID: 435d84d6-c959-4bf6-ad37-fb6e524b54ff 99 :ID: 15115ab8-e7b4-4050-9567-bf026cc140d1 101 Next, we want the
=cbindgen= program which we'll use to generate header files for
102 C/C++. This step isn't necessary at all, we just want it for further experimentation.
104 src_sh[:exports code]{cargo install --force cbindgen}
106 We append the
=cbindgen= crate as a
/build dependency/ to our
=Cargo.toml= like so:
107 #+begin_src conf-toml 113 :ID: 21d776b7-9f8d-445c-83c2-8e0c28751827 115 #+begin_src conf-toml :tangle cbindgen.toml 117 autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
118 include_version = true
121 after_includes = "#define DYSK_VERSION \"0.1.0\""
125 documentation_style = "c99"
126 usize_is_size_t = true
132 :ID: 469dd142-bafa-4076-8ba9-baf088e60260 134 #+begin_src rust :tangle build.rs 135 fn main() -> Result<(), cbindgen::Error> {
136 if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
137 b.write_to_file("dysk.h"); Ok(())}
138 else { panic!("failed to generate dysk.h from cbindgen.toml") } }
142 :ID: e9d50e77-4d1e-4d38-90aa-9977d8c6c888 144 #+begin_src rust :tangle lib.rs 145 //! lib.rs --- dysk library
146 use std::ffi::{c_char, c_int, CString};
148 pub extern "C" fn hello() -> *const c_char {
149 CString::new("hello from rust").unwrap().into_raw()}
151 pub extern "C" fn plus(a:c_int,b:c_int) -> c_int {a+b}
153 pub extern "C" fn plus1(n:c_int) -> c_int {n+1}
157 :ID: 84b5dffe-7171-4d72-9788-f288e3185070 159 #+begin_src rust :tangle test.rs 160 //! test.rs --- dysk test
161 fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::plus1(2 as core::ffi::c_int);}}
165 :ID: 30f4fb3e-0757-4df6-947f-8d1e11edabf9 168 cargo build --release
172 :ID: 69b517f2-648d-4e44-b956-7879b3dadf99 174 #+begin_src lisp :tangle dysk.lisp 176 ;; (dysk:hello) ;; => "hello from rust"
179 (:export :hello :plus :plus1))
181 (load-shared-object #P"target/release/libdysk.so")
182 (define-alien-routine hello c-string)
183 (define-alien-routine plus int (a int) (b int))
184 (define-alien-routine plus1 int (n int))
188 :ID: 86e4195d-601b-4736-b852-1209a6d38bdf 191 time target/release/dysk-test
193 #+begin_src lisp :tangle test.lisp 194 (time (dotimes (_ 500000000) (dysk:plus1 2)))