1 #+title: Shared Library Skeletons
2 #+setupfile: ../../clean.theme
3 #
+header-args: :eval never 4 #+property: header-args :eval no-export 7 :ID: 748ba6e4-60db-4ff7-9d78-12c3f67644d8 9 + CODE ::
[[https://lab.rwest.io/packy/stash/dysk][packy/stash/dysk]] 10 Our core languages are
[[https://www.rust-lang.org/][Rust]] and
[[https://lisp-lang.org/][Lisp]] - this is the killer combo which will allow NAS-T
11 to rapidly develop high-quality software. As such, it's crucial that these two very
12 different languages (i.e. compilers) are able to interoperate seamlessly.
14 Some interop methods are easy to accomodate via the OS - such as IPC or data sharing,
15 but others are a bit more difficult.
17 In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something
18 that
/can/ be difficult, due to some complications with Rust and because this is not the
19 most popular software stack (yet ;). This is an experiment and may not make it to our
20 code-base, but it's definitely something worth adding to the toolbox in case we need it.
23 :ID: 0e490b3b-0c15-4963-9d43-7d2a689ec371 25 The level of interop we're after in this case is
[[https://en.wikipedia.org/wiki/Foreign_function_interface][FFI]].
27 Basically, calling Rust code from Lisp and vice-versa. There's an article about calling
28 Rust from Common Lisp
[[https://dev.to/veer66/calling-rust-from-common-lisp-45c5][here]] which shows the basics and serves as a great starting point
32 :ID: 6c24dc95-eea6-44fb-8c7f-bb49227ca7bf 34 The complication(s) with Rust I mentioned early is really just that
/it is not C/.
=C= 35 is old, i.e. well-supported with a stable ABI, making the process of creating bindings
36 for a C library a breeze in many languages.
38 For a Rust library we need to first appease the compiler, as explained in
[[https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c][this section]] 39 of the Rustonomicon. Among other things it involves changing the calling-convention of
40 functions with a type signature and editing the Cargo.toml file to produce a
41 C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used
44 [[https://github.com/rodrimati1992/abi_stable_crates][abi_stable_crates]] is a project which addresses some of the ABI concerns, presenting a
45 sort of ABI-API as a Rust library. Perhaps this is the direction the ecosystem will go
46 with in order to maintain an unstable ABI, but for now there is no 'clear' pathway for a
47 friction-less FFI development experience in Rust.
51 :ID: faf8fbcb-3403-4e2f-b881-ef1404ad9780 53 Using FFI involves some overhead. Check
[[https://github.com/dyu/ffi-overhead][here]] for an example benchmark across a few
54 languages. While building the NAS-T core, I'm very much aware of this, and will need a
55 few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular,
56 I'm concerned about crossing multiple language barriers (Rust
<->C
<->Lisp).
60 :ID: ab04fff7-51c9-4f1c-b61b-fa4739932343 64 :ID: 08420b18-3856-4b62-9523-98fb3398afca 66 For starters, I'm going to assume we all have Rust (via
[[https://rustup.rs/][rustup]]) and Lisp (
[[https://www.sbcl.org/][sbcl]] only)
67 installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
71 :ID: cfeb1299-67c5-4a64-863e-f214e195e176 73 Create a new library crate. For this example we're focusing on a 'skeleton' for
74 /dynamic/ libraries only, so our experiment will be called
=dylib-skel= or
*dysk* for
76 src_sh[:exports code]{cargo init dysk --lib && cd dysk}
78 A
=src/lib.rs= will be generated for you. Go ahead and delete that. We're going to be
79 making our own
=lib.rs= file directly in the root directory (just to be cool).
81 The next step is to edit your
=Cargo.toml= file. Add these lines after the
=[package]= 82 section and before
=[dependencies]=:
85 crate-type = ["cdylib","rlib"]
92 This tells Rust to generate a shared C-compatible object with a
=.so= extension which we
93 can open using
[[https://man.archlinux.org/man/dlopen.3.en][dlopen]].
96 :ID: 435d84d6-c959-4bf6-ad37-fb6e524b54ff 100 :ID: 15115ab8-e7b4-4050-9567-bf026cc140d1 102 Next, we want the
=cbindgen= program which we'll use to generate header files for
103 C/C++. This step isn't necessary at all, we just want it for further experimentation.
105 src_sh[:exports code]{cargo install --force cbindgen}
107 We append the
=cbindgen= crate as a
/build dependency/ to our
=Cargo.toml= like so:
108 #+begin_src conf-toml 114 :ID: 21d776b7-9f8d-445c-83c2-8e0c28751827 116 #+begin_src conf-toml :tangle cbindgen.toml 118 autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
119 include_version = true
122 after_includes = "#define DYSK_VERSION \"0.1.0\""
126 documentation_style = "c99"
127 usize_is_size_t = true
133 :ID: 469dd142-bafa-4076-8ba9-baf088e60260 135 #+begin_src rust :tangle build.rs 136 fn main() -> Result<(), cbindgen::Error> {
137 if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
138 b.write_to_file("dysk.h"); Ok(())}
139 else { panic!("failed to generate dysk.h from cbindgen.toml") } }
143 :ID: e9d50e77-4d1e-4d38-90aa-9977d8c6c888 145 #+begin_src rust :tangle lib.rs 146 //! lib.rs --- dysk library
147 use std::ffi::{c_char, c_int, CString};
149 pub extern "C" fn hello() -> *const c_char {
150 CString::new("hello from rust").unwrap().into_raw()}
152 pub extern "C" fn plus(a:c_int,b:c_int) -> c_int {a+b}
154 pub extern "C" fn plus1(n:c_int) -> c_int {n+1}
158 :ID: 84b5dffe-7171-4d72-9788-f288e3185070 160 #+begin_src rust :tangle test.rs 161 //! test.rs --- dysk test
162 fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::plus1(2 as core::ffi::c_int);}}
166 :ID: 30f4fb3e-0757-4df6-947f-8d1e11edabf9 169 cargo build --release
173 :ID: 69b517f2-648d-4e44-b956-7879b3dadf99 175 #+begin_src lisp :tangle dysk.lisp 177 ;; (dysk:hello) ;; => "hello from rust"
180 (:export :hello :plus :plus1))
182 (load-shared-object #P"target/release/libdysk.so")
183 (define-alien-routine hello c-string)
184 (define-alien-routine plus int (a int) (b int))
185 (define-alien-routine plus1 int (n int))
189 :ID: 86e4195d-601b-4736-b852-1209a6d38bdf 192 time target/release/dysk-test
194 #+begin_src lisp :tangle test.lisp 195 (time (dotimes (_ 500000000) (dysk:plus1 2)))