1 #+title: Shared Library Skeletons
2 #+setupfile: ../../../clean.theme
5 :ID: 748ba6e4-60db-4ff7-9d78-12c3f67644d8 7 + CODE :: [[https://lab.rwest.io/packy/stash/dysk][packy/stash/dysk]] 8 Our core languages are
[[https://www.rust-lang.org/][Rust]] and
[[https://lisp-lang.org/][Lisp]] - this is the killer combo which will allow NAS-T
9 to rapidly develop high-quality software. As such, it's crucial that these two very
10 different languages (i.e. compilers) are able to interoperate seamlessly.
12 Some interop methods are easy to accomodate via the OS - such as IPC or data sharing,
13 but others are a bit more difficult.
15 In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something
16 that
/can/ be difficult, due to some complications with Rust and because this is not the
17 most popular software stack (yet ;). This is an experiment and may not make it to our
18 code-base, but it's definitely something worth adding to the toolbox in case we need it.
21 :ID: 0e490b3b-0c15-4963-9d43-7d2a689ec371 23 The level of interop we're after in this case is
[[https://en.wikipedia.org/wiki/Foreign_function_interface][FFI]].
25 Basically, calling Rust code from Lisp and vice-versa. There's an article about calling
26 Rust from Common Lisp
[[https://dev.to/veer66/calling-rust-from-common-lisp-45c5][here]] which shows the basics and serves as a great starting point
30 :ID: 6c24dc95-eea6-44fb-8c7f-bb49227ca7bf 32 The complication(s) with Rust I mentioned early is really just that
/it is not C/.
=C= 33 is old, i.e. well-supported with a stable ABI, making the process of creating bindings
34 for a C library a breeze in many languages.
36 For a Rust library we need to first appease the compiler, as explained in
[[https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c][this section]] 37 of the Rustonomicon. Among other things it involves changing the calling-convention of
38 functions with a type signature and editing the Cargo.toml file to produce a
39 C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used
42 [[https://github.com/rodrimati1992/abi_stable_crates][abi_stable_crates]] is a project which addresses some of the ABI concerns, presenting a
43 sort of ABI-API as a Rust library. Perhaps this is the direction the ecosystem will go
44 with in order to maintain an unstable ABI, but for now there is no 'clear' pathway for a
45 friction-less FFI development experience in Rust.
49 :ID: faf8fbcb-3403-4e2f-b881-ef1404ad9780 51 Using FFI involves some overhead. Check
[[https://github.com/dyu/ffi-overhead][here]] for an example benchmark across a few
52 languages. While building the NAS-T core, I'm very much aware of this, and will need a
53 few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular,
54 I'm concerned about crossing multiple language barriers (Rust
<->C
<->Lisp).
58 :ID: ab04fff7-51c9-4f1c-b61b-fa4739932343 62 :ID: 08420b18-3856-4b62-9523-98fb3398afca 64 For starters, I'm going to assume we all have Rust (via
[[https://rustup.rs/][rustup]]) and Lisp (
[[https://www.sbcl.org/][sbcl]] only)
65 installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
69 :ID: cfeb1299-67c5-4a64-863e-f214e195e176 71 Create a new library crate. For this example we're focusing on a 'skeleton' for
72 /dynamic/ libraries only, so our experiment will be called
=dylib-skel= or
*dysk* for
74 src_sh[:exports code]{cargo init dysk --lib && cd dysk}
76 A
=src/lib.rs= will be generated for you. Go ahead and delete that. We're going to be
77 making our own
=lib.rs= file directly in the root directory (just to be cool).
79 The next step is to edit your
=Cargo.toml= file. Add these lines after the
=[package]= 80 section and before
=[dependencies]=:
83 crate-type = ["cdylib","rlib"]
90 This tells Rust to generate a shared C-compatible object with a
=.so= extension which we
91 can open using
[[https://man.archlinux.org/man/dlopen.3.en][dlopen]].
94 :ID: 435d84d6-c959-4bf6-ad37-fb6e524b54ff 98 :ID: 15115ab8-e7b4-4050-9567-bf026cc140d1 100 Next, we want the
=cbindgen= program which we'll use to generate header files for
101 C/C++. This step isn't necessary at all, we just want it for further experimentation.
103 src_sh[:exports code]{cargo install --force cbindgen}
105 We append the
=cbindgen= crate as a
/build dependency/ to our
=Cargo.toml= like so:
106 #+begin_src conf-toml 112 :ID: 21d776b7-9f8d-445c-83c2-8e0c28751827 114 #+begin_src conf-toml :tangle cbindgen.toml 116 autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
117 include_version = true
120 after_includes = "#define DYSK_VERSION \"0.1.0\""
124 documentation_style = "c99"
125 usize_is_size_t = true
131 :ID: 469dd142-bafa-4076-8ba9-baf088e60260 133 #+begin_src rust :tangle build.rs 134 fn main() -> Result<(), cbindgen::Error> {
135 if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
136 b.write_to_file("dysk.h"); Ok(())}
137 else { panic!("failed to generate dysk.h from cbindgen.toml") } }
141 :ID: e9d50e77-4d1e-4d38-90aa-9977d8c6c888 143 #+begin_src rust :tangle lib.rs 144 //! lib.rs --- dysk library
145 use std::ffi::{c_char, c_int, CString};
147 pub extern "C" fn hello() -> *const c_char {
148 CString::new("hello from rust").unwrap().into_raw()}
150 pub extern "C" fn plus(a:c_int,b:c_int) -> c_int {a+b}
152 pub extern "C" fn plus1(n:c_int) -> c_int {n+1}
156 :ID: 84b5dffe-7171-4d72-9788-f288e3185070 158 #+begin_src rust :tangle test.rs 159 //! test.rs --- dysk test
160 fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::plus1(2 as core::ffi::c_int);}}
164 :ID: 30f4fb3e-0757-4df6-947f-8d1e11edabf9 167 cargo build --release
171 :ID: 69b517f2-648d-4e44-b956-7879b3dadf99 173 #+begin_src lisp :tangle dysk.lisp 175 ;; (dysk:hello) ;; => "hello from rust"
178 (:export :hello :plus :plus1))
180 (load-shared-object #P"target/release/libdysk.so")
181 (define-alien-routine hello c-string)
182 (define-alien-routine plus int (a int) (b int))
183 (define-alien-routine plus1 int (n int))
187 :ID: 86e4195d-601b-4736-b852-1209a6d38bdf 190 time target/release/dysk-test
192 #+begin_src lisp :tangle test.lisp 193 (time (dotimes (_ 500000000) (dysk:plus1 2)))