Mercurial > org > notes / 20231105.org

 * DRAFT dylib-skel-1
 - State "DRAFT"      from              [2023-11-05 Sun 22:23]
 ** Overview
 Our core languages are [[https://www.rust-lang.org/][Rust]] and [[https://lisp-lang.org/][Lisp]] - this is the killer combo which will allow NAS-T
 to rapidly develop high-quality software. As such, it's crucial that these two very
 different languages (i.e. compilers) are able to interoperate seamlessly.
 
 Some interop methods are easy to accomodate via the OS - such as IPC or data sharing,
 but others are a bit more difficult.
 
 In this 2-part series we'll build a FFI bridge between Rust and Lisp, which is something
 that /can/ be difficult, due to some complications with Rust and because this is not the
 most popular software stack (yet ;). This is an experiment and may not make it to our
 code-base, but it's definitely something worth adding to the toolbox in case we need it.
 
 ** FFI
 The level of interop we're after in this case is [[https://en.wikipedia.org/wiki/Foreign_function_interface][FFI]].
 
 Basically, calling Rust code from Lisp and vice-versa. There's an article about calling
 Rust from Common Lisp [[https://dev.to/veer66/calling-rust-from-common-lisp-45c5][here]] which shows the basics and serves as a great starting point
 for those interested.
 *** Rust != C
 The complication(s) with Rust I mentioned early is really just that /it is not C/. =C=
 is old, i.e. well-supported with a stable ABI, making the process of creating bindings
 for a C library a breeze in many languages.
 
 For a Rust library we need to first appease the compiler, as explained in [[https://doc.rust-lang.org/nomicon/ffi.html#calling-rust-code-from-c][this section]]
 of the Rustonomicon. Among other things it involves changing the calling-convention of
 functions with a type signature and editing the Cargo.toml file to produce a
 C-compatible ABI binary. The Rust default ABI is unstable and can't reliably be used
 like the C ABI can.
 
 *** Overhead
 Using FFI involves some overhead. Check [[https://github.com/dyu/ffi-overhead][here]] for an example benchmark across a few
 languages. While building the NAS-T core, I'm very much aware of this, and will need a
 few sanity benchmarks to make sure the cost doesn't outweigh the benefit. In particular,
 I'm concerned about crossing multiple language barriers (Rust<->C<->Lisp).
 
 ** Rust -> C -> Lisp
 *** Setup
 For starters, I'm going to assume we all have Rust (via =rustup=) and Lisp (=sbcl= only)
 installed on our GNU/Linux system (some tweaks needed for Darwin/Windows, not covered in
 this post).
 **** Cargo
 Create a new library crate. For this example we're focusing on a 'skeleton' for
 /dynamic/ libraries only, so our experiment will be called =dylib-skel= or *dysk* for
 short.
 src_sh[:exports code]{cargo init dysk --lib && cd dysk} 
 
 A =src/lib.rs= will be generated for you. Go ahead and delete that. We're going to be
 making our own =lib.rs= file directly in the root directory (just to be cool).
 
 The next step is to edit your =Cargo.toml= file. Add these lines after the =[package]=
 section and before =[dependencies]=:
 #+begin_src conf-toml
 [lib]
 crate-type = ["cdylib","rlib"]
 path = "lib.rs"
 [[bin]]
 name="dysk-test"
 path="test.rs"
 #+end_src
 
 This tells Rust to generate a shared C-compatible object with a =.so= extension which we
 can open using [[https://man.archlinux.org/man/dlopen.3.en][dlopen]].
 **** cbindgen
 ***** install
 Next, we want the =cbindgen= program which we'll use to generate header files for
 C/C++. This step isn't necessary at all, we just want it for further experimentation.
 
 src_sh[:exports code]{cargo install --force cbindgen}
 
 We append the =cbindgen= crate as a /build dependency/ to our =Cargo.toml= like so:
 #+begin_src conf-toml
 [build-dependencies]
 cbindgen = "0.24"
 #+end_src
 ***** cbindgen.toml
 #+begin_src conf-toml :tangle cbindgen.toml
 language = "C"
 autogen_warning = "/* Warning, this file is autogenerated by cbindgen. Don't modify this manually. */"
 include_version = true
 namespace = "dysk"
 cpp_compat = true
 after_includes = "#define DYSK_VERSION \"0.1.0\""
 line_length = 88
 tab_width = 2
 documentation = true
 documentation_style = "c99"
 usize_is_size_t = true
 [cython]
 header = '"dysk.h"'
 #+end_src
 ***** build.rs
 #+begin_src rust :tangle build.rs
 fn main() -> Result<(), cbindgen::Error> {
   if let Ok(b) = cbindgen::generate(std::env::var("CARGO_MANIFEST_DIR").unwrap()) {
     b.write_to_file("dysk.h"); Ok(())}
   else { panic!("failed to generate dysk.h from cbindgen.toml") } }
 #+end_src
 *** lib.rs
 #+begin_src rust :tangle lib.rs
 //! lib.rs --- dysk library
 use std::ffi::{c_char, c_int, CString};
 #[no_mangle]
 pub extern "C" fn dysk_hello() -> *const c_char {
   CString::new("hello from rust").unwrap().into_raw()}
 #[no_mangle]
 pub extern "C" fn dysk_plus(a:c_int,b:c_int) -> c_int {a+b}
 #[no_mangle]
 pub extern "C" fn dysk_plus1(n:c_int) -> c_int {n+1}
 #+end_src
 *** test.rs
 #+begin_src rust :tangle test.rs
 //! test.rs --- dysk test
 fn main() { let mut i = 0u32; while i < 500000000 {i+=1; dysk::dysk_plus1(2 as core::ffi::c_int);}}
 #+end_src
 *** compile
 #+begin_src sh
 cargo build --release
 #+end_src
 *** load from SBCL
 #+begin_src lisp :tangle dysk.lisp
 (load-shared-object #P"target/release/libdysk.so")
 (define-alien-routine dysk-hello c-string)
 (define-alien-routine dysk-plus int (a int) (b int))
 (define-alien-routine dysk-plus1 int (n int))
 (dysk-hello) ;; => "hello from rust"
 #+end_src
 *** benchmark
 #+begin_src shell
 time target/release/dysk-test
 #+end_src
 #+begin_src lisp :tangle test.lisp
 (time (dotimes (_ 500000000) (dysk-plus1 2)))
 #+end_src