MAGE Got One More Wizard Called Rust
MAGE is an open-source repository that contains many Memgraph modules in various programming languages. The overall idea is to extend Memgraph with capabilities that are not part of the core engine. One of the critical design decisions early on was the compatibility with various programming languages to support different application stacks and developers with varying skill sets. Up until now, only C/C++ and Python were first-class citizens of the Memgraph ecosystem, but Rust is quickly running through the front doors!
I'm happy to announce the full support for writing Memgraph Rust Query Modules in the format of rsmgp-sys. You'll hopefully find the crate exciting and helpful!
The Vision
Rust is a relatively new programming language suitable for various software engineering challenges. People use Rust for low-level system engineering tasks but also to build high-level stuff, like web applications. From the Memgraph perspective, C/C++ is great but required deep knowledge about memory allocation and a lot of time to get the module implemented correctly. On the other hand, Python is excellent for prototyping stuff and shipping new capabilities with lightning speed, but performance is often a considerable bottleneck.
Right in the middle (ok, maybe closer to either side) stands Rust. Rust's compiler ensures the implemented module is correct, but also that the program runs fast. As a language, Rust is quite appealing and easy to write. A developer still has to understand some low-level concepts (it's not as easy as writing Python). Still, the combination of modern language design and a powerful compiler brings to the table an optimal tradeoff between development time and runtime characteristics.
Details
The complete implementation is quite large and took way longer than initially expected (as usual :D). This GitHub PR contains all the code versions. I've tried to follow good coding practices but also deliver in the shortest possible time. Fun fact! It took me 16 weeks to finish this project because I mostly coded in my spare time and over the weekends. Overall, I think I've spent around 20 effective days to get this all working.
The most exciting part was obviously to design and implement the Rust API for the underlying C API. That sounds easy, but there are various challenges from getting bindgen initialized, setting up mockall to unit test the code, and thinking about how best to expose Memgraph capabilities in the standard Rusty way. If you are interested in all the details, check out our YouTube playlist, which documents most of the process!
The implementation is quite extensive, but not all things are optimal yet. Because of the code simplicity and underlying C API, it's challenging to get everything working without value copies. However, it's possible to optimize the wrapper so it becomes more performant and the copying overhead is minimized.
Future Plans
The Memgraph team plans to continue working on the library code but also add many exciting query modules. If you are interested in the low-level stuff, feel free to contribute to the library code inside MAGE. On the other hand, if you are looking to solve some real-world problems, feel free to download Memgraph here, implement the module by utilizing rsmgp-sys, plug these things together, and enjoy!
