WASM to the Moon - Introducing the Very First WASM Based Client

Hi folks,

It’s Kostas again, and this is the last article of my WebAssembly series. In case you missed it, here is Part 1 and Part 2. Today I will finally talk about how jsmgclient, Memgraph’s WASM-based JavaSript client adapter, came to life.

For the compiler toolchain, I decided to use Emscripten since it bundles nicely with JS (if you recall from the previous article, the system libraries are exported in JS, which provides a nice native integration). Moreover, Emscripten integrates nicely with cmake, making it easy to extend the current build scripts of mgclient. Therefore, after a set of small changes to the codebase, I managed to add WASM as a compilation target of mgclient. The best part is that the integration is so smooth that you can build it in one go with cmake .. -DWASM=ON && make -j4, and if you are curious about how this all works internally, check out our cmake files GitHub repository.

With a brand new shiny WASM module at my disposal, it was time to consume it and build the very first WASM-based Memgraph client. But before I dive into that, I think it's important to talk about WASM modules and what they actually contain a bit more. A WASM module is nothing more than an IR that contains the functions and data types exported during the compilation of a target. For example, mgclient exposes a connect() method, which establishes a connection between the client and Memgraph, effectively opening a communication channel with the database instance. Therefore, if the function connect() is exported while compiling mgclient to WASM, thenconnect() is publicly available to any of its consumers. At its core, the WASM module bundles system libraries and specifies the module's interface with the outside world. Of course, there are other important details of a WASM module, but each of them deserves a couple of articles on their own.

So back in action, it's time to consume the module and have some fun. I quickly wrote wrappers for the data types supported by the mgclient's communication protocol (Memgraph supports the Bolt protocol). These wrappers are trivial to implement because the data types already implemented inmgclient are now accessible by JS via the WASM-exported interface. Finally, I quickly wrapped the networking interface, pretty much doing the same thing as above with trivial wrappers (remember the connect() above?). Filled with anticipation and excited to see the results of my little experiment, I fired up a Memgraph instance and wrote a typical Hello world example that establishes a connection with Memgraph and runs a create node query. I used Node.js to run my JavaScript client example, and BOOM - the client froze, and the connection never reached the other end. After debugging, skimming through the Emscripten docs and asking the Slack community it finally struck me. Remember the limitations imposed by JS I mentioned in my previous article? The networking system calls are all failing because they are being emulated as Websockets, which are async, but our networking stack is synchronous, meaning that the event loop of the JavaScript engine must yield to allow the async calls to proceed. Even worst, at that time, Memgraph didn't support Websocket connections (it does now :D).

After a few stitches to our mgclient I adapted the networking stack to be asynchronous with the help of Emscripten functions (see emscripten.h and emscripten_sleep) and I finally managed to establish a connection with Memgraph and run the very first Hello world example. Everything was mostly working, but it must be noted that during the development of the rest of the wrappers, I stumbled on a few peculiarities of WASM, like BigInt support (remember it’s Javascript?) but nothing too major to mention here. In the following week, I played and migrated a large chunk of mgclient to JavaScript, bringing the first WASM-based client to life.

Something that I found really noteworthy is how simple and trivial the JavaScript wrappers are to implement. It was as simple as delegating calls to the exported functions of the WASM module. Simply put, the inter-op was almost (as briefly said with BigInts) seamless. And at that point is when I thought - if wrappers are trivial, that is, if they are just calls to exported functions, and if wasmer works with multiple languages via the WASI standard, then maybe we could write a generator that basically uses the wasmer runtime on each client, and the wrappers are generated to the targeted language. And you might say: "Hey Kostas, this is SWIG all over again!" I would say it is not. There are no bindings, and there is only one layer of abstraction, that is, WebAssembly. All in all, this is not something I experimented a lot with, but I suspect it could have a huge impact on how database client libraries are implemented in the future.

And that's how my story comes to an end with a happy and successful little experiment and a new client joining the Memgraph family. Check its status at the jsmgclient repository.

To my readers, I hope you all enjoyed my WebAssembly series and as always,

Cheers with a cold, mate!