Custom query modules

Many graph algorithms and uitility procedures have already been developed and are available as part of the MAGE library you can add to your Memgraph installation. The library is already included if you are using Memgraph Platform or Memgraph MAGE Docker images to run Memgraph.

But, you can also implement custom query modules. Memgraph supports extending the query language with user-written procedures in C, **C++*, Python, and Rust. These procedures are grouped into modules - query module files (either *.so or *.py files).

Every single Memgraph installation comes with the example.so and py_example.py query modules located in the /usr/lib/memgraph/query_modules directory. They were provided as examples of query modules for you to examine and learn from.

Each query module file corresponds to one query module, and file names are mapped as query module names. For example, example.so will be mapped as example module, and py_example.py will be mapped as py_example module. If each module file has a procedure called procedure defined, those procedures would be mapped in the Cypher query language as example.procedure() and py_example.procedure() respectively.

You can develop custom procedures from Memgraph itself.

Develop custom query modules

If you want to develop Python procedures you can do so from Memgraph Lab, the visual user interface. The files will be saved at /var/lib/memgraph/internal_modules. Just navigate to Query Modules and select + New Module.

If you don't won't to develop from Memgraph Lab, you can:

• install MAGE development environment from Docker Hub
• build MAGE using the docker build command
• build MAGE from source

Then, select a language you want to develop in.

Install MAGE and import query modules

docker run -p 7687:7687 memgraph/memgraph-mage:<version>-dev

docker run --rm -p 7687:7687 --name mage memgraph-mage:version-dev

docker run --rm -p 7687:7687 --name mage memgraph-mage:1.4-dev

docker cp . mage:/mage/

docker exec -u root -it mage /bin/bash

rm -rf cpp/build

python3 setup build -p /usr/lib/memgraph/query_modules/

exit

Querying

Query modules are loaded into Memgraph on startup, so if your instance was already running, you need to execute the following query inside one of the querying platforms (mgonsole, Memgraph Lab or a client library) to load them:

CALL mg.load_all();

Run a query and test your module. All procedures are called using the CALL clause:

MERGE (start:Node {id: 0})-[:RELATION]->(:Node {id: 1})-[:RELATION]->(:Node {id: 2})
CALL random_walk.get(start, 2) YIELD path
RETURN path

Since every query module is run as one transaction in Memgraph, the user can stop the query module by terminating the corresponding transaction, but only if crucial parts of the code, such as while and until loops, or similar points where the procedure might become costly, are preceded with CheckMustAbort() function. The user can then find out the transaction ID using SHOW TRANSACTIONS command and terminate it using the TERMINATE TRANSACTIONS <tid> command.

Testing

Test decoupled parts of your code that don't depend on Memgraph like you would in any other setting. E2e (end to end) tests, on the other hand, depend on internal Memgraph data structures, like nodes and edges. After running Memgraph, we need to prepare the testing environment on the host machine. Position yourself in the mage directory you cloned from GitHub. The expected folder structure for each module is the following:

mage
└── e2e
    └── random_walk_test
        └── test_base
            ├── input.cyp
            └── test.yml

input.cyp represents a Cypher script for entering the data into the database. To simplify this tutorial, we'll leave the database empty. test.yml specifies which test query should be run by the database and what should be the result or exception. Create the files following the aforementioned directory structure.

input.cyp

MATCH (n) DETACH DELETE n;

test.yml

query: >
  MATCH (start:Node {id: 0})
    CALL random_walk.get(start, 2) YIELD path
    RETURN path
 
output: []

Lastly, run the e2e tests with python:

python test_e2e