Custom query modules
Many graph algorithms and uitility procedures have already been developed and are available as part of the MAGE library you can add to your Memgraph installation. The library is already included if you are using Memgraph Platform or Memgraph MAGE Docker images to run Memgraph.
But, you can also implement custom query modules. Memgraph supports extending
the query language with user-written procedures in C, **C++*, Python,
and Rust. These procedures are grouped into modules - query module files
(either *.so
or *.py
files).
Every single Memgraph installation comes with the example.so
and
py_example.py
query modules located in the /usr/lib/memgraph/query_modules
directory. They were provided as examples of query modules for you to examine
and learn from.
Each query module file corresponds to one query module, and file names are
mapped as query module names. For example, example.so
will be mapped as
example
module, and py_example.py
will be mapped as py_example
module. If
each module file has a procedure called procedure
defined, those procedures
would be mapped in the Cypher query language as example.procedure()
and
py_example.procedure()
respectively.
You can develop custom procedures from Memgraph itself.
Develop custom query modules
If you want to develop Python procedures you can do so from Memgraph
Lab, the visual user interface. The files will be saved at
/var/lib/memgraph/internal_modules
. Just navigate to Query Modules and
select + New Module.
If you don't won't to develop from Memgraph Lab, you can:
- install MAGE development environment from Docker Hub
- build MAGE using the
docker build
command - build MAGE from source
Then, select a language you want to develop in.
Install MAGE and import query modules
The following steps will guide you through installing MAGE development environment from Docker Hub and importing develooped query modules.
Download the MAGE image
Run the following command to get the Memgraph and MAGE development Docker image:
docker run -p 7687:7687 memgraph/memgraph-mage:<version>-dev
By running this command, you will get an image with the following tools
installed: Python3
, Rust
, Clang
, Make
, and CMake
.
Develop a query
Develop query modules in Python, C, C++ or Rust (opens in a new tab).
Start the MAGE container
Use the following command to start the MAGE container:
docker run --rm -p 7687:7687 --name mage memgraph-mage:version-dev
Be sure to replace the version with the specific version, for example:
docker run --rm -p 7687:7687 --name mage memgraph-mage:1.4-dev
Copy the files to the container
Copy the files to the container named mage
:
docker cp . mage:/mage/
Enter the container
Position yourself inside the container as root
:
docker exec -u root -it mage /bin/bash
If you performed the build locally, make sure to delete the cpp/build
directory because you might be dealing with different architectures or problems with CMakeCache.txt
.
To delete it, run:
rm -rf cpp/build
Build MAGE
Build MAGE with the option to copy executables from mage/dist
to /usr/lib/memgraph/query_modules
:
python3 setup build -p /usr/lib/memgraph/query_modules/
Exit the container
Everything should be ready to exit the container and load the query modules:
exit
Querying
Query modules are loaded into Memgraph on startup, so if your instance was already running, you need to execute the following query inside one of the querying platforms (mgonsole, Memgraph Lab or a client library) to load them:
CALL mg.load_all();
Run a query and test your module. All procedures are called using the CALL
clause:
MERGE (start:Node {id: 0})-[:RELATION]->(:Node {id: 1})-[:RELATION]->(:Node {id: 2})
CALL random_walk.get(start, 2) YIELD path
RETURN path
Since every query module is run as one transaction in Memgraph, the user can
stop the query module by terminating the corresponding
transaction, but only if crucial parts
of the code, such as while and until loops, or similar points where the
procedure might become costly, are preceded with CheckMustAbort() function. The
user can then find out the transaction ID using SHOW TRANSACTIONS
command and
terminate it using the TERMINATE TRANSACTIONS <tid>
command.
Testing
Test decoupled parts of your code that don't depend on Memgraph like you would in any other setting. E2e (end to end) tests, on the other hand, depend on internal Memgraph data structures, like nodes and edges. After running Memgraph, we need to prepare the testing environment on the host machine. Position yourself in the mage directory you cloned from GitHub. The expected folder structure for each module is the following:
mage
└── e2e
└── random_walk_test
└── test_base
├── input.cyp
└── test.yml
input.cyp
represents a Cypher script for entering the data into the database.
To simplify this tutorial, we'll leave the database empty. test.yml
specifies
which test query should be run by the database and what should be the result or
exception. Create the files following the aforementioned directory structure.
input.cyp
MATCH (n) DETACH DELETE n;
test.yml
query: >
MATCH (start:Node {id: 0})
CALL random_walk.get(start, 2) YIELD path
RETURN path
output: []
Lastly, run the e2e tests with python:
python test_e2e