System configuration

System configuration

Enhancing Memgraph's performance can sometimes involve optimizing operating system-level configurations. These adjustments primarily relate to the settings governing resource allocation and process management.

The following configuration steps are written based on the Ubuntu 22.04 operating system. The steps may vary slightly depending on the Linux distribution you are using.

Handle numerous open files simultaneously

The fs.file-max parameter in Linux sets the maximum number of file handles that the Linux kernel will allocate. This adjustment is important when the system needs to handle numerous open files simultaneously. To adjust this parameter, do the following:

Open `/etc/sysctl.conf` in a text editor

You need root privileges to edit this file, which is used for configuring kernel parameters at runtime.

Set the file handle limit

Add the line fs.file-max=100000 to sysctl.conf.

Apply the changes

Save your changes and exit the text editor. Run sudo sysctl -p to apply the changes immediately without rebooting.

Verify the changes

Run sysctl fs.file-max to check that the new value has been applied. It should return 100000.

Increasing memory map areas

If users are working on a larger scale, there is a possibility that a transaction is hanging because Memgraph tries to allocate memory and it couldn’t as it was out of virtual memory space. User can adjust the vm.max_map_count parameter to define the maximum number of memory map areas a process can have.

Recommended values for the vm.max_map_count parameter

For optimal system performance, the vm.max_map_count value should be chosen in accordance with your system's RAM, aiming for approximately one memory map area per 128 KB of system memory.

Amount of RAMvm.max_map_count value
8GB - 32GBvm.max_map_count=262144
32GB - 64GBvm.max_map_count=524288
64GB - 128GBvm.max_map_count=1048576
128GB - 256GBvm.max_map_count=2097152
256GB - 512GBvm.max_map_count=4194304
512GB - 1TBvm.max_map_count=8388608
1TB - 1.5TBvm.max_map_count=12582912
1.5TB - 2TBvm.max_map_count=16777216
2TB - 2.5TBvm.max_map_count=20971520
2.5TB - 3TBvm.max_map_count=25165824
3TB - 3.5TBvm.max_map_count=29360128
3.5TB - 4TBvm.max_map_count=33554432

Immediate adjustment

For instance, if your recommended vm.max_map_count value is 262144, run sudo sysctl -w vm.max_map_count=262144 to immediately increase the memory map area limit to 262,144. This change takes effect right away but lasts only until the next reboot.

Persistent adjustment

To ensure that changes to vm.max_map_count persist after a reboot, follow these steps:

Edit `/etc/sysctl.conf`

Add vm.max_map_count=262144 to /etc/sysctl.conf. Root privileges are required for editing.

Apply the changes

Save your changes and run sudo sysctl -p to apply them right away.

Verify the changes

Run sysctl vm.max_map_count to ensure the new value is in effect. It should return 262144.

Addressing stack overflow issues

Stack overflow issues may occur due to a large query in Memgraph. To address these issues, there are two possible approaches:

  1. Increase the stack size
  2. Split the query into smaller chunks

Increase the stack size

With Docker

Run Docker image

Run the Memgraph using:

docker run --rm -it --name memgraph --ulimit stack=33554432:33554432 -p 7687:7687 -p 7444:7444 memgraph/memgraph

Replace 33554432 with the required stack size in bytes.

Verify the changes

To check the new stack size limits, run:

docker exec memgraph sh -c "ulimit -a | grep stack"

This command should return stack(kbytes) 32768, confirming that the new stack size limits have been applied. This value is equivalent of 32MB

Without Docker

Open /etc/security/limits.conf in a text editor

You need root privileges to edit this file. This file is used to set user or group-specific soft and hard limits for various system resources.

Set the stack size limit

Add the following lines to set both the soft and hard stack size limits for the user:

memgraph soft stack 32768
memgraph hard stack 32768

The value 32768 represents the stack size in kilobytes, equivalent to 32MB.

Apply the changes

Save your changes and exit the text editor. For the changes to take effect, the user may need to log out and then log back in.

Verify the changes

To check the new stack size limits, switch to the user account and run:

ulimit -s

This command should return 32768, confirming that the new stack size limits have been applied.

Split the query into smaller chunks

To avoid stack overflow, divide your large query into smaller parts. This approach requires careful handling of variables and node references across queries. For creating edges between nodes created in separate queries, use the MATCH clause to reference nodes from previous queries:

// First Query
CREATE (a:Node {id: 1});
// Second Query
MATCH (a:Node {id: 1})
CREATE (b:Node {id: 2}), (a)-[:RELATES_TO]->(b);

This method reduces the stack size required for each query, potentially avoiding the stack overflow issue.