Set up HA cluster with Docker ^Enterprise

This example will show how to set up a highly available cluster in Memgraph using 3 coordinators and 3 data instances. This setup example has been created locally using one server.

💡

In production, it is always advised to run each instance on its dedicated server. The setup however is very similar. Make sure that the IPs of the machines are accessible inside the cluster in order to do proper registration of the cluster.

Start all instances

1. Start coordinator1:

docker run  --name coord1 --network=host -p 7691:7691 -p 7444:7444
    -v mg_lib1:/var/lib/memgraph
    -v mg_log1:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7691
    --log-level=TRACE
    --also-log-to-stderr
    --coordinator-id=1
    --coordinator-port=10111
    --management-port=12121
    --coordinator-hostname=localhost
    --nuraft-log-file=/var/log/memgraph/nuraft

2. Start coordinator2:

docker run  --name coord2 --network=host -p 7692:7692 -p 7445:7444
    -v mg_lib2:/var/lib/memgraph
    -v mg_log2:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7692
    --log-level=TRACE
    --also-log-to-stderr
    --coordinator-id=2
    --coordinator-port=10112
    --management-port=12122
    --coordinator-hostname=localhost
    --nuraft-log-file=/var/log/memgraph/nuraft

3. Start coordinator3:

docker run  --name coord3 --network=host -p 7693:7693 -p 7446:7444
    -v mg_lib3:/var/lib/memgraph
    -v mg_log3:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7693
    --log-level=TRACE
    --also-log-to-stderr
    --coordinator-id=3
    --coordinator-port=10113
    --management-port=12123
    --coordinator-hostname=localhost
    --nuraft-log-file=/var/log/memgraph/nuraft

4. Start instance1:

docker run  --name instance1 --network=host -p 7687:7687 -p 7447:7444
    -v mg_lib4:/var/lib/memgraph
    -v mg_log4:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7687
    --log-level=TRACE
    --also-log-to-stderr
    --management-port=13011

5. Start instance2:

docker run --name instance2 --network=host -p 7688:7688 -p 7448:7444
    -v mg_lib5:/var/lib/memgraph
    -v mg_log5:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7688
    --log-level=TRACE
    --also-log-to-stderr
    --management-port=13012

6. Start instance3:

docker run --name instance3 --network=host -p 7689:7689 -p 7449:7444
    -v mg_lib6:/var/lib/memgraph
    -v mg_log6:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7689
    --log-level=TRACE
    --also-log-to-stderr
    --management-port=13013

Register instances

1. Connect to any coordinator

Here we connect to coordinator1:

mgconsole --port=7691

2. Add coordinator instances to the cluster:

ADD COORDINATOR 1 WITH CONFIG {
    "bolt_server": "localhost:7691",
    "coordinator_server": "localhost:10111",
    "management_server": "localhost:12121"
};
ADD COORDINATOR 2 WITH CONFIG {
    "bolt_server": "localhost:7692",
    "coordinator_server": "localhost:10112",
    "management_server": "localhost:12122"
};
ADD COORDINATOR 3 WITH CONFIG {
    "bolt_server": "localhost:7693",
    "coordinator_server": "localhost:10113",
    "management_server": "localhost:12123"
};

3. Register 3 data instances:

Replace <ip_address> with the container’s IP address. This is necessary for Docker deployments where instances are not on the local host.

REGISTER INSTANCE instance_1 WITH CONFIG {
    "bolt_server": "localhost:7687",
    "management_server": "localhost:13011",
    "replication_server": "localhost:10001"
};
REGISTER INSTANCE instance_2 WITH CONFIG {
    "bolt_server": "localhost:7688",
    "management_server": "localhost:13012",
    "replication_server": "localhost:10002"
};
REGISTER INSTANCE instance_3 WITH CONFIG {
    "bolt_server": "localhost:7689",
    "management_server": "localhost:13013",
    "replication_server": "localhost:10003"
};

4. Elect a MAIN instance:

SET INSTANCE instance_3 TO MAIN;

Check cluster state

Connect to the leader coordinator and check cluster state with SHOW INSTANCES;

name	bolt_server	coordinator_server	management_server	health	role	last_succ_resp_ms
coordinator_1	localhost:7691	localhost:10111	localhost:12121	up	leader	0
coordinator_2	localhost:7692	localhost:10112	localhost:12122	up	follower	16
coordinator_3	localhost:7693	localhost:10113	localhost:12123	up	follower	25
instance_1	localhost:7687	""	localhost:13011	up	replica	39
instance_2	localhost:7688	""	localhost:13012	up	replica	21
instance_3	localhost:7689	""	localhost:13013	up	main	91

Querying the cluster in HA Set up HA cluster with Docker Compose

Set up HA cluster with Docker Enterprise

Start all instances

Register instances

Check cluster state

Set up HA cluster with Docker ^Enterprise