# Set up HA cluster with Docker <sup style={{ fontSize: '0.6em', color: '#888' }}>Enterprise</sup>

This example will show how to set up a highly available cluster in Memgraph
using 3 coordinators and 3 data instances. This setup example has been created
locally using one server.

> **Note**
>
> **In production, it is always advised to run each instance on its dedicated
> server.** The setup however is very similar. Make sure that the IPs of the
> machines are accessible inside the cluster in order to do proper registration of
> the cluster.

## Start all instances

**1. Start `coordinator1`:**

```plaintext
docker run  --name coord1 --network=host -p 7691:7691 -p 7444:7444
    -v mg_lib1:/var/lib/memgraph
    -v mg_log1:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7691
    --log-level=TRACE
    --also-log-to-stderr
    --coordinator-id=1
    --coordinator-port=10111
    --management-port=12121
    --coordinator-hostname=localhost
    --nuraft-log-file=/var/log/memgraph/nuraft
```

---

**2. Start `coordinator2`:**

```plaintext
docker run  --name coord2 --network=host -p 7692:7692 -p 7445:7444
    -v mg_lib2:/var/lib/memgraph
    -v mg_log2:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7692
    --log-level=TRACE
    --also-log-to-stderr
    --coordinator-id=2
    --coordinator-port=10112
    --management-port=12122
    --coordinator-hostname=localhost
    --nuraft-log-file=/var/log/memgraph/nuraft
```

---

**3. Start `coordinator3`:**

```plaintext
docker run  --name coord3 --network=host -p 7693:7693 -p 7446:7444
    -v mg_lib3:/var/lib/memgraph
    -v mg_log3:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7693
    --log-level=TRACE
    --also-log-to-stderr
    --coordinator-id=3
    --coordinator-port=10113
    --management-port=12123
    --coordinator-hostname=localhost
    --nuraft-log-file=/var/log/memgraph/nuraft
```

---

**4. Start `instance1`:**

```plaintext
docker run  --name instance1 --network=host -p 7687:7687 -p 7447:7444
    -v mg_lib4:/var/lib/memgraph
    -v mg_log4:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7687
    --log-level=TRACE
    --also-log-to-stderr
    --management-port=13011
```

---

**5. Start `instance2`:**

```plaintext
docker run --name instance2 --network=host -p 7688:7688 -p 7448:7444
    -v mg_lib5:/var/lib/memgraph
    -v mg_log5:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7688
    --log-level=TRACE
    --also-log-to-stderr
    --management-port=13012
```

---

**6. Start `instance3`:**

```plaintext
docker run --name instance3 --network=host -p 7689:7689 -p 7449:7444
    -v mg_lib6:/var/lib/memgraph
    -v mg_log6:/var/log/memgraph
    -e MEMGRAPH_ORGANIZATION_NAME=<YOUR ORGANIZATION NAME>
    -e MEMGRAPH_ENTERPRISE_LICENSE="<YOUR ENTERPRISE LICENSE>"
    memgraph/memgraph-mage
    --bolt-port=7689
    --log-level=TRACE
    --also-log-to-stderr
    --management-port=13013
```

## Register instances

**1. Connect to any coordinator**

Here we connect to coordinator1:
```plaintext
mgconsole --port=7691
```
---

**2. Add coordinator instances to the cluster:**

```plaintext
ADD COORDINATOR 1 WITH CONFIG {
    "bolt_server": "localhost:7691",
    "coordinator_server": "localhost:10111",
    "management_server": "localhost:12121"
};
ADD COORDINATOR 2 WITH CONFIG {
    "bolt_server": "localhost:7692",
    "coordinator_server": "localhost:10112",
    "management_server": "localhost:12122"
};
ADD COORDINATOR 3 WITH CONFIG {
    "bolt_server": "localhost:7693",
    "coordinator_server": "localhost:10113",
    "management_server": "localhost:12123"
};
```
---
**3. Register 3 data instances:**

> **Info**
>
> Replace `<ip_address>` with the container's IP address. This is necessary for Docker deployments where instances are not on the local host.

```plaintext
REGISTER INSTANCE instance_1 WITH CONFIG {
    "bolt_server": "localhost:7687",
    "management_server": "localhost:13011",
    "replication_server": "localhost:10001"
};
REGISTER INSTANCE instance_2 WITH CONFIG {
    "bolt_server": "localhost:7688",
    "management_server": "localhost:13012",
    "replication_server": "localhost:10002"
};
REGISTER INSTANCE instance_3 WITH CONFIG {
    "bolt_server": "localhost:7689",
    "management_server": "localhost:13013",
    "replication_server": "localhost:10003"
};
```
---
**4. Elect a MAIN instance:**
  
```plaintext
SET INSTANCE instance_3 TO MAIN;
```

## Check cluster state

Connect to any coordinator and check cluster state with `SHOW INSTANCES`. The
query is always answered by the leader, so followers report the same state:

| name          | bolt_server    | coordinator_server | management_server | health | role     | last_succ_resp_ms |
| ------------- | -------------- | ------------------ | ----------------- | ------ | -------- | ----------------  |
| coordinator_1 | localhost:7691 | localhost:10111    | localhost:12121   | up     | leader   |  0                |
| coordinator_2 | localhost:7692 | localhost:10112    | localhost:12122   | up     | follower |  16               |
| coordinator_3 | localhost:7693 | localhost:10113    | localhost:12123   | up     | follower |  25               |
| instance_1    | localhost:7687 |       ""           | localhost:13011   | up     | replica  |  39               |
| instance_2    | localhost:7688 |       ""           | localhost:13012   | up     | replica  |  21               |
| instance_3    | localhost:7689 |       ""           | localhost:13013   | up     | main     |  91               |