Data durability

Memgraph uses two mechanisms to ensure the durability of stored data and make disaster recovery possible:

• write-ahead logging (WAL)
• periodic snapshot creation

These mechanisms generate durability files and save them in the respective wal and snapshots folders in the data directory. Data directory stores permanent data on disk.

Memgraph cannot be used with only WAL files enabled. You can either have only snapshots or snapshots and WAL files.

The default data directory path is /var/lib/memgraph but the path can be changed by modifying the --data-directory configuration flag. To learn how to modify configuration flags, head over to the Configuration page.

With Memgraph Enterprise, the data_directory holds databases directory which splits durability files by database name. The reason for that is the multi-tenant architecture in Memgraph Enterprise, where the durability files for each database are stored under /data_directory/databases/<db_name>. The databases directory will exist even if you’re not using the multi-tenancy feature.

Durability files are deleted when certain events are triggered, for example, exceeding the maximum number of snapshots, defined by the --storage-snapshot-retention-count=3 flag.

To prevent the deletion of durability files, you need to lock the data directory, and enable it again by unlocking the directory.

To manage this behavior, use the following queries:

LOCK DATA DIRECTORY;
UNLOCK DATA DIRECTORY;

To show the status of the data directory, run:

DATA DIRECTORY LOCK STATUS;

To encrypt the data directory, use LUKS as it works with Memgraph out of the box and is undetectable from the application perspective so it shouldn’t break any existing applications.

Durability mechanisms

To configure the durability mechanisms, check their respective configuration flags.

Write-ahead logging

Write-ahead logging (WAL) is a technique applied in providing atomicity and durability to database systems.

In the default IN_MEMORY_TRANSACTIONAL storage mode, Memgraph creates a Delta object each time data is changed. By using Deltas, Memgraph creates write-ahead logs. Each database modification is therefore recorded in a log file before being written to the DB, and in the end the log file contains all steps needed to reconstruct the DB’s most recent state.

Memgraph has WAL enabled by default. To switch it on and off, use the boolean --storage-wal-enabled flag. For other WAL-related flags check the configuration reference guide.

By default, WAL files are located at /var/lib/memgraph/wal.

WAL file lifecycle

Older WAL files are deleted automatically after a snapshot is created since the snapshot contains the full database state up to that point. Only WAL files containing changes after the latest snapshot are retained.

To control WAL file cleanup indirectly, you can limit the number of snapshots via --storage-snapshot-retention-count.

It is not possible to use WAL files exclusively without snapshots. Memgraph enforces periodic snapshots when WAL is enabled and will fail to start if WAL is enabled with snapshot interval set to zero.

WAL data integrity

To guard against silent on-disk corruption, Memgraph protects WAL files with CRC32 checksums:

• The WAL file header (offsets and metadata such as the UUID, epoch ID and sequence number) is protected by its own checksum.
• Each transaction is protected by a 4-byte checksum covering the transaction’s bytes (transaction start, deltas and transaction end).

Checksums are verified automatically during recovery. If a transaction’s stored checksum does not match the recomputed value, the WAL is considered corrupted at that point and recovery stops, so corrupted data is never applied to the database. A mismatch in the WAL header causes recovery from that file to fail.

The same checksums protect WAL files that are buffered on disk on a replica before being applied, so corruption introduced between the main and the replica is detected before the data is committed. Deltas streamed during the commit (PrepareCommitRpc) are not checksummed because the TCP transport already provides integrity guarantees.

Snapshots

Snapshots provide a faster way to restore the states of your database. Snapshots are created periodically based on the value defined with the --storage-snapshot-interval configuration flags, as well as upon exit based on the value of the --storage-snapshot-on-exit configuration flag. When a snapshot creation is triggered, the entire data storage is written to the drive. Nodes and relationships are divided into groups called batches.

Snapshot creation can be made faster by using multiple threads. See Parallelized execution for more information.

On startup, the database state is recovered from the most recent snapshot file. Memgraph can read the data and build the indexes on multiple threads, using batches as a parallelization unit: each thread will recover one batch at a time until there are no unhandled batches.

This means the same batch size might not be suitable for every dataset. A smaller dataset might require a smaller batch size to utilize a multi-threaded processor, while bigger datasets might use bigger batches to minimize the synchronization between the worker threads. Therefore, the size of batches and the number of used threads are configurable similarly to other durability-related settings.

The timestamp of the snapshot is compared with the latest update recorded in the WAL file and, if the snapshot is less recent, the state of the DB will be recovered using the WAL file.

Memgraph has snapshot creation enabled by default. You can configure the exact snapshot creation behavior by defining the relevant flags. Alternatively, you can make one directly by running the following query:

CREATE SNAPSHOT;

By default, snapshot files are saved inside the var/lib/memgraph/snapshots directory. The CREATE SNAPSHOT query will return the path of the newly created snapshot file.

To query which snapshots currently exist in the data directory, execute:

SHOW SNAPSHOTS;

Snapshot and WAL recovery logic

During recovery, Memgraph always attempts to use the fastest and most efficient method to restore the database state:

• If the snapshot has a more recent timeline than the WAL, the database is fully recovered from the latest snapshot.
• If the snapshot has a less recent timeline than the WAL, Memgraph first recovers from the snapshot, and then replays WAL files containing changes made after the snapshot was taken. This ensures recovery to the most recent state.
• Snapshot recovery is typically faster than recovery from WAL because snapshots store the complete state of the database in a single file, while WAL files store incremental changes and need to be replayed sequentially.

Periodic snapshots

IN_MEMORY_TRANSACTIONAL mode supports periodic snapshot creation. The interval can be set at startup via the --storage-snapshot-interval flag or at run-time via the database settings:

SET DATABASE SETTING "storage.snapshot.interval" TO "1200";
SET DATABASE SETTING "storage.snapshot.interval" TO "* * 12 * * *";
SET DATABASE SETTING "storage.snapshot.interval" TO "";

If the interval string is an integer, then it’s treated as the execution period in seconds. Interval can also be defined as a 6-field CRON expression (seconds, minute, hour, day of month, month, day of week). Standard 5-field cron references (e.g. crontab.guru) describe the last five fields; Memgraph adds seconds as the first field. By setting the value to an empty string, the background process is paused and any currently active snapshot creation will finish. Please note that defining the interval via a CRON expression is an Enterprise feature.

If the database is started in or migrated into IN_MEMORY_ANALYTICAL mode, the background thread will pause and no snapshots will be created as long as that mode is active. The job will continue with the last defined interval when the storage mode is changed to IN_MEMORY_TRANSACTIONAL storage mode.

Parallelized execution

Snapshot creation in Memgraph can be optimized using multiple threads, which significantly reduces the time required to create snapshots for large datasets.

This behavior can be controlled using the following flags:

• --storage-parallel-snapshot-creation: This flag determines whether snapshot creation is performed in a multi-threaded fashion. By default, it is set to false. To enable parallelized execution, set this flag to true.
• --storage-snapshot-thread-count: This flag specifies the number of threads to be used for snapshot creation. By default, Memgraph uses the system’s maximum thread count. You can override this value to fine-tune performance based on your system’s resources.

When parallelized execution is enabled, Memgraph divides the data into batches, where the batch size is defined via --storage-items-per-batch. The optimal batch size and thread count may vary depending on the dataset size and system configuration.

When parallelization helps

Parallel execution is especially beneficial when CPU-bound operations dominate the snapshot creation process, such as serialization or compression of in-memory structures. As a general guideline, parallel snapshot creation provides the most significant performance improvement when disk I/O constitutes 25% or less of the total snapshot creation time.

To take full advantage of parallelization, it’s also important to set the --storage-items-per-batch flag appropriately. This value determines how the dataset is split into work units for threads. A good rule of thumb is: Total number of items (vertices + edges) ≈ 4 × number of threads × —storage-items-per-batch This ensures that each thread has enough batches to work on without idling, helping maximize CPU utilization during snapshot creation.

When using multi-threaded snapshot creation with the correct batch size, the disk will once again become the bottleneck. At that point, more threads will not necessarily yield better performance.

Measuring disk write speed on Linux

To determine how fast your disk can handle writes (which influences the I/O bottleneck), you can use the dd command:

dd if=/dev/zero of=testfile bs=1G count=1 oflag=direct

This writes a 1 GB file directly to disk and reports the write speed. After the test, remove the file.

You can also monitor real-time disk utilization during snapshot creation using tools like iostat, iotop, or dstat.

Recovery failure handling

By default, if a database fails durability recovery on startup — because of a truncated WAL, a corrupt snapshot, a missing prefix WAL file, or any other malformed durability file — the entire Memgraph process refuses to boot. Recovery hits a fatal error deep in the durability layer and the process exits. In a multi-tenant deployment, a single corrupt tenant therefore takes down every other (healthy) tenant on the instance, and in a clustered deployment it takes down that instance’s role in the cluster. The only recourse is to manually restore the whole data directory from a backup before the process will start at all.

Setting the --storage-allow-recovery-failure flag to true changes this: a database that fails recovery no longer crashes the process. Instead it comes up in a broken state — an empty, inert in-memory placeholder whose on-disk durability files are left byte-for-byte untouched. The instance boots, all healthy databases work normally, and the operator can inspect which database is broken and recover it online.

The flag defaults to false, so upgrading Memgraph does not silently change how corruption is handled — you keep the existing fail-stop-on-corruption guarantee unless you deliberately opt into availability over it.

Broken database behavior

A broken database:

• comes up empty — it holds no data, and its on-disk snapshots/ and wal/ files are left exactly as they were, so no recovery option is lost;
• rejects every query that touches its data with a clear, actionable error (see below), so you are never served silently wrong (empty) results from a database that failed to load;
• suppresses background durability while broken — periodic snapshots and the on-exit snapshot are skipped and no WAL is written, so an empty placeholder can never overwrite your corrupt durability files with an empty snapshot;
• is re-derived on every startup. The broken state is never persisted — if you fix the underlying files out-of-band, the database simply recovers normally on the next restart.

Any query that operates on a broken database’s data (Cypher reads and writes, DDL, CREATE SNAPSHOT, SHOW INDEX INFO / SHOW CONSTRAINT INFO and other graph-metadata SHOW ... INFO variants) is rejected with:

Database is in the broken state because the recovery process failed. Please recover your database using the RECOVER SNAPSHOT query. If you have a backup of the whole data directory, please replace the current data directory with the backup one and restart the process.

Meta and admin queries that operate on instance-level or system state rather than the database graph remain available so you can diagnose and recover — for example USE DATABASE, SHOW DATABASES, SHOW STORAGE INFO, SHOW CONFIG, RECOVER SNAPSHOT, authentication and replication queries, LOCK/UNLOCK DATA DIRECTORY, and FREE MEMORY.

Identifying a broken database

Two operator-facing queries report each database’s health as ready or broken:

• SHOW DATABASES has a Health column (alongside Name and State), so you can immediately see which database did not load:

  +------------+-------+---------+
  | Name       | State | Health  |
  +------------+-------+---------+
  | memgraph   | HOT   | ready   |
  | mydb       | HOT   | broken  |
  +------------+-------+---------+

• SHOW STORAGE INFO ON CURRENT DATABASE (and ON DATABASE <name>) includes a health row with the value ready or broken.

Restoration paths

Once a database is broken, you have several ways to restore it. Choose based on whether you have a good snapshot, whether you are running in a cluster, and whether you want to keep the database’s data at all.

Recover in place with RECOVER SNAPSHOT

The primary cure is RECOVER SNAPSHOT, which loads a known-good snapshot into the broken placeholder. On a broken current database (switch to it first with USE DATABASE <name> in Enterprise multi-tenant deployments; in Community the default memgraph database is recovered directly):

RECOVER SNAPSHOT "/path/to/good.snapshot";

On success, Memgraph clears the broken flag and the database resumes normal operation and background durability. The existing RECOVER SNAPSHOT behavior moves all prior/corrupt snapshots and WAL files to the .old directory (or deletes them when --storage-enable-backup-dir is off), leaving a clean single-snapshot directory. As a result, the database recovers cleanly on the next restart and does not re-enter the broken state.

RECOVER SNAPSHOT is available in both Community and Enterprise editions, so single-tenant deployments can recover a corrupt default database.

Restore the whole data directory from a backup

If you do not have a usable snapshot to load with RECOVER SNAPSHOT, replace the instance’s data directory with a backup copy of the whole data directory and restart the process. Because the broken placeholder never wrote to the corrupt files, the original durability files are still intact if you prefer to recover them out-of-band before restarting.

Replica self-heal (High Availability)

In a clustered (HA) deployment, a broken replica database heals automatically. Because the broken placeholder comes up empty with a fresh epoch, the main detects that the replica is not caught up, drives it into recovery, and sends a full snapshot (plus WAL). Loading that snapshot clears the broken state, after which the database resumes normal replication and serves reads. No operator intervention is required on replicas, and a broken database on one instance does not disrupt replication of the other healthy databases.

Broken main and failover (High Availability)

A main that is broken for one database but otherwise healthy remains main — the cluster does not automatically fail over based on per-database broken state, so it never thrashes over a single corrupt tenant. Meanwhile, replicas keep serving reads of their healthy copy of that database, so read availability survives main-side corruption.

Recovery of a broken main is operator-driven, and you have two options:

• Cure it in place with RECOVER SNAPSHOT (as above), after which the recovered database participates in replication normally and the cluster returns to a fully healthy state.
• Fail over to a healthy replica. If a replica holds a good copy of the broken database, promote it with a manual failover — DEMOTE INSTANCE the old main, then SET INSTANCE ... TO MAIN on the replica. The demoted old main rejoins as a replica, and its broken database then self-heals from the new main through the replica self-heal path described above.

Drop and recreate the database

If you want to abandon a broken database rather than recover its data, you can DROP DATABASE it and create it again:

DROP DATABASE mydb;
CREATE DATABASE mydb;

DROP DATABASE works on a broken database like any other and also removes its corrupt on-disk files, reclaiming the disk. It does not affect other databases.

Storage modes

Memgraph has the option to work in IN_MEMORY_ANALYTICAL, IN_MEMORY_TRANSACTIONAL or ON_DISK_TRANSACTIONAL storage modes.

Memgraph always starts in the IN_MEMORY_TRANSACTIONAL mode in which it creates periodic snapshots and write-ahead logging as durability mechanisms, and also enables creating manual snapshots.

In the IN_MEMORY_ANALYTICAL mode, Memgraph offers no periodic snapshots and write-ahead logging. Users can create a snapshot with the CREATE SNAPSHOT; Cypher query. During the process of snapshot creation, other transactions will be prevented from starting until the snapshot creation is completed.

In the ON_DISK_TRANSACTIONAL mode, durability is supported by RocksDB since it keeps its own WAL files. Memgraph persists the metadata used in the implementation of the on-disk storage.