periodic
The periodic module enables users to execute a query periodically in batches. In this case, the name periodic doesn't indicate that the query is executed after a time interval, but rather that, due to the complexity of the query, the results of some input source are batched to speed up execution.
⚠️
As the results are batched and executed in different transactions, every executed batch is committed by itself. If an issue occurs while running this procedure, the already committed batches cannot be rolled back.
Trait | Value |
---|---|
Module type | module |
Implementation | C++ |
Parallelism | sequential |
Procedures
iterate(input_query, running_query, params)
Input:
input_query: string
➡ the input query which will yield the results that need to be batchedrunning_query: string
➡ query which will be executed on the batched resultsparams: Map[string, string]
➡ parameters for the procedurebatch_size: Integer
➡ key specifying how many results should a batch contain
Output:
success: boolean
➡true
if the procedure executed successfully,false
otherwisenumber_of_executed_batches: Integer
➡ number of executed batches (possibly a fraction of the full number if the procedure returnedsuccess: false
)
Usage:
CALL periodic.iterate(
"LOAD CSV FROM '/tmp/file.csv' WITH HEADER AS row RETURN row.node_id AS node_id, row.supernode_id AS supernode_id",
"MATCH (s:SuperNode {supernode_id: supernode_id}), (n:Node {node_id: node_id}) CREATE (s)-[:HAS_REL_TO]->(n)",
{batch_size: 5000})
YIELD * RETURN *;
Example
Cypher load commands
CREATE INDEX ON :SuperNode;
CREATE INDEX ON :SuperNode(supernode_id);
CREATE INDEX ON :Node;
CREATE INDEX ON :Node(node_id);
CREATE (:SuperNode {supernode_id: 1});
FOREACH (i IN range(1, 1000000) | CREATE (:Node {id: i}));
CSV file present
The CSV file should contain the following data:
supernode_id,node_id
1,1
1,2
1,3
1,4
1,5
1,6
...
1,999998
1,999999
1,1000000
Running command
CALL periodic.iterate(
"LOAD CSV FROM '/tmp/file.csv' WITH HEADER AS row RETURN row.node_id AS node_id, row.supernode_id AS supernode_id",
"MATCH (s:SuperNode {supernode_id: supernode_id}), (n:Node {node_id: node_id}) CREATE (s)-[:HAS_REL_TO]->(n)",
{batch_size: 5000})
YIELD * RETURN *;
Results
+------------------+----------------------------+
| success | number_of_executed_batches |
+------------------+----------------------------+
| true | 200 |
+------------------+----------------------------+