How To Build Agentic GraphRAG?

By Sara Tilly

8 min readFebruary 18, 2025

Last week, Memgraph hosted a community call to discuss an exciting topic in AI—Agentic GraphRAG. If you missed it, here’s a deep dive into the webinar highlights presented by Ante Javor, our Developer Experience Engineer. The discussion covered everything from the fundamentals of GraphRAG to Memgraph’s new capabilities in version 3.0 and why agents are crucial for advancing GraphRAG. Let’s break it down.

Watch the full webinar recording on our YouTube channel:

GraphRAG Is Great—But What If It Had a Brain?

To understand Agentic GraphRAG, you first need to know the basics of RAG (Retrieval-Augmented Generation) and its graph-based cousin, GraphRAG.

RAG turned the tables when it came to overcoming the limitations of large language models (LLMs). Fine-tuning an LLM is expensive, slow, and often impractical unless you have an army of GPUs and a deep-pocketed budget. Your other solution is RAG. Instead of fine-tuning, RAG dynamically feeds relevant, real-time context into an LLM to generate personalized answers. It's faster, more scalable, and infinitely more flexible.

But RAG has limitations. It depends on the quality of the context provided. If you stuff too much irrelevant data into the prompt or fail to curate the information, your LLM will return subpar results. You must also to manage limited context windows and the inherent constraints of LLMs acting as "text predictors" rather than true reasoning systems.

And then you have GraphRAG, as the name suggests, takes it up a notch by using a knowledge graph as its foundation. Developed initially by Microsoft, GraphRAG structures data into nodes and relationships, delivering richer and more precise insights. It’s perfect for situations where data complexity and relationships matter, such as healthcare, finance, and beyond.

Why Make GraphRAG Agentic?

While GraphRAG is great, it has one flaw: rigidity. Traditional GraphRAG systems hard-code pipelines for data retrieval. This is great for simple use cases but limiting for complex, evolving scenarios. It’s where you’d use the agentic aspect.

Agents, as Ante explains, are simply decision-making systems that use LLMs to interact with external data sources. Think of them as orchestrators who dynamically interact with your data and decide the best retrieval strategy for each query.

At its core, an agent is simply an LLM call with a specific context. It takes your input and reasons through what it knows (or doesn’t know), and it interacts with external tools or databases (like Memgraph) to retrieve the most relevant answers. In this case, agents don’t just retrieve data—they figure out how to retrieve it in the most effective way possible.

For example, instead of always relying on vector search, an agent might use a graph traversal algorithm like BFS or PageRank to find the most relevant nodes in a knowledge graph. This dynamic decision-making process unlocks new flexibility, scalability, and intelligence levels in AI-driven systems.

Here’s how Agentic GraphRAG improves knowledge retrieval:

Flexibility: Agents aren’t tied to one approach—they select the best tool or algorithm for the task.
Autonomy: Once set up, the system operates independently, requiring minimal human intervention. Just you keeping an eye on it.
Error handling: Feedback loops allow agents to dynamically retry, diagnose issues, and recover from failures.

Agentic GraphRAG is seen as a state machine where agents chain decisions and actions together, sharing context and results at every step. This architecture enables a smarter, more responsive system.

Memgraph 3.0 and Agentic GraphRAG

The recent release of Memgraph 3.0 was a big deal for us—and for good reason. With features like vector search, dynamic algorithms, and high availability, building production-ready AI applications’s easier than ever.

What makes these features so pivotal for GraphRAG systems?

Vector search: You can now search for data using embeddings. For instance, searching for similar nodes in a graph—like finding a patient named Mark and traversing one or two hops to discover associated symptoms—is seamless with Memgraph’s vector capabilities.
Dynamic algorithms: No more rigid pipelines. Memgraph lets you use algorithms like BFS, DFS, PageRank, or community detection to extract relevant data based on your use case.
High availability: Building production systems demands reliability. Memgraph 3.0 ensures your GraphRAG-based applications can scale without hiccups.

How Agentic GraphRAG Works: The Core Workflow

Building an Agentic GraphRAG system involves creating a pipeline of agents, each responsible for a specific step in the process. Here’s a typical workflow, which you’ll also see in the demo.

Question classification

The first agent analyzes the user query and determines its type. For example:
- Retrieval questions involve direct lookups, such as “What is John’s age?”
- Structured questions explore relationships, like “Is Coca-Cola connected to stock market trends?”
- Global queries analyze trends such as “What are the top 10 most important nodes?”
- Database questions address system configurations, such as “What logging level is Memgraph using?”
Tool selection

Based on the query type, another agent selects the best tool to retrieve the required data. This might include vector search, PageRank, BFS (breadth-first search), DFS (depth-first search), or a database schema query.
Execution

The selected tool processes the query, retrieving relevant data or results. Agents can adapt tools to match the query’s specific needs. For example, a PageRank query may return the top 10 nodes for a broad query or a single node for a specific question.
Feedback loops

If a tool fails (e.g., due to a misconfigured query or missing data), the agent diagnoses the issue, retries, or switches to an alternative tool. These feedback loops ensure robust error handling.
Response

The processed data is returned to the user as a meaningful answer.

Building Agentic GraphRAG with Memgraph

Thanks to its advanced graph database capabilities, Memgraph is uniquely suited to power Agentic GraphRAG. With Memgraph, you can create a flexible system that adapts to various queries and retrieval tasks. The demo Ante presented in the Memgraph community call showcases how to build Agentic GraphRAG with Memgraph. It’s not just a proof of concept—it’s a working model you can replicate and extend.

Watch the full webinar recording—How To Build Agentic GraphRAG?

Here’s what you need to know about the demo:

It’s dataset agnostic.
The demo doesn’t rely on external tools like LangChain; instead, it focuses on raw OpenAI API calls for faster iteration.
Every step is automated, from question classification to tool selection. Based on the question type, agents choose tools like PageRank, BFS, or schema queries.
We’ve implemented error recovery so feedback loops ensure the handling of failures.
All features used in the demo are within Memgraph, including vector search for similarity-based queires, text-to-cypher translation for natural language queries, schema info for database structure exploration, graph traversal algorithms (BFS, DFS), and PageRank.

DEMO RESOURCES

Ante’s presentation: Agentic GraphRAG PDF
Demo: Memgraph AI Demos
Useful Memgraph AI resources: Memgraph AI Demos

The Future of Agentic GraphRAG

We conclude the webinar with some thoughts and perspectives on the the vision for Agentic GraphRAG.

Right now, the demo does a solid job of selecting the right tool based on the type of question being asked. But there’s plenty of room to make it even smarter. One of the first things on my mind is expanding the toolset. While it already supports vector search, PageRank, BFS/DFS, and text-to-Cypher, we could throw a lot more into the mix. Geo-search would be a great addition, allowing for spatial queries, and integrating more advanced text-based queries could help make the system more flexible for unstructured data.

Another big leap would be fully automatic tool selection. Currently, the tool selection process is somewhat constrained by predefined mappings between question types and tools. But what if we didn’t have to predefine anything? What if the agent itself could figure out which tool to use?

And then there’s the big idea: a universal knowledge retrieval agent. Imagine an agent capable of pulling knowledge from any graph, seamlessly switching between retrieval methods, graph algorithms, and even external data sources without manual intervention. A system like that wouldn’t just answer questions—it would reason through them, dynamically navigating graphs, selecting the best tools, and structuring responses in a way that makes sense given the context. Instead of thinking about "which tool should I use," we’d have an agent that just knows what to do.

The goal isn’t just to build something that works—it’s to continuously iterate, improve, and push the boundaries of what’s possible with Agentic GraphRAG. And with Memgraph 3.0, you have the perfect platform to build your own Agentic GraphRAG. Whether you’re working in healthcare, finance, or any data-driven field, the flexibility and power of this system will unlock new possibilities.

Memgraph Academy

If you are new to the GraphRAG scene, check out a few short and easy-to-follow lessons from our subject matter experts. For free. Start with: