# SQL2Graph

SQL2Graph migrates a relational database, MySQL or PostgreSQL, to Memgraph.
It reads the relational schema, proposes a graph model, lets you review and
change it, migrates the data and validates the result.

SQL2Graph can design the graph model in two ways:

- **Deterministic strategy:** Maps tables to nodes and foreign keys to
  relationships by rules.
- **LLM strategy:** An LLM reads the schema and proposes a model with
  meaningful labels and relationship types.

It runs in one of two modes:

- **Automatic mode:** Migrates without asking you anything.
- **Incremental mode:** Shows the graph model step by step, so you can review
  and change it before the migration.

You choose the strategy and the mode with command-line options or environment
variables. SQL2Graph validates the data before and after the migration and
reports the result.

## Supported databases

| Type | Supported options |
|------|-------------------|
| **Source databases** | PostgreSQL, MySQL |
| **Target database** | Memgraph |

## Prerequisites

You need a running PostgreSQL or MySQL database to migrate, and Memgraph
started with schema tracking enabled. To install Memgraph, see [Install
Memgraph](https://memgraph.com/docs/getting-started/install-memgraph). To start it in Docker with
schema tracking:

```bash
docker run -p 7687:7687 memgraph/memgraph --schema-info-enabled=true
```

> **Warning**
>
> Start Memgraph with `--schema-info-enabled=true`. Without it, SQL2Graph can't
> read the graph schema to validate the migration.

## Installation

In order to use you first need to clone the repository and install the
dependencies:

```bash
# Clone the repository
git clone https://github.com/memgraph/ai-toolkit

# Navigate to the sql2graph directory
cd agents/sql2graph

# Install dependencies using uv
uv pip install -e .
```

## Configuration

The configuration enables you to control the agent flow via environment
variables. The key information needed are the source database connection
details, target Memgraph connection details, and LLM API keys and agent
configuration. 

Create a `.env` and fill the following variables:

```bash
# Source Database
SOURCE_DB_TYPE=postgresql  # or mysql

# PostgreSQL Configuration
POSTGRES_HOST=localhost
POSTGRES_PORT=5432
POSTGRES_DATABASE=mydb
POSTGRES_USER=username
POSTGRES_PASSWORD=password
POSTGRES_SCHEMA=public

# MySQL Configuration (if using MySQL)
MYSQL_HOST=localhost
MYSQL_PORT=3306
MYSQL_DATABASE=mydb
MYSQL_USER=username
MYSQL_PASSWORD=password

# Target Memgraph Database
MEMGRAPH_URL=bolt://localhost:7687
MEMGRAPH_USERNAME=
MEMGRAPH_PASSWORD=
MEMGRAPH_DATABASE=memgraph

# LLM API Keys (for AI-powered features)
# Only provide the key for your chosen provider
OPENAI_API_KEY=sk-...          # For GPT models
# ANTHROPIC_API_KEY=sk-ant-...  # For Claude models
# GOOGLE_API_KEY=AI...          # For Gemini models

# Optional: Specify LLM model (defaults shown)
# LLM_MODEL=gpt-4o-mini         # OpenAI default
# LLM_MODEL=claude-3-5-sonnet-20241022  # Anthropic default
# LLM_MODEL=gemini-2.0-flash-exp        # Google default

# Migration Defaults (can be overridden via CLI flags)
SQL2MG_MODE=automatic # Options: automatic, incremental
SQL2MG_STRATEGY=deterministic # Options: deterministic, llm
SQL2MG_META_POLICY=auto # Options: auto, reset, skip
SQL2MG_LOG_LEVEL=INFO

```
To start, copy `.env.example` in `agents/sql2graph` to `.env`.

## Run a migration

You can run SQL2Graph in four ways, from fully automatic to fully reviewed.

### Automatic migration

Run with default settings (automatic mode, deterministic strategy):

```bash
uv run main.py
```

The agent will:
1. Validate your environment and database connections
2. Analyze the source database schema
3. Generate a complete graph model
4. Execute the migration
5. Validate the results

In **automatic mode**, no user interaction is required, and the entire process
is automated. This means the `SQL2MG_MODE` is set to `automatic`, and the
SQL2MG_STRATEGY is set to `deterministic`. `SQL2MG_MODE` refers **modeling
mode**  and represents how much user interaction is involved, while
`SQL2MG_STRATEGY` refers to how the graph model is generated.

### Refinement with incremental mode

For more control, run in incremental mode to review and refine the model
step-by-step:

```bash
uv run main.py --mode incremental
```

The agent will:
1. Analyze the source database schema
2. Generate an initial graph model
3. Present each table's proposed transformation for review
4. Allow you to accept, skip, or modify each table's mapping
5. After reviewing all tables, optionally enter a refinement loop for final
   adjustments
6. Execute the migration
7. Validate the results

This is predictable and repeatable flow so you can iteratively improve the graph
model before migration. Each table is processed one at a time, and you have full
control over the transformations, the table will show you all the proposed
nodes, relationships, and properties for that table, and you can choose to
accept them as-is, skip the table entirely, or modify the mapping details.

### Interactive migration with LLM

Use LLM-powered modeling for AI driven design:

```bash
uv run main.py --strategy llm
```

The agent auto-detects which LLM provider to use based on available API keys. In
this strategy, the agent will:
1. Analyze your SQL schema semantically using LLM
2. Generate an initial graph model with AI-optimized design
3. Execute the migration
4. Validate the results

Keep in mind that in this mode, the entire migration is still automatic and LLM
driven. 

### Incremental migration with review

Control each step of the transformation:

```bash
uv run main.py --mode incremental --strategy llm
```

In incremental mode:
1. The AI generates a complete graph model for all tables
2. You review each table's mapping one at a time
3. Accept or modify individual table transformations
4. After processing all tables, optionally enter a refinement loop
5. Interactively adjust the entire model before final migration

In this mode the LLM is used to generate the initial model, but you have full
control to review and refine each table's mapping before migration. After each
modification, the LLM will try to regenerate based on your feedback and
validation errors to improve the model iteratively.

## CLI reference

### Command-line options

| Flag | Environment Variable | Description | Default |
|------|---------------------|-------------|---------|
| `--mode` | `SQL2MG_MODE` | `automatic` or `incremental` | interactive prompt |
| `--strategy` | `SQL2MG_STRATEGY` | `deterministic` or `llm` | interactive prompt |
| `--provider` | _(none)_ | `openai`, `anthropic`, or `gemini` | auto-detect from API keys |
| `--model` | `LLM_MODEL` | Specific model name | provider default |
| `--meta-graph` | `SQL2MG_META_POLICY` | `auto`, `skip`, or `reset` | `auto` |
| `--log-level` | `SQL2MG_LOG_LEVEL` | `DEBUG`, `INFO`, `WARNING`, `ERROR` | `INFO` |

### Usage examples

```bash
# Use specific Gemini model
uv run main.py --strategy llm --provider gemini --model gemini-2.0-flash-exp

# Skip meta-graph comparison (treat as fresh migration)
uv run main.py --meta-graph skip

# Enable debug logging
uv run main.py --log-level DEBUG

# Fully configured non-interactive run
uv run main.py \
  --mode automatic \
  --strategy deterministic \
  --meta-graph reset \
  --log-level INFO
```

## LLM provider support

| Provider | Models |
|----------|--------|
| **OpenAI** | GPT-4o, GPT-4o-mini |
| **Anthropic** | Claude 3.5 Sonnet |
| **Google** | Gemini 2.0 Flash |

### Provider selection

The agent automatically selects a provider based on available API keys:
1. Checks for `OPENAI_API_KEY`
2. Falls back to `ANTHROPIC_API_KEY`
3. Falls back to `GOOGLE_API_KEY`

Override with `--provider` flag:

```bash
# Force Anthropic even if OpenAI key exists
uv run main.py --strategy llm --provider anthropic
```

### Model selection

Each provider has sensible defaults:
- **OpenAI**: `gpt-4o-mini` 
- **Anthropic**: `claude-3-5-sonnet-20241022`
- **Google**: `gemini-2.0-flash-exp` 

Override with `--model` or `LLM_MODEL` env variable:

```bash
# Use more powerful OpenAI model
uv run main.py --strategy llm --model gpt-4o

# Or via environment variable
export LLM_MODEL=claude-3-opus-20240229
uv run main.py --strategy llm --provider anthropic
```

## Architecture overview

If you like the implementation details, here is a high-level overview of the
project structure:

```plaintext
sql2graph/
├── main.py                    # CLI entry point
├── core/
│   ├── migration_agent.py     # Main orchestration
│   └── hygm/                  # Graph modeling engine
│       ├── hygm.py            # HyGM core
│       ├── models/            # Data models
│       ├── strategies/        # Modeling strategies
│       └── validation/        # Validation system
├── database/
│   ├── analyzer.py            # Schema analysis
│   ├── factory.py             # Database adapter factory
│   └── adapters/              # DB-specific adapters
├── query_generation/
│   ├── cypher_generator.py    # Cypher query builder
│   └── schema_utilities.py   # Schema helpers
└── utils/
    ├── environment.py         # Env validation
    └── config.py              # Configuration
```