CUSTOMER STORY
Real-time Data Processing for Relationship Mapping: Network Risk Analysis
Customer
Sayari
Use case
Criminal Risk Detection
Industry
Risk Intelligence
Profile
Building with Memgraph
15x
reduction in query time-outs
16k+
queries executed with 99.8% completed in less than 10s
1.5k+
community members
Challenge
Sayari needed a graph database capable of addressing complex data processing for detailed relationship mapping. They also needed to ensure that the graph database could handle real-time, high-performance demands crucial for timely threat detection and response.
Solution
Sayari's switch to Memgraph significantly improved their data management capabilities, and they now offer high-performance real-time analysis and enhanced reliability by reducing query timeouts by 15x. These advantages come from Memgraph's in-memory processing and optimized storage, crucial for Sayari's cybersecurity efforts and operational efficiency.
Reading time: 5min
About Sayari
Sayari is in the industry of counterparty and supply chain risk intelligence. It provides global visibility into the relationships between businesses and individuals, helping to uncover risks within corporate and trade networks. Sayari offers a platform for users to access a large database of public records and financial intelligence.
Impact highlights
Real-time insights and processing speed
With Memgraph, Sayari can process real-time data and offer instant intelligence on global corporate risks for their end users. This capability leveled up Sayari's competitive edge in the risk intelligence sector by providing swift and reliable information.
Cost efficiency
Switching from TigerGraph to Memgraph, Sayari significantly reduced their total cost of ownership. They navigated away from a licensing deadlock that would have seen costs surge by 20x with TigerGraph
End-user satisfaction
The improvements in data freshness, query performance, and the overall product experience led to increased customer satisfaction and loyalty among the Sayari client base.
“With 40 million new entities and 50 million new connections being added into Sayari platforms each month, no other graph database is able to handle the performance needs like Memgraph does”
William Hurley, Director of Infrastructure, Sayari
Key Memgraph Features for Sayari
- In-memory data storageIn Memgraph’s in-memory graph storage, data is stored in main memory (RAM) allowing for much faster access and analysis. High speed data access means high-speed response times - an essential requirement for Sayari.
- Custom query modulesMemgraph allows the use of custom query modules to extend the Cypher query language and expand beyond the pre-developed algorithms and utility procedures available in MAGE (Memgraph Algorithm Graph Extensions) by modifying existing and/or writing new procedures in C, C++, Python, and Rust. Using Memgraph, Sayari developed custom query modules to implement specialized logic for entity resolution and relationship analysis.
- Live streaming data ingestionMemgraph’s ability to ingest streaming data enables Sayari to continuously update their graph database with new transactions and entity relationships.
Backstory
Sayari pulls together data that covers years of litigations, property transactions, intellectual property filings, regulatory disclosures, commercial and trade activity, vital records registrations, corporate formations and changes and much more from federal, state and local public records in high-risk markets and uses that to illuminate commercial and supplier networks, expose hidden risks and provide the necessary context needed to make informed decisions.
“Most of the other vendors we tested are heavily reliant on disks and hold indices on the disk. Even with the fastest disks on GCP, it was taking days to write. Keeping data in memory was really crucial for those performance characteristics.”
William Hurley, Director of Infrastructure, Sayari
With a database encompassing over 5 billion records, 640 million companies, and 614 million key individuals, Sayari supports a wide range of users—from financial crime analysts to defense and intelligence agencies.
This wide-variety data repository enables end users, such as investigators, regulators, or financial institutions, to gain a deep understanding of the activities of individuals or companies. By analyzing this information, users can make informed decisions about their next steps. Also, this dataset allows users to trace the connections between entities, identifying not just direct relationships but also the extent of separation between suspicious entities and individuals, including all intermediary connections.
The Sayari Graph is a multi-tenant cloud application with an intuitive user interface, scalable API feeds, and offers on-premise deployment models. Currently, Sayari Graph has over 1 billion nodes and 1.3 billion connections and is constantly growing.
Challenge:
Quickly processing large amounts of data coming in and rebuilding the database. All while trying to avoid performance issues.
Sayari maps out relationships between entities by continually collecting diverse data. Such a dataset, with frequent updates, has 20 million new entities and 50 million relationships every two weeks. Integrating new data through entity resolution —verifying and linking records across various sources— is important for keeping the database reliable and up-to-date at all times.
The crux of the matter was speed. Traditional vendors bogged down by disk-based storage and indexing couldn't meet Sayari's demands. They took days to update, causing delays and outdated information.
In addition, Sayari was running into performance issues during data queries and the lengthy duration required to rebuild the graph database from scratch with updated information. This impacted user experience and posed business risks. There was also the challenge of a staggering 20x price increase from their previous graph database vendor.
Why Memgraph?
Despite evaluating other options, only Memgraph stood out for its exceptional speed and efficiency, particularly in-memory data handling and custom query module capabilities, which were unmatched by competitors
"We use Memgraph to rebuild a 1.4B-node graph every 2 weeks from scratch and query it live making our data easily accessible and current for users," said Bill Hurley.
Here’s why Sayari chose Memgraph:
- High-performance real-time analysis and in-memory data processing for a more advanced and faster query execution.
- Memgraph enables real-time ETL (Extract, Transform, Load) processes with minimal synchronization time.
- Faster processing means there’s no need for data batching and validation during low demand hours.
- Graph storage model with high multi-hop traversal capabilities
- Sayari allows querying up to 10th level of depth, but anything beyond that first hop has to go through the graph database. Memgraph offers efficient exploration of complex relationships within large datasets and fast execution of deep queries across extensive graphs.
- For Sayari, the multi-hop traversal capabilities provide an edge over competitors who process data in less depth.
- Custom Query Modules and Memgraph Algorithm Graph Extensions (MAGE)
- Memgraph’s MAGE modules met Sayari's requirements, but the flexibility to create custom query modules for entity resolution and relationship analysis was particularly valuable, offering capabilities not available with other vendors.
- Easy to get started and use
- Getting started with Memgrah is straightforward with a Docker container setup, up-to-date documentation, and support for Cypher query language.
- Community edition
- Memgraph's community edition allows users to evaluate the product using more than 100 nodes and test extensively prior to going into production.
Results
Before switching to Memgraph, Sayari faced challenges with delays and the risk of timeouts, which negatively affected user interactions. However, the transition to Memgraph transformed their service—offering quick and reliable responses with virtually no timeouts. This was a win for Sayari in terms of performance and reliability.
Moreover, using sophisticated algorithms like Breadth-First Search (BFS) and other search techniques, Sayari has been able to tackle more complex queries and analyses, further enhancing their service offering.
Moreover, using sophisticated algorithms like Breadth-First Search (BFS) and other search techniques, Sayari has been able to tackle more complex queries and analyses, further enhancing their service offering.
By implementing Memgraph, Sayari significantly enhanced their user experience, evidenced by their ability to execute over 16,000 queries while keeping more than 99.8% of them under 10 seconds in response time. This improvement led to a faster and more efficient graph exploration experience for users, enabling Sayari to expand their product's capabilities effectively.
The improvements in data freshness, query performance, and the overall product experience led to increased customer satisfaction and loyalty among the Sayari client base.
“The Community edition isn't Crippleware—it's not about being restricted to, say, 100 nodes and then having to get in touch with the sales team. Rather, we were actually able to kick the tires and verify it satisfied our needs before there was any need to contact your team. Which is great!”
William Hurley, Director of Infrastructure, Sayari
Find out how Memgraph performs compared to Neo4j
Let’s see how Memgraph fits into your environment
