Optimizing Real-Time Payment Authorization with Memgraph
The webinar featuring Martin Vo, CTO of Paysure Solutions, focuses on real-time payment authorization and the company's journey in implementing Memgraph as a graph database. Paysure Solutions uses Memgraph to handle complex payment transactions efficiently. Martin discussed the operational challenges Paysure faced and how Memgraph provided solutions that significantly enhanced their authorization services.
Go and watch the full webinar recording–Real-Time Payments Authorization with Memgraph.
In the meantime, here are the key talking points from the webinar.
Talking Point 1: Introduction Paysure Solutions
The webinar starts with their background and expertise in the insurance domain, For example, for payment authorizations by determining the approvability of transactions based on domain-specific data, like insurance policies.
Talking Point 2: Payment Authorization Flow
Martin described the typical payment process, emphasizing the role of Paysure in the chain, where they receive data from payment networks and decide on transaction approval based on several factors. This is done under a time constraint of 200 milliseconds.
Talking Point 3: Challenges With Previous Architectures
Paysure’s initial system used a combination of PostgreSQL and Redis, which posed challenges with data locking and synchronization. This led to a realization that a graph database could manage their needs more effectively.
Talking Point 4: Transition to Memgraph
After experiencing issues with another graph database provider, Paysure switched to Memgraph due to its stability and speed. Memgraph's capability to handle complex queries efficiently and maintain atomicity was necessary for their real-time needs.
Talking point 5: Technical implementation and Optimization
Martin detailed their technical challenges, such as handling complex queries, managing database connections, and optimizing performance through indexing and caching strategies.
Talking Point 6: Operational Challenges and Solutions
The team encountered and overcame numerous obstacles, including handling Memgraph updates during peak times around holidays and optimizing system performance that degraded due to authentication methods.
Talking Point 7: Performance Results
With Memgraph, Paysure managed to simplify its system architecture, eliminating the need for separate data fetching, locking, and updating services. The performance enhancements allowed them to handle transactions within tens of milliseconds.
Q&A
We’ve compiled the questions and answers from the webinar's Q&A session. Note that for brevity, we’ve paraphrased them slightly. For complete details, please refer to the full webinar recording.
1. Could you explain in more detail why you chose not to continue using Redis? You mentioned replacing it with Memgraph because Redis is a key-value store and lacks a graph solution. Can you elaborate on why a graph database was a better option for you?
-
Answer: Yes, Redis doesn't have an out-of-the-box graph solution, although there was RedisGraph, which isn't supported anymore. In Redis, designing our graph database would have required numerous requests to fetch data and further requests to get additional node IDs, leading to potential issues and race conditions. While Redis is excellent as a key-value store and quite fast, maintaining a custom solution built around it would have been too complicated. Also, Redis doesn't provide guarantees about data structure, which we needed. With Memgraph, we can implement constraints where necessary, helping us balance between a restricted schema and the flexibility of a graph database.
2. Apart from writing your procedures, what kind of graph algorithms do you utilize, particularly in analyzing transaction data?
-
Answer: We don't use complex graph algorithms during the payment authorization process as it needs to be quick, and we must know immediately why a payment failed. Implementing AI might be possible for fraud protection, but it’s more about quick data analysis rather than running complex graph algorithms. The most advanced thing we've done is a wildcard query to navigate from one node to all connected nodes, which might not strictly qualify as a graph algorithm.
3. How did you manage to avoid duplicating payments?
-
Answer: There are two main scenarios for payment duplication. First, if the same payment request comes from the network with identical identifiers, we have constraints to prevent storing such duplicates. Secondly, two similar-looking payments might not be identical due to different identifiers, possibly indicating fraud. For these cases, we have rules to detect potential issues, but it's crucial to align with client expectations, as what might be considered fraud in one scenario could be legitimate in another.
4. Have you considered clearing older transactions from the graph database over time?
-
Answer: Currently, we haven't needed to remove old transactions because there hasn't been a noticeable impact on performance, even with about 10,000 nodes in the database. However, if performance issues arise, we might consider aggregating historical payment data into a single record, which would be one way to manage database growth.
5. Did you incorporate a temporal dimension in modeling your graph, specifically regarding the timing of payments?
-
Answer: Yes, we include timestamps from the payment network directly on the payment nodes using Memgraph’s datetime capabilities. Initially, we faced some issues when comparing dates as strings, which could disrupt payment processing. Moving forward, using constraints and triggers helps ensure accurate datetime entries.
6. How did you handle data replication with Memgraph? Did you use a single server?
- Answer: We currently do not use replication. Our main challenge was optimizing the authorization query, which involves both reading and writing data, making read replicas less useful. High availability and data replication in a Kubernetes environment poses additional challenges, but we may explore these areas in the future.
7. Why did you choose a graph database when you are not using graph algorithms to solve problems that a relational database could handle?
- Answer: We initially tried using PostgreSQL and found it inadequate for handling self-referencing allowances during payment authorization, which are much easier to manage with a graph database. The graph database allows us to seamlessly link and check all related allowances and other conditions in real-time, which would be cumbersome with traditional SQL joins.
8. What general tips would you give regarding indexing in Memgraph?
-
Answer: I recommend running your query and then reviewing the suggestions provided by Memgraph, which can significantly help optimize performance. Previously, we had to manually profile our queries and check for inefficiencies, like something being off with Cartesian products, but Memgraph’s updates have simplified this process by providing helpful indexing suggestions.
Pro Tips
Some best practices from our Memgraph DX team:
- Use temporal data if possible because it uses less memory than string.
- Pro-tip on data replication and high availability - Within the Memgraph Community edition, you can replicate the data and create a cluster, but regarding high availability, you have to manage it yourself. However, we have high availability for the Memgraph Enterprise version.
Memgraph DX team is active within the Memgraph Discord community, so feel free to reach out or book a call if you have any questions.
Further Reading
- For more tips and tricks, you can refer to the Memgraph querying best practices on our docs.
- Memgraph DX team is active within the Memgraph Discord community, so feel free to reach out if you have any questions.