Resiliency against DB unavailability in SPRING-BOOT + MongoDB

Question

I have a micro-service that captures callbacks from the payment gateway once the user completes the payment and stores it in the Database. Currently, if there is an issue with the database (DB down, not enough available connection ..) the method fails and the service is unable to capture the details of the payment. What are possible approaches that I may take to improve the resiliency to the service in such scenario. One possible solution might be to store the data in a queue and attempt to load the data back from the queue once the DB is back up. However, it would be helpful to have few other alternative approaches.

Answer 1

Making your microservice strong in dealing with database issues are crucial for the reliability on any payment system. we can follow below to enhance the same :

Retry System: Create a retry system for database tasks. If a task fails due to a temporary problem, like a short database issue, set your microservice to try again after a pause. Use a set timeout approach to avoid overloading the database when it's back online.

Circuit Breaker Technique: Use the Circuit Breaker approach in your microservice. It can pause database operations temporarily if it detects repeated failures. This prevents your microservice from continuously trying to write to the database during a long issue, giving the database time to recover.

Async Processing: Instead of directly adding payment details to the database, handle payment processing asynchronously. Save payment details in a message queue and then process them separately. This way, your microservice can acknowledge payments quickly without being stuck by potential database problems.

Connection Pooling: Ensure your microservice uses a database connection pool. This helps manage and reuse connections efficiently, avoiding problems with running out of available connections. Many modern database tools support connection pooling, I guess MongoDB but I am not sure.

Graceful Downgrading: Plan for a graceful way to handle issues. If the database is having problems, let your microservice offer a limited set of features, like logging payment details to a file or another storage system, without affecting the user experience.

Fallback Plan: Have a backup plan for critical operations. If the database is unreachable, temporarily store payment information in another storage and retry writing to the database when it's back.

Health Checks and Monitoring: Regularly check the health of your microservice and the database. Set up monitoring to catch problems early, so you can fix them before a major failure.

Database Copying: Think about setting up database replication with a standby or read-only copy. If the main database is down, your microservice can still read from the replica, ensuring at least read operations can continue.

Backup Systems: If possible, consider having backup databases in different places. Your microservice can switch to a secondary database if the main one is not available.

Rollback Strategy: Set up a rollback mechanism that can undo partially completed tasks if an issue happens during payment processing. This keeps things consistent and maintains data integrity.

You can finalise the strategies that fits your needs to build a strong and dependable microservice for payment processing.