I'm implementing the circuit breaker design pattern for a large scale distributed system (on AWS).
Netflix gives a great description here: http://techblog.netflix.com/2011/12/making-netflix-api-more-resilient.html
I need some advice for the backend monitoring database implementation. Basically, I need a database/daemon/service that can do the following:
- Store thousands of writes per second.
- Quickly sum thousands of records (last few seconds) or otherwise maintain a rolling average calculation of statuses, etc (preferably with some caching).
- Store two weeks worth of data or the results of the calculations in step 2.
Is there an existing database or other software that's good for this task? I doubt Netflix will give me theirs ;)
They use Cassandra at the backend. Check this article about Netflix architecture...