How to manage large data in java web services

Question

I have a problem my java web service. How can i manage big data (more than 10 GB). I have to send to consumer this data. Please give me some advise.

Answer 1

It depends on many things :

1. Response time required
2. Is data computation heavy or are you generating data
3. Where is data stored

Most probably your data will be stored somewhere and I assume you have map-reduced it. You can provide this data to your clients using pagination.

Answer 2

-If it is really not a necessity to serve all the data at one go,I would suggest creating filter based API's .

-Even batch processing this data serving let's say 50 records at a time with pagination at the front end will help you with the response time.

-You can use compression techniques like gzip compression on the webserver