morphia mongodb retreive huge data batchwise

660 Views Asked by At

I want to retrieve 200 000 data from db from mongodb using morphia.

Earlier I was using

query.asList()

which gave me out of memory exception I tried changing it to

query.batchSize(50).asList()

but with no luck.

I update my code to

Iterable<DataStreams> iterable= query.fetch();
            Iterator<DataStreams> iterator=iterable.iterator();
            while (iterator.hasNext()) {
               dataStreamsList.add(iterator.next());                
            }           
           System.out.println("iteration done");
1

There are 1 best solutions below

1
On

query.asList() will pull everything in to memory. query.fetch() will return an Iterator allowing you to process each entity (in batches of 20 by default) without loading the entire result set in to memory first.