I'm querying a SQL database and want to save the query results in a local mongo db repeatedly (e.g. every 30 minutes). The SQL data is not very consistent that is to say that there are many null values.
The data which I want to save has this format:
{
"attribute1": value,
"attribute2": value,
"hugeArray": [], // 30.000 entries
}
The array has around 100MB and will not get much larger in the future.
Every 30 minutes the sql query will almost give the same result.
I'm a newbie and I don't know whether patching and updating the few old values in the mongodb is faster than dropping the collection and (re-) adding all values.
I do not really care about the order of which the entries are inserted, I will only make one SQL query and store that data in my mongodb database.
I can't make use of documents because of this restriction:
The maximum BSON document size is 16 megabytes.
Is a capped collection a good choice for my circumstances or are there any better solutions and can you reason that?