NSManagedObjectContext Reset appears to be losing data?

Question

My core data store contains 51 Entry entities which have a message attribute reading "Test". Doing an NSFetchRequest for this confirms that they're all there.

I have another part of my method however, which will be used for memory intensive purposes, dealing with large chunks of NSData, and so I need to call [oldContext reset]; quite often.

I have a memory intensive method which accesses a lot of NSData from my MOC. As such, it regularly calls [oldContext reset];. Without this line, it runs out of memory.

I've discovered that by using this though, it's not returning the correct results. To test this, I commented out the data intensive code, leaving me with code which returned the message attribute, 51 of which are set to "Test" (confirmed by a separate NSFetchRequest).

Using [oldContext reset]; however, it only returns 6 results with the message set to "Test". This is the code I'm using:

    NSFetchRequest *oldFetchRequest = [[NSFetchRequest alloc] init];
    NSEntityDescription *oldEntryEntity = [NSEntityDescription entityForName:@"Entry"
                                                      inManagedObjectContext:oldContext];
    [oldFetchRequest setEntity:oldEntryEntity];
    [oldFetchRequest setFetchBatchSize:10];
    [oldFetchRequest setIncludesPropertyValues:NO];
    NSArray *entrys = [oldContext executeFetchRequest:oldFetchRequest error:&error];

    int totalEntries = [oldContext countForFetchRequest:oldFetchRequest error:nil];

    int i = 0;

    while (i < totalEntries) {
        @autoreleasepool {

            Entry *entry = [entrys objectAtIndex:i];

            NSLog(@"message 1: %@", [entry valueForKey:@"message"]);


            [oldContext reset];


            i++;
        }
    }

Any thoughts as to why it's not giving the 51 "Test" results it should do?

Answer 1

NSFetchRequest *oldFetchRequest = [[NSFetchRequest alloc] init];
NSEntityDescription *oldEntryEntity =
    [NSEntityDescription entityForName:@"Entry"
                inManagedObjectContext:oldContext];
[oldFetchRequest setEntity:oldEntryEntity];
[oldFetchRequest setFetchBatchSize:10];
[oldFetchRequest setIncludesPropertyValues:NO];
NSArray *entrys = [oldContext executeFetchRequest:oldFetchRequest error:&error];
int totalEntries = [oldContext countForFetchRequest:oldFetchRequest error:nil];

The above code executes a fetch request on the MOC oldContext. As a result, the entrys array will contain managed objects for each object in the database that matches the request. In addition, as a result of setting the batch size, the fetch will fault your batch size of objects.

As an experiment to assure yourself of what is happening, add these log statements...

NSLog(@"entrys count = %u", entrys.count);
for (NSManagedObject *entry in entrys) {
    NSLog(@"entry: %@", entry);
}

Do you see what is in that array? Does it make sense now?

Let's look at the rest of the code.

int i = 0;
while (i < totalEntries) {
    @autoreleasepool {
        // You get the i-th entry.  It will be a managed object.  It could be a fault
        // or it could be a fully hydrated object.  Based on your batch size, the
        // first ten (0 <= i < 10) will be complete objects.
        Entry *entry = [entrys objectAtIndex:i];

        // Log the "message" attribute.  By calling valueForKey, the object will be
        // faulted into memory if it is a fault.  Since your batch size is 10,
        // this will make sure 10 objects are faulted if one is needed.
        NSLog(@"message 1: %@", [entry valueForKey:@"message"]);

        // Resetting the entire context blows away everything in the context.
        // Calling reset is a hard call, and should not be done if you have
        // references to the objects in the context.
        [oldContext reset];

        i++;
    }
}

I would suggest a different method. Calling reset is not designed for contexts in which you will retain objects.

There are several alternatives. You could create a child context, do your scratch work in there, and then release the context. It will release all memory it used.

You could selectively use

- (void)refreshObject:(NSManagedObject *)object mergeChanges:(BOOL)flag

which will turn an object back into a fault if flag is NO, freeing its memory. Note, there are also some inherent dangers with this as well, especially if you have managed relationships.

There are a few other options, but without having knowledge of what your ultimate goal is... it's kinda hard to tell what would be most beneficial.

I highly suggest you read all the documentation associated with these calls. In fact, while Core Data is very sophisticated, it does have some "brittle" inter-workings that you should know about if you do anything non-trivial.

I highly recommend reading all the Core Data documentation. It addresses all the issues you have been having with your project.

Answer 2

As per the Apple docs, this is what ManagedObjectContext reset does:

• All the receiver's managed objects are “forgotten.” If you use this method, you should ensure that you also discard references to any managed objects fetched using the receiver, since they will be invalid afterwards.

So in your code, you are telling oldContext to forget all of its objects every time that while loop iterates.

You don't say where in the MOC that the NSData is coming from, but if it's from the Entry entity, I'd say you have two options:

1) Instead of resetting the MOC every time, use NSManagedObjectContext's refreshObject:mergeChanges: method. This will re-fault the entity and free up memory. So for example:

while (i < totalEntries) {
    @autoreleasepool {

        Entry *entry = [entrys objectAtIndex:i];

        NSLog(@"message 1: %@", [entry valueForKey:@"message"]);


        [oldContext refreshObject:entry mergeChanges:NO];


        i++;
    }
}

2) Don't store large amounts of data inside a Core Data entity. Instead, write the data out to the file system and keep a reference to the data's path in the entity. That would be my preferred approach. Take a look at Apple's Core Data Performance under the section "Large Data Objects"

Answer 3

Try setting fetch batch size to 1 instead of current 10.

My interpretation of this situation is as follows.

1. By executeFetchRequest... it fetches 10 records to the context.
2. However, by resetting the context, all records will be lost. So, only the first record in 10 counts. Note that first record is logged before context reset.
3. Since we have 51 records in total, 51 % 10 = 6 records remains.