iOS11 watchdog timeout crashes (0x8badf00d) but code not on stack

Question

I am debugging user reports of our app repeatedly exiting while in the background starting in iOS11, even during active use (for example, the user backgrounds us and returns within a few seconds or a minute, only to find it restarted). Crash logs all reveal the same cause: a watchdog timeout. Here is the relevant bit from one such crash log:

Exception Type:  EXC_CRASH (SIGKILL)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note:  EXC_CORPSE_NOTIFY
Termination Reason: Namespace <0xF>, Code 0x8badf00d
Triggered by Thread:  0

I realize that our code has a limited period of time during which to operation when receiving push notifications or when backgrounded. We definitely do use UIBackgroundTasks (with Alamofire Networking, FWIW), and we do have expiration handlers that do this:

backgroundTask = [application beginBackgroundTaskWithExpirationHandler:^{
    [application endBackgroundTask:backgroundTask];
    backgroundTask = UIBackgroundTaskInvalid; // Set the task to be invalid
    DebugLog(@"Ended because expiration");
}];

The most perplexing thing about these crash reports is that our code is nowhere on the stacks. We can see from this Apple discussion of the 0x8badf00d exception code, the offending code is, in fact, being actively executed on the main thread.

However, in my case, none of the stacks ever have any of my code being executed. Here is one representative sample:

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib          0x0000000183208bc4 0x183208000 + 3012
1   libsystem_kernel.dylib          0x0000000183208a3c 0x183208000 + 2620
2   CoreFoundation                  0x00000001836b9c4c 0x1835d0000 + 957516
3   CoreFoundation                  0x00000001836b7818 0x1835d0000 + 948248
4   CoreFoundation                  0x00000001835d7e78 0x1835d0000 + 32376
5   GraphicsServices                0x000000018546cf84 0x185462000 + 44932
6   UIKit                           0x000000018d37a0bc 0x18d307000 + 471228
7   MyApp                           0x0000000102a6572c main + 87852 (main.m:22)
8   libdyld.dylib                   0x00000001830fa56c 0x1830f9000 + 5484

Thread 1 name:  com.apple.uikit.eventfetch-thread
Thread 1:
0   libsystem_kernel.dylib          0x0000000183208bc4 0x183208000 + 3012
1   libsystem_kernel.dylib          0x0000000183208a3c 0x183208000 + 2620
2   CoreFoundation                  0x00000001836b9c4c 0x1835d0000 + 957516
3   CoreFoundation                  0x00000001836b7818 0x1835d0000 + 948248
4   CoreFoundation                  0x00000001835d7e78 0x1835d0000 + 32376
5   Foundation                      0x00000001840006e4 0x183ff4000 + 50916
6   Foundation                      0x000000018401fafc 0x183ff4000 + 178940
7   UIKit                           0x000000018ded9630 0x18d307000 + 12396080
8   Foundation                      0x0000000184101860 0x183ff4000 + 1103968
9   libsystem_pthread.dylib         0x000000018333c31c 0x18333a000 + 8988
10  libsystem_pthread.dylib         0x000000018333c1e8 0x18333a000 + 8680
11  libsystem_pthread.dylib         0x000000018333ac28 0x18333a000 + 3112

Thread 2 name:  com.twitter.crashlytics.ios.MachExceptionServer
Thread 2:
0   libsystem_kernel.dylib          0x0000000183208bc4 0x183208000 + 3012
1   libsystem_kernel.dylib          0x0000000183208a3c 0x183208000 + 2620
2   MyApp                           0x0000000102cdaad8 CLSMachExceptionServer + 100
3   libsystem_pthread.dylib         0x000000018333c31c 0x18333a000 + 8988
4   libsystem_pthread.dylib         0x000000018333c1e8 0x18333a000 + 8680
5   libsystem_pthread.dylib         0x000000018333ac28 0x18333a000 + 3112

Thread 3 name:  com.apple.NSURLConnectionLoader
Thread 3:
0   libsystem_kernel.dylib          0x0000000183208bc4 0x183208000 + 3012
1   libsystem_kernel.dylib          0x0000000183208a3c 0x183208000 + 2620
2   CoreFoundation                  0x00000001836b9c4c 0x1835d0000 + 957516
3   CoreFoundation                  0x00000001836b7818 0x1835d0000 + 948248
4   CoreFoundation                  0x00000001835d7e78 0x1835d0000 + 32376
5   CFNetwork                       0x0000000183d41de0 0x183c93000 + 716256
6   Foundation                      0x0000000184101860 0x183ff4000 + 1103968
7   libsystem_pthread.dylib         0x000000018333c31c 0x18333a000 + 8988
8   libsystem_pthread.dylib         0x000000018333c1e8 0x18333a000 + 8680
9   libsystem_pthread.dylib         0x000000018333ac28 0x18333a000 + 3112

Thread 4 name:  AVAudioSession Notify Thread
Thread 4:
0   libsystem_kernel.dylib          0x0000000183208bc4 0x183208000 + 3012
1   libsystem_kernel.dylib          0x0000000183208a3c 0x183208000 + 2620
2   CoreFoundation                  0x00000001836b9c4c 0x1835d0000 + 957516
3   CoreFoundation                  0x00000001836b7818 0x1835d0000 + 948248
4   CoreFoundation                  0x00000001835d7e78 0x1835d0000 + 32376
5   AVFAudio                        0x0000000189615774 0x189591000 + 542580
6   AVFAudio                        0x0000000189640018 0x189591000 + 716824
7   libsystem_pthread.dylib         0x000000018333c31c 0x18333a000 + 8988
8   libsystem_pthread.dylib         0x000000018333c1e8 0x18333a000 + 8680
9   libsystem_pthread.dylib         0x000000018333ac28 0x18333a000 + 3112

Thread 5:
0   libsystem_kernel.dylib          0x0000000183229150 0x183208000 + 135504
1   libsystem_pthread.dylib         0x000000018333ed30 0x18333a000 + 19760
2   libc++.1.dylib                  0x00000001828e3ea4 0x1828dc000 + 32420
3   JavaScriptCore                  0x000000018b157d00 0x18a812000 + 9723136
4   JavaScriptCore                  0x000000018b157c28 0x18a812000 + 9722920
5   JavaScriptCore                  0x000000018b157f8c 0x18a812000 + 9723788
6   libsystem_pthread.dylib         0x000000018333c31c 0x18333a000 + 8988
7   libsystem_pthread.dylib         0x000000018333c1e8 0x18333a000 + 8680
8   libsystem_pthread.dylib         0x000000018333ac28 0x18333a000 + 3112

Thread 6 name:  WebThread
Thread 6:
0   libsystem_kernel.dylib          0x0000000183208bc4 0x183208000 + 3012
1   libsystem_kernel.dylib          0x0000000183208a3c 0x183208000 + 2620
2   CoreFoundation                  0x00000001836b9c4c 0x1835d0000 + 957516
3   CoreFoundation                  0x00000001836b7818 0x1835d0000 + 948248
4   CoreFoundation                  0x00000001835d7e78 0x1835d0000 + 32376
5   WebCore                         0x000000018bc1c75c 0x18bbdb000 + 268124
6   libsystem_pthread.dylib         0x000000018333c31c 0x18333a000 + 8988
7   libsystem_pthread.dylib         0x000000018333c1e8 0x18333a000 + 8680
8   libsystem_pthread.dylib         0x000000018333ac28 0x18333a000 + 3112

Thread 7 name:  WebCore: LocalStorage
Thread 7:
0   libsystem_kernel.dylib          0x0000000183229150 0x183208000 + 135504
1   libsystem_pthread.dylib         0x000000018333ed30 0x18333a000 + 19760
2   JavaScriptCore                  0x000000018a81fa18 0x18a812000 + 55832
3   JavaScriptCore                  0x000000018b13da04 0x18a812000 + 9615876
4   WebKitLegacy                    0x000000018d00f5fc 0x18cf7c000 + 603644
5   WebKitLegacy                    0x000000018d01226c 0x18cf7c000 + 615020
6   WebKitLegacy                    0x000000018d011998 0x18cf7c000 + 612760
7   JavaScriptCore                  0x000000018a81c010 0x18a812000 + 40976
8   JavaScriptCore                  0x000000018a81bf50 0x18a812000 + 40784
9   libsystem_pthread.dylib         0x000000018333c31c 0x18333a000 + 8988
10  libsystem_pthread.dylib         0x000000018333c1e8 0x18333a000 + 8680
11  libsystem_pthread.dylib         0x000000018333ac28 0x18333a000 + 3112

Thread 8 name:  com.apple.CFSocket.private
Thread 8:
0   libsystem_kernel.dylib          0x0000000183229570 0x183208000 + 136560
1   CoreFoundation                  0x00000001836c2184 0x1835d0000 + 991620
2   libsystem_pthread.dylib         0x000000018333c31c 0x18333a000 + 8988
3   libsystem_pthread.dylib         0x000000018333c1e8 0x18333a000 + 8680
4   libsystem_pthread.dylib         0x000000018333ac28 0x18333a000 + 3112

Thread 9:
0   libsystem_pthread.dylib         0x000000018333ac1c 0x18333a000 + 3100

Thread 10:
0   libsystem_kernel.dylib          0x0000000183229dbc 0x183208000 + 138684
1   libsystem_pthread.dylib         0x000000018333afa0 0x18333a000 + 4000
2   libsystem_pthread.dylib         0x000000018333ac20 0x18333a000 + 3104

Thread 11:
0   libsystem_pthread.dylib         0x000000018333ac1c 0x18333a000 + 3100

Thread 0 crashed with ARM Thread State (64-bit):
    x0: 0x0000000010004005   x1: 0x0000000007000806   x2: 0x0000000000000000   x3: 0x0000000000000c00
    x4: 0x0000000000002b03   x5: 0x00000000ffffffff   x6: 0x0000000000000000   x7: 0x0000000000000000
    x8: 0x00000000fffffbbf   x9: 0x0000000007000000  x10: 0x0000000007000100  x11: 0x0000000000000040
   x12: 0xffffffffffffffff  x13: 0x0000000000000001  x14: 0x01e8540001e85400  x15: 0x0000000000000000
   x16: 0xffffffffffffffe1  x17: 0x00000000ffffffff  x18: 0x0000000000000000  x19: 0x0000000000000000
   x20: 0x00000000ffffffff  x21: 0x0000000000002b03  x22: 0x0000000000000c00  x23: 0x000000016d3aed38
   x24: 0x0000000007000806  x25: 0x0000000000000000  x26: 0x0000000007000806  x27: 0x0000000000000c00
   x28: 0x0000000000000001   fp: 0x000000016d3aec30   lr: 0x0000000183208a3c
    sp: 0x000000016d3aebe0   pc: 0x0000000183208bc4 cpsr: 0x60000000

The only part of my code that is running is main.m, line 22, which is

int retVal = UIApplicationMain(argc, argv, nil, @"PSSMyAppDelegate");  

Thus, I am perplexed as to how my app could be repeatedly killed for running time violations when it would seem that none of my code is, in fact, running. Is there anything new in iOS 11 that could change the behavior of the watchdog process? If not, how can I tell what part of my code is the part that is in violation for running too long?

Answer 1

I spoke with an engineer at Apple who described the problem as follows -- imagine we had a method like this:

- (void)startTask {
   self.bgTask = [application beginBackgroundTaskWithExpirationHandler:^{
      [application endBackgroundTask:self.bgTask];
      self.bgTask = UIBackgroundTaskInvalid;
   }];
}

The code then calls it twice (say, because we received two calls to didReceiveRemoteNotification):

[self startTask];
[self startTask];

According to the engineer we spoke with, the app will then crash exactly the way we are seeing.

Moreover, technically, our app will crash any time we create a second task before ending the first task. Unless background tasks are directly attached to some other object that directly manages their lifetime and is incapable of "owning" more than one (for example, a standalone operation that begins 1 single task at creation and ends it at completion/destruction), it's very difficult to avoid "task reentrancy" issue.

The better option is to rely on local variables for the task expiration, keeping your object member variables out of the expiration handler.

Answer 2

Your Thread 0 looks a lot like the crash we are experiencing in iOS 11. This is ours:

Thread 0 name:  Dispatch queue: com.apple.main-thread
Thread 0 Crashed:
0   libsystem_kernel.dylib         0x00000001853c4bc4 mach_msg_trap + 8
1   libsystem_kernel.dylib         0x00000001853c4a3c mach_msg + 72
2   CoreFoundation                 0x0000000185875c4c __CFRunLoopServiceMachPort + 196
3   CoreFoundation                 0x0000000185873818 __CFRunLoopRun + 1424
4   CoreFoundation                 0x0000000185793e78 CFRunLoopRunSpecific + 436
5   GraphicsServices               0x0000000187628f84 GSEventRunModal + 100
6   UIKit                         0x000000018f5360bc UIApplicationMain + 208
7   TestCrashInBackground         0x0000000100e8dbac 0x100e88000 + 23468
8   libdyld.dylib                 0x00000001852b656c start + 4

For us the problem is that displaying notifications causes the app to crash (when the app is backgrounded). We have even created a test app (TestCrashInBackground that you see above) in which we can reproduce this issue. The app only displays a notification and doesn't have any background tasks. The test case is:

1. The app shows a notification
2. The user backgrounds the app (presses the home key).
3. The user locks the screen

Result: The OS kills the app after some (short) time.

We have taken a sysdiagnose after this happens and we can see that when we display a notification an assertion with the name "will present notification" is added.

default 2017-10-03 14:32:16.280562 +0200    assertiond  [SpringBoard:53] Attempting to acquire assertion for TestCrashInBack:507: <BKProcessAssertion: 0x101235c90; "will present notification" (notificationAction:30s); id:…E1D79D51D1D9>
default 2017-10-03 14:32:16.281283 +0200    assertiond  [TestCrashInBack:507] Add assertion: <BKProcessAssertion: 0x101235c90; id: 53-96A5F4EA-4C42-4675-97E4-E1D79D51D1D9; name: "will present notification"; state: active; reason: notificationAction; duration: 30.0s> {
    owner = <BSProcessHandle: 0x10110a810; SpringBoard:53; valid: YES>;
    flags = preventSuspend, preventThrottleDownUI, preventIdleSleep, preventSuspendOnSleep;
}

After about 45 seconds the app is killed because it "has active assertions beyond permitted time":

default 2017-10-03 14:33:00.436085 +0200    assertiond  [TestCrashInBack:507] Forcing crash report with description: TestCrashInBack:507 has active assertions beyond permitted time: 
<BKProcessAssertion: 0x101235c90; "will present notification" (notificationAction:30s); id:…E1D79D51D1D9> (owner: SpringBoard:53)

All this just for displaying a notification... pretty serious bug!

This is the code that adds the notification:

UNMutableNotificationContent *content = [[UNMutableNotificationContent alloc] init];
content.body = NSLocalizedString(@"This is test notification", nil);

UNNotificationRequest *request = [UNNotificationRequest requestWithIdentifier:content.body content:content trigger:[UNTimeIntervalNotificationTrigger triggerWithTimeInterval:1.0 repeats:NO]];
[[UNUserNotificationCenter currentNotificationCenter] addNotificationRequest:request withCompletionHandler:^(NSError * _Nullable error) {
    NSLog(@"display notification error:%@", error);
}];

I have filed a bug report to Apple (id: 34788843) with our test project and the sysdiagnose attached. Hopefully they will adress this issue soon.