WinDbg of .Net application shows 2 threads with locks, but empty !locks and !syncblk

35 Views Asked by At

I'm using WinDbg to look at a deadlock of a .Net application. !threads shows me two threads are holding on to a lock (check lock count column):

0:000> !threads
ThreadCount:      34
UnstartedThread:  0
BackgroundThread: 25
PendingThread:    0
DeadThread:       0
Hosted Runtime:   no
                                                                                                        Lock  
       ID OSID ThreadOBJ           State GC Mode     GC Alloc Context                  Domain           Count Apt Exception
   0    1 2f5c 000002df099a99c0    26020 Preemptive  000002DF11A22560:000002DF11A23F98 000002df0999d060 0     STA 
   2    2 3560 000002df099d46d0    2b220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Finalizer) 
   5    3  930 000002df24c2c2f0  1029220 Preemptive  000002DF119A1FC8:000002DF119A3F98 000002df0999d060 0     MTA (Threadpool Worker) 
   6    4 1278 000002df24c3ce70  3029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Worker) 
   7    5 2f6c 000002df24c45cf0  1029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Worker) 
   8    6 3134 000002df24c469a0  1029220 Preemptive  000002DF11937680:000002DF11937F68 000002df0999d060 0     MTA (Threadpool Worker) 
   9    7  7dc 000002df24c4a340  1029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Worker) 
  10    8 3038 000002df24b022b0    2b020 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA 
  11    9 2944 000002df29d3a960  102a220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Worker) 
  13   10 3930 000002df29d6d850  202b020 Preemptive  000002DF119A1ED8:000002DF119A1F98 000002df0999d060 0     MTA 
  14   11  838 000002df29d51620    21220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 1     Ukn (GC) 
  16   12 3e1c 000002df29cf9420    2b220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 1     MTA 
  17   13  74c 000002df29cda690  1020220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     Ukn (Threadpool Worker) 
  27   17  a1c 000002df29e9e620    2b020 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA 
  28   18 1cb4 000002df29e9ceb0    2b020 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA 
  30   19 2734 000002df29e9f5c0    2b020 Preemptive  000002DF11A300C8:000002DF11A31F98 000002df0999d060 0     MTA 
  29   21 3658 000002df29ea0560    20220 Preemptive  000002DF11A2E970:000002DF11A2FF98 000002df0999d060 0     Ukn 
  31   20 2cdc 000002df29e9c6e0  1029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Worker) 
  32   22  9c0 000002df29e9de50  1029220 Preemptive  000002DF11A265C0:000002DF11A27F98 000002df0999d060 0     MTA (Threadpool Worker) 
  33   23 13b0 000002df29ea0d30    2b020 Preemptive  000002DF11A2C978:000002DF11A2DF98 000002df0999d060 0     MTA 
  34   24 2b34 000002df29ea1cd0  202b020 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA 
  37   25 1710 000002df2a0143d0  202b220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA 
  38   27 11d8 000002df2a00f5b0    20220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     Ukn 
  39   30 16ac 000002df2a010d20  1029220 Preemptive  000002DF11A23FF0:000002DF11A25F98 000002df0999d060 0     MTA (Threadpool Worker) 
  40   31 2c70 000002df2a012c60  8029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Completion Port) 
  44   32 3884 000002df2d7ffa70  1029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Worker) 
  43   33  c24 000002df2d7fd360  1029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Worker) 
  49   28 1424 000002df2a010550    2b020 Preemptive  000002DF11928028:000002DF11929F68 000002df0999d060 0     MTA 
  70   14 3af0 000002df3677cb30    2b220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA 
  71   34  550 000002df3677ea70    2b220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA 
  68   35 2b10 000002df5c986850  1029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Worker) 
  69   36  120 000002df36a708e0  1029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Worker) 
  76   29  824 000002df29d6f530  8029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Completion Port) 
  77   16 27fc 000002df3616eca0  8029220 Preemptive  0000000000000000:0000000000000000 000002df0999d060 0     MTA (Threadpool Completion Port) 

However, !syncblk comes up empty, locks are normally listed below the header and above the line:

0:000> !syncblk
Index SyncBlock MonitorHeld Recursion Owning Thread Info  SyncBlock Owner
-----------------------------
Total           2168
CCW             718
RCW             414
ComClassFactory 1
Free            261

Also !locks shows nothing:

0:000> !locks

Scanned 43 critical sections

And using the sosex !dlk command to scan for deadlocks:

.load C:\path_to_where_sosex_dll_lives\sosex
0:000> !dlk
Examining SyncBlocks...
Scanning for ReaderWriterLock(Slim) instances...
Scanning for holders of ReaderWriterLock locks...
Scanning for holders of ReaderWriterLockSlim locks...
Examining CriticalSections...
Scanning for threads waiting on SyncBlocks...
Scanning for threads waiting on ReaderWriterLock locks...
*** WARNING: Unable to verify checksum for xxxxx.dll
Scanning for threads waiting on ReaderWriterLocksSlim locks...
Scanning for threads waiting on CriticalSections...
*** WARNING: Unable to verify checksum for xxxxx.dll
No deadlocks detected.

The finalizer thread might be stuck here (thread 2 above). It is doing some GDI work in this particular dump, but not in another one (see below):

OS Thread Id: 0x3560 (2)
Current frame: ntdll!NtWaitForSingleObject+0x14
Child-SP         RetAddr          Caller, Callee
000000fd4fafefe0 000002df2f74cc71 000002df2f74cc71
000000fd4fafeff0 00007ffd62dd71b1 clr!ETW::GCLog::SendFinalizeObjectEvent+0x55, calling clr!_security_check_cookie
000000fd4faff010 000002df2f6943a5 000002df2f6943a5, calling 000002df2f74cc40
000000fd4faff040 000002df2f6a0208 000002df2f6a0208, calling 000002df2f6942c0
000000fd4faff080 000002df2f773056 000002df2f773056
000000fd4faff150 00007ffd62e4912e clr!SafeHandle::Release+0x9e, calling clr!_security_check_cookie
000000fd4faff190 00007ffd7cc4ab11 ntdll!RtlFreeHeap+0x51, calling ntdll!RtlpFreeHeapInternal
000000fd4faff200 00007ffd4fdfa761 GdiPlus!GpBitmap::`vector deleting destructor'+0x21, calling GdiPlus!GpFree
000000fd4faff220 00007ffd62ed67ae clr!SafeHandle::Dispose+0x86, calling clr!guard_dispatch_icall_nop
000000fd4faff290 00007ffd62ed6616 clr!SafeHandle::Finalize+0xc6, calling clr!HelperMethodFrameRestoreState
000000fd4faff300 00007ffd7a220479 KERNELBASE!WaitForMultipleObjectsEx+0xe9, calling 00007ffd7ac50000
000000fd4faff3b0 000002df2f772e30 000002df2f772e30, calling 000002df2f721c50
000000fd4faff3e0 00007ffd62dddb53 clr!CrstBase::Leave+0x2b, calling ntdll!RtlLeaveCriticalSection
000000fd4faff3f0 00007ffd7a22186c KERNELBASE!SetThreadLocale+0x7c, calling KERNELBASE!_security_check_cookie
000000fd4faff430 00007ffd7a1f497e KERNELBASE!WaitForSingleObjectEx+0x8e, calling 00007ffd7ac70000
000000fd4faff4d0 00007ffd62dc320a clr!CLREventWaitHelper2+0x32, calling kernel32!WaitForSingleObjectEx
000000fd4faff4e0 00007ffd62dd6fb2 clr!CallFinalizer+0x22, calling clr!StressLog::LogOn
000000fd4faff510 00007ffd62dc31ab clr!CLREventWaitHelper+0x1f, calling clr!CLREventWaitHelper2
000000fd4faff570 00007ffd62dc3113 clr!CLREventBase::WaitEx+0x67, calling clr!CLREventWaitHelper
000000fd4faff5b0 00007ffd62dc1cbc clr!FinalizerThread::ProcessProfilerAttachIfNecessary+0x34, calling kernel32!WaitForSingleObject
000000fd4faff5e0 00007ffd62dc1d6f clr!FinalizerThread::WaitForFinalizerEvent+0x9f, calling kernel32!WaitForMultipleObjectsEx
000000fd4faff620 00007ffd62dc1b20 clr!FinalizerThread::FinalizerThreadWorker+0x40, calling clr!FinalizerThread::WaitForFinalizerEvent
000000fd4faff650 00007ffd62e0230b clr!ManagedThreadBase_DispatchInner+0x33, calling clr!guard_dispatch_icall_nop
000000fd4faff670 00007ffd62edb453 clr!ClrFlsIncrementValue+0x1f, calling clr!guard_dispatch_icall_nop
000000fd4faff690 00007ffd62e0222f clr!ManagedThreadBase_DispatchMiddle+0x83, calling clr!ManagedThreadBase_DispatchInner
000000fd4faff6a0 00007ffd62dc375b clr!ThreadSuspend::UnlockThreadStore+0x53, calling clr!ClrFlsIncrementValue
000000fd4faff6b0 00007ffd62da0975 clr!REGUTIL::RegCacheValueNameSeenPerhaps+0x29, calling clr!HashiStringKnownLower80
000000fd4faff6d0 00007ffd62e51e13 clr!ThreadStore::TransferStartedThread+0x7b, calling clr!ThreadStore::CheckForEEShutdown
000000fd4faff6e0 00007ffd62f3116b clr!REGUTIL::GetConfigInteger+0xc7, calling clr!REGUTIL::RegCacheValueNameSeenPerhaps
000000fd4faff710 00007ffd6307bb93 clr!EEConfig::GetConfiguration_DontUse_+0x2dc657, calling clr!HashString
000000fd4faff770 00007ffd62d9f4de clr!EEConfig::GetConfigDWORD_DontUse_+0x56, calling clr!EEConfig::GetConfiguration_DontUse_
000000fd4faff780 00007ffd62e020fb clr!ManagedThreadBase_DispatchOuter+0x87, calling clr!ManagedThreadBase_DispatchMiddle
000000fd4faff810 00007ffd62e9a2a7 clr!FinalizerThread::FinalizerThreadStart+0x107, calling clr!ManagedThreadBase_DispatchOuter
000000fd4faff880 00007ffd62eb18d7 clr!operator delete+0x1b, calling clr!guard_dispatch_icall_nop
000000fd4faff8b0 00007ffd62f058ea clr!Thread::intermediateThreadProc+0x8a, calling clr!guard_dispatch_icall_nop
000000fd4faff930 00007ffd62f058c7 clr!Thread::intermediateThreadProc+0x67, calling clr!_chkstk
000000fd4faff970 00007ffd7bb8257d kernel32!BaseThreadInitThunk+0x1d, calling kernel32!BaseThreadInitXfgThunk
000000fd4faff9a0 00007ffd7cc6aa58 ntdll!RtlUserThreadStart+0x28, calling ntdll!guard_xfg_dispatch_icall_nop

The finalizer thread of annother occurrence of the same deadlock was not doing GDI work, nor anything else that looks suspicious to me:

OS Thread Id: 0x2f98 (2)
Current frame: ntdll!NtWaitForSingleObject+0x14
Child-SP         RetAddr          Caller, Callee
000000dafc4ff250 00000280aef8cc71 00000280aef8cc71
000000dafc4ff260 00007ffd62dd71b1 clr!ETW::GCLog::SendFinalizeObjectEvent+0x55, calling clr!_security_check_cookie
000000dafc4ff280 00000280aeed43a5 00000280aeed43a5, calling 00000280aef8cc40
000000dafc4ff2b0 00000280aeee0208 00000280aeee0208, calling 00000280aeed42c0
000000dafc4ff2f0 00000280aefb3056 00000280aefb3056
000000dafc4ff3c0 00007ffd62e4912e clr!SafeHandle::Release+0x9e, calling clr!_security_check_cookie
000000dafc4ff460 00007ffd7cc4ab11 ntdll!RtlFreeHeap+0x51, calling ntdll!RtlpFreeHeapInternal
000000dafc4ff490 00007ffd62ed67ae clr!SafeHandle::Dispose+0x86, calling clr!guard_dispatch_icall_nop
000000dafc4ff4d0 00007ffd6313357c clr!WKS::GCHeap::RegisterForFinalization+0x218dac, calling clr!WKS::CFinalize::RegisterForFinalization
000000dafc4ff4f0 00007ffd63379f68 clr!AppDomainNative::IsFinalizingForUnload+0x118, calling clr!HelperMethodFrameRestoreState
000000dafc4ff500 00007ffd62f1427e clr!GCInterface::ReRegisterForFinalize+0xde, calling clr!HelperMethodFrameRestoreState
000000dafc4ff510 00007ffd62ecbee3 clr!StubHelpers::SetLastError+0x13, calling clr!GetThread
000000dafc4ff570 00007ffd7a220479 KERNELBASE!WaitForMultipleObjectsEx+0xe9, calling 00007ffd7ac50000
000000dafc4ff620 00000280aefb2e30 00000280aefb2e30, calling 00000280aef61c50
000000dafc4ff650 00007ffd5bbb2374 (MethodDesc 00007ffd5b983930 +0xa4 System.Gen2GcCallback.Finalize()), calling (MethodDesc 00007ffd5b983930 +0xd0 System.Gen2GcCallback.Finalize())
000000dafc4ff660 00007ffd7a22186c KERNELBASE!SetThreadLocale+0x7c, calling KERNELBASE!_security_check_cookie
000000dafc4ff6a0 00007ffd7a1f497e KERNELBASE!WaitForSingleObjectEx+0x8e, calling 00007ffd7ac70000
000000dafc4ff740 00007ffd62dc320a clr!CLREventWaitHelper2+0x32, calling kernel32!WaitForSingleObjectEx
000000dafc4ff750 00007ffd62dd6fee clr!CallFinalizer+0x5e, calling clr!MethodTable::CallFinalizer
000000dafc4ff780 00007ffd62dc31ab clr!CLREventWaitHelper+0x1f, calling clr!CLREventWaitHelper2
000000dafc4ff7e0 00007ffd62dc3113 clr!CLREventBase::WaitEx+0x67, calling clr!CLREventWaitHelper
000000dafc4ff820 00007ffd62dc1cbc clr!FinalizerThread::ProcessProfilerAttachIfNecessary+0x34, calling kernel32!WaitForSingleObject
000000dafc4ff850 00007ffd62dc1d6f clr!FinalizerThread::WaitForFinalizerEvent+0x9f, calling kernel32!WaitForMultipleObjectsEx
000000dafc4ff890 00007ffd62dc1b20 clr!FinalizerThread::FinalizerThreadWorker+0x40, calling clr!FinalizerThread::WaitForFinalizerEvent
000000dafc4ff8c0 00007ffd62e0230b clr!ManagedThreadBase_DispatchInner+0x33, calling clr!guard_dispatch_icall_nop
000000dafc4ff8e0 00007ffd62edb453 clr!ClrFlsIncrementValue+0x1f, calling clr!guard_dispatch_icall_nop
000000dafc4ff900 00007ffd62e0222f clr!ManagedThreadBase_DispatchMiddle+0x83, calling clr!ManagedThreadBase_DispatchInner
000000dafc4ff910 00007ffd62dc375b clr!ThreadSuspend::UnlockThreadStore+0x53, calling clr!ClrFlsIncrementValue
000000dafc4ff920 00007ffd62da0975 clr!REGUTIL::RegCacheValueNameSeenPerhaps+0x29, calling clr!HashiStringKnownLower80
000000dafc4ff940 00007ffd62e51e13 clr!ThreadStore::TransferStartedThread+0x7b, calling clr!ThreadStore::CheckForEEShutdown
000000dafc4ff950 00007ffd62f3116b clr!REGUTIL::GetConfigInteger+0xc7, calling clr!REGUTIL::RegCacheValueNameSeenPerhaps
000000dafc4ff980 00007ffd6307bb93 clr!EEConfig::GetConfiguration_DontUse_+0x2dc657, calling clr!HashString
000000dafc4ff9e0 00007ffd62d9f4de clr!EEConfig::GetConfigDWORD_DontUse_+0x56, calling clr!EEConfig::GetConfiguration_DontUse_
000000dafc4ff9f0 00007ffd62e020fb clr!ManagedThreadBase_DispatchOuter+0x87, calling clr!ManagedThreadBase_DispatchMiddle
000000dafc4ffa80 00007ffd62e9a2a7 clr!FinalizerThread::FinalizerThreadStart+0x107, calling clr!ManagedThreadBase_DispatchOuter
000000dafc4ffaf0 00007ffd62eb18d7 clr!operator delete+0x1b, calling clr!guard_dispatch_icall_nop
000000dafc4ffb20 00007ffd62f058ea clr!Thread::intermediateThreadProc+0x8a, calling clr!guard_dispatch_icall_nop
000000dafc4ffba0 00007ffd62f058c7 clr!Thread::intermediateThreadProc+0x67, calling clr!_chkstk
000000dafc4ffbe0 00007ffd7bb8257d kernel32!BaseThreadInitThunk+0x1d, calling kernel32!BaseThreadInitXfgThunk
000000dafc4ffc10 00007ffd7cc6aa58 ntdll!RtlUserThreadStart+0x28, calling ntdll!guard_xfg_dispatch_icall_nop

How I can I see which locks are involved here?

0

There are 0 best solutions below