Using ADPlus to create dumps on a specific .NET exception

Sometimes an application throws an exception and you need to have a look at the environment to be able to find the cause of the exception. To do this, it is convenient if it would be possible to capture a snapshot of the process precisely at the point when the exception is thrown.

The example at hand is an issue where a customer reported that the application threw an ArgumentException:

ArgumentException: '', hexadecimal value 0x1B, is an invalid character.]
   System.Xml.XmlEncodedRawTextWriter.InvalidXmlChar(Int32 ch, Char* pDst, Boolean entitize) +1702938
   System.Xml.XmlEncodedRawTextWriter.WriteElementTextBlock(Char* pSrc, Char* pSrcEnd) +493
   System.Xml.XmlEncodedRawTextWriter.WriteString(String text) +88
   System.Xml.XmlWellFormedWriter.WriteString(String text) +87
   System.Xml.XmlWrappedWriter.WriteString(String text) +15
   System.ServiceModel.Syndication.TextSyndicationContent.WriteContentsTo(XmlWriter writer) +62
   System.ServiceModel.Syndication.SyndicationContent.WriteTo(XmlWriter writer, String outerElementName, String outerElementNamespace) +430
   System.ServiceModel.Syndication.Atom10FeedFormatter.WriteItemContents(XmlWriter dictWriter, SyndicationItem item, Uri feedBaseUri) +840
   System.ServiceModel.Syndication.Atom10ItemFormatter.WriteItem(XmlWriter writer) +42
   System.ServiceModel.Syndication.Atom10ItemFormatter.WriteTo(XmlWriter writer) +58
   EPiServer.Search.IndexItemBase.ToSyndicationItemXml() +4104
   EPiServer.Search.IndexRequestItem.ToSyndicationItemXml() +139
   EPiServer.Search.Data.SearchFactory.AddToQueue(IndexRequestItem item, String namedIndexingService) +117
   EPiServer.Search.SearchHandler.UpdateIndex(IndexRequestItem item, String namedIndexingService) +70
   EPiServer.Community.Search.SearchHandler.UpdateIndex(Int32 entityId, IEntity entity, IndexAction action) +312
   EPiServer.Community.Search.SearchHandler.AddEntitiesToIndex(GetCollection getCollection, Boolean overwrite) +233
   EPiServer.Community.Search.SearchHandler.AddCommunityEntitiesToIndex(Type type, Boolean overwrite) +1536
   EPiServer.Community.Search.SearchHandler.AddCommunityEntitiesToIndex(Boolean overwrite) +20

Interpreting the exception, we know that the problem occurs during the indexing of some community entity. However, the stack trace leaves us no hint of which entity, nor the field that contain the 0x1B character that the XML writer class is unable to handle.

Again, Debugging Tools for Windows comes to the rescue. This time, I'll show you how to use ADPlus to attach to a process and perform a dump each time an ArgumentException is thrown.

Create a file, let's call it "dump-on-argumentexception.cfg", with the following content:

<ADPlus>
    <!-- Configuring ADPlus to log only exceptions we're interested in -->
  <Exceptions>
     <Config>
        <!-- This is for the CLR exception -->
       <Code> clr </Code>
       <Actions1> Log </Actions1>
       <CustomActions1> .loadby sos mscorwks; !StopOnException System.ArgumentException 1; j ($t1 = 1) '.dump /ma /u ArgumentException.dmp; gn' ; 'gn'</CustomActions1>
       <ReturnAction1> VOID  </ReturnAction1>
       <Actions2> Log </Actions2>
     </Config>
  </Exceptions>
</ADPlus>

Note that when you're troubleshooting a .NET 4 process, you need to replace ".loadby sos mscorwks" with ".loadby sos clr" in the CustomActions1-element of the configuration above.

Then, invoke ADPlus with the following set of parameters.

C:\Program Files (x86)\Debugging Tools for Windows (x86)>cscript adplus.vbs -crash –p <PID> -FullOnFirst –c dump-on-argumentexception.cfg

Replace <PID> with the process ID you want to monitor, for example the w3wp.exe. ADPlus will launch, attach a console debugger to the process and take a memory dump each time the process throws an ArgumentException, even if it's handled (try/catched).

Debugging the created memory dump(s)

Open up WinDbg, and select File->Open Crash Dump... Navigate to one of the created dumps.

Load the .NET debugging extension:

0:020> .loadby sos mscorwks

Note the 0:020? This is WinDbg's syntax of telling us which process:thread is currently active. In dumps with an active exception on one of its threads, WinDbg will automatically switch to the thread with the exception.

Now, to see the stack trace including all available parameters and local variables, issue the following command:

0:020> !clrstack –a
OS Thread Id: 0xdd1c (20)
ESP       EIP 
0b8ff600 66083e1c System.Web.HttpRuntime.ProcessRequestInternal(System.Web.HttpWorkerRequest)
    PARAMETERS:
        this = 0x02611d4c
...
...

Given this information, you may call SOS' !DumpObject command ("!do" for short) to examine the contents of 'this' or any other parameter that are still available (release mode binaries often optimize them away), and !DumpStackObjects ("!dso" for short) to enumerate all objects on the stack.

Unfortunately, I don't have a memory dump that is perfectly suitable to demonstrate this. An out-of-context example would be this examination of the LanguageManager's FileSystemWatcher instance on the current thread.

0:020:x86> !dso
OS Thread Id: 0x1524 (20)
ESP/REG  Object   Name
000000000f16a624 000000000358c1d4 System.IO.FileSystemWatcher
000000000f16a67c 000000000358c1d4 System.IO.FileSystemWatcher
000000000f16a690 000000000358c1d4 System.IO.FileSystemWatcher
000000000f16a694 000000000358c1d4 System.IO.FileSystemWatcher
000000000f16a6f8 000000000358c1d4 System.IO.FileSystemWatcher
000000000f16a714 000000000358c1d4 System.IO.FileSystemWatcher
000000000f16a730 000000000358c1d4 System.IO.FileSystemWatcher
000000000f16a740 000000000358c1d4 System.IO.FileSystemWatcher
000000000f16a750 000000000358c030 EPiServer.Core.LanguageManager
000000000f16a75c 000000000358c030 EPiServer.Core.LanguageManager
000000000f16a768 000000000358bf28 System.Object
000000000f16a7a0 000000000357f0f4 EPiServer.Web.InitializeEngine+ActionState
000000000f16a7a4 000000000357ec8c EPiServer.Web.InitializeEngine

0:020:x86> !do 000000000358c1d4
Name: System.IO.FileSystemWatcher
MethodTable: 000000000a2de9a4
EEClass: 000000000adb82b0
Size: 92(0x5c) bytes
(C:\windows\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
0000000060ad078c  400018a        4        System.Object  0 instance 0000000000000000 __identity
000000000a2dc594  40008e0        8 ...ponentModel.ISite  0 instance 0000000000000000 site
000000000ae0b3a4  40008e1        c ....EventHandlerList  0 instance 0000000000000000 events
0000000060ad078c  40008df      108        System.Object  0   static 000000001365be64 EventDisposed
0000000060ad0b70  400315f       10        System.String  0 instance 000000000358c044 directory
0000000060ad0b70  4003160       14        System.String  0 instance 000000000358c098 filter
0000000060aceb3c  4003161       18 ...es.SafeFileHandle  0 instance 0000000000000000 directoryHandle
000000000a2de708  4003162       34         System.Int32  1 instance       81 notifyFilters
0000000060aa463c  4003163       40       System.Boolean  1 instance        0 includeSubdirectories
0000000060aa463c  4003164       41       System.Boolean  1 instance        0 enabled
0000000060aa463c  4003165       42       System.Boolean  1 instance        0 initializing
0000000060ad2dbc  4003166       38         System.Int32  1 instance     8192 internalBufferSize
000000000a2de93c  4003167       48 ...tForChangedResult  1 instance 000000000358c21c changedResult
0000000060aa463c  4003168       43       System.Boolean  1 instance        0 isChanged
00000000084dd30c  4003169       1c ...SynchronizeInvoke  0 instance 0000000000000000 synchronizingObject
0000000060aa463c  400316a       44       System.Boolean  1 instance        0 readGranted
0000000060aa463c  400316b       45       System.Boolean  1 instance        1 disposed
0000000060ad2dbc  400316c       3c         System.Int32  1 instance        2 currentSession
000000000a2deb5c  400316d       20 ...ystemEventHandler  0 instance 0000000000000000 onChangedHandler
000000000a2deb5c  400316e       24 ...ystemEventHandler  0 instance 0000000000000000 onCreatedHandler
000000000a2deb5c  400316f       28 ...ystemEventHandler  0 instance 0000000000000000 onDeletedHandler
000000000a2dec40  4003170       2c ...namedEventHandler  0 instance 0000000000000000 onRenamedHandler
000000000a2dfd74  4003171       30 ...ErrorEventHandler  0 instance 0000000000000000 onErrorHandler
0000000060aa463c  4003172       46       System.Boolean  1 instance        1 stopListening
0000000060aa463c  4003173       47       System.Boolean  1 instance        0 runOnce
0000000060ad17a0  4003174      bc0        System.Char[]  0   static 000000001366d540 wildcards
0000000060ad2dbc  4003175      b08         System.Int32  1   static      383 notifyFiltersValidMask
0:020:x86> !do 000000000358c044
Name: System.String
MethodTable: 0000000060ad0b70
EEClass: 000000006088d66c
Size: 82(0x52) bytes
(C:\windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
String: C:\EPiServer\Sites\composer\lang
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
0000000060ad2dbc  4000096        4         System.Int32  1 instance       33 m_arrayLength
0000000060ad2dbc  4000097        8         System.Int32  1 instance       32 m_stringLength
0000000060ad1850  4000098        c          System.Char  1 instance       43 m_firstChar
0000000060ad0b70  4000099       10        System.String  0   shared   static Empty
    >> Domain:Value  0000000000798d20:00000000025b1198 0000000007f817e0:00000000025b1198 0000000007f81ca8:00000000025b1198 <<
0000000060ad17a0  400009a       14        System.Char[]  0   shared   static WhitespaceChars
    >> Domain:Value  0000000000798d20:00000000025b1788 0000000007f817e0:000000000266e0e0 0000000007f81ca8:00000000035a0794 <<