Introduction To Debugging ASP.

NET applications with

WinDBG and Dump Analysis

AHMET MİTHAT BOSTANCI

MICROSOFT, GTSC
IIS/WEB DEVELOPMENT SUPPORT
[email protected]
 SCOPE

This is NOT a full dump analysis workshop.

The goal is to introduce the dump analysis as one of
the ASP.NET 2.0 web application debugging technics
and introduce the tools that help to achieve this task.
 AGENDA

ASP.NET Debugging Technics

What is WinDBG
When are WinDBG and dump analysis useful?
What are dump files?
How to collect dump?
Basic commands
Toolbox & References
 ASP.NET Debugging

Old School Debugging

 Using Response.Write
 Mostly not useful in production
 Not useful in hang scenario
 Usful in development
 Useful in production if there is a repro
 Looks easy but not useful
Debugging in VS.NET
 Good for debugging in development
 Useful in production with remote debugging / attaching to w3wp.exe
 Not useful in most production env. issues.
Debugging with WinDBG
 Live debugging / attaching to w3wp.exe
 Offline debugging: Collecting dumps of w3wp.exe and analysing in WinDBG
 What is WinDBG

A GUI Debugger with commands.

Part of Debugging Tools For Windows
User mode and kernel mode debugging.
Its commands are mainly useful in unmanaged
debugging.
Extensions are sets of commands to ease debugging.
SOS and PSSCOR2 extensions for .NET debugging.
 Debugging with WinDBG

Offline debugging
 Collecting dump files with specific tools for different scenarios
 Analysing in WinDBG.
Live debugging
 Attaching a process
 Using breakpoints, catching exceptions, dumping the process
out to a file, etc... when necessary.
 Debugging Scenarios

Slow performance
 Web site is responsive but slow.
Hang
 Web site is unresponsive.
Crash
 Exceptions
 Exceptions and ASP.NET

Exceptions are important objects in debugging.

First Chance exceptions
 Too many exceptions may cause high CPU.
Second Chance
 Exceptions which are not handled
 Causes crash.
First chance exceptions may cause w3wp.exe
crashes, if it is thrown in a background thread, such
as finalizer threads.
 When do we need WinDBG?

Hang / Performance Problems – Manuel

Web site is running slow / performance issues.
Web site is leaking memory - ends with performance
problems, hang or crash.
Web site is running with high memory but not not
leaking.
Web site is running with 100% CPU.
Web site is running with high CPU or CPU usage spikes.
Web site is completely hung - Does not response any
(dynamic) requests.
 When do we need WinDBG?

Crash
Web site or a specific page / method is throwing
exceptions but not crashing the w3wp.exe process,
first chance exceptions.
Web site is crashing - w3wp.exe is crashing. Second
chance exceptions.
 When do we NOT need WinDBG?

When there is an easier way for debugging, such as a

specific page / method is not functioning as expected
 In this case, debugging in VS.NET with a repro code might be
easier.
«Page cannot be displayed» type issues.
 In this case, troubleshooting IIS or network side is the correct
approach.
 Dump

A dump is a snapshot of a process (user mode) or a

snapshot of all of the memory in kernel mode.
Usermode dump: Process specific, such as w3wp.exe
Kernel mode dump: Snapshot of the kernel memory
Complete memory dump: Snapshot of the whole
memory including kernel and user mode.
 Types of Dump

Minidump
 Contains only selected parts of the memory used by the
process.
Fulldump
 Contains the full copy of the process memory. Size of the
fulldump is somewhat equal to the virtual memory used by the
process.
 This is the dump type we are interested in most of the times.
 What a dump contains

All the things in the memory of the process.

Information about the objects in heaps, threads,
information about the callstack, stack memory, etc...
Limited information about the dead objects which
are not garbage collected yet.
 What a dump does NOT contain

Other processes' information.

What was the CPU or memory usage in time. It is
only a snapshot.
Also: Minidump is the smallest in size and does not
contain some useful information such as stack
objects, local variables, etc...
 Tools for collecting dumps

ADPlus
 Part of Debugging Tools For Windows
 Command line
 adplus -hang -pn w3wp.exe
 adplus -crash –iis
Debug Diagnostic Tool
 GUI based
 Easy to configure and use
ProcDump
 Command line
 Specific for high CPU usage but can be used for collecting manuel dumps as well
 procdump.exe -c 80 -s 30 -n 5 -ma NNNN >procdump.log
Task manager
 Can be used to collect manuel dump
 Right click and create userdump
 Only on Win 7 and Win 2008
 32bit vs 64bit

If the process is 32bit then the dump should be taken

by the 32bit version of the related tools. The dump
should be opened with the WinDBG’s 32bit version.
Same rule applies to 64bit process.
 Which dump in which scenario?

Web site is crashing / throwing exceptions

 Crash rule with Debug Diagnostic Tool, ADPlus
Web site is slow, hanging, using high memory
 Manuel dump(s) with Debug Diagnostic Tool, ADPlus.
Web site is using high CPU or CPU is spiking
 Manuel dump(s) with Debug Diagnostic Tool, ADPlus.
 ProcDump for CPU spikes
Web site is leaking memory
 Manuel hang dumps.
 Memory leak rule created in Debug Diagnostic Tool.
Full IIS dumps
 Manuel dump
 Collects dumps for all of w3wp.exe, all of dllhost.exe and inetinfo.exe process.
 How many dump files should I collect?

Dump is a snapshot of the memory and does not

contain the process history.
Most of the crash scenarios
 One crash dump is enough to start debugging and find the
reason of the crash.
Performance, hang
 Consecutive dump files of the same process.
 For example taking three dumps of w3wp.exe, each are taken
one minute apart.
 Symbol Files

Footprint information of functions in EXE or DLL

files
Necessary to see useful information in dumps, such
as callstacks or variables.
 Private vs Public symbols

Private symbols contain local variables, structure

type information, functions, source line no., etc...
Public symbols are the stripped version of the
private symbols.
 Symbols and WinDBG

Symbol server
 http://msdl.microsoft.com/download/symbols
Setting symbol server in WinDBG
 srv*c:\symbols\public*http://msdl.microsoft.com/download/
symbols
Symbols are downloaded to the local folder when
needed.
 Commands in WinDBG

WinDBG commands
 Mostly used in unmanaged debugging.
Extensions
 DLL files
 Command sets for specific debugging requirements
 Extensions

Copied to the WinDBG installation directory

.NET related extensions
 SOS
 Comes with .NET framework
 PSSCOR2
 Superset of SOS
 .NET 2.0 specific extension
 http://www.microsoft.com/downloads/details.aspx?FamilyID=5c
068e9f-ebfe-48a5-8b2f-0ad6ab454ad4&displayLang=en
 Extract and copy to the WinDBG installation folder.
 Load from WinDBG:
 .load psscor2
 WinDBG and ASP.NET 2.0 dump analysis

Open dump file

Check and set symbol server
Load the PSSCOR2 extenstion
 .load psscor2
 Some WinDBG commands

Vertarget
 Shows information about the system on which you are debugging
Lm
 Displays loaded module information
~
 Lists all OS threads
~Ns
 Switch to thread #N
K
 Shows unmanaged call stack of the thread switched.
.hh
 Opens help file
.reload /f
 Forces to reload the symbols.
!runaway
 Shows the thread’s CPU time
 Some PSSCOR2 commands

!DumpAllExceptions (!dae)
 Shows managed exceptions thrown.
!DumpDataTables
 Shows data tables objects in memory. Useful when debugging memory issues.
!DumpColumnNames (!dcn)
 Prints out the column names for a data table given.
!ASPXPages
 Shows ASPX pages running on all threads.
!threadpool
 Shows ASP.NET thread pool information including the requests in the queue and the current CPU usage of OS.
!FindDebugTrue
 Shows the web applications running in debug mode.
!FindDebugModules
 Shows the modules built in debug mode.
!SaveAllModules (!sam)
 Saves the all modules in the process. Useful when using the reflector to get the code from dump file.
!eeversion
 Prints the .NET framework version. Useful when checking the fixes or service packs installed.
!help
!help <command>
 Toolbox

Debugging Tools For Windows (WinDBG and

ADPlus)
 http://www.microsoft.com/whdc/DevTools/Debugging/defaul
t.mspx
Debug Diagnostic Tool v1.1
 http://www.microsoft.com/downloads/details.aspx?FamilyID
=28BD5941-C458-46F1-B24D-F60151D875A3&displaylang=e
n
ProcDump
 http://technet.microsoft.com/en-us/sysinternals/dd996900.a
spx
PSSCOR2
 http://www.microsoft.com/downloads/details.aspx?familyid=
5C068E9F-EBFE-48A5-8B2F-0AD6AB454AD4&displaylang=
 Refernces

http://blogs.msdn.com/tess/ (İngilizce)
http://blogs.msdn.com/amb/
http://blogs.msdn.com/farukceliktr/
http://blogs.msdn.com/cenkiscan/
http://www.codeproject.com/KB/debug/windbg_pa
rt1.aspx
http://www.cengizhan.com/
Q&A