.natvis files and type templates in WinDbg


When we work with binary data we often use the dt command to group the bytes into meaningful fields, eg.

0:000> dt ntdll!_PEB @$peb
   +0x000 InheritedAddressSpace : 0 ''
   +0x001 ReadImageFileExecOptions : 0 ''
   +0x002 BeingDebugged    : 0x1 ''
   +0x003 BitField         : 0x8 ''
   +0x003 ImageUsesLargePages : 0y0
   +0x003 IsProtectedProcess : 0y0
   +0x003 IsLegacyProcess  : 0y0
   +0x003 IsImageDynamicallyRelocated : 0y1
   +0x003 SkipPatchingUser32Forwarders : 0y0
    ...

The problem arises when the library owner does not provide type information in the symbol files. We are usually left with a manual decomposition of the bytes in a binary editor (010 Editor has a nice template system). Wouldn’t it be great if we had some template system available also in the debugger? I have some good news for you: with the latest release of WinDbg we received a very powerful feature: .natvis files. There were even two Defrag Tools episodes dedicated to this functionality: Defrag Tools #138 and Defrag Tools #139. Let’s first analyze how the .natvis files are built, to later use them in our binary data analysis.

Continue reading

Understanding the Thread Time view in PerfView


Recently while examining a slow request issue (I have a plan to describe this investigation in a seperate post) it came to me that every time I open the Thread Time view it takes a moment to understand what this view actually contains. I decided to write this post for me and for any of you who share the same feeling about this window :).

Continue reading

Diagnosing a Windows Service timeout with PerfView


Today I would like to share with you an interesting (I hope) diagnostics case in one of our system services. The IngestService (that is its name) was not starting properly for the first time – it was being killed because of exceeding the default 30s timeout. But the second try was always successful. No exception was thrown and no logs could be found in the event logs. It’s a situation when ETW traces might shed some light on what’s going on. As it was a .NET service I used PerfView to record the trace file. An important checkbox to select when diagnosing thread wait times is the Thread Time box:

perfview-collect

After collecting the traces on production, I merged them and copied to my developer machine.

Continue reading

NetExt – SOS on steroids


I have been playing recently with quite a new windbg extension (released by Rodney Viana from Microsoft) called NetExt. Rodney Viana published an introductory post about it, which you may find on his blog. In this post I would like to show you my usage samples as well as encourage you to start using it by yourself. Netext documentation is thorough and nicely organized which is good because at the beginning you probably will spend a lot of time on this page 🙂 In paragraphs that follow I will focus mainly on dump debugging, but most of the techniques presented here should work as well in live debugging sessions.

Continue reading

How to debug Windows Services written in .NET? (part II)


This post is the second and final one dedicated to debugging .NET Windows services (you may read the first one here). The inwrap tool (presented in the first part) is not very friendly to use and I myself haven’t used it since then 🙂 It’s not the best advertisement of someone’s own work, but it did motivate me to write another tool which maybe will gain your interest. The winsvcdiag is a simple application that allows you to debug a start of a Windows service from Visual Studio (or any other debugger – even the remote one).

Continue reading

A case of a deadlock in a .NET application


I recently had an interesting issue in one of our applications. The SMS router, responsible for sending and receiving SMSes, hanged – there was no CPU usage and we haven’t observed any activity in the application logs. I collected a full memory dump and restarted the service, which seemed to come back to its normal state. Curious what happened I opened the dump in WinDbg, loaded PDE and SOS and started an investigation

Continue reading

Timeouts when making web requests in .NET


In one of our applications I recently observed timeouts in code performing HTTP requests to the REST service. While investigating this issue I discovered few interesting facts about System.Net namespace and would like to share them with you. We were using objects of type System.Net.HttpWebRequest in our code, but some of the information presented in this post will also apply to the newer System.Net.HttpClient implementation.

Continue reading

How to debug Windows Services written in .NET? (part I)


Diagnosing Windows Services might sometimes be cumbersome – especially when errors occur during the service start. In this two-parts series I am going to show you different ways how to handle such problems in production. In the first part we will focus on “exceptions discovery” techniques which very often are enough to figure out why our service is not working. In the second part we will setup a debugging environment and attach a debugger to our service. Let’s start then.

Continue reading

Collect .NET applications traces with sysinternals tools


In this short post I would like to show you how, with sysinternals tools, you may noninvasively trace .NET applications. This is especially useful in production environment where you can’t install your favorite debugger and hang whole IIS to diagnose an issue. We will work with three tools: dbgview, procdump and procmon. Let’s start with the first one.

Continue reading

Reference Source, dotPeek and source code debugging


Not so long ago Microsoft has made .NET source code browsable through a really nice page: http://referencesource.microsoft.com/. Additionally, they promised that the .NET Framework source code debugging will finally work in Visual Studio. At almost the same time JetBrains published EAP of its dotPeek tool with some great features that make “reverse-engineered debugging” extremely easy. And for other DLLs we still have the old Microsoft Public Symbols server. In this post I am going to show you how I configure my system and Visual Studio for different debugging scenarios.

Continue reading