Synthetic types and tracing syscalls in WinDbg


Recently at work, I needed to trace several syscalls to understand what SQL Server was doing. My usual tool for this purpose on Windows was API Monitor, but, unfortunately, it hasn’t been updated for a few years already and became unstable for me. Thus, I decided to switch back to WinDbg. In the past, my biggest problem with tracing the system API in WinDbg was the missing symbols for the internal NT objects. Moreover, I discovered some messy ways to work around it. Fortunately, with synthetic types in WinDbg Preview it’s no longer a problem. In this post, I will show you how to create a breakpoint that nicely prints the arguments to a sample NtOpenFile syscall.

Continue reading

A story of fixing a memory leak in MiniDumper


MiniDumper is a diagnostic tool for collecting memory dumps of .NET applications. Dumps created by MiniDumper are significantly smaller than full-memory dumps collected by, for example, procdump. However, they contain enough information to diagnose most of the issues in the managed applications. MiniDumper was initially developed by Sasha Goldstein, and I made few contributions to its code base. You may learn more about this tool from Sasha’s or my blog posts.

Recently, one of MiniDumper users reported a memory leak in the application. The numbers looked scary as there was a 20MB leak on each memory dump. The issue stayed opened a few weeks before I finally found a moment to look at it. As it was quite a compelling case, I decided to share with you the diagnostics steps in the hope it proves useful in your investigations.

Continue reading

Debug.Assert Everyone!


Every developer knows that unit testing improves the quality of the code. We also profit from static code analysis and use tools such as SonarQube in our build pipelines. Yet, I still find that many developers are not aware of a much older way of checking the validity of the code: assertions. In this post, I will present you the benefits of using assertions, as well as some configuration tips for .NET applications. We will also learn how .NET and Windows support them.

Continue reading

Few facts about OutputDebugString and default settings of the System.Diagnostics.Trace


I recently spent some time analyzing OutputDebugString method. For my another experiment I needed a version of OutputDebugString which depends only on Native API. While implementing it, I discovered few interesting facts about OutputDebugString that maybe will interest you too. The title mentions System.Diagnostics.Trace. It is because the default trace configuration in .NET sends trace messages to an instance of the DefaultTraceListener class, which uses OutputDebugString. And if you do not remove it explicitly from the trace listener collection, your logs will always go through it. You will later see why sometimes it might not be a good idea.

Continue reading

How to decode managed stack frames in procmon traces


Recently, the idea of protected variables in TFS struck my attention and pushed me to do some more research on how exactly those variables are stored. I hope I will write a separate post on that subject, but today I would like to share with you a small trick I use whenever I need to work with managed application traces (and TFS is one of them).

On Windows, when I want to know how things work internally, I usually start with procmon. Seeing which paths and registry keys are accessed, combined with TCP/IP connections is often enough to get an idea where to put breakpoints in further analysis. My TFS investigation was no exception to this rule. I collected a trace while saving a protected build variable – this is how such a variable looks like (in case you are interested :)):

tfs_protected_variable

Continue reading

When procmon trace is not enough


Process Monitor (or procmon) is a very powerful diagnostics tool for Windows. I remember when I first run it, a few years ago, I was overwhelmed by the number of events happening on my system each and every second. Thankfully, procmon has features to help you deal with this bulk of data, including filters, highlighters, and bookmarks. But sometimes even a filtered log is not enough to find the root cause of a problem. On such occasions, other log sources may contain the missing bits of information. We only need to correlate them with our procmon trace. And today, I would like to show you how I do that.

Continue reading

How dependency calls are tracked by the Application Insights


Application Insights is a performance monitoring service, created by Microsoft and available on Azure. It gives you space to store the performance metrics and logs of your application (1GB for free!), as well as functionalities to search and manage them. In this post I am not going to present you the whole platform – Microsoft already did it in the Azure documentation, but rather focus on an element of the log collection, named dependency calls tracking. I did some analysis on the Application Insights libraries, and decided to publish my findings, in the hope that the results might interest some of you too.

Dependency calls are requests, which your application makes to the external services (such as databases or REST services). When this telemetry type is enabled, all the dependent actions form a timeline within the scope of the parent action. Using this timeline we may easily verify whether the delay in our application is caused by an external service, or the application itself. Let’s analyze in detail how this data is collected.

Continue reading

‘AutoMapper’ already has a dependency defined for ‘Microsoft.CSharp’


This is how the story begins. On our build server we are using a JetBrains Resharper CLT to generate a code analysis report. In one of the projects build we started getting the following exception log:

Executing the powershell script: C:\install\TFS \1.0.691\resharp.ps1
JetBrains Inspect Code 2016.1.2
Running in 64-bit mode, .NET runtime 4.0.30319.42000 under Microsoft Windows NT 6.2.9200.0
'AutoMapper' already has a dependency defined for 'Microsoft.CSharp'.
--- EXCEPTION #1/2 [InvalidOperationException]
Message = "'AutoMapper' already has a dependency defined for 'Microsoft.CSharp'."
ExceptionPath = Root.InnerException
ClassName = System.InvalidOperationException
HResult = COR_E_INVALIDOPERATION=80131509
Source = NuGet.Core
StackTraceString = "
  at NuGet.Manifest.ValidateDependencySets(IPackageMetadata metadata)
     at NuGet.Manifest.ReadFrom(Stream stream, IPropertyProvider propertyProvider, Boolean validateSchema)
     at NuGet.LocalPackage.ReadManifest(Stream manifestStream)
     at NuGet.OptimizedZipPackage.EnsureManifest()
     at NuGet.SharedPackageRepository.OpenPackage(String path)
     at NuGet.LocalPackageRepository.GetPackage(Func`2 openPackage, String path)
     at NuGet.LocalPackageRepository.<>c__DisplayClass13.<FindPackage>b__f(String path)
     at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
     at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
     at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)
     at NuGet.LocalPackageRepository.FindPackage(Func`2 openPackage, String packageId, SemanticVersion version)
     at NuGet.SharedPackageRepository.FindPackage(String packageId, SemanticVersion version)
     at JetBrains.ProjectModel.Packages.SharedPackageRepositoryInTemp.FindPackage(String packageId, SemanticVersion version)
     at NuGet.PackageRepositoryExtensions.FindPackage(IPackageRepository repository, String packageId, SemanticVersion version, IPackageConstraintProvider constraintProvider, Boolean allowPrereleaseVersions, Boolean allowUnlisted)
     at NuGet.PackageReferenceRepository.GetPackage(PackageReference reference)
     at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
     at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
     at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
     at JetBrains.ProjectModel.Packages.NuGetSolutionManager.<>c__DisplayClass7.<GetInstalledPackages>b__6()
     at JetBrains.Util.ILoggerEx.Catch[TValue](ILogger th?s, Func`1 F, ExceptionOrigin origin, LoggingLevel loggingLevel)

Continue reading

Enumerating AppDomains in a remote process


I am working on adding a support for ASP.NET performance counters into Musketeer. Compared to other .NET performance counters they have quite surprising instance names. ASP.NET developers decided that their performance counter instances will be identified by names derived from the AppDomain names (more information can be found here). This is probably due to a fact that one process may host multiple ASP.NET applications, thus one counter instance per process won’t be enough. Consequently, in order to match collected metrics with process ids we need to know which AppDomain belongs to which process. How can we do that?

Continue reading