On the occasion of releasing wtrace 2.2, I decided to write a short post about new functionalities I added to this tool in the recent months. I hope you will find them interesting. Wtrace is a command line application which collects ETW traces from the system and the selected processes and outputs them to the console. It is very simple to use and runs on Windows 7+. Currently, it supports the collection of File I/O, TCP, ALPC, RPC, ISR, DPC, and PowerShell events.
In today’s short post I would like to present you a new tool in my diagnostics toolkit: wtrace, and an update to procgov (or Process Governor). Let’s start with wtrace.
On Linux, when I need to check what a given process is doing, I usually use strace. I was always missing such a tool for Windows. We have procmon (which is great), but it does not run in a console, and thus can’t be used in the command line scripts, or on a Nano server. This might change soon, as in one of the latest episodes of the Defrag Tools show, Mark Russinovich shared the plan of releasing the procmon version for Nano. Till then though we don’t have much choice when it comes to real-time tracing. You may think of xperf or wpr, but those tools only record ETW events for further analysis. However, we may use the same ETW events in a realtime session, and print information they provide to the console output. This is how the idea for wtrace was born in my head. Few weeks ago Sasha Goldshtein released another tool for ETW processing named etrace, which basically does something very similar and has many interesting options. I decided to publish wtrace nonetheless, as my point was to create a tool with an extremely simple interface. Wtrace is collecting only a small subset of events (FileIO, TcpIp, Process/Thread Start) from the kernel provider. It may either start a process, or trace one that is already running. At the end of the trace it also shows some statistics (unless you use the –nosummary switch). Trace session will end either when you press Ctrl+C, or when the traced process terminates. Events are printed in the console window. An example session might look as follows:
For some time I have been working on a monitoring solution for developers. Today I have a pleasure to announce its first official release. It is a set of tools which should help you better diagnose your applications. As there are many monitoring solutions on the market you may be using one of them (and that’s great). However, I’ve observed that it’s still uncommon for developers to collect application logs in one place. Therefore OPS monitor IIS logs and developers are checking application-specific targets. This is not the best approach as you can’t see at first sight if something is going wrong with your application. Few years ago I had an idea of an application board which will show statuses of applications on all the servers. This is one of the central part of the Diagnostics Kit and I named it the Diagnostics Castle. A sample board might look as follows:
Some time ago Sasha Goldstein wrote on his blog a post about a new open-source tool he created: minidumper. The idea is great and allows you to create more compact memory dumps for .NET processes. I completely agree with Sasha that most of the time when you are working with .NET processes memory dumps you are focused on .NET memory heap. All native elements are usually not necessary and take a lot of space in dumps. Having said that, I need to admit that there were times when those native parts were invaluable in my diagnosis, such as diagnosing native memory leaks (when using pinvoke) or handle leaks. But those are quite rare and in such situations we have no choice but to use the full memory dumps. For all other .NET problems you are welcome to use minidumper. If you are interested in minidumper internals there is another post on Sasha’s blog which explains them very well.
In today’s short post I would like to present you three tools that I use frequently in diagnosing services that use MS Message Queues. These are:
- MessageDumper – downloads and removes messages from queue
- MessagePeeker – downloads but does not remove messages from queue
- MessagePusher – sends collected messages to a given queue
MessageDumper and MessagePeeker gather messages in batches, storing each batch in a separate file. The size of the batch and the number of files is configurable from the command line. Output files can be then processed by MessagePusher and send to a different queue, for example on a developer’s machine.
Case of diagnostics
Imagine you have a production Windows Service that processes statistics. Statistics are generated by web applications on people actions and sent to your service using MS Message Queues (let’s assume the service queue is private and its name is LowLevelDesign.Stats). Some day you observe that for a specific set of statistics messages your service breaks. In order to debug the issue locally you need those messages. You may then ask your admin to stop the service, wait for the statistics to come and run:
MessagePeeker -q .\private$\LowLevelDesign.Stats -o brokenset
This command should generate 2 files on output: brokenset.headers and brokenset.1. The first one is a header file which contains information about messages stored in other files. Copy the generated files to your local machine and run:
MessagePusher -q .\private$\LowLevelDesign.Local.Stats -i brokenset
and all the saved messages will be sent to your local queue. As said previously when you have many messages to process you may gather them in batches. The presented tools are available for download on my .NET Diagnostics Toolkit page.
Today I would like to introduce you to Process Governor – a new tool in my .NET diagnostics toolkit. This application allows you to set a limit on a memory committed by a process. On Windows committed memory is actually all private memory that the process uses. I wrote this tool to test my .NET applications (including web applications) for memory leaks. With it I can check if under heavy load they won’t throw OutOfMemoryException.
Some time ago I developed a simple diagnosing tool to monitor our production SQL Server and today I have a pleasure to present it to the public:) It’s a web application that you can deploy on IIS or run locally on IIS Express. On the main page it shows all currently running requests and open sessions. The screenshot in this case will serve better than a text description so just have a look: