Recently I have spent some time on learning the internals of HTTPS. I wanted to know what makes it secure and how the communication actually looks like. Today I would like to show you the steps required to decrypt a sample HTTPS request. Imagine you got a .pcap file recorded by one of your company clients who complains that your application returned 500 HTTP status code with a strange error message. The client forgot to copy the error message but luckily had a Wireshark instance running in the background (I know it’s highly hypothetical, but just close your eyes to that :)) and he/she sent you the collected traces. Let’s then assume that your server has a certificate with a public RSA key and you are in possession of its private key. Finally the client was using a slightly outdated browser which supports TLS 1.0 (though I will inform you what would have been different if it had been TLS 1.2) and does not use ephemeral keys. My main point in writing this post is to present you the steps of the TLS communication. This post is not a guidance on how to create a secure TLS configuration, but a walk-through on how this protocol works and I will purposely use less secure ciphers to make things easier to explain.
Some time ago Sasha Goldstein wrote on his blog a post about a new open-source tool he created: minidumper. The idea is great and allows you to create more compact memory dumps for .NET processes. I completely agree with Sasha that most of the time when you are working with .NET processes memory dumps you are focused on .NET memory heap. All native elements are usually not necessary and take a lot of space in dumps. Having said that, I need to admit that there were times when those native parts were invaluable in my diagnosis, such as diagnosing native memory leaks (when using pinvoke) or handle leaks. But those are quite rare and in such situations we have no choice but to use the full memory dumps. For all other .NET problems you are welcome to use minidumper. If you are interested in minidumper internals there is another post on Sasha’s blog which explains them very well.
Today’s short post was inspired by a recent memory leak in Nancy. I thought it’s worth to describe it in detail as the reason why the memory was leaking was not so obvious and many of us could commit the same mistake as Nancy authors. The leak was present in the
NancyEngine class, which is the central point in the Nancy request handling process. In other words: each request served by the Nancy application must pass through this class instance.
NancyEngine processes the requests asynchronously and accepts as a parameter a
CancellationToken instance – thus making it possible to cancel the request from the “outside”. At the same time it uses an internal
CancellationTokenSource instance which cancels the current requests when the engine is getting disposed. As you see there are two cancellation tokens involved and the
HandleRequest method needs to respect their statuses. For such an occasion there is a method in the
CancellationTokenSource class in .NET Framework which creates for you a special “linked”
CancellationTokenSource that will depend on values in the related tokens. From now on you don’t need to worry about the other tokens as whenever they get cancelled your linked token will become cancelled too. With this introduction the prologue of the
HandleRequest becomes clear:
Recently while examining a slow request issue (I have a plan to describe this investigation in a seperate post) it came to me that every time I open the Thread Time view it takes a moment to understand what this view actually contains. I decided to write this post for me and for any of you who share the same feeling about this window :).
Today I would like to share with you an interesting (I hope) diagnostics case in one of our system services. The IngestService (that is its name) was not starting properly for the first time – it was being killed because of exceeding the default 30s timeout. But the second try was always successful. No exception was thrown and no logs could be found in the event logs. It’s a situation when ETW traces might shed some light on what’s going on. As it was a .NET service I used PerfView to record the trace file. An important checkbox to select when diagnosing thread wait times is the Thread Time box:
After collecting the traces on production, I merged them and copied to my developer machine.
I have been playing recently with quite a new windbg extension (released by Rodney Viana from Microsoft) called NetExt. Rodney Viana published an introductory post about it, which you may find on his blog. In this post I would like to show you my usage samples as well as encourage you to start using it by yourself. Netext documentation is thorough and nicely organized which is good because at the beginning you probably will spend a lot of time on this page 🙂 In paragraphs that follow I will focus mainly on dump debugging, but most of the techniques presented here should work as well in live debugging sessions.
This post is the second and final one dedicated to debugging .NET Windows services (you may read the first one here). The inwrap tool (presented in the first part) is not very friendly to use and I myself haven’t used it since then 🙂 It’s not the best advertisement of someone’s own work, but it did motivate me to write another tool which maybe will gain your interest. The winsvcdiag is a simple application that allows you to debug a start of a Windows service from Visual Studio (or any other debugger – even the remote one).
I recently had an interesting issue in one of our applications. The SMS router, responsible for sending and receiving SMSes, hanged – there was no CPU usage and we haven’t observed any activity in the application logs. I collected a full memory dump and restarted the service, which seemed to come back to its normal state. Curious what happened I opened the dump in WinDbg, loaded PDE and SOS and started an investigation
In one of our applications I recently observed timeouts in code performing HTTP requests to the REST service. While investigating this issue I discovered few interesting facts about System.Net namespace and would like to share them with you. We were using objects of type System.Net.HttpWebRequest in our code, but some of the information presented in this post will also apply to the newer System.Net.HttpClient implementation.
This one would be short 🙂 While learning new things I write notes, collect help files and sample code. I use my Google Drive to store them. I have decided recently that some of the folders may be worth publishing and this is how Debug Recipes repository was born. I have a plan to store in it:
- instructions how to resolve various problems in applications
- instructions how to use debugging/profiling tools and features available in them (example for Visual Studio)
- scripts that may help you in debugging (example for collecting .NET exceptions in production)
I’m still working on a better navigation (each section will have a README.md file), but for now the Github search and folder navigation are the only options. As you can imagine it will always be a work in progress, but I hope that some recipes will prove useful to you. As always comments and suggestions are welcome.