Fixing empty paths in FileIO events (ETW)

This month marks ten years since I started this blog πŸ₯‚πŸ₯‚πŸ₯‚. On this occasion, I would like to thank you for being my reader! Let’s celebrate with a new post on ETW πŸ™‚

Empty paths issue in the wtrace output has been bugging me for quite some time. As I started working on a new wtrace release (coming soon!), there came the right moment to fix it. I’ve seen other people struggling with this problem too, so I thought that maybe it’s worth a blog post πŸ™‚ Wtrace uses the TraceEvent library to interact with the ETW API, and in this post, I will use this library as well. Note that this issue affects only the real-time ETW sessions.

Continue reading

Decrypting PerfView’s OSExtensions.cs file

While analyzing the PerfView source code, I stumbled upon an interesting README file in the src/OSExtensions folder:

// The OSExtensions.DLL is a DLL that contains a small number of extensions
// to the operating system that allow it to do certain ETW operations.  
//
// However this DLL is implemented using private OS APIs, and as such should
// really be considered part of the operating system (until such time as
// the OS provide the functionality in public APIs).
//
// To discourage taking dependencies on these internal details we do not 
// provide the source code for this DLL in the open source repo. 
//
// IF YOU SIMPLY WANT TO BUILD PERFIVEW YOU DO NOT NEED TO BUILD OSExtensions!
// A binary copy of this DLL is included in the TraceEvent\OSExtensions.  
//*************************************************************************** 
// However we don't want this source code to be lost.  So we check it in
// with the rest of the code but in an encrypted form for only those few
// OS developers who may need to update this interface.   These people 
// should have access to the password needed to unencrpt the file.    
//
// As part of the build process for OSExtension.dll, we run the command 'syncEncrypted.exe'.
// This command keeps a encrypted and unencrypted version of a a file  in sync.
// Currently it is run on this pair
//
//  OSExtensions.cs   <-->  OSExtesions.cs.crypt
//
// Using a password file 'password.txt'  
//
// Thus if the password.txt exists and OSExtesions.cs.crypt exist, it will
// unencrypt it to OSExtesions.cs.   If OSExtesions.cs is newer, it will
// be reencrypted to OSExtesions.cs.crypt. 

Hmm, private OS APIs seem pretty exciting, right? A simple way to check these APIs would be to disassemble the OSExtensions.dll (for example, with dnSpy). But this method would not show us comments. And for internal APIs, they might contain valuable information. So let’s see if we can do better.

Continue reading

How Ansible impersonates users on Windows

Recently, I hit an interesting error during a deployment orchestrated by Ansible. One of the deployment steps was to execute a custom .NET application. Unfortunately, the application was failing on each run with an ACCESS DENIED error. After collecting the stack trace, I found that the failing code was ProtectedData.Protect(messageBytes, null, DataProtectionScope.CurrentUser), so a call to the Data Protection API. To pinpoint a problem I created a simple playbook:

- hosts: all
  gather_facts: no
  vars:
    ansible_user: testu
    ansible_connection: winrm
    ansible_winrm_transport: basic
    ansible_winrm_server_cert_validation: ignore
  tasks:
    - win_shell: |
        Add-Type -AssemblyName "System.Security"; \
        [System.Security.Cryptography.ProtectedData]::Protect([System.Text.Encoding]::GetEncoding(
            "UTF-8").GetBytes("test12345"), $null, [System.Security.Cryptography.DataProtectionScope]::CurrentUser)
      args:
        executable: powershell
      register: output

    - debug:
        var: output

When I run it I get the following error:

fatal: [192.168.0.30]: FAILED! => {"changed": true, "cmd": "Add-Type -AssemblyName \"System.Security\"; [System.Security.Cryptography.ProtectedData]::Protect([System.Text.Encoding]::GetEncoding(\n    \"UTF-8\").GetBytes(\"test\"), $null, [System.Security.Cryptography.DataProtectionScope]::CurrentUser)", "delta": "0:00:00.807970", "end": "2020-05-04 11:34:29.469908", "msg": "non-zero return code", "rc": 1, "start": "2020-05-04 11:34:28.661938", "stderr": "Exception calling \"Protect\" with \"3\" argument(s): \"Access is denied.\r\n\"\r\nAt line:1 char:107\r\n+ ... .Security\"; [System.Security.Cryptography.ProtectedData]::Protect([Sy ...\r\n+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException\r\n    + FullyQualifiedErrorId : CryptographicException", "stderr_lines": ["Exception calling \"Protect\" with \"3\" argument(s): \"Access is denied.", "\"", "At line:1 char:107", "+ ... .Security\"; [System.Security.Cryptography.ProtectedData]::Protect([Sy ...", "+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~", "    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException", "    + FullyQualifiedErrorId : CryptographicException"], "stdout": "", "stdout_lines": []}

The workaround to make it always work was to use the Ansible become parameters:

...
  tasks:
    - win_shell: |
        Add-Type -AssemblyName "System.Security"; \
        [System.Security.Cryptography.ProtectedData]::Protect([System.Text.Encoding]::GetEncoding(
            "UTF-8").GetBytes("test12345"), $null, [System.Security.Cryptography.DataProtectionScope]::CurrentUser)
      args:
        executable: powershell
      become_method: runas
      become_user: testu
      become: yes
      register: output
...

Interestingly, the original playbook succeeds if the testu user has signed in to the remote system interactively (for example, by opening an RDP session) and encrypted something with DPAPI before running the script.

It only made me even more curious about what is happening here. I hope it made you too πŸ™‚

Continue reading

Writing network proxies for development purposes in C#

If you are developing, testing, or supporting web applications, you probably encounter situations when you need to record or modify HTTP traffic. Quite often, the browser request viewer might be enough, but what if you need to modify the traffic on the fly? Another challenging task is testing how your application behaves when put behind a load balancer or an edge server. There are many great HTTP proxies available in the market, including mitmproxy, Burp Suite, or Fiddler and they may be perfect in diagnosing/testing your applications. In this post, however, I am encouraging you to write small tools for your specific needs. There are many reasons why you may want to do so, such as the need for complex requests modifications, better control over the request processing, or customizations of the certificate creation. Of course, implementing the HTTP protocol could be demanding so, don’t worry; we won’t do that πŸ™‚ Instead, we will use the open-source Titanium Web Proxy. The code samples in this post are meant to be run in LINQPad, which is my favorite tool for writing and running .NET code snippets, but you should have no difficulties in porting the samples to a C# script or a console application.

Continue reading

Synthetic types and tracing syscalls in WinDbg

Recently at work, I needed to trace several syscalls to understand what SQL Server was doing. My usual tool for this purpose on Windows was API Monitor, but, unfortunately, it hasn’t been updated for a few years already and became unstable for me. Thus, I decided to switch back to WinDbg. In the past, my biggest problem with tracing the system API in WinDbg was the missing symbols for the internal NT objects. Moreover, I discovered some messy ways to work around it. Fortunately, with synthetic types in WinDbg Preview it’s no longer a problem. In this post, I will show you how to create a breakpoint that nicely prints the arguments to a sample NtOpenFile syscall.

Continue reading