In today’s post I will describe different security settings of the application pool and the IIS6 directory. It’s not always easy to guess which permissions must be set on system folders and files in order to make the application run correctly. I will also show you how to diagnose those pesky security problems using available tracing options.
Each process in Windows, including w3wp.exe (Application Pool), runs with a specific security identity. By default IIS6 uses the NetworkService account. You can change this setting for a particular pool by using its properties dialog. The next security option (Directory Security) available in IIS configures which credentials will be used to access physical files and folders of the application. Finally you can define some custom security settings for your ASP.NET application (including Basic/Forms/Windows authentication, roles authorization etc.). In this post I will focus mainly on the first two options, mentioning ASP.NET security only when necessary.
Application Pool Identity
Application pool identity can be set in its properties dialog. You can either select one of the system accounts or choose a custom one. We will a create the testpool account and use the second option. A detailed tutorial about how to create an application pool account can be found here. For our first exercise LEAVE THE PASSWORD BOX EMPTY as shown on the screenshot below:
Let’s now create an ASP.NET application that will use this application pool. The application will be composed of two files:
- Default.aspx
<%@ Page Language="C#" AutoEventWireup="true" %> <%@ Import Namespace="System.Threading" %> <%@ Import Namespace="System.Security.Principal" %> <!DOCTYPE html> <html> <head> </head> <body> <%= "ASP.NET logged user: " + (User != null ? User.Identity.Name : "anonymous") %><br/> <%= "ASP.NET process identity: " + (WindowsIdentity.GetCurrent().Name) %> </body> </html> - web.config
<?xml version="1.0"?> <configuration> <system.web> <compilation debug="true" /> <customErrors mode="Off" /> </system.web> </configuration> <configuration>
After creating a new web application in IIS (I will use port 8090) enable anonymous access for it using one of the system user accounts (I will use testdir user in my sample):
Finally deploy the application to some folder (I will use c:\websites\sectest) and apply full permissions on it only to the Administrators group (don’t bother yet about testpool or testdir accounts). You can check permissions using icacls command:
C:\websites>icacls sectest sectest BUILTIN\Administrators:(OI)(CI)(F) Successfully processed 1 files; Failed processing 0 files
503 Service Unavailable
We are now ready to launch the browser:
As you can see we have 503 – Service Unavailable error message. You may already know what’s wrong, but let’s check which logs may provide some insight into the situation. There is no info about this error in the IIS log (c:\WINDOWS\system32\LogFiles\W3SVC1716510118\ex120815.log on my machine) which means that it’s severe enough to have a line in http.sys log. Quick look at c:\WINDOWS\system32\ogFiles\HTTPERR\httperr1.log confirms our suspicions:
2012-08-15 08:30:11 127.0.0.1 1072 127.0.0.1 8090 HTTP/1.1 GET / 503 1716510118 AppOffline TestPool
The Reason Phrase states: AppOffline which indicates (after http://support.microsoft.com/?id=820729):
A service unavailable error occurred (an HTTP error 503). The service is not available because application errors caused the application to be taken offline.
We can check that the pool is offline either in IIS manager or using iisapp /a TestPool command. So the next question is: why the application became offline? An answer to it can be found in the System event log:
Event Type: Error Event Source: W3SVC Event Category: None Event ID: 1059 Date: 8/15/2012 Time: 11:04:20 AM User: N/A Computer: WIN2003SRV Description: A failure was encountered while launching the process serving application pool 'TestPool'. The application pool has been disabled. For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Following it there should be a few warning logs with the following content:
Event Type: Warning Event Source: W3SVC Event Category: None Event ID: 1021 Date: 8/15/2012 Time: 11:04:18 AM User: N/A Computer: WIN2003SRV Description: The identity of application pool, 'TestPool' is invalid. If it remains invalid when the first request for the application pool is processed, the application pool will be disabled. The data field contains the error number. For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp. Data: 0000: 8007052e
We can easily find that the error 8007052e description is Logon Failure: unknown user name or bad password. So we figured out that the problem lied in either incorrect username or password of the application pool user. If we have a login failure audits enabled in our system an adequate log entry should appear also in the Security Event Log:
Event Type: Failure Audit Event Source: Security Event Category: Account Logon Event ID: 680 Date: 8/16/2012 Time: 9:22:15 PM User: NT AUTHORITY\SYSTEM Computer: WIN2003SRV Description: Logon attempt by: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0 Logon account: testpool Source Workstation: WIN2003SRV Error Code: 0xC000006A
500 Server Application Unavailable
Let’s fill the password box for the application pool account, move back to the browser window and press refresh. We should now receive 500 Server Application Unavailable error:
This time we have info about this error in IIS log:
2012-08-16 19:25:53 W3SVC1716510118 127.0.0.1 GET / - 8090 - 127.0.0.1 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.2;+Trident/4.0;+.NET+CLR+1.1.4322;+.NET4.0C;+.NET4.0E) 500 0 0
Unfortunately, except for an error number we don’t get much information. Error number 500 usually indicates that the error lies in the application itself so let’s check if ASP.NET Health Monitoring logged something (by default ASP.NET is configured to log any error messages to the Application Event Log). There are two new entries describing why the application failed to start:
Event Type: Error Event Source: ASP.NET 4.0.30319.0 Event Category: None Event ID: 1088 Date: 8/16/2012 Time: 9:56:31 PM User: N/A Computer: WIN2003SRV Description: Failed to execute request because the App-Domain could not be created. Error: 0x80070005 Access is denied.
Event Type: Error Event Source: ASP.NET 4.0.30319.0 Event Category: None Event ID: 1325 Date: 8/16/2012 Time: 9:56:30 PM User: N/A Computer: WIN2003SRV Description: Failed to initialize the AppDomain:/LM/W3SVC/1716510118/Root Exception: System.IO.FileLoadException Message: Could not load file or assembly 'System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. Access is denied. StackTrace: at System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks) at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks) ... at System.Web.Hosting.ApplicationManager.CreateAppDomainWithHostingEnvironment(String appId, IApplicationHost appHost, HostingEnvironmentParameters hostingParameters) at System.Web.Hosting.ApplicationManager.CreateAppDomainWithHostingEnvironmentAndReportErrors(String appId, IApplicationHost appHost, HostingEnvironmentParameters hostingParameters)
From the above log we can see that we got Access Denied while creating appdomain in the hosting environment. As the ASP.NET process is running with the same credentials as the Application Pool we might guess it’s the testpool account that is missing permissions on c:\websites\sectest folder. Let’s add Read Permissions for the testpool account:
C:\websites>icacls sectest
sectest BUILTIN\Administrators:(OI)(CI)(F)
WIN2003SRV\testpool:(OI)(CI)(R)
Successfully processed 1 files; Failed processing 0 files
and reload the browser once again
Directory Security
401.5 Access denied
You should receive a slightly different Access Denied error. In IIS log we can check that its code is 401.5:
2012-08-16 22:32:26 W3SVC1716510118 127.0.0.1 GET / - 8090 - 127.0.0.1 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.2;+Trident/4.0;+.NET+CLR+1.1.4322;+.NET4.0C;+.NET4.0E) 401 5 0
The subcode is really important and informs us that Authorization failed by ISAPI/CGI application (after http://blog.crowe.co.nz/archive/2006/06/15/231.aspx). In this case we have no information in the Event Log so we need to try something more verbose. Here Windows event tracing comes into play. We will use the general “IIS: WWW Server” provider (more on providers and IIS6 tracing can be found on this technet site):
C:\temp>logman start iistrace -p "IIS: WWW Server" 0xFFFFFFFE 0x5 -o test4.etl -
ets
Name: iistrace
Age Limit: 15
Buffer Size: 8
Buffers Written: 1
Clock Type: System
Events Lost: 0
Flush Timer: 0
Buffers Free: 2
Buffers Lost: 0
File Mode: Sequential
File Name: C:\temp\test4.etl
Logger Id: 3
Logger Thread Id: 2952
Maximum Buffers: 25
Maximum File Size: 0
Minimum Buffers: 3
Number of buffers: 3
Real Time Buffers Lost: 0
Provider Flags Level
-------------------------------------------------------------------------------
* "IIS: WWW Server" (IISAuthentication,IISSecurity,IISFilter,IISStaticFile,IISCGI,IISCompression,IISCache,IISAll) 0x05
{3A2A4E84-4C21-4981-AE10-3FDA0D9B0F83} 0xfffffffe 0x05
The command completed successfully.
Just after the above command finished, repeat the failing request and issue C:\temp>logman stop iistrace -ets to stop the tracing session. A new trace4.etl should be created. We can convert it to, for example, .csv file using Log Parser (we can also use tracerpt or any other tool that can read .etl files):
C:\temp>"c:\tools\logging\Log Parser 2.2\LogParser.exe" "select * from test4.etl" -o:CSV > test4.csv
The generated file looks as follows (for brevity I copied only the interesting lines):
16,IISAuthentication,AUTH_START,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|AuthTypeSupported=Anonymous
17,IISAuthentication,AUTH_REQUEST_AUTH_TYPE,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|RequestAuthType=Anonymous
18,IISAuthentication,AUTH_SUCCEEDED,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|AuthType=Anonymous|NTLMUsed=0|RemoteUserName=|AuthUserName=|TokenImpersonationLevel=0x00000002
19,IISAuthentication,AUTH_END,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}
20,IISISAPI,ISAPI_START,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}
21,IISGeneral,GENERAL_ISAPI_HANDLER,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}
22,IISCache,URL_CACHE_ACCESS_START,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|RequestURL=/
23,IISGeneral,GENERAL_GET_URL_METADATA,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|PhysicalPath=C:\websites\sectest\|AccessPerms=Read+Exec+Script
24,IISCache,URL_CACHE_ACCESS_END,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|PhysicalPath=C:\websites\sectest\|URLInfoFromCache=0|URLInfoAddedToCache=0|ErrorCode=0x00000000
25,IISCache,HTTPSYS_CACHEABLE,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|HttpsysCacheable=0|Reason=URL_CHANGE_BY_FILTER
26,IISISAPI,ISAPI_END,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}
27,IISFilter,FILTER_START,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|FilterName=c:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\\aspnet_filter.dll
28,IISFilter,FILTER_LOG_START,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|OrigClientHostName=127.0.0.1|OrigClientUserName=|OrigServerName=127.0.0.1|OrigOperation=GET|OrigTarget=/eurl.axd/8b86b9faa4046348912ac0adbc7fb2c7/|OrigParameters=|OrigHttpStatus=401|OrigWin32Status=0
29,IISFilter,FILTER_LOG_END,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|FinalClientHostName=127.0.0.1|FinalClientUserName=|FinalServerName=127.0.0.1|FinalOperation=GET|FinalTarget=/|FinalParameters=|FinalHttpStatus=401|FinalWin32Status=0
30,IISFilter,FILTER_END,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}
31,IISGeneral,GENERAL_REQUEST_END,2012-08-16 23:02:30,ContextId={00000000-0000-0000-0600-0060000000fb}|BytesSent=2032|BytesReceived=249|HttpStatus=401|HttpSubStatus=5
The AUTH_SUCCEEDED event informs us that IIS (and thus ASP.NET) is accessing files using Anonymous account (the one that we configured on the Directory Security tab for our application) and from the last GENERAL_REQUEST_END we can read that it did not succeed.
200 OK
Let’s then grant Read Permissions to the testdir account:
C:\websites>icacls sectest
sectest BUILTIN\Administrators:(OI)(CI)(F)
WIN2003SRV\testdir:(OI)(CI)(R)
WIN2003SRV\testpool:(OI)(CI)(R)
Successfully processed 1 files; Failed processing 0 files
and reload the browser for the last time:) :
I hope that you will benefit from information provided in this post in your own IIS security struggles. In further posts I have a plan to dwell on a WCF activation process and IIS7 tracing options so stay tuned 🙂 If you have any questions or maybe better ideas for tracing issues presented in today’s post please drop me a message or leave a comment.
Try to remove “testdir” from ACL and use ASP.NET MVC app (it would probably also work if you’d use Begin_Request event, output some HTML there and invoke Response.End()). You will see proper HTML but static content will get 401 (or 302->404 if you use Forms auth). It seems, that this is because static content is delegated back to IIS and “testdir” has no rights, and it seems only….
To see static content without “testdir” ACL set on app folder try:
– use IIS Windows Integrated, Windows Auth in ASP.NET and you provide credentials that have access to folder – thats seems to make sense and to be obvious
– use IIS Windows Integrated and NONE authentication (identity impersonation is not relevant here) – browser should ask you for credentials, if you provide them (and have access) – static content will arrive with HTTP 200; that also looks right, since static content seems to be requested from IIS directly
– now the best part: use Forms authentication in ASP.NET and login using whatever login – you will see static content! If you log off (from forms) – content disappears (302 and then 404 response). Why?
I tested on IIS 7.5 and I wonder what would happen on IIS 6 in these cases…
Have you ever thought about creating an ebook or guest authoring on other blogs?
I have a blog based upon on the same topics you discuss and would really like to
have you share some stories/information. I know my readers would appreciate your work.
If you’re even remotely interested, feel free to send me an email.