Prevent Denial of Service Attacks with Lockout Guard

One of the unmentioned issues regarding ISA Firewall security for published Web sites relates to the potential for denial of service attacks on user accounts due to pre-authentication at the ISA Firewall. How can this happen? If your company has implemented an account lockout policy for user accounts that fail to provide a correct password, attackers who can guess user names can use the pre-authentication feature provided by the ISA Firewall to lock out user accounts.

For example, suppose you published your OWA site and are using forms-based authentication at the ISA Firewall. An attacker might have looked at your corporate Web site and guessed some user names based on e-mail addresses included on your public Web site. The attacker can then try to log on using those user names. Of course, it’s unlikely that he’ll guess the correct password within your set number of failed log on attempts. So, while the attacker wasn’t able to get into the user’s mailbox, he was able to successfully lock that user’s account out for the timeout period you’ve configured in the Active Directory.

What would be nice is if you could set a special account lock out attempts number at the Web Listener that is lower than the one you set in the Active Directory. In this way, you would have a “soft” lock out for that user account at the ISA Firewall. While that account would not be able to log on via the Web Publishing Rule on the ISA Firewall, the account would not be locked out on the Internal network and could still access the network through a remote access VPN connection.

Well, the good news is that we now have this functionality! Collective Software has put together a fine Web authentication filter that gives you exactly this type of functionality. The product, known as LockoutGuard includes the following features:

  • LockoutGuard can be configured to deny authentication attempts from external users before the Active Directory limit is reached
  • While the account is locked out from external connections, this account is still available on the internal network and through remote access VPN connections
  • Attackers won’t be able to perform brute force attacks during the “soft” lockout at the ISA Firewall, as authentication attempts are automatically rejected once the custom lock out number if reached

LockoutGuard is simple to install and configure, and there are very few requirements. All you need is:

  • ISA 2006
  • Authentication to the Active Directory, which can be done by making the ISA Firewall a domain member, or if the ISA Firewall is not a domain member, you can use LDAP authentication
  • The Web Publishing Rule needs to use a Web Listener that’s configured to use either Basic or Forms-based Authentication
  • The Microsoft .NET Framework 2.0 needs to be installed on the ISA Firewall

In this article we’ll take a look at the installation and configuration process to show how easy it is to get things working with LockoutGuard.


First thing you’ll want to do is download the LockoutGuard App. You can run LockoutGuard as a demo version for a while before you decide to buy the app. Make sure you do the download to your workstation. Remember, you never want to use your ISA Firewall as a workstation, which means that you never use the Web browser on the ISA Firewall. After you download the app to your workstation, copy it to a USB key or CD and then take it to the ISA Firewall and copy it to the hard disk on the ISA Firewall.

Now that the LockoutGuard app is on the ISA Firewall, double click on the LockoutGuard.msi file. Make sure that the ISA Firewall console isn’t open! On the Welcome to the LockoutGuard Setup Wizard page, make sure there is a checkmark in the Stop/start service automatically checkbox so that the firewall service restarts automatically. Click Next.

Figure 1

On the End-User License Agreement page, put a checkmark in the I accept the terms in the License Agreement checkbox and click Next.

Figure 2

On the Choose Setup Type page, click the Complete button to install all the features of the product.

Figure 3

On the Ready to Install page, click the Install button.

Figure 4

Whoops! Me, being the typical ISA Firewall admin, forgot the system requirements. No problem! The installer checked the system for me and found that I forgot to install the .NET Framework 2.0. At this point I’ll stop the installation and install the .NET framework.

Figure 5

The wizard give me the chance to exit the installation so that I can install the .NET framework.

Figure 6

Now that I have the .NET Framework installed, I’ll restart the LockoutGuard app again. This time things go more smoothly.

Figure 7

Installation is successful!

Figure 8

Configure LDAP Server Settings on the ISA Firewall

While LockoutGuard is easy to install and configure, the hardest part of the setup is to configure the ISA Firewall to use LDAP authentication. Even if the ISA Firewall is a domain member, we still need to configure the LDAP settings. The reason for this is that the filter needs to learn the current failed password count for the user who is trying to authenticate, that this requires an LDAP lookup.

In the ISA Firewall console, go to the Configuration\General node in the left pane. On the General tab in the middle pane of the console, click the Specify RADIUS and LDAP Servers link.

Figure 9

In the Authentication Servers dialog box, click the LDAP Servers tab. As you see here, I have already added my LDAP servers. Click the Add button to add a new LDAP Server Set.

Figure 10

You will see a dialog box similar to this one. The first thing you need to do is add an LDAP server, which is a domain controller, in the domain that contains that users who will authenticate with the ISA Firewall. Click the Add button and add the name of the LDAP server. In the example you see below, I added the server This name is VERY important. The LDAP server must have a certificate installed in its machine certificate store that matches the name you put here. If the name you put here doesn’t match the common/subject name on the certificate, certificate authentication for the LDAPS connection will fail.

In addition, the ISA Firewall must have the CA certificate of the issuer of the server certificate that issued the certificate installed on the domain controller. The CA certificate needs to be installed in the ISA Firewall’s Trusted Root Certification Authorities machine certificate store. If the CA certificate isn’t installed on the ISA Firewall, the ISA Firewall won’t trust the server certificate presented by the DC and the LDAPS connection will fail.

You should add more than one domain controller to the list if possible. This way if the first DC is down, LockoutGuard can still use the next one.

Make sure you put in the name of the domain in the Type the Active Directory domain name text box. Enter only the domain name. Do not enter the FQDN of the domain controller.

Also, make sure that you put a checkmark in the Connect LDAP servers over a secure connection. This is sensitive information, and you don’t want to connect to the DC over an unsecured connection!

Finally, enter a user name that the ISA Firewall can use to connect to the LDAP server. This can be a normal user account and does not need to be a domain or enterprise admin account. However, make sure that this is not an account that has access from an external location, because ISA already has the correct password that you configured in the dialog box. Because of this, the LDAP user you specify here can never be locked out (each time ISA connects to LDAP its bad password count will be reset to zero!)

Figure 11

The last thing you need to do is configure the Login Expression. Click the New button and enter * for the expression and select the LDAP Server Set you created.

Figure 12

Configure Account Lockout in Group Policy

If you haven’t configured an Account Lockout Policy in Group Policy, then you should do that now. You can open the Group Policy Management Console on the domain controller (I’m using a Windows Server 2008 DC in this example), and then click on the Default Domain Policy as seen in the figure below. Then right click on it and click Edit.

Figure 13

This opens the Group Policy Management Editor for the default domain policy. Go to the Computer Configuration\Windows Settings\Security Settings\Account Policies\Account Lockout Policy node. Double click on the Account lockout threshold entry in the right pane and put a checkmark in the Define this policy setting checkbox. In this example we’ll set the threshold at 5. When you click OK, it will automatically set the account lockout duration and reset account lockout counter after settings, although you can change them from the defaults.

Figure 14

Create the Web Listener and Enable LockoutGuard

Now the LockoutGuard is installed on the ISA Firewall and with a lockout policy in place, we’ll create a Web Listener that will have LockoutGuard enabled on it.

In the ISA Firewall console, click the Firewall Policy node in the left pane of the console, and then click the Toolbox tab in the Task Pane. Click the Network Objects header and then click New and then click Web Listener.

Figure 15

On the Welcome to the New Web Listener Wizard page, enter a name for the Web Listener. We’ll make an SSL Listener in this example, we we’ll name it SSL Listener. Click Next.

Figure 16

On the Client Connection Security page, select the Require SSL secured connections with clients option and click Next.

Figure 17

On the Web Listener IP Addresses, I’ve selected a specific address on the external interface of the ISA Firewall for this Web Listener to accept incoming connections. Click Next.

Figure 18

Since SSL connections are required on this listener, we need to bind a certificate to this listener. Click the Select Certificate button and select a valid certificate. I see that I have a certificate named on this machine, so I’ll use that one.

Figure 19

Click Next on the Listener SSL Certificates page after binding the certificate to the Web Listener.

Figure 20

LockoutGuard only works for forms-based authentication or Basic authentication. So, in this example we’ll select HTML Form Authentication. Since this ISA Firewall is a domain member (I always make my ISA Firewalls domain members to benefit from enhanced security available only on domain member machines), we’ll use the Windows (Active Directory) option. Click Next.

Figure 21

We don’t need Single Sign on for this example, so we’ll remove the Enable SSO for Web site published with this Web listener checkbox. Click Next on the Single Sign On Settings page.

Figure 22

Click Finish on the Completing the New Web Listener Wizard page.

Figure 23

Click Apply to save the changes and update the firewall policy. Click OK in the Saving Configuration Changes dialog box.

In the Task Pane, click on the Toolbox tab. Click the Network Objects section header and click on the Web Listeners folder. Double Click on the SSL Listener Web Listener.

You’ll see a new tab on the SSL Listener Properties dialog box. This is the LockoutGuard tab. Here you put a checkmark in the Enable LockoutGuard checkbox. Then you put in the value for the Soft Lockout when bad password count is text box. In this example, we’ll set the soft lockout to be one less than the “hard” lockout we have in the Active Directory, which is 4. The Active Directory “hard” lockout value is 5.

Figure 24

Remember to apply the changes after making the change to the Soft Lockout value.

Just for fun, you can use the ldp.exe tool to check the lockout settings and status for the user account. In the example below, you can see that the user has a bad password count of 5 and the time when the account locked out appears here too.

Note that for an account protected by LockoutGuard, the bad password count value should not exceed the limit you entered above, no matter how many times the login form is submitted.

Figure 25

At this point LockoutGuard is enabled and will let you know if there is an account that is subjected to the software lockout through entries in the Alerts tab in the ISA Firewall console.

Figure 26


In this article we introduced the problem of potential denial of service attacks that can take place through authenticating Web Publishing Rules. In order to solve this problem, we needed a method that would allow us to set a “soft” lockout policy that would allow the ISA Firewall to stop authentication attempts for user accounts before the “hard” lockout number set in Active Directory is hit. This would prevent attackers from locking out user accounts and allow those accounts to still be accessed on the internal network and through RAS connections, while preventing further brute force attempts at that account through the ISA Firewall. We found that Collective Software’s LockoutGuard was the ideal solution for this problem. We then took a look at how to install and configure the software, including a brief discussion on how to configure LDAPS settings to support the LDAP lookups required by the LockoutGuard authentication filter.


Download LockoutGuard

About The Author

Leave a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top