This tip was submitted by Jack Johnson who has been working with Platforms Dedicated Support Engineering (DSE) at Microsoft for over fourteen years.
A Cautionary Tale of Server Hardening
Server hardening is a key component to the infrastructure of many enterprises. It is widely used and very effective against preventing attacks. One of the main tools of server hardening is to block high RPC ports. There are the ports normally in the 5000 and up range. However, careful planning should be taken to ensure the enterprise doesn’t “shoot itself in the foot” so to speak when doing this. I came across one sure situation where a company was having issues with account lockouts. Based on the data, the account lockouts were due to bad passwords.
Viewing a Netlogon.log file would show something similar to this:
12/12 09:53:31 [LOGON] US: SamLogon: Transitive Network logon of US\User1 from WORKSTATION23 (via DOMAINCONTROLLER45) Returns 0xC000006A
So once we started to dig into the issue we discovered that the users were actually trying to change their passwords when this issue was occurring. Once the password was successfully changed, they were being prompted by their Outlook client for the correct password. They enter the password that was just changed but it doesn’t work and now their account is locked out. So why did they account get locked out after only one “bad password”? The answer is in the screen capture below.
So in taking a look at a connection from Outlook we see multiple connections being made via RPC. The multiple connections are all using the port of 6001 which is one of those higher ports that are usually blocked via server hardening. There are 3 connections shows here but I have seen as many as 5-7 different connections. So if all of these connections are sending the same bad password, then it’s possible each connection is being counted as a bad password, therefore exceeding the count and locking the user out.
So why is the password is considered “bad” when it was just changed on the users workstation successfully? Well, when a password is changed, the change is sent to the authenticating domain controller. If that domain controller is the PDCE (Primary Domain Controller Emulator), then the password change is recorded. If the authenticating domain controller is not the PDCE then that DC will forward the change to its PDCE. However, that forward is done via RPC on one of those high ports we just mentioned. But wait, those ports are blocked via server hardening. Correct, which is why the change is not successfully forwarded to the PDCE immediately so as far as the PDCE knows, the user’s password has not changed. The PDCE does get the change eventually but only after normal replication has taken place. That was the other piece to this problem, the user’s accounts would NOT get locked out if they waited for an hour or so, hence the normal replication cycle.
Once we had all the pieces to the puzzle, we were able to create a static port to allow the domain controller to forward the password change to the PDCE without breaking the rules of server hardening. This type of issue is one that could easily be overlooked when constructing your server hardening plan so be mindful and design accordingly.
The above tip was previously published in an issue of WServerNews, a weekly newsletter from TechGenix that focuses on the administration, management and security of the Windows Server platform in particular and cloud solutions in general. Subscribe to WServerNews today by going to http://www.wservernews.com/subscribe.htm and join almost 100,000 other IT professionals around the world who read our newsletter!
Mitch Tulloch is an eleven-time recipient of the Microsoft Most Valuable Professional (MVP) award and a widely recognized expert on Windows Server and cloud computing technologies. Mitch is also Senior Editor of WServerNews. For more information about him see http://www.mtit.com.
