While investigating the “Quick Mode SA idle timeout” problem mentioned in my previous blog A new IPSec Quick Mode Security Association is negotiated every 5 minutes when you use an IPSec tunnel mode connection on a Windows 2003 SP1 based server, we noticed that every time the ISA 2004 server initiate the negotiation of a new Quick Mode Security Association, the packet that triggered that event was lost. If we replaced the ISA 2004 with a Windows 2003 SP1 server as VPN endpoint, we didn’t saw that behavior.
Microsoft PSS further analysed this problem too. It turned out that in Windows Server 2003 SP1 some changes were made in the way the ipsec.sys driver handles traffic during an IKE negotiation. The bottom line is that the packets will be blocked and queued by the IPSec driver. There will be a special flag set on those packets and if the negotiation is over the packets are re-injected into the IP Stack. The Firewall engine Kernel Mode driver fweng.sys is currently not treating this special flag correctly and therefore drops the packets with error code FWX_E_FWE_SPOOFING_PACKET_DROPPED which is a bit misleading in this stage.
However, the ISA development team does not think this is a major area of concern since the IKE negotiation is not happening that often – at least if we assume that the “Quick Mode SA idle timeout” problem has been fixed – and that packet loss must be expected in a networked environment in anyway. Moreover, in all our repro’s we never had a single instance that a TCP connection was dropped due to this issue. The TCP Retransmission took care of the dropped packets. There will be certainly a performance hit for TCP connections due to the Slow Start Algorithm. However, the ISA development team does not see this as a justification for a fix at this moment.
To mitigate the problem, Microsoft PSS suggest to increase the timeout value for the IPSec Security Association Idle Timer to 3600 seconds. Sounds familiar, isn’t it? For more info, check out the Knowledge Base article KB917025 Error message in ISA Server 2004 when you configure an IPsec tunnel mode site-to-site VPN on an ISA Server 2004-based computer: “0xc0040014 FWX_E_FWE_SPOOFING_PACKET_DROPPED”.
BTW — I wonder if this issue has been fixed in ISA Server 2006 RTM. So, any comment on this is more than welcome.
HTH,
Stefaan