An IPSec tunnel mode connection is not implemented as a routable interface on a Windows based server

Imagine you have to connect to a Partner Network with an IPSec tunnel mode VPN connection. For performance and/or reliability reasons you do so through a dedicated Internet or leased line connection. For this scenario we can draw the following simplified network diagrams:

The important characteristic of the above scenario is that the traffic destined for the Partner Network will not follow the default gateway routing path which is the Internet connection (RT1 in the above diagrams). At first sight you would expect that you only have to define a persistent static route for the VPN gateway at the Partner Network (the outer IP address) pointing to the router (RT2 in the above diagrams) handling that connection to the Partner Network. However that’s not enough in the case of an IPSec tunnel mode connection. Although the IPSec tunnel mode connection itself will succeed, the traffic destined for the Partner Network (the inner IP addresses) will not reach the Partners VPN gateway (the outer IP address) unless you have also defined a persistent static route pointing to the router (RT2 in the above diagrams) handling the connection to the Partner Network for all the destinations (the inner IP addresses) reachable through the Partners VPN gateway (the outer IP address).

The reason for this is the following. When IPsec was architectured for Windows 2000, it turned out that the IPsec RFC’s were rather vague regarding IPSec tunnel mode. Microsoft worked together with other vendors implementing IPsec tunnel mode and found that half of them were implementing the IPsec tunnel as a routable interface, while the other half was not. The goals of the Microsoft implementation were to interoperate with as many other implementations as possible, to perform as well as possible, and to work in as many scenarios as possible, to name a few. So the decision was made that IPsec tunnel mode should not be implemented as a routable interface. When the architecture was done for Server 2003 this decision was upheld. 

What does this mean? It means that for outbound traffic the IP routing decision is taken before any IPSec processing. To understand it better, I strongly suggest you read the Microsoft Cable Guy article TCP/IP Packet Processing Paths dated June 2005. For the above scenario it means that if no static route is defined for the destinations reachable through the Partners VPN gateway, then Windows will choose the default gateway as next hop.  Next, the IPSec encapsulation will be done and the resulting packet with the Partners VPN gateway IP address as destination will be sent to the MAC address of the default gateway. Obviously not the intended behavior. Therefore, you need to specify those static routes for the destinations reachable through the Partners VPN gateway. Take note that this also means that all ISA server versions running on Windows 2000/2003 will have this same behavior.

In the case of a L2TP/IPSec VPN connection the situation is completely different. Here the L2TP packets are protected with IPSec transport mode. Because this doesn’t add an new IP header or changes the current IP header, the routing decision taken on the L2TP packets are still valid after the IPSec processing. So, no extra persistent static routes are needed for a L2TP/IPSec VPN connection.


About The Author

Leave a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top