Selecting the best TMG Firewall Remote Access VPN Protocol for your network

Introduction

You know that Virtual Private Networks (VPNs) enable a virtual link-layer connection between the VPN client and VPN server that terminates the VPN connection. The VPN connection is an encrypted tunnel that carries traffic to a remote access VPN server and routes the traffic into a remote private network. VPN protocols that are available in a given situationmay vary, based on the capabilities of the VPN client and server as well as limitations related to the network devices that are located between the VPN client and server.

VPN connections are typically used to provide private and secure access to a corporate network. A common misconception is that a VPN connection requires that you allow unrestricted access to the corporate network. In actuality, this depends on the VPN server that you’re using. If you’re using the TMG firewall, there is no necessity to open up everything to VPN users; you can setpolicies that control who can access which servers using pre-defined protocols.

Varieties of VPN Tunnels

Before we get into the types of VPN tunnels and the protocols that enable them, there are a few things you need to remember about VPN tunneling in general. Because VPN tunnels are intended to be private, authenticated and authorized, they must provide for strong link encryption and authentication. Additionally, they must employ tunnel management to control the traffic flow through the tunnel – this helps maintain the integrity of the tunnel itself.

Recent updates in VPN technologies have led to the emergence of a VPN tunnel option often referred to as SSLVPN. It should be noted that the term SSL VPN is a generic term that is often applied to reverse web proxies and various forms for port and socket forwarding over an SSL encrypted session. The TMG firewall does not provide full SSL VPN capabilities, but it does include a network layer SSL VPN solution called SSTP (Secure Socket Tunneling Protocol).  You can read more about it here.

VPN tunnels fall into two operational modes: transport mode and tunnel mode. It’s important, however,  to note that not all VPN tunnel technologies offer both modes. Let’s look at the differences between the two modes:

• Transport mode – This mode enables a point to point connection between two endpoints and does not route traffic between an endpoint and a network. Transport mode is typically used for dial-on-demand VPN connections between individual remote access VPN clients and a VPN terminator (such as the TMG firewall when it has been configured as a remote access VPN server).
• Tunnel mode – This mode enables routing between two networks as well as between the two endpoints. This mode is typically used for site-to-site VPN connections, where disjointed networks need to communicate. In this mode, the hosts in each network must use their VPN gateways on their own networks to route traffic to the remote VPN gateway and to the remote network.

Virtual Private Networking Protocols

Windows Server 2008 R2 and the TMG firewall support three VPN protocols, which are described in this section.

Point-to-Point Tunneling Protocol (PPTP)

PPTP is defined in RFC 2637 and uses two protocols: PPTP, which uses TCP port 1723, and the Generic Routing Encapsulation (GRE) protocol, which uses IP protocol 47 (not TCP or UDP port 47). PPTP serves as the management protocol, while GRE provides the encrypted and authenticated tunnel between the client and server.

A PPTP editor in the NAT device is required for PPTP NAT traversal. A PPTP connection begins with the VPN client establishing a PPTP control channel connection with the VPN server. The PPTP client authenticates and then negotiates encryption keys that are to be used for the GRE tunnel. The PPTP client then establishes a second connection, using GRE. All traffic except the PPTP control channel is carried within the encrypted GRE tunnel. PPTP depends on a NAT editor and NAT devices for NAT traversal, and it’s important to keep in mind that the level of support for NAT traversal across various devices can vary widely.

Layer-Two Tunneling Protocol over IPsec (L2TP/IPsec)

L2TP is described in RFC 2661 and works with Internet Key Exchange (IKE) protocol and the IPsec protocol to enable anencrypted and authenticated tunnel. IPsec allows for machine authentication and encryption using IP protocol 50 (ESP or Encrypted Security Payload) or IP protocol 51 (Authentication Header or AH). L2TP performs tunnel management over UDP port 1701.

When you use L2TP, NAT traversal is enabled by IPsec NAT-T, which uses UDP ports 500 (for IKE) and 4500 (for NAT-T). For non-NAT L2TP/IPsec sessions, the client uses UDP port 500. Encryption keys are exchanged using certificates (the preferred and more secure method) or pre-shared keys. After the exchange of encryption keys, the client establishes an L2TP session within the encrypted IPsec tunnel over IP protocol 50 (ESP) or protocol 51 (AH), depending on how the client and server are configured. All traffic, including the L2TP management traffic, is carried within the IPsec tunnel. L2TP/IPsec also supports NAT traversal, in which case the client will connect using UDP port 500 to negotiate the encryption (IKE), then will pass the L2TP and client traffic through UDP port 4500.

Secure Socket Tunneling Protocol (SSTP)

SSTP encapsulates the Point-to-Point Protocol (PPP) over HTTP and then encrypts it with Transport Layer Security (TLS is the latest version of SSL). HTTP supports tunnel management for the traffic tunnel. SSTP works from behind a Web proxy so that the VPN client can connect to the remote access VPN server. However, be aware that SSTP cannot authenticate with a Web proxy, so if the local Web proxy requires authentication, the SSTP connection attempt will fail. SSTP clients connect over TCP port 443, negotiate the SSL handshake, and then send an HTTP request to the VPN server. All traffic is passed through this HTTPS tunnel.

Address Assignment

IP address assignment for VPN connections does not use Dynamic Host Configuration Protocol (DHCP). The Internet Protocol Control Protocol (IPCP) is used instead of DHCP to assign IP addressing information to VPN clients. IPCP is defined in RFC 1332. You can configure RRAS to obtain IP addresses from a DHCP server (by default, RRAS obtains blocks of 10 addresses at a time as needed) or you can assign IP addresses using a static pool. IPCP provides only the IP address itself and unlike DHCP, it does not have a provision for “options”. In addition, the VPN client assigns itself a 32-bit subnet mask.

Authentication

Each VPN server solution will enable a different collection of authentication options. The TMG VPN server offers the following (presented here in order of increasing relative security):

• Password Authentication Protocol (PAP) –PAP is the weakest user authentication protocol because the credentials are sent in plain text so that anyone with a network sniffers could capture and  view this information.This is only a problem if authentication takes place before the tunnel is established. If it takes place after tunnel establishment, then the use of clear text is less problematic.
• Challenge Handshake Authentication Protocol (CHAP) – CHAP uses the MD5 hash algorithm to protect credentials when they arepassed between the client and server. Similar to Digest authentication, this authentication method requires that passwords stored in Active Directory use reversible encryption.
• Microsoft Challenge Handshake Authentication Protocol Version 2 (MS-CHAPv2) – MS-CHAPv2 solves some of the problems related to CHAP protocol security by adding one-way salted encryption. This involves using a unique “salt value” each time the encryption key is calculated, which results in a different value being calculated each time. This mitigates any hash-based password-guessing attacks.
• Extensible Authentication Protocol (EAP) – EAP is made up of three different authentication schemes:
  MD5-Challenge – Similar to PPP-CHAP, but is sent as an EAP message.
  EAP-TLS – Uses mutual certificate authentication and is required for smart cards withVPN smart card authentication.
  EAP-RADIUS – Uses a RADIUS server as the authentication repository. The actual credentials are passed using MD5-Challenge or TLS and are satisfied through a call to the specified RADIUS server.
• Internet Key Exchange Version 2 (IKEv2) – IKEv2 enables the remote access VPN client to encrypt the tunnel first, so that all subsequent communications will take place over an encrypted and authenticated connection and thus will not be exposed to network sniffers and other similar attacks. This requires that the remote access VPN client and server share a common trusted root CA and that both have certificates that were issued for the purpose of IPsec negotiation.

Comparing the VPN Technologies

The following table compares the VPN tunnel technologies that are supported by Windows Server 2008 R2 and the TMG firewall.

Summary

In this article, we went over some characteristics of the basic VPN protocols that are supported by Windows Server 2008 R2 and the TMG firewall, to help you better select the protocols that are appropriate for a given VPN scenario on your network. As you learned, the TMG firewall supports three VPN protocols: PPTP, L2TP/IPsec and SSTP. Although supported by Windows Server 2008 R2, the IKEv2 protocol does not have built in support in the TMG firewall. PPTP and L2TP/IPsec can be used for both remote access VPN client connections and for site-to-site VPN connections. In contrast, the SSTP VPN protocol can only be used for remote access VPN client connections and not for site-to-site connections. All protocols support NAT traversal, although the level of support varies and in some cases, as in the case of PPTP, will depend on software technologies that will support NAT for that protocol.