With businesses increasing the volume of transactions, processes, and operations they conduct over the Internet, security is always a major concern. As your data traverses across the Internet, it’ll certainly face various threats. These threats are capable of stealing, tampering with, or performing other malicious acts on your data. To address these threats, businesses, government agencies, and various organizations constantly work together or independently to come up with solutions to secure Internet traffic. One of the most successfully developed solutions is the Internet Protocol Security or IPsec suite.
In this post, I’m going to introduce you to the basic concepts of IPsec, explain how it works, go over the key IPsec protocols, and discuss other relevant information that’ll help you get a good grasp on this important bundle of technology.
First, let me explain what IPsec is.
What Is IPsec?
Developed by the Internet Engineering Task Force (IETF) in the 1990s and used in VPNs, IPsec is a suite of protocols that secures Internet connections at the network layer. It does this by providing encryption, authentication, data integrity, and replay protection. Let me explain each of these security functions first. This will give you a better appreciation of how IPsec works and its importance in business operations.
Encryption
Encryption is a way of rendering text unreadable to people who don’t have the required decryption key. In effect, it preserves data confidentiality. This is important because, without it, a threat actor can intercept your packets. For instance, because your packets go through the Internet, a cyberattacker can initiate a man-in-the-middle attack. This means they can extract any confidential data you might have.
Another example is that a cyberattacker can also extract usernames and passwords whenever remote users log in to your system. They can then use those stolen credentials to perform unauthorized logins themselves. Threat actors can’t do this if you encrypt your packets, as this restricts them from making out the contents of those packets.
Authentication
Authentication refers to the process of verifying the authenticity of a particular set of data—i.e., whether the information is true. In the context of IPsec, you’re verifying the source IP or the origin of the packet.
It’s important to ascertain the packet’s origin. If it doesn’t match what you were expecting, it could mean the packet you just received might actually be coming from a malicious site spoofing a legitimate source.
Data Integrity
A data integrity check is a process that verifies whether the data in question hasn’t gotten tampered with. In the context of IPsec, you’re checking the integrity of the packet. The part of the packet that’s checked depends on the IPsec header, AH, or ESP (more on these two later). But you can always check the payload or the transmitted data.
You need to conduct an integrity check on the data you receive. That’s because a threat actor always can intercept your packets, alter them, and then send them along. Data integrity checks will help you detect any tampering and prevent you from processing maliciously modified data.
Replay Protection
Replay protection is a security mechanism that thwarts replay attacks. In this context, a replay attack is a cyber-attack where a threat actor intercepts an IPsec packet, captures its contents, and then resends or replays it to the same destination at a later time. Note that the replay attack will still work even if you encrypt the IPsec packets. This happens because the attacker doesn’t have to alter or even obtain the packet’s contents.
So why is this a cause of concern? Well, if that packet contains a valid request that prompts a valid response, the replayed packet will also elicit a valid response. For example, if the replayed packet requests access to your system, your system won’t notice anything suspicious and will simply grant access. IPsec’s replay protection mechanism detects these replay attacks and generates an error message, thereby preventing the malicious packet from getting a response.
The combination of all of these features makes IPsec suitable for Internet connections that require a high level of security. It’s the reason why IPsec gets used in securing VPNs, which are then called IPsec VPNs; more on those in the next section!
IPsec VPNs
Although you do have other ways to secure a VPN (e.g., by using SSL/TLS), IPsec is arguably the most popular choice. That’s why IPsec VPNs are likewise the most widely used VPNs. Before I define what an IPsec VPN is, it might be best to review what a VPN is first.
What Is a VPN?
A virtual private network, or VPN, is a service that creates a private network within a public network like the Internet. VPN solutions achieve this through a process known as tunneling. It’s a technique where packets originally intended for a private network get encapsulated so that they can traverse a public network.
Although tunneling alone can already create a VPN and provide some form of privacy, it doesn’t offer enough protection against today’s threats such as man-in-the-middle attacks. You need a VPN with security mechanisms such as encryption and authentication for that. Today, most VPN solutions support these security capabilities. And of these solutions, IPsec VPNs are arguably the most popular.
What Is an IPsec VPN?
As the name suggests, an IPsec VPN is a VPN that leverages IPsec protocols for security. IPsec VPNs use cryptographic algorithms to provide confidentiality, authentication, and data integrity, as well as a counter and number sequencing for detecting replayed packets.
You can implement IPSec VPNs using the following VPN architectures:
Site-to-Site
A site-to-site VPN connects two or more networks, or sites, using IPsec VPN gateways. Each network should have one VPN gateway. The gateway may be a router or a firewall that supports an IPsec VPN . It can also be a separate device specifically designed for IPsec VPN operations. You’d typically use a site-to-site VPN to establish private communications and data exchanges between a central office and a branch office, or between two different companies.
Remote Access
A remote access (a.k.a. client-to-site) VPN enables users to connect from VPN clients to a central location (e.g., a data center) and access resources hosted there. This type of VPN can be particularly useful in companies that need to support remote workers. Note that if you have multiple devices that need to connect to your VPN gateway, then each device must have a VPN client on it.
Host-to-Host
This architecture is the least popular among the three. It usually comes into play when an IT administrator wants to perform certain tasks on a remote server. In this case, the server itself provides VPN services (notice you have no VPN gateway here) and the IT administrator’s device acts as a VPN client.
IPsec VPNs fully function based on the rules of various IPsec protocols. Let’s talk about the most important ones now.
Key IPsec Protocols
As stated earlier, IPsec isn’t just one protocol. Rather, it contains several protocols. The four that I’ll talk about in this next section are IP Authentication Header (AH) protocol, Encapsulation Security Payload (ESP) protocol, Internet Key Exchange (IKE) protocol, and Internet Security Association and Key Management Protocol (ISAKMP). Let’s start with the IP AH protocol.
1. IP AH
Defined in RFC 4302, the IP Authentication Header, or IP AH, protocol provides origin authentication, data integrity, and replay protection services to IPsec traffic. Because this IPsec protocol doesn’t support encryption, many organizations consider it obsolete and use ESP instead. However, because AH performs data integrity checks on certain parts of a packet (specifically the outer IP header) that ESP does not, you do have some organizations that still use both AH and ESP.
2. IP ESP
Specified in RFC 4303, the Encapsulation Security Payload (ESP) protocol provides packet payload encryption in addition to authentication, data integrity, and replay protection services. Even if ESP doesn’t provide integrity protection to the outer IP header, many organizations don’t perceive this as a critical deficiency. Most organizations worry more about the confidentiality of their traffic. Hence, they prefer this IPsec protocol over AH because of its packet payload encryption capabilities.
3. IKE
Defined in RFC 7296, the Internet Key Exchange Protocol (IKE) is primarily responsible for the negotiation, creation, and management of security associations. A security association (SA) is a set of parameters that define IPsec security and other functionality pertinent to two parties trying to establish a common IPsec connection. The entire IKE process consists of two phases: Phase 1 and Phase 2.
The goal of Phase 1 is for the two parties to establish a secure channel for management-related functions (e.g., health status of the IPsec tunnel, renegotiation of cryptographic keys, etc). This channel is often called IKE SA. To build this channel, the two parties negotiate security parameters such as the encryption algorithm, authentication method, integrity protection algorithm, etc.
The goal of Phase 2, on the other hand, is to establish the actual IPsec tunnel where the data packets will go through. This is the IPsec SA. So, later in this article, when I say IKE SA, know that I’m referring to the management channel. And when I say IPsec SA, I’m referring to the IPsec VPN tunnel itself.
4. ISAKMP
Originally specified in RFC 2408, Internet Security Association and Key Management Protocol, or ISAKMP, is a framework for authentication and key exchange. It defines the procedures as well as the packet formats involved in building, negotiating, changing, and terminating SAs. A lot of what you see in IKE (e.g., syntax and attributes of SAs) is based on ISAKMP.
When you use either the AH or ESP IPsec protocol, you can configure them to encapsulate packets using one of two modes: Tunnel Mode or Transport Mode. Let’s get into those now.
IPsec Modes: Tunnel Mode vs Transport Mode
AH and ESP can encapsulate packets using either tunnel mode or transport mode. It’s important to get familiar with these two modes because each one works best with certain architectures. Let’s start by talking about tunnel mode.
IPsec Tunnel Mode
Tunnel mode is the more widely used between the two and is the default mode. In this mode, AH or ESP creates a new IP header when they encapsulate a packet. This new IP header contains the source and destination IP addresses of the endpoints involved, whether they’re gateways, clients, or hosts. For this reason, tunnel mode can work with all three VPN architectures mentioned previously.
IPsec Transport Mode
Unlike in tunnel mode, AH and ESP don’t have to create a new IP header when they encapsulate a packet in transport mode. Transport mode simply uses the original IP header. For this reason, this mode is best for implementations of the host-to-host architecture because you have direct communication between the two endpoints, and you don’t have to change the IP addresses in the IP header.
That just about covers it regarding IPsec tunnel and transport mode. Now, I’ll talk about how IPsec actually works.
How Does IPsec Work?
Let’s now put everything you’ve learned so far together and talk about how IPsec works. I’ll go over the steps of a site-to-site scenario, as it’s the more popular architecture or use case. Let me bring up the site-to-site diagram I showed you earlier so you can use it as a point of reference.
Now, without further ado, here’s a brief overview of the steps involved in a site-to-site scenario:
- The IPsec process starts once a packet arrives at the IPsec VPN gateway, let’s say Gateway 1, and the gateway identifies the packet as one that requires IPsec protection.
- Gateway 1 begins negotiating a suitable IKE SA with Gateway 2 until the IKE SA is ultimately established. The two gateways also authenticate with each other. Remember that this is IKE Phase 1.
- Once you establish the IKE SA, IKE Phase 2 comes into play. Here, parameters determined by the IKE SA are in turn used to negotiate a suitable IPsec SA. Once the IPsec SA is also established, the packet can then get protected using AH or ESP, whichever protocol the VPN gateway was configured for.
- Gateway 1 then secures the packet using AH or ESP and the IPsec SA parameters, and then sends the IPsec packet to Gateway 2 through the IPsec tunnel. All encryption, decryption, authentication, and so on, get carried out in accordance with the IPsec SA parameters.
- The IPsec tunnel terminates if one of these occurs: the session times out or it reaches the value of the maximum bytes transmitted.
Before we end, let me briefly touch upon one of the most contentious topics about VPNs.
IPsec VPN vs SSL VPN
Whenever you have a discussion on IPSec VPN, a comparison with SSL VPN is usually not far behind. That’s because SSL VPN is slowly replacing IPSec VPN in some organizations. You can find a more detailed discussion comparing these two types of VPNs in the works here at TechGenix. It should be out very soon, so check it out when it’s available. In the meantime, here are my two cents on the subject.
What Is an SSL VPN?
An SSL VPN is a VPN that uses Transport Layer Security (TLS) for security. It’s called SSL VPN because Secure Socket Layer (SSL) is the predecessor of TLS. While SSL is already obsolete, it was originally used for this type of VPN and the name stuck. You can find SSL/TLS built into every modern web browser, which is why, in most cases, users only need a web browser to use an SSL VPN.
One major advantage of SSL VPNs over IPsec VPNs is their ease of deployment and management in client-to-site scenarios. Compared to IPsec VPN clients, which require complex processes and technical know-how, SSL VPN clients are much easier to deploy, configure, and manage. In fact, SSL VPN users only need to install a plugin on any modern web browser and they’ll be ready to go.
When Would You Choose an IPsec VPN over an SSL VPN?
From a security standpoint, IPsec VPNs have certain advantages over SSL VPNs. I won’t dive deep into the details here (that’s for another article), but here are a couple of points to consider:
- If you’ve been keeping track of highly publicized critical vulnerabilities, many of them (e.g., POODLE, BEAST, Heartbleed, and several others) involve the SSL and TLS protocols
- IPsec operates at Layer 3 of the OSI model, while SSL/TLS operates at Layers 4-7. That means, the security coverage a single IPsec tunnel provides encompasses more than SSL/TLS does. For example:
- IPsec can protect IP information (e.g., IP addresses), while TLS can’t
- IPsec protects both TCP and UDP traffic, while TLS only protects TCP traffic
- Data exchanges between applications are only protected by TLS if those applications support TLS. IPsec, on the other hand, supports all network communications, including any data exchanges between hosts, networks, and applications
So does that mean that IPSec VPNs are better than SSL VPNs? Not necessarily. Because IPsec VPN clients are quite difficult to configure and maintain, IPSec VPNs are more suitable for site-to-site use cases. In a site-to-site scenario, you only need to deploy a VPN gateway on each site. Thus, it’s much easier from a management perspective compared to client-to-site scenarios. Your IT team should be easily able to handle that.
That was a lot of information, wasn’t it? Look at the bright side, you’ve gained tons of knowledge in a short span of time! Let’s wrap up now, as we always do.
Final Words
Confidentiality, data integrity, and authenticity are three qualities that are crucial to every business transaction. As many of these transactions are now getting conducted over the Internet, solutions that support these qualities have become integral to business operations. Among the most widely used of this breed of solutions are IPsec VPNs.
In this article, I introduced you to the basic concepts of IPsec. You learned about the qualities of IPsec, VPN architectures, IPsec protocols, IPsec modes, and other related concepts. I also briefly compared IPsec VPNs with SSL VPNs. While this is by no means a deep dive into the inner workings of IPsec, I’m hoping I’ve covered enough to give you a good grasp on the subject.
I’m sure some of these discussions opened up a couple of questions regarding IPsec in your mind. See if you can find some answers in the FAQ and Resources sections below.
FAQ
Is there a VPN that doesn’t support encryption?
Yes, Layer 2 Tunneling Protocol, or L2TP, which is a tunneling protocol used for VPNs, doesn’t encrypt data on its own. Rather, most implementations of L2TP get paired with IPsec, which then provides encryption functionality. If you’re using, or about to use a VPN solution that uses L2TP, check first if it’s actually paired with IPsec.
What is SSL/TLS?
SSL/TLS stands for Secure Sockets Layer or Transport Layer Security (TLS), which are technologies that protect various application layer protocols such as HTTP and FTP using encryption. Encryption protects data by making them unreadable. SSL is the older technology and TLS is the current one, but most users keep using the acronym SSL as it’s still more widely known.
What are encryption algorithms?
Encryption algorithms are the rules and instructions needed to transform cleartext (non-encrypted text) to ciphertext (encrypted text). Some of the most commonly used algorithms include AES, 3DES, RC6, IDEA, and Blowfish. Most cryptographic algorithms use complex math such as modular arithmetic, number theory, prime factorization, elliptic curves, and others.
What is POODLE?
Padding Oracle on Downgraded Legacy Encryption, or POODLE, is a vulnerability that can get exploited when clients perform a protocol downgrade from TLS to SSLv3. As explained earlier in the article, SSL is already obsolete. One of the reasons is that it has several vulnerabilities. Unfortunately, endpoints sometimes perform protocol downgrades to work around server-side interoperability issues. The best way to counter a POODLE exploit is by disabling SSLv3 altogether. The method varies from one application to another, so you’ll have to contact your software vendor for the proper steps.
What is Heartbleed?
Heartbleed was a bug in OpenSSL (which is an implementation of SSL/TLS) that allowed a threat actor to send a malformed heartbeat request in order to elicit a response that might contain more data than usual. A heartbeat request is an OpenSSL feature that allows one endpoint to check if its link with another endpoint is still alive. Note that this bug only affects systems that use OpenSSL. Assuming you do use OpenSSL, you can address Heartbleed by simply applying the latest patch. This is such an old bug (appeared in 2012), so chances are, you’ve already addressed it a long time ago.
Resources
TechGenix: Newsletters
Subscribe to our newsletters for more quality content.
TechGenix: Article on Cloud Network Security
Understand the importance, challenges, and types of cloud network security.
TechGenix: Review on KerioControl
Weigh the pros and cons of using KerioControl.
TechGenix: Guide for Deploying a VPN on Windows
Learn how to deploy a VPN in a Windows environment.
TechGenix: Article on SIEM Software
Discover the infosec policy-enforcement benefits of a SIEM.
TechGenix: Article on Cloud Computing Trends
Discover the top 6 trends in cloud computing in 2022.
