Understanding the ISA 2004 Connectivity Verifiers
By Stefaan Pouseele
March 2005
Last Update: 11/03/2005
1. Summary
A very nice feature of the ISA Server 2004 is the ability to verify the connectivity by regularly monitoring connections from the ISA Server computer to any specific computer or URL on any network. To accomplish that you have to configure connectivity verifiers. However, did you ever wonder how they exactly work, which access rules are involved and how this activity is logged? If you are interested in that kind of stuff, this article might give you some more background information.
With the help of the connectivity verifiers you can monitor the connections between the ISA Server and specific servers on any network by FQDN or IP address, or between the ISA Server and a specific Web Server by URL. You can configure which method to use to determine the connectivity:
- Ping: When you configure this method, ISA Server sends an ICMP ECHO_REQUEST to the specified server and waits for an ICMP ECHO_REPLY. Use this method to verify that a server is running and can be reached by ISA Server.
- TCP connect: When you configure this method, ISA Server tries to establish a TCP connection to a specific port on the specified server. Use this method to verify that a specific service is running on the server and can be reached by ISA Server.
- HTTP request: When you configure this method, ISA Server sends an HTTP Get request and waits for the reply. Use this method to verify that a Web server is running and can be reached by ISA Server.
Note: By default, the connectivity is verified every thirty seconds. This a global setting and therefore applies to all connectivity verifiers. According to the ISA help file, you can change this interval by using the Refresh rate script, available on the ISA Server Coding Corner. However, this Refresh rate script seems not to be available yet.
In the next sections we will explore each of the above mentioned connectivity verifier methods in more detail. Take note that the lab environment we use for this article is running ISA Server 2004 Standard Edition with Service Pack 1, released on March 1, 2005. Also, a good understanding of how ISA Server 2004 processes the access rules is recommended. For more info, check out my article Understanding the ISA 2004 Access Rule Processing.
2. Ping
To be able to investigate the ping method, let us first configure a simple Ping connectivity verifier to the destination www.cevi.be. Once that is done and you have applied the changes, you should see something similar as in the figure below when reviewing the Ping connectivity verifier properties.
When the Ping connectivity verifier get a valid answer within the specified timeout value, that is an echo reply message, then the result shows the ping response time. In all other cases an error will be shown. The measured response time is the time between the sending of the ping request (ICMP ECHO_REQUEST packet) and the receiving of the ping reply (ICMP ECHO_REPLY packet).
If we start now the ISA live logging and filter the traffic on the destination 193.75.143.142 (www.cevi.be), nothing is shown in the logging. How is that possible although a ping request is sent every 30 seconds? You can verify the latter easely with a Network Monitor capture as shown in the figure below.
To understand that behavior, you should know that any stateful packet inspection mechanism is based on the concept of connections. It is the initiating and closing of a connection that is logged, not every individual packet. However, not all protocols are connection oriented, for example the ICMP and UDP based protocols. To solve that problem a simulated connection state is maintained for those protocols. It works as follows:
- When the very first packet is sent or received and that packet is allowed by the firewall policy, a connection entry is made in the connection table with the connection parameters source and destination IP addresses, protocol, source and destination port numbers (UDP) or ICMP type, etc.. Next, a timer is started with a timeout value of 60 seconds and finally a log entry is written that a new connection is initiated.
- When a follow-up packet arrives, first the connection table is checked to see if the connection parameters matches those of an already established connection. If they match, the packet is accepted as belonging to the same connection and the timer is restarted. Nothing is logged because the packet belongs to the same conversation.
- When the timer expires, the connection entry is removed from the connection table and a log entry is written that an existing connection is closed. Therefore, any subsequent packet with the same connection parameters will be considered as a first packet of a new connection and the whole process starts over.
You can verify the above behavior by disabling the Ping connectivity verifier, apply the changes, wait at least one minute and restart the ISA live logging. Next enable the Ping connectivity verifier and apply the changes. If you look now at the ISA log you should see an entry similar to the first one in the figure below (Initiated Connection).
Now, let it run for a while and finally disable the Ping connectivity verifier again and apply the changes. Return to the ISA live log viewer and after a short time you should see a new entry similar to the second one in the above figure (Closed Connection). In the above example, the simulated connection was active for about a 21:30 minutes.
If you look more closely to the log, you will see that the Ping connectivity verifier is allowed by the access rule Allow ICMP requests from ISA Server to selected servers. Did you create that access rule? No, instead it is the System Policy rule #11 which allowed the traffic. This System Policy rule is enabled by default. To find out how the Ping connectivity verifier interact with the access rules, let us disable the System Policy rule #11 and then create our own Firewall Policy rule Ping Access to allow the Ping protocol from Local Host to All Networks. The sequence of events that will be logged is shown in the figure below.
After disabling the System Policy rule #11 and applying the changes, no event at all is logged. This is normal because modifications to the firewall policy are only applied to new connections. Remember that the Ping connectivity verifier is still active and therefore also the simulated connection entry in the connection table. So, you should disable the verifier first and wait until the simulated connection times out. Then you will see the first event logged as shown in the above figure (Closed Connection). After this event, enable again the Ping connectivity verifier and you should see that each individual request will be denied by the default rule Last as expected (Denied Connection). Once you have configured your own Firewall Policy rule Ping Access and apply the changes you will see that the subsequent Ping connectivity verifier requests are allowed by your own Ping Access rule as shown in the last event logged in the above figure (Initiated Connection).
Although the Ping connectivity verifier does not create much noise in the ISA logging, you can get rid of all those entries if you like. To accomplish that, right click on the System Policy rule #11, select Properties and disable the Log requests matching this rule in the Action tab.
3. TCP Connect
To be able to investigate the TCP method, let us first configure a simple TCP connectivity verifier for the FTP protocol to the destination www.cevi.be. Once that is done and you have applied the changes, you should see something similar as in the figure below when reviewing the TCP connectivity verifier properties.
When the TCP connectivity verifier get a valid answer within the specified timeout value, that is a TCP connection accept, then the result shows the TCP connection response time. In all other cases an error will be shown. The measured response time is the time between the sending of the TCP connection request (SYN packet) and the receiving of the TCP connection accept (SYN, ACK packet).
If we start now the ISA live logging and filter the traffic on the destination 193.75.143.142 (www.cevi.be) you should see something similar as shown in the figure below.
If you look very carefully to the above log entries, you should see two weird things. First of all the sequence in the Log Time field and secondly the empty Rule field. For the Log Time field, the time stamp of the action Closed Connection is each time the same as the time stamp of the subsequent action Initiated Connection. That means that the TCP connectivity verifier keeps the connection active until the next interval. Then the connection is torn down, immediately followed by a new connection setup You can verify that easely with a Network Monitor capture as shown in the figure below. Also, take note that an active connection is forcibly terminated, in other words aborted, with an RST packet.
The second weird thing is the empty Rule field. That means to me that neither a known System Policy rule nor a known Firewall Policy rule explicitly allowed that traffic. Therefore we could say that a TCP connectivity verifier does not follow the normal ISA 2004 Access rule processing and that some sort of hidden dynamic System Policy rule is created when you configure a connectivity verifier with the TCP method. In my opinion this is very bad and should be fixed in an upcoming service pack because the firewall administrator has no control over it. Moreover, that behavior have some unwanted site effects.
The traffic generated by the TCP connectivity verifiers is not unambiguous identified in the logging. This could have been mitigated by logging the Rule field with for example Allow TCP requests from Connectivity Verifiers. At least that would identify clearly that kind of traffic. Moreover, the TCP connectivity verifiers do create a lot of noise in the ISA logging and as far as I know, there is no method to mitigate that problem. As clearly shown in the above example, one TCP connectivity verifier generates every 30 seconds two entries in the ISA log. That is obviously too much noise for traffic you implicitly allow by configuring a TCP connectivity verifier. Therefore, be very sparingly in configuring connectivity verifiers with the TCP method.
4. HTTP Connect
To be able to investigate the HTTP method, let us first configure a simple HTTP connectivity verifier to the destination www.cevi.be. Take note that at the end of the configuration wizard, you have to confirm that a System Policy rule will be enabled. Once that is done and you have applied the changes, you should see something similar as in the figure below when reviewing the HTTP connectivity verifier properties.
By configuring a HTTP connectivity verifier, the System Policy rule #18 is enabled which allows HTTP/HTTPS requests from Local Host to All Networks. Instead of allowing HTTP/HTTPS requests to All Networks, you may consider configuring the System Policy rule #18 to use a custom Computer Set that only includes the computers for which you have defined a HTTP connectivity verifier. After you delete all connectivity verifiers that use HTTP or HTTPS, the system policy rule will be automatically disabled as a security measure.
When the HTTP connectivity verifier get a valid answer within the specified timeout value, that is a HTTP response code 1xx, 2xx, 3xx or 401, then the result shows the HTTP Get response time. In all other cases an error will be shown. The measured response time is the time between the sending of the HTTP Get request and the receiving of the HTTP response code 1xx, 2xx, 3xx or 401.
If you look now at the ISA logging and filter the traffic on the destination 193.75.143.142 (www.cevi.be) you should see something similar as shown in the figure below for a complete HTTP connectivity verifier session (enable the verifier, let it run for a while and then disable the verifier).
The log entries with HTTP (in capitals) as value in the Protocol field are written by the Firewall component. Those with http (in lower case) as value in the Protocol field are coming from the Web Proxy filter. As far as I know, the Firewall component processes the TCP connection handling and the Web Proxy filter the real HTTP requests, in this case the HTTP Get request.
Another interesting point is the fact that we see two entries with Initiated Connection and two entries with Close Connection as value in the Action field. For the first entry with Initiated Connection and Close Connection as action, the Rule field is populated with the value Allow HTTP/HTTPS requests from ISA Server to specified sites. That’s very strange because that is the System Policy rule #17 and that rule has nothing to do with the connectivity verifiers. To prove it, disable the System Policy rule #17 and the HTTP connectivity verifier will still work. Moreover, for the second entry with Initiated Connection and Close Connection as action, the value of the Rule field is empty.
As already mentioned before, the HTTP connectivity verifier is allowed by the System Policy rule #18 named Allow HTTP/HTTPS requests from ISA Server to selected servers for connectivity verifiers and that is consistent with the value written in the Rule field for the log entries with Allowed Connection in the Action field.
To have a little bit more fun, let’s try to match the above logging with the Network Monitor trace of the same session as shown in the figure below.
You can clearly see the TCP connection request handshake (SYN, SYN-ACK and ACK packets), followed by five times the HTTP Get request and the corresponding HTTP response 200 and finally the TCP connection close handshake (FIN-ACK and ACK packets in both directions). The log entry matching the TCP connection request in the Network Monitor trace (TCP source port 1146) is the second HTTP Initiated Connection entry with an empty Rule field. You can check that out by verifying the Source Port field in the ISA log. Therefore, the logic behind the log entry with the System Policy Rule #17 in the Rule field escapes me completely.
To find out how the HTTP connectivity verifier interact with the access rules, let us disable the System Policy rule #18 and then create our own Firewall Policy rule HTTP Access to allow the HTTP/HTTPS protocol from Local Host to All Networks. The sequence of events logged is shown in the figure below, assuming that the System Policy rule #17 is still enabled.
After disabling the System Policy rule #18 and applying the changes, you will see that each individual request will be denied by the default rule Last as expected (sequence Initiated Connection, Denied Connection, Closed Connection). Once you have configured your own Firewall Policy rule HTTP Access and apply the changes you will see that the subsequent HTTP connectivity verifier requests are allowed by your own HTTP Access rule as shown in the last events logged in the above figure (sequence two times Initiated Connection, Allowed Connection, Allowed Connection, etc.).
Finally, the HTTP connectivity verifiers do create a lot of noise in the ISA logging. As clearly shown in the above example, one HTTP connectivity verifier generates every 30 seconds one entry in the ISA log. However, you can get rid of all those entries if you like. To accomplish that, right click on the System Policy rule #18, select Properties and disable the Log requests matching this rule in the Action tab.
5. Conclusion
In this article we examined most of the characteristics of the ISA Server 2004 Connectivity Verifiers. On the basis of the Ping method we have learned how ISA Server handles non-TCP traffic by using a simulated connection state. When the TCP connect method is used, we have observed that the connectivity verifier does not follow the normal ISA 2004 Access rule processing and therefore be aware of some unwanted site effects. Finally, we have detected some unresolved logging mysteries when the HTTP request method is used.
I hope you enjoyed this article and found something in it that you can apply to your own environment. If you have any questions on anything I discussed in this article, head on over to http://forums.isaserver.org/ultimatebb.cgi?ubb=get_topic;f=31;t=000015 and post a message. I’ll be informed of your post and will answer your questions ASAP. Thanks! – Stefaan.
