Exchange 2010 Domain Security (Part 3)

If you would like to read the other parts in this article series please go to:


Over the first two parts of this article, we have been configuring’s Exchange 2010 infrastructure for domain security. Here in part three we will be continuing with the configuration as there are still quite a few things to do. Specifically, we will be configuring the Send and Receive connectors such that they are ready for domain security. Once that has been done, we will be attempting to send test messages to see what errors and issues we get with the self-signed certificates that are present on the default configuration of the Edge Transport servers located in both the and Exchange 2010 organizations. So, let’s get going with part three and complete the configuration for domain security.

Domain Security Configuration – Continued

Back in Figure 2-1 in part two of this article, we saw the properties of the Send connector configured on the internal Exchange 2010 server. Specifically, we saw the check box titled Enable Domain Security (Mutual Auth TLS). This check box now needs to be selected and we can either do this using the Exchange Management Console, or we can run the following command:

Set-SendConnector –DomainSecureEnabled $true

In this command, we specify the name of the Send connector that we wish to modify and in this case the name of the connector is The DomainSecureEnabled parameter represents the Enable Domain Security (Mutual Auth TLS) check box, which we enable. Remember, this change must be made on the Send Connector seen on the internal Exchange 2010 server and not the Edge Transport server; the Edge synchronization process will ensure that the configuration change is made on the Send connector on the Edge Transport server. Figure 3-1 shows the completed configuration on the Send connector on the Edge Transport server after Edge synchronization has completed. Once you have made the configuration change on the internal Exchange 2010 server, it is always worth checking that the Edge synchronization process is working correctly and that the change has replicated to the Edge Transport server; the Test-EdgeSynchronization cmdlet can help with this.

Figure 3-1: Enabling Domain Security on the Send Connector

The Enable Domain Security option shown in Figure 3-1 is known as Mutual Auth TLS because the Edge Transport servers from and use their certificates to mutually authenticate each other as well as encrypt the communications session. Normally when connecting to a server, clients use Transport Layer Security (TLS) to authenticate to that server. With Mutual TLS (MTLS), we are looking at server-to-server communications and in this article we are referring to the two Edge Transport servers communicating with each other.

If you try and modify the Send connector directly on the Edge Transport server, you will receive the error message shown in Figure 3-2, informing you to make the changes on the internal Exchange 2010 server.

Figure 3-2: Error Message When Modifying the Edge Transport Server Directly

Additionally, you may remember back in part two of this article that I discussed the need to ensure that the Send connector uses DNS MX records for mail routing rather than using a smart host. The reason for this is because this is a requirement to use domain security; if a smart host is chosen the domain security option is unavailable as you can see from Figure 3-3.

Figure 3-3: Domain Security Unavailable

Aside from certificates which will be discussed later, the final part of the configuration is to check the configuration of the Receive connector that is responsible for processing messages from the Internet. Like the Send connector, the Receive connector must also be configured to support domain security which it was by default in my lab environment.

To check the configuration, run the Exchange Management Console on the Edge Transport server and navigate to the Receive Connectors tab. Here, bring up the properties of the default Receive connector and select the Authentication tab. You should see that both the Transport Layer Security (TLS) and Enable Domain Security (Mutual Auth TLS) options are selected; if they are not, select them now. This is shown in Figure 3-4.

Figure 3-4: Setting Domain Security on the Receive Connector

You can also configure these parameters using the Exchange Management Shell. The Set-ReceiveConnector cmdlet is used with the DomainSecureEnabled and AuthMechanism parameters. The command to use is as follows, noting that in the lab environment the Receive connector name is Default internal receive connector DS-FEDGE.

Set-ReceiveConnector “Default internal receive connector DS-FEDGE” –DomainSecureEnabled $true –AuthMechanism tls

That completes the configuration of the internal Exchange 2010 server and the Edge Transport server as far as the Exchange Management Console or Exchange Management Shell is concerned. It is important, before going any further, that the corresponding configuration is made on’s internal Exchange 2010 server and Edge Transport server. In other words, ensure that the Edge subscription is complete, the appropriate Send connector is created and configured accordingly.

With both environments configured, note that we still haven’t paid any attention to certificates so far but I want to show what happens when this is the case as it gives you an idea of what certificate errors or issues you could get, and where to check for these errors or issues.

Test Messages

Let’s now send a message from to However, we won’t expect the user to receive this message since we have not done anything with certificates yet. Using the Queue Viewer utility on’s Edge Transport server, we can see what the Edge Transport server makes of this scenario. To launch the Queue Viewer utility, run the Exchange Management Console on’s Edge Transport server and navigate to the Toolbox option. From there, open the Queue Viewer utility and you should see a screen similar to the one shown in Figure 3-5.

Figure 3-5: Messages Queued for

You can see from Figure 3-5 that there is a queued message for By default, the Last Error column is somewhat obscured. If we hover the mouse over the Last Error column, we see the following error text:

451 4.4.0 Primary target IP address responded with: “454 4.7.5 Certificate validation failure.”

Another way to see this error message is to use the Get-Queue cmdlet and examine the contents of the LastError parameter. In Figure 3-6, you can see that we have examined the LastError parameter in conjunction with the NextHopDomain parameter for the domain

Figure 3-6: Messages Queued for in the Exchange Management Shell

Therefore, it’s clear from this error message that there is a problem validating the certificate on’s Edge Transport server. This is, of course, because we are using a self-signed certificate that’s Edge Transport server does not trust. In a production environment, we will likely be using certificates from a 3rd party trusted certificate authority and hence will not see this problem.


That completes part three of this article series on domain security. We now have everything set up with the exception of the certificates, which is what we will be covering in the fourth and final part of this article. After we have made the necessary certificate changes in part four, we will also be looking at message headers and transport rules as they relate to domain security.

If you would like to read the other parts in this article series please go to:

Leave a Comment

Your email address will not be published.

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top