Securing SMTP Connections to Specific Servers
Exchange 2000/3 allows you to encrypt mail on the server side. This is useful when you want all mail sent to a certain business partner to be secured, while still being able to send and receive unencrypted mail from and to the rest of the Internet.
You would need an active Certificate Authority (CA) to generate encryption keys. This is outside of this articles' scope but you can have a look at the step-by-step guide provided by Microsoft at the following link:
Once your CA is up and running you can generate a certificate for mail encryption using the Exchange System Manager. The certificate is a piece of information that is linked to its sources, in this case, the CA and the Exchange server that is used to encrypt the data. The part of the certificate that is actually used to encrypt the data is called "encryption key".
In the following dialog box you can specify the length of the key used to encrypt the data. Increasing key length makes the encrypted data more difficult to break but also increases the load on the server's CPU and the size of the mail that is sent.
Don't fuss too much about the data entered in the coming three dialog boxes. The data eventually appears on your certificate but is not that important for this scenario.
Now your server is ready to accept secure mail. To send encrypted mail we now need to configure the Exchange connectors. To send encrypted mail to a particular domain, we need a second dedicated connector. Since the Internet mail outbound connector usually sends mail to all external domains we need to first of all designate that this connector cost be higher. This means that outbound mail is first evaluated using the lower cost connector. If it matches the address space for this connector it will be encrypted, if not it will be sent to the Internet using the higher cost connector, typically with an address space of "*" designating all outbound domains.
You can directly forward the mail to the remote server if you know the name or IP address, but I prefer using DNS which is better in case your business partner changes the name of the mail server of IP address and forgets to tell you.
Mail encryption uses TLS, an official Internet standard based and improving on the more commonly known SSL 3.0 protocol. Mail is encrypted in the same way web transactions are encrypted when you visit a secure web site, only this is done both ways. To make sure that no one is posing as the designated mail server I usually configure Basic Authentication with a username and a complex password on both sides. Integrated Windows Authentication might be more secure but typically does not work over Firewalls.
After completing this on both sides you are all set to go and your e-mail is transparently encrypted between the servers, safe from tampering and theft.