When you request a certificate using the Certificate MMC snap-in or the certificate auto-enrollment mechanism, the request fails. Of course we assume here that the CA is started and you have sufficient permissions to request a certificate. This issue occurs because DCOM is required to acquire a certificate. However, for access rules a default “Enable strict RPC compliance” setting is configured on each RPC rule. With this setting in place, DCOM traffic is blocked.
To workaround this issue without using an access rule that allows all outbound protocols between the host requesting the certificate (the source) and the CA (the destination), you can apply the following steps:
- On the CA, configure the RPC application or DCOM endpoint to use a custom TCP protocol port as a static port.
- On the ISA, turn off the “Enable strict RPC compliance” setting on the RPC access rule.
- On the ISA, create the custom protocol for outbound use.
- On the ISA, create an access rule to allow the custom protocol between the required source and destination.
On the CA, configure the RPC application or DCOM endpoint to use a custom protocol port as a static port.
The CA is primarily implemented as a DCOM application. By default, DCOM uses high-numbered TCP ports to respond to client requests. With the DCOM Configuration MMC Snap-In, Certificate Services can be enforced to use a custom TCP port.
- While logged on to the issuing CA, from a command-line prompt, run the following command:
dcomcnfg.exe - In the left pane of the Component Services MMC Snap-In, expand Component Services, Computers, My Computer, and then DCOM Config.
- In the right pane, select CertSrv Request.
- On the Action menu, click Properties.
- On the Endpoints tab, click Add.
- Select Use static endpoint, enter an unused TCP port number, for example 789 (my choice), and then click OK twice.
- Close the MMC Snap-In.
- Restart the certification authority service through the MMC Snap-In or by the following commands:
net stop certsvc & net start certsvc
To undo this change at a later stage, remove the added entry from the list of endpoints.
On the ISA, turn off the “Enable strict RPC compliance” setting on the RPC access rule.
If ISA Server is requesting the certificate, disable the “Enforce strict RPC compliance setting” on the system policy rule.
- To do this, on the Firewall Policy tab of ISA Server Management, click Edit System Policy on the Tasks tab.
- Select the Active Directory group in the Configuration Groups list.
- On the General tab, clear the Enforce strict RPC compliance checkbox.
If a host is requesting the certificate from another network through ISA Server, do the following: in the Firewall Policy tab of ISA Server Management, right-click the access rule allowing the traffic, and then click Configure RPC protocol. On the Protocol tab, clear the Enforce strict RPC compliance checkbox.
On the ISA, create the custom protocol for outbound use.
I assume everybody is able to accomplish this task 😉
My choice was to use TCP port 789 outbound for the CertSrv Request protocol definition.
On the ISA, create an access rule to allow the custom protocol between the required source and destination.
Again, I don’t think that creating an access rule requires much explanation. The important thing to note is that if ISA Server is requesting the certificate, the rule must apply to all users. Obviously the source is Local Host in this case and the destination can be further restricted to only the computer object representing the CA.
For publishing rules ISA Server blocks DCOM traffic, and as far as I know this setting cannot be modified. Therefore, if a host is requesting the certificate from another network through ISA Server, you must have a route relationship between the required source and destination, and create the appropriate access rule.
For more information, check out the following resources:
- RPC Filter and “Enable strict RPC compliance”
- Configuring and Troubleshooting Windows 2000 and Windows Server 2003 Certificate Services Web Enrollment
HTH,
Stefaan