The spirit of this document is to provide you with a quick guide to troubleshooting System Center 2012 Data Protection Manager (DPM) Certificate authentication issues. This document assumes that you are already familiar with DPM 2012 and have a healthy certificate infrastructure. This document also assumes that you have setup certificates in accordance with the following blog post:
How to use certificates to authenticate computers in workgroups or untrusted domains with Data Protection Manager:
http://blogs.technet.com/b/dpm/archive/2012/04/23/how-to-use-certificates-to-authenticate-computers-in-workgroups-or-untrusted-domains-with-data-protection-manager.aspx
We will cover briefly at a high level some of the common caveats that you will come across when using certificates with DPM protection and how to assess what may be the root cause. I have included screenshots of some of the DPM GUI errors, event log errors and some log snippets. Admittedly the log reading is not very intuitive so I just highlighted the relevant portions.
Services
1. Make sure the DPMRA service can be started.
2. Make sure the DPM CPWrapper Service can be started. I can’t stress this enough. Upon my testing I performed various actions to simulate a failure.
a.) Removed the DPM cert
b.) Removed the client cert
c.)Removed the DPM reg key on the DPM server
d.) Removed the Member Server regkey on the DPM server
e.) Removed the DPM regkey on the Member server
f.) Removed the Member Server regkey on the Member Server.
After each failure I would either place the cert or regkey back and almost each time I would have to restart the DPM CPWrapper Service. In light of this you should make it a very common practice to restart the DPM CPWrapper Service during your troubleshooting.
3. Make sure the Cryptographic Services are started
Ports
1.) Remember that DPM certificate use relies on port 6076 for Certificate protection. You may have to adjust any intermediate firewall settings to allow for this port to be opened for certificate based protection. You can use the netstat command to verify if port 6076 is listening for communication from both the ends.
Type in: netstat –ano
or
netstat –ano |findstr 6076
a = Displays all connections and listening ports
n = Displays addresses and port numbers in numerical form.
o = Displays the owning process ID associated with each connection.
|findstr 6076 will show only the associations for that port.
Using the following: netstat –ano |findstr 6076
We see that the port 6076 is listening:
You could also use TCPView found at http://technet.microsoft.com/en-us/sysinternals/bb897437.aspx for a GUI interface to show what services are listening to which ports.
Example:
The use of certificates for authentication DOES NOT remove the need for other ports for types of domain communication such as name resolution, Kerberos and LDAP. These ports will still be needed for proper DNS or NetBIOS name resolution and AD authentication.
http://technet.microsoft.com/en-us/library/ff399341.aspx
Firewall
If on the target server you have the firewall turned off and you go through the setdpmserver command, the command will NOT create the necessary firewall rule for TCP port 6076. If you leave the firewall turned off , then there will be no issue. If at a later date you turn the firewall back on, your communication will fail because this rule is NOT created. To correct this you can do one of three things:
a.) Leave the integrated firewall off
b.) Manually create the rule yourself
c.) Re-run the setdpmserver command.
Verification that the ports have been added.
Firewall Rule Created
This firewall rule created specifies a local port of 6076 and a remote port of “all ports”.
4.) If testing with DPM beta If the firewall is turned off on the client you will get the following error:
You will have to have the firewall turned on. Again, this has been fixed in RTM for DPM 2012.
Certificate
By default, with web enrollment the certificate is saved in the Current User store but needs to exported with the private key and imported into the Local computer store. Again, this is if you are using web enrollment. If certs are configured for “enroll” then it can be specified to be placed in the local computer store.
The thumbprint in the command syntax (AttachProductionServerWithCertificate and the Set-DPMCredentials commands) needs to NOT have spaces when specifying it in the commands.
An example would be as follows.
Here is the DPM server cert:
Notice the spaces in the thumbprint.
When we use the certificate thumbprint to generate the bin file we need to remove the spaces.
Notice the Set-DPMCredentials command syntax used in this case.
Set-DPMCredentials –DPMServerName DPM2012.contoso.com –Type Certificate –Action Configure –OutputFilePath C:\Temp -Thumbprint 493f27f35b2105804afbd49bb5a59bf2e380e00
This is the thumbprint for the DPM server certificate without the spaces.
The certificate must specify certain parameters:
X.509 V3 certificates
Enhance Key Usage should have client authentication and server authentication.
Key length should be at least 1024 bits.
Key type should be exchange.
Certificate can NOT be self signed.
Subject name of the certificate and root certificate should not be empty.
Certificates shouldn’t be of Cryptography API Next Generation (CNG) Keys. DPM doesn’t support
certificates with CNG Keys.
The revocation servers of the associated Certificate Authorities are online and accessible by both the
protected server and DPM server.
The certificate has an associated private key
You can use the following command to verify the certificate parameters of the certs in use on a server.
certutil –store –v my
C:\>certutil -store -v my
================ Certificate 1 ================
X509 Certificate: <<<<<Denotes x.509>>>>>
Version: 3 <<<<<Denotes V3>>>>>
Serial Number: 5da52bdc000226d4c235
Signature Algorithm:
Algorithm ObjectId: 1.2.840.113549.1.1.5 sha1RSA
Algorithm Parameters:
05 00
Issuer:
CN=Corp NAP CA 1
NotBefore: 9/14/2011 7:31 AM
NotAfter: 9/17/2011 7:31 AM
……….
Public Key Length: 2048 bits <<<<<Denotes length>>>>>
Public Key: UnusedBits = 0
0000 30 82 01 0a 02 82 01 01 00 97 3c 11 94 27 58 47
0010 4a 51 55 60 a5 b6 32 8a 4e 4b 59 1d 56 1f ac 53
……….
Application Policies
[1]Application Certificate Policy:
Policy Identifier=Server Authentication <<<<<Denotes server>>>>>
[2]Application Certificate Policy:
Policy Identifier=Client Authentication <<<<<Denotes client>>>>>
……….
CERT_KEY_PROV_INFO_PROP_ID(2):
Key Container = {57CE5453-2951-4AE2-A036-E685FC52AB83}
Unique container name: bed058e40c5ed733d5da8a6655583c3d_d5520479-582f-4563-8c84-e153a68e8fe2
Provider = Microsoft Enhanced Cryptographic Provider v1.0 <<<<<Denotes provider - must be cryptographic provider and NOT Key Storage Provider >>>>>
ProviderType = 1
Flags = 60
KeySpec = 1 -- AT_KEYEXCHANGE <<<<<Denotes type is Exchange>>>>>
……….
Private key is NOT exportable
Encryption test passed
This output has been trimmed down and the key points have been bolded above.
If the certificate is invalid then when you run the command you may see an error like this.
Example Error
***********
C:\Program Files\Microsoft Data Protection Manager\DPM\bin>SetDPMServer -dpmCredential CertificateConfiguration_DPM2012.contoso.com.bin -Outputfilepath c:\temp
-Thumbprint 4301114a1d05b44bc834f34f04f4cb4333433bac
Error(Id= 33234), Details : The certificate provided with thumbprint 4301114a1d05b44bc834f34f04f4cb4333433bac on the personal machine store of machine MemberServerTest does not correspond to the requirements of DPM. The following requirements are not met for the certificate.
The certificate is not trusted on the local machine.
Please make sure certificate fulfills the following requirements:
1) The certificate is trusted on the local machine and has not expired.
2) The revocation servers of the associated Certificate Authorities are online.
3) The certificate has an associated private key with a valid exchange algorithm.
4) The certificate's public key length is greater than or equal to 1024 bits.
5) The certificate should have both Server and Client Authentication if EnhancedKey Usage is enabled.
6) The subject of the certificate and its root CA should not be empty.
7) DPM does not support certificates with Cryptography API Next Generation (CNG)keys.
For more details see help.
SetDpmServer failed with errorcode =0x809909b4, error says: (null)
Note the 33234 error which equates to an invalid cert. Most likely the cert used does NOT meet our requirements. Again you can use the command certutil –store –v my to verify our certs in use.
Troubleshooting the Attach-ProductionServerWithCertificate and the SetDPMServer commands
Attach-ProductionServerWithCertificate
1.) On the DPM server upon the attach attempt If you get the following error:
You will need to place the client bin file on the DPM server system32 directory OR specify the full path of the bin file. In the example above we specified:
DPMServerName: DPM2012
PSCredential: CertificateConfiguration_MemberServer.Contoso.com.bin <----This is not the full path so it will, by default, search the system32 directory.
If we placed the cert in a folder named C:\Cert then we would specifiy:
DPMServerName: DPM2012
PSCredential: C:\Cert\CertificateConfiguration_MemberServer.Contoso.com.bin <--This is a full path to the certificate we wish to use.
2.) On the DPM server the Attach-ProductionServerWithCertificate on the DPM server creates a registry key for the protected computer with the certificate.
HKLM\Software\Microsoft\Microsoft Data Protection Manager\Agent\2.0\Certificates\<Protected ComputerName>
Note the Certificate name and port number.
If the Attach-ProductionServerWithCertificate fails, then the following needs to be looked at:
a.) There is a network issue between the DPM Server and Protected Computer. You can use the telnet command to verify if port 6076 is open for communication from both the ends.
b.) Certificate used for DPM server is not trusted on the Protected Computer. In Certificates MMC verify that ROOT CA Certificate is present in the Trusted Root Certification Authorities. Go to the Workstation and check the DPMRACurr.errlog files for failures.
SetDPMServer
1.) Specifying a Wrong Bin file on the Target server
In this case the SetDPMServer command was used to setup protection. We purposely used the wrong bin file for the DPM server to simulate an error.
2.) On the DPM server, when running the Set-DPMCredential the following registry key is created:
HKLM\Software\Microsoft\Microsoft Data Protection Manager\Agent\2.0\Certificates\<DPMServerName>
Note the Certificate name, port number and thumbprint specified.
The Set-DPMCredentail command also enables the DPM CPWrapper Service and configures it to use the certificate.
3.) Failures while running this are logged in the DPM management Shell, MSDPM*.errlog and the CAP12 event viewer logs.
Sample Errors
Error in DPM Management Console:
***************************
Set-DPMCredentials : Unable to find certificate with the thumbprint 8d8bddbc15d73f3c20c3faf3faab9b69075e582c on the personal machine store of machine DPM2012.contoso.com. (ID: 33231)
Error in MSDPMCurr.errlog
************************
ConfigureCertificates.cs(400) NORMAL Getting certificate for thumbPrint : 8d8bddbc15d73f3c20c3faf3faab9b69075e582c
CertificatesHelper.cs(51) NORMAL Looking for Certificate with thumbprint: 8d8bddbc15d73f3c20c3faf3faab9b69075e582c in store: My at location: LocalMachine
CertificatesHelper.cs(88) NORMAL Could not find Certificate with thumbPrint: 8d8bddbc15d73f3c20c3faf3faab9b69075e582c in store :My at location :LocalMachine
ConfigureCertificates.cs(133) WARNING Getting certificate for thumbPrint : 8d8bddbc15d73f3c20c3faf3faab9b69075e582c failed
ConfigureCertificates.cs(256) WARNING Failed to configure the dpm credentials with exception: Microsoft.Internal.EnterpriseStorage.Dls.Utils.DlsException: Getting certificate for thumbPrint : 8d8bddbc15d73f3c20c3faf3faab9b69075e582c failed
ConfigureCertificates.cs(256) WARNING at Microsoft.Internal.EnterpriseStorage.Dls.CertificateHelper.ConfigureCertificates.GetCertificateByThumbPrint(String thumbPrint)
ConfigureCertificates.cs(256) WARNING at Microsoft.Internal.EnterpriseStorage.Dls.CertificateHelper.ConfigureCertificates.ConfigureDPMCredentials(String certificateThumbPrint, String authCAThumbprint, String outputFilePath, Boolean generateFileOnly)
Conclusion
This concludes Part 1 of DPM Certificate Based Authentication. Part 2 will entail troubleshooting missing or corrupt registry keys and their symptom and Part 3 will go over missing or invalid certificates.
Shane Brasher | Senior Support Escalation Engineer
Get the latest System Center news on Facebook and Twitter:
App-V Team blog: http://blogs.technet.com/appv/
ConfigMgr Support Team blog: http://blogs.technet.com/configurationmgr/
DPM Team blog: http://blogs.technet.com/dpm/
MED-V Team blog: http://blogs.technet.com/medv/
Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
Operations Manager Team blog: http://blogs.technet.com/momteam/
SCVMM Team blog: http://blogs.technet.com/scvmm
Server App-V Team blog: http://blogs.technet.com/b/serverappv
Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center Essentials Team blog: http://blogs.technet.com/b/systemcenteressentials
WSUS Support Team blog: http://blogs.technet.com/sus/
The Forefront Server Protection blog: http://blogs.technet.com/b/fss/
The Forefront Endpoint Security blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms-identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/