WCF-Java interop failure due to certificate string mismatch

Here we describe an interoperability issue with a WCF client consuming a Java web service and using MutualCertificateDuplex binding with X509 certificates. 

In this case, after doing the required configuration, the WCF client was throwing the following exception:
"The EncryptedKey clause was not wrapped with the required encryption token 'System.IdentityModel.Tokens.X509SecurityToken'."

The problem only occurred when the service was using certificates issued by a third party Certificate Authority (CA), in this case the CA was VeriSign. The same problem did not reproduce when using certificates issued by a test CA. Therefore, if the response from the Java service contained a CA certificate issued by a test Microsoft Certificate Server, things worked fine. However, when the CA certificate was from VeriSign, the WCF client failed with the above error. It is worth noting that the final issue is not VeriSign-specific; the issue was reproducible with several other common CAs.

Here is the exact exception from the WCF traces:
<ExceptionString>System.ServiceModel.Security.MessageSecurityException: The EncryptedKey clause was not wrapped with the required encryption token 'System.IdentityModel.Tokens.X509SecurityToken'.</ExceptionString>

This exception usually indicates that the certificate used in the incoming message is not what the client configured.  We re-verified the configuration to ensure that correct certificates were set on both ends.

Upon detailed investigation, we found an underlying cryptography exception being thrown when attempting to find/load the incoming certificate:
Exception type: System.Security.Cryptography.CryptographicExceptionMessage: The string contains an invalid X500 name attribute key, oid, value or delimiter.
Furthermore, the reason for the failure was that the incoming CA name in the message contained a comma (,) which was being normalized differently by the java side - as an escaped comma (\,) versus the windows side which does not support the escaped comma.

Here is the soap message from the java service containing a reference to a certificate issued by VeriSign CA (this fails):
<X509Data> <X509IssuerSerial> <X509IssuerName>CN=VeriSign Class 1 Individual Subscriber CA - G3,OU=VeriSign Trust Network,O=VeriSign\, Inc.,C=US</X509IssuerName> <X509SerialNumber>11223344556677889900112233445566778899</X509SerialNumber> </X509IssuerSerial></X509Data>
Note the comma in the name is escaped (O=VeriSign \, Inc.)

Here is the soap message from the java service containing reference to certificate issued by a Microsoft Test CA (this works):
<X509Data> <X509IssuerSerial> <X509IssuerName>CN=Test Employer,OU=Test Employer HR,O=Test Employer,L=Test City,ST=Test State,C=US</X509IssuerName> <X509SerialNumber>11223344556677889900</X509SerialNumber> </X509IssuerSerial></X509Data>This works. Note that name of the CA does not contain any commas or other special characters.


The First Fix – Use the Certificate’s ‘Thumbprint’

One way to get around this issue would be to simply change referencing the certificate using thumbprint instead of IssuerSerial on the java service side.  You can then make the same change on the WCF side to use ThumbPrint. This solution is reasonably easy to implement.  However, it requires a change on either side, which may not always be possible.

Reference: https://msdn.microsoft.com/en-us/library/system.servicemodel.security.tokens.x509keyidentifierclausetype.aspx

To make the change on the WCF client side you can specify the binding in code similar to the following SAMPLE. The change to note here is the specification of the X509KeyIdentifierClauseType to Thumbprint:

 namespace ConsoleApplication1


    class Program


        static void Main(string[] args)


            ServiceReference1.TestServiceClient prxy = new ServiceReference1.TestServiceClient(CreateClientBinding(), new System.ServiceModel.EndpointAddress(new Uri("https://www.myserver.com:443/ServiceWeb/services/TestService"), EndpointIdentity.CreateDnsIdentity("www.myserver.com"), new AddressHeaderCollection()));

            prxy.ClientCredentials.ClientCertificate.SetCertificate(StoreLocation.CurrentUser, StoreName.My, X509FindType.FindByThumbprint, "92 9e 78 89 b0 53 92 9e 78 89 b0 53 92 9e 78 89 b0 53 92 9e");

            prxy.ClientCredentials.ServiceCertificate.SetDefaultCertificate(StoreLocation.CurrentUser, StoreName.TrustedPeople, X509FindType.FindByThumbprint, "b3 89 a9 69 b3 89 a9 69 b3 89 a9 69 b3 89 a9 69 b3 89 a9 69");

            Console.WriteLine(prxy.TestMethod("xxxxx123", "1234567"));


        public static Binding CreateClientBinding()


            AsymmetricSecurityBindingElement abe = (AsymmetricSecurityBindingElement)SecurityBindingElement.CreateMutualCertificateDuplexBindingElement(



            abe.DefaultAlgorithmSuite = System.ServiceModel.Security.SecurityAlgorithmSuite.TripleDesRsa15;

            X509SecurityTokenParameters istp = abe.InitiatorTokenParameters as X509SecurityTokenParameters;

            if (istp != null)


                istp.X509ReferenceStyle = X509KeyIdentifierClauseType.Thumbprint;


            X509SecurityTokenParameters rstp = abe.RecipientTokenParameters as X509SecurityTokenParameters;

            if (rstp != null)


                rstp.X509ReferenceStyle = X509KeyIdentifierClauseType.Thumbprint;


            TextMessageEncodingBindingElement soapVersion = new TextMessageEncodingBindingElement(MessageVersion.Soap11, System.Text.Encoding.UTF8);

            HttpsTransportBindingElement transport = new HttpsTransportBindingElement();

            return new CustomBinding(abe, soapVersion, transport);




This resolves the issue.

A Host Only Fix – Use a WCF Custom Message Encoder (https://msdn.microsoft.com/en-us/library/ee960159.aspx)

Another way to resolve the issue would be to make use of a Custom Message Encoder. A Custom Message Encoder will allow you to modify and change the format of the incoming certificate identifier to a format that will be understood by WCF.  For example if the incoming certificate format was:
CN=VeriSign Class 1 Individual Subscriber CA - G3,OU=Persona Not Validated,OU=Terms of use at https://www.verisign.com/rpa (c)09,OU=VeriSign Trust Network,O=VeriSign\, Inc.,C=US

And the expected certificate name string was:
CN=VeriSign Class 1 Individual Subscriber CA - G3, OU=Persona Not Validated, OU=Terms of use at https://www.verisign.com/rpa (c)09, OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US

(Note the quotes around VeriSign, Inc.
If the incoming string and expected strings do not match, we’ll see the exception.)

Then you could use a custom message encoder to change the incoming certificate string to match what is expected.  Alternatively, you could choose to write a custom message encoder that changes the reference from an issuer serial to a thumbprint on the incoming message.

We resolved this issue by using a custom message encoder. This message encoder will look at the security header in the envelope and find all X509IssuerName Elements.  It then finds all instances of ‘\,’ in the incoming string and wraps quotes around the affected attributes.

Such as the <X509IssuerName> below:
<X509Data> <X509IssuerSerial> <X509IssuerName>CN=VeriSign Class 1 Individual Subscriber CA - G3,OU=VeriSign Trust Network,O=VeriSign\, Inc.,C=US </X509IssuerName> ``<X509SerialNumber>22677986226779862267798622677986226779</X509SerialNumber> </X509IssuerSerial></X509Data>

The certificate string above would change to:
<X509Data> <X509IssuerSerial> <X509IssuerName>CN=VeriSign Class 1 Individual Subscriber CA - G3,OU=VeriSign Trust Network,O=”VeriSign, Inc.”,C=US</X509IssuerName> <X509SerialNumber>22677986226779862267798622677986226779</X509SerialNumber> </X509IssuerSerial></X509Data>

The code for the custom message encoder used here is attached for reference. The ModifyIssuerName function does the string manipulation work of wrapping quotes around the affected attributes.  

You can set up a custom message encoder as follows:
1) Put the CertFixEscapedComma.dll in the bin directory.2) Add the bindingElementExtension in the config file:<system.serviceModel> <extensions> <bindingElementExtensions> <add name="CertFixEscapedComma" type="CertFixEscapedComma.CertRefBEExtentionElement, CertFixEscapedComma, Version=, Culture=neutral, PublicKeyToken=null" /> </bindingElementExtensions> </extensions>``3) Using a custom binding in your endpoint, create it using the new Encoder:``<customBinding> <binding name="CertFixer"> <CertFixEscapedComma/> <httpTransport/> </binding></customBinding>