C#/IRS ACA - Sending a web service request with both MTOM attachment and GZip encoding using WCF 4.5

Question

We are trying to send data to IRS through an exposed web service for the ACA data transmission and we could not make the WSE 3.0 approach to work because of the order of Timestamp and Signature elements in the Security Header. XSD validation at the IRS side is showing an error when the TimeStamp element is appearing before Signature element. When we used policy assertion for manually generating the security header with the right order of Signature and Timestamp elements, IRS web service is showing "Invalid WS Security Header" Error.

Can anyone having the same problem, please let us know what could be a possible fix. Using WCF 4.5 instead of WSE 3.0 is the solution, can you please provide a working sample which handles both MTOM attachment & GZip encoding using WCF 4.5

Any help would be greatly appreciated.

EDIT: Following is the Status Request Soap Envelope we are sending now with the whole XML created by manually. It still shows TPE - 1122 Invalid Security Header Error. Are there any mistakes with this request?

<?xml version="1.0" encoding="utf-8"?>
<soapenv:Envelope xmlns:oas1="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:us:gov:treasury:irs:msg:irstransmitterstatusrequest" xmlns:urn1="urn:us:gov:treasury:irs:ext:aca:air:7.0" xmlns:urn2="urn:us:gov:treasury:irs:common" xmlns:urn3="urn:us:gov:treasury:irs:msg:acasecurityheader">
    <soapenv:Header xmlns:wsa="http://www.w3.org/2005/08/addressing">
        <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <ds:Signature Id="SIG-E68EBBF1696C5DD4AA143353323390579" xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                <ds:SignedInfo>
                    <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#WithComments" />
                    <ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
                    <ds:Reference URI="#TS-82E7E6716E615C14D6144736030985954">
                        <ds:Transforms>
                            <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
                                <InclusiveNamespaces PrefixList="wsse wsa oas1 soapenv urn urn1 urn2 urn3" xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" />
                            </ds:Transform>
                        </ds:Transforms>
                        <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <ds:DigestValue>[Digest_Value_Replaced]</ds:DigestValue>
                    </ds:Reference>
                    <ds:Reference URI="#id-82E7E6716E615C14D6144736030986559">
                        <ds:Transforms>
                            <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
                                <InclusiveNamespaces PrefixList="oas1 soapenv urn1 urn2 urn3" xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" />
                            </ds:Transform>
                        </ds:Transforms>
                        <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <ds:DigestValue>[Digest_Value_Replaced]</ds:DigestValue>
                    </ds:Reference>
                    <ds:Reference URI="#id-82E7E6716E615C14D6144736030986558">
                        <ds:Transforms>
                            <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
                                <InclusiveNamespaces PrefixList="wsa oas1 soapenv urn1 urn2 urn3" xmlns="http://www.w3.org/2001/10/xml-exc-c14n#" />
                            </ds:Transform>
                        </ds:Transforms>
                        <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <ds:DigestValue>[Digest_Value_Replaced]</ds:DigestValue>
                    </ds:Reference>
                </ds:SignedInfo>
                <ds:SignatureValue>[Signature_Value_Replaced]</ds:SignatureValue>
                <ds:KeyInfo Id="KI-82E7E6716E615C14D6144736030986456">
                    <wsse:SecurityTokenReference wsu:Id="STR-82E7E6716E615C14D6144736030986457" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
                        <wsse:KeyIdentifier EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary" ValueType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3">[Cert_Value_Replaced]</wsse:KeyIdentifier>
                    </wsse:SecurityTokenReference>
                </ds:KeyInfo>
            </ds:Signature>
            <wsu:Timestamp wsu:Id="TS-82E7E6716E615C14D6144736030985954">
                <wsu:Created>2016-03-23T09:53:23:55Z</wsu:Created>
                <wsu:Expires>2016-03-23T10:03:23:55Z</wsu:Expires>
            </wsu:Timestamp>
        </wsse:Security>
        <urn:ACABusinessHeader wsu:Id="id-82E7E6716E615C14D6144736030986558" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <urn1:UniqueTransmissionId>uuid:SYS12:tcc_cd::T</urn1:UniqueTransmissionId>
            <urn2:Timestamp>2016-03-23T09:53:23:55Z</urn2:Timestamp>
        </urn:ACABusinessHeader>
        <urn3:ACASecurityHeader/>
        <wsa:Action>RequestSubmissionStatusDetail</wsa:Action>
    </soapenv:Header>
    <soapenv:Body>
        <urn:ACABulkRequestTransmitterStatusDetailRequest version="1.0" wsu:Id="id-82E7E6716E615C14D6144736030986559" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <urn1:ACABulkReqTrnsmtStsReqGrpDtl>
                <urn2:ReceiptId>[ReceiptId_Value_Replaced]</urn2:ReceiptId>
            </urn1:ACABulkReqTrnsmtStsReqGrpDtl>
        </urn:ACABulkRequestTransmitterStatusDetailRequest>
    </soapenv:Body>
</soapenv:Envelope>

EDIT 2:: Here is the method I am using to sign the envelope and create the Signature Element. STILL GETTING SECURITY HEADER ERROR... :(

 public static string getSignedXML(XmlDocument xmlDoc, RSACryptoServiceProvider key, string signatureNamespacePrefix,
                                        string sTimeStampId, string sManifestId, string sBusHeaderId)
        {
            xmlDoc.PreserveWhitespace = false;  //Ignore the whitespace in XML   

            SignedXml signedXml = new CustomIdSignedXml(xmlDoc); //If Id attribute needs to have a prefix. This is not needed as per latest Reference Guide Info.
            //SignedXml signedXml = new SignedXml(xmlDoc);

            // Add the key to the SignedXml document.
            signedXml.SigningKey = key;
            signedXml.SignedInfo.SignatureMethod = SignedXml.XmlDsigRSASHA1Url;
            signedXml.SignedInfo.CanonicalizationMethod = SignedXml.XmlDsigExcC14NWithCommentsTransformUrl;
            signedXml.SignedInfo.SignatureMethod = SignedXml.XmlDsigRSASHA1Url;
            signedXml.Signature.Id = "SIG-E68EBBF1696C5DD4AA143353323390579";

            //------------------------------------------------------------------------
            //START OF: Adding Manifest, BusinessHeader and TimeStamp References
            //------------------------------------------------------------------------
            //Adding Timestamp Reference
            XmlDsigExcC14NTransform timeStampTransform = new XmlDsigExcC14NTransform();
            timeStampTransform.InclusiveNamespacesPrefixList = "wsse wsa oas1 soapenv urn urn1 urn2 urn3";
            Reference reference = new Reference("#" + sTimeStampId);
            reference.AddTransform(timeStampTransform);
            // Add the reference to the SignedXml object.
            signedXml.AddReference(reference);

            //Adding Business Header Reference
            XmlDsigExcC14NTransform busHeaderTransform = new XmlDsigExcC14NTransform();
            busHeaderTransform.InclusiveNamespacesPrefixList = "wsa oas1 soapenv urn1 urn2 urn3";
            reference = new Reference("#" + sBusHeaderId);
            reference.AddTransform(busHeaderTransform);
            // Add the reference to the SignedXml object.
            signedXml.AddReference(reference);

            //Adding Manifest Request Dtl Reference
            XmlDsigExcC14NTransform manifestTransform = new XmlDsigExcC14NTransform();
            manifestTransform.InclusiveNamespacesPrefixList = "oas1 soapenv urn1 urn2 urn3";
            reference = new Reference("#" + sManifestId);
            reference.AddTransform(manifestTransform);
            // Add the reference to the SignedXml object.
            signedXml.AddReference(reference);
            //------------------------------------------------------------------------
            //END OF: Adding Manifest, BusinessHeader and TimeStamp References
            //------------------------------------------------------------------------

            signedXml.ComputeSignature();

            XmlElement xmlSignature = signedXml.GetXml(); //Get the singed XML block

            if (!string.IsNullOrEmpty(signatureNamespacePrefix))
            {
                //Here we set the namespace prefix on the signature element and all child elements to "ds", invalidating the signature.
                AssignNameSpacePrefixToElementTree(xmlSignature, "ds");

                //So let's recompute the SignatureValue based on our new SignatureInfo...

                //For XPath
                XmlNamespaceManager namespaceManager = new XmlNamespaceManager(xmlDoc.NameTable);
                namespaceManager.AddNamespace("ds", "http://www.w3.org/2000/09/xmldsig#"); //this prefix is arbitrary and used only for XPath
                XmlElement xmlSignedInfo = xmlSignature.SelectSingleNode("ds:SignedInfo", namespaceManager) as XmlElement;

                //Canonicalize the SignedInfo element
                XmlDsigC14NTransform transform = new XmlDsigC14NTransform();
                XmlDocument signedInfoDoc = new XmlDocument();
                signedInfoDoc.LoadXml(xmlSignedInfo.OuterXml);
                transform.LoadInput(signedInfoDoc);

                //Compute the new SignatureValue
                string signatureValue = Convert.ToBase64String(key.SignData(transform.GetOutput() as MemoryStream, new SHA1CryptoServiceProvider()));
                //Set it in the xml
                XmlElement xmlSignatureValue = xmlSignature.SelectSingleNode("ds:SignatureValue", namespaceManager) as XmlElement;
                xmlSignatureValue.InnerText = signatureValue;
            }

            //xmlDoc.DocumentElement.AppendChild(xmlDoc.ImportNode(xmlSignature, true));

            //----------------------------------------------------------------------------------
            //START OF: Add Key Info Element also to the XML after SignatureValue Node.
            //----------------------------------------------------------------------------------
            X509Certificate2 mycert = new X509Certificate2(<Cert_Path>, <Cert_Password>);
            //bulkReqTransmitService.ClientCertificates.Add(mycert);
            var exported = mycert.Export(X509ContentType.Cert, <Cert_Password>);
            var base64 = Convert.ToBase64String(exported);

            StringBuilder sbKeyInfo = new StringBuilder();
            string dsStartTagPrefix = "";
            string dsEndTagPrefix = "/";
            if (!string.IsNullOrEmpty(signatureNamespacePrefix))
            {
                dsStartTagPrefix = "ds:";
                dsEndTagPrefix = "/ds:";
            }
            sbKeyInfo.Append("<root xmlns:wsse=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd\" xmlns:wsu=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\" xmlns:ds=\"http://www.w3.org/2000/09/xmldsig#\">");
            sbKeyInfo.Append("<" + dsStartTagPrefix + "KeyInfo Id=\"KI-82E7E6716E615C14D6144736030986456\">");
            sbKeyInfo.Append("<wsse:SecurityTokenReference wsu:Id=\"STR-82E7E6716E615C14D6144736030986457\">");
            sbKeyInfo.Append("<wsse:KeyIdentifier EncodingType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\" ValueType=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509v3\">" + base64.ToString());
            sbKeyInfo.Append("</wsse:KeyIdentifier>");
            sbKeyInfo.Append("</wsse:SecurityTokenReference>");
            sbKeyInfo.Append("<" + dsEndTagPrefix + "KeyInfo>");
            sbKeyInfo.Append("</root>");

            XmlDocument tempDoc = new XmlDocument();
            tempDoc.LoadXml(sbKeyInfo.ToString());
            XmlNode oNode = tempDoc.DocumentElement;
            //necessary for crossing XmlDocument contexts
            XmlNode importNode = xmlSignature.OwnerDocument.ImportNode(oNode.FirstChild, true);
            xmlSignature.AppendChild(importNode);
            //----------------------------------------------------------------------------------
            //END OF: Add Key Info Element also to the XML after SignatureValue Node.
            //----------------------------------------------------------------------------------

            return xmlSignature.OuterXml;
        }


public class CustomIdSignedXml : SignedXml
    {
        public CustomIdSignedXml(XmlDocument xml) : base(xml)
        {
        }

        public CustomIdSignedXml(XmlElement xmlElement)
            : base(xmlElement)
        {
        }

        public override XmlElement GetIdElement(XmlDocument doc, string id)
        {
            // check to see if it's a standard ID reference
            XmlElement idElem = base.GetIdElement(doc, id);

            if (idElem == null)
            {
                XmlNamespaceManager nsManager = new XmlNamespaceManager(doc.NameTable);
                nsManager.AddNamespace("wsu", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd");

                //idElem = doc.SelectSingleNode("//*[@p5:Id=\"" + id + "\"]", nsManager) as XmlElement;

                string xml = doc.SelectSingleNode("//*[@wsu:Id=\"" + id + "\"]", nsManager).OuterXml;
                XmlDocument tempDoc = new XmlDocument();
                tempDoc.LoadXml(xml);
                XmlElement xEle = tempDoc.DocumentElement;

                idElem = xEle;
            }

            return idElem;
        }
    }

Answer 1

I do not yet have the reputation to actually post a comment on your posts >:(

Here are a couple of follow-up questions for you regarding the process you are using.

1. Are you using the WSDL Service Reference/Proxy classes to make your calls to the IRS through the ISS-A2A Channel?
2. After implementing the three class files from the samples, I am receiving the following error. Do you see anything out of the ordinary that I am doing incorrectly?

Additional information: The type '[Project].GZipEncoder.GzipMessageEncodingElement, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' registered for extension 'gzipMessageEncoding' could not be loaded.

The <gzipMessageEncoding> element in my <customBinding> is giving me a warning which I think is related to the above error.

app.config

<system.serviceModel>
  <extensions>
    <bindingElementExtensions>
      <add name="gzipMessageEncoding" type="[ProjectName].GZipEncoder.GzipMessageEncodingElement, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
    </bindingElementExtensions>
  </extensions>
  <client>
    <endpoint />
    <metadata>
      <policyImporters>
        <extension type="[ProjectName].GZipEncoder.GZipMessageEncodingBindingElementImporter, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
      </policyImporters>
    </metadata>
  </client>
  <bindings>
    <customBinding>
      <binding name="BulkRequestTransmitterBinding">
        <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />
        <httpsTransport />
      </binding>
    </customBinding>
  </bindings>
</system.serviceModel>

Answer 2

EDIT: Verified none of this will work due to WCF's messaging architecture. IRS structure requires modification of HTTP headers for MTOM to work properly which cannot be modified in WCF after the point during which XML serialization occurs. You will have to construct your XML manually and send it via HttpClient and do GZip and MTOM encoding yourself. Sorry for the bad news :(

WCF gzip: Take the sample here and integrate the classes and client config with your app. https://msdn.microsoft.com/en-us/library/ms751458.aspx

You'll have to tweak the content type to be "text/xml" instead of "application/x-gzip".

This still will be rejected by the IRS because this doesn't set the "Content-Encoding" header. You can set the content encoding this way:

using (new OperationContextScope(transmitterClient.InnerChannel))
{
// Add a HTTP Header to an outgoing request
HttpRequestMessageProperty requestMessage = new HttpRequestMessageProperty();
requestMessage.Headers["Content-Encoding"] = "gzip";
OperationContext.Current.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = requestMessage;
return port.BulkRequestTransmitter(request);
}

Thanks to this blog post for that: http://blogs.msdn.com/b/wsdevsol/archive/2014/02/07/adding-custom-messageheader-and-http-header-to-a-wcf-method-call.aspx

Additionally, IRS's response is NOT gzipped, so you need to modify the class to not try to decompress the response.

Additionally, additionally, the innerMessageEncoding messageVersion default is NOT compatible with the IRS. You have to modify the ApplyConfiguration method of the GZipMessageEncodingElement to instantiate the TextMessageEncodingBindingElement to the MessageVersion.Soap11WSAddressing10 enum with Encoding.UTF8

MTOM My bindings look like this:

<bindings>
  <customBinding>
    <binding name="BulkRequestTransmitterBinding">
      <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />
      <security enableUnsecuredResponse="true" authenticationMode="MutualCertificate" messageSecurityVersion="WSSecurity10WSTrust13WSSecureConversation13WSSecurityPolicy12BasicSecurityProfile10"/>
      <httpsTransport />
    </binding>
  </customBinding>
</bindings>  

And then a custom behavior aiming at my cert.