How to configure SignedXML to sign XML document with prefixes and other ceremonies?

Question

I need to sign XML documents with a cert, using .NET Core and SignedXml, but I don't know how to configure some of the elements expected by the other side in the signature tag. I've checked many articles, one of this, authored by Rick Stahl, is pretty similar, but I am still validated with errors. Unfortunately, I cannot send any good log message, I have only a general answer, that signature is invalid.

The expected content of the signature looks like below. It looks like all details - tags, prefixes, identifiers, schemas - are required.

        <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <ds:Signature Id="{{someValue}}" xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
                <ds:SignedInfo>
                    <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
                        <ec:InclusiveNamespaces PrefixList="soapenv urn urn1 urn2 urn3 urn4" xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#"/>
                    </ds:CanonicalizationMethod>
                    <ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/>
                    <ds:Reference URI="#{{someValue}}">
                        <ds:Transforms>
                            <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
                                <ec:InclusiveNamespaces PrefixList="urn urn1 urn2 urn3 urn4" xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#"/>
                            </ds:Transform>
                        </ds:Transforms>
                        <ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/>
                        <ds:DigestValue>VALUEOFDIGIESTVALUE</ds:DigestValue>
                    </ds:Reference>
                </ds:SignedInfo>
                <ds:SignatureValue>VALUEOFSIGNATUREVALUE</ds:SignatureValue>
                <ds:KeyInfo Id="{{someValue}}">
                    <wsse:SecurityTokenReference wsu:Id="{{someValue}}">
                        <ds:X509Data>
                            <ds:X509IssuerSerial>
                                <ds:X509IssuerName>{{someValue}}</ds:X509IssuerName>
                                <ds:X509SerialNumber>{{someValue}}</ds:X509SerialNumber>
                            </ds:X509IssuerSerial>
                        </ds:X509Data>
                    </wsse:SecurityTokenReference>
                </ds:KeyInfo>
            </ds:Signature>
        </wsse:Security>

EVERY TIP IS HELPFUL, I don't expect overall answer, but I am interested the most of these questions:

• How to define prefixes ds and ec?
• How to configure wsse:SecurityTokenReference?
• Where to find all identifiers, marked as {{someValue}}

After two weeks of fighting with SignedXml I need to confirm the same state in 2021, which was described by Rick Stahl in 2008:

It really is no wonder that security for signatures and encryption with certificates is so lightly used on Windows. It's a royal pain in the ass, horribly documented and requires a host of tools.

Although colleagues from the Java county say, that this is much simpler abroad, I think the problem is overcomplicated specifications of SOAP services.

Answer 1

This is a partial solution, based on mentioned Rick Stahl's article. Let's suppose we have that XML document to sign:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP:Envelope xmlns:r1="http://www.routeone.com/namespace.messaging.diag#"
               xmlns:star="http://www.starstandards.org/STAR"
               xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
               xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/"
               xmlns:oa="http://www.openapplications.org/oagis">
  <SOAP:Body>
    <!-- data to be signed here -->
  </SOAP:Body>
</SOAP:Envelope>

With that code:

using System;
using System.IO;
using System.Security.Cryptography.X509Certificates;
using System.Security.Cryptography.Xml;
using System.Xml;

namespace XmlSigner
{
    class Program
    {
        static void Main(string[] args)
        {
            // Create a new XML document.
            var doc = new XmlDocument();

            // Format the document to ignore white spaces.
            doc.PreserveWhitespace = false;

            // Load the passed XML file using it's name.
            doc.Load(new XmlTextReader("C:\\Repos\\request.xml"));

            // Initialize the certificate.
            var certificate = new X509Certificate2("C:\\Repos\\certificate.pfx", "youdontguessthispassword");

            // Sign the document
            var signedDoc = SignSoapBody(doc, certificate);

            // Write to the output file.
            File.WriteAllText("C:\\Repos\\signed.xml", signedDoc.OuterXml);

            // Validate the result.
            if (ValidateSoapBodySignature(signedDoc, certificate))
                Console.WriteLine("Everything looks good, keep going.");
            else
                Console.WriteLine("You've screwed, back to work.");
        }

        private const string STR_SOAP_NS = "http://schemas.xmlsoap.org/soap/envelope/";
        private const string STR_SOAPSEC_NS = "http://schemas.xmlsoap.org/soap/security/2000-12";

        /// <summary>
        /// Signs the SOAP document and adds a digital signature to it.
        /// 
        /// Note a lot of optional settings are applied against
        /// key and certificate info to match the required XML document
        /// structure the server requests.
        /// </summary>
        /// <param name="xmlDoc"></param>
        /// <param name="certFriendlyName">Friendly Name of Cert installed in the Certificate Store under CurrentUser | Personal</param>
        /// <returns></returns>
        static public XmlDocument SignSoapBody(XmlDocument xmlDoc, X509Certificate2 cert)
        {
            // Add search Namespaces references to ensure we can reliably work 
            // against any SOAP docs regardless of tag naming
            XmlNamespaceManager ns = new XmlNamespaceManager(xmlDoc.NameTable);
            ns.AddNamespace("SOAP", STR_SOAP_NS);
            ns.AddNamespace("SOAP-SEC", STR_SOAPSEC_NS);

            // Grab the body element - this is what we create the signature from
            XmlElement body = xmlDoc.DocumentElement.SelectSingleNode(@"//SOAP:Body", ns) as XmlElement;
            if (body == null)
                throw new ApplicationException("No body tag found");

            // We'll only encode the <SOAP:Body> - add id: Reference as #Body
            var bodyId = $"id-{Guid.NewGuid()}";
            body.SetAttribute("id", bodyId);

            // Signed XML will create Xml Signature - Xml fragment
            SignedXml signedXml = new SignedXml(xmlDoc);
            signedXml.Signature.Id = $"sid-{Guid.NewGuid()}";
            signedXml.SignedInfo.SignatureMethod = "http://www.w3.org/2000/09/xmldsig#rsa-sha1";

            // Create a KeyInfo structure
            KeyInfo keyInfo = new KeyInfo();
            keyInfo.Id = $"kid-{Guid.NewGuid()}";

            // The actual key for signing - MAKE SURE THIS ISN'T NULL!
            signedXml.SigningKey = cert.PrivateKey;

            // Specifically use the issuer and serial number for the data rather than the default
            KeyInfoX509Data keyInfoData = new KeyInfoX509Data();
            keyInfoData.AddIssuerSerial(cert.Issuer, cert.GetSerialNumberString());
            keyInfo.AddClause(keyInfoData);


            // provide the certficate info that gets embedded - note this is only
            // for specific formatting of the message to provide the cert info
            signedXml.KeyInfo = keyInfo;

            // Again unusual - meant to make the document match template
            signedXml.SignedInfo.CanonicalizationMethod = SignedXml.XmlDsigExcC14NTransformUrl;

            // Now create reference to sign: Point at the Body element
            Reference reference = new Reference();
            reference.DigestMethod = "http://www.w3.org/2000/09/xmldsig#sha1";
            reference.Uri = $"#{bodyId}";  // reference id=body section in same doc
            reference.AddTransform(new XmlDsigExcC14NTransform());  // required to match doc
            signedXml.AddReference(reference);

            // Finally create the signature
            signedXml.ComputeSignature();

            // Result is an XML node with the signature detail below it
            // Now let's add the sucker into the SOAP-HEADER
            XmlElement signedElement = signedXml.GetXml();

            // Create SOAP-SEC:Signature element
            XmlElement soapSignature = xmlDoc.CreateElement("Security", STR_SOAPSEC_NS);
            soapSignature.Prefix = "wsse";
            //soapSignature.SetAttribute("MustUnderstand", "", "1");

            // And add our signature as content
            soapSignature.AppendChild(signedElement);

            // Now add the signature header into the master header
            XmlElement soapHeader = xmlDoc.DocumentElement.SelectSingleNode("//SOAP:Header", ns) as XmlElement;
            if (soapHeader == null)
            {
                soapHeader = xmlDoc.CreateElement("Header", STR_SOAP_NS);
                soapHeader.Prefix = "SOAP";
                xmlDoc.DocumentElement.InsertBefore(soapHeader, xmlDoc.DocumentElement.ChildNodes[0]);
            }
            soapHeader.AppendChild(soapSignature);

            return xmlDoc;
        }

        /// <summary>
        /// Validates the Xml Signature in a document.
        /// 
        /// This routine is significantly simpler because the key parameters
        /// are embedded into the signature itself. All that's needed is a
        /// certificate to provide the key - the rest can be read from the
        /// Signature itself.
        /// </summary>
        /// <param name="doc"></param>
        /// <param name="publicCertFileName"></param>
        /// <returns></returns>
        static public bool ValidateSoapBodySignature(XmlDocument doc, X509Certificate2 cert)
        {
            // Load the doc this time
            SignedXml sdoc = new SignedXml(doc);

            // Find the signature and load it into SignedXml
            XmlNodeList nodeList = doc.GetElementsByTagName("Signature");
            sdoc.LoadXml((XmlElement)nodeList[0]);

            // Now read the actual signature and validate
            bool result = sdoc.CheckSignature(cert, true);

            return result;
        }
    }
}

I have this result:

<?xml version="1.0" encoding="UTF-8"?>
<SOAP:Envelope xmlns:r1="http://www.routeone.com/namespace.messaging.diag#" xmlns:star="http://www.starstandards.org/STAR" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/" xmlns:oa="http://www.openapplications.org/oagis">
    <SOAP:Header>
        <wsse:Security xmlns:wsse="http://schemas.xmlsoap.org/soap/security/2000-12">
            <Signature Id="sid-f3d74053-b47a-4740-b105-74ba1a550d38" xmlns="http://www.w3.org/2000/09/xmldsig#">
                <SignedInfo>
                    <CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
                    <SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1" />
                    <Reference URI="#id-9718651b-c7a4-47a0-bcd1-f89e1a5f8395">
                        <Transforms>
                            <Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
                        </Transforms>
                        <DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
                        <DigestValue>SOME_SECRET_VALUES</DigestValue>
                    </Reference>
                </SignedInfo>
                <SignatureValue>SOME_SECRET_VALUES</SignatureValue>
                <KeyInfo Id="kid-626cac63-9f43-46a9-80e3-ed350ef6cea2">
                    <X509Data>
                        <X509IssuerSerial>
                            <X509IssuerName>CN=XX, OU=XX, O=XX, L=XX, C=XX</X509IssuerName>
                            <X509SerialNumber>SOME_SECRET_VALUES</X509SerialNumber>
                        </X509IssuerSerial>
                    </X509Data>
                </KeyInfo>
            </Signature>
        </wsse:Security>
    </SOAP:Header>
    <SOAP:Body id="id-9718651b-c7a4-47a0-bcd1-f89e1a5f8395"> <!-- data to be signed here -->
    </SOAP:Body>
</SOAP:Envelope>