Tuesday, March 18, 2014

Apache Santuario - XML Security for Java 2.0.0

In recent posts I have described some of the new features of the forthcoming Apache WSS4J 2.0.0 release. In particular, I focused on the changes and improvements to the existing "in-memory" (DOM-based) WS-Security implementation. However, the biggest new feature of WSS4J 2.0.0 will be a new streaming (StAX-based) WS-Security stack. In the next couple of posts, we will examine the core streaming XML Security functionality that will be available in the Apache Santuario - XML Security for Java 2.0.0 release.

The existing Apache Santuario - XML Security for Java implementation is DOM-based. What this essentially means is that the complete XML Document must be read into memory before being signed or verified. For large XML Documents this carries a corresponding performance penalty. In addition, it means that XML Signature is not feasible for very large documents. In contrast, the StAX API uses a streaming approach, resulting in very low memory usage.

The streaming XML/WS-Security functionality is split between the Apache Santuario and WSS4J projects. The Apache Santuario project contains the core streaming capabilities for XML Signature and Encryption. The Apache WSS4J project builds on this functionality to deliver a streaming WS-Security stack. The entire streaming security functionality is the work of Marc Giger, who donated his SWSSF project to Apache. Marc has done an outstanding job to implement this technically difficult task, and also to integrate the functionality into existing Apache projects. If you wish to experiment with the new functionality, it is available via the 2.0.0-rc1 release.

1) XML Signature test-cases

In this post we will focus on XML Signature only. I have uploaded some test-cases to github to show how to use the new StAX-based API. There are currently three junit tests in this project:
2) Outbound StAX configuration

To see how to configure the new outbound StAX-based XML Signature functionality, take a look at the "signUsingStAX" method used by the tests. The streaming XML Security functionality is configured by populating a XMLSecurityProperties Object. You must typically call the following methods:
  • properties.setAction(XMLSecurityConstants.Action) - an "Action" to perform, which for XML Signature purposes is XMLSecurityConstants.SIGNATURE. 
  • properties.setSignatureKey(Key) - The signing key. Can be a PrivateKey or SecretKey.
  • properties.setSigningCerts(X509Certificate[]) - The signing certs. May be inserted into the Signature KeyInfo depending on the Signature KeyIdentifier.
  • properties.addSignaturePart(SecurePart) - Add a SecurePart to sign, e.g. sign a given QName. You can also specify a digest method or transform algorithm here. You can also sign the entire request via SecurePart.setSecureEntireRequest(boolean).
  • properties.setSignatureAlgorithm(String) - Signature Algorithm to use. The default depends on the signing key.
  • properties.setSignatureDigestAlgorithm(String) - Signature Digest Algorithm to use. The default is the URI for SHA-1.
  • properties.setSignatureCanonicalizationAlgorithm(String) - The canonicalization algorithm to use. The default is XMLSecurityConstants.NS_C14N_EXCL_OMIT_COMMENTS.
  • properties.setSignatureKeyIdentifier(SecurityTokenConstants.KeyIdentifier) - How to reference the signing key/cert. The default is SecurityTokenConstants.KeyIdentifier_IssuerSerial.  
The valid Signature KeyIdentifier types are:
  • SecurityTokenConstants.KeyIdentifier_KeyValue
  • SecurityTokenConstants.KeyIdentifier_KeyName
  • SecurityTokenConstants.KeyIdentifier_IssuerSerial
  • SecurityTokenConstants.KeyIdentifier_SkiKeyIdentifier
  • SecurityTokenConstants.KeyIdentifier_X509KeyIdentifier
  • SecurityTokenConstants.KeyIdentifier_X509SubjectName
  • SecurityTokenConstants.KeyIdentifier_NoKeyInfo
By default, the Signature Element is attached as the first child of the root document.

3) Inbound StAX configuration

To see how to configure the new inbound StAX-based XML Signature functionality, take a look at the "verifyUsingStAX" method used by the tests. As with signature creation, it is necessary to create a XMLSecurityProperties Object, and to tell it what "Action" to perform. In addition you must call the following method, unless the complete signing key is contained in the Signature KeyInfo:
  • properties.setSignatureVerificationKey(Key) - a key to use to verify the Signature.
In addition, it is possible to pass a SecurityEventListener to record what security events were performed (see TestSecurityEventListener used in the tests). This allows the user to check what keys were used for signature verification, what algorithms were used, what was signed etc. See the verifyUsingStAX method as above for some simple code to check the signing key and the elements that were signed.

55 comments:

  1. Hello,

    I'm trying to prove the signature verification with StAX.
    In my case if I change the content of XML after the signature has not reported any problems. It should return an error indicating that the document is invalid.

    How I can check if the content has been modified after signing and get the error?

    Thank you very much

    ReplyDelete
  2. How are you modifying the signature? Do you have a test-case?

    Colm.

    ReplyDelete
  3. You could create a github project + mail a link to it here. Alternatively send it to the CXF user list.

    Colm.

    ReplyDelete
  4. or you can try to document.getDocumentElement().getFirstChild().setNodeValue("NODE MODIFIED"); after the signature

    ReplyDelete

  5. That is just changing the value of the first node. If you look at the test-case, it is signing the "PaymentInfo" node. Hence changing the value of the first node does not affect signature verification.

    Colm.

    ReplyDelete
    Replies
    1. I want to check signature integrity: the content hash vs signatureValue hash.

      Xml signature is not only to know the identy of signer, you have to know if document is not manipulated after has signed.


      Delete
    2. Yes, but I am telling you that changing the first node does not affect digest calculation, as that node is not included in the signature. If the example signed the entire document, and the modification was made, then signature validation would fail.

      Colm.

      Delete
  6. This comment has been removed by the author.

    ReplyDelete
  7. I have the next example: ....

    After signature I have changed data of Person.

    The same invalid XML with SignatureDomTest:

    oct 16, 2014 1:54:28 PM org.apache.xml.security.signature.Reference verify
    Advertencia: Verification failed for URI ""
    oct 16, 2014 1:54:28 PM org.apache.xml.security.signature.Reference verify
    Advertencia: Expected Digest: Oyyx+K28+cp7kuUgcnANtTBdUwg=
    oct 16, 2014 1:54:28 PM org.apache.xml.security.signature.Reference verify
    Advertencia: Actual Digest: lE7NVwAKFheZoXEVLdxJXLWSY28=

    With SignatureStAXTest is not reporting any error.

    ReplyDelete
    Replies
    1. How are you changing the XML for the SignatureStAXTest though? It doesn't use DOM between signing +verifying, so if you are changing the DOM here it has no effect.

      Colm.

      Delete
  8. I have xml file in resources with data modified.
    I has commented the signature part SignatureUtils.signUsingStAX or DOM

    In other words I have declared inputstream with xml modified and call SignatureUtils.verifyUsingStAX() or SignatureUtils.verifyUsingDOM()

    The same document with StAX is not reporting errors and with DOM is reporting Verification failed

    ReplyDelete
  9. Could you create a pull request for the project on github that reproduces the problem, + I will take a look?

    Colm.

    ReplyDelete
  10. Hi, I have created fork for Digest Value Test

    ReplyDelete
  11. Working on it. Can you give me an email address to contact you at privately?

    Colm.

    ReplyDelete
  12. This comment has been removed by the author.

    ReplyDelete
  13. I need to sign a xml document provided by a customer. How can I using the stax api to sign a full xml file and add the signature at the end? How can I changed the element dsig:Signature to a ds:Signature element?

    ReplyDelete
  14. You can customize where the Signature is added via the following: https://issues.apache.org/jira/browse/SANTUARIO-405 - this has not yet been released though. For now, the prefix is hard-coded to "dsig". Please raise a JIRA if you want to be able to change it.

    Colm.

    ReplyDelete
  15. With this version I`ve two issues.

    Stax signer
    - When I set XMLSecurityProperties.setSignaturePosition(1); no signature is being written, with "0" the signature is written on the top of the file.

    Dom Verify
    - With this new version 2.0.3 I`m getting an exception, it was working with 2.0.2, but I need the RSA-PSS algorithm support, therefore I want to upgrade
    Caused by: org.apache.xml.security.exceptions.XMLSecurityException: Invalid digest of reference #ID_097f0764-9f73-4fb2-b2e0-7de370930288

    ReplyDelete
  16. Another question is, why does that code:

    String id = "ID_" + UUID.randomUUID().toString();
    elementToSign.setAttributeNS(null, "Id", id);
    elementToSign.setIdAttributeNS(null, "Id", true);

    transforms = new Transforms(document);
    transforms
    .addTransform("http://www.w3.org/2001/10/xml-exc-c14n#");
    xmlSignature.addDocument("#" + id, transforms,
    "http://www.w3.org/2000/09/xmldsig#sha1");

    set the id on the top of the xml document and also to the reference field?




    Is that a correct behaviour?

    ReplyDelete
  17. I tried to make a signature on root element (PurchaseOrder), and then a verification.
    But signature verification failed :(

    I have just replaced this line :
    namesToSign.add(new QName("urn:example:po", "PaymentInfo"));
    by :
    namesToSign.add(new QName("urn:example:po", "PurchaseOrder"));

    And it raises this error :
    org.apache.xml.security.exceptions.XMLSecurityException: Invalid digest of reference #G33e45112-ac8f-4ed2-9da9-12c5af4a782e

    I think that, when computing document digest, it doesn't remove "Signature" node.

    Can you say what to change to make a "root element signature verification" ?

    ReplyDelete
  18. You need to use an "enveloped signature transform" when you are signing the root node, and including the Signature inside the root node (as the test-case is doing). This tells the validation code not to include the Signature in the digest calculation.

    So for the DOM code in the test-case, add the following line in SignatureUtils.signUsingDOM:

    transforms.addTransform("http://www.w3.org/2000/09/xmldsig#enveloped-signature");

    For the streaming code in SignatureUtils.signUsingStAX:

    String[] transforms = new String[2];
    transforms[0] = "http://www.w3.org/2000/09/xmldsig#enveloped-signature";
    transforms[1] = "http://www.w3.org/2001/10/xml-exc-c14n#";
    securePart.setTransforms(transforms);

    Colm.

    ReplyDelete
  19. Hi,
    I've tested what you say, and it works great !

    Thank you very much !

    ReplyDelete
  20. Hi,

    At first thanks for the helpful test examples.

    I am not sure If the signature transform is the "thing" I am looking for.
    I have to nest the signature inside of the paymentInfo element as child. Is this somehow possible with santuario?

    Best Regards

    ReplyDelete
  21. Yes, see the enveloped signature transform comment above.

    Colm.

    ReplyDelete
    Replies
    1. Hi Colm,

      the transform doesn't have any effect on the Data. The only thing I see are some extra Debug lines in the logs.

      I am adding the transform by:

      SecurePart securePart = new SecurePart(namesToSign.get(0),
      SecurePart.Modifier.Content);

      String[] transforms = new String[2];
      transforms[0] = "http://www.w3.org/2000/09/xmldsig#enveloped-signature";
      transforms[1] = "http://www.w3.org/2001/10/xml-exc-c14n#";
      securePart.setTransforms(transforms);

      Best Regards

      Delete
    2. I am using the STAX-API btw.

      Delete
  22. I furthermore realized that there is put an ID field in the Tag .

    Is it somehow possible to avoid? Since the server I am communicating with, isn't able to handle this.

    ReplyDelete
  23. Hi,

    Is it possible to validate a XML with 2 signature with the Stax API ?
    By my side, I got a stack trace...

    To reproduce :
    - take the Stax example(SignatureStAXTest)
    - generate a 2nd keypair, and a new signature on the document (new key ID, new key pair)
    - re run the test

    java.lang.NullPointerException
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:386)
    at com.sun.xml.internal.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:374)
    at org.apache.xml.security.stax.ext.AbstractInputSecurityHeaderHandler.parseStructure(AbstractInputSecurityHeaderHandler.java:46)
    at org.apache.xml.security.stax.impl.processor.input.AbstractSignatureInputHandler.handle(AbstractSignatureInputHandler.java:58)
    at org.apache.xml.security.stax.impl.processor.input.XMLSecurityInputProcessor.processNextEvent(XMLSecurityInputProcessor.java:129)
    at org.apache.xml.security.stax.impl.InputProcessorChainImpl.processEvent(InputProcessorChainImpl.java:193)
    at org.apache.xml.security.stax.impl.XMLSecurityStreamReader.next(XMLSecurityStreamReader.java:78)
    at com.diehl.metering.izar.com.lib.security.XadesManager.verifyUsingStAX(XadesManager.java:261)
    ...


    :-(

    ReplyDelete
  24. Could you submit a pull request for the code in github with your change + I'll take a look?

    ReplyDelete
  25. The patch is quiet simple :

    diff --git a/apache/santuario/santuario-xml-signature/src/test/java/org/apache/coheigea/santuario/xmlsignature/SignatureStAXTest.java b/apache/santuario/santuario-xml-signature/src/test/java/org/apache/coheigea/santuario/xmlsignature/SignatureStAXTest.java
    index 220a9ef..7515dd0 100644
    --- a/apache/santuario/santuario-xml-signature/src/test/java/org/apache/coheigea/santuario/xmlsignature/SignatureStAXTest.java
    +++ b/apache/santuario/santuario-xml-signature/src/test/java/org/apache/coheigea/santuario/xmlsignature/SignatureStAXTest.java
    @@ -58,9 +58,19 @@
    sourceDocument, namesToSign, "http://www.w3.org/2000/09/xmldsig#rsa-sha1", key, cert
    );

    + // Sign using StAX
    + InputStream sourceDocument2 = new ByteArrayInputStream(baos.toByteArray());
    + List namesToSign2 = new ArrayList();
    + namesToSign2.add(new QName("urn:example:po", "PaymentInfo"));
    + ByteArrayOutputStream baos2 = SignatureUtils.signUsingStAX(
    + sourceDocument2, namesToSign2, "http://www.w3.org/2000/09/xmldsig#rsa-sha1", key, cert
    + );
    +
    + System.out.println(new String(baos2.toByteArray()));
    +
    // Verify using StAX
    SignatureUtils.verifyUsingStAX(
    - new ByteArrayInputStream(baos.toByteArray()), namesToSign, cert);
    + new ByteArrayInputStream(baos2.toByteArray()), namesToSign2, cert);
    }



    I already tried with 2 keys / certificats... same problem : NPE in the StAX parser. The processing of the second is in error.

    My usecase :
    - a client generate a XML file (after GZ 50 MB)
    - the client sign the content
    - the client send the file to the server
    - the server check the signature
    - the server generate a timestamp a sign the XML with the timestamp in the metadata
    - the server archivate the XML (with the 2 signature and the timestamp)
    And now, I want the check the content of the archive !
    -> with DOM : Ok
    -> with StAX : Ko

    ReplyDelete
  26. Can someone reproduce ?

    ReplyDelete
  27. Yes I can reproduce. I raised the following JIRA:

    https://issues.apache.org/jira/browse/SANTUARIO-424

    Colm.

    ReplyDelete
  28. This comment has been removed by a blog administrator.

    ReplyDelete
  29. First many thanks for the sample programmes, without which it is really hard to put all the pieces together from reading the Javadoc.
    Regarding the EncryptionStAXtest, I have two questions:
    1. Does StAX API supports multiple recipients? i.e. Can I add more than one properties.setEncryptionTransportKey (wrappingKeyX)
    2. I have a use case, where I have to encrypt an arbitrary (big) data file and
    have it enveloped in as CipherValue (not CipherReference) in EncryptedData. What are the XMLSecruityProperties value to apply?
    It cannot be SecurePart I presume. (There is no input XML file)


    ReplyDelete
  30. Setting more than one key transport key is not supported. Please feel free to submit a JIRA for the Santuario project for this if if it something you'd like to see supported.

    I'm not sure I understand you're second point. If it is not XML then what format is it?

    Colm.

    ReplyDelete
    Replies
    1. Thansk for the quick reply.
      1. It mean the actual Santuario StAX API does not support multi-recipeints for Encryption. right. Then does its counterpart Santuario DOM API do it ?
      2.Let me put it in another way. Suppose I get a XML element whose content (to be encrypted) comes from a external file. Is there any way in Santuario API to indicate it via XmlSecurityProperties ?

      Delete
    2. 1) Yes. See this test I just added: http://svn.apache.org/viewvc?view=revision&revision=1697004

      2) I don't follow you. The EncryptionStAXTest does exactly that - it reads in an XML Document "plaintext.xml" and encrypts it using a streaming approach. SecurePart just refers to the QName of the Element to encrypt in the stream/document.

      Delete
    3. Thank you for the new example.
      I got two questions on XMLSecurityProperties class.
      1. In case of KeyIdentifier.KeyName used, how can I set a value for the "keyname" value for the generated EncryptedKey?
      2. The field "EncryptionUseThisCertificate" is for transport Key encryption or for Data encryption ? If it is for Transport key, then will the EncryptionTransportKey ignored in case both are given ?

      Delete
    4. Please ask any future questions on the Apache Santuario mailing lists...

      1) You can't - it's not supported at the moment. Feel free to create a JIRA for this.

      2) It's for key transport. The EncryptionTransportKey is used if it is specified, otherwise it falls back to the Cert. The Cert is required if you are using a key referencing mechanism such as IssuerSerial.

      Colm.

      Delete
  31. This comment has been removed by the author.

    ReplyDelete
  32. I was hoping for a performance boost when signing with the Stax method vs using DOM but found no such difference. Should not the Stax method perform better?
    This was done using Santuario 2.0.5

    ReplyDelete
    Replies
    1. I reached the same conclusion, the CPU-usage performance is the same in DOM and Stax

      Delete
  33. See here:

    http://coheigea.blogspot.ie/2014/04/apache-santuario-xml-security-for-java_25.html

    ReplyDelete
  34. Do you have any solution for android xml digital signature?

    ReplyDelete
  35. Hi,

    I found that the StAX signing always sign the xml by Id even for entire document. However, according to XML digital signature, for the whole xml signing, the reference URI can be "".
    Could StAX be set to sign the entire document without adding Id attribute? As my existing XML schema doesn't support any Id attribute.

    Thanks a lot.

    ReplyDelete
  36. No, but there's a patch being reviewed at the moment to support this: https://issues.apache.org/jira/browse/SANTUARIO-458

    ReplyDelete
  37. This comment has been removed by the author.

    ReplyDelete
  38. Hi sir,

    Any idea, pointers, how we can inject the "Signature" part via StAX streaming API into an arbitrary xml tag ? So not as a child from the root tag, but more down deep ? I played with the signaturePosition attribute, but no luck here....

    ReplyDelete
    Replies
    1. You can either use signaturePosition or else the new "signaturePositionQName" which will be available in the next Santuario release (currently under vote):

      https://issues.apache.org/jira/browse/SANTUARIO-468

      Delete
  39. Can you elaborate how the streaming API works in terms of caching and memory usage, if the entire XML is signed (enveloped mode). How is the part of the XML after the "Signature" part handled while canonicalizing/transforming/digestCalculation ?

    Suppose, you have a big XML file, where the "Signature" part is in the header ?

    Regards, Bert.

    ReplyDelete
  40. Colm

    We're using Santuario 2.1.1 canonicalise, digest and then encode an XML signature as part of a non-repudiation service. However we're finding that any document >4MB has sever performance issues when calling performTransforms. By severe, think hours taken to do a transform! Do you have any thoughts or pointers as to what could be the issue?

    ReplyDelete
    Replies
    1. No, could you create a JIRA here and attach a test-case to reproduce the issue? https://issues.apache.org/jira/projects/SANTUARIO. Have you tried with the streaming implementation - it might give better performance?

      Delete
  41. Hi Stax team..
    My problem is to decipher some xml fields wihtout specifying the list of them but based on the xsd type : "xenc:EncryptedDataType"
    Is it possible ?

    ReplyDelete