Talend Integration Factory released

Following in the footsteps of the first Talend Service Factory release last year, the first release of the Talend Integration Factory has been announced:

Talend Integration Factory, based on Apache Camel, uses well known Enterprise Integration Patterns to make message-based system integration easier to implement, yet more powerful and scalable.

For an in-depth analysis of what the Talend Integration Factory offers, see Christian Schneider's blog entry here. For some other comments on this release, see here and here.

[WSS4J 1.6] Changes to the Crypto interface

WSS4J uses the Crypto interface to provide a pluggable way of retrieving and converting certificates, verifying trust on certificates etc. A previous blog post described how verifying trust on PublicKeys was also moved into the Crypto interface for WSS4J 1.6.

WSS4J 1.5.x shipped with a Crypto implementation based around a KeyStore, which was configured via a properties file. A previous blog post describes some changes that were made to the property config names, as well as describing the separation between a keystore and a truststore. The code was also modified as part of this work so that the Crypto instance can be instantiated as normal, and not just via a properties file.

A big issue with the Crypto interface is that it was too closely tied in to the Java KeyStore objects which back the Merlin implementation. This makes it difficult to subclass the interface, for example to write a Crypto implementation that just stores an array of X509Certificates for signature verification, or to write a client to obtain the X509Certificates from a remote store (e.g. XKMS).

The fix for WSS-269 attempts to address these shortcomings. All of the KeyStore specific method names and arguments have been abstracted. All of the methods which locate X509Certificates have been consolidated into a single method, which takes an argument a new class which supplies the data (e.g. a SHA1 thumbprint of the certificate) used to retrieve the certificates. The abstract functionality (i.e. KeyStore independent) has been moved into the CryptoBase class, so implementation can just subclass this to avoid having to re-implement all of the generic methods in the Crypto interface.

WSS4J now ships with another Crypto implementation. CertificateStore holds an array of X509Certificates that can be used out of the box, for signature verification, or encryption. The PrivateKey methods are not implemented, so it cannot be used for decryption, or signature creation. However, CertificateStore could easily be subclassed.

[WSS4J 1.6] Change to PublicKey validation

This entry describes a relatively minor change to how trust validation is performed on PublicKeys in WSS4J 1.6.

When the KeyInfo element of a Signature does not have a SecurityTokenReference child, WSS4J tries to extract a PublicKey via a KeyValue child. In WSS4J 1.5.x, it then constructed a PublicKeyCallback instance, passing it the PublicKey object, and invoked the CallbackHandler. It then called a "isVerified" method on the Callback to check to see whether the CallbackHandler had verified the PublicKey or not. The CallbackHandler implementation needed to call the "verifyTrust" method on the PublicKeyCallback, passing in a KeyStore object. This method iterates through each Certificate in the KeyStore, and checks to see whether the PublicKeys match.

There are a number of problems with this approach:

• It is inconsistent with how Certificate validation is done (i.e. via a Crypto object).
• It relies on the CallbackHandler implementation calling "verifyTrust" on the Callback object, thus putting the onus on the end-user to write the CallbackHandler implementation properly.

As part of the fix for WSS-266, the PublicKeyCallback class was removed from WSS4J. Instead, the "verifyTrust" method was moved to the Crypto interface, whether the argument is now a PublicKey object, rather than a KeyStore. In this way, validation is done in the same way as for Certificates, and the end-user has no need to consider the special-case of verifying public keys in the CallbackHandler, it is taken care of internally by WSS4J.

[WSS4J 1.6] Specifying elements to sign or encrypt

The signature and encryption creation code in WSS4J uses the WSEncryptionPart class to find DOM elements to sign and encrypt. There are a number of minor changes to how elements are located from a WSEncryptionPart in WSS4J 1.6:

1. WSEncryptionPart now stores an optional DOM element, which will be used as the element to sign/encrypt if it is non-null.
2. Failing this, it finds the SOAP body and compares the wsu:Id with the stored Id, or if there is no stored Id in WSEncryptionPart, it checks the stored localname/namespace.
3. Failing this, if the stored Id in WSEncryptionPart is not null, it tries to find the first element in the SOAP envelope that has a matching wsu:Id.
4. If the stored Id is null, it tries to find *all* DOM Elements that match the stored localname/namespace.

WSEncryptionPart is intended to refer to a single Element for encryption/signature. However, as a localname/namespace is not necessarily unique, point 4 will return all matching Elements. An important implication of the order of the steps given above, is that client code should set the DOM element on the WSEncryptionPart if it is accessible, and if not, it should set the wsu:Id. Otherwise, a localname/namespace (which is not referring to the SOAP Body) will result in a traversal of the DOM tree.

The DOM element(s) that is(are) found are stored for retrieval, so that we don't need to traverse the SOAP envelope multiple times, when e.g. doing an STR Transform, or for element location in the XML Security code.

WSPasswordCallback changes in WSS4J 1.6

Following the previous post on Username Token processing changes in WSS4J 1.6, this post gives some additional details on related changes that have been made to the WSPasswordCallback class in WSS4J.

WSPasswordCallback defines a set of integers which correspond to usage instructions for the CallbackHandler. In WSS4J 1.5.x they were often used in an inconsistent or ad hoc manner, leading to confusion for users. The following integers have been deprecated in WSS4J 1.6 and are no longer used internally by WSS4J:

• WSPasswordCallback.KEY_NAME
• WSPasswordCallback.USERNAME_TOKEN_UNKNOWN
• WSPasswordCallback.ENCRYPTED_KEY_TOKEN

In WSS4J 1.6, the following WSPasswordCallback identifiers are used:

• WSPasswordCallback.DECRYPT - DECRYPT usage is used when the calling code needs a password to get the private key of this identifier (alias) from a keystore. This is only used for the inbound case of decrypting a session (symmetric) key, and not for the case of getting a private key to sign the message. The CallbackHandler must set the password via the setPassword(String) method.
• WSPasswordCallback.USERNAME_TOKEN - USERNAME_TOKEN usage is used to obtain a password for either creating a Username Token, or for validating it. It is also used for the case of deriving a key from a Username Token. The CallbackHandler must set the password via the setPassword(String) method.
• WSPasswordCallback.SIGNATURE - SIGNATURE usage is used on the outbound side only, to get a password to get the private key of this identifier (alias) from a keystore. The CallbackHandler must set the password via the setPassword(String) method.
• WSPasswordCallback.SECURITY_CONTEXT_TOKEN - SECURITY_CONTEXT_TOKEN usage is for the case of when we want the CallbackHandler to supply the key associated with a SecurityContextToken. The CallbackHandler must set the key via the setKey(byte[]) method.
• WSPasswordCallback.CUSTOM_TOKEN - CUSTOM_TOKEN usage is used for the case that we want the CallbackHandler to supply a token as a DOM Element. For example, this is used for the case of a reference to a SAML Assertion or Security Context Token that is not in the message. The CallbackHandler must set the token via the setCustomToken(Element) method.
• WSPasswordCallback.SECRET_KEY - SECRET_KEY usage is used for the case that we want to obtain a secret key for encryption or signature on the outbound side, or for decryption or verification on the inbound side. The CallbackHandler must set the key via the setKey(byte[]) method.

The DECRYPT, SIGNATURE and SECURITY_CONTEXT_TOKEN identifiers are used more or less the same as they were used in WSS4J 1.5.x. The CUSTOM_TOKEN identifier is also used in a similar way, apart from the fact that some of the processors incorrectly used it to obtain a secret key. The changes to USERNAME_TOKEN were covered in a previous blog post.

SECRET_KEY is a new identifier for finding secret keys. It replaces the occasionally incorrect use of CUSTOM_TOKEN, as well as KEY_NAME and ENCRYPTED_KEY_TOKEN.

UsernameToken processing changes in WSS4J 1.6

The forthcoming WSS4J 1.6 release contains some significant changes relating to how wsse UsernameTokens are processed. In WSS4J 1.5.x, the following processing rules applied:

• For a digest password, the CallbackHandler implementation was given the username and an identifier of WSPasswordCallback.USERNAME_TOKEN. It was then expected to set the password on the callback, and the processor did the comparison.
• For a plaintext password, the CallbackHandler implementation was given the username, password, and an identifier of WSPasswordCallback.USERNAME_TOKEN_UNKNOWN, and was expected to do all validation of the plaintext password itself, throwing an exception if validation failed.
• For a password of some unspecified non-standard type, WSS4J would throw an exception by default. However, if wssConfig.getHandleCustomPasswordTypes() returned true, then it would again dispatch the username, password, and an identifier of WSPasswordCallback.USERNAME_TOKEN_UNKNOWN to the CallbackHandler implementation for validation.
• For the case of a UsernameToken with no password element, it would again dispatch the username, password, and an identifier of WSPasswordCallback.USERNAME_TOKEN_UNKNOWN to the CallbackHandler implementation for validation.

The reason that the processor does not validate the plaintext password, as per the digest case, is to accommodate dispatching the username/password to (for example) a directory store for authentication. The custom password type is treated as a separate case namely for WCF interoperability (see here).

There are a couple of several obvious problems with this set-up:

• The standard plaintext password case is treated exactly the same as a non-standard password type, or the case of no password at all, by sending the callback handler a type of WSPasswordCallback.USERNAME_TOKEN_UNKNOWN.
• The treatment of passwords for both the standard digest and plaintext cases are inconsistent.
• A potential security hole exists where the user may not be aware that the CallbackHandler implementation *must* throw an exception on WSPasswordCallback.USERNAME_TOKEN_UNKNOWN when the password is not recognised.
• The CallbackHandler interface is being used in a non-standard way - it is only meant to supply a password, not do the validation.
• UsernameTokens with no password (i.e. used for key derivation) are stored using the same result (WSConstants.UT) as UsernameTokens that have been validated. This could lead to a security hole where a user assumes that because a UsernameToken has been processed, password validation has taken place.

WSS4J 1.6 fixes these issues, at the cost of breaking backwards compatibility in terms of the semantics of the CallbackHandler implementation. The UsernameTokenProcessor now does no validation of the password, beyond making sure it is well formed. A new Validator implementation (more on this at a later date) does the validation of the token. The default behaviour is as follows:

• For the digest case, the CallbackHandler is given the username, password type and an identifier of WSPasswordCallback.USERNAME_TOKEN. It must set the password on the callback, and the validator does the comparison. This is the same as the old behaviour.
• The plaintext case has exactly the same behaviour as the digest case. The identifier is now WSPasswordCallback.USERNAME_TOKEN and not WSPasswordCallback.USERNAME_TOKEN_UNKNOWN, and the CallbackHandler does not do any authentication, but must set the password on the callback.
• The custom password type case defaults to the same behaviour as the plaintext case, assuming wssConfig.getHandleCustomPasswordTypes() returns true.
• For the case of a username token with no password element, the default behaviour is simply to ignore it, and to store it as a new result of type WSConstants.UT_NOPASSWORD.

So we now have the correct use of the CallbackHandler interface - it does no authentication, and is only used to retrieve password for validation. The plaintext and digest cases are treated the same way by default. There is no security hole in that the user does not need to throw exceptions in the CallbackHandler implementation for various cases. Also, the case of no password at all is not validated, but is stored as a separate type, for use by other processors to derive keys for decryption or signature verification.

So what if you want to validate the plaintext password against a directory store, rather than have the CallbackHandler set the password? Instead of implementing this behaviour in your CallbackHandler implementation, you can simply @Override the verifyPlaintextPassword(UsernameToken usernameToken) method in the validator instead. By simply plugging in a validator on the UsernameTokenProcessor (such as the NoOpValidator), it is possible to do any kind of custom validation (or none at all) on the token. This is a much better solution than having to write a custom processor, and replace the existing UsernameTokenProcessor.

Support for SAML2 assertions in WSS4J 1.6

Support for SAML2 assertions has finally arrived in WSS4J, via the forthcoming 1.6 release. This has been a long-standing feature request (see here). WSS4J 1.5.x only supports SAML 1.1 assertions via the deprecated Opensaml1, and it supports them in a very limited manner, namely:

• It only supports the creation of Authentication statements.
• Processing essentially involves saving the assertions, it did not support validating enveloped signatures, or trust on the signatures, etc.

Several patches were submitted to WSS-146 to upgrade WSS4J to use Opensaml2. I took a patch from Todd Michael Dunst as a starting point for the port, as his patch had the nice idea of using a CallbackHandler implementation to construct outbound SAML assertions. About a months work later (mainly in testing, refactoring, and porting the patch to trunk) the SAML2 port is more or less ready on trunk. SAML2 support in WSS4J 1.6 consists of:

• Support for creating signed/unsigned SAML 1.1/2.0 assertions, containing authentication, authorization, attribute statements etc.
• This extensibility is achieved by letting the user implement a CallbackHandler instance.
• The SAMLTokenProcessor can now process any type of assertion, verify an enveloped signature on it, and verify trust on the signature. It also verifies some holder-of-key requirements, e.g. that the Subject contains a KeyInfo element, and that the assertion is signed and trusted etc.

WSS4J 1.6 contains an extensive set of tests for both creating and processing different type of assertions, you can browse them here. To illustrate the flexibility and simplicity of the CallbackHandler approach for constructing assertions, take a look at an abstract CallbackHandler here, as well as the concrete implementations (SAML 1.1 and SAML 2). As you can see, a fairly small amount of code can create a large variety of assertions.

Opensaml2 has a very large set of dependencies, but through some judicious pom exclusions, as well replacing the Opensaml DefaultBootstrap code to avoid loading velocity, the following dependencies are introduced in WSS4J via Opensaml (snippet from mvn dependency):

+- org.opensaml:opensaml:jar:2.4.1:compile
 |  \- org.opensaml:openws:jar:1.4.1:compile
 |     \- org.opensaml:xmltooling:jar:1.3.1:compile
 |        +- org.slf4j:slf4j-api:jar:1.6.1:compile
 |        \- joda-time:joda-time:jar:1.6.2:compile

The WSS4J 1.6 pom currently has a dependency on the Shibboleth repo, where the Opensaml2 artifacts live. I plan on getting the Opensaml2 artifacts into Maven central in time for the 1.6 release - this is slightly complicated by the fact that some of the Opensaml2 dependencies are themselves not in Maven Central.

One known issue is that WSS4J cannot create an Assertion which has an EncryptedKey element in the Subject. This is due to a bug in Opensaml2 which has been fixed, but not released yet.

The Opensaml2 port has a large impact on existing code for *creating* assertions, however I suspect that very few people used that code. It has a minimal impact on existing code for processing assertions, with several caveats:

• WSS4J 1.5.x ignored (enveloped) signatures on SAML (1.1) assertions - this is no longer the case, so deployments which do not set the correct keystore/truststore config for dealing with signature verification will fail
• The SAMLTokenProcessor no longer saves all tokens as an "WSConstants.ST_UNSIGNED" action. It saves tokens that do not have an enveloped signature as this action, and token which *do* have an enveloped signature are saved as a "WSConstants.ST_SIGNED" action.
• The object that is saved as part of the action above has changed, from an Opensaml1 specific Assertion object, to an AssertionWrapper instance, which is a WSS4J specific object which encapsulates an Assertion, as well as some information corresponding to signature verification, etc.