Securing web services using Talend's Open Studio for ESB - part III

This is the third article in a series on securing web services using Talend's Open Studio for ESB. In the first article, we looked at how to design and test a SOAP web service in the Studio, and how to create a client job to invoke on it. In the second article we looked at deploying the jobs in the Talend ESB runtime container. In this article, we will look at how to secure the SOAP webservice we are deploying in the container, by requiring the client to authenticate using a WS-Security UsernameToken.

1) Secure the "double-it" webservice by requiring clients to authenticate

First we will secure the "double-it" webservice we have designed in the Studio in the first article, by requiring clients to authenticate using a WS-Security UsernameToken. Essentially what this means is that the client adds a SOAP header to the request containing username and password values, which then must be authenticated by the service. UsernameToken authentication can be configured for a service in the Studio, by right-clicking on the "DoubleIt 0.1" Service in the left-hand menu and selecting "ESB Runtime Options". Under "ESB Service Security" select "Username/Password". Select "OK" and export the service again as detailed in the second article.

Now start the container and deploy the modified service. Note that what selecting the "Username/Password" actually does in the container is to enforce the policy that is stored in 'etc/org.talend.esb.job.token.policy', which is a WS-SecurityPolicy assertion that requires that a UsernameToken must always be sent to the service. Now deploy the client job - you will see an error in the Console along the lines of:

{http://schemas.xmlsoap.org/soap/envelope/}Server|These policy alternatives can not be satisfied:
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}SupportingTokens
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}UsernameToken

This is due to the fact that we have not yet configured the client job to send a
UsernameToken in the request.

2) How authentication works in the container

So far we have required clients to authenticate to the service, but we have not said anything about how the service actually authenticates the credentials that it receives. Apache Karaf uses JAAS realms to handle authentication and authorization. Typing "jaas:realm-list" in the container shows the list of JAAS realms that are installed:

Here we can see that the (default) JAAS realm of "karaf" has been configured with a number of JAAS Login Modules. In particular, in index 1, the PropertiesLoginModule authenticates users against entries in 'etc/users.properties'. This file contains entries that map a username to a password, as well as an optional number of groups. It also contains entries mapping groups to roles. In this example though we are solely concerned with authentication. The service will extract the username and password from the security header of the request and will compare them to the values in 'etc/users.properties'. If there is a match then a user is deemed to be authenticated and the request can proceed.

In a real-world deployment, we can authenticate to users stored in a database or in an LDAP directory server, by configuring a JAAS Realm with the appropriate LoginModules (see the Karaf security guide for a list of available Login Modules).

3) Update the client job to include a UsernameToken

Finally we have to update the client job to include a UsernameToken in the Studio. Open the "tESBConsumer" component and select "Use Authentication", and then select the "Username Token" authentication type. Enter "tesb" for the username and password values (this is one of the default users defined in 'etc/users.properties' in the container).

Now save the job and build and deploy it as per the second tutorial. The job request should succeed, with the response message printed in the console. Examining 'log/tesb.log' it is possible to see what the client request looks like:

In the next article we'll look at authentication using SAML tokens.

Securing web services using Talend's Open Studio for ESB - part II

This is the second article in a series on securing web services using Talend's Open Studio for ESB. In the first article, we looked at how Talend's Open Studio for ESB can be used to design and test a SOAP web service, and also how we can create a client job that invokes on this service. In this article, we will show how to deploy the service and client we created previously in the Talend ESB runtime container.

1) The Talend ESB runtime container

When we downloaded Talend Open Studio for ESB (see the first article), we launched the Talend Studio via the "Studio" directory to design and test our "double it" SOAP service. However, the ability to "Run" the SOAP Service in the Studio is only suitable for testing the design of the service. Once we are ready to deploy a service or client we have created in the Studio, we will need a suitable runtime container, something that is available in the "Runtime_ESBSE" directory in the Talend Open Studio for ESB distribution. The runtime container in question is a powerful and enterprise-ready container based on Apache Karaf. We can start it in the "Runtime_ESBSE/container" directory via "bin/trun":

By default, the Talend ESB runtime starts with a set of default "bundles" (which can be viewed with "la"). All of the libraries that we require will be started automatically, so no further work is required here.

2) Export the service and client job from the Studio

To deploy the SOAP "double it" service, and client job, we need to export them from the Studio. Right click on the "Double It" service in the left-hand menu, and first select "ESB Runtime Options", ticking "Log Messages" so that we can see the input/output messages of the service when we look at the logs. Then, right click again on "Double It" and select "Export Service" and save the resulting .kar file locally.

Before exporting the client job, we need to make one minor change. The default port that the Studio used for the "double it" SOAP service (8090) is different to that of Karaf (8040). Click on "tESBConsumer" and change the port number in the address to "8040". Then after saving, right click on the double it client job and select "Build job". Under "Build Type" select "OSGI bundle for ESB", and click "Finish" to export the job:

3) Deploy the service and client jobs to the Runtime Container

Finally, we need to deploy the service and client jobs to the Runtime Container. First, copy the service .kar file into "Runtime_ESBSE/container/deploy". This will automatically deploy the service in Karaf (something that can be verified by running "la" in the console - you should see the service as the last bundle on the list). Then also copy the client jar into the "deploy" directory. The response will be output in the console window (due to the tLogRow component), and the full message can be seen in the server logs ("log/tesb.log"):

SAML SSO support for the Apache CXF Fediz plugins

Apache CXF Fediz originated as a way of securing web applications using Single Sign-On via the WS-Federation Passive Requestor Profile. Plugins were written to support the most popular web application containers, such as Apache Tomcat, Jetty, Spring, Websphere, etc. Fediz then shipped an IdP which could be used to perform authentication using the container plugins. For the 1.3.0 release, support was added to the IdP to also support the SAML SSO protocol. From the next 1.4.4 release, the Tomcat 8 plugin can authenticate to a SAML SSO IdP, instead of using WS-Federation, with a few simple configuration changes. This makes it very easy to upgrade your Fediz-secured containers to use SAML SSO instead of WS-Federation.

In this article, we will secure the 'fedizhelloworld' application example that ships with Fediz, which is deployed in Apache Tomcat, using Keycloak as the SAML SSO IdP. We will show how to deploy and secure the application both manually and also by a docker image that I have created for quick deployment.

1) Download and configure Keycloak

Download and install the latest Keycloak distribution (tested with 3.4.3).

1.1) Create users in Keycloak

Start keycloak in standalone mode by running 'sh bin/standalone.sh'. First we need to create an admin user by navigating to the following URL, and entering a password:

• http://localhost:8080/auth/

Click on the "Administration Console" link, logging on using the admin user credentials. You will see the configuration details of the "Master" realm. For the purposes of this demo, we will create a new realm. Hover the mouse pointer over "Master" in the top left-hand corner, and click on "Add realm". Create a new realm called "fediz-samlsso". Now we will create a new user in this realm. Click on "Users" and select "Add User", specifying "alice" as the username. Click "save" and then go to the "Credentials" tab for "alice", and specify a password, unselecting the "Temporary" checkbox, and reset the password.

1.2) Create a new client application in Keycloak

Now we will create a new client application for 'fedizhelloworld' in Keycloak. Select "Clients" in the left-hand menu, and click on "Create". Specify the following values:

• Client ID: urn:org:apache:cxf:fediz:fedizhelloworld
• Client protocol: saml
• Client SAML Endpoint: https://localhost:9443/fedizhelloworld/secure

Once the client is created you will see more configuration options:

• Select "Sign Assertions"
• Select "Force Name ID Format".
• Valid Redirect URIs: https://localhost:9443/*

Click 'Save'. Now go to the "SAML Keys" tab of the newly created client. Here we will have to import the certificate of the Fediz RP so that Keycloak can validate the signed SAML requests. Click "Import" and specify:

• Archive Format: JKS
• Key Alias: mytomrpkey
• Store password: tompass
• Import file: rp-ssl-key.jks

1.3) Export the Keycloak signing certificate

Finally, we need to export the Keycloak signing certificate so that the Fediz plugin can validate the signed SAML Response from Keycloak. Select "Realm Settings" (for "fediz-samlsso") and click on the "Keys" tab. Copy and save the value specified in the "Certificate" textfield. 

2) Manually configure 'fedizhelloworld' application in Apache Tomcat

In this section, we'll look at manually configuring the 'fedizhelloworld' application in Apache Tomcat. To use a docker image skip to the next section. Download and extract Apache Tomcat 8 (tested with 8.5.32) to ${catalina.home}. Download Fediz 1.4.4 and build the source with "mvn clean install -DskipTests". Copy 'apache-fediz/target/apache-fediz-1.4.4' to a new directory (${fediz.home}).

2.1) Secure the Apache Tomcat container with the Fediz plugin

First we will secure the Apache Tomcat container with the Fediz plugin:

• Create a new directory: ${catalina.home}/lib/fediz
• Edit ${catalina.home}/conf/catalina.properties and append ',${catalina.home}/lib/fediz/*.jar' to the 'common.loader' property.
• Copy ${fediz.home}/plugins/tomcat8/lib/* to ${catalina.home}/lib/fediz
• Edit the TLS Connector in ${catalina.home}/conf/server.xml', and change the ports to avoid conflict with Keycloak, i.e. switch 8080 to 9080, 8443 to 9443, etc.
• In the same file, add configuration for the TLS port: <Connector port="9443" protocol="org.apache.coyote.http11.Http11NioProtocol" maxThreads="150" SSLEnabled="true" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" keystoreFile="rp-ssl-key.jks" keystorePass="tompass" />

2.2) Deploy 'fedizhelloworld' to Tomcat

• Do a "mvn clean install" in ${fediz.home}/examples/simpleWebapp
• Copy ${fediz.home}/examples/simpleWebapp/target/fedizhelloworld.war to ${catalina.home}/webapps.
• Copy ${fediz.home}/examples/samplekeys/rp-ssl-key.jks to ${catalina.home}.
• Copy ${fediz.home}/examples/simpleWebapp/src/main/config/fediz_config.xml to ${catalina.home}/conf/

2.3) Configure 'fediz_config.xml'
 
Now we need to configure '${catalina.home}/conf/fediz_config.xml' that we copied from the Fediz example in the previous section:

• Under 'contextConfig', specify the key we are using to sign the SAML Request: <signingKey keyAlias="mytomrpkey" keyPassword="tompass">
              <keyStore file="rp-ssl-key.jks" password="tompass" type="JKS" />
          </signingKey>
• Change the trustManager keystore to: <keyStore file="keycloak.cert" type="PEM" />
• In the 'Protocol' section, change 'federationProtocolType' to 'samlProtocolType'.
• Change the 'Issuer' value to: http://localhost:8080/auth/realms/fediz-samlsso/protocol/saml
• Under 'Protocol' add: <signRequest>true</signRequest and <disableDeflateEncoding>true</disableDeflateEncoding>

Finally, create a file called 'keycloak.cert' in ${catalina.home}. In between "-----BEGIN CERTIFICATE----- / -----END CERTIFICATE-----" tags, paste the Keycloak signing certificate as retrieved in step "1.3" above. 

3) Using a docker image to deploy 'fedizhelloworld'

I have created a simple docker project which can be used to package and deploy the 'fedizhelloworld.war' into Apache Tomcat, which is secured with Fediz. The project is available on github here. Clone the project and build and run with the following steps:

• Edit 'keycloak.cert' and paste in the signing certificate as retrieved in step 1.3 above.
• Build with: docker build -t coheigea/fediz-samlsso-rp .
• Run with: docker run -p 9443:8443 coheigea/fediz-samlsso-rp

At the time of writing, the docker file references a SNAPSHOT build of Fediz, as 1.4.4 is not yet released.

4) Testing the service

To test the service navigate to:

• https://localhost:9443/fedizhelloworld/secure/fedservlet

You should be redirected to the Keycloak authentication page. Enter the user credentials you have created, and you will be redirected back to the 'fedizhelloworld' application successfully.

Securing web services using Talend's Open Studio for ESB - part I

This is the first part in a series of posts on securing web services using Talend's Open Studio for ESB. Talend's Open Studio for ESB is a freely available tool that comprises an eclipse-based studio to design and test web services, as well as a runtime based on Apache Karaf which can be used to deploy the web services built in the studio. In this post we will show how a simple SOAP web service can be designed and tested in the Studio.

1) Download and start the Studio and design a simple SOAP web service

Firstly download and extract Talend's Open Studio for ESB. Version 7.0.1 was used for the purpose of this blog post. The "Runtime_ESBSE" directory contains the runtime container, and the "Studio" directory the Open Studio for ESB. Launch the Studio and create a new project.

First we will create a simple SOAP "double-it" web service using the Studio. Right click on "Services" in the left-hand menu and then "Create Service". Call the service "DoubleIt" and click "Next" to create a new WSDL. The Studio then displays the design of the service, and the WSDL can be seen if you click on the "Source" tab. We will make a few changes to the default service.

Click on "DoubleItPortType" and change the operation name from "DoubleItOperation" to "DoubleIt". Next click on the right arrow next to "DoubleItRequest" and change the request type from "string" to "int". Do the same for the Response type. Now save the service and we are ready to move on to the next step.

2) Implement the "DoubleIt" service we have designed

After having designed the service above, now we need to implement the service by assigning a job to it. Right click on the "DoubleIt" service we have created ("Services/DoubleIt 0.1/DoubleItPortType 0.1/DoubleIt 0.1" in the left-hand menu) and select "Assign Job", and click "Next" to create a new job for this service. Now under "Job Designs" in the left-hand menu we see a new job has been created and the main window has been updated with tESBProviderRequest and tESBProviderResponse components.

Drag the tESBProviderResponse component over to the right hand side of the window. Now we need to think about how to handle the service logic. In our service, we want to take an input number, double it, and assign it to an output number. A Talend component available in the Studio that allows us to map XML is the "tXMLMap" component.

Find the "tXMLMap" component in the palette on the right hand side (under "XML"), and drag it onto the main window in between the two existing components. Now right click on tESBProviderRequest and select "Row" and "Main", and map the arrow onto the tXMLMap component. Do the same from tXMLMap to tESBProviderResponse, giving a name "Response" for the output when prompted.

Next we need to configure the "tXMLMap" component to implement the mapping logic. Double click on "tXMLMap". The input request is available on the left hand side, with a "root" payload. We want to map the request payload to the response payload. Hold the left mouse button down on the left-hand side over the request "root" payload, and move the mouse over to the right hand side and release on the response "root" payload, selecting "Add linker to target node".

Before we try to implement the "doubling" logic, we need to change the payload type from "String" to "int". Click on the "Tree Schema Editor" tab at the bottom of the screen and change both the request and response payload types to "int". Back up on the Response tab, click on "[row1.payload:/root]" and edit it to be "2 * [row1.payload:/root]".

Finally, we need to change the request/response elements to conform to the WSDL. Right-click on "root" in the left-hand request column and rename it to "ns2:DoubleItRequest". When this is done, right click again on this element and "Set a Namespace", with the namespace "http://www.talend.org/service/" and prefix "ns2". Similarly, on the right hand side, rename "root" to "ns2:DoubleItResponse", and set the namespace in the same way as for the request. Now click "Apply" and save the job.

Now click on the "Run" tab at the bottom of the screen and run the service. The service should now be deployed at "http://localhost:8090/services/DoubleIt".

3) Implement a client for the "DoubleIt" service

As well as designing, creating and testing web services, Talend Open Studio for ESB can also be used to create clients for these web services. To do this we need a new job. Right click on "Job Designs" in the left-hand menu and create a new job. Drag the "tESBConsumer" component to the main screen, as well as two "tLogRow" components. Right click on "tESBConsumer" and select "Row/Response" and drag the arrow to the first tLogRow component. In addition, select "Row/Fault" and drag the arrow to the second tLogRow component. This way we are logging both the response + faults from the remote service.

Left-click on "tESBConsumer" and specify "http://localhost:8090/services/DoubleIt?wsdl" as the WSDL location (the WSDL of our service is available at this address due to WSDL publish). Then click the "reload" button on the right-hand side and click "Finish". Next we need to implement the client logic - namely, to supply a number to double. Drag the "tFixedFlowInput" and "tXMLMap" components to the screen. Map "Row/Main" from "tFixedFlowInput" to "tXMLMap", and "Row" from "tXMLMap" to "tESBConsumer" with a new output name of "Request".

Now click on "tFixedFlowInput" and "Edit Schema". Add a new column of type "int" called "numberToDouble". Back in the component screen for "tFixedFlowInput" select "Use inline table" and enter a number (e.g. 200). Now click on "tXMLMap" to configure our mapping. Drag "numberToDouble" on the left-hand side over to "root" on the right-hand side, selecting "Add linker to target node". Right click on "root" on the right-hand side, and rename it to "ns2:DoubleItRequest", and again "Set a Namespace" with namespace "http://www.talend.org/service/" and prefix "ns2".

Click OK and save the job, and run it via the "Run" tab. In the console window we should see the response from the service, informing us that 200 doubled is "400". The job is also updated so that you can see the flow along with the throughput:

In the next tutorial we'll look at how to deploy our service and client jobs in the runtime container.

Streaming WS-Security MTOM support in Apache CXF

Apache CXF 3.0.0 introduced the new streaming (StAX-based) WS-security implementation via new functionality available in the core libraries - Apache WSS4J 2.0.0 and Apache Santuario 2.0.0. The StAX-based approach is more limited than the older DOM-based alternative, and in general slightly slower. However it can come into its own when sending or processing very large documents due to the low memory footprint of the library.

In addition, support was added in Apache CXF 3.2.0 for the DOM library to send and process WS-Security messages using MTOM. Essentially what this means is that we can compress WS-Security secured SOAP messages, by storing binary content in message attachments, instead of inlining them in the message (via BASE-64 encoding). When MTOM is enabled, Apache CXF will automatically use this functionality for WS-Security. However up until now, this functionality has not been available for the streaming WS-Security library.

This is set to change in Apache CXF 3.2.5. Support has been added in Apache Santuario to process CipherValue message elements in the streaming XML Security code that contain a "xop:Include" reference to a message attachment. Some fixes in Apache WSS4J build on this support, also adding support for processing BinarySecurityToken Elements that include 'xop:Include' instead of the inlined bytes. Both of these sets of changes are supported in Apache CXF in the following JIRA.

What this means is that Apache CXF 3.2.5 onwards will be able to process WS-Security enabled SOAP messages over MTOM. Please note however that support is limited to processing messages. The streaming code still inlines message bytes on the outbound side, unlike the DOM implementation. This could perhaps be implemented in the future if there is sufficient demand.