Securing web services using Talend's Open Studio for ESB - part VII

This is the seventh and final article in a series on securing web services using Talend's Open Studio for ESB. First we covered how to create and secure a SOAP service, client job and route in the Studio, and how to deploy them to the Talend runtime container. In the previous post we looked instead at how to implement a REST service and client in the Studio. In this post we will build on the previous post by showing some different ways to secure our REST service when it is deployed in the Talend container.

1) Secure the REST "double-it" webservice using HTTP B/A

Previously we saw how to secure the SOAP "double-it" service in the container using WS-Security UsernameTokens. In this section we'll also secure our REST service using a username and password that the client supplies - this time using HTTP Basic Authentication. Open the REST service we have created in the Studio, and click on the 'tRESTRequest' component. Select "Use Authentication" and then pick the default "Basic HTTP" option. Save the job and build it by right clicking on the job name and selecting "Build job".

Start the runtime container and deploy the job. Now open our REST client job in the Studio. Click on the 'tRESTClient' component and select "Use Authentication" as per 'tRESTRequest' above. Select 'tesb' for the username and password (see section 2 of the SAML tutorial for an explanation of how authentication works in the container). Now build the job and deploy it to the container. The client job should successfully run. See below for a log of a successful request where the client credentials can be seen in the "Basic" HTTP header:

2) Secure the REST "double-it" webservice using SAML

As for SOAP services, we can also secure our REST webservice using SAML. Instead of having the REST client to create a SAML Assertion, we will leverage the Talend Security Token Service (STS). The REST client will use the same mechanism (WS-Trust) to authenticate and obtain a SAML Token from the Talend STS as for the SOAP-case. Then the REST client inserts the SAML Token into the authorization header of the service request. The service parses the header and validates the signature on the SAML Token in exactly the same way as for the REST request.

In the Studio, edit the 'tRESTRequest' and 'tRESTClient' components in our jobs as for the "Basic Authentication" example above, except this time select "SAML Token" for "Use Authentication". Save the jobs and build them and deploy the service to the container. Before deploying the client job, we need to start the STS via:

• tesb:start-sts

Then deploy the client job and it should work correctly:

Securing web services using Talend's Open Studio for ESB - part VI

This is the sixth article in a series on securing web services using Talend's Open Studio for ESB. Up to now we have seen how to create and secure a SOAP service, client job and route in the Studio, and how to deploy them to the Talend runtime container. For the remaining articles in this series, we will switch our focus to REST web services instead. In this article we will look at how to implement a REST service and client in the Studio.

1) Implement a "double-it" REST Service in the Studio

First let's look at how we can create implement the "double-it" service as a REST service instead. Open the Studio and right click on "Job Designs" and select "Create job". Create a new job called "DoubleItRESTService". Drag the 'tRESTRequest', 'tXMLMap' and 'tRESTResponse' components from the palette into the central window. Connect them by right-clicking on 'tRESTRequest' and selecting "Row / New Output" and drag the link to 'tXMLMap', calling the output 'Request'. Right-click on 'tXMLMap' and select "Row / New Output" and drag the link to 'tRESTResponse', calling the output 'Response':

Now let's design the REST endpoint by clicking on 'tRESTRequest'. Our simple "double-it" service will accept a path parameter corresponding to the number to double. It will return an XML or JSON response containing the doubled number wrapped in a "result" tag. Edit the 'REST endpoint' to add "/doubleit" at the end of the URL. In the REST API mapping, edit the "URI Pattern" to be "/{number}". Now click on the "Output Flow" for "Request" and click on the three dots that appear. Click the "+" button and change the column name to "number" and the Type to "Integer":

Click "OK" and then double-click on 'tXMLMap'. Left-click on the "Number" column on the left-hand side, and drag it over to the right-hand side to the "body" column. Select "Add Linker to Target Node". Now click on "Request.number" on the right-hand side and then on the three dots. Change the expression to "2 * Request.number" to implement the "doubling" logic. Finally, rename the "root" element to "result":

Finally click "OK", save the job and run it. We can test via a console that the job is working OK using a tool such as curl:

• curl -H "Accept: application/xml" http://localhost:8088/doubleit/15
• Response: <?xml version="1.0" encoding="UTF-8"?><result>30</result>
• Response if we ask for JSON: {"result":30}

2) Implement a "double-it" REST client in the Studio

Now we'll design a client job for the "double-it" REST service in the Studio. Right-click on "Job Designs" and create a new job called "DoubleItRESTClient". Drag a 'tFixedFlowInput', 'tRESTClient' and two 'tLogRow' components from the palette into the central window. Link the components, sending the 'tRESTClient'
"Response" to one 'tLogRow' component and the "Error" to the other:

Now click on 'tFixedFlowInput' and then 'Edit Schema'. Add a new column called "number" of type "Integer", and click "yes" to propagate the changes. In the inline table, add a value for the number. Finally, click on 'tRESTClient' and specify "http://localhost:8088/doubleit/" for the URL, and row1.number for the relative path. Keep the default HTTP Method of "GET" and "XML" for the "Accept Type":

Now save the job and run it. The service response should be displayed in the window of the run tab. In the next article, we'll look at how to secure this REST service in the Studio when deploying it to the Talend runtime container.

Securing web services using Talend's Open Studio for ESB - part V

This is the fifth article in a series on securing web services using Talend's Open Studio for ESB. So far we have seen how to design a SOAP service and client in the Studio, how to deploy them to the Talend runtime container, and how to secure them using a UsernameToken and SAML token. In addition to designing 'jobs', the Studio also offers the ability to create a 'route'. Routes leverage the capabilities and components of Apache Camel, which is a popular integration framework. In this article, we will design a route to invoke on the SAML-secured service we configured in the previous tutorial, instead of using a job.

1) Create a route to invoke on the "double-it" service

In the Studio, right-click on 'Routes' in the left-hand pane, and select 'Create Route' and create a new route called 'DoubleItClientRoute'. Select the 'cTimer', 'cSetBody', 'cSOAP' and 'cLog' components from the palette on the right-hand side and drag them into the route window from left to right. Link the components up by right clicking on each component, and selecting 'Row' and then 'Route' and left-clicking on the next component over:

Now let's configure each component in turn. The 'cTimer' component is used to start the route. You can run the route an arbitrary number of times with a specified delay, or else specify a start time to run the route. For now just enter '1' for 'Repeat' as we want to run the route once. Now click on the 'cSetBody' component. This is used to specify the Body of the request we are going to make on the remote (SOAP) service. For simplicity we will just hard-code the SOAP Body, so select 'CONSTANT' as the Language and input '"<ns2:DoubleItRequest xmlns:ns2=\"http://www.talend.org/service/\">60</ns2:DoubleItRequest>"' for the expression:

Now we will configure the 'cSOAP' component. First, deploy the SAML-secured SOAP service on the container (see previous tutorial) so that we have access to the WSDL. Double-click 'cSOAP' and enter 'http://localhost:8040/services/DoubleIt?wsdl' for the WSDL and hit the reload icon on the right-hand side and click 'Finish'. We will use the default dataformat of 'PAYLOAD' (the SOAP Body contents we set in 'cSetBody'). Select 'Use Authentication' and then pick "SAML Token". Input 'tesb' for the Username and Password values, and save the route.

2) Deploy the route to the container

Right click on the route name in the left-hand pane and select 'Build Route' to build the .kar file. In the container where the SAML-secured service should already be running, start the STS with 'tesb:start-sts', and then copy the client route .kar file into the 'deploy' folder. Consult the log in 'log/tesb.log' and you will see the successful service response as follows:

Combining Keycloak with the Apache CXF STS

The Apache CXF STS (Security Token Service) is a web service (both SOAP and REST are supported) that issues tokens (e.g. SAML, JWT) to authenticated users. It can also validate, renew and cancel tokens. To invoke successfully on the STS, a user must present credentials to the STS for authentication. The STS must be configured in turn to authenticate the user credentials to some backend. Another common requirement is to retrieve claims relating to the authenticated user from some backend to insert into the issued token.

In this post we will look at how the STS could be combined with Keycloak to both authenticate users and to retrieve the roles associated with a given user. Typically, Keycloak is used as an IdM for authentication using the SAML SSO or OpenId Connect protocols. However in this post we will leverage the Admin REST API.

I have created a project on github to deploy the CXF STS and Keycloak via docker here.

1) Configuring the STS

Checkout the project from github. The STS is configured is a web application that is contained in the 'src' folder. The WSDL defines a single endpoint with a security policy that requires the user to authenticate via a WS-Security UsernameToken. The STS is configured in spring. Essentially we define a custom 'validator' to validate the UsernameToken, as well as a custom ClaimsHandler to handle retrieving role claims from Keycloak. We also configure the STS to issue SAML tokens.

UsernameTokens are authenticated via the KeycloakUTValidator in the project source. This class is configured with the Keycloak address and realm and authenticates received tokens as follows:

Here we use the Keycloak REST API to search for the user matching the given username, using the given username and password as credentials. What the client API is actually doing behind the scenes here is to obtain an access token from Keycloak using the OAuth 2.0 resource owner password credentials grant, something that can be replicated with a tool like curl as follows:

• curl --data "client_id=admin-cli&grant_type=password&username=admin&password=password" http://localhost:9080/auth/realms/master/protocol/openid-connect/token -v
• curl -H "Authorization: bearer <access token>" http://localhost:9080/auth/admin/realms/master/users -H "Accept: application/json" -v

Keycloak will return a HTTP status code of 401 if authentication fails. We allow the case that Keycloak returns 403 unauthorized, as the user may not be authorized to invoke on the admin-cli client. A better approach would be to emulate Apache Syncope and have a "users/self" endpoint to allow users to retrieve information about themselves, but I could not find an analogous endpoint in Keycloak.

Role claims are retrieved via the KeycloakRoleClaimsHandler. This uses the admin credentials to search for the (already authenticated) user, and obtains the effective "realm-level" roles to add to the claim.

2) Running the testcase in docker

First build the STS war and create a docker image for the STS as follows:

• mvn clean install
• docker build -t coheigea/cxf-sts-keycloak . 

This latter command just deploys the war that was built into a Tomcat docker image via this Dockerfile. Then pull the official Keycloak docker image and start both via docker-compose (see here):

• docker pull jboss/keycloak
• docker-compose up

This starts the STS on port 8080 and Keycloak on port 9080. Log on to the Keycloak administration console at http://localhost:9080/auth/ using the username "admin" and password "password". Click on "Roles" and add a role for a user (e.g. "employee"). The click on "Users" and add a new user. After saving, click on "Credentials" and specify a password (unselecting "Temporary"). Then click on "Role Mappings" and select the role you created above for the user.

Now we will use SoapUI to invoke on the STS. Download it and create a new SOAP project using the WSDL of the STS (http://localhost:8080/cxf-sts-keycloak/UT?wsdl). Click on 'Issue' and select the request. We need to edit the SOAP Body of the request to instruct the STS to issue a SAML Token with a Role Claim using the standard WS-Trust parameters:

<ns:RequestSecurityToken>
     <t:TokenType xmlns:t="http://docs.oasis-open.org/ws-sx/ws-trust/200512">http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV2.0</t:TokenType>
     <t:KeyType xmlns:t="http://docs.oasis-open.org/ws-sx/ws-trust/200512">http://docs.oasis-open.org/ws-sx/ws-trust/200512/Bearer</t:KeyType>
     <t:RequestType xmlns:t="http://docs.oasis-open.org/ws-sx/ws-trust/200512">http://docs.oasis-open.org/ws-sx/ws-trust/200512/Issue</t:RequestType>
     <t:Claims xmlns:ic="http://schemas.xmlsoap.org/ws/2005/05/identity" xmlns:t="http://docs.oasis-open.org/ws-sx/ws-trust/200512" Dialect="http://schemas.xmlsoap.org/ws/2005/05/identity">
        <ic:ClaimType xmlns:ic="http://schemas.xmlsoap.org/ws/2005/05/identity" Uri="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/role"/>
     </t:Claims>
</ns:RequestSecurityToken>

Click in the Properties box in the lower left-hand corner and specify the username and password for the user you created in Keycloak. Finally, right click on the request and select "Add WSS UsernameToken" and hit "OK" and send the request. If the request was successful you should see the SAML Assertion issued by the STS on the right-hand side. In particular, note that the Assertion contains a number of Attributes corresponding to the roles of that particular user.

Running the Apache Kerby KDC in docker

Apache Kerby is a subproject of the Apache Directory project, and is a complete open-source KDC written entirely in Java. Apache Kerby 1.1.1 has been released recently. Last year I wrote a blog post about how to configure and launch Apache Kerby, by first obtaining the source distribution and building it using Apache Maven. In this post we will cover an alternative approach, which is to download and run a docker image I have prepared which is based on Apache Kerby 1.1.1.

The project is available on github here and the resulting docker image is available here. Note that this is not an official docker image - and so it provided just for testing or experimentation purposes. First clone the github repository and either build the image from scratch or download it from dockerhub:

• docker build . -t coheigea/kerby

 or:

• docker pull coheigea/kerby

The docker image builds a KDC based on Apache Kerby and runs it when started. However, it expects a directory to be supplied as the first argument (defaults to '/kerby-data/conf') containing the configuration files for Kerby. The github repository contains the relevant files in the 'kerby-data' directory. As well as the configuration files, it stores the admin keytab and a JSON file containing the default principals for the KDC.

Start the KDC by mapping the kerby-data directory to a volume on the container:

• docker run -it -p 4000:88 -v `pwd`/kerby-data:/kerby-data coheigea/kerby

Now we can log into the docker image and create a user for our tests:

• docker exec -it <id> bash
• stty rows 24 columns 80 (required to run jline in docker)
• sh bin/kadmin.sh /kerby-data/conf/ -k /kerby-data/keytabs/admin.keytab
• Then: addprinc -pw password alice@EXAMPLE.COM

To test the KDC from outside the container you can use the MIT kinit tool. Set the KRB5_CONFIG environment variable to point to the "krb5.conf" file included in the github repository, e.g:

• export KRB5_CONFIG=`pwd`/krb5.conf
• kinit alice

This will get you a ticket for "alice", that can be inspected via "klist".

Securing web services using Talend's Open Studio for ESB - part IV

This is the fourth article in a series on securing web services using Talend's Open Studio for ESB. In the previous article, we looked at how to secure a SOAP webservice in the Talend container, by requiring the client to authenticate using a WS-Security UsernameToken. In this post we will look at an alternative means of authenticating clients using a SAML token, which the client obtains from a Security Token Service (STS) also deployed in the Talend container. This is more sophisticated than the UsernameToken approach, as we can embed claims as attributes in the SAML Assertion, thus allowing the service provider to also make authorization decisions. However, in this article we will just focus on authentication.

1) Secure the "double-it" webservice by requiring clients to authenticate

As in the previous article, first we will secure the "double-it" webservice we have designed in the Studio in the first article, by requiring clients to authenticate using a SAML Token, which is conveyed in the security header of the request. SAML authentication can be configured for a service in the Studio, by right-clicking on the "DoubleIt 0.1" Service in the left-hand menu and selecting "ESB Runtime Options". Under "ESB Service Security" select "SAML Token". Select "OK" and export the service again as detailed in the second article.

Now start the container and deploy the modified service. Note that what selecting the "SAML Token" actually does in the container is to enforce the policy that is stored in 'etc/org.talend.esb.job.saml.policy', which is a WS-SecurityPolicy assertion that requires that a SAML 2.0 token containing an X.509 certificate associated with the client (subject) must be sent to the service. In addition, a Timestamp must be included in the security header of the request, and signed by the private key associated with the X.509 certificate in the Assertion.

2) Update the client job to include a SAML Token in the request

Next we have to update the client job to include a SAML Token in the Studio. Open the "tESBConsumer" component and select "Use Authentication", and then select the "SAML Token" authentication type. The propagation options are not required for this task - they are used when a SOAP Service is an intermediary service, and wishes to get a new SAML Token "On Behalf Of" a token that it received. Enter "tesb" for the username and password values (this is one of the default users defined in 'etc/users.properties' in the container). Now save the job and build it.

3) Start the STS in the container and deploy the client job

Once the client job has been deployed to the container, it will first attempt to get a SAML Token from the STS. Various properties used by the client to communicate with the STS are defined in 'etc/org.talend.esb.job.client.sts.cfg'. The Talend runtime container ships with a fully fledged STS. Clients can obtain a SAML Token by including a username/password in the request, which the STS in turn authenticates using JAAS (see section 2 in the previous article). Start the STS in container via:

• tesb:start-sts

Now deploy the client job, and it should succeed, with the response message printed in the console. The log 'log/tesb.log' includes the client request and service response messages - in the client request you can see the SAML Assertion included in the security header of the message.