Monday, June 18, 2018

Securing web services using Talend's Open Studio for ESB - part VI

This is the sixth article in a series on securing web services using Talend's Open Studio for ESB. Up to now we have seen how to create and secure a SOAP service, client job and route in the Studio, and how to deploy them to the Talend runtime container. For the remaining articles in this series, we will switch our focus to REST web services instead. In this article we will look at how to implement a REST service and client in the Studio.

1) Implement a "double-it" REST Service in the Studio

First let's look at how we can create implement the "double-it" service as a REST service instead. Open the Studio and right click on "Job Designs" and select "Create job". Create a new job called "DoubleItRESTService". Drag the 'tRESTRequest', 'tXMLMap' and 'tRESTResponse' components from the palette into the central window. Connect them by right-clicking on 'tRESTRequest' and selecting "Row / New Output" and drag the link to 'tXMLMap', calling the output 'Request'. Right-click on 'tXMLMap' and select "Row / New Output" and drag the link to 'tRESTResponse', calling the output 'Response':


Now let's design the REST endpoint by clicking on 'tRESTRequest'. Our simple "double-it" service will accept a path parameter corresponding to the number to double. It will return an XML or JSON response containing the doubled number wrapped in a "result" tag. Edit the 'REST endpoint' to add "/doubleit" at the end of the URL. In the REST API mapping, edit the "URI Pattern" to be "/{number}". Now click on the "Output Flow" for "Request" and click on the three dots that appear. Click the "+" button and change the column name to "number" and the Type to "Integer":


Click "OK" and then double-click on 'tXMLMap'. Left-click on the "Number" column on the left-hand side, and drag it over to the right-hand side to the "body" column. Select "Add Linker to Target Node". Now click on "Request.number" on the right-hand side and then on the three dots. Change the expression to "2 * Request.number" to implement the "doubling" logic. Finally, rename the "root" element to "result":


Finally click "OK", save the job and run it. We can test via a console that the job is working OK using a tool such as curl:
  • curl -H "Accept: application/xml" http://localhost:8088/doubleit/15
  • Response: <?xml version="1.0" encoding="UTF-8"?><result>30</result>
  • Response if we ask for JSON: {"result":30}
2) Implement a "double-it" REST client in the Studio

Now we'll design a client job for the "double-it" REST service in the Studio. Right-click on "Job Designs" and create a new job called "DoubleItRESTClient". Drag a 'tFixedFlowInput', 'tRESTClient' and two 'tLogRow' components from the palette into the central window. Link the components, sending the 'tRESTClient'
"Response" to one 'tLogRow' component and the "Error" to the other:

Now click on 'tFixedFlowInput' and then 'Edit Schema'. Add a new column called "number" of type "Integer", and click "yes" to propagate the changes. In the inline table, add a value for the number. Finally, click on 'tRESTClient' and specify "http://localhost:8088/doubleit/" for the URL, and row1.number for the relative path. Keep the default HTTP Method of "GET" and "XML" for the "Accept Type":


Now save the job and run it. The service response should be displayed in the window of the run tab. In the next article, we'll look at how to secure this REST service in the Studio when deploying it to the Talend runtime container.

Friday, June 15, 2018

Securing web services using Talend's Open Studio for ESB - part V

This is the fifth article in a series on securing web services using Talend's Open Studio for ESB. So far we have seen how to design a SOAP service and client in the Studio, how to deploy them to the Talend runtime container, and how to secure them using a UsernameToken and SAML token. In addition to designing 'jobs', the Studio also offers the ability to create a 'route'. Routes leverage the capabilities and components of Apache Camel, which is a popular integration framework. In this article, we will design a route to invoke on the SAML-secured service we configured in the previous tutorial, instead of using a job.

1) Create a route to invoke on the "double-it" service

In the Studio, right-click on 'Routes' in the left-hand pane, and select 'Create Route' and create a new route called 'DoubleItClientRoute'. Select the 'cTimer', 'cSetBody', 'cSOAP' and 'cLog' components from the palette on the right-hand side and drag them into the route window from left to right. Link the components up by right clicking on each component, and selecting 'Row' and then 'Route' and left-clicking on the next component over:


Now let's configure each component in turn. The 'cTimer' component is used to start the route. You can run the route an arbitrary number of times with a specified delay, or else specify a start time to run the route. For now just enter '1' for 'Repeat' as we want to run the route once. Now click on the 'cSetBody' component. This is used to specify the Body of the request we are going to make on the remote (SOAP) service. For simplicity we will just hard-code the SOAP Body, so select 'CONSTANT' as the Language and input '"<ns2:DoubleItRequest xmlns:ns2=\"http://www.talend.org/service/\">60</ns2:DoubleItRequest>"' for the expression:


Now we will configure the 'cSOAP' component. First, deploy the SAML-secured SOAP service on the container (see previous tutorial) so that we have access to the WSDL. Double-click 'cSOAP' and enter 'http://localhost:8040/services/DoubleIt?wsdl' for the WSDL and hit the reload icon on the right-hand side and click 'Finish'. We will use the default dataformat of 'PAYLOAD' (the SOAP Body contents we set in 'cSetBody'). Select 'Use Authentication' and then pick "SAML Token". Input 'tesb' for the Username and Password values, and save the route.


2) Deploy the route to the container

Right click on the route name in the left-hand pane and select 'Build Route' to build the .kar file. In the container where the SAML-secured service should already be running, start the STS with 'tesb:start-sts', and then copy the client route .kar file into the 'deploy' folder. Consult the log in 'log/tesb.log' and you will see the successful service response as follows:


Wednesday, June 13, 2018

Combining Keycloak with the Apache CXF STS

The Apache CXF STS (Security Token Service) is a web service (both SOAP and REST are supported) that issues tokens (e.g. SAML, JWT) to authenticated users. It can also validate, renew and cancel tokens. To invoke successfully on the STS, a user must present credentials to the STS for authentication. The STS must be configured in turn to authenticate the user credentials to some backend. Another common requirement is to retrieve claims relating to the authenticated user from some backend to insert into the issued token.

In this post we will look at how the STS could be combined with Keycloak to both authenticate users and to retrieve the roles associated with a given user. Typically, Keycloak is used as an IdM for authentication using the SAML SSO or OpenId Connect protocols. However in this post we will leverage the Admin REST API.

I have created a project on github to deploy the CXF STS and Keycloak via docker here.

1) Configuring the STS

Checkout the project from github. The STS is configured is a web application that is contained in the 'src' folder. The WSDL defines a single endpoint with a security policy that requires the user to authenticate via a WS-Security UsernameToken. The STS is configured in spring. Essentially we define a custom 'validator' to validate the UsernameToken, as well as a custom ClaimsHandler to handle retrieving role claims from Keycloak. We also configure the STS to issue SAML tokens.

UsernameTokens are authenticated via the KeycloakUTValidator in the project source. This class is configured with the Keycloak address and realm and authenticates received tokens as follows:

Here we use the Keycloak REST API to search for the user matching the given username, using the given username and password as credentials. What the client API is actually doing behind the scenes here is to obtain an access token from Keycloak using the OAuth 2.0 resource owner password credentials grant, something that can be replicated with a tool like curl as follows:
  • curl --data "client_id=admin-cli&grant_type=password&username=admin&password=password" http://localhost:9080/auth/realms/master/protocol/openid-connect/token -v
  • curl -H "Authorization: bearer <access token>" http://localhost:9080/auth/admin/realms/master/users -H "Accept: application/json" -v
Keycloak will return a HTTP status code of 401 if authentication fails. We allow the case that Keycloak returns 403 unauthorized, as the user may not be authorized to invoke on the admin-cli client. A better approach would be to emulate Apache Syncope and have a "users/self" endpoint to allow users to retrieve information about themselves, but I could not find an analogous endpoint in Keycloak.

Role claims are retrieved via the KeycloakRoleClaimsHandler. This uses the admin credentials to search for the (already authenticated) user, and obtains the effective "realm-level" roles to add to the claim.

2) Running the testcase in docker

First build the STS war and create a docker image for the STS as follows:
  • mvn clean install
  • docker build -t coheigea/cxf-sts-keycloak . 
This latter command just deploys the war that was built into a Tomcat docker image via this Dockerfile. Then pull the official Keycloak docker image and start both via docker-compose (see here):
  • docker pull jboss/keycloak
  • docker-compose up
This starts the STS on port 8080 and Keycloak on port 9080. Log on to the Keycloak administration console at http://localhost:9080/auth/ using the username "admin" and password "password". Click on "Roles" and add a role for a user (e.g. "employee"). The click on "Users" and add a new user. After saving, click on "Credentials" and specify a password (unselecting "Temporary"). Then click on "Role Mappings" and select the role you created above for the user.

Now we will use SoapUI to invoke on the STS. Download it and create a new SOAP project using the WSDL of the STS (http://localhost:8080/cxf-sts-keycloak/UT?wsdl). Click on 'Issue' and select the request. We need to edit the SOAP Body of the request to instruct the STS to issue a SAML Token with a Role Claim using the standard WS-Trust parameters:

<ns:RequestSecurityToken>
     <t:TokenType xmlns:t="http://docs.oasis-open.org/ws-sx/ws-trust/200512">http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV2.0</t:TokenType>
     <t:KeyType xmlns:t="http://docs.oasis-open.org/ws-sx/ws-trust/200512">http://docs.oasis-open.org/ws-sx/ws-trust/200512/Bearer</t:KeyType>
     <t:RequestType xmlns:t="http://docs.oasis-open.org/ws-sx/ws-trust/200512">http://docs.oasis-open.org/ws-sx/ws-trust/200512/Issue</t:RequestType>
     <t:Claims xmlns:ic="http://schemas.xmlsoap.org/ws/2005/05/identity" xmlns:t="http://docs.oasis-open.org/ws-sx/ws-trust/200512" Dialect="http://schemas.xmlsoap.org/ws/2005/05/identity">
        <ic:ClaimType xmlns:ic="http://schemas.xmlsoap.org/ws/2005/05/identity" Uri="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/role"/>
     </t:Claims>
</ns:RequestSecurityToken>

Click in the Properties box in the lower left-hand corner and specify the username and password for the user you created in Keycloak. Finally, right click on the request and select "Add WSS UsernameToken" and hit "OK" and send the request. If the request was successful you should see the SAML Assertion issued by the STS on the right-hand side. In particular, note that the Assertion contains a number of Attributes corresponding to the roles of that particular user.


Monday, June 11, 2018

Running the Apache Kerby KDC in docker

Apache Kerby is a subproject of the Apache Directory project, and is a complete open-source KDC written entirely in Java. Apache Kerby 1.1.1 has been released recently. Last year I wrote a blog post about how to configure and launch Apache Kerby, by first obtaining the source distribution and building it using Apache Maven. In this post we will cover an alternative approach, which is to download and run a docker image I have prepared which is based on Apache Kerby 1.1.1.

The project is available on github here and the resulting docker image is available here. Note that this is not an official docker image - and so it provided just for testing or experimentation purposes. First clone the github repository and either build the image from scratch or download it from dockerhub:
  • docker build . -t coheigea/kerby
 or:
  • docker pull coheigea/kerby
The docker image builds a KDC based on Apache Kerby and runs it when started. However, it expects a directory to be supplied as the first argument (defaults to '/kerby-data/conf') containing the configuration files for Kerby. The github repository contains the relevant files in the 'kerby-data' directory. As well as the configuration files, it stores the admin keytab and a JSON file containing the default principals for the KDC.

Start the KDC by mapping the kerby-data directory to a volume on the container:
  • docker run -it -p 4000:88 -v `pwd`/kerby-data:/kerby-data coheigea/kerby
Now we can log into the docker image and create a user for our tests:
  • docker exec -it <id> bash
  • stty rows 24 columns 80 (required to run jline in docker)
  • sh bin/kadmin.sh /kerby-data/conf/ -k /kerby-data/keytabs/admin.keytab
  • Then: addprinc -pw password alice@EXAMPLE.COM
To test the KDC from outside the container you can use the MIT kinit tool. Set the KRB5_CONFIG environment variable to point to the "krb5.conf" file included in the github repository, e.g:
  • export KRB5_CONFIG=`pwd`/krb5.conf
  • kinit alice
This will get you a ticket for "alice", that can be inspected via "klist".

Friday, June 8, 2018

Securing web services using Talend's Open Studio for ESB - part IV

This is the fourth article in a series on securing web services using Talend's Open Studio for ESB. In the previous article, we looked at how to secure a SOAP webservice in the Talend container, by requiring the client to authenticate using a WS-Security UsernameToken. In this post we will look at an alternative means of authenticating clients using a SAML token, which the client obtains from a Security Token Service (STS) also deployed in the Talend container. This is more sophisticated than the UsernameToken approach, as we can embed claims as attributes in the SAML Assertion, thus allowing the service provider to also make authorization decisions. However, in this article we will just focus on authentication.

1) Secure the "double-it" webservice by requiring clients to authenticate

As in the previous article, first we will secure the "double-it" webservice we have designed in the Studio in the first article, by requiring clients to authenticate using a SAML Token, which is conveyed in the security header of the request. SAML authentication can be configured for a service in the Studio, by right-clicking on the "DoubleIt 0.1" Service in the left-hand menu and selecting "ESB Runtime Options". Under "ESB Service Security" select "SAML Token". Select "OK" and export the service again as detailed in the second article.

Now start the container and deploy the modified service. Note that what selecting the "SAML Token" actually does in the container is to enforce the policy that is stored in 'etc/org.talend.esb.job.saml.policy', which is a WS-SecurityPolicy assertion that requires that a SAML 2.0 token containing an X.509 certificate associated with the client (subject) must be sent to the service. In addition, a Timestamp must be included in the security header of the request, and signed by the private key associated with the X.509 certificate in the Assertion.

2) Update the client job to include a SAML Token in the request

Next we have to update the client job to include a SAML Token in the Studio. Open the "tESBConsumer" component and select "Use Authentication", and then select the "SAML Token" authentication type. The propagation options are not required for this task - they are used when a SOAP Service is an intermediary service, and wishes to get a new SAML Token "On Behalf Of" a token that it received. Enter "tesb" for the username and password values (this is one of the default users defined in 'etc/users.properties' in the container). Now save the job and build it.



3) Start the STS in the container and deploy the client job

Once the client job has been deployed to the container, it will first attempt to get a SAML Token from the STS. Various properties used by the client to communicate with the STS are defined in 'etc/org.talend.esb.job.client.sts.cfg'. The Talend runtime container ships with a fully fledged STS. Clients can obtain a SAML Token by including a username/password in the request, which the STS in turn authenticates using JAAS (see section 2 in the previous article). Start the STS in container via:
  • tesb:start-sts
Now deploy the client job, and it should succeed, with the response message printed in the console. The log 'log/tesb.log' includes the client request and service response messages - in the client request you can see the SAML Assertion included in the security header of the message.

Tuesday, May 29, 2018

Securing web services using Talend's Open Studio for ESB - part III

This is the third article in a series on securing web services using Talend's Open Studio for ESB. In the first article, we looked at how to design and test a SOAP web service in the Studio, and how to create a client job to invoke on it. In the second article we looked at deploying the jobs in the Talend ESB runtime container. In this article, we will look at how to secure the SOAP webservice we are deploying in the container, by requiring the client to authenticate using a WS-Security UsernameToken.

1) Secure the "double-it" webservice by requiring clients to authenticate

First we will secure the "double-it" webservice we have designed in the Studio in the first article, by requiring clients to authenticate using a WS-Security UsernameToken. Essentially what this means is that the client adds a SOAP header to the request containing username and password values, which then must be authenticated by the service. UsernameToken authentication can be configured for a service in the Studio, by right-clicking on the "DoubleIt 0.1" Service in the left-hand menu and selecting "ESB Runtime Options". Under "ESB Service Security" select "Username/Password". Select "OK" and export the service again as detailed in the second article.

Now start the container and deploy the modified service. Note that what selecting the "Username/Password" actually does in the container is to enforce the policy that is stored in 'etc/org.talend.esb.job.token.policy', which is a WS-SecurityPolicy assertion that requires that a UsernameToken must always be sent to the service. Now deploy the client job - you will see an error in the Console along the lines of:

{http://schemas.xmlsoap.org/soap/envelope/}Server|These policy alternatives can not be satisfied:
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}SupportingTokens
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}UsernameToken

This is due to the fact that we have not yet configured the client job to send a
UsernameToken in the request.

2) How authentication works in the container

So far we have required clients to authenticate to the service, but we have not said anything about how the service actually authenticates the credentials that it receives. Apache Karaf uses JAAS realms to handle authentication and authorization. Typing "jaas:realm-list" in the container shows the list of JAAS realms that are installed:

Here we can see that the (default) JAAS realm of "karaf" has been configured with a number of JAAS Login Modules. In particular, in index 1, the PropertiesLoginModule authenticates users against entries in 'etc/users.properties'. This file contains entries that map a username to a password, as well as an optional number of groups. It also contains entries mapping groups to roles. In this example though we are solely concerned with authentication. The service will extract the username and password from the security header of the request and will compare them to the values in 'etc/users.properties'. If there is a match then a user is deemed to be authenticated and the request can proceed.

In a real-world deployment, we can authenticate to users stored in a database or in an LDAP directory server, by configuring a JAAS Realm with the appropriate LoginModules (see the Karaf security guide for a list of available Login Modules).

3) Update the client job to include a UsernameToken

Finally we have to update the client job to include a UsernameToken in the Studio. Open the "tESBConsumer" component and select "Use Authentication", and then select the "Username Token" authentication type. Enter "tesb" for the username and password values (this is one of the default users defined in 'etc/users.properties' in the container).



Now save the job and build and deploy it as per the second tutorial. The job request should succeed, with the response message printed in the console. Examining 'log/tesb.log' it is possible to see what the client request looks like:

In the next article we'll look at authentication using SAML tokens.

Monday, May 21, 2018

Securing web services using Talend's Open Studio for ESB - part II

This is the second article in a series on securing web services using Talend's Open Studio for ESB. In the first article, we looked at how Talend's Open Studio for ESB can be used to design and test a SOAP web service, and also how we can create a client job that invokes on this service. In this article, we will show how to deploy the service and client we created previously in the Talend ESB runtime container.

1) The Talend ESB runtime container

When we downloaded Talend Open Studio for ESB (see the first article), we launched the Talend Studio via the "Studio" directory to design and test our "double it" SOAP service. However, the ability to "Run" the SOAP Service in the Studio is only suitable for testing the design of the service. Once we are ready to deploy a service or client we have created in the Studio, we will need a suitable runtime container, something that is available in the "Runtime_ESBSE" directory in the Talend Open Studio for ESB distribution. The runtime container in question is a powerful and enterprise-ready container based on Apache Karaf. We can start it in the "Runtime_ESBSE/container" directory via "bin/trun":

By default, the Talend ESB runtime starts with a set of default "bundles" (which can be viewed with "la"). All of the libraries that we require will be started automatically, so no further work is required here.

2) Export the service and client job from the Studio

To deploy the SOAP "double it" service, and client job, we need to export them from the Studio. Right click on the "Double It" service in the left-hand menu, and first select "ESB Runtime Options", ticking "Log Messages" so that we can see the input/output messages of the service when we look at the logs. Then, right click again on "Double It" and select "Export Service" and save the resulting .kar file locally.

Before exporting the client job, we need to make one minor change. The default port that the Studio used for the "double it" SOAP service (8090) is different to that of Karaf (8040). Click on "tESBConsumer" and change the port number in the address to "8040". Then after saving, right click on the double it client job and select "Build job". Under "Build Type" select "OSGI bundle for ESB", and click "Finish" to export the job:

3) Deploy the service and client jobs to the Runtime Container

Finally, we need to deploy the service and client jobs to the Runtime Container. First, copy the service .kar file into "Runtime_ESBSE/container/deploy". This will automatically deploy the service in Karaf (something that can be verified by running "la" in the console - you should see the service as the last bundle on the list). Then also copy the client jar into the "deploy" directory. The response will be output in the console window (due to the tLogRow component), and the full message can be seen in the server logs ("log/tesb.log"):