Friday, August 12, 2016

OpenId Connect in Apache CXF Fediz 1.3.0

Previous blog posts have described support for OpenId Connect protocol bridging in the Apache CXF Fediz IdP. What this means is that the Apache CXF Fediz IdP can bridge between the WS-Federation protocol and OpenId Connect third party IdPs, when the user must be authenticated in a different security domain. However, the 1.3.0 release of Apache CXF Fediz also sees the introduction of a new OpenId Connect Idp which is independent of the existing (WS-Federation and SAML-SSO based) IdP, and based on Apache CXF. This post will introduce the new IdP via an example.

The example code is available on github:
  • cxf-fediz-oidc: This project shows how to use interceptors of Apache CXF to authenticate and authorize clients of a JAX-RS service using OpenId Connect.
1) The secured service

The first module available in the example contains a trivial JAX-RS Service based on Apache CXF which "doubles" a number that is passed as a path parameter via HTTP GET. The service defines via a @RolesAllowed annotation that only users allowed in roles "User", "Admin" or "Manager" can access the service.

The service is configured via spring. The endpoint configuration references the service bean above, as well as the CXF SecureAnnotationsInterceptor which enforces the @RolesAllowed annotation on the service bean. In addition, the service is configured with the CXF OidcRpAuthenticationFilter, which ensures that only users authenticated via OpenId Connect can access the service. The filter is configured with a URL to redirect the user to. It also explicitly requires a role claim to enforce authorization.

The OidcRpAuthenticationFilter redirects the browser to a separate authentication endpoint, defined in the same spring file for convenience. This endpoint has a filter called OidcClientCodeRequestFilter, which initiates the OpenId Connect authorization code flow to a remote OpenId Connect IdP (in this case, the new Fediz IdP). It is also responsible for getting an IdToken after successfully getting an authorization code from the IdP.

2) The Fediz OpenId Connect IdP

The second module contains an integration test which deploys a number of wars into an Apache Tomcat container:
  • The "double-it" service as described above
  • The Apache CXF Fediz IdP which authenticates users via WS-Federation
  • The Apache CXF Fediz STS which performs the underlying authentication of users
  • The Apache CXF Fediz OpenId Connect IdP
The way the Apache CXF Fediz OpenId Connect IdP works (at least for 1.3.x) is that user authentication is actually delegated to the WS-Federation based IdP via a Fediz plugin. So when the user is redirected to the Fediz IdP, (s)he gets redirected to the WS-Federation based IdP for authentication, and then gets redirected back to the OpenId Connect IdP with a WS-Federation Response. The OpenId Connect IdP parses this (SAML) Response and converts it into a JWT IdToken. Future releases will enable authentication directly at the OpenId Connect service.

After deploying all of the services, the test code makes a series of REST calls to create a client in the OpenId Connect IdP so that we can run the test without having to manually enter information in the client UI of the Fediz IdP. To run the test, simply remove the @org.junit.Ignore assertion on the "testInBrowser" method. The test code will create the clients in Fediz and then print out a URL in the console before sleeping. Copy the URL and paste it into a browser. Authenticate using the credentials "alice/ecila".

Thursday, August 11, 2016

Introducing Apache Syncope 2.0.0

Apache Syncope is a powerful and flexible open-source Identity Management system that has been developed at the Apache Software Foundation for several years now. The Apache Syncope team has been busy developing a ton of new features for the forthcoming new major release (2.0.0), which will really help to cement Apache Syncope's position as a first class Identity Management solution. If you wish to experiment with these new features, a 2.0.0-M4 release is available. In this post we will briefly cover some of the new features and changes. For a more comprehensive overview please refer to the reference guide.

1) Domains

Perhaps the first new concept you will be introduced to in Syncope 2.0.0 after starting the (Admin) console is that of a domain. When logging in, as well as specifying a username, password, and language, you can also specify a configured domain. Domains are a new concept in Syncope 2.0.0 that facilitate multi-tenancy. Domains allow the physical separation of all data stored in Syncope (by storing the data in different database instances). Therefore, Syncope can facilitate users, groups etc. that are in different domains in a single Syncope instance.

2) New Console layout

After logging in, it becomes quickly apparent that the Syncope Console is quite different compared to the 1.2.x console. It has been completely rewritten and looks great. Connectors and Resources are now managed under "Topology" in the menu on the left-hand side. Users and Groups (formerly Roles) are managed under "Realms" in the menu. The Schema types are configured under "Configuration". A video overview of the new Console can be seen here.


3) AnyType Objects

With Syncope 1.2.x, it was possible to define plain/derived/virtual Schema Types for users, roles and memberships, but no other entities. In Syncope 2.0.0, the Schema Types are decoupled from the entity that uses them. Instead, a new concept called an AnyType class is available which is a collection of schema types. In turn, an AnyType object can be created which consists of any number of AnyType classes. AnyType objects represent the type of things that Apache Syncope can model. Besides the predefined Users and Groups, it can also represent physical things such as printers, workstations, etc. With this new concept, Apache Syncope 2.0.0 can model many different types of identities.

4) Realms

Another new concept in Apache Syncope 2.0.0 is that of a realm. A realm encapsulates a number of Users, Groups and Any Objects. It is possible to specify account and password policies per-realm (see here for a blog entry on custom policies in Syncope 2.0.0). Each realm has a parent realm (apart from the pre-defined root realm identified as "/"). The realm tree is hierarchical, meaning that Users, Groups etc. defined in a sub-realm, are also defined on a parent realm. Combined with Roles (see below), realms facilitate some powerful access management scenarios.

5) Groups/Roles

In Syncope 2.0.0, what were referred to as "roles" in Syncope 1.2.x are now called "groups". In addition, "roles" in Syncope 2.0.0 are a new concept which associate a number of entitlements with a number of realms. Users assigned to a role can exercise the defined entitlements on any of the objects in the given realms (any any sub-realms).

Syncope 2.0.0 also has the powerful concept of dynamic membership, which means that users can be assigned to groups or roles via a conditional expression (e.g. if an attribute matches a given value).

6) Apache Camel Provisioning

An exciting new feature of Apache Syncope 2.0.0 is the new Apache Camel provisioning engine, which is available under "Extensions/Camel Routes" in the Console. Apache Syncope comes pre-loaded with some Camel routes that are executed as part of the provisioning implementation for Users, Groups and Any Objects. The real power of this new engine lies is the ability to modify the routes to perform some custom provisioning rules. For example, on creating a new user, you may wish to send an email to an administrator. Or if a user is reactivated, you may wish to reactivate the user's home page on a web server. All these things and more are possible using the myriad of components that are available to be used in Apache Camel routes. I'll explore this feature some more in future blog posts.

7) End-User UI

As well as the Admin console (available via /syncope-console), Apache Syncope 2.0.0 also ships with an Enduser console (available via /syncope-enduser). This allows a user to edit only details pertaining to his/her-self, such as editing the user attributes, changing the password, etc. See the following blog entry for more information on the new End-User UI.


8) Command Line Interface (CLI) client

Another new feature of Apache Syncope 2.0.0 is that of the CLI client. It is available as a separate download. Once downloaded, extract it and run (on linux): ./syncopeadm.sh install --setup. Answer the questions about where Syncope is deployed and the credentials required to access it. After installation, you can run queries such as: ./syncopeadm.sh user --list.

9) Apache CXF-based testcases

I updated the testcases that I wrote before to use Apache Syncope 2.0.0 to authenticate and authorize web services calls using Apache CXF. The new test-cases are available here

Monday, August 8, 2016

Installing the Apache Ranger Key Management Server (KMS)

The previous couple of blog entries have looked at how to install the Apache Ranger Admin Service as well as the Usersync Service. In this post we will look at how to install the Apache Ranger Key Management Server (KMS). KMS is a component of Apache Hadoop to manage cryptographic keys. Apache Ranger ships with its own KMS implementation, which allows you to store the (encrypted) keys in a database. The Apache Ranger KMS is also secured via policies defined in the Apache Ranger Admin Service.

1) Build the source code

The first step is to download the source code, as well as the signature file and associated message digests (all available on the download page). Verify that the signature is valid and that the message digests match. Now extract and build the source, and copy the resulting KMS archive to a location where you wish to install it:
  • tar zxvf apache-ranger-incubating-0.6.0.tar.gz
  • cd apache-ranger-incubating-0.6.0
  • mvn clean package assembly:assembly 
  • tar zxvf target/ranger-0.6.0-kms.tar.gz
  • mv ranger-0.6.0-kms ${rangerkms.home}
2) Install the Apache Ranger KMS Service

As the Apache Ranger KMS Service stores the cryptographic keys in a database, we will need to setup and configure a database. We will also configure the KMS Service to store audit logs in the database. Follow the steps given in section 2 of the tutorial on the Apache Ranger Admin Service to set up MySQL. We will also need to create a new user 'rangerkms':
  • CREATE USER 'rangerkms'@'localhost' IDENTIFIED BY 'password';
  • FLUSH PRIVILEGES; 
You will need to install the Apache Ranger KMS Service using "sudo". If the root user does not have a JAVA_HOME property defined, then edit ${rangerkms.home}/setup.sh + add in, e.g.:
  • export JAVA_HOME=/opt/jdk1.8.0_91
Next edit ${rangerkms.home}/install.properties and make the following changes:
  • Change SQL_CONNECTOR_JAR to point to the MySQL JDBC driver jar (see previous tutorial).
  • Set (db_root_user/db_root_password) to (admin/password)
  • Set (db_user/db_password) to (rangerkms/password)
  • Change KMS_MASTER_KEY_PASSWD to a secure password value.
  • Set POLICY_MGR_URL=http://localhost:6080
  • Set XAAUDIT.DB.IS_ENABLED=true
  • Set XAAUDIT.DB.FLAVOUR=MYSQL 
  • Set XAAUDIT.DB.HOSTNAME=localhost 
  • Set XAAUDIT.DB.DATABASE_NAME=ranger_audit 
  • Set XAAUDIT.DB.USER_NAME=rangerlogger
  • Set XAAUDIT.DB.PASSWORD=password
Now you can run the setup script via "sudo ./setup.sh".

3) Starting the Apache Ranger KMS service

After a successful installation, first start the Apache Ranger admin service with "sudo ranger-admin start". Then start the Apache Ranger KMS Service via "sudo ranger-kms start". Now open a browser and go to "http://localhost:6080/". Log on with "keyadmin/keyadmin". Note that these are different credentials to those used to log onto the Apache Ranger Admin UI in the previous tutorial. Click on the "+" button on the "KMS" tab to create a new KMS Service. Specify the following values:
  • Service Name: kmsdev
  • KMS URL: kms://http@localhost:9292/kms
  • Username: keyadmin
  • Password: keyadmin
Click on "Test Connection" to make sure that the KMS Service is up and running. If it is showing a connection failure, log out and log into the Admin UI using credentials "admin/admin". Go to the "Audit" section and click on "Plugins". You should see a successful message indicating that the KMS plugin can successfully download policies from the Admin Service:


After logging back in to the UI as "keyadmin" you can start to create keys. Click on the "Encryption/Key Manager" tab. Select the "kmsdev" service in the dropdown list and click on "Add New Key". You can create, delete and rollover keys in the UI:



Friday, July 22, 2016

Syncing users and groups from LDAP into Apache Ranger

The previous post covered how to install the Apache Ranger Admin service. The Apache Ranger Admin UI supports creating authorization policies for various Big Data components, by giving users and/or groups permissions on resources. This means that we need to import users/groups into the Apache Ranger Admin service from some backend service in order to create meaningful authorization policies. Apache Ranger supports syncing users into the Admin service from both unix and ldap. In this post, we'll look at syncing in users and groups from an OpenDS LDAP backend.

1) The OpenDS backend

For the purposes of this tutorial, we will use OpenDS as the LDAP server. It contains a domain called "dc=example,dc=com", and 5 users (alice/bob/dave/oscar/victor) and 2 groups (employee/manager). Victor, Oscar and Bob are employees, Alice and Dave are managers. Here is a screenshot using Apache Directory Studio:



2) Build the Apache Ranger usersync module

Follow the steps in the previous tutorial to build Apache Ranger and to setup and start the Apache Ranger Admin service. Once this is done, go back to the Apache Ranger distribution that you have built and copy the usersync module:
  • tar zxvf target/ranger-0.6.0-usersync.tar.gz
  • mv ranger-0.6.0-usersync ${usersync.home}
3) Configure and build the Apache Ranger usersync service 

You will need to install the Apache Ranger Usersync service using "sudo". If the root user does not have a JAVA_HOME property defined, then edit ${usersync.home}/setup.sh + add in, e.g.:
  • export JAVA_HOME=/opt/jdk1.8.0_91
Next edit ${usersync.home}/install.properties and make the following changes:
  • POLICY_MGR_URL = http://localhost:6080
  • SYNC_SOURCE = ldap
  • SYNC_INTERVAL = 1 (just for testing purposes....)
  • SYNC_LDAP_URL = ldap://localhost:2389
  • SYNC_LDAP_BIND_DN = cn=Directory Manager,dc=example,dc=com
  • SYNC_LDAP_BIND_PASSWORD = test
  • SYNC_LDAP_SEARCH_BASE = dc=example,dc=com
  • SYNC_LDAP_USER_SEARCH_BASE = ou=users,dc=example,dc=com
  • SYNC_GROUP_SEARCH_BASE=ou=groups,dc=example,dc=com
Now you can run the setup script via "sudo ./setup.sh". 

4) Start the Usersync service

The Apache Ranger Usersync service can be started via "sudo ./ranger-usersync-services.sh start". After 1 minute (see SYNC_INTERVAL above), it should successfully copy the users/groups from the OpenDS backend into the Apache Ranger Admin. Open a browser and go to "http://localhost:6080", and click on "Settings" and then "Users/Groups". You should see the users and groups synced successfully from OpenDS.

Tuesday, July 19, 2016

Installing the Apache Ranger Admin UI

Apache Ranger 0.6 has been released, featuring new support for securing Apache Atlas and Nifi, as well as a huge amount of bug fixes. It's easiest to get started with Apache Ranger by downloading a big data sandbox with Ranger pre-installed. However, the most flexible way is to grab the Apache Ranger source and to build and deploy the artifacts yourself. In this tutorial, we will look into building Apache Ranger from source, setting up a database to store policies/users/groups/etc. as well as Ranger audit information, and deploying the Apache Ranger Admin UI.

1) Build the source code

The first step is to download the source code, as well as the signature file and associated message digests (all available on the download page). Verify that the signature is valid and that the message digests match. Now extract and build the source, and copy the resulting admin archive to a location where you wish to install the UI:
  • tar zxvf apache-ranger-incubating-0.6.0.tar.gz
  • cd apache-ranger-incubating-0.6.0
  • mvn clean package assembly:assembly 
  • tar zxvf target/ranger-0.6.0-admin.tar.gz
  • mv ranger-0.6.0-admin ${rangerhome}
2) Install MySQL

The Apache Ranger Admin UI requires a database to keep track of users/groups as well as policies for various big data projects that you are securing via Ranger. In addition, we will use the database for auditing as well. For the purposes of this tutorial, we will use MySQL. Install MySQL in $SQL_HOME and start MySQL via:
  • sudo $SQL_HOME/bin/mysqld_safe --user=mysql
Now you need to log on as the root user and create three users for Ranger. We need a root user with admin privileges (let's call this user "admin"), a user for the Ranger Schema (we'll call this user "ranger"), and finally a user to store the Ranger audit logs in the DB as well ("rangerlogger"):
  • CREATE USER 'admin'@'localhost' IDENTIFIED BY 'password';
  • GRANT ALL PRIVILEGES ON * . * TO 'admin'@'localhost' WITH GRANT OPTION;
  • CREATE USER 'ranger'@'localhost' IDENTIFIED BY 'password';
  • CREATE USER 'rangerlogger'@'localhost' IDENTIFIED BY 'password'; 
  • FLUSH PRIVILEGES;
Finally,  download the JDBC driver jar for MySQL and put it in ${rangerhome}.

3) Install the Apache Ranger Admin UI

You will need to install the Apache Ranger Admin UI using "sudo". If the root user does not have a JAVA_HOME property defined, then edit ${rangerhome}/setup.sh + add in, e.g.:
  • export JAVA_HOME=/opt/jdk1.8.0_91
Next edit ${rangerhome}/install.properties and make the following changes:
  • Change SQL_CONNECTOR_JAR to point to the MySQL JDBC driver jar that you downloaded above.
  • Set (db_root_user/db_root_password) to (admin/password)
  • Set (db_user/db_password) to (ranger/password)
  • Change "audit_store" from "solr" to "db"
  • Set "audit_db_name" to "ranger_audit"
  • Set (audit_db_user/audit_db_password) to (rangerlogger/password).
Now you can run the setup script via "sudo ./setup.sh".

4) Starting the Apache Ranger admin service

After a successful installation, we can start the Apache Ranger admin service with "sudo ranger-admin start". Now open a browser and go to "http://localhost:6080/". Log on with "admin/admin" and you should be able to create authorization policies for a desired big data component.


Friday, June 17, 2016

A new REST interface for the Apache CXF Security Token Service - part II

The previous blog entry introduced the new REST interface of the Apache CXF Security Token Service. It covered issuing, renewing and validating tokens via HTTP GET and POST with a focus on SAML tokens. In this post, we'll cover the new support in the STS to issue and validate JWT tokens via the REST interface, as well as how we can transform SAML tokens into JWT tokens, and vice versa. For information on how to configure the STS to support JWT tokens (via WS-Trust), please look at a previous blog entry.

1) Issuing JWT Tokens

We can retrieve a  JWT Token from the REST interface of the STS simply by making a HTTP GET request to "/token/jwt". Specify the "appliesTo" query parameter to insert the service address as the audience of the generated token. You can also include various claims in the token via the "claim" query parameter.

If an "application/xml" accept type is specified, or if multiple accept types are specified (as in a browser), the JWT token is wrapped in a "TokenWrapper" XML Element by the STS:

If "application/json" is specified, the JWT token is wrapped in a simple JSON Object as follows:

If "text/plain" is specified, the raw JWT token is returned. We can also get a JWT token using HTTP POST by constructing a WS-Trust RequestSecurityToken XML fragment, specifying "urn:ietf:params:oauth:token-type:jwt" as the WS-Trust TokenType.

2) Validating JWT Tokens and token transformation

We can also validate JWT Tokens by POSTing a WS-Trust RequestSecurityToken to the STS. The raw (String) JWT Token must be wrapped in a "TokenWrapper" XML fragment, which in turn is specified as the value of "ValidateTarget". Also, an "action" query parameter must be added with value "validate".

A powerful feature of the STS is the ability to transform tokens from one type to another. This is done by making a validate request for a given token, and specifying a value for "TokenType" that corresponds to a token type that is desired.  In this way we can validate a SAML token and issue a JWT Token, and vice versa.

To see some examples of how to do token validation as well as token transformation, please take a look at the following tests, which use the CXF WebClient to invoke on the REST interface of the STS.

Thursday, June 16, 2016

A new REST interface for the Apache CXF Security Token Service - part I

Apache CXF ships a Security Token Service (STS) that can issue/validate/renew/cancel tokens via the (SOAP based) WS-Trust interface. The principal focus of the STS is to deal with SAML tokens, although other token types are supported as well. JAX-RS clients can easily obtain tokens using helper classes in CXF (see Andrei Shakirin's blog entry for more information on this).

However, wouldn't it be cool if a JAX-RS client could dispense with SOAP altogether and obtain tokens via a REST interface? Starting from the 3.1.6 release, the Apache CXF STS now has a powerful and flexible REST API which I'll describe in this post. The next post will cover how the STS can now issue and validate JWT tokens, and how it can transform JWT tokens into SAML tokens and vice versa.

One caveat - this REST interface is obviously not a standard, as compared to the WS-Trust interface, and so is specific to CXF.

1) Configuring the JAX-RS endpoint of the CXF STS

Firstly, let's look at how to set up the JAX-RS endpoint needed to support the REST interface of the STS. The STS is configured in exactly the same way as for the standard WS-Trust based interface, the only difference being that we are setting up a JAX-RS endpoint instead of a JAX-WS endpoint. Example configuration can be seen here in a system test.


2) Issuing Tokens

The new JAX-RS interface supports a number of different methods to obtain tokens. In this post, we will just focus on the methods used to obtain XML based tokens (such as SAML).

2.1) Issue tokens via HTTP GET

The easiest way to get a token is by doing a HTTP GET on "/token/{tokenType}". Here is a simple example using a web browser:

 The supported tokenType values are:
  • saml
  • saml2.0
  • saml1.1
  • jwt
  • sct
A number of optional query parameters are supported:
  • keyType - The standard WS-Trust based URIs to indicate whether a Bearer, HolderOfKey, etc. token is required. The default is to issue a Bearer token.
  • claim - A list of requested claims to include in the token. See below for the list of acceptable values.
  • appliesTo - The AppliesTo value to use
  • wstrustResponse - A boolean parameter to indicate whether to return a WS-Trust Response or just the desired token. The default is false. This parameter only applies to the XML case.
Note that for the "Holder of Key" keytype, the STS will try to get the client key via the TLS client certificate, if it is available. The (default) supported claim values are:
  • emailaddress
  • role
  • surname
  • givenname
  • name
  • upn
  • nameidentifier
The format of the returned token depends on the HTTP application type:
  • application/xml - Returns a token in XML format
  • application/json - JSON format for JWT tokens only.
  • text/plain - The "plain" token is returned. For JWT tokens it is just the raw token. For XML-based tokens, the token is BASE-64 encoded and returned.
The default is to return XML if multiple types are specified (e.g. in a browser).

2.2) Issue tokens via HTTP POST

While the GET method above is very convenient to use, it may be that you want to pass other parameters to the STS in order to issue a token. In this case, you can construct a WS-Trust RequestSecurityToken XML fragment and POST it to the STS. The response will be the standard WS-Trust Response, and not the raw token as you have the option of receiving via the GET method.

3) Validating/Renewing/Cancelling tokens

It is only possible to renew or validate or cancel a token using the POST method detailed in section 2.2 above. You construct the RequestSecurityToken XML fragment in the same way as for Issue, except including the token in question under "ValidateTarget", "RenewTarget", etc. For the non-issue use case, you must specify a "action" query parameter, which can be "issue" (default), "validate", "renew", "cancel".