Wednesday, August 31, 2016

Integrating Apache Camel with Apache Syncope - part I

Apache Syncope is an open-source Identity Management solution. A key feature of Apache Syncope is the ability to pull Users, Groups and Any Objects from multiple backend resources (such as LDAP, RDMBS, etc.) into Syncope's internal storage, where they can then be assigned roles, pushed to other backend resources, exposed via Syncope's REST API, etc.

However, what if you wanted to easily perform some custom task as part of the Identity Management process? Wouldn't it be cool to be able to plug in a powerful integration framework such as Apache Camel, so that you could exploit Camel's huge list of messaging components and routing + mediation rules? Well with Syncope 2.0.0 you can do just this with the new Apache Camel provisioning manager. This is a unique and very powerful selling point of Apache Syncope in my opinion. In this article, we will introduce the new Camel provisioning manager, and show a simple example of how to use it.

1) The new Apache Camel provisioning manager

As stated above, a new provisioning manager is available in Apache Syncope 2.0.0 based on Apache Camel. A set of Camel routes are available by default which are invoked when the User, Groups and Any Objects in question are changed in some way. So for example, if a new User is created, then the corresponding Camel route is invoked at the same time. This allows the administrator to plug in custom logic on any of these state changes. The routes can be viewed and edited in the Admin Console by clicking on "Extensions" and then "Camel Routes".

Each of the Camel routes uses a new "propagate" Camel component available in Syncope 2.0.0. This component encapsulates some common logic involved in using the Syncope PropagationManager to create some tasks, and to execute them via the PropagationTaskExecutor. All of the routes invoke this propagate component via something like:
  • <to uri="propagate:<propagateType>?anyTypeKind=<anyTypeKind>&options"/>
Where propagateType is one of:
  • create
  • update
  • delete
  • provision
  • deprovision
  • status
  • suspend
  • confirmPasswordReset
and anyTypeKind is one of:
  • USER
  • GROUP
  • ANY
2) The use-case

In this post, we will look at a simple use-case of sending an email to an administrator when a User is created, with some details about the created User in the email. Of course, this could be handled by a Notification Task, but we'll discuss some more advanced scenarios in future blog posts. Also note that a video available on the Tirasa blog details more or less the same use-case. For the purposes of the demo, we will set up a mailtrap account where we will receive the emails sent by Camel.

3) Configure Apache Syncope

Download and install  Apache Syncope (I used the "standalone" download for the purposes of this demo). Before starting Apache Syncope, we need to copy a few jars that are required by Apache Camel to actually send emails. Copy the following jars to $SYNCOPE/webapps/syncope/WEB-INF/lib:
  • http://repo1.maven.org/maven2/org/apache/camel/camel-mail/2.17.3/camel-mail-2.17.3.jar
  • http://repo1.maven.org/maven2/com/sun/mail/javax.mail/1.5.5/javax.mail-1.5.5.jar
Now start Apache Syncope and log on to the admin console. Click on "Extensions" and then "Camel Routes". As we want to change the default route when users are created, click on the "edit" image for the "createUser" route. Add the following information just above the "bean method=" line:
  • <setHeader headerName="subject"><simple>New user ${body.username} created in realm ${body.realm}</simple></setHeader> 
  • <setBody><simple>User full name: ${body.plainAttrMap[fullname].values[0]}</simple></setBody>
  • <to uri="smtp://mailtrap.io?username=<username>&amp;password=<password>&amp;contentType=text/html&amp;to=dummy@apache-recipient.org"/>
Let's examine what each of these statements do. The first statement is setting the Camel header "Subject" which corresponds to the Subject of the Email. It simply states that a new user with a given name is created in a given realm. The second statement sets the message Body, which is used as the content of the message by Camel. It just shows the User's full name, extracted from the "fullname" attribute, as an example of how to access User attributes in the route.

The third statement invokes on the Camel smtp component. You'll need to substitute in the username + password you configured when setting up the mailtrap account. The recipient is configured using the "to" part of the URI. One more change is required to the existing route. As we have overridden the message Body in the second statement above, we need to change the ${body} in the create call to ${exchangeProperty.actual}, which is the saved Body. Click on "save" to save the modified route.


Before creating a User, we need to add a "fullname" User attribute as the route expects. Go to "Configuration" and "Types", and click on the "Schemas" tab. Click on the "+" button under "PLAIN" and add a new attribute called "fullname". Then click on "AnyTypeClasses", and add the "fullname" attribute to the BaseUser AnyTypeClass.

Finally, go to the "/" realm and create a new user, specifying a fullname attribute. A new email should be available in the mailtrap account as follows:

Friday, August 26, 2016

Pulling users and groups from LDAP into Apache Syncope 2.0.0

A previous tutorial showed how to synchronize (pull) users and roles into Apache Syncope 1.2.x from an LDAP backend (Apache Directory). Interacting with an LDAP backend appears to be a common use-case for Apache Syncope users. For this reason, in this tutorial we will cover how to pull users and groups (previously roles) into Apache Syncope 2.0.0 from an LDAP backend via the Admin Console, as it is a little different from the previous 1.2.x releases.

1) Apache DS

The basic scenario is that we have a directory that stores user and group information that we would like to import into Apache Syncope 2.0.0. For the purposes of this tutorial, we will work with Apache DS. The first step is to download and launch Apache DS. I recommend installing Apache Directory Studio for an easy way to create and view the data stored in your directory.

Create two new groups (groupOfNames) in the default domain ("dc=example,dc=com") called "cn=employee,ou=groups,ou=system" and "cn=boss,ou=groups,ou=system". Create two new users (inetOrgPerson) "cn=alice,ou=users,ou=system" and "cn=bob,ou=users,ou=system". Now edit the groups you created such that both alice and bob are employees, but only alice is a boss. Specify "sn" (surname) and "userPassword" attributes for both users.

2) Pull data into Apache Syncope

The next task is to import (pull) the user data from Apache DS into Apache Syncope. Download and launch an Apache Syncope 2.0.x instance. Make sure that an LDAP Connector bundle is available (see here).

a) Define a 'surname' User attribute

The inetOrgPerson instances we created in Apache DS have a "sn" (surname) attribute. We will map this into an internal User attribute in Apache Syncope. The Schema configuration is quite different in the Admin Console compared to Syncope 1.2.x. Select "Configuration" and then "Types" in the left hand menu. Click on the "Schemas" tab and then the "+" button associated with "PLAIN". Add "surname" for the Key and click "save". Now go into the "AnyTypeClasses" tab and edit the "BaseUser" item. Select "surname" from the list of available plain Schema attributes. Now the users we create in Syncope can have a "surname" attribute.




b) Define a Connector

The next thing to do is to define a Connector to enable Syncope to talk to the Apache DS backend. Click on "Topology" in the left-hand menu, and on the ConnId instance on the map. Click "Add new connector" and create a new Connector of type "net.tirasa.connid.bundles.ldap". On the next tab select:
  • Host: localhost
  • TCP Port: 10389
  • Principal: uid=admin,ou=system
  • Password: <password>
  • Base Contexts: ou=users,ou=system and ou=groups,ou=system
  • LDAP Filter for retrieving accounts: cn=*
  • Group Object Classes: groupOfNames
  • Group member attribute: member
  • Click on "Maintain LDAP Group Membership".
  • Uid attribute: cn
  • Base Context to Synchronize: ou=users,ou=system and ou=groups,ou=system
  • Object Classes to Synchronize: inetOrgPerson and groupOfNames
  • Status Management Class: net.tirasa.connid.bundles.ldap.commons.AttributeStatusManagement
  • Tick "Retrieve passwords with search".
Click on the "heart" icon at the top of tab to check to see whether Syncope is able to connect to the backend resource. If you don't see a green "Successful Connection" message, then consult the logs. On the next tab select all of the available capabilities and click on "Finish".

c) Define a Resource

Next we need to define a Resource that uses the LDAP Connector.  The Resource essentially defines how we use the Connector to map information from the backend into Syncope Users and Groups. Click on the Connector that was created in the Topology map and select "Add new resource". Just select the defaults and finish creating the new resource. When the new resource is created, click on it and add some provisioning rules via "Edit provision rules".

Click the "+" button and select the "USER" type to create the mapping rules for users. Click "next" until you come to the mapping tab and create the following mappings:


Click "next" and enable "Use Object Link" and enter "'cn=' + username + ',ou=users,ou=system'". Click "Finish" and "save". Repeat the process above for the "GROUP" type to create a mapping rule for groups as follows:
Similar to creating the user mappings, we also need to enable "Use Object Link" and enter "'cn=' + name + ',ou=groups,ou=system'". Click "Finish" and "save".

d) Create a pull task

Having defined a Connector and a Resource to use that Connector, with mappings to map User/Group information to and from the backend, it's time to import the backend information into Syncope.  Click on the resource and select "Pull Tasks". Create a new Pull Task via the "+" button. Select "/" as the destination realm to create the users and groups in. Choose "FULL_RECONCILIATION" as the pull mode. Select "LDAPMembershipPullActions"  (this will preserve the fact that users are members of a group in Syncope) and "LDAPPasswordPullActions" from the list of available actions. Select "Allow create/update/delete". When the task is created,  click on the "execute" button (it looks like a cogged wheel). Now switch to the "Realms" tab in the left-hand menu and look at the users and groups that have been imported in the "/" realm from Apache DS.


Friday, August 12, 2016

OpenId Connect in Apache CXF Fediz 1.3.0

Previous blog posts have described support for OpenId Connect protocol bridging in the Apache CXF Fediz IdP. What this means is that the Apache CXF Fediz IdP can bridge between the WS-Federation protocol and OpenId Connect third party IdPs, when the user must be authenticated in a different security domain. However, the 1.3.0 release of Apache CXF Fediz also sees the introduction of a new OpenId Connect Idp which is independent of the existing (WS-Federation and SAML-SSO based) IdP, and based on Apache CXF. This post will introduce the new IdP via an example.

The example code is available on github:
  • cxf-fediz-oidc: This project shows how to use interceptors of Apache CXF to authenticate and authorize clients of a JAX-RS service using OpenId Connect.
1) The secured service

The first module available in the example contains a trivial JAX-RS Service based on Apache CXF which "doubles" a number that is passed as a path parameter via HTTP GET. The service defines via a @RolesAllowed annotation that only users allowed in roles "User", "Admin" or "Manager" can access the service.

The service is configured via spring. The endpoint configuration references the service bean above, as well as the CXF SecureAnnotationsInterceptor which enforces the @RolesAllowed annotation on the service bean. In addition, the service is configured with the CXF OidcRpAuthenticationFilter, which ensures that only users authenticated via OpenId Connect can access the service. The filter is configured with a URL to redirect the user to. It also explicitly requires a role claim to enforce authorization.

The OidcRpAuthenticationFilter redirects the browser to a separate authentication endpoint, defined in the same spring file for convenience. This endpoint has a filter called OidcClientCodeRequestFilter, which initiates the OpenId Connect authorization code flow to a remote OpenId Connect IdP (in this case, the new Fediz IdP). It is also responsible for getting an IdToken after successfully getting an authorization code from the IdP.

2) The Fediz OpenId Connect IdP

The second module contains an integration test which deploys a number of wars into an Apache Tomcat container:
  • The "double-it" service as described above
  • The Apache CXF Fediz IdP which authenticates users via WS-Federation
  • The Apache CXF Fediz STS which performs the underlying authentication of users
  • The Apache CXF Fediz OpenId Connect IdP
The way the Apache CXF Fediz OpenId Connect IdP works (at least for 1.3.x) is that user authentication is actually delegated to the WS-Federation based IdP via a Fediz plugin. So when the user is redirected to the Fediz IdP, (s)he gets redirected to the WS-Federation based IdP for authentication, and then gets redirected back to the OpenId Connect IdP with a WS-Federation Response. The OpenId Connect IdP parses this (SAML) Response and converts it into a JWT IdToken. Future releases will enable authentication directly at the OpenId Connect service.

After deploying all of the services, the test code makes a series of REST calls to create a client in the OpenId Connect IdP so that we can run the test without having to manually enter information in the client UI of the Fediz IdP. To run the test, simply remove the @org.junit.Ignore assertion on the "testInBrowser" method. The test code will create the clients in Fediz and then print out a URL in the console before sleeping. Copy the URL and paste it into a browser. Authenticate using the credentials "alice/ecila".

Thursday, August 11, 2016

Introducing Apache Syncope 2.0.0

Apache Syncope is a powerful and flexible open-source Identity Management system that has been developed at the Apache Software Foundation for several years now. The Apache Syncope team has been busy developing a ton of new features for the forthcoming new major release (2.0.0), which will really help to cement Apache Syncope's position as a first class Identity Management solution. If you wish to experiment with these new features, a 2.0.0-M4 release is available. In this post we will briefly cover some of the new features and changes. For a more comprehensive overview please refer to the reference guide.

1) Domains

Perhaps the first new concept you will be introduced to in Syncope 2.0.0 after starting the (Admin) console is that of a domain. When logging in, as well as specifying a username, password, and language, you can also specify a configured domain. Domains are a new concept in Syncope 2.0.0 that facilitate multi-tenancy. Domains allow the physical separation of all data stored in Syncope (by storing the data in different database instances). Therefore, Syncope can facilitate users, groups etc. that are in different domains in a single Syncope instance.

2) New Console layout

After logging in, it becomes quickly apparent that the Syncope Console is quite different compared to the 1.2.x console. It has been completely rewritten and looks great. Connectors and Resources are now managed under "Topology" in the menu on the left-hand side. Users and Groups (formerly Roles) are managed under "Realms" in the menu. The Schema types are configured under "Configuration". A video overview of the new Console can be seen here.


3) AnyType Objects

With Syncope 1.2.x, it was possible to define plain/derived/virtual Schema Types for users, roles and memberships, but no other entities. In Syncope 2.0.0, the Schema Types are decoupled from the entity that uses them. Instead, a new concept called an AnyType class is available which is a collection of schema types. In turn, an AnyType object can be created which consists of any number of AnyType classes. AnyType objects represent the type of things that Apache Syncope can model. Besides the predefined Users and Groups, it can also represent physical things such as printers, workstations, etc. With this new concept, Apache Syncope 2.0.0 can model many different types of identities.

4) Realms

Another new concept in Apache Syncope 2.0.0 is that of a realm. A realm encapsulates a number of Users, Groups and Any Objects. It is possible to specify account and password policies per-realm (see here for a blog entry on custom policies in Syncope 2.0.0). Each realm has a parent realm (apart from the pre-defined root realm identified as "/"). The realm tree is hierarchical, meaning that Users, Groups etc. defined in a sub-realm, are also defined on a parent realm. Combined with Roles (see below), realms facilitate some powerful access management scenarios.

5) Groups/Roles

In Syncope 2.0.0, what were referred to as "roles" in Syncope 1.2.x are now called "groups". In addition, "roles" in Syncope 2.0.0 are a new concept which associate a number of entitlements with a number of realms. Users assigned to a role can exercise the defined entitlements on any of the objects in the given realms (any any sub-realms).

Syncope 2.0.0 also has the powerful concept of dynamic membership, which means that users can be assigned to groups or roles via a conditional expression (e.g. if an attribute matches a given value).

6) Apache Camel Provisioning

An exciting new feature of Apache Syncope 2.0.0 is the new Apache Camel provisioning engine, which is available under "Extensions/Camel Routes" in the Console. Apache Syncope comes pre-loaded with some Camel routes that are executed as part of the provisioning implementation for Users, Groups and Any Objects. The real power of this new engine lies is the ability to modify the routes to perform some custom provisioning rules. For example, on creating a new user, you may wish to send an email to an administrator. Or if a user is reactivated, you may wish to reactivate the user's home page on a web server. All these things and more are possible using the myriad of components that are available to be used in Apache Camel routes. I'll explore this feature some more in future blog posts.

7) End-User UI

As well as the Admin console (available via /syncope-console), Apache Syncope 2.0.0 also ships with an Enduser console (available via /syncope-enduser). This allows a user to edit only details pertaining to his/her-self, such as editing the user attributes, changing the password, etc. See the following blog entry for more information on the new End-User UI.


8) Command Line Interface (CLI) client

Another new feature of Apache Syncope 2.0.0 is that of the CLI client. It is available as a separate download. Once downloaded, extract it and run (on linux): ./syncopeadm.sh install --setup. Answer the questions about where Syncope is deployed and the credentials required to access it. After installation, you can run queries such as: ./syncopeadm.sh user --list.

9) Apache CXF-based testcases

I updated the testcases that I wrote before to use Apache Syncope 2.0.0 to authenticate and authorize web services calls using Apache CXF. The new test-cases are available here

Monday, August 8, 2016

Installing the Apache Ranger Key Management Server (KMS)

The previous couple of blog entries have looked at how to install the Apache Ranger Admin Service as well as the Usersync Service. In this post we will look at how to install the Apache Ranger Key Management Server (KMS). KMS is a component of Apache Hadoop to manage cryptographic keys. Apache Ranger ships with its own KMS implementation, which allows you to store the (encrypted) keys in a database. The Apache Ranger KMS is also secured via policies defined in the Apache Ranger Admin Service.

1) Build the source code

The first step is to download the source code, as well as the signature file and associated message digests (all available on the download page). Verify that the signature is valid and that the message digests match. Now extract and build the source, and copy the resulting KMS archive to a location where you wish to install it:
  • tar zxvf apache-ranger-incubating-0.6.0.tar.gz
  • cd apache-ranger-incubating-0.6.0
  • mvn clean package assembly:assembly 
  • tar zxvf target/ranger-0.6.0-kms.tar.gz
  • mv ranger-0.6.0-kms ${rangerkms.home}
2) Install the Apache Ranger KMS Service

As the Apache Ranger KMS Service stores the cryptographic keys in a database, we will need to setup and configure a database. We will also configure the KMS Service to store audit logs in the database. Follow the steps given in section 2 of the tutorial on the Apache Ranger Admin Service to set up MySQL. We will also need to create a new user 'rangerkms':
  • CREATE USER 'rangerkms'@'localhost' IDENTIFIED BY 'password';
  • FLUSH PRIVILEGES; 
You will need to install the Apache Ranger KMS Service using "sudo". If the root user does not have a JAVA_HOME property defined, then edit ${rangerkms.home}/setup.sh + add in, e.g.:
  • export JAVA_HOME=/opt/jdk1.8.0_91
Next edit ${rangerkms.home}/install.properties and make the following changes:
  • Change SQL_CONNECTOR_JAR to point to the MySQL JDBC driver jar (see previous tutorial).
  • Set (db_root_user/db_root_password) to (admin/password)
  • Set (db_user/db_password) to (rangerkms/password)
  • Change KMS_MASTER_KEY_PASSWD to a secure password value.
  • Set POLICY_MGR_URL=http://localhost:6080
  • Set XAAUDIT.DB.IS_ENABLED=true
  • Set XAAUDIT.DB.FLAVOUR=MYSQL 
  • Set XAAUDIT.DB.HOSTNAME=localhost 
  • Set XAAUDIT.DB.DATABASE_NAME=ranger_audit 
  • Set XAAUDIT.DB.USER_NAME=rangerlogger
  • Set XAAUDIT.DB.PASSWORD=password
Now you can run the setup script via "sudo ./setup.sh".

3) Starting the Apache Ranger KMS service

After a successful installation, first start the Apache Ranger admin service with "sudo ranger-admin start". Then start the Apache Ranger KMS Service via "sudo ranger-kms start". Now open a browser and go to "http://localhost:6080/". Log on with "keyadmin/keyadmin". Note that these are different credentials to those used to log onto the Apache Ranger Admin UI in the previous tutorial. Click on the "+" button on the "KMS" tab to create a new KMS Service. Specify the following values:
  • Service Name: kmsdev
  • KMS URL: kms://http@localhost:9292/kms
  • Username: keyadmin
  • Password: keyadmin
Click on "Test Connection" to make sure that the KMS Service is up and running. If it is showing a connection failure, log out and log into the Admin UI using credentials "admin/admin". Go to the "Audit" section and click on "Plugins". You should see a successful message indicating that the KMS plugin can successfully download policies from the Admin Service:


After logging back in to the UI as "keyadmin" you can start to create keys. Click on the "Encryption/Key Manager" tab. Select the "kmsdev" service in the dropdown list and click on "Add New Key". You can create, delete and rollover keys in the UI: