Apache Syncope tutorial - part II

In the previous tutorial on Apache Syncope, we described how to create a standalone application deployed in Apache Tomcat, and using MySQL as the persistent storage. In this tutorial we will show how to set up a basic schema for Syncope that describes the users that will be created in Syncope. Then we will show how to import users from a Database backend, which will be Apache Derby for the purposes of this tutorial.

1) Creating a Schema attribute

The first thing we will do is add a simple attribute for all users that will exist in Syncope. Launch Apache Syncope as per tutorial I. Click on the "Schema" tab, and then "Create New Attribute" in the Users/Normal subsection. Create a new attribute called "surname" which is of type "String" and "mandatory". So users in our Syncope application must have a "surname". Obviously, the schema allows you to do far more complex and interesting things, but this will suffice for the purposes of this tutorial.

2) Apache Derby

The basic scenario is that we have a SQL database that stores user information that we would like to import into Apache Syncope, to integrate into a BPEL workflow, expose via a RESTful interface, associate with roles, etc. For the purposes of this tutorial, we will work with Apache Derby. The first step is to download and launch Apache Derby, and then to populate it with a table with some user data. Hat tip to my Apache CXF colleague Glen Mazza for an excellent tutorial on setting up Apache Derby.

a) Launch Apache Derby

Download Apache Derby and extract it into a new directory ($DERBY_HOME). Create a directory to use to store Apache Derby databases ($DERBY_DATA). In $DERBY_DATA, create a file called 'derby.properties' with the content:

derby.connection.requireAuthentication=true
derby.user.admin=security

In other words, authentication is required, and a valid user is "admin" with password "security". Now launch Apache Derby in network mode via:

java -Dderby.system.home=$DERBY_DATA/ -jar $DERBY_HOME/lib/derbyrun.jar server start

b) Create user data

Create a new file called 'create-users.sql' with the following content:

SET SCHEMA APP;
DROP TABLE USERS;

CREATE TABLE USERS (
  NAME   VARCHAR(20) NOT NULL PRIMARY KEY,
  PASSWORD  VARCHAR(20) NOT NULL,
  STATUS  VARCHAR(20) NOT NULL,
  SURNAME  VARCHAR(20) NOT NULL
);

INSERT INTO USERS VALUES('dave', 'password', 'true', 'yellow');
INSERT INTO USERS VALUES('harry', 'password', 'true', 'blue');

Launch Apache Derby via $DERBY_HOME/bin/ij. Then connect to the server via:

connect 'jdbc:derby://localhost:1527/SYNCOPE;create=true;user=admin;password=security;';

Populate user data via: run 'create-users.sql';

You can now see the user data via: select * from users;

3) Synchronize user data into Apache Syncope

The next task is to import (synchronize) the user data from Apache Derby into Apache Syncope. See the Syncope wiki for more information on this topic.

a) Define a Connector

The first thing to do is to define a Connector. In tutorial I we configured two Connector bundles to use for Syncope, one for a DB backend, and one for an LDAP backend. In this section we select the DB Connector, and configure it to connect to the Derby instance we have set up above. Go to "Resources/Connectors", and create a new Connector of name "org.connid.bundles.db.table". In the "Configuration" tab select:

• User: admin
• User Password: security
• Table: app.users
• Key Column: name
• Password Column: password
• Status Column: status
• JDBC Driver: org.apache.derby.jdbc.ClientDriver
• JDBC Connection URL: jdbc:derby://localhost:1527/SYNCOPE
• Enable 'Retrieve Password'

Note that the Derby JDBC driver must be available on the classpath as per tutorial II. In the "Capabilities" tab select the following properties:

• ONE_PHASE_CREATE
• ONE_PHASE_UPDATE
• ONE_PHASE_DELETE
• SEARCH
• SYNC

Click on the "helmet" icon in the "Configuration" tab to check to see whether Syncope is able to connect to the backend resource. If you don't see a green "Successful Connection" message, then consult the logs.

b) Define a Resource

Next we need to define a Resource that uses the DB Connector.  The Resource essentially defines how we use the Connector to map information from the backend into Syncope Users and Roles. Go into the "Resources" tab and select "Create New Resource". In the "Resource Details" tab select:

• Name: (Select a name)
• Connector: (Connector display name you have configured previously)
• Enforce mandatory condition
• Propagation Primary
• Propagation Mode (see here): ONE_PHASE
• Select "DefaultPropagationActions" for the "Actions class"

The next step is to create User mappings. Click on the "User mapping" tab, and create the following mappings:

c) Create a synchronization task

Having defined a Connector and a Resource to use that Connector, with mappings to map User information to and from the backend, it's time to import the backend information into Syncope.  Go to "Tasks" and select the "Synchronization Tasks" tab. Click on "Create New Task". On the "Profile" tab enter:

• Name: (Select a name)
• Resource Name: (The Resource name you have created above)
• Actions class: DefaultSyncActions
• Create new identities
• Updated matched identities
• Delete matching identities
• Status
• Full reconciliation

Save the task and then click on the "Execute" button. Now switch to the Users tab. You should see the users stored in the backend. Click on one of the users, and see that the "surname" attribute is populated with the value mapped from the column stored in the backend:

Apache Syncope tutorial - part I

Apache Syncope is a new open source Identity Management project at Apache. This is the first of a planned four-part set of tutorials on how to get Apache Syncope up and running, how to integrate it with various backends, and how to interact with its REST API.

In this tutorial we will explain how to create a new Apache Syncope project and how to deploy it to a container. We will also cover how to set up internal storage with a database, and how to install Connector bundles to communicate with backend resources. Please note that if you wish to download and play around with Apache Syncope, without going through the steps detailed in this tutorial to set it up for a standalone deployment, it is possible to download the 1.1.3 distribution and use an embedded Tomcat instance. In that case you can completely skip this tutorial and go straight to the next one.

1) Set up a database for Internal Storage

The first step in setting up a standalone deployment of Apache Syncope is to decide what database to use for Internal Storage. Apache Syncope persists internal storage to a database via Apache OpenJPA. In this article we will set up MySQL, but see here for more information on using PostgreSQL, Oracle, etc. Install MySQL in $SQL_HOME and create a new user for Apache Syncope. We will create a new user "syncope_user" with password "syncope_pass". Start MySQL and create a new Syncope database:

• Start: sudo $SQL_HOME/bin/mysqld_safe --user=mysql
• Log on: $SQL_HOME/bin/mysql -u syncope_user -p
• Create a Syncope database: create database syncope; 

2) Set up a container to host Apache Syncope

The next step is to figure out in what container to deploy Syncope to. In this demo we will use Apache Tomcat, but see here for more information about installing Syncope in other containers. Install Apache Tomcat to $CATALINA_HOME. Now we will add a datasource for internal storage in Tomcat's 'conf/context.xml'. When Syncope does not find a datasource called 'jdbc/syncopeDataSource', it will connect to internal storage by instantiating a new connection per request, which carries a performance penalty. Add the following to 'conf/context.xml':

<Resource name="jdbc/syncopeDataSource" auth="Container"
    type="javax.sql.DataSource"
    factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
    testWhileIdle="true" testOnBorrow="true" testOnReturn="true"
    validationQuery="SELECT 1" validationInterval="30000"
    maxActive="50" minIdle="2" maxWait="10000" initialSize="2"
    removeAbandonedTimeout="20000" removeAbandoned="true"
    logAbandoned="true" suspectTimeout="20000"
    timeBetweenEvictionRunsMillis="5000" minEvictableIdleTimeMillis="5000"
    jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
    org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
    username="syncope_user" password="syncope_pass"
    driverClassName="com.mysql.jdbc.Driver"
    url="jdbc:mysql://localhost:3306/syncope?characterEncoding=UTF-8"/>

Uncomment the "<Manager pathname="" />" configuration in context.xml as well. Next, download the JDBC driver jar for MySQL and put it in Tomcat's 'lib' directory. As we will be configuring a connector for a Derby resource in a future tutorial, also download the JDBC driver jar for Apache Derby and put it in Tomcat's 'lib' directory as well.

3) Create a new Apache Syncope project

To create a new Apache Syncope project, we must start with a Maven Archetype. For the purposes of this tutorial we will use Apache Syncope 1.1.3. Create a new directory to hold all of the project artifacts ($SYNCOPE_HOME). Create a new project in this directory as follows:

mvn archetype:generate
    -DarchetypeGroupId=org.apache.syncope
    -DarchetypeArtifactId=syncope-archetype
    -DarchetypeRepository=http://repos1.maven.apache.org/maven2/
    -DarchetypeVersion=1.1.3

Enter values as requested for the 'groupId', 'artifactId', 'version', 'package' and 'secretKey'. A new Syncope project will be created in a directory corresponding to the value entered for 'artifactId'. Build the project by going into the newly created directory and typing "mvn clean package". To launch Syncope in "embedded" mode, go into the "console" subdirectory and type "mvn -Pembedded". This provides a quick way to start up Syncope with some test data and connectors pre-installed, along with a H2 instance for internal storage. However, instead of doing this we will set up a proper standalone deployment.

4) Install ConnId bundles

Apache Syncope uses bundles supplied by the ConnId project to communicate with backend resources. In part III of this series of tutorials we will cover importing users from a database backend, and part IV will cover using a directory backend. Therefore, we need two ConnId bundles to handle these scenarios. Create two new directories in $SYNCOPE_HOME, one called "bundles" to store the ConnId bundles, and another called "logs" to store logging information. Go to the ConnId download page, and download the relevant jars to the bundles directory you have created. It should have the following files:

• org.connid.bundles.ldap-1.3.6.jar
• org.connid.bundles.db.table-2.1.5.jar.

5) Configure the Apache Syncope project

After creating the Apache Syncope project, we need to configure it to use the ConnId bundles we have downloaded, the logging directory we have created, and the MySQL instance for internal storage.

Edit 'core/src/main/resources/persistence.properties' in your Syncope project, and replace the existing configuration with the following:

jpa.driverClassName=com.mysql.jdbc.Driver
jpa.url=jdbc:mysql://localhost:3306/syncope?characterEncoding=UTF-8
jpa.username=syncope_user
jpa.password=syncope_pass
jpa.dialect=org.apache.openjpa.jdbc.sql.MySQLDictionary
quartz.jobstore=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
quartz.sql=tables_mysql.sql
logback.sql=mysql.sql

Edit 'core/src/main/webapp/WEB-INF/web.xml' and uncomment the "resource-ref" section (this is required as we are using a DataSource in Tomcat). If your Tomcat instance is starting on a port other than 8080, edit 'console/src/main/resources/configuration.properties' and change the port number.

6) Deploy Apache Syncope to Tomcat

Package everything up by executing the following command from the Syncope project:

mvn clean package -Dbundles.directory=${bundles.dir} -Dlog.directory=${log.dir}

Deploy 'core/target/syncope.war' and 'console/target/syncope-console.war' to the Tomcat 7 container. Start Tomcat and point a browser at "localhost:8080/syncope-console", logging in as "admin/password". You should see the following:

At this point you should have successfully deployed Apache Syncope in the Apache Tomcat container, using MySQL as the internal storage. If this is not the case then consult the Tomcat logs, as well as the Syncope logs in the directory you have configured. In the next couple of tutorials we will look at importing data into our Syncope application from various backend resources.