Apache Syncope tutorial - part II

In the previous tutorial on Apache Syncope, we described how to create a standalone application deployed in Apache Tomcat, and using MySQL as the persistent storage. In this tutorial we will show how to set up a basic schema for Syncope that describes the users that will be created in Syncope. Then we will show how to import users from a Database backend, which will be Apache Derby for the purposes of this tutorial.

1) Creating a Schema attribute

The first thing we will do is add a simple attribute for all users that will exist in Syncope. Launch Apache Syncope as per tutorial I. Click on the "Schema" tab, and then "Create New Attribute" in the Users/Normal subsection. Create a new attribute called "surname" which is of type "String" and "mandatory". So users in our Syncope application must have a "surname". Obviously, the schema allows you to do far more complex and interesting things, but this will suffice for the purposes of this tutorial.

2) Apache Derby

The basic scenario is that we have a SQL database that stores user information that we would like to import into Apache Syncope, to integrate into a BPEL workflow, expose via a RESTful interface, associate with roles, etc. For the purposes of this tutorial, we will work with Apache Derby. The first step is to download and launch Apache Derby, and then to populate it with a table with some user data. Hat tip to my Apache CXF colleague Glen Mazza for an excellent tutorial on setting up Apache Derby.

a) Launch Apache Derby

Download Apache Derby and extract it into a new directory ($DERBY_HOME). Create a directory to use to store Apache Derby databases ($DERBY_DATA). In $DERBY_DATA, create a file called 'derby.properties' with the content:

derby.connection.requireAuthentication=true
derby.user.admin=security

In other words, authentication is required, and a valid user is "admin" with password "security". Now launch Apache Derby in network mode via:

java -Dderby.system.home=$DERBY_DATA/ -jar $DERBY_HOME/lib/derbyrun.jar server start

b) Create user data

Create a new file called 'create-users.sql' with the following content:

SET SCHEMA APP;
DROP TABLE USERS;

CREATE TABLE USERS (
  NAME   VARCHAR(20) NOT NULL PRIMARY KEY,
  PASSWORD  VARCHAR(20) NOT NULL,
  STATUS  VARCHAR(20) NOT NULL,
  SURNAME  VARCHAR(20) NOT NULL
);

INSERT INTO USERS VALUES('dave', 'password', 'true', 'yellow');
INSERT INTO USERS VALUES('harry', 'password', 'true', 'blue');

Launch Apache Derby via $DERBY_HOME/bin/ij. Then connect to the server via:

connect 'jdbc:derby://localhost:1527/SYNCOPE;create=true;user=admin;password=security;';

Populate user data via: run 'create-users.sql';

You can now see the user data via: select * from users;

3) Synchronize user data into Apache Syncope

The next task is to import (synchronize) the user data from Apache Derby into Apache Syncope. See the Syncope wiki for more information on this topic.

a) Define a Connector

The first thing to do is to define a Connector. In tutorial I we configured two Connector bundles to use for Syncope, one for a DB backend, and one for an LDAP backend. In this section we select the DB Connector, and configure it to connect to the Derby instance we have set up above. Go to "Resources/Connectors", and create a new Connector of name "org.connid.bundles.db.table". In the "Configuration" tab select:

• User: admin
• User Password: security
• Table: app.users
• Key Column: name
• Password Column: password
• Status Column: status
• JDBC Driver: org.apache.derby.jdbc.ClientDriver
• JDBC Connection URL: jdbc:derby://localhost:1527/SYNCOPE
• Enable 'Retrieve Password'

Note that the Derby JDBC driver must be available on the classpath as per tutorial II. In the "Capabilities" tab select the following properties:

• ONE_PHASE_CREATE
• ONE_PHASE_UPDATE
• ONE_PHASE_DELETE
• SEARCH
• SYNC

Click on the "helmet" icon in the "Configuration" tab to check to see whether Syncope is able to connect to the backend resource. If you don't see a green "Successful Connection" message, then consult the logs.

b) Define a Resource

Next we need to define a Resource that uses the DB Connector.  The Resource essentially defines how we use the Connector to map information from the backend into Syncope Users and Roles. Go into the "Resources" tab and select "Create New Resource". In the "Resource Details" tab select:

• Name: (Select a name)
• Connector: (Connector display name you have configured previously)
• Enforce mandatory condition
• Propagation Primary
• Propagation Mode (see here): ONE_PHASE
• Select "DefaultPropagationActions" for the "Actions class"

The next step is to create User mappings. Click on the "User mapping" tab, and create the following mappings:

c) Create a synchronization task

Having defined a Connector and a Resource to use that Connector, with mappings to map User information to and from the backend, it's time to import the backend information into Syncope.  Go to "Tasks" and select the "Synchronization Tasks" tab. Click on "Create New Task". On the "Profile" tab enter:

• Name: (Select a name)
• Resource Name: (The Resource name you have created above)
• Actions class: DefaultSyncActions
• Create new identities
• Updated matched identities
• Delete matching identities
• Status
• Full reconciliation

Save the task and then click on the "Execute" button. Now switch to the Users tab. You should see the users stored in the backend. Click on one of the users, and see that the "surname" attribute is populated with the value mapped from the column stored in the backend:

Apache Syncope tutorial - part I

Apache Syncope is a new open source Identity Management project at Apache. This is the first of a planned four-part set of tutorials on how to get Apache Syncope up and running, how to integrate it with various backends, and how to interact with its REST API.

In this tutorial we will explain how to create a new Apache Syncope project and how to deploy it to a container. We will also cover how to set up internal storage with a database, and how to install Connector bundles to communicate with backend resources. Please note that if you wish to download and play around with Apache Syncope, without going through the steps detailed in this tutorial to set it up for a standalone deployment, it is possible to download the 1.1.3 distribution and use an embedded Tomcat instance. In that case you can completely skip this tutorial and go straight to the next one.

1) Set up a database for Internal Storage

The first step in setting up a standalone deployment of Apache Syncope is to decide what database to use for Internal Storage. Apache Syncope persists internal storage to a database via Apache OpenJPA. In this article we will set up MySQL, but see here for more information on using PostgreSQL, Oracle, etc. Install MySQL in $SQL_HOME and create a new user for Apache Syncope. We will create a new user "syncope_user" with password "syncope_pass". Start MySQL and create a new Syncope database:

• Start: sudo $SQL_HOME/bin/mysqld_safe --user=mysql
• Log on: $SQL_HOME/bin/mysql -u syncope_user -p
• Create a Syncope database: create database syncope; 

2) Set up a container to host Apache Syncope

The next step is to figure out in what container to deploy Syncope to. In this demo we will use Apache Tomcat, but see here for more information about installing Syncope in other containers. Install Apache Tomcat to $CATALINA_HOME. Now we will add a datasource for internal storage in Tomcat's 'conf/context.xml'. When Syncope does not find a datasource called 'jdbc/syncopeDataSource', it will connect to internal storage by instantiating a new connection per request, which carries a performance penalty. Add the following to 'conf/context.xml':

<Resource name="jdbc/syncopeDataSource" auth="Container"
    type="javax.sql.DataSource"
    factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
    testWhileIdle="true" testOnBorrow="true" testOnReturn="true"
    validationQuery="SELECT 1" validationInterval="30000"
    maxActive="50" minIdle="2" maxWait="10000" initialSize="2"
    removeAbandonedTimeout="20000" removeAbandoned="true"
    logAbandoned="true" suspectTimeout="20000"
    timeBetweenEvictionRunsMillis="5000" minEvictableIdleTimeMillis="5000"
    jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
    org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer"
    username="syncope_user" password="syncope_pass"
    driverClassName="com.mysql.jdbc.Driver"
    url="jdbc:mysql://localhost:3306/syncope?characterEncoding=UTF-8"/>

Uncomment the "<Manager pathname="" />" configuration in context.xml as well. Next, download the JDBC driver jar for MySQL and put it in Tomcat's 'lib' directory. As we will be configuring a connector for a Derby resource in a future tutorial, also download the JDBC driver jar for Apache Derby and put it in Tomcat's 'lib' directory as well.

3) Create a new Apache Syncope project

To create a new Apache Syncope project, we must start with a Maven Archetype. For the purposes of this tutorial we will use Apache Syncope 1.1.3. Create a new directory to hold all of the project artifacts ($SYNCOPE_HOME). Create a new project in this directory as follows:

mvn archetype:generate
    -DarchetypeGroupId=org.apache.syncope
    -DarchetypeArtifactId=syncope-archetype
    -DarchetypeRepository=http://repos1.maven.apache.org/maven2/
    -DarchetypeVersion=1.1.3

Enter values as requested for the 'groupId', 'artifactId', 'version', 'package' and 'secretKey'. A new Syncope project will be created in a directory corresponding to the value entered for 'artifactId'. Build the project by going into the newly created directory and typing "mvn clean package". To launch Syncope in "embedded" mode, go into the "console" subdirectory and type "mvn -Pembedded". This provides a quick way to start up Syncope with some test data and connectors pre-installed, along with a H2 instance for internal storage. However, instead of doing this we will set up a proper standalone deployment.

4) Install ConnId bundles

Apache Syncope uses bundles supplied by the ConnId project to communicate with backend resources. In part III of this series of tutorials we will cover importing users from a database backend, and part IV will cover using a directory backend. Therefore, we need two ConnId bundles to handle these scenarios. Create two new directories in $SYNCOPE_HOME, one called "bundles" to store the ConnId bundles, and another called "logs" to store logging information. Go to the ConnId download page, and download the relevant jars to the bundles directory you have created. It should have the following files:

• org.connid.bundles.ldap-1.3.6.jar
• org.connid.bundles.db.table-2.1.5.jar.

5) Configure the Apache Syncope project

After creating the Apache Syncope project, we need to configure it to use the ConnId bundles we have downloaded, the logging directory we have created, and the MySQL instance for internal storage.

Edit 'core/src/main/resources/persistence.properties' in your Syncope project, and replace the existing configuration with the following:

jpa.driverClassName=com.mysql.jdbc.Driver
jpa.url=jdbc:mysql://localhost:3306/syncope?characterEncoding=UTF-8
jpa.username=syncope_user
jpa.password=syncope_pass
jpa.dialect=org.apache.openjpa.jdbc.sql.MySQLDictionary
quartz.jobstore=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
quartz.sql=tables_mysql.sql
logback.sql=mysql.sql

Edit 'core/src/main/webapp/WEB-INF/web.xml' and uncomment the "resource-ref" section (this is required as we are using a DataSource in Tomcat). If your Tomcat instance is starting on a port other than 8080, edit 'console/src/main/resources/configuration.properties' and change the port number.

6) Deploy Apache Syncope to Tomcat

Package everything up by executing the following command from the Syncope project:

mvn clean package -Dbundles.directory=${bundles.dir} -Dlog.directory=${log.dir}

Deploy 'core/target/syncope.war' and 'console/target/syncope-console.war' to the Tomcat 7 container. Start Tomcat and point a browser at "localhost:8080/syncope-console", logging in as "admin/password". You should see the following:

At this point you should have successfully deployed Apache Syncope in the Apache Tomcat container, using MySQL as the internal storage. If this is not the case then consult the Tomcat logs, as well as the Syncope logs in the directory you have configured. In the next couple of tutorials we will look at importing data into our Syncope application from various backend resources.

Denial of Service attacks on Apache CXF

A significant new paper has emerged called "A new Approach towards DoS Penetration Testing on Web Services" by Andreas Falkenberg of SEC Consult Deutschland GmbH, and Christian Mainka, Juraj Somorovsky and Joerg Schwenk of Ruhr-University Bochum. In this paper, the authors developed a suite of automated tests for various Denial of Service (DoS) attacks on Web Services, and ran them against different web service stacks. In this post I will describe the attacks that were successful on Apache CXF and how they were fixed.

The authors found that Apache CXF (prior to 2.7.4/ 2.6.7/ 2.5.10) was vulnerable (see CVE-2013-2160) to the following attacks:

• Coercive Parsing Attack: The attacker sends a deeply nested XML document to the service.
• Attribute Count Attack: The attacker sends a message with a (very) high attribute count.
• Element Count Attack: The attacker sends a message with a (very) high number of non-nested elements.
• DJBX31A Hash Collision: A specific hash collision attack.

The effects of these attacks can vary from causing high CPU usage, to causing the JVM to run out of memory. Clearly the latter is a critical vulnerability. Prior to CXF 2.7.4 / 2.6.7 / 2.5.10, a CXF service was vulnerable to these attacks "out of the box". However, it was possible to avoid some of the attacks using CXF's DepthRestrictingInterceptor. If this was added to the InInterceptor chain, then it was possible to control the stack depth + the number of elements in the request. However it was not possible to control the number of attributes with this interceptor, and it also came with a performance cost.

CXF uses Woodstox by default as the StAX XML Processor. It was decided that the best place to fix the vulnerabilities was at this level, both to offer protection to other stacks that use Woodstox, and also to remove the performance penalties associated with the DepthRestrictingInterceptor. From Woodstox 4.2.0, new functionality has been added to prevent Denial of Service attacks by restricting the size of XML. It uses the following defaults:

• Maximum Attributes per Element: 1000.
• Maximum Children per Element: (no effective bound).
• Maximum Stack Depth: 1000.
• Maximum Element Count: (no effective bound).
• Maximum Number of Characters: (no effective bound).

These bounds are quite loose to preserve backwards compatibility. CXF 2.7.4 / 2.6.7 / 2.5.10 pick up Woodstox 4.2.0. Other parser implementations are defined as "insecure parsers" and are rejected by default from CXF 2.7.5 unless a System property is set. CXF restricts the Woodstox default values further:

• Maximum Attributes per Element: 500.
• Maximum Children per Element: 50000.
• Maximum Stack Depth: 100.

CXF 2.7.4 / 2.6.7 / 2.5.10 are not vulnerable to any of the DoS attacks listed above due to these restrictions.

Apache XML Security for Java 1.4.8 and 1.5.5 released

Two new versions of the Apache XML Security for Java project have been released and are available for download. These releases contain a fix for a critical security advisory CVE-2013-2172, which involves an XML Signature spoofing attack. Thanks to James Forshaw for reporting the vulnerability to the Apache Santuario project.

Apache CXF 2.7.5 released

Apache CXF 2.7.5 has been released. The list of issues fixed is available here. The following security fixes of note have been made in this release:

• The OpenSAML dependency has been upgraded from 2.5.1 to 2.5.3.
• A change was made to the logic the STS uses to encrypt tokens that it issues. Previously it threw an exception if a key could not be found (at either service or at a more generic level) to use to encrypt the token. Now it only encrypts the token if a matching key can be found. This allows the ability to only encrypt tokens to specific "AppliesTo" endpoint addresses.
• LDAP groups are now (better) supported as claims in the STS. See the following blog entry for more detail.
• The CryptoCoverageChecker interceptor has been enhanced so that you can disable coverage checking for SOAP Faults. This is useful for testing/debugging if you want to figure out the root cause of a remote exception.

Apache CXF 2.7.4 released

Apache CXF 2.7.4 (and 2.6.7 + 2.5.10) have been released. Users are strongly encouraged to upgrade to the latest versions, due to a critical security issue which must remain undisclosed for the moment. These latest releases pick up Apache Santuario 1.5.4 and Apache WSS4J 1.6.10. In addition to the fixes in these projects, CXF 2.7.4 contains a number of security fixes of interest.

1) WS-SecurityPolicy fixes

A large number of negative tests for WS-Security(Policy) were added to CXF to try to smoke out some remaining issues surrounding validating a request against a defined security policy. The following issues were fixed as a result:

• Layout policies "LaxTimestampFirst" and "LaxTimestampLast" were not validated correctly.
• X509Token "PKI" policies were not validated correctly.
• The "OnlySignEntireHeadersAndBody" policy was not validated correctly.
• The "ProtectTokens" policy was not validated correctly in conjunction with EndorsingSupportingTokens.
• SAML Token versions weren't validated against policy versions in certain circumstances.

2) Populating the SecurityContext from a JAAS Subject from WSS4J

An additional improvement to the WS-Security layer in CXF is that the SecurityContext is now populated from a JAAS Subject from WSS4J, if one is available. For example, if you are using a custom UsernameTokenValidator with WSS4J to validate a received UsernameToken via JAAS, and are returning the Subject (as per WSS4J's JAASUsernameTokenValidator), then CXF will attempt to extract roles from the Subject and populate the SecurityContext accordingly. The advantage of this is that a user can check the standard SecurityContext methods (e.g. "isUserInRole") to perform authorization.

This is controlled by two JAX-WS properties (see the documentation for more information):

• ws-security.role.classifier - The Subject Role Classifier to use.  If this value is not specified, then it tries to get roles using the DefaultSecurityContext in cxf-rt-core. Otherwise it uses this value in combination with the "ws-security.role.classifier.type" property to get the roles from the Subject.
• ws-security.role.classifier.type - The Subject Role Classifier Type to use. Currently accepted values are "prefix" or "classname". Must be used in conjunction with the "ws-security.role.classifier". The default value is "prefix".

3) STS fixes

The SecurityTokenService (STS) fixes contained in this release are:

• The STS Client was not always sending an "AppliesTo" address in a request to the STS, depending on how the STS Client was deployed.
• The STS Client was always using the "old" WS-Policy namespace for "AppliesTo", instead of getting the namespace from the policy.
• The STS now supports processing Claims in a request that are retrieved from a security policy as "IssuedToken/Claims". Previously, it would only issue claims that were contained in a "RequestSecurityTokenTemplate" policy. 

4) CVE-2012-5575

Finally, a note on security advisory CVE-2012-5575 was added to the CXF security advisories page. This attack exploits the fact that Apache CXF will attempt to decrypt arbitrary ciphertexts, without first checking to see if the algorithm corresponds to the given encryption algorithm defined by the WS-SecurityPolicy AlgorithmSuite definition. This can be exploited by chosen ciphertext attacks to retrieve the plaintext. Please note that this issue has been fixed since CXF 2.5.7, 2.6.4, and 2.7.1.

Apache Santuario 1.5.4 and Apache WSS4j 1.6.10 released

Two new bug-fix releases of note in Apache security products:

Apache Santuario 1.5.4 has been released. Amongst the issues fixed is a thread-safety problem when secure validation is enabled, and a possible NPE due to ThreadLocal storage when an application is deployed in certain containers.

Apache WSS4J 1.6.10 has also been released. The issues fixed are available here. A performance issue was fixed in the MemoryReplayCache, which is used to guard against replay attacks. An interop issue was fixed with older Axis 1.x stacks. UsernameTokens with no password elements have been explicitly disallowed by default (although this is configurable). Finally, "time-to-live" functionality has been added to disallow "stale" UsernameTokens (with older Created values).