Deploying an Apache Camel route to Apache Karaf

In the previous blog post, we showed how to use Apache Camel to query an Apache Kafka broker, which is secured using kerberos. In this post, we will build on the previous blog post by showing how to deploy our Camel route to Apache Karaf. Karaf is an application runtime container that makes it incredibly easy to deploy simple applications via its "hot deploy" feature. As always, there are a few slightly tricky considerations when using kerberos, which is the purpose of this post.

As a pre-requisite to this article, please follow the previous blog post to set up Apache Kafka using kerberos, and test that the Camel route can retrieve from the topic we created successfully.

1) Configuring the Kerberos JAAS Login Module in Karaf

Download and extract the latest version of the Apache Karaf runtime (4.2.3 was used in this post). Before starting Karaf, we need to pass through a system property pointing to the krb5.conf file created in our Kerby KDC. This step is not necessary if you are using the standard location in the filesystem for krb5.conf. Open 'bin/karaf' and add the following to the list of system properties:

• -Djava.security.krb5.conf=/path.to.kerby.project/target/krb5.conf \

Now start Karaf via "bin/karaf". Karaf uses JAAS for authentication (see the documentation here). In the console, enter "jaas:" and hit 'tab' to see the possibilities. For example, "jaas:realm-list" displays the JAAS realms that are currently configured.

Recall that our Camel route needs to configure a JAAS LoginModule for Kerberos. In the example given in the previous post, this was configured by setting the Java System property "java.security.auth.login.config" to point to the JAAS configuration file. We don't want to do that with Karaf, as otherwise we will end up overriding the other JAAS LoginModules that are installed.

Instead, we will take advantage of Karaf's "hot deploy" feature to add the Kerberos Login Module we need to Karaf. Drop the following blueprint XML file into Karaf's deploy directory, changing the keytab location with the correct path to the keytab file:

	<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
	xmlns:jaas="http://karaf.apache.org/xmlns/jaas/v1.0.0"
	xsi:schemaLocation="
	http://www.osgi.org/xmlns/blueprint/v1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
	">
	<jaas:config name="KafkaClient" rank="2">
	<jaas:module className="com.sun.security.auth.module.Krb5LoginModule" flags="required">
	refreshKrb5Config=true
	useKeyTab=true
	keyTab=/home/coheig/src/testcases/apache/bigdata/kerberos/target/client.keytab
	storeKey=true
	principal=client
	</jaas:module>
	</jaas:config>
	</blueprint>

view raw kafka-jaas.xml hosted with ❤ by GitHub

For Karaf to pick this up, we need to register the blueprint feature via "feature:install aries-blueprint". Now we should see our LoginModule configured via "jaas:realm-list":

2) Configuring the Camel route in Karaf

Next we will hot deploy our Camel route as a blueprint file in Karaf. Copy the following file into the deploy directory:

	<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
	xmlns:camel="http://camel.apache.org/schema/blueprint"
	xmlns:jaas="http://karaf.apache.org/xmlns/jaas/v1.0.0"
	xsi:schemaLocation="
	http://www.osgi.org/xmlns/blueprint/v1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
	">
	<camelContext xmlns="http://camel.apache.org/schema/blueprint">
	<route>
	<from uri="kafka:test?brokers=localhost:9092&amp;groupId=test-consumer-group&amp;seekTo=beginning&amp;autoCommitEnable=true&amp;sessionTimeoutMs=30000&amp;autoCommitIntervalMs=1000&amp;saslKerberosServiceName=kafka&amp;securityProtocol=SASL_PLAINTEXT"/>
	<log message="Received: $simple{body}" loggingLevel="INFO"/>
	<log message="Headers: $simple{headers}" loggingLevel="INFO"/>
	<convertBodyTo type="java.lang.String"/>
	<to uri="file:target/results"/>
	</route>
	</camelContext>
	</blueprint>

view raw camel-kafka.xml hosted with ❤ by GitHub

Then we need to install a few dependencies in Karaf. Add the Camel repo via "repo-add camel 2.23.1", and install the relevant Camel dependencies via: "feature:install camel camel-kafka". Our Camel route should then automatically start, and will retrieve the messages from the Kafka topic and write them to the filesystem, as configured in the route. The message payload and headers are logged in "data/log/karaf.log".

Using the Apache Camel Kafka component with Kerberos

Apache Camel is a well-known integration framework available at the Apache Software Foundation. It comes with a huge number of components to integrate with pretty much anything you can think of. Naturally, it has a dedicated component to communicate with the popular Apache Kafka project. In this blog entry, we'll show first how to use Apache Camel as a consumer for a Kafka topic. Then we will show how to configure things when we are securing the Kafka broker with kerberos, something that often causes problems.

1) Setting up Apache Kafka

First let's set up Apache Kafka. Download and install it (this blog post uses Kafka 2.0.0), and then start up Zookeeper and the broker, as well as creating a "test" topic and a producer for that topic as follows:

• bin/zookeeper-server-start.sh config/zookeeper.properties
• bin/kafka-server-start.sh config/server.properties
• bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
• bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test --producer.config config/producer.properties

Type a few messages into the producer console to make sure that it is working.

2) Consuming from Kafka using Apache Camel

Now we'll look at how to set up Apache Camel to consume from Kafka. I put a project up on github here for this purpose. The Camel route is defined in Spring, and uses the Camel Kafka component to retrieve messages from the broker, and to write them out to the target/results folder:

	<camelContext xmlns="http://camel.apache.org/schema/spring">
	<route>
	<from uri="kafka:test?brokers=localhost:9092&amp;groupId=test-consumer-group&amp;seekTo=beginning&amp;autoCommitEnable=true&amp;sessionTimeoutMs=30000&amp;autoCommitIntervalMs=1000"/>
	<log message="Received: $simple{body}" loggingLevel="INFO"/>
	<log message="Headers: $simple{headers}" loggingLevel="INFO"/>
	<convertBodyTo type="java.lang.String"/>
	<to uri="file:target/results"/>
	</route>
	</camelContext>

view raw camel-spring.xml hosted with ❤ by GitHub

Simply run "mvn clean install" and observe the logs indicating that Camel has retrieved the messages you put into the topic with the producer above. Then check "target/results" to see the files containing the message bodies.

3) Securing Apache Kafka with Kerberos

So far so good. Now let's look at securing the Kafka broker using kerberos. I wrote a previous blog post to show how to use Apache Kerby as a KDC with Kafka, so please follow the steps outlined here, skipping the parts about configuring the consumer.

4) Consuming from Kafka using Apache Camel and Kerberos

To make our Camel route work with Kafka and Kerberos, a few changes are required. Just as we did for the Kafka producer, we need to set the "java.security.auth.login.config" and "java.security.krb5.conf" system properties for Camel. You can do this in the example by editing the "pom.xml" and adding something like this under "systemPropertyVariables" of the surefire configuration:

• <java.security.auth.login.config>/path.to.kafka.project/config/client.jaas</java.security.auth.login.config
• <java.security.krb5.conf>/path.to.kerby.project/target/krb5.conf</java.security.krb5.conf>

Replacing the paths to Kafka and Kerby appropriately (refer to the previous blog post on Kafka + Kerberos if this does not make sense). Next we need to make some changes to the Camel route itself. Add the following configuration to the Camel configuration for the Kafka component:

• &amp;saslKerberosServiceName=kafka&amp;securityProtocol=SASL_PLAINTEXT

Camel uses "GSSAPI" as the default SASL mechanism, and so we don't have to configure that. Now re-run "mvn clean install" and you will see the Camel route get a ticket from the Kerby KDC and consuming messages successfully from the Kafka topic.

Validating kerberos tokens from different realms in Apache CXF

We've covered on this blog before how to configure an Apache CXF service to validate kerberos tokens. However, what if we have a use-case where we want to have multiple endpoints validate kerberos tokens that are in different realms? As Java uses system properties to configure kerberos, things can get a bit tricky if we want to co-locate the services in the same JVM. In this article we'll show how it's done.

1) The test scenario

The scenario is that we have two KDCs. The first KDC has realm "realma.apache.org", with users "alice" and "bob/service.realma.apache.org". The second KDC has realm "realmb.apache.org", with users "carol" and "dave/service.realmb.apache.org". We have a single service with two different endpoints - one which will authenticate users in "realma.apache.org", and the second that will authenticate users in "realmb.apache.org". Both endpoints have keytabs that we have exported from the KDC for "bob" and "dave".

2) Kerberos configuration

Both endpoints have to share the same Kerberos configuration, due to the fact that Java uses system properties to set up JAAS with the Krb5LoginModule. We need to set the following system properties:

• java.security.auth.login.config - The path to the JAAS configuration file for the Krb5LoginModule
• java.security.krb5.conf - The path to the krb5.conf kerberos configuration file

The JAAS configuration file for our service looks like the following:

Here we have two entries for "bob" and "dave", each pointing to a keytab file. Note that the principal contains the realm name. This is important as we have no default_realm in the krb5.conf file. The krb5.conf file looks like this:

Here we configure how to reach both KDCs for our different realms.

3) Service configuration

Next, we'll look at how to configure the services. We will show how it's done for a JAX-WS service, but similar configuration exists for JAX-RS. The client will pass the kerberos token in a BinarySecurityToken security header in the message, according to the WS-Security specs. We'll assume the service is using a WS-SecurityPolicy that requires a kerberos token (for more details see here). Here is a sample spring configuration for an endpoint for "dave":

	<bean id="kerberosCallbackHandler2"
	class="org.apache.cxf.systest.kerberos.common.KerberosServicePasswordCallback">
	<property name="username" value="dave"/>
	<property name="password" value="dave"/>
	</bean>
	<bean id="kerberosValidator2" class="org.apache.wss4j.dom.validate.KerberosTokenValidator">
	<property name="contextName" value="dave"/>
	<property name="serviceName" value="dave/service.realmb.apache.org@service.realmb.apache.org"/>
	<property name="usernameServiceNameForm" value="true"/>
	<property name="callbackHandler" ref="kerberosCallbackHandler2"/>
	</bean>
	<jaxws:endpoint ... >
	<jaxws:properties>
	<entry key="ws-security.bst.validator" value-ref="kerberosValidator2"/>
	</jaxws:properties>
	</jaxws:endpoint>

view raw service.xml hosted with ❤ by GitHub

We have a JAX-WS endpoint with a "ws-security.bst.validator" property which points to a KerberosTokenValidator instance. This tells CXF to process a received BinarySecurityToken with the KerberosTokenValidator.

The KerberosTokenValidator is configured with a CallbackHandler implementation, to supply a username and password (see here for a sample implementation). Note that this is not required normally when we have a keytab file, but it appears to be required when we do not define a default realm. The KerberosTokenValidator instance also defines the JAAS context name, as well as the fully qualified principal name. As this is in service name form, we have to set the property "usernameServiceNameForm" to "true" as well.

If we set up the endpoint for "bob" with similar configuration, then our krb5.conf doesn't need the "default_realm" property and we can successfully validate tickets for both realms.