Securing Apache Storm - part III

This is the third in a series of blog posts on securing Apache Storm. The first post looked at setting up a simple Storm cluster that authenticates users via Kerberos, and deploying a topology. The second post looked at deploying the Storm UI using Kerberos, and accessing it via a REST client. Thus far we have only looked at how to authenticate users to Storm using Kerberos. In this post we will look at how to use Apache Ranger to authorize access to Apache Storm.

1) Install the Apache Ranger Storm plugin
 
Follow the steps in the first tutorial (parts 1 - 3) to setup the Apache Kerby testcase, Apache Zookeeper instance, and the Apache Storm distribution, if you have not done this already. Now we will install the Apache Ranger Storm plugin. If you want to be able to download the topologies from Storm to Ranger when creating policies, then follow the second tutorial to start the Storm UI.

Download Apache Ranger and verify that the signature is valid and that the message digests match. Due to some bugs that were fixed for the installation process, I am using version 1.0.0-SNAPSHOT in this post. Now extract and build the source, and copy the resulting plugin to a location where you will configure and install it:

• mvn clean package assembly:assembly -DskipTests
• tar zxvf target/ranger-1.0.0-SNAPSHOT-storm-plugin.tar.gz
• mv ranger-1.0.0-SNAPSHOT-storm-plugin ${ranger.storm.home}

Now go to ${ranger.storm.home} and edit "install.properties". You need to specify the following properties:

• POLICY_MGR_URL: Set this to "http://localhost:6080"
• REPOSITORY_NAME: Set this to "cl1_storm".
• COMPONENT_INSTALL_DIR_NAME: The location of your Apache Storm installation

Save "install.properties" and install the plugin as root via "sudo ./enable-storm-plugin.sh". The Apache Ranger Storm plugin should now be successfully installed. Now start Kerby, Zookeeper and Storm as covered in the first tutorial.

2) Create authorization policies in the Apache Ranger Admin console

Next we will use the Apache Ranger admin console to create authorization policies for Apache Storm. Follow the steps in this tutorial to install the Apache Ranger admin service. To retrieve the running topologies from Apache Storm, then you need to configure Kerberos appropriately for Apache Ranger. You can first point to the Kerby krb5.conf via:

• export JAVA_OPTS="-Djava.security.krb5.conf=/path.to./kerby.project/target/krb5.conf"

Start the Apache Ranger admin service with "sudo -E ranger-admin start" and open a browser and go to "http://localhost:6080/" and log on with "admin/admin". Add a new Storm service with the following configuration values:

• Service Name: cl1_storm
• Username: storm-client
• Password: storm-client
• Nimbus URL: http://localhost:8080

Click on "Test Connection" to verify that we can connect successfully to Storm  + then save the new service. Now click on the "cl1_storm" service that we have created. Edit the existing policy for the "*" Storm topology, adding the user "alice" (create this user if you have not done so already under "Settings, Users/Groups") to all of the available permissions.

3) Testing authorization in Storm

Now let's test the Ranger authorization policy we created above in action. The Ranger authorization plugin will pull policies from the Admin service every 30 seconds by default. For the "cl1_storm" example above, they are stored in "/etc/ranger/cl1_storm/policycache/" by default. Make sure that the user you are running Storm as can access this directory. To test authorization follow step 4 in the first tutorial, but use the user "storm-client" instead (and "storm_client.keytab"). You should see an authorization exception. Now try again with user "alice" (and "alice.keytab") and authorization should succeed.