Driven Agent Guide: Advanced Installation

version 2.2.6

Advanced Installation

Advanced users may wish to script the Driven Agent installation, or use the Driven Agent with Amazon Elastic MapReduce.

Scripted Installation

For advanced users, the Driven Agent can be installed with the following script:

# to download the script
$ wget

# for Hive installation
$ bash --hive

# for MapReduce installation
$ bash --mapreduce

# for Spark installation
$ bash --spark

Alternately, as a one-liner:

# for Hive installation
$ export AGENT=hive; curl | sh

# for MapReduce installation
$ export AGENT=mr; curl | sh

# for Spark installation
$ export AGENT=spark; curl | sh

This script will create a .driven-plugin directory in the current user’s home directory, download the latest Driven Agent JAR, and create a symbolic link referencing the latest versions of the driven-agent-[framework].jar.

Re-running the script can be used to safely upgrade the agent.

driven-agent-[framework].jar is a unix symbolic link to the latest downloaded version of the agent jar file. This link is created or updated by the install script.

Amazon Elastic MapReduce

For Amazon Elastic MapReduce users, the, introduced above, doubles as a bootstrap action.

Use this script as you would any AWS EMR bootstrap action.

EMR 4.x

Amazon introduced a set of changes in EMR version 4.0, that have a direct influence on how to install the Driven agent.

One important change is that bootstrap actions can no longer modify the installation of Hadoop, since Hadoop is only deployed after all bootstrap actions have been executed. The new way of changing the hadoop installation with user defined settings is using the application configuration feature.

Driven provides a set configurations to be used with the different agents. This needs to be done IN ADDITION to configuring the install-driven-plugin bootstrap action above. On the command line simply add the --configurations switch for your framework:


Replace [framework] with hive for Hive, mr for MapReduce, and spark for Spark.

AWS EMR Before 4.x

For EMR versions prior to 4.x, the only addition is to also use the Amazon provided configure-daemons (s3://elasticmapreduce/bootstrap-actions/configure-daemons) bootstrap action with the following arguments:


Replace [framework] with hive for Hive, mr for MapReduce, and spark for Spark.