Driven Agent Guide: Prerequisitesversion 2.2.6
- 1. Prerequisites
- 2. Installing the Driven Agent
- 3. Configuring the Driven Agent
3.1. Testing the Agent
3.2. Agent Common Options
- 4. Driven Agent for MapReduce
4.2. MapReduce Versions
- 5. Driven Agent for Hive
5.2. Metadata Support
- 6. Driven Agent for Apache Spark
6.2. Supported APIs
6.3. Spark Runtimes
6.4. Supported Runtimes
- 7. Using Driven Agent with Apache Oozie
- 8. Advanced Installation
- 9. Troubleshooting the Driven Agent
The Driven Agent is a collection of JVM libraries that enables monitoring of the following Hadoop applications in Driven:
The Agent works with any job scheduler that can launch an Apache YARN or Apache Spark application. Some schedulers may have specific instructions like:
There is one agent JAR file for Hive, another JAR file for MapReduce, and
another JAR file for Spark. Thus the appropriate Agent must be downloaded to the
host machine the above applications are launched from, and the launch scripts
need to be modified to force the JVM binary (
$JAVA_HOME/bin/java) to load the
Agent. See Installing the Agent for details.
Driven defines an application context as the JVM instance driving and orchestrating the client side of Hadoop applications. Each Hive query or MapReduce job appears as a single Unit of Work in that application. In a single application context, there can be thousands of queries or Units of Work. Each instance of the application entails a shutdown and restart.
|To monitor only Cascading applications with Driven, the Driven Agent cannot be used, an installation of the Driven Plugin JAR is required. See Driven Plugin for details.|
To download the necessary Driven Agent, you will need Internet access from the host machine it will be installed, or sufficient privileges to upload the Agent jar file to the host machine the applications to be monitored will be launched from.
If using a hosted version of Driven Server not downloaded and installed on the local network, the Driven Agent will need direct internet access to the remote Driven Server installation.
If your applications run behind a secure firewall, it may be necessary to download and install a local copy of the Driven Server. See the Driven Administrator Guide for details.