Driven Administrator Guide
version 2.0.5Driven Plugin Configuration Parameters
The Driven Plugin is the component that collects data from a running Cascading application and sends the information to the Driven Server for processing and analysis.
In general, you do not need to customize the Driven Plugin beyond that needed to make the plugin receive data from running applications. The following information is a reference for the Driven Plugin configuration parameters, which you might need depending on your environment and preferences.
Make the plugin available to the job by placing a file named cascading-service.properties in Hadoop’s classpath with the following contents:
cascading.management.service.jar=${PATH}/driven-plugin.jar
|
Tip
|
If installation and configuration of the plugin appear to fail, check your Cascading application logs to ensure that the ServiceLoader, CascadingService, and DrivenDocumentService properties point to the same path for the Driven Plugin JAR file. |
Driven Server URL
If not using the default hosted Driven Server, the Driven Plugin must be configured to connect to your own installation of the Driven Server.
Either configure the cascading-service.properties file:
cascading.management.document.service.hosts=http<s>://${HOST_NAME}<:${PORT}>
Or set the environment variable:
$ export DRIVEN_SERVER_HOSTS=http<s>://${HOST_NAME}<:${PORT}>
API Key
API keys map to teams in Driven. An application that is associated with an API key is searchable by members of the team that own the key. See Configuring Teams for Collaboration in the User Guide for more information.
Either configure the cascading-service.properties file:
cascading.management.document.service.apikey=${API_KEY}
Or set the environment variable:
$ export DRIVEN_API_KEY=${API_KEY}
Default Application Tags
As part of the Driven Plugin configuration, you can specify a tag or group of tags to associate with all applications that run with the plugin. See Managing Applications with Tags in the User Guide for best practices with tags and for how to apply tags on particular applications.
In the following lines, replace CATEGORY:INSTANCE1 with your real tag. It is recommended to use the "category label:specific value" capability of tags, which requires using the colon as a separator. To specify multiple tags, enter them as comma-separated values.
Either configure the cascading-service.properties file:
driven.protocol.tags=CATEGORY:INSTANCE1
Or set the environment variable:
$ export DRIVEN_DEFAULT_TAGS=CATEGORY:INSTANCE1
Slice Data Suppression
Slice data transmission can be suppressed or resumed by setting a configuration parameter. The default setting of this parameter is false, which means slice data is transmitted unless you change the setting. Setting this property to true overrides a server-side configuration for slice-data transmission.
Either configure the cascading-service.properties file:
driven.protocol.slice.suppress=[true|false]
Or configure the environment variable:
$ export DRIVEN_SUPPRESS_SLICE_DATA=[true|false]
Slice Data on Completion
A configuration parameter can be set to send or to not send data when the slice state changes to completed. The default setting for this parameter is true, which means data is sent only when the slice state reaches completed state.
Either configure the cascading-service.properties file:
driven.protocol.slice.state_change_only=[true|false]
Or set the environment variable:
$ export DRIVEN_SLICE_STATE_CHANGE_ONLY=[true|false]
Java Command Suppression
You can suppress the command-line arguments used to launch the Cascading job by setting the parameter for Java command suppression to true. This property overrides any server-side configuration for Java command suppression. The default setting for this parameter is false.
Either configure the cascading-service.properties file:
driven.protocol.command.suppress=[true|false]
Or set the environment variable:
$ export DRIVEN_SUPPRESS_JAVA_COMMAND=[true|false]
Archive Mode
The Driven Plugin can be run in archive mode. If archive mode is enabled, all records that are sent to the Driven Server are written to disk even if the server is unreachable. This can be useful if the Driven Server is not available or unreachable as the archive can later be replayed when the server is reachable.
Either configure the cascading-service.properties file:
cascading.management.document.service.archive.dir=/${ARCHIVE_DIRECTORY_PATH}
Or set the environment variable:
$ export DRIVEN_ARCHIVE_DIR=${ARCHIVE_DIRECTORY_PATH}
Failed Units of Work (formerly Failed Flows)
By default, a Unit of Work that has FAILED status does not automatically set the end state of the application to FAILED. You can configure Driven so that an application is put in FAILED state when a Unit of Work fails by setting the following parameter to true.
Configure the cascading-service.properties file:
cascading.management.state.service.appstats.failonflow=[true|false]
Pending Flows (formerly Pending Units of Work)
By default, if an application finishes running but still has a Unit of Work that has PENDING status, the end state of the application is SUCCESSFUL. You can configure Driven so that an application is put in FAILED state when a Unit of Work remains in PENDING state by setting the following parameter to true.
Configure the cascading-service.properties file:
cascading.management.state.service.appstats.failonpending=[true|false]
Next
Learn about how a Driven admin can further customize and monitor the deployment in Using the Driven Admin Console.