Driven Plugin Guide: Configuring the Driven Plugin

The Driven hosts property configures the Driven Plugin to send data to one or more Driven Server instances. If multiple values are provided, each server is used as a failover host for the forer.

Either configure the cascading-service.properties file:

Or set the environment variable:

To set multiple values, separate the urls with a comma.

API keys map to teams in Driven.

If a Driven configuration URL has been retrieved from a team page and set, this property is not required.

An application that is associated with an API key is searchable by members of the team that own the key. See Working with Teams in the User Guide for more information.

Either configure the cascading-service.properties file:

Or set the environment variable:

The following properties control the Driven Plugin’s behavior as it attempts to communicate with the Driven Server.

Set the flush wait time using the following property:

This property has a default value of 400 milliseconds, which means that the Driven Plugin will wait 400 milliseconds between attempts to flush the internal buffer to the Driven Server.

If the Driven Server is unavailable, the flush will back-off gradually until driven.management.document.service.max.retry.millis is met.

To change the default maximum wait time, use the following property:

Once the interval between retries reaches the value of driven.management.document.service.max.retry.millis, the Driven Plugin will retry at an interval equal to that value until the current application begins to shutdown.

During shutdown, the Driven Plugin will continue to attempt to reach the Driven Server until driven.management.document.service.max.server.unreachable.millis is reached. By default the Driven Plugin will retry indefinitely.

To set the Driven Plugin to stop retrying after a certain number of milliseconds, use the following property:

By default, the Driven Plugin will retry indefinitely (set to "-1") until the Driven Server becomes available and the buffer completely flushes.

In case of failed connections, the Driven Plugin keeps a disk buffer of all unsent items until they can be successfully sent.

Previous versions of the Driven Plugin relied on an 'in memory' buffer requiring complex and large applications to allocate more memory to the client applications. As of Driven 2.2, the Driven Plugin uses a disk backed buffer by default.

To restore the original behavior when disk space is expected to be low or unavailable on the host the Driven Plugin is installed, set:

By default, this value is false, forcing all data to be buffered to disk to minimize the chance of an OutOfMemoryError if the application default memory settings are not adjusted.

At startup the plugin attempts to find a suitable directory to store the disk buffer. A suitable directory is one that has sufficient usable storage available, and has write permissions.

Suitable available (usable) disk space for a disk partition containing the target path is by default 200MB of free space. To change this value, change the following parameter, for example to 500MB.

The following paths are verified, in order:

A specific root directory for all buffer files can be provided via:

If this path is unavailable, the above paths will be attempted unless the following property is true:

Alternately, each of the paths above can be removed from the search by disabling them via one of the following properties:

By default, the maximum size of the disk buffer will grow from 4K to a maximum of 100MB. This can only happen if the Driven Server becomes unavailable or overloaded preventing the buffer from being flushed. Once a buffer is flushed, it will be reduced back to 4k.

This value can be changes by setting:

Keep in mind that every active Driven Plugin disk buffer will begin to grow larger as long as the Driven Server is unavailable. Once the disk buffer reaches its maximum size, or the usable storage space on the partition reaches the configured minimum, the Driven Plugin will shutdown and reclaim any used space without shutting down the host application.

It is important to both; provide failover Driven Server hosts for the Driven Plugin, and configure a partition with suitable disk space in case of unforeseen network failure.

As part of the Driven Plugin configuration, you can specify a tag or group of tags to associate with all applications that run with the plugin. See Managing Applications with Tags in the User Guide for best practices with tags and for how to apply tags on particular applications.

In the following lines, replace CATEGORY:INSTANCE1 with your real tag. It is recommended to use the "category label:specific value" capability of tags, which requires using the colon as a separator. To specify multiple tags, enter them as comma-separated values.

Either configure the cascading-service.properties file:

Or set the environment variable:

Once an application JVM is running and initial data is sent by the Driven Plugin, then an application heartbeat signal is sent by the Driven Plugin to the Driven Server. To set the frequency (or interval, in milliseconds) of this signal, use the following property:

Slice data transmission can be suppressed or resumed by setting a configuration parameter. The default setting of this parameter is false, which means slice data is transmitted unless you change the setting. Setting this property to true overrides a server-side configuration for slice-data transmission.

Either configure the cascading-service.properties file:

Or configure the environment variable:

A configuration parameter can be set to send or to not send data when the slice state changes to completed. The default setting for this parameter is true, which means data is sent only when the slice state reaches completed state.

Either configure the cascading-service.properties file:

Or set the environment variable:

You can suppress the command-line arguments used to launch the Cascading job by setting the parameter for Java command suppression to true. This property overrides any server-side configuration for Java command suppression. The default setting for this parameter is false.

Either configure the cascading-service.properties file:

Or set the environment variable:

By default, a flow that has FAILED status does not automatically set the end state of the application to FAILED. You can configure Driven so that an application is put in FAILED state when a flow fails by setting the following parameter to true.

Configure the cascading-service.properties file:

By default, if an application finishes running but still has a flow that has PENDING status, the end state of the application is SUCCESSFUL. You can configure Driven so that an application is put in FAILED state when a flow remains in PENDING state by setting the following parameter to true.

Configure the cascading-service.properties file:

By default, the Driven Plugin logs events. To disable all Driven Plugin logging, set the following property:

Or set the environment variable:

To enable the Driven Plugin debug logging, set the following property:

Or set the environment variable:

To force the Driven Plugin to log INFO messages, set the following property:

To disable the Driven Plugin while leaving it installed, set the following property:

Or set the environment variable:

All properties can be set in the cascading-service.properties file, as property arguments to the Java VM (java -Dcascading..enabled=true), or if specified, as an environment variable set via the export or equivalent shell command.