Driven User Guide

This document explains the Driven process object and state model.

The Driven process object model consists of an Application as the root parent. An Application can have any number of child Units of Work, which are analogous to Flows for those familiar with the Cascading model. Units of Work are composed of child Steps, which are composed of Nodes in turn. Slices are data concerning the actual instantiation and execution of logical nodes on the compute cluster. For example a logical mapper is a Node of the mapper type, which will then be instantiated multiple times and executed in parallel on the compute cluster in whatever slots can be allocated. Slice objects contain actual execution data such as counters and durations.

All Driven process objects, such as App, Unit of Work, and Step have a state, and progress through those states using a state machine explained below.

As soon as the Application client JVM starts, the Application is created in a "Pending" state. As the other logical application objects such as Units of Work, Steps and Nodes are created, they enter the "Pending" state.

Once processing is initiated, process objects move from "Pending" to "Started". When the client actually submits work to the compute cluster the related process objects move into the "Submitted" state indicating that they are in the cluster queue. Applications can not be in a "Submitted" state as they represent the JVM that is submitting the work to the cluster.

Once an instance of a logical Node is actually executing on the compute cluster, the associated child and parents move into the "Running" state.

When all children have a completed state (Stopped, Successful, Failed, Skipped), The parent is also moved into a completed state.

The final Application state is based on the final state of the child Units of Work. Because there may be handled errors, Applications with failing Units of Work can be considered successful. The user can configure whether an Application with any failing Units of Work should be considered a failure using Driven plugin properties; see Plugin Failure configuration

Because Slice data concerns actual execution data, Slices are really only ever in a running or completed state. For this reason no Slice data shows up in the Unit of Work detail view until they are running.