Driven User Guide
version 1.3.8Application Views
As business demand for data analytics increases, so does the demand to retrieve application execution data and telemetry information punctually for statistical reporting, forecasting, resource allocation, and other purposes. Driven’s graphical overlays on detailed data sets help parse large volumes and multiple layers of application performance information.
After searching or filtering the information in a Status View such that the applications, time range, or other dimensions focus on what you want to explore in more depth, save the parameters as an Application View. You can also create a new Application View by modifying the search or filter criteria of an existing Application View and saving it with a different name. Figure 1 shows an example of the Application View, which is followed by a description of how to gather and customize information in the view.
|
Tip
|
The Application View is ideal for monitoring instances of a single application that executes at periodic intervals. Adjust the search and filter criteria to focus on a particular application, and then save the criteria as an Application View for quick retrieval later. A look at the graph can provide a visual cue to spot application run outliers. |
The Application View displays the following:
-
Quality-of Service Statistics: Various statistics that can help gauge compliance with service-level agreements or other benchmarks — longitudinal data is presented side-by-side with data for selectable time intervals.
-
Timeline Selector: Directly under the statistics, there is a timeline selector area. The row of boxes represents a timeline charting the degree of application activity chronologically so that you can compare the activity by period in one glance. Using the controls in the timeline selector area, you can filter the information that appears in the interval statistics, graph, and table to help focus on relevant data.
-
Runtime Graph: An interactive Marey diagram of individual application runs that illustrate the different runtime states with color coding.
-
Table: Execution details for current and past application runs that are hyperlinked to underlying flow metrics, which in turn contain hyperlinks to even more granular slice performance information.
The graphical, interdependent representation of application performance metrics can help pinpoint problematic application runs and correlate instances with root causes of potential or existing bottlenecks.
Timeline Selector
Search-term and filter parameters can narrow down data for specific applications. But monitoring and troubleshooting often entails breaking down runtime data to separate application instances, especially for applications that run repeatedly. The Application View provides a platform for inspecting application runs both by discrete blocks of time and cumulatively. The timeline selector area (see Figure 2) provides a way to control time parameters.
The horizontal grid of boxes represents a period of time in the span set by the search time filter. The grid is often only one snapshot period in the range of time set by the search time-filter parameters, but it can fit the entire range depending on settings. Click the arrowheads under the timeline to navigate to other periods within the Application View time span. Notice that by doing this the date/time range above the timeline boxes changes accordingly.
Each box in the grid is an interval, which enables you to focus on application runs during a particular period of time. The gradient white-to-dark gray coloring of intervals reflects the relative degree of application activity for each period, where shading is in proportion to number of application executions. Notice that when you click on an interval, the selected box turns blue and the date/time range in the graph changes.
After you select an interval, Driven dynamically plots detailed changes in the Interval Statistics, graph, and tabular data that reflect the period that you select. Define the amount of time that each interval represents by clicking on one of the Time Interval periods.
Quality-of-Service Statistics
The Application View can automatically display a range of statistics about application execution. To display all metrics or specific metrics that Driven can calculate, click the configuration wheel above the timeline (
).
| Metric | Calculation Method |
|---|---|
Effective Parallelism |
Total runtime of all slices divided by runtime of the app, averaged over all app instances |
Failed Ratio |
Ratio of number of FAILED app runs to number of all app runs |
Time Between Failures |
Average of time periods between one app failure and the next app failure |
Pending Duration |
Average time between when apps enter PENDING status and when apps reach RUNNING status |
Periodicity |
Average of time periods between when one app run enters PENDING status and the next app run enters PENDING status |
Running Duration |
Average time between when apps enter RUNNING status and when apps reach a finished state |
Stop Ratio |
Ratio of number of stopped app runs to number of all app runs |
Total Duration |
Average time between when apps start (enter PENDING or RUNNING status) and when apps reach a finished state |
After you select statistical metrics, the information is displayed in two ways:
-
Aggregated Statistics is an accumulation of your selected metrics for application run data that spans the entire search time-filter parameters of the Application View.
-
Interval Statistics displays the metrics based on data only from the selected interval.
Main Insights from the Graph
The graph is a Marey diagram, which plots a dynamic, visual representation of various application execution metrics.
Application-Instance Status and Runtimes
Each line in the graph represents an instance of an application run for the selected time interval. Where an application run touches the top horizontal line represents the start date/time of the run, while the point touching the bottom horizontal line represents the ending date/time. Consequently, the more vertically graphed an application instance is, the shorter is the total duration.
By default, the graph colors each application run with the status when runtime ended. (You might need to expand your browser window to the right to see the color legend above the graph.) Application instances that are still executing at the current time are rendered as dashed lines and are colored to reflect the current status. Hover over an application instance line to view information about various timing factors, as shown in Figure 3.
Variances in the runtimes of a particular application can indicate that there are issues requiring attention. The linear graphic representation of application runs is one way that Driven facilitates this type of troubleshooting. By scanning the graph to see if runs of the same application are sloped differently relative to one another, you can spot whether further investigation into persistence and reliability is needed.
To highlight application runs that exceed a certain period of time, click the Set Threshold icon. The graph renders all application runs that exceed the threshold as red lines. This can help with pinpointing application runs that do not comply with service-level agreements.
Historical Application Performance
While viewing the application runs for a time period, you can overlay the graph with data from runs of the same application prior to the period. The rendering enables you to compare how an application performed at repeated time intervals in a unified view.
Relative Status Durations
Click the State Transition icon to toggle the graph to a mode where each application run is colored proportionately to the times of three different runtime states (PENDING, STARTED, and RUNNING). In addition, each status is sloped proportionately to the amount of time that the application was in the state. State transitions on the graph enable visual cues for comparing how multiple instances of the same application perform because you can view relative times of runtime status.
Anomalous Application Behavior
Use the various visualizations available in the graph to help detect and analyze anomalies when applications execute. Table 2 lists a few types of anomalous application behavior along with some possible causes to consider. The information in the table is not exhaustive and not applicable to every situation.
| Unexpected Behavior | Possible Causes |
|---|---|
App started as expected, but finishes later than expected |
1) Cluster overload; 2) App processed exceptionally large data set |
App started late, but finished on time |
1) Dependency on another app that finished late; 2) Cluster overload |
App not executed |
1) Dependency on another app that did not complete successfully; 2) Scheduling error |
Fine-Tuning the Graph
Several graph controls let you toggle the different visual representations of application performance data and chronological markers that are discussed above.
The following table explains the graph controls. Examples of how most controls effect the graph are immediately after the table.
| Element | Drop-down Menu Icon | Action |
|---|---|---|
Hide/Show toggle button |
|
Toggles between hiding and showing the graph in the current Application View. Tip: Hiding the graph can be helpful if you want to move the table in closer proximity to the statistics and timeline selector part of the window. |
Historical Intervals |
|
Graphs the app instances for the specified number of preceding intervals. |
Set Threshold |
|
Colors lines for application runs that exceed specified threshold in red, and colors lines that run in less time than the specified threshold in green. |
Show Guides |
|
Displays grid-like vertical lines that demarcate the start and end of equal time periods on the graph. Use the drop-down menu to set the amount of time between the guide lines. |
State Transitions |
|
Renders application execution lines to change color when the status of the application changes in the duration of time that is represented from top horizontal axis to bottom horizontal axis. |
- Example 1: Historical Intervals
-
Historical Intervals setting = 2; Time Interval setting = 1 day
The gray lines in the graph below represent apps that started executing between 2:00 and 4:00 pm on June 24 and June 25 (2 intervals of 1 day each), while the lines with other colors display app runs on June 26 marked with state transitions.
The screenshot cuts off the parts of the graph that diagram any app activity before 2:00 pm and after 4:00 pm of June 26. On the actual Driven screen, the whole day is graphed. The RUNNING status portions of the app runs and end times of app executions are also omitted.
|
Note
|
The graphs in Examples 1 through 4 are excerpts of whole Driven diagrams that appear on the screen. Only parts of the graphs are shown to pinpoint specific elements for illustrative purposes. |
- Example 2: Set Threshold
-
Set Threshold setting: Total Duration, 6 minutes
The green lines in the graph below represent application instances that completed in less than 6 minutes. The red lines show application runtimes that exceeded 6 minutes.
Graph with threshold setting:
- Example 3: Show Guides
-
Show Guides setting: 1 day
The dotted gray lines mark 1-day (24-hour) intervals.
Graph with dotted-gray guide lines:
- Example 4: State Transitions
-
State Transitions is toggled on (there is no value setting for this control).
Each application execution is color-coded to reflect changing app states in proportion to the amount of time that passed in each status. Dark blue = PENDING status; yellow = STARTED; Light blue = RUNNING. In the graph below, you can see that the application instances were usually in RUNNING status longer than other states because the light-blue parts of the lines are sloped more horizontally.
Click the colored segment of a line can be clicked to reveal underlying application details.
Graph showing the status changes when State Transitions is selected:
Auto-Refreshed Data
Driven can refresh the displayed information as updates stream in from the plugin. Ensure that the Refresh toggle in the top right corner is enabled to allow the displayed Driven data to auto-refresh in real time. If the Refresh toggle is disabled, you must manually refresh the browser window to see real-time updates. Generally, this feature is useful if you want to monitor applications as they run.
Click and slide the Refresh icon to toggle between enabled and disabled.
Tabular Data
Information about each application run is also displayed in the table under the graph. The table mirrors data in the graph but can provide additional details about each application instance. If the information that you seek does not appear in the table, use the column chooser to see what additional details about application runs can be brought into the display of the Application View table.
Some key monitoring assets of the tabular interface include the following capabilities:
-
Click on a hyperlinked application instance to view an app details page, which includes visualization of flows and steps as directed acyclic graphs (DAGs) and hyperlinks to flow details pages that contain slice performance information
-
Export application-level tabular data to a tab-separated values text file
-
Add or remove metrics that are displayed
-
View status transitions of an application instance on a timeline
The Driven page displays a maximum of 25 rows. Use the pagination arrows to navigate a table that spans more than one page.
|
Tip
|
The columns can be arranged by drag and drop to your preferred order after clicking on a column heading. You can also sort the items within a column by clicking on the bidirectional arrow. |
Exporting Data to a .tsv File
As part of your analytical process, the application data that is presented in a Driven table can be downloaded as a tab-separated values (.tsv) file, which then can populate a spreadsheet for detecting patterns, metrics, and usage.
Click the download icon 
to capture the Driven table data and download it to a file.
Track Cascading Application Instances by Additional Metrics
Driven lets you customize most of the information that the table displays. Click the column chooser icon 
to reveal or conceal columnar metrics. The Status and Name columns cannot be hidden.
The columnar metrics are categorized in the column chooser. Each category can be collapsed or expanded.
A key feature of the table and column chooser is the ability to import and view counter attributes. See Counter Data and Other Metrics in Tables for more information.