Driven User Guide: Searches, Saved Views, and Accessing Relevant Data

version 2.2.6

Driven User Guide: Searches, Saved Views, and Accessing Relevant Data

1. Overview of Monitored Applications: 1.1. Logging In

1.2. Status Views
2. Searches, Saved Views, and Accessing Relevant Data: 2.1. Starting a Search

2.2. Saving Search Queries as Views

2.3. My Teams Views

2.4. Case Examples of Searches, Views, and Teams

2.5. Customizing Searches

2.6. Periodic Views

2.7. Counter Data and Other Metrics in Tables

2.8. Sharing and Limiting Access to Application Information
3. Using the App Details Page: 3.1. Searching App Details

3.2. Viewing Application Details

3.3. Understanding the Directed Acyclic Graph

3.4. Viewing the Graph

3.5. Real-Time Visibility into Your Application

3.6. Details Table
4. Understanding the Unit of Work Details Page: 4.1. Viewing Unit-of-Work Details

4.2. The Directed Acyclic Graph

4.3. Step Table and Slice Histograms

4.4. Viewing the Hadoop Dashboard
5. Managing Applications with Tags: 5.1. Best Practice for Tags

5.2. Assigning Tags as Application-Level Properties

5.3. Assigning Tags in the Driven Plugin Configuration File

5.4. Using Tags in a Search Query

5.5. Finding Tagged Applications with the Column Chooser
6. Configuring Teams for Collaboration: 6.1. Creating and Managing Teams

6.2. Team Details

6.3. Associating an Application with a Team
7. Using Annotations: 7.1. Creating Custom Annotations

7.2. Data Visibility
8. Execute Hive Queries as Cascading HiveFlow: 8.1. Using HiveFlow

8.2. Driven for HiveFlow
9. Execute Cascading MapReduce Flows
10. User Profile: 10.1. User Actions

10.2. User Credentials

10.3. User Statistics

10.4. Invitations

10.5. Teams

Searches, Saved Views, and Accessing Relevant Data

The graphs of a Status View provide an overall profile of application processing. But you are also likely to want to explore applications in more depth and with a sharper focus than the displayed accumulation of all runtime information provides. The Driven Plugin collects a rich set of operational data with each application run, which is indexed in the persistence layer of the Driven server. Searching, filtering, and using saved views help focus on the dimensions of application runs that are of most interest to you.

You can save searches with particular filters and search terms as views so that you and other members of your Driven team can retrieve the search criteria and apply to data later with one click. Depending on how permissions are set, you might also be able to share views with users outside your team. When you save a view, you can choose to save it as a Status View or to focus on specific application runtimes by saving it as an Periodic View.

The circled elements in Searching and filtering controls of application data, including saved views show the areas of the Status View that allow you to filter what data populates the metrics on the page when you want to adjust the scope of reported performance data or to see different metrics. The Periodic View has the same filtering and search handles.

Figure 1. Searching and filtering controls of application data, including saved views

Note	An app details page also has search capabilities, but the search filters are different to better focus on finding relevant details after you identify an application run that you want to inspect. See Using the App Details Page.

Starting a Search

Searches that you can initiate at top of the Driven window support several application-, statement-, process-, and resource-level query attributes. This is the starting point for mining performance data if you do not have access to any helpful saved views in the navigation panel.

Note	You can use the asterisk (*) as a wildcard character. The search feature does not support other special characters ("",@, #, $, <, >, ?, etc.) in search strings and does not support spaces.

There are multiple parameters that you can invoke to query the application runs:

Application, statement, process, and resource metadata - You can use a filter to specify a category for your search term. Click the All drop-down menu to select a parameter. The available search filter parameters are shown in Search filter parameters.
Date - Click the All dates drop-down menu to select a predefined date range or to customize the date or dates.
Status - Click the Status drop-down menu to filter on an application-run parameter that is listed in Status filter parameters. The predefined Active states filter queries for applications that are in Pending, Started, Submitted, or Running state. The predefined Finished states filter queries for applications that are in Successful, Failed, or Stopped state.
Teams - Click the All Teams filter button to filter application data based on association with one or more Driven teams. See Configuring Teams for more information about how teams are associated with applications.

Figure 3. Status filter parameters

Tip	If the application ID, tag, or owner is not displayed in the Details Table of the search results, use the column chooser to bring these dimensions in view. Column chooser icon and excerpt of selectable attributes shows how to access column-chooser attributes.

Figure 4. Column chooser icon and excerpt of selectable attributes

Statement and Process ID Filters

You can search by filters that focus on components that are more granular than an application. Searches that are filtered by Statement and Process ID return application instances that have matching criteria on the step level. Information about the step level is on the Unit of Work Details page.

The Statement search filter is useful for finding applications containing select statements at the unit-of-work level. Generally, applications can contain SQL statements, such as Hive-based apps with Hive Query Language (HQL). For example, consider the select statement for the Hive Flow -CalculateAverageQuantity flow, which contains:

Tip	You can use the asterisk symbol (*) as the wildcard at the front and end of the text within the SELECT statement you want to search.

The Process ID search filter is helpful for finding applications with objects that are executed at the step level. The process ID is correlated with the job ID of Hadoop’s JobTracker.

Note	In a search using the Process ID filter, the wildcard character () is NOT* supported. Such a search would potentially return all components from every slice in all applications on the system, which can be a very expensive operation.

Resource Filters

The result set of searches filtered by Resource Identifier and Resource Field display applications that run with Resources matching the search criteria. Each application in the results has a Resource identifier or field name (depending on which filter is selected) that matches the string that is typed in the search field.

Resource identifier refers to a uniform resource identifier (URI) for the Resource. The Resource Identifier search filter returns applications that have processed data from a resource matching the identifier-string criteria as entered in the search field. A use case for a resource-identifier search is to discover which applications are accessing particular data resources. For example, a database administrator (DBA) who plans to bring down the system for maintenance could want to schedule a period for the outage by consulting with the people who run or use the affected database resources. A list of applications helps the DBA identify which users and groups access the data resources so that they can be contacted.

A Resource field is usually a descriptor for a field of sourced or sinked data. Searching with the Resource Field filter can help you detect whether or not applications are accessing particular types of data sets. Because a field is a grouping of data by a criterion, the field name in many cases reflects the nature of the data. For example, a field with the name "CC_number" could be used for a field containing credit card numbers.

The ability to easily diagnose application processing on the resource-field level can be useful in environments where a field is known to contain sensitive or confidential data that should not be exposed by running applications. By searching for the field name with the Resource Field filter selected, Driven parses runtime information to focus on whether or not applications process data from the field. Such a search can be a practical use case in an environment that needs to demonstrate or audit for data-security compliance. In this type of use case, the absence of search hits for an application with the given resource field indicates separation from the protected data set.

Saving Search Queries as Views

After searching for applications based on the parameters you have defined, you can save the query as a view. You can then return to the view to retrieve all applications that match the search criteria and filters. Search results of a saved view are dynamic, displaying applications with matching search parameters at the time that the view is opened.

When you save the view, select what information about the matched applications to display:

Status View highlights accumulated data about application states, graphically displaying the total numbers and relative frequencies of application statuses.
Periodic View provides canned quality-of-service statistics and heat maps by selectable timelines, which can serve as a gateway to discovering relevant details about repeated executions of specific applications (see [application-views]).

After you save a view, it appears as a link under a My Views heading in the navigation panel. If you do not see the link, you might need to expand STATUS VIEWS, PERIODIC VIEWS, or the appropriate My Views heading.

Figure 5. Saving a view

My Teams Views

Each link in the MY TEAMS area of the navigation panel is an entry point to a Status View that is associated with a team of which you are a member. A link in the MY TEAMS area of the side panel opens to a Status View that has the same parameters as the SHOW MY APPS page, except that the data is filtered to show only information that is correlated with the selected team.

The purpose of this type of view is to isolate metrics of application activity that is monitored by a single team. By opening a link under MY TEAMS, you get a broad view of a specific team’s application performance. The view can then be used as a gateway to further refine filter parameters or to start focused searches without extraneous data from applications that are associated with other teams.

Note	MY TEAMS links are generated automatically by virtue of belonging to teams and not actively "saved" by a user.

Case Examples of Searches, Views, and Teams

Joanne is a member of the ServerStar and EasternIT teams. She sees links for both of the teams under MY TEAMS. Other members of the ServerStar team see the same ServerStar link in their MY TEAMS navigation panels and access the same information if they click the view. The same applies to the EasternIT team.

Saving a Status View

After Joanne opens the ServerStar link under MY TEAMS, she finds the information useful in general but she wants to refine the filter parameters to focus on application status for the past five days.
She clicks the drop-down menu for filtering dates and selects Custom Date Range to set the date range to the last five days.
After naming and saving the search, the view link appears in the STATUS VIEWS > My Views section of the navigation panel.

Saving and Sharing an Periodic View

Joanne wants to troubleshoot why instances of an application named Cost_Analysis, which is administered by the EasternIT team, fail intermittently. She clicks the EasternIT link under MY TEAMS.
She searches the EasternIT Status View with the following parameters:
- App Name: Cost_Analysis
- Status: Failed
The graphs display failed application runs for the whole history stored by Driven.
As a way to gather the most recent root causes of application failure, Joanne further filters the view to 1 week.
To help herself and others find the relevant application execution details in Driven, Joanne saves the view as an Periodic View called Recent_CAFail.
She wants other people on the EasternIT team to see the data to help troubleshoot the problem. After navigating to STATUS VIEWS > My Views > Recent_CAFail in the navigation panel to share the view, she selects only the EasternIT team because no other users should access the application data. When EasternIT teammates are viewing Driven, they will see a Recent_CAFail link in the PERIODIC VIEWS > Team Views section of the navigation panel.

Note	See Sharing and Limiting Access to Application Information for more information about sharing views.

Customizing Searches

You can create custom searches for application runs that have specific attributes, such as having a certain time range for processing or populating a counter with a defined value range. Custom searches are based on Lucene query syntax, which is entered in the search field of a Status View or Application View.

As with searches that do not use Lucene query syntax, custom searches return applications that match your search-parameter values and that are associated with your Driven teams.

Table 1 shows some examples of the types of information that can be retrieved with a custom search and the query statements to obtain the application search results. Refer to the statement syntax examples in Table 1 for guidance on how to construct some types of Lucene queries. For detailed information about the required syntax in search queries, see Apache Lucene - Query Parser Syntax documentation.

Note	The letters in the query statement syntax are case-sensitive.

Table 1. Examples of Custom Search Goals and Statement Syntax
Application Attributes	Sample Values for Parameters	Statement Syntax
Processing duration; Tag identifier	Duration more than or equals 5 minutes; Tag identifier = production	`duration:[300000 TO *] AND tags:production`
Pending status time; Runtime	Pending time does not equal 0; Runtime > 0	`NOT pendingTime:0 AND (NOT runTime:0)`
Application name; Processing duration	Name = Cascading-Hive; Duration more than or equals 5 minutes	`name:Cascading-Hive* AND duration:[300000 TO *]`
Application name containing spaces	Name = sales region	`sales_region*`
Counter with a particular value	The BYTES_WRITTEN counter equals 814186673	`counters.org\:apache\:hadoop:\mapreduce\:lib\:output\: FileOutputFormatCounter.BYTES_WRITTEN:814186673`
Counters with a value in a particular range	The BYTES_WRITTEN counter equals or is greater than 814186000	`counters.org\:apache\:hadoop\:mapreduce\:lib\:output\: FileOutputFormatCounter.BYTES_WRITTEN:[814186000 TO *]`
Path to user directory	Path is /Users/smith/company/code/project	`userDir:\/Users\/smith\/company\/code\/project`

Note	You must prepend a custom search query for counter values with `counters`.

As also shown in the counter examples, backslashes are required as escape sequences to comment out colons in the paths so that they are not parsed as Lucene syntax.

Table 2 lists application attributes that are most relevant to searches in Driven. The most commonly used search targets, such as app name, are integrated in the Driven GUI so that you do not need to run a Lucene query to find matches. When an attribute can be located by a search filter or column chooser, Table 2 lists the part of the user interface that can be used to track the information.

Although an application attribute might be searchable with the GUI controls, you might prefer to query for the attribute and value in a Lucene query. This is particularly true when you want to find matches based on a range of values or if you want to run a complex query.

Elasticsearch truncates field values that exceed 16,000 characters. If a search parameter value includes characters that appear only in a string beyond the character limit (such as a very long classpath), Driven does not return a match because it is unable to search for possible matches after 16,000 characters of a string.

Table 2. Searchable Application Attributes
Attribute	Displayed in GUI?
(Duration and other time-based attributes are listed at bottom of table.)
Version	App Details page
classpath	No
command	Searchable with App Command filter; Displayed on App Details page
counter (Note: `counters.` must prepend the path in the query)	App Details and Unit-of-Work Details tables
finished	Searchable with Status filter
frameworks	App Details page
id (application ID)	Searchable with the App ID filter; Displayed as the last node of the URL for the App Details page
jarName	App Details page
jarPath	App Details page (click on JAR Information link to display the path)
javaClassVersion	No
javaCompiler	No
javaHome	No
javaInterpreterVersion	No
javaIoTmpdir	No
javaVendor	No
javaVendorUrl	No
jvmMaxMemory	No
localeCountry	No
messagingProtocolVersion	No
name	Searchable with App Name filter; Displayed on Periodic View and Application Details pages
osArch	No
osName	No
osVersion	No
owner	Searchable with the App Owner filter; Displayed on the App Details page
pid	No
pluginVersion	Displayed on the App Details page when you hover over the information icon
status	Searchable with one of the Status filters; Displayed in Status View and Periodic View
tags	Searchable with the App Tags filter; Displayed on the App Details page
type	Types that are identified in the Driven user interface are applications, units of work, and steps
userDir	No
userHome	No
userLanguage	No
userRegion	No
userTimezone	No
version	No
Time Attributes	All time attributes with corresponding values can be displayed in the App Details and Unit-of-Work Details tables, except for the lastCounterFetchTime and statusTime attributes.
Absolute-time attributes Values for these attributes are in milliseconds:	finishedTime lastCounterFetchTime pendingTime runTime startTime statusTime submitTime
Duration-time attributes Values for these attributes are in Unix time:	duration (duration = Amount of time from STARTED to FINISHED status. Equivalent to the startTime:finishedTime attribute.) pendingTime:finishedTime pendingTime:runTime pendingTime:startTime pendingTime:submitTime pendingTime:finishedTime runTime:finishedTime startTime:finishedTime startTime:runTime startTime:submitTime

Periodic Views

Monitoring and troubleshooting often entails breaking down runtime data to separate application instances and comparing them, especially for applications that run repeatedly. The Periodic View provides a platform for inspecting application runs both by discrete blocks of time and cumulatively.

After searching or filtering the information in a Status View such that the applications, time range, or other dimensions focus on what you want to explore in more depth, save the parameters as an Periodic View. Periodic View: Filtered to executions of an app in the last 3 months shows an example of the Periodic View, which is followed by a description of how to gather and customize information in the view. You can also create a new Periodic View by finding an application execution and clicking on its History button, and then saving the view it brings you to.

Figure 6. Periodic View: Filtered to executions of an app in the last 3 months

Tip

The Periodic View is ideal for monitoring instances of a single application that executes at periodic intervals. Adjust the search and filter criteria to focus on a particular application, and then save the criteria as an Periodic View for quick retrieval later. A look at the graph can provide a visual cue to spot application run outliers.

The Periodic View displays the following:

Summary Statistics: The Summary Statistics section of this page contains a graph and a table outlining application activity chronologically so that you can compare the activity among periods in one glance. Using the controls in the timeline selector area, you can filter the information that appears in the Interval Statistics, the App Execution Graph, and the Details Table to help focus on relevant data.
App Execution Graph: The App Execution Graph is an interactive railroad diagram of individual application runs that highlights anomalous setup, queueing, and execution durations for a periodic application. The graphical, interdependent representation of application performance metrics can help pinpoint problematic application runs and correlate instances with root causes of potential or existing bottlenecks.
Details Table: The Details Table is a table of execution details for current and past application runs that are linked to underlying metrics and contain hyperlinks to even more granular slice performance information.

Metric Graph

The metric graph plots a variety of metrics, including durations and the number of application executions. The gray shading represents the time range for which Driven displays data on the rest of the page. Nodes on the graph represent the beginning and ending of time intervals, the size of which is set with the buttons above and to the right of the graph. Navigating the metric graph to select time intervals illustrates how you can control the timeline.

Figure 7. Navigating the metric graph to select time intervals

Quality-of-Service Statistics

The Periodic View can automatically display a range of statistics about application execution. To display all metrics or specific metrics that Driven can calculate, click the configuration wheel above the statistics table ( app runtime settings icon ).

Table 3. Statistics of the Periodic View
Metric	Calculation Method
Failed Ratio	Ratio of number of FAILED app runs to number of all app runs
Mean Effective Parallelism	Total runtime of all slices divided by runtime of the app, averaged over all app instances
Mean Pending Duration	Average time between when apps enter PENDING status and when apps reach RUNNING status
Mean Periodicity	Average of time periods between when one app run enters PENDING status and the next app run enters PENDING status
Mean Running Duration	Average time between when apps enter RUNNING status and when apps reach a finished state
Mean Total Duration	Average time between when apps start (enter PENDING or RUNNING status) and when apps reach a finished state
Stop Ratio	Ratio of number of stopped app runs to number of all app runs
Total Count	Total number of app executions

After you select statistical metrics, the information is displayed in two ways:

Aggregated Statistics is an accumulation of your selected metrics for application run data that spans the entire period as specified in the search time parameters for the Periodic View.
Interval Statistics displays the metrics based on data only from the selected interval size.

The Application Execution Graph

The Application Execution Graph is a railroad diagram, which plots a dynamic representation of application executions over time. Each line in the graph represents a phase of an application run over the selected time interval: setup, queue, and execution. The setup phase is when an application is launched and is setting up its work. The queue phase is when the application has sent jobs to be executed on a cluster. Finally, the execution phase is when the cluster jobs are executing. These phases correspond to the application statuses Pending, Started, and Running in the search filters on the top of the page and the Details Table below.

Application phases that are still executing at the current time are rendered as dashed lines and phases that fail or are manually stopped are colored either red or orange, respectively.

Figure 8. Hovering over a line and clicking on it will redirect your browser to the App Details Page for that application execution

Gaining Insights from the Application Execution Graph

In this graph, each status is sloped proportionately to the amount of time that the application was in the state. State transitions on the graph enable visual cues for comparing how multiple instances of the same application perform because you can view relative times of runtime status.

Table 2 lists a few types of anomalous application behavior along with some possible causes to consider. The information in the table is not exhaustive and not applicable to every situation.

Table 4. Possible Causes for Anomalous Application Behavior
Unexpected Behavior	Possible Causes
App started as expected, but finishes later than expected	1) Cluster overload; 2) App processed exceptionally large data set
App started late, but finished on time	1) Dependency on another app that finished late; 2) Cluster overload
App not executed	1) Dependency on another app that did not complete successfully; 2) Scheduling error

Auto-Updated Data

Driven can refresh the displayed information as updates stream in from the plugin. Ensure that the Auto Update toggle in the top right corner is enabled to allow the displayed Driven data to auto-update in real time. If the Auto Update toggle is disabled, you must manually refresh the browser window to see real-time updates. Generally, this feature is useful if you want to monitor applications as they run.

Click the circle in the Auto Update slider to toggle between off and on.

Figure 9. Auto Update slider

Details Table

The Details Table under the graphs provides a breakdown of application execution data by instance of each application run. Use the table to drill down and gain insights to application performance from your cluster. Some key monitoring assets of the tabular interface include the following capabilities:

Export application-level tabular data to a tab-separated values text file
Add or remove metrics that are displayed
Click on a hyperlinked application name to view app data on more granular levels, including visualization of units of work and steps as directed acyclic graphs (DAGs)

Figure 10. First three rows of a sample table

The Driven page displays a maximum of 25, 50, or 100 rows. Use the Page Size drop-down menu if you want to change the maximum number of rows.

If the number of rows spans more than one page after you have set the Page Size to your preference, use the pagination arrows to navigate to other pages of the table.

Tip	You can reorder the columns by clicking column headings and dragging them to different locations. You can also sort the information in a column by ascending or descending order by clicking on the bidirectional arrow next to the column heading.

Track Applications by Various Metrics

Driven lets you customize most of the information that the table displays. Click the column chooser icon Counter Chooser to reveal or conceal columnar metrics. The Status and Name columns cannot be hidden.

The columnar metrics are categorized in the column chooser. Each category can be collapsed or expanded. This helps with retrievability because the number of choosable columns can grow large.

A key feature of the table and column chooser is the ability to import and view counter attributes. See Counter Data and Other Metrics in Tables for more information.

Exporting Data to a .tsv File

As part of your analytical process, the application data that is presented in a Driven table can be downloaded as a tab-separated values (.tsv) file, which then can populate a spreadsheet for detecting patterns, metrics, and usage.

Click the download icon tsv icon to capture the Driven table data and download it to a file.

Counter Data and Other Metrics in Tables

Tables map a lot of detailed information to all areas of application performance on Hadoop clusters. Tables show some basic information, which are listed as Common metrics in the column chooser. For example, a table on an Application View page provides application name and status in each row. You can also show or hide Time, Duration, and Tags (if applicable) metrics by using the column chooser.

Driven can provide deeper insights in the tables if you choose to display data from counters. Counters feed big data application execution metrics to the Driven Plugin from Hadoop, Cascading, and other frameworks. Standard counters are listed under the Counter Aliases heading in the column chooser. In addition, if custom counters are programmed in the application, Driven also surfaces these counters in the column chooser as importable attributes to your tables.

Raw counter data is usually about a very granular level of an application run. For example, a counter could exist to report on the number of user profiles that are running an application. But the counter might be reporting metrics from separate steps of application instances one-by-one. You receive accurate data about the number of user profiles running the application at step level, but you do not get a total number of user profiles running applications cumulatively. The problem with receiving fragmented data like this is that you have an incomplete picture of the application. In addition, Hadoop holds on to the counter data for a short period of time. Without a storage mechanism, people and systems do not have one-stop access to historical data. The best workaround is to check logs, which is unwieldy and time-consuming.

A value of Driven is that it aggregates all the data reported from a counter into an application level, while also reporting on the unit of work and step level if the user wants to stay on the granular level. Driven maintains the historical data and makes it easily, dynamically retrievable. Driven allows you to filter so that you can slice and dice what you see from the counter data. On a higher level, you can easily compare counter data from one application to another application.

Note	For information about importing counters to tables in the slice performance view, see Understanding the Unit of Work Details Page.

To select or remove metrics in the displayed table:

Click the Select table columns icon.
Indicate which metrics to display in the Details Table by selecting and de-selecting columns. You might need to expand counter group headings to display particular attributes that you want to include or exclude. You might also need to minimize a group of metric attributes to more easily view the full list. See Column chooser window example for examples.
Click UPDATE.

Figure 11. Column chooser window example

The following sections provide a brief explanation of the column metrics:

Common

The common metrics vary depending on whether you are viewing a table listing application runs or units of work.

Effective Parallelism - Total runtime of all slices divided by runtime of the app, averaged over all app instances.
Host Name - The name of the machine where the application jar was launched.
ID - The numeric identifier (ID) of the application.
IP Address - The IP address of the host where the application ran.
JVM Max Memory - The JVM Max Memory settings where the application ran.
Owner - The owner (by name) of the application.
PID - The PID of the process where the application ran.
Slide Rate - A small line graph that plots the changing number of slices involved in an active application at different points over a short period of time. The slice rate is depicted by the vertical fluctuations of the line. This column attribute also displays a timestamped count of slices for the last point in time range for the slice rate line graph.
Team - The team name that is associated with the application. The Owner of the team or appointed Team Leader is able to manage the team member details through a link in this column.
UoW Active Count - The currently active OuW (units of work) in the application.
UoW Count - The total number of UoW in the application.
UoW Failed Count - The number of failed OuW in the application.
Version - The version of the application as set by the developer.
Timeline - A timeline representation of each row with color coding to reflect the status. Hover over the graphed bar to display a pop-up of the status information.

Tip	Slice Rate is displayed only for an application instance that is still executing as of the time range endpoint for the displayed data. If you have Auto Update turned ON and an application is executing, the slice rate refreshes itself in real time until the app reaches a Finished state.

Time

Pending Time - The point in time when an application JVM is started up or a unit of work is created.
Start Time - The point in time when the first unit of work was requested to begin executing on the cluster for an application, or when the unit of work was requested to begin execution.
Submit Time - This is when the unit of work is actually submitted to the cluster for execution.
Run Time - The point in time when the first unit of work begins executing on the cluster for an application, or when the current unit of work began execution.
Finished Time - The point in time when the application JVM or unit of work completes or fails.

Counter Aliases

Many Apache Hadoop ecosystem projects, like Hadoop MapReduce, Apache YARN, and Apache Tez, provide counter names that are slightly different from other platforms. In many cases, the counter "key name" may be the same across the platforms, but the "group name" may live under a different name making comparisons across platforms difficult and tedious.

Counter aliases provide a shorthand for getting equivalent counter metrics across platforms into a single table column for easier comparison.

Bytes Read - The total number of bytes read during the execution of the application across all file systems — an alias for *.FileSystemCounter.*_BYTES_READ counters.
Bytes Written - The number of bytes written during the execution of the application across all file systems — an alias for */FileSystemCounter.*_BYTES_WRITTEN counters.
CPU Time - The total amount of CPU time (in milliseconds) across all slices for the application or unit of work being displayed — an alias for *.CPU_MILLISECONDS counters.
Records Read - The total number of records read during the execution of the application or unit of work — an alias for *.MAP_INPUT_RECORDS and *.REDUCE_INPUT_RECORDS.
Records Written - The total number of records written during the execution of the application or unit of work — an alias for *.MAP_OUTPUT_RECORDS, *.REDUCE_OUTPUT_RECORDS, or for Tez *.OUTPUT_RECORDS
Task Retries - The number of retries that Hadoop executed for a task, when known.

Additionally, many counters are framework specific. A few examples of counters for the Cascading framework are listed below.

CoGroup Tuples Spilled - The number of Tuples spilled to disk during the Cascading CoGroup operation — this metric can help determine if the spill threshold should be increased or decreased.
HashJoin Tuples Spilled - The number of Tuples spilled to disk during the Cascading HashJoin operation — this metric can help determine if the spill threshold should be increased or decreased.
Tuples Read - The number of Tuples read during the execution of the Cascading application or unit of work.
Tuples Trapped - The number of trapped tuples (data placed into an output file because of error in processing) during the execution of the Cascading application or unit of work.
Tuples Written - The number of Tuples written during the execution of the Cascading application or unit of work.

Duration

The Duration metric is the product of measured time during a particular state and the matrix of ensuing states when processing the application.

Duration - The time when the application is in the Started status subtracted from the time when the application reaches a finished state.
Pending:Run Duration - The time when the application is in the Pending status subtracted from the time when the application is in the Running status.
Pending Duration - The total time that the application is in Pending status.
Pending:Submit Duration - The time when the application is in the Pending status subtracted from the time when the application is in the Submitted status.
Running Duration - The total time that the application is in the Running status.
Start:Finish Duration - The time when the application is in the Started status subtracted from the time when the application reaches a finished state.
Start:Run Duration - The time when the application is in the Started status subtracted from the time when the application is in the Running status.
Started Duration - The total time that the application is in the Started status.
Submitted:Finish Duration - The time when the application is in the Submitted status subtracted from the time when the application reaches a finished state.
Submitted Duration - The total time that the application is in the Submitted status.
Total Duration - The time when the application is in the Pending status subtracted from the time when the application reaches a finished state.

Tip	See Understanding the Driven State Mode documentation for more information about semantics for the counters.

As a member of a Driven team, you have access to all aspects of runtime metrics of an application that is associated with the team. You also can share a view of data that you create and save with other Driven users who belong to one or more of the same teams that you do.

Depending on the sharing permissions that are configured at your site, you can also share a view with users who do not belong to the same teams as you do.

There are two types of access to information that can be shared across teams:

Discoverability access to an application lets users see the Status View and Periodic View information for the application.
Visibility access to an application lets users see application details and unit of work details.

The access levels are not mutually dependent. If an administrator or team leader enables or disables discoverability, this does not affect visibility, and vice versa.

See Configuring Access Levels for details about how a Driven administrator or team leader can set discoverability and visibility permissions.

After you save a view, you can share it with other Driven users. When you share a view, you must either select some or all of your teams or grant access to all other users of the Driven deployment.

To share a view:

Hover over the filter bar to expand it
Click the share button ().
Select how you want to share the view:
- If you want to share with all users who can log in to Driven, select Public.
- If you want to share the view with teammates only, select Teams and one or more of the entries in the list.
- If you prefer to share the view more selectively, select Link.
Click Share.
If you selected Teams or Public sharing, the saved view appears in the Team Views or Public Views list of the other users. If you selected Link sharing, then copy the URL that is generated in the window and send it to other users of your Driven deployment who need access to the view. (Selecting Link sharing does not populate the saved views that appear in the side bars of other users.)

Note

Discoverability and visibility settings on an application take precedence over how you share a view. As a result, views that are shared with other teams, marked public, or saved as distributed links might contain applications that are not discoverable or visible by users outside your team. Contact the team leader or system administrator if you have concerns about the permission settings.

After sharing a view, you can either change who else can access the view or revoke access from all other users. To modify the share settings, click the share button to configure and submit changes to view access.

Saved views in the header shows how saved views are categorized and arranged in the header of your Driven window.

Figure 12. Saved views in the header

Alternatively, you can hover click Saved Views in the header to open a separate page listing all saved views that you can access.

Configuring Access Levels

If the Driven Server is configured to allow a team leader or the system administrator to control discoverability or visibility, then the access level can be controlled on the team details page. See Configuring Teams For Collaboration.

For information about how the access levels can be configured for the whole system when the Driven Server is deployed, see the Access to Monitored Application Information section in the Driven Administrator Guide.

The following list outlines all the permissions settings in Driven pertaining to discoverability and visibility:

Driven Server configuration parameters
- Establish a default setting for discoverability
- Establish a default setting for visibility
- Enable a Driven admin to change the default system-wide discoverability setting on a team-by-team basis
- Enable a Driven admin to change the default system-wide visibility setting on a team-by-team basis
Team settings (only possible if Driven Server configuration allows change of access settings on team details page)
- Change default discoverability setting that was set during Driven deployment
- Change default visibility setting that was set during Driven deployment