Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-11889

Visualization of Pipeline Summary Metrics

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: UI
    • Labels:
    • Sprint:
      UP5 Sprint 2
    • Release Notes:
      Adds visualization for pipeline in UI. This helps visualizing runs, logs/warnings and data flowing through each node for each run in the pipeline.
    • Rank:
      1|hzy13a:ai

      Description

      As a user, I want to be able to view relevant metrics related to a pipeline run history, after the pipeline has been created.
      In the Pipeline Details view, I select "summary" to open a modeless window.
      The modeless is organized as follows:

      • Summary Header, which contains:
        • The total number of runs the pipeline has done (both success and failures)
        • Run times: Minimum (expressed in hrs/min/sec); Maximum (expressed in hrs/min/sec) and Average Run Time (expressed in hrs/min/sec).
        Success and failure runs are taken into consideration
        • Current Schedule: surface a summarized version of the current schedule which includes, frequency of runs (every 5 min, every x hours, every x days, etc), time of runs (# or minutes past the hour) and concurrency information
      • "View" history dropdown: As a user, I want to be able to select which groups of runs I want to display. I can select the following pre-selected options:
        BY RUN #
        • Last 10 runs: x axis displays last 10 run numbers e.g. 100; 101; 102; 103; 104; 105; 106; 107; 108; 109; 110
        • Last 50 runs: x axis displays last 50 runs organized as follows: 100; 105; 110; 115; 120; 125; 130; 135; 140; 145; 150
        • Last 100 runs: organized as follows: 100; 110; 120; 130; 140; 150; 160; 170; 180; 190; 200
        BY TIME
        • Last 24 hours: it displays all the runs in the last 24 hours. The x axis is organized to surface hours at intervals of 4 hours (e.g. 00:00am; 04:00am; 08:00am; 12:00pm; 4:00pm; etc.).
        • Last 7 days: it displays the runs in the last 7 days. The x axis displays every other day of the week and dates (e.g. Mon 10/10/17; Wed 10/12/17; etc).
        • Last 30 days: it displays the runs in the last 30 days. The x axis displays first day or each week. (e.g. Mon 10/10/17; Mon 10/18/17; etc).
        • Since Inception: it displays all the runs since the creation of the pipeline. The x axis dynamically displays by either months, weeks, days or hours depending from the life-length of the pipeline.
      • Last Updated: The metrics are updated in realtime. The "Last Updated" displays the day and time of the last update
      • Displays the following metrics:
        • Run History
        • Logs Errors and Warnings
        • Numbers of Records In
        • Numbers of Records Out

      Refer to https://invis.io/E7C3OMF5K#/238236490_Pipeline_Summar_-_Chart_Copy_5

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ajai Ajai Narayan
                Reporter:
                lea Lea
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: