Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-9091 Revisit macro handling in pipelines
  3. CDAP-11524

Configure - setting up Pipeline Config for both Batch and Realtime Pipelines

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.2.0
    • Component/s: UI
    • Labels:
    • Sprint:
      UP4 Sprint 1, UP4 Sprint 2, UP4 Sprint 3
    • Rank:
      1|i001k7:

      Description

      As a user, I want to be able to set configuration to a pipeline:

      BATCH PIPELINE
      Stage Logging - stageLoggingEnabled - true/false
      Tooltip: Allows logs from each stage in the pipeline to be queried individually.
      Instrumentation - processTimingEnabled - true/false
      Tooltip: Emits timing metrics such as total time, mean, standard deviation for pipeline stages.

      REALTIME PIPELINE
      Stage Logging - stageLoggingEnabled - true/false
      Tooltip: Allows logs from each stage in the pipeline to be queried individually.
      Instrumentation - processTimingEnabled - true/false
      Tooltip: Emits timing metrics such as total time, mean, standard deviation for pipeline stages.
      Checkpointing - disableCheckpoints - true/false - Note that this should be set to true if the switch is disabled.
      Tooltip: Allows Apache Spark Streaming to checkpoint data (RDDs) to persistent storage so that the pipeline can recover from failures
      Graceful Stop - stopGracefully - true/false
      Tooltip: When the pipeline is stopped, waits for the processing of all received data to be completed

        Attachments

          Activity

            People

            • Assignee:
              tbach Tony Bach
              Reporter:
              lea Lea
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: