Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-9091 Revisit macro handling in pipelines
  3. CDAP-11541

Configure - setting up Resources in both Batch and Realtime pipelines

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.2.0
    • Component/s: UI
    • Labels:
    • Sprint:
      UP4 Sprint 1, UP4 Sprint 2, UP4 Sprint 3
    • Rank:
      1|i001nz:

      Description

      As a CDAP pipeline user, I would like to specify the resources for my pipeline, so that I can specify dynamically allocate/deallocate resources for my pipeline when more/less resources are required.

      Specs:

      • BATCH MAPREDUCE:
        Label: Driver; Pipeline Spec Property: driverResources
        Tooltip: Resources for the MapReduce driver process, which initializes the pipeline
        Label: Executor (Mapper/Reducer); Pipeline Spec Property: resources
        Tooltip: Resources for Map and Reduce Tasks of the MapReduce program
      • BATCH SPARK:
        Label: Driver; Pipeline Spec Property: driverResources
        Tooltip: Resources for the Apache Spark driver process which initializes the pipeline
        Label: Executor; Pipeline Spec Property: resources
        Tooltip: Resources for executor processes which run tasks in an Apache Spark pipeline.
      • DATA STREAMS:
        Label: Driver; Pipeline Spec Property: driverResources
        Tooltip: Resources for the Apache Spark driver process which initializes the pipeline
        Label: Client; Pipeline Spec Property: clientResources
        Tooltip: Resources for the Apache Spark client process
        Label: Executor; Pipeline Spec Property: resources
        Tooltip: Resources for executor processes which run tasks in an Apache Spark Streaming pipeline.

        Attachments

          Activity

            People

            • Assignee:
              tbach Tony Bach
              Reporter:
              lea Lea
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: