Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-11113

Add a way to configure number of mappers for db source

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.0
    • Component/s: Pipeline Plugins, Pipelines
    • Labels:
      None
    • Release Notes:
      Added a numSplits config property to the Database source to allow users to configure how many splits should be used for the import query.
    • Rank:
      1|hzz7cv:

      Description

      It would be good to be able to specify the number of splits to generate in the db source. This can help prevent lots of small files on hdfs.

        Attachments

          Activity

            People

            • Assignee:
              ashau Albert Shau
              Reporter:
              ashau Albert Shau
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: