Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-3742

Metrics for mapreduce jobs are not correct while multiple sinks are used

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: ETL
    • Labels:
      None
    • Release Notes:
      Fix a bug where certain MapReduce metrics were not being properly emitted when using multiple outputs.
    • Rank:
      1|hzz07j:

      Description

      While using multiple sinks - the ones that use Outputformats (eg: s3). The mapper output metrics are incorrect. The map input records show the right number but the map output records are not.

      To reproduce - create an ETL batch reading from stream, writing to S3 and Table.

        Attachments

          Activity

            People

            • Assignee:
              ali.anwar Ali Anwar
              Reporter:
              sree Sreevatsan Raman
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: