Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-12514

PartitionConsumer onFinish should be called within the MR's transaction

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: MapReduce
    • Labels:
      None
    • Rank:
      1|i007dr:

      Description

      PartitionConsumer onFinish should be called within the MR's transaction.
      Currently, it runs in its own transaction in the MapReduce's destroy method.
      Because of this, it may actually fail, whereas the MapReduce has actually consumed those partitions and committed its output.

        Attachments

          Activity

            People

            • Assignee:
              bhooshan Bhooshan Mogal
              Reporter:
              ali.anwar Ali Anwar
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: