Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-6099

Writing to dataset in mapper destroy method does not work

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.4.0
    • Fix Version/s: None
    • Component/s: MapReduce
    • Labels:
    • Rank:
      1|hzzdrb:

      Description

      I have a mapper that implement ProgramLifeCycle<MapReduceTaskContext> and I am attempting to write to a Table in the mapper’s destroy method. The writes to the table are not available when I scan the table later. I have also tried calling close on the table and that does not fix the problem. I have tried moving the code to the mapper's cleanup method and there is a NPE when the dataset is accessed.

      My setup is a workflow where my MapReduce job runs first and writes to a local dataset created by the workflow and then a workflow action is executed next and attempts to read from the table. If I write to the table in the mapper's map method the workflow action is able to read those rows.

        Attachments

          Activity

            People

            • Assignee:
              andreas Andreas Neumann
              Reporter:
              jd565c Jeff Dix
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: