Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-7624

Readless Increments do not work from MapReduce

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.5.1
    • Fix Version/s: 4.0.0
    • Component/s: Datasets, MapReduce
    • Labels:
    • Release Notes:
      Fixes an issue where readless increments from different MapReduce tasks cancel out each other.
    • Rank:
      1|hzzohz:

      Description

      The following table:

          createDataset("counts", Table.class,
                        DatasetProperties.builder().add(Table.PROPERTY_READLESS_INCREMENT, "true").build());
      

      written to from a mapper:

          @Override
          protected void map(Long key, String value, Context context)
            throws IOException, InterruptedException {
            table.increment(new Increment("counts").add("incr", 1L));
            context.write(ONE, ONE);
          }
      

      The increments should not cancel each other out, because readless increments are enabled. But for N mappers, I only see the effect of the increments of one of them.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                andreas Andreas Neumann
                Reporter:
                andreas Andreas Neumann
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: