Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-1887

Mapreduce cannot accept custom classes as mapper output class

    Details

    • Rank:
      1|hzypiv:

      Description

      When doing so, the following exception is thrown in the mapper:

      2015-03-20 18:52:29,513 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.io.IOException: Type mismatch in value from map: expected co.cask.cdap.examples.sportresults.ScoreCounter$GameStat, received co.cask.cdap.examples.sportresults.ScoreCounter$GameStat
              at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1054)
              at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:692)
              at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
              at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
              at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
              at co.cask.cdap.examples.sportresults.ScoreCounter$ResultsMapper.map(ScoreCounter.java:112)
              at co.cask.cdap.examples.sportresults.ScoreCounter$ResultsMapper.map(ScoreCounter.java:99)
              at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
              at co.cask.cdap.internal.app.runtime.batch.MapperWrapper.run(MapperWrapper.java:109)
              at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
              at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
              at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:415)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594)
              at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
      

        Attachments

          Activity

            People

            • Assignee:
              terence Terence Yim
              Reporter:
              julien Julien Guery
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: