Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-14342

ExploreTest is consistently failing on MapR4.1

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.1.0
    • Fix Version/s: 5.1.0
    • Component/s: Explore
    • Labels:
      None
    • Rank:
      1|i00hrj:

      Description

      The ExploreTest integration test is consistently failing on MapR4.1 here (5 out of 5 times so far):
      https://github.com/caskdata/cdap-integration-tests/blob/develop/integration-test-remote/src/test/java/co/cask/cdap/apps/explore/ExploreTest.java#L100-L102

      I took a look at the explore logs and I see this NullPointerException:

      Diagnostic Messages for this Task:
      Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"word":"bar","count":1,"ts":1537347835870}
      	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:185)
      	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
      	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:435)
      	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:347)
      	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
      	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"word":"bar","count":1,"ts":1537347835870}
      	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:503)
      	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:176)
      	... 8 more
      Caused by: java.lang.NullPointerException
      	at co.cask.cdap.hive.serde.ObjectSerializer.serialize(ObjectSerializer.java:86)
      	at co.cask.cdap.hive.datasets.DatasetSerDe.serialize(DatasetSerDe.java:174)
      	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:660)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
      	at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
      	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
      	at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
      	at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
      	at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:493)
      	... 9 more 

      This regression is as of https://github.com/caskdata/cdap/pull/10660,which introduces the line of code which causes the NullPointerException on MapR 4.1:
      https://github.com/caskdata/cdap/blob/release/5.1/cdap-explore/src/main/java/co/cask/cdap/hive/serde/ObjectSerializer.java#L86.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vinisha Vinisha Shah
                Reporter:
                ali.anwar Ali Anwar
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: