Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-9914

Hydrator warning is actually an error

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: Pipeline Plugins, Pipelines
    • Labels:
      None
    • Release Notes:
      Batch Hydrator pipelines will now log an error instead of a warning if they fail in the sdk.
    • Rank:
      1|hzzd2v:

      Description

      This exception stops the pipeline but is being logged as a warning instead of an error.

      2016-05-11 21:55:19,720 - WARN  [Thread-254:o.a.h.m.LocalJobRunnerWithFix@562] - Error cleaning up job: job_local1176050516_0008
      java.lang.Exception: java.lang.IllegalArgumentException: Row key cannot be null.
      	at org.apache.hadoop.mapred.LocalJobRunnerWithFix$Job.runTasks(LocalJobRunnerWithFix.java:465) ~[co.cask.cdap.cdap-app-fabric-3.4.0.jar:na]
      	at org.apache.hadoop.mapred.LocalJobRunnerWithFix$Job.run(LocalJobRunnerWithFix.java:524) ~[co.cask.cdap.cdap-app-fabric-3.4.0.jar:na]
      java.lang.IllegalArgumentException: Row key cannot be null.
      	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) ~[guava-13.0.1.jar:na]
      	at co.cask.cdap.format.RecordPutTransformer.createPut(RecordPutTransformer.java:147) ~[cdap-formats-3.4.0.jar:na]
      	at co.cask.cdap.format.RecordPutTransformer.toPut(RecordPutTransformer.java:72) ~[cdap-formats-3.4.0.jar:na]
      	at co.cask.hydrator.plugin.batch.sink.TableSink.transform(TableSink.java:98) ~[1463028914987-0/:na]
      	at co.cask.hydrator.plugin.batch.sink.TableSink.transform(TableSink.java:43) ~[1463028914987-0/:na]
      	at co.cask.cdap.etl.common.TrackedTransform.transform(TrackedTransform.java:59) ~[cdap-etl-core-3.4.0.jar:na]
      	at co.cask.cdap.etl.common.TransformExecutor.executeTransformation(TransformExecutor.java:86) ~[cdap-etl-core-3.4.0.jar:na]
      	at co.cask.cdap.etl.common.TransformExecutor.executeTransformation(TransformExecutor.java:90) ~[cdap-etl-core-3.4.0.jar:na]
      	at co.cask.cdap.etl.common.TransformExecutor.executeTransformation(TransformExecutor.java:90) ~[cdap-etl-core-3.4.0.jar:na]
      	at co.cask.cdap.etl.common.TransformExecutor.executeTransformation(TransformExecutor.java:90) ~[cdap-etl-core-3.4.0.jar:na]
      	at co.cask.cdap.etl.common.TransformExecutor.runOneIteration(TransformExecutor.java:49) ~[cdap-etl-core-3.4.0.jar:na]
      	at co.cask.cdap.etl.batch.mapreduce.TransformRunner.transform(TransformRunner.java:154) ~[cdap-etl-batch-3.4.0.jar:na]
      	at co.cask.cdap.etl.batch.mapreduce.ETLMapReduce$ETLMapper.map(ETLMapReduce.java:299) ~[cdap-etl-batch-3.4.0.jar:na]
      	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na]
      	at co.cask.cdap.internal.app.runtime.batch.MapperWrapper.run(MapperWrapper.java:117) ~[co.cask.cdap.cdap-app-fabric-3.4.0.jar:na]
      	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na]
      	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na]
      	at org.apache.hadoop.mapred.LocalJobRunnerWithFix$Job$MapTaskRunnable.run(LocalJobRunnerWithFix.java:243) ~[co.cask.cdap.cdap-app-fabric-3.4.0.jar:na]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_79]
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_79]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_79]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_79]
      	at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_79]
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ashau Albert Shau
                Reporter:
                russellsavage Russ Savage
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: