Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-12040

On Standalone, CDAP Master dies if MR/Spark pipeline fails with ClassNotFoundException

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 6.1.0
    • Component/s: Master
    • Labels:
    • Rank:
      1|i004jr:

      Description

      On Standalone, CDAP Master dies if MR/Spark pipeline fails with ClassNotFoundException.

      2017-07-05 12:20:43,364 - DEBUG [WorkflowDriver:c.c.c.c.l.c.UncaughtExceptionHandler@42] - Uncaught exception in thread Thread[WorkflowDriver,5,appfabric-executor-thread]
      java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.avro.LogicalType
              at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.executeAll(WorkflowDriver.java:596) ~[na:na]
              at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.run(WorkflowDriver.java:572) ~[na:na]
              at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52) ~[com.google.guava.guava-13.0.1.jar:na]
              at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_131]
      Caused by: java.lang.ClassNotFoundException: org.apache.avro.LogicalType
              at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_131]
              at co.cask.cdap.common.lang.InterceptableClassLoader.findClass(InterceptableClassLoader.java:46) ~[na:na]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_131]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[na:1.8.0_131]
              at org.apache.parquet.avro.AvroWriteSupport.init(AvroWriteSupport.java:117) ~[na:na]
              at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:389) ~[na:na]
              at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:350) ~[na:na]
              at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1107) ~[na:na]
              at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1091) ~[na:na]
              at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66) ~[na:na]
              at org.apache.spark.scheduler.Task.run(Task.scala:89) ~[na:na]
              at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214) ~[na:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_131]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_131]
      

        Attachments

          Activity

            People

            • Assignee:
              poorna Poorna Chandra
              Reporter:
              dshau David Shau
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: