Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-11125

Joiner plugin is failing, when using Spark engine

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.5.0
    • Fix Version/s: 3.5.1
    • Labels:
      None
    • Release Notes:
      Fixed an issue where the Joiner plugin was failing in Hydrator pipelines executing in a Spark environment.
    • Rank:
      1|hzzl3j:

      Description

      I'm using the Joiner plugin, with Spark engine. It fails; see below for stacktrace.

      2016-08-25 14:40:09,225 - ERROR [Driver:o.a.s.d.y.ApplicationMaster@95] - User class threw exception: org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: javax/ws/rs/BadRequestException
      org.apache.tephra.TransactionFailureException: Exception raised in transactional execution. Cause: javax/ws/rs/BadRequestException
              at co.cask.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:87) ~[co.cask.cdap.cdap-data-fabric-3.5.0.jar:na]
              at co.cask.cdap.data2.transaction.Transactions.asTransactionFailure(Transactions.java:73) ~[co.cask.cdap.cdap-data-fabric-3.5.0.jar:na]
              at co.cask.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:216) ~[co.cask.cdap.cdap-spark-core-3.5.0.jar:na]
              at co.cask.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:133) ~[co.cask.cdap.cdap-spark-core-3.5.0.jar:na]
              at co.cask.cdap.app.runtime.spark.DefaultSparkExecutionContext.execute(DefaultSparkExecutionContext.scala:153) ~[co.cask.cdap.cdap-spark-core-3.5.0.jar:na]
              at co.cask.cdap.app.runtime.spark.DefaultJavaSparkExecutionContext.execute(DefaultJavaSparkExecutionContext.scala:79) ~[co.cask.cdap.cdap-spark-core-3.5.0.jar:na]
              at co.cask.cdap.etl.spark.batch.BatchSparkPipelineDriver.run(BatchSparkPipelineDriver.java:78) ~[hydrator-spark-core-3.5.0.jar:na]
              at co.cask.cdap.app.runtime.spark.SparkMainWrapper$.main(SparkMainWrapper.scala:103) ~[co.cask.cdap.cdap-spark-core-3.5.0.jar:na]
              at co.cask.cdap.app.runtime.spark.SparkMainWrapper.main(SparkMainWrapper.scala) ~[co.cask.cdap.cdap-spark-core-3.5.0.jar:na]
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_67]
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_67]
              at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_67]
              at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542) ~[spark-assembly.jar:na]
      Caused by: java.lang.NoClassDefFoundError: javax/ws/rs/BadRequestException
              at java.lang.Class.getDeclaredFields0(Native Method) ~[na:1.7.0_67]
              at java.lang.Class.privateGetDeclaredFields(Class.java:2436) ~[na:1.7.0_67]
              at java.lang.Class.getDeclaredField(Class.java:1946) ~[na:1.7.0_67]
              at co.cask.cdap.internal.lang.Fields.findField(Fields.java:39) ~[co.cask.cdap.cdap-api-3.5.0.jar:na]
              at co.cask.cdap.internal.app.runtime.plugin.PluginInstantiator.newInstance(PluginInstantiator.java:206) ~[co.cask.cdap.cdap-app-fabric-3.5.0.jar:na]
              at co.cask.cdap.internal.app.runtime.DefaultPluginContext.newPluginInstance(DefaultPluginContext.java:95) ~[co.cask.cdap.cdap-app-fabric-3.5.0.jar:na]
              at co.cask.cdap.internal.app.runtime.AbstractContext.newPluginInstance(AbstractContext.java:322) ~[co.cask.cdap.cdap-app-fabric-3.5.0.jar:na]
              at co.cask.cdap.app.runtime.spark.SparkPluginContext.newPluginInstance(SparkPluginContext.java:68) ~[co.cask.cdap.cdap-spark-core-3.5.0.jar:na]
              at co.cask.cdap.etl.spark.SparkPipelineDriver.runPipeline(SparkPipelineDriver.java:145) ~[hydrator-spark-core-3.5.0.jar:na]
              at co.cask.cdap.etl.spark.batch.BatchSparkPipelineDriver.run(BatchSparkPipelineDriver.java:93) ~[hydrator-spark-core-3.5.0.jar:na]
              at co.cask.cdap.app.runtime.spark.SparkTransactional$2.run(SparkTransactional.java:225) ~[co.cask.cdap.cdap-spark-core-3.5.0.jar:na]
              at co.cask.cdap.app.runtime.spark.SparkTransactional.execute(SparkTransactional.java:198) ~[co.cask.cdap.cdap-spark-core-3.5.0.jar:na]
              ... 11 common frames omitted
      Caused by: java.lang.ClassNotFoundException: javax.ws.rs.BadRequestException
              at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_67]
              at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_67]
              at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_67]
              at java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_67]
              at co.cask.cdap.common.lang.InterceptableClassLoader.findClass(InterceptableClassLoader.java:36) ~[co.cask.cdap.cdap-common-3.5.0.jar:na]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_67]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_67]
              ... 23 common frames omitted
      

        Attachments

          Activity

            People

            • Assignee:
              vinisha Vinisha Shah
              Reporter:
              ali.anwar Ali Anwar
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: