Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.2.0
    • Component/s: App Fabric
    • Labels:
    • Release Notes:
      Added support for Spark2. Currently, CDAP can only support a single version of Spark. In environments where multiple Spark versions exist, CDAP must be configured to use one or the other.
    • Rank:
      1|hzzqmf:

      Description

      After disabling Explore, starting CDAP Master we're unable to load Spark plugins:

      2016-12-15 22:23:21,674 - WARN  [Endure-Service-:c.c.c.i.a.r.a.ArtifactRepository@708] - Could not add system artifact 'spark-plugins-1.5.0-SNAPSHOT.jar' because it is invalid.
      co.cask.cdap.common.InvalidArtifactException: Class could not be found while inspecting artifact for plugins. Please check dependencies are available, and that the correct parent artifact was specified. Error class: class java.lang.NoClassDefFoundError, message: org/apache/spark/api/java/function/Function.
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactInspector.inspectPlugins(ArtifactInspector.java:249) ~[na:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactInspector.inspectArtifact(ArtifactInspector.java:135) ~[na:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository.inspectArtifact(ArtifactRepository.java:583) [na:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository.addArtifact(ArtifactRepository.java:473) [na:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository.addSystemArtifact(ArtifactRepository.java:697) [na:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactRepository.addSystemArtifacts(ArtifactRepository.java:676) [na:na]
              at co.cask.cdap.internal.app.runtime.artifact.SystemArtifactLoader$1$1.doStart(SystemArtifactLoader.java:44) [na:na]
              at com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) [com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.common.service.RetryOnStartFailureService$1.run(RetryOnStartFailureService.java:63) [na:na]
      Caused by: java.lang.NoClassDefFoundError: org/apache/spark/api/java/function/Function
              at java.lang.Class.getDeclaredFields0(Native Method) ~[na:1.7.0_75]
              at java.lang.Class.privateGetDeclaredFields(Class.java:2499) ~[na:1.7.0_75]
              at java.lang.Class.getDeclaredFields(Class.java:1811) ~[na:1.7.0_75]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactInspector.getProperties(ArtifactInspector.java:412) ~[na:na]
              at co.cask.cdap.internal.app.runtime.artifact.ArtifactInspector.inspectPlugins(ArtifactInspector.java:238) ~[na:na]
              ... 8 common frames omitted
      Caused by: java.lang.ClassNotFoundException: org.apache.spark.api.java.function.Function
              at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_75]
              at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_75]
              at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_75]
              at java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_75]
              at co.cask.cdap.common.lang.InterceptableClassLoader.findClass(InterceptableClassLoader.java:46) ~[na:na]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_75]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_75]
              ... 13 common frames omitted
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ashau Albert Shau
                Reporter:
                chris Chris Gianelloni
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: