Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-11335

Support application written scala

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.1.0, 4.0.0
    • Fix Version/s: None
    • Component/s: App Fabric, Spark
    • Labels:
      None
    • Rank:
      1|i000hr:

      Description

      User writing a spark program might choose to write other related classes such as application and program config classes in scala. This seems to fail because we don't expose scala classes everywhere. For example, a workflow whose config class is written in scala fail with following error:

      java.lang.NoClassDefFoundError: scala/collection/Seq
              at java.lang.Class.forName0(Native Method) ~[na:1.7.0_79]
              at java.lang.Class.forName(Class.java:274) ~[na:1.7.0_79]
              at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.initializeWorkflow(WorkflowDriver.java:208) ~
              at co.cask.cdap.internal.app.runtime.workflow.WorkflowDriver.startUp(WorkflowDriver.java:203) ~[na:na]
              at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService
              at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
      Caused by: java.lang.ClassNotFoundException: scala.collection.Seq
              at java.net.URLClassLoader$1.run(URLClassLoader.java:366) ~[na:1.7.0_79]
              at java.net.URLClassLoader$1.run(URLClassLoader.java:355) ~[na:1.7.0_79]
              at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_79]
              at java.net.URLClassLoader.findClass(URLClassLoader.java:354) ~[na:1.7.0_79]
              at co.cask.cdap.common.lang.InterceptableClassLoader.findClass(InterceptableClassLoader.java:46) ~[na:na]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:425) ~[na:1.7.0_79]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ~[na:1.7.0_79]
              ... 6 common frames omitted
      

      We should support scala as a programming language for classes other than actual spark program either by exposing these classes in workflow driver and other places or some other feasible way

        Attachments

          Activity

            People

            • Assignee:
              bhooshan Bhooshan Mogal
              Reporter:
              rsinha Rohit Sinha
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: