Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-5854

Spark programs fails on SDK running on windows machine

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.4.0
    • Fix Version/s: 3.4.0
    • Component/s: Spark
    • Labels:
      None
    • Rank:
      1|hzzcav:

      Description

      Spark programs fails on SDK running on windows machine with the following error:

      2016-04-28 17:19:40,331 - INFO  [pcontroller-program:default.WikipediaPipelineApp.workflow.WikipediaPipelineWorkflow-fdff0a91-0d9f-11e6-87a5-0000005701a3:c.c.c.i.a.s.ProgramLifecycleService$1@343] - Program stopped with error program:default.WikipediaPipelineApp.workflow.WikipediaPipelineWorkflow, fdff0a91-0d9f-11e6-87a5-0000005701a3
      java.nio.file.InvalidPathException: Illegal char <*> at index 24: $PWD\cdap-spark.jar\lib\*
      	at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182) ~[na:1.8.0_65]
      	at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153) ~[na:1.8.0_65]
      	at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77) ~[na:1.8.0_65]
      	at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94) ~[na:1.8.0_65]
      	at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255) ~[na:1.8.0_65]
      	at java.nio.file.Paths.get(Paths.java:84) ~[na:1.8.0_65]
      	at co.cask.cdap.app.runtime.spark.SparkRuntimeService.createSubmitConfigs(SparkRuntimeService.java:390) ~[na:na]
      	at co.cask.cdap.app.runtime.spark.SparkRuntimeService.startUp(SparkRuntimeService.java:210) ~[na:na]
      	at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47) ~[com.google.guava.guava-13.0.1.jar:na]
      	at co.cask.cdap.app.runtime.spark.SparkRuntimeService$3$1.run(SparkRuntimeService.java:299) ~[na:na]
      	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
      2016-04-28 17:19:40,378 - DEBUG [pcontroller-program:default.WikipediaPipelineApp.workflow.WikipediaPipelineWorkflow-fdff0a91-0d9f-11e6-87a5-0000005701a3:c.c.c.a.r.AbstractProgramRuntimeService@383] - Removing RuntimeInfo: Workflow WikipediaPipelineWorkflow fdff0a91-0d9f-11e6-87a5-0000005701a3
      2016-04-28 17:19:40,378 - DEBUG
      

      This is because on Windows machine * is reserved character: https://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx

        Attachments

          Activity

            People

            • Assignee:
              rsinha Rohit Sinha
              Reporter:
              rsinha Rohit Sinha
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: