Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-12122

Unable to create workflow for complex pipeline

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: Pipelines
    • Labels:
      None
    • Release Notes:
      Fixed a bug in the pipeline planner that caused some pipelines to fail to deploy with a NoSuchElementException
    • Rank:
      1|i0051b:

      Description

      There looks to be some sort of planner bug with the attached pipeline. During publish, it fails with:

      2017-07-13 14:12:05,944 - ERROR [appfabric-executor-30:c.c.c.g.h.AppLifecycleHttpHandler$1@405] - Deploy failure
      java.util.concurrent.ExecutionException: java.util.NoSuchElementException
      	at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294) ~[com.google.guava.guava-13.0.1.jar:na]
      	at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:267) ~[com.google.guava.guava-13.0.1.jar:na]
      	at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:96) ~[com.google.guava.guava-13.0.1.jar:na]
      	at co.cask.cdap.internal.app.deploy.pipeline.LocalArtifactLoaderStage.process(LocalArtifactLoaderStage.java:105) ~[na:na]
      	at co.cask.cdap.internal.app.deploy.pipeline.LocalArtifactLoaderStage.process(LocalArtifactLoaderStage.java:55) ~[na:na]
      	at co.cask.cdap.pipeline.AbstractStage.process(AbstractStage.java:52) ~[na:na]
      	at co.cask.cdap.internal.pipeline.SynchronousPipeline.execute(SynchronousPipeline.java:55) ~[na:na]
      	at co.cask.cdap.internal.app.deploy.LocalApplicationManager.deploy(LocalApplicationManager.java:133) ~[na:na]
      	at co.cask.cdap.internal.app.services.ApplicationLifecycleService.deployApp(ApplicationLifecycleService.java:684) ~[na:na]
      	at co.cask.cdap.internal.app.services.ApplicationLifecycleService.deployApp(ApplicationLifecycleService.java:499) ~[na:na]
      	at co.cask.cdap.gateway.handlers.AppLifecycleHttpHandler$1.onFinish(AppLifecycleHttpHandler.java:387) ~[na:na]
      	at co.cask.cdap.common.http.AbstractBodyConsumer.finished(AbstractBodyConsumer.java:65) [na:na]
      	at co.cask.http.HttpMethodInfo.bodyConsumerFinish(HttpMethodInfo.java:149) [co.cask.http.netty-http-0.16.0.jar:na]
      	at co.cask.http.HttpMethodInfo.chunk(HttpMethodInfo.java:108) [co.cask.http.netty-http-0.16.0.jar:na]
      	at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:40) [co.cask.http.netty-http-0.16.0.jar:na]
      	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [io.netty.netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [io.netty.netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [io.netty.netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [io.netty.netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [io.netty.netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [io.netty.netty-3.6.6.Final.jar:na]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_121]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
      	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121]
      Caused by: java.util.NoSuchElementException: null
      	at java.util.HashMap$HashIterator.nextNode(HashMap.java:1439) ~[na:1.8.0_121]
      	at java.util.HashMap$KeyIterator.next(HashMap.java:1461) ~[na:1.8.0_121]
      	at com.google.common.collect.AbstractMultimap$WrappedCollection$WrappedIterator.next(AbstractMultimap.java:540) ~[com.google.guava.guava-13.0.1.jar:na]
      	at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1042) ~[na:1.8.0_121]
      	at co.cask.cdap.datapipeline.SmartWorkflow.addBranchPrograms(SmartWorkflow.java:276) ~[na:na]
      	at co.cask.cdap.datapipeline.SmartWorkflow.addBranchPrograms(SmartWorkflow.java:276) ~[na:na]
      	at co.cask.cdap.datapipeline.SmartWorkflow.addPrograms(SmartWorkflow.java:250) ~[na:na]
      	at co.cask.cdap.datapipeline.SmartWorkflow.configure(SmartWorkflow.java:174) ~[na:na]
      	at co.cask.cdap.api.workflow.AbstractWorkflow.configure(AbstractWorkflow.java:58) ~[na:na]
      	at co.cask.cdap.app.DefaultAppConfigurer.addWorkflow(DefaultAppConfigurer.java:182) ~[na:na]
      	at co.cask.cdap.api.app.AbstractApplication.addWorkflow(AbstractApplication.java:128) ~[na:na]
      	at co.cask.cdap.datapipeline.DataPipelineApp.configure(DataPipelineApp.java:76) ~[na:na]
      	at co.cask.cdap.api.app.AbstractApplication.configure(AbstractApplication.java:63) ~[na:na]
      	at co.cask.cdap.internal.app.deploy.InMemoryConfigurator.getSpecJson(InMemoryConfigurator.java:162) ~[na:na]
      	at co.cask.cdap.internal.app.deploy.InMemoryConfigurator.createResponse(InMemoryConfigurator.java:128) ~[na:na]
      	at co.cask.cdap.internal.app.deploy.InMemoryConfigurator.config(InMemoryConfigurator.java:118) ~[na:na]
      	at co.cask.cdap.internal.app.deploy.pipeline.LocalArtifactLoaderStage.process(LocalArtifactLoaderStage.java:104) ~[na:na]
      	... 20 common frames omitted
      

      It is dying while trying to create the workflow. It looks like some sort of bug in the ControlDag, which converts the generic dag of phases into a fork-join type of dag.

        Attachments

          Activity

            People

            • Assignee:
              ashau Albert Shau
              Reporter:
              ashau Albert Shau
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: