Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-4580

On multinode cluster if the clocks are not in sync an ETL pipeline fail with not a very helpful message

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.0
    • Fix Version/s: 3.5.0
    • Component/s: CDAP, ETL
    • Labels:
    • Rank:
      1|hzz507:

      Description

      On multinode cluster if the clocks are not sync running an etl pipeline fails with the following error:

      2016-01-12 00:35:24,756 ERROR co.cask.cdap.common.HttpExceptionHandler: Unexpected error: request=POST /v3/namespaces/default/apps/HiveProJection/workflows/ETLWorkflow/start user=<null>:
      java.lang.IllegalArgumentException: Newer program update time than the specification update time. Application must be redeployed
              at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) ~[com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.internal.app.store.DefaultStore.loadProgram(DefaultStore.java:220) ~[co.cask.cdap.cdap-app-fabric-3.3.0-SNAPSHOT.jar:na]
              at co.cask.cdap.internal.app.services.ProgramLifecycleService.getProgram(ProgramLifecycleService.java:114) ~[co.cask.cdap.cdap-app-fabric-3.3.0-SNAPSHOT.jar:na]
              at co.cask.cdap.internal.app.services.ProgramLifecycleService.start(ProgramLifecycleService.java:135) ~[co.cask.cdap.cdap-app-fabric-3.3.0-SNAPSHOT.jar:na]
              at co.cask.cdap.gateway.handlers.ProgramLifecycleHttpHandler.start(ProgramLifecycleHttpHandler.java:1361) ~[co.cask.cdap.cdap-app-fabric-3.3.0-SNAPSHOT.jar:na]
              at co.cask.cdap.gateway.handlers.ProgramLifecycleHttpHandler.startStopProgram(ProgramLifecycleHttpHandler.java:1330) ~[co.cask.cdap.cdap-app-fabric-3.3.0-SNAPSHOT.jar:na]
              at co.cask.cdap.gateway.handlers.ProgramLifecycleHttpHandler.performAction(ProgramLifecycleHttpHandler.java:331) ~[co.cask.cdap.cdap-app-fabric-3.3.0-SNAPSHOT.jar:na]
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_67]
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_67]
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_67]
              at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_67]
              at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) ~[co.cask.http.netty-http-0.13.0.jar:na]
              at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) [co.cask.http.netty-http-0.13.0.jar:na]
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [io.netty.netty-3.6.6.Final.jar:na]
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [io.netty.netty-3.6.6.Final.jar:na]
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [io.netty.netty-3.6.6.Final.jar:na]
              at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [io.netty.netty-3.6.6.Final.jar:na]
              at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [io.netty.netty-3.6.6.Final.jar:na]
              at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [io.netty.netty-3.6.6.Final.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_67]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_67]
              at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67]
      

      It will be nice to give a more helpful message and point user to sync the clocks.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                terence Terence Yim
                Reporter:
                rsinha Rohit Sinha
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: