Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-11957

Inconsistency in schedule store can silently prevent scheduler from starting up

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 4.3.3, 4.2.0, 4.1.0
    • Fix Version/s: None
    • Component/s: Master, Scheduler
    • Labels:
    • Rank:
      1|i00447:

      Description

      If the trigger exists for a schedule, but the job doesn't exist, the scheduler will repeatedly restart and have the following exception:

      2017-06-20 20:03:20,742 - DEBUG [Endure-Service-:c.c.c.c.s.RetryOnStartFailureService$1@71] - Exception raised when starting service 
      java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.quartz.JobPersistenceException: The job (DEFAULT.<NS>:<APP>:-SNAPSHOT:WORKFLOW:<WORKFLOW>) referenced by the trigger does not exist.
              at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294) ~[com.google.guava.guava-13.0.1.jar:na]
              at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:281) ~[com.google.guava.guava-13.0.1.jar:na]
              at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) ~[com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.common.service.RetryOnStartFailureService$1.run(RetryOnStartFailureService.java:64) ~[na:na]
      Caused by: java.lang.RuntimeException: org.quartz.JobPersistenceException: The job (DEFAULT.<NS>:<APP>:-SNAPSHOT:WORKFLOW:<WORKFLOW>) referenced by the trigger does not exist.
              at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.internal.app.runtime.schedule.store.DatasetBasedTimeScheduleStore.initialize(DatasetBasedTimeScheduleStore.java:103) ~[na:na]
              at co.cask.cdap.app.guice.AppFabricServiceRuntimeModule$AppFabricServiceModule.getScheduler(AppFabricServiceRuntimeModule.java:440) ~[na:na]
              at co.cask.cdap.app.guice.AppFabricServiceRuntimeModule$AppFabricServiceModule.access$200(AppFabricServiceRuntimeModule.java:316) ~[na:na]
              at co.cask.cdap.app.guice.AppFabricServiceRuntimeModule$AppFabricServiceModule$4.get(AppFabricServiceRuntimeModule.java:399) ~[na:na]
              at co.cask.cdap.app.guice.AppFabricServiceRuntimeModule$AppFabricServiceModule$4.get(AppFabricServiceRuntimeModule.java:392) ~[na:na]
              at co.cask.cdap.internal.app.runtime.schedule.TimeScheduler.init(TimeScheduler.java:103) ~[na:na]
              at co.cask.cdap.internal.app.runtime.schedule.AbstractSchedulerService.startSchedulers(AbstractSchedulerService.java:67) ~[na:na]
              at co.cask.cdap.internal.app.runtime.schedule.DistributedSchedulerService$1$1.doStart(DistributedSchedulerService.java:51) ~[na:na]
              at com.google.common.util.concurrent.AbstractService.start(AbstractService.java:170) ~[com.google.guava.guava-13.0.1.jar:na]
              ... 1 common frames omitted
      Caused by: org.quartz.JobPersistenceException: The job (DEFAULT.<NS>:<APP>:-SNAPSHOT:WORKFLOW:<WORKFLOW>) referenced by the trigger does not exist.
              at org.quartz.simpl.RAMJobStore.storeTrigger(RAMJobStore.java:422) ~[org.quartz-scheduler.quartz-2.2.0.jar:na]
              at co.cask.cdap.internal.app.runtime.schedule.store.DatasetBasedTimeScheduleStore.readSchedulesFromPersistentStore(DatasetBasedTimeScheduleStore.java:389) ~[na:na]
              at co.cask.cdap.internal.app.runtime.schedule.store.DatasetBasedTimeScheduleStore.initialize(DatasetBasedTimeScheduleStore.java:101) ~[na:na]
              ... 9 common frames omitted
      

        Attachments

          Activity

            People

            • Assignee:
              bhooshan Bhooshan Mogal
              Reporter:
              ali.anwar Ali Anwar
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: