Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-7155

Calls to delete system modules should not be performed in one short transaction

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 4.0.0
    • Fix Version/s: 4.1.0
    • Component/s: Security
    • Labels:
      None
    • Release Notes:
      Fixed a problem where the Dataset Service failed to start up if authorization was enabled and the authorization plugin was slow to respond.
    • Rank:
      1|hzzkxb:

      Description

      When master starts up there is a call to delete system modules and it is performed in one transaction which might timeout. Revisit this in 4.0

      This results in CDAP master continuously restarting DatasetService. Following is the stacktrace of one such failure.

      2016-11-05 01:49:27,956 ERROR co.cask.cdap.data2.datafabric.dataset.service.DatasetService: Uncaught exception from Thread[DatasetService,5,main]
      com.google.common.util.concurrent.UncheckedExecutionException: org.apache.tephra.TransactionFailureException: Transaction 1478310522767000000 is not in progress.
              at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015) ~[com.google.guava.guava-13.0.1.jar:na]
              at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) ~[com.google.guava.guava-13.0.1.jar:na]
              at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) ~[com.google.guava.guava-13.0.1.jar:na]
              at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106) ~[com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.data2.datafabric.dataset.service.DatasetService.startUp(DatasetService.java:114) ~[co.cask.cdap.cdap-data-fabric-4.0.0-SNAPSHOT.jar:na]
              at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47) ~[com.google.guava.guava-13.0.1.jar:na]
              at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_67]
      Caused by: org.apache.tephra.TransactionFailureException: Transaction 1478310522767000000 is not in progress.
              at org.apache.tephra.TransactionContext.checkForConflicts(TransactionContext.java:277) ~[org.apache.tephra.tephra-core-0.10.0-incubating-SNAPSHOT.jar:0.10.0-incubating-S
      NAPSHOT]
              at org.apache.tephra.TransactionContext.finish(TransactionContext.java:147) ~[org.apache.tephra.tephra-core-0.10.0-incubating-SNAPSHOT.jar:0.10.0-incubating-SNAPSHOT]
              at co.cask.cdap.data2.dataset2.SingleThreadDatasetCache$DelayedDiscardingTransactionContext.finish(SingleThreadDatasetCache.java:423) ~[co.cask.cdap.cdap-data-fabric-4.0
      .0-SNAPSHOT.jar:na]
              at co.cask.cdap.data2.transaction.DynamicTransactionExecutor.executeOnce(DynamicTransactionExecutor.java:144) ~[co.cask.cdap.cdap-data-fabric-4.0.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data2.transaction.DynamicTransactionExecutor.executeWithRetry(DynamicTransactionExecutor.java:104) ~[co.cask.cdap.cdap-data-fabric-4.0.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data2.transaction.DynamicTransactionExecutor.execute(DynamicTransactionExecutor.java:61) ~[co.cask.cdap.cdap-data-fabric-4.0.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data2.transaction.DynamicTransactionExecutor.execute(DynamicTransactionExecutor.java:89) ~[co.cask.cdap.cdap-data-fabric-4.0.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data2.datafabric.dataset.service.DatasetTypeService.deleteSystemModules(DatasetTypeService.java:455) ~[co.cask.cdap.cdap-data-fabric-4.0.0-SNAPSHOT.jar:n
      a]
              at co.cask.cdap.data2.datafabric.dataset.service.DatasetTypeService.startUp(DatasetTypeService.java:155) ~[co.cask.cdap.cdap-data-fabric-4.0.0-SNAPSHOT.jar:na]
              at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ~[com.google.guava.guava-13.0.1.jar:na]
              ... 1 common frames omitted
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                andreas Andreas Neumann
                Reporter:
                sree Sreevatsan Raman
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: