Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.0.0
    • Fix Version/s: 4.1.0
    • Component/s: None
    • Labels:
      None
    • Release Notes:
      Fixed an issue where the CDAP Master process would hang during a graceful shutdown.
    • Rank:
      1|hzzrxz:

      Description

      When I try to shutdown cdap master - /etc/init.d/cdap-master stop,
      it doesn't stop. When I check the master logs, I see that the operational stats collection for 'transactions' is still trying to get info from the tx master but it is not found since the MasterTwillApplication has been stopped.

      2017-01-05 18:56:10,703 - WARN  [operational-stats-collector-0:c.c.c.o.OperationalStatsService@107] - Error while collecting stats for service: CDAP; type: transactions
      java.lang.RuntimeException: org.apache.thrift.TException: Unable to discover tx service.
              at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[com.google.guava.guava-13.0.1.jar:na]
              at org.apache.tephra.distributed.TransactionServiceClient.getInvalidSize(TransactionServiceClient.java:472) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at co.cask.cdap.operations.cdap.CDAPTransactions.collect(CDAPTransactions.java:99) ~[na:na]
              at co.cask.cdap.operations.OperationalStatsService.runOneIteration(OperationalStatsService.java:105) ~[na:na]
              at com.google.common.util.concurrent.AbstractScheduledService$1$1.run(AbstractScheduledService.java:170) [com.google.guava.guava-13.0.1.jar:na]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_75]
              at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_75]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_75]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_75]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
              at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
      Caused by: org.apache.thrift.TException: Unable to discover tx service.
              at org.apache.tephra.distributed.AbstractClientProvider.newClient(AbstractClientProvider.java:106) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at org.apache.tephra.distributed.AbstractClientProvider.newClient(AbstractClientProvider.java:85) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at org.apache.tephra.distributed.PooledClientProvider$TxClientPool.create(PooledClientProvider.java:48) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at org.apache.tephra.distributed.PooledClientProvider$TxClientPool.create(PooledClientProvider.java:41) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at org.apache.tephra.distributed.ElasticPool.getOrCreate(ElasticPool.java:138) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at org.apache.tephra.distributed.ElasticPool.obtain(ElasticPool.java:125) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at org.apache.tephra.distributed.PooledClientProvider.getCloseableClient(PooledClientProvider.java:101) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at org.apache.tephra.distributed.TransactionServiceClient.execute(TransactionServiceClient.java:217) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at org.apache.tephra.distributed.TransactionServiceClient.execute(TransactionServiceClient.java:188) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              at org.apache.tephra.distributed.TransactionServiceClient.getInvalidSize(TransactionServiceClient.java:464) ~[org.apache.tephra.tephra-core-0.10.0-incubating.jar:0.10.0-incubating]
              ... 10 common frames omitted
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                terence Terence Yim
                Reporter:
                gokul Gokul Gunasekaran
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: