Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-8445

Worker stream writes fail to retry if dataset service is down

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.0
    • Component/s: None
    • Labels:
      None
    • Release Notes:
      The stream endpoint to enqueue messages now returns a 503 instead of a 500 if it failed because the dataset service was unavailable.
    • Rank:
      1|hzzuo7:

      Description

      I have a worker that continuously writes to a Stream. The method to do so is supposed to retry if the required services are unavailable. It retries if the Stream service is not available, but if the call to the Stream service fails because the Dataset service is unavailable, it is not counted as a retryable failure.

      The program ends up seeing an exception like:

      2017-02-10 22:40:18,978 - ERROR [worker-StreamWriterWorker-0:c.c.r.AdminWorker@27] - error writing to stream
      java.io.IOException: Writing to Stream stream:default.streem did not succeed. Stream Service ResponseCode : 500
              at co.cask.cdap.app.stream.DefaultStreamWriter.writeToStream(DefaultStreamWriter.java:146) ~[na:na]
              at co.cask.cdap.app.stream.DefaultStreamWriter.write(DefaultStreamWriter.java:157) ~[na:na]
              at co.cask.cdap.app.stream.DefaultStreamWriter.write(DefaultStreamWriter.java:167) ~[na:na]
              at co.cask.cdap.app.stream.DefaultStreamWriter.write(DefaultStreamWriter.java:162) ~[na:na]
              at co.cask.cdap.internal.app.runtime.worker.BasicWorkerContext.write(BasicWorkerContext.java:116) ~[na:na]
              at co.cask.resiliency.StreamWriterWorker.run(StreamWriterWorker.java:23) ~[1486765962734-0/:na]
              at co.cask.cdap.internal.app.runtime.worker.WorkerDriver$1.run(WorkerDriver.java:85) [na:na]
              at co.cask.cdap.internal.app.runtime.AbstractContext.executeChecked(AbstractContext.java:492) [na:na]
              at co.cask.cdap.internal.app.runtime.worker.WorkerDriver.run(WorkerDriver.java:82) [na:na]
              at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:52) [com.google.guava.guava-13.0.1.jar:na]
              at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
      

      with the real cause hidden in the cdap master log:

      2017-02-10 22:41:00,585 - ERROR [streams-worker-thread-0:c.c.c.d.s.s.ConcurrentStreamWriter@283] - Failed to write to stream streem
      java.io.IOException: org.apache.tephra.TransactionFailureException: Exception raised from TxRunnable.run() co.cask.cdap.data2.transaction.Transactions$4@338c009
              at co.cask.cdap.data.stream.service.ConcurrentStreamWriter.getEventQueue(ConcurrentStreamWriter.java:283)
              at co.cask.cdap.data.stream.service.ConcurrentStreamWriter.enqueue(ConcurrentStreamWriter.java:165)
              at co.cask.cdap.data.stream.service.StreamHandler.enqueue(StreamHandler.java:266)
              at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80)
              at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124)
              at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at co.cask.http.RequestRouter.messageReceived(RequestRouter.java:84)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
              at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
              at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
              at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.handler.codec.http.HttpContentEncoder.messageReceived(HttpContentEncoder.java:69)
              at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at co.cask.http.NettyHttpService$2.handleUpstream(NettyHttpService.java:205)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
              at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
              at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
              at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
              at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
              at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      org.apache.tephra.TransactionFailureException: Exception raised from TxRunnable.run() co.cask.cdap.data2.transaction.Transactions$4@338c009
              at co.cask.cdap.data2.transaction.Transactions$CacheBasedTransactional.finishExecute(Transactions.java:237)
              at co.cask.cdap.data2.transaction.Transactions$CacheBasedTransactional.execute(Transactions.java:223)
              at co.cask.cdap.data2.transaction.Transactions$5.executeInternal(Transactions.java:295)
              at co.cask.cdap.data2.transaction.Transactions$5.execute(Transactions.java:282)
              at co.cask.cdap.data2.transaction.Transactions.execute(Transactions.java:259)
              at co.cask.cdap.data.stream.service.MDSStreamMetaStore.getStream(MDSStreamMetaStore.java:98)
              at co.cask.cdap.data.stream.service.MDSStreamMetaStore.streamExists(MDSStreamMetaStore.java:118)
              at co.cask.cdap.data2.transaction.stream.FileStreamAdmin.exists(FileStreamAdmin.java:387)
              at co.cask.cdap.data.stream.service.ConcurrentStreamWriter.getEventQueue(ConcurrentStreamWriter.java:266)
              at co.cask.cdap.data.stream.service.ConcurrentStreamWriter.enqueue(ConcurrentStreamWriter.java:165)
              at co.cask.cdap.data.stream.service.StreamHandler.enqueue(StreamHandler.java:266)
              at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80)
              at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124)
              at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at co.cask.http.RequestRouter.messageReceived(RequestRouter.java:84)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
              at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
              at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
              at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.handler.codec.http.HttpContentEncoder.messageReceived(HttpContentEncoder.java:69)
              at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at co.cask.http.NettyHttpService$2.handleUpstream(NettyHttpService.java:205)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
              at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
              at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
              at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
              at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
              at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      co.cask.cdap.common.ServiceUnavailableException: Service 'dataset.service' is not available. Please wait until it is up and running.
              at co.cask.cdap.common.internal.remote.RemoteClient.resolve(RemoteClient.java:113)
              at co.cask.cdap.common.internal.remote.RemoteClient.requestBuilder(RemoteClient.java:78)
              at co.cask.cdap.data2.datafabric.dataset.DatasetServiceClient.doGet(DatasetServiceClient.java:275)
              at co.cask.cdap.data2.datafabric.dataset.DatasetServiceClient.getInstance(DatasetServiceClient.java:116)
              at co.cask.cdap.data2.datafabric.dataset.RemoteDatasetFramework.getDataset(RemoteDatasetFramework.java:252)
              at co.cask.cdap.data2.dataset2.ForwardingDatasetFramework.getDataset(ForwardingDatasetFramework.java:172)
              at co.cask.cdap.data2.metadata.writer.LineageWriterDatasetFramework.access$101(LineageWriterDatasetFramework.java:61)
              at co.cask.cdap.data2.metadata.writer.LineageWriterDatasetFramework$2.call(LineageWriterDatasetFramework.java:184)
              at co.cask.cdap.data2.metadata.writer.LineageWriterDatasetFramework$2.call(LineageWriterDatasetFramework.java:181)
              at co.cask.cdap.data2.dataset2.DefaultDatasetRuntimeContext.execute(DefaultDatasetRuntimeContext.java:121)
              at co.cask.cdap.data2.metadata.writer.LineageWriterDatasetFramework.getDataset(LineageWriterDatasetFramework.java:179)
              at co.cask.cdap.data.dataset.SystemDatasetInstantiator.getDataset(SystemDatasetInstantiator.java:101)
              at co.cask.cdap.data2.dataset2.SingleThreadDatasetCache$1.load(SingleThreadDatasetCache.java:97)
              at co.cask.cdap.data2.dataset2.SingleThreadDatasetCache$1.load(SingleThreadDatasetCache.java:93)
              at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3589)
              at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2374)
              at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2337)
              at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2252)
              at com.google.common.cache.LocalCache.get(LocalCache.java:3990)
              at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3994)
              at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4878)
              at com.google.common.cache.ForwardingLoadingCache.get(ForwardingLoadingCache.java:48)
              at co.cask.cdap.data2.dataset2.SingleThreadDatasetCache$LineageRecordingDatasetCache.get(SingleThreadDatasetCache.java:149)
              at co.cask.cdap.data2.dataset2.SingleThreadDatasetCache$LineageRecordingDatasetCache.get(SingleThreadDatasetCache.java:132)
              at co.cask.cdap.data2.dataset2.SingleThreadDatasetCache.getDataset(SingleThreadDatasetCache.java:175)
              at co.cask.cdap.data2.dataset2.MultiThreadDatasetCache.getDataset(MultiThreadDatasetCache.java:114)
              at co.cask.cdap.data2.dataset2.DynamicDatasetCache.getDataset(DynamicDatasetCache.java:203)
              at co.cask.cdap.data2.dataset2.DynamicDatasetCache.getDataset(DynamicDatasetCache.java:135)
              at co.cask.cdap.data2.dataset2.DynamicDatasetCache.getDataset(DynamicDatasetCache.java:106)
              at co.cask.cdap.data2.dataset2.DynamicDatasetCache.getDataset(DynamicDatasetCache.java:94)
              at co.cask.cdap.data2.datafabric.dataset.DatasetsUtil.getOrCreateDataset(DatasetsUtil.java:87)
              at co.cask.cdap.data.stream.service.MDSStreamMetaStore.getMetadataStore(MDSStreamMetaStore.java:75)
              at co.cask.cdap.data.stream.service.MDSStreamMetaStore.access$200(MDSStreamMetaStore.java:53)
              at co.cask.cdap.data.stream.service.MDSStreamMetaStore$2.call(MDSStreamMetaStore.java:101)
              at co.cask.cdap.data.stream.service.MDSStreamMetaStore$2.call(MDSStreamMetaStore.java:98)
              at co.cask.cdap.data2.transaction.Transactions$4.run(Transactions.java:262)
              at co.cask.cdap.data2.transaction.Transactions$CacheBasedTransactional.finishExecute(Transactions.java:235)
              at co.cask.cdap.data2.transaction.Transactions$CacheBasedTransactional.execute(Transactions.java:223)
              at co.cask.cdap.data2.transaction.Transactions$5.executeInternal(Transactions.java:295)
              at co.cask.cdap.data2.transaction.Transactions$5.execute(Transactions.java:282)
              at co.cask.cdap.data2.transaction.Transactions.execute(Transactions.java:259)
              at co.cask.cdap.data.stream.service.MDSStreamMetaStore.getStream(MDSStreamMetaStore.java:98)
              at co.cask.cdap.data.stream.service.MDSStreamMetaStore.streamExists(MDSStreamMetaStore.java:118)
              at co.cask.cdap.data2.transaction.stream.FileStreamAdmin.exists(FileStreamAdmin.java:387)
              at co.cask.cdap.data.stream.service.ConcurrentStreamWriter.getEventQueue(ConcurrentStreamWriter.java:266)
              at co.cask.cdap.data.stream.service.ConcurrentStreamWriter.enqueue(ConcurrentStreamWriter.java:165)
              at co.cask.cdap.data.stream.service.StreamHandler.enqueue(StreamHandler.java:266)
              at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80)
              at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124)
              at co.cask.cdap.common.http.AuthenticationChannelHandler.messageReceived(AuthenticationChannelHandler.java:64)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at co.cask.http.RequestRouter.messageReceived(RequestRouter.java:84)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
              at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
              at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
              at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.handler.codec.http.HttpContentEncoder.messageReceived(HttpContentEncoder.java:69)
              at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.messageReceived(SimpleChannelUpstreamHandler.java:124)
              at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
              at co.cask.http.NettyHttpService$2.handleUpstream(NettyHttpService.java:205)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
              at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
              at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
              at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
              at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
              at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
              at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
              at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
              at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      

        Attachments

          Activity

            People

            • Assignee:
              ashau Albert Shau
              Reporter:
              ashau Albert Shau
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: