Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-745

Lots of metrics exceptions in master log

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.7.1
    • Component/s: Metrics
    • Labels:
      None

      Description

      After beaming 2.6 snapshot to a cluster, I see lots of the following in the logs. Not sure where they come from...

      2014-11-24T23:31:00,681Z INFO c.c.c.m.p.MetricsMessageCallback [anew535-1000.dev.continuuity.net] [message-callback-0] MetricsMessageCallback:onReceived(MetricsMessageCallba
      ck.java:80) - No records to process.
      2014-11-24T23:31:00,683Z WARN c.c.c.m.p.MetricsMessageCallback [anew535-1000.dev.continuuity.net] [message-callback-0] MetricsMessageCallback$1:apply(MetricsMessageCallback.
      java:73) - Failed to decode message to MetricsRecord. Skipped. null
      java.io.EOFException: null
      at co.cask.cdap.common.io.BinaryDecoder.readByte(BinaryDecoder.java:155)
      at co.cask.cdap.common.io.BinaryDecoder.readLong(BinaryDecoder.java:66)
      at co.cask.cdap.internal.io.ReflectionDatumReader.resolveType(ReflectionDatumReader.java:360)
      at co.cask.cdap.internal.io.ReflectionDatumReader.doRead(ReflectionDatumReader.java:118)
      at co.cask.cdap.internal.io.ReflectionDatumReader.read(ReflectionDatumReader.java:80)
      at co.cask.cdap.internal.io.ReflectionDatumReader.readRecord(ReflectionDatumReader.java:217)
      at co.cask.cdap.internal.io.ReflectionDatumReader.doRead(ReflectionDatumReader.java:112)
      at co.cask.cdap.internal.io.ReflectionDatumReader.read(ReflectionDatumReader.java:80)
      at co.cask.cdap.internal.io.ReflectionDatumReader.read(ReflectionDatumReader.java:63)
      at co.cask.cdap.metrics.process.MetricsMessageCallback$1.apply(MetricsMessageCallback.java:71)
      at co.cask.cdap.metrics.process.MetricsMessageCallback$1.apply(MetricsMessageCallback.java:67)
      at com.google.common.collect.Iterators$9.transform(Iterators.java:893)
      at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
      at com.google.common.collect.Iterators$8.computeNext(Iterators.java:735)
      at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
      at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
      at com.google.common.collect.ImmutableList.copyOf(ImmutableList.java:263)
      at co.cask.cdap.metrics.process.MetricsMessageCallback.onReceived(MetricsMessageCallback.java:66)
      at co.cask.cdap.metrics.process.PersistedMessageCallback.onReceived(PersistedMessageCallback.java:63)
      at org.apache.twill.internal.kafka.client.SimpleKafkaConsumer$SimplePreparer$2$1.run(SimpleKafkaConsumer.java:282)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:744)

        Attachments

          Activity

            People

            • Assignee:
              alexb Alex Baranau
              Reporter:
              andreas Andreas Neumann
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: