Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-4097

Fix AbstractDatasetProvider.getOrCreate

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.3.0, 3.2.1, 3.1.2
    • Fix Version/s: 3.3.0, 3.2.2
    • Component/s: App Fabric, Test
    • Labels:
      None
    • Rank:
      1|hzz2a7:

      Description

      I see lots of these in the logs:

      2015-10-26 16:14:21,570 - WARN  [executor-8:c.c.c.d.d.d.s.DatasetInstanceService@145] - Failed to register usage of program:default.SparkAppUsingObjectStore.spark.SparkCharCountProgram -> dataset:default.keys
      2015-10-26 16:14:21,587 - ERROR [executor-4:c.c.c.d.r.UsageRegistry$1@73] - Failed to access usage table
      co.cask.cdap.common.NotFoundException: 'dataset:system.usage.registry' was not found.
      	at co.cask.cdap.data2.datafabric.dataset.service.DatasetInstanceService.get(DatasetInstanceService.java:124) [classes/:na]
      	at co.cask.cdap.data2.datafabric.dataset.service.DatasetInstanceService$1.getMeta(DatasetInstanceService.java:82) ~[classes/:na]
      	at co.cask.cdap.data2.datafabric.dataset.AbstractDatasetProvider.get(AbstractDatasetProvider.java:101) ~[classes/:na]
      	at co.cask.cdap.data2.datafabric.dataset.AbstractDatasetProvider.getOrCreate(AbstractDatasetProvider.java:84) ~[classes/:na]
      	at co.cask.cdap.data2.registry.UsageRegistry$1.get(UsageRegistry.java:62) [classes/:na]
      	at co.cask.cdap.data2.registry.UsageRegistry$1.get(UsageRegistry.java:58) [classes/:na]
      	at co.cask.cdap.data2.dataset2.tx.Transactional.execute(Transactional.java:104) [classes/:na]
      	at co.cask.cdap.data2.dataset2.tx.Transactional.executeUnchecked(Transactional.java:80) [classes/:na]
      	at co.cask.cdap.data2.registry.UsageRegistry.register(UsageRegistry.java:149) [classes/:na]
      	at co.cask.cdap.data2.datafabric.dataset.service.DatasetInstanceService.registerUsage(DatasetInstanceService.java:142) [classes/:na]
      	at co.cask.cdap.data2.datafabric.dataset.service.DatasetInstanceService.get(DatasetInstanceService.java:134) [classes/:na]
      	at co.cask.cdap.data2.datafabric.dataset.service.DatasetInstanceHandler.get(DatasetInstanceHandler.java:110) [classes/:na]
      	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source) ~[na:na]
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_80]
      	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_80]
      	at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:80) [netty-http-0.13.0.jar:na]
      	at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:38) [netty-http-0.13.0.jar:na]
      	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [netty-3.6.6.Final.jar:na]
      	at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [netty-3.6.6.Final.jar:na]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_80]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_80]
      	at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
      

      This seems to happen with tests based on AppFabricTestHelper, for example, SparkProgramRunnerTest. This pollutes the logs and makes it hard to see the logs and debug actual issues.

      Do we need this in 3.1.x?

        Attachments

          Activity

            People

            • Assignee:
              alvin Alvin Wang
              Reporter:
              andreas Andreas Neumann
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: