Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-2343

In UI, Mapreduce logs are convoluted with system logs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.1.0
    • Component/s: App Fabric, Log, UI
    • Labels:
    • Sprint:
      FE Sprint ending 06/22
    • Rank:
      1|hzy7lr:

      Description

      See below for a MapReduce log that is convoluted with messages caused by the UI trying to fetch job history (these messages are due to CDAP-2342).

      2015-04-28 20:21:50,918 - INFO [executor-334:c.c.c.i.a.r.b.MapReduceProgramRunner@175] - Starting MapReduce Job: job=HiveCounter,=namespaceId=default, applicationId=SportResults, program=HiveCounter, runid=dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9
      2015-04-28 20:21:52,214 - INFO [MapReduceRunner-HiveCounter:c.c.c.h.s.HiveCounter@69] - input: nfl, output: nfl
      2015-04-28 20:21:52,215 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@547] - Using Dataset totals as output for MapReduce Job
      2015-04-28 20:21:52,217 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@803] - Set output key class to class org.apache.hadoop.io.Text
      2015-04-28 20:21:52,217 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@808] - Set output value class to class java.lang.String
      2015-04-28 20:21:52,219 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@844] - Set map output key class to class org.apache.hadoop.io.Text
      2015-04-28 20:21:52,220 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@849] - Set map output value class to class org.apache.hadoop.io.Text
      2015-04-28 20:21:52,222 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@699] - Creating Job jar: file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.jar
      2015-04-28 20:21:52,224 - INFO [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@713] - InputFormat class: class org.apache.hive.hcatalog.mapreduce.HCatInputFormat co.cask.cdap.common.lang.ProgramClassLoader@5dc90316
      2015-04-28 20:21:52,224 - INFO [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@730] - OutputFormat class: class org.apache.hadoop.mapreduce.lib.output.TextOutputFormat sun.misc.Launcher$AppClassLoader@1d9f953d
      2015-04-28 20:21:52,225 - WARN [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@743] - Not including HBaseTableUtil classes in submitted Job Jar since they are not available
      2015-04-28 20:21:52,225 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.c.u.ApplicationBundler@129] - start creating bundle mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.jar. building a temporary file locally at first
      2015-04-28 20:22:01,650 - INFO [executor-334:c.c.c.i.a.r.b.i.LocalClientProtocolProvider@42] - Using framework: local
      2015-04-28 20:22:01,651 - INFO [executor-334:c.c.c.i.a.r.b.i.LocalClientProtocolProvider@50] - Using tracker: clocal
      2015-04-28 20:22:01,662 - WARN [executor-334:c.c.c.a.m.MRJobClient@77] - JobClient failed to get all jobs. java.lang.NullPointerException: null at org.apache.hadoop.mapred.JobClient.getAllJobs(JobClient.java:814) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na] at co.cask.cdap.app.mapreduce.MRJobClient.getMRJobInfo(MRJobClient.java:75) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at co.cask.cdap.gateway.handlers.ProgramLifecycleHttpHandler.getMapReduceInfo(ProgramLifecycleHttpHandler.java:237) [co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_65] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_65] at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:85) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:41) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [na:1.6.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [na:1.6.0_65] at java.lang.Thread.run(Thread.java:695) [na:1.6.0_65]
      2015-04-28 20:22:01,665 - DEBUG [executor-334:c.c.c.g.h.ProgramLifecycleHttpHandler@239] - Failed to get run history from JobClient for runId: namespace:default/application:SportResults/program:mapreduce:HiveCounter/run:dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9. Falling back to Metrics system. java.io.IOException: java.lang.NullPointerException at co.cask.cdap.app.mapreduce.MRJobClient.getMRJobInfo(MRJobClient.java:78) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at co.cask.cdap.gateway.handlers.ProgramLifecycleHttpHandler.getMapReduceInfo(ProgramLifecycleHttpHandler.java:237) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_65] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_65] at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:85) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:41) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [na:1.6.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [na:1.6.0_65] at java.lang.Thread.run(Thread.java:695) [na:1.6.0_65] Caused by: java.lang.NullPointerException: null at org.apache.hadoop.mapred.JobClient.getAllJobs(JobClient.java:814) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na] at co.cask.cdap.app.mapreduce.MRJobClient.getMRJobInfo(MRJobClient.java:75) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] ... 15 common frames omitted
      2015-04-28 20:22:10,423 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.c.u.ApplicationBundler@146] - copying temporary bundle to destination file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.jar (60659688 bytes)
      2015-04-28 20:22:11,172 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.c.u.ApplicationBundler@159] - finished creating bundle at file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.jar
      2015-04-28 20:22:11,184 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.c.u.ApplicationBundler@162] - cleaned up local temporary for bundle file:/var/folders/84/29j9c2z57lnbcxnpjgsp7t980000gq/T/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.jar3729636414414052011.tmp
      2015-04-28 20:22:11,189 - INFO [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@751] - Built MapReduce Job Jar at file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.jar
      2015-04-28 20:22:11,428 - INFO [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@873] - Copied Program Jar to file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.program.jar, source: file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/namespaces/default/programs/SportResults/MapReduce/HiveCounter.jar
      2015-04-28 20:22:11,453 - INFO [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@277] - Submitting MapReduce Job: job=HiveCounter,=namespaceId=default, applicationId=SportResults, program=HiveCounter, runid=dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9
      2015-04-28 20:22:11,455 - INFO [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.i.LocalClientProtocolProvider@42] - Using framework: local
      2015-04-28 20:22:11,455 - INFO [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.i.LocalClientProtocolProvider@50] - Using tracker: clocal
      2015-04-28 20:22:11,487 - WARN [MapReduceRunner-HiveCounter:o.a.h.m.JobSubmitter@150] - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
      2015-04-28 20:22:11,745 - INFO [executor-334:c.c.c.i.a.r.b.i.LocalClientProtocolProvider@42] - Using framework: local
      2015-04-28 20:22:11,746 - INFO [executor-334:c.c.c.i.a.r.b.i.LocalClientProtocolProvider@50] - Using tracker: clocal
      2015-04-28 20:22:11,749 - WARN [executor-334:c.c.c.a.m.MRJobClient@77] - JobClient failed to get all jobs. java.lang.NullPointerException: null at org.apache.hadoop.mapred.JobClient.getAllJobs(JobClient.java:814) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na] at co.cask.cdap.app.mapreduce.MRJobClient.getMRJobInfo(MRJobClient.java:75) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at co.cask.cdap.gateway.handlers.ProgramLifecycleHttpHandler.getMapReduceInfo(ProgramLifecycleHttpHandler.java:237) [co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_65] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_65] at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:85) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:41) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [na:1.6.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [na:1.6.0_65] at java.lang.Thread.run(Thread.java:695) [na:1.6.0_65]
      2015-04-28 20:22:11,754 - DEBUG [executor-334:c.c.c.g.h.ProgramLifecycleHttpHandler@239] - Failed to get run history from JobClient for runId: namespace:default/application:SportResults/program:mapreduce:HiveCounter/run:dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9. Falling back to Metrics system. java.io.IOException: java.lang.NullPointerException at co.cask.cdap.app.mapreduce.MRJobClient.getMRJobInfo(MRJobClient.java:78) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at co.cask.cdap.gateway.handlers.ProgramLifecycleHttpHandler.getMapReduceInfo(ProgramLifecycleHttpHandler.java:237) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_65] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_65] at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:85) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:41) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [na:1.6.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [na:1.6.0_65] at java.lang.Thread.run(Thread.java:695) [na:1.6.0_65] Caused by: java.lang.NullPointerException: null at org.apache.hadoop.mapred.JobClient.getAllJobs(JobClient.java:814) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na] at co.cask.cdap.app.mapreduce.MRJobClient.getMRJobInfo(MRJobClient.java:75) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] ... 15 common frames omitted
      2015-04-28 20:22:12,641 - WARN [MapReduceRunner-HiveCounter:o.a.h.c.Configuration@2345] - file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/tmp/mapred/staging/yarn1443035616/.staging/job_local1443035616_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
      2015-04-28 20:22:12,644 - WARN [MapReduceRunner-HiveCounter:o.a.h.c.Configuration@2345] - file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/tmp/mapred/staging/yarn1443035616/.staging/job_local1443035616_0008/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
      2015-04-28 20:22:13,978 - WARN [MapReduceRunner-HiveCounter:o.a.h.f.FileUtil@829] - Command 'ln -s /opt/cdap-sdk-3.0.0-SNAPSHOT/data/tmp/mapred/local/job_local1443035616_0008_1430277732925/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.program.jar /opt/cdap-sdk-3.0.0-SNAPSHOT/data/programs/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.program.jar' failed 1 with: ln: /opt/cdap-sdk-3.0.0-SNAPSHOT/data/programs/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.program.jar: No such file or directory
      2015-04-28 20:22:13,980 - WARN [MapReduceRunner-HiveCounter:o.a.h.m.LocalDistributedCacheManagerWithFix@214] - Failed to create symlink: /opt/cdap-sdk-3.0.0-SNAPSHOT/data/tmp/mapred/local/job_local1443035616_0008_1430277732925/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.program.jar <- /opt/cdap-sdk-3.0.0-SNAPSHOT/data/programs/mapreduce.default.SportResults.HiveCounter.dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9.program.jar
      2015-04-28 20:22:14,332 - WARN [MapReduceRunner-HiveCounter:o.a.h.f.FileUtil@829] - Command 'ln -s /opt/cdap-sdk-3.0.0-SNAPSHOT/data/tmp/mapred/local/job_local1443035616_0008_1430277732926/job.jar /opt/cdap-sdk-3.0.0-SNAPSHOT/data/programs/job.jar' failed 1 with: ln: /opt/cdap-sdk-3.0.0-SNAPSHOT/data/programs/job.jar: No such file or directory
      2015-04-28 20:22:14,333 - WARN [MapReduceRunner-HiveCounter:o.a.h.m.LocalDistributedCacheManagerWithFix@214] - Failed to create symlink: /opt/cdap-sdk-3.0.0-SNAPSHOT/data/tmp/mapred/local/job_local1443035616_0008_1430277732926/job.jar <- /opt/cdap-sdk-3.0.0-SNAPSHOT/data/programs/job.jar
      2015-04-28 20:22:14,460 - WARN [MapReduceRunner-HiveCounter:o.a.h.c.Configuration@2345] - file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/tmp/mapred/local/localRunner/yarn/job_local1443035616_0008/job_local1443035616_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
      2015-04-28 20:22:14,462 - WARN [MapReduceRunner-HiveCounter:o.a.h.c.Configuration@2345] - file:/opt/cdap-sdk-3.0.0-SNAPSHOT/data/tmp/mapred/local/localRunner/yarn/job_local1443035616_0008/job_local1443035616_0008.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
      2015-04-28 20:22:14,465 - DEBUG [MapReduceRunner-HiveCounter:c.c.c.i.a.r.AbstractProgramController@230] - Program started: HiveCounter dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9
      2015-04-28 20:22:14,816 - WARN [LocalJobRunner Map Task Executor #0:c.c.c.c.c.Configuration@570] - security.auth.server.address is deprecated. Instead, use security.auth.server.bind.address
      2015-04-28 20:22:14,835 - WARN [LocalJobRunner Map Task Executor #0:c.c.c.c.c.Configuration@1748] - java.io.ByteArrayInputStream@35db58b8:an attempt to override final parameter: stream.instance.file.prefix; Ignoring.
      2015-04-28 20:22:14,865 - WARN [LocalJobRunner Map Task Executor #1:c.c.c.c.c.Configuration@570] - security.auth.server.address is deprecated. Instead, use security.auth.server.bind.address
      2015-04-28 20:22:14,870 - WARN [LocalJobRunner Map Task Executor #1:c.c.c.c.c.Configuration@1748] - java.io.ByteArrayInputStream@2a698a14:an attempt to override final parameter: stream.instance.file.prefix; Ignoring.
      2015-04-28 20:22:17,235 - WARN [LocalJobRunner Reducer Task Executor #0:c.c.c.c.c.Configuration@570] - security.auth.server.address is deprecated. Instead, use security.auth.server.bind.address
      2015-04-28 20:22:17,264 - WARN [LocalJobRunner Reducer Task Executor #0:c.c.c.c.c.Configuration@1748] - java.io.ByteArrayInputStream@1c112ba8:an attempt to override final parameter: stream.instance.file.prefix; Ignoring.
      2015-04-28 20:22:20,550 - INFO [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@319] - MapReduce Job is complete, status: true, job: job=HiveCounter,=namespaceId=default, applicationId=SportResults, program=HiveCounter, runid=dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9
      2015-04-28 20:22:21,810 - INFO [executor-334:c.c.c.i.a.r.b.i.LocalClientProtocolProvider@42] - Using framework: local
      2015-04-28 20:22:21,810 - INFO [executor-334:c.c.c.i.a.r.b.i.LocalClientProtocolProvider@50] - Using tracker: clocal
      2015-04-28 20:22:21,812 - WARN [executor-334:c.c.c.a.m.MRJobClient@77] - JobClient failed to get all jobs. java.lang.NullPointerException: null at org.apache.hadoop.mapred.JobClient.getAllJobs(JobClient.java:814) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na] at co.cask.cdap.app.mapreduce.MRJobClient.getMRJobInfo(MRJobClient.java:75) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at co.cask.cdap.gateway.handlers.ProgramLifecycleHttpHandler.getMapReduceInfo(ProgramLifecycleHttpHandler.java:237) [co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_65] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_65] at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:85) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:41) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [na:1.6.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [na:1.6.0_65] at java.lang.Thread.run(Thread.java:695) [na:1.6.0_65]
      2015-04-28 20:22:21,815 - DEBUG [executor-334:c.c.c.g.h.ProgramLifecycleHttpHandler@239] - Failed to get run history from JobClient for runId: namespace:default/application:SportResults/program:mapreduce:HiveCounter/run:dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9. Falling back to Metrics system. java.io.IOException: java.lang.NullPointerException at co.cask.cdap.app.mapreduce.MRJobClient.getMRJobInfo(MRJobClient.java:78) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at co.cask.cdap.gateway.handlers.ProgramLifecycleHttpHandler.getMapReduceInfo(ProgramLifecycleHttpHandler.java:237) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source) ~[na:na] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) ~[na:1.6.0_65] at java.lang.reflect.Method.invoke(Method.java:597) ~[na:1.6.0_65] at co.cask.http.HttpMethodInfo.invoke(HttpMethodInfo.java:85) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at co.cask.http.HttpDispatcher.messageReceived(HttpDispatcher.java:41) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at org.jboss.netty.handler.execution.OrderedMemoryAwareThreadPoolExecutor$ChildExecutor.run(OrderedMemoryAwareThreadPoolExecutor.java:314) [co.cask.cdap.cdap-explore-jdbc-3.0.0-SNAPSHOT.jar:na] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) [na:1.6.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) [na:1.6.0_65] at java.lang.Thread.run(Thread.java:695) [na:1.6.0_65] Caused by: java.lang.NullPointerException: null at org.apache.hadoop.mapred.JobClient.getAllJobs(JobClient.java:814) ~[org.apache.hadoop.hadoop-mapreduce-client-core-2.3.0.jar:na] at co.cask.cdap.app.mapreduce.MRJobClient.getMRJobInfo(MRJobClient.java:75) ~[co.cask.cdap.cdap-app-fabric-3.0.0-SNAPSHOT.jar:na] ... 15 common frames omitted
      2015-04-28 20:22:22,562 - INFO [MapReduceRunner-HiveCounter:c.c.c.i.a.r.b.MapReduceRuntimeService@339] - Committing MapReduce Job transaction: job=HiveCounter,=namespaceId=default, applicationId=SportResults, program=HiveCounter, runid=dfbe7cbc-ee1e-11e4-afda-0e3dd7e64ad9
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                edwin Edwin Elia
                Reporter:
                andreas Andreas Neumann
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: