Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-6455

Explore query fails sometimes with java.lang.NoSuchMethodError

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.5.0
    • Fix Version/s: 3.5.0
    • Component/s: Explore
    • Labels:
      None
    • Release Notes:
      Fixed the classpath of the MapReduce program launched by the explore service to include cdap-common.jar at the beginning.
    • Rank:
      1|hzzfwv:

      Description

      Explore query fails sometimes with NoSuchMethodError.
      Following is the stack trace:

      08:39:54.383 [main] ERROR org.apache.hadoop.mapred.YarnChild - Error running child : java.lang.NoSuchMethodError: org.apache.twill.filesystem.FileContextLocationFactory.<init>(Lorg/apache/hadoop/conf/Configuration;Lorg/apache/hadoop/fs/FileContext;Ljava/lang/String;)V
      	at co.cask.cdap.common.guice.LocationRuntimeModule$HDFSLocationModule.providesLocationFactory(LocationRuntimeModule.java:102)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.
      

      The value of the mapreduce.application.classpath from corresponding container is

      $PWD/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*.
      

      The current working directory contains both jars from Apache Twill as well as CDAP. We want map tasks to pick up FileContextLocationFactory from CDAP instead of Apache Twill. However because the classpath contains "$PWD/*" there is no guarantee about which version of the FileContextLocationFactory would get picked up.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sagar Sagar Kapare
                Reporter:
                sagar Sagar Kapare
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: