Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-7648

Apache HTTP Client may conflict with other versions

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: Master
    • Labels:
      None
    • Release Notes:
      Support Amazon EMR 4.6.0.
    • Rank:
      1|hzzopb:

      Description

      When spinning up CDAP on EMR AMI 4.6.0, I noticed an error which prevented the CDAP Master from successfully starting up.

      Exception in thread " STARTING" java.lang.NoSuchMethodError: org.apache.http.impl.client.DefaultHttpClient.execute(Lorg/apache/http/client/methods/HttpUriRequest;)Lorg/apache/http/client/methods/CloseableHttpResponse;
              at amazon.emr.metrics.ClientUtil.getInstanceId(ClientUtil.java:115)
              at amazon.emr.metrics.MetricsConfig.getInstanceId(MetricsConfig.java:294)
              at amazon.emr.metrics.MetricsConfig.<init>(MetricsConfig.java:195)
              at amazon.emr.metrics.MetricsConfig.<init>(MetricsConfig.java:182)
              at amazon.emr.metrics.MetricsConfig.<init>(MetricsConfig.java:177)
              at amazon.emr.metrics.MetricsSaver.ensureSingleton(MetricsSaver.java:652)
              at amazon.emr.metrics.MetricsSaver.addInternal(MetricsSaver.java:332)
              at amazon.emr.metrics.MetricsSaver.addValue(MetricsSaver.java:178)
              at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1667)
              at org.apache.hadoop.hdfs.DFSClient.primitiveCreate(DFSClient.java:1753)
              at org.apache.hadoop.fs.Hdfs.createInternal(Hdfs.java:102)
              at org.apache.hadoop.fs.Hdfs.createInternal(Hdfs.java:58)
              at org.apache.hadoop.fs.AbstractFileSystem.create(AbstractFileSystem.java:584)
              at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:686)
              at org.apache.hadoop.fs.FileContext$3.next(FileContext.java:682)
              at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
              at org.apache.hadoop.fs.FileContext.create(FileContext.java:682)
              at org.apache.twill.filesystem.FileContextLocation.getOutputStream(FileContextLocation.java:113)
              at org.apache.twill.internal.ApplicationBundler.createBundle(ApplicationBundler.java:165)
              at org.apache.twill.internal.ApplicationBundler.createBundle(ApplicationBundler.java:126)
              at org.apache.twill.yarn.YarnTwillPreparer.createAppMasterJar(YarnTwillPreparer.java:501)
              at org.apache.twill.yarn.YarnTwillPreparer.access$200(YarnTwillPreparer.java:113)
              at org.apache.twill.yarn.YarnTwillPreparer$1.call(YarnTwillPreparer.java:363)
              at org.apache.twill.yarn.YarnTwillPreparer$1.call(YarnTwillPreparer.java:353)
              at org.apache.twill.yarn.YarnTwillController.doStartUp(YarnTwillController.java:116)
              at org.apache.twill.internal.AbstractZKServiceController.startUp(AbstractZKServiceController.java:75)
              at org.apache.twill.internal.AbstractExecutionServiceController$ServiceDelegate.startUp(AbstractExecutionServiceController.java:175)
              at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43)
              at java.lang.Thread.run(Thread.java:745)
      

      Further investigation shows that the version of the Apache HTTP Core/Client shipped with CDAP was older than the version in the CLASSPATH as derived from HBase.

      Removing these JARs allows CDAP to start up, successfully.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ali.anwar Ali Anwar
                Reporter:
                chris Chris Gianelloni
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: