Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-10191

S3 Sources throw ClassNotFoundException: org.apache.http.HttpEntity

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.0.1
    • Fix Version/s: 4.1.0
    • Component/s: Pipelines
    • Labels:
      None
    • Release Notes:
      Fixed an issue with the S3 source and sink plugins in the CDAP Standalone.
    • Rank:
      1|hzzvh3:

      Description

      Seeing the following error message when running a pipeline with an S3 source.

      2017-02-17 01:14:18,756 - WARN  [HiveServer2-Background-Pool: Thread-353:c.c.c.c.l.RedirectedPrintStream@91] - OK
      Exception in thread "MapReduceRunner-phase-1" java.lang.NoClassDefFoundError: org/apache/http/HttpEntity
      	at org.apache.hadoop.fs.s3native.Jets3tNativeFileSystemStore.initialize(Jets3tNativeFileSystemStore.java:66)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
      	at org.apache.hadoop.fs.s3native.$Proxy67.initialize(Unknown Source)
      	at org.apache.hadoop.fs.s3native.NativeS3FileSystem.initialize(NativeS3FileSystem.java:272)
      	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2316)
      	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:90)
      	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2350)
      	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2332)
      	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:369)
      	at co.cask.hydrator.plugin.batch.source.FileBatchSource.prepareRun(FileBatchSource.java:178)
      	at co.cask.hydrator.plugin.batch.source.FileBatchSource.prepareRun(FileBatchSource.java:69)
      	at co.cask.cdap.etl.batch.LoggedBatchConfigurable$1.call(LoggedBatchConfigurable.java:44)
      	at co.cask.cdap.etl.batch.LoggedBatchConfigurable$1.call(LoggedBatchConfigurable.java:41)
      	at co.cask.cdap.etl.log.LogContext.run(LogContext.java:59)
      	at co.cask.cdap.etl.batch.LoggedBatchConfigurable.prepareRun(LoggedBatchConfigurable.java:41)
      	at co.cask.cdap.etl.batch.mapreduce.ETLMapReduce.initialize(ETLMapReduce.java:185)
      	at co.cask.cdap.api.mapreduce.AbstractMapReduce.initialize(AbstractMapReduce.java:122)
      	at co.cask.cdap.api.mapreduce.AbstractMapReduce.initialize(AbstractMapReduce.java:33)
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService.doInitialize(MapReduceRuntimeService.java:561)
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService.access$100(MapReduceRuntimeService.java:143)
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService$2.run(MapReduceRuntimeService.java:539)
      	at co.cask.cdap.internal.app.runtime.AbstractContext$2.run(AbstractContext.java:427)
      	at co.cask.cdap.data2.transaction.Transactions$CacheBasedTransactional.finishExecute(Transactions.java:242)
      	at co.cask.cdap.data2.transaction.Transactions$CacheBasedTransactional.execute(Transactions.java:230)
      	at co.cask.cdap.internal.app.runtime.AbstractContext.execute(AbstractContext.java:422)
      	at co.cask.cdap.internal.app.runtime.AbstractContext.execute(AbstractContext.java:410)
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService.beforeSubmit(MapReduceRuntimeService.java:536)
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService.startUp(MapReduceRuntimeService.java:225)
      	at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:47)
      	at co.cask.cdap.internal.app.runtime.batch.MapReduceRuntimeService$1$1.run(MapReduceRuntimeService.java:440)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.ClassNotFoundException: org.apache.http.HttpEntity
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      	at co.cask.cdap.common.lang.InterceptableClassLoader.findClass(InterceptableClassLoader.java:46)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      	... 36 more
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ali.anwar Ali Anwar
                Reporter:
                russellsavage Russ Savage
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: