Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-3693

Support HDP 2.3

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0
    • Component/s: Explore
    • Labels:
      None
    • Release Notes:
      Added support for CDAP to work with HDP 2.3
    • Rank:
      1|hzyzxb:

      Description

      HDP 2.3 ships with Hive 1.2, for which CDAP Master throws an exception:

      2015-09-17 00:51:27,184 - INFO  [main:c.c.c.d.r.m.MasterServiceMain@145] - Starting MasterServiceMain
      2015-09-17 00:51:27,292 - WARN  [main:c.c.c.c.c.Configuration@570] - security.auth.server.address is deprecated. Instead, use security.auth.server.bind.address
      2015-09-17 00:51:28,394 - WARN  [main:o.a.h.u.NativeCodeLoader@62] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      2015-09-17 00:51:29,501 - INFO  [main:c.c.c.c.g.LocationRuntimeModule$HDFSLocationModule@105] - HDFS namespace is /cdap
      2015-09-17 00:51:29,790 - INFO  [main:c.c.c.d.d.InMemoryDatasetFramework@122] - Adding Default module basicKVTable to system namespace
      2015-09-17 00:51:30,209 - INFO  [main:c.c.c.d.r.m.TokenSecureStoreUpdater@150] - Setting token renewal time to: 86100000 ms
      2015-09-17 00:51:30,211 - INFO  [main:c.c.c.e.s.ExploreServiceUtils@198] - Hadoop version is: 2.7.1.2.3.0.0-2557
      Exception in thread "main" java.lang.RuntimeException: Hive distribution not supported. Set the configuration 'explore.enabled' to false to start up without Explore.
              at co.cask.cdap.explore.service.ExploreServiceUtils.checkHiveSupport(ExploreServiceUtils.java:227)
              at co.cask.cdap.explore.service.ExploreServiceUtils.checkHiveSupport(ExploreServiceUtils.java:189)
              at co.cask.cdap.data.runtime.main.MasterServiceMain.checkExploreRequirements(MasterServiceMain.java:435)
              at co.cask.cdap.data.runtime.main.MasterServiceMain.init(MasterServiceMain.java:169)
              at co.cask.cdap.common.runtime.DaemonMain.doMain(DaemonMain.java:36)
              at co.cask.cdap.data.runtime.main.MasterServiceMain.main(MasterServiceMain.java:146)
      
      # rpm -qa | grep -i -e hadoop -e hive
      hadoop_2_3_0_0_2557-mapreduce-2.7.1.2.3.0.0-2557.el6.x86_64
      hadooplzo_2_3_0_0_2557-native-0.6.0.2.3.0.0-2557.el6.x86_64
      hadoop-libhdfs-2.7.1.2.3.0.0-2557.el6.noarch
      hadoop_2_3_0_0_2557-yarn-2.7.1.2.3.0.0-2557.el6.x86_64
      hadoop_2_3_0_0_2557-hdfs-2.7.1.2.3.0.0-2557.el6.x86_64
      hive_2_3_0_0_2557-jdbc-1.2.1.2.3.0.0-2557.el6.noarch
      hive_2_3_0_0_2557-1.2.1.2.3.0.0-2557.el6.noarch
      hadooplzo-0.6.0.2.3.0.0-2557.el6.noarch
      hadooplzo-native-0.6.0.2.3.0.0-2557.el6.noarch
      hadoop_2_3_0_0_2557-libhdfs-2.7.1.2.3.0.0-2557.el6.x86_64
      ranger_2_3_0_0_2557-hive-plugin-0.5.0.2.3.0.0-2557.el6.x86_64
      hadoop_2_3_0_0_2557-2.7.1.2.3.0.0-2557.el6.x86_64
      hadoop_2_3_0_0_2557-client-2.7.1.2.3.0.0-2557.el6.x86_64
      hive-1.2.1.2.3.0.0-2557.el6.noarch
      hadooplzo_2_3_0_0_2557-0.6.0.2.3.0.0-2557.el6.x86_64
      hadoop-client-2.7.1.2.3.0.0-2557.el6.noarch
      
      # hive --version
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/usr/hdp/2.3.0.0-2557/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/usr/hdp/2.3.0.0-2557/spark/lib/spark-assembly-1.3.1.2.3.0.0-2557-hadoop2.7.1.2.3.0.0-2557.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
      WARNING: Use "yarn jar" to launch YARN applications.
      SLF4J: Class path contains multiple SLF4J bindings.
      SLF4J: Found binding in [jar:file:/usr/hdp/2.3.0.0-2557/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: Found binding in [jar:file:/usr/hdp/2.3.0.0-2557/spark/lib/spark-assembly-1.3.1.2.3.0.0-2557-hadoop2.7.1.2.3.0.0-2557.jar!/org/slf4j/impl/StaticLoggerBinder.class]
      SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
      SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
      Hive 1.2.1.2.3.0.0-2557
      Subversion git://ip-10-0-0-89.ec2.internal/grid/0/jenkins/workspace/HDP-dal-centos6/bigtop/build/hive/rpm/BUILD/hive-1.2.1.2.3.0.0 -r 727487c3d35ff6ed9abdc531317d80d1ef08e29f
      Compiled by jenkins on Tue Jul 14 09:48:43 EDT 2015
      From source with checksum 981fdb96a23d0014586ca7a7f2824e6c
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sree Sreevatsan Raman
                Reporter:
                derek Derek Wood
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: