CDAP
  1. CDAP
  2. CDAP-4006

obscure init script failure if hive scratch dir not created

    Details

    • Type: Improvement Improvement
    • Status: Resolved Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0
    • Component/s: Build System
    • Labels:
      None
    • Rank:
      1|hzz1tz:

      Description

      If explore is enabled, but the hive hdfs scratch dir is not yet created, master will fail to start with the following error:

      Exception in thread "main" java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hive.common.util.HiveVersionInfo
              at com.google.common.base.Throwables.propagate(Throwables.java:160)
              at co.cask.cdap.explore.service.ExploreServiceUtils.checkHiveSupport(ExploreServiceUtils.java:227)
              at co.cask.cdap.explore.service.ExploreServiceUtils.checkHiveSupport(ExploreServiceUtils.java:190)
              at co.cask.cdap.data.runtime.main.MasterServiceMain.checkExploreRequirements(MasterServiceMain.java:440)
              at co.cask.cdap.data.runtime.main.MasterServiceMain.init(MasterServiceMain.java:174)
              at co.cask.cdap.common.runtime.DaemonMain.doMain(DaemonMain.java:36)
              at co.cask.cdap.data.runtime.main.MasterServiceMain.main(MasterServiceMain.java:147)
      Caused by: java.lang.ClassNotFoundException: org.apache.hive.common.util.HiveVersionInfo
              at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
              at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
              at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
              at co.cask.cdap.explore.service.ExploreServiceUtils.checkHiveSupport(ExploreServiceUtils.java:213)
              ... 5 more
      

      root cause is the following command invoked by the init scripts fails:

      $ hive -e 'set -v'
      
      Logging initialized using configuration in jar:file:/opt/mapr/hive/hive-1.0/lib/hive-common-1.0.0-mapr-1508.jar!/hive-log4j.properties
      2015-10-16 05:23:10,3392 ERROR JniCommon fs/client/fileclient/cc/jni_MapRClient.cc:1306 Thread: 27662 mkdirs failed for /user/cdap/tmp/hive, error 13
      Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.security.AccessControlException: User cdap(user id 525)  has been denied access to create hive
      	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
      	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:626)
      	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:570)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
      Caused by: org.apache.hadoop.security.AccessControlException: User cdap(user id 525)  has been denied access to create hive
      	at com.mapr.fs.MapRFileSystem.makeDir(MapRFileSystem.java:1143)
      	at com.mapr.fs.MapRFileSystem.mkdirs(MapRFileSystem.java:1163)
      	at org.apache.hadoop.hive.ql.exec.Utilities.createDirsWithPermission(Utilities.java:3545)
      	at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:524)
      	at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:478)
      	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:430)
      
      

      We need to check exit code of this and fail gracefully

        Activity

        Show
        Chris Gianelloni added a comment - https://github.com/caskdata/cdap/pull/4688

          People

          • Assignee:
            Chris Gianelloni
            Reporter:
            Derek Wood
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: