Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-2953

HiveServer2 support in a secure cluster

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.1.0
    • Fix Version/s: 3.1.0
    • Component/s: None
    • Labels:
      None
    • Rank:
      1|hzyvq7:

      Description

      In a secure Hadoop cluster, CDAP Explore doesn't work:

      2015-07-08 00:28:50,571 - INFO  [leader-election-election-master.services:c.c.c.d.r.m.MasterTwillApplication@76] - Adding explore runnable.
      2015-07-08 00:29:50,288 - INFO  [leader-election-election-master.services:c.c.c.e.s.HiveTokenUtils@44] - Obtaining delegation token for Hive
      2015-07-08 00:29:50,578 - ERROR [leader-election-election-master.services:o.a.h.h.q.m.Hive@2578] - MetaException(message:java.lang.NullPointerException)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_delegation_token_result$get_delegation_token_resultStandardScheme.read(ThriftHiveMetastore.java)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_delegation_token_result$get_delegation_token_resultStandardScheme.read(ThriftHiveMetastore.java)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_delegation_token_result.read(ThriftHiveMetastore.java)
              at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_delegation_token(ThriftHiveMetastore.java:3293)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_delegation_token(ThriftHiveMetastore.java:3279)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDelegationToken(HiveMetaStoreClient.java:1521)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
              at com.sun.proxy.$Proxy46.getDelegationToken(Unknown Source)
              at org.apache.hadoop.hive.ql.metadata.Hive.getDelegationToken(Hive.java:2576)        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at co.cask.cdap.explore.security.HiveTokenUtils.obtainToken(HiveTokenUtils.java:56)
              at co.cask.cdap.data.runtime.main.TokenSecureStoreUpdater.refreshCredentials(TokenSecureStoreUpdater.java:81)
              at co.cask.cdap.data.runtime.main.TokenSecureStoreUpdater.update(TokenSecureStoreUpdater.java:167)
              at co.cask.cdap.data.runtime.main.MasterServiceMain.startTwillApplication(MasterServiceMain.java:603)
              at co.cask.cdap.data.runtime.main.MasterServiceMain.monitorTwillApplication(MasterServiceMain.java:490)
              at co.cask.cdap.data.runtime.main.MasterServiceMain.access$500(MasterServiceMain.java:120)
              at co.cask.cdap.data.runtime.main.MasterServiceMain$1.leader(MasterServiceMain.java:371)
              at org.apache.twill.internal.zookeeper.LeaderElection.becomeLeader(LeaderElection.java:229)
              at org.apache.twill.internal.zookeeper.LeaderElection.access$1800(LeaderElection.java:53)
              at org.apache.twill.internal.zookeeper.LeaderElection$5.onSuccess(LeaderElection.java:207)
              at org.apache.twill.internal.zookeeper.LeaderElection$5.onSuccess(LeaderElection.java:186)
              at com.google.common.util.concurrent.Futures$6.run(Futures.java:799)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      2015-07-08 00:29:50,594 - ERROR [leader-election-election-master.services:c.c.c.e.s.HiveTokenUtils@66] - Exception when fetching delegation token from Hive MetaStore
      java.lang.reflect.InvocationTargetException: null
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_75]
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_75]
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_75]
              at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_75]
              at co.cask.cdap.explore.security.HiveTokenUtils.obtainToken(HiveTokenUtils.java:56) ~[co.cask.cdap.cdap-explore-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.TokenSecureStoreUpdater.refreshCredentials(TokenSecureStoreUpdater.java:81) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.TokenSecureStoreUpdater.update(TokenSecureStoreUpdater.java:167) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.MasterServiceMain.startTwillApplication(MasterServiceMain.java:603) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.MasterServiceMain.monitorTwillApplication(MasterServiceMain.java:490) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.MasterServiceMain.access$500(MasterServiceMain.java:120) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.MasterServiceMain$1.leader(MasterServiceMain.java:371) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at org.apache.twill.internal.zookeeper.LeaderElection.becomeLeader(LeaderElection.java:229) [org.apache.twill.twill-zookeeper-0.6.0-incubating-SNAPSHOT.jar:0.6.0-incubating-SNAPSHOT]
              at org.apache.twill.internal.zookeeper.LeaderElection.access$1800(LeaderElection.java:53) [org.apache.twill.twill-zookeeper-0.6.0-incubating-SNAPSHOT.jar:0.6.0-incubating-SNAPSHOT]
              at org.apache.twill.internal.zookeeper.LeaderElection$5.onSuccess(LeaderElection.java:207) [org.apache.twill.twill-zookeeper-0.6.0-incubating-SNAPSHOT.jar:0.6.0-incubating-SNAPSHOT]
              at org.apache.twill.internal.zookeeper.LeaderElection$5.onSuccess(LeaderElection.java:186) [org.apache.twill.twill-zookeeper-0.6.0-incubating-SNAPSHOT.jar:0.6.0-incubating-SNAPSHOT]
              at com.google.common.util.concurrent.Futures$6.run(Futures.java:799) [com.google.guava.guava-13.0.1.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
              at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.NullPointerException)
              at org.apache.hadoop.hive.ql.metadata.Hive.getDelegationToken(Hive.java:2579) ~[hive-exec-0.13.0.2.1.7.0-784.jar:0.13.0.2.1.7.0-784]
              ... 19 common frames omitted
      Caused by: org.apache.hadoop.hive.metastore.api.MetaException: java.lang.NullPointerException
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_delegation_token_result$get_delegation_token_resultStandardScheme.read(ThriftHiveMetastore.java) ~[hive-metastore-0.13.0.2.1.7.0-784.jar:0.13.0.2.1.7.0-784]
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_delegation_token_result$get_delegation_token_resultStandardScheme.read(ThriftHiveMetastore.java) ~[hive-metastore-0.13.0.2.1.7.0-784.jar:0.13.0.2.1.7.0-784]
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_delegation_token_result.read(ThriftHiveMetastore.java) ~[hive-metastore-0.13.0.2.1.7.0-784.jar:0.13.0.2.1.7.0-784]
              at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) ~[org.apache.thrift.libthrift-0.9.0.jar:0.9.0]
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_delegation_token(ThriftHiveMetastore.java:3293) ~[hive-metastore-0.13.0.2.1.7.0-784.jar:0.13.0.2.1.7.0-784]
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_delegation_token(ThriftHiveMetastore.java:3279) ~[hive-metastore-0.13.0.2.1.7.0-784.jar:0.13.0.2.1.7.0-784]
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDelegationToken(HiveMetaStoreClient.java:1521) ~[hive-metastore-0.13.0.2.1.7.0-784.jar:0.13.0.2.1.7.0-784]
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_75]
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_75]
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_75]
              at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_75]
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89) ~[hive-metastore-0.13.0.2.1.7.0-784.jar:0.13.0.2.1.7.0-784]
              at com.sun.proxy.$Proxy46.getDelegationToken(Unknown Source) ~[na:na]
              at org.apache.hadoop.hive.ql.metadata.Hive.getDelegationToken(Hive.java:2576) ~[hive-exec-0.13.0.2.1.7.0-784.jar:0.13.0.2.1.7.0-784]
              ... 19 common frames omitted
      2015-07-08 00:29:50,595 - WARN  [leader-election-election-master.services:c.c.c.d.r.m.MasterServiceMain@640] - Failed to cleanup directory data/tmp/master4486846145495862353
      java.io.IOException: Not a directory: data/tmp/master4486846145495862353
              at co.cask.cdap.common.utils.DirUtils.deleteDirectoryContents(DirUtils.java:61) ~[co.cask.cdap.cdap-common-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.common.utils.DirUtils.deleteDirectoryContents(DirUtils.java:49) ~[co.cask.cdap.cdap-common-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.MasterServiceMain.startTwillApplication(MasterServiceMain.java:638) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.MasterServiceMain.monitorTwillApplication(MasterServiceMain.java:490) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.MasterServiceMain.access$500(MasterServiceMain.java:120) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at co.cask.cdap.data.runtime.main.MasterServiceMain$1.leader(MasterServiceMain.java:371) [co.cask.cdap.cdap-master-3.1.0-SNAPSHOT.jar:na]
              at org.apache.twill.internal.zookeeper.LeaderElection.becomeLeader(LeaderElection.java:229) [org.apache.twill.twill-zookeeper-0.6.0-incubating-SNAPSHOT.jar:0.6.0-incubating-SNAPSHOT]
              at org.apache.twill.internal.zookeeper.LeaderElection.access$1800(LeaderElection.java:53) [org.apache.twill.twill-zookeeper-0.6.0-incubating-SNAPSHOT.jar:0.6.0-incubating-SNAPSHOT]
              at org.apache.twill.internal.zookeeper.LeaderElection$5.onSuccess(LeaderElection.java:207) [org.apache.twill.twill-zookeeper-0.6.0-incubating-SNAPSHOT.jar:0.6.0-incubating-SNAPSHOT]
              at org.apache.twill.internal.zookeeper.LeaderElection$5.onSuccess(LeaderElection.java:186) [org.apache.twill.twill-zookeeper-0.6.0-incubating-SNAPSHOT.jar:0.6.0-incubating-SNAPSHOT]
              at com.google.common.util.concurrent.Futures$6.run(Futures.java:799) [com.google.guava.guava-13.0.1.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_75]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_75]
              at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
      2015-07-08 00:29:50,597 - INFO  [leader-election-election-master.services:c.c.c.d.r.m.MasterServiceMain$1@382] - Became follower for master services
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                alvin Alvin Wang
                Reporter:
                alvin Alvin Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: