Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-12937

HiveSink does not work with Spark as an engine

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.3.0, 4.2.0, 4.1.0, 4.0.0
    • Fix Version/s: None
    • Component/s: Spark
    • Labels:
      None
    • Rank:
      1|i009tr:

      Description

      HiveSink fails to write if the execution engine is hive with the following exception:

      2017-11-20 20:34:32,990 - WARN  [main:o.a.h.s.UserGroupInformation@1923] - PriviledgedActionException as:cdap (auth:SIMPLE) cause:org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN]
      2017-11-20 20:34:32,992 - WARN  [main:o.a.h.i.Client@713] - Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN]
      

      This might be because HiveSink uses HCatOutputFormat which does not work well spark-hive in kerberos. Spark expects to use HiveContext.

        Attachments

          Activity

            People

            • Assignee:
              bhooshan Bhooshan Mogal
              Reporter:
              rsinha Rohit Sinha
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: