Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-5793

Explore should use updated credentials while launching Hive jobs

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.0, 3.2.0, 3.1.0, 3.0.0, 2.8.0
    • Fix Version/s: 3.3.4, 3.4.0
    • Component/s: Explore
    • Labels:
    • Release Notes:
      Explore jobs properly use the latest/updated delegation tokens
    • Rank:
      1|hzzbxr:

      Description

      When Twill launches a YARN job on a secure Hadoop cluster, it creates an initial credentials file with delegation tokens. This file is used during the initial run of the container. Environment variable HADOOP_TOKEN_FILE_LOCATION for the container points to this file.

      To refresh credentials, a new credentials file is created on HDFS with new delegation tokens (note that the old tokens are not refreshed). And containers load the new credentials directly from HDFS file. This works fine for all CDAP programs.

      However when Explore launches Hive jobs, it still uses HADOOP_TOKEN_FILE_LOCATION as credentials file. This leads to Explore jobs failing after the initial tokens expire. We'll need Explore to use the latest credentials while launching new jobs.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ali.anwar Ali Anwar
                Reporter:
                poorna Poorna Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: