Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-5855

HDFS delegation token of CDAP Master is getting cancelled by RM after an explore query run

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.4.0, 3.3.0
    • Fix Version/s: 3.3.4, 3.4.1, 3.4.0
    • Component/s: Explore, Master
    • Labels:
      None
    • Release Notes:
      Avoid the cancellation of delegation tokens upon completion of Explore-launched MapReduce and Spark jobs, as these delegation tokens are shared by CDAP system services.
    • Rank:
      1|hzzcb3:

      Description

      We had seen issues with shared delegation tokens getting cancelled by RM after MR jobs finished. In CDAP-4767, we stared providing each program with separate delegation tokens.

      However, we could not provide separate delegation tokens to every MR job started by Explore. Hence Explore MR jobs use the same delegation tokens as of Explore Service container. Also, all Master services share the same delegation tokens due to TWILL-166.

      We are now seeing delegation tokens of Master getting cancelled after Explore queries are run. We need to figure out a way to provide every Explore query its own set of delegation tokens.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                poorna Poorna Chandra
                Reporter:
                poorna Poorna Chandra
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: