Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-12649

prevent error logging in standby resourceManager

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Master
    • Labels:
      None
    • Rank:
      1|i0086v:

      Description

      It appears in an HA ResourceManager configuration, when CDAP Master is starting up, the following exception is repeatedly logged in the standby ResourceManager:

      2017-09-20 22:04:09,124 INFO  ipc.Server (Server.java:logException(2435)) - IPC Server handler 0 on 8033, call org.apache.hadoop.ha.HAServiceProtocol.getServiceStatus from 10.250.0.17:46428 Call#2366 Retry#0
      org.apache.hadoop.security.AccessControlException: User cdap doesn't have permission to call 'getServiceState'
              at org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.verifyAdminAccess(RMServerUtils.java:191)
              at org.apache.hadoop.yarn.server.resourcemanager.RMServerUtils.verifyAdminAccess(RMServerUtils.java:157)
              at org.apache.hadoop.yarn.server.resourcemanager.AdminService.checkAccess(AdminService.java:230)
              at org.apache.hadoop.yarn.server.resourcemanager.AdminService.getServiceStatus(AdminService.java:370)
              at org.apache.hadoop.ha.protocolPB.HAServiceProtocolServerSideTranslatorPB.getServiceStatus(HAServiceProtocolServerSideTranslatorPB.java:131)
              at org.apache.hadoop.ha.proto.HAServiceProtocolProtos$HAServiceProtocolService$2.callBlockingMethod(HAServiceProtocolProtos.java:4464)
              at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:640)
              at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2351)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2347)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:422)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
              at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2345)
      

      This is a big potential red-herring when troubleshooting why CDAP doesn't startup or similar. I know the yarn client does things in a round-robin fashion, but we should see if we can avoid calling this method or similar.

        Attachments

          Activity

            People

            • Assignee:
              shankar Shankar Selvam
              Reporter:
              derek Derek Wood
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: