Uploaded image for project: 'CDAP'
  1. CDAP
  2. CDAP-12690

Better error message if 'java.security.auth.login.config' is misconfigured.

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Security
    • Labels:
    • Rank:
      1|i008g7:

      Description

      Users can configure 'java.security.auth.login.config' in cdap-env.sh. If this is misconfigured, such as if the file doesn't exist, then the following error logs show up in auth server logs. Auth server does not fail, but simply becomes follower.

      We should do a file existence check and a check on the validity of the config (keytab, principal, etc).

      Also, the following log line is not visible in the auth server logs:

      17/10/03 19:04:34 INFO zookeeper.Login: successfully logged in.
      
      2017-10-03 10:19:08,292 - WARN  [leader-election-leader:o.a.t.i.z.LeaderElection@236] - Exception thrown when calling leader() method. Withdraw from the leader election process.
      java.lang.RuntimeException: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$InvalidACLException: KeeperErrorCode = InvalidACL for /keys/676805029
              at com.google.common.base.Throwables.propagate(Throwables.java:160) ~[com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.security.zookeeper.SharedResourceCache.put(SharedResourceCache.java:195) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at co.cask.cdap.security.auth.DistributedKeyManager.addKey(DistributedKeyManager.java:135) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at co.cask.cdap.security.auth.AbstractKeyManager.generateKey(AbstractKeyManager.java:125) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at co.cask.cdap.security.auth.DistributedKeyManager.rotateKey(DistributedKeyManager.java:142) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at co.cask.cdap.security.auth.DistributedKeyManager.access$100(DistributedKeyManager.java:51) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at co.cask.cdap.security.auth.DistributedKeyManager$1.leader(DistributedKeyManager.java:104) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at org.apache.twill.internal.zookeeper.LeaderElection.becomeLeader(LeaderElection.java:234) [org.apache.twill.twill-zookeeper-0.12.0.jar:0.12.0]
              at org.apache.twill.internal.zookeeper.LeaderElection.access$1900(LeaderElection.java:54) [org.apache.twill.twill-zookeeper-0.12.0.jar:0.12.0]
              at org.apache.twill.internal.zookeeper.LeaderElection$5.onSuccess(LeaderElection.java:212) [org.apache.twill.twill-zookeeper-0.12.0.jar:0.12.0]
              at org.apache.twill.internal.zookeeper.LeaderElection$5.onSuccess(LeaderElection.java:191) [org.apache.twill.twill-zookeeper-0.12.0.jar:0.12.0]
              at com.google.common.util.concurrent.Futures$6.run(Futures.java:799) [com.google.guava.guava-13.0.1.jar:na]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_66]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_66]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
      Caused by: java.util.concurrent.ExecutionException: org.apache.zookeeper.KeeperException$InvalidACLException: KeeperErrorCode = InvalidACL for /keys/676805029
              at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:294) ~[com.google.guava.guava-13.0.1.jar:na]
              at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:281) ~[com.google.guava.guava-13.0.1.jar:na]
              at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116) ~[com.google.guava.guava-13.0.1.jar:na]
              at co.cask.cdap.security.zookeeper.SharedResourceCache.put(SharedResourceCache.java:192) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              ... 13 common frames omitted
      org.apache.zookeeper.KeeperException$InvalidACLException: KeeperErrorCode = InvalidACL for /keys/676805029
              at org.apache.zookeeper.KeeperException.create(KeeperException.java:121) ~[zookeeper-3.4.6.2.6.1.0-129.jar:3.4.6-129--1]
              at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) ~[zookeeper-3.4.6.2.6.1.0-129.jar:3.4.6-129--1]
              at org.apache.twill.internal.zookeeper.DefaultZKClientService$Callbacks$1.processResult(DefaultZKClientService.java:574) ~[org.apache.twill.twill-zookeeper-0.12.0.jar:0.12.0]
              at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:617) ~[zookeeper-3.4.6.2.6.1.0-129.jar:3.4.6-129--1]
              at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) ~[zookeeper-3.4.6.2.6.1.0-129.jar:3.4.6-129--1]
      2017-10-03 10:22:19,542 - WARN  [qtp1232973470-71:o.e.j.s.ServletHandler@563] - /token
      org.jboss.resteasy.spi.UnhandledException: java.lang.NullPointerException
              at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:76) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:212) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:149) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:372) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:179) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:220) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at javax.servlet.http.HttpServlet.service(HttpServlet.java:770) ~[javax.servlet.javax.servlet-api-3.0.1.jar:3.0.1]
              at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684) ~[org.eclipse.jetty.jetty-servlet-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) [org.eclipse.jetty.jetty-servlet-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:522) [org.eclipse.jetty.jetty-security-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1088) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429) [org.eclipse.jetty.jetty-servlet-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.Server.handle(Server.java:370) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:971) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1033) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644) [org.eclipse.jetty.jetty-http-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) [org.eclipse.jetty.jetty-http-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82) [org.eclipse.jetty.jetty-server-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.io.nio.SslConnection.handle(SslConnection.java:196) [org.eclipse.jetty.jetty-io-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696) [org.eclipse.jetty.jetty-io-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53) [org.eclipse.jetty.jetty-io-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) [org.eclipse.jetty.jetty-util-8.1.15.v20140411.jar:8.1.15.v20140411]
              at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) [org.eclipse.jetty.jetty-util-8.1.15.v20140411.jar:8.1.15.v20140411]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
      Caused by: java.lang.NullPointerException: null
              at co.cask.cdap.security.auth.AbstractKeyManager.generateMAC(AbstractKeyManager.java:147) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at co.cask.cdap.security.auth.TokenManager.signIdentifier(TokenManager.java:58) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at co.cask.cdap.security.server.GrantAccessToken.grantToken(GrantAccessToken.java:126) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at co.cask.cdap.security.server.GrantAccessToken.token(GrantAccessToken.java:100) ~[co.cask.cdap.cdap-security-4.3.1.jar:na]
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_66]
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_66]
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_66]
              at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
              at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:137) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:296) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:250) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:237) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:356) ~[org.jboss.resteasy.resteasy-jaxrs-3.0.8.Final.jar:na]
              ... 28 common frames omitted
      

        Attachments

          Activity

            People

            • Assignee:
              bhooshan Bhooshan Mogal
              Reporter:
              ali.anwar Ali Anwar
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: