Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.1
    • Component/s: CDAP
    • Labels:
      None
    • Release Notes:
      Added support for CDH 5.11.0.
    • Rank:
      1|hzzzaf:

      Description

      Add support for CDH 5.11.0, which comes with hbase version '1.2.0-cdh5.11.0'.
      http://bits.cloudera.com/f08dc57a/cdh/5/

      Currently, with CDAP 4.1.0, CDAP master fails to start up with the following in master logs:

      2017-04-06 00:31:55,915 INFO co.cask.cdap.common.startup.ConfigurationLogger: Important config settings:
      2017-04-06 00:31:56,051 WARN co.cask.cdap.common.conf.Configuration: cdap-site.xml:an attempt to override final parameter: messaging.container.instances;  Ignoring.
      2017-04-06 00:31:56,051 WARN co.cask.cdap.common.conf.Configuration: cdap-site.xml:an attempt to override final parameter: messaging.max.instances;  Ignoring.
      2017-04-06 00:31:56,054 INFO co.cask.cdap.common.startup.ConfigurationLogger:   ssl.external.enabled: false
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   ssl.internal.enabled: false
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   security.enabled: false
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   explore.enabled: true
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   router.bind.port: 11015
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   router.ssl.bind.port: 10443
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   app.ssl.bind.port: 30443
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   dashboard.bind.port: 11011
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   dashboard.ssl.bind.port: 9443
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   security.auth.server.bind.port: 10009
      2017-04-06 00:31:56,055 INFO co.cask.cdap.common.startup.ConfigurationLogger:   security.auth.server.ssl.bind.port: 10010
      2017-04-06 00:31:56,055 INFO co.cask.cdap.master.startup.MasterStartupTool: Hadoop subsystem versions:
      2017-04-06 00:31:56,064 INFO co.cask.cdap.master.startup.MasterStartupTool:   Hadoop version: 2.6.0-cdh5.11.0
      2017-04-06 00:31:56,079 INFO co.cask.cdap.master.startup.MasterStartupTool:   HBase version: 1.2.0-cdh5.11.0
      2017-04-06 00:31:56,080 INFO co.cask.cdap.master.startup.MasterStartupTool:   ZooKeeper version: 3.4.5.1405704
      2017-04-06 00:31:56,082 INFO co.cask.cdap.master.startup.MasterStartupTool:   Kafka version: 0.8.2.2
      2017-04-06 00:31:56,280 INFO co.cask.cdap.master.startup.MasterStartupTool:   Hive version: 1.1.0-cdh5.11.0
      2017-04-06 00:31:56,282 INFO co.cask.cdap.master.startup.MasterStartupTool: CDAP version: 4.1.0-1488315006729
      2017-04-06 00:31:56,282 INFO co.cask.cdap.master.startup.MasterStartupTool: CDAP HBase compat version: HBASE_11
      2017-04-06 00:31:56,284 INFO co.cask.cdap.master.startup.MasterStartupTool: Tephra HBase compat version: HBASE_12
      2017-04-06 00:31:56,626 INFO co.cask.cdap.common.kerberos.SecurityUtil: Logging in as: principal=cdap/<HOSTNAME>@<REALM> keytab=/var/run/c
      loudera-scm-agent/process/48-cdap-CDAP_MASTER/cdap.keytab
      2017-04-06 00:31:58,400 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      2017-04-06 00:31:58,426 INFO co.cask.cdap.master.startup.ConfigurationCheck: Checking that config settings are valid.
      2017-04-06 00:31:58,633 INFO co.cask.cdap.master.startup.ConfigurationCheck:   Configuration successfully verified.
      2017-04-06 00:31:58,633 INFO co.cask.cdap.master.startup.FileSystemCheck: Checking FileSystem availability.
      2017-04-06 00:31:58,778 INFO co.cask.cdap.master.startup.FileSystemCheck:   FileSystem availability successfully verified.
      2017-04-06 00:31:58,780 INFO co.cask.cdap.master.startup.FileSystemCheck: Checking that user cdap has permission to write to /cdap on the FileSystem.
      2017-04-06 00:31:58,856 INFO co.cask.cdap.master.startup.FileSystemCheck:   FileSystem permissions successfully verified.
      2017-04-06 00:31:59,106 WARN co.cask.cdap.common.conf.Configuration: cdap-site.xml:an attempt to override final parameter: messaging.container.instances;  Ignoring.
      2017-04-06 00:31:59,108 WARN co.cask.cdap.common.conf.Configuration: cdap-site.xml:an attempt to override final parameter: messaging.max.instances;  Ignoring.
      2017-04-06 00:31:59,141 WARN co.cask.cdap.common.conf.Configuration: cdap-site.xml:an attempt to override final parameter: messaging.container.instances;  Ignoring.
      2017-04-06 00:31:59,141 WARN co.cask.cdap.common.conf.Configuration: cdap-site.xml:an attempt to override final parameter: messaging.max.instances;  Ignoring.
      2017-04-06 00:31:59,144 WARN co.cask.cdap.common.conf.Configuration: java.io.ByteArrayInputStream@3c5e76f7:an attempt to override final parameter: messaging.max.instances;  Ignoring
      .
      2017-04-06 00:31:59,144 WARN co.cask.cdap.common.conf.Configuration: java.io.ByteArrayInputStream@3c5e76f7:an attempt to override final parameter: messaging.container.instances;  Ig
      noring.
      2017-04-06 00:31:59,145 WARN co.cask.cdap.common.conf.Configuration: java.io.ByteArrayInputStream@3c5e76f7:an attempt to override final parameter: stream.instance.file.prefix;  Igno
      ring.
      2017-04-06 00:31:59,181 INFO co.cask.cdap.master.startup.LogPipelineCheck: Log pipeline configurations verified.
      2017-04-06 00:31:59,181 INFO co.cask.cdap.master.startup.HBaseCheck: Checking HBase version.
      2017-04-06 00:31:59,189 INFO co.cask.cdap.master.startup.HBaseCheck:   HBase version successfully verified.
      2017-04-06 00:31:59,189 INFO co.cask.cdap.master.startup.HBaseCheck: Checking HBase availability.
      2017-04-06 00:31:59,907 INFO co.cask.cdap.master.startup.HBaseCheck:   HBase availability successfully verified.
      2017-04-06 00:31:59,920 INFO co.cask.cdap.master.startup.HBaseCheck: Hbase authorization is disabled. Skipping global admin check for transaction pruning.
      2017-04-06 00:31:59,921 INFO co.cask.cdap.master.startup.YarnCheck: Checking YARN availability -- may take up to 60 seconds.
      2017-04-06 00:32:00,238 INFO co.cask.cdap.master.startup.YarnCheck:   YARN availability successfully verified.
      2017-04-06 00:32:00,238 INFO co.cask.cdap.master.startup.YarnCheck: Checking that YARN has enough resources to run all system services.
      2017-04-06 00:32:00,239 INFO co.cask.cdap.master.startup.YarnCheck:   YARN resources successfully verified.
      2017-04-06 00:32:04,548 INFO co.cask.cdap.common.io.URLConnections: Turning off default caching in URLConnection
      2017-04-06 00:32:05,162 INFO co.cask.cdap.common.io.URLConnections: Turning off default caching in URLConnection
      2017-04-06 00:32:05,332 WARN co.cask.cdap.common.conf.Configuration: cdap-site.xml:an attempt to override final parameter: messaging.container.instances;  Ignoring.
      2017-04-06 00:32:05,332 WARN co.cask.cdap.common.conf.Configuration: cdap-site.xml:an attempt to override final parameter: messaging.max.instances;  Ignoring.
      2017-04-06 00:32:05,825 INFO co.cask.cdap.common.kerberos.SecurityUtil: Logging in as: principal=cdap/<HOSTNAME>@<REALM>, keytab=/var/run/c
      loudera-scm-agent/process/48-cdap-CDAP_MASTER/cdap.keytab
      2017-04-06 00:32:07,717 INFO co.cask.cdap.logging.appender.LogAppenderInitializer: Initializing log appender KafkaLogAppender
      2017-04-06 00:32:08,669 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      2017-04-06 00:32:10,426 INFO co.cask.cdap.data.runtime.main.MasterServiceMain: Stopping master.services
      2017-04-06 00:32:10,438 INFO co.cask.cdap.data.runtime.main.MasterServiceMain: Saved CDAP shutdown time 1491438730438 at file /var/tmp/cdap/data/cdap_shutdown_time
      2017-04-06 00:32:10,443 ERROR co.cask.cdap.common.runtime.DaemonMain: Exception when shutting down: Not connected to zooKeeper.
      java.lang.IllegalArgumentException: Not connected to zooKeeper.
              at com.google.common.base.Preconditions.checkArgument(Preconditions.java:92) ~[com.google.guava.guava-13.0.1.jar:na]
              at org.apache.twill.internal.zookeeper.DefaultZKClientService.getZooKeeper(DefaultZKClientService.java:344) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.internal.zookeeper.DefaultZKClientService.doCreate(DefaultZKClientService.java:147) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.internal.zookeeper.DefaultZKClientService.create(DefaultZKClientService.java:137) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.zookeeper.ForwardingZKClient.create(ForwardingZKClient.java:61) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.internal.zookeeper.FailureRetryZKClient.create(FailureRetryZKClient.java:66) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.zookeeper.ForwardingZKClient.create(ForwardingZKClient.java:61) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.zookeeper.ZKClientServices$1.create(ZKClientServices.java:84) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.zookeeper.AbstractZKClient.create(AbstractZKClient.java:42) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.internal.zookeeper.ReentrantDistributedLock.acquire(ReentrantDistributedLock.java:233) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.internal.zookeeper.ReentrantDistributedLock.acquire(ReentrantDistributedLock.java:176) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at org.apache.twill.internal.zookeeper.ReentrantDistributedLock.lock(ReentrantDistributedLock.java:81) ~[org.apache.twill.twill-zookeeper-0.10.0.jar:0.10.0]
              at co.cask.cdap.data.runtime.main.MasterServiceMain.stop(MasterServiceMain.java:307) ~[na:na]
              at co.cask.cdap.common.runtime.DaemonMain$1.run(DaemonMain.java:44) ~[na:na]
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ali.anwar Ali Anwar
                Reporter:
                ali.anwar Ali Anwar
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: