630305 - Change hbase.regionserver.maxlogs to 16 for Socorro hbase

Status: RESOLVED FIXED

(Mozilla Metrics :: Hadoop/HBase Operations, defect)

Description

[Daniel Einspanjer [:dre] [:deinspanjer]]

Currently defaults to 128.  This is per server which means that we have a very long startup time for the master to churn through 128*30 hlog files.

We need to test this setting with our secondary cluster to be sure it works properly, then we should deploy to production as soon as reasonable to cut down on downtimes for restart.

Comment 1

[Xavier Stevens [:xstevens]]

I did a shutdown and startup with this setting on secondary cluster twice, because I wasn't sure how the setting would take.

First startup was 12 minutes and the second was 10 minutes to load all of the regions.

Comment 2

[Xavier Stevens [:xstevens]]

Also investigate modifying:

<property>
    <name>zookeeper.session.timeout</name>
    <value>60000</value>
    <description>ZooKeeper session timeout.
      HBase passes this to the zk quorum as suggested maximum time for a
      session.  See http://hadoop.apache.org/zookeeper/docs/current/zookeeperProgrammers.html#ch_zkSessions
      "The client sends a requested timeout, the server responds with the
      timeout that it can give the client. "
      In milliseconds.
    </description>
  </property>

Comment 3

[Daniel Einspanjer [:dre] [:deinspanjer]]

What is the status of this one?

Comment 4

[T [:tmary] Meyarivan]

Changes merged with current config for both pri-phx1 and sec-phx1 HBase clusters.

--