Hbase1.1.2的HTablePool已經被棄用，用什麼來代替HTablePool呢？

01-15

需要多線程的網hbase插入數據，網上有很多實用htablepool的例子，但是我的hbase版本是1.1.2，HTablePool已經被棄用，請問有什麼其他類可以用的？

糾正

準確而言不是被棄用是不建議client用都成了developers api

所有client端被棄用的api都有相對應的interface 用那個就好

這樣做的好處是將具體實現隔離留給開發者更大的開發空間以後隨意抽換更好的實現而不影響client端代碼

說白了就是bridge pattern

I don"t know what version are you using.

From HBase 0.98.x you don"t need to pool HTable object anymore.

Just get Table object from Connection object when you need, all the resource caching and management is done internally by Connection class.

Apache HBase ? Reference Guide

For example, in multi-threaded environment the standard method can be described like this:

import org.apache.hadoop.conf.Configuration;

import org.apache.hadoop.hbase.HBaseConfiguration;

import org.apache.hadoop.hbase.client.ConnectionFactory;

import org.apache.hadoop.hbase.client.Connection;

import org.apache.hadoop.hbsae.client.Table;

int main() {

// create a new hbase configuration object. (singleton)

Configuration conf = HBaseConfiguration.create();

// create a new hbase connection object. (singleton)

Connection connection = ConnectionFactory.createConnection(conf);

try {

while (true) {

// create a table instance everytime you need.

// you don"t have to pool this object, because hbase client implementation

// do necessary resource caching control internally.

Table table = connection.getTable(TableName.valueOf("table"));

...

table.close();

}

} finally {

// close the hbase connection on application shutdown.

connection.close();

}

機器配置太低了。

首先hbase內存至少要12G以上開給JVM, CMS回收，解決JVM hang問題。

台數得根據自己的業務需求壓力測試，評估負載。用途不一樣，優化方向不一向

發生的問題第一個是memstore, 第二個是compaction. 第三個JVM hang.

memstore, compaction優化後的參數。當然，環境不一樣，參數不一定適合，以下給個思路。下面是一個高並發，數據本身小的參數優化。

&hbase.regionserver.global.memstore.upperLimit&

&0.2&&

&hbase.regionserver.global.memstore.lowerLimit&

&0.15&&

&hbase.regionserver.optionalcacheflushinterval&

&3600000&&

&hbase.hregion.memstore.mslab.chunksize&

&2097152&

&hbase.hregion.memstore.mslab.max.allocation&

&1024768&

&hbase.hregion.memstore.flush.size&

&134217728&&

&hbase.hregion.majorcompaction&

&86400000&&

&hbase.hstore.compaction.min&

&3&

&hbase.hstore.compaction.max&

&10&&

&hbase.hstore.blockingStoreFiles&

&30&&

&hbase.regionserver.thread.compaction.large&

&2&&

98的 API 上有說明可以直接用

// Create a connection to the cluster. HConnection connection = HConnectionManager.createConnection(Configuration); HTableInterface table = connection.getTable("myTable"); // use table as needed, the table returned is lightweight table.close(); // use the connection for other access to the cluster connection.close();

共享一個 Connection，HTableInterface 實例化是一個非常輕量級的操作，我們現在已經這麼用了，貌似沒什麼問題。

1.1.2中HConnectionManager 也被 Deprecated

/** @deprecated */ @Public @Evolving @Deprecated public class HConnectionManager extends ConnectionFactory { /** @deprecated */ }

直接用org.apache.hadoop.hbase.client.Connection;就可以

可以用hconnection代替，線程安全

有人嗎？