1.连接
HBaseConfiguration conf = HBaseConfiguration.create();
HTable table1 = new HTable(conf, "myTable");
HTable table2 = new HTable(conf, "myTable");
而不是这样的代码:
HBaseConfiguration conf1 = HBaseConfiguration.create();
HTable table1 = new HTable(conf1, "myTable");
HBaseConfiguration conf2 = HBaseConfiguration.create();
HTable table2 = new HTable(conf2, "myTable");
2.连接池
当面对多线程访问需求时,我们可以预先建立HConnection,参见以下代码:
Example 9.1. Pre-Creating a HConnection // Create a connection to the cluster.
HConnection connection = HConnectionManager.createConnection(Configuration);
HTableInterface table = connection.getTable("myTable");
// use table as needed, the table returned is lightweight
table.close();
// use the connection for other access to the cluster
connection.close();
构建HTableInterface实现是非常轻量级的,并且资源是可控的。
HConnection connection = HConnectionManager.createConnection(config);
HTableInterface table = connection.getTable("tablename");
try {
// Use the table as needed, for a single operation and a single thread
} finally {
table.close();
connection.close();
}
package com.bigdata.dbhbase096; import java.io.IOException; import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.client.HConnection;
import org.apache.hadoop.hbase.client.HConnectionManager;
import org.apache.hadoop.hbase.client.HTableInterface;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.client.ResultScanner;
import org.apache.hadoop.hbase.client.Scan;
import org.apache.hadoop.hbase.util.Bytes; public class ConnectionPoolTest {
private static final String QUORUM = "compute1";
private static final String CLIENTPORT = "2181";
private static final String TABLENAME = "minifiletable4";
private static Configuration conf = null;
private static HConnection conn = null; static{
try {
conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.quorum", QUORUM);
conf.set("hbase.zookeeper.property.clientPort", CLIENTPORT);
conn = HConnectionManager.createConnection(conf);
} catch (IOException e) {
e.printStackTrace();
}
} public static void main(String[] args) throws IOException {
HTableInterface htable = ConnectionPoolTest.conn.getTable(TABLENAME);
try {
Scan scan = new Scan();
ResultScanner rs = htable.getScanner(scan);
for (Result r : rs.next(5)) {
for (Cell cell : r.rawCells()) {
System.out.println("Rowkey : " + Bytes.toString(r.getRow())
+ " Familiy:Quilifier : "
+ Bytes.toString(CellUtil.cloneQualifier(cell))
+ " Value : "
+ Bytes.toString(CellUtil.cloneValue(cell))
+ " Time : " + cell.getTimestamp());
}
}
} finally {
htable.close();
} }
}
/**
* Get the connection that goes with the passed <code>conf</code> configuration instance.
* If no current connection exists, method creates a new connection and keys it using
* connection-specific properties from the passed {@link Configuration}; see
* {@link HConnectionKey}.
* @param conf configuration
* @return HConnection object for <code>conf</code>
* @throws ZooKeeperConnectionException
*/
@Deprecated
public static HConnection getConnection(final Configuration conf)
throws IOException {
HConnectionKey connectionKey = new HConnectionKey(conf);
synchronized (CONNECTION_INSTANCES) {
HConnectionImplementation connection = CONNECTION_INSTANCES.get(connectionKey);
if (connection == null) {
connection = (HConnectionImplementation)createConnection(conf, true);
CONNECTION_INSTANCES.put(connectionKey, connection);
} else if (connection.isClosed()) {
HConnectionManager.deleteConnection(connectionKey, true);
connection = (HConnectionImplementation)createConnection(conf, true);
CONNECTION_INSTANCES.put(connectionKey, connection);
}
connection.incCount();
return connection;
}
}
根据传入的conf构建HConnectionKey,然后以HConnectionKey实例为key到连接池Map对象CONNECTION_INSTANCES中去查找connection,如果找到就返回connection,如果找不到就新建,如果找到但已被关闭,就删除再新建。
我们来看HConnectionKey的构造函数:
HConnectionKey(Configuration conf) {
Map<String, String> m = new HashMap<String, String>();
if (conf != null) {
for (String property : CONNECTION_PROPERTIES) {
String value = conf.get(property);
if (value != null) {
m.put(property, value);
}
}
}
this.properties = Collections.unmodifiableMap(m);
try {
UserProvider provider = UserProvider.instantiate(conf);
User currentUser = provider.getCurrent();
if (currentUser != null) {
username = currentUser.getName();
}
} catch (IOException ioe) {
HConnectionManager.LOG.warn("Error obtaining current user, skipping username in HConnectionKey", ioe);
}
}
由以上源码可知,接收conf构造HConnectionKey实例时,其实是将conf配置文件中的属性赋值给HConnectionKey自身的属性,换句话说,不管你new几次,只要conf的属性相同,new出来的HConnectionKey实例的属性都相同。
结论一:conf的属性 --》 HConnectionKey实例的属性
接下来,回到getConnection源码中看到这样一句话:
HConnectionImplementation connection = CONNECTION_INSTANCES.get(connectionKey);
该代码是以HConnectionKey实例为key来查找CONNECTION_INSTANCES这个LinkedHashMap中是否已经包含了HConnectionKey实例为key的键值对,这里要注意的是,map的get方法,其实获取的是key的hashcode,这个自己读JDK源码就能看到。
而HConnectionKey已经重载了hashcode方法:
@Override
public int hashCode() {
final int prime = 31;
int result = 1;
if (username != null) {
result = username.hashCode();
}
for (String property : CONNECTION_PROPERTIES) {
String value = properties.get(property);
if (value != null) {
result = prime * result + value.hashCode();
}
}
return result;
}
在该代码中,最终返回的hashcode取决于当前用户名及当前conf配置文件的属性。所以,只要conf配置文件的属性和用户相同,HConnectionKey实例的hashcode就相同!
结论二:conf的属性 --》HConnectionKey实例的hashcode
Example 9.1. Pre-Creating a HConnection
// Create a connection to the cluster.
HConnection connection = HConnectionManager.createConnection(Configuration);
HTableInterface table = connection.getTable("myTable");
// use table as needed, the table returned is lightweight
table.close();
// use the connection for other access to the cluster
connection.close();
/**
* Get the connection that goes with the passed <code>conf</code> configuration instance.
* If no current connection exists, method creates a new connection and keys it using
* connection-specific properties from the passed {@link Configuration}; see
* {@link HConnectionKey}.
* @param conf configuration
* @return HConnection object for <code>conf</code>
* @throws ZooKeeperConnectionException
*/
public static HConnection getConnection(final Configuration conf) throws IOException {
return ConnectionManager.getConnectionInternal(conf);
}
这个不是重点,重点是最新版本代码的pom:
<groupId>org.apache.hbase</groupId> 40 <artifactId>hbase</artifactId> <packaging>pom</packaging> <version>2.0.0-SNAPSHOT</version> <name>HBase</name> <description> Apache HBase\99 is the Hadoop database. Use it when you need random, realtime read/write access to your Big Data. This project's goal is the hosting of very large tables -- billions of rows X millions of columns -- atop clusters of commodity hardware. </description>