搭建 Hbase 集群
- 一、准备工作
- 二、三台服务器之间的 SSH 免密登录
- 1.修改hosts文件添加DNS映射
- 2.在每台服务器上生成 SSH 密钥对
- 3.将公共密钥（通常为 ~/.ssh/id_rsa.pub）复制到目标服务器上
- 4.从本地使用 SSH 命令无需密码连接到目标服务器
- 二、安装JDK
- 1.执行解压更名
- 2.配置环境 source生效
- 3.分发环境配置
- 三、安装ZooKeeper 来协调和管理集群中的各个节点
- 1.解压更名
- 2.修改配置文件
- 2-1.配置文件更名修改配置
- 2-2.根据配置创建数据所需文件夹
- 3.配置环境
- 4.分发环境
- 5.分发环境
- 5-1.文件远程拷贝
- 5-2. lvresun-server2、lvrensun-server3 source生效环境变量
- 6. 每台zookeeper服务分配myid
- 7.启动验证
- 8.验证宕机选举是否正常
在内容开始之前需要了解一下几个方面的知识会更好的帮助学习和搭建Hbase集群
Hadoop：Hbase 是建立在 Hadoop基础之上的分布式数据库。了解 Hadoop 的基本概念和架构将有助于您理解 Hbase 的工作原理。
分布式系统：Hbase是一个分布式数据库，它将数据存储在多台服务器上。了解分布式系统的概念、优点、挑战和常见的解决方案，将帮助您更好地理解和管理
Hbase集群。ZooKeeper：Hbase 使用 ZooKeeper 来协调和管理集群中的各个节点。了解 ZooKeeper的基本概念、角色和功能，以及如何安装、配置和运行 ZooKeeper，将有助于您正确地配置和管理 Hbase 的
ZooKeeper。网络和主机配置：在搭建 Hbase集群之前，需要确保服务器之间能够相互通信，并具有足够的网络带宽和稳定性。此外，还需要了解如何设置主机名解析和网络配置，以便在集群中使用主机名进行通信。
数据模型和表设计：Hbase 使用键值对存储和访问数据，了解如何设计良好的数据模型和表结构将有助于提高性能和查询效率。
安全性配置：在生产环境中搭建 Hbase集群时，安全性是一个重要的考虑因素。了解如何配置安全认证和访问控制，以保护您的数据和集群是很重要的。
一、准备工作
- Hadoop 官方网站 https://archive.apache.org/dist/hadoop/common/
- Hbase：官方网站 https://archive.apache.org/dist/hbase/
- ZooKeeper：Hbase 使用 ZooKeeper进行协调和管理。官方网站:https://archive.apache.org/dist/zookeeper/
- HDFS：Hbase 使用 Hadoop HDFS存储其数据。因此，您需要安装和配置 Hadoop HDFS。一般情况下，Hadoop 的安装包中已经包含了 HDFS。
- JDK：确保服务器上安装了 Java Development Kit（JDK）。Hadoop 和 Hbase 都需要运行在 Java环境中，因此需要安装并正确配置 JDK。
我们在这里使用的是一下版本：
lvrensun-server1
[root@lvrensun-server1 soft]# pwd
/root/soft
[root@lvrensun-server1 soft]# ll -h
总用量 700M
-rw-r--r--. 1 root root 396M 2月 2 23:09 hadoop-2.10.2.tar.gz
-rw-r--r--. 1 root root 79M 1月 30 01:50 hbase-0.98.6-hadoop2-bin.tar.gz
-rw-r--r--. 1 root root 177M 3月 16 2017 jdk-8u131-linux-x64.tar.gz
-rw-r--r--. 1 root root 16M 1月 30 01:50 zookeeper-3.4.5.tar.gz
[root@lvrensun-server1 soft]#
lvrensun-server2
[root@lvrensun-server2 soft]# pwd
/root/soft
[root@lvrensun-server2 soft]# ll -h
总用量 0
[root@lvrensun-server2 soft]#
lvrensun-server3
[root@lvrensun-server3 soft]# pwd
/root/soft
[root@lvrensun-server3 soft]# ll -h
总用量 0
[root@lvrensun-server3 soft]#
可以看到上方显示有三台服务器分别是：
| hostname | IP | 资源路径 | 是否有资源包 |
|---|---|---|---|
| lvrensun-server1 | 192.168.0.11 | /root/soft | 是 |
| lvrensun-server2 | 192.168.0.22 | /root/soft | 否 |
| lvrensun-server3 | 192.168.0.33 | /root/soft | 否 |
二、三台服务器之间的 SSH 免密登录
1.修改hosts文件添加DNS映射
vim /etc/hosts
如下图所示
2.在每台服务器上生成 SSH 密钥对
[root@lvrensun-server1 soft]# ssh-keygen -t rsa
[root@lvrensun-server2 soft]# ssh-keygen -t rsa
[root@lvrensun-server3 soft]# ssh-keygen -t rsa
在生成密钥对时，您可以选择存储位置和设置密钥的密码。如果留空密码，则可以实现完全无密码的免密登录。
3.将公共密钥（通常为 ~/.ssh/id_rsa.pub）复制到目标服务器上
[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.11
[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.22
[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.33
[root@lvrensun-server2 ~]# ssh-copy-id root@192.168.0.11
[root@lvrensun-server2 ~]# ssh-copy-id root@192.168.0.22
[root@lvrensun-server2 ~]# ssh-copy-id root@192.168.0.33
[root@lvrensun-server3 ~]# ssh-copy-id root@192.168.0.11
[root@lvrensun-server3 ~]# ssh-copy-id root@192.168.0.22
[root@lvrensun-server3 ~]# ssh-copy-id root@192.168.0.33
前两步其中一台服务器详细执行步骤如下：
[root@lvrensun-server1 soft]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:QeDtQ6BMG5mIM2rk2DjM87H5Qe9uCv2i16bF55CI3fI root@lvrensun-server1
The key's randomart image is:
+---[RSA 2048]----+
| . .ooo.. |
|+..oo= + |
|B= + . + |
|=*.. . o . |
|..o = . S |
| ++.+... |
| o.=+* . |
| .+==+ |
| .ooBE . |
+----[SHA256]-----+
[root@lvrensun-server1 ~]# cat ~/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNW0i7f2CAa/rq55WqUtJBc8igbOSTrsJGlFjX2s1XmONL1nPHk3eKEy1HrqC5kpEwgjYcIawtIq/apERkjqeGNRbjV9/Sq7zUvwiONA1+WJTwI80RyH84ZhV6aHuXTrly+9cx8UF1TbVRh0WgWZcjzFZdzDVIcB18zsBmWXZsGZxxVYD4fjkfXjMKJwOcAGXlUWb3J7O30uB9GcJhasTN98QcFmVp+cGldNJbTlXgUSUffD0HTOAPhlWDgWI3dFq+jCnCI4XzugyF0vU4dv81FiBTgcTxJ25tJvnw2PwtxhSEpklfK9UuYLBrOlDMsDBlkKBITTddJkLOvCsLr1Sr root@lvrensun-server1
[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.11
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.0.11 (192.168.0.11)' can't be established.
ECDSA key fingerprint is SHA256:SPQuwBgvdNmYL9PVBgcGTkuynS9iqvn3pBvXv6ZcePw.
ECDSA key fingerprint is MD5:db:a1:53:83:6c:52:e5:6c:d8:75:6a:05:47:b7:26:5b.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.0.11's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@192.168.0.11'"
and check to make sure that only the key(s) you wanted were added.
[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.22
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.0.22 (192.168.0.22)' can't be established.
ECDSA key fingerprint is SHA256:SPQuwBgvdNmYL9PVBgcGTkuynS9iqvn3pBvXv6ZcePw.
ECDSA key fingerprint is MD5:db:a1:53:83:6c:52:e5:6c:d8:75:6a:05:47:b7:26:5b.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.0.22's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@192.168.0.22'"
and check to make sure that only the key(s) you wanted were added.
[root@lvrensun-server1 ~]# ssh-copy-id root@192.168.0.33
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host '192.168.0.33 (192.168.0.33)' can't be established.
ECDSA key fingerprint is SHA256:SPQuwBgvdNmYL9PVBgcGTkuynS9iqvn3pBvXv6ZcePw.
ECDSA key fingerprint is MD5:db:a1:53:83:6c:52:e5:6c:d8:75:6a:05:47:b7:26:5b.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.0.33's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh 'root@192.168.0.33'"
and check to make sure that only the key(s) you wanted were added.
[root@lvrensun-server1 ~]#
4.从本地使用 SSH 命令无需密码连接到目标服务器
lvrensun-server1
[root@lvrensun-server1 ~]# ssh lvrensun-server1
Last login: Fri Feb 2 12:54:38 2024 from lvrensun-server2
[root@lvrensun-server1 ~]# exit
登出
Connection to lvrensun-server1 closed.
[root@lvrensun-server1 ~]# ssh lvrensun-server2
Last login: Fri Feb 2 12:54:51 2024 from lvrensun-server2
[root@lvrensun-server2 ~]# exit
登出
Connection to lvrensun-server2 closed.
[root@lvrensun-server1 ~]# ssh lvrensun-server3
Last login: Fri Feb 2 12:55:14 2024 from lvrensun-server2
[root@lvrensun-server3 ~]#
lvrensun-server2
[root@lvrensun-server2 ~]# ssh lvrensun-server1
Last login: Fri Feb 2 12:53:50 2024 from lvrensun-server2
[root@lvrensun-server1 ~]# exit
登出
Connection to lvrensun-server1 closed.
[root@lvrensun-server2 ~]# ssh lvrensun-server2
Last login: Fri Feb 2 12:54:09 2024 from lvrensun-server2
[root@lvrensun-server2 ~]# exit
登出
Connection to lvrensun-server2 closed.
[root@lvrensun-server2 ~]# ssh lvrensun-server3
Last login: Fri Feb 2 12:54:21 2024 from lvrensun-server2
[root@lvrensun-server3 ~]#
lvrensun-server3
[root@lvrensun-server3 ~]# ssh lvrensun-server1
Last login: Fri Feb 2 12:56:09 2024 from lvrensun-server1
[root@lvrensun-server1 ~]# exit
登出
Connection to lvrensun-server1 closed.
[root@lvrensun-server3 ~]# ssh lvrensun-server2
Last login: Fri Feb 2 12:56:19 2024 from lvrensun-server1
[root@lvrensun-server2 ~]# exit
登出
Connection to lvrensun-server2 closed.
[root@lvrensun-server3 ~]# ssh lvrensun-server3
Last login: Fri Feb 2 12:56:27 2024 from lvrensun-server1
[root@lvrensun-server3 ~]#
二、安装JDK
Hadoop 和 Hbase 都需要运行在 Java环境中因此JDK属于基础环境
1.执行解压更名
tar -xzvf jdk-8u131-linux-x64.tar.gz
mv jdk-1.8.0-u131/ jdk1.8
如下：
-rw-r--r--. 1 root root 34687765 1月 30 01:50 hadoop-2.10.2-site.tar.gz
-rw-r--r--. 1 root root 82347017 1月 30 01:50 hbase-0.98.6-hadoop2-bin.tar.gz
drwxr-xr-x. 8 10 143 255 3月 15 2017 jdk1.8
-rw-r--r--. 1 root root 185540433 3月 16 2017 jdk-8u131-linux-x64.tar.gz
-rw-r--r--. 1 root root 16402010 1月 30 01:50 zookeeper-3.4.5.tar.gz
2.配置环境 source生效
vim /etc/profile
[root@lvrensun-server1 soft]# vim /etc/profile
添加配置
export JAVA_HOME=/root/soft/jdk1.8
export PATH=$PATH:$JAVA_HOME/bin
source生效
[root@lvrensun-server1 soft]# source /etc/profile
[root@lvrensun-server1 soft]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
[root@lvrensun-server1 soft]#
3.分发环境配置
分发JDK
[root@lvrensun-server1 soft]# scp -r jdk1.8/ lvrensun-server2:/root/soft/
[root@lvrensun-server1 soft]# scp -r jdk1.8/ lvrensun-server3:/root/soft/
分发环境变量
[root@lvrensun-server1 soft]# scp /etc/profile lvrensun-server2:/etc/
[root@lvrensun-server1 soft]# scp /etc/profile lvrensun-server3:/etc/
再到 lvresun-server2、lvrensun-server3 source生效环境变量
[root@lvrensun-server2 ~]# source /etc/profile
[root@lvrensun-server2 ~]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
[root@lvrensun-server2 ~]#
[root@lvrensun-server3 soft]# source /etc/profile
[root@lvrensun-server3 soft]# java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)
[root@lvrensun-server3 soft]#
三、安装ZooKeeper 来协调和管理集群中的各个节点
1.解压更名
[root@lvrensun-server1 soft]# tar -xzvf zookeeper-3.4.5.tar.gz
.......
-rw-r--r--. 1 root root 34687765 1月 30 01:50 hadoop-2.10.2-site.tar.gz
-rw-r--r--. 1 root root 82347017 1月 30 01:50 hbase-0.98.6-hadoop2-bin.tar.gz
drwxr-xr-x. 8 10 143 255 3月 15 2017 jdk1.8
-rw-r--r--. 1 root root 185540433 3月 16 2017 jdk-8u131-linux-x64.tar.gz
drwxr-xr-x. 10 501 games 4096 11月 5 2012 zookeeper-3.4.5
-rw-r--r--. 1 root root 16402010 1月 30 01:50 zookeeper-3.4.5.tar.gz
[root@lvrensun-server1 soft]# mv zookeeper-3.4.5 zookeeper
[root@lvrensun-server1 soft]# ll
总用量 311512
-rw-r--r--. 1 root root 34687765 1月 30 01:50 hadoop-2.10.2-site.tar.gz
-rw-r--r--. 1 root root 82347017 1月 30 01:50 hbase-0.98.6-hadoop2-bin.tar.gz
drwxr-xr-x. 8 10 143 255 3月 15 2017 jdk1.8
-rw-r--r--. 1 root root 185540433 3月 16 2017 jdk-8u131-linux-x64.tar.gz
drwxr-xr-x. 10 501 games 4096 11月 5 2012 zookeeper
-rw-r--r--. 1 root root 16402010 1月 30 01:50 zookeeper-3.4.5.tar.gz
[root@lvrensun-server1 soft]#
2.修改配置文件
2-1.配置文件更名修改配置
[root@lvrensun-server1 conf]# pwd
/root/soft/zookeeper/conf
[root@lvrensun-server1 conf]# mv zoo_sample.cfg zoo.cfg
[root@lvrensun-server1 conf]# vim zoo.cfg
zoo.cfg 修改内容如下：
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/root/soft/zookeeper/data
dataLogDir=/root/soft/zookeeper/log
# the port at which the clients will connect
clientPort=2181
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=lvrensun-server1:2888:2299
server.2=lvrensun-server2:2888:2299
server.3=lvrensun-server3:2888:2299
2-2.根据配置创建数据所需文件夹
[root@lvrensun-server1 conf]# mkdir -p /root/soft/zookeeper/data
[root@lvrensun-server1 conf]# mkdir -p /root/soft/zookeeper/log
[root@lvrensun-server1 conf]# ll ../
总用量 1488
drwxr-xr-x. 2 501 games 149 2月 2 13:18 bin
-rw-r--r--. 1 501 games 75988 10月 1 2012 build.xml
-rw-r--r--. 1 501 games 70223 10月 1 2012 CHANGES.txt
drwxr-xr-x. 2 501 games 70 2月 2 13:32 conf
drwxr-xr-x. 10 501 games 130 2月 2 13:18 contrib
drwxr-xr-x. 2 root root 6 2月 2 13:34 data
drwxr-xr-x. 2 501 games 4096 2月 2 13:18 dist-maven
drwxr-xr-x. 6 501 games 4096 2月 2 13:18 docs
-rw-r--r--. 1 501 games 1953 10月 1 2012 ivysettings.xml
-rw-r--r--. 1 501 games 3120 10月 1 2012 ivy.xml
drwxr-xr-x. 4 501 games 235 2月 2 13:18 lib
-rw-r--r--. 1 501 games 11358 10月 1 2012 LICENSE.txt
drwxr-xr-x. 2 root root 6 2月 2 13:34 log
-rw-r--r--. 1 501 games 170 10月 1 2012 NOTICE.txt
-rw-r--r--. 1 501 games 1770 10月 1 2012 README_packaging.txt
-rw-r--r--. 1 501 games 1585 10月 1 2012 README.txt
drwxr-xr-x. 5 501 games 47 2月 2 13:18 recipes
drwxr-xr-x. 8 501 games 154 2月 2 13:18 src
-rw-r--r--. 1 501 games 1315806 11月 5 2012 zookeeper-3.4.5.jar
-rw-r--r--. 1 501 games 833 11月 5 2012 zookeeper-3.4.5.jar.asc
-rw-r--r--. 1 501 games 33 11月 5 2012 zookeeper-3.4.5.jar.md5
-rw-r--r--. 1 501 games 41 11月 5 2012 zookeeper-3.4.5.jar.sha1
[root@lvrensun-server1 conf]#
3.配置环境
[root@lvrensun-server1 zookeeper]# vim /etc/profile
添加内容：
export ZOOKEEPER_HOME=/root/soft/zookeeper
export PATH=$PATH:$ZOOKEEPER_HOME/bin
生效
[root@lvrensun-server1 zookeeper]# source /etc/profile
4.分发环境
分发包：
[root@lvrensun-server1 soft]# pwd
/root/soft
[root@lvrensun-server1 soft]# scp -r zookeeper lvrensun-server2:/root/soft/
......
[root@lvrensun-server1 soft]# scp -r zookeeper lvrensun-server3:/root/soft/
......
[root@lvrensun-server1 soft]#
5.分发环境
5-1.文件远程拷贝
[root@lvrensun-server1 soft]# scp /etc/profile lvrensun-server2:/etc/
[root@lvrensun-server1 soft]# scp /etc/profile lvrensun-server3:/etc/
5-2. lvresun-server2、lvrensun-server3 source生效环境变量
[root@lvrensun-server2 soft]# source /etc/profile
[root@lvrensun-server3 soft]# source /etc/profile
6. 每台zookeeper服务分配myid
[root@lvrensun-server1 bin]# echo "1" > /root/soft/zookeeper/data/myid
[root@lvrensun-server2 bin]# echo "2" > /root/soft/zookeeper/data/myid
[root@lvrensun-server3 bin]# echo "3" > /root/soft/zookeeper/data/myid
其中 myid中的 1、2、3 分别对应的是 zoo.cfg 文件中配置项server.1、server.2、server.3中一一对应的数字，知道就行了，对应即可
7.启动验证
从
[root@lvrensun-server1 bin]# cd /root/soft/zookeeper/bin/
[root@lvrensun-server1 bin]# zkServer.sh start
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@lvrensun-server1 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@lvrensun-server1 bin]#
从
[root@lvrensun-server2 bin]# cd /root/soft/zookeeper/bin/
[root@lvrensun-server2 bin]# zkServer.sh start
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@lvrensun-server2 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@lvrensun-server2 bin]#
主
[root@lvrensun-server3 bin]# cd /root/soft/zookeeper//bin/
[root@lvrensun-server3 bin]# zkServer.sh start
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[root@lvrensun-server3 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: leader
启动一切正产 lvrensun-server3 为主节点
8.验证宕机选举是否正常
把主服务器停掉
[root@lvrensun-server3 bin]# ./zkServer.sh stop
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
[root@lvrensun-server3 bin]#
查看其他两台状态
从
[root@lvrensun-server1 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: follower
[root@lvrensun-server1 bin]#
主
[root@lvrensun-server2 bin]# zkServer.sh status
JMX enabled by default
Using config: /root/soft/zookeeper/bin/../conf/zoo.cfg
Mode: leader
[root@lvrensun-server2 bin]#
可以看到 lvrensun-server2 服务器中zookeeper节点已经被选为主服务节点
四、安装 Hadoop Hbase 是建立在 Hadoop基础之上的分布式数据库
