问题：初始化hive元数据报错
[atguigu@hadoop102 software]$ schematool -initSchema -dbType mysql -verbose
Error: Table 'CTLGS' already exists (state=42S01,code=1050)
Closing: 0: jdbc:mysql://hadoop102:3306/metastore?useSSL=false
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:594)
at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: java.io.IOException: Schema script failed, errorcode 2
at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1226)
at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:1204)
at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:590)
... 8 more
*** schemaTool failed ***
- 首先想到的是删除hive在MySQL中创建的元数据库Metastore
- 和HDFS中创建的目录
- 结果还是报错
- 仔细一看是hive.metastore连接问题
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:94)
at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:169)
at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:475)
at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:581)
at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:567)
at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1517)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown database 'metastore'
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
at com.mysql.jdbc.Util.getInstance(Util.java:387)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:941)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3870)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3806)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:871)
at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1686)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1207)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2254)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2285)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2084)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:795)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:400)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:327)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.apache.hadoop.hive.metastore.tools.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88)
查看MySQL中的用户权限
- 参考：https://blog.csdn.net/peterchan88/article/details/78341852
1.首先登陆用root用户登录mysql
mysql -u root -p
2.查看root下
mysql> select user,host from mysql.user where user='root';
grant all on *.* to 'root'@'hadoop102' identified by 'hadoop';
4.测试是否能登录
mysql -u root -h 'hadoop102' -phadoop
- 使用初始化命令还是报错，继续往下查
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: java.sql.SQLException : Access denied for user 'root'@'hadoop102' (using password: YES)
SQL Error code: 1045
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
检查 MySQL 服务的状态
显示 MySQL 服务的状态以及相关的日志信息，包括任何错误消息或警告
sudo systemctl status mysqld.service
start 命令启动
sudo systemctl start mysqld.service
- 之前由于内存原因，删掉了MySQL解压后的相关文件，于是开始重新走一遍安装步骤
问题：登入MySQL报错
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
- 解决：使用“ln -s /storage/db/mysql/mysql.sock /var/lib/mysql/mysql.sock”命令，将正确的socket文件位置，软链接到提示错误的socket文件路径位置
- 试了还是不行，在此基础上重启mysql就好了，systemctl restart mysqld
- 参考文章：https://blog.csdn.net/Aria_Miazzy/article/details/92803246
- 新建 Hive 元数据库
初始化 Hive 元数据库，成功
schematool -initSchema -dbType mysql -verbose
hiveserver2 启动失败
metastore未正常启动
- [atguigu@hadoop202 hive]$ nohup hive --service hiveserver2 2>&1 &
java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
...
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thri
ft.transport.TTransportException: java.net.ConnectException: 拒绝连接 (Connection refused)
...
Caused by: java.net.ConnectException: 拒绝连接 (Connection refused)
...
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
- 查看元数据库是否启动
ps aux | grep metastore
- 一开始查到的
- 启动 metastore
[atguigu@hadoop202 hive]$ hive --service metastore
HiveServer2 未正常启动
当时的原因是引擎问题，安装配置了tez引擎
<property>
<name>hive.execution.engine</name>
<value>tez</value>
</property>
bin/beeline连接不了
[atguigu@hadoop102 hive]$ bin/beeline -u jdbc:hive2://hadoop102:10000 -n 用户名 -p 密码
Error: Could not open client transport with JDBC Uri: jdbc:hive2://hadoop102:10000: Failed to open new session: java.lang.RuntimeExce
ption: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: link999 is not allowed to impersonate anonymous (state=08S01,code=0)Beeline version 3.1.2 by Apache Hive
- 在hadoop的配置文件core-site.xml增加如下配置，重启hdfs，其中“xxx”是连接beeline的用户，将“xxx”替换成自己的用户名即可。最关键的是一定要重启hadoop
<property>
<name>hadoop.proxyuser.xxx.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.xxx.groups</name>
<value>*</value>
</property>
-
“*”表示可通过超级代理“xxx”操作hadoop的用户、用户组和主机
-
主要原因是hadoop引入了一个安全伪装机制，使得hadoop 不允许上层系统直接将实际用户传递到hadoop层，而是将实际用户传递给一个超级代理，由此代理在hadoop上执行操作，避免任意客户端随意操作hadoop
-
参考文章https://blog.csdn.net/qq_35746739/article/details/119032729
-
beeline 的常用参数
Option Description
--autoCommit=[true/false] ---进入一个自动提交模式：beeline --autoCommit=true
--autosave=[true/false] ---进入一个自动保存模式：beeline --autosave=true
--color=[true/false] ---显示用到的颜色：beeline --color=true
--delimiterForDSV= DELIMITER ---分隔值输出格式的分隔符。默认是“|”字符。
--fastConnect=[true/false] ---在连接时，跳过组建表等对象：beeline --fastConnect=false
--force=[true/false] ---是否强制运行脚本：beeline--force=true
--headerInterval=ROWS ---输出的表间隔格式，默认是100: beeline --headerInterval=50
--help ---帮助 beeline --help
--hiveconf property=value ---设置属性值，以防被hive.conf.restricted.list重置：beeline --hiveconf prop1=value1
--hivevar name=value ---设置变量名：beeline --hivevar var1=value1
--incremental=[true/false] ---输出增量
--isolation=LEVEL ---设置事务隔离级别：beeline --isolation=TRANSACTION_SERIALIZABLE
--maxColumnWidth=MAXCOLWIDTH ---设置字符串列的最大宽度：beeline --maxColumnWidth=25
--maxWidth=MAXWIDTH ---设置截断数据的最大宽度：beeline --maxWidth=150
--nullemptystring=[true/false] ---打印空字符串：beeline --nullemptystring=false
--numberFormat=[pattern] ---数字使用DecimalFormat：beeline --numberFormat="#,###,##0.00"
--outputformat=[table/vertical/csv/tsv/dsv/csv2/tsv2] ---输出格式：beeline --outputformat=tsv
--showHeader=[true/false] ---显示查询结果的列名：beeline --showHeader=false
--showNestedErrs=[true/false] ---显示嵌套错误：beeline --showNestedErrs=true
--showWarnings=[true/false] ---显示警告：beeline --showWarnings=true
--silent=[true/false] ---静默方式执行，不显示执行过程信息：beeline --silent=true
--truncateTable=[true/false] ---是否在客户端截断表的列
--verbose=[true/false] ---显示详细错误信息和调试信息：beeline --verbose=true
-d <driver class> ---使用一个驱动类：beeline -d driver_class
-e <query> ---使用一个查询语句：beeline -e "query_string"
-f <file> ---加载一个文件：beeline -f filepath 多个文件用-e file1 -e file2
-n <username> ---加载一个用户名：beeline -n valid_user
-p <password> ---加载一个密码：beeline -p valid_password
-u <database URL> ---加载一个JDBC连接字符串：beeline -u db_URL
![AI论文速读 | 2024[WWW]不只是路线：联合 GPS 和路线建模的轨迹表示学习](https://img-blog.csdnimg.cn/direct/298821f3e9e84d95bc42cb4a680b52e7.jpeg#pic_center)
