I'm running through the Cloudera Manager (free edition) and I reached the point where the wizard is creating the Hive Metastore Database. This error is shown and halts the configuration process.
我正在运行Cloudera Manager（免费版），我到达了向导创建Hive Metastore数据库的程度。显示此错误并暂停配置过程。
using /var/run/cloudera-scm-agent/process/40-hive-metastore-create-tables/hadoop-conf as HADOOP_CONF_DIR
使用/ var / run / cloudera-scm-agent / process / 40-hive-metastore-create-tables / hadoop-conf作为HADOOP_CONF_DIR
I cant seem to find any information that might cause this?
我似乎无法找到任何可能导致这种情况的信息？
Every thing has been configured correctly up to this point, everything installed and user names and passwords are correct.
到目前为止，每件事情都已正确配置，所有安装的内容以及用户名和密码都是正确的。
Has anybody seen this error before? Thoughts?
以前有人见过这个错误吗？思考？
Error Log:
错误日志：
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:688)
at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1094)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2337)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2370)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2154)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:49)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at com.cloudera.enterprise.dbutil.SqlRunner.open(SqlRunner.java:109)
at com.cloudera.enterprise.dbutil.SqlRunner.runSingleQuery(SqlRunner.java:80)
at com.cloudera.cmf.service.hive.HiveMetastoreDbUtil.countTables(HiveMetastoreDbUtil.java:191)
... 2 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2540)
at com.mysql.jdbc.MysqlIO.readPacket(MysqlIO.java:612)
... 20 more
2 个解决方案
#1
2
ok cloudera is using version HIVE 0.10, that doesn't support remote login
ok cloudera使用版本HIVE 0.10，不支持远程登录
but you need to go around that bug, logging to the server that is getting the error the cloudera manager will tell you the ip
但你需要绕过那个bug，登录到服务器，这个错误会让cloudera经理告诉你ip
1) create login in to the Server that fails to install HIVE 2) Create a $HADDOP_HOME
1）创建登录到无法安装HIVE的服务器2）创建$ HADDOP_HOME
$HADOOP_HOME="/usr/lib/hadoop/"
3) INSTALL postgres in the server that fails
3）在失败的服务器中安装postgres
`$ sudo apt-get install postgresql`
$ cat /etc/postgresql/9.1/main/postgresql.conf | grep -e listen -e standard_conforming_strings
modify this to lines in the file
将其修改为文件中的行
listen_addresses = '*'
standard_conforming_strings = off
You also need to configure authentication for your network in pg_hba.conf. You need to make sure that the PostgreSQL user that you will create in the next step will have access to the server from a remote host. To do this, add a new line into pg_hba.con that has the following information:
您还需要在pg_hba.conf中为网络配置身份验证。您需要确保将在下一步中创建的PostgreSQL用户可以从远程主机访问服务器。为此，请在pg_hba.con中添加一个包含以下信息的新行：
host <database> <user> <network address> <mask> password
Start PostgreSQL Server
启动PostgreSQL服务器
$ sudo service postgresql start
Use chkconfig utility to ensure that your PostgreSQL server will start at a boot time:
使用chkconfig实用程序确保PostgreSQL服务器在启动时启动：
chkconfig postgresql on
You can use the chkconfig utility to verify that PostgreSQL server will be started at boot time, for example:
您可以使用chkconfig实用程序验证PostgreSQL服务器是否将在引导时启动，例如：
chkconfig --list postgresql
Step 2: Install the Postgres JDBC Driver
第2步：安装Postgres JDBC驱动程序
Before you can run the Hive metastore with a remote PostgreSQL database, you must configure a JDBC driver to the remote PostgreSQL database, set up the initial database schema, and configure the PostgreSQL user account for the Hive user.
在使用远程PostgreSQL数据库运行Hive Metastore之前，必须为远程PostgreSQL数据库配置JDBC驱动程序，设置初始数据库模式，并为Hive用户配置PostgreSQL用户帐户。
To install the PostgreSQL JDBC Driver on a Debian/Ubuntu system:
要在Debian / Ubuntu系统上安装PostgreSQL JDBC驱动程序：
Install libpostgresql-jdbc-java and symbolically link the file into the /usr/lib/hive/lib/ directory.
安装libpostgresql-jdbc-java并将该文件符号链接到/ usr / lib / hive / lib /目录。
$ sudo apt-get install libpostgresql-jdbc-java
$ ln -s /usr/share/java/postgresql-jdbc4.jar /usr/lib/hive/lib/postgresql-jdbc4.jar
Step 3: Create the metastore database and user account
第3步：创建Metastore数据库和用户帐户
bash# sudo –u postgres psql
bash$ psql
postgres=# CREATE USER hiveuser WITH PASSWORD 'mypassword';
postgres=# CREATE DATABASE metastore;
postgres=# \c metastore;
You are now connected to database 'metastore'.
postgres=# \i /usr/lib/hive/scripts/metastore/upgrade/postgres/hive-schema-0.10.0.postgres.sql
SET
SET
...
Now you need to grant permission for all metastore tables to user hiveuser. PostgreSQL does not have statements to grant the permissions for all tables at once; you'll need to grant the permissions one table at a time. You could automate the task with the following SQL script:
现在，您需要将所有Metastore表的权限授予用户hiveuser。 PostgreSQL没有语句一次授予所有表的权限;您需要一次授予一个表的权限。您可以使用以下SQL脚本自动执行该任务：
bash# sudo –u postgres psql
metastore=# \o /tmp/grant-privs
metastore=# SELECT 'GRANT SELECT,INSERT,UPDATE,DELETE ON "' || schemaname || '"."' || tablename || '" TO hiveuser ;'
metastore-# FROM pg_tables
metastore-# WHERE tableowner = CURRENT_USER and schemaname = 'public';
metastore=# \o
metastore=# \i /tmp/grant-privs
You can verify the connection from the machine where you'll be running the metastore service as follows:
您可以从将运行Metastore服务的计算机验证连接，如下所示：
psql –h myhost –U hiveuser –d metastore
metastore=#
Step 4: Configure the Metastore Service to Communicate with the PostgreSQL Database
步骤4：配置Metastore服务以与PostgreSQL数据库通信
change the IP of the AWS amazon master Server, or your master server, don't use DNS name
更改AWS amazon主服务器或主服务器的IP，请勿使用DNS名称
$find / -name hive-site.xml
$nano /run/cloudera-scm-agent/process/27-hive-metastore-create-tables/hive-site.xml
in the File search for:
在文件中搜索：
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:postgresql://myhost/metastore</value>
</property>
and change to the correct IP that is your Master Hadoop Server where u are running Cloudera Manager also every link in that file that is not correctly write to the hadoop master Cloudera manager connector , you will have to change to the correct IP
并更改为正确的IP是你师父的Hadoop服务器，其中u正在运行的Cloudera Manager还该文件中的每一个环节没有被正确地写入Hadoop的主Cloudera的经理连接器，你将不得不更改为正确的IP
after all this just get back to the autoinstall of cloudera manager and run again and it will be all good :)
毕竟这只是回到cloudera经理的自动安装并再次运行，这将是很好的:)
that it all the installation that you have to work around our contract cloudera support (that's their business) :)
你必须解决我们的合同cloudera支持（这是他们的业务）所有的安装:)
all this it works fine for me when i have this problem in de cloudera CDH 4.X with sorl
所有这一切对我来说都很好，当我在de cloudera CDH 4.X与sorl有这个问题
Regards
问候
#2
0
Go to this link : http://www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_ig_mysql.html
转到此链接：http：//www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_ig_mysql.html
Go to this topic: Installing the MySQL JDBC Driver
转到此主题：安装MySQL JDBC驱动程序
Follow the instruction. Finally Restart your hive service
按照说明操作。最后重启你的配置单元服务
Thx Kumar
Thx Kumar
#1
2
ok cloudera is using version HIVE 0.10, that doesn't support remote login
ok cloudera使用版本HIVE 0.10，不支持远程登录
but you need to go around that bug, logging to the server that is getting the error the cloudera manager will tell you the ip
但你需要绕过那个bug，登录到服务器，这个错误会让cloudera经理告诉你ip
1) create login in to the Server that fails to install HIVE 2) Create a $HADDOP_HOME
1）创建登录到无法安装HIVE的服务器2）创建$ HADDOP_HOME
$HADOOP_HOME="/usr/lib/hadoop/"
3) INSTALL postgres in the server that fails
3）在失败的服务器中安装postgres
`$ sudo apt-get install postgresql`
$ cat /etc/postgresql/9.1/main/postgresql.conf | grep -e listen -e standard_conforming_strings
modify this to lines in the file
将其修改为文件中的行
listen_addresses = '*'
standard_conforming_strings = off
You also need to configure authentication for your network in pg_hba.conf. You need to make sure that the PostgreSQL user that you will create in the next step will have access to the server from a remote host. To do this, add a new line into pg_hba.con that has the following information:
您还需要在pg_hba.conf中为网络配置身份验证。您需要确保将在下一步中创建的PostgreSQL用户可以从远程主机访问服务器。为此，请在pg_hba.con中添加一个包含以下信息的新行：
host <database> <user> <network address> <mask> password
Start PostgreSQL Server
启动PostgreSQL服务器
$ sudo service postgresql start
Use chkconfig utility to ensure that your PostgreSQL server will start at a boot time:
使用chkconfig实用程序确保PostgreSQL服务器在启动时启动：
chkconfig postgresql on
You can use the chkconfig utility to verify that PostgreSQL server will be started at boot time, for example:
您可以使用chkconfig实用程序验证PostgreSQL服务器是否将在引导时启动，例如：
chkconfig --list postgresql
Step 2: Install the Postgres JDBC Driver
第2步：安装Postgres JDBC驱动程序
Before you can run the Hive metastore with a remote PostgreSQL database, you must configure a JDBC driver to the remote PostgreSQL database, set up the initial database schema, and configure the PostgreSQL user account for the Hive user.
在使用远程PostgreSQL数据库运行Hive Metastore之前，必须为远程PostgreSQL数据库配置JDBC驱动程序，设置初始数据库模式，并为Hive用户配置PostgreSQL用户帐户。
To install the PostgreSQL JDBC Driver on a Debian/Ubuntu system:
要在Debian / Ubuntu系统上安装PostgreSQL JDBC驱动程序：
Install libpostgresql-jdbc-java and symbolically link the file into the /usr/lib/hive/lib/ directory.
安装libpostgresql-jdbc-java并将该文件符号链接到/ usr / lib / hive / lib /目录。
$ sudo apt-get install libpostgresql-jdbc-java
$ ln -s /usr/share/java/postgresql-jdbc4.jar /usr/lib/hive/lib/postgresql-jdbc4.jar
Step 3: Create the metastore database and user account
第3步：创建Metastore数据库和用户帐户
bash# sudo –u postgres psql
bash$ psql
postgres=# CREATE USER hiveuser WITH PASSWORD 'mypassword';
postgres=# CREATE DATABASE metastore;
postgres=# \c metastore;
You are now connected to database 'metastore'.
postgres=# \i /usr/lib/hive/scripts/metastore/upgrade/postgres/hive-schema-0.10.0.postgres.sql
SET
SET
...
Now you need to grant permission for all metastore tables to user hiveuser. PostgreSQL does not have statements to grant the permissions for all tables at once; you'll need to grant the permissions one table at a time. You could automate the task with the following SQL script:
现在，您需要将所有Metastore表的权限授予用户hiveuser。 PostgreSQL没有语句一次授予所有表的权限;您需要一次授予一个表的权限。您可以使用以下SQL脚本自动执行该任务：
bash# sudo –u postgres psql
metastore=# \o /tmp/grant-privs
metastore=# SELECT 'GRANT SELECT,INSERT,UPDATE,DELETE ON "' || schemaname || '"."' || tablename || '" TO hiveuser ;'
metastore-# FROM pg_tables
metastore-# WHERE tableowner = CURRENT_USER and schemaname = 'public';
metastore=# \o
metastore=# \i /tmp/grant-privs
You can verify the connection from the machine where you'll be running the metastore service as follows:
您可以从将运行Metastore服务的计算机验证连接，如下所示：
psql –h myhost –U hiveuser –d metastore
metastore=#
Step 4: Configure the Metastore Service to Communicate with the PostgreSQL Database
步骤4：配置Metastore服务以与PostgreSQL数据库通信
change the IP of the AWS amazon master Server, or your master server, don't use DNS name
更改AWS amazon主服务器或主服务器的IP，请勿使用DNS名称
$find / -name hive-site.xml
$nano /run/cloudera-scm-agent/process/27-hive-metastore-create-tables/hive-site.xml
in the File search for:
在文件中搜索：
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:postgresql://myhost/metastore</value>
</property>
and change to the correct IP that is your Master Hadoop Server where u are running Cloudera Manager also every link in that file that is not correctly write to the hadoop master Cloudera manager connector , you will have to change to the correct IP
并更改为正确的IP是你师父的Hadoop服务器，其中u正在运行的Cloudera Manager还该文件中的每一个环节没有被正确地写入Hadoop的主Cloudera的经理连接器，你将不得不更改为正确的IP
after all this just get back to the autoinstall of cloudera manager and run again and it will be all good :)
毕竟这只是回到cloudera经理的自动安装并再次运行，这将是很好的:)
that it all the installation that you have to work around our contract cloudera support (that's their business) :)
你必须解决我们的合同cloudera支持（这是他们的业务）所有的安装:)
all this it works fine for me when i have this problem in de cloudera CDH 4.X with sorl
所有这一切对我来说都很好，当我在de cloudera CDH 4.X与sorl有这个问题
Regards
问候
#2
0
Go to this link : http://www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_ig_mysql.html
转到此链接：http：//www.cloudera.com/documentation/enterprise/5-7-x/topics/cm_ig_mysql.html
Go to this topic: Installing the MySQL JDBC Driver
转到此主题：安装MySQL JDBC驱动程序
Follow the instruction. Finally Restart your hive service
按照说明操作。最后重启你的配置单元服务
Thx Kumar
Thx Kumar