CDH6.2安装配置第二篇：CDH安装的前期配置

时间：2023-03-08 18:54:55

本篇介绍cdh安装之前需要的一些必要配置，当然这些配置也可以用shell脚本来配置。在安装之前请先配置好yum源，在文中用的统一都是阿里源。在安装的时候，要确保主机的内存是4G以上，要不然会无限重启cm服务

另外2篇传送门：

CDH6.2安装配置第一篇：CDH配置本地http服务

CDH6.2安装配置第三篇：前台页面配置讲解

其他配置的传送门：

LINUX的ssh免密码配置

LINUX之ntp时间同步服务配置

CDH版本：6.2

系统：centos7（el6是代表centos6系统，centos7则用el7表示）

主机3台：

master.com.cn 192.168.1.110   2C4G

salver1.com.cn 192.168.1.111   1C2G

salver2.com.cn 192.168.1.112  1C2G

yum服务器：yum.com.cn 192.168.1.100。这里如果不知道怎么配置的朋友，可以参考第一篇：https://www.cnblogs.com/yaowentao/p/12296329.html

基本配置

1. 主机名在系统安装的配置完毕，如果没有配置，可以进系统在修改

 $ vi /etc/hostname

2. 配置host文件

 $ vi /etc/hosts

 192.168.1.100 yum.com.cn

 192.168.1.110 master.com.cn

 192.168.1.111 salver1.com.cn

 192.168.1.112 salver2.com.cn

3. 关闭防火墙，selinux。配置完后重启

 $ sudo systemctl disable firewalld        #停止防火墙

 $ sudo systemctl stop firewalld           #禁止开机启动防火墙

 $ vi /etc/selinux/config

 SELINUX=disable　　　　　　　　　　　　　　　　#关闭selinux

4. 配置ntp服务与ssh免密码登陆配置，请参考我的另外2篇文档

ssh免密码配置传送门：https://www.cnblogs.com/yaowentao/p/12300321.html

ntp配置传送门：https://www.cnblogs.com/yaowentao/p/12300408.html

5. 如果有本地的yum源，就配置。如果没有就使用公网的yum源。

5.1 配置yum 本地库,也可以下载官网的repo文件，一个是从网上下载，一个是从本地yum库下载
 [cloudera-repo]

 name=cloudera-repo

 baseurl=http://192.168.10.225/cloudera-repos/cm6/6.2.1/redhat7/yum/

 enabled=

 gpgcheck=
5.2 网上yum源，直接下载即可
 $ sudo wget https://archive.cloudera.com/cm6/6.2.1/redhat7/yum/cloudera-manager.repo -P /etc/yum.repos.d/ 

6. 安装cdh专用jdk（需要在所有主机上都要安装）

 $ sudo yum -y install oracle-j2sdk1.8
CDH6.2安装配置第二篇：CDH安装的前期配置

7. 配置本地的mysql数据库，便于存放cdh服务器的一些配置，以及hadoop组件中需要的一些配置，如hive的源数据库。这里一般推荐用mysql，当然oracle等其他的数据库也可以。

  1. 直接用yum源安装
  2.  $ wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
    
 $ sudo rpm -ivh mysql-community-release-el7-.noarch.rpm
    
 $ sudo yum update
    
 $ sudo yum install mysql-server
    
 $ sudo systemctl start mysqld

    CDH6.2安装配置第二篇：CDH安装的前期配置

  3. 配置mysql数据，并进行初始化，设置root密码
  4. $vi /etc/my.cnf

[mysqld]
    
datadir=/var/lib/mysql
    
socket=/var/lib/mysql/mysql.sock
    
transaction-isolation = READ-COMMITTED
    
# Disabling symbolic-links is recommended to prevent assorted security risks;
    
# to do so, uncomment this line:
    
symbolic-links = 

key_buffer_size = 32M
    
max_allowed_packet = 32M
    
thread_stack = 256K
    
thread_cache_size =
    
query_cache_limit = 8M
    
query_cache_size = 64M
    
query_cache_type = 

max_connections =
    
#expire_logs_days =
    
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space.
    
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
    
#system and chown the specified folder to the mysql user.
    
log_bin=/var/lib/mysql/mysql_binary_log

#In later versions of MySQL, if you enable the binary log and do not set
    
#a server_id, MySQL will not start. The server_id must be unique within
    
#the replicating group.
    
server_id=

binlog_format = mixed

read_buffer_size = 2M
    
read_rnd_buffer_size = 16M
    
sort_buffer_size = 8M
    
join_buffer_size = 8M

# InnoDB settings
    
innodb_file_per_table =
    
innodb_flush_log_at_trx_commit  =
    
innodb_log_buffer_size = 64M
    
innodb_buffer_pool_size = 4G
    
innodb_thread_concurrency =
    
innodb_flush_method = O_DIRECT
    
innodb_log_file_size = 512M

[mysqld_safe]
    
log-error=/var/log/mysqld.log
    
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES

  5. 配置开机自启动

  6.  $ sudo systemctl enable mysqld
    
 $ sudo systemctl start mysqld
  7. 初始化mysql，设置root密码
  8.  $ sudo /usr/bin/mysql_secure_installation
    
 [...]
    
 Enter current password for root (enter for none):
    
 OK, successfully used password, moving on...
    
 [...]
    
 Set root password? [Y/n] Y
    
 New password:
    
 Re-enter new password:
    
 Remove anonymous users? [Y/n] Y
    
 [...]
    
 Disallow root login remotely? [Y/n] N
    
 [...]
    
 Remove test database and access to it [Y/n] Y
    
 [...]
    
 Reload privilege tables now? [Y/n] Y
    
 All done!

    CDH6.2安装配置第二篇：CDH安装的前期配置

  9. 安装mysql的jdbc

  10.  $ wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz
    
 $ tar zxvf mysql-connector-java-5.1..tar.gz

 $ sudo mkdir -p /usr/share/java/
    
 $ cd mysql-connector-java-5.1.
    
 $ sudo cp mysql-connector-java-5.1.-bin.jar /usr/share/java/mysql-connector-java.jar
  11. 给root授远程登陆权限
  12.  mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '' WITH GRANT OPTION;
    
 mysql> FLUSH PRIVILEGES;
  13. 创建cm服务，hadoop组件需要的库。用户统一用root。
  14.  CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    
 GRANT ALL ON scm.* TO 'root'@'%' IDENTIFIED BY '';
    
 CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    
 GRANT ALL ON amon.* TO 'root'@'%' IDENTIFIED BY '';
    
 CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    
 GRANT ALL ON rman.* TO 'root'@'%' IDENTIFIED BY '';
    
 CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    
 GRANT ALL ON hue.* TO 'root'@'%' IDENTIFIED BY '';
    
 CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    
 GRANT ALL ON metastore.* TO 'root'@'%' IDENTIFIED BY '';
    
 CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    
 GRANT ALL ON sentry.* TO 'root'@'%' IDENTIFIED BY '';
    
 CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    
 GRANT ALL ON nav.* TO 'root'@'%' IDENTIFIED BY '';
    
 CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    
 GRANT ALL ON navms.* TO 'root'@'%' IDENTIFIED BY '';
    
 CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
    
 GRANT ALL ON oozie.* TO 'root'@'%' IDENTIFIED BY '';

    CDH6.2安装配置第二篇：CDH安装的前期配置

  15. 安装cm服务器文件

  16.  master:
    
 $ sudo yum -y install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

    CDH6.2安装配置第二篇：CDH安装的前期配置

     salver:
    
 $ sudo yum -y install cloudera-manager-daemons cloudera-manager-agent

    CDH6.2安装配置第二篇：CDH安装的前期配置 CDH6.2安装配置第二篇：CDH安装的前期配置

  17. scm初始化
    1. 　　 
       $ sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm root

      CDH6.2安装配置第二篇：CDH安装的前期配置

  18. 完成以上配置，就能正常登陆cm服务器的前台了。下面进入服务器的配置。请看第三篇。
    1. 　　启动cm服务 
       $ sudo systemctl start cloudera-scm-server
      
 $ sudo tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

      CDH6.2安装配置第二篇：CDH安装的前期配置

启动成功：

CDH6.2安装配置第二篇：CDH安装的前期配置

各位小伙伴，对以上如有问题，请在下方留言沟通。以上都是我亲自测试通过安装成功的。

 

转发，请注明出处！

相关文章