cdh 安装系列2--cdh manager product 安装

时间:2022-06-29 03:42:12

前期准备工作:

准备三台centos7 地址分别为 192.168.20.163;192.168.20.162;192.168.20.161

用163这台机器链接外网,并安装manager以及manager需要的数据库mysql

1.关闭防火墙

iptables-save > /root/firewall.rules
systemctl stop firewalld.service
systemctl disable firewalld.service

2.检查 selinux

getenforce 
如果返回disabled就可以不用处理。
如果不是则要修改
/etc/selinux/config
中修改
SELINUX=permissive
保存以后执行
setenforce 0

3.ntp

如果是主机,配置ntp server

yum install ntp
编辑 /etc/ntp.conf file to add NTP servers, as in the following example.
server 0.pool.ntp.org
server 1.pool.ntp.org
server 2.pool.ntp.org 在命令行下执行
sudo systemctl start ntpd
sudo systemctl enable ntpd

其他机器配置成和ntp server 同步。

ntpdate -u 192.168.20.163
hwclock --systohc

4.安装python 2.7 系统自带了,这里省略安装步骤。

一、安装 repo库

1.安装repo库

sudo wget https://archive.cloudera.com/cm6/6.0.1/redhat7/yum/cloudera-manager.repo  -P /etc/yum.repos.d/

2.Import the repository signing GPG key:

sudo rpm --import https://archive.cloudera.com/cm6/6.0.0/redhat7/yum/RPM-GPG-KEY-cloudera

二、安装jdk 这里不描述

用的版本是:1.8

[root@host03 java]# java -version
openjdk version "1.8.0_171"

安装步骤:

1.sudo yum install oracle-j2sdk1.8

三、安装manager server

sudo yum install cloudera-manager-daemons cloudera-manager-agent cloudera-manager-server

四.安装需要的数据库,这里我选择的是mysql数据库作为元数据库

1.wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm

2.sudo rpm -ivh mysql-community-release-el7-5.noarch.rpm

3.sudo yum update

4.sudo yum install mysql-server

5.sudo systemctl start mysqld

配置数据库,

6.sudo systemctl stop mysqld

7.Move old InnoDB log files /var/lib/mysql/ib_logfile0 and /var/lib/mysql/ib_logfile1 out of /var/lib/mysql/ to a backup location.

8.Determine the location of the option file, my.cnf (/etc/my.cnf by default).参考文件如下,可以直接copy

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
symbolic-links = 0

key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1

max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M

#log_bin should be on a disk with enough free space.
#Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your
#system and chown the specified folder to the mysql user.
log_bin=/var/lib/mysql/mysql_binary_log

#In later versions of MySQL, if you enable the binary log and do not set
#a server_id, MySQL will not start. The server_id must be unique within
#the replicating group.
server_id=1

binlog_format = mixed

read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M

# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

sql_mode=STRICT_ALL_TABLES

9.保证机器启动的时候MySQL自己启动

sudo systemctl enable mysqld

10.启动本次mysql服务器

sudo systemctl start mysqld

11.设置mysql的密码

sudo /usr/bin/mysql_secure_installation

(我的环境中设置的密码是root)

安装jdbc驱动

12.下载,注意这里不能yum安装,会使得jdk变成openjdk

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.46.tar.gz

13.tar zxvf mysql-connector-java-5.1.46.tar.gz

14.

sudo mkdir -p /usr/share/java/
cd mysql-connector-java-5.1.46
sudo cp mysql-connector-java-5.1.46-bin.jar /usr/share/java/mysql-connector-java.jar

配置MySQL里面的数据库

1.登录

mysql -u root -p

(我的环境中设置的密码是root)

2.建立数据库脚本

CREATE DATABASE scm DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON scm.* TO 'scm'@'%' IDENTIFIED BY 'scm';
CREATE DATABASE amon DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON amon.* TO 'amon'@'%' IDENTIFIED BY 'amon';
CREATE DATABASE rman DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON rman.* TO 'rman'@'%' IDENTIFIED BY 'rman';
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON hue.* TO 'hue'@'%' IDENTIFIED BY 'hue';
CREATE DATABASE metastore DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON metastore.* TO 'hive'@'%' IDENTIFIED BY 'hive';
CREATE DATABASE sentry DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON sentry.* TO 'sentry'@'%' IDENTIFIED BY 'centry';
CREATE DATABASE nav DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON nav.* TO 'nav'@'%' IDENTIFIED BY 'nav';
CREATE DATABASE navms DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON navms.* TO 'navms'@'%' IDENTIFIED BY 'navms';
CREATE DATABASE oozie DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
GRANT ALL ON oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie';

3.SHOW DATABASES;

可以用来检查是否成功

五。安装 cdh manager数据库

1.由于我的cdh manager和 scm数据库都是安装在一台192.168.20.163机器上所以

sudo /opt/cloudera/cm/schema/scm_prepare_database.sh mysql scm scm

2.sudo systemctl start cloudera-scm-server

启动scm server 估计要等待5分钟,指导jetty服务器完全启动

3.http://192.168.20.163:7180

登录 admin admin测试是否安全成功。

cdh 安装系列2--cdh manager product 安装

cloudera-manager.repo