Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

时间:2024-01-06 08:52:56

其实,基本思路跟如下差不多,我就不多详细说了,贴出主要图。

  博主,我是直接借鉴下面这位博主,来进行安装的!(灰常感谢他们!)

在线和离线安装Cloudera CDH 5.6.0

 Cloudera Manager 5和CDH5离线安装

Ubuntu14.04用apt在线/离线安装CDH5.1.2[Apache Hadoop 2.3.0

https://www.zybuluo.com/ncepuwanghui/note/474966

 ClouderManager官网

https://www.cloudera.com/documentation/manager/5-1-x/Cloudera-Manager-Installation-Guide/cm5ig_mysql.html#id_trn_pps_xm_unique_1

说在前面的话

  我的机器是总共4台,分别为ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。

  在ubuntucmbigdata1上执行了

sudo apt-get install cloudera-manager-daemons cloudera-manager-server 

sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent 

  即,ubuntucmbigdata1是既做server,又做agent。

  在ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4上执行了

sudo apt-get install cloudera-manager-daemons  cloudera-manager-agent 

  即,ubuntucmbigdata2、3、和4是只做agent。

  在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

新建文件sudo vi /etc/apt/sources.list.d/cloudera-manager.list,复制以下内容到cloudera.list里

# Packages for Cloudera Manager, Version , on Ubuntu 14.04 x86_64
deb [arch=amd64] http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib
deb-src http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm trusty-cm5 contrib

  并且在在ubuntucmbigdata1、ubuntucmbigdata2、ubuntucmbigdata3、ubuntucmbigdata4上

  向apt添加cloudera源公钥,在所有节点执行如下命令:

curl -s http://archive.cloudera.com/cm5/ubuntu/trusty/amd64/cm/archive.key | sudo apt-key add -

然后,再

sudo apt-get update

  具体在后面我会详细教大家怎么安装。

  

  的确,我也跟这位博主一样,也需要在实验室里部署。以及也是4台机器。(至于原因,大家都懂得,中国高校研究生喜欢4、3、2、1来进行加速比做实验论文)

在实验室安装过社区版的Hadoop,当时用的是普通PC机,安装的过程遇到了好多问题。当解决了大部分的安装问题之后由于实验室的需求,需要安装一个商业发行版的hadoop ,我们就采用的Cloudera 公司发行的版本。首先要说明的是CDH对内存要求比较高,当我们采用原来的4G内存的PC机,只要启动基本内存就满了,并且会拖慢整个集群的运算速度。所以说安装CDH对硬件的要求比较高。安装CDH 有三种安装方式:在线安装、半离线安装、离线安装。在线安装和离线安装我都安装过,虽然在线安装貌似简单但是实际用起来还是没有离线安装方便,我还是建议大家采用离线安装的方式。不过下面都会介绍在线安装的方式和离线安装的方式。希望对别人有所帮助

  

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

  其它的,ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4。

  

 参考 第一步:

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5)(一)

Ubuntu14.04安装之后的一些配置

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

参考

ubuntu 下 visudo的使用

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

  在文件末尾(按快捷键,Shift + g)添加bigdata  ALL=(ALL) NOPASSWD:ALL

ctrl+o 保存

ctrl+x 退出。

  最后,得到如下

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

  我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

  防火墙

  参考

hadoop-2.6.0.tar.gz + spark-1.5.2-bin-hadoop2.6.tgz的集群搭建(单节点)(Ubuntu系统)

 Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

  由此,表明Ubuntu14.04是默认没开启防火墙的。

 我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

相关服务的安装

  如果没有安装的话可以使用下面的命令安装一下。

  参考

Ubuntu 14.04 安装 Cloudera Manager 5 And CDH 5 (在线安装)

bigdata@ubuntucmbigdata1:~$ sudo apt-get install lrzsz

bigdata@ubuntucmbigdata1:~$ sudo apt-get install openssh-server 

  这个工具要装(你懂得),2个理由:1,可以远程管理系统。2,Cloudera Manages 集群管理需要用该服务。

 我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

ssh 服务配置、vim的安装

  参考

Ubuntu14.04安装之后的一些配置

  打开sshd_config文件, 在  /etc/ssh 目录下。

# Authentication:
LoginGraceTime 120
#PermitRootLogin without-password
PermitRootLogin yes
StrictModes yes

  注释掉 #PermitRootLogin without-password 行。

  添加 PermitRootLogin yes 行

  修改后,如下:

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

 我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

修改hosts文件

  vi /etc/hosts

在里面添加主机名和IP之间的对应关系

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

#127.0.1.1      ubuntucmbigdata1

192.168.86.31  ubuntucmbigdata1
192.168.86.32 ubuntucmbigdata2
192.168.86.33 ubuntucmbigdata3
192.168.86.34 ubuntucmbigdata4

   我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

  主机名

bigdata@ubuntucmbigdata1:~$ sudo vim /etc/hostname
bigdata@ubuntucmbigdata2:~$ sudo vim /etc/hostname
bigdata@ubuntucmbigdata3:~$ sudo vim /etc/hostname
bigdata@ubuntucmbigdata4:~$ sudo vim /etc/hostname

  这里肯定是对的。

配置ssh免密码登录 (对于这一步,其实可以不做,因为,在CM时会自动安装,大家可以跳过)

  见

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
bigdata@ubuntucmbigdata1:~$ mkdir .ssh
bigdata@ubuntucmbigdata1:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/bigdata/.ssh/id_rsa):(回车)
Enter passphrase (empty for no passphrase):(回车)
Enter same passphrase again: (回车)
Your identification has been saved in /home/bigdata/.ssh/id_rsa.
Your public key has been saved in /home/bigdata/.ssh/id_rsa.pub.
The key fingerprint is:
58:c2:3f:25:cb:82:5b:d5:b4:b2:ea:54:ac:e2:ea:c8 bigdata@ubuntucmbigdata1
The key's randomart image is:
+--[ RSA 2048]----+
| . |
| . o . |
| o = + |
| . O * |
| . + S |
| o = . |
| o + |
|.. . + |
|.Eo.. . |
+-----------------+
bigdata@ubuntucmbigdata1:~$ cd .ssh
bigdata@ubuntucmbigdata1:~/.ssh$ cat id_rsa.pub >> authorized_keys
bigdata@ubuntucmbigdata1:~/.ssh$ cd ..
bigdata@ubuntucmbigdata1:~$ chmod 700 .ssh
bigdata@ubuntucmbigdata1:~$ chmod 600 .ssh/*
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata1
The authenticity of host 'ubuntucmbigdata1 (127.0.1.1)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 14:45:01 2017 from 192.168.86.1
bigdata@ubuntucmbigdata1:~$ exit;
logout
Connection to ubuntucmbigdata1 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata1
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:52:15 2017 from localhost
bigdata@ubuntucmbigdata1:~$ exit;
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

   我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

  将所有节点(ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4)中的共钥id_ras.pub拷贝到ubuntucmbigdata1中的authorized_keys文件中。

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
bigdata@ubuntucmbigdata2:~$ cat ~/.ssh/id_rsa.pub | ssh bigdata@ubuntucmbigdata1 'cat >> ~/.ssh/authorized_keys'
The authenticity of host 'ubuntucmbigdata1 (192.168.86.31)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1,192.168.86.31' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata1's password:
bigdata@ubuntucmbigdata2:~$
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
bigdata@ubuntucmbigdata3:~$ cat ~/.ssh/id_rsa.pub | ssh bigdata@ubuntucmbigdata1 'cat >> ~/.ssh/authorized_keys'
The authenticity of host 'ubuntucmbigdata1 (192.168.86.31)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1,192.168.86.31' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata1's password:
bigdata@ubuntucmbigdata3:~$
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
bigdata@ubuntucmbigdata4:~$ cat ~/.ssh/id_rsa.pub | ssh bigdata@ubuntucmbigdata1 'cat >> ~/.ssh/authorized_keys'
The authenticity of host 'ubuntucmbigdata1 (192.168.86.31)' can't be established.
ECDSA key fingerprint is 1c:85:65:a1:d7:f0:fe:56:4d:86:0f:6e:25:e3:80:46.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata1,192.168.86.31' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata1's password:
bigdata@ubuntucmbigdata4:~$
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

  

  得到

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
bigdata@ubuntucmbigdata1:~$ cd .ssh
bigdata@ubuntucmbigdata1:~/.ssh$ ls
authorized_keys id_rsa id_rsa.pub known_hosts
bigdata@ubuntucmbigdata1:~/.ssh$ cat authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCrEPrX5HHQCnwWoavXUoM8N+fINUckzZ7EcDjJqs1EYkrzltdmMrKQv9VeGGj2BPaGoxzmgit+DPFGuPmPgftBOPWQnLVoILqtQd+H5m0wrK67W7Y98cUpZ+0thCbnmUZRuAES1mt2xS/bL2ibw8AvRrd6HUHOhH17Tcd/2gok32ximn48kPzs6li5d2ay8rPqgqD3QQw3NlAhVmK7RVmpsvlHCTnhrQIoNCz8A+adtK5r1jSgH7uKmPAJTqMUP2Zgv02XQTf/kjW3eacNAWR96Khdh0HsktaK6HuC+FFC+8bXqAj4Hih25EX8RTMFMl3nhGT5FPMLyt7C0Z65QhKV bigdata@ubuntucmbigdata1
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDi7mZhUMzB1h7XjWPIZo3w/Tps0Oxr3mgOeKVfoD4pme/lzgTz61FpEmgggc2WB+mNLKSG/WiazTzuSXKUdBXnM5LNeCGV3ZcWBxFrwp2+wAYH6L5wOGqRq+oQzKUHIsvqEc7TBpN3JmaqZOw4zVCwRBBzQ4cg1B2SjSsjDFLBpjlfM+N8UDG+g9OaZDlEu1z5oUyz0KsKbBtvkkSott+sB+EX4zYSDz/cwUPmSf76RJLk89AI8aX4wGS+P4Eahs5/J2wfLOnAWBG2NQoYVPKyN4sXj7OWw+aErriUJXOqLayYV5xSM2yJjT9WuzbNOiR/D7HGZBlC4wXWG+11GdO5 bigdata@ubuntucmbigdata2
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZtE7Z4cXPjv/t0bC4Ozi3HuovI6XiNe+tJ7cQ6WKp/m44a8w/FqgB4HVZLLQusuXwQM/mEzHKiL1EdPsi5om/s7A7xyDakl2OSUOqki871+/TSjFVTUOOz/H/wwHvl2QzBkb8/dMZLphU1/E4odrFhhlhGGpjK5WUdY8cYH6yyli6W3oDPZ55UG+8EnM89T1AyhC1E81xmyMzGMCj4603oZFwckOeff5n2dqeXg0CRhXO9HhsF9kSZQnE2GmR5pLZlORMkitsCKIhaf5UOBzrGENeJylHGCV1F+KlYjQ0ND4ufTUq14aAnUwBOCqCMhB1tBOBGI4O3S0+j27xumwN bigdata@ubuntucmbigdata3
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXPtqzCU+AmRi+l5lysCEG8gbFKPItYjczvSZIe6+arOfgE/3Ozu2cwr9MABTjgd80hk7ZG4YNRxBPxKvXKtcZxGAe6poe1G5UOrlKOMLeqv1mnUiwtRVsiWp0qnPFcmMt17mpCyN7XnCOLTsdTcl+BVl0c2S2i1xoC6vRKvqV1C/nneJUmuD3KRR1zmR3+a9Wj4yhRRahO6ro/LhgqdwrEaR+NOuT+/1d058fXANToaqyUMa72pz5M2kdxZprumzmHAJ5KlOYImlGrahdLk1wPFckA2m1BMKKycxZHFm1X1O9HeSFrD9N+pdtZY1CXdc6N9hmWKzjBDqw71rZgdUd bigdata@ubuntucmbigdata4
bigdata@ubuntucmbigdata1:~/.ssh$
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

  说明成功!

  然后将中ubuntucmbigdata1的authorized_keys文件分发到所有节点上面。

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
bigdata@ubuntucmbigdata1:~/.ssh$ scp -r authorized_keys bigdata@ubuntucmbigdata2:~/.ssh/
The authenticity of host 'ubuntucmbigdata2 (192.168.86.32)' can't be established.
ECDSA key fingerprint is 17:a2:fb:1e:47:dc:79:d5:2d:5a:aa:79:9b:10:c2:eb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata2,192.168.86.32' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata2's password:
authorized_keys 100% 1624 1.6KB/s 00:00
bigdata@ubuntucmbigdata1:~/.ssh$ scp -r authorized_keys bigdata@ubuntucmbigdata3:~/.ssh/
The authenticity of host 'ubuntucmbigdata3 (192.168.86.33)' can't be established.
ECDSA key fingerprint is 6a:32:65:61:98:78:3f:ad:08:2c:f3:a1:bc:c6:c4:91.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata3,192.168.86.33' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata3's password:
authorized_keys 100% 1624 1.6KB/s 00:00
bigdata@ubuntucmbigdata1:~/.ssh$ scp -r authorized_keys bigdata@ubuntucmbigdata4:~/.ssh/
The authenticity of host 'ubuntucmbigdata4 (192.168.86.34)' can't be established.
ECDSA key fingerprint is f4:d5:92:17:dc:b1:8f:95:61:11:95:cd:d3:fa:2e:57.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ubuntucmbigdata4,192.168.86.34' (ECDSA) to the list of known hosts.
bigdata@ubuntucmbigdata4's password:
authorized_keys 100% 1624 1.6KB/s 00:00
bigdata@ubuntucmbigdata1:~/.ssh$
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

  大家通过ssh 相互访问,如果都能无密码访问,代表ssh配置成功。这里很简单,我就不多赘述了。

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata1
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:52:23 2017 from localhost
bigdata@ubuntucmbigdata1:~$ exit;Connection to ubuntucmbigdata1 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata2
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:56:00 2017 from localhost
bigdata@ubuntucmbigdata2:~$ exit;
logout
Connection to ubuntucmbigdata2 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata3
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:56:14 2017 from localhost
bigdata@ubuntucmbigdata3:~$ exit;
logout
Connection to ubuntucmbigdata3 closed.
bigdata@ubuntucmbigdata1:~$ ssh ubuntucmbigdata4
Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 4.2.0-27-generic x86_64) * Documentation: https://help.ubuntu.com/ New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it. Last login: Sun Jun 25 15:56:27 2017 from localhost
bigdata@ubuntucmbigdata4:~$ exit;
logout
Connection to ubuntucmbigdata4 closed.
bigdata@ubuntucmbigdata1:~$
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

  这样,就说明是成功!

  

JDK的安装(建议1.8)(对于这一步,其实可以不做,因为,在CM时会自动安装,大家可以跳过)(但是要先卸载linux系统自带的openjdk,因为自动也是安装的是orcal公司的)

  见

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5)(一)

  这里,因为是ubuntu14.04版本,作为专业人员来说,

Ubuntu14.04下Neo4j图数据库官网安装部署步骤(图文详解)(博主推荐)

bigdata@ubuntucmbigdata1:~$ sudo apt-get remove openjdk* 

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
bigdata@ubuntucmbigdata1:/usr/local$ pwd
/usr/local
bigdata@ubuntucmbigdata1:/usr/local$ ll
total 40
drwxr-xr-x 10 root root 4096 2月 18 2016 ./
drwxr-xr-x 10 root root 4096 2月 18 2016 ../
drwxr-xr-x 2 root root 4096 2月 18 2016 bin/
drwxr-xr-x 2 root root 4096 2月 18 2016 etc/
drwxr-xr-x 2 root root 4096 2月 18 2016 games/
drwxr-xr-x 2 root root 4096 2月 18 2016 include/
drwxr-xr-x 4 root root 4096 2月 18 2016 lib/
lrwxrwxrwx 1 root root 9 6月 24 15:00 man -> share/man/
drwxr-xr-x 2 root root 4096 2月 18 2016 sbin/
drwxr-xr-x 7 root root 4096 2月 18 2016 share/
drwxr-xr-x 2 root root 4096 2月 18 2016 src/
bigdata@ubuntucmbigdata1:/usr/local$ sudo mkdir jdk
bigdata@ubuntucmbigdata1:/usr/local$ ll
total 44
drwxr-xr-x 11 root root 4096 6月 25 16:08 ./
drwxr-xr-x 10 root root 4096 2月 18 2016 ../
drwxr-xr-x 2 root root 4096 2月 18 2016 bin/
drwxr-xr-x 2 root root 4096 2月 18 2016 etc/
drwxr-xr-x 2 root root 4096 2月 18 2016 games/
drwxr-xr-x 2 root root 4096 2月 18 2016 include/
drwxr-xr-x 2 root root 4096 6月 25 16:08 jdk/
drwxr-xr-x 4 root root 4096 2月 18 2016 lib/
lrwxrwxrwx 1 root root 9 6月 24 15:00 man -> share/man/
drwxr-xr-x 2 root root 4096 2月 18 2016 sbin/
drwxr-xr-x 7 root root 4096 2月 18 2016 share/
drwxr-xr-x 2 root root 4096 2月 18 2016 src/
bigdata@ubuntucmbigdata1:/usr/local$ sudo chown -R bigdata:bigdata jdk/
bigdata@ubuntucmbigdata1:/usr/local$ cd jdk/
bigdata@ubuntucmbigdata1:/usr/local/jdk$ pwd
/usr/local/jdk
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo rz
rz waiting to receive.
bigdata@ubuntucmbigdata1:/usr/local/jdk$ ll
total 177000
drwxr-xr-x 2 bigdata bigdata 4096 6月 25 16:09 ./
drwxr-xr-x 11 root root 4096 6月 25 16:08 ../
-rw-r--r-- 1 root root 181238643 9月 3 2016 jdk-8u60-linux-x64.tar.gz
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo tar -zxvf jdk-8u60-linux-x64.tar.gz
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
bigdata@ubuntucmbigdata1:/usr/local/jdk$ ll
total 177004
drwxr-xr-x 3 bigdata bigdata 4096 6月 25 16:09 ./
drwxr-xr-x 11 root root 4096 6月 25 16:08 ../
drwxr-xr-x 8 uucp 143 4096 8月 5 2015 jdk1.8.0_60/
-rw-r--r-- 1 root root 181238643 9月 3 2016 jdk-8u60-linux-x64.tar.gz
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo rm jdk-8u60-linux-x64.tar.gz
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo chown -R bigdata:bigdata jdk1.8.0_60/
bigdata@ubuntucmbigdata1:/usr/local/jdk$ ll
total 12
drwxr-xr-x 3 bigdata bigdata 4096 6月 25 16:10 ./
drwxr-xr-x 11 root root 4096 6月 25 16:08 ../
drwxr-xr-x 8 bigdata bigdata 4096 8月 5 2015 jdk1.8.0_60/
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo vim /etc/profile
bigdata@ubuntucmbigdata1:/usr/local/jdk$ sudo source /etc/profile
sudo: source: command not found
bigdata@ubuntucmbigdata1:/usr/local/jdk$ source /etc/profile
bigdata@ubuntucmbigdata1:/usr/local/jdk$ java -version
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
bigdata@ubuntucmbigdata1:/usr/local/jdk$
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

   我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

以及,我们还可以设置文件打开数量和用于最大进程数

  文件打开数量

  参考

Cloudera Manager安装之Cloudera Manager安装前准备(CentOS6.5)(一)

bigdata@ubuntucmbigdata1:~$ sudo vi /etc/security/limits.conf 

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

* soft nofile 65535
* hard nofile 65535
* soft nproc 32000
* hard nproc 32000

   我这里,仅拿ubuntucmbigdata1来说,其他的ubuntucmbigdata2、ubuntucmbigdata3和ubuntucmbigdata4是一样的。

   ubuntu系统的root用户开启

  参考

Ubuntu14.04安装之后的一些配置

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

bigdata@ubuntucmbigdata1:~$ sudo -s
root@ubuntucmbigdata1:~# pwd
/home/bigdata
root@ubuntucmbigdata1:~#

 MySQL的安装(以下步骤仅仅在ubuntucmbigdata1这台机器上做)

  注意:Cloudera-Manages-Server 端 需要安装 MySQL,Cloudera-Manages-Agent 端 可以不装。

  即,在我这里就是,Cloudera-Manages-Server 是cmbigdata1。同时,我也是喜欢把oozie、hive等这些,都是安装在cmbigdata1这台机器上。

  Cloudera-Manages-Agent,在我这里就是,cmbigdata2、cmbigdata3和cmbigdata4。

  参考

60分钟内从零起步驾驭Hive实战学习笔记(Ubuntu里安装mysql)

Ubuntu 14.04 MySQL 安装配置 For Cloudera Manages 5

root@ubuntucmbigdata1:~# apt-get install mysql-server

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
。。。
。。。
Setting up libdbd-mysql-perl (4.025-1ubuntu0.1) ...
Setting up libterm-readkey-perl (2.31-1) ...
Setting up mysql-client-5.5 (5.5.55-0ubuntu0.14.04.1) ...
Setting up mysql-server-5.5 (5.5.55-0ubuntu0.14.04.1) ...
170625 16:28:33 [Warning] Using unique option prefix key_buffer instead of key_buffer_size is deprecated and will be removed in a future release. Please use the full name instead.
170625 16:28:33 [Note] Ignoring --secure-file-priv value as server is running with --bootstrap.
170625 16:28:33 [Note] /usr/sbin/mysqld (mysqld 5.5.55-0ubuntu0.14.04.1) starting as process 7930 ...
mysql start/running, process 8062
Setting up libhtml-template-perl (2.95-1) ...
Processing triggers for ureadahead (0.100.0-16) ...
Setting up mysql-server (5.5.55-0ubuntu0.14.04.1) ...
Processing triggers for libc-bin (2.19-0ubuntu6.7) ...
root@ubuntucmbigdata1:~#
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

安装成功!

  启动mysql服务

安装MySql成功后,mysql服务会自动启动

root@ubuntucmbigdata1:~# service mysql start

或者
root@ubuntucmbigdata1:~# sudo /etc/init.d/mysql start

  安装mysql JDBC驱动

root@ubuntucmbigdata1:~# apt-get install libmysql-java

设置MySQL的root用户设置密码,这一步,就不需要做了,在CentOS版本里,需要做。在Ubuntu版本,在安装时,就已经设置了。

  跳过,就是

创建Cloudera-Manages-Server 需要用到的数据库表 for MySql。

  Creating Databases for Activity Monitor, Reports Manager, Hive Metastore Server, Sentry Server, Cloudera Navigator AuditServer, and Cloudera Navigator Metadata Server

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
root@ubuntucmbigdata1:~# mysql -root -p
Enter password: (密码为rootroot)
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 42
Server version: 5.5.55-0ubuntu0.14.04.1 (Ubuntu) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql>
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

  执行创建数据库脚本语句。

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)
mysql> create database hive DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on hive.* TO 'hive'@'localhost' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> grant all on *.* TO 'hive'@'ubuntucmbigdata1' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> grant all on hive.* TO 'hive'@'%' IDENTIFIED BY 'hive_password';
Ignoring query to other database
mysql> create database oozie DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on oozie.* TO 'oozie'@'localhost' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> grant all on *.* TO 'oozie'@'ubuntucmbigdata1' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> grant all on oozie.* TO 'oozie'@'%' IDENTIFIED BY 'oozie_password';
Ignoring query to other database
mysql> create database amon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on amon.* TO 'amon'@'localhost' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'amon'@'ubuntucmbigdata1' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> grant all on amon.* TO 'amon'@'%' IDENTIFIED BY 'amon_password';
Ignoring query to other database
mysql> create database smon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on smon.* TO 'smon'@'localhost' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'smon'@'ubuntucmbigdata1' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> grant all on smon.* TO 'smon'@'%' IDENTIFIED BY 'smon_password';
Ignoring query to other database
mysql> create database rman DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on rman.* TO 'rman'@'localhost' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> grant all on *.* TO 'rman'@'ubuntucmbigdata1' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> grant all on rman.* TO 'rman'@'%' IDENTIFIED BY 'rman_password';
Ignoring query to other database
mysql> create database hmon DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on hmon.* TO 'hmon'@'localhost' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> grant all on *.* TO 'hmon'@'ubuntucmbigdata1' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> grant all on hmon.* TO 'hmon'@'%' IDENTIFIED BY 'hmon_password';
Ignoring query to other database
mysql> create database nav DEFAULT CHARACTER SET utf8;
Ignoring query to other database
mysql> grant all on nav.* TO 'nav'@'localhost' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql> grant all on *.* TO 'nav'@'ubuntucmbigdata1' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql> grant all on nav.* TO 'nav'@'%' IDENTIFIED BY 'nav_password';
Ignoring query to other database
mysql>
Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

mysql> flush privileges;
Query OK, rows affected (0.08 sec) mysql> select user,host,password from mysql.user;
+------------------+------------------+-------------------------------------------+
| user | host | password |
+------------------+------------------+-------------------------------------------+
| root | localhost | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root | ubuntucmbigdata1 | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root | 127.0.0.1 | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root | :: | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| debian-sys-maint | localhost | *0C134AA3949C6DEE1804B1D1D641A7C2824C599A |
| scm | % | *E2D6A4ADCCA7B38098E85EAF9BB785AB21451139 |
| scm | ubuntucmbigdata1 | *E2D6A4ADCCA7B38098E85EAF9BB785AB21451139 |
| scm | localhost | *E2D6A4ADCCA7B38098E85EAF9BB785AB21451139 |
| hive | localhost | *8AC2E431CC7A9F2C4C0E51A65B8D8175892D9F22 |
| hive | ubuntucmbigdata1 | *8AC2E431CC7A9F2C4C0E51A65B8D8175892D9F22 |
| hive | % | *8AC2E431CC7A9F2C4C0E51A65B8D8175892D9F22 |
| oozie | localhost | *81A1BB46F79EBD0AA76E6EFAA31D62458CFCAF62 |
| oozie | ubuntucmbigdata1 | *81A1BB46F79EBD0AA76E6EFAA31D62458CFCAF62 |
| oozie | % | *81A1BB46F79EBD0AA76E6EFAA31D62458CFCAF62 |
| amon | localhost | *4A1243A12C2625553EC0465DD4E640DB056AA07A |
| amon | ubuntucmbigdata1 | *4A1243A12C2625553EC0465DD4E640DB056AA07A |
| amon | % | *4A1243A12C2625553EC0465DD4E640DB056AA07A |
| smon | localhost | *F65C49868BBCA6D143F5AF81D1864B1C64DF25B4 |
| smon | ubuntucmbigdata1 | *F65C49868BBCA6D143F5AF81D1864B1C64DF25B4 |
| smon | % | *F65C49868BBCA6D143F5AF81D1864B1C64DF25B4 |
| rman | localhost | *AEF345BFE495D8E678EA9D3D5708FD110AD2F08E |
| rman | ubuntucmbigdata1 | *AEF345BFE495D8E678EA9D3D5708FD110AD2F08E |
| rman | % | *AEF345BFE495D8E678EA9D3D5708FD110AD2F08E |
| hmon | localhost | *DC736DB1383C5AC16BFE895C856F8D30214AF19A |
| hmon | ubuntucmbigdata1 | *DC736DB1383C5AC16BFE895C856F8D30214AF19A |
| hmon | % | *DC736DB1383C5AC16BFE895C856F8D30214AF19A |
| nav | localhost | *993A16D875A9AB1932CC2F57E7DADA76AD35C3B2 |
| nav | ubuntucmbigdata1 | *993A16D875A9AB1932CC2F57E7DADA76AD35C3B2 |
| nav | % | *993A16D875A9AB1932CC2F57E7DADA76AD35C3B2 |
+------------------+------------------+-------------------------------------------+
rows in set (0.00 sec) mysql> exit;
欢迎大家,加入我的微信公众号:大数据躺过的坑     免费给分享
 

同时,大家可以关注我的个人博客

 http://www.cnblogs.com/zlslch/   和  http://www.cnblogs.com/lchzls/ 

  人生苦短,我愿分享。本公众号将秉持活到老学到老学习无休止的交流分享开源精神,汇聚于互联网和个人学习工作的精华干货知识,一切来于互联网,反馈回互联网。
  目前研究领域:大数据、机器学习、深度学习、人工智能、数据挖掘、数据分析。 语言涉及:Java、Scala、Python、Shell、Linux等 。同时还涉及平常所使用的手机、电脑和互联网上的使用技巧、问题和实用软件。 只要你一直关注和呆在群里,每天必须有收获

以及对应本平台的QQ群:161156071(大数据躺过的坑)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)

Cloudera Manager安装之Cloudera Manager安装前准备(Ubuntu14.04)(一)