搭建本地Yum软件源安装Cloudera Manager 5

时间:2021-07-02 17:28:08

    记录操作过程,不过多介绍相关知识以及原理,有问题自行百度或留言讨论。

一、搭建web服务服务器

    有两台虚拟机,elephant用于安装cdh-server相关作为管理者,monkey用于安装cdh-agent相关,作为被管理者。

    我选择的是nginx,网上安装教程较多,不在赘述,把用于放置Cloudera安装文件的conf配置贴上来。关闭防火墙之类的也不多讲,注意如果是centos7,防火墙服务不是iptables了,也许如下命令会用的着。

 firewall-cmd --state systemctl stop firewalld.service
 systemctl disable firewalld.service
[root@elephant cloudera-cdh5]# cat /etc/nginx/conf.d/elephant.confserver {    listen       80;    server_name elephant www.elephant.com;    location / {        root   /data/webapps/elephant;        autoindex on;  #允许浏览文件    }    #error_page  404              /404.html;    # redirect server error pages to the static page /50x.html    #    error_page   500 502 503 504  /50x.html;    location = /50x.html {        root   /usr/share/nginx/html;    }}

通过如上的操作,我在宿主机和monkey虚拟机添加host,就可以通过http协议来浏览文件了。

[root@elephant cloudera-cdh5]# cat /etc/hosts192.168.1.109 elephant192.168.1.110 monkey[root@elephant cloudera-cdh5]# curl http://elephant
<html><head><title>Index of /</title></head><body bgcolor="white"><h1>Index of /</h1><hr><pre><a href="../">../</a><a href="cdh/">cdh/</a>                                               22-Mar-2017 15:12                   -<a href="cm/">cm/</a>                                                24-Mar-2017 13:02                   -<a href="index.html_txt">index.html_txt</a>                                     19-Mar-2017 07:15                  40</pre><hr></body></html>

二、下载CM资源包

在/data/webapps/elephant下新建目录,最终目录结构如下:

[root@elephant cm]# pwd/data/webapps/elephant/cm

浏览网页 ,找到适合你自己环境的安装包,我选择的是 cm5.10.0-centos7.tar.gz

http://archive-primary.cloudera.com/cm5/repo-as-tarball/5.10.0/

注意版本要和你的cdh一致。800多M,虚拟机中直接wget很慢,于是宿主机中打开迅雷,添加下载地址 http://archive-primary.cloudera.com/cm5/repo-as-tarball/5.10.0/cm5.10.0-centos7.tar.gz ,然后去睡觉了。

第二天晚上下班回来继续操作。

把下载的软件包复制到虚拟机的 web目录,也就是上面的cm目录,解压

tar -zxvf cm5.10.0-centos7.tar.gz

解压后的目录结构,进入到 5.10.0 目录可以看到下载的安装文件。

[root@elephant cm]# lltotal 8lrwxrwxrwx 1 1106 592    6 Jan 21 03:38 5 -> 5.10.0lrwxrwxrwx 1 1106 592    6 Jan 21 03:38 5.10 -> 5.10.0drwxrwxr-x 4 1106 592   49 Mar 24 21:06 5.10.0-rw-rw-r-- 1 1106 592 1690 Jan 21 03:35 RPM-GPG-KEY-cloudera-rw-rw-r-- 1 1106 592  347 Jan 26 05:35 cloudera-cm.repo

三、重新生成repodata

进入到/data/webapps/elephant/cm/5.10 目录,执行如下命令,重新生成repodata。下一步配置yum源的baseurl,一定是有repodata的目录。

[root@elephant 5.10]# createrepo .

如果你的虚拟机没有repodata命令,请执行下面命令安装。

sudo yum install yum-utils createrepo

四、配置 yum 源

把 上面cm目录中的 cloudera-cm.repo 复制到  /etc/yum.repos.d 目录,并进行编辑

[root@elephant yum.repos.d]# vim cloudera-cm.repo[cloudera-cm]# Packages for Cloudera's Distribution for cm, Version 5, on RedHat     or CentOS 7 x86_64name=Cloudera's Distribution for cm, Version 5#baseurl=http://repos.jenkins.cloudera.com/cm5.10.0-release/redhat/7/x86_64/cm/5/#gpgkey = http://repos.jenkins.cloudera.com/cm5.10.0-release/redhat/7/x86_64/cm/RPM-GPG-KEY-cloudera#gpgcheck = 1baseurl=http://elephant/cm/5.10#gpgkey = http://repos.jenkins.cloudera.com/cm5.10.0-release/redhat/7/x86_64/cm/RPM-GPG-KEY-clouderagpgcheck = 0

在管理机上运行 yum clean all 后,就可以如下命令安装cm了。

[root@elephant x86_64]# yum install -y cloudera-manager-serverLoaded plugins: fastestmirrorbase                                                                                                                                                          | 3.6 kB  00:00:00cloudera-cdh5                                                                                                                                                 | 2.9 kB  00:00:00cloudera-cm                                                                                                                                                   | 2.9 kB  00:00:00extras                                                                                                                                                        | 3.4 kB  00:00:00nginx                                                                                                                                                         | 2.9 kB  00:00:00updates                                                                                                                                                       | 3.4 kB  00:00:00(1/7): cloudera-cdh5/primary_db                                                                                                                               |  77 kB  00:00:00(2/7): cloudera-cm/primary_db                                                                                                                                 |  11 kB  00:00:00(3/7): base/7/x86_64/group_gz                                                                                                                                 | 155 kB  00:00:00(4/7): extras/7/x86_64/primary_db                                                                                                                             | 139 kB  00:00:00(5/7): updates/7/x86_64/primary_db                                                                                                                            | 3.8 MB  00:00:03(6/7): nginx/x86_64/primary_db                                                                                                                                |  21 kB  00:00:06(7/7): base/7/x86_64/primary_db                                                                                                                               | 5.6 MB  00:00:21Determining fastest mirrors * base: mirrors.tuna.tsinghua.edu.cn * extras: mirrors.tuna.tsinghua.edu.cn * updates: mirrors.tuna.tsinghua.edu.cnResolving Dependencies--> Running transaction check---> Package cloudera-manager-server.x86_64 0:5.10.0-1.cm5100.p0.85.el7 will be installed--> Processing Dependency: cloudera-manager-daemons = 5.10.0 for package: cloudera-manager-server-5.10.0-1.cm5100.p0.85.el7.x86_64--> Running transaction check---> Package cloudera-manager-daemons.x86_64 0:5.10.0-1.cm5100.p0.85.el7 will be installed--> Finished Dependency Resolution…………Downloading packages:(1/2): cloudera-manager-server-5.10.0-1.cm5100.p0.85.el7.x86_64.rpm                                                                                           | 8.4 kB  00:00:00(2/2): cloudera-manager-daemons-5.10.0-1.cm5100.p0.85.el7.x86_64.rpm                                                                                          | 570 MB  00:00:09-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Total                                                                                                                                                 60 MB/s | 570 MB  00:00:09Running transaction checkRunning transaction testTransaction test succeededRunning transaction  Installing : cloudera-manager-daemons-5.10.0-1.cm5100.p0.85.el7.x86_64                                                                                                         1/2  Installing : cloudera-manager-server-5.10.0-1.cm5100.p0.85.el7.x86_64                                                                                                          2/2  Verifying  : cloudera-manager-server-5.10.0-1.cm5100.p0.85.el7.x86_64                                                                                                          1/2  Verifying  : cloudera-manager-daemons-5.10.0-1.cm5100.p0.85.el7.x86_64                                                                                                         2/2Installed:  cloudera-manager-server.x86_64 0:5.10.0-1.cm5100.p0.85.el7Dependency Installed:  cloudera-manager-daemons.x86_64 0:5.10.0-1.cm5100.p0.85.el7Complete!

为了验证是从本地源下载,特意查看了nginx访问日志

127.0.0.1 - - [24/Mar/2017:22:19:55 +0800] "GET /cdh/cloudera-cdh5/repodata/repomd.xml HTTP/1.1" 200 2987 "-" "urlgrabber/3.10 yum/3.4.3" "-"127.0.0.1 - - [24/Mar/2017:22:19:55 +0800] "GET /cm/5.10/repodata/repomd.xml HTTP/1.1" 200 2980 "-" "urlgrabber/3.10 yum/3.4.3" "-"127.0.0.1 - - [24/Mar/2017:22:20:05 +0800] "GET /cdh/cloudera-cdh5/repodata/09b008ab02bc0b4f4b4279e2f0c0a5fb8ce53c5d0dcbd5258d32657a390f64fb-primary.sqlite.bz2 HTTP/1.1" 200 78773 "-" "urlgrabber/3.10 yum/3.4.3" "-"127.0.0.1 - - [24/Mar/2017:22:20:05 +0800] "GET /cm/5.10/repodata/600a57f21246adc3f5d159a4daa0ce8f74d860234b6943b7ce978682b2f6f0ad-primary.sqlite.bz2 HTTP/1.1" 200 11313 "-" "urlgrabber/3.10 yum/3.4.3" "-"127.0.0.1 - - [24/Mar/2017:22:20:30 +0800] "GET /cm/5.10/RPMS/x86_64/cloudera-manager-server-5.10.0-1.cm5100.p0.85.el7.x86_64.rpm HTTP/1.1" 200 8648 "-" "urlgrabber/3.10 yum/3.4.3" "-"127.0.0.1 - - [24/Mar/2017:22:20:37 +0800] "GET /cm/5.10/RPMS/x86_64/cloudera-manager-daemons-5.10.0-1.cm5100.p0.85.el7.x86_64.rpm HTTP/1.1" 200 597309256 "-" "urlgrabber/3.10 yum/3.4.3" "-"

搭建本地Yum软件源安装Cloudera Manager 5


到此,yum本地源方式安装Cloudera Manager 5已经完成。


补充:

关于如何以类似的方式安装CDH,可以参考官网的博文,如果有问题可以留言。https://www.cloudera.com/documentation/enterprise/5-9-x/topics/cdh_ig_yumrepo_local_create.html
这个命令会下载cdh的全部包,想到耗时间,做好心理准备reposync -r cloudera-cdh5


本文出自 “韩宇斌-大道源于生活” 博客,请务必保留此出处http://byteh.blog.51cto.com/141786/1910197