概念了解
Ambair介绍
Apache Ambari是一个用于支持大数据软件供应 管理与监控软件.它也是一个分布式软件,分为Ambair-Server与Ambari-Client两个部分.在生产环境下一般单独用一台服务器安装ambari-server来确保服务的稳定性.然后需要安装大数据服务组件的服务器上均安装ambari-client组件,server发送命令与client进行交互完成任务.
HDP介绍
HDP是hortonworks公司的Hadoop发行版,其中包括大多数Hadoop生态下的重要组件,可以作为Ambari进行部署与安装.从官网上可以查看HDP中版本与其他组件的版本对应关系:https://zh.hortonworks.com/products/data-platforms/hdp/
准备工作
硬件准备
这里使用的是阿里云中Ubuntu16.06版本服务器,预装有Python2.7环境,机器三台,配置为一台2H8G与两台2H4G服务器作为测试搭建环境.系统准备好后防火墙及端口是全开放的,直接启用的root用户.若环境与之不符作为测试环境可以先匹配一下.选用配置最好的作为master,剩下的两台作为两个slave节点.
准备安装包
这边因为是要测试Spark的一些新特性,所以所准备的安装包均为当前时间点最新的,直接从hortonworks官网进行下载;
其中要安装的组件有JDK-1.8 确认一下Python版本为2.7 然后下载Ambari2.7.3对应的组件,其中包括:ambari-2.7.3.0 HDP HDP-UTILS HDP-GPL
首先是Ambari安装包位置:
选择下载最新版,这样子在版本控制方面选择余地更大
主页:
https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/ambari_repositories.html
https://docs.hortonworks.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/hdp_31_repositories.html
下载链接:
ambari-2.7.3.0 http://public-repo-1.hortonworks.com/ambari/ubuntu16/2.x/updates/2.7.3.0/ambari-2.7.3.0-ubuntu16.tar.gz
HDP http://public-repo-1.hortonworks.com/HDP/ubuntu16/3.x/updates/3.1.0.0/HDP-3.1.0.0-ubuntu16-deb.tar.gz
HDP-UTILS http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu16/HDP-UTILS-1.1.0.22-ubuntu16.tar.gz
HDP-GPL http://public-repo-1.hortonworks.com/HDP-GPL/ubuntu16/3.x/updates/3.1.0.0/HDP-GPL-3.1.0.0-ubuntu16-gpl.tar.gz
查看配置
首先查看机器硬件配置信息,包括内存及CPU配置信息,根据配置分配不同的角色:
cat /proc/meminfo |grep MemTotal
cat /proc/cpuinfo |grep “model name” && cat /proc/cpuinfo |grep “physical id”
主机命名 master.example.com node1.example.com node2.example.com(生产环境下建议使用域名服务器进行解析)
三步操作:1 )hostname 主机名 2)修改/etc/hosts 文件对应DNS解析 3)修改/etc/hostname文件 最后使用hostname hostname -A对结果进行检测,要求与设置值一致
修改/etc/hosts 添加对应的域名配置解析,将主机名与IP地址相对应(注意IP地址与DNS要做到一一对应,会存在反解析操作,否则有坑)
ssh免密码登录 ssh-keygen -t rsa 然后id_rsa.pub 导入到authorized_keys文件中 scp将文件发送到远程服务器对应目录上
配置环境
配置所有机器的JDK环境 首先是安装Ambari Server服务器,这边因为墙的原因需要自己配置本地源 : mkdir -p /var/www/html 将上述安装包解压到该目录下,通过Python服务器在该目录启动访问链接:( 以静默的方式启动Python服务器 nohup python -m SimpleHTTPServer 1>out.log 2>&1 &)
mkdir -p /var/www/html
ls /var/www/html
ambari HDP HDP-GPL HDP-UTILS
将安装包解压后放入上述目录
配置Ubuntu启动源
指定仓库地址:
https://www.jianshu.com/p/e33e6d9a0655
/etc/apt/sources.list.d cat /etc/apt/sources.lost.d/ambari.list
deb http://10.1.51.17:8000/ambari/ubuntu16/2.7.3.0-139/ Ambari main
etc/apt/sources.list.d/ambari-hdp.list
deb http://10.1.51.17:8000/HDP/ubuntu16/3.1.0.0-78/ HDP main
deb http://10.1.51.17:8000/HDP-GPL/ubuntu16/3.1.0.0-78/ HDP-GPL main
deb http://10.1.51.17:8000/HDP-UTILS/ubuntu16/1.1.0.22/ HDP-UTILS main apt-key adv --recv-keys --keyserver keyserver.ubuntu.com B9733A7A07513CAD
apt-get update
apt-cache showpkg ambari-server
最后在apt-server服务器上执行:
apt-get install ambari-server
可以顺利安装ambari-server;
之后执行ambari-server setup对其进行安装配置操作,包括设置jdk与存储数据库等配置
最后ambari-server start启动,启动后可以通过该机器的8080端口访问Ambari页面
当打开以下页面后,Ambari的安装流程就告一段落,可以先打一个镜像了;
当打开以上界面后证明Ambari Server已经可以成功启动了,后边现在需要利用它进行Ambari agent配置工作,同时包括大数据的一些服务组件配置:
首先在以上界面对该集群进行命名;
选择对应的HDP版本,同时配置一些源服务.这个地方如果不需要修改源码可以初略处理,后边我想要更换HDP源还需要进行一些替换操作;
填好需要安装Ambari agent服务器的hostname,同时下边贴上ambari server服务器的id_rsa文件,注意没有.pub后缀,否则在下一步安装的时候会提示ssh连接失败的错误.(如下图所示)
填好之后下一步就会进行Ambari agent安装工作了.
安装完成后选择自己所需要的服务组件,进行安装,注意要将服务分配均匀 同时中间如果有出错需要及时解决错误;
出坑1:
Ubuntu16.04安装Ambari2.6.2.0配置HDP2.5.5.0-292遇见大坑,安装Ambari Client有个坎一直过不去.
[code=csharp]
==========================
Creating target directory...
==========================
Command start time 2019-01-04 12:38:12
chmod: cannot access '/var/lib/ambari-agent/data': No such file or directory
Warning: Permanently added 'master.example.com,10.10.80.131' (ECDSA) to the list of known hosts.
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:12
==========================
Copying ambari sudo script...
==========================
Command start time 2019-01-04 12:38:12
scp /var/lib/ambari-server/ambari-sudo.sh
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying common functions script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_commons
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:13
scp /var/lib/ambari-server/create-python-wrap.sh
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying OS type check script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_server/os_check_type.py
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Running create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:13
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Running OS type check...
==========================
Command start time 2019-01-04 12:38:14
Cluster primary/cluster OS family is ubuntu16 and local/current OS family is ubuntu16
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Checking 'sudo' package on remote host...
==========================
Command start time 2019-01-04 12:38:14
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Copying repo file to 'tmp' folder...
==========================
Command start time 2019-01-04 12:38:14
scp /etc/apt/sources.list.d/ambari.list
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Moving file to repo dir...
==========================
Command start time 2019-01-04 12:38:15
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Changing permissions for ambari.repo...
==========================
Command start time 2019-01-04 12:38:15
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Update apt cache of repository...
==========================
Command start time 2019-01-04 12:38:15
0% [Working]
Get:1 http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155 Ambari InRelease [7,394 B]
0% [1 InRelease 7,394 B/7,394 B 100%]
0% [Working]
Get:2 http://10.10.80.131:8000/HDP/ubuntu16/2.6.5.0-292 HDP InRelease [7,376 B]
Get:3 http://10.10.80.131:8000/HDP-GPL/ubuntu16/2.6.5.0-292 HDP-GPL InRelease [7,395 B]
Get:4 http://10.10.80.131:8000/HDP-UTILS/ubuntu16/1.1.0.22 HDP-UTILS InRelease [7,405 B]
0% [Working]
0% [1 InRelease gpgv 7,394 B]
0% [Working]
0% [2 InRelease gpgv 7,376 B]
0% [Working]
0% [3 InRelease gpgv 7,395 B]
0% [Working]
0% [4 InRelease gpgv 7,405 B]
100% [Working]
Fetched 29.6 kB in 0s (148 kB/s)
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 8%
Reading package lists... 19%
Reading package lists... 19%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... Done
W: http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155/dists/Ambari/InRelease: Signature by key DF52ED4F7A3A5882C0994C66B9733A7A07513CAD uses weak digest algorithm (SHA1)
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Copying setup script file...
==========================
Command start time 2019-01-04 12:38:16
scp /usr/lib/ambari-server/lib/ambari_server/setupAgent.py
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Running setup agent script...
==========================
Command start time 2019-01-04 12:38:16
dpkg-query: no packages found matching ambari-agent
('WARNING 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7f0e12cad490>; currently running: False
INFO 2019-01-04 12:38:30,685 hostname.py:106 - Read public hostname \'master.example.com\' using socket.getfqdn()
INFO 2019-01-04 12:38:30,686 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:30,737 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:30,748 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'master.example.com\' using socket.getfqdn().
INFO 2019-01-04 12:38:30,750 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources.
INFO 2019-01-04 12:38:30,752 Hardware.py:73 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'example.com\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.4.0-117-generic\', \'uptime_days\': \'0\', \'memorytotal\': 8174812, \'swapfree\': \'0.00 GB\', \'memorysize\': 8174812, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': \'255.255.255.0\', \'timezone\': \'CST\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 241164, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.4\', \'kernelversion\': \'4.4.0\', \'macaddress\': \'00:16:3E:10:7E:5C\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'10.10.80.131\', \'hostname\': \'master\', \'uptime_hours\': \'1\', \'fqdn\': \'master.example.com\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'4068960\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'4068960\'}, {\'available\': \'24108320\', \'used\': \'14930060\', \'percent\': \'39%\', \'device\': \'/dev/vda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'41151808\'}, {\'available\': \'88747784\', \'used\': \'9072256\', \'percent\': \'10%\', \'device\': \'/dev/vdb1\', \'mountpoint\': \'/mnt/disk1\', \'type\': \'ext4\', \'size\': \'103079200\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'7017\', \'interfaces\': \'eth0,lo\'}
INFO 2019-01-04 12:38:30,953 Controller.py:170 - Registering with master.example.com (10.10.80.131) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "example.com", "physicalprocessorcount": 4, "kernelrelease": "4.4.0-117-generic", "uptime_days": "0", "memorytotal": 8174812, "swapfree": "0.00 GB", "memorysize": 8174812, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": "255.255.255.0", "timezone": "CST", "hardwareisa": "x86_64", "memoryfree": 241164, "operatingsystem": "ubuntu", "kernelmajversion": "4.4", "kernelversion": "4.4.0", "macaddress": "00:16:3E:10:7E:5C", "operatingsystemrelease": "16.04", "ipaddress": "10.10.80.131", "hostname": "master", "uptime_hours": "1", "fqdn": "master.example.com", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "4068960", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "4068960"}, {"available": "24108320", "used": "14930060", "percent": "39%", "device": "/dev/vda1", "mountpoint": "/", "type": "ext4", "size": "41151808"}, {"available": "88747784", "used": "9072256", "percent": "10%", "device": "/dev/vdb1", "mountpoint": "/mnt/disk1", "type": "ext4", "size": "103079200"}], "hardwaremodel": "x86_64", "uptime_seconds": "7017", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.6.2.0", "agentEnv": {"transparentHugePage": "madvise", "hostHealth": {"agentTimeStampAtReporting": 1546576710951, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1546576710753, "hostname": "master.example.com", "responseId": -1, "publicHostname": "master.example.com"}\')
INFO 2019-01-04 12:38:30,953 NetUtil.py:70 - Connecting to https://master.example.com:8440/connection_info
INFO 2019-01-04 12:38:30,981 security.py:93 - SSL Connect being called.. connecting to the server
ERROR 2019-01-04 12:38:30,984 Controller.py:226 - Unable to connect to: https://master.example.com:8441/agent/v1/register/master.example.com
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 175, in registerWithServer
ret = self.sendRequest(self.registerUrl, data)
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 549, in sendRequest
raise IOError(\'Request to {0} failed due to {1}\'.format(url, str(exception)))
IOError: Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
ERROR 2019-01-04 12:38:30,984 Controller.py:227 - Error:Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
WARNING 2019-01-04 12:38:30,984 Controller.py:228 - Sleeping for 27 seconds and then trying again
', None)
('WARNING 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:28,678 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7f0e12cad490>; currently running: False
INFO 2019-01-04 12:38:30,685 hostname.py:106 - Read public hostname \'master.example.com\' using socket.getfqdn()
INFO 2019-01-04 12:38:30,686 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:30,737 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:30,748 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname \'master.example.com\' using socket.getfqdn().
INFO 2019-01-04 12:38:30,750 Facter.py:202 - Directory: \'/etc/resource_overrides\' does not exist - it won\'t be used for gathering system resources.
INFO 2019-01-04 12:38:30,752 Hardware.py:73 - Host system information: {\'kernel\': \'Linux\', \'domain\': \'example.com\', \'physicalprocessorcount\': 4, \'kernelrelease\': \'4.4.0-117-generic\', \'uptime_days\': \'0\', \'memorytotal\': 8174812, \'swapfree\': \'0.00 GB\', \'memorysize\': 8174812, \'osfamily\': \'ubuntu\', \'swapsize\': \'0.00 GB\', \'processorcount\': 4, \'netmask\': \'255.255.255.0\', \'timezone\': \'CST\', \'hardwareisa\': \'x86_64\', \'memoryfree\': 241164, \'operatingsystem\': \'ubuntu\', \'kernelmajversion\': \'4.4\', \'kernelversion\': \'4.4.0\', \'macaddress\': \'00:16:3E:10:7E:5C\', \'operatingsystemrelease\': \'16.04\', \'ipaddress\': \'10.10.80.131\', \'hostname\': \'master\', \'uptime_hours\': \'1\', \'fqdn\': \'master.example.com\', \'id\': \'root\', \'architecture\': \'x86_64\', \'selinux\': False, \'mounts\': [{\'available\': \'4068960\', \'used\': \'0\', \'percent\': \'0%\', \'device\': \'udev\', \'mountpoint\': \'/dev\', \'type\': \'devtmpfs\', \'size\': \'4068960\'}, {\'available\': \'24108320\', \'used\': \'14930060\', \'percent\': \'39%\', \'device\': \'/dev/vda1\', \'mountpoint\': \'/\', \'type\': \'ext4\', \'size\': \'41151808\'}, {\'available\': \'88747784\', \'used\': \'9072256\', \'percent\': \'10%\', \'device\': \'/dev/vdb1\', \'mountpoint\': \'/mnt/disk1\', \'type\': \'ext4\', \'size\': \'103079200\'}], \'hardwaremodel\': \'x86_64\', \'uptime_seconds\': \'7017\', \'interfaces\': \'eth0,lo\'}
INFO 2019-01-04 12:38:30,953 Controller.py:170 - Registering with master.example.com (10.10.80.131) (agent=\'{"hardwareProfile": {"kernel": "Linux", "domain": "example.com", "physicalprocessorcount": 4, "kernelrelease": "4.4.0-117-generic", "uptime_days": "0", "memorytotal": 8174812, "swapfree": "0.00 GB", "memorysize": 8174812, "osfamily": "ubuntu", "swapsize": "0.00 GB", "processorcount": 4, "netmask": "255.255.255.0", "timezone": "CST", "hardwareisa": "x86_64", "memoryfree": 241164, "operatingsystem": "ubuntu", "kernelmajversion": "4.4", "kernelversion": "4.4.0", "macaddress": "00:16:3E:10:7E:5C", "operatingsystemrelease": "16.04", "ipaddress": "10.10.80.131", "hostname": "master", "uptime_hours": "1", "fqdn": "master.example.com", "id": "root", "architecture": "x86_64", "selinux": false, "mounts": [{"available": "4068960", "used": "0", "percent": "0%", "device": "udev", "mountpoint": "/dev", "type": "devtmpfs", "size": "4068960"}, {"available": "24108320", "used": "14930060", "percent": "39%", "device": "/dev/vda1", "mountpoint": "/", "type": "ext4", "size": "41151808"}, {"available": "88747784", "used": "9072256", "percent": "10%", "device": "/dev/vdb1", "mountpoint": "/mnt/disk1", "type": "ext4", "size": "103079200"}], "hardwaremodel": "x86_64", "uptime_seconds": "7017", "interfaces": "eth0,lo"}, "currentPingPort": 8670, "prefix": "/var/lib/ambari-agent/data", "agentVersion": "2.6.2.0", "agentEnv": {"transparentHugePage": "madvise", "hostHealth": {"agentTimeStampAtReporting": 1546576710951, "activeJavaProcs": [], "liveServices": [{"status": "Healthy", "name": "ntp or chrony", "desc": ""}]}, "reverseLookup": true, "alternatives": [], "hasUnlimitedJcePolicy": null, "umask": "18", "firewallName": "ufw", "stackFoldersAndFiles": [], "existingUsers": [], "firewallRunning": false}, "timestamp": 1546576710753, "hostname": "master.example.com", "responseId": -1, "publicHostname": "master.example.com"}\')
INFO 2019-01-04 12:38:30,953 NetUtil.py:70 - Connecting to https://master.example.com:8440/connection_info
INFO 2019-01-04 12:38:30,981 security.py:93 - SSL Connect being called.. connecting to the server
ERROR 2019-01-04 12:38:30,984 Controller.py:226 - Unable to connect to: https://master.example.com:8441/agent/v1/register/master.example.com
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 175, in registerWithServer
ret = self.sendRequest(self.registerUrl, data)
File "/usr/lib/ambari-agent/lib/ambari_agent/Controller.py", line 549, in sendRequest
raise IOError(\'Request to {0} failed due to {1}\'.format(url, str(exception)))
IOError: Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
ERROR 2019-01-04 12:38:30,984 Controller.py:227 - Error:Request to https://master.example.com:8441/agent/v1/register/master.example.com failed due to EOF occurred in violation of protocol (_ssl.c:590)
WARNING 2019-01-04 12:38:30,984 Controller.py:228 - Sleeping for 27 seconds and then trying again
', None)
Connection to master.example.com closed.
SSH command execution finished
host=master.example.com, exitcode=0
Command end time 2019-01-04 12:38:31
Registering with the server...
Registration with the server failed.
==========================
Creating target directory...
==========================
Command start time 2019-01-04 12:38:12
chmod: cannot access '/var/lib/ambari-agent/data': No such file or directory
Warning: Permanently added 'node1.example.com,10.10.80.130' (ECDSA) to the list of known hosts.
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:12
==========================
Copying ambari sudo script...
==========================
Command start time 2019-01-04 12:38:12
scp /var/lib/ambari-server/ambari-sudo.sh
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying common functions script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_commons
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:13
scp /var/lib/ambari-server/create-python-wrap.sh
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:13
==========================
Copying OS type check script...
==========================
Command start time 2019-01-04 12:38:13
scp /usr/lib/ambari-server/lib/ambari_server/os_check_type.py
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Running create-python-wrap script...
==========================
Command start time 2019-01-04 12:38:14
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Running OS type check...
==========================
Command start time 2019-01-04 12:38:14
Cluster primary/cluster OS family is ubuntu16 and local/current OS family is ubuntu16
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Checking 'sudo' package on remote host...
==========================
Command start time 2019-01-04 12:38:14
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:14
==========================
Copying repo file to 'tmp' folder...
==========================
Command start time 2019-01-04 12:38:14
scp /etc/apt/sources.list.d/ambari.list
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Moving file to repo dir...
==========================
Command start time 2019-01-04 12:38:15
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Changing permissions for ambari.repo...
==========================
Command start time 2019-01-04 12:38:15
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:15
==========================
Update apt cache of repository...
==========================
Command start time 2019-01-04 12:38:15
0% [Working]
Get:1 http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155 Ambari InRelease [7,394 B]
0% [1 InRelease 7,394 B/7,394 B 100%]
0% [Working]
Get:2 http://10.10.80.131:8000/HDP/ubuntu16/2.6.5.0-292 HDP InRelease [7,376 B]
Get:3 http://10.10.80.131:8000/HDP-GPL/ubuntu16/2.6.5.0-292 HDP-GPL InRelease [7,395 B]
Get:4 http://10.10.80.131:8000/HDP-UTILS/ubuntu16/1.1.0.22 HDP-UTILS InRelease [7,405 B]
0% [Working]
0% [1 InRelease gpgv 7,394 B]
0% [Working]
0% [2 InRelease gpgv 7,376 B]
0% [Working]
0% [3 InRelease gpgv 7,395 B]
0% [Working]
0% [4 InRelease gpgv 7,405 B]
100% [Working]
Fetched 29.6 kB in 0s (158 kB/s)
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 0%
Reading package lists... 8%
Reading package lists... 19%
Reading package lists... 19%
Reading package lists... 38%
Reading package lists... 38%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... 39%
Reading package lists... Done
W: http://10.10.80.131:8000/ambari/ubuntu16/2.6.2.0-155/dists/Ambari/InRelease: Signature by key DF52ED4F7A3A5882C0994C66B9733A7A07513CAD uses weak digest algorithm (SHA1)
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Copying setup script file...
==========================
Command start time 2019-01-04 12:38:16
scp /usr/lib/ambari-server/lib/ambari_server/setupAgent.py
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:16
==========================
Running setup agent script...
==========================
Command start time 2019-01-04 12:38:16
dpkg-query: no packages found matching ambari-agent
("INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,808 DataCleaner.py:39 - Data cleanup thread started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:120 - Data cleanup started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:122 - Data cleanup finished
INFO 2019-01-04 12:38:25,809 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:25,834 PingPortListener.py:50 - Ping port listener started on port: 8670
INFO 2019-01-04 12:38:25,834 main.py:439 - Connecting to Ambari server at https://master.example.com:8440 (10.10.80.131)
INFO 2019-01-04 12:38:25,834 NetUtil.py:70 - Connecting to https://master.example.com:8440/ca
INFO 2019-01-04 12:38:26,256 main.py:449 - Connected to Ambari server master.example.com
INFO 2019-01-04 12:38:26,257 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads
WARNING 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7ff4bfa91490>; currently running: False
INFO 2019-01-04 12:38:28,265 hostname.py:106 - Read public hostname 'node1.example.com' using socket.getfqdn()
INFO 2019-01-04 12:38:28,266 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:28,310 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:28,319 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:28,321 Facter.py:202 - Directory: '/etc/resource_overrides' does not exist - it won't be used for gathering system resources.
INFO 2019-01-04 12:38:28,323 Hardware.py:73 - Host system information: {'kernel': 'Linux', 'domain': 'example.com', 'physicalprocessorcount': 4, 'kernelrelease': '4.4.0-117-generic', 'uptime_days': '0', 'memorytotal': 4046044, 'swapfree': '0.00 GB', 'memorysize': 4046044, 'osfamily': 'ubuntu', 'swapsize': '0.00 GB', 'processorcount': 4, 'netmask': '255.255.255.0', 'timezone': 'CST', 'hardwareisa': 'x86_64', 'memoryfree': 1011008, 'operatingsystem': 'ubuntu', 'kernelmajversion': '4.4', 'kernelversion': '4.4.0', 'macaddress': '00:16:3E:0E:F0:53', 'operatingsystemrelease': '16.04', 'ipaddress': '10.10.80.130', 'hostname': 'node1', 'uptime_hours': '1', 'fqdn': 'node1.example.com', 'id': 'root', 'architecture': 'x86_64', 'selinux': False, 'mounts': [{'available': '2004576', 'used': '0', 'percent': '0%', 'device': 'udev', 'mountpoint': '/dev', 'type': 'devtmpfs', 'size': '2004576'}, {'available': '35189292', 'used': '3849088', 'percent': '10%', 'device': '/dev/vda1', 'mountpoint': '/', 'type': 'ext4', 'size': '41151808'}, {'available': '97758996', 'used': '61044', 'percent': '1%', 'device': '/dev/vdb1', 'mountpoint': '/mnt/disk1', 'type': 'ext4', 'size': '103079200'}], 'hardwaremodel': 'x86_64', 'uptime_seconds': '7180', 'interfaces': 'eth0,lo'}
", None)
("INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,807 main.py:147 - loglevel=logging.INFO
INFO 2019-01-04 12:38:25,808 DataCleaner.py:39 - Data cleanup thread started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:120 - Data cleanup started
INFO 2019-01-04 12:38:25,809 DataCleaner.py:122 - Data cleanup finished
INFO 2019-01-04 12:38:25,809 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:25,834 PingPortListener.py:50 - Ping port listener started on port: 8670
INFO 2019-01-04 12:38:25,834 main.py:439 - Connecting to Ambari server at https://master.example.com:8440 (10.10.80.131)
INFO 2019-01-04 12:38:25,834 NetUtil.py:70 - Connecting to https://master.example.com:8440/ca
INFO 2019-01-04 12:38:26,256 main.py:449 - Connected to Ambari server master.example.com
INFO 2019-01-04 12:38:26,257 threadpool.py:58 - Started thread pool with 3 core threads and 20 maximum threads
WARNING 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:280 - [AlertScheduler] /var/lib/ambari-agent/cache/alerts/definitions.json not found or invalid. No alerts will be scheduled until registration occurs.
INFO 2019-01-04 12:38:26,257 AlertSchedulerHandler.py:175 - [AlertScheduler] Starting <ambari_agent.apscheduler.scheduler.Scheduler object at 0x7ff4bfa91490>; currently running: False
INFO 2019-01-04 12:38:28,265 hostname.py:106 - Read public hostname 'node1.example.com' using socket.getfqdn()
INFO 2019-01-04 12:38:28,266 Hardware.py:68 - Initializing host system information.
INFO 2019-01-04 12:38:28,310 Hardware.py:188 - Some mount points were ignored: /run, /dev/shm, /run/lock, /sys/fs/cgroup, /run/user/0
INFO 2019-01-04 12:38:28,319 hostname.py:67 - agent:hostname_script configuration not defined thus read hostname 'node1.example.com' using socket.getfqdn().
INFO 2019-01-04 12:38:28,321 Facter.py:202 - Directory: '/etc/resource_overrides' does not exist - it won't be used for gathering system resources.
INFO 2019-01-04 12:38:28,323 Hardware.py:73 - Host system information: {'kernel': 'Linux', 'domain': 'example.com', 'physicalprocessorcount': 4, 'kernelrelease': '4.4.0-117-generic', 'uptime_days': '0', 'memorytotal': 4046044, 'swapfree': '0.00 GB', 'memorysize': 4046044, 'osfamily': 'ubuntu', 'swapsize': '0.00 GB', 'processorcount': 4, 'netmask': '255.255.255.0', 'timezone': 'CST', 'hardwareisa': 'x86_64', 'memoryfree': 1011008, 'operatingsystem': 'ubuntu', 'kernelmajversion': '4.4', 'kernelversion': '4.4.0', 'macaddress': '00:16:3E:0E:F0:53', 'operatingsystemrelease': '16.04', 'ipaddress': '10.10.80.130', 'hostname': 'node1', 'uptime_hours': '1', 'fqdn': 'node1.example.com', 'id': 'root', 'architecture': 'x86_64', 'selinux': False, 'mounts': [{'available': '2004576', 'used': '0', 'percent': '0%', 'device': 'udev', 'mountpoint': '/dev', 'type': 'devtmpfs', 'size': '2004576'}, {'available': '35189292', 'used': '3849088', 'percent': '10%', 'device': '/dev/vda1', 'mountpoint': '/', 'type': 'ext4', 'size': '41151808'}, {'available': '97758996', 'used': '61044', 'percent': '1%', 'device': '/dev/vdb1', 'mountpoint': '/mnt/disk1', 'type': 'ext4', 'size': '103079200'}], 'hardwaremodel': 'x86_64', 'uptime_seconds': '7180', 'interfaces': 'eth0,lo'}
", None)
Connection to node1.example.com closed.
SSH command execution finished
host=node1.example.com, exitcode=0
Command end time 2019-01-04 12:38:28
Registering with the server...
Registration with the server failed.
[/code]
解决方案(谷歌比百度好用)
https://community.hortonworks.com/questions/97831/registration-to-ambari-server-via-the-ambari-agent.html
/etc/ambari-agent/conf修改ambari.ini文件
We have over come the problem by adding
following option to security section in ambari-agent.ini in all the hosts in the cluster:
[security] force_https_protocol=PROTOCOL_TLSv1_2
Ubuntu16.04安装Ambari 2.7.3的更多相关文章
-
ubuntu16.04安装jdk，tomcat
ubuntu16.04安装jdk,tomcat 最近装了一下tomcat,网上的教程很多,我也试了很多次,但是有一些教程关于tomcat配置是错误的,让我走上了歧途.差点重装系统,还好王总及时出手帮助 ...
-
Ubuntu16.04 安装openjdk-7-jdk
Ubuntu16.04 安装openjdk-7-jdk sudo apt-get install openjdk-7-jre 或者sudo apt-get install openjdk-7-jdk ...
-
Ubuntu16.04安装GTK3主题:OSX-Arc
Ubuntu16.04安装GTK3主题:OSX-Arc GTK3主题:OSX-Arc描述: 前几个月,Gnome3.20升3.22的时候,出现了大量主题崩溃的现象,其中包括Arc.Flatabulou ...
-
Ubuntu16.04安装opencv for python/c++
Ubuntu16.04安装opencv for python/c++ 网上关于opencv的安装已经有了不少资料,但是没有一篇资料能让我一次性安装成功,因此花费了大量时间去解决各种意外,希望这篇能给一 ...
-
ubuntu16.04安装不上有道词典的解决办法
转自:http://www.linuxdiyf.com/linux/21143.html ubuntu16.04安装不上有道词典,提示: le@hu-pc:~/下载$ sudo dpkg -i you ...
-
Ubuntu16.04安装mongodb
Ubuntu16.04安装mongodb copy from: http://blog.csdn.net/zhushh/article/details/52451441 1.导入软件源的公钥 sudo ...
-
【Tools】ubuntu16.04安装搜狗输入法
Ubuntu16,04 安装搜狗输入法 1.下载搜狗输入法的安装包 下载地址为:http://pinyin.sogou.com/linux/ 2.按键Ctr+Alt+T打开终端,输入以下命令切换到下载 ...
-
Ubuntu16.04安装cuda9.0+cudnn7.0
Ubuntu16.04安装cuda9.0+cudnn7.0 这篇记录拖了好久,估计是去年6月份就已经安装过几遍,然后一方面因为俺比较懒,一方面后面没有经常在自己电脑上跑算法,比较少装cuda和cudn ...
-
Ubuntu16.04安装TensorFlow及Mnist训练
版权声明:本文为博主原创文章,欢迎转载,并请注明出处.联系方式:460356155@qq.com TensorFlow是Google开发的开源的深度学习框架,也是当前使用最广泛的深度学习框架. 一.安 ...
随机推荐
-
柯朗数(Courant number)研究
在数值计算过程中,对于计算结果的准确性和效率有很高的要求,但是这两者之间往往互相矛盾:而使用柯朗数可用于平衡两者. 1.柯朗数的定义: C = sqrt(gh)*t/s 其中,t是时间步长,s是网格在 ...
-
第一章 jQuery基础方法回顾
jQuery即JavaScript,它是一个.js文件(官网下载).使用时须将jQuery库的声明写在HTML文档的head标签里. 章节内容: 1.选择DOM节点 2.延迟的JavaScript的执 ...
-
Application.Exit()结束程序，但线程还在的解决方法。
出现此情况大多原因是使用了多线程编程,或者你所调用的dll使用了多线程.我们知道,一般情况下的线程执行完指定的任务之后是会关闭了的,但是如果对于一些循环类线程,或者忘记关掉的线程时,这个时候就需要我们 ...
-
Apache Shiro系列教程之三：Shiro的结构
Shiro的设计目标是简化应用的安全管理工作.软件通常是以用户为基础设计的.也就是说,我们经常是根据用户是怎样和我们的软件交互的来设计相关的用户接口.比如,你可能会说"如果是已经登录的用户与 ...
-
javascript --- 继承初探七日谈 （一）
原型链是javascript实现继承的默认方式.下面来一个例子看一下她的魔法吧: 首先定义三个构造函数: function her(){ this.name = 'Anna'; this.toStri ...
-
PE格式第四讲,数据目录表之导入表,以及IAT表
PE格式第四讲,数据目录表之导入表,以及IAT表 一丶IAT(地址表) 首先我们思考一个问题,程序加载的时候会调用API,比如我们以前写的标准PE 那么他到底是怎么去调用的? 他会Call 下边的Jm ...
-
Ribbon负载均衡策略配置
在这里吐槽一句:网上很多文章真是神坑,你不看还好,看了只会问题越来越多,就连之前的问题都没有解决!!! 不多说了,Ribbon作为后端负载均衡器,比Nginx更注重的是请求分发而不是承担并发,可以直接 ...
-
List进行排序
使用Java中提供的对集合进行操作的工具类Collections,其中的sort方法,重写比较方法 /** * 根据序号对列席人进行排序 * @param lxrUser * @return */ p ...
-
python-django rest framework框架之分页
1. 以前django做的分页组件当数据量特别大的时候,性能不是很高,有以下三种方式处理: a. 记录当前访问页的最后一条数据id,往后取多少条 b. 最多显示120页 ...
-
LoadRuner常见错误
LoadRuner常见错误 一.Step download timeout (120 seconds) 这是一个经常会遇到的问题,解决得办法走以下步骤: 1. 修改run time setting中的 ...