Ubuntu18.04.6 server 安装 HDP3.1.0

标签（空格分隔）：大数据运维专栏

1.1 系统介绍

系统：
  Ubuntu18.04.6 server

主机名：
   cat /etc/hosts

172.16.10.61    flyfish61
172.16.10.62    flyfish62
172.16.10.63    flyfish63
172.16.10.64    flyfish64
172.16.10.65    flyfish65

本次安装前面3台机器： flyfish61/flyfish62/flyfish63

1.2 系统初始化

1. 关闭防火墙：
  service ufw stop
  update-rc.d ufw disable

2. 时间服务器：

   timedatectl set-timezone Asia/Shanghai
 
  apt-get install chrony -y

  vim /etc/chrony/chrony.conf

 ---
server ntp1.aliyun.com iburst


 ---

 service chrony stop 
 service chrony start 
 update-rc.d chrony defaults
 chronyc sources -v

修改系统句柄
vim /etc/security/limits.conf
---
*               soft    nofile          65535
*               hard    nofile          1029345
*               soft    nproc           unlimited
*               hard    nproc           unlimited
*               soft    memlock         unlimited
*               hard    memlock         unlimited

---

ulimit -a

1.3 准备Ubuntu的源

1. 安装Ubuntu的环境
sudo apt-get install dpkg-dev
sudo apt-get install apache2
service apache2 start
sudo  systemctl enable apache2

2. 解压ambari 与 HDP的包：
tar -zxvf ambari-2.7.3.0-ubuntu18.tar.gz
tar -zxvf HDP-3.1.0.0-ubuntu18-deb.tar.gz
tar -zxvf HDP-GPL-3.1.0.0-ubuntu18-gpl.tar.gz
tar -zxvf HDP-UTILS-1.1.0.22-ubuntu18.tar.gz
mv ambari /var/www/html/
mv HDP /var/www/html/
mv HDP-GPL /var/www/html/
mv HDP-UTILS /var/www/html/

cd /var/www/html/ambari/ubuntu18/2.7.3.0-139

mv ambari.list /etc/apt/sources.list.d/

cd /var/www/html/HDP/ubuntu18/3.1.0.0-78

mv hdp.list /etc/apt/sources.list.d/

cd /var/www/html/HDP-GPL/ubuntu18/3.1.0.0-78

mv hdp.gpl.list /etc/apt/sources.list.d/

cd /var/www/html/HDP-UTILS/ubuntu18/1.1.0.22

mv hdp-utils.list /etc/apt/sources.list.d/

sudo chmod -R 777 /var/www/html/

ambari的源修改
cd /etc/apt/sources.list.d/

vim ambari.list
---
#VERSION_NUMBER=2.7.3.0-139
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
deb http://172.16.10.61/ambari/ubuntu18/2.7.3.0-139 Ambari main
---

vim hdp.list

---
#VERSION_NUMBER=3.1.0.0-78
deb http://172.16.10.61/HDP/ubuntu18/3.1.0.0-78 HDP main
#deb http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/ubuntu18 HDP-UTILS main
----

vim hdp.gpl.list

---
#VERSION_NUMBER=3.1.0.0-78
deb http://172.16.10.61/HDP-GPL/ubuntu18/3.1.0.0-78 HDP-GPL main
---

vim hdp-utils.list
---
#VERSION_NUMBER=HDP-UTILS-1.1.0.22}
deb http://172.16.10.61/HDP-UTILS/ubuntu18/1.1.0.22 HDP-UTILS main
---

同步所有节点：

 scp *.list root@flyfish62:/etc/apt/sources.list.d/
 scp *.list root@flyfish63:/etc/apt/sources.list.d/

所有节点执行key

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys B9733A7A07513CAD

   apt-get update

所有安装jdk 环境
  tar -zxvf jdk-8u381-linux-x64.tar.gz
   mv jdk1.8.0_381/ /usr/local/jdk

设置jdk 环境变量
  vim /etc/profile 
  ---
export JAVA_HOME=/usr/local/jdk
export CLASSPATH=.:$JAVA_HOME/jre/lib:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin

  ----

 source /etc/profile
 java -version

所有节点安装supervisor
apt-get install -y supervisor

1.4 安装MySQL

安装MySQL 5.7 
apt-get install mysql-server

vim /etc/mysql/mysql.conf.d/mysqld.cnf

---
#bind-address           = 127.0.0.1    这行注释掉
---


service mysql start

ps -ef |grep mysql

mysql

 set password = password("flyfish225");
 flush privileges;

 grant all privileges on root.* to 'root'@'%' identified by 'flyfish225' with grant option;
 
 flush privileges;

创建ambari 的 账号与授权 

create user 'ambari'@'*' identified by 'Ambari123';
 
create database ambari;
 
grant all privileges on ambari.* to 'ambari'@'%' identified by 'Ambari123' with grant option;

flush privileges;

二：安装ambari

sudo apt install ambari-server

配置jdbc连接

unzip mysql-connector-java-5.1.49.zip

cp -ap mysql-connector-java-5.1.49-bin.jar /usr/share/java/mysql-connector-java.jar

vim /etc/ambari-server/conf/ambari.properties

----
增加：
  server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

---

ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

root@flyfish61:~# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
WARNING: Could not run /usr/sbin/sestatus: OK
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):ambari
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/local/jdk
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database name (ambari): ambari
Username (ambari): ambari
Enter Database Password (bigdata):
Re-enter password:
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.7.3.0.139.jar
....
Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

mysql -uroot -pflyfish225

use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;

修改ambari 配置文件

vim /etc/ambari-server/conf/ambari.properties
---
将所有涉及jdbc 连接的主机名称全部改为localhost

server.jdbc.rca.url=jdbc:mysql://localhost:3306/ambari

server.jdbc.url=jdbc:mysql://localhost:3306/ambari
---

启动ambari-server 

ambari-server start

打开web :

 172.16.10.61:8080

用户名密码：

 admin/admin

本地源：
   http://172.16.10.61/HDP/ubuntu18/3.1.0.0-78
   http://172.16.10.61/HDP-GPL/ubuntu18/3.1.0.0-78
   http://172.16.10.61/HDP-UTILS/ubuntu18/1.1.0.22

先安装hdfs/zookeeper/ambari metric

启动报错

报错： ambari-hdp-1.repo 源的baseurl 为空值， 这个是因为 官方源封闭了需要改MySQL 的表：
   
 select * from repo_definition;

   http://172.16.10.61/HDP/ubuntu18/3.1.0.0-78
   http://172.16.10.61/HDP-GPL/ubuntu18/3.1.0.0-78
   http://172.16.10.61/HDP-UTILS/ubuntu18/1.1.0.22


将表的base_url值加入：
 update repo_definition set base_url='http://172.16.10.61/HDP/ubuntu18/3.1.0.0-78' where id=25;
 update repo_definition set base_url='http://172.16.10.61/HDP-GPL/ubuntu18/3.1.0.0-78' where id=26;

删掉ambari-hdp-1.repo
cd /etc/apt/sources.list.d
rm -rf ambari-hdp-1.repo 
从新启动ambari-server 

ambari-server restart

这一步分发要等很长的一段时间

仍然报错：
  select * from repo_definition;

将HDP-3.1-GPL的源一个更改一下：
    update repo_definition set base_url='http://172.16.10.61/HDP-UTILS/ubuntu18/1.1.0.22' where id=51;

再次删了：
  删掉ambari-hdp-1.repo
rm -rf ambari-hdp-1.repo 
从新启动ambari-server 

ambari-server restart

秒客网

Ubuntu18.04.6 server 安装 HDP3.1.0

1.1 系统介绍

1.2 系统初始化

1.3 准备Ubuntu的源

1.4 安装MySQL

二：安装ambari

相关文章

Ubuntu18.04.6 server 安装 HDP3.1.0

1.1 系统介绍

1.2 系统初始化

1.3 准备Ubuntu的源

1.4 安装MySQL

二： 安装ambari

相关文章

二：安装ambari