环境
系统：centos7
hdp版本：2.5
hue版本：3.11.0
1. 依赖包安装
yum install krb5-devel cyrus-sasl-gssapi cyrus-sasl-deve libxml2-devel libxslt-devel mysql mysql-devel openldap-devel python-devel python-simplejson sqlite-devel libffi-devel gmp-devel
2. hue下载
官网：http://gethue.com/category/release/
安装
1. 解压到/opt
cd /opt
tar -zxvf hue-3.11.0.tgz
cd hue-3.11.02. 新建用户组hue和用户hue
groupadd hue
useradd -g hue -G root hue3. 编译并更改文件所属用户用户组
cd hue-3.11.0
make apps
chown -R hue:hue /opt/hue-3.11.0 //不然会报数据库不能读的问题4. 配置
vim /opt/hue-3.11.0/desktop/conf/hue.ini
4.1 hdfs
[[hdfs_clusters]]
# HA support by using HttpFs
[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://hadoop1.b3.com:8020
# NameNode logical name.
## logical_name=
# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
# Default port is 14000 for HttpFs.
webhdfs_url=http://hadoop1.b3.com:50070/webhdfs/v1
# Change this if your HDFS cluster is Kerberos-secured
## security_enabled=false
# In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
# have to be verified against certificate authority
## ssl_cert_ca_verify=True
# Directory of the Hadoop configuration
## hadoop_conf_dir=$HADOOP_CONF_DIR when set or '/etc/hadoop/conf'
4.2 yarn
[[yarn_clusters]]
[[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=hadoop2.b3.com
# The port where the ResourceManager IPC listens on
## resourcemanager_port=8032
# Whether to submit jobs to this cluster
submit_to=True
# Resource Manager logical name (required for HA)
## logical_name=
# Change this if your YARN cluster is Kerberos-secured
## security_enabled=false
# URL of the ResourceManager API
resourcemanager_api_url=http://hadoop2.b3.com:8088
# URL of the ProxyServer API
proxy_api_url=http://hadoop2.b3.com:8088
# URL of the HistoryServer API
history_server_api_url=http://hadoop2.b3.com:19888
# URL of the Spark History Server
spark_history_server_url=http://hadoop4.b3.com:18088
# In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
# have to be verified against certificate authorit4.3 hive
[beeswax]
# Host where HiveServer2 is running.
# If Kerberos security is enabled, use fully-qualified domain name (FQDN).
hive_server_host=hadoop2.b3.com
# Port where HiveServer2 Thrift server runs on.
hive_server_port=10000
# Hive configuration directory, where hive-site.xml is located
## hive_conf_dir=/etc/hive/conf
4.4 hbase
[hbase]
# Comma-separated list of HBase Thrift servers for clusters in the format of '(name|host:port)'.
# Use full hostname with security.
# If using Kerberos we assume GSSAPI SASL, not PLAIN.
hbase_clusters=(Cluster|hadoop1.b3.com:16010)
其他的组件看注释安装就行了
5. 启动
/opt/hue-3.11.0/build/env/bin/supervisor -d //-d为后台启动将自带的sqlite3换为postgresql
1. 先结束掉hue的进程
2. 在postgresql新建用户hue，数据库hue,密码hue，授予读取权限
3. 对sqlite3数据备份（新安装的可跳过）
/opt/hue-3.11.0/build/env/bin/hue dumpdata > /tmp/hue_db_dump.json4. 安装psycopg2
yum install python-psycopg25. 修改hue.ini
[[database]]
# Database engine is typically one of:
# postgresql_psycopg2, mysql, sqlite3 or oracle.
#
# Note that for sqlite3, 'name', below is a path to the filename. For other backends, it is the database name
# Note for Oracle, options={"threaded":true} must be set in order to avoid crashes.
# Note for Oracle, you can use the Oracle Service Name by setting "host=" and "port=" and then "name=<host>:<port>/<service_name>".
# Note for MariaDB use the 'mysql' engine.
engine=postgresql_psycopg2
host=hadoop1.b3.com
port=5432
user=hue
password=hue
name=hue //这行是数据库名，要写，我就在这里卡了很久
# Execute this script to produce the database password. This will be used when 'password' is not set.
## password_script=/path/script
## name=desktop/desktop.db
## options={}
6. 迁移数据库
这里切记要用hue用户，不然会失败
sudo -u hue /opt/hue-3.11.0/build/env/bin/hue syncdb --noinput
sudo -u hue /opt/hue-3.11.0/build/env/bin/hue migrate下面是输出
[root@hadoop4 conf]# sudo -u hue /opt/hue-3.11.0/build/env/bin/hue syncdb --noinput
Syncing...
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_openid_auth_nonce
Creating table django_openid_auth_association
Creating table django_openid_auth_useropenid
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table django_admin_log
Creating table south_migrationhistory
Creating table axes_accessattempt
Creating table axes_accesslog
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
Synced:
> django.contrib.auth
> django_openid_auth
> django.contrib.contenttypes
> django.contrib.sessions
> django.contrib.sites
> django.contrib.staticfiles
> django.contrib.admin
> south
> axes
> about
> filebrowser
> help
> impala
> jobbrowser
> metastore
> proxy
> rdbms
> zookeeper
> indexer
Not synced (use migrations):
- django_extensions - desktop - beeswax - hbase - jobsub - oozie - pig - search - security - spark - sqoop - useradmin - notebook (use ./manage.py migrate to migrate these)
[root@hadoop4 conf]# sudo -u hue /opt/hue-3.11.0/build/env/bin/hue migrate
Running migrations for django_extensions:
- Migrating forwards to 0001_empty. > django_extensions:0001_empty
- Loading initial data for django_extensions. Installed 0 object(s) from 0 fixture(s)
Running migrations for desktop:
- Migrating forwards to 0024_auto__add_field_document2_is_managed. > pig:0001_initial
> oozie:0001_initial
> oozie:0002_auto__add_hive
> oozie:0003_auto__add_sqoop
> oozie:0004_auto__add_ssh
> oozie:0005_auto__add_shell
> oozie:0006_auto__chg_field_java_files__chg_field_java_archives__chg_field_sqoop_f
> oozie:0007_auto__chg_field_sqoop_script_path
> oozie:0008_auto__add_distcp
> oozie:0009_auto__add_decision
> oozie:0010_auto__add_fs
> oozie:0011_auto__add_email
> oozie:0012_auto__add_subworkflow__chg_field_email_subject__chg_field_email_body
> oozie:0013_auto__add_generic
> oozie:0014_auto__add_decisionend
> oozie:0015_auto__add_field_dataset_advanced_start_instance__add_field_dataset_ins
> oozie:0016_auto__add_field_coordinator_job_properties
> oozie:0017_auto__add_bundledcoordinator__add_bundle
> oozie:0018_auto__add_field_workflow_managed
> oozie:0019_auto__add_field_java_capture_output
> oozie:0020_chg_large_varchars_to_textfields
> oozie:0021_auto__chg_field_java_args__add_field_job_is_trashed
> oozie:0022_auto__chg_field_mapreduce_node_ptr__chg_field_start_node_ptr
> oozie:0022_change_examples_path_format
> oozie:0023_auto__add_field_node_data__add_field_job_data
> oozie:0024_auto__chg_field_subworkflow_sub_workflow
> oozie:0025_change_examples_path_format
> desktop:0001_initial
> desktop:0002_add_groups_and_homedirs
> desktop:0003_group_permissions
> desktop:0004_grouprelations
> desktop:0005_settings
> desktop:0006_settings_add_tour
> beeswax:0001_initial
> beeswax:0002_auto__add_field_queryhistory_notify
> beeswax:0003_auto__add_field_queryhistory_server_name__add_field_queryhistory_serve
> beeswax:0004_auto__add_session__add_field_queryhistory_server_type__add_field_query
> beeswax:0005_auto__add_field_queryhistory_statement_number
> beeswax:0006_auto__add_field_session_application
> beeswax:0007_auto__add_field_savedquery_is_trashed
> beeswax:0008_auto__add_field_queryhistory_query_type
> desktop:0007_auto__add_documentpermission__add_documenttag__add_document
> desktop:0008_documentpermission_m2m_tables
> desktop:0009_auto__chg_field_document_name
> desktop:0010_auto__add_document2__chg_field_userpreferences_key__chg_field_userpref
> desktop:0011_auto__chg_field_document2_uuid
> desktop:0012_auto__chg_field_documentpermission_perms
> desktop:0013_auto__add_unique_documenttag_owner_tag
> desktop:0014_auto__add_unique_document_content_type_object_id
> desktop:0015_auto__add_unique_documentpermission_doc_perms
> desktop:0016_auto__add_unique_document2_uuid_version_is_history
> desktop:0017_auto__add_document2permission__add_unique_document2permission_doc_perm
> desktop:0018_auto__add_field_document2_parent_directory
> desktop:0019_auto
> desktop:0020_auto__del_field_document2permission_all
> desktop:0021_auto__add_defaultconfiguration__add_unique_defaultconfiguration_app_is
> desktop:0022_auto__del_field_defaultconfiguration_group__del_unique_defaultconfigur
> desktop:0023_auto__del_unique_defaultconfiguration_app_is_default_user__add_field_d
> desktop:0024_auto__add_field_document2_is_managed
- Loading initial data for desktop. Installed 0 object(s) from 0 fixture(s)
Running migrations for beeswax:
- Migrating forwards to 0014_auto__add_field_queryhistory_is_cleared. > beeswax:0009_auto__add_field_savedquery_is_redacted__add_field_queryhistory_is_reda
> beeswax:0009_auto__chg_field_queryhistory_server_port
> beeswax:0010_merge_database_state
> beeswax:0011_auto__chg_field_savedquery_name
> beeswax:0012_auto__add_field_queryhistory_extra
> beeswax:0013_auto__add_field_session_properties
> beeswax:0014_auto__add_field_queryhistory_is_cleared
- Loading initial data for beeswax. Installed 0 object(s) from 0 fixture(s)
Running migrations for hbase:
- Migrating forwards to 0001_initial. > hbase:0001_initial
- Loading initial data for hbase. Installed 0 object(s) from 0 fixture(s)
Running migrations for jobsub:
- Migrating forwards to 0006_chg_varchars_to_textfields. > jobsub:0001_initial
> jobsub:0002_auto__add_ooziestreamingaction__add_oozieaction__add_oozieworkflow__ad
> jobsub:0003_convertCharFieldtoTextField
> jobsub:0004_hue1_to_hue2
> jobsub:0005_unify_with_oozie
> jobsub:0006_chg_varchars_to_textfields
- Loading initial data for jobsub. Installed 0 object(s) from 0 fixture(s)
Running migrations for oozie:
- Migrating forwards to 0027_auto__chg_field_node_name__chg_field_job_name. > oozie:0026_set_default_data_values
> oozie:0027_auto__chg_field_node_name__chg_field_job_name
- Loading initial data for oozie. Installed 0 object(s) from 0 fixture(s)
Running migrations for pig:
- Nothing to migrate. - Loading initial data for pig. Installed 0 object(s) from 0 fixture(s)
Running migrations for search:
- Migrating forwards to 0003_auto__add_field_collection_owner. > search:0001_initial
> search:0002_auto__del_core__add_collection
> search:0003_auto__add_field_collection_owner
- Loading initial data for search. Installed 0 object(s) from 0 fixture(s)
? You have no migrations for the 'security' app. You might want some.
Running migrations for spark:
- Migrating forwards to 0001_initial. > spark:0001_initial
- Loading initial data for spark. Installed 0 object(s) from 0 fixture(s)
Running migrations for sqoop:
- Migrating forwards to 0001_initial. > sqoop:0001_initial
- Loading initial data for sqoop. Installed 0 object(s) from 0 fixture(s)
Running migrations for useradmin:
- Migrating forwards to 0006_auto__add_index_userprofile_last_activity. > useradmin:0001_permissions_and_profiles
> useradmin:0002_add_ldap_support
> useradmin:0003_remove_metastore_readonly_huepermission
> useradmin:0004_add_field_UserProfile_first_login
> useradmin:0005_auto__add_field_userprofile_last_activity
> useradmin:0006_auto__add_index_userprofile_last_activity
- Loading initial data for useradmin. Installed 0 object(s) from 0 fixture(s)
Running migrations for notebook:
- Migrating forwards to 0001_initial. > notebook:0001_initial
- Loading initial data for notebook. Installed 0 object(s) from 0 fixture(s)
错误
1. hue 开启notebook，pyspack 时会出现 CSRF问题
解决：通过 ambari，使livy的csrf为false