1,在cdh的spark页面 中添加如下配置:
spark-conf/spark-env.sh 的 Spark 服务高级配置代码段(安全阀):
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
export SPARK_LIBRARY_PATH=$SPARK_LIBRARY_PATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
export SPARK_CLASSPATH=$SPARK_CLASSPATH:/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*
2.yarn 添加配置;
yarn.app.mapreduce.am.admin.user.env 追加 :/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/native
yarn.application.classpath追加/opt/cloudera/parcels/HADOOP_LZO/lib/hadoop/lib/*