运行Spark-shell,解决Unable to load native-hadoop library for your platform

时间:2023-03-08 17:38:20

启动spark后,运行bin/spark-shell会出现一个警告

提君博客原创

 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

运行Spark-shell,解决Unable to load native-hadoop library for your platform

>>提君博客原创  http://www.cnblogs.com/tijun/  <<

虽然不影响运行,但是看着不舒服。下面我整理的解决方法。

方法有两种,亲测可以解决这个问题。

第一种,在linux环境变量里设置linux共享库

vi /etc/profile
export LD_LIBRARY_PATH=$HADOOP_HOME/lib/native/:$LD_LIBRARY_PATH

source一下,问提解决了,就这么的简单。

提君博客原创

运行Spark-shell,解决Unable to load native-hadoop library for your platform

>>提君博客原创  http://www.cnblogs.com/tijun/  <<

第二种方法,需要设置环境变量和conf/spark-env.sh

环境变量

vi /etc/profile
export JAVA_LIBRARY_PATH=$HADOOP_HOME/lib/native/

conf/spark-env.sh

vi conf/spark-env.sh
export LD_LIBRARY_PATH=$JAVA_LIBRARY_PATH

>>提君博客原创  http://www.cnblogs.com/tijun/  <<

提君博客原创