分别在两台linux系统上安装hadoop时,./start-dfs.sh启动hdfs时均出现报错
19/02/02 22:24:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [hadoop001]
hadoop001: Warning: Permanently added 'hadoop001,139.199.101.84' (ECDSA) to the list of known hosts.
chao@hadoop001's password:
hadoop001: starting namenode, logging to /home/chao/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-chao-namenode-hadoop001.out
localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
chao@localhost's password:
localhost: starting datanode, logging to /home/chao/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-chao-datanode-hadoop001.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
chao@0.0.0.0's password:
0.0.0.0: starting secondarynamenode, logging to /home/chao/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-chao-secondarynamenode-hadoop001.out
19/02/02 22:24:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
尝试了很多解决方法,后来根据博客
https://blog.csdn.net/github_37643896/article/details/82463455
配置本地ssh 免密登录
即在当前登录用户/.ssh 文件下生成新的秘钥文件 ssh-keygen -t rsa
然后将公钥导入授权文件
cat id_rsa.pub >> authorized_keys
此时终于不再报错了
但也出现了新的错误
Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.
Starting namenodes on [2019-02-02 22:31:18,183 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable]
Error: Cannot find configuration directory: /etc/hadoop
Error: Cannot find configuration directory: /etc/hadoop
Starting secondary namenodes [2019-02-02 22:31:19,921 WARN [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
0.0.0.0]
Error: Cannot find configuration directory: /etc/hadoop
修改hadoop-env.sh文件中的环境变量 export HADOOP_CONF_DIR=/home/用户名/app/hadoop-2.6.0-cdh5.7.0/etc/hadoop/
重新source hadoop-env.sh
重新启动hdfs:./start-dfs.sh
使用jps查看看到hdfs已启动,但是namenode没有启动
找了很久之后发现因为我用的是腾讯云服务器,无法绑定公网ip的地址的原因,修改/etc/hosts下的文件即后解决解决:
内网IP地址 你的hostname
公网IP地址 别的hostname
https://blog.csdn.net/u011521382/article/details/81432968
至此,搞了两三天,这个小破问题终于解决了。
此外还解决了native下文件夹为空的问题,解决方法是到网上下载native按压缩包,将其中的文件解压到lib文件夹下后解决。