CentOS 7.2 64位 安装hadoop之后无法正常启动hdfs问题

在两台CentOS 7.2系统上安装Hadoop 2.6.0-cdh5.7.0后,启动hdfs时遇到警告和错误,包括无法加载native-hadoop库、namenode服务未配置、配置目录找不到等问题。通过配置SSH免密登录、修改HADOOP_CONF_DIR环境变量、更新hosts文件以及添加native库文件,最终成功启动HDFS。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

分别在两台linux系统上安装hadoop时,./start-dfs.sh启动hdfs时均出现报错

19/02/02 22:24:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

Starting namenodes on [hadoop001]

hadoop001: Warning: Permanently added 'hadoop001,139.199.101.84' (ECDSA) to the list of known hosts.

chao@hadoop001's password: 

hadoop001: starting namenode, logging to /home/chao/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-chao-namenode-hadoop001.out

localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.

chao@localhost's password: 

localhost: starting datanode, logging to /home/chao/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-chao-datanode-hadoop001.out

Starting secondary namenodes [0.0.0.0]

0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.

chao@0.0.0.0's password: 

0.0.0.0: starting secondarynamenode, logging to /home/chao/app/hadoop-2.6.0-cdh5.7.0/logs/hadoop-chao-secondarynamenode-hadoop001.out

19/02/02 22:24:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

 

尝试了很多解决方法,后来根据博客

https://blog.csdn.net/github_37643896/article/details/82463455

配置本地ssh 免密登录 

即在当前登录用户/.ssh 文件下生成新的秘钥文件   ssh-keygen -t rsa

然后将公钥导入授权文件 
cat id_rsa.pub >> authorized_keys

此时终于不再报错了

但也出现了新的错误

Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured.

Starting namenodes on [2019-02-02 22:31:18,183 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable]

Error: Cannot find configuration directory: /etc/hadoop

Error: Cannot find configuration directory: /etc/hadoop

Starting secondary namenodes [2019-02-02 22:31:19,921 WARN  [main] util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

0.0.0.0]

Error: Cannot find configuration directory: /etc/hadoop

 

修改hadoop-env.sh文件中的环境变量 export HADOOP_CONF_DIR=/home/用户名/app/hadoop-2.6.0-cdh5.7.0/etc/hadoop/

重新source hadoop-env.sh

重新启动hdfs:./start-dfs.sh

使用jps查看看到hdfs已启动,但是namenode没有启动

找了很久之后发现因为我用的是腾讯云服务器,无法绑定公网ip的地址的原因,修改/etc/hosts下的文件即后解决解决:

内网IP地址  你的hostname
公网IP地址  别的hostname

https://blog.csdn.net/u011521382/article/details/81432968

至此,搞了两三天,这个小破问题终于解决了。

 

此外还解决了native下文件夹为空的问题,解决方法是到网上下载native按压缩包,将其中的文件解压到lib文件夹下后解决。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值