Hadoop启动后，datanode 和namenode节点有了，但是不能往hdfs上上传文件的原因

最新推荐文章于 2025-01-04 20:45:01 发布

原创最新推荐文章于 2025-01-04 20:45:01 发布 · 876 阅读

2 ·

CC 4.0 BY-SA版权

文章标签：

#hadoop #datanode #namenode #不能往hdfs上上传文件

博客记录了使用Hadoop时无法往HDFS上传文件的报错信息，显示文件只能复制到0个节点，无运行的datanode。给出解决办法，即删除Hadoop安装目录下的tmp和dfs文件并重新创建，格式化集群，若集群处于安全模式，可执行命令退出。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

报错

[root@master mapreduce_wordcount_python]# hadoop dfs -put The_Man_of_Property.txt /
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

19/03/20 06:42:00 WARN hdfs.DFSClient: DataStreamer Exception
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /The_Man_of_Property.txt.COPYING could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation.
at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1562)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3245)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:663)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:482)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol $2 . c a l l B l o c k i n g M e t h o d (C l i e n t N a m e n o d e P r o t o c o l P r o t o s . j a v a) a t o r g . a p a c h e . h a d o o p . i p c . P r o t o b u f R p c E n g i n e$ Server $P r o t o B u f R p c I n v o k e r . c a l l (P r o t o b u f R p c E n g i n e . j a v a : 619) a t o r g . a p a c h e . h a d o o p . i p c . R P C$ Server.call(RPC.java:962)
at org.apache.hadoop.ipc.Server$Handler $1 . r u n (S e r v e r . j a v a : 2040) a t o r g . a p a c h e . h a d o o p . i p c . S e r v e r$ Handler $1 . r u n (S e r v e r . j a v a : 2036) a t j a v a . s e c u r i t y . A c c e s s C o n t r o l l e r . d o P r i v i l e g e d (N a t i v e M e t h o d) a t j a v a x . s e c u r i t y . a u t h . S u b j e c t . d o A s (S u b j e c t . j a v a : 422) a t o r g . a p a c h e . h a d o o p . s e c u r i t y . U s e r G r o u p I n f o r m a t i o n . d o A s (U s e r G r o u p I n f o r m a t i o n . j a v a : 1656) a t o r g . a p a c h e . h a d o o p . i p c . S e r v e r$ Handler.run(Server.java:2034)

at org.apache.hadoop.ipc.Client.call(Client.java:1469)
at org.apache.hadoop.ipc.Client.call(Client.java:1400)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy9.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:399)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy10.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1532)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1349)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:588)

put: File /The_Man_of_Property.txt.COPYING could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation.
[root@master mapreduce_wordcount_python]# hadoop dfs -put The_Man_of_Property.txt /
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

解决办法
删除hadoop安装目录下的，tmp和dfs文件，重新创建，master和slave1，slave2上都执行相同的操作，然后在格式化集群
hadoop namenode -format，重新启动./sbin/start-all.sh

如过遇到集群处于安全模式
[root@master mapreduce_wordcount_python]# hadoop dfs -put The_Man_of_Property.txt /
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.

put: Cannot create file/The_Man_of_Property.txt.COPYING. Name node is in safe mode.

可以通过执行命令退出安全模式

[root@master hadoop-2.6.1]# bin/hadoop dfsadmin -safemode leave
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.