Ubuntu系统安装hadoop3_ubuntu hadoop3安装-CSDN博客

本文链接：https://blog.csdn.net/growing1224/article/details/110264927

本文详细介绍如何在多节点环境下安装并配置Hadoop3集群，包括服务器名称设置、Hadoop软件下载与安装、环境变量配置、核心配置文件修改等步骤，并提供启动与验证方法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

在上文的基础上安装hadoop3

修改服务器名称

sudo vi /etc/hostname #主是Maste 从为Slave1 Slave2

然后重启虚拟机

下载安装Hadoop

下载安装包

https://mirrors.cnnic.cn/apache/hadoop/common/hadoop-3.1.4/

上传到/home/shower/目录下

tar -xvf hadoop-3.1.4.tar.gz
mv hadoop-3.1.4 hadoop

修改环境变量

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64
export JAVA_BIN=/usr/lib/jvm/java-1.8.0-openjdk-amd64/bin
export HADOOP_HOME=/home/shower/hadoop
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export JAVA_HOME JAVA_BIN PATH CLASSPATH
export HDFS_NAMENODE_USER=shower
export HDFS_DATANODE_USER= shower
export HDFS_SECONDARYNAMENODE_USER=shower
export YARN_RESOURCEMANAGER_USER=shower
export YARN_NODEMANAGER_USER=shower

source /etc/profile

修改配置

core-site.xml

<configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://node1:9000</value>
        </property>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>file:///home/shower/hadoop/tmp</value>
                <description>Abase for other temporary directories.</description>
        </property>
</configuration>

hadoop-env.sh

JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64

hdfs-site.xml

<configuration>
        <property>
                <name>dfs.namenode.secondary.http-address</name>
                <value>node1:50090</value>
        </property>
                <property>
                <name>dfs.replication</name>
                <value>2</value>
        </property>
        <property>
                <name>dfs.namenode.name.dir</name>
                <value>file:///home/shower/hadoop/tmp/dfs/name</value>
        </property>
        <property>
                <name>dfs.datanode.data.dir</name>
                <value>file:///home/shower/hadoop/tmp/dfs/data</value>
        </property>
</configuration>

mapred-site.xml

<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
        <property>
                <name>mapreduce.jobhistory.address</name>
                <value>node1:10020</value>
        </property>
        <property>
                <name>mapreduce.jobhistory.webapp.address</name>
                <value>node1:19888</value>
        </property>
</configuration>

works

node2
node3

yarn-site.xml

<configuration>
        <property>
                <name>yarn.resourcemanager.hostname</name>
                <value>node1</value>
        </property>
        <property>
                <name>yarn.nodemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
        </property>
</configuration>

复制到其他机器

tar -zcf hadoop.master.tar.gz hadoop

scp /home/shower/hadoop-3.1.4.tar.gz shower@node2:/home/shower/

scp /home/shower/hadoop-3.1.4.tar.gz shower@node3:/home/shower/

关闭防火墙

service ufw status

service ufw stop