搭建Ambari集群

最新推荐文章于 2023-12-29 00:00:00 发布
最新推荐文章于 2023-12-29 00:00:00 发布
阅读量280 收藏
点赞数
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/xiaoxiongaa0/article/details/104061897

 

[root@hadoop001 ~]# visudo



[root@hadoop001 ssh]# useradd hadoop

SSH免密

[hadoop@hadoop001 ~]$ ssh-keygen

[hadoop@hadoop001 ~]$ cd .ssh

[hadoop@hadoop001 .ssh]$ pwd
/home/hadoop/.ssh

[hadoop@hadoop001 .ssh]$ cat id_rsa.pub >> authorized_keys

[hadoop@hadoop001 .ssh]$ chmod 700 ~/.ssh

[hadoop@hadoop001 .ssh]$ chmod 600 ~/.ssh/authorized_keys


[hadoop@hadoop001 .ssh]$ ssh hadoop001
The authenticity of host 'hadoop001 (172.31.36.137)' can't be established.
ECDSA key fingerprint is SHA256:AAM1VixV4qWn6aVj1liWEOFzmsYKTYxqOFKokwPIPwI.
ECDSA key fingerprint is MD5:2d:1b:1d:d2:c2:32:34:ea:fe:ba:52:37:c4:a3:c8:27.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop001,172.31.36.137' (ECDSA) to the list of known hosts.
Last login: Tue Jan 21 14:55:08 2020

Welcome to Alibaba Cloud Elastic Compute Service !

安装Java

[root@hadoop001 software]# sudo tar -zxvf jdk-8u231-linux-x64.tar.gz -C /usr/local/

[root@hadoop001 software]# sudo ln -s /usr/local/jdk1.8.0_231/ /usr/local/jdk

[root@hadoop001 software]# sudo vi /etc/profile

export JAVA_HOME=/usr/local/jdk
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

[root@hadoop001 software]# source /etc/profile

 

[hadoop@hadoop001 software]$ wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/ambari-2.7.4.0-centos 7.tar.gz

http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/HDP-3.1.4.0-centos7-rpm.tar.gz

http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz

http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.4.0/HDP-GPL-3.1.4.0-centos7gpl.tar.gz

此处http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.1.4.0/HDP-GPL-3.1.4.0-centos7gpl.tar.gz这条链接已失效

安装Apache

[hadoop@hadoop001 software]$ sudo yum -y install httpd

[hadoop@hadoop001 software]$ sudo systemctl enable httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.

[hadoop@hadoop001 software]$ sudo systemctl start httpd


[hadoop@hadoop001 ~]$ sudo mv ~/software/HDP-3.1.4.0-centos7-rpm.tar.gz /var/www/html/

[hadoop@hadoop001 ~]$ sudo mv ~/software/HDP-GPL-3.1.4.0-centos7-gpl.tar.gz /var/www/html/

[hadoop@hadoop001 ~]$ sudo mv ~/software/HDP-UTILS-1.1.0.22-centos7.tar.gz /var/www/html/

[hadoop@hadoop001 ~]$ sudo mv ~/software/ambari-2.7.4.0-centos7.tar.gz /var/www/html/


[hadoop@hadoop001 software]$ cd /var/www/html/

[hadoop@hadoop001 html]$ sudo tar -zxvf ambari-2.7.4.0-centos7.tar.gz

[hadoop@hadoop001 html]$ sudo tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz

[hadoop@hadoop001 html]$ sudo tar -zxvf HDP-GPL-3.1.4.0-centos7-gpl.tar.gz

[hadoop@hadoop001 html]$ sudo tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz

 

[hadoop@hadoop001 html]$ sudo chmod -R ugo+rX /var/www/html/

 

[hadoop@hadoop001 html]$ cd /etc/yum.repos.d/

[hadoop@hadoop001 yum.repos.d]$ sudo vim ambari.repo


[ambari-repo]
name=ambari
baseurl=http://hadoop001/ambari/centos7/2.7.4.0-118/
gpgcheck=0
enabled=1

安装MySQL

[hadoop@hadoop001 ~]$ sudo yum -y localinstall https://dev.mysql.com/get/mysql57-community-release-el7-8.noarch.rpm

[hadoop@hadoop001 ~]$ sudo yum -y install mysql-community-server

启动MySQL

[hadoop@hadoop001 ~]$ sudo systemctl enable mysqld
[hadoop@hadoop001 ~]$ sudo systemctl start mysqld
[hadoop@hadoop001 ~]$ sudo systemctl status mysqld
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2020-01-21 16:13:56 CST; 4s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 12030 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=0/SUCCESS)
  Process: 11980 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 12034 (mysqld)
   CGroup: /system.slice/mysqld.service
           └─12034 /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid

Jan 21 16:13:51 hadoop001 systemd[1]: Starting MySQL Server...
Jan 21 16:13:56 hadoop001 systemd[1]: Started MySQL Server.


[hadoop@hadoop001 ~]$ sudo grep 'temporary password' /var/log/mysqld.log
2020-01-21T08:13:54.177137Z 1 [Note] A temporary password is generated for root@localhost: :l%1LM8iddA?

[hadoop@hadoop001 ~]$ mysql -uroot -p

mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)

mysql> set global validate_password_mixed_case_count=0;
Query OK, 0 rows affected (0.00 sec)

mysql> set global validate_password_number_count=3;
Query OK, 0 rows affected (0.00 sec)

mysql> set global validate_password_special_char_count=0;
Query OK, 0 rows affected (0.00 sec)

mysql> set global validate_password_length=3;
Query OK, 0 rows affected (0.00 sec)

mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.00 sec)


[hadoop@hadoop001 ~]$ sudo mkdir -p /usr/share/java

[hadoop@hadoop001 ~]$ exit
logout

[root@hadoop001 software]# mv mysql-connector-java-8.0.18.jar /usr/share/java/


[hadoop@hadoop001 ~]$ sudo yum -y install ambari-server


mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)

mysql>   set global validate_password_mixed_case_count=0;
Query OK, 0 rows affected (0.00 sec)

mysql>   set global validate_password_number_count=3;
Query OK, 0 rows affected (0.00 sec)

mysql>   set global validate_password_special_char_count=0;
Query OK, 0 rows affected (0.00 sec)

mysql>   set global validate_password_length=3;
Query OK, 0 rows affected (0.00 sec)

mysql> create database ambari;
Query OK, 1 row affected (0.01 sec)

mysql> CREATE USER 'ambari'@'%' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL ON ambari.* TO 'ambari'@'%';
Query OK, 0 rows affected (0.01 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

mysql> use ambari;
Database changed
mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;


[hadoop@hadoop001 ~]$ sudo ambari-server setup

启动Ambari

[hadoop@hadoop001 ~]$ sudo ambari-server start

如果出现错误，一定要查看日志: /var/log/ambari-server/ambari-server.log

[hadoop@hadoop001 ~]$ cat ~/.ssh/id_rsa

加入Ambari管理

 

 

如果hadoop002和hadoop003没有安装jdk就会安装不成功

 

 

安装Kafka

 

安装Storm

部署HBase

部署Hive

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>
mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)

mysql>   set global validate_password_mixed_case_count=0;
Query OK, 0 rows affected (0.00 sec)

mysql>   set global validate_password_number_count=3;
Query OK, 0 rows affected (0.00 sec)

mysql>   set global validate_password_special_char_count=0;
Query OK, 0 rows affected (0.00 sec)

mysql>   set global validate_password_length=3;
Query OK, 0 rows affected (0.00 sec)

mysql> create database hive;
Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'hive'@'%' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.01 sec)

mysql> GRANT ALL ON hive.* TO 'hive'@'%';
Query OK, 0 rows affected (0.02 sec)

mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)

mysql> exit
Bye


[hadoop@hadoop001 ~]$ sudo ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java-8.0.18.jar
Using python  /usr/bin/python
Setup ambari-server
Copying /usr/share/java/mysql-connector-java-8.0.18.jar to /var/lib/ambari-server/resources/mysql-connector-java-8.0.18.jar
Creating symlink /var/lib/ambari-server/resources/mysql-connector-java-8.0.18.jar to /var/lib/ambari-server/resources/mysql-connector-java.jar
If you are updating existing jdbc driver jar for mysql with mysql-connector-java-8.0.18.jar. Please remove the old driver jar, from all hosts. Restarting services that need the driver, will automatically copy the new jar to the hosts.
JDBC driver was successfully initialized.
Ambari Server 'setup' completed successfully.

注意，启动Timeline Service V2.0 Reader 这个服务内存得够

 

 

 

 

 

 

 

 

 

 

AaronLwx
关注
2、ambari集群搭建完整详细步骤（每一步都有实操）
Davina_yu的博客
12-12 395
在windows浏览器地址栏中输入。在 master 上执行。在 master 上执行。在 master 上执行。在 master 上执行。在 master 上执行。在 master 上执行。在 master 上执行。
Ambari集群部署全流程
m0_70082703的博客
05-01 807
安装Apache sudo yum -y install httpd 将Apache设置为开机启动 sudo systemctl enable httpd 启动Apache sudo systemctl start httpd 测试访问 [http://192.168.186.164](() 3.2 HDP安装部署到apache 通过FTP上传RPM安装包到服务器上 移动安装包到 /var/www/html/目录下 sudo mv ambari-2.7.4.0-cento.
使用Ambari搭建Hadoop集群
08-20
Apache Ambari是一种基于Web的工具，支持Apache Hadoop集群的供应、管理和监控。Ambari已支持大多数Hadoop组件，包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeeper、Sqoop和Hcatalog等。
Ambari集群搭建
LSur_king的博客
08-03 1019
【集群规划】 虚拟机1 虚拟机2 虚拟机3 内 存 5GB 5GB 5GB 磁 盘 115GB 115GB 115GB host名 vm01 vm02 vm03 ip地址 192.168.1.100 192.168.1.102 192.168.1.104 ssh信息 root/polyv123 root/polyv123
Ambari简述及集群搭建（上）
lixinkuan的博客
03-14 1868
一、Ambari简述 1.1、什么是Ambari Apache Ambari项目旨在通过开发用于配置，管理和监控Apache Hadoop集群的软件来简化Hadoop管理。Ambari提供了一个直观，易用的Hadoop管理Web UI。 1.2、Ambari的功能 提供了跨任意数量的主机安装Hadoop服务的分步向导。 处理群集的Hadoop服务配置。 提供集中管理，用于在整个集群中启...
全网最详细的ambari集群搭建详细步骤
luo981695830的博客
05-01 8468
1、服务器环境说明 集群三台节点 可以互相访问 使用root用户 配置免密 centos7 ambari-2.6.2.2 hdp-2.6.5 我的环境如下： 192.168.8.105 my-hdp1 192.168.8.106 my-hdp2 192.168.8.107 my-hdp3 2、防火墙设置 #查看防火墙状态 systemctl status firewalld #关闭防火墙 systemctl stop firewalld #设置防火墙服务开机不自启 system.
大数据环境构建：从虚拟机创建到Ambari集群部署的技术指南
最新发布
04-03
内容概要：本文档详细介绍了基于Linux系统的大数据环境搭建流程，涵盖从虚拟机创建到集群建立的全过程。首先，通过一系列步骤创建并配置虚拟机，包括设置IP地址、安装MySQL数据库等操作。接着，重点讲解了Ambari的...
Ambari HDP集群搭建全攻略
张伟的专栏
01-12 643
（就是一个开源的hadoop一键式安装服务）此外，Ambari能够安装安全的（基于Kerberos）Hadoop集群，以此实现了对Hadoop 安全的支持，提供了基于角色的用户认证、授权和审计功能，并为用户管理集成了LDAP和Active Directory。此外，Ambari能够安装安全的（基于Kerberos）Hadoop集群，以此实现了对Hadoop 安全的支持，提供了基于角色的用户认证、授权和审计功能，并为用户管理集成了LDAP和Active Directory。（说白了就是可以偷好多懒）
使用ambari安装Hadoop集群
九万里
07-09 4935
一、环境准备 1.操作系统检查 操作系统版本检查 cat /etc/redhat-release 操作系统默认语言检查 echo $LANG echo 'export LANG=en_US.UTF-8' >> ~/.bashrc 操作系统时区设置 cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime 时钟同步 4.1 安装ntp服务： rpm –qa|grep ntp yum install ntp systemctl is
基于Ambari的Hadoop集群的搭建（上）
m0_74973672的博客
12-27 2013
安装并利用Ambari搭建Hadoop集群（上）
Ambari2.7.5集群搭建详细流程
不以物喜的博客
02-17 2478
本文基于本地虚拟机从零开始搭建ambari集群。
2018最新大数据Ambari+HDP搭建
maoqun55的博客
06-03 2089
01.HDP介绍02.环境准备--ntp--java--selinux--hosts--hostname--ip03.环境准备--克隆机器--配置IP和HostName03.环境准备--克隆机器--配置IP和HostName04.环境准备--安装MySQL--配置免密04.环境准备--安装MySQL--配置免密05.配置本地Ambari-HDP的yum源05.配置本地Ambari-HDP的yum源...
基于Ambari的Hadoop集群的搭建（下）
m0_74973672的博客
12-29 1566
基于ambari的Hadoop集群的搭建
Ambari集群搭建(CentOS7)
bigdata_player
04-28 2239
本文以 ambari-2.6.2.2 、 HDP-2.6.5.0 、HDP-UTILS-1.1.0.22 、HDP-GPL-2.6.5.0 为例搭建ambari集群。 官网地址： ambari ：https://docs.cloudera.com/HDPDocuments/Ambari-2.6.2.2/bk_ambari-installation/content/ambari_...
Ambari大数据集群的搭建
wujin8589的博客
03-22 3410
Ambari——大数据平台的搭建利器   虽然大数据越来越流行，但其学习的门槛却一直阻碍着很多的分布式应用初学者或者大数据的业务应用开发者。多个产品之间的不兼容问题，快速集成和维护也显得比较困难。不管是Hadoop V1 或者V2 的安装，又或者Spark/YARN 等的集成，都不是几行简单的命令可以完成的，而是需要手工修改很多的集群配置，这进一步增加了业务开发者的学习和使用难度。有
使用Ambari轻松搭建大数据集群
"Ambari-大数据平台的搭建利器，通过本地源搭建Ambari服务，适用于华为公有云，涉及操作系统配置、JDK卸载、SSH免密登录、防火墙和SELinux关闭、NTP同步、本地源制作等步骤，旨在简化Hadoop生态系统的管理和监控。...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值