Clickhouse：clickhouse集群化安装分片及副本_clickhouse replicatedmergetree 修改-CSDN博客

本文链接：https://blog.csdn.net/Joseph25/article/details/125070380

vim /etc/security/limits.conf
* soft nofile 65536 
* hard nofile 65536 
* soft nproc 131072 
* hard nproc 131072

vim /etc/security/limits.d/20-nproc.conf 
* soft nofile 65536 
* hard nofile 65536 
* soft nproc 131072 
* hard nproc 131072

scp -r /etc/security/limits.conf node02:/etc/security/
scp -r /etc/security/limits.d/20-nproc.conf node02:/etc/security/limits.d/

各节点：
yum install -y libtool
yum install -y *unixODBC*

1，安装:https://clickhouse.com/
cd /opt/soft
mkdir clickhouse上传软件

2，rpm -ivh *.rpm 安装

2.,1 修改配置文件
[root@node01 clickhouse]$ vim /etc/clickhouse-server/config.xml
（1）把 <listen_host>::</listen_host> 的注释打开，

这样的话才能让 ClickHouse 被除本机以外的服务器访问

（2）将文件分发到其他机器

scp -r /etc/clickhouse-server/config.xml node02:/etc/clickhouse-server/

在这个文件中，有 ClickHouse 的一些默认路径配置，比较重要的
数据文件路径：<path>/var/lib/clickhouse/</path>
日志文件路径：<log>/var/log/clickhouse-server/clickhouse-server.log</log>

开启：systemctl status clickhouse-server 查看clickhouse状态

systemctl start clickhouse-server 开启clickhouse

systemctl disable clickhouse-server 关闭开机自启

连接：clickhouse client -m 默认端口9000连接

如果9000端口北站修改端口，vim /etc/clickhouse-server/config.xml

修改<tcp_port>19000</tcp_port>

然后启动指定修改后的端口 clickhouse client --port 19000

3，修改副本安装

副本的目的主要是保障数据的高可用性，即使一台 ClickHouse 节点宕机，那么也可以从
其他服务器获得相同的数据。写入流程

3.1 配置步骤
（1）启动 zookeeper 集群
（2）在 node01 的/etc/clickhouse-server/config.d 目录下创建一个名为 metrika.xml
的配置文件,内容如下：
注：也可以不创建外部文件，直接在 config.xml 中指定<zookeeper>

<?xml version="1.0"?>
<yandex>
        <zookeeper-servers>
                <node index="1">
                        <host>node01</host>
                        <port>2181</port>
                </node>
                <node index="2">
                        <host>node02</host>
                        <port>2181</port>
                </node>
                <node index="3">
                        <host>node03</host>
                        <port>2181</port>
                </node>
                <node index="4">
                        <host>node04</host>
                        <port>2181</port>
                </node>
                <node index="5">
                        <host>node05</host>
                        <port>2181</port>
                </node>
        </zookeeper-servers>
</yandex>

（3）将metrika.xml同步到node02上：

scp -r /etc/clickhouse-server/config.d/metrika.xml node02:/etc/clickhouse-server/config.d/

（4）修改node01上的/etc/clickhouse-server/config.xml

<zookeeper incl="zookeeper-servers" optional="true" />
<include_from>/etc/clickhouse-server/config.d/metrika.xml</include_from>

（5）将config.xml分发到node02上

scp -r /etc/clickhouse-server/config.xml node02:/etc/clickhouse-server/

（6）重启clickhouse

systemctl restart clickhouse-server

（7）建表插入数据验证

        ①node01上
        create table t_order_rep2 (
        id UInt32,
        sku_id String,
        total_amount Decimal(16,2),
        create_time Datetime
        )engine =ReplicatedMergeTree('/clickhouse/table/01/t_order_rep','rep_102')
        partition by toYYYYMMDD(create_time)
        primary key (id)
        order by (id,sku_id);

        ②node02上
        create table t_order_rep2 (
        id UInt32,
        sku_id String,
        total_amount Decimal(16,2),
        create_time Datetime
        )engine =ReplicatedMergeTree('/clickhouse/table/01/t_order_rep','rep_103')
        partition by toYYYYMMDD(create_time)
        primary key (id)
        order by (id,sku_id);

③参数解释
ReplicatedMergeTree 中，
第一个参数是分片的 zk_path 一般按照： /clickhouse/table/{shard}/{table_name} 的格式，如果只有一个分片就写 01 即可。
第二个参数是副本名称，相同的分片副本名称不能相同。

        在node01上执行insert语句
        insert into t_order_rep2 values
        (101,'sku_001',1000.00,'2020-06-01 12:00:00'),
        (102,'sku_002',2000.00,'2020-06-01 12:00:00'),
        (103,'sku_004',2500.00,'2020-06-01 12:00:00'),
        (104,'sku_002',2000.00,'2020-06-01 12:00:00'),
        (105,'sku_003',600.00,'2020-06-02 12:00:00');

4，分片，修改node01 的/etc/clickhouse-server/config.d 目录下创建一个名为 metrika-shard.xml

<?xml version="1.0"?>
<yandex>
   <remote_servers>
       <gmall_cluster>
           <shard>
               <internal_replication>true</internal_replication>
               <replica>
                   <host>node01</host>
                   <port>9000</port>
               </replica>
               <replica>
                   <host>node02</host>
                   <port>9000</port>
               </replica>
           </shard>
           <shard>
               <internal_replication>true</internal_replication>
               <replica>
                   <host>node03</host>
                   <port>9000</port>
               </replica>
               <replica>
                   <host>node04</host>
                   <port>9000</port>
               </replica>
           </shard>
       </gmall_cluster>
   </remote_servers>

   <zookeeper-servers>
           <node index="1">
                   <host>node01</host>
                   <port>2181</port>
           </node>
           <node index="2">
                   <host>node02</host>
                   <port>2181</port>
           </node>
           <node index="3">
                   <host>node03</host>
                   <port>2181</port>
           </node>
           <node index="4">
                   <host>node04</host>
                   <port>2181</port>
           </node>
           <node index="5">
                   <host>node05</host>
                   <port>2181</port>
           </node>
   </zookeeper-servers>

   <macros>
       <shard>01</shard>
       <replica>rep_1_1</replica>
   </macros>
</yandex>