跨机房数据双向同步工具之DTLE（更新中）

亮总说数

已于 2024-10-15 15:51:22 修改

阅读量1.2k

点赞数

分类专栏： CDC 文章标签： java 开发语言

于 2020-09-07 17:43:57 首次发布

本文链接：https://blog.csdn.net/leonpenn/article/details/108449095

版权

CDC 专栏收录该内容

1 篇文章

订阅专栏

本文详细介绍了DTLE集群的环境搭建、安装、配置及启动流程，通过具体实例展示了跨数据中心的数据同步配置，适用于追求高可用性的分布式系统架构师和运维工程师。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

博主首页：「亮总说数」，同名公众号 :「亮总说数」，wx号 : 「leonpenn」

一、环境说明

2节点DTLE：

test_node1：192.168.222.76

角色：server+client

test_node2：192.168.222.77

角色：client

二、安装

准备安装包：

Releases · actiontech/dtle · GitHub

rpm -ivh dtle-3.20.08.0.x86_64.rpm

三、配置

配置文件位于：/etc/dtle/consul.hcl、/etc/dtle/nomad.hcl

3.1、consul.hcl

修改：node_name = "consul_node1" # 剩余节点分别为consul_node2

3.2、nomad.hcl

node1：

name = "nomad1" # rename for each node
datacenter = "dc1"
data_dir  = "//var/lib/nomad"
plugin_dir = "//usr/share/dtle/nomad-plugin"

log_level = "Info"
log_file = "//var/log/nomad/"

disable_update_check = true

bind_addr = "192.168.222.76"
# change ports if multiple nodes run on a same machine
ports {
  http = 4646
  rpc  = 4647
  serf = 4648
}
addresses {
  # Default to `bind_addr`. Or set individually here.
}
advertise {
  http = "192.168.222.76:4646"
  rpc  = "192.168.222.76:4647"
  serf = "192.168.222.76:4648"
}

server {
  enabled          = true

  bootstrap_expect = 1
  # Set bootstrap_expect to 3 for multiple (high-availablity) nodes.
  # Multiple nomad nodes will join with consul.
}

client {
  enabled = true
  options = {
    "driver.blacklist" = "docker,exec,java,mock,qemu,rawexec,rkt"
  }

  # Will auto join other server with consul.
}

consul {
  # dtle-plugin and nomad itself use consul separately.
  # nomad uses consul for server_auto_join and client_auto_join.
  # Only one consul can be set here. Write the nearest here,
  #   e.g. the one runs on the same machine with the nomad server.
  address = "192.168.222.76:8500"
}

plugin "dtle" {
  config {
    data_dir = "//var/lib/nomad"
    nats_bind = "192.168.222.76:8193"
    nats_advertise = "192.168.222.76:8193"
    # Repeat the consul address above.
    consul = "192.168.222.76:8500"

    # By default, API compatibility layer is disabled.
    #api_addr = "127.0.0.1:8190"   # for compatibility API
    nomad_addr = "192.168.222.76:4646" # compatibility API need to access a nomad server

    publish_metrics = false
    stats_collection_interval = 15
  }
}

node2：

name = "nomad2" # rename for each node
datacenter = "dc1"
data_dir  = "//var/lib/nomad"
plugin_dir = "//usr/share/dtle/nomad-plugin"

log_level = "Info"
log_file = "//var/log/nomad/"

disable_update_check = true

bind_addr = "192.168.222.77"
# change ports if multiple nodes run on a same machine
ports {
  http = 4646
  rpc  = 4647
  serf = 4648
}
addresses {
  # Default to `bind_addr`. Or set individually here.
}
advertise {
  http = "192.168.222.77:4646"
  rpc  = "192.168.222.77:4647"
  serf = "192.168.222.77:4648"
}

server {
  enabled          = false

  bootstrap_expect = 1
  # Set bootstrap_expect to 3 for multiple (high-availablity) nodes.
  # Multiple nomad nodes will join with consul.
}

client {
  enabled = true
  options = {
    "driver.blacklist" = "docker,exec,java,mock,qemu,rawexec,rkt"
  }

  # Will auto join other server with consul.
}

consul {
  # dtle-plugin and nomad itself use consul separately.
  # nomad uses consul for server_auto_join and client_auto_join.
  # Only one consul can be set here. Write the nearest here,
  #   e.g. the one runs on the same machine with the nomad server.
  address = "192.168.222.77:8500"
}

plugin "dtle" {
  config {
    data_dir = "//var/lib/nomad"
    nats_bind = "192.168.222.77:8193"
    nats_advertise = "192.168.222.77:8193"
    # Repeat the consul address above.
    consul = "192.168.222.77:8500"

    # By default, API compatibility layer is disabled.
    #api_addr = "127.0.0.1:8190"   # for compatibility API
    nomad_addr = "192.168.222.77:4646" # compatibility API need to access a nomad server

    publish_metrics = false
    stats_collection_interval = 15
  }
}

四、启动

服务启动命令：

systemctl start dtle-nomad

systemctl start dtle-consul # 单server模式下，node2无需执行

systemctl enable dtle-consul dtle-nomad # 开机自动启动

五、测试

5.1、测试实例

mysql dc1：192.168.222.76:3306

mysql dc2：192.168.222.77:3306

5.2、job配置

dc1 => dc2

{
  "Job": {
    "ID":"demo-dc1-dc2",
    "Datacenters": ["dc1"],
    "TaskGroups": [{
        "Name": "src",
        "Tasks": [{
          "Name": "src",
          "Driver": "dtle",
          "Affinities": [{
            "LTarget": "${node.unique.name}",
            "RTarget": "nomad1",
            "Operand": "="
          }],
          "Config": {
            "Gtid":"41f102d4-d29f-11e8-8de7-0242ac130002:1-5",
            "ConnectionConfig": {
              "Host":"mysql-dc1",
              "Port": 3306,
              "User": "cdc_user",
              "Password": "123456"
            }
          }
        }]
      }, {
        "Name": "dest",
        "Tasks": [{
          "Name": "dest",
          "Driver": "dtle",
          "Affinities": [{
            "LTarget": "${node.unique.name}",
            "RTarget": "nomad2",
            "Operand": "="
          }],
          "Config": {
            "ConnectionConfig": {
              "Host":"mysql-dc2",
              "Port": 3306,
              "User": "cdc_user",
              "Password": "123456"
            }
          }
        }]
    }]
  }
}