PyZMQ 是 ZeroMQ (ØMQ) 的 Python 绑定,它提供了一个简单而强大的消息传递库,用于构建分布式或并发应用程序。
ZeroMQ 简介
ZeroMQ 是一个高性能异步消息库,它提供了多种消息模式:
-
请求-应答 (Request-Reply)
-
发布-订阅 (Publish-Subscribe)
-
推送-拉取 (Push-Pull)
-
排他对 (Exclusive Pair)
安装 PyZMQ
pip install pyzmq
基本概念
上下文 (Context)
所有 ZeroMQ 套接字都在一个上下文中创建。
import zmq context = zmq.Context()
套接字 (Socket)
ZeroMQ 提供了多种套接字类型:
-
zmq.REQ
/zmq.REP
- 请求/应答 -
zmq.PUB
/zmq.SUB
- 发布/订阅 -
zmq.PUSH
/zmq.PULL
- 推送/拉取 -
zmq.PAIR
- 排他对
常见模式示例
请求-应答模式
服务端 (REP):
import zmq context = zmq.Context() socket = context.socket(zmq.REP) socket.bind("tcp://*:5555") while True: message = socket.recv_string() print(f"Received request: {message}") socket.send_string(f"World from {message}")
客户端 (REQ):
import zmq context = zmq.Context() socket = context.socket(zmq.REQ) socket.connect("tcp://localhost:5555") for request in range(10): socket.send_string(f"Hello {request}") message = socket.recv_string() print(f"Received reply: {message}")
发布-订阅模式
发布者 (PUB):
import zmq
import time
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:5556")
topic = "news"
for i in range(10):
message = f"{topic} Message {i}"
socket.send_string(message)
print(f"Sent: {message}")
time.sleep(1)
订阅者 (SUB):
import zmq
context = zmq.Context()
socket = context.socket(zmq.SUB)
socket.connect("tcp://localhost:5556")
socket.setsockopt_string(zmq.SUBSCRIBE, "news")
for i in range(10):
message = socket.recv_string()
print(f"Received: {message}")
推送-拉取模式 (管道模式)
推送者 (PUSH):
import zmq import time context = zmq.Context() socket = context.socket(zmq.PUSH) socket.bind("tcp://*:5557") for i in range(10): socket.send_string(f"Task {i}") print(f"Sent task {i}") time.sleep(1)
拉取者 (PULL):
import zmq context = zmq.Context() socket = context.socket(zmq.PULL) socket.connect("tcp://localhost:5557") for i in range(10): message = socket.recv_string() print(f"Received task: {message}")
高级特性
多部分消息
# 发送多部分消息 socket.send_multipart([b"part1", b"part2", b"part3"]) # 接收多部分消息 parts = socket.recv_multipart()
轮询 (Polling)
import zmq
context = zmq.Context()
receiver = context.socket(zmq.PULL)
receiver.connect("tcp://localhost:5557")
subscriber = context.socket(zmq.SUB)
subscriber.connect("tcp://localhost:5556")
subscriber.setsockopt_string(zmq.SUBSCRIBE, "")
poller = zmq.Poller()
poller.register(receiver, zmq.POLLIN)
poller.register(subscriber, zmq.POLLIN)
while True:
socks = dict(poller.poll())
if receiver in socks:
message = receiver.recv_string()
print(f"Received task: {message}")
if subscriber in socks:
message = subscriber.recv_string()
print(f"Received update: {message}")
代理模式
import zmq
context = zmq.Context()
frontend = context.socket(zmq.ROUTER)
backend = context.socket(zmq.DEALER)
frontend.bind("tcp://*:5559")
backend.bind("tcp://*:5560")
zmq.proxy(frontend, backend)
性能优化
-
使用 inproc 传输:当通信在同一进程内时
socket.bind("inproc://my_endpoint")
-
批量发送消息:使用
send_multipart
代替多次send
-
调整高水位标记 (HWM):
socket.setsockopt(zmq.SNDHWM, 100) # 发送高水位 socket.setsockopt(zmq.RCVHWM, 100) # 接收高水位
-
使用非阻塞操作:
try: socket.send(b"message", zmq.NOBLOCK) except zmq.Again: print("Send would block")
安全特性
加密通信 (CurveZMQ)
# 服务端
server = context.socket(zmq.REP)
server.curve_secretkey = b'server_secret_key'
server.curve_publickey = b'server_public_key'
server.curve_server = True
server.bind("tcp://*:5555")
# 客户端
client = context.socket(zmq.REQ)
client.curve_secretkey = b'client_secret_key'
client.curve_publickey = b'client_public_key'
client.curve_serverkey = b'server_public_key'
client.connect("tcp://localhost:5555")
常见问题
-
REQ-REP 必须严格交替:REQ 必须先发送后接收,REP 必须先接收后发送
-
SUB 套接字必须设置订阅:默认不接收任何消息
-
上下文管理:确保正确关闭套接字和上下文
socket.close() context.term()
PyZMQ 提供了强大而灵活的消息传递能力,适用于各种分布式系统场景。通过组合不同的套接字类型,可以构建复杂的通信模式。