openmpi跨节点报错tcp_peer_send_blocking: send() to socket 9 failed: Broken pipe (32)

最新推荐文章于 2025-06-24 17:25:16 发布

wjcaiyf

最新推荐文章于 2025-06-24 17:25:16 发布

阅读量2.8k

点赞数 2

CC 4.0 BY-SA版权

分类专栏：工作笔记 parallel 文章标签： openmpi tcp_peer_sen

本文链接：https://blog.csdn.net/wjciayf/article/details/75656027

工作笔记同时被 2 个专栏收录

19 篇文章

订阅专栏

parallel

6 篇文章

订阅专栏

本文介绍了一个关于OpenMPI跨节点运行时遇到的问题及解决方案。客户反馈在使用mpirun命令进行跨节点作业时出现了错误，通过添加特定参数来诊断连接问题，并给出了可能的原因及其对应的解决办法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

客户反应作业无法跨节点，运行测试命令如下

mpirun -np 8 -hostfile hostfilt.txt sleep 5

运行后报错如下：

[test02:01719] [[24772,0],1] tcp_peer_send_blocking: send() to socket 9 failed: Broken pipe (32)
--------------------------------------------------------------------------
ORTE was unable to reliably start one or more daemons.
This usually is caused by:

* not finding the required libraries and/or binaries on
  one or more nodes. Please check your PATH and LD_LIBRARY_PATH
  settings, or configure OMPI with --enable-orterun-prefix-by-default

* lack of authority to execute on one or more specified nodes.
  Please verify your allocation and authorities.

* the inability to write startup files into /tmp (--tmpdir/orte_tmpdir_base).
  Please check with your sys admin to determine the correct location to use.

*  compilation of the orted with dynamic libraries when static are required
  (e.g., on Cray). Please check your configure cmd line and consider using
  one of the contrib/platform definitions for your system type.

* an inability to create a connection back to mpirun due to a
  lack of common network interfaces and/or no route found between
  them. Please check network connectivity (including firewalls
  and network routing requirements).
--------------------------------------------------------------------------

网上查到的解决方法：

I had this same problem on Cygwin with OpenMPI 1.10.4.

Try adding "-report-uri -" to your mpirun command to see what IP address it's trying to use for connection:

mpirun -report-uri - -np 2 a.exe
It should print out a line that looks something like this:

568328192.0;tcp://192.168.10.103,169.254.247.11,0.0.0.0,0.0.0.0,0.0.0.0:55600
If the first IP address after the "tcp://" is not a current valid address for your machine, that's the problem and things are likely to break (even if the correct IP appears later in the list). Apparently ORTE is not smart enough to order the interfaces based on what is actually enabled and online.

If the wrong IP corresponds to an old/disabled interface, uninstall it (if possible)

Reference: https://stackoverflow.com/questions/34032655/cygwin-error-tcp-peer-send-blocking-send-to-socket