Container killed on request. Exit code is 143

背景

只是一条INSERT语句插入一条数据,然后就报错

报错如下

Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

分析

之前写过这个错误,这只是个表象错误,具体问题还是需要看日志分析。Z

正常日志里没有报错,在这个任务的一个POST日志文件中发现如下报错

Container container_e19_1577953713473_0001_01_000003 exited with diagnostics set to Container failed, exitCode=-105. Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

因某种原因通过度娘无法找到exitCode=-105的相关,只能找到Exit code is 143的相关问题。大部分都是在说是yarn的container分配的内存不够,需要进行yarn资源调整。但是详细的可以看到具体报错就是内存问题的报错,跟自己遇到的并不是同一种情况。

还是回过头来分析语句,因为是往特定的一个分区插数,且之前出现过分区丢失的情况,于是试一下换个分区插入数据,结果成功了。那么原因就找到了,原来分区映射存在问题也会导致这种错误,涨姿势了。修复该分区后,问题解决。

2025-04-02 11:24:49,591 ERROR org.apache.spark.scheduler.cluster.YarnScheduler: Lost executor 5 on dominos-usdp-v3-pro02: Container from a bad node: container_e35_1709867558487_220002_02_000002 on host: dominos-usdp-v3-pro02. Exit status: 143. Diagnostics: [2025-04-02 11:24:49.169]Container killed on request. Exit code is 143 [2025-04-02 11:24:49.170]Container exited with a non-zero exit code 143. [2025-04-02 11:24:49.170]Killed by external signal . 2025-04-02 11:24:49,592 WARN org.apache.spark.scheduler.TaskSetManager: Lost task 0.0 in stage 2.0 (TID 5) (dominos-usdp-v3-pro02 executor 5): ExecutorLostFailure (executor 5 exited caused by one of the running tasks) Reason: Container from a bad node: container_e35_1709867558487_220002_02_000002 on host: dominos-usdp-v3-pro02. Exit status: 143. Diagnostics: [2025-04-02 11:24:49.169]Container killed on request. Exit code is 143 [2025-04-02 11:24:49.170]Container exited with a non-zero exit code 143. [2025-04-02 11:24:49.170]Killed by external signal . 2025-04-02 11:24:49,594 WARN org.apache.spark.scheduler.cluster.YarnSchedulerBackend$YarnSchedulerEndpoint: Requesting driver to remove executor 5 for reason Container from a bad node: container_e35_1709867558487_220002_02_000002 on host: dominos-usdp-v3-pro02. Exit status: 143. Diagnostics: [2025-04-02 11:24:49.169]Container killed on request. Exit code is 143 [2025-04-02 11:24:49.170]Container exited with a non-zero exit code 143. [2025-04-02 11:24:49.170]Killed by external signal . 2025-04-02 11:26:37,187 ERROR org.apache.spark.scheduler.cluster.YarnScheduler: Lost executor 6 on dominos-usdp-v3-pro03: Container from a bad node: container_e35_1709867558487_220002_02_000003 on host: dominos-usdp-v3-pro03. Exit status: 143. Diagnostics: [2025-04-02 11:26:36.792]Container killed on request. Exit code is 143 [2025-04-02 11:26:36.793]Container exited with a non-zero exit code 143.
最新发布
04-03
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值