Hive报错, [Fatal Error] total number of created files now is 100028, which exceeds 100000.

二十六画生的博客

于 2019-03-15 14:05:32 发布

阅读量4.5k

点赞数 1

CC 4.0 BY-SA版权

本文链接：https://blog.csdn.net/u010002184/article/details/88572872

Hive Spark Hadoop MapReduce 专栏收录该内容

162 篇文章 ¥29.90 ¥99.00

订阅专栏

超级会员免费看

当使用Hive进行动态分区操作时，遇到错误提示'[Fatal Error] total number of created files now is 100028, which exceeds 100000.'。解决方案是通过调整配置，增加默认的分区和文件创建限制。" 117868037,10536963,HTML5制作逼真卷轴动画详解,"['前端开发', 'HTML', 'CSS', '动画制作']

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

问题1：

Hive Dynamic partition Error : [Fatal Error] total number of created files now is 100028, which exceeds 100000. Killing the job.

Solution:

So my config increases the default partitions and files created limit:

 set hive.exec.dynamic.partition=true;
 set hive.exec.max.dynamic.partitions=100000;
 SET hive.exec.max.dynamic.partitions.pernode=100000;
 set hive.exec.dynamic.partition.mode=nonstrict;
 SET hive.exec.max.created.files=900000;

转自：