HIVE执行过程中java.lang.ArrayIndexOutOfBoundsException【待解决】

最新推荐文章于 2022-04-17 21:33:20 发布

冉小苏

最新推荐文章于 2022-04-17 21:33:20 发布

阅读量3k

点赞数

CC 4.0 BY-SA版权

分类专栏： hive 文章标签： hive

本文链接：https://blog.csdn.net/zoe_ranxiaosu/article/details/116945415

HIVE执行过程中java.lang.ArrayIndexOutOfBoundsException

表tmp_1

+-----------------------+-------------+
| tmp_1.id  | tmp_1.cl  | tmp_1.plat  |
+-----------------------+-------------+
| 293       | IOS_i01   | IOS         |
| 553       | IOS_i01   | IOS         |
| 559       | AND_a01   | AND         |
| 711       | AND_a01   | AND         |
+---------------+---------------------+

简单通过plat进行id关联

=============
示例1
=============
select 
a.plat,a.cnt,b.cnt
from 
(select
plat,count(*) cnt
from tmp_1
group by plat)a
left join
(select split(cl,'_')[0] as plat,count(*) cnt
from tmp_1 group by split(cl,'_')[0])b
on a.plat=b.plat;

 查询结果：
 +-------------+---------+---------+--+
| a.plat       |  a.cnt  |  b.cnt  |
+-------------+---------+---------+--+
| ALIPAY      | 371     | 371     |
| AND         | 783199  | 783199  |
| IOS         | 659319  | 659319  |
| WECHAT      | 2054    | 2054    |
+-------------+---------+---------+--+

执行正常，且查看执行过程

+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                                                           Explain                                                                                                           |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| Plan not optimized by CBO.                                                                                                                                                                                                  |                                                                                                                                                                                                                            |
| Vertex dependency in root stage                                                                                                                                                                                             |
| Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (BROADCAST_EDGE)                                                                                                                                                                |
| Reducer 4 <- Map 3 (SIMPLE_EDGE)                                                                                                                                                                                            |                                                                                                                                                                                                                            |
| Stage-0                                                                                                                                                                                                                     |
|    Fetch Operator                                                                                                                                                                                                           |
|       limit:-1                                                                                                                                                                                                              |
|       Stage-1                                                                                                                                                                                                               |
|          Reducer 2                                                                                                                                                                                                          |
|          File Output Operator [FS_6313316]                                                                                                                                                                                  |
|             compressed:false                                                                                                                                                                                                |
|             Statistics:Num rows: 794718 Data size: 27599734 Basic stats: COMPLETE Column stats: NONE                                                                                                                        |
|             table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}  |
|             Select Operator [SEL_6313315]                                                                                                                                                                                   |
|                outputColumnNames:["_col0","_col1","_col2"]                                                                                                                                                                  |
|                Statistics:Num rows: 794718 Data size: 27599734 Basic stats: COMPLETE Column stats: NONE                                                                                                                     |
|                Map Join Operator [MAPJOIN_6313319]                                                                                                                                                                          |
|                |  condition map:[{"":"Left Outer Join0 to 1"}]                                                                                                                                                              |
|                |  HybridGraceHashJoin:true                                                                                                                                                                                  |
|                |  keys:{"Reducer 2":"_col0 (type: string)","Reducer 4":"_col0 (type: string)"}                                                                                                                              |
|                |  outputColumnNames:["_col0","_col1","_col3"]                                                                                                                                                               |
|                |  Statistics:Num rows: 794718 Data size: 27599734 Basic stats: COMPLETE Column stats: NONE                                                                                                                  |
|                |<-Reducer 4 [BROADCAST_EDGE]                                                                                                                                                                                |
|                |  Reduce Output Operator [RS_6313313]                                                                                                                                                                       |
|                |     key expressions:_col0 (type: string)                                                                                                                                                                   |
|                |     Map-reduce partition columns:_col0 (type: string)                                                                                                                                                      |
|                |     sort order:+                                                                                                                                                                                           |
|                |     Statistics:Num rows: 722471 Data size: 25090667 Basic stats: COMPLETE Column stats: NONE                                                                                                               |
|