flume与hadoop jar包冲突

MJK祺

已于 2023-05-14 21:03:27 修改

阅读量1k

点赞数 2

CC 4.0 BY-SA版权

文章标签： java hadoop 大数据 flume

于 2023-05-14 20:59:47 首次发布

本文链接：https://blog.csdn.net/qq_58056343/article/details/130673363

在运行Flume时遇到错误，原因是Flume的guava-11.0.2.jar与Hadoop的guava-27.0-jre.jar版本冲突。解决方法是将Hadoop中的高版本guava包复制到Flume的lib目录，删除Flume中的低版本guava包，之后重新运行Flume，错误消失。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

在运行flume时报了以下错误：

2023-05-14 20:45:44,445 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache. flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:459)] process failed
java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1679)
at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:221)
at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:572)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.NoSuchMethodError:com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1679)
at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:221)
at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:572)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
at java.lang.Thread.run(Thread.java:748)

原因: flume和hadoop的guava包冲突

(flume的版本是guava-11.0.2.jar，hadoop的是guava-27.0-jre.jar)

解决办法：将hadoop的guava包替换掉flume的低版本包即可

(hadoop的guava包在$HADOOP_HOME/share/hadoop/common/lib/)

cp /opt/servers/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar /opt/servers/flume/lib/

rm -rf /opt/servers/flume/lib/guava-11.0.2.jar

然后再运行就不会报错了