tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.mapre

最新推荐文章于 2023-05-16 21:00:25 发布

疯狂的程序猿88888

最新推荐文章于 2023-05-16 21:00:25 发布

阅读量8k

点赞数 3

CC 4.0 BY-SA版权

分类专栏： spark项目实战总结

本文链接：https://blog.csdn.net/u012940753/article/details/85775754

在尝试运行一个简单的Spark程序时遇到了'java.lang.IllegalAccessError'，错误指出试图从类org.apache.hadoop.mapred.FileInputFormat访问com.google.common.base.Stopwatch的构造方法。错误出现在FileInputFormat类的getSplits方法中。通过添加两个Hadoop的包到pom.xml文件后，问题得到解决。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Exception in thread "main" java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>()V from class org.apache.hadoop.mapred.FileInputFormat
   at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:312)
   at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:199)
   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)
   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)
   at scala.Option.getOrElse(Option.scala:121)
   at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)
   at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)
   at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)
   at scala.Option.getOrElse(Option.scala:121)
   at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)
   at org.apache.spark.SparkContext.runJob(SparkContext.scala:2094)
   at org.apache.spark.rdd.RDD.count(RDD.scala:1158)
   at com.anchiva.Day01.LinearRegression$.main(LinearRegression.scala:13)
   at com.anchiva.Day01.LinearRegression.main(LinearRegression.scala)

idea中运行一个简单的spark程序，报上面的错，加了两个hadoop的包，就好了

<dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-common</artifactId>
      <version>2.7.7</version>
    </dependency>

    <!-- https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-mapreduce-client-core -->
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-mapreduce-client-core</artifactId>
      <version>2.7.7</version>
    </dependency>

所有pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://m