java-org.apache.hadoop.hdfs.DFSInputStream上的NoSuchMethodError

我有一个非常简单的Java代码,可以从hdfs中读取数据

try{
    InputStream s = new GzipCompressorInputStream(hdfsFileSystem.open(filePath), false);
    ByteStreams.copy(s, outputStream);
    s.close();
}
catch (Exception ex){
    logger.error("Problem with file "+ filePath,ex);
}

有时(并非总是如此)会引发异常

java.lang.NoSuchMethodError: org.apache.commons.io.IOUtils.closeQuietly(Ljava/io/Closeable;)V
        at org.apache.hadoop.hdfs.DFSInputStream.getBlockReader(DFSInputStream.java:1099)
        at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:533)
        at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:749)
        at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:793)
        at java.io.DataInputStream.read(DataInputStream.java:149)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
        at org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream.init(GzipCompressorInputStream.java:136)
        at org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream.<init>(GzipCompressorInputStream.java:129)
        [...]

在下面的行中：

InputStream s = new GzipCompressorInputStream(hdfsFileSystem.open(filePath), false);

我正在使用波纹管Maven依赖项来加载hadoop客户端：

    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-client</artifactId>
        <version>2.2.0</version>
    </dependency>

有人知道如何解决此问题吗？当然,我可以将catch(Exception e)更改为catch(Error e),但这不是解决方法.

解决方法:

看起来在您的类路径中,出现了几个具有不同版本的“ commons-io.jar”.
方法“ closeQuietly(Ljava / io / Closeable;)”出现在2.0版中.
有时会先加载旧版本的“ commons-io.jar”,然后出现异常.
需要类路径修复.

【说明】：本文章由站长整理发布，文章内容不代表本站观点，如文中有侵权行为，请与本站客服联系(QQ:254677821)！