问题描述
在HBase上运行MapReduce作业时，报如下异常：IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString
使用HBase环境如下：CDH5.0.1, HBase版本：0.96.1
问题原因
This isssue occurs because of an optimization introduced in HBASE-9867 that inadvertently introduced a classloader dependency. This affects both jobs using the -libjars option and "fat jar," jobs which package their runtime dependencies in a nested lib folder.
这个问题的发生是由于优化了HBASE-9867 引起的，无意间引进了一个依赖类加载器。它影响使用-libjars参数和使用 fat jar两种模式的job.
fat jar模式Hadoop的一个特殊功能：可以读取操作目录中/lib目录下包含的所有库的JAR文件，把运行job依赖的jar放在jar中的lib目录下。
解决方案
1、永久解决方案
cd /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hadoop
ln -s /opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar hbase-protocol-0.96.1.1-cdh5.0.0.jar
2: 临时解决方案
HADOOP_CLASSPATH=/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/hbase/lib/hbase-protocol-0.96.1.1-cdh5.0.0.jar:/etc/hbase/conf hadoop jar myjar.jar MyJobMainClass
详细报错信息
14/07/16 14:40:50 WARN client.HConnectionManager$HConnectionImplementation: Encountered problems when prefetch hbase:meta table:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=10, exceptions:
Wed Jul 16 14:32:49 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405,java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString
Wed Jul 16 14:32:52 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:32:57 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:33:07 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:33:28 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:34:08 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:35:49 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:37:29 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:39:10 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
Wed Jul 16 14:40:50 CST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@52380405,java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:705)
at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:144)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1102)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1162)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1054)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1011)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:326)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:192)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150)
at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:206)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:455)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
at mapreduce.ImportFromFile.run(ImportFromFile.java:228)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at mapreduce.ImportFromFile.main(ImportFromFile.java:185)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152)
at mapreduce.Driver.main(Driver.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
at org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:897)
at org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest(RequestConverter.java:131)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1402)
at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:701)
at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:699)
at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
... 38 more
Exception in thread "main" java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
at org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:897)
at org.apache.hadoop.hbase.protobuf.RequestConverter.buildGetRowOrBeforeRequest(RequestConverter.java:131)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1402)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1176)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1054)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1011)
at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:326)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:192)
at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150)
at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:206)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:455)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
at mapreduce.ImportFromFile.run(ImportFromFile.java:228)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at mapreduce.ImportFromFile.main(ImportFromFile.java:185)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:152)
at mapreduce.Driver.main(Driver.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
相关推荐
NULL 博文链接：https://bnmnba.iteye.com/blog/2322332
2. http://labs.google.com/papers/bigtable.html 3. http://hadoop.apache.org 4. http://www.apache.org/dyn/closer.cgi/hbase/ 5. http://hbase.apache.org/docs/current/source-repository.html 6. ...
Google的核心竞争技术是它的计算平台。Google的大牛们用了下面5篇文章，介绉了它们的计算设施。 GoogleCluster： ...http://labs.google.com/papers/mapreduce.html 径快，Apache上就出现了一个类似的解决方案，目前...
https://dlcdn.apache.org/hbase/2.4.11/ hbase-2.4.11 稳定版安装部署包
本文将HBase-2.2.1安装在Hadoop-3.1.2上，关于Hadoop-3.1.2的安装，请参见《基于zookeeper-3.5.5安装hadoop-3.1.2》一文。安装环境为64位CentOS-Linux 7.2版本。 本文将在HBase官方提供的quickstart.html文件的指导...
HBase 元数据修复工具包。 ①修改 jar 包中的application.properties，重点是 zookeeper.address、zookeeper.nodeParent、hdfs....③开始修复 `java -jar -Drepair.tableName=表名 hbase-meta-repair-hbase-2.0.2.jar`
https://www.tutorialspoint.com/apache_drill/apache_drill_pdf_version.htm https://www.tutorialspoint.com/apache_drill/apache_drill_pdf_version.htm ...
资源分类：Python库 所属语言：Python 资源全名：thumbor_hbase-0.11.tar.gz 资源来源：官方 安装方法：https://lanzao.blog.csdn.net/article/details/101784059
hbase-1.2.6.1-bin.tar.gz,hbase-1.2.6.1-bin.tar.gz,hbase-1.2.6.1-bin.tar.gz,hbase-1.2.6.1-bin.tar.gz,hbase-1.2.6.1-bin.tar.gz,hbase-1.2.6.1-bin.tar.gz,hbase-1.2.6.1-bin.tar.gz,hbase-1.2.6.1-bin.tar.gz
hbase-0.90.5.tar.gz与hadoop0.20.2版本匹配，我在我本地虚拟机已经安装成功可以使用。请放心下载！！！
/root/hbase-0.98.12.1-hadoop2/bin/./start-hbase.sh 12、通过浏览器访问hbase管理页面 Node11:60010 Node12:60010 Node11:50070 Node12:50070 13、为保证集群的可靠性，要启动多个HMaster hbase-daemon.sh ...
hbase-0.94.27.tar.gz
搭建pinpoint需要的hbase初始化脚本hbase-create.hbase
hbase-1.0.1.1-bin.tar.gz，官方版本，请尽情下载使用。
首先下载hbase安装包 wget http://mirror.bit.edu.cn/apache/hbase/stable/hbase-2.2.3-bin.tar.gz 解压压缩包 tar -zxvf hbase-2.2.3-bin.tar.gz 修改/opt/hbase-2.2.3/conf/hbse-env.sh文件 第一步 设置javahome ...
ambari-2.7.5 编译过程中四个大包下载很慢，所以需要提前下载，包含：hbase-2.0.2.3.1.4.0-315-bin.tar.gz ，hadoop-3.1.1.3.1.4.0-315.tar.gz ， grafana-6.4.2.linux-amd64.tar.gz ，phoenix-5.0.0.3.1.4.0-315....