Hive darf nicht imitieren Bienenstock

Ich habe einen Hadoop-Cluster mit Hadoop 2.7.2 und Hive 2.1.0 konfiguriert.Hive darf nicht imitieren Bienenstock

ich Beeline bin mit verbinden mit dem folgenden Befehl hive:

beeline 
beeline> !connect jdbc:hive2://localhost:10000 
Enter username for jdbc:hive2://localhost:10000:

Zuerst geben Sie einen beliebigen Benutzernamen hive erfolgreich verbinden können. Nach der Konfiguration der folgenden Einstellung wird jedoch ein Fehler angezeigt.

//within hive-site.xml and hiveserver2-site.xml 
<property> 
<name>hive.server2.enable.doAs</name> 
<value>true</value> 
<description> 
    Setting this property to true will have HiveServer2 execute 
    Hive operations as the user making the calls to it. 
</description> 
</property> 

//within core-site.xml 
<property> 
<name>hadoop.proxyuser.hive.hosts</name> 
<value>*</value> 
</property> 

<property> 
    <name>hadoop.proxyuser.hive.groups</name> 
    <value>*</value> 
</property>

Ich habe hadoop Cluster neu gestartet, aber die folgende Meldung erscheint nach wie vor:

Error: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive (state=,code=0)

Die Debug-Ausgabe für hiveserver2 ist wie folgt:

16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive: starting, having connections 1 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive got value #-3 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: closing ipc connection to /10.104.90.40:8020: User: hive is not allowed to impersonate hive 
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.ipc.Client.call(Client.java:1475) 
     at org.apache.hadoop.ipc.Client.call(Client.java:1412) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) 
     at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
     at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) 
     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:674) 
     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513) 
     at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:165) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
     at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
     at com.sun.proxy.$Proxy45.open(Unknown Source) 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:327) 
     at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:279) 
     at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:189) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:414) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:310) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) 
     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
     at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) 
     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
     at java.lang.Thread.run(Thread.java:745) 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive: closed 
16/11/15 11:28:46 [IPC Client (241742811) connection to /10.104.90.40:8020 from hive]: DEBUG ipc.Client: IPC Client (241742811) connection to /10.104.90.40:8020 from hive: stopped, remaining connections 0 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: WARN service.CompositeService: Failed to open session 
java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
     at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
     at com.sun.proxy.$Proxy45.open(Unknown Source) 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:327) 
     at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:279) 
     at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:189) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:414) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:310) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) 
     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
     at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) 
     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
     at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513) 
     at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:165) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
     ... 21 more 
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.ipc.Client.call(Client.java:1475) 
     at org.apache.hadoop.ipc.Client.call(Client.java:1412) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) 
     at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
     at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) 
     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:674) 
     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) 
     ... 28 more 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG security.UserGroupInformation: PrivilegedAction as:hive (auth:PROXY) via hive (auth:SIMPLE) from:org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO session.SessionState: Updating thread name to 74dcfeae-04c1-494d-b491-df53f0d20039 HiveServer2-Handler-Pool: Thread-38 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO session.SessionState: Resetting thread name to HiveServer2-Handler-Pool: Thread-38 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG session.SessionState: Removing resource dir /tmp/hive_resources 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Trying to connect to metastore with URI thrift://10.104.90.40:9083 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Opened a connection to metastore, current connections: 3 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Connected to metastore. 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG metadata.Hive: Closing current thread's connection to Hive Metastore. 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: INFO hive.metastore: Closed a connection to metastore, current connections: 2 
16/11/15 11:28:46 [HiveServer2-Handler-Pool: Thread-38]: DEBUG ipc.Client: stopping client from cache: [email protected] 
16/11/15 11:28:47 [HiveServer2-Handler-Pool: Thread-38]: WARN thrift.ThriftCLIService: Error opening session: 
org.apache.hive.service.cli.HiveSQLException: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:336) 
     at org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:279) 
     at org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:189) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:414) 
     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:310) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1377) 
     at org.apache.hive.service.rpc.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1362) 
     at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
     at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
     at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56) 
     at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
     at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:89) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) 
     at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) 
     at com.sun.proxy.$Proxy45.open(Unknown Source) 
     at org.apache.hive.service.cli.session.SessionManager.createSession(SessionManager.java:327) 
     ... 13 more 
Caused by: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:513) 
     at org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:165) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78) 
     ... 21 more 
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: hive is not allowed to impersonate hive 
     at org.apache.hadoop.ipc.Client.call(Client.java:1475) 
     at org.apache.hadoop.ipc.Client.call(Client.java:1412) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) 
     at com.sun.proxy.$Proxy27.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) 
     at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
     at com.sun.proxy.$Proxy28.getFileInfo(Unknown Source) 
     at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305) 
     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) 
     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301) 
     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424) 
     at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:674) 
     at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:622) 
     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) 
     ... 28 more

Ich suchte online und die meisten Lösung gefunden Dieser Fehler bezieht sich auf die konfigurierte Einstellung, die das Problem verursacht.

Weiß jemand, wie man dieses Problem löst?

Update:

Nach einigen Versuchen, fand ich heraus, dass dieser Fehler zu beziehen, die Benutzerstruktur Dienst starten verwendet.

Bevor ich Benutzer Bienenstock habe, um Bienenstockmetastre und hiveserver2 zu beginnen, der zu die Fehlermeldung führt.

Aber mit Benutzer hadoop das ist der Benutzer, der hadoop namenode gestartet hat, um hiveserver2 loszuwerden der Fehlermeldung.

Ich habe nicht den Grund herausgefunden, warum die Verwendung anderer Benutzer eine temporäre Problemumgehung ist.

Quelle

2016-11-15 Heyang Wang

Warum möchten Sie den Metastore mit einem bestimmten Benutzer starten (warum nicht einfach Hadoop?). - Nur eine Vermutung, aber vielleicht definiert der Benutzer, der den Metastore startet, wo Configs/Logs abgelegt werden. Und für einige Benutzer existieren diese Orte möglicherweise nicht oder sind möglicherweise nicht genug zugänglich. –

Zuerst ist es, weil die meisten dieser Tutorial Online-Hive als Benutzer verwenden. Aber später, ich dachte, möglicherweise mit Bienenstock für alle Bienenstock-Dienste ist nicht unnötig sice Bienenstock Benutzer nur das Privileg auf Bienenstock Ordner haben, kann es nicht die Hadoop oder Funken Ordner zu vermasseln. –

Vielleicht erhalte ich aus Ihrem Kommentar die falsche Schlussfolgerung, aber wenn Sie sich Sorgen machen, auf die falschen Ordner zuzugreifen, würde ich Ihnen empfehlen, nur die Standardbenutzerkonten für die Ersteinrichtung zu verwenden und dann die Arbeit mit benutzerdefinierten Benutzerkonten durchzuführen . (Wie Wang und adminWang). –

Ich konfrontiert ein ähnliches Problem auch. Das habe ich getan, damit es funktioniert. Im Bienenstock-site.xml, haben Sie diese Eigenschaft:

<property> 
     <name>hive.conf.restricted.list</name> 
     <value>hive.security.authenticator.manager,hive.security.authorization.manager,**hive.users.in.admin.role**</value> 
    </property>

I entfernt nur der Teil oberhalb fett markiert: hive.users.in.admin.role.

Es begann für mich zu arbeiten.

Quelle

2017-02-08 07:22:41

Hive darf nicht imitieren Bienenstock

Antwort

Verwandte Themen