How to access Namenode UI with KnoxSSO

Question

How to access Namenode UI with KnoxSSO

540 Views Asked by venom13k At 20 October 2024 at 22:47

My goal

I want to access Namenode UI through KnoxSSO and browse hdfs files without Ambari.

What have I already done

I've already set up Security cluster and HDFS HA with QJM.

Apache Knox uses Pac4J provider to authenticate users with Openid Connect.

There are 3 instances of Apache Knox beyond Nginx reverse proxy.

I've also have several services working fine with KnoxSSO:

Resourcemanager UI
Jobhistory UI
WebHDFS
Apache Zeppelin
Spark History Server

The problem

Despite of all other services are accessable by their UI through knox, I can't achieve it with Namenode IU (HDFSUI). I folowed instructions by official documentation but when I try to follow hdfsui link it's cause probably infinite loop redirecting and show: HTTP ERROR 500 java.io.IOException: Service connectivity error.

In the gateway.log file I have numerous exceptions, like these:

2022-03-19 18:15:35,509 WARN  knox.gateway (IdentityAsserterHttpServletRequestWrapper.java:scrubOfExistingPrincipalParams(202)) - Possible identity spoofing attempt - impersonation parameter removed: doAs
2022-03-19 18:15:35,521 WARN  knox.gateway (DefaultDispatch.java:executeOutboundRequest(183)) - Connection exception dispatching request: https://my.hadoop.domain:443/gateway/default/hdfs?doAs=username java.net.SocketTimeoutException: Read timed out
java.net.SocketTimeoutException: Read timed out
        at java.base/java.net.SocketInputStream.socketRead0(Native Method)
        at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
        at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
        at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
        at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
        at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)
        at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1374)
        at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:985)
        at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
        at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
        at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
        at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
        at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
        at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
        at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
        at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
        at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
        at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:166)
        at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:152)
        at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequestWrapper(DefaultDispatch.java:135)
        at org.apache.knox.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:300)
        at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:174)
        at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:125)
        at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
        at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:193)
        at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:149)
        at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:94)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
        at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:57)
        at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
        at org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter$1.run(AbstractJWTFilter.java:249)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
        at org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter.continueWithEstablishedSecurityContext(AbstractJWTFilter.java:244)
        at org.apache.knox.gateway.provider.federation.jwt.filter.SSOCookieFederationFilter.doFilter(SSOCookieFederationFilter.java:164)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
        at org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:50)
        at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
        at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:166)
        at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:93)
        at org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:135)
        at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1443)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
        at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
        at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:228)
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:106)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.websocket.server.WebSocketHandler.handle(WebSocketHandler.java:115)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:516)
        at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
        at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
        at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
        at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
        at java.base/java.lang.Thread.run(Thread.java:829)
2022-03-19 18:15:35,522 ERROR knox.gateway (AbstractGatewayFilter.java:doFilter(60)) - Failed to execute filter: java.io.IOException: Service connectivity error.
2022-03-19 18:15:35,522 ERROR knox.gateway (AbstractGatewayFilter.java:doFilter(60)) - Failed to execute filter: java.io.IOException: Service connectivity error.
2022-03-19 18:15:35,522 ERROR knox.gateway (AbstractGatewayFilter.java:doFilter(60)) - Failed to execute filter: java.io.IOException: Service connectivity error.
2022-03-19 18:15:35,523 ERROR knox.gateway (GatewayFilter.java:doFilter(168)) - Gateway processing failed: java.io.IOException: Service connectivity error.
java.io.IOException: Service connectivity error.
        at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:184)
        at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:152)
        at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequestWrapper(DefaultDispatch.java:135)
        at org.apache.knox.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:300)
        at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:174)
        at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:125)
        at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
        at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:193)
        at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:149)
        at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:94)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
        at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:57)
        at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
        at org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:50)
        at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
        at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
        at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
        at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:166)
        at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:93)
        at org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:135)
        at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1443)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
        at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
        at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:228)
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582)
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:106)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.websocket.server.WebSocketHandler.handle(WebSocketHandler.java:115)
        at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
        at org.eclipse.jetty.server.Server.handle(Server.java:516)
        at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
        at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
        at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
        at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
        at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
        at java.base/java.lang.Thread.run(Thread.java:829)

In debug logs I saw some strange headers like that

X-Forwarded-Context: /gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default

Could anyone know, what's going wrong?

Original Q&A

There are 1 best solutions below

**pyOwner** · Answer 1

Try to add the following line for HDFSUI role in your topology file:

<version>2.7.0</version>

It should be look like this:

<service>
    <role>HDFSUI</role>
    <version>2.7.0</version>
    <url>http://NAMENODE:50070</url>
</service>

https://issues.apache.org/jira/browse/KNOX-1644

How to access Namenode UI with KnoxSSO

My goal

What have I already done

The problem

There are 1 best solutions below

Related Questions in HADOOP

Related Questions in HDFS

Related Questions in NAMENODE

Related Questions in APACHE-KNOX

Trending Questions

Popular # Hahtags

Popular Questions