From 17eddb35a280e77da7520343e0bf2a86b329ed62 Mon Sep 17 00:00:00 2001
From: jerryshao <sshao@hortonworks.com>
Date: Tue, 28 Mar 2017 10:41:11 -0700
Subject: [PATCH] [SPARK-19995][YARN] Register tokens to current UGI to avoid
re-issuing of tokens in yarn client mode
## What changes were proposed in this pull request?
In the current Spark on YARN code, we will obtain tokens from provided services, but we're not going to add these tokens to the current user's credentials. This will make all the following operations to these services still require TGT rather than delegation tokens. This is unnecessary since we already got the tokens, also this will lead to failure in user impersonation scenario, because the TGT is granted by real user, not proxy user.
So here changing to put all the tokens to the current UGI, so that following operations to these services will honor tokens rather than TGT, and this will further handle the proxy user issue mentioned above.
## How was this patch tested?
Local verified in secure cluster.
vanzin tgravescs mridulm dongjoon-hyun please help to review, thanks a lot.
Author: jerryshao <sshao@hortonworks.com>
Closes #17335 from jerryshao/SPARK-19995.
---
.../src/main/scala/org/apache/spark/deploy/yarn/Client.scala | 3 +++
1 file changed, 3 insertions(+)
diff --git a/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala b/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
index ccb0f8fdbb..3218d22114 100644
--- a/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
+++ b/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
@@ -371,6 +371,9 @@ private[spark] class Client(
val nearestTimeOfNextRenewal = credentialManager.obtainCredentials(hadoopConf, credentials)
if (credentials != null) {
+ // Add credentials to current user's UGI, so that following operations don't need to use the
+ // Kerberos tgt to get delegations again in the client side.
+ UserGroupInformation.getCurrentUser.addCredentials(credentials)
logDebug(YarnSparkHadoopUtil.get.dumpTokens(credentials).mkString("\n"))
}
--
GitLab