Skip to content
Snippets Groups Projects
  • Nathan Howell's avatar
    23146a67
    spark-assembly.jar fails to authenticate with YARN ResourceManager · 23146a67
    Nathan Howell authored
    sbt-assembly is setup to pick the first META-INF/services/org.apache.hadoop.security.SecurityInfo file instead of merging them. This causes Kerberos authentication to fail, this manifests itself in the "info:null" debug log statement:
    
        DEBUG SaslRpcClient: Get token info proto:interface org.apache.hadoop.yarn.api.ApplicationClientProtocolPB info:null
        DEBUG SaslRpcClient: Get kerberos info proto:interface org.apache.hadoop.yarn.api.ApplicationClientProtocolPB info:null
        ERROR UserGroupInformation: PriviledgedActionException as:foo@BAR (auth:KERBEROS) cause:org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
        DEBUG UserGroupInformation: PrivilegedAction as:foo@BAR (auth:KERBEROS) from:org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:583)
        WARN Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
        ERROR UserGroupInformation: PriviledgedActionException as:foo@BAR (auth:KERBEROS) cause:java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
    
    This previously would just contain a single class:
    
    $ unzip -c assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-SNAPSHOT-hadoop2.2.0.jar META-INF/services/org.apache.hadoop.security.SecurityInfo
    Archive:  assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-SNAPSHOT-hadoop2.2.0.jar
      inflating: META-INF/services/org.apache.hadoop.security.SecurityInfo
    
        org.apache.hadoop.security.AnnotatedSecurityInfo
    
    And now has the full list of classes:
    
    $ unzip -c assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-SNAPSHOT-hadoop2.2.0.jar META-INF/services/org.apache.hadoop.security.SecurityInfoArchive:  assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-SNAPSHOT-hadoop2.2.0.jar
      inflating: META-INF/services/org.apache.hadoop.security.SecurityInfo
    
        org.apache.hadoop.security.AnnotatedSecurityInfo
        org.apache.hadoop.mapreduce.v2.app.MRClientSecurityInfo
        org.apache.hadoop.mapreduce.v2.security.client.ClientHSSecurityInfo
        org.apache.hadoop.yarn.security.client.ClientRMSecurityInfo
        org.apache.hadoop.yarn.security.ContainerManagerSecurityInfo
        org.apache.hadoop.yarn.security.SchedulerSecurityInfo
        org.apache.hadoop.yarn.security.admin.AdminSecurityInfo
        org.apache.hadoop.yarn.server.RMNMSecurityInfoClass
    23146a67
    History
    spark-assembly.jar fails to authenticate with YARN ResourceManager
    Nathan Howell authored
    sbt-assembly is setup to pick the first META-INF/services/org.apache.hadoop.security.SecurityInfo file instead of merging them. This causes Kerberos authentication to fail, this manifests itself in the "info:null" debug log statement:
    
        DEBUG SaslRpcClient: Get token info proto:interface org.apache.hadoop.yarn.api.ApplicationClientProtocolPB info:null
        DEBUG SaslRpcClient: Get kerberos info proto:interface org.apache.hadoop.yarn.api.ApplicationClientProtocolPB info:null
        ERROR UserGroupInformation: PriviledgedActionException as:foo@BAR (auth:KERBEROS) cause:org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
        DEBUG UserGroupInformation: PrivilegedAction as:foo@BAR (auth:KERBEROS) from:org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:583)
        WARN Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
        ERROR UserGroupInformation: PriviledgedActionException as:foo@BAR (auth:KERBEROS) cause:java.io.IOException: org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS]
    
    This previously would just contain a single class:
    
    $ unzip -c assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-SNAPSHOT-hadoop2.2.0.jar META-INF/services/org.apache.hadoop.security.SecurityInfo
    Archive:  assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-SNAPSHOT-hadoop2.2.0.jar
      inflating: META-INF/services/org.apache.hadoop.security.SecurityInfo
    
        org.apache.hadoop.security.AnnotatedSecurityInfo
    
    And now has the full list of classes:
    
    $ unzip -c assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-SNAPSHOT-hadoop2.2.0.jar META-INF/services/org.apache.hadoop.security.SecurityInfoArchive:  assembly/target/scala-2.10/spark-assembly-0.9.0-incubating-SNAPSHOT-hadoop2.2.0.jar
      inflating: META-INF/services/org.apache.hadoop.security.SecurityInfo
    
        org.apache.hadoop.security.AnnotatedSecurityInfo
        org.apache.hadoop.mapreduce.v2.app.MRClientSecurityInfo
        org.apache.hadoop.mapreduce.v2.security.client.ClientHSSecurityInfo
        org.apache.hadoop.yarn.security.client.ClientRMSecurityInfo
        org.apache.hadoop.yarn.security.ContainerManagerSecurityInfo
        org.apache.hadoop.yarn.security.SchedulerSecurityInfo
        org.apache.hadoop.yarn.security.admin.AdminSecurityInfo
        org.apache.hadoop.yarn.server.RMNMSecurityInfoClass