Skip to content
Snippets Groups Projects
  • Sandy Ryza's avatar
    9dd8b916
    SPARK-1252. On YARN, use container-log4j.properties for executors · 9dd8b916
    Sandy Ryza authored
    container-log4j.properties is a file that YARN provides so that containers can have log4j.properties distinct from that of the NodeManagers.
    
    Logs now go to syslog, and stderr and stdout just have the process's standard err and standard out.
    
    I tested this on pseudo-distributed clusters for both yarn (Hadoop 2.2) and yarn-alpha (Hadoop 0.23.7)/
    
    Author: Sandy Ryza <sandy@cloudera.com>
    
    Closes #148 from sryza/sandy-spark-1252 and squashes the following commits:
    
    c0043b8 [Sandy Ryza] Put log4j.properties file under common
    55823da [Sandy Ryza] Add license headers to new files
    10934b8 [Sandy Ryza] Add log4j-spark-container.properties and support SPARK_LOG4J_CONF
    e74450b [Sandy Ryza] SPARK-1252. On YARN, use container-log4j.properties for executors
    9dd8b916
    History
    SPARK-1252. On YARN, use container-log4j.properties for executors
    Sandy Ryza authored
    container-log4j.properties is a file that YARN provides so that containers can have log4j.properties distinct from that of the NodeManagers.
    
    Logs now go to syslog, and stderr and stdout just have the process's standard err and standard out.
    
    I tested this on pseudo-distributed clusters for both yarn (Hadoop 2.2) and yarn-alpha (Hadoop 0.23.7)/
    
    Author: Sandy Ryza <sandy@cloudera.com>
    
    Closes #148 from sryza/sandy-spark-1252 and squashes the following commits:
    
    c0043b8 [Sandy Ryza] Put log4j.properties file under common
    55823da [Sandy Ryza] Add license headers to new files
    10934b8 [Sandy Ryza] Add log4j-spark-container.properties and support SPARK_LOG4J_CONF
    e74450b [Sandy Ryza] SPARK-1252. On YARN, use container-log4j.properties for executors