Skip to content
Snippets Groups Projects
  • Andrew Or's avatar
    41dc5987
    [SPARK-3264] Allow users to set executor Spark home in Mesos · 41dc5987
    Andrew Or authored
    The executors and the driver may not share the same Spark home. There is currently one way to set the executor side Spark home in Mesos, through setting `spark.home`. However, this is neither documented nor intuitive. This PR adds a more specific config `spark.mesos.executor.home` and exposes this to the user.
    
    liancheng tnachen
    
    Author: Andrew Or <andrewor14@gmail.com>
    
    Closes #2166 from andrewor14/mesos-spark-home and squashes the following commits:
    
    b87965e [Andrew Or] Merge branch 'master' of github.com:apache/spark into mesos-spark-home
    f6abb2e [Andrew Or] Document spark.mesos.executor.home
    ca7846d [Andrew Or] Add more specific configuration for executor Spark home in Mesos
    41dc5987
    History
    [SPARK-3264] Allow users to set executor Spark home in Mesos
    Andrew Or authored
    The executors and the driver may not share the same Spark home. There is currently one way to set the executor side Spark home in Mesos, through setting `spark.home`. However, this is neither documented nor intuitive. This PR adds a more specific config `spark.mesos.executor.home` and exposes this to the user.
    
    liancheng tnachen
    
    Author: Andrew Or <andrewor14@gmail.com>
    
    Closes #2166 from andrewor14/mesos-spark-home and squashes the following commits:
    
    b87965e [Andrew Or] Merge branch 'master' of github.com:apache/spark into mesos-spark-home
    f6abb2e [Andrew Or] Document spark.mesos.executor.home
    ca7846d [Andrew Or] Add more specific configuration for executor Spark home in Mesos