Skip to content
  • Josh Rosen's avatar
    b9573263
    Do not inherit master's PYTHONPATH on workers. · b9573263
    Josh Rosen authored
    This fixes SPARK-832, an issue where PySpark
    would not work when the master and workers used
    different SPARK_HOME paths.
    
    This change may potentially break code that relied
    on the master's PYTHONPATH being used on workers.
    To have custom PYTHONPATH additions used on the
    workers, users should set a custom PYTHONPATH in
    spark-env.sh rather than setting it in the shell.
    b9573263
    Do not inherit master's PYTHONPATH on workers.
    Josh Rosen authored
    This fixes SPARK-832, an issue where PySpark
    would not work when the master and workers used
    different SPARK_HOME paths.
    
    This change may potentially break code that relied
    on the master's PYTHONPATH being used on workers.
    To have custom PYTHONPATH additions used on the
    workers, users should set a custom PYTHONPATH in
    spark-env.sh rather than setting it in the shell.
Loading