Skip to content
Snippets Groups Projects
  • Davies Liu's avatar
    7fe08b43
    [SPARK-4415] [PySpark] JVM should exit after Python exit · 7fe08b43
    Davies Liu authored
    When JVM is started in a Python process, it should exit once the stdin is closed.
    
    test: add spark.driver.memory in conf/spark-defaults.conf
    
    ```
    daviesdm:~/work/spark$ cat conf/spark-defaults.conf
    spark.driver.memory       8g
    daviesdm:~/work/spark$ bin/pyspark
    >>> quit
    daviesdm:~/work/spark$ jps
    4931 Jps
    286
    daviesdm:~/work/spark$ python wc.py
    943738
    0.719928026199
    daviesdm:~/work/spark$ jps
    286
    4990 Jps
    ```
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #3274 from davies/exit and squashes the following commits:
    
    df0e524 [Davies Liu] address comments
    ce8599c [Davies Liu] address comments
    050651f [Davies Liu] JVM should exit after Python exit
    7fe08b43
    History
    [SPARK-4415] [PySpark] JVM should exit after Python exit
    Davies Liu authored
    When JVM is started in a Python process, it should exit once the stdin is closed.
    
    test: add spark.driver.memory in conf/spark-defaults.conf
    
    ```
    daviesdm:~/work/spark$ cat conf/spark-defaults.conf
    spark.driver.memory       8g
    daviesdm:~/work/spark$ bin/pyspark
    >>> quit
    daviesdm:~/work/spark$ jps
    4931 Jps
    286
    daviesdm:~/work/spark$ python wc.py
    943738
    0.719928026199
    daviesdm:~/work/spark$ jps
    286
    4990 Jps
    ```
    
    Author: Davies Liu <davies@databricks.com>
    
    Closes #3274 from davies/exit and squashes the following commits:
    
    df0e524 [Davies Liu] address comments
    ce8599c [Davies Liu] address comments
    050651f [Davies Liu] JVM should exit after Python exit