Skip to content
Snippets Groups Projects
  • Davies Liu's avatar
    ccd5ab5f
    [SPARK-2580] [PySpark] keep silent in worker if JVM close the socket · ccd5ab5f
    Davies Liu authored
    During rdd.take(n), JVM will close the socket if it had got enough data, the Python worker should keep silent in this case.
    
    In the same time, the worker should not print the trackback into stderr if it send the traceback to JVM successfully.
    
    Author: Davies Liu <davies.liu@gmail.com>
    
    Closes #1625 from davies/error and squashes the following commits:
    
    4fbcc6d [Davies Liu] disable log4j during testing when exception is expected.
    cc14202 [Davies Liu] keep silent in worker if JVM close the socket
    ccd5ab5f
    History
    [SPARK-2580] [PySpark] keep silent in worker if JVM close the socket
    Davies Liu authored
    During rdd.take(n), JVM will close the socket if it had got enough data, the Python worker should keep silent in this case.
    
    In the same time, the worker should not print the trackback into stderr if it send the traceback to JVM successfully.
    
    Author: Davies Liu <davies.liu@gmail.com>
    
    Closes #1625 from davies/error and squashes the following commits:
    
    4fbcc6d [Davies Liu] disable log4j during testing when exception is expected.
    cc14202 [Davies Liu] keep silent in worker if JVM close the socket