Skip to content
Snippets Groups Projects
  • cocoatomo's avatar
    fdcad6ef
    [SPARK-8763] [PYSPARK] executing run-tests.py with Python 2.6 fails with... · fdcad6ef
    cocoatomo authored
    [SPARK-8763] [PYSPARK] executing run-tests.py with Python 2.6 fails with absence of subprocess.check_output function
    
    Running run-tests.py with Python 2.6 cause following error:
    
    ```
    Running PySpark tests. Output is in python//Users/tomohiko/.jenkins/jobs/pyspark_test/workspace/python/unit-tests.log
    Will test against the following Python executables: ['python2.6', 'python3.4', 'pypy']
    Will test the following Python modules: ['pyspark-core', 'pyspark-ml', 'pyspark-mllib', 'pyspark-sql', 'pyspark-streaming']
    Traceback (most recent call last):
      File "./python/run-tests.py", line 196, in <module>
        main()
      File "./python/run-tests.py", line 159, in main
        python_implementation = subprocess.check_output(
    AttributeError: 'module' object has no attribute 'check_output'
    ...
    ```
    
    The cause of this error is using subprocess.check_output function, which exists since Python 2.7.
    (ref. https://docs.python.org/2.7/library/subprocess.html#subprocess.check_output)
    
    Author: cocoatomo <cocoatomo77@gmail.com>
    
    Closes #7161 from cocoatomo/issues/8763-test-fails-py26 and squashes the following commits:
    
    cf4f901 [cocoatomo] [SPARK-8763] backport process.check_output function from Python 2.7
    fdcad6ef
    History
    [SPARK-8763] [PYSPARK] executing run-tests.py with Python 2.6 fails with...
    cocoatomo authored
    [SPARK-8763] [PYSPARK] executing run-tests.py with Python 2.6 fails with absence of subprocess.check_output function
    
    Running run-tests.py with Python 2.6 cause following error:
    
    ```
    Running PySpark tests. Output is in python//Users/tomohiko/.jenkins/jobs/pyspark_test/workspace/python/unit-tests.log
    Will test against the following Python executables: ['python2.6', 'python3.4', 'pypy']
    Will test the following Python modules: ['pyspark-core', 'pyspark-ml', 'pyspark-mllib', 'pyspark-sql', 'pyspark-streaming']
    Traceback (most recent call last):
      File "./python/run-tests.py", line 196, in <module>
        main()
      File "./python/run-tests.py", line 159, in main
        python_implementation = subprocess.check_output(
    AttributeError: 'module' object has no attribute 'check_output'
    ...
    ```
    
    The cause of this error is using subprocess.check_output function, which exists since Python 2.7.
    (ref. https://docs.python.org/2.7/library/subprocess.html#subprocess.check_output)
    
    Author: cocoatomo <cocoatomo77@gmail.com>
    
    Closes #7161 from cocoatomo/issues/8763-test-fails-py26 and squashes the following commits:
    
    cf4f901 [cocoatomo] [SPARK-8763] backport process.check_output function from Python 2.7