Skip to content
Snippets Groups Projects
  • Nicholas Chammas's avatar
    f17d43b0
    [SPARK-6219] [Build] Check that Python code compiles · f17d43b0
    Nicholas Chammas authored
    This PR expands the Python lint checks so that they check for obvious compilation errors in our Python code.
    
    For example:
    
    ```
    $ ./dev/lint-python
    Python lint checks failed.
    Compiling ./ec2/spark_ec2.py ...
      File "./ec2/spark_ec2.py", line 618
        return (master_nodes,, slave_nodes)
                             ^
    SyntaxError: invalid syntax
    
    ./ec2/spark_ec2.py:618:25: E231 missing whitespace after ','
    ./ec2/spark_ec2.py:1117:101: E501 line too long (102 > 100 characters)
    ```
    
    This PR also bumps up the version of `pep8`. It ignores new types of checks introduced by that version bump while fixing problems missed by the older version of `pep8` we were using.
    
    Author: Nicholas Chammas <nicholas.chammas@gmail.com>
    
    Closes #4941 from nchammas/compile-spark-ec2 and squashes the following commits:
    
    75e31d8 [Nicholas Chammas] upgrade pep8 + check compile
    b33651c [Nicholas Chammas] PEP8 line length
    f17d43b0
    History
    [SPARK-6219] [Build] Check that Python code compiles
    Nicholas Chammas authored
    This PR expands the Python lint checks so that they check for obvious compilation errors in our Python code.
    
    For example:
    
    ```
    $ ./dev/lint-python
    Python lint checks failed.
    Compiling ./ec2/spark_ec2.py ...
      File "./ec2/spark_ec2.py", line 618
        return (master_nodes,, slave_nodes)
                             ^
    SyntaxError: invalid syntax
    
    ./ec2/spark_ec2.py:618:25: E231 missing whitespace after ','
    ./ec2/spark_ec2.py:1117:101: E501 line too long (102 > 100 characters)
    ```
    
    This PR also bumps up the version of `pep8`. It ignores new types of checks introduced by that version bump while fixing problems missed by the older version of `pep8` we were using.
    
    Author: Nicholas Chammas <nicholas.chammas@gmail.com>
    
    Closes #4941 from nchammas/compile-spark-ec2 and squashes the following commits:
    
    75e31d8 [Nicholas Chammas] upgrade pep8 + check compile
    b33651c [Nicholas Chammas] PEP8 line length