-
- Downloads
SPARK-1352: Improve robustness of spark-submit script
1. Better error messages when required arguments are missing. 2. Support for unit testing cases where presented arguments are invalid. 3. Bug fix: Only use environment varaibles when they are set (otherwise will cause NPE). 4. A verbose mode to aid debugging. 5. Visibility of several variables is set to private. 6. Deprecation warning for existing scripts. Author: Patrick Wendell <pwendell@gmail.com> Closes #271 from pwendell/spark-submit and squashes the following commits: 9146def [Patrick Wendell] SPARK-1352: Improve robustness of spark-submit script
Showing
- core/src/main/scala/org/apache/spark/deploy/Client.scala 3 additions, 0 deletionscore/src/main/scala/org/apache/spark/deploy/Client.scala
- core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala 43 additions, 24 deletions.../src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
- core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala 52 additions, 22 deletions.../scala/org/apache/spark/deploy/SparkSubmitArguments.scala
- core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala 59 additions, 2 deletions...test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
- yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/Client.scala 3 additions, 0 deletions.../src/main/scala/org/apache/spark/deploy/yarn/Client.scala
- yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala 3 additions, 0 deletions.../src/main/scala/org/apache/spark/deploy/yarn/Client.scala
Loading
Please register or sign in to comment