-
- Downloads
Various fixes to configuration code
- Got rid of global SparkContext.globalConf - Pass SparkConf to serializers and compression codecs - Made SparkConf public instead of private[spark] - Improved API of SparkContext and SparkConf - Switched executor environment vars to be passed through SparkConf - Fixed some places that were still using system properties - Fixed some tests, though others are still failing This still fails several tests in core, repl and streaming, likely due to properties not being set or cleared correctly (some of the tests run fine in isolation).
Showing
- core/src/main/scala/org/apache/spark/Accumulators.scala 4 additions, 4 deletionscore/src/main/scala/org/apache/spark/Accumulators.scala
- core/src/main/scala/org/apache/spark/MapOutputTracker.scala 2 additions, 2 deletionscore/src/main/scala/org/apache/spark/MapOutputTracker.scala
- core/src/main/scala/org/apache/spark/Partitioner.scala 2 additions, 4 deletionscore/src/main/scala/org/apache/spark/Partitioner.scala
- core/src/main/scala/org/apache/spark/SparkConf.scala 123 additions, 35 deletionscore/src/main/scala/org/apache/spark/SparkConf.scala
- core/src/main/scala/org/apache/spark/SparkContext.scala 76 additions, 62 deletionscore/src/main/scala/org/apache/spark/SparkContext.scala
- core/src/main/scala/org/apache/spark/SparkEnv.scala 6 additions, 5 deletionscore/src/main/scala/org/apache/spark/SparkEnv.scala
- core/src/main/scala/org/apache/spark/api/java/JavaSparkContext.scala 14 additions, 1 deletion...in/scala/org/apache/spark/api/java/JavaSparkContext.scala
- core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala 3 additions, 3 deletions...rc/main/scala/org/apache/spark/api/python/PythonRDD.scala
- core/src/main/scala/org/apache/spark/broadcast/HttpBroadcast.scala 20 additions, 13 deletions...main/scala/org/apache/spark/broadcast/HttpBroadcast.scala
- core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala 19 additions, 19 deletions...n/scala/org/apache/spark/broadcast/TorrentBroadcast.scala
- core/src/main/scala/org/apache/spark/deploy/ApplicationDescription.scala 1 addition, 1 deletion...cala/org/apache/spark/deploy/ApplicationDescription.scala
- core/src/main/scala/org/apache/spark/deploy/LocalSparkCluster.scala 4 additions, 3 deletions...ain/scala/org/apache/spark/deploy/LocalSparkCluster.scala
- core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala 8 additions, 6 deletions.../main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala
- core/src/main/scala/org/apache/spark/deploy/client/TestClient.scala 5 additions, 4 deletions...ain/scala/org/apache/spark/deploy/client/TestClient.scala
- core/src/main/scala/org/apache/spark/deploy/master/Master.scala 20 additions, 16 deletions...rc/main/scala/org/apache/spark/deploy/master/Master.scala
- core/src/main/scala/org/apache/spark/deploy/master/SparkZooKeeperSession.scala 1 addition, 1 deletion...rg/apache/spark/deploy/master/SparkZooKeeperSession.scala
- core/src/main/scala/org/apache/spark/deploy/master/ZooKeeperLeaderElectionAgent.scala 1 addition, 1 deletion...he/spark/deploy/master/ZooKeeperLeaderElectionAgent.scala
- core/src/main/scala/org/apache/spark/deploy/master/ZooKeeperPersistenceEngine.scala 1 addition, 1 deletion...ache/spark/deploy/master/ZooKeeperPersistenceEngine.scala
- core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala 8 additions, 6 deletions...rc/main/scala/org/apache/spark/deploy/worker/Worker.scala
- core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala 2 additions, 2 deletions.../apache/spark/executor/CoarseGrainedExecutorBackend.scala
Loading
Please register or sign in to comment