Skip to content
Snippets Groups Projects
Commit a78a91f4 authored by Andrew Or's avatar Andrew Or
Browse files

Revert "[SPARK-12345][MESOS] Filter SPARK_HOME when submitting Spark jobs with Mesos cluster mode."

This reverts commit ad8c1f0b.
parent 8a9417bc
No related branches found
No related tags found
No related merge requests found
......@@ -94,12 +94,7 @@ private[mesos] class MesosSubmitRequestServlet(
val driverMemory = sparkProperties.get("spark.driver.memory")
val driverCores = sparkProperties.get("spark.driver.cores")
val appArgs = request.appArgs
// We don't want to pass down SPARK_HOME when launching Spark apps
// with Mesos cluster mode since it's populated by default on the client and it will
// cause spark-submit script to look for files in SPARK_HOME instead.
// We only need the ability to specify where to find spark-submit script
// which user can user spark.executor.home or spark.home configurations.
val environmentVariables = request.environmentVariables.filter(!_.equals("SPARK_HOME"))
val environmentVariables = request.environmentVariables
val name = request.sparkProperties.get("spark.app.name").getOrElse(mainClass)
// Construct driver description
......
......@@ -34,7 +34,7 @@ import org.apache.spark.util.Utils
/**
* Shared trait for implementing a Mesos Scheduler. This holds common state and helper
* methods the Mesos scheduler will use.
* methods and Mesos scheduler will use.
*/
private[mesos] trait MesosSchedulerUtils extends Logging {
// Lock used to wait for scheduler to be registered
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment