Skip to content
Snippets Groups Projects
Commit 792d9084 authored by Patrick Wendell's avatar Patrick Wendell
Browse files

Merge pull request #470 from tgravescs/fix_spark_examples_yarn

Only log error on missing jar to allow spark examples to jar.

Right now to run the spark examples on Yarn you have to use the --addJars option and put the jar in hdfs.  To make that nicer  so the user doesn't have to specify the --addJars option change it to simply log an error instead of throwing.
parents 256a3553 dd56b212
No related branches found
No related tags found
No related merge requests found
......@@ -708,8 +708,11 @@ class SparkContext(
env.httpFileServer.addJar(new File(fileName))
} catch {
case e: Exception => {
// For now just log an error but allow to go through so spark examples work.
// The spark examples don't really need the jar distributed since its also
// the app jar.
logError("Error adding jar (" + e + "), was the --addJars option used?")
throw e
null
}
}
} else {
......@@ -722,8 +725,10 @@ class SparkContext(
path
}
}
addedJars(key) = System.currentTimeMillis
logInfo("Added JAR " + path + " at " + key + " with timestamp " + addedJars(key))
if (key != null) {
addedJars(key) = System.currentTimeMillis
logInfo("Added JAR " + path + " at " + key + " with timestamp " + addedJars(key))
}
}
}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment