Skip to content
Snippets Groups Projects
Commit 12c24e78 authored by Charles Reiss's avatar Charles Reiss
Browse files

Set default uncaught exception handler to exit.

Among other things, should prevent OutOfMemoryErrors in some daemon threads
(such as the network manager) from causing a spark executor to enter a state
where it cannot make progress but does not report an error.
parent c23a74df
No related branches found
No related tags found
No related merge requests found
...@@ -68,7 +68,6 @@ object SparkEnv extends Logging { ...@@ -68,7 +68,6 @@ object SparkEnv extends Logging {
isMaster: Boolean, isMaster: Boolean,
isLocal: Boolean isLocal: Boolean
) : SparkEnv = { ) : SparkEnv = {
val (actorSystem, boundPort) = AkkaUtils.createActorSystem("spark", hostname, port) val (actorSystem, boundPort) = AkkaUtils.createActorSystem("spark", hostname, port)
// Bit of a hack: If this is the master and our port was 0 (meaning bind to any free port), // Bit of a hack: If this is the master and our port was 0 (meaning bind to any free port),
......
...@@ -43,6 +43,21 @@ private[spark] class Executor extends Logging { ...@@ -43,6 +43,21 @@ private[spark] class Executor extends Logging {
urlClassLoader = createClassLoader() urlClassLoader = createClassLoader()
Thread.currentThread.setContextClassLoader(urlClassLoader) Thread.currentThread.setContextClassLoader(urlClassLoader)
// Make any thread terminations due to uncaught exceptions kill the entire
// executor process to avoid surprising stalls.
Thread.setDefaultUncaughtExceptionHandler(
new Thread.UncaughtExceptionHandler {
override def uncaughtException(thread: Thread, exception: Throwable) {
try {
logError("Uncaught exception in thread " + thread, exception)
System.exit(1)
} catch {
case t: Throwable => System.exit(2)
}
}
}
)
// Initialize Spark environment (using system properties read above) // Initialize Spark environment (using system properties read above)
env = SparkEnv.createFromSystemProperties(slaveHostname, 0, false, false) env = SparkEnv.createFromSystemProperties(slaveHostname, 0, false, false)
SparkEnv.set(env) SparkEnv.set(env)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment