-
- Downloads
[SPARK-7767] [STREAMING] Added test for checkpoint serialization in StreamingContext.start()
Currently, the background checkpointing thread fails silently if the checkpoint is not serializable. It is hard to debug and therefore its best to fail fast at `start()` when checkpointing is enabled and the checkpoint is not serializable. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #6292 from tdas/SPARK-7767 and squashes the following commits: 51304e6 [Tathagata Das] Addressed comments. c35237b [Tathagata Das] Added test for checkpoint serialization in StreamingContext.start()
Showing
- core/src/main/scala/org/apache/spark/serializer/SerializationDebugger.scala 1 addition, 1 deletion...a/org/apache/spark/serializer/SerializationDebugger.scala
- streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala 42 additions, 28 deletions...rc/main/scala/org/apache/spark/streaming/Checkpoint.scala
- streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala 23 additions, 3 deletions...n/scala/org/apache/spark/streaming/StreamingContext.scala
- streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala 23 additions, 4 deletions...la/org/apache/spark/streaming/StreamingContextSuite.scala
Loading
Please register or sign in to comment