-
- Downloads
[SPARK-1617] and [SPARK-1618] Improvements to streaming ui and bug fix to socket receiver
1617: These changes expose the receiver state (active or inactive) and last error in the UI 1618: If the socket receiver cannot connect in the first attempt, it should try to restart after a delay. That was broken, as the thread that restarts (hence, stops) the receiver waited on Thread.join on itself! Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #540 from tdas/streaming-ui-fix and squashes the following commits: e469434 [Tathagata Das] Merge remote-tracking branch 'apache-github/master' into streaming-ui-fix dbddf75 [Tathagata Das] Style fix. 66df1a5 [Tathagata Das] Merge remote-tracking branch 'apache/master' into streaming-ui-fix ad98bc9 [Tathagata Das] Refactored streaming listener to use ReceiverInfo. d7f849c [Tathagata Das] Revert "Moved BatchInfo from streaming.scheduler to streaming.ui" 5c80919 [Tathagata Das] Moved BatchInfo from streaming.scheduler to streaming.ui da244f6 [Tathagata Das] Fixed socket receiver as well as made receiver state and error visible in the streamign UI.
Showing
- core/src/main/scala/org/apache/spark/ui/UIUtils.scala 1 addition, 1 deletioncore/src/main/scala/org/apache/spark/ui/UIUtils.scala
- streaming/src/main/scala/org/apache/spark/streaming/dstream/SocketInputDStream.scala 21 additions, 28 deletions...g/apache/spark/streaming/dstream/SocketInputDStream.scala
- streaming/src/main/scala/org/apache/spark/streaming/receiver/ActorReceiver.scala 10 additions, 2 deletions...a/org/apache/spark/streaming/receiver/ActorReceiver.scala
- streaming/src/main/scala/org/apache/spark/streaming/receiver/Receiver.scala 4 additions, 1 deletion.../scala/org/apache/spark/streaming/receiver/Receiver.scala
- streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverMessage.scala 2 additions, 2 deletions...org/apache/spark/streaming/receiver/ReceiverMessage.scala
- streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverSupervisor.scala 34 additions, 24 deletions.../apache/spark/streaming/receiver/ReceiverSupervisor.scala
- streaming/src/main/scala/org/apache/spark/streaming/receiver/ReceiverSupervisorImpl.scala 15 additions, 9 deletions...che/spark/streaming/receiver/ReceiverSupervisorImpl.scala
- streaming/src/main/scala/org/apache/spark/streaming/scheduler/BatchInfo.scala 3 additions, 0 deletions...cala/org/apache/spark/streaming/scheduler/BatchInfo.scala
- streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverInfo.scala 37 additions, 0 deletions...a/org/apache/spark/streaming/scheduler/ReceiverInfo.scala
- streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala 25 additions, 15 deletions...rg/apache/spark/streaming/scheduler/ReceiverTracker.scala
- streaming/src/main/scala/org/apache/spark/streaming/scheduler/StreamingListener.scala 22 additions, 3 deletions.../apache/spark/streaming/scheduler/StreamingListener.scala
- streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingJobProgressListener.scala 15 additions, 3 deletions...che/spark/streaming/ui/StreamingJobProgressListener.scala
- streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala 14 additions, 6 deletions...n/scala/org/apache/spark/streaming/ui/StreamingPage.scala
- streaming/src/test/scala/org/apache/spark/streaming/NetworkReceiverSuite.scala 6 additions, 2 deletions...ala/org/apache/spark/streaming/NetworkReceiverSuite.scala
- streaming/src/test/scala/org/apache/spark/streaming/StreamingListenerSuite.scala 8 additions, 7 deletions...a/org/apache/spark/streaming/StreamingListenerSuite.scala
Loading
Please register or sign in to comment