-
- Downloads
[SPARK-12299][CORE] Remove history serving functionality from Master
Remove history server functionality from standalone Master. Previously, the Master process rebuilt a SparkUI once the application was completed which sometimes caused problems, such as OOM, when the application event log is large (see SPARK-6270). Keeping this functionality out of the Master will help to simplify the process and increase stability. Testing for this change included running core unit tests and manually running an application on a standalone cluster to verify that it completed successfully and that the Master UI functioned correctly. Also added 2 unit tests to verify killing an application and driver from MasterWebUI makes the correct request to the Master. Author: Bryan Cutler <cutlerb@gmail.com> Closes #10991 from BryanCutler/remove-history-master-SPARK-12299.
Showing
- core/src/main/scala/org/apache/spark/deploy/master/ApplicationInfo.scala 0 additions, 9 deletions...cala/org/apache/spark/deploy/master/ApplicationInfo.scala
- core/src/main/scala/org/apache/spark/deploy/master/Master.scala 1 addition, 108 deletions...rc/main/scala/org/apache/spark/deploy/master/Master.scala
- core/src/main/scala/org/apache/spark/deploy/master/MasterMessages.scala 0 additions, 2 deletions...scala/org/apache/spark/deploy/master/MasterMessages.scala
- core/src/main/scala/org/apache/spark/deploy/master/ui/ApplicationPage.scala 5 additions, 1 deletion...a/org/apache/spark/deploy/master/ui/ApplicationPage.scala
- core/src/main/scala/org/apache/spark/deploy/master/ui/HistoryNotFoundPage.scala 0 additions, 73 deletions...g/apache/spark/deploy/master/ui/HistoryNotFoundPage.scala
- core/src/main/scala/org/apache/spark/deploy/master/ui/MasterPage.scala 7 additions, 1 deletion.../scala/org/apache/spark/deploy/master/ui/MasterPage.scala
- core/src/main/scala/org/apache/spark/deploy/master/ui/MasterWebUI.scala 2 additions, 38 deletions...scala/org/apache/spark/deploy/master/ui/MasterWebUI.scala
- core/src/main/scala/org/apache/spark/status/api/v1/ApiRootResource.scala 1 addition, 1 deletion...cala/org/apache/spark/status/api/v1/ApiRootResource.scala
- core/src/main/scala/org/apache/spark/status/api/v1/ApplicationListResource.scala 0 additions, 30 deletions.../apache/spark/status/api/v1/ApplicationListResource.scala
- core/src/test/scala/org/apache/spark/deploy/master/ui/MasterWebUISuite.scala 70 additions, 48 deletions.../org/apache/spark/deploy/master/ui/MasterWebUISuite.scala
- docs/monitoring.md 0 additions, 5 deletionsdocs/monitoring.md
Loading
Please register or sign in to comment