-
- Downloads
[SPARK-12885][MINOR] Rename 3 fields in ShuffleWriteMetrics
This is a small step in implementing SPARK-10620, which migrates TaskMetrics to accumulators. This patch is strictly a cleanup patch and introduces no change in functionality. It literally just renames 3 fields for consistency. Today we have: ``` inputMetrics.recordsRead outputMetrics.bytesWritten shuffleReadMetrics.localBlocksFetched ... shuffleWriteMetrics.shuffleRecordsWritten shuffleWriteMetrics.shuffleBytesWritten shuffleWriteMetrics.shuffleWriteTime ``` The shuffle write ones are kind of redundant. We can drop the `shuffle` part in the method names. I added backward compatible (but deprecated) methods with the old names. Parent PR: #10717 Author: Andrew Or <andrew@databricks.com> Closes #10811 from andrewor14/rename-things.
Showing
- core/src/main/java/org/apache/spark/shuffle/sort/BypassMergeSortShuffleWriter.java 2 additions, 2 deletions...ache/spark/shuffle/sort/BypassMergeSortShuffleWriter.java
- core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java 2 additions, 2 deletions.../org/apache/spark/shuffle/sort/ShuffleExternalSorter.java
- core/src/main/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java 3 additions, 3 deletions...va/org/apache/spark/shuffle/sort/UnsafeShuffleWriter.java
- core/src/main/java/org/apache/spark/storage/TimeTrackingOutputStream.java 5 additions, 5 deletions...va/org/apache/spark/storage/TimeTrackingOutputStream.java
- core/src/main/scala/org/apache/spark/executor/ShuffleWriteMetrics.scala 24 additions, 13 deletions...scala/org/apache/spark/executor/ShuffleWriteMetrics.scala
- core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala 1 addition, 1 deletion...main/scala/org/apache/spark/scheduler/SparkListener.scala
- core/src/main/scala/org/apache/spark/shuffle/FileShuffleBlockResolver.scala 1 addition, 1 deletion...a/org/apache/spark/shuffle/FileShuffleBlockResolver.scala
- core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleWriter.scala 1 addition, 1 deletion...ala/org/apache/spark/shuffle/sort/SortShuffleWriter.scala
- core/src/main/scala/org/apache/spark/status/api/v1/AllStagesResource.scala 6 additions, 6 deletions...la/org/apache/spark/status/api/v1/AllStagesResource.scala
- core/src/main/scala/org/apache/spark/storage/DiskBlockObjectWriter.scala 6 additions, 6 deletions...cala/org/apache/spark/storage/DiskBlockObjectWriter.scala
- core/src/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala 1 addition, 1 deletion...rc/main/scala/org/apache/spark/ui/exec/ExecutorsTab.scala
- core/src/main/scala/org/apache/spark/ui/jobs/JobProgressListener.scala 4 additions, 4 deletions.../scala/org/apache/spark/ui/jobs/JobProgressListener.scala
- core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala 7 additions, 7 deletionscore/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala
- core/src/main/scala/org/apache/spark/util/JsonProtocol.scala 7 additions, 6 deletionscore/src/main/scala/org/apache/spark/util/JsonProtocol.scala
- core/src/main/scala/org/apache/spark/util/collection/ExternalAppendOnlyMap.scala 2 additions, 2 deletions.../apache/spark/util/collection/ExternalAppendOnlyMap.scala
- core/src/main/scala/org/apache/spark/util/collection/ExternalSorter.scala 2 additions, 2 deletions...ala/org/apache/spark/util/collection/ExternalSorter.scala
- core/src/test/java/org/apache/spark/shuffle/sort/UnsafeShuffleWriterSuite.java 10 additions, 10 deletions...g/apache/spark/shuffle/sort/UnsafeShuffleWriterSuite.java
- core/src/test/scala/org/apache/spark/ShuffleSuite.scala 2 additions, 2 deletionscore/src/test/scala/org/apache/spark/ShuffleSuite.scala
- core/src/test/scala/org/apache/spark/metrics/InputOutputMetricsSuite.scala 1 addition, 1 deletion...la/org/apache/spark/metrics/InputOutputMetricsSuite.scala
- core/src/test/scala/org/apache/spark/scheduler/SparkListenerSuite.scala 1 addition, 1 deletion...scala/org/apache/spark/scheduler/SparkListenerSuite.scala
Loading
Please register or sign in to comment