-
- Downloads
[SPARK-18670][SS] Limit the number of...
[SPARK-18670][SS] Limit the number of StreamingQueryListener.StreamProgressEvent when there is no data ## What changes were proposed in this pull request? This PR adds a sql conf `spark.sql.streaming.noDataReportInterval` to control how long to wait before outputing the next StreamProgressEvent when there is no data. ## How was this patch tested? The added unit test. Author: Shixiong Zhu <shixiong@databricks.com> Closes #16108 from zsxwing/SPARK-18670.
Showing
- sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala 17 additions, 1 deletion...pache/spark/sql/execution/streaming/StreamExecution.scala
- sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala 10 additions, 0 deletions...rc/main/scala/org/apache/spark/sql/internal/SQLConf.scala
- sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingQueryListenerSuite.scala 44 additions, 0 deletions...che/spark/sql/streaming/StreamingQueryListenerSuite.scala
Loading
Please register or sign in to comment