-
- Downloads
[SPARK-14555] Second cut of Python API for Structured Streaming
## What changes were proposed in this pull request? This PR adds Python APIs for: - `ContinuousQueryManager` - `ContinuousQueryException` The `ContinuousQueryException` is a very basic wrapper, it doesn't provide the functionality that the Scala side provides, but it follows the same pattern for `AnalysisException`. For `ContinuousQueryManager`, all APIs are provided except for registering listeners. This PR also attempts to fix test flakiness by stopping all active streams just before tests. ## How was this patch tested? Python Doc tests and unit tests Author: Burak Yavuz <brkyvz@gmail.com> Closes #12673 from brkyvz/pyspark-cqm.
Showing
- python/pyspark/sql/context.py 9 additions, 0 deletionspython/pyspark/sql/context.py
- python/pyspark/sql/readwriter.py 1 addition, 1 deletionpython/pyspark/sql/readwriter.py
- python/pyspark/sql/streaming.py 124 additions, 11 deletionspython/pyspark/sql/streaming.py
- python/pyspark/sql/tests.py 75 additions, 34 deletionspython/pyspark/sql/tests.py
- python/pyspark/sql/utils.py 8 additions, 0 deletionspython/pyspark/sql/utils.py
Loading
Please register or sign in to comment