Skip to content
Snippets Groups Projects
  • aokolnychyi's avatar
    3fdce814
    [SPARK-16046][DOCS] Aggregations in the Spark SQL programming guide · 3fdce814
    aokolnychyi authored
    ## What changes were proposed in this pull request?
    
    - A separate subsection for Aggregations under “Getting Started” in the Spark SQL programming guide. It mentions which aggregate functions are predefined and how users can create their own.
    - Examples of using the `UserDefinedAggregateFunction` abstract class for untyped aggregations in Java and Scala.
    - Examples of using the `Aggregator` abstract class for type-safe aggregations in Java and Scala.
    - Python is not covered.
    - The PR might not resolve the ticket since I do not know what exactly was planned by the author.
    
    In total, there are four new standalone examples that can be executed via `spark-submit` or `run-example`. The updated Spark SQL programming guide references to these examples and does not contain hard-coded snippets.
    
    ## How was this patch tested?
    
    The patch was tested locally by building the docs. The examples were run as well.
    
    ![image](https://cloud.githubusercontent.com/assets/6235869/21292915/04d9d084-c515-11e6-811a-999d598dffba.png)
    
    Author: aokolnychyi <okolnychyyanton@gmail.com>
    
    Closes #16329 from aokolnychyi/SPARK-16046.
    3fdce814
    History
    [SPARK-16046][DOCS] Aggregations in the Spark SQL programming guide
    aokolnychyi authored
    ## What changes were proposed in this pull request?
    
    - A separate subsection for Aggregations under “Getting Started” in the Spark SQL programming guide. It mentions which aggregate functions are predefined and how users can create their own.
    - Examples of using the `UserDefinedAggregateFunction` abstract class for untyped aggregations in Java and Scala.
    - Examples of using the `Aggregator` abstract class for type-safe aggregations in Java and Scala.
    - Python is not covered.
    - The PR might not resolve the ticket since I do not know what exactly was planned by the author.
    
    In total, there are four new standalone examples that can be executed via `spark-submit` or `run-example`. The updated Spark SQL programming guide references to these examples and does not contain hard-coded snippets.
    
    ## How was this patch tested?
    
    The patch was tested locally by building the docs. The examples were run as well.
    
    ![image](https://cloud.githubusercontent.com/assets/6235869/21292915/04d9d084-c515-11e6-811a-999d598dffba.png)
    
    Author: aokolnychyi <okolnychyyanton@gmail.com>
    
    Closes #16329 from aokolnychyi/SPARK-16046.