Skip to content
Snippets Groups Projects
  • Sean Owen's avatar
    649e9d0f
    [SPARK-3369][CORE][STREAMING] Java mapPartitions Iterator->Iterable is... · 649e9d0f
    Sean Owen authored
    [SPARK-3369][CORE][STREAMING] Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator
    
    Fix Java function API methods for flatMap and mapPartitions to require producing only an Iterator, not Iterable. Also fix DStream.flatMap to require a function producing TraversableOnce only, not Traversable.
    
    CC rxin pwendell for API change; tdas since it also touches streaming.
    
    Author: Sean Owen <sowen@cloudera.com>
    
    Closes #10413 from srowen/SPARK-3369.
    649e9d0f
    History
    [SPARK-3369][CORE][STREAMING] Java mapPartitions Iterator->Iterable is...
    Sean Owen authored
    [SPARK-3369][CORE][STREAMING] Java mapPartitions Iterator->Iterable is inconsistent with Scala's Iterator->Iterator
    
    Fix Java function API methods for flatMap and mapPartitions to require producing only an Iterator, not Iterable. Also fix DStream.flatMap to require a function producing TraversableOnce only, not Traversable.
    
    CC rxin pwendell for API change; tdas since it also touches streaming.
    
    Author: Sean Owen <sowen@cloudera.com>
    
    Closes #10413 from srowen/SPARK-3369.