Skip to content
  • Brian O'Neill's avatar
    81e3bfc1
    [SPARK-14421] Upgrades protobuf dependency to 2.6.1 for the new version of KCL, and… · 81e3bfc1
    Brian O'Neill authored
    ## What changes were proposed in this pull request?
    
    When running with Kinesis Consumer Library (KCL), against a stream that contains aggregated data, the KCL needs access to protobuf to de-aggregate the records.   Without this patch, that results in the following error message:
    
    ```
       Caused by: java.lang.ClassNotFoundException: com.google.protobuf.ProtocolStringList
    ```
    
    This PR upgrades the protobuf dependency within the kinesis-asl-assembly, and relocates that package (as not to conflict with Spark's use of 2.5.0), which fixes the above CNFE.
    
    ## How was this patch tested?
    
    Used kinesis word count example against a stream containing aggregated data.
    
    See: SPARK-14421
    
    Author: Brian O'Neill <bone@alumni.brown.edu>
    
    Closes #13054 from boneill42/protobuf-relocation-for-kcl.
    81e3bfc1
    [SPARK-14421] Upgrades protobuf dependency to 2.6.1 for the new version of KCL, and…
    Brian O'Neill authored
    ## What changes were proposed in this pull request?
    
    When running with Kinesis Consumer Library (KCL), against a stream that contains aggregated data, the KCL needs access to protobuf to de-aggregate the records.   Without this patch, that results in the following error message:
    
    ```
       Caused by: java.lang.ClassNotFoundException: com.google.protobuf.ProtocolStringList
    ```
    
    This PR upgrades the protobuf dependency within the kinesis-asl-assembly, and relocates that package (as not to conflict with Spark's use of 2.5.0), which fixes the above CNFE.
    
    ## How was this patch tested?
    
    Used kinesis word count example against a stream containing aggregated data.
    
    See: SPARK-14421
    
    Author: Brian O'Neill <bone@alumni.brown.edu>
    
    Closes #13054 from boneill42/protobuf-relocation-for-kcl.
Loading