python/pyspark/context.py · cbb7f04aef2220ece93dea9f3fa98b5db5f270d6 · cs525-sp18-g07 / spark

"...rodinia/git@gitlab.engr.illinois.edu:llvm/hercules.git" did not exist on "72a61f40f290d9b5ced10b2c2caa406998b7c861"

11 years ago

Add custom serializer support to PySpark. · cbb7f04a

Josh Rosen authored 11 years ago

For now, this only adds MarshalSerializer, but it lays the groundwork
for other supporting custom serializers.  Many of these mechanisms
can also be used to support deserialization of different data formats
sent by Java, such as data encoded by MsgPack.

This also fixes a bug in SparkContext.union().

cbb7f04a

History

Add custom serializer support to PySpark.

Josh Rosen authored 11 years ago

For now, this only adds MarshalSerializer, but it lays the groundwork
for other supporting custom serializers.  Many of these mechanisms
can also be used to support deserialization of different data formats
sent by Java, such as data encoded by MsgPack.

This also fixes a bug in SparkContext.union().