-
- Downloads
Add custom serializer support to PySpark.
For now, this only adds MarshalSerializer, but it lays the groundwork for other supporting custom serializers. Many of these mechanisms can also be used to support deserialization of different data formats sent by Java, such as data encoded by MsgPack. This also fixes a bug in SparkContext.union().
Showing
- core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala 1 addition, 22 deletions...rc/main/scala/org/apache/spark/api/python/PythonRDD.scala
- python/epydoc.conf 1 addition, 1 deletionpython/epydoc.conf
- python/pyspark/accumulators.py 4 additions, 2 deletionspython/pyspark/accumulators.py
- python/pyspark/context.py 45 additions, 16 deletionspython/pyspark/context.py
- python/pyspark/rdd.py 47 additions, 39 deletionspython/pyspark/rdd.py
- python/pyspark/serializers.py 243 additions, 67 deletionspython/pyspark/serializers.py
- python/pyspark/tests.py 2 additions, 1 deletionpython/pyspark/tests.py
- python/pyspark/worker.py 19 additions, 22 deletionspython/pyspark/worker.py
- python/run-tests 1 addition, 0 deletionspython/run-tests
Loading
Please register or sign in to comment