python/pyspark/serializers.py · c84946fe210069259f5d42ab8fd22a5ddae91d12 · cs525-sp18-g07 / spark

"README.md" did not exist on "0ad4f77d88a0107af6c654752523fd667f691ddb"

11 years ago

Fixing SPARK-602: PythonPartitioner · c84946fe

Andre Schumacher authored 11 years ago

Currently PythonPartitioner determines partition ID by hashing a
byte-array representation of PySpark's key. This PR lets
PythonPartitioner use the actual partition ID, which is required e.g.
for sorting via PySpark.

c84946fe

History

Fixing SPARK-602: PythonPartitioner

Andre Schumacher authored 11 years ago

Currently PythonPartitioner determines partition ID by hashing a
byte-array representation of PySpark's key. This PR lets
PythonPartitioner use the actual partition ID, which is required e.g.
for sorting via PySpark.