Skip to content
Snippets Groups Projects
Commit c8748c25 authored by Ewen Cheslack-Postava's avatar Ewen Cheslack-Postava
Browse files

Add notes to python documentation about using SparkContext.setSystemProperty.

parent 317a9eb1
No related branches found
No related tags found
No related merge requests found
...@@ -131,6 +131,17 @@ sc = SparkContext("local", "App Name", pyFiles=['MyFile.py', 'lib.zip', 'app.egg ...@@ -131,6 +131,17 @@ sc = SparkContext("local", "App Name", pyFiles=['MyFile.py', 'lib.zip', 'app.egg
Files listed here will be added to the `PYTHONPATH` and shipped to remote worker machines. Files listed here will be added to the `PYTHONPATH` and shipped to remote worker machines.
Code dependencies can be added to an existing SparkContext using its `addPyFile()` method. Code dependencies can be added to an existing SparkContext using its `addPyFile()` method.
You can set [system properties](configuration.html#system-properties)
using `SparkContext.setSystemProperty()` class method *before*
instantiating SparkContext. For example, to set the amount of memory
per executor process:
{% highlight python %}
from pyspark import SparkContext
SparkContext.setSystemProperty('spark.executor.memory', '2g')
sc = SparkContext("local", "App Name")
{% endhighlight %}
# API Docs # API Docs
[API documentation](api/pyspark/index.html) for PySpark is available as Epydoc. [API documentation](api/pyspark/index.html) for PySpark is available as Epydoc.
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment