From f38fab97c7970168f1bd81d4dc202e36322c95e3 Mon Sep 17 00:00:00 2001
From: Rob O'Dwyer <odwyerrob@gmail.com>
Date: Wed, 27 Aug 2014 19:47:33 -0700
Subject: [PATCH] SPARK-3265 Allow using custom ipython executable with pyspark
Although you can make pyspark use ipython with `IPYTHON=1`, and also change the python executable with `PYSPARK_PYTHON=...`, you can't use both at the same time because it hardcodes the default ipython script.
This makes it use the `PYSPARK_PYTHON` variable if present and fall back to default python, similarly to how the default python executable is handled.
So you can use a custom ipython like so:
`PYSPARK_PYTHON=./anaconda/bin/ipython IPYTHON_OPTS="notebook" pyspark`
Author: Rob O'Dwyer <odwyerrob@gmail.com>
Closes #2167 from robbles/patch-1 and squashes the following commits:
d98e8a9 [Rob O'Dwyer] Allow using custom ipython executable with pyspark
---
bin/pyspark | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bin/pyspark b/bin/pyspark
index 01d42025c9..59cfdfa7c5 100755
--- a/bin/pyspark
+++ b/bin/pyspark
@@ -104,7 +104,7 @@ if [[ "$1" =~ \.py$ ]]; then
else
# Only use ipython if no command line arguments were provided [SPARK-1134]
if [[ "$IPYTHON" = "1" ]]; then
- exec ipython $IPYTHON_OPTS
+ exec ${PYSPARK_PYTHON:-ipython} $IPYTHON_OPTS
else
exec "$PYSPARK_PYTHON"
fi
--
GitLab