See the discussion at https://github.com/mesos/spark/pull/355 for why
default profile activation is a problem.

Minor improvements to PySpark docs

Also, adds a line in doc explaining how to use.

It's nicer if all the commands you need are made explicit.

Remember ConnectionManagerId used to initiate SendingConnections

Make ExecutorIDs include SlaveIDs when running Mesos

Include message and exitStatus if availalbe.

the Mesos ExecutorID as a Spark ExecutorID.

This prevents ConnectionManager from getting confused if a machine
has multiple host names and the one getHostName() finds happens
not to be the one that was passed from, e.g., the BlockManagerMaster.

Conflicts:
	core/src/main/scala/spark/deploy/master/Master.scala

Add RDD.toDebugString.

Replace old 'master' term with 'driver'.

- RDD's getDependencies and getSplits methods are now guaranteed to be
  called only once, so subclasses can safely do computation in there
  without worrying about caching the results.

- The management of a "splits_" variable that is cleared out when we
  checkpoint an RDD is now done in the RDD class.

- A few of the RDD subclasses are simpler.

- CheckpointRDD's compute() method no longer assumes that it is given a
  CheckpointRDDSplit -- it can work just as well on a split from the
  original RDD, because it only looks at its index. This is important
  because things like UnionRDD and ZippedRDD remember the parent's
  splits as part of their own and wouldn't work on checkpointed parents.

- RDD.iterator can now reuse cached data if an RDD is computed before it
  is checkpointed. It seems like it wouldn't do this before (it always
  called iterator() on the CheckpointRDD, which read from HDFS).

SPARK-658: Adding logging of stage duration

Original idea by Nathan Kronenfeld.

Conflicts:
	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/SparkEnv.scala
	core/src/main/scala/spark/deploy/LocalSparkCluster.scala
	core/src/main/scala/spark/executor/StandaloneExecutorBackend.scala
	core/src/main/scala/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
	core/src/main/scala/spark/scheduler/cluster/StandaloneClusterMessage.scala
	core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala
	core/src/main/scala/spark/storage/BlockManagerMaster.scala
	core/src/main/scala/spark/storage/ThreadingTest.scala
	core/src/test/scala/spark/MapOutputTrackerSuite.scala

Some DEBUG-level log cleanup.

add long and float accumulatorparams

A few changes to make the DEBUG-level logs less
noisy and more readable.

- Moved a few very frequent messages to Trace
- Changed some BlockManger log messages to make them
  more understandable

SPARK-666 #resolve

Directives, and bind to a random port)

executors per machine and remove the need for multiple IP addresses in
unit tests.

Detect whether we run on EC2 using ec2-metadata as well

Blockmanager ui

Fix BlockManager reregistration deadlock; do BlockManager reregistration more asynchronously