- SizeEstimator was following through a ClassLoader field of Hadoop
  JobConfs, which referenced the whole interpreter, Scala compiler, etc.
  Chaos ensued, giving an estimated size in the tens of gigabytes.
- Broadcast variables in local mode were only stored as MEMORY_ONLY and
  never made accessible over a server, so they fell out of the cache when
  they were deemed too large and couldn't be reloaded.

This reverts commit 08579ffa.

development.

of a weird file dediated to accessing this variable).

I'm not entirely sure why this broke in the jump from Scala 2.9.0.1 to
2.9.1 -- maybe something about name resolution changed?

Interestingly, the version in Maven is 2.9.1, but SBT outputs file to the 2.9.1.final
directory inside target.

A couple of small changes in SparkIMain were also required.

All tests pass and ./spark-shell launches successfully.

missed before and setting SparkEnv properly on the threads that execute
each line in the 2.9 interpreter).

each line's object rather than a singleton object so that we can ship
these classes to worker nodes). This is pretty hairy stuff, which would
be nice to avoid in the future by integrating with the interpreter some
other way.