-
- Downloads
SPARK-1294 Fix resolution of uppercase field names using a HiveContext.
Fixing this bug required the following: - Creation of a new logical node that converts a schema to lowercase. - Generalization of the subquery eliding rule to also elide this new node - Fixing of several places where too tight assumptions were made on the types of `InsertIntoTable` children. - I also removed an API that was left in by accident that exposed catalyst data structures, and fix the logic that pushes down filters into hive tables scans to correctly compare attribute references. Author: Michael Armbrust <michael@databricks.com> Closes #202 from marmbrus/upperCaseFieldNames and squashes the following commits: 15e5265 [Michael Armbrust] Support for resolving mixed case fields from a reflected schema using HiveQL. 5aa5035 [Michael Armbrust] Remove API that exposes internal catalyst data structures. 9d99cb6 [Michael Armbrust] Attributes should be compared using exprId, not TreeNode.id.
Showing
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala 18 additions, 1 deletion...ala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala 0 additions, 14 deletions...a/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicOperators.scala 29 additions, 1 deletion...che/spark/sql/catalyst/plans/logical/basicOperators.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ConstantFoldingSuite.scala 4 additions, 3 deletions...e/spark/sql/catalyst/optimizer/ConstantFoldingSuite.scala
- sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala 7 additions, 6 deletions...he/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala 10 additions, 8 deletions...rc/main/scala/org/apache/spark/sql/hive/HiveContext.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala 4 additions, 2 deletions...cala/org/apache/spark/sql/hive/HiveMetastoreCatalog.scala
- sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala 2 additions, 2 deletions...main/scala/org/apache/spark/sql/hive/HiveStrategies.scala
- sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveResolutionSuite.scala 13 additions, 0 deletions...apache/spark/sql/hive/execution/HiveResolutionSuite.scala
Loading
Please register or sign in to comment