Skip to content
Snippets Groups Projects
Commit 25c2776d authored by Xingbo Jiang's avatar Xingbo Jiang Committed by gatorsmile
Browse files

[SPARK-23523][SQL][FOLLOWUP] Minor refactor of OptimizeMetadataOnlyQuery

## What changes were proposed in this pull request?

Inside `OptimizeMetadataOnlyQuery.getPartitionAttrs`, avoid using `zip` to generate attribute map.
Also include other minor update of comments and format.

## How was this patch tested?

Existing test cases.

Author: Xingbo Jiang <xingbo.jiang@databricks.com>

Closes #20693 from jiangxb1987/SPARK-23523.
parent 476a7f02
No related branches found
No related tags found
No related merge requests found
......@@ -83,7 +83,7 @@ case class OptimizeMetadataOnlyQuery(catalog: SessionCatalog) extends Rule[Logic
private def getPartitionAttrs(
partitionColumnNames: Seq[String],
relation: LogicalPlan): Seq[Attribute] = {
val attrMap = relation.output.map(_.name.toLowerCase(Locale.ROOT)).zip(relation.output).toMap
val attrMap = relation.output.map(a => a.name.toLowerCase(Locale.ROOT) -> a).toMap
partitionColumnNames.map { colName =>
attrMap.getOrElse(colName.toLowerCase(Locale.ROOT),
throw new AnalysisException(s"Unable to find the column `$colName` " +
......
......@@ -67,9 +67,9 @@ case class HadoopFsRelation(
}
}
// When data schema and partition schema have the overlapped columns, the output
// schema respects the order of data schema for the overlapped columns, but respect
// the data types of partition schema
// When data and partition schemas have overlapping columns, the output
// schema respects the order of the data schema for the overlapping columns, and it
// respects the data types of the partition schema.
val schema: StructType = {
StructType(dataSchema.map(f => overlappedPartCols.getOrElse(getColName(f), f)) ++
partitionSchema.filterNot(f => overlappedPartCols.contains(getColName(f))))
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment