Skip to content

Minor: Remove unnecessary clone in datafusion_proto - #7921

Merged
alamb merged 1 commit into
apache:mainfrom
ongchi:proto-unnecessary-clone
Oct 24, 2023
Merged

Minor: Remove unnecessary clone in datafusion_proto#7921
alamb merged 1 commit into
apache:mainfrom
ongchi:proto-unnecessary-clone

Conversation

@ongchi

@ongchi ongchi commented Oct 24, 2023

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

Remove unnecessary clone in LogicalPlanNode::try_into_logical_plan and PhysicalPlanNode::try_into_physical_plan.

Are these changes tested?

Are there any user-facing changes?

Comment on lines 461 to 464
use protobuf::repartition_node::PartitionMethod;
let pb_partition_method = repartition.partition_method.clone().ok_or_else(|| {
let pb_partition_method = repartition.partition_method.as_ref().ok_or_else(|| {
DataFusionError::Internal(String::from(
"Protobuf deserialization error, RepartitionNode was missing required field 'partition_method'",
))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be returned early and the clone could be avoided.

Comment on lines -397 to +402
let input_schema = hash_agg
.input_schema
.as_ref()
.ok_or_else(|| {
DataFusionError::Internal(
"input_schema in AggregateNode is missing.".to_owned(),
)
})?
.clone();
let physical_schema: SchemaRef =
SchemaRef::new((&input_schema).try_into()?);
let input_schema = hash_agg.input_schema.as_ref().ok_or_else(|| {
DataFusionError::Internal(
"input_schema in AggregateNode is missing.".to_owned(),
)
})?;
let physical_schema: SchemaRef = SchemaRef::new(input_schema.try_into()?);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SchemaRef seems able create from ref of input_schema.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I is confusing, but SchemaRef is an alias for Arc<Schema> which is fast and quick to copy

@ongchi ongchi Oct 26, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The input_schema here is a &protobuf::Schema, and the SchemaRef is an alias of Arc<arrow_schema::Schema>.

https://github.com/apache/arrow-datafusion/blob/0911f1523ec7088bae88684ecb9bca94aa553693/datafusion/proto/src/logical_plan/from_proto.rs#L372-L379

https://github.com/apache/arrow-datafusion/blob/0911f1523ec7088bae88684ecb9bca94aa553693/datafusion/proto/src/logical_plan/from_proto.rs#L609-L620

This really confusing me at first glance, it's consume a &protobuf::Schema when calling try_into(). Seems fields will be recreated anyway, I think the clone of input_schema could be eliminated.

I don't understand why these convert trait methods convert into U from a &T rather than a T, this makes implicit clones in convert methods.

https://github.com/apache/arrow-datafusion/blob/0911f1523ec7088bae88684ecb9bca94aa553693/datafusion/proto/src/physical_plan/mod.rs#L490

And I just notice this line could be replaced by physical_schema.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree it is confusing. Any help to make it better would be most appreciated

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @ongchi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants