Skip to content

ARROW-1246: [Format] Draft Flatbuffer metadata description for Map#876

Closed
wesm wants to merge 3 commits into
apache:masterfrom
wesm:ARROW-1246
Closed

ARROW-1246: [Format] Draft Flatbuffer metadata description for Map#876
wesm wants to merge 3 commits into
apache:masterfrom
wesm:ARROW-1246

Conversation

@wesm

@wesm wesm commented Jul 21, 2017

Copy link
Copy Markdown
Member

No description provided.

wesm added 2 commits July 20, 2017 21:30
Change-Id: I335c5fc820dda617c46c36b30935ed554d892bca
Change-Id: I54ef62c27410a2de681f82b1e2fb1eb6b93c9f68

@xhochy xhochy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, sounds reasonable from my side.

@BryanCutler

Copy link
Copy Markdown
Member

Sounds good to me. I was thinking about how to store Map data from Spark, which is just (key, value) pairs, in Arrow and assumed I'd have to make the types myself into something like List<entry: Struct<key: K, value: V>>. So if I understand this change correctly, when reading back this type of data, I would know from the Map type that it was meant to be (key, value) pairs?

@jacques-n jacques-n left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good. One small question

Comment thread format/Schema.fbs Outdated
/// - child[0] key: K
/// - child[1] value: V
///
/// As a matter of convention, neither values of the "entry" nor "key" fields

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure your meaning here. Seems like this should be a formal specification, not a convention.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point; I think this is just a language issue. What I mean is that "neither entry nor key fields may have nulls"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm +1 after you clarify. Thanks!

Change-Id: I813e52b97f1d773d975ac8df66c566e770edab9c
@wesm

wesm commented Jul 25, 2017

Copy link
Copy Markdown
Member Author

Thanks for the review, all. @jacques-n is there specialized code for handling maps in this form in the Arrow Java library, or Dremio?

@asfgit asfgit closed this in 11c92bf Jul 25, 2017
@jacques-n

Copy link
Copy Markdown
Contributor

No, we should open a jira to support

@wesm
wesm deleted the ARROW-1246 branch July 25, 2017 03:16
@wesm

wesm commented Jul 25, 2017

Copy link
Copy Markdown
Member Author

ARROW-1261

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants