Skip to content

[SPARK-12198] [SparkR] SparkR support read.parquet and deprecate parquetFile - #10191

Closed
yanboliang wants to merge 2 commits into
apache:masterfrom
yanboliang:spark-12198
Closed

[SPARK-12198] [SparkR] SparkR support read.parquet and deprecate parquetFile#10191
yanboliang wants to merge 2 commits into
apache:masterfrom
yanboliang:spark-12198

Conversation

@yanboliang

@yanboliang yanboliang commented Dec 8, 2015

Copy link
Copy Markdown
Contributor

SparkR support read.parquet and deprecate parquetFile. This change is similar with #10145 for jsonFile.

@sun-rui

sun-rui commented Dec 8, 2015

Copy link
Copy Markdown
Contributor

LGTM

@yanboliang

yanboliang commented Dec 8, 2015

Copy link
Copy Markdown
Contributor Author

Jenkins, test this please.

@SparkQA

SparkQA commented Dec 8, 2015

Copy link
Copy Markdown

Test build #47321 has finished for PR 10191 at commit e507c31.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

felixcheung
felixcheung reviewed Dec 8, 2015
View reviewed changes
Comment thread R/pkg/NAMESPACE Outdated

@felixcheung felixcheung Dec 8, 2015

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: sort - this should go after read.df

@felixcheung

felixcheung commented Dec 8, 2015

Copy link
Copy Markdown
Member

looks good otherwise. are we adding a generic read() method?

@sun-rui

sun-rui commented Dec 9, 2015

Copy link
Copy Markdown
Contributor

@felixcheung, what do you mean "a generic read() method"? we have read.df() yet?

@felixcheung

felixcheung commented Dec 9, 2015

Copy link
Copy Markdown
Member

sure - read.df doesn't match DataFrameReader so there are certain sources that it couldn't handle

@sun-rui

sun-rui commented Dec 9, 2015

Copy link
Copy Markdown
Contributor

@felixcheung, could you detail more about "read.df doesn't match DataFrameReader"? The only exception is JDBC source?

@SparkQA

SparkQA commented Dec 9, 2015

Copy link
Copy Markdown

Test build #47401 has finished for PR 10191 at commit 9e0fd63.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@yanboliang

yanboliang commented Dec 9, 2015

Copy link
Copy Markdown
Contributor Author

+1 @sun-rui Only jdbc can not fit in read.df.

@felixcheung

felixcheung commented Dec 9, 2015

Copy link
Copy Markdown
Member

sorry that's unrelated to this PR. Opened SPARK-12224 and SPARK-12232

@felixcheung

felixcheung commented Dec 9, 2015

Copy link
Copy Markdown
Member

looks good.

@shivaram

shivaram commented Dec 10, 2015

Copy link
Copy Markdown
Contributor

LGTM. Thanks @yanboliang -- Merging this to master and branch-1.6

@SparkQA

SparkQA commented Dec 10, 2015

Copy link
Copy Markdown

Test build #47518 has finished for PR 10191 at commit 9e0fd63.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@asfgit asfgit closed this in eeb5872 Dec 10, 2015
asfgit pushed a commit that referenced this pull request Dec 10, 2015
@yanboliang @shivaram
[SPARK-12198][SPARKR] SparkR support read.parquet and deprecate parqu…
b7b9f77 
…etFile

SparkR support ```read.parquet``` and deprecate ```parquetFile```. This change is similar with #10145 for ```jsonFile```.

Author: Yanbo Liang <ybliang8@gmail.com>

Closes #10191 from yanboliang/spark-12198.

(cherry picked from commit eeb5872)
Signed-off-by: Shivaram Venkataraman <shivaram@cs.berkeley.edu>
@yanboliang
yanboliang deleted the spark-12198 branch December 11, 2015 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@yanboliang @sun-rui @SparkQA @felixcheung @shivaram