Skip to content

feat(experimental): Add support for using a private S3 bucket#29

Merged
JakePartusch merged 10 commits into
mainfrom
feature/oai-private-s3
Jul 24, 2021
Merged

feat(experimental): Add support for using a private S3 bucket#29
JakePartusch merged 10 commits into
mainfrom
feature/oai-private-s3

Conversation

@JakePartusch

@JakePartusch JakePartusch commented Mar 10, 2021

Copy link
Copy Markdown
Owner

TODO:

  • Put behind opt-in __experimental feature flag
  • Add documentation

@github-actions

Copy link
Copy Markdown

✅ Your deploy preview is ready: https://96d03a4f.serverlessui.app

@damc-dev

Copy link
Copy Markdown
Contributor

Hey @JakePartusch, I have a need for deploying a static site without opening public access to the S3 bucket and this change seems to fit my needs perfectly.

Is there any way I can help get this change merged into main?

@JakePartusch

Copy link
Copy Markdown
Owner Author

Hey @damc-dev, I didn't end up merging because of the issue here: #26 (reply in thread)

I wasn't able to find a good workaround other than deploying an Edge Lambda... And the Edge lambda would be tricky because it would need to make some assumptions about how the routing in the app would work 😕

@damc-dev

damc-dev commented Jun 10, 2021

Copy link
Copy Markdown
Contributor

So I was able to get it working with a CloudFront function in my fork serverless-ui.construct.ts#L171 in a pretty generic way that works for both of your example applications.

simple+api: Site | CloudFront to API

gatsby: Site

I am sure there are other cases that I am missing, would you be able to elaborate on the cases were this assumption wouldn't work?

@JakePartusch

Copy link
Copy Markdown
Owner Author

Yeah @damc-dev , I should've been more specific about the problem that I've seen when using OAI.

For example:

As far as I understand, this happens because page-2 is not an object in the bucket, as the object is actually /page-2/index.html. Typical public S3 websites are smart enough to figure this out, but OAI breaks it 😕. So the way that I would normally go about fixing that is to use an Edge lambda to reroute the path to /page-2/index.html if the resource receives an error on the initial route.

@damc-dev

Copy link
Copy Markdown
Contributor

Interesting, I had no idea that Gatsby routing compensated for that, but would error when navigating there directly.

I updated the CloudFront function to perform the same action that is done by default in S3 (copied from github.com/aws-samples/amazon-cloudfront-functions/url-rewrite-single-page-apps)

I think it should solve the problem!

updated simple+api: Site | CloudFront to API

updated gatsby: Site

@damc-dev

Copy link
Copy Markdown
Contributor

I created a PR #38 to update your feature branch, if you would prefer the PR request was against main let me know!

* updated to latest stable cdk

* Added CloudFront function to rewrite URLs

* Modified CloudFront function to add /index.html to any request without a filename

Co-authored-by: David McElligott <dmmcelli@amazon.com>
@JakePartusch

Copy link
Copy Markdown
Owner Author

Thanks, @damc-dev! I'm thinking about putting this behind a feature flag for now... Also, it looks like I have some conflicts to resolve. Hopefully I can get this updated over the weekend. Thanks for your help and feedback!

@damc-dev

damc-dev commented Jun 11, 2021

Copy link
Copy Markdown
Contributor

Yeah I had to update to the latest CDK to support deploying CloudFront functions, in hindsight a separate PR to just update the CDK might have been a good idea. Sorry for dumping the extra integration work in your lap.

I know a lot of organizations have control policies to disable the ability to make buckets public so this pattern will allow them to use your awesome solution.

I wouldn't be surprised if there are other challenges introduced via this approach we haven't ran into yet, so I like the idea of a feature flag at-least temporarily!

Thanks for all your hard work on this great solution @JakePartusch!

@github-actions

Copy link
Copy Markdown

✅ Your deploy preview is ready: https://bd9abf26.serverlessui.app

@JakePartusch JakePartusch changed the title feat: using a private s3 bucket with origin access identity feat(experimental): Add support for using a private S3 bucket Jun 12, 2021
@github-actions

Copy link
Copy Markdown

✅ Your deploy preview is ready: https://864b79d4.serverlessui.app

@github-actions

Copy link
Copy Markdown

✅ Your deploy preview is ready: https://9d7201b1.serverlessui.app

@github-actions

Copy link
Copy Markdown

✅ Your deploy preview is ready: https://57a34edc.serverlessui.app

@JakePartusch
JakePartusch merged commit 415a0d2 into main Jul 24, 2021
@JakePartusch
JakePartusch deleted the feature/oai-private-s3 branch July 24, 2021 20:04
@github-actions

Copy link
Copy Markdown

✅ Your deploy preview is ready: https://c3d7aede.serverlessui.app

@github-actions

Copy link
Copy Markdown

✅ Your deploy preview is ready: https://73405bac.serverlessui.app

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants