fix(destination-bigquery): use CREATE TABLE IF NOT EXISTS for non-replace table creation by sunil-kuruba · Pull Request #82102 · airbytehq/airbyte

Sunil Kuruba (sunil-kuruba) · 2026-07-14T21:04:31Z

Summary

Uses CREATE TABLE IF NOT EXISTS instead of CREATE TABLE when replace=false in BigqueryDirectLoadSqlGenerator
This prevents errors when the table already exists during non-truncate sync modes (Append, Dedup)
Complements the CDK change in fix(Bulk CDK): use replace=false for real table creation #81549 which switched non-truncate modes to pass replace=false

Part of https://github.com/airbytehq/oncall/issues/13059
Supersedes #81550 (for BigQuery portion)

Important

⚡ Auto-merge enabled.

This PR is set to merge automatically when all requirements are met.

…lace table creation When creating real tables with replace=false (non-truncate sync modes), use CREATE TABLE IF NOT EXISTS instead of CREATE TABLE to prevent errors when the table already exists. This complements the CDK change in #81549 which switched non-truncate modes to use replace=false. Co-authored-by: Claude <noreply@anthropic.com>

github-actions · 2026-07-14T21:05:04Z

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

PR Slash Commands

Airbyte Maintainers (that's you!) can execute the following slash commands on your PR:

🛠️ Quick Fixes
- /format-fix - Fixes most formatting issues.
- /bump-version - Bumps connector versions, scraping changelog description from the PR title.
  - Bump types: patch (default), minor, major, major_rc, rc, promote.
  - The rc type is a smart default: applies minor_rc if stable, or bumps the RC number if already RC.
  - The promote type strips the RC suffix to finalize a release.
  - Example: /bump-version type=rc or /bump-version type=minor
- /bump-progressive-rollout-version - Alias for /bump-version type=rc. Bumps with an RC suffix and enables progressive rollout.
❇️ AI Testing and Review (internal link: AI-SDLC Docs):
- /ai-prove-fix - Runs prerelease readiness checks, including testing against customer connections.
- /ai-canary-prerelease - Rolls out prerelease to 5-10 connections for canary testing.
- /ai-review - AI-powered PR review for connector safety and quality gates.
📝 AI Documentation:
- /ai-docs-review - AI-powered documentation review for PRs with connector changes.
- /ai-create-docs-pr - Creates a documentation PR for connector changes, stacked on the current PR.
🚀 Connector Releases:
- /publish-connectors-prerelease - Publishes pre-release connector builds (tagged as {version}-preview.{git-sha}) for all modified connectors in the PR.
- /enable-autopilot-rollouts - Enables autopilot progressive rollouts for the modified connector(s) in the PR, remediating "autopilot rollouts not enabled for {connector-name}" auto-merge blockers. Sets defaultRolloutMode: autopilot and enableProgressiveRollout: true, preserving any existing autopilotConfig.
  - Optional args: connector=<CONNECTOR_NAME> (defaults to the modified connectors in the PR), strategy=fast|slow|default (defaults to fast).
  - Example: /enable-autopilot-rollouts or /enable-autopilot-rollouts connector=source-faker strategy=slow
☕️ JVM connectors:
- /update-connector-cdk-version connector=<CONNECTOR_NAME> - Updates the specified connector to the latest CDK version.
  Example: /update-connector-cdk-version connector=destination-bigquery
🐍 Python connectors:
- /poe connector source-example lock - Run the Poe lock task on the source-example connector, committing the results back to the branch.
- /poe source example lock - Alias for /poe connector source-example lock.
- /poe source example use-cdk-branch my/branch - Pin the source-example CDK reference to the branch name specified.
- /poe source example use-cdk-latest - Update the source-example CDK dependency to the latest available version.
⚙️ Admin commands:
- /force-merge reason="<REASON>" - Force merges the PR using admin privileges, bypassing CI checks. Requires a reason.
  Example: /force-merge reason="CI is flaky, tests pass locally"

📚 Show Repo Guidance

Helpful Resources

Breaking Changes Guide - Breaking changes, migration guides, and upgrade deadlines
Developing Connectors Locally
Managing Connector Secrets
On-Demand Regression Tests
#connector-ci-issues
#connector-publish-updates
#connector-build-statuses

📝 Edit this welcome message.

github-actions · 2026-07-14T21:05:22Z

Note

✅ All modified connector(s) now have autopilot progressive rollouts enabled. Thanks!

Upgrade CDK to 1.0.16 which includes the replace=false change (#81549) for non-truncate sync modes. Without this bump, the CREATE TABLE IF NOT EXISTS path would never be exercised. Co-authored-by: Claude <noreply@anthropic.com>

Co-authored-by: Claude <noreply@anthropic.com>

github-actions · 2026-07-14T21:16:31Z

Note

Detected that there are differences in the Gradle dependencies.

https://github.com/airbytehq/airbyte/runs/87436356090

github-actions · 2026-07-14T21:32:37Z

`destination-bigquery` Connector Test Results

464 tests 413 ✅ 1h 33m 43s ⏱️
22 suites 51 💤
22 files 0 ❌

Results for commit 7fb213e.

♻️ This comment has been updated with latest results.

github-actions · 2026-07-14T23:36:29Z

Deploy preview for airbyte-docs ready!

Project:	`airbyte-docs`
Status:	✅ Deploy successful!
Preview URL:	https://airbyte-docs-dgbhyzc41-airbyte-growth.vercel.app
Latest Commit:	`7fb213e`

Deployed with vercel-action

Airbyte Support Bot (airbyte-support-bot) · 2026-07-15T11:38:30Z

↪️ Triggering /ai-review per Hands-Free AI Triage Project triage next step.

Reason: Ready non-draft fix PR with CI largely green and no prior AI review. Running AI review is the next pipeline stage.

Devin session

octavia-bot · 2026-07-15T11:38:52Z

AI PR Review starting...

Reviewing PR for connector safety and quality.
View playbook

🔗 Session URL: https://app.devin.ai/sessions/951af5056b3640ae80099a811fbbace3

✅ Devin AI session created successfully!

Airbyte Support Bot (airbyte-support-bot) · 2026-07-15T11:41:37Z

🛡️ AI PR Review Report

🔴 Review Action: REQUEST CHANGES

Gate	Status
Test Coverage	FAIL
Code Hygiene	WARNING
Live / E2E Tests	UNKNOWN

🔶 Risk Level: 3/5

Behavioral change to existing table-creation DDL in destination-bigquery (CREATE TABLE IF NOT EXISTS for replace=false); a typical functional connector logic change.

🔧 Remediation Required

Test Coverage (enforced → REQUEST CHANGES)

This is a bug fix (fix(...) title, linked to airbytehq/oncall#13059) that changes SQL generation in BigqueryDirectLoadSqlGenerator.kt, but no test was added.
Add at least one unit test asserting the generated createTable SQL uses CREATE TABLE IF NOT EXISTS when replace=false and plain CREATE TABLE when replace=true. The existing BigqueryDirectLoadSqlGeneratorTest.kt only covers clustering columns and was not updated — extending it is the natural place.
Alternatively, if you believe existing coverage is sufficient, add an ## AI PR Review Justification section to the PR description explaining why no new test is needed, then re-run /ai-review.

Live / E2E Tests (UNKNOWN — not blocking, but recommended)

No /ai-prove-fix conclusion was found. Run /ai-prove-fix on this PR to validate the change (it complements CDK behavior from fix(Bulk CDK): use replace=false for real table creation #81549 that now passes replace=false for Append/Dedup).

📋 PR Details

Connector(s): destination-bigquery
PR: #82102
HEAD SHA: 1df50bf119bf56a7c07cfc847e4b2416b7269fac
Session: https://app.devin.ai/sessions/951af5056b3640ae80099a811fbbace3

🔍 Gate Evaluation Details

Gate-by-Gate Analysis

Gate	Status	Enforced?	Details
PR Hygiene	PASS	Yes	Description has a meaningful Summary + linked issue (`oncall#13059`); changelog row added in `docs/integrations/destinations/bigquery.md`; no unresolved human review comments (only bot comments).
Code Hygiene	WARNING	WARNING	Source file `BigqueryDirectLoadSqlGenerator.kt` modified with no test files modified. Consider adding tests for the new code path.
Test Coverage	FAIL	Yes	Behavioral fix with no new test content. See Remediation.
Code Security	PASS	Yes	No security-sensitive file paths touched. `metadata.yaml` diff changes only `dockerImageTag`; no `allowedHosts`/`connectorBuildOptions`/`dockerRepository` keywords in the +/- diff lines.
Per-Record Performance	PASS	WARNING	Change is in one-time `CREATE TABLE` DDL generation, not a per-record path.
Breaking Dependencies	PASS	WARNING	CDK `1.0.13 → 1.0.16` (patch bump) in `gradle.properties`, intended to pair with CDK #81549. No documented breaking behavior.
Backwards Compatibility	PASS	Yes (enforced)	No spec, schema, primary-key, or cursor changes. The change is strictly safer (adds `IF NOT EXISTS` only when `replace=false`). `metadata.yaml` change is a version bump only.
Forwards Compatibility	PASS	Warning (elevates Risk Level)	No state/cursor/pagination/transformation keywords in the changed hunks.
Behavioral Changes	PASS	Warning (elevates Risk Level)	No rate-limit/retry/backoff/timeout/error-handler keywords in the changed hunks.
Out-of-Scope Changes	PASS	Skip	Connector (`airbyte-integrations/connectors/destination-bigquery/**`) + docs changes — in scope.
CI Checks	PASS	Yes	All core checks green: `Test destination-bigquery Connector`, `Lint destination-bigquery Connector`, `Build and Verify Artifacts`, `Format Check`, and the Connector CI summary.
Live / E2E Tests	UNKNOWN	Yes	Validation is required (bug fix touching sync/table-creation behavior) but no `/ai-prove-fix` conclusion was found. Not run ⇒ UNKNOWN, not FAIL.

Changed files (4):

airbyte-integrations/connectors/destination-bigquery/src/main/kotlin/.../BigqueryDirectLoadSqlGenerator.kt — functional (adds createPrefix logic)
airbyte-integrations/connectors/destination-bigquery/gradle.properties — CDK version bump
airbyte-integrations/connectors/destination-bigquery/metadata.yaml — dockerImageTag 3.0.22 → 3.0.23
docs/integrations/destinations/bigquery.md — changelog entry

📚 Evidence Consulted

Evidence

Changed files: 4 files
CI checks: 38 passed, 0 failed, 0 pending (core: Test/Lint/Build/Format all green)
PR labels: connectors/destination/bigquery, dependencies-change, hyd-review
PR description: present (Summary + linked oncall#13059)
Existing bot reviews: none (no prior AI PR review for this HEAD SHA)

❓ How to Respond

Providing Context or Justification

You can add explanations that the bot will see on the next review:

Option 1: PR Description (recommended)
Add a section to your PR description:

## AI PR Review Justification

### Test Coverage
[Your explanation here]

Option 2: PR Comment
Add a comment starting with:

AI PR Review Justification:
[Your explanation here]

After adding your response, re-run /ai-review to have the bot evaluate it.

Note: For Test Coverage, a sufficient justification (e.g. existing tests already cover the changed behavior, or the change is a trivial refactor) can flip FAIL → PASS. For the Live / E2E Tests gate, run /ai-prove-fix to obtain validation evidence.

Devin session

Sunil Kuruba (sunil-kuruba) · 2026-07-15T18:12:39Z

/enable-autopilot-rollouts

Enable Autopilot Rollouts job started... Check job output.

✅ Autopilot rollouts enabled. (7fb213e)

Sunil Kuruba (sunil-kuruba) requested a review from a team as a code owner July 14, 2026 21:04

Octavia Squidington III (octavia-squidington-iii) added the connectors/destination/bigquery label Jul 14, 2026

Sunil Kuruba (sunil-kuruba) mentioned this pull request Jul 14, 2026

fix: use CREATE TABLE IF NOT EXISTS for non-replace table creation in connectors #81550

Closed

chore(destination-bigquery): bump CDK version to 1.0.16

9ebe697

Upgrade CDK to 1.0.16 which includes the replace=false change (#81549) for non-truncate sync modes. Without this bump, the CREATE TABLE IF NOT EXISTS path would never be exercised. Co-authored-by: Claude <noreply@anthropic.com>

Sunil Kuruba (sunil-kuruba) self-assigned this Jul 14, 2026

chore: remove redundant comments

1df50bf

Co-authored-by: Claude <noreply@anthropic.com>

github-actions Bot added the dependencies-change label Jul 14, 2026

Sunil Kuruba (sunil-kuruba) changed the title ~~[ready] fix(destination-bigquery): use CREATE TABLE IF NOT EXISTS for non-replace table creation~~ fix(destination-bigquery): use CREATE TABLE IF NOT EXISTS for non-replace table creation Jul 14, 2026

Rodi Reich Zilberman (rodireich) approved these changes Jul 14, 2026

View reviewed changes

devin-ai-integration Bot added the hyd-review label Jul 15, 2026

chore: enable autopilot rollouts

7fb213e

Sunil Kuruba (sunil-kuruba) enabled auto-merge (squash) July 15, 2026 18:29

Sunil Kuruba (sunil-kuruba) merged commit 607716b into master Jul 16, 2026
59 of 60 checks passed

Sunil Kuruba (sunil-kuruba) deleted the fix/bigquery-create-table-if-not-exists branch July 16, 2026 02:27

devin-ai-integration Bot mentioned this pull request Jul 16, 2026

docs(destination-bigquery): fix changelog, correct output schema, remove obsolete config steps #78303

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(destination-bigquery): use CREATE TABLE IF NOT EXISTS for non-replace table creation#82102

fix(destination-bigquery): use CREATE TABLE IF NOT EXISTS for non-replace table creation#82102
Sunil Kuruba (sunil-kuruba) merged 4 commits into
masterfrom
fix/bigquery-create-table-if-not-exists

Sunil Kuruba (sunil-kuruba) commented Jul 14, 2026 •

edited by github-actions Bot

Loading

Uh oh!

github-actions Bot commented Jul 14, 2026

PR Slash Commands

Helpful Resources

Uh oh!

github-actions Bot commented Jul 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 14, 2026 •

edited

Loading

Uh oh!

Airbyte Support Bot (airbyte-support-bot) commented Jul 15, 2026

Uh oh!

octavia-bot Bot commented Jul 15, 2026 •

edited

Loading

Uh oh!

Airbyte Support Bot (airbyte-support-bot) commented Jul 15, 2026 •

edited

Loading

Gate-by-Gate Analysis

Evidence

Providing Context or Justification

Uh oh!

Sunil Kuruba (sunil-kuruba) commented Jul 15, 2026 •

edited by github-actions Bot

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Sunil Kuruba (sunil-kuruba) commented Jul 14, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

github-actions Bot commented Jul 14, 2026

👋 Greetings, Airbyte Team Member!

PR Slash Commands

Helpful Resources

Uh oh!

github-actions Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

destination-bigquery Connector Test Results

Uh oh!

github-actions Bot commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Airbyte Support Bot (airbyte-support-bot) commented Jul 15, 2026

Uh oh!

octavia-bot Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Airbyte Support Bot (airbyte-support-bot) commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛡️ AI PR Review Report

🔴 Review Action: REQUEST CHANGES

🔶 Risk Level: 3/5

Gate-by-Gate Analysis

Evidence

Providing Context or Justification

Uh oh!

Sunil Kuruba (sunil-kuruba) commented Jul 15, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Sunil Kuruba (sunil-kuruba) commented Jul 14, 2026 •

edited by github-actions Bot

Loading

github-actions Bot commented Jul 14, 2026 •

edited

Loading

github-actions Bot commented Jul 14, 2026 •

edited

Loading

github-actions Bot commented Jul 14, 2026 •

edited

Loading

`destination-bigquery` Connector Test Results

github-actions Bot commented Jul 14, 2026 •

edited

Loading

octavia-bot Bot commented Jul 15, 2026 •

edited

Loading

Airbyte Support Bot (airbyte-support-bot) commented Jul 15, 2026 •

edited

Loading

Sunil Kuruba (sunil-kuruba) commented Jul 15, 2026 •

edited by github-actions Bot

Loading