Skip to content

fix(core): handle parameter limits in files metadata deletion - #62331

Open
gitedmond wants to merge 1 commit into
nextcloud:masterfrom
gitedmond:fix/db-parameter-limits
Open

fix(core): handle parameter limits in files metadata deletion#62331
gitedmond wants to merge 1 commit into
nextcloud:masterfrom
gitedmond:fix/db-parameter-limits

Conversation

@gitedmond

@gitedmond gitedmond commented Jul 19, 2026

Copy link
Copy Markdown

Summary

dropMetadataForFiles() was already intended to split file IDs into chunks using IQueryBuilder::MAX_IN_PARAMETERS, but the DELETE query mistakenly bound the original $fileIds array instead of the current $chunk. As a result, every query still contained the full ID list and could exceed database parameter limits.

This PR passes $chunk to each query and adds regression coverage that verifies the exact chunks sent to the query builder.

Testing

Added a regression test using more than two parameter-limit batches. It verifies that every ID is bound exactly once, in order, and no query receives more than MAX_IN_PARAMETERS IDs.

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

@gitedmond
gitedmond force-pushed the fix/db-parameter-limits branch from a56d71c to 48551eb Compare July 19, 2026 18:38
@gitedmond
gitedmond marked this pull request as ready for review July 19, 2026 18:42
@gitedmond
gitedmond requested a review from a team as a code owner July 19, 2026 18:42
@gitedmond
gitedmond requested review from come-nc, icewind1991, leftybournes and salmart-dev and removed request for a team July 19, 2026 18:42
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

@come-nc come-nc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello,

Please switch back to using the const MAX_IN_PARAMETERS, we do not want to have a magic number 500 in there.

Nice catch on the wrong var being used.

Regarding the transaction I lack knowledge to judge in it makes sense here. Is a partial metadata deletion an issue?

@come-nc come-nc added the 3. to review Waiting for reviews label Aug 3, 2026
@come-nc come-nc added this to the Nextcloud 35 milestone Aug 3, 2026
@gitedmond
gitedmond requested a review from come-nc August 3, 2026 13:42
@gitedmond

gitedmond commented Aug 3, 2026

Copy link
Copy Markdown
Author

Hello,

Please switch back to using the const MAX_IN_PARAMETERS, we do not want to have a magic number 500 in there.

Nice catch on the wrong var being used.

Regarding the transaction I lack knowledge to judge in it makes sense here. Is a partial metadata deletion an issue?

You're right: the transaction is not needed for this fix, and it would not make metadata and index deletion atomic because those happen in separate calls. I've removed it. The PR now only binds each existing MAX_IN_PARAMETERS chunk instead of the full $fileIds array, with a regression test that verifies the exact chunks passed to the query builder.

@gitedmond
gitedmond force-pushed the fix/db-parameter-limits branch 4 times, most recently from 6a4e471 to 071b333 Compare August 3, 2026 22:00
Bind each existing MAX_IN_PARAMETERS chunk in dropMetadataForFiles instead of passing the full file ID array to every query. Add a regression test that verifies the exact chunks sent to the query builder.

Resolves: nextcloud#62325

Signed-off-by: Edmond <edmnd@users.noreply.github.com>
@gitedmond
gitedmond force-pushed the fix/db-parameter-limits branch from 071b333 to 12ff385 Compare August 3, 2026 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: files:scan creates oversized files_metadata DELETE query and causes severe resource usage

3 participants