Skip to content

HDDS-11205. Implement a search feature for users to locate keys pending Deletion within the OM Deleted Keys Insights section - #6969

Merged
devmadhuu merged 22 commits into
apache:masterfrom
ArafatKhan2198:HDDS-11205
Oct 18, 2024
Merged

HDDS-11205. Implement a search feature for users to locate keys pending Deletion within the OM Deleted Keys Insights section#6969
devmadhuu merged 22 commits into
apache:masterfrom
ArafatKhan2198:HDDS-11205

Conversation

@ArafatKhan2198

@ArafatKhan2198 ArafatKhan2198 commented Jul 19, 2024

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This pull request enhances the search functionality for deleted pending keys in the Deleted Keys Insights section. The previous implementation was limited to a subset of the available data returned by the API. This update ensures accurate retrieval of specific key information by improving pagination and handling of both prefix-based searches and general search scenarios.

The patch implements the ability to search for deleted keys using a specified prefix, efficiently returning results that match the prefix across both FSO and Object Store layouts. The logic has been refactored to the OMDBInsightEndpoint, unifying the method for handling all cases where the startPrefix is provided or not.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-11205

How was this patch tested?

  • Unit tests were created and updated to cover various scenarios, including pagination, searches with and without startPrefix, and for both FSO and non-FSO layouts.
  • Manual testing was conducted to ensure that the search functionality works as expected across different scenarios.
  • We can run the following command on docker container running OM.
// Use the Freon tool to create 5 directories with 10 keys each 
for i in {1..5}; do
  ozone freon dfsg --path=ofs://om/s3v/fso-bucket/ -s=10000 -n=10
done

// Delete all the directories under the fso-bucket which will result in records being added to deletedTable 
ozone fs -rm -r -skipTrash ofs://om/s3v/fso-bucket
  • Using the given API endpoint which will list out all the deleted keys under the bucket directory 1inkjkisd which is under fso-bucket.
http://localhost:9888/api/v1/keys/deletePending?startPrefix=/s3v/fso-bucket/1inkjkisd
image

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants