diff options
| author | Levi Tamasi <ltamasi@meta.com> | 2024-09-19 15:47:13 -0700 |
|---|---|---|
| committer | Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> | 2024-09-19 15:47:13 -0700 |
| commit | 54ace7f34083090dcefa51c0fd9381436ccb8fa0 (patch) | |
| tree | e377e606eefa5452ed89879e29cdb63ecf3caca3 /include | |
| parent | 98c33cb8e39412ffdc4aa9b0ec97204dbd28acb9 (diff) | |
Change the semantics of blob_garbage_collection_force_threshold to provide better control over space amp (#13022)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13022
Currently, `blob_garbage_collection_force_threshold` applies to the oldest batch of blob files, which is typically only a small subset of the blob files currently eligible for garbage collection. This can result in a form of head-of-line blocking: no GC-triggered compactions will be scheduled if the oldest batch does not currently exceed the threshold, even if a lot of higher-numbered blob files do. This can in turn lead to high space amplification that exceeds the soft bound implicit in the force threshold (e.g. 50% would suggest a space amp of <2 and 75% would imply a space amp of <4). The patch changes the semantics of this configuration threshold to apply to the entire set of blob files that are eligible for garbage collection based on `blob_garbage_collection_age_cutoff`. This provides more intuitive semantics for the option and can provide a better write amp/space amp trade-off. (Note that GC-triggered compactions still pick the same SST files as before, so triggered GC still targets the oldest the blob files.)
Reviewed By: jowlyzhang
Differential Revision: D62977860
fbshipit-source-id: a999f31fe9cdda313de513f0e7a6fc707424d4a3
Diffstat (limited to 'include')
| -rw-r--r-- | include/rocksdb/advanced_options.h | 13 |
1 files changed, 6 insertions, 7 deletions
diff --git a/include/rocksdb/advanced_options.h b/include/rocksdb/advanced_options.h index 309d0c510..0805e2aab 100644 --- a/include/rocksdb/advanced_options.h +++ b/include/rocksdb/advanced_options.h @@ -929,13 +929,12 @@ struct AdvancedColumnFamilyOptions { // Dynamically changeable through the SetOptions() API double blob_garbage_collection_age_cutoff = 0.25; - // If the ratio of garbage in the oldest blob files exceeds this threshold, - // targeted compactions are scheduled in order to force garbage collecting - // the blob files in question, assuming they are all eligible based on the - // value of blob_garbage_collection_age_cutoff above. This option is - // currently only supported with leveled compactions. - // Note that enable_blob_garbage_collection has to be set in order for this - // option to have any effect. + // If the ratio of garbage in the blob files currently eligible for garbage + // collection exceeds this threshold, targeted compactions are scheduled in + // order to force garbage collecting the oldest blob files. This option is + // currently only supported with leveled compactions. Note that + // enable_blob_garbage_collection has to be set in order for this option to + // have any effect. // // Default: 1.0 // |