diff options
| author | Sagar Vemuri <svemuri@fb.com> | 2019-04-10 19:24:25 -0700 |
|---|---|---|
| committer | Facebook Github Bot <facebook-github-bot@users.noreply.github.com> | 2019-04-10 19:31:18 -0700 |
| commit | d3d20dcdca9dc79d893a03dfa611fb1055c28f96 (patch) | |
| tree | 769dc30a406acdea51335909ae9305ce49cc4d39 /HISTORY.md | |
| parent | ef0fc1b46189c51da8799ee79574fd37ced93752 (diff) | |
Periodic Compactions (#5166)
Summary:
Introducing Periodic Compactions.
This feature allows all the files in a CF to be periodically compacted. It could help in catching any corruptions that could creep into the DB proactively as every file is constantly getting re-compacted. And also, of course, it helps to cleanup data older than certain threshold.
- Introduced a new option `periodic_compaction_time` to control how long a file can live without being compacted in a CF.
- This works across all levels.
- The files are put in the same level after going through the compaction. (Related files in the same level are picked up as `ExpandInputstoCleanCut` is used).
- Compaction filters, if any, are invoked as usual.
- A new table property, `file_creation_time`, is introduced to implement this feature. This property is set to the time at which the SST file was created (and that time is given by the underlying Env/OS).
This feature can be enabled on its own, or in conjunction with `ttl`. It is possible to set a different time threshold for the bottom level when used in conjunction with ttl. Since `ttl` works only on 0 to last but one levels, you could set `ttl` to, say, 1 day, and `periodic_compaction_time` to, say, 7 days. Since `ttl < periodic_compaction_time` all files in last but one levels keep getting picked up based on ttl, and almost never based on periodic_compaction_time. The files in the bottom level get picked up for compaction based on `periodic_compaction_time`.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5166
Differential Revision: D14884441
Pulled By: sagar0
fbshipit-source-id: 408426cbacb409c06386a98632dcf90bfa1bda47
Diffstat (limited to 'HISTORY.md')
| -rw-r--r-- | HISTORY.md | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/HISTORY.md b/HISTORY.md index f2bdcda11..3abe5a79f 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -3,6 +3,8 @@ ### Unreleased ### New Features * When reading from option file/string/map, customized comparators and/or merge operators can be filled according to object registry. +* Introduce Periodic Compaction for Level style compaction. Files are re-compacted periodically and put in the same level. + ### Public API Change ### Bug Fixes * Fix a bug in 2PC where a sequence of txn prepare, memtable flush, and crash could result in losing the prepared transaction. |