diff options
15 files changed, 22 insertions, 15 deletions
diff --git a/HISTORY.md b/HISTORY.md index f3ad192da..d4fc843a2 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -1,6 +1,26 @@ # Rocksdb Change Log > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt` +## 9.7.0 (09/20/2024) +### New Features +* Make Cache a customizable class that can be instantiated by the object registry. +* Add new option `prefix_seek_opt_in_only` that makes iterators generally safer when you might set a `prefix_extractor`. When `prefix_seek_opt_in_only=true`, which is expected to be the future default, prefix seek is only used when `prefix_same_as_start` or `auto_prefix_mode` are set. Also, `prefix_same_as_start` and `auto_prefix_mode` now allow prefix filtering even with `total_order_seek=true`. +* Add a new table property "rocksdb.key.largest.seqno" which records the largest sequence number of all keys in file. It is verified to be zero during SST file ingestion. + +### Behavior Changes +* Changed the semantics of the BlobDB configuration option `blob_garbage_collection_force_threshold` to define a threshold for the overall garbage ratio of all blob files currently eligible for garbage collection (according to `blob_garbage_collection_age_cutoff`). This can provide better control over space amplification at the cost of slightly higher write amplification. +* Set `write_dbid_to_manifest=true` by default. This means DB ID will now be preserved through backups, checkpoints, etc. by default. Also add `write_identity_file` option which can be set to false for anticipated future behavior. +* In FIFO compaction, compactions for changing file temperature (configured by option `file_temperature_age_thresholds`) will compact one file at a time, instead of merging multiple eligible file together (#13018). +* Support ingesting db generated files using hard link, i.e. IngestExternalFileOptions::move_files/link_files and IngestExternalFileOptions::allow_db_generated_files. +* Add a new file ingestion option `IngestExternalFileOptions::link_files` to hard link input files and preserve original files links after ingestion. +* DB::Close now untracks files in SstFileManager, making avaialble any space used +by them. Prior to this change they would be orphaned until the DB is re-opened. + +### Bug Fixes +* Fix a bug in CompactRange() where result files may not be compacted in any future compaction. This can only happen when users configure CompactRangeOptions::change_level to true and the change level step of manual compaction fails (#13009). +* Fix handling of dynamic change of `prefix_extractor` with memtable prefix filter. Previously, prefix seek could mix different prefix interpretations between memtable and SST files. Now the latest `prefix_extractor` at the time of iterator creation or refresh is respected. +* Fix a bug with manual_wal_flush and auto error recovery from WAL failure that may cause CFs to be inconsistent (#12995). The fix will set potential WAL write failure as fatal error when manual_wal_flush is true, and disables auto error recovery from these errors. + ## 9.6.0 (08/19/2024) ### New Features * *Best efforts recovery supports recovering to incomplete Version with a clean seqno cut that presents a valid point in time view from the user's perspective, if versioning history doesn't include atomic flush. diff --git a/include/rocksdb/version.h b/include/rocksdb/version.h index fbbd9765c..2a9b1aff3 100644 --- a/include/rocksdb/version.h +++ b/include/rocksdb/version.h @@ -12,7 +12,7 @@ // NOTE: in 'main' development branch, this should be the *next* // minor or major version number planned for release. #define ROCKSDB_MAJOR 9 -#define ROCKSDB_MINOR 7 +#define ROCKSDB_MINOR 8 #define ROCKSDB_PATCH 0 // Do not use these. We made the mistake of declaring macros starting with diff --git a/tools/check_format_compatible.sh b/tools/check_format_compatible.sh index 98b8ca094..16309f18c 100755 --- a/tools/check_format_compatible.sh +++ b/tools/check_format_compatible.sh @@ -125,7 +125,7 @@ EOF # To check for DB forward compatibility with loading options (old version # reading data from new), as well as backward compatibility -declare -a db_forward_with_options_refs=("8.6.fb" "8.7.fb" "8.8.fb" "8.9.fb" "8.10.fb" "8.11.fb" "9.0.fb" "9.1.fb" "9.2.fb" "9.3.fb" "9.4.fb" "9.5.fb" "9.6.fb") +declare -a db_forward_with_options_refs=("8.6.fb" "8.7.fb" "8.8.fb" "8.9.fb" "8.10.fb" "8.11.fb" "9.0.fb" "9.1.fb" "9.2.fb" "9.3.fb" "9.4.fb" "9.5.fb" "9.6.fb" "9.7.fb") # To check for DB forward compatibility without loading options (in addition # to the "with loading options" set), as well as backward compatibility declare -a db_forward_no_options_refs=() # N/A at the moment diff --git a/unreleased_history/behavior_changes/blob_garbage_collection_force_threshold.md b/unreleased_history/behavior_changes/blob_garbage_collection_force_threshold.md deleted file mode 100644 index 0c4b8bba2..000000000 --- a/unreleased_history/behavior_changes/blob_garbage_collection_force_threshold.md +++ /dev/null @@ -1 +0,0 @@ -Changed the semantics of the BlobDB configuration option `blob_garbage_collection_force_threshold` to define a threshold for the overall garbage ratio of all blob files currently eligible for garbage collection (according to `blob_garbage_collection_age_cutoff`). This can provide better control over space amplification at the cost of slightly higher write amplification. diff --git a/unreleased_history/behavior_changes/db_id.md b/unreleased_history/behavior_changes/db_id.md deleted file mode 100644 index 5536f3ce5..000000000 --- a/unreleased_history/behavior_changes/db_id.md +++ /dev/null @@ -1 +0,0 @@ -* Set `write_dbid_to_manifest=true` by default. This means DB ID will now be preserved through backups, checkpoints, etc. by default. Also add `write_identity_file` option which can be set to false for anticipated future behavior. diff --git a/unreleased_history/behavior_changes/fifo-temp-compaction.md b/unreleased_history/behavior_changes/fifo-temp-compaction.md deleted file mode 100644 index 7ba5f643d..000000000 --- a/unreleased_history/behavior_changes/fifo-temp-compaction.md +++ /dev/null @@ -1 +0,0 @@ -* In FIFO compaction, compactions for changing file temperature (configured by option `file_temperature_age_thresholds`) will compact one file at a time, instead of merging multiple eligible file together (#13018).
\ No newline at end of file diff --git a/unreleased_history/behavior_changes/ingest-live-file-with-move.md b/unreleased_history/behavior_changes/ingest-live-file-with-move.md deleted file mode 100644 index 444a7a45e..000000000 --- a/unreleased_history/behavior_changes/ingest-live-file-with-move.md +++ /dev/null @@ -1 +0,0 @@ -* Support ingesting db generated files using hard link, i.e. IngestExternalFileOptions::move_files/link_files and IngestExternalFileOptions::allow_db_generated_files.
\ No newline at end of file diff --git a/unreleased_history/behavior_changes/link-file-ingest.md b/unreleased_history/behavior_changes/link-file-ingest.md deleted file mode 100644 index e9f909eee..000000000 --- a/unreleased_history/behavior_changes/link-file-ingest.md +++ /dev/null @@ -1 +0,0 @@ -* Add a new file ingestion option `IngestExternalFileOptions::link_files` to hard link input files and preserve original files links after ingestion.
\ No newline at end of file diff --git a/unreleased_history/behavior_changes/sst_file_manager_untrack_close.md b/unreleased_history/behavior_changes/sst_file_manager_untrack_close.md deleted file mode 100644 index 7251b547e..000000000 --- a/unreleased_history/behavior_changes/sst_file_manager_untrack_close.md +++ /dev/null @@ -1,2 +0,0 @@ -DB::Close now untracks files in SstFileManager, making avaialble any space used -by them. Prior to this change they would be orphaned until the DB is re-opened. diff --git a/unreleased_history/bug_fixes/bug-refit-level.md b/unreleased_history/bug_fixes/bug-refit-level.md deleted file mode 100644 index f41e699e6..000000000 --- a/unreleased_history/bug_fixes/bug-refit-level.md +++ /dev/null @@ -1 +0,0 @@ -* Fix a bug in CompactRange() where result files may not be compacted in any future compaction. This can only happen when users configure CompactRangeOptions::change_level to true and the change level step of manual compaction fails (#13009).
\ No newline at end of file diff --git a/unreleased_history/bug_fixes/memtable_prefix.md b/unreleased_history/bug_fixes/memtable_prefix.md deleted file mode 100644 index d7b45c65e..000000000 --- a/unreleased_history/bug_fixes/memtable_prefix.md +++ /dev/null @@ -1 +0,0 @@ -* Fix handling of dynamic change of `prefix_extractor` with memtable prefix filter. Previously, prefix seek could mix different prefix interpretations between memtable and SST files. Now the latest `prefix_extractor` at the time of iterator creation or refresh is respected. diff --git a/unreleased_history/bug_fixes/wal-error-recovery.md b/unreleased_history/bug_fixes/wal-error-recovery.md deleted file mode 100644 index 07d87473d..000000000 --- a/unreleased_history/bug_fixes/wal-error-recovery.md +++ /dev/null @@ -1 +0,0 @@ -* Fix a bug with manual_wal_flush and auto error recovery from WAL failure that may cause CFs to be inconsistent (#12995). The fix will set potential WAL write failure as fatal error when manual_wal_flush is true, and disables auto error recovery from these errors.
\ No newline at end of file diff --git a/unreleased_history/new_features/customizable_cache.md b/unreleased_history/new_features/customizable_cache.md deleted file mode 100644 index b50d03c24..000000000 --- a/unreleased_history/new_features/customizable_cache.md +++ /dev/null @@ -1 +0,0 @@ -Make Cache a customizable class that can be instantiated by the object registry. diff --git a/unreleased_history/new_features/prefix_seek_opt_in_only.md b/unreleased_history/new_features/prefix_seek_opt_in_only.md deleted file mode 100644 index 71c3cd689..000000000 --- a/unreleased_history/new_features/prefix_seek_opt_in_only.md +++ /dev/null @@ -1 +0,0 @@ -* Add new option `prefix_seek_opt_in_only` that makes iterators generally safer when you might set a `prefix_extractor`. When `prefix_seek_opt_in_only=true`, which is expected to be the future default, prefix seek is only used when `prefix_same_as_start` or `auto_prefix_mode` are set. Also, `prefix_same_as_start` and `auto_prefix_mode` now allow prefix filtering even with `total_order_seek=true`. diff --git a/unreleased_history/new_features/tp_largest_seqno.md b/unreleased_history/new_features/tp_largest_seqno.md deleted file mode 100644 index 6776cfcf2..000000000 --- a/unreleased_history/new_features/tp_largest_seqno.md +++ /dev/null @@ -1 +0,0 @@ -* Add a new table property "rocksdb.key.largest.seqno" which records the largest sequence number of all keys in file. It is verified to be zero during SST file ingestion.
\ No newline at end of file |