summaryrefslogtreecommitdiff
path: root/db/repair.cc
diff options
context:
space:
mode:
authorPeter Dillinger <peterd@fb.com>2021-04-30 13:49:24 -0700
committerFacebook GitHub Bot <facebook-github-bot@users.noreply.github.com>2021-04-30 13:50:13 -0700
commitd2ca04e3ed237e202306865db6201be1161cbdc2 (patch)
treeae875a5da3b1cd0032b1020fd852e9cbe4b4141b /db/repair.cc
parent85becd94c169b311cbf73d9dacd38f4d38206084 (diff)
Add more LSM info to FilterBuildingContext (#8246)
Summary: Add `num_levels`, `is_bottommost`, and table file creation `reason` to `FilterBuildingContext`, in anticipation of more powerful Bloom-like filter support. To support this, added `is_bottommost` and `reason` to `TableBuilderOptions`, which allowed removing `reason` parameter from `rocksdb::BuildTable`. I attempted to remove `skip_filters` from `TableBuilderOptions`, because filter construction decisions should arise from options, not one-off parameters. I could not completely remove it because the public API for SstFileWriter takes a `skip_filters` parameter, and translating this into an option change would mean awkwardly replacing the table_factory if it is BlockBasedTableFactory with new filter_policy=nullptr option. I marked this public skip_filters option as deprecated because of this oddity. (skip_filters on the read side probably makes sense.) At least `skip_filters` is now largely hidden for users of `TableBuilderOptions` and is no longer used for implementing the optimize_filters_for_hits option. Bringing the logic for that option closer to handling of FilterBuildingContext makes it more obvious that hese two are using the same notion of "bottommost." (Planned: configuration options for Bloom-like filters that generalize `optimize_filters_for_hits`) Recommended follow-up: Try to get away from "bottommost level" naming of things, which is inaccurate (see VersionStorageInfo::RangeMightExistAfterSortedRun), and move to "bottommost run" or just "bottommost." Pull Request resolved: https://github.com/facebook/rocksdb/pull/8246 Test Plan: extended an existing unit test to exercise and check various filter building contexts. Also, existing tests for optimize_filters_for_hits validate some of the "bottommost" handling, which is now closely connected to FilterBuildingContext::is_bottommost through TableBuilderOptions::is_bottommost Reviewed By: mrambacher Differential Revision: D28099346 Pulled By: pdillinger fbshipit-source-id: 2c1072e29c24d4ac404c761a7b7663292372600a
Diffstat (limited to 'db/repair.cc')
-rw-r--r--db/repair.cc12
1 files changed, 6 insertions, 6 deletions
diff --git a/db/repair.cc b/db/repair.cc
index 5d3a0c097..9d103a710 100644
--- a/db/repair.cc
+++ b/db/repair.cc
@@ -444,8 +444,9 @@ class Repairer {
TableBuilderOptions tboptions(
*cfd->ioptions(), *cfd->GetLatestMutableCFOptions(),
cfd->internal_comparator(), cfd->int_tbl_prop_collector_factories(),
- kNoCompression, default_compression, false /* skip_filters */,
- cfd->GetID(), cfd->GetName(), -1 /* level */, current_time,
+ kNoCompression, default_compression, cfd->GetID(), cfd->GetName(),
+ -1 /* level */, false /* is_bottommost */,
+ TableFileCreationReason::kRecovery, current_time,
0 /* oldest_key_time */, 0 /* file_creation_time */,
"DB Repairer" /* db_id */, db_session_id_, 0 /*target_file_size*/);
status = BuildTable(
@@ -453,10 +454,9 @@ class Repairer {
env_options_, table_cache_.get(), iter.get(),
std::move(range_del_iters), &meta, nullptr /* blob_file_additions */,
{}, kMaxSequenceNumber, snapshot_checker,
- false /* paranoid_file_checks*/, nullptr /* internal_stats */,
- TableFileCreationReason::kRecovery, &io_s, nullptr /*IOTracer*/,
- nullptr /* event_logger */, 0 /* job_id */, Env::IO_HIGH,
- nullptr /* table_properties */, write_hint);
+ false /* paranoid_file_checks*/, nullptr /* internal_stats */, &io_s,
+ nullptr /*IOTracer*/, nullptr /* event_logger */, 0 /* job_id */,
+ Env::IO_HIGH, nullptr /* table_properties */, write_hint);
ROCKS_LOG_INFO(db_options_.info_log,
"Log #%" PRIu64 ": %d ops saved to Table #%" PRIu64 " %s",
log, counter, meta.fd.GetNumber(),