diff options
author | Akanksha Mahajan <akankshamahajan@fb.com> | 2022-06-08 14:16:43 -0700 |
---|---|---|
committer | Facebook GitHub Bot <facebook-github-bot@users.noreply.github.com> | 2022-06-08 14:16:43 -0700 |
commit | f85b31a2e958a65e73797932f23985e398562aa7 (patch) | |
tree | 6d77c54b089857ba3e0460b0aa49ea278e55dfc6 /db/wal_manager.cc | |
parent | 9efae1442812d3646cb5c7cfb178f22fb883678c (diff) |
Fix bug for WalManager with compressed WAL (#10130)
Summary:
RocksDB uses WalManager to manage WAL files. In WalManager::ReadFirstLine(), the assumption is that reading the first record of a valid WAL file will return OK status and set the output sequence to non-zero value.
This assumption has been broken by WAL compression which writes a `kSetCompressionType` record which is not associated with any sequence number.
Consequently, WalManager::GetSortedWalsOfType() will skip these WALs and not return them to caller, e.g. Checkpoint, Backup, causing the operations to fail.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10130
Test Plan: - Newly Added test
Reviewed By: riversand963
Differential Revision: D36985744
Pulled By: akankshamahajan15
fbshipit-source-id: dfde7b3be68b6a30b75b49479779748eedf29f7f
Diffstat (limited to 'db/wal_manager.cc')
-rw-r--r-- | db/wal_manager.cc | 18 |
1 files changed, 14 insertions, 4 deletions
diff --git a/db/wal_manager.cc b/db/wal_manager.cc index 83a3636fb..ed76905d4 100644 --- a/db/wal_manager.cc +++ b/db/wal_manager.cc @@ -507,10 +507,20 @@ Status WalManager::ReadFirstLine(const std::string& fname, } } - // ReadRecord might have returned false on EOF, which means that the log file - // is empty. Or, a failure may have occurred while processing the first entry. - // In any case, return status and set sequence number to 0. - *sequence = 0; + if (status.ok() && reader.IsCompressedAndEmptyFile()) { + // In case of wal_compression, it writes a `kSetCompressionType` record + // which is not associated with any sequence number. As result for an empty + // file, GetSortedWalsOfType() will skip these WALs causing the operations + // to fail. + // Therefore, in order to avoid that failure, it sets sequence_number to 1 + // indicating those WALs should be included. + *sequence = 1; + } else { + // ReadRecord might have returned false on EOF, which means that the log + // file is empty. Or, a failure may have occurred while processing the first + // entry. In any case, return status and set sequence number to 0. + *sequence = 0; + } return status; } |