Skip to content

Commit 520436a

Browse files
committed
Add a temporary hook for custom yielding in long-running op (#13103)
Summary: This is a simplified version of #13096, which called for a way to hook into long-running loops completely within RocksDB to change their thread priority (or similar). The current prime hook point is `DBIter::FindNextUserEntryInternal` likely because of iterating over tombstones. This is implemented using the weak symbol hack for ease of back-porting/patching, and while we get to know potential future requirements better for integration into the public API. (Consider potential relationships to `Env::GetThreadStatusUpdater()` and `TransactionDBMutexFactory`.) Pull Request resolved: #13103 Test Plan: Performance validated with db_bench and DEBUG_LEVEL=0: `./db_bench --benchmarks=fillseq,deleterandom,readseq[-X100] --value_size=1 --num=1000000` No consistent difference seen; variances likely in how DB / executable / memory were laid out. ``` With an empty hook: readseq [AVG 100 runs] : 1753018 (± 8850) ops/sec; 28.4 (± 0.1) MB/sec readseq [MEDIAN 100 runs] : 1763746 ops/sec; 28.6 MB/sec (recompile) readseq [AVG 100 runs] : 1789019 (± 10260) ops/sec; 29.0 (± 0.2) MB/sec readseq [MEDIAN 100 runs] : 1801849 ops/sec; 29.2 MB/sec Base: readseq [AVG 100 runs] : 1772196 (± 8240) ops/sec; 28.7 (± 0.1) MB/sec readseq [MEDIAN 100 runs] : 1780453 ops/sec; 28.9 MB/sec (recompile) readseq [AVG 100 runs] : 1777637 (± 7613) ops/sec; 28.8 (± 0.1) MB/sec readseq [MEDIAN 100 runs] : 1786657 ops/sec; 29.0 MB/sec With a functional hook (count number of calls into it): readseq [AVG 100 runs] : 1796733 (± 8854) ops/sec; 29.1 (± 0.1) MB/sec readseq [MEDIAN 100 runs] : 1804690 ops/sec; 29.3 MB/sec RocksDbThreadYield: 126915800 (recompile) readseq [AVG 100 runs] : 1775371 (± 10529) ops/sec; 28.8 (± 0.2) MB/sec readseq [MEDIAN 100 runs] : 1789046 ops/sec; 29.0 MB/sec RocksDbThreadYield: 126977000 Base: readseq [AVG 100 runs] : 1773071 (± 10657) ops/sec; 28.7 (± 0.2) MB/sec readseq [MEDIAN 100 runs] : 1783414 ops/sec; 28.9 MB/sec (recompile) readseq [AVG 100 runs] : 1750852 (± 10184) ops/sec; 28.4 (± 0.2) MB/sec readseq [MEDIAN 100 runs] : 1763587 ops/sec; 28.6 MB/sec ``` Reviewed By: george-reynya Differential Revision: D65235379 Pulled By: pdillinger fbshipit-source-id: 7829e4cc25a56d4c1801b8adf9c7f7aa49ab7aca
1 parent 0e2801a commit 520436a

File tree

3 files changed

+20
-0
lines changed

3 files changed

+20
-0
lines changed

db/db_impl/db_impl.cc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3227,6 +3227,8 @@ Status DBImpl::MultiGetImpl(
32273227
s = Status::Aborted();
32283228
break;
32293229
}
3230+
// This could be a long-running operation
3231+
ROCKSDB_THREAD_YIELD_HOOK();
32303232
}
32313233

32323234
// Post processing (decrement reference counts and record statistics)

db/db_iter.cc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -540,6 +540,8 @@ bool DBIter::FindNextUserEntryInternal(bool skipping_saved_key,
540540
} else {
541541
iter_.Next();
542542
}
543+
// This could be a long-running operation due to tombstones, etc.
544+
ROCKSDB_THREAD_YIELD_HOOK();
543545
} while (iter_.Valid());
544546

545547
valid_ = false;

port/port.h

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,19 @@
1919
#elif defined(OS_WIN)
2020
#include "port/win/port_win.h"
2121
#endif
22+
23+
#ifdef OS_LINUX
24+
// A temporary hook into long-running RocksDB threads to support modifying their
25+
// priority etc. This should become a public API hook once the requirements
26+
// are better understood.
27+
extern "C" void RocksDbThreadYield() __attribute__((__weak__));
28+
#define ROCKSDB_THREAD_YIELD_HOOK() \
29+
{ \
30+
if (RocksDbThreadYield) { \
31+
RocksDbThreadYield(); \
32+
} \
33+
}
34+
#else
35+
#define ROCKSDB_THREAD_YIELD_HOOK() \
36+
{}
37+
#endif

0 commit comments

Comments
 (0)