Watt merge #20
Open
Watt merge #20
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix error, if n < hw_threads Better Balancing: - we now give each thread ceil(n/hw_threads) work, therefore we steal a bit from last threads - floor (n/hw_threads) would add work to the last thread.
- PageProvider: Total Evictions, total Touches, total Writes, - Worker: Jumps, Misse, new_pages, tx Refactor Profiling thread: - Extract anonymous function to named function - put configy_table and result_table in extra category (untimed_tables)
Refactor BufferManager and BTreeNode a bit Add missing sum_no_reset from previous commit (profiling table)
Persist ssd_path Persist some other value, that was earlier derived from the table, but strangely had some errors on multiple persist and recover run.... Add Functionality to Leanstore, to persist and retrieve persisted Strings by name.
Dont run tpcc tx if run for sec == 0 and run_until_tx == 0 Here just the data is created.
Remove "failed_attempts" from phase1 (unneeded, because only read before change) Add a few comments to PageProvider Thread
Has its own latch -> update does not lock page, only parent pointer.
Put logic of AsyncWriteBuffer in AsyncWriteBuffer and only offer "add" and "flush" function. Now the AsyncWriteBuffer needs a "getPartition" to get a Partition for a specific page And offers a "pageCallback", to call on handled pages
recover tuple_count in ycsb
…ent strategy); Fix Target_gib
There was a problem hiding this comment.
There is a slight regression for in-memory SI tpc-c
./frontend/tpcc --tpcc_warehouse_count=2 --tpcc_warehouse_affinity --ssd_path=./ssd --worker_threads=2 --pp_threads=1 --dram_gib=20 --csv_path=./log --free_pct=1 --contention_split --xmerge --print_tx_console --run_for_seconds=60 --isolation_level=si
+-----+------------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+
| t | OLTP TX | RF % | Abort% | OLAP TX | W MiB | R MiB | Instrs/- | Branch | Cycles/- | CPUs | L1/TX | LLC/TX | GHz | WAL | GCT | Space G | GCT | touches | evictio- |
| | | | | | | | TX | miss/TX | TX | | | | | GiB/s | GiB/s | | Rounds | | ns |
+-----+------------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+
| 1 | 46091 | 22.4035- | 0.000000 | 0 | 0.0 | 0.0 | 218277.- | 444.161- | 132171.- | 1.940588 | 1832.90- | 0.000000 | 6.227894 | 0.1 | 0.0 | 0.5 | 179115 | 0 | 0 |
+-----+------------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+----------+
| t | OLTP TX | RF % | Abort% | OLAP TX | W MiB | R MiB | Instrs/- | Cycles/- | CPUs | L1/TX | LLC/TX | GHz | WAL | GCT | Space G | GCT |
| | | | | | | | TX | TX | | | | | GiB/s | GiB/s | | Rounds |
1 | 52789 | 22.5084- | 0.000000 | 0 | 0.0 | 0.0 | 190514.- | 125488.- | 1.970990 | 1675.06- | 0.000000 | 6.678050 | 0.1 | 0.0 | 0.5 | 177638 |
backend/leanstore/Config.cpp
Outdated
| @@ -22,6 +22,7 @@ DEFINE_bool(profile_latency, false, ""); | |||
| DEFINE_bool(crc_check, false, ""); | |||
| // ------------------------------------------------------------------------------------- | |||
| DEFINE_uint32(worker_threads, 4, ""); | |||
| DEFINE_uint32(creator_threads, 0, ""); | |||
There was a problem hiding this comment.
Maybe add a comment to describe the difference between worker and creator thread?
| @@ -142,52 +142,42 @@ u32 BTreeNode::spaceUsedBySlot(u16 s_i) | |||
| // ------------------------------------------------------------------------------------- | |||
| // right survives, this gets reclaimed | |||
| // left(this) into right | |||
| bool BTreeNode::merge(u16 slotId, ExclusivePageGuard<BTreeNode>& parent, ExclusivePageGuard<BTreeNode>& right) | |||
| bool BTreeNode::merge(ExclusivePageGuard<BTreeNode>& self, u16 slotId, ExclusivePageGuard<BTreeNode>& parent, ExclusivePageGuard<BTreeNode>& right) | |||
There was a problem hiding this comment.
Was it buggy or why you made this change?
There was a problem hiding this comment.
I need access to both (left and right) bfs.header.tracker (To merge the two Timestamp-Trackers)
| @@ -43,25 +45,139 @@ struct BufferFrame { | |||
| ContentionTracker contention_tracker; | |||
| // ------------------------------------------------------------------------------------- | |||
| struct OptimisticParentPointer { | |||
There was a problem hiding this comment.
Are you now using OptimisticParentPointer? It was used only in micro experiments
6D4E
There was a problem hiding this comment.
I just added a guard to remove some race conditions. it still is behind a flag
frontend/micro-benchmarks/queue.cpp
Outdated
| @@ -96,7 +96,10 @@ int main(int argc, char** argv) | |||
| ensure(ret); | |||
| table.insert({newest + 1}, {}); | |||
| cr::Worker::my().commitTX(); | |||
| COUNTERS_BLOCK() { WorkerCounters::myCounters().tx++; } | |||
| COUNTERS_BLOCK() { | |||
| WorkerCounters::myCounters().tx++; | |||
There was a problem hiding this comment.
What is the difference between tx and tx_counter?
There was a problem hiding this comment.
tx is for the statistics per second, tx_counter for the run statistics (not reset), see ResultsTable
I can reproduce this.. (~10%) will have a look into it |
align globalTrackerTime (epochs) to reduce cacheline invalidation simplify mergo of two trackers
|
Fixed Efficiency problems... Now there seems to be an problem with the effectiveness... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reimplemented WATT on Master
Additionally: