8000 Bump unconfirmed ancestor transactions to target feerate by murchandamus · Pull Request #26152 · bitcoin/bitcoin · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Bump unconfirmed ancestor transactions to target feerate #26152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Sep 14, 2023

Conversation

murchandamus
Copy link
Contributor
@murchandamus murchandamus commented Sep 21, 2022

Includes some commits to address follow-ups from #27021: #27021 (comment)

Reduces the effective value of unconfirmed UTXOs by the fees necessary to bump their ancestor transactions to the same feerate.

While the individual UTXOs always account for their full ancestry before coin-selection, we can correct potential overestimates with a second pass where we establish the ancestry and bump fee for the whole input set collectively.

Fixes #9645
Fixes #9864
Fixes #15553

@murchandamus murchandamus force-pushed the 2022-09-ancestor-aware-funding branch from db2c0f7 to 54fd46b Compare September 21, 2022 21:16
@DrahtBot
Copy link
Contributor
DrahtBot commented Sep 21, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK S3RK, brunoerg, ismaelsadeeq, t-bast, achow101
Concept ACK fanquake, glozow, jonatack, ishaanam, andrewtoth, stickies-v, LarryRuane, josibake

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #28391 (refactor: Simplify CTxMempool/BlockAssembler fields, remove some external mapTx access by TheCharlatan)
  • #28335 (RFC: Remove boost usage from kernel api / headers by TheCharlatan)
  • #27865 (wallet: Track no-longer-spendable TXOs separately by achow101)
  • #27601 (wallet: don't duplicate change output if already exist by furszy)
  • #27286 (wallet: Keep track of the wallet's own transaction outputs in memory by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Member
@fanquake fanquake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

@murchandamus
Copy link
Contributor Author

Thanks @fanquake, I fixed the two issues.

I also added a test for a transaction using subtractfeefromamount

Copy link
Member
@glozow glozow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HUGE Concept ACK obviously 🥳 🥳

This needs to obey -maxtxfee, so I'd suggest that coin selection keeps track of what ancestors the tx has and how much of the fees is ancestors vs this transaction. And we'd want a test where the target feerate is just below maxtxfee and there are low-feerate ancestors to bump, and so total fees paid / size of this tx is higher than maxtxfee, but actually you're just paying to bump.

Also, given that there is a (small) chance of overpayment, it would be good to check that -maxtxfee protects against something drastic. For example, a test where all the coins share 1 large parent? It will overestimate and -maxtxfee should make sure the tx isn't sent? edit: this is totally wrong

Copy link
Member
@jonatack jonatack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

Copy link
Contributor
@t-bast t-bast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK, thanks a lot for working on this, it will be very helpful!
And this set of internal utility functions will very likely be useful for many things in the future.
I'll create a set of E2E tests in eclair to run against this branch when I have a bit more time, I'll let you know the results.

@glozow glozow added the Wallet label Sep 23, 2022
@t-bast
Copy link
Contributor
t-bast commented Sep 23, 2022

I ran a first set of tests from within eclair against a07ac02, and everything is looking good 👍

Copy link
Contributor
@ishaanam ishaanam left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

@murchandamus murchandamus force-pushed the 2022-09-ancestor-aware-funding branch from a07ac02 to a720b50 Compare September 27, 2022 19:15
@murchandamus
Copy link
Contributor Author
murchandamus commented Sep 27, 2022

@jonatack, @t-bast: Thanks for the review and testing. I made an attempt of getting rid of the getters on MockMempoolEntry, but what I did interfered with the calls made on properties of actual mempool entries. Will have to shift my approach.

@murchandamus murchandamus force-pushed the 2022-09-ancestor-aware-funding branch from a720b50 to 337a24e Compare September 27, 2022 20:25
@murchandamus
Copy link
Contributor Author
murchandamus commented Sep 27, 2022

@glozow: Maybe for

This needs to obey -maxtxfee

Maybe we can add a maxtxfee check to the filter introduced in #25729 for max weight after input sets are produced for different subsets of the available coins. Perhaps a separate PR that builds both on this one here and #25729.

@jonatack, @t-bast:
Fixed whitespace issues, applied the propose change to a class for the struct MockMempoolEntry, amended comments in Chain interface. Thanks!

Copy link
Member
@glozow glozow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to obey -maxtxfee

Maybe we can add a maxtxfee check to the filter introduced in #25729 for max weight after input sets are produced for different subsets of the available coins. Perhaps a separate PR that builds both on this one here and #25729.

Not sure if this is scope creep, but seems like breaking maxtxfee would be a bug and should probably be done in the same PR? Why not build this on top of #25729? ignore

@murchandamus murchandamus force-pushed the 2022-09-ancestor-aware-funding branch from 970c848 to f9979ef Compare October 27, 2022 18:53
@murchandamus
Copy link
Contributor Author
  • Reordered members vs initialization
  • Removed special casing of UTXOs without relatives in CalculateBumpFee
  • Call CalculateBumpFee once for the whole UTXO pool instead of introducing chain-interface dependency on every UTXO

Todos:

  • Prevent exceeding maxtxfee
  • Add test for bumpfee RPC
  • Add secondary modus for CalculateBumpFee that treats the provided UTXOs as being spent together. This allows us to recalculate the bumpfee of all inputs together to resolve our overpayment caveat.

@murchandamus
Copy link
Contributor Author

@glozow: looking more into this, I realized that the maxtxfee refers to an absolute fee, not a feerate. maxtxfee is checked after a transaction is built, so I don't see how ancestor aware funding changes anything in regard to maxtxfee—we still check at the end whether the amount of fee is allowed, regardless how we calculated the fee. If you meant maxtxfeerate, that is used to check raw transactions on submission in sendrawtx, so it doesn't apply here either.

@glozow
Copy link
Member
glozow commented Nov 2, 2022

maxtxfee refers to an absolute fee, not a feerate. maxtxfee is checked after a transaction is built, so I don't see how ancestor aware funding changes anything in regard to maxtxfee

Ah for some reason I thought it was a feerate, apologies. Question: is it better to only enforce -maxtxfee on the fees paid for the tx itself and not on the fees used to bump its ancestors? Or would the user expect that it's applied to any tx, bumping or not?

@achow101
Copy link
Member
achow101 commented Nov 2, 2022

Question: is it better to only enforce -maxtxfee on the fees paid for the tx itself and not on the fees used to bump its ancestors? Or would the user expect that it's applied to any tx, bumping or not?

I think that -maxtxfee should be expected to behave the same regardless of bumping, it's a context-free check.

@murchandamus
Copy link
Contributor Author

@S3RK noticed that SelectionResult::GetBumpFeeDiscount() was now obsolete and no longer used. (AFAIR, this would have been a bi-product of moving GetSelectionWaste into SelectionResult.)

I removed the obsolete method.

@S3RK
Copy link
Contributor
S3RK commented Sep 13, 2023

ACK f18f9ef

Great work! This PR not only makes our wallet work properly with unconfirmed inputs, but it also improved the factoring of the code. Happy to see waste calculations as a member function of SelectionResult

@murchandamus
Copy link
Contributor Author
murchandamus commented Sep 13, 2023

@t-bast, @ismaelsadeeq, @achow101: Changes since 1b2fb54d5c18579e013b1dfd061c2b29d8cffdc2

  • Pass chain interface to AttemptSelection(…) instead of whole wallet
  • Remove obsolete method SelectionResult::GetBumpFeeDiscount()
  • Add waste calculation test for changeless transactions with bump fees
  • Fix typo

@DrahtBot DrahtBot requested review from ismaelsadeeq and removed request for ismaelsadeeq September 13, 2023 20:27
@DrahtBot DrahtBot requested review from ismaelsadeeq and removed request for ismaelsadeeq September 13, 2023 21:48
Copy link
Contributor
@brunoerg brunoerg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

crACK f18f9ef

code lgtm, nice improvement!

@DrahtBot DrahtBot requested review from ismaelsadeeq and removed request for ismaelsadeeq September 13, 2023 22:11
@ismaelsadeeq
Copy link
Member

ACK f18f9ef

@DrahtBot DrahtBot removed the request for review from ismaelsadeeq September 14, 2023 11:49
@fanquake fanquake added this to the 26.0 milestone Sep 14, 2023
Copy link
Contributor
@t-bast t-bast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK f18f9ef, I reviewed the latest changes and run e2e tests against eclair, everything looks good 👍

@achow101
Copy link
Member

ACK f18f9ef

@DrahtBot DrahtBot removed the request for review from achow101 September 14, 2023 20:00
@achow101 achow101 merged commit 459272d into bitcoin:master Sep 14, 2023
Frank-GER pushed a commit to syscoin/syscoin that referenced this pull request Sep 19, 2023
achow101 added a commit that referenced this pull request Jun 4, 2024
bd34dd8 Use `exact_target` shorthand in coinselector_tests (Murch)
7aa7e30 Fold GetSelectionWaste() into ComputeAndSetWaste() (Murch)

Pull request description:

  PR #26152 moved waste calculation into SelectionResult to be able to correct the waste score on basis of the bump_fee_group_discount for overlapping ancestries. This left two functions with largely overlapping purpose, where one was simply a wrapper of the other. This PR cleans up the overlap, and fixes the double-meaning of `change_cost` where the `GetChange()` function assumed that no change was created when `change_cost` was set to 0. This behavior was exploited in a bunch of tests, but is problematic, because a `change_cost` of 0 is permitted with custom settings for feerate and discard_feerate (i.e. when they’re both 0).

ACKs for top commit:
  achow101:
    ACK bd34dd8
  furszy:
    Code ACK bd34dd8
  ismaelsadeeq:
    Code Review ACK bd34dd8

Tree-SHA512: 83a2688d45d719dc61a64b5180fe136107faccf401a59df65245c05d701748a03e85ed56fde8c9b7ef39a3ab54374dd3718c559bda5b3f55dafedfd7fed25161
@bitcoin bitcoin locked and limited conversation to collaborators Sep 13, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
0