8000 Remove Sock::Get() and Sock::Sock() by vasild · Pull Request #26312 · bitcoin/bitcoin · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Remove Sock::Get() and Sock::Sock() #26312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 3, 2023
Merged

Conversation

vasild
Copy link
Contributor
@vasild vasild commented Oct 14, 2022

This is a piece of #21878, chopped off to ease review.

Peeking at the underlying socket file descriptor of Sock and checkig if it is INVALID_SOCKET is bad encapsulation and stands in the way of testing/mocking/fuzzing.

Instead use an empty unique_ptr to denote that there is no valid socket where appropriate or outright remove such checks where they are not necessary.

The default constructor Sock::Sock() is unnecessary now after recent changes, thus remove it.

@vasild vasild mentioned this pull request Oct 14, 2022
12 tasks
@DrahtBot
Copy link
Contributor
DrahtBot commented Oct 14, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK ajtowns, jonatack
Concept ACK theStack

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #27375 (net: support unix domain sockets for -proxy and -onion by pinheadmz)
  • #26812 (test: add end-to-end tests for CConnman and PeerManager by vasild)
  • #21878 (Make all networking code mockable by vasild)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@yancyribbens
Copy link
Contributor

Why not add additional testing/mocking/fuzzing in this PR since this seems to be laying the groundwork for that?

@vasild
Copy link
Contributor Author
vasild commented Oct 17, 2022

@yancyribbens, to ease review. Right now, #21878 consists of this PR + the tests (it is almost complete 🎈). If somebody has spare review cycles they can review/ACK #21878 directly.

@yancyribbens
Copy link
Contributor
yancyribbens commented Oct 17, 2022

@vasild If the tests are not yet complete, then that may change this PR, right? Besides adding more safety, seeing the accompanying tests would make this easier (for me anyway) to review.

@vasild
Copy link
Contributor Author
vasild commented Oct 17, 2022

If the tests are not yet complete, then that may change this PR, right?

The tests are "complete" in a sense that they are "production ready" and can be merged. "Not complete" in a sense that more tests can be added. I do not think that tweaking the tests can change this PR. The point of #21878 is to have high level code not deal with int sockets (file descriptor), but to deal with Sock instead, which can be mocked together with all the relevant operations (connect, accept, read, write, etc).

Besides adding more safety, seeing the accompanying tests would make this easier (for me anyway) to review.

The tests are in #21878, feel free to look at them ❤️

@yancyribbens
Copy link
Contributor

.. to have high level code not deal with int sockets (file descriptor), but to deal with Sock instead

Thanks for the additional explanation. I was wondering if you could explain this a bit more. It's been a while since i've done anything with int sockets but if I remember, it was in C (not C++). Is Sock a C++ construct that makes it easier to Mock?

The tests are in #21878, feel free to look at them heart

I looked over 21878 and I see there's some additional fuzz testing however I don't see the Mocks that this PR enables. Am I missing the place where the Mocks are tested or is it yet to be added? Would this PR be merged before 21878?

@vasild
Copy link
Contributor Author
vasild commented Oct 25, 2022

The int sockets is the API provided by the OS. It is in C. We call it from the C++ code.

Before #21878 we would use that OS C API directly from all over the place. E.g.:

int s = socket(...); // create the socket, see man 2 socket
...
connect(s, to_addr); // see man 2 connect

The problem with this is that in order to test it we have to create a real socket that really connects to the given address which is very inconvenient. It cannot be mocked.

#21878 aims to replace all such calls to the operating system with constructs like

Sock sock = CreateSock(); // can return Sock, FuzzedSock, StaticContentsSock or another class that implements the interface
sock->Connect(to_addr);

This way a mock Sock class can be used and the high level code avoids calling the OS socket API at all in tests or dealing/touching int sockets at all. The last piece of that is the removal of Sock::Get() and changing the high level code which checks whether the int socket stored in Sock is INVALID_SOCKET.

@yancyribbens
Copy link
Contributor

@vasild

The problem with this is that in order to test it we have to create a real socket that really connects to the given address which is very inconvenient. It cannot be mocked.

I understand. What I am asking is if you wouldn't mind adding the Mock tests to this PR, or create a new PR with the Mocks before merging this one?

@vasild
Copy link
Contributor Author
vasild commented Oct 25, 2022

What I am asking is if you wouldn't mind adding the Mock tests to this PR,

I assume that by "the Mock tests" you mean the tests that are in the last 4 commits of #21878, right? If I add them to this PR then this PR would be the same as #21878.

I created this PR as a smaller piece of #21878 to ease review. If you want to peek or fully review the tests, then see the last 4 commits in #21878.

or create a new PR with the Mocks before merging this one?

Because those tests depend on removing the Sock::Get() call which is done in this PR. If the tests get merged first, this may happen: CConnman::OpenNetworkConnection() -> CConnman::ConnectNode() -> i2p::sam::Session::Connect() and later i2p::sam::Session::~Session() -> i2p::sam::Session::Disconnect() -> Sock::Get(). Calling Get() on FuzzedSock is a bit fishy, the high level code shouldn't peek at the internal int socket and make decisions based on its value.

@yancyribbens
Copy link
Contributor

@vasild

Are there any Mock tests that are not part of fuzz tests?

@vasild
Copy link
Contributor Author
vasild commented Oct 25, 2022

There is one in src/test/i2p_tests.cpp.

@yancyribbens
Copy link
Contributor

Which PR is the one Mock in src/test/i2p_tests.cpp, or is it already merged?

@vasild
Copy link
Contributor Author
vasild commented Oct 26, 2022

It is already merged.

@yancyribbens
Copy link
Contributor

I'm confused. I thought this PR was required before a Mock could be used.

@vasild
Copy link
Contributor Author
vasild commented Oct 26, 2022

FuzzedSock and StaticContentsSock are mock implementations of Sock that are already used in master.

Depending on the code being tested various methods of Sock (or a mocked implementation) are going to be called. For example, if you write an unit or fuzz test for CConnman::SocketSendData() that is only going to invoke Sock::Send().

@vasild
Copy link
Contributor Author
vasild commented Dec 12, 2022

7b6fe6761a...666c7584cf: rebase due to conflicts

@jonatack
Copy link
Member
jonatack commented Feb 1, 2023

ACK 666c758 modulo trivial rebase

@vasild
Copy link
Contributor Author
vasild commented Feb 9, 2023

666c7584cf...c40c25763c: rebase due to conflicts

Invalidates ACK from @jonatack

8000 Copy link
Member
@jonatack jonatack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-ACK c40c257 per git range-diff dc905f6 666c758 c40c257

It's good to have them in place, just noting that the Sock::IsConnected() virtual member function overrides in src/test/util/net.h and src/test/fuzz/util/net.{h.cpp} aren't used currently or here or in the parent yet.

@jonatack
Copy link
Member

re-ACK e536b3a per git range-diff be2e748 c40c257 e536b3a

@jonatack
Copy link
Member

This is the last part of parent #21878 opened 2 years ago, anyone else like to have a look?

Copy link
Contributor
@theStack theStack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

Copy link
Contributor
@ajtowns ajtowns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach ACK

vasild added 3 commits August 24, 2023 14:39
Peeking at the underlying socket file descriptor of `Sock` and checkig
if it is `INVALID_SOCKET` is bad encapsulation and stands in the way of
testing/mocking/fuzzing.

Instead use an empty unique_ptr to denote that there is no valid socket.
The socket is always valid (the underlying file descriptor is not
`INVALID_SOCKET`) when `GetBindAddress()` is called.
The socket is always valid (the underlying file descriptor is not
`INVALID_SOCKET`) when `ConnectSocketDirectly()` is called.
@vasild
Copy link
Contributor Author
vasild commented Aug 24, 2023

e536b3a67b...b44e493890: address suggestions

vasild added 2 commits August 25, 2023 14:41
`Sock::Get()` was used only in `sock.{cpp,h}`. Remove it and access
`Sock::m_socket` directly.

Unit tests that used `Get()` to test for equality still verify that the
behavior is correct by using the added `operator==()`.
@vasild
Copy link
Contributor Author
vasild commented Aug 25, 2023

b44e493890...384e9e3552: reword a commit message, see #26312 (comment)

Copy link
Member
@jonatack jonatack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 384e9e3

Use also a socket for the moved-to object and check which one is closed when.
@vasild
Copy link
Contributor Author
vasild commented Aug 28, 2023

384e9e3552...7df4508369: add one more check in the unit test: #26312 (comment)

@ajtowns
Copy link
Contributor
ajtowns commented Aug 29, 2023

ACK 7df4508

@DrahtBot DrahtBot requested a review from jonatack August 29, 2023 18:04
@jonatack
Copy link
Member

ACK 7df4508

@DrahtBot DrahtBot removed the request for review from jonatack August 30, 2023 16:47
@jonatack
Copy link
Member
jonatack commented Oct 2, 2023

rfm or is anything left to be done here?

@ryanofsky
Copy link
Contributor

rfm or is anything left to be done here?

@vasild pinged me on irc, and agree looks this looks ready for merge. Will merge shortly

@ryanofsky ryanofsky merged commit d0b928b into bitcoin:master Oct 3, 2023
@vasild vasild deleted the remove_Sock_Get branch October 3, 2023 16:09
@vasild
Copy link
Contributor Author
vasild commented Oct 4, 2023

Thank you! This concludes #21878

Some more fuzz tests for review in #28584.

Frank-GER pushed a commit to syscoin/syscoin that referenced this pull request Oct 13, 2023
7df4508 test: improve sock_tests/move_assignment (Vasil Dimov)
5086a99 net: remove Sock default constructor, it's not necessary (Vasil Dimov)
7829272 net: remove now unnecessary Sock::Get() (Vasil Dimov)
944b21b net: don't check if the socket is valid in ConnectSocketDirectly() (Vasil Dimov)
aeac68d net: don't check if the socket is valid in GetBindAddress() (Vasil Dimov)
5ac1a51 i2p: avoid using Sock::Get() for checking for a valid socket (Vasil Dimov)

Pull request description:

  _This is a piece of bitcoin#21878, chopped off to ease review._

  Peeking at the underlying socket file descriptor of `Sock` and checkig if it is `INVALID_SOCKET` is bad encapsulation and stands in the way of testing/mocking/fuzzing.

  Instead use an empty `unique_ptr` to denote that there is no valid socket where appropriate or outright remove such checks where they are not necessary.

  The default constructor `Sock::Sock()` is unnecessary now after recent changes, thus remove it.

ACKs for top commit:
  ajtowns:
    ACK 7df4508
  jonatack:
    ACK 7df4508

Tree-SHA512: 9742aeeeabe8690530bf74caa6ba296787028c52f4a3342afd193b05dbbb1f6645935c33ba0a5230199a09af01c666bd3c7fb16b48692a0d185356ea59a8ddbf
HiHat added a commit to HiHat/pocketnet.core that referenced this pull request Sep 29, 2024
p2p: Refactor sock to add I2P unit tests (backported from bitcoin/bitcoin#21387)
refactor: replace CConnman::SocketEvents() with mockable Sock::WaitMany() (backported from bitcoin/bitcoin#24356)
refactor: make setsockopt() and SetSocketNoDelay() mockable/testable (backported from bitcoin/bitcoin#24357)
I2P: add support for transient addresses for outbound connections (backported from bitcoin/bitcoin#25355)
Remove Sock::Get() and Sock::Sock() (backported from bitcoin/bitcoin#26312)
Remove now-unnecessary poll, fcntl includes from net(base).cpp (backported from bitcoin/bitcoin#27530)
@bitcoin bitcoin locked and limited conversation to collaborators Oct 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants
0