8000 Provide and retrieve artifacts via the kademlia p2p network by tiainen · Pull Request #477 · pyrsia/pyrsia · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Provide and retrieve artifacts via the kademlia p2p network #477

New issue

Have a question about 8000 this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Mar 10, 2022
Merged

Provide and retrieve artifacts via the kademlia p2p network #477

merged 4 commits into from
Mar 10, 2022

Conversation

tiainen
Copy link
Collaborator
@tiainen tiainen commented Mar 9, 2022

PR Checklist

  • I've built the code cargo build. For major changes, cargo build --workspace --release is recommended.
  • I've run the automated unit tests cargo test. This executes our automated unit tests.
  • I've not broken any existing tests or functionality. In addition to cargo test, I've run cargo clippy
  • I've not introduced any new known security vulnerabilities. I've run cargo audit
  • I've included a brief description and a good title of the proposed changes and how to test them.
  • I've read the contributing guidelines.
  • I've associated an issue with this Pull Request.
  • I've checked that the code is up-to-date with the pyrsia/main branch.
  • I've read "What is a Good PR?"
  • I've assigned this Pull Request to "pyrsia/collaborators"

Description

Fixes #444

This PR introduces the following changes:

  1. A node can provide an artifact via its hash into the p2p network.
  2. When a node needs an artifact that does not yet exist locally, it retrieves a list of providers in the p2p network and requests it from the first provider peer found. If no providers exist, it falls back to fetching the artifact from docker.io.
  3. Providing the boot node with --peer no longer requires the PeerId of that node. One can simple pass the IP and port, i.e.: --peer /ip4/127.0.0.1/tcp/41105
  4. Added the local PeerId in the status command: 
    $ curl http://127.0.0.1:7888/status
{
  "peers_count": 0,
  "peer_id": "12D3KooWRGFnz3uYujWgD3jTMey2cazzfMYYfSdUhf1MNJKgsgcu",
  "artifact_count": 0,
  "disk_allocated": "10.84 GB",
  "disk_usage": "0.0000"
}

@codecov
Copy link
codecov bot commented Mar 9, 2022

Codecov Report

Merging #477 (fa8b8f2) into main (95badf1) will decrease coverage by 0.13%.
The diff coverage is 0.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #477      +/-   ##
==========================================
- Coverage   60.55%   60.42%   -0.14%     
==========================================
  Files          24       24              
  Lines        2183     2188       +5     
==========================================
  Hits         1322     1322              
- Misses        861      866       +5
Impacted Files Coverage Δ
src/docker/v2/handlers/blobs.rs 0.00% <ø> (ø)
src/docker/v2/routes.rs 0.00% <0.00%> (ø)
src/network/handlers.rs 0.00% <0.00%> (ø)
src/network/p2p.rs 0.00% <0.00%> (ø)
src/node_api/handlers/swarm.rs 0.00% <ø> (ø)
src/node_manager/model/cli.rs 0.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95badf1...fa8b8f2. Read the comment docs.

@tiainen tiainen requested review from a team, MitaliBo and fishseabowl and removed request for a team March 9, 2022 12:01
prince-chrismc
prince-chrismc reviewed Mar 9, 2022
View reviewed changes
prince-chrismc
prince-chrismc approved these changes Mar 10, 2022
View reviewed changes
Copy link
Contributor
@prince-chrismc prince-chrismc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

@erwin1 erwin1 mentioned this pull request Mar 10, 2022
Closed
@erwin1 erwin1 merged commit 1496cce into pyrsia:main Mar 10, 2022
AbhijithGanesh added a commit to AbhijithGanesh/pyrsia that referenced this pull request Mar 16, 2022
@AbhijithGanesh @erwin1
@tiainen @betarelease @dependabot
Zap openssl (#20)
50b17d6 
* keep all logs in release build and default to DEBUG log output for demo (part of pyrsia#442) (pyrsia#480)

* Provide and retrieve artifacts via the kademlia p2p network (pyrsia#477)

* lookup blobs via kademlia content provisioning
* use dial and identify to add new node to dht
* add local peer_id to status

* Update source folder readme with more terminology and sub-crate (pyrsia#446)



Co-authored-by: Sudhindra Rao <41690+betarelease@users.noreply.github.com>

Co-authored-by: Erwin Morrhey <erwin@lodgon.com>

Co-authored-by: Sudhindra Rao <41690+betarelease@users.noreply.github.com>
Co-authored-by: Erwin Morrhey <erwin@lodgon.com>

* Bump clap from 3.1.5 to 3.1.6 (pyrsia#474)

Bumps [clap](https://github.com/clap-rs/clap) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](clap-rs/clap@v3.1.5...v3.1.6)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump once_cell from 1.9.0 to 1.10.0 (pyrsia#485)

Bumps [once_cell](https://github.com/matklad/once_cell) from 1.9.0 to 1.10.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](matklad/once_cell@v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Introduce the `KeyBox` for consensus engine (pyrsia#464)

* delete signed structs

build passes

* more traces of openssl

* cargo fmt

Co-authored-by: Erwin Morrhey <erwin@lodgon.com>
Co-authored-by: Joeri Sykora <joeri@sertik.net>
Co-authored-by: Chris Mc <christopherm@jfrog.com>
Co-authored-by: Sudhindra Rao <41690+betarelease@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
AbhijithGanesh added a commit that referenced this pull request Mar 29, 2022
@AbhijithGanesh @erwin1
@tiainen @betarelease @dependabot @prince-chrismc
Removal of unused signed_struct to reduce OpenSSL requirement (#494)
677cb7a 
* Removal of OpenSSL dependencies - Signed_struct and Struct
* Zap openssl (#20)
* Keep all logs in release build and default to DEBUG log output for demo (part of #442) (#480)
* Provide and retrieve artifacts via the kademlia p2p network (#477)
* lookup blobs via kademlia content provisioning
* use dial and identify to add new node to dht
* add local peer_id to status
* Update source folder readme with more terminology and sub-crate (#446)
* Bump clap from 3.1.5 to 3.1.6 (#474)
Bumps [clap](https://github.com/clap-rs/clap) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](clap-rs/clap@v3.1.5...v3.1.6)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

* Bump once_cell from 1.9.0 to 1.10.0 (#485)

Bumps [once_cell](https://github.com/matklad/once_cell) from 1.9.0 to 1.10.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](matklad/once_cell@v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

* Introduce the `KeyBox` for consensus engine (#464)
* delete signed structs
* more traces of openssl
* cargo fmt
* add `now_as_iso8601_string` for testing
* Resolution of errors
* Restoring ArtifactBuilder
* Removal of _hash
* Suggested changes - Chris
* Update Dockerfile
* fix clippy warning by move import to correct spot
* fix import
* Update src/docker/v2/handlers/manifests.rs
* Update src/metadata_manager/metadata.rs
* Update src/docker/v2/handlers/manifests.rs
* Updates to signed functions
* Cargo format fix
* Updates to dev-dep
* Changes of methods to fields


Signed-off-by: Abhijith Ganesh <67182544+AbhijithGanesh@users.noreply.github.com>

Co-authored-by: Erwin Morrhey <erwin@lodgon.com>
Co-authored-by: Joeri Sykora <joeri@sertik.net>
Co-authored-by: Chris Mc <christopherm@jfrog.com>
Co-authored-by: Sudhindra Rao <41690+betarelease@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Chris Mc <prince.chrismc@gmail.com>
Co-authored-by: Joeri Sykora <joeri.sykora@gluonhq.com>
@tiainen tiainen deleted the 444-provide-blobs-in-p2p branch December 8, 2022 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@chb0github chb0github chb0github approved these changes

@prince-chrismc prince-chrismc prince-chrismc approved these changes

@MitaliBo MitaliBo Awaiting requested review from MitaliBo MitaliBo is a code owner automatically assigned from pyrsia/collaborators

@fishseabowl fishseabowl Awaiting requested review from fishseabowl fishseabowl is a code owner automatically assigned from pyrsia/collaborators

Assignees

@tiainen tiainen

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

A Pyrsia node finds any p2p node that offers the requested artifact
4 participants
@tiainen @chb0github @prince-chrismc @erwin1
0