8000 feat: gitlab fetch by lmphil · Pull Request #229 · bomctl/bomctl · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: gitlab fetch #229

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 25 commits into from
Dec 18, 2024
Merged

feat: gitlab fetch #229

merged 25 commits into from
Dec 18, 2024

Conversation

lmphil
Copy link
Contributor
@lmphil lmphil commented Nov 27, 2024 •
edited
Loading

Description

This pull request adds the ability to fetch an SBOM from GitLab using the Dependency List Export web API endpoint.

Fixes #131

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How Has This Been Tested?

  • Manual tests.
  • Unit tests.

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • I have checked my code and corrected any misspellings

@lmphil
feat: add gitlab fetch
95c986a 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
Merge branch 'main' of https://github.com/bomctl/bomctl into gitlab-c…
61f5037 
…lient-support

Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil lmphil self-assigned this Nov 27, 2024
@ghost
Copy link
ghost commented Nov 27, 2024 •
edited by ghost
Loading

Minder Vulnerability Report ✅

Minder analyzed this PR and found it does not add any new vulnerable dependencies.

Vulnerability scan of c0c623ec:

  • 🐞 vulnerable packages: 0
  • 🛠 fixes available for: 0

@lmphil lmphil mentioned this pull request Nov 27, 2024
Closed
@lmphil
fix linter warning
1518d41 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
add unit tests (wip)
a8ea8e4 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
fix linter warnings
2078e05 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
jhoward-lm
jhoward-lm previously requested changes Dec 5, 2024
View reviewed changes
8000
lmphil added 3 commits December 5, 2024 19:06
@lmphil
update gitlab fetch unit tests
4a42c8c 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
refactor
98303b1 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
Merge branch 'main' of https://github.com/bomctl/bomctl into gitlab-c…
573c105 
…lient-support

Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@jhoward-lm
Copy link
Contributor
jhoward-lm commented Dec 6, 2024 •
edited
Loading

I didn't explain what I was getting at for implementing a mockable pattern very well during our call, so hopefully this will make more sense. In my opinion, this type of implementation might make the code easier to read, maintain, and test/mock.

The TL;DR version is even with nested types and interfaces, it might simplify things to abstract everything into a common flattened interface. This interface can then be implemented as a mock much more easily.

  1. Define interfaces for methods you want to mock and embed these interfaces as fields on your Client struct definition

    Example 
    // internal/pkg/client/gitlab/client.go

type (
	BranchesService interface {
		GetBranch(pid any, branch string, options ...gitlab.RequestOptionFunc) (*gitlab.Branch, *gitlab.Response, error)
	}

	ProjectsService interface {
		GetProject(pid any, opt *gitlab.GetProjectOptions, options ...gitlab.RequestOptionFunc) (*gitlab.Project, *gitlab.Response, error)
	}

	CommitsService interface {
		GetCommit(pid any, sha string, opt *gitlab.GetCommitOptions, options ...gitlab.RequestOptionFunc) (*gitlab.Commit, *gitlab.Response, error)
	}

	DependencyListExportService interface {
		CreateDependencyListExport(pipelineID int, opt *gitlab.CreateDependencyListExportOptions, options ...gitlab.RequestOptionFunc) (*gitlab.DependencyListExport, *gitlab.Response, error)
		DownloadDependencyListExport(id int, options ...gitlab.RequestOptionFunc) (io.Reader, *gitlab.Response, error)
		GetDependencyListExport(id int, options ...gitlab.RequestOptionFunc) (*gitlab.DependencyListExport, *gitlab.Response, error)
	}

	Client struct {
		BranchesService
		ProjectsService
		CommitsService
		DependencyListExportService
	
		glClient *gitlab.Client
	}
)

Note

It might be more conventional or idiomatic to name the interfaces as "what they do" instead, such as BranchGetter, ProjectGetter, etc., which would also distinguish them from their concrete type counterparts in the upstream go-gitlab library.

  1. If one of these interfaces is unset in the code before trying to call its interface method, set it to a default value

    Example 
    func (client *Client) createExport(projectName, branchName string) error {
	if client.ProjectsService == nil {
		client.ProjectsService = client.glClient.Projects
	}

	project, response, err := client.ProjectsService.GetProject(projectName, nil)
	// ...
}

  2. (optional) Explicitly implement interface methods if it's more convenient, although this might cause embedded method name conflicts

    Example 
    func (client *Client) GetBranch(pid any, branch string, options ...gitlab.RequestOptionFunc) (*gitlab.Branch, *gitlab.Response, error) {
	result, resp, err := client.glClient.Branches.GetBranch(pid, branch, options...)
	if err != nil {
		return nil, nil, fmt.Errorf("%w", err)
	}

	return result, resp, nil
}

ashearin and others added 5 commits December 6, 2024 10:39
@ashearin
Merge branch 'main' into gitlab-client-support
57b5e05
@lmphil
Merge branch 'main' of https://github.com/bomctl/bomctl into gitlab-c…
f81a8f6 
…lient-support

Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
Merge branch 'gitlab-client-support' of https://github.com/lmphil/bomctl
e93681c 
 into gitlab-client-support
@lmphil
refactor test
0fb839c 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
refactor unit test
aea9562 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil lmphil force-pushed the gitlab-client-support branch from 40dc4d9 to aea9562 Compare December 10, 2024 03:37
lmphil added 2 commits December 9, 2024 23:38
@lmphil
modify revive disable for mocks
de7a196 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
Merge branch 'main' of https://github.com/bomctl/bomctl into gitlab-c…
fdd473a 
…lient-support
@lmphil lmphil marked this pull request as ready for review December 10, 2024 03:44
@lmphil lmphil requested a review from a team as a code owner December 10, 2024 03:44
lallevato-lm
lallevato-lm reviewed Dec 10, 2024
View reviewed changes
jhoward-lm
jhoward-lm reviewed Dec 10, 2024
View reviewed changes
@lmphil
refactor
5b36981 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@ashearin ashearin requested a review from jhoward-lm December 10, 2024 21:23
@ashearin
Copy link
Member

Heads up: the go-gitlab package was deprecated....today: https://pkg.go.dev/github.com/xanzy/go-gitlab#section-readme

@ashearin
Copy link
Member
ashearin commented Dec 11, 2024 •
edited
Loading

What am I doing wrong: (running on your branch obvs)

$ go run . fetch https://www.gitlab.com/hoppr/hoppr@dev
INFO  fetch: Fetching from GitLab URL url=https://www.gitlab.com/hoppr/hoppr@dev
FATAL fetch: failed to save document: adding document: parsing SBOM data: detecting SBOM format: detecting format: unknown SBOM format
exit status 1

@lmphil
update deprecated dependency and fix gitlab fetch
64e3212 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
Copy link
Contributor Author
lmphil commented Dec 11, 2024

What am I doing wrong: (running on your branch obvs)

$ go run . fetch https://www.gitlab.com/hoppr/hoppr@dev
INFO  fetch: Fetching from GitLab URL url=https://www.gitlab.com/hoppr/hoppr@dev
FATAL fetch: failed to save document: adding document: parsing SBOM data: detecting SBOM format: detecting format: unknown SBOM format
exit status 1

It was a bug on my part. Sorry. It's fixed now.

@lmphil
remove debug launch config
c28ef42 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil lmphil marked this pull request as draft December 12, 2024 23:00
@lmphil
refactor gitlab client for tests
fde1ecd 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
Merge branch 'main' of https://github.com/bomctl/bomctl into gitlab-c…
9eacb4e 
…lient-support
@lmphil lmphil marked this pull request as ready for review December 13, 2024 03:45
jhoward-lm
jhoward-lm reviewed Dec 16, 2024
View reviewed changes
@ashearin ashearin self-requested a review December 16, 2024 20:48
@lmphil
refactor gitlab client initialization
2bd9933 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
Merge branch 'main' of https://github.com/bomctl/bomctl into gitlab-c…
c6b5ed5 
…lient-support

Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
jhoward-lm
jhoward-lm reviewed Dec 17, 2024 F438
View reviewed changes
ashearin
ashearin reviewed Dec 17, 2024
View reviewed changes
@ashearin ashearin self-requested a review December 17, 2024 22:17
ashearin
ashearin reviewed Dec 17, 2024
View reviewed changes
ashearin
ashearin reviewed Dec 17, 2024
View reviewed changes
@lmphil
implement the new PrepareFetch method
986e7ac 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
use gitref instead of fragment for branch name
3e41ab7 
Signed-off-by: Philippe <philippe.a.aviles@lmco.com>
@lmphil
Merge branch 'main' of https://github.com/bomctl/bomctl into gitlab-c…
c0c623e 
…lient-support
@ashearin ashearin merged commit aff491d into bomctl:main Dec 18, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ashearin ashearin ashearin approved these changes

@lallevato-lm lallevato-lm lallevato-lm approved these changes

@jhoward-lm jhoward-lm jhoward-lm approved these changes

@pkwiatkowski1 pkwiatkowski1 Awaiting requested review from pkwiatkowski1

Assignees

@lmphil lmphil

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Initial Gitlab Client Skeleton and Sbom api Fetch
4 participants
@lmphil @jhoward-lm @ashearin @lallevato-lm
0