Conversation
Benchmark Test ResultsBenchmark results from the latest changes vs base branch |
705eaf2 to
c741b05
Compare
modify spdx22 to include relationships export spdxJson constuctor for poweruser consumption Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
latest patch for json schema: https://github.com/spdx/spdx-spec/blob/development/v2.2.1/schemas/spdx-schema.json See this pr: spdx/spdx-spec#528 See this comment: spdx/spdx-spec#528 (comment) Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Add basic functionality for generating contains relationships for packages. Code needs to be refactored with a second pass to clean up signatures. The right format will be discovered when we try to build multiple types of relationship for the relationship field. Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
c741b05 to
db373d0
Compare
|
This is a very exciting addition to Syft! Keep up the great work, @spiffcs :) |
| @@ -1,3 +1,4 @@ | |||
| //go:build ignore | |||
There was a problem hiding this comment.
I think you mentioned this line is getting added by accident. You might want to make a local adjustment to avoid accidentally checking this in 🤔
There was a problem hiding this comment.
Yep - thought I removed this configuration but it seems like it's back with a vengence. I'll look at what magic nonsense exists within my defaults that is appending this.
There was a problem hiding this comment.
@luhring looks like my tooling is trying to migrate us to the current spec:
https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
There was a problem hiding this comment.
Better explanation of the transition
https://stackoverflow.com/questions/68360688/whats-the-difference-between-gobuild-and-build-directives
I'm going to remove it for now, but will keep an eye on the transition going forward. I don't know if it's my LSP that is doing this or if it's some configuration I updated in my dotfiles when go1.17 dropped which has slipped my mind.
| @@ -39,7 +39,8 @@ func (pres *SPDXJsonPresenter) Present(output io.Writer) error { | |||
| return enc.Encode(&doc) | |||
| } | |||
|
|
|||
| func newSPDXJsonDocument(catalog *pkg.Catalog, srcMetadata source.Metadata) spdx22.Document { | |||
| // NewSPDXJsonDocument creates and populates a new JSON document struct that follows the SPDX 2.2 spec from the given cataloging results. | |||
| func NewSPDXJsonDocument(catalog *pkg.Catalog, srcMetadata source.Metadata) spdx22.Document { | |||
There was a problem hiding this comment.
Just curious: Why did this get exported?
There was a problem hiding this comment.
We export NewJsonDocument as it is consumed by the poweruser package. It seemed incorrect to have one Document constructor exposed while the other was private. I also think it's something that should have an official Documentation string highlighted above the function since its counterpart would also end up in the godoc if generated.
There was a problem hiding this comment.
Gotcha, interesting! My thoughts, in no particular order:
- 100% agree re: doc comments 🌟
- Architectural question: Do we plan on creating SPDX documents for any non-package commands?
- My pref in general is not to export something unless it's being used internally in another package, or we expect it to be used by a consumer of our module. In this case, it looks like this code is under
internal/so I don't think it would be consumed by another module. Is that right?
There was a problem hiding this comment.
lowercased - if we start doing anything in poweruser with the function we can export it then.
This is good information. I think today we have some catalogers that raise up transitive dependencies and others that only raise up direct dependencies. This implies a couple of things:
|
Yea this is the direction we should proceed. I can start tonight looking at how to make those changes as part of this PR. I don't want this going in if |
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
75ae899 to
8ebd51d
Compare
|
After talking with @wagoodman we're going to keep some parts of this branch. We'll table the cc @luhring for an update on progress/direction |
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
57a28f4 to
45cadf6
Compare
010268a to
46ee410
Compare
SPDX json Schema documentation: https://github.com/spdx/spdx-spec/blob/a08ffa4b883b35574ae0df8f4566cd031341ba3e/schemas/spdx-schema.json Required fields: [ "SPDXID", "fileName", "copyrightText", "licenseConcluded" ] Add `omitempty` to non required fields. Add depricated fields to old SPDX fields Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
46ee410 to
ee19181
Compare
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
There was a problem hiding this comment.
We tend to add information onto to common model that gets used by the presenters when possible. This PR is adding the relationship within a specific presenter --I think this is the only way to go to keep this PR small, but that does hint at future refactor that we can iterate on.
| // it might to use a matcher like https://github.com/gabriel-vasile/mimetype. | ||
| // This would need to be implemented when we move into reading things from disk | ||
| // rather than digesting package metadata from the different package managers. | ||
| func matchFileTypes(baseFileName string) []string { |
There was a problem hiding this comment.
just for fyi, no action requested: there is a prototype branch for stereoscope to add MIME type detection https://github.com/anchore/stereoscope/compare/add-mimetype , this would need to be supported for the directory-resolver as well, but this at least covers containers.
|
|
||
| for _, ownedFilePath := range pkgFileOwner.OwnedFiles() { | ||
| baseFileName := filepath.Base(ownedFilePath) | ||
| fileSpdxID := spdx22.ElementID(fmt.Sprintf("File-%s-%s", p.Name, baseFileName)).String() |
There was a problem hiding this comment.
I'm not certain the base name will be good enough for an ID --what if there are two files with the same names in different paths for the same package? Could we use the whole path? or a hash of the path?
There was a problem hiding this comment.
That's a good point - I can hash the full path and add it as a suffix to this Sprintf
|
|
||
| files = append(files, spdx22.File{ | ||
| FileName: ownedFilePath, | ||
| FileTypes: matchFileTypes(baseFileName), |
There was a problem hiding this comment.
since this function isn't implemented yet we could probably delete it and the FileTypes entry here (if the spec permits it). (would the basename be enough to lookup the filetype here? I'm not certain it would)
There was a problem hiding this comment.
The basename is not enough in this case for file detection. I'll remove it as it's not a required type.
d9af81e to
5f4544e
Compare
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
5f4544e to
95861d9
Compare
) * update spdx22 Document model to include relationships field Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com> * update document and relationship to match current JSON spec https://github.com/spdx/spdx-spec/blob/development/v2.2.1/schemas/spdx-schema.json spdx/spdx-spec#528 spdx/spdx-spec#528 (comment) Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com> * update File struct based on SPDX schema Required fields: [ "SPDXID", "fileName", "copyrightText", "licenseConcluded" ] Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Work for #476
Initial Support for
RELATIONSHIPSin thespdx-jsonoutput.This PR adds support for the following:
filesfield of the spdxJSON outputPackageswith the new File resources in thehasFilesfield.Relationshipfield with the vertices associating Files and PackagesFile support only comes from
catalogerswho support theFileOwnerinterface.The current list is
Link to JSON schema:
https://github.com/spdx/spdx-spec/blob/a08ffa4b883b35574ae0df8f4566cd031341ba3e/schemas/spdx-schema.json#L423-L558
Snippets of the new output generated with the
spdxJsonoption - pullfrom node:latest