8000 Implement glob support for enumerators by moadibfr Β· Pull Request #353 Β· snyk/driftctl Β· GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Implement glob support for enumerators #353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 7, 2021
Merged

Conversation

moadibfr
Copy link
Contributor
@moadibfr moadibfr commented Mar 18, 2021
Q A
πŸ› Bug fix? no
πŸš€ New feature? yes
⚠ Deprecations? no
❌ BC Break yes
πŸ”— Related issues #316
❓ Documentation yes

Description

Limited glob support.
Support go glob syntax + ** syntax on file enumerator
Support go glob syntax on s3 enumerator

@moadibfr moadibfr requested a review from a team as a code owner March 18, 2021 17:49
@codecov
Copy link
codecov bot commented Mar 18, 2021

Codecov Report

Merging #353 (144ccd4) into v0.8 (8486419) will decrease coverage by 0.01%.
The diff coverage is 76.74%.

Impacted file tree graph

@@            Coverage Diff             @@
##             v0.8     #353      +/-   ##
==========================================
- Coverage   70.98%   70.96%   -0.02%     
==========================================
  Files         288      289       +1     
  Lines        6545     6562      +17     
==========================================
+ Hits         4646     4657      +11     
- Misses       1522     1525       +3     
- Partials      377      380       +3     
Impacted Files Coverage Ξ”
pkg/iac/terraform/state/enumerator/glob.go 63.63% <63.63%> (ΓΈ)
pkg/iac/terraform/state/enumerator/file.go 90.00% <85.71%> (+4.28%) ⬆️
pkg/iac/terraform/state/enumerator/s3.go 100.00% <100.00%> (ΓΈ)

@moadibfr moadibfr marked this pull request as draft March 18, 2021 17:51
@moadibfr moadibfr force-pushed the fea/glob_files_list branch from 5e6688d to 33033f7 Compare March 22, 2021 14:29
@moadibfr moadibfr marked this pull request as ready for review March 22, 2021 14:29
@moadibfr moadibfr force-pushed the fea/glob_files_list branch from 33033f7 to 0500da0 Compare March 22, 2021 14:37
sundowndev
sundowndev previously approved these changes Mar 23, 2021
Copy link
Contributor
@sundowndev sundowndev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@moadibfr moadibfr force-pushed the fea/glob_files_list branch 3 times, most recently from 22918a3 to c2a8d2a Compare March 26, 2021 17:45
@moadibfr moadibfr marked this pull request as draft March 29, 2021 15:04
@moadibfr moadibfr force-pushed the fea/glob_files_list branch from c2a8d2a to 7bd3591 Compare March 29, 2021 15:49
@moadibfr moadibfr changed the title glob filtering for s3 and file enumerators Limited glob support for enumerators Mar 29, 2021
@moadibfr moadibfr force-pushed the fea/glob_files_list branch from 7bd3591 to f25792e Compare March 29, 2021 16:29
@moadibfr
Copy link
Contributor Author
moadibfr commented Apr 6, 2021

user mwarkentin on discord had multiple files on the same s3 bucket were his statefile lays and listing take too much time. Progress bar is being stopped.
I have to send tics to progressbar so it dont stop when listing from s3...

@eliecharra eliecharra added this to the v0.8.0 milestone Apr 16, 2021
@eliecharra
Copy link
Contributor

To rebase on v0.8 and change target branch

@sundowndev sundowndev added kind/enhancement New feature or improvement priority/2 labels Apr 28, 2021
Support go glob syntax + ** syntaxe on file enumerator
Support go glob syntax on s3 enumerator
@sundowndev sundowndev changed the base branch from main to v0.8 April 28, 2021 12:28
@sundowndev sundowndev force-pushed the fea/glob_files_list branch from f25792e to 5141f1e Compare April 28, 2021 12:31
@sundowndev sundowndev marked this pull request as ready for review April 28, 2021 14:53
wbeuil
wbeuil previously requested changes Apr 30, 2021
Copy link
Contributor
@wbeuil wbeuil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too bad for the ** syntax on S3. Small typo in one sentence.

return nil, err
}

// filpath match does not compile so we try to match to be able to report the pattern error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filepath match .... so we use the match method to report the pattern error

@sundowndev sundowndev self-assigned this Apr 30, 2021
@sundowndev sundowndev force-pushed the fea/glob_files_list branch from 3e9e07e to 2136c8b Compare May 6, 2021 08:47
@sundowndev sundowndev force-pushed the fea/glob_files_list branch from 2136c8b to 7838728 Compare May 6, 2021 08:58
@sundowndev sundowndev requested a review from a team May 6, 2021 09:04
Copy link
Contributor Author
@moadibfr moadibfr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I wonder if you could have use just doublestar.glob instead of testing double start presence in the file glob but it's a details.

eliecharra
eliecharra previously approved these changes May 7, 2021
eliecharra
eliecharra previously approved these changes May 7, 2021
info, err := os.Lstat(path)
if err != nil {
if isGlob := strings.Contains(path, "*"); !isGlob && err != nil {
Copy link
Contributor Author
@moadibfr moadibfr May 7, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't you use HasMeta?

@sundowndev sundowndev force-pushed the fea/glob_files_list branch from 845ca11 to 144ccd4 Compare May 7, 2021 09:43
@eliecharra elie 6D40 charra dismissed wbeuil’s stale review May 7, 2021 09:49

martin reviewed it

@eliecharra eliecharra merged commit 717ffae into v0.8 May 7, 2021
@eliecharra eliecharra deleted the fea/glob_files_list branch May 7, 2021 09:51
@sundowndev sundowndev changed the title Limited glob support for enumerators Implement glob support for enumerators May 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement New feature or improvement priority/2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0