8000 Added Run.SkipRemainingOnFailure option by ArmaanMcleod · Pull Request #2023 · pester/Pester · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Added Run.SkipRemainingOnFailure option #2023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

< 8000 /details>
Jump to bottom
Merged
merged 21 commits into from
Jul 22, 2021
Merged

Added Run.SkipRemainingOnFailure option #2023

merged 21 commits into from
Jul 22, 2021

Conversation

ArmaanMcleod
Copy link
Contributor
@ArmaanMcleod ArmaanMcleod commented Jul 3, 2021 •
edited
Loading

PR Summary

Fix #360

I've added a Run.SkipRemainingOnFailure option. The following scopes are supported:

Block -> Skip all remaining tests in current block(including child tests) after a failed test.
Container -> Skip all remainng tests in the container(file or scriptblock) after a failed test.
Run -> Skip all tests across all containers in a run after a failed test.
None -> default, keep original behaviour.

Sample Test 1

# C:\Users\Armaan\Documents\powershell-dev\sample.Tests.ps1
Describe "d1" { 
    Describe "d2" {
        It "i2" {
            $true | Should -BeTrue
        }
        Describe "d3" { 
            It "i3" {
                $false | Should -BeTrue # Fail
            }
            It "i4" {
                $true | Should -BeTrue
            }
        }
        It "i5" {
            $true | Should -BeTrue
        }
    }
    Describe "d4" {
        It "i6" {
            $true | Should -BeTrue
        }
    }
}

Sample Test 2

# C:\Users\Armaan\Documents\powershell-dev\sample2.Tests.ps1
Describe "d5" { 
    It "i7" {
        $false | Should -BeTrue # Fail
    }
    It "i8" {
        $false | Should -BeTrue # Fail
    }
}

Setup code

$pesterConfig = New-PesterConfiguration
$pesterConfig.Run.Container = @((New-PesterContainer -Path C:\Users\Armaan\Documents\powershell-dev\sample.Tests.ps1), (New-PesterContainer -Path C:\Users\Armaan\Documents\powershell-dev\sample2.Tests.ps1))
$pesterConfig.Output.Verbosity = "Detailed"

Block

$pesterConfig.Run.SkipRemainingOnFailure = 'Block'
Invoke-Pester -Configuration $pesterConfig

Block

Container

$pesterConfig.Run.SkipRemainingOnFailure = 'Container'
Invoke-Pester -Configuration $pesterConfig

Container

Run

$pesterConfig.Run.SkipRemainingOnFailure = 'Run'
Invoke-Pester -Configuration $pesterConfig

Run

PR Checklist

  • PR has meaningful title
  • Summary describes changes
  • PR is ready to be merged
    • If not, use the arrow next to Create Pull Request to mark it as a draft. PR can be marked Ready for review when it's ready.
  • Tests are added/update (if required)
  • Documentation is updated/added (if required)

@ArmaanMcleod ArmaanMcleod changed the title Fail test run option Added Run.SkipRemainingOnFailure option Jul 3, 2021
@ArmaanMcleod ArmaanMcleod force-pushed the fail-test-run-option branch from 6495dbd to 034dbd7 Compare July 3, 2021 17:04
@ArmaanMcleod ArmaanMcleod force-pushed the fail-test-run-option branch from 034dbd7 to 6a650d4 Compare July 3, 2021 17:23
@ArmaanMcleod ArmaanMcleod marked this pull request as ready for review July 3, 2021 17:38
@ArmaanMcleod
Copy link
Contributor Author
ArmaanMcleod commented Jul 3, 2021 •
edited
Loading

This should be ready for review.

I've attached screenshots in the PR description to show behaviour of the different options. Let me know if I've interpreted something incorrectly 👍 .

Also added a message to the skipped tests. Not sure if it adds too much noise, but seems useful to see why the tests were skipped. Perhaps It would be better to limit this to just one message at the bottom, instead of under every skipped test? It might also be too generic, and could probably also say which test failed instead of just Skipped due to previous failure and Run.SkipRemainingOnFailure set to <option>. Didn't want to overcomplicate this so just left it as is for now. Happy to change this if needed 🙂 .

fflaten
fflaten reviewed Jul 3, 2021
View reviewed changes
Copy link
Collaborator
@fflaten fflaten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good start! Added some thoughts

@ArmaanMcleod ArmaanMcleod marked this pull request as draft July 4, 2021 03:06
@ArmaanMcleod ArmaanMcleod force-pushed the fail-test-run-option branch from 4838243 to 25ac249 Compare July 4, 2021 06:45
@ArmaanMcleod ArmaanMcleod force-pushed the fail-test-run-option branch from 25ac249 to 598465f Compare July 6, 2021 12:13
@ArmaanMcleod ArmaanMcleod marked this pull request as ready for review July 7, 2021 15:15
@ArmaanMcleod
Copy link
Contributor Author

I think this is ready for another review. Thanks @fflaten for reviewing this change up till now, really helpful feedback 🙂 .

One of the conversations above regarding how to handle skipped messages has not been resolved, leaving this for the maintainer @nohwnd to comment on since we are not sure if a message should be included at the bottom for Detailed/Diagnostic view.

Other than this, happy to hear more feedback for anything else that needs to be changed.

@nohwnd
Copy link
Member
nohwnd commented Jul 13, 2021

@ArmaanMcleod sorry I am slammed with work, and don't have any time for Pester nowadays. I will review as soon as I can.

@ArmaanMcleod
Copy link
Contributor Author

@nohwnd No worries 🙂. I am also getting slammed at work. Must be that busy time of year again 😄

@nohwnd nohwnd added this to the 5.3 milestone Jul 20, 2021
@nohwnd
Copy link
Member
nohwnd commented Jul 20, 2021

@ArmaanMcleod I think adding a message to the bottom of detailed + diag output would be nice. Much like when a block fails there is summary at the bottom. We can change the behavior later as we learn how useful or tragically oververbose (like my original filter output in diag) it is.

@nohwnd
Copy link
Member
nohwnd commented Jul 22, 2021

LGTM.

@nohwnd nohwnd merged commit 1515194 into pester:main Jul 22, 2021
@fflaten fflaten mentioned this pull request Jul 22, 2021
Closed
@ArmaanMcleod ArmaanMcleod deleted the fail-test-run-option branch July 22, 2021 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fflaten fflaten fflaten left review comments

@nohwnd nohwnd nohwnd left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
5.3
Development

Successfully merging this pull request may close these issues.

How to stop Pester at first failed test?
3 participants
@ArmaanMcleod @nohwnd @fflaten
0