This repository is an integration testing environment for the auto-comments reusable GitHub Actions workflow.
The auto-comments repository contains a reusable workflow that automatically posts comments on Pull Requests from external contributors. This test repository helps verify that the workflow functions correctly in real-world scenarios before changes are merged into the main workflow.
The core functionality of the auto-comments workflow can only be properly tested with real user accounts, as it relies on checking organization membership status. For comprehensive testing, you should:
| Test Case | User Type | PR Action | Expected Result |
|---|---|---|---|
| 1 | Internal (org member) | Open PR | No comments |
| 2 | External (non-member) | Open first PR | First PR comment appears |
| 3 | External (non-member) | Open second PR | No first PR comment |
| 4 | External (non-member) | Mark PR ready for review | Ready for review comment appears |
| 5 | External (non-member) | Merge PR | Merged PR comment appears |
- Create or use a GitHub account that is not a member of the RequestNetwork organization
- Have this account create a PR to the auto-comments-test repository
- Check that the first PR comment appears
- Have the same account create a second PR
- Verify that the first PR comment doesn't appear this time
- Mark the PR as ready for review (after creating it as a draft)
- Verify the ready for review comment appears
- Merge the PR
- Verify the merged PR comment appears
In all comments, verify that:
{{username}}correctly shows the PR author's username{{repository}}correctly shows "auto-comments-test"{{org}}correctly shows "RequestNetwork"
This repository includes a simplified integration-test.yml workflow that:
- Is triggered by actual pull request events (opened, ready for review, closed)
- Calls the reusable workflow from the auto-comments repository
- Provides test-specific comment messages clearly marked with "[TEST]"
By using real pull requests, we can accurately test the workflow's core functionality:
- Organization membership detection
- First PR detection
- Event-specific comments
- Variable substitution
- Local Validation: First validate workflow syntax locally
- Real User Testing: Have a non-org member create a PR to verify the full functionality
- Review Results: Check that comments appear correctly and variables are substituted properly
Remember: The most crucial aspects (first-PR detection, org membership detection) can only be properly verified with actual GitHub accounts that are not members of your organization.
For thorough testing, consider:
- Creating a dedicated testing GitHub account
- Keeping it outside your organization
- Using it exclusively for testing external contributor experiences
- Documenting test results with screenshots
MIT