Description
User story
As a user, I want to get feedback on my pull requests, which tells me how the proposed changes will impact my Code PushUp reports. Specifically, I want a comment to be posts a report diff summary in Markdown. The diff should list which categories and audits have improved or regressed, and by how much.
As a paying customer, I also want to view the full diff in the portal UI. The PR comment should include a link to the portal diff page. I also want to be able to access this diff from the portal directly, by selecting source and target Git refs (branches/commits/tags).
Architecture
Free version
graph TD;
user(["🤓 User"])
pr(["📑 Pull Request"])
comment(["🗨️ PR comment"])
user -- creates --> pr
user -- "👁️🗨️ views" --> comment
pr -- triggers --> ghAction
postComment -- creates --> comment
subgraph ghAction ["🏗️ code-pushup/github-action"]
collectSrc[["git checkout $GITHUB_HEAD_REF\ncode-pushup collect"]]
collectTgt[["git checkout $GITHUB_BASE_REF\ncode-pushup collect"]]
compare[["code-pushup compare source-report.json target-report.json"]]
postComment[["github.issues.createComment({ ... })"]]
srcReport("source-report.json")
tgtReport("target-report.json")
diffJson("reports-diff.json")
diffMd("reports-diff.md")
collectSrc --> srcReport
collectTgt --> tgtReport
srcReport & tgtReport --> compare
compare --> diffJson & diffMd
diffMd --> postComment
end
Paid version
graph TD;
user(["🤓 User"])
pr(["📑 Pull Request"])
comment(["🗨️ PR comment"])
ghAction{{"🏗️ code-pushup/github-action"}}
portal[("🌈 Portal")]
user -. "💰 pays for" .-> portal
user -- creates --> pr
user -- "👁️🗨️ views" --> comment
pr -- triggers --> ghAction
ghAction -- "🗘 uploads to\n& downloads from" --> portal
ghAction -- creates --> comment
comment -- "🔗 links to" --> portal
The GitHub Action will accept optional parameters for interacting with portal. The flow will then differ from the free version in a few ways:
- Since it's likely that the target branch (e.g.
main) already has an uploaded report, we can download it from the portal API instead of runningcollectagain. This report caching should speed up CI runs. - Because we want the reports to be accessible in the portal as well (primarily via link in PR comment), all reports collected by the GitHub Action are also uploaded to the portal (
autoruninstead ofcollect). - The Markdown comment will have an additional link added, for accessing the full diff in the portal.
In order to compare any two reports by branch/commit/etc., the portal API will implement the diffing logic as well. In this case, the diff could be more complex, e.g. to compare which issues were introduced or fixed (would be too much detail to include in MD comment).
Acceptance criteria
Free features:
- CLI compare command (2 report.json files => reports-diff.json) #482
- CLI report diff MD formatting (reports-diff.json => reports-diff.md) #559
- GitHub Action for posting PR comments (collect source, collect target, compare) #575
- use GitHub Annotations API to link source file issues in PR #605
Paid features:
- portal
- code-pushup/portal#268
- code-pushup/portal#332
- Report download in portal-client
- code-pushup/portal#376
- CLI and other
- CLI option to download cached report instead of collect
- Link to portal comparison page in markdown comment #681
- GitHub Action portal connection (download and upload reports)
Follow-up improvements
- use report artifact as cache