Benchmark BNF examples via Criterion #91
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CrockAgile
commented
May 4, 2022
|
|
||
| fn examples(c: &mut Criterion) { | ||
| c.bench_function("parse postal", |b| { | ||
| let input = std::include_str!("../tests/fixtures/postal_address.terminated.input.bnf"); |
There was a problem hiding this comment.
importing from the integration tests makes this a little brittle, but it seemed preferable to duplication
|
this is rad, really glad to see it incorporated, totally agree with 😍 |
1e24c4c to
3dd3171
Compare
Codecov Report
@@ Coverage Diff @@
## main #91 +/- ##
=======================================
Coverage 91.94% 91.94%
=======================================
Files 10 10
Lines 869 869
=======================================
Hits 799 799
Misses 70 70 Continue to review full report at Codecov.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
I hope the README explains this PR. But one additional nice thing to look at is a screenshot from the Criterion report 😍
Obviously big caveat to the numbers. Running on my personal laptop, with other process activity, etc. But nice to see what the reports can look like!
Issues
One potential issue with these benchmarks is that
Grammar::generaterelies on random seeds. Currently there is no public API for generating with a static, controlled seed for reliable benchmarking. This introduces some chance to that benchmark, but hopefully it is small noise in the big picture. On my machine, I observed ~2% of performance drift between randomGrammar::generatebenchmarks.