8000 Add continuous performance testing · Issue #194 · s-mandra/qflex · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Add continuous performance testing #194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

8000 Already on GitHub? Sign in to your account

Jump to bottom
Open
95-martin-orion opened this issue Oct 28, 2019 · 0 comments
Open

Add continuous performance testing #194

95-martin-orion opened this issue Oct 28, 2019 · 0 comments
Labels
enhancement New feature or request testing Improve test coverage

Comments

@95-martin-orion
Copy link
Contributor

We should periodically (e.g. once a day?) run a set of sample circuits to track performance of qFlex, both in terms of memory cost and simulation runtime. Some options for this:

  1. Run each circuit that has a hand-crafted ordering, at a reasonable depth (e.g. runtime < 30s)
    • Pros: using a fixed circuit set gives clearer comparisons between runs
    • Cons: might be biased towards these specific circuits
  2. Generate circuits and orderings for each grid (via generator.py and order_circuit_simulation.py) and run several for each (e.g. 30 each, runtime < 1s)
    • Pros: more varied coverage for different circuit layouts (but still restricted to QS circuits)
    • Cons: more difficult to compare between runs, still biased towards QS circuits
  3. Create a tool for generating truly random circuits (i.e. no fixed placement for 2-qubit gates) and follow the plan in (2), using the new tool to generate circuits
    • Pros: best option for covering various circuit layouts
    • Cons: time cost of writing tool, potential for untested layouts to misbehave (e.g. excessive runtime or memory usage), very difficult to compare between runs
@95-martin-orion 95-martin-orion added enhancement New feature or request testing Improve test coverage labels Oct 28, 2019
@95-martin-orion 95-martin-orion mentioned this issue Oct 28, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request testing Improve test coverage
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@95-martin-orion
0