8000 feat: `testbed` by McPatate · Pull Request #39 · huggingface/llm-ls · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

feat: testbed #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 59 commits into from
Nov 6, 2023
Merged

feat: testbed #39

merged 59 commits into from
Nov 6, 2023

Conversation

McPatate
Copy link
Member

testbed will be evaluating completion quality

@McPatate McPatate force-pushed the feat/test_framework branch from b66825b to 54da8aa Compare October 24, 2023 16:17
@github-actions
Copy link
github-actions bot commented Oct 30, 2023 •
edited
Loading
Repository name Source type Average hole completion time (s) Pass percentage
simple local 3.11255 93%
gcanti/io-ts github 109.3835 0%
lancedb/lance github 96.7374 69.46386%
lancedb/lancedb github 78.153206 90%
mmaitre314/picklescan github 4.4968 100%
smol-rs/async-executor github 10.5044 50%
colinhacks/zod github 52.4536 80%
helix-editor/helix github 86.479706 70%
encode/starlette github 37.8284 89.73298%
tkaitchuck/constrandom github 2.444 10%
tiangolo/fastapi github 88.779205 80%
huggingface/huggingface_hub github 20.571901 100%
jaemk/cached github 34.3235 100%
Total -- 29.694595 80.41803%

Note: The "hole completion time" represents the full process of:

  • replacing the code from the file with a completion from the model
  • building the project
  • running the tests

@huggingface huggingface deleted a comment from github-actions bot Oct 30, 2023
@huggingface huggingface deleted a comment from github-actions bot Oct 30, 2023
@huggingface huggingface deleted a comment from github-actions bot Oct 30, 2023
@McPatate McPatate force-pushed the feat/test_framework branch from b675e52 to 1198992 Compare November 3, 2023 14:20
@McPatate McPatate force-pushed the feat/test_framework branch from 1644939 to 61ad1a4 Compare November 3, 2023 19:18
@McPatate McPatate merged commit c7affd0 into main Nov 6, 2023
@McPatate McPatate deleted the feat/test_framework branch November 6, 2023 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects 4227
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@McPatate @glegendre01
0