8000 Question About BLIVA-Vicuna Performance on VSR-Zeroshot-Test · Issue #31 · mlpc-ucsd/BLIVA · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Question About BLIVA-Vicuna Performance on VSR-Zeroshot-Test #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up 9992 for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Yinhance opened this issue Feb 22, 2025 · 1 comment
Open

Question About BLIVA-Vicuna Performance on VSR-Zeroshot-Test #31

Yinhance opened this issue Feb 22, 2025 · 1 comment

Comments

@Yinhance
Copy link

Thank you for sharing your work on BLIVA! I have been running the provided code locally and evaluating bliva-vicuna on the VSR-Zeroshot-Test dataset. However, I obtained an accuracy of 0.5158 (539/1045) using a strict hard-matching strategy for answers.

To ensure my setup aligns with yours, could you clarify what prompt was used when evaluating accuracy on this dataset?
@weiyueli7
Copy link

We used the same prompt as InstructBLIP

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Yinhance @weiyueli7
0