8000 perf(coord): Timeout query immediately if it stayed in mailbox for too long by vishramachandran · Pull Request #761 · filodb/FiloDB · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

perf(coord): Timeout query immediately if it stayed in mailbox for too long #761

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
May 29, 2020
Merged

perf(coord): Timeout query immediately if it stayed in mailbox for too long #761

merged 7 commits into from
May 29, 2020

Conversation

vishramachandran
Copy link
Member
@vishramachandran vishramachandran commented May 27, 2020 •
edited
Loading

Pull Request checklist

  • The commit(s) message(s) follows the contribution guidelines ?
  • Tests for the changes have been added (for bug fixes / features) ?
  • Docs have been added / updated (for bug fixes / features) ?

Using certain Occurs clauses enables scoring/ranking of search results which appears to be CPU intensive in profiler results. This PR uses Occurs.FILTER that explicitly disables scoring and weighting of results.

Also do timeout check before doing anything for query execution. This accounts for delays in delivery from actor mailbox.

TODO:

  • JMH benchmark to measure effect. Right now it seems to be broken. Needs to be revived.
  • More changes to possibly optimize other Lucene calls.

@vishramachandran vishramachandran force-pushed the remove-lucene-scoring branch from d51d311 to 1db479e Compare May 28, 2020 18:53
velvia
velvia approved these changes May 28, 2020
View reviewed changes
Copy link
Member
@velvia velvia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking forward to this change. Would you mind adding comments about where the scoring is disabled? Not obvious in the code.... just comment so we are aware why the code is the way it is and the impact scoring has.

@vishramachandran
Copy link
Member Author

I am undoing most of this PR, retaining just the timeout checks. The JMH is not showing any improvement. It appears that Lucene's "score" methods actually do the job of search instead of scoring. Scoring is disabled already. I was able to debug through the steps and confirm it.

@vishramachandran vishramachandran force-pushed the remove-lucene-scoring branch from 2b813e5 to 4908e00 Compare May 29, 2020 14:41
@vishramachandran vishramachandran changed the title perf(core): Remove options that enable scoring of documents during queries perf(coord): Timeout query immediately if it stayed in mailbox for too long May 29, 2020
@vishramachandran vishramachandran merged commit 1142c72 into filodb:develop May 29, 2020
@vishramachandran vishramachandran deleted the remove-lucene-scoring branch May 29, 2020 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@velvia velvia velvia approved these changes

@sherali42 sherali42 sherali42 approved these changes

@whizkido whizkido Awaiting requested review from whizkido

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@vishramachandran @velvia @sherali42
0