8000 Refactor ES logger for bulk requests by arvchristos · Pull Request #382 · dmachard/DNS-collector · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Refactor ES logger for bulk requests #382

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 27, 2023

Conversation

arvchristos
Copy link
Contributor
@arvchristos arvchristos commented Aug 23, 2023

This PR is introducing bulk requests for the ES logger, introducing a new parameter called bulk-size and splits the previous url parameter in server and index.

Additionally, a new parameter (flush-interval) is introduced, to flush the ES buffer on a defined period.

@dmachard
Copy link
Owner

Many thanks for this enhancement :)
It will be nice to clear the buffer also after some time as below
https://github.com/dmachard/go-dnscollector/blob/d00bd08b8c401db84ecd4f2c140a1197ccb343b3/loggers/redispub.go#L294

@arvchristos arvchristos force-pushed the feat/elasticsearch_bulk branch from 065e77e to 0063438 Compare August 27, 2023 14:17
@arvchristos arvchristos force-pushed the feat/elasticsearch_bulk branch from 0063438 to c80ceb2 Compare August 27, 2023 14:22
@arvchristos
Copy link
Contributor Author

Many thanks for this enhancement :) It will be nice to clear the buffer also after some time as below

https://github.com/dmachard/go-dnscollector/blob/d00bd08b8c401db84ecd4f2c140a1197ccb343b3/loggers/redispub.go#L294

This is indeed a nice idea. Done :)

@dmachard
Copy link
Owner
dmachard commented Aug 27, 2023

lgtm, thanks!

@dmachard dmachard merged commit f885f04 into dmachard:main Aug 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0