amqp: allow multiple hosts for failover #3410
Conversation
There was a problem hiding this comment.
Hi Märt,
thank you very much for your PR!
I'm wondering whether round-robin, i.e. trying the hosts in order, would be a better strategy. If the number of hosts is small (two or three), the chances of trying to reconnect to the failing host are high. If we moved on to the next host, we'd avoid that.
This would also allow the user to specify a preference, which would make sense for a broker running locally. For a set of external brokers, this might create an imbalance since all clients (with the same config) are going to use the same broker …
What do you think?
Best regards,
—octo
src/collectd.conf.pod
Outdated
| @@ -540,6 +540,7 @@ B<Synopsis:> | |||
| # Send values to an AMQP broker | |||
| <Publish "some_name"> | |||
| Host "localhost" | |||
| # Host "amqp1.example.com" "amqp2.example.com" "amqp3.example.com" | |||
There was a problem hiding this comment.
I find this comment a bit distracting. How about amending the existing line, e.g. to:
Host "localhost"
Host "fallback-amqp.example.com"
|
Thank you @octo for the review! I considered round robin and thought that using random might balance the load better. It was also a bit easier to implement. I'm not aware of any use cases where setting a preference for a clustered broker node is useful. However if you think that round robin would be better, then I can make the change. Should be quite simple and I don't have a strong preference. |
There was a problem hiding this comment.
Thanks for the feedback @mbakhoff!
Preferring one instance over others is typically done when one instance is running in the same data center or metro, while others are "far away" (from the network's POV). That said, we'd only have this benefit until the connection fails for the first time, i.e. you can't really rely on it.
FTR, we could get the best of both worlds by randomizing the order after reading the config, then doing round-robin on the randomized list. If anybody thinks this is a good idea, send a PR :)
Thanks for your work Märt!
—octo
|
Thank you! |
ChangeLog: amqp plugin: allow multiple hosts to support failover
With this implementation all the hosts must use the same port number. Specifying a different port for each host would be tricky wrt config backwards compatibility and it's not clear how important it is.