Tags: QuantamHD/brod
Tags
Support extra socket options (kafka4beam#273) 3.5.1. Add extra_sock_opts client config. This should allow users tune socket performance depending on their specific use cases.
Introduce roundrobin_v2 group protocol (brod version 3.3.3)
Prior to this change, brod's 'roundrobin' group protocol commits
acknowledged offsets to kafka, then +1 as begin-offset when assign
to group members to continue after restart/rebalance.
Starting from this version, brod will commit 'next offset to fetch',
to be compliant with the protocol spec and other kafka client
implementations, as well as monitoring tools.
For offsets committed to kafka:
'roundrobin_v2' will upgrade old commits to new version by +1.
This is done by making use of the metadata committed together
with the offsets:
'roundrobin' metadata should be a string containing the Erlang node
name and the group coordinator pid. e.g. "myconsumer@myhost/<0,6,0>"
'roundrogin_v2' uses the same but with a "+1/" prefix.
NOTE: Since it's a new protocol, it does not support rolling upgrade
from a running 'roundrobin' group. i.e. all 'roundrobin' members
will have to be stopped before starting 'roundrobin_v2' members.
Nonetheless, there is no risk even if 'roudrobin' and
'roundrobin_v2' members are started at the same time,
because kafka would not allow a member supporting different
protocol to join an active group.
It is however NOT OK for 'roundrobin' members to pick up
'roundrobin_v2' commits. i.e. downgrade is not supported.
For consumer managed offsets (offset_commit_policy = consumer_managed):
There is no easy way to smooth upgrade consumer managed offsets,
therefor we will keep commiting 'acked offsets' instead of
'begin offset' until next major release.
PreviousNext