Description
As for #3864 we realized that send and receive rate limiters have impact on the overall performance of connections between nodes, and therefore of the nodes themselves (consensus duration, in particular).
We need some more data to understand the behavior of the flow rate limiter implementation. It is actually based on this package https://pkg.go.dev/github.com/mxk/go-flowrate/flowrate?utm_source=godoc that has not been recently updated, the last release is from 2014.
The package itself produce some data using the Monitor.Status() method. This method is invoked by the Multiples connection to provide its own Status(), which is consumed by p2p.Peer instances to produce some metrics. But at the moment, none of the data produced by the rate limiter is considered or reported as a metric.
In addition, the Status produced by the rate limiter does not include the data we find most relevant in our investigations, that is the time that each limiter spend on time.Sleep calls, as part of the waitNextSample method.
We should collect the total time spent on those time.Sleep calls and possibly the maximum amount of time spent on a request to the limiter during a given time interval. This information could be used to produce metrics in the p2p package and understand the impact of those idle periods/sleeps in the overall performance of the system.