8000
HTTP connection pooling (using HTTP connection keep-alives) · buger goreplay · Discussion #1258 · GitHub
More Web Proxy on the site http://driver.im/
You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Here at my $dayjob, it appears that we've hit what appears to be a limitation of goreplay. I'm not sure it's really a limitation and not some misunderstanding on our part, so I'm reaching out for help.
The setup is as follows: we've recorded a dump of some 15 minutes of HTTP traffic using goreplay, which is saved in its native format. The traffic was created by multiple clients making requests against a single service. Hence, the dump was intended to be replayed against a single service as well.
Dry-running the dump using gor -input-file $filename -input-file-dry-run -output-null -output-http-stats shows sensbile number of requests and estimates the play time to be ca. 12m30s.
Dividing the number of requests by the estimated time to replay them shows that gor should perform the requests at roughly 2500 RPS.
Now we actually play the dump using gor -input-file $filename -output-http=$service_url -stats -output-http-stats and see that the actual RPS fluctuates between 650 and 950 — far below the estimated 2500; this can also be easily seen by observing that the replaying does not complete within the estimated time (and twice that time etc).
We've tried to debug the problem by replacing the service by a no-nonsense configured nginx instance told to return 200 OK with a small precanned response to any request, made sure it had sheer limits in terms of nofile rlimit and so on, but gor was still unable to go above 950 RPS or so.
Attempting to force it go faster by appending |1000% to the input file name made it reach ca. 1k RPS but not above.
There are no EMFILE errors (all processes have sheer limits set), and CPU/memory usage is low.
Using a build of the master's tip (to have this problem solved) and multiple -http-output= command-line parameters (targeting different IP addresses all listened on by the receiving nginx instance) does not change anything w.r.t. the sustained RPS.
So, we've recorded a TCP dump of the exchanges between gor and nginx and saw that goreplay handles every single request using the "connect(), send request, read response, close()" sequence, does not place Connection: keep-alive in its requests and ignores the same field sent by nginx in its responses.
In other words, it does not reuse TCP sessions it creates, and it looks like it's exactly the limiting factor.
I have read about the PRO version's capability of exactly replaying TCP sessions as they were captured but in our case, it's too advanced a feature to have; what we need is just the ability to use HTTP keep-alives to be able to maintain the RPS as recorded in the dump.
I fail to find any command-line option controlling this aspect of the goreplay behavior.
So, the question: does gorplay support connection pooling?
Or maybe we cannot see anything obvious to make it what we want some using some other approach?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi!
Here at my $dayjob, it appears that we've hit what appears to be a limitation of
goreplay. I'm not sure it's really a limitation and not some misunderstanding on our part, so I'm reaching out for help.The setup is as follows: we've recorded a dump of some 15 minutes of HTTP traffic using
goreplay, which is saved in its native format. The traffic was created by multiple clients making requests against a single service. Hence, the dump was intended to be replayed against a single service as well.Dry-running the dump using
gor -input-file $filename -input-file-dry-run -output-null -output-http-statsshows sensbile number of requests and estimates the play time to be ca. 12m30s.Dividing the number of requests by the estimated time to replay them shows that
gorshould perform the requests at roughly 2500 RPS.Now we actually play the dump using
gor -input-file $filename -output-http=$service_url -stats -output-http-statsand see that the actual RPS fluctuates between 650 and 950 — far below the estimated 2500; this can also be easily seen by observing that the replaying does not complete within the estimated time (and twice that time etc).We've tried to debug the problem by replacing the service by a no-nonsense configured
nginxinstance told to return 200 OK with a small precanned response to any request, made sure it had sheer limits in terms ofnofilerlimit and so on, butgorwas still unable to go above 950 RPS or so.Attempting to force it go faster by appending
|1000%to the input file name made it reach ca. 1k RPS but not above.There are no
EMFILEerrors (all processes have sheer limits set), and CPU/memory usage is low.Using a build of the
master's tip (to have this problem solved) and multiple-http-output=command-line parameters (targeting different IP addresses all listened on by the receivingnginxinstance) does not change anything w.r.t. the sustained RPS.So, we've recorded a TCP dump of the exchanges between
gorandnginxand saw that goreplay handles every single request using the "connect(), send request, read response,close()" sequence, does not placeConnection: keep-alivein its requests and ignores the same field sent bynginxin its responses.In other words, it does not reuse TCP sessions it creates, and it looks like it's exactly the limiting factor.
I have read about the PRO version's capability of exactly replaying TCP sessions as they were captured but in our case, it's too advanced a feature to have; what we need is just the ability to use HTTP keep-alives to be able to maintain the RPS as recorded in the dump.
I fail to find any command-line option controlling this aspect of the goreplay behavior.
So, the question: does
gorplaysupport connection pooling?Or maybe we cannot see anything obvious to make it what we want some using some other approach?
Beta Was this translation helpful? Give feedback.
All reactions