Ticket #115 (new defect)

Opened 2 months ago

Last modified 2 months ago

Random failures when sending out lots of requests (using em-http)

Reported by: nerochiaro Assigned to: tmm1
Priority: major Milestone:
Keywords: Cc:

Description

When sending out many em-http requests at the same time (i.e. in the same EM tick) some of those requests return an errback without any apparent cause.

Tested this on Linux (Ubuntu 8.10 and Debian unstable)

The number of requests is less than the per-user limit (1024 on my machine), yet they still fail. The failure happens both on localhost, 127.0.0.1 and any other site out there (e.g. google.com).

Throttling the number of requests by sending out one per EM tick is fixing the problem entirely when i try localhost, but when trying other sites there are still failures (less, but still some).

A script showing the unthrottled behaviour is at http://gist.github.com/82287 I always have failures running it with --url=http://www.google.com/ --reqs=1000

Change History

03/20/09 05:03:38 changed by tmm1

Happens with and without epoll. Using strace makes it both faster and the errors go away:

aman@pluto:~$ ruby testhttp.rb --url http://amazon.com/ --reqs 1000
I, [2009-03-20T01:51:59.167738 #15821]  INFO -- : All done: 997 ok, 3 failed, 0 conn-err. Elapsed: 8.154226

aman@pluto:~$ ruby testhttp.rb --url http://amazon.com/ --reqs 1000
I, [2009-03-20T01:52:23.156699 #15843]  INFO -- : All done: 861 ok, 139 failed, 0 conn-err. Elapsed: 7.484148

aman@pluto:~$ strace -ttTo /tmp/http.strace ruby testhttp.rb --url http://amazon.com/ --reqs 1000
I, [2009-03-20T01:52:51.166085 #15852]  INFO -- : All done: 1000 ok, 0 failed, 0 conn-err. Elapsed: 3.403963