Revision 788, 6.0 kB (checked in by raggi, 8 months ago)

Merge of branches/raggi
Most notable work and patches by Aman Gupta, Roger Pack, and James Tucker.
Patches / Tickets also submitted by: Jeremy Evans, aanand, darix, mmmurf,
danielaquino, macournoyer.

  • Moved docs into docs/ dir
  • Major refactor of rakefile, added generic rakefile helpers in tasks
  • Added example CPP build rakefile in tasks/cpp.rake
  • Moved rake tests out to tasks/tests.rake
  • Added svn ignores where appropriate
  • Fixed jruby build on older java platforms
  • Gem now builds from Rakefile rather than directly via extconf
  • Gem unified for jruby, C++ and pure ruby.
  • Correction for pure C++ build, removing ruby dependency
  • Fix for CYGWIN builds on ipv6
  • Major refactor for extconf.rb
  • Working mingw builds
  • extconf optionally uses pkg_config over manual configuration
  • extconf builds for 1.9 on any system that has 1.9
  • extconf no longer links pthread explicitly
  • looks for kqueue on all *nix systems
  • better error output on std::runtime_error, now says where it came from
  • Fixed some tests on jruby
  • Added test for general send_data flaw, required for a bugfix in jruby build
  • Added timeout to epoll tests
  • Added fixes for java reactor ruby api
  • Small addition of some docs in httpclient.rb and httpcli2.rb
  • Some refactor and fixes in smtpserver.rb
  • Added parenthesis where possible to avoid excess ruby warnings
  • Refactor of $eventmachine_library logic for accuracy and maintenance, jruby
  • EM::start_server now supports unix sockets
  • EM::connect now supports unix sockets
  • EM::defer @threadqueue now handled more gracefully
  • Added better messages on exceptions raised
  • Fix edge case in timer fires
  • Explicitly require buftok.rb
  • Add protocols to autoload, rather than require them all immediately
  • Fix a bug in pr_eventmachine for outbound_q
  • Refactors to take some of the use of defer out of tests.
  • Fixes in EM.defer under start/stop conditions. Reduced scope of threads.
  • Property svn:keywords set to Id
1 EventMachine now supports epoll, bringing large increases in performance and scalability to Ruby programs.
3 Epoll(7) is a alternative mechanism for multiplexed I/O that is available in Linux 2.6 kernels.
4 It features significantly greater performance than the standard select(2) mechanism, when used in
5 applications that require very large numbers of open I/O descriptors.
7 EventMachine has always used select(2) because its behavior is well standardized and broadly supported.
8 But select becomes unreasonably slow when a program has a
9 very large number of file descriptors or sockets. Ruby's version of select hardcodes a limit
10 of 1024 descriptors per process, but heavily loaded processes will start to show performance
11 degradation even after only a few hundred descriptors are in use.
13 Epoll is an extended version of the poll(2) call, and it solves the problems with select. Programs
14 based on epoll can easily scale past Ruby's 1024-descriptor limit, potentially to tens of thousands
15 of connectors, with no significant impact on performance.
17 (Another alternative which is very similar to epoll in principle is kqueue, supplied on BSD and its
18 variants.)
22 This note shows you how to use epoll in your programs.
24 === Compiling EventMachine to use epoll.
26 You don't have to do anything to get epoll support in EventMachine.
27 When you compile EventMachine on a platform that supports epoll, EM will
28 automatically generate a Makefile that includes epoll. (At this writing, this will only work
29 on Linux 2.6 kernels.) If you compile EM on a platform without epoll, then epoll support will
30 be omitted from the Makefile, and EM will work just as it always has.
32 === Using epoll in your programs.
34 First, you need to tell EventMachine to use epoll instead of select (but see below, as this requirement
35 will be removed in a future EventMachine version). Second, you need to prepare your program to use
36 more than 1024 descriptors, an operation that generally requires superuser privileges. Third, you will probably
37 want your process to drop the superuser privileges after you increase your process's descriptor limit.
39 === Using EventMachine#epoll
41 Call the method EventMachine#epoll anytime before you call EventMachine#run, and your program will
42 automatically use epoll, if available. It's safe to call EventMachine#epoll on any platform because
43 it compiles to a no-op on platforms that don't support epoll.
45   require 'rubygems'
46   require 'eventmachine'
48   EM.epoll
49 {
50     ...
51   }
54 EventMachine#epoll was included in this initial release only to avoid changing the behavior of existing
55 programs. However, it's expected that a future release of EM will convert EventMachine#epoll to a no-op,
56 and run epoll by default on platforms that support it.
58 === Using EventMachine#set_descriptor_table_size
60 In Linux (as in every Unix-like platform), every process has a internal table that determines the maximum
61 number of file and socket descriptors you may have open at any given time. The size of this table is
62 generally fixed at 1024, although it may be increased within certain system-defined hard and soft limits.
64 If you want your EventMachine program to support more than 1024 total descriptors, you must use
65 EventMachine#set_descriptor_table_size, as follows:
67   require 'rubygems'
68   require 'eventmachine'
70   new_size = EM.set_descriptor_table_size( 60000 )
71   $>.puts "New descriptor-table size is #{new_size}"
73 {
74     ...
75   }
77 If successful, this example will increase the maximum number of descriptors that epoll can use to 60,000.
78 Call EventMachine#set_descriptor_table_size without an argument at any time to find out the current
79 size of the descriptor table.
81 Using EventMachine#set_descriptor_table_size ONLY affects the number of descriptors that can be used
82 by epoll. It has no useful effect on platforms that don't support epoll, and it does NOT increase the
83 number of descriptors that Ruby's own I/O functions can use.
85 #set_descriptor_table_size can fail if your process is not running as superuser, or if you try to set a
86 table size that exceeds the hard limits imposed by your system. In the latter case, try a smaller number.
89 === Using EventMachine#set_effective_user
91 In general, you must run your program with elevated or superuser privileges if you want to increase
92 your descriptor-table size beyond 1024 descriptors. This is easy enough to verify. Try running the
93 sample program given above, that increases the descriptor limit to 60,000. You will probably find that
94 the table size will not be increased if you don't run your program as root or with elevated privileges.
96 But of course network servers, especially long-running ones, should not run with elevated privileges.
97 You will want to drop superuser privileges as soon as possible after initialization. To do this,
98 use EventMachine#set_effective_user:
100   require 'rubygems'
101   require 'eventmachine'
103   # (Here, program is running as superuser)
105   EM.set_descriptor_table_size( 60000 )
106   EM.set_effective_user( "nobody" )
107   # (Here, program is running as nobody)
109 {
110     ...
111   }
113 Of course, you will need to replace "nobody" in the example with the name of an unprivileged user
114 that is valid on your system. What if you want to drop privileges after opening a server socket
115 on a privileged (low-numbered) port? Easy, just call #set_effective_user after opening your sockets:
117   require 'rubygems'
118   require 'eventmachine'
120   # (Here, program is running as superuser)
122   EM.set_descriptor_table_size( 60000 )
124 {
125     EM.start_server( "", 80, MyHttpServer )
126     EM.start_server( "", 443, MyEncryptedHttpServer )
128     EM.set_effective_user( "nobody" )
129     # (Here, program is running as nobody)
131     ...
132   }
135 Because EventMachine#set_effective_user is used to enforce security
136 requirements, it has no nonfatal errors. If you try to set a nonexistent or invalid effective user,
137 #set_effective_user will abort your program, rather than continue to run with elevated privileges.
139 EventMachine#set_effective_user is a silent no-op on platforms that don't support it, such as Windows.
Note: See TracBrowser for help on using the browser.