Re: performance of select(), poll and kqueue on large number of sockets / was: increase Number of filedescriptors

[Ian Lister <openbsd-lists@lister.dnsalias.net>]

On Wed, 18 Sep 2002, Torsten Valentin wrote:
>I am writing this, because I am not sure if my calculations are correct.
>Ian, is this right:

[snip]

Your reasoning and figures seem about right.

>(Btw: The number of active pipes doesn't make a difference in poll(),
>according to the table you referenced above.)

That's pretty much true - it's O(num_fds) rather than O(active_fds) but
obviously if you have more active sockets it will take a bit more time to
service each of them.

>This means I have to set up two larger (modern, say 2-2,5 GHz) machines
>with 5.000 Clients each, and will most probably have a load well below
>50 %, right?

Yes, assuming your traffic is smooth (not bursty). If you have control
over the client software you might want to consider making it schedule
its next `ping' based on the time at which it received the last `pong'
rather than the time it sent the last `ping' in order to naturally smooth
out any bursts from synchronised clients.

If you can split the clients across two machines (using round robin DNS or
whatever) then you might want to consider running multiple instances of
pen on the _same_ machine (e.g. with multiple addresses). If you work it
out you'll see that splitting the clients between two processes on the
same machine will halve your load (each process will be approximately
twice as fast to poll, and will only need to poll about half as often).
And if you can split them between two processes you can split them between
three. Or a hundred.

>In terms of redundancy and HA it would make sense to set up two machines
>that work together and share the load of all 10.000 clients anyway. In
>case of a failure of one machine, the remaining machine would take over
>the IP of the faulty machine. It would then suffer from heavy load, but
>it probably would work.

Yes. FWIW I still stand by my recommendation of writing or obtaining a
program (or version of that program) that uses kqueue rather than select
or poll - having done the figures you'll see you could probably handle
those 10 000 clients on a *much* lower specced machine.

Ian