better way using IO::Select and pool of non-blockig socks

[Uri Guttman]

use Event.pm instead. it is much better than IO::Select when you have
many read/write sockets. if you use the OO callbacks, then you can
associate an object with each socket and that object can tell the
callback what to do.

uri

 

[Andrew Tkachenko]

Hello.

Lets suppose I have a pool of about 300-400 non-blocking sockets and I want to
monitor their state and do some actions depending on some events.
So this is how I do it now:

my ($rd_set, $wr_set) = (new IO::Select, new IO::Select);

while(1)
{
my ($can_read, $can_write) = IO::Select->select($rd_set, $wr_set, undef, $TIMEOUT);
my %readers = (map {$_, 1} @$can_read);
my %writers = (map {$_, 1} @$can_write);
for (my $i = 0; $i < $#socks; $i++) {
# do something
if($readers{$socks[$i]}) {
# read from socket
delete $readers{$sock[$i]}
}
if($writers{$socks[$i]}) {
# write to socket (if needed)
delete $writers{$sock[$i]}
}
}
}

now for some reasons this scheme is not quite suitable for me, so I wonder if I could do it
in another way:

while(1)
{
for (my $i = 0; $i < $#socks; $i++) {
# do something
my $ios =IO::Select->new($socks[$i]);
if ($ios->can_read(0)) {
# read from socket
}
if ($ios->can_write(0)) {
# write to socket (if needed)
}
}
}

So assuming that I have quite big set of sockets (@socks >= 300 ) I wonder if
second example would be inefficient one ?
Thanks for advance,
Andrew.

P.S. Sorry for poor english

 

[Stuart Moore]

Hello.

Lets suppose I have a pool of about 300-400 non-blocking sockets and I want to
monitor their state and do some actions depending on some events.

From what I can see, your aim is to find out which element of @socks
each socket you can read/write from is in. Am I right?

If so, I'd use Tie::RefHash (included in my distrib of 5.8, possibly not
in older versions), which allows you to have hash references as keys.
Then somewhere (probably where you set up @socks) create a RefHash with
the sockets as keys and their array index as values. I think the
following will work:

my %sock_index;
@sock_index{@socks}=(0..@socks);

but you'd better check that as I'm not certain I remember the syntax.
Basically what I'm trying to do is something along the lines of

for(my $i=0;$i<@socks;$i++){
$sock_index{$socks[$i]}=$i;
}

Then a modified version of your code would be:

while(1)
{
my ($can_read, $can_write) = IO::Select->select($rd_set,
$wr_set, undef, $TIMEOUT);

foreach my $reader(@$can_read){
my $i = $sock_index{$reader};
#read from socket
}
foreach my $writer(@$can_write){
my $i = $sock_index{$writer};
#write to socket
}

}

which avoids the massive loop through all of @socks. I'm assuming
there's no reason to write/read in the order they are in @socks. If
there is, then you can hopefully tweak mine to do what you want.

 

[Stuart Moore]

Stuart Moore wrote:

Oops, missed a bit...

my %sock_index;

tie %sock_index, 'Tie::RefHash';

 

[Andrew Tkachenko]

thanks a lot for hint - unfortunately I've missed this module before.
Very good and handy job. I'm sure I'll use it in future, but
unfortunately, for my current task this module is not quite suitable
SO, the question is still remains actual.

Thanks, Andrew

 

[Uri Guttman]

AT> thanks a lot for hint - unfortunately I've missed this module before.
AT> Very good and handy job. I'm sure I'll use it in future, but
AT> unfortunately, for my current task this module is not quite suitable
AT> SO, the question is still remains actual.

why don't you state your actual reason for not using it? IO::Select has
a very poor api for doing read and write events. if your event code is
all isolated to one area, it should be easy to convert it to event.pm.

uri

 

[Uri Guttman]

AT> I don't need much from IO::Select - just to know if socket if
AT> ready for write or has something to read. Nothing more. I already

but that is what event.pm will tell you. and you don't have the clunky
api of io::select when it needs to do read AND write events. it is fine
for simple event loops with read or write but the need to create these
objects for each r/w instance is nuts. and the proper way to handle them
is to loop over your sockets and build up the args to the event loop.

AT> have engine generating appropriate events, and after some
AT> expirements and reading of Event.pm pod and tutorial I've found
AT> that this would be quite expensive for me to embed Event.pm into
AT> existing project. As I said, I've missed this quite useful module
AT> while I planned this task. Otherwise probably I'd use it, and I'm
AT> sure I'll use it in future.

expensive makes little sense here. your engine doesn't generate events,
io::select or event.pm does. what your engine does is handle the
events. now if you wrote this system in a procedural style and only call
io::select when you feel like it, then i can understand why you are
reticent to redesign it. that is a poor event loop design since it can
cause timeouts and possible lost data when you are doing other computing
and don't check back often enough. it is akin to a classic polling
design where you actually ask each socket if it has data. you waste so
much cpu in the polling loop.

but i can't convince anyone about anything today.

uri

 

[Andrew Tkachenko]

I don't need much from IO::Select - just to know if socket if ready for write or has something to read.
Nothing more. I already have engine generating appropriate events, and after some expirements and
reading of Event.pm pod and tutorial I've found that this would be quite expensive for me to embed Event.pm
into existing project. As I said, I've missed this quite useful module while I planned this task. Otherwise
probably I'd use it, and I'm sure I'll use it in future.

Thanks,
Andrew

 

[Andrew Tkachenko]

Thanks for you response, Stuart.

I'm sorry, probably I've missed something, but I've thought that there is no
need to use additional tools/modules/classes to create hash of sockets?

%socks = ();
for (0 .. 10) {
$socks{$sock} = new IO::Socket::INET(<skipped>);
}

<..skipped..>

@readers = $ios->can_read(0);
for my $r(@readers) {
if ($socks{$r}) {
# then do something
}
}

I've used it lots of time and never thought that it is impossible or erroneus
Please correct me if I've missed something or did it wrong ?

Thanks for advance,
Andrew

Stuart Moore wrote:

 

[Andrew Tkachenko]

Sorry, mistaken a bit.
I've meant:

%socks = ();
for (0 .. 10) {
my $sock = new IO::Socket::INET(..);
$socks{$sock} = $_;
}

 

[Ala Qumsieh]

%socks = ();
for (0 .. 10) {
my $sock = new IO::Socket::INET(..);
$socks{$sock} = $_;

That won't do what you think it will. You can NOT access the IO::Socket
objects using 'keys %socks' since the keys of the hash are stringified.
I would go with:

$socks[$_] = $sock;

--Ala

 

[Stuart Moore]

Sorry, mistaken a bit.
I've meant:

%socks = ();
for (0 .. 10) {
my $sock = new IO::Socket::INET(..);
$socks{$sock} = $_;
}

Without Tie::RefHash, any hash keys are strings, so the key would be a
stringified version of the reference to the socket. This might well seem
to work for most stuff, if something else (IO::Select for example) is
keeping a copy of the socket, but e.g. you won't be able to use the
output of 'keys' as a reference to the socket.

Stuart

 

[Stuart Moore]

Without Tie::RefHash, any hash keys are strings, so the key would be a
stringified version of the reference to the socket. This might well seem
to work for most stuff, if something else (IO::Select for example) is
keeping a copy of the socket, but e.g. you won't be able to use the
output of 'keys' as a reference to the socket.

Just a thought, would attempting to use a reference as a hash key be a
suitable thing to mention when "use warnings;" is on? Neither warnings
nor diagnostics seems to do anything

 

[Andrew Tkachenko]

event.pm is a really good thing, but unfortunately not in my case.
I'm still guessing how to embed it into existing project and still didn't find any not 'expensive' solution, because:

1. event.pm forces me to switch to async. way of event processing.
2. not all of events generated by current code may be represented as io,var,timer and idle events.
3. my 'engine' is inherited by wide range of classes and I'd like to have event processing to be transparent for
all of them.

So when I say 'expensive' I mean just amount of time needed to rewrite existing code to match event.pm demands.
Of course I'd be happy to upload base concepts of my proj. and ask for help to improve it, but I'm not sure
this is a right place for such a things, besides, I'm not sure that this would be a good idea to worry local community with my
terrible english

Thanks for your attention,
Andrew

 

[Tassilo v. Parseval]

Also sprach Stuart Moore:

Just a thought, would attempting to use a reference as a hash key be a
suitable thing to mention when "use warnings;" is on? Neither warnings
nor diagnostics seems to do anything

And they better don't. It's perfectly ok to use a reference as a
hash-key. Andrew's suggestion does make quite a bit of sense and I used
a similar approach in a select-loop in one of my programs. I had the
reference both as key and value. When a filehandle became readable and
the program read a sign-off string from it, all I had to do was

while (<$fh>) {
...
if (/^q$/) {
$select->remove($sockets{ $fh });
...
}
}

Most of the time you need the key as a lookup-means. For that it doesn't
matter that you cannot really do anyting useful with the key itself. For
that you have the value. And your complain that you cannot use 'keys' to
get references to the handle is moot in this setup, too: You just use
'values' to achieve the same.

Tassilo