Kqueue interface module implementation

[U. A. R. Ruirarchzatrea]

Hello,

I have been trying to get a Kqueue module written for Perl with XS
that would allow the Kqueue event notification facility to be accessed
from Perl. Kqueue is a scalable event notification facility often
found on BSD OSs that allows you to recieve events from sockets for
instance, and corrects the performance limitations of select, allowing
for multiplexed servers to be written which can scale to very large
numbers of connections. It does the same thing as epoll on Linux but
does have a broader range of applications and features than epoll.

Having a module avialable to access Kqueue would be of use to anyone
wanting to write scalable network code on BSD OSs.


Has anyone out there looked into implemented a Kqueue module?
I was looking into doing it myself, but got confused how to do it. The goal
is for a Kqueue module written which can be contributed to CPAN so
other people who need to use Kqueue do not have to reinvent the wheel.

The Kqueue module should i think basically a straightforward
mapping of the kqueue and kevent functions and the kevent structures
to Perl. Later I can writing a higher level OO IO::Kqueue interface
around the Kqueue module, which would be no problem for me, since that
wouldnt directly use XS.

Of course, the Kqueue module involves writing an XS wrapper and that
is where I am completely confused as to how to properly implement the
interface.

I am sure for a Perl XS guru this would probably take a short bit to
implement with a few lines of code, but despite my reading the xs
documentation it is still beyond my understanding how to to implement
the wrapper.

Documentation for kqueue can be found in the kqueue man page "man
kqueue", the man pages can also be read online if you search for
kqueue.

Is there anyone out there who might be interested and know how to
write the code to do this?

I do have some of the files from my attempt including an XS wrapper
and and a .pm file in the following archive:
http://www.geocities.com/millueradfa/Kqueue.tar.gz. The archive also
includes the kqueue manual page and a sample kqueue echo server I
found. The man pages explains the kqueue API. A pointer to an array of
kevent structures is passed to kevent().

Perhaps this would map to
perl as a reference to an array containing arrays whos elements would
contain the values in the kevent structure. I have written some
documentation of what I was thinking of for the API for the module at
the end of the .pm file. Here is a synopsis of the API i was
considering, this is the low level API and will be wrapper with a
higher level interface later in IO::Kqueue:


use Kqueue;

($kq, $errno)=kqueue();

$timeout=12;
$filter='EVFILT_READ';
$flags='EV_ADD';
$fflags='';
$data='';
$udata='';

@kevent=($sockid, $filter, $flags, $fflags, $data, $udata);
@changelist=(\@kevent);
$changelist=\@changelist;

@eventlist=();
$eventlist=\@eventlist;

($errorval, $errno)=kevent($kq, $changelist, $eventlist, $timeout);

I think Changelist contains any changes to the configuration, eventlist, is used
to return pending events.

I am posting some of the documentation for Kqueue here and the
incomplete XS file I need help with getting to work. For the full
files (including .pm) you can look at the archive download I
previously referenced.


part of Kqueue man page (for full manpage look at :
http://people.freebsd.org/~jmg/kqueue.man.html)

SYNOPSIS
#include <sys/types.h>
#include <sys/event.h>
#include <sys/time.h>

int
kqueue(void);

int
kevent(int kq, const struct kevent *changelist, int nchanges,
struct kevent *eventlist, int nevents,
const struct timespec *timeout);

EV_SET(&kev, ident, filter, flags, fflags, data, udata);

The kevent structure is defined as:

struct kevent {
uintptr_t ident; /* identifier for this event */
short filter; /* filter for event */
u_short flags; /* action flags for kqueue */
u_int fflags; /* filter flag value */
intptr_t data; /* filter data value */
void *udata; /* opaque user data identifier */
};

RETURN VALUES
kqueue() creates a new kernel event queue and returns a file
descriptor.
If there was an error creating the kernel event queue, a value of
-1 is
returned and errno set.

kevent() returns the number of events placed in the eventlist, up
to the
value given by nevents. If an error occurs while processing an
element
of the changelist and there is enough room in the eventlist, then
the
event will be placed in the eventlist with EV_ERROR set in flags
and the
system error in data. Otherwise, -1 will be returned, and errno
will be
set to indicate the error condition. If the time limit expires,
then
kevent() returns 0.


Kqueue.xs:

#include "ppport.h"

#include <sys/event.h>
#include <sys/types.h>
#include <sys/time.h>

#include "const-c.inc"

MODULE = Kqueue PACKAGE = Kqueue

INCLUDE: const-xs.inc

int
kqueue ()


int
kevent (kq, changelist, nchanges, eventlist, nevents, timeout)
int kq
const struct kevent *changelist
int nchanges
struct kevent *eventlist
int nevents
const struct timespec *timeout


Thank you for any assistance in this.

U. A. R. Ruirarchzatrea

 

[Uri Guttman]

UARR> I have been trying to get a Kqueue module written for Perl with
UARR> XS that would allow the Kqueue event notification facility to be
UARR> accessed from Perl. Kqueue is a scalable event notification
UARR> facility often found on BSD OSs that allows you to recieve
UARR> events from sockets for instance, and corrects the performance
UARR> limitations of select, allowing for multiplexed servers to be
UARR> written which can scale to very large numbers of connections. It
UARR> does the same thing as epoll on Linux but does have a broader
UARR> range of applications and features than epoll.

UARR> Having a module avialable to access Kqueue would be of use to
UARR> anyone wanting to write scalable network code on BSD OSs.


UARR> Has anyone out there looked into implemented a Kqueue module? I
UARR> was looking into doing it myself, but got confused how to do
UARR> it. The goal is for a Kqueue module written which can be
UARR> contributed to CPAN so other people who need to use Kqueue do
UARR> not have to reinvent the wheel.

check out using libevent (at http://monkey.org/~provos/libevent/)
instead and wrapping that in perl via xs/swig/inline. that will be more
effective in that it gives you /dev/poll, kqueue(2), select(2), poll(2)
and epoll(4) support already and with one api wrapper you get all in
perl. i would be willing to help out with this as i have written and
wrapped event loops and done some (small) xs work (actually in swig).

you shouldn't post the kqueue api here as it can be found on the
net. and proposing a perl api for it is too early. first check out
libevent and think about wrapping that. then get the extending and
embedding perl book and read that. then work on the api (i can help
there for sure). also look at event.pm (on cpan) and its api. there is
plenty to do before you create the api and the actual wrapper code.

uri

 

[U. A. R. Ruirarchzatrea]

check out using libevent (at http://monkey.org/~provos/libevent/)
instead and wrapping that in perl via xs/swig/inline. that will be more
effective in that it gives you /dev/poll, kqueue(2), select(2), poll(2)
and epoll(4) support already and with one api wrapper you get all in
perl. i would be willing to help out with this as i have written and
wrapped event loops and done some (small) xs work (actually in swig).

Thank you for the advice. I think doing a libevent interface i an
excellent idea. But I also think that there is still is a need for a
interface to kqueue, and that is on my list of things to study doing.
There are actually many event engines written in Perl that could
benefit from it, such as POE. Doing an engine in Perl has its
advantages, as it is eisier to understand the code and to improve upon
it, and there is less chance of memory buffer overflows.

 

[Uri Guttman]

UARR> Thank you for the advice. I think doing a libevent interface i an
UARR> excellent idea. But I also think that there is still is a need for a
UARR> interface to kqueue, and that is on my list of things to study doing.
UARR> There are actually many event engines written in Perl that could
UARR> benefit from it, such as POE. Doing an engine in Perl has its
UARR> advantages, as it is eisier to understand the code and to improve upon
UARR> it, and there is less chance of memory buffer overflows.

i don't think you got what i meant. wrap libevent in perl and then stem
and poe and others can use libevent which already wraps kqueue and
several other things. for the amount of work to just wrap kqueue in
perl, you get several event interfaces in perl by wrapping libevent.

focus your effort where it will do the most good and libevent is that
for you.

uri

 

[U. A. R. Ruirarchzatrea]

UARR> Uri Guttman <[email protected]> wrote in message news:<[email protected]>...

i don't think you got what i meant. wrap libevent in perl and then stem
and poe and others can use libevent which already wraps kqueue and
several other things. for the amount of work to just wrap kqueue in
perl, you get several event interfaces in perl by wrapping libevent.

focus your effort where it will do the most good and libevent is that
for you.

uri

I think that doing a libevent wrapper is a good idea, however, there
still needs to be a direct interface to Kqueue for various reasons,
especially if a Perl programmer wants to be able to write the event
loop in Perl. It should be a purely trivial task to wrap Kqueue in a
wrapper for an XS expert, I see no reason not to.

Furthermore, it does not seem that libevent supports all of Kqueue
features.

 

[Rocco Caputo]

Thank you for the advice. I think doing a libevent interface i an
excellent idea. But I also think that there is still is a need for a
interface to kqueue, and that is on my list of things to study doing.
There are actually many event engines written in Perl that could
benefit from it, such as POE. Doing an engine in Perl has its
advantages, as it is eisier to understand the code and to improve upon
it, and there is less chance of memory buffer overflows.

Have a look at the POE::Loop documentation if you're considering
interfacing POE to BSD's kqueue. For portability's sake, all it really
requires is I/O events and a timeout.