Timer

[Anthony Briggs]

Anthony Briggs wrote:
...

Intercal. Lawrence Clark's 1973 article about "comefrom" in Fortran
was satire (just as all of Intercal is).

Ah. I just read the article at

Note that, in Python, you have that 'after' available any time
you're running a Tkinter GUI: any Tkinter widget has an 'after'
method that takes a delay (in milliseconds), a callable object,
and optionally some arguments, and schedules the callable to
be called with those arguments after that delay.

It works a charm, btw.

I'll take your word for it. It sounds like a recipe for a lot of hair
pulling to me. What happens if there's some part of your code that
you don't want interrupted? eg. database commits, file access, any
sort of time critical stuff, that sort of thing. Is there a way to
switch it off?

my money, the event-oriented model behind the [after]
above is at least as robust as any other.

Yes, that's what I said - only with fewer words

I thought you were arguing AGAINST the 'after' functionality,
and therefore against event-driven programming...?!

Perhaps I was taking his words a little too literally. As far as I
can tell with threads, etc, they're little better than voodoo. So the
after statement is at least as robust, even if it's a complete
horror...

Anthony

 

[Cameron Laird]

.
.
.

Note that, in Python, you have that 'after' available any time
you're running a Tkinter GUI: any Tkinter widget has an 'after'
method that takes a delay (in milliseconds), a callable object,
and optionally some arguments, and schedules the callable to
be called with those arguments after that delay.

It works a charm, btw.

.
.
.
I want to refine Alex' description: it's possible to
use after with only the slenderest connection to a
Tkinter GUI. You can "import Tkinter", and exploit
after(), without ever having to make Tkinter widgets
or windows visible on the screen.

 

[Alan Kennedy]

[Alan Kennedy]
[Alex Martelli]

Hmmm, what's wrong with Twisted+pyOpenSSL...? Seems to work fine...

I haven't used that particular combination.

I was going to followup my own post to mention that my statement was
not in any way meant to be slight to any of the providers of python
SSL. For example, Ng Pheng Siong has put a lot of work into getting
M2Crypto SSL working in non-blocking mode.

My understanding is that using SSL in non-blocking mode is problematic
because SSL-level packets can trigger readiness notification on an SSL
socket when in fact there is no application-level data available.

Here is a link that discusses the problem

http://www.openssl.org/support/faq.html#PROG9

I honestly don't know if all of the asynch models implemented in
python address this problem, and would be happy to be enlightened
and/or proven completely wrong if they have.

Java nio has this problem, which will be addressed in java 1.5

http://www.itworld.com/nl/java_sec/09142001/

[Alan Kennedy]

Another point in favour of threads: On a multi-processor box, the
thread abstraction generally "automagically" gives you free migration
to other processors. You (generally) don't have to change a line of
your code: the underlying [OS|VM] takes care of it.

[Alex Martelli]

*blink*? Uh, doesn't that depend on how you choose to have your
threads communicate with each other?

There is assumption behind my statement: that the underlying [OS|VM]
also maps the memory space of threads running on different processors
so that they coincide, and all objects are shared between threads as
if they were local to the "process" containing the threads.

If I use a Queue.Queue in jython, (AFAICT) the different threads can
be running on different processors, and the Queue.Queue will work just
fine as a synch mechanism.

The most Pythonic solution
is generally to have threads communicate via Queue instances. Can
you point me to a "cross-process Queue" class for Linux and Windows?

I'd love to. However, python's Queue.Queue class doesn't work across
processes, only across threads. But those threads could still
potentially be running on multiple processors in the same box (see
above re jython).

It'd be nice if we could have a Queue.Queue style IPC mechanism that
worked between processes, and that could be included in the set of
"selectables" for an event-driven framework. Without copious
serialization/pickling. I have stated that several times in this
forum.

Not a rhetoric question -- I'd LIKE to be able to scale things up that
way, but see some problems with translating typical Queue use idioms,
particularly in a cross-platform way, to cross-process:
-- N1 client-threads posting workrequests to Q and N2 worker-threads
peeling workrequests off Q and working on them;
-- a workrequest including a reference to the Queue instance on
which the worker-thread is going to post the response/result

I was interested to read about TupleSpaces from a recent post. Though
I haven't looked into TupleSpaces in detail, I can imagine that
there's lots of de/serialization going on.

Lastly, another area which seems to have patchy support across
operating systems is that of non-blocking file-locks, which would be
required for many event-driven apps.

regards,

 

[Cameron Laird]

.
.
.

I'll take your word for it. It sounds like a recipe for a lot of hair
pulling to me. What happens if there's some part of your code that
you don't want interrupted? eg. database commits, file access, any
sort of time critical stuff, that sort of thing. Is there a way to
switch it off?

.
.
.
Ah! There's confusion, just the sort that plagues *everyone*
on his or her first acquaintance with this concurrency model.

If you set an after() timer for, let's say, five seconds, then
launch a big atomic database operation, the database transaction
is NOT interrupted after five seconds. The five seconds means,
"check on me whenever you get a chance, and, if five seconds
have elapsed, then wake me up and I'll process for a while." In
the case at hand, the database transaction starts, and runs
through to completion, without interruption, even if it requires
*fifty* seconds. Once it's done (more-or-less, depending on
details of its invocation), the after callback gets its chance.
There's no hazard that needs to be "switched off".

This is how event processing works within a thread. A simple
program can use both threads and events, but they don't "cross".

 

[Alex Martelli]

Anthony Briggs wrote:
...

I'll take your word for it. It sounds like a recipe for a lot of hair
pulling to me. What happens if there's some part of your code that
you don't want interrupted? eg. database commits, file access, any
sort of time critical stuff, that sort of thing. Is there a way to
switch it off?

Event-driven processing, AKA asynchronous processing, is not
pre-emptive: on the contrary, it relies on each event handler
(callback) terminating reasonably promptly and thus returning
to the main loop. Hmmm -- ever programmed a GUI...?

Perhaps I was taking his words a little too literally. As far as I
can tell with threads, etc, they're little better than voodoo. So the

Threads are best kept in their place by keeping them well separated
with Queue instances, although I wouldn't be surprised to hear that
the blood of a black roster strangled at midnight is also effective.

after statement is at least as robust, even if it's a complete
horror...

Nope -- the callback is done in the same thread that requested
it, so "thread separation" is not at issue.


Alex

 

[Alex Martelli]

.
.
.
.
.
.
I want to refine Alex' description: it's possible to
use after with only the slenderest connection to a
Tkinter GUI. You can "import Tkinter", and exploit
after(), without ever having to make Tkinter widgets
or windows visible on the screen.

and I'll further refine yours: "as long as you run
Tkinter's mainloop". Yes, you can hide Tkinter's root
window and still run its mainloop (I think you have
to be explicit about it though, as long as you don't
need any of the normal processing flow used by most
non-GUI applications (including other event-driven apps
that don't use Tkinter's mainloop -- e.g., network ones
that use asyncore or select). And the .after callback
will be able to get into play only if other callbacks
(presumably also from .after, if you have no GUI) return
reasonably promptly.

In short, I doubt using Tkinter just for the sake of
getting .after makes much sense -- you're probably at
least as well off with the much lighter-weight scheduler
module, in that case. (OTOH, if it IS possible to
give fileobjects to Tkinter to cause callbacks when
they're ready, as it is in Tcl/Tk, then using Tkinter
w/o GUI may indeed be easier than merging e.g scheduler
and select -- it depends... but last time I checked I
saw no way to get the "callback on file activity" from
Tkinter as I could back when I used Tcl/Tk...).


Alex

 

[Michael Hudson]

In short, I doubt using Tkinter just for the sake of
getting .after makes much sense -- you're probably at
least as well off with the much lighter-weight scheduler
module, in that case. (OTOH, if it IS possible to
give fileobjects to Tkinter to cause callbacks when
they're ready, as it is in Tcl/Tk, then using Tkinter
w/o GUI may indeed be easier than merging e.g scheduler
and select -- it depends... but last time I checked I
saw no way to get the "callback on file activity" from
Tkinter as I could back when I used Tcl/Tk...).

Huh? _tkinter.createfilehandler() is there (pyrepl uses it). I think
only on Unix, but I think that also applies to Tcl_CreateFileHandler,
so that's not much of a surprise.

Cheers,
mwh

 

[Itamar Shtull-Trauring]

My understanding is that using SSL in non-blocking mode is problematic
because SSL-level packets can trigger readiness notification on an SSL
socket when in fact there is no application-level data available.

Here is a link that discusses the problem

http://www.openssl.org/support/faq.html#PROG9

Twisted does the right thing and makes sure the user doesn't have to
worry about this. Using SSL is just as easy as using TCP in Twisted.

There are other issues with event loops, like the fact the RSA
handshake is very expensive so you want to run it in another thread.
If Twisted ever has performance issues due to this I'll probably
change the code to do just that, move the handshake stage to a
threadpool.

 

[Alan Kennedy]

[Alan Kennedy]
[Itamar Shtull-Trauring]

Twisted does the right thing and makes sure the user doesn't have to
worry about this. Using SSL is just as easy as using TCP in Twisted.

There are other issues with event loops, like the fact the RSA
handshake is very expensive so you want to run it in another thread.
If Twisted ever has performance issues due to this I'll probably
change the code to do just that, move the handshake stage to a
threadpool.

Thanks for that Itamar.

IMHO, another good solution to the SSL/Non-Blocking IO problem might
be to run stunnel (www.stunnel.org) to do the SSL decode, and then
forward the decrypted requests to an non-SSL asynch server. Although I
have never tried this.

Stunnel can use OpenSSL, which means that support for hardware crypto
accelerators comes for free, etc, etc.

regards,

 

[Alex Martelli]

Michael Hudson wrote:
...

Huh? _tkinter.createfilehandler() is there (pyrepl uses it). I think
only on Unix, but I think that also applies to Tcl_CreateFileHandler,
so that's not much of a surprise.

I could have sworn I got "callbacks on file activity" cross-platform
when I used Tcl/Tk, but maybe it's just a memory grown rosy with age.


Alex

 

[Alexandre Fayolle]

Derek Fountain a écrit :

Does Python have a timer mechanism? i.e. an "after 500 milliseconds, run
this bit of code" sort of thing?

The sched module in the standard library may be what you're looking for.

 

[Michael Hudson]

Michael Hudson wrote:
...

I could have sworn I got "callbacks on file activity" cross-platform
when I used Tcl/Tk, but maybe it's just a memory grown rosy with age.

Well, maybe. I haven't been near a Windows system in months & months
and have never used Tcl/Tk in isolation, so please don't be surprised
when I'm wrong about Tcl on windows

Cheers,
mwh