Will python ever have signalhandlers in threads?

[Tim Peters]

....

[Antoon Pardon]

Sure but that is not enough is it? The work for letting one thread
raise an exception in an other thread is done.

Are you talking about the internal PyThreadState_SetAsyncExc gimmick?
If so, that's got nothing to do with signals. If that's all you want,
say so, and leave signals out of it.

As to whether the work is done, emphatically no, it isn't. I hadn't
looked at its implementation before, so just did. That uncovered a
critical bug "by eyeball", which may delay the release of Python 2.4.
Happy now <wink>?

Seriously, that's just an error in the C coding. The work for a
Python-level feature hasn't even begun. I'd guess it's about 5% of
the way there (believe it or not, writing C code is typically the
least time-consuming part of any language feature).

If you want to do that, fine. Then because this functionality is
controversial, it needs a PEP. That isn't "punishment", it's that
controversial decisions need a record of design rationale, and PEPs
are Python's way of making such records. The Python-level API also
needs design and debate. For example, the C-level API takes a thread
ID as argument, but that's a poor fit with threading.py's higher-level
view of threads. In this case it probably needs a higher-level API.
The C-level API also a bizarre gimmick where you're supposed to do an
obscure dance if the function returns a value greater than 1. Offhand
I have no idea what that's all about, but it's clearly unacceptable
for a Python-level function (maybe some good I suspect that
when the critical bug just found is fixed, that fix will automatically
stop the indeterminism driving the need for the goofy API dance).
Thought also needs to be given to whether other implementations of
Python *can* supply this gimmick. It's unclear to me. The CPython
implementation of it is very simple, but relies in all respects on
implementation details unique to CPython (primarily the GIL, and that
there's an exhaustive linked list of thread states to crawl over). A
new Python-level function also needs documentation, and a good test
suite. The C-level API here isn't tested at all, which isn't good.

But it still didn't get really in the language.

Of course it wasn't. As above, this isn't anywhere yet near being a
language-level feature. To judge from Guido's checkin comment, he was
also reluctant to add it even at the C level, which may (or may not)
be another battle. FWIW, it's fine by me if this got exposed at the
Python level. People will certainly get themselves in trouble by
using it, but the CPython implementation of this is so toothless I
don't think they'll be able to screw up internal states of the
platform C library by, e.g., leaving *its* synch gimmicks in
ill-defined states when they provoke a Python thread "externally" into
async surprises.

All that was provided was the C interface and people who want to use
it must provide the python interface themselves.

Yes. And looking at the history, the C interface was in fact all that
was asked for at the time.

....

I understand, but that also means one can't expect short term
contributors to put time in this, because they are likely to
see their time invested in this, going to waste.

That's their call. Beyond noting that few things are achieved by
those who give up easily, "that's life".

Now I may invest some time in this anyway, purely for my own
interest. The question I have is, will I just have to fight
the normal signal behaviour or do I have to fight python as
well.

I'm confused now. Up until this point, it appeared that "this", in
this message, was talking about the lack of access to
PyThreadState_SetAsyncExc from the Python level. If so, that's got
nothing to do with signals, at least not in CPython. The
implementation doesn't use signals, and is 100% portable exactly the
way it is. And it's very simple. Nobody can object to *this*
function on porting or maintenance-burden grounds. They may object to
it on other grounds.

To be more specific, the documentation of the signal
module states the following.

... only the main thread can set a new signal handler, and the main
thread will be the only one to receive signals (this is enforced by
the Python signal module, even if the underlying thread implementa-
tion supports sending signals to individual threads).

The question I have is the following. That the main thread is the only
one to receive signals, is that purely implemented in the signal module
or is there some interpreter magic that supports this, which can
cause problems for an alternative signal module.

Read the code? As I said before, I pay no attention to the signal
module, and the only way I could answer these questions is by studying
the code too. Maybe someone else here already knows, and will chip
in.

 

[Antoon Pardon]

...

[Antoon Pardon]

Sure but that is not enough is it? The work for letting one thread
raise an exception in an other thread is done.

Are you talking about the internal PyThreadState_SetAsyncExc gimmick?
If so, that's got nothing to do with signals. If that's all you want,
say so, and leave signals out of it.

I'm talking about PyThreadState_SetAsyncExc only as a throw in
response to your previous remark. The main subject is signals.

As to whether the work is done, emphatically no, it isn't. I hadn't
looked at its implementation before, so just did. That uncovered a
critical bug "by eyeball", which may delay the release of Python 2.4.
Happy now <wink>?

Well wether the work is done or not hardly seems to matter. The
documentation as worded now seems to make it clear, this isn't
going get into the language

Seriously, that's just an error in the C coding. The work for a
Python-level feature hasn't even begun. I'd guess it's about 5% of
the way there (believe it or not, writing C code is typically the
least time-consuming part of any language feature).

As I understand the doc, one doesn't plan to begin working on a
Python-level feature and even if someone else implements it, it
has no chance of getting in the language. As I read the docs it
is not so much a question of the feature not being ready for the
langauge, but a question of Guido not wanting the feature to be
in the language.


This just to show that having someone implement it, is not
the biggest hurdle as your prevous remark seemed to suggest
to me.

But that is enough of this tangent (although I did find your
remarks about it interesting)

Read the code? As I said before, I pay no attention to the signal
module, and the only way I could answer these questions is by studying
the code too. Maybe someone else here already knows, and will chip
in.

Well I'll wait a few days to see if someone does, otherwise I'll see
I can find enough time to dive into the code.


Although you can't help me with my main question, I like to express
my appreciation for your responses. They have been very insighfull
and I'll be sure to use them to my advantage.

 

[Tim Peters]

[Antoon Pardon]

I'm talking about PyThreadState_SetAsyncExc only as a throw in
response to your previous remark. The main subject is signals.
OK.
....

Well wether the work is done or not hardly seems to matter. The
documentation as worded now seems to make it clear, this isn't
going get into the language.

But you're still talking about PyThreadState_SetAsyncExc here, right?

Explained last time that it can't possibly "get into the language" in
the state it's in now. The people who orginally did the work got
everything they wanted at the time: the C API function, and the
then-new thread.interrupt_main() function in Python. They didn't ask
for more than that, and they didn't work on more than that.

Doing more work is a necessary prerequisite if people want more than
that in the language. Making the case that it should be in the
language is part of that work, but, if you haven't noticed, PEPs that
have working implementations fare much better than PEPs that don't.
Indeed, no PEP without an implementation has ever been released
<wink>.

....

As I understand the doc, one doesn't plan to begin working on a
Python-level feature and even if someone else implements it, it
has no chance of getting in the language.

Sorry, couldn't make sense of that sentence.

As I read the docs it is not so much a question of the feature not
being ready for the langauge, but a question of Guido not wanting the
feature to be in the language.

You're still talking about PyThreadState_SetAsyncExc? I haven't asked
Guido about it, and I can't find any design discussion of that
function anywhere. I think it got added during a European sprint. If
you want to know what he thinks, ask him. If you want a definitive
ruling, write a PEP.

This just to show that having someone implement it, is not
the biggest hurdle as your prevous remark seemed to suggest
to me.

An implementation is prerequisite to release, but isn't necessarily
sufficient. I explained in detail last time why the current C code
has no claim to being "an implementation" of a *Python*-level spelling
of this functionality, so neither a suitable implementation nor the
necessary design discusiion have been done in this case.

If this is something you want, but you also want guaranteed acceptance
in advance of doing more than writing about it, that won't work. If
you ask Guido, and he can make time to answer, you may get guaranteed
rejection in advance -- or you may not. A PEP would be a good thing
to have in *either* case.

...

Although you can't help me with my main question, I like to express
my appreciation for your responses. They have been very insighfull
and I'll be sure to use them to my advantage.

My fondest hope is that you'll be able to use them to crush Guido into
submission <wink>.

 

[Antoon Pardon]

[Antoon Pardon]

I'm talking about PyThreadState_SetAsyncExc only as a throw in
response to your previous remark. The main subject is signals.
OK.
...

Well wether the work is done or not hardly seems to matter. The
documentation as worded now seems to make it clear, this isn't
going get into the language.

But you're still talking about PyThreadState_SetAsyncExc here, right?
Yes.

Explained last time that it can't possibly "get into the language" in
the state it's in now.

And my response to that is that the documentation suggests that
that is not the main reason why it is not in the langauage.

The people who orginally did the work got
everything they wanted at the time: the C API function, and the
then-new thread.interrupt_main() function in Python. They didn't ask
for more than that, and they didn't work on more than that.

Doing more work is a necessary prerequisite if people want more than
that in the language. Making the case that it should be in the
language is part of that work, but, if you haven't noticed, PEPs that
have working implementations fare much better than PEPs that don't.
Indeed, no PEP without an implementation has ever been released
<wink>.

Look the documentation states this:

To prevent naive misuse, you must write your own C extension to call
this


IMO this says the reason for not having it in the language has
nothing to do with lack of implementation. In this case it seems
lack of implementation is the consequence of Guido and others
finding it to be too dangerous too be in the language. Not that
a lack of implementation is the reason it didn't make the language
yet.

...


Sorry, couldn't make sense of that sentence.

That raising an exception from one thread in an other thread doesn't
made the language is a so by design not by lack of implementation.
As the documentation is now worded it suggest very strongly that if
someone does implement it tomorrow it will simply rejected because
they want to prevent naive missuse.

You're still talking about PyThreadState_SetAsyncExc?
Yes.

I haven't asked
Guido about it, and I can't find any design discussion of that
function anywhere.

Wel that sentence came in the documentation, I don't know, but
something like: "To prevent ..." does sound like a design decision
to me.

I think it got added during a European sprint. If
you want to know what he thinks, ask him. If you want a definitive
ruling, write a PEP.

That seems a bit idiot to me. The information seems to be available
in the documentation. So why should I ask?

An implementation is prerequisite to release, but isn't necessarily
sufficient. I explained in detail last time why the current C code
has no claim to being "an implementation" of a *Python*-level spelling
of this functionality, so neither a suitable implementation nor the
necessary design discusiion have been done in this case.

If this is something you want, but you also want guaranteed acceptance
in advance of doing more than writing about it, that won't work. If
you ask Guido, and he can make time to answer, you may get guaranteed
rejection in advance -- or you may not. A PEP would be a good thing
to have in *either* case.

I don't want guaranteed acceptance in advance. But if I have the
impression that there is rejection in advance then there seems to
be no point in starting. The documentation as worded now seems to
imply such a rejection in advance.

So for the sake of argument even if I wanted this, I would be reluctant
to spend time in it, because I only have limited amounts of it and
this battle seems to be lost from the beginning. I would think my
time would be better spend fighting other battles.

 

[Michael Hudson]

I'm not really reading comp.lang.python at the moment, but found this
thread via the Python-URL...

The only currently active contributor I can think of who cares about
signals enough to actually work on them is Michael Hudson, but his
interest seems limited to keeping the interactions been Python's GNU
readline wrapper and signals working across incompatible details of
signal semantics on Unixish boxes. It's possible I'm wrong, and he'd
really love to work much more on signals, but feels inhibited by
Guido's well-known dislike of the beasts.

I think you're more or less right. A couple of years ago I got pretty
badly discouraged attempting to support sigprocmask and friends from
Python, and came to conclusion that noone who cares about portability
-- not just no Python programmers, but *noone at all* -- mixes threads
and signals, because the cross platform behaviour is a total mess, and
in many cases is just plain buggy (I believe the FreeBSD version of
libc_r for 5.something contains a fix that is a direct consequence of
my efforts, for example). Even linking with the threading libraries
was enough to stuff things up mightily in some circumstances.

We Do Not Want To Go There.

Cheers,
mwh

 

[Antoon Pardon]

I'm not really reading comp.lang.python at the moment, but found this
thread via the Python-URL...



I think you're more or less right. A couple of years ago I got pretty
badly discouraged attempting to support sigprocmask and friends from
Python, and came to conclusion that noone who cares about portability
-- not just no Python programmers, but *noone at all* -- mixes threads
and signals, because the cross platform behaviour is a total mess, and
in many cases is just plain buggy (I believe the FreeBSD version of
libc_r for 5.something contains a fix that is a direct consequence of
my efforts, for example). Even linking with the threading libraries
was enough to stuff things up mightily in some circumstances.

We Do Not Want To Go There.

Well for the moment I don't care about portability that much. I do
understand this is a big issue for something in the standard library
and with your remarks consider my question in the subject answered
with no.

I do think it is a pity that the crossplatform behaviour is such a
total mess but for the moment that can't be helped.

I'll just experiment a bit my self on the boxes I'm interested in.