What c.l.py's opinions about Soft Exception?

[Lie]

I'm asking about people in c.l.py's opinion about a _probably_ very
Pythonic way of doing something if such features is implemented. It is
to be known that I'm not a Python expert and actually relatively new
to Python programming, so probably I'm just not thinking pythonic
enough yet or this feature might already exist somewhere in a
different name.
Anyway, I'm just asking for opinions, tell me problems I haven't
foreseen, or whether such things would be hard to implement, or
whether you think the idea is great or plain bad (and why).

Soft Exception
What is "Soft Exception"?
Soft Exception is an exception that if is unhandled, pass silently as
if nothing happened. For example, if a variable turns into NoneType,
it'll raise Soft Exception that it have become NoneException,
programmers that wants to handle it can handle it with a try...except
block while programmers that doesn't care about it (or know it won't
be a problem to his code) can just leave the code as it is.

Soft Exception differs from Hard Exceptions (the regular Exception) in
a way that Hard Exception must be handled at all cost or the program
will be terminated while Soft Exception allow programmers not to
handle it if they don't want to.

Soft Exception is similar to an event-based system, although Soft
Exception can only be handled by codes above it while Event-based
system can be handled by anyone aware of the handle pool. The Soft
Exception can also be thought as a Warning to the codes above it that
it has done something that the codes above might want to know.

Implementation:
Simple implementation might be done by catching all exceptions at the
highest level, then filtering which exceptions would be stopped (Soft
Exception) and which exceptions will be reraised and terminate the
program (Hard Exception). This is simple and can be easily implemented
but is inefficient as that means all soft exceptions must bubble
through its way to the top to find out if it is Soft or Hard.

A more through implementation would start from the raiser inspecting
the execution stack and finding whether there are any try block above
it, if no try block exist it pass silently and if one exist it will
check whether it have a matching except clause. This also circumvents
a problem that simple implementation have, as described below.

Syntax change is probably unneeded and a Soft Exception class may be
created by inheriting from BaseSoftException class.

Problems:
- If the raising statement is a complex statement (like function call)
instead of just a simple statement (like assignment from an
expression) the exception might catch a similar soft exceptions from
deep _inside_ the function call and handle it as if it happened in the
code it's protecting. This problem can be solved by raising Soft
Exception only once except if explicitly reraised (perhaps through
except SoftException: raise).
This can be done by modifying a flag that is turned off (Falsed) if
the Soft Exception is raised and turned on again (Trued) if the Soft
Exception is reraised. Normal Exceptions (Hard Exceptions) would have
this flag turned off (Falsed) if it is handled and turned on (Trued)
again if it is reraised.

- To be half useful, soft exceptions have to be raised everywhere here
and there, not just here and here only. This might require a massive
change in current codes, or at least in Python's official libraries.

- Irresponsible programmers. Sometimes lazy programmers might decides
to be lazy and make all exceptions soft so he doesn't have to handle
it.

Ideology Base:
- EAAP: Easier to apologize than to ask permission.

Others:
- Sometimes it might be useful to convert a Soft Exception into Hard
Exception or vice versa.

 

[Kay Schluehr]

A more through implementation would start from the raiser inspecting
the execution stack and finding whether there are any try block above
it, if no try block exist it pass silently and if one exist it will
check whether it have a matching except clause. This also circumvents
a problem that simple implementation have, as described below.

This will not be easy in particular in the presence of inheritance and
dynamism. There is no way to statically decide whether an exception
BException has the type AException and will be caught by the except
clause in

try:
BLOCK
except AException, e:
print "SoftException %s caught"%e

A feasible solution was to invert the try...except statement and
creating a continuation.

catch AException, a:
print "SoftException A: %s"%a
catch BException , b:
print "SoftException B: %s"%b
....
in:
BLOCK

Here each SoftException is raised initially when a catch clause is
entered and a continuation is created that returns to the catch block
of the raised SoftException if required. When a SoftException is
raised within BLOCK a lookup will be made and if a corresponding
SoftException was found that was raised by a catch-clause the current
control flow will be suspended and the continuation is called.

 

[Steven D'Aprano]

Soft Exception
What is "Soft Exception"?
Soft Exception is an exception that if is unhandled, pass silently as if
nothing happened. For example, if a variable turns into NoneType, it'll
raise Soft Exception that it have become NoneException, programmers that
wants to handle it can handle it with a try...except block while
programmers that doesn't care about it (or know it won't be a problem to
his code) can just leave the code as it is.

Soft Exception differs from Hard Exceptions (the regular Exception) in a
way that Hard Exception must be handled at all cost or the program will
be terminated while Soft Exception allow programmers not to handle it if
they don't want to.

I don't think that there are very many cases where exceptions can be
ignored safely. There are two main reasons for using exceptions:

(1) Signaling an exceptional event.

(2) An error occurred.

I can't think of many cases where you would wish to ignore either, and
just continue processing. The only examples I can think of are in loops,
where you are doing the same thing over and over again with just a little
change, and you wish to skip any problematic data, e.g.:

def plot_graph(func, domain):
for x in domain:
plot(x, func(x))

If an error occurs in plot() for one particular x value, you would want
to ignore it and go on to the next point. But that's easy enough to do
with a regular try...except block.

Simply put, you're suggesting the following two alternatives:

Hard Exceptions: terminate the program unless explicitly silenced
Soft Exceptions: pass silently unless explicitly caught

In this case, I agree with the Zen of Python ("import this"):

Errors should never pass silently.
Unless explicitly silenced.

The cost of explicitly silencing exceptions is tiny, the risk of misuse
of Soft Exceptions is very high, and the benefit of them is negligible.

 

[Kay Schluehr]

Hard Exceptions: terminate the program unless explicitly silenced
Soft Exceptions: pass silently unless explicitly caught

In this case, I agree with the Zen of Python ("import this"):

Errors should never pass silently.
Unless explicitly silenced.

Exceptions in Python don't necessarily signal errors. Just think about
StopIteration.

Note also that the common practice of letting *possible* errors passed
silently is to return None instead of raising an exception. Moreove
people create boilerplate like this

try:
k = lst.index(elem)
...
except IndexError:
pass

instead of

with lst.index(elem) as k:
...

It would be interesting to think about SoftException semantics for
such clauses: lst.index would neither raises a HardException nor does
it return None but leads to skipping the with-block.

Is it really so exotic that it requires the demand for more use
cases?

 

[Steven D'Aprano]

Exceptions in Python don't necessarily signal errors. Just think about
StopIteration.

I know that. That's why I said that exceptions were used for signaling
exceptional events.

Note also that the common practice of letting *possible* errors passed
silently is to return None instead of raising an exception.

"Common"? Who does that?

I know re.search() etc. return None in the event the regex doesn't match.
That's not an error.

Moreove people create boilerplate like this

try:
k = lst.index(elem)
...
except IndexError:
pass

instead of

with lst.index(elem) as k:
...

Possibly because the with keyword is quite new and many people don't know
it, and much code was written before it even existed, or they have to
support Python 2.4 or older.

It would be interesting to think about SoftException semantics for such
clauses: lst.index would neither raises a HardException nor does it
return None but leads to skipping the with-block.

Is it really so exotic that it requires the demand for more use cases?

Are the existing solutions really so incomplete that we need yet another
solution?

What problem are you trying to solve with SoftExceptions?

How would changing lst.index() to raise a soft exception help here?

pos = lst.index('foo')
lst[pos] = 'bar'

What is that code going to do if 'foo' isn't found in lst and it raises a
silent, unhandled SoftException? Do you expect Python to be context
sensitive, and raise HardExceptions in some places and SoftExceptions in
others? Who controls that?

 

[Lie]

This will not be easy in particular in the presence of inheritance and
dynamism. There is no way to statically decide whether an exception
BException has the type AException and will be caught by the except
clause in

try:
BLOCK
except AException, e:
print "SoftException %s caught"%e


A feasible solution was to invert the try...except statement and
creating a continuation.

catch AException, a:
print "SoftException A: %s"%a
catch BException , b:
print "SoftException B: %s"%b
...
in:
BLOCK

Here each SoftException is raised initially when a catch clause is
entered and a continuation is created that returns to the catch block
of the raised SoftException if required. When a SoftException is
raised within BLOCK a lookup will be made and if a corresponding
SoftException was found that was raised by a catch-clause the current
control flow will be suspended and the continuation is called.

I'd rather want to avoid any syntax changes, as I wished that Soft
Exception can be added to the language silently[1] so that new
programmers doesn't _need_ to know about it (although knowing it could
change the way they write codes to be more structured and simple).

[1] Definition of silently: Codes that aren't aware of this
functionality shouldn't break. Adding new syntax usually means adding
keywords, making possible break in current program.

I don't think that there are very many cases where exceptions can be
ignored safely. There are two main reasons for using exceptions:

(1) Signaling an exceptional event.

In that case, programmers might decide whether to raise Soft or Hard
Exception. Hard Exception is much preferred.

(2) An error occurred.

Which must always be handled with Hard Exception.

Adding another thing
(3) Informing codes above it about what's currently happening inside,
the thing is just a mundane report that might be useful to codes above

Which might be a useful place to use SoftExceptions

I can't think of many cases where you would wish to ignore either, and
just continue processing. The only examples I can think of are in loops,
where you are doing the same thing over and over again with just a little
change, and you wish to skip any problematic data, e.g.:

def plot_graph(func, domain):
    for x in domain:
        plot(x, func(x))

If an error occurs in plot() for one particular x value, you would want
to ignore it and go on to the next point. But that's easy enough to do
with a regular try...except block.

No, you're misunderstanding the purpose of Soft Exception, it's not
for silencing errors and not so much for exceptional cases. It's for
the more mundane tasks such as:

from __future__ import division

class SomeNumeric(object):
def __div__(a, b):
if b == 0: raise ZeroDivisionError ## Hard Exception, don't
ignore me!
if a == 0: raise ZeroNumerator ## Soft Exception
f = a / b
i = a // b
if f == float(i):
raise IntegerDivision ## Soft Exception
return a // b
else:
raise FloatDivision ## Soft Exception
return a / b

Most people can ignore the ZeroNumerator, IntegerDivision, and
FloatDivision exceptions (warnings) because they're excessive and
unnecessary, but some people might want to catch them and do something
else (especially ZeroNumerator). Practicle example, far below.

The example is actually quite bad at demonstrating the purpose of Soft
Exception as it is very simple while Soft Exception is generally more
useful in complex operations. But I want to avoid giving complex
examples since it'll be more difficult to explain the complex examples
instead of the simple examples.

Simply put, you're suggesting the following two alternatives:

Hard Exceptions: terminate the program unless explicitly silenced
Soft Exceptions: pass silently unless explicitly caught

In this case, I agree with the Zen of Python ("import this"):

Errors should never pass silently.
Unless explicitly silenced.

That's what sloppy programmers do, silently pass errors. OTOH, Soft
exceptions are not used for errors (perhaps the wording can be better
phrased: must not be used for errors), they're used for events that
some might have interest in, but some would consider it as normal.
That's why I mentioned to think of it as a Warning. Operations that
raise Soft Exceptions should be able to run normally even when the
exception isn't handled (although it might generate garbage out that
can be handled at later time).

The cost of explicitly silencing exceptions is tiny, the risk of misuse
of Soft Exceptions is very high, and the benefit of them is negligible.

Perhaps relabeling it as Warning, and renaming raise SoftException as
give Warning might make it more acceptable?
And I agree that the probability for misuse is quite high, but the
benefits is also quite high, it's just that you can't see it since
you're not used to using such exceptions. The benefit of
SoftExceptions lies mostly on the regular programmings tasks, not the
exceptional programming tasks

Practical Example:
This example takes _case ideas_ from this simple gravity simulator
http://www.pygame.org/project/617/ BUT _no line of code is taken from
it_. I only give this link so you can easily know what the case is
about without lengthy explanation.

A particle machine.
The particle machine calculates gravity created by the particles in a
field. Additionaly, it clumps together two particles that happens to
be near enough (less than the sum of their radiuses).

The force two particle is expressing to each other is calculated with:
def calculateforce(P1, P2):
return (P1.mass - P2.mass) / distance(P1, P2)

and this is done to every particle in the field against the current
particle.

And the distance is calculated by:
def distance(P1, P2)
return (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2

The problem happens when the distance is small enough and we want to
clump them together.

A possible solution to this problem might be to check whether distance
is less than P1.radius + P2.radius in the calculateforce.
But, this obfuscate the code since we have to separate distance
calculation from the main formula (see !s), and this also insist that
clumping be done on force calculation level (see @s), shortly this
piece of code is plain bad:
def distance(P1, P2):
return (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2

def calculateforce(P1, P2):
## Separating this dist calculation into its own line is
## necessary if we want to check the value of dist
## Personally I think that's a bit obfuscated.
## Well known formulas should be kept to one line if possible
! dist = distance(P1, P2)

if dist <= P1.radius + P2.radius:
## Calling clump() here is bad, because
## there are occasions where we only want to
## calculate force but doesn't want to
## clump it
@ clump(P1, P2)
else:
! return (P1.mass - P2.mass) / dist

## Codes calling calculateforce()
# Note: this code is located inside a loop

F = calculateforce(P1, P2)
# Do something else, acceleration calculation, movement
calculations, etc


A better refactoring would be like this, but this requires calculating
distance twice (see !s):
def distance(P1, P2):
return (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2

def calculateforce(P1, P2):
## Here distance is calculated once
! return (P1.mass - P2.mass) / distance(P1, P2)

## Codes calling calculateforce()
# Note: this code is located inside a loop

## Here distance is calculated again
! if distance(P1, P2) <= P1.radius + P2.radius:
clump(P1, P2)
break
F = calculateforce(P1, P2)
# Do something else, acceleration calculation, movement
calculations, etc


A much better solution would be to use SoftException
def distance(P1, P2):
D = (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2
if D <= P1.radius + P2.radius: raise Collision
return D

def calculateforce(P1, P2):
try:
F = (P1.mass - P2.mass) / distance(P1, P2)
except Collision:
raise

## Codes calling calculateforce()
# Note: this code is located inside a loop
try:
F = calculateforce(P1, P2)
except Collision:
clump(P1, P2)
break # Calculate the next particle pair
else:
# Do something else, acceleration calculation,
# speed calculation, movement calculations, etc

This results in a cleaner code. And it also allow _other part of
codes_ that uses calculate distance and force to easily ignore or
handle the Collision Exception. If this code had used Hard Exception,
other codes would have to explicitly silence the exception or the
program terminates. That would be too much since Collision is
technically not an Error, but just a normal events that you might be
interested to know. Soft Exception allows events of interest to be
noticed or be ignored depending on the requirement.

It's like a notice board: In the notice board, there are notices about
the Maths Test next week, which you must not ignore (Hard Exception),
but there are also notices about Part-time Job Advertisement, if
you're interested about it you can look at it further, else you could
just ignore it and do nothing (Soft Exception)

 

[Kay Schluehr]

This will not be easy in particular in the presence of inheritance and
dynamism. There is no way to statically decide whether an exception
BException has the type AException and will be caught by the except
clause in

try:
BLOCK
except AException, e:
print "SoftException %s caught"%e
A feasible solution was to invert the try...except statement and
creating a continuation.

catch AException, a:
print "SoftException A: %s"%a
catch BException , b:
print "SoftException B: %s"%b
...
in:
BLOCK

Here each SoftException is raised initially when a catch clause is
entered and a continuation is created that returns to the catch block
of the raised SoftException if required. When a SoftException is
raised within BLOCK a lookup will be made and if a corresponding
SoftException was found that was raised by a catch-clause the current
control flow will be suspended and the continuation is called.

I'd rather want to avoid any syntax changes, as I wished that Soft
Exception can be added to the language silently[1] so that new
programmers doesn't _need_ to know about it (although knowing it could
change the way they write codes to be more structured and simple).

I just tried to save your proposal from being unimplementable. Maybe
you can comment on it?

I know of course that syntax is Pythons holy cow but I'm not Guidos
mouthpiece and have a different perspective on some aspects of the
system for obvious reasons [1]. I appreciate when people in the Python
community talk about what is useful for them and not what the
imaginary Pythonista superego thinks about them and how it will be
standardized in the end - if anyhow.

I'd like to know if SoftExceptions would make programs more reliable
in the end because people won't use them sparingly but for all kinds
of events that are somewhat "irregular" ( i.e. warnings ) not just for
obvious errors. It's an interesting idea to have an advanced warning
system that is more responsive than just this kind of logging which is
currently implemented. And I'd surely discriminate them from the
current exception handling at least in the design phase. Later
language changes could still be "dissolved".

[1] http://www.fiber-space.de/EasyExtend/doc/EE.html

 

[Mel]

Lie wrote:
[ ... ]

Soft Exception
What is "Soft Exception"?
Soft Exception is an exception that if is unhandled, pass silently as
if nothing happened. For example, if a variable turns into NoneType,
it'll raise Soft Exception that it have become NoneException,
programmers that wants to handle it can handle it with a try...except
block while programmers that doesn't care about it (or know it won't
be a problem to his code) can just leave the code as it is.

Soft Exception differs from Hard Exceptions (the regular Exception) in
a way that Hard Exception must be handled at all cost or the program
will be terminated while Soft Exception allow programmers not to
handle it if they don't want to. [ ... ]
Ideology Base:
- EAAP: Easier to apologize than to ask permission.

Sort of like a COME FROM statement. Drastic effects on program
execution: a `raise SoftException` in the middle of a loop would break
the loop if a catcher existed, or leave the loop running if not. It
would really need the ability to resume after catching an exception.
You can't really talk about 'apologize' around something that's so
irreparable.

I'd try for this effect by creating a class of objects with
well-defined callbacks.

Mel.

 

[Bryan Olson]

Lie wrote:
[...]

Soft Exception is an exception that if is unhandled, pass silently as
if nothing happened. [...]

Implementation:
Simple implementation might be done by catching all exceptions at the
highest level, then filtering which exceptions would be stopped (Soft
Exception) and which exceptions will be reraised and terminate the
program (Hard Exception). This is simple and can be easily implemented
but is inefficient as that means all soft exceptions must bubble
through its way to the top to find out if it is Soft or Hard.

If I'm following what you want, that "simple implementation" does
not work. If a function, possibly deep in a call stack, raises a
soft exception that no caller above catches, what executes next?

Correct me if I'm misunderstanding your idea: You want raising
an un-caught soft exception to be equivalent to a 'pass';
execution continues as if the 'raise' never happened.

Catching everything at a high level does nothing like that. The
exception causes execution to leave the function invoking the
raise statement, and abolishes the stack state from the point of
the raise to the point of the catch.

As Python exceptions currently work, "catching all exceptions
at the highest level" is either what Python already does, or
ill-defined nonsense. When an exception is uncaught, there is
no useful "highest level", other than the level at which the
program simply fails. Python *must* terminate execution upon
an unhanded exception, because the program defined no state
from which executions could correctly continue.

A more through implementation would start from the raiser inspecting
the execution stack and finding whether there are any try block above
it, if no try block exist it pass silently and if one exist it will
check whether it have a matching except clause. This also circumvents
a problem that simple implementation have, as described below.

The described problems do not include the "where should execution
resume" problem, which I think is central to the issue. Correct me
if I've misunderstood: A "soft" exception is one that gets raised
only if some calling frame has arranged to catch it.

The if-exception-would-be-unhanded-then-pass logic strikes me as
interesting. It would be a huge change to Python, so I doubt it
will get traction here. Still, I'd say it's worth more
consideration and discussion.

 

[Lie]

Lie wrote:

[...]> Soft Exception is an exception that if is unhandled, pass silently as

if nothing happened.
[...]

Implementation:
Simple implementation might be done by catching all exceptions at the
highest level, then filtering which exceptions would be stopped (Soft
Exception) and which exceptions will be reraised and terminate the
program (Hard Exception). This is simple and can be easily implemented
but is inefficient as that means all soft exceptions must bubble
through its way to the top to find out if it is Soft or Hard.

If I'm following what you want, that "simple implementation" does
not work.  If a function, possibly deep in a call stack, raises a
soft exception that no caller above catches, what executes next?

The highest possible code that could catch exceptions would have
something like this:

try:
## Everything is happening inside me
except SoftException:
pass

Correct me if I'm misunderstanding your idea: You want raising
an un-caught soft exception to be equivalent to a 'pass';
execution continues as if the 'raise' never happened.

Catching everything at a high level does nothing like that. The
exception causes execution to leave the function invoking the
raise statement, and abolishes the stack state from the point of
the raise to the point of the catch.

The high-level mentioned here is outside our own code, it'll be inside
Python's internal. When CPython (or any implementation of Python) sees
that a particular exception is raised to a certain point, it'll check
whether it is of type SoftException, and if it is, it'll return
program state to the place where the exception is raised before. I
don't know how Python's exception system works (as I tell you I know
next to nothing about Python's internal, although I'm interested to
know), so probably if there is any defect in the implementation
schemes I mentioned, it is simply caused by my ignorance.

As Python exceptions currently work, "catching all exceptions
at the highest level" is either what Python already does, or
ill-defined nonsense. When an exception is uncaught, there is
no useful "highest level", other than the level at which the
program simply fails. Python *must* terminate execution upon
an unhanded exception, because the program defined no state
from which executions could correctly continue.

That's where the difference between Soft Exceptions and Hard
Exceptions lies. Soft Exception must be continued while Hard
exceptions must terminate programs. Possibly this is impossible at the
absolutely highest level, perhaps it should be done at the level that
can guarantee

The described problems do not include the "where should execution
resume" problem, which I think is central to the issue.

I didn't mentioned it?
I think it's obvious when I say: Execution should resume as if nothing
happened, as if there is nothing raised, that means execution resumes
at the point after the raise statement. If something was caught,
execution isn't resumed and execution continues at the level of the
handler (possibly we could also add a "resume" to explicitly resume
the statement that was caught)

Correct me
if I've misunderstood: A "soft" exception is one that gets raised
only if some calling frame has arranged to catch it.

That could be a way to implement it. And btw, that's a one-line-
explanation that hits the right note (although from a different view I
initially mentioned).

 

[Lie]

I'd rather want to avoid any syntax changes, as I wished that Soft
Exception can be added to the language silently[1] so that new
programmers doesn't _need_ to know about it (although knowing it could
change the way they write codes to be more structured and simple).

I just tried to save your proposal from being unimplementable. Maybe
you can comment on it?

Perhaps I'm not the appropriate person to talk about whether unchanged
syntax is feasible for such implementation since I basically have no
idea about how Python's internals works. It's only that I think if
current syntax could be used, we could just use it (except if there is
a strong reason why it shouldn't be done).

I know of course that syntax is Pythons holy cow but I'm not Guidos
mouthpiece and have a different perspective on some aspects of the
system for obvious reasons [1].

I agree, everybody have different perspective. But that differences is
what makes languages evolves as it'd be continuously searching for the
most optimal perspective.

 

[Steven D'Aprano]

(3) Informing codes above it about what's currently happening inside,
the thing is just a mundane report that might be useful to codes above

Which might be a useful place to use SoftExceptions

Okay, now we're getting somewhere.

So, I have a function foo() which raises a HardException on error, but it
also raises a SoftException if it wants to notify me of something
"mundane".

def foo(sequence):
if args == []:
raise SoftException("empty list")
return len(args)

Now, how do I use that?

try:
x = foo(something)
except TypeError:
print "you should pass a sequence"
sys.exit() # unrecoverable error
except SoftException:
print "you passed an empty list"
print x

Am I close?

But there's a problem. Once the SoftException is caught, execution will
continue from the line "print x" -- but the function foo() never got a
chance to actually return a result!

In order to make that work, you would need a significant change to
Python's internals. I don't know how difficult that would be, but I'm
guess that it would be a lot of work for not much benefit.

But even if that happened, it would mean that the one mechanism has TWO
different effects:

try:
x = foo(sequence)
except SoftException:
print x # this is okay, because foo() did return
except TypeError:
print x # this is NOT okay, because foo() never returned


That is a recipe for confusion.



[...]

No, you're misunderstanding the purpose of Soft Exception, it's not for
silencing errors and not so much for exceptional cases. It's for the
more mundane tasks such as: [...]
Perhaps relabeling it as Warning, and renaming raise SoftException as
give Warning might make it more acceptable?

Do you realise that Python already has a warnings module?

And I agree that the
probability for misuse is quite high, but the benefits is also quite
high, it's just that you can't see it since you're not used to using
such exceptions. The benefit of SoftExceptions lies mostly on the
regular programmings tasks, not the exceptional programming tasks

Practical Example:
This example takes _case ideas_ from this simple gravity simulator
http://www.pygame.org/project/617/ BUT _no line of code is taken from
it_. I only give this link so you can easily know what the case is about
without lengthy explanation.

A particle machine.
The particle machine calculates gravity created by the particles in a
field. Additionaly, it clumps together two particles that happens to be
near enough (less than the sum of their radiuses).

The force two particle is expressing to each other is calculated with:
def calculateforce(P1, P2):
return (P1.mass - P2.mass) / distance(P1, P2)

and this is done to every particle in the field against the current
particle.

And the distance is calculated by:
def distance(P1, P2)
return (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2

The problem happens when the distance is small enough and we want to
clump them together.

A possible solution to this problem might be to check whether distance
is less than P1.radius + P2.radius in the calculateforce. But, this
obfuscate the code since we have to separate distance calculation from
the main formula (see !s),

I don't agree that this obfuscates the code.

and this also insist that clumping be done on
force calculation level (see @s), shortly this piece of code is plain
bad:
def distance(P1, P2):
return (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2

def calculateforce(P1, P2):
## Separating this dist calculation into its own line is
## necessary if we want to check the value of dist
## Personally I think that's a bit obfuscated.
## Well known formulas should be kept to one line if possible
! dist = distance(P1, P2)

if dist <= P1.radius + P2.radius:
## Calling clump() here is bad, because
## there are occasions where we only want to
## calculate force but doesn't want to
## clump it
@ clump(P1, P2)
else:
! return (P1.mass - P2.mass) / dist

I agree that calling clump() there is bad.

## Codes calling calculateforce()
# Note: this code is located inside a loop

F = calculateforce(P1, P2)
# Do something else, acceleration calculation, movement
calculations, etc


A better refactoring would be like this, but this requires calculating
distance twice (see !s):

That's not better. Don't do it.

def distance(P1, P2):
return (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2

def calculateforce(P1, P2):
## Here distance is calculated once
! return (P1.mass - P2.mass) / distance(P1, P2)

## Codes calling calculateforce()
# Note: this code is located inside a loop

## Here distance is calculated again
! if distance(P1, P2) <= P1.radius + P2.radius:
clump(P1, P2)
break
F = calculateforce(P1, P2)
# Do something else, acceleration calculation, movement
calculations, etc

And here's a way to do it that doesn't calculate anything twice, and
doesn't require any exceptions:


def calculateforce(P1, P2, dist):
return (P1.mass - P2.mass)/dist


And then for all pairs of particles:

dist = distance(P1, P2)
if dist <= P1.radius + P2.radius:
clump(P1, P2)
break
F = calculateforce(P1, P2, dist)

A much better solution would be to use SoftException
def distance(P1, P2):
D = (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2
if D <= P1.radius + P2.radius:
raise Collision
return D

But look at that line, "raise Collision". You could replace that with a
callback function, and have the same result:

DO_NOTHING = lambda : None

def distance(P1, P2, callback=DO_NOTHING):
D = (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2
if D <= P1.radius + P2.radius:
callback()
return D


That gives you virtually everything you want. If you want to ignore the
signal, you simply call distance(P1, P2). If you want to do something on
the signal, you set the callback.

But frankly, the function distance() is NOT the place for that. Why
should the distance() function decide what's a special result and what
isn't? With your plan, you end up with functions like this:

def distance(P1, P2):
D = (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2
if D <= P1.radius + P2.radius:
raise Collision
elif D > 1000:
raise FredsModuleEscape
elif D <= 1:
raise MagnetismModuleUnity
elif D == 0:
raise AnotherFunctionSingularity
elif ...
...
...
else:
raise NothingSpecialHappened
return D


Every module and function that calls distance() will start demanding that
it raises the SoftExceptions that *it* wants, and before you know it,
your distance() function has a hundred SoftExceptions covering all sorts
of things that most people don't care about.

No. This is a terrible idea. If the caller wants to treat a particular
result as special, the caller should be responsible for detecting that,
not the callee.

This results in a cleaner code.

I don't agree.

And it also allow _other part of codes_
that uses calculate distance and force to easily ignore or handle the
Collision Exception.

And demand their own SoftExceptions.


Okay, now that I understand your intention better, I've gone from -1 to
-2 on the idea. I no longer think it's a bad idea, I think it's a
TERRIBLE idea.

Just out of curiosity, are there any existing languages that do something
like this, or did you invent it yourself?

 

[Diez B. Roggisch]

I'm asking about people in c.l.py's opinion about a _probably_ very
Pythonic way of doing something if such features is implemented. It is
to be known that I'm not a Python expert and actually relatively new
to Python programming, so probably I'm just not thinking pythonic
enough yet or this feature might already exist somewhere in a
different name.
Anyway, I'm just asking for opinions, tell me problems I haven't
foreseen, or whether such things would be hard to implement, or
whether you think the idea is great or plain bad (and why).

Soft Exception
What is "Soft Exception"?
Soft Exception is an exception that if is unhandled, pass silently as
if nothing happened. For example, if a variable turns into NoneType,
it'll raise Soft Exception that it have become NoneException,
programmers that wants to handle it can handle it with a try...except
block while programmers that doesn't care about it (or know it won't
be a problem to his code) can just leave the code as it is.

Soft Exception differs from Hard Exceptions (the regular Exception) in
a way that Hard Exception must be handled at all cost or the program
will be terminated while Soft Exception allow programmers not to
handle it if they don't want to.

<snip/>

Is this soft-exception implemented anywhere, so that one can see what
experiences and best practices have evolved around using it?

Diez

 

[Kay Schluehr]

A more through implementation would start from the raiser inspecting
the execution stack and finding whether there are any try block above
it, if no try block exist it pass silently and if one exist it will
check whether it have a matching except clause. This also circumvents
a problem that simple implementation have, as described below.
This will not be easy in particular in the presence of inheritance and
dynamism. There is no way to statically decide whether an exception
BException has the type AException and will be caught by the except
clause in
try:
BLOCK
except AException, e:
print "SoftException %s caught"%e
A feasible solution was to invert the try...except statement and
creating a continuation.
catch AException, a:
print "SoftException A: %s"%a
catch BException , b:
print "SoftException B: %s"%b
...
in:
BLOCK
Here each SoftException is raised initially when a catch clause is
entered and a continuation is created that returns to the catch block
of the raised SoftException if required. When a SoftException is
raised within BLOCK a lookup will be made and if a corresponding
SoftException was found that was raised by a catch-clause the current
control flow will be suspended and the continuation is called.
I'd rather want to avoid any syntax changes, as I wished that Soft
Exception can be added to the language silently[1] so that new
programmers doesn't _need_ to know about it (although knowing it could
change the way they write codes to be more structured and simple).

I just tried to save your proposal from being unimplementable. Maybe
you can comment on it?

Perhaps I'm not the appropriate person to talk about whether unchanged
syntax is feasible for such implementation since I basically have no
idea about how Python's internals works. It's only that I think if
current syntax could be used, we could just use it (except if there is
a strong reason why it shouldn't be done).

You are an appropriate person to consider the workflow in a dynamic
language, no matter how the language is implemented internally.

Just start with function calls

maybe_raise(ZeroDivisionError)

The only requirement is that maybe_raise has to know when it shall
raise ZeroDivisionError. This depends on whether the exception is
caught. How do the program knows this in advance? There are no static
analysis techniques available.

When maybe_raise is entered the system must know that the exception is
handled in the future. You can't inspect the call stack for this
purpose because the call stack represents past events and
continuations ( and you can't rely on names ).

So you need something like this

do_softexception(ZeroDivisionError)
try:
TRY_BLOCK
except ZeroDivisionError:
EXCEPT_BLOCK

But this looks odd and the solution isn't DRY. So better one macro-
transforms a new statement into this form.

 

[Lie]

(3) Informing codes above it about what's currently happening inside,
the thing is just a mundane report that might be useful to codes above

Which might be a useful place to use SoftExceptions

Okay, now we're getting somewhere.

So, I have a function foo() which raises a HardException on error, but it
also raises a SoftException if it wants to notify me of something
"mundane".

def foo(sequence):
    if args == []:
        raise SoftException("empty list")
    return len(args)

Now, how do I use that?

try:
   x = foo(something)
except TypeError:
   print "you should pass a sequence"
   sys.exit()  # unrecoverable error
except SoftException:
   print "you passed an empty list"
print x

Am I close?

But there's a problem. Once the SoftException is caught, execution will
continue from the line "print x" -- but the function foo() never got a
chance to actually return a result!

In that particular case above, you don't need to handle the Soft
Exception. As stated in it's original purpose, you don't need to
handle a Soft Exception, it exists there if you need to handle the
exceptional case, but ignorable on other cases. _IF_ printing the "you
passed an empty list" is an important operation, you'd make it like
this:
try:
x = foo(something)
except TypeError:
print "you should pass a sequence"
sys.exit() # unrecoverable error
except SoftException:
print "you passed an empty list"
x = 0
print x

or alternatively:

try:
x = foo(something)
except TypeError:
print "you should pass a sequence"
sys.exit() # unrecoverable error
except SoftException:
print "you passed an empty list"
else:
print x

The case you states above only mentions that you haven't fully grasped
the workflow in exception-based system.

In order to make that work, you would need a significant change to
Python's internals. I don't know how difficult that would be, but I'm
guess that it would be a lot of work for not much benefit.

I too don't know how difficult that'd be, especially because the only
thing I know about Python's internal is it implements its garbage
collector by ref counting (which is barely useful here).

But even if that happened, it would mean that the one mechanism has TWO
different effects:

try:
    x = foo(sequence)
except SoftException:
    print x  # this is okay, because foo() did return
except TypeError:
    print x  # this is NOT okay, because foo() never returned

That is a recipe for confusion.

Both are not okay since foo() never return on both cases (soft
statement doesn't return to finish the try clause except if explicitly
resumed).

(snip)

Do you realise that Python already has a warnings module?

Ah... I completely forget about it, but anyway the Warning module is
unrelated to this. Perhaps we could just think of another name, but
names doesn't seems to be important in Python modules anyway, can you
guess what pickle/zip/mutex/quopri/curses is if you're new to Python
without looking at the documentations? There are some modules in
Python that have weird names, we ought to reduce it, but we can't
reduce it... we ought to live with it.

(snip)

I don't agree that this obfuscates the code.

For a formula as complex as this:
http://www.mapleprimes.com/blog/axel-vogt/computing-the-complex-gamma-function-using-spouges-formula
it might be useful to fragment the formula to smaller pieces

but in a simple to moderately complex that still fits in one line,
fragmenting the code confuses the eye.

And here's a way to do it that doesn't calculate anything twice, and
doesn't require any exceptions:

def calculateforce(P1, P2, dist):
    return (P1.mass - P2.mass)/dist

And then for all pairs of particles:

dist = distance(P1, P2)
if dist <= P1.radius + P2.radius:
    clump(P1, P2)
    break
F = calculateforce(P1, P2, dist)

That... is the worst solution that could ever be suggested. The
Particle objects already contain their own position, supplying dist
overrides their real distance of the particles and also it may make
bug holes by supplying bogus distance:
def calculateforce(P1, P2, dist):
return (P1.mass - P2.mass) / dist

calculateforce(P1, P2, 'bogus data')

or simply by passing dist that isn't equal (P1.X - P2.X) ** 2 + (P1.Y
- P2.Y) ** 2.

If you want to do it that way, it'd be much better if you go fully
functional:
def calculateforce(P1_mass, P2_mass, dist):
return (P1_mass * P2_mass) / dist ** 2

but it would be a pain in the neck to manually dismantle the P1.M,
P2.M, dist every time you call the calculateforce.

btw, on an unrelated note, the original formula I supplied for gravity
calculation is incorrect, it should be (P1.M * P2.M) / dist ** 2
instead of (P1.M - P2.M) / dist, but that's just physics not python.

But look at that line, "raise Collision". You could replace that with a
callback function, and have the same result:

    DO_NOTHING = lambda : None

    def distance(P1, P2, callback=DO_NOTHING):
        D = (P1.X - P2.X) ** 2 - (P1.Y - P2.Y) ** 2
        if D <= P1.radius + P2.radius:
            callback()
        return D

That gives you virtually everything you want. If you want to ignore the
signal, you simply call distance(P1, P2). If you want to do something on
the signal, you set the callback.

Guess where it goes after the callback finishes? It goes to "return
D". And if the calling code hasn't been notified that P1 and P2 has
already been clumped by the callback, it'll refer to an inexistent
object.
Soft Exception don't do that, it terminates the distance calculation
and goes to the handler and won't go back to finish distance
calculation unless explicitly told to (by using resume).
And another weakness, it'd be a pain in the neck to handle more than
two or three separate callbacks. You can define as many Soft Exception
as you want without cluttering the function's "interface" (please find
a better diction for "interface")

But frankly, the function distance() is NOT the place for that. Why
should the distance() function decide what's a special result and what
isn't? With your plan, you end up with functions like this:

(snip)
Because when a calculation of distance is lower than the sum of their
radius, it means the two objects shared the same space (!this is
impossible but not problematic for calculations if ignored!), that
would be an event of interest to codes that handles the movement,
clumping, etc but codes that just wants to print the distance to
screen wouldn't want to care about it.

Every module and function that calls distance() will start demanding that
it raises the SoftExceptions that *it* wants, and before you know it,
your distance() function has a hundred SoftExceptions covering all sorts
of things that most people don't care about.

No. This is a terrible idea. If the caller wants to treat a particular
result as special, the caller should be responsible for detecting that,
not the callee.

No the callee isn't responsible for detecting the exceptional result,
the caller is responsible for handling the exceptional result, while
the callee is responsible to notify the caller. In some cases it might
be impossible for the caller code to determine the exceptional cases
or probably the cost of doing so is nearly equal to the cost of doing
the whole calculations (but completing the calculation would result in
garbage out).

(snip)

Steven says:

Just out of curiosity, are there any existing languages that do something
like this, or did you invent it yourself?

Is this soft-exception implemented anywhere, so that one can see what
experiences and best practices have evolved around using it?

Actually I "invent" it myself (or probably I'm just ignorant enough
not to know any), that's why I expect there'll be a lot of weaknesses
I haven't foreseen (two heads are better than one, and a bunch of
heads are better than two). And it is also the reason why I'm asking
here for others' opinions, to bake the ideas, and possibly getting it
implemented when it's baked enough or to have other's opinion on why
it's bad and why it shouldn't exist.

<rant>If there is a reason why I chose here particularly, it'll be
because I think Python (AFAICS) have a lot of interesting features:
list comprehension, generator functions, for-loop semantic, just to
name a few of them (well, maybe it's not *that* unique to you, but
coming from a VB background[1], such features are extremely unique to
me) and I love them and use them heavily.</rant>

Another reason might be that Python is a dynamic language that have
adopted preference for try block compared to if block. And I think the
idea fits well enough with Python's ideologies.

<rant>Still another reason is because Python is not controlled by a
Impotent Company For Life but only by a Benevolent Dictator For Life.</
rant>

<rant>Yet another reason is simply because Python is currently my
active language.</rant>

[1] <rant mode="inherited">Although since I picked Python, I've yet to
touch VB again, I'm sure I'll still be Thinking In Python if I go back
to do some VB.</rant>

 

[Lie]

You are an appropriate person to consider the workflow in a dynamic
language, no matter how the language is implemented internally.

I agree, but the only thing I'm not confident to talk about is how
it'll be implemented, since I think an idea should suggest how it can
be implemented in practice to show that it's not just a nonsense paper
idea that's not possible in reality.

Just start with function calls

   maybe_raise(ZeroDivisionError)

The only requirement is that maybe_raise has to know when it shall
raise ZeroDivisionError. This depends on whether the exception is
caught. How do the program knows this in advance? There are no static
analysis techniques available.

Perhaps similar technique the compiler uses to determine whether a
function is a normal function or a generator function? Positive
forward lookup for any soft exceptions, which would then activate
matching soft exceptions inside the code?

 

[castironpi]

D'Aprano suggested callbacks. How does this work for you?

class SomeNumeric(object):
def __div__(a, b):
if b == 0: raise ZeroDivisionError ## Hard Exception...
if a == 0: msgboard- ZeroNumerator()
f = a / b
i = a // b
if f == float(i):
msgboard- IntegerDivision()
return a // b #? return i?
else:
msgboard- FloatDivision()
return a / b #? return f?


If it works, I can write some implementations of msgboard and discuss
its scope.

It sounded at one point like you were augmenting control flow, but you
cleared that up by saying, raise SoftException is not a true raise.
Correct?

There is a limit to how many behaviors you can fit in a single
statement in any language. What behaviors is determined by other
choices in the language design. Those choices have many consequences,
some linguistic, some social, and some personal. If a dictator always
makes impersonal decisions, which linguistic + social ones should he
make? If the answer is, "No, we will not strike note J and K because
we elected to strike notes L and M instead", then it's disappointing,
but it might ease that to know what L and M are.

Lastly, that notation I wrote was kind of sly. I might favor
msgboard( ZeroNumerator() ) or even msgboard.add( ZeroNumerator() ).

How far are you willing to come from the OP's proposal to get it to
work? If they add something half-way there, but you don't get exactly
what you want, is that fair?

 

[Diez B. Roggisch]

Perhaps similar technique the compiler uses to determine whether a

function is a normal function or a generator function? Positive
forward lookup for any soft exceptions, which would then activate
matching soft exceptions inside the code?

The difference between generators and functions is made on the
yield-keyword.

However, the exception-mechanism isn't governed by the compiler, but at
runtime. You can do things like this:


eclass = HardException if full_moon() else: SoftException

raise eclass()

Which means that you don't stand a chance determining
soft-exception-usage at compiletime.

What would work is most probably to register soft-exception-handlers
when encountering them at runtime, thus making raising-code aware of
them and execute it only if there are one (or several) present.

However, IMHO it's not a worthy extension to the language - for the same
reasons Steven gave. It seems only useful for tightly coupled code, not
as a general mechanism. And callbacks or maybe even thread-local state
are sufficient to deal with that I'd say.


Diez

 

[castironpi]

Perhaps similar technique the compiler uses to determine whether a
function is a normal function or a generator function? Positive
forward lookup for any soft exceptions, which would then activate
matching soft exceptions inside the code?

What would work is most probably to register soft-exception-handlers
when encountering them at runtime, thus making raising-code aware of
them and execute it only if there are one (or several) present.

[HO snip]

Mr. Roggisch, I just said that. You can unplonk me now.

 

[John Nagle]

Are the existing solutions really so incomplete that we need yet another
solution?

What problem are you trying to solve with SoftExceptions?

I actually implemented something like "soft exceptions" in a LISP
program long ago. I called them "gripes". They were a way that a function
could complain about something it wanted the caller to fix. The caller
could either fix it or decline to do so.

This was for a robotic planning application, where one module had detected
that some precondition that it needed wasn't satisfied. This was usually
something like some physical object being in the wrong place for a later
operation, and a previously planned move needed to be modified. Passing
the problem back to the caller sometimes allowed an easy fix, rather than
aborting the whole plan and building a new one.

This isn't a common problem.

In the rare cases that it is needed, it can be implemented with callbacks.
It doesn't require a language extension.

John Nagle