My Python annoyances

  • Thread starter =?iso-8859-1?B?QW5kcuk=?=
  • Start date
R

Ross Ridge

Thorsten Kampe said:
He was using /Windows/ Python in Cygwin *chuckle*... Windows Python
says Ctrl-Z because it doesn't know that it's been run from bash where
Ctrl-Z is for job control.

No, if you run Windows Python from Cygwin bash CTRL-Z works as the
EOF character:

~$ /cygdrive/e/util/python24/python
Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> quit
'Use Ctrl-Z plus Return to exit.'
>>> ^Z

~$ jobs
~$ python
Python 2.5 (r25:51908, Mar 13 2007, 08:13:14)
[GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> quit
Use quit() or Ctrl-D (i.e. EOF) to exit
>>>
[1]+ Stopped python
~$

Apparently though the Cygwin version of Python now prints the correct
message for quit.

Ross Ridge
 
B

Ben Collver

Paul said:
I'm sorry to hear about that. If by "macho" you mean people who insist
that things are good enough as they are, and that newcomers should
themselves adapt to whatever they may discover, instead of things
being improved so that they are more intuitive and reliable for
newcomers and experienced developers alike, then I'd certainly be
interested in undermining that culture.

That was the sort of response I got on freenode #python, which I realize
I should not take as representative of the whole community. Thank you
for the thoughtful response.
Can you provide the bug identifier? Bug reports are generally welcome,
and despite complaints about patch reviews, I've found people
reviewing things I've submitted.

It is problem report #1678102. I understand the problem: that a 32 bit
number looks different in a 32 bit signed int than in a 64 bit signed
int. However, the workaround of dropping a bit seems to defeat the
purpose of using a CRC.
Yes, Ctrl-Z exits Python in the standard Windows edition. Since Cygwin
provides a POSIX-like environment, Ctrl-D should be used instead. If
the documentation is wrong, a bug report or patch should be filed
against the software.

This morning I could not reproduce the problem. When the user types
"quit" at the Python prompt, the Cygwin port instructs the user to press
Control-D, which works. Even if you SSH in to Cygwin, and run the win32
port, it instructs the user to press Control-Z plus Return, which works.
Maybe it was fixed after I had the problem.
Try using isinstance or relying on "deeper" knowledge of how the
object will be used.

Thank you for the hint. I just want my function to validate that one of
its arguments is a file-like object. It isn't necessarily going to be a
temporary file, but it might be.
False

My opinions, already expressed, include the observation that the core
development community is more interested in extending the language
than in strengthening the standard library (and its documentation). It
should be noted that the proposed standard library reorganisation,
which is a very conservative affair, has actually been postponed until
after the release of Python 3.0a1 according to a message I read
recently. And yet, if you read people's lists about what they "hate"
about Python (amongst actual users of Python), guess which thing
almost always comes up?

I guess you cannot blame folks for working on what they find interesting.

Ben
 
C

Chris Mellon

That was the sort of response I got on freenode #python, which I realize
I should not take as representative of the whole community. Thank you
for the thoughtful response.

#python is one of the most accepting communities around. If the bug
reports here and the way you've presented them in this thread (vs the
way that they appear to an outside observer) are any indication,
though, I'm not surprised that you might have left in a huff.

Bear in mind that #python has no special status with regards to python
development and is primarily a community of *users*. If you go in with
some sort of thing you consider a problem, you are likely to be shown
a solution. Debate over whether it should be fixed in the core is
likely to be met with "patches accepted".
It is problem report #1678102. I understand the problem: that a 32 bit
number looks different in a 32 bit signed int than in a 64 bit signed
int. However, the workaround of dropping a bit seems to defeat the
purpose of using a CRC.

That's a valid point. Maybe you should have responded on the tracker
with that viewpoint. Your characterization of what happened in your
original post borders on dishonest - how can you possibly view what
happened there as "bug reports not welcomed"?
This morning I could not reproduce the problem. When the user types
"quit" at the Python prompt, the Cygwin port instructs the user to press
Control-D, which works. Even if you SSH in to Cygwin, and run the win32
port, it instructs the user to press Control-Z plus Return, which works.
Maybe it was fixed after I had the problem.

Huh.


Thank you for the hint. I just want my function to validate that one of
its arguments is a file-like object. It isn't necessarily going to be a
temporary file, but it might be.

False

Code like this is working directly against Python philosophy. You
probably got told this on #python, too. There's hardly any
circumstance where you should need to validate the exact class of an
object, and as long as they have the same interface theres no harm
whatsoever in tempfile changing it's return value between Python
versions.
 
B

Ben Collver

Terry said:
Three days after you posted, 'gagenellina' explained that he thought your
complaint was invalid.
"py> -531560245 & 0xffffffff
3763407051L

It's the same number (actually, the same bit pattern). ..."

A few weeks later, noticing that you had not challenged his explanation, I
closed after changing the Resolution box to Invalid. THAT WAS MY COMMENT.

A month later, I notice that you still have not directly challenged G's
claim of invalidity. Instead, you ignored it and simply repeated your
claim here. WHO IS IGNORING WHO?
...
Real bug reports are quite welcome, as any honest person could determine by
looking thru the tracker.

Hi Terry,

I understand and agree that the number was the same bit pattern. I
don't remember being asked to challenge this. I must have missed the
status change notification.

I do wonder whether the diagnosis is accurate: is the sparc64 port
actually using an unsigned int where the i386 port is using a signed int?

Either way, I don't see how it reflects on the validity of the report.
I reported that the resulting numbers were different. To me that seems
a trap for the unwary.

All I saw was a comment on what might cause my problem, and then I saw
that the problem report was closed. Now I am told that I didn't even
file a real bug report. I don't know whether to take that as "this is a
trivial problem not worth reporting" or "this is a poorly filed bug report".

I am an honest person, honestly!

Ben
 
B

Ben Collver

Thorsten said:
He was using /Windows/ Python in Cygwin *chuckle*... Windows Python
says Ctrl-Z because it doesn't know that it's been run from bash where
Ctrl-Z is for job control.

And the lesson we learn from that: if you're using Windows Python use
a Windows shell. If you're using a Cygwin shell use Cygwin Python -
unless you know what you're doing (which he wasn't).

The reason I tried to do this: Cygwin python lacks _winreg, but I wanted
to SSH into Cygwin to run this script.

I suppose the folks who know what they are doing probably stick to
wscript and WMI for this sort of stuff.

Ben
 
B

Ben Collver

Chris said:
#python is one of the most accepting communities around. If the bug
reports here and the way you've presented them in this thread (vs the
way that they appear to an outside observer) are any indication,
though, I'm not surprised that you might have left in a huff.

Bear in mind that #python has no special status with regards to python
development and is primarily a community of *users*. If you go in with
some sort of thing you consider a problem, you are likely to be shown
a solution. Debate over whether it should be fixed in the core is
likely to be met with "patches accepted".

I generally use IRC for idle chat and mulling over problems, and I
realize it would be the wrong place to ask for a change. At the time I
was talking about XML in the Python library. I was informed that I was
unwise to read 3rd party documentation for the Python library. I get
"Don't complain about documentation we didn't write" instead of "Yeah
it's broken, use pyxml instead."
That's a valid point. Maybe you should have responded on the tracker
with that viewpoint. Your characterization of what happened in your
original post borders on dishonest - how can you possibly view what
happened there as "bug reports not welcomed"?

I made a mistake when I first read the response: it does not drop any bits.

In the bug report itself, I saw a diagnosis of my problem's cause, and
then I saw the bug report closed as invalid. I did not know why the bug
was flagged invalid and closed, because I received no comment from the
person who closed it. I assumed that I should not have filed the bug
report.

Feedback in this newsgroup names my bug report as a "hobby horse", a
"wart", and "not a real bug report". I apologize for this noise over
such a small issue. It is clear now that real bug reports are welcome.
Code like this is working directly against Python philosophy. You
probably got told this on #python, too. There's hardly any
circumstance where you should need to validate the exact class of an
object, and as long as they have the same interface theres no harm
whatsoever in tempfile changing it's return value between Python
versions.

I am unqualified to comment on the Python philosophy, but I would like
for my function to do some basic error checking on its arguments. I
will read up on the Python philosophy.

Ben
 
B

Ben Collver

Ben said:
I am unqualified to comment on the Python philosophy, but I would like
for my function to do some basic error checking on its arguments.

By "basic error checking" I mean "verify that the file argument actually
is a file-like object". By same interface, do you mean that I should
check for the methods that I depend on? That sounds easy enough.

Thanks for the hint,

Ben
 
C

Chris Mellon

By "basic error checking" I mean "verify that the file argument actually
is a file-like object". By same interface, do you mean that I should
check for the methods that I depend on? That sounds easy enough.

You should "check" for the methods by calling them. If the object
doesn't support the method in question, you will get a runtime
exception. Premature inspection of an object is rarely useful and
often outright harmful.
 
A

Ant

Chris Mellon wrote: ....

I am unqualified to comment on the Python philosophy, but I would like
for my function to do some basic error checking on its arguments. I
will read up on the Python philosophy.

The basic point here is that the code will do it's own error checking.
If you pass in a string to your function, and it tries to call
write("xxx") on it, then you will get an exception thrown:

AttributeError: 'str' object has no attribute 'write

If your goal is to provide feedback to a potential user that they are
using the wrong arguments, then you can use something like the
following (the "Easier to ask for forgiveness than for permission"
idiom):
.... arg.write("")
.... except AttributeError:
.... print "You need to pass in a file like object!"
....
You need to pass in a file like object!
 
B

Ben Collver

Chris said:
You should "check" for the methods by calling them. If the object
doesn't support the method in question, you will get a runtime
exception. Premature inspection of an object is rarely useful and
often outright harmful.

That makes sense, thank you for the response.

What about the case where I have an array of objects that represent some
particular binary file format. If the object is a file, then I want to
copy its contents. If the object is a string, then I want to write the
string. And so forth.

Should I assume that an object is file-like if it has a read method, and
that I can call the read method without unexpected side effects?

Ben
 
A

Alex Martelli

Chris Mellon said:
You should "check" for the methods by calling them. If the object
doesn't support the method in question, you will get a runtime
exception. Premature inspection of an object is rarely useful and
often outright harmful.

However, hoisting method extraction out of the loop is often useful,
because it may speed up operations, and when you do that you might as
well use a try/except AttributeError to provide better diagnostics if
something is missing. E.g., suppose you have a need to slurp away one
char at a time from a file until you get a 'Z':

def readToZ(f):
while True:
c = f.read(1)
if c == 'Z': return
elif not c: raise ValueError, "No Z in file f"

this may also raise IOError if the read fails, AttributeError if f does
not have a read method, TypeError (or who knows what) if f.read is not
callable, or does not let you call it with one int argument, etc, etc.

A slightly faster approach:

def readToZ(f):
read = f.read
while True:
c = read(1)
if c == 'Z': return
elif not c: raise ValueError, "No Z in file f"

this has exactly the same exception-behavior as the previous version.
It's hardly an error to tweak that behavior slightly:

def readToZ(f):
try: read = f.read
except AttributeError: raise TypeError, "%s has no read" % type(f)
while True:
c = read(1)
if c == 'Z': return
elif not c: raise ValueError, "No Z in file f"

going to horrid amounts of trouble to check that the call to read(1)
won't produce weird failures would be unwarranted, but making one case
of AttributeError into a TypeError in this way is quite OK, for example.


Alex
 
A

Alex Martelli

Ben Collver said:
That makes sense, thank you for the response.

What about the case where I have an array of objects that represent some
particular binary file format. If the object is a file, then I want to
copy its contents. If the object is a string, then I want to write the
string. And so forth.

"Type-switching" in this way is a rather dubious practice in any
language (it can't respect the "open-closed" principle). Can't you have
those objects wrapped in suitable wrappers with a "copyorwrite" method
that knows what to do? For example, StringIO.StringIO is a standard
library wrapper type that makes a string look like a file.

It's a reasonable request you can make to whatever code is putting stuff
in that container: make the container "weakly homogeneous" by having it
stuffed only with "suitably file-like" objects. Dealing with totally
and weirdly heterogeneous containers is not a sensible task, because
there will be infinite types that COULD be in the container and about
which your code just can't be expected to guess right what to do.


Alex
 
B

Ben Collver

Alex said:
"Type-switching" in this way is a rather dubious practice in any
language (it can't respect the "open-closed" principle). Can't you have
those objects wrapped in suitable wrappers with a "copyorwrite" method
that knows what to do? For example, StringIO.StringIO is a standard
library wrapper type that makes a string look like a file.

That is very reasonable. Thanks again,

Ben
 
T

Thorsten Kampe

* Ben Collver (Fri, 04 May 2007 06:40:50 -0700)
The reason I tried to do this: Cygwin python lacks _winreg, but I wanted
to SSH into Cygwin to run this script.

I suppose the folks who know what they are doing probably stick to
wscript and WMI for this sort of stuff.

No, they stick to Python and WMI for this sort of stuff:
http://tgolden.sc.sabren.com/python/wmi.html
 
T

Terry Reedy

| Hi Terry,
|
| I understand and agree that the number was the same bit pattern.

OK

| I don't remember being asked to challenge this.

You don't need an invitation to disagree with another person's tracker
comment. I assumed you knew this and took non-response as acquiesence.
That (closing no response by item submitter) is a fairly typical pattern ,
by the way. I wish it were otherwise.

| I must have missed the status change notification.

The SF item-change emails indicate the changes, but you have to look.

| I do wonder whether the diagnosis is accurate: is the sparc64 port
| actually using an unsigned int where the i386 port is using a signed int?

I have no idea. I was essentially acting on GG's behalf since I know him
to be techically competent and stronly suspected (correctly, it turns out)
that could not put the close button himself.

| Either way, I don't see how it reflects on the validity of the report.

A bug is a discrepancy between promise and performance

| I reported that the resulting numbers were different.

I understand CRC checks to produce bit patterns rather than 'numbers'.

| To me that seems a trap for the unwary.

That is a different issue. If, for instance, you think the docs could and
should be improved to make people more wary, reopen the item, change the
appropriate field to 'documentation' and please give a suggested addition
or change.

| All I saw was a comment on what might cause my problem,

I understood (correctly, as it turns out) GG as saying 'invalid'.

| and then I saw that the problem report was closed.

Think of GG and I as a team acting as one. I had nothing to add and other
bug reports to attend to.

| Now I am told that I didn't even file a real bug report.

Invalid reports are more common than I wish. We try to be polite and
accurate in rejecting them. Once a seemingly competant judgement is made
in this direction, the ball is in the submitter's court, so to speak.

| don't know whether to take that as "this is a
| trivial problem not worth reporting" or "this is a poorly filed bug
report".

Your choice. See above.

| I am an honest person, honestly!

I can see that there was some non-understanding both ways, which is why I
have tried to clarify things.

Terry Jan Reedy
 
T

Terry Reedy

| In the bug report itself,

See my other response to you.

| Feedback in this newsgroup names my bug report as a "hobby horse",

That was not directed as you but the claim by someone else that I and other
reviewers are in the 'denial' stage of the 5 Stages of Grieving.

tjr
 
R

Ross Ridge

Ben Collver said:
It is problem report #1678102. I understand the problem: that a 32 bit
number looks different in a 32 bit signed int than in a 64 bit signed
int. However, the workaround of dropping a bit seems to defeat the
purpose of using a CRC.

The workaround doesn't drop any bits, it converts the value to a Python
long and extracts the lower 32 bits.

There's really no good reason for Python to give two different results
here. It should either return a signed 32-bit CRC value in a Python int,
or return a unsigned 32-bit CRC value in either Python long or a Python
int, if it's big enough. What it's doing now, returning unsigned value in
a Python int, even when it's not big enough to hold the result, is wrong.

Ross Ridge
 
B

Ben Collver

Terry said:
You don't need an invitation to disagree with another person's tracker
comment. I assumed you knew this and took non-response as acquiesence.
That (closing no response by item submitter) is a fairly typical pattern ,
by the way. I wish it were otherwise.

I (incorrectly) took the comment to support rather than invalidate my
report, and did not see anything to challenge. Email is not 100%
reliable, but I understand you don't have the time to hound submitters.
Do you think it might help to ask a question when you expect a
response from the submitter? It might act as a prompt.
That is a different issue. If, for instance, you think the docs could and
should be improved to make people more wary, reopen the item, change the
appropriate field to 'documentation' and please give a suggested addition
or change.

I trust the experts to take the appropriate action. It seems equally
reasonable to ignore the report for its triviality, or to treat the
checksum as a long, since that is what zlib returns.

Ben
 
S

Steven D'Aprano

"Type-switching" in this way is a rather dubious practice in any
language (it can't respect the "open-closed" principle).

What do people think about functions that accept either a file name or a
file object?


def handle_file(obj):
if type(obj) == str:
need_to_close = True
obj = file(obj, 'r')
else:
need_to_close = False
do_something_with(obj.read())
if need_to_close:
data.close()


Good idea? Bad idea? Just a matter of personal preference?
 
P

Paul Boddie

Steven said:
What do people think about functions that accept either a file name or a
file object?


def handle_file(obj):
if type(obj) == str:
need_to_close = True
obj = file(obj, 'r')
else:
need_to_close = False
do_something_with(obj.read())
if need_to_close:
data.close()


Good idea? Bad idea? Just a matter of personal preference?

I sometimes write functions like this myself. However, the matter of
testing for file-like objects can obviously vary somewhat in terms of
personal preference and correctness. Some would argue that this is a
situation which would benefit from interfaces:

if isinstance(obj, FileLike): # or obj.implements(FileLike), perhaps
do_something_with(obj.read())

In the original example, we can intuitively see that a file-like
object need only support the read and close methods, and in the case
of receiving a file-like object, only the read method need exist on
the object. Consequently, we can write the following:

if hasattr(obj, "read"):
do_something_with(obj.read())

Some would rightly say that this is ridiculous: you're testing
something which will be discovered straight away. However, there can
be situations where you might want to know in advance whether the
object is suitable, especially if you may perform more than one kind
of operation on the object and where side-effects may occur - the "let
the code fail" attitude arguably doesn't hold up very well in such
cases.

The problem can then be framed in terms of being able to express the
set of required operations and whether something like interfaces is a
flexible enough means of doing so. We might have something like this:

if test_for_file(obj):
do_something_with(obj) # not the string but the object itself

Now, we have the choice of explicitly phrasing the test ourselves...

def test_for_file(obj):
return hasattr(obj, "read") and hasattr(obj, "close") # and ...

....or relying on an interface mechanism to do this for us, with the
possible additional overhead of declaring such interface usage when
defining or adopting classes.

It seems to me that there's a gulf between the use of interfaces, with
the cost of introducing declarations in the code and the benefit of
relatively easy verification of object "capabilities", and the current
situation where one might like to try and deduce the required
capabilities of an object at any given point in the code. Without
interfaces, such verification is difficult but there's less overhead
for the programmer; with interfaces, verification is easier but the
programmer has to work harder to do most of the work. I can't really
see a good compromise.

Paul
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
474,157
Messages
2,570,879
Members
47,414
Latest member
djangoframe

Latest Threads

Top