while c = f.read(1)

[Greg McIntyre]

That is both clever and useful! I never would have thought of doing
that.

This seems to me like a general way to "workaround" the Python
statement/expression separation woes I've been having, in cases where I
really really want it.

Now, where can I copy this out to so I will be able to find it when the
occasion arises? Hmm... *jot jot jot*

Thanks muchly!

 

[Robert Kern]

The 2nd option has real potential for me. Although the total amount of
code is greater, it factors out some complexity away from the actual
job, so that code is not obscured by unnecessary compexity. IMHO that's
great practice.

Please quote the message you are replying to. We have no idea what "the
2nd option" is.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Steve Holden]

But '', {}, [] and () are not nothing. They are empty containers.

Oh come on, "empty" is all about nothing.

No it is not. There are situation where False or None should
be treated differently from an empty sequence.

Empty can mean, "nothing yet" which should be treated
differently from "nothomg more".

Sure, if your function's type is "None | int", then certainly
you must explicitly check for None.

The fact is that python doesn't know which type a function is.
So why does python guess that zero should be treated as false.

Python doesn't guess. There are a range of values that will be treated,
in a Boolean context (how perlish) as equivalent to False. If your
function is capable of validly returning any of these values then your
calls must be prepared to discriminate between (say) zero and False or
None. If not, the calling experession in the "if" can be simpler.

No is is not. The comparison with the specific false value
makes it easier for the reader of the code to find out what
to expect. I also find the use of '' as false in this context
wrong. A read can be used on all kind of objects including
a network connection. Returning '' on a network read would
be IMO the most natural answer to say, the network connection
is still open but no data is available for the moment. '' here
would mean "nothing yet" while '' is now made into "nothing more"

Yes, but you are restricting the programmer's range of expression if you
promote this as a general rule. Sometimes it's important to discriminate
between "", (), [] and None, sometimes there is no possiblity that
confusion will arise.

When there's no possibility of confusion you arae just picking nits
(which I know your sense of intellectual tidiness encourages you to do ;-).

regards
Steve

 

[Steve Holden]

While that is true, I had high hopes that I could do this:

while c = f.read(1): # ...

And relative to that, it is more complex. And although I am nit-picking
to try to simplify this code, I wanted to understand why Python works
in this way (even if that's just "historical reasons"), and check to
see if there was not some shorter more modern Pythonic alternative.

Guido explicitly decided he wanted a clear differentiation between
expressions and statements, and that assignment was to be a statement.
The logic behind this design decision is documented in


http://www.python.org/doc/faq/general.html#why-can-t-i-use-an-assignment-in-an-expression

which may have already been mentioned in this thread.

I did actually like Robert Kern's suggestion which used an iterator and
a function to factor out the complexity of setting it up. I think that
is actually better code than the original.

There are many elegant ways to express iterative solutions in Python,
and generator expressions are frequently useful in this context as well.

It matches my philosophy in programming of pushing complexity *out* of
the code which does the actual job, even if it means writing a few
support functions/classes/whatever. I know they can be re-used and
refined and I know that it is the code that does the actual job that is
most likely to be rewritten in future revisions of the code with shifts
in requirements.

You are likely to be happy with Python, then. Most Python programmers
are pragmatists.

I'm sorry! I realise that now and I'm sorry to have caused the traffic
I did. Thank you for pointing it out to me though - it's pretty
fundamental Python!

Yes, but it's a common question from newcomers (hence its appearance in
the FAQ), and there's usually some interesting discussion when it comes
up. Fortunately c.l.py isn't the kind of place you will usually
experience "Read the FAQ!" abuse. We try to remember that Python's
popularity is growing so fast that about a third of readers at any time
are likely to be newbies to Python (albeit many of them are experienced
in other programming languages).

*Greg thumbtacks a note to his forehead*
Ouch!



I would if only I had any kind of guarrantee on the file size but I
don't - this code is for reading a header out of a binary file which
uses delimiters and escape characters to mark out its fields. I didn't
design the format, but after cleaning up the code that deals with it, I
may *re*design it.

If you *know* that the header is MAX characters maximum you could try

for c in f.read(MAX):
...

Our company is designing a new file type. *sigh*. Confidentiality
prevents me from saying any more, too. If that bugs you because it's
not open source, sorry I need a job. Don't worry though, I'm developing
an open source remote GUI for the code management system we're using
called Aegis (http://aegis.sf.net). It's not sufficiently implemented
yet to warrant posting publically (I'd describe its current state as
"framework") but if anybody reading this is interested then give me a
yell and it'll have a public project page or something overnight.

Good luck.

regards
Steve

 

[Antoon Pardon]

But '', {}, [] and () are not nothing. They are empty containers.

Oh come on, "empty" is all about nothing.

No it is not. There are situation where False or None should
be treated differently from an empty sequence.

Empty can mean, "nothing yet" which should be treated
differently from "nothomg more".

And 0 is not nothing either it is a number. Suppose I have
a variable that is either None if I'm not registered and a
registration number if I am. In this case 0 should be treated
as any other number.

Such possibilities, make me shy away from just using 'nothing'
as false and writing out my conditionals more explicitly.

Sure, if your function's type is "None | int", then certainly
you must explicitly check for None.

The fact is that python doesn't know which type a function is.
So why does python guess that zero should be treated as false.

Python doesn't guess. There are a range of values that will be treated,
in a Boolean context (how perlish) as equivalent to False.

Yes it does. Python has no way to know what would be the most
usefull Boolean interpretation of these values in a particular
context. That it picks one out is guessing. Lisp imterprets
an empty list as false, scheme interprets it as true. So
both seem usable interpretations.

If your
function is capable of validly returning any of these values then your
calls must be prepared to discriminate between (say) zero and False or
None. If not, the calling experession in the "if" can be simpler.

But that doesn't seem to be accepted practice in this newsgroup.
Whenever someone shows code that does something like:

if var != []:
...

or

if var is True:

Someone else is almost bound to react that it is better to
write this as:

if var:


IMO it is the fact that python accepts almost anything as
true that may make it necessary to use "var is True"

No is is not. The comparison with the specific false value
makes it easier for the reader of the code to find out what
to expect. I also find the use of '' as false in this context
wrong. A read can be used on all kind of objects including
a network connection. Returning '' on a network read would
be IMO the most natural answer to say, the network connection
is still open but no data is available for the moment. '' here
would mean "nothing yet" while '' is now made into "nothing more"

Yes, but you are restricting the programmer's range of expression if you
promote this as a general rule. Sometimes it's important to discriminate
between "", (), [] and None, sometimes there is no possiblity that
confusion will arise.

When there's no possibility of confusion you arae just picking nits
(which I know your sense of intellectual tidiness encourages you to do ;-).

IMO there is more possibility of confusion then you think off.

Sure if you restrict your attention to one specific part of code, you
can conclude that there is no possibility of confusion whether '' is
used as false or not. But maybe sometime in the future you would want
that piece of code to work with other pieces of code where it is not
usefull to have '' interpreted as false in a Boolean context.
Now suddenly whether '' is false or not depends on where it is coming
from and your code with no possibilty of confusion is part of
a project where it is contributing to the confusion.

 

[Antoon Pardon]

Guido explicitly decided he wanted a clear differentiation between
expressions and statements, and that assignment was to be a statement.
The logic behind this design decision is documented in

http://www.python.org/doc/faq/general.html#why-can-t-i-use-an-assignment-in-an-expression

which may have already been mentioned in this thread.

I can't help but find this a very poor reason for this decision.
Sure it is a hard to find bug. but that is not because the
assignment is also an expression but because the assigment operator
looks so much like an equality comparator. ':=' was already in use
as an assignment is a number of languages and using it would
have handled the typo problem just as well while still allowing
assignments in expressions. This answer gives me the impression
the matter wasn't thought through thoroughly.

 

[Magnus Lycka]

Yes it does.

No it doesn't!

Python has no way to know what would be the most
usefull Boolean interpretation of these values in a particular
context.

It's hardly the task of the interpreter to try to do that.

That it picks one out is guessing.

No, it simply follows a well defined specification.
See http://docs.python.org/ref/Booleans.html
There is no guessing involved in that.

Lisp imterprets
an empty list as false, scheme interprets it as true. So
both seem usable interpretations.

You might argue that *Guido* was guessing when he decided what
the most useful behaviour of Python would be in this case, and
there's probably some truth in that, just as there is some
guesswork involved in most programming language design decisions,
but that's another thing. That's not Python guessing, it's
Guido using his excellent lanugage design skills. It seems most
Python programmers agree that he "guessed" right here, as usual.

(Perhaps you thought that "Python" was the name of the language
designer. It's not. Python's design is led by Guido van Rossum,
and the name Python comes from a (mostly) British comedy group.)

You might also argue that this behaviour is counter to the
Python dogma of "explicit is better than implicit". Python also
allows you to get a float out of an expression such as "2*3.1"
without forcing an explicit cast, as in "float(2)*3.1".

You should note that the first Python tenet is "Beautiful is
better than ugly" and that's probably what we have to blame here.

There seems to be close to a consensus, that "if users:" is more
beautiful than e.g. "if len(users) > 0:" or
"if (len(users)==0)==False" or for that matter
"if ((len(users)==0)==False)==True" or
"if (((len(users)==0)==False)==True)==True" etc.

What's true and false for Python, belongs to the few things you
actually have to learn, and I can appreciate that it's annoying
for a frequent schemer to remember that it's not the same in
Python, but it seems that very few people argue with the way
Python behaves in this respect.

 

[Mike Meyer]

My files are large, hence 1 character at a time, not f.read().

There are alternatives between the two. You could read in a chunk of
reasonable size for your platform. Say 10meg on a recent workstation,
or 100meg on a current workstation.

This is code from another employee and I'm just in the stages of going
through it and doing a basic clean-up before I get on to a proper
efficiency assessment, hence I don't want to change the way it works,
just make it as short and lucid as I can.

Well, the thing that has been hinted around but not explicitly stated
is that doing thing one character at a time in Python is a "code
smell", by which I mean it's an indication that there could be a
better way to do things, and it's probably worthwhile spending a
little time looking for it. On the other hand, if you've already
planned another pass over the code, that might be the time to look
into this.

<mike

 

[Donn Cave]

Before leaving this topic, I wanted to make a rare visit
to the ruins of Google's USENET archive and pull out this
giant post on the subject of True and False, when they were
being considered for adoption into Python. There is some
stuff to ignore, where she addresses questions that didn't
go anywhere, but she goes on to write a well articulated
case that makes very interesting reading, and possibly has
had some effect on how people think about it around here.

http://groups-beta.google.com/group/comp.lang.python/msg/2de5e1c8384c0360

Donn Cave, (e-mail address removed)

 

[Steve Holden]

You can get a good start by killfiling me.

*plonk*

Same goes to you. You don't like the "smart-questions.html" response, so
why do you respond? Because you want to change people's behavior. Or
perhaps because it pleases you to express your displeasure about it
(regardless of any effect that expression might have on other people).

Coincidentally, those are exactly the reasons why I posted it in the
first place. I care not a whit about decluttering the newgroup, an
impossible task.

It's clear that you care not a whit about it. Unfortunately the only way
to preserve bandwidth on this (or any other) chanell is for those with
nothing to say to not say it.

regards
Steve

 

[Robert Kern]

Robert Kern wrote:

It's clear that you care not a whit about it. Unfortunately the only way
to preserve bandwidth on this (or any other) chanell is for those with
nothing to say to not say it.

Folks, if you want to preserve bandwidth from being wasted on irrelevant
posts (like this one), delete your newsreader.

USENET *is* clutter. Sometimes useful clutter. Entertaining clutter.
*Shiny* clutter. But clutter. If you care about efficient use of
bandwidth, this is not the place for you.

James asked a question in such a way that I didn't think it would get
answered. Judging from the other non-responses to his post, I was right.
I showed him the way to ask questions such that they *will* get
answered, and he came back, did so, and got his questions answered.

That's worth 5 kb.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Greg McIntyre]

On the other hand, if you've already
planned another pass over the code, that might be the time to look
into this.

Exactly. And when I do that pass I will definitely try buffering the
data 10 or 100 meg at a time before entring the 1 char-at-a-time loop,
or using mmap to similar ends.

 

[John Hunter]

Robert> Please quote the message you are replying to. We have no
Robert> idea what "the 2nd option" is.

I think he means the second option you presented

If you must read one character at a time,

def reader(fileobj, blocksize=1):
"""Return an iterator that reads blocks of a given size from a
file object until EOF.
...snip

With a decent threaded news/mail reader, the thread provides
sufficient context, no?

JDH

 

[Robert Kern]

Robert> Please quote the message you are replying to. We have no
Robert> idea what "the 2nd option" is.

I think he means the second option you presented

If you must read one character at a time,

def reader(fileobj, blocksize=1):
"""Return an iterator that reads blocks of a given size from a
file object until EOF.
...snip

With a decent threaded news/mail reader, the thread provides
sufficient context, no?

Not taking into account the python-list gateway or GMane. I see his
message threaded directly under his original one.

And dammit, I'm vain enough that if people are complimenting my code, I
want to be sure about it. ;-)

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Antoon Pardon]

No it doesn't!


It's hardly the task of the interpreter to try to do that.

Indeed ir is not, so the interpreter should not pretend it can
by providing a Boolean interpretation.

No, it simply follows a well defined specification.
See http://docs.python.org/ref/Booleans.html
There is no guessing involved in that.

Following a well defined specification is not contradictory
to guessing. It may just mean that the guess was formalized
into the specification.

You might argue that *Guido* was guessing when he decided what
the most useful behaviour of Python would be in this case, and
there's probably some truth in that, just as there is some
guesswork involved in most programming language design decisions,
but that's another thing. That's not Python guessing, it's
Guido using his excellent lanugage design skills. It seems most
Python programmers agree that he "guessed" right here, as usual.

I don't see a big difference. Guessing plays its roles when
different people can have different expectations in cases
where not all the details are known/specified. Whether that
is in the language design or in program design doesn't make
a big difference and I would expect that a language that
discourages guessing in the software that its written in it,
would follow its own rules in its design.

(Perhaps you thought that "Python" was the name of the language
designer. It's not. Python's design is led by Guido van Rossum,
and the name Python comes from a (mostly) British comedy group.)

You might also argue that this behaviour is counter to the
Python dogma of "explicit is better than implicit". Python also
allows you to get a float out of an expression such as "2*3.1"
without forcing an explicit cast, as in "float(2)*3.1".

You should note that the first Python tenet is "Beautiful is
better than ugly" and that's probably what we have to blame here.

There seems to be close to a consensus, that "if users:" is more
beautiful than e.g. "if len(users) > 0:" or
"if (len(users)==0)==False" or for that matter
"if ((len(users)==0)==False)==True" or
"if (((len(users)==0)==False)==True)==True" etc.

Well then I must say people here give beauty too high a priority.
Because there seems a tendency to beautify others code when
snippets are posted here. Often enough such snippets
don't give enough inoformation to know whether "if var is True:"
can be replaced by "if var:" or whether other beautifications are
appropiate.

What's true and false for Python, belongs to the few things you
actually have to learn, and I can appreciate that it's annoying
for a frequent schemer to remember that it's not the same in
Python, but it seems that very few people argue with the way
Python behaves in this respect.

I care very little for what someone with experience in an other
language can expect. I care about the consistence of the language
and the programs written in it. I just mentioned scheme to show
that [] interpreted as false is not so obvious as a lot of people
seem to think. Since there are situations where using '', (), []
or 0 as false would be not appropiate I advise against using them
as such, because you never know when your software that does has
to cooperate with software that doesn't.

 

[Antoon Pardon]

Before leaving this topic, I wanted to make a rare visit
to the ruins of Google's USENET archive and pull out this
giant post on the subject of True and False, when they were
being considered for adoption into Python. There is some
stuff to ignore, where she addresses questions that didn't
go anywhere, but she goes on to write a well articulated
case that makes very interesting reading, and possibly has
had some effect on how people think about it around here.

http://groups-beta.google.com/group/comp.lang.python/msg/2de5e1c8384c0360

Well I guess she and I disagree here. I also don't understand some
of her arguments. e.g. about code like: "if var == True"

| Aaargh!
| I already see too much code like this. It's mostly written by people
| who come from other languages. They define their own True and False so
| they can do this.

I would argue that this would be bad code even in these other languages.
Heck when I was still a student and getting a course in PASCAL, people
with no programming experience whatsoever would write code like that
and PASCAL does have a real BOOLEAN type.

So where she gets the idea that "if var == True" is a symptom of a
language that has no real BOOLEAN type (as python now has IHO) is
beyond me.

 

[Magnus Lycka]

Following a well defined specification is not contradictory
to guessing. It may just mean that the guess was formalized
into the specification.

If you want the behaviour of Python to change, you should
write a PEP. It always felt natural to me to interpret empty
as false, but I could be wrong. It's strange that this flaw
managed to go unnoticed for so long though...

If Python is too "wild" for your taste, you might like OCaml.

Well then I must say people here give beauty too high a priority.
Because there seems a tendency to beautify others code when
snippets are posted here. Often enough such snippets
don't give enough inoformation to know whether "if var is True:"
can be replaced by "if var:" or whether other beautifications are
appropiate.

You might be right about that. I didn't really notice.

I think the typical comment is to replace "if s != '':" with
"if s:" in contexts where s is always a string, or to replace
"if expr != False:" with "if expr": in cases where "expr" is an
expression that returns True or False etc. In some cases, obvious
bugs, such as "if (a and b) == True:" where the programmer
should have written "if (a and b):" are pointed out.

Many of us have a solid mathematical education, and in that world
it's considered good behaviour to simplify expressions and remove
redundancy.

If we see things such as redundant pieces in logic expressions,
functions ending in "return None", returns directly after a raise,
or other meaningless constructs, it suggests that someone might be
doing something they don't understand, and then it's helpful to
try to point that out.

It's really important to understand what we do when we're
programming, not just repeat mantras or wave dead chickens.

If people mechanically learn "I shouldn't use '!= False' in
if-statements in Python", we've failed in trying to help them.
If people learn what Python considers true and false, what the
boolean operators return, and understands how to use these
things in an effective way, we've succeeded.

Another aspect of this particular idiom is that it's often better
to use the exception system in Python for exceptional situations,
than to use one variable to carry several different kinds of
information.

Concerning interfaces between different pieces of code, it's a
good approach to assume as little as possible, but to be clear
about what we assume.

 

[Antoon Pardon]

If you want the behaviour of Python to change, you should
write a PEP.

Such a PEP would have no chance of being accepted, since
it would break to much existing code.

It always felt natural to me to interpret empty
as false, but I could be wrong. It's strange that this flaw
managed to go unnoticed for so long though...

The problem with interpreting empty as false is that empty
just means no data now. But no data now can mean no data
yet or it can mean no more data. The problem is not so
much as having empty interpreted as false but that people
don't seem to think about which false value would be
more appropiate in particular circumstances. IMO reading
'' from network connection is the most natural result
when no data is ready and this should be treated differently
from an EOF which would indicate the connection was closed.

But how can you do this when somewhere else '' is used as
an indication for an EOF.

If Python is too "wild" for your taste, you might like OCaml.


You might be right about that. I didn't really notice.

I think the typical comment is to replace "if s != '':" with
"if s:" in contexts where s is always a string,

And it is IMO this kind of comments that lead to '' being used
as an EOF.

or to replace
"if expr != False:" with "if expr": in cases where "expr" is an
expression that returns True or False etc. In some cases, obvious
bugs, such as "if (a and b) == True:" where the programmer
should have written "if (a and b):" are pointed out.

This is not such an obvious bug. Because python allows non boolean
operands with "and" the two don't behave the same. How do you know
which behaviour the other person wants?

Many of us have a solid mathematical education, and in that world
it's considered good behaviour to simplify expressions and remove
redundancy.

I have yet to see a mathematical work where 0, or any kind of
empty sequence is treated as false. In mathematics accuracy
is considered vitaly important and won't be sacrified to
remove redundancy. Boolean expression are always written out
fully.

If we see things such as redundant pieces in logic expressions,
functions ending in "return None", returns directly after a raise,
or other meaningless constructs, it suggests that someone might be
doing something they don't understand, and then it's helpful to
try to point that out.

But you don't know if the logic expression are redundant. The
suggestions made are usually not equivallent.

It's really important to understand what we do when we're
programming, not just repeat mantras or wave dead chickens.

If people mechanically learn "I shouldn't use '!= False' in
if-statements in Python", we've failed in trying to help them.
If people learn what Python considers true and false, what the
boolean operators return, and understands how to use these
things in an effective way, we've succeeded.

Another aspect of this particular idiom is that it's often better
to use the exception system in Python for exceptional situations,
than to use one variable to carry several different kinds of
information.

In that case you wouldn't return an empty sequence if you wanted
a false value in a sequence context but would throw an exception.
So this would be fine by me, I just don't understand how this
would support the use of empty sequences as false.

I also don't see how a read from a network connection returning
either:

a bytestring when data is available,
'' when no data is available
None when the connection was closed

As so much different kinds of information.

Besides sometimes different kinds of information is not that
exceptional, so why should I throw an exception in such a
case?

 

[rafi]

In that case you wouldn't return an empty sequence if you wanted
a false value in a sequence context but would throw an exception.
So this would be fine by me, I just don't understand how this
would support the use of empty sequences as false.

I also don't see how a read from a network connection returning
either:

a bytestring when data is available,
'' when no data is available
None when the connection was closed

I do not get why returning '' when no data is available? If no data is
available then nothing is returned, the function hangs. (Which is the
cas for receive)

 

[Antoon Pardon]

I do not get why returning '' when no data is available? If no data is
available then nothing is returned, the function hangs. (Which is the
cas for receive)

Network connections can be configured to either block or return
immediatly when no data is available.