while c = f.read(1)

[Sybren Stuvel]

Antoon Pardon enlightened us with:

The problem with interpreting empty as false is that empty just
means no data now. But no data now can mean no data yet or it can
mean no more data. The problem is not so much as having empty
interpreted as false but that people don't seem to think about which
false value would be more appropiate in particular circumstances.

By no means can you take all possible future alterations in account.
You have to make assumptions somewhere.

IMO reading '' from network connection is the most natural result
when no data is ready and this should be treated differently from an
EOF which would indicate the connection was closed.

But how can you do this when somewhere else '' is used as an
indication for an EOF.

That's called "a protocol". If another protocol is used than the
software is written for, it'll break. This has nothing to do with
accepting something as False or True.

And it is IMO this kind of comments that lead to '' being used as an
EOF.

And who cares if it is? In a blocking environment, it seems pretty
okay to me. A read() call could block, waiting for more data to
arrive. If there is none because the connection is down, for instance,
it could return ''. Of course, raising an exception would be better in
such a case, and it would also remove any ambiguity.

I have yet to see a mathematical work where 0, or any kind of empty
sequence is treated as false.

In that case, properly define your variables to hold either booleans
or numbers, and only test booleans in an "if var:" clause, and only
test numbers in an "if var != 0:" clause.

a bytestring when data is available,
'' when no data is available
None when the connection was closed

Seems pretty nice to me. In such a case, one could do:

data = network.read()
if data:
handleData(data)

if data is None:
handleClosedConnection()

I don't see a problem here.

Sybren

 

[Antoon Pardon]

Antoon Pardon enlightened us with:

By no means can you take all possible future alterations in account.
You have to make assumptions somewhere.

But having to write out a boolean expression fully, would force
one to make his assumptions explicite. My impression is that
people don't actually make assumptions but are happy to just
throw a variable in a conditional context. They only begin
to think about the assumptions this imply when things break.

That's called "a protocol". If another protocol is used than the
software is written for, it'll break. This has nothing to do with
accepting something as False or True.

Yes it has. That empty sequences are treated as false in a conditional
context, influences what kind of protocols are used.

And who cares if it is? In a blocking environment, it seems pretty
okay to me.

Not all environments are blocking and it seems to be me it is better
to use protocols that are are as much as possible independant from
whether the environment is blocking or not.

A read() call could block, waiting for more data to
arrive. If there is none because the connection is down, for instance,
it could return ''. Of course, raising an exception would be better in
such a case, and it would also remove any ambiguity.


In that case, properly define your variables to hold either booleans
or numbers, and only test booleans in an "if var:" clause, and only
test numbers in an "if var != 0:" clause.


Seems pretty nice to me. In such a case, one could do:

data = network.read()
if data:
handleData(data)

if data is None:
handleClosedConnection()

I don't see a problem here.

That is because you are looking at it too much in isolation.
What if you have a function that among things handles bytes
coming from a stream. Whether that stream is a regular file
or a network connection may be irrelevant for the function,
but if regular files return '' for EOF and other streams
don't, you are in trouble. And IMO that '' is treated as
false played its parts in deciding for it as indicating EOF.

 

[Dennis Lee Bieber]

If you want the behaviour of Python to change, you should
write a PEP. It always felt natural to me to interpret empty
as false, but I could be wrong. It's strange that this flaw
managed to go unnoticed for so long though...

Seemed to fit my world view too... So many other languages used the
"non-zero" is True, that it made sense to take "non-empty" is True --
and if "non-empty" is true, then empty must be False.

Of course, there is always VMS -- where status codes were checked in
LSB... odd number => true, even number => false
--

 

[Steve Holden]

Seemed to fit my world view too... So many other languages used the
"non-zero" is True, that it made sense to take "non-empty" is True --
and if "non-empty" is true, then empty must be False.

Of course, there is always VMS -- where status codes were checked in
LSB... odd number => true, even number => false

.... and most *nix shells, where a return code of zero implies "success".
Of course, this still leaves wiggle room to discuss whether success is
True or False.

regards
Steve

 

[Bengt Richter]

I often end up doing something like this:

class foo:
def set(self, x):
self.x = x
return x
c = foo()

while c.set(f.read(1)):
# do stuff with c.x

In that file example, it's too much nuisance, but when you're
comparing some input against a series of regexps and you'd otherwise
need a multi-line construction for each one, this method comes in
quite handy.

if you use def __call__ instead of def set, you can also simplify the
subsequent spelling e.g,

while c(f.read(1)):
# do stuff with c.x

though something more menmonic than c might be desirable in that case.

Regards,
Bengt Richter

 

[John Machin]

I would if only I had any kind of guarrantee on the file size but I
don't - this code is for reading a header out of a binary file which
uses delimiters and escape characters to mark out its fields. I didn't
design the format, but after cleaning up the code that deals with it, I
may *re*design it.





Our company is designing a new file type. *sigh*.

Sigh indeed. If you need to read it a character at a time to parse it,
the design is f***ed.

Confidentiality
prevents me from saying any more, too. If that bugs you because it's
not open source, sorry I need a job.

Don't be so sensitive; it has strong Biblical precedent. You can believe
in the one true god but still bow down in the house of Rimmon or Redmond.

 

[Magnus Lycka]

Such a PEP would have no chance of being accepted, since
it would break to much existing code.

What's the point of this thread then?

But how can you do this when somewhere else '' is used as
an indication for an EOF.

If that's your problem, I guess that's what you should attack,
not that Python considers nothing to be nothing even it might
some time become something.

Network programming with Python works pretty well though, so
it seems this is a non-issue too.

And it is IMO this kind of comments that lead to '' being used
as an EOF.

Huh? Aren't the Python APIs just mirroring the behaviour of the
underlying C APIs?

This is not such an obvious bug. Because python allows non boolean
operands with "and" the two don't behave the same. How do you know
which behaviour the other person wants?

I think you misread my text. If the programmer should have written
"if (a and b):", adding "==True" will cause different behaviour
unless True (or 1) is the only non-False value that b can have.
This would not be obvious for someone who expects that the results
of logical operations will return boolean values.

I have yet to see a mathematical work where 0, or any kind of
empty sequence is treated as false. In mathematics accuracy
is considered vitaly important and won't be sacrified to
remove redundancy. Boolean expression are always written out
fully.

Dear Antoon. The "if" statement interprets the result of an
expression to determine whether or not to execute a block of
code. If "x" is an expression that returns True or False, then
"x==True" is an equivalent expression. It's just not written in
its minimal form.

It's like writing "a / b * 100 %" instead of just "a / b" in a
mathematical equation. The first version contains some kind of
noise that just looks ugly for people who know that 100%==1.
Why multiply with 1? At least in my home town, the MBA students
write stuff like that, but mathematicians and engineers would
just think it was ugly.

But you don't know if the logic expression are redundant. The
suggestions made are usually not equivallent.

I think I know. Please point out if I made some mistake.

It's pretty common that people fail to reduce logical expressions.
I've seen C++ code checking for overlapping periods looking roughly
like this:

if ((start1<=start2 and stop1>=start2 and stop1<=stop2) or
(start1<=start2 and stop1>=stop2) or
(start1>=start2 and stop1<=stop2) or
(start1>=start2 and start1<=stop2 and stop1>stop2))

For that person, his code might actually have been clearer than
the less cluttered version I changed it to:

if (start1<=stop2 and start2<=stop1)

At least he spent a few minutes staring at it before he could
accept that they were equivalent. (Maybe he just gave up.)

Of course, it might well be that different brains work in different
ways, and that different presentations are preferred by different
people. Python seems to fit my brain.

In the overlapping case above it's really a shift of perspective.
He saw overlapping time periods as occuring in four different
cases:
* A starts before B and ends during B
* A starts before B and ends after B
* B starts before A and ends during A
* B starts before A and ends after A.

I guess my first impulse (before I asked what his code did) was
that it looked redundant, but the thing is that my short version
describes overlapping time periods in a different way: Both
periods start before the other period ends. That's all!

The kinds of simplifications in code that we talk about often has
this quality. They show us that things are really simpler than the
coder thought. I'm sure it happens that some people try to over-
simplify things, but most systems can be made simpler, and trying
to do that is a noble cause.

In that case you wouldn't return an empty sequence if you wanted
a false value in a sequence context but would throw an exception.

Huh? If I want False I use False. If a sequence is empty, it's empty.
It seems to me that you make things more complicated than they have
to be. Perhaps you're just confused over the C APIs for network
programming. I can understand that, but you seem to be pretty far
away from the real target now.

So this would be fine by me, I just don't understand how this
would support the use of empty sequences as false.

I also don't see how a read from a network connection returning
either:

a bytestring when data is available,
'' when no data is available
None when the connection was closed

As so much different kinds of information.

Besides sometimes different kinds of information is not that
exceptional, so why should I throw an exception in such a
case?

I don't know what you are trying to say. If you want more
Pythonic network API, I'd rather use an iterator. Just make
it return StopIterator when the connection closes. You would
use it like this:

for chunk in NetworkReader(ip, port):
if chunk: print chunk

This could be blocking, or use a timer and return '' on
timeout. Thus the if-statement. chunk would always be a
string. As you suggested, an empty string would then mean
no data yet. When the connection is closed, the for loop
simply ends.

In this context, writing "if chunk != '':" instead of simply
"if chunk:" is just a waste of resources.

 

[Antoon Pardon]

What's the point of this thread then?

I only hope to impress people that the way python
treats 'nothing' in a condional context is not
so obvious as seems to be accepted here.

If that's your problem, I guess that's what you should attack,
not that Python considers nothing to be nothing even it might
some time become something.

IMO the two are linked. People use '' as an EOF because the
call they are working with returns strings and they need a
way to end a loop. Since "if var:" is considered beautifull
they search for a nothing value and since they were working
with strings, '' gets chosen.

Network programming with Python works pretty well though, so
it seems this is a non-issue too.


Huh? Aren't the Python APIs just mirroring the behaviour of the
underlying C APIs?

That may depend on how strict the meaning of mirroring is, you
are using. I would say no. Even if the answer is yes I would
say that chosing such values on that basis was a bad design
choice.

I think you misread my text. If the programmer should have written
"if (a and b):", adding "==True" will cause different behaviour
unless True (or 1) is the only non-False value that b can have.
This would not be obvious for someone who expects that the results
of logical operations will return boolean values.

So? How do you know what the writer of the code expects. You
originaly wrote it was an obvious bug, how do you come to
that conclusion.

Dear Antoon. The "if" statement interprets the result of an
expression to determine whether or not to execute a block of
code. If "x" is an expression that returns True or False, then
"x==True" is an equivalent expression. It's just not written in
its minimal form.

But we were talking about interpreting 0, '', (), [], and {}
directly in a conditional context. No mathematical text
will just contain a line like

a => b > 10

when what is meant is:

a != 0 => b > 10

It's like writing "a / b * 100 %" instead of just "a / b" in a
mathematical equation. The first version contains some kind of
noise that just looks ugly for people who know that 100%==1.
Why multiply with 1? At least in my home town, the MBA students
write stuff like that, but mathematicians and engineers would
just think it was ugly.

But you can't transfer this situation to python, because python
allows non Boolean values to be interpreted in a conditional
context. I have code somewhere that looks like this:

if var is True:

and that is exactly how it should be. The if branch should not
be taken if var would be 5. I even have code that looks like:

if var is not False:

And although in a logical context those two would be equivallent
to each other and to just "if var:", they are not equivallent
in a python context.

Yet I see a lot of suggestions here to change logical expressions
in python code, seemingly based on the fact that they would
be equivallent in a logical context.

I think I know. Please point out if I made some mistake.

It's pretty common that people fail to reduce logical expressions.
I've seen C++ code checking for overlapping periods looking roughly
like this:

if ((start1<=start2 and stop1>=start2 and stop1<=stop2) or
(start1<=start2 and stop1>=stop2) or
(start1>=start2 and stop1<=stop2) or
(start1>=start2 and start1<=stop2 and stop1>stop2))

For that person, his code might actually have been clearer than
the less cluttered version I changed it to:

if (start1<=stop2 and start2<=stop1)

At least he spent a few minutes staring at it before he could
accept that they were equivalent. (Maybe he just gave up.)

I think he did, because both expression are not equivallent
unless some implicite constraints make them so. Values where
both expressions differ are:

start1=67, stop1=9, start2=10, stop2=29

Huh? If I want False I use False. If a sequence is empty, it's empty.
It seems to me that you make things more complicated than they have
to be.

You can't just decide to use False if you want False. What if
you depend on a library that returns '', because '' behaves
as false in a conditional context. So returning it was good
enough for those who wrote it.

 

[Magnus Lycka]

I think he did, because both expression are not equivallent
unless some implicite constraints make them so. Values where
both expressions differ are:

start1=67, stop1=9, start2=10, stop2=29

Ouch! That didn't occur to me. How sloppy to just assume that
time periods can't end before they start. I'll shut up now.
You win, I'm obviously the idiot here, and Python's must be
redesigned from ground up. Pyrdon maybe?

 

[Greg McIntyre]

Sigh indeed. If you need to read it a character at a time to parse it,
the design is f***ed.

There is always the potential to do 2k buffered reads and once in
memory pick the contents apart character-wise.

I assume something similar would happen for tokenising XML and HTML
which would presumably often 'read until "<"'.

 

[Greg McIntyre]

Not taking into account the python-list gateway or GMane. I see his
message threaded directly under his original one.

And dammit, I'm vain enough that if people are complimenting my code, I
want to be sure about it. ;-)

Sorry Robert, I'm using Google Groups until I figure out the news
settings for our ISP at work (which is most unhelpful). I'm not used to
using it and the default 'Reply' option doesn't quote. :\ Not a good
excuse, I know.

Let's see... to summarise the responses I got, I liked yours the best,
Robert. It was:

def reader(fileobj, blocksize=1):
"""Return an iterator that reads blocks of a given size from a
file object until EOF.
"""
# Note that iter() can take a function to call repeatedly until
it
# receives a given sentinel value, here ''.
return iter(lambda: fileobj.read(blocksize), '')


f = open('blah.txt', 'r')
try:
for c in reader(f):
# ...
finally:
f.close()

I like it because I can make 'reader' a stock library function I can
potentially re-use and it removes complexity from the area where I want
to place the domain-specific logic (where I call reader()), which I
have a personal preference for.

Potentially the added comlexity of buffering larger chunks at a time
for efficiency could also be put into the reader() function to keep the
rest of the code super clean and neat.

 

[Steve Holden]

Sorry Robert, I'm using Google Groups until I figure out the news
settings for our ISP at work (which is most unhelpful). I'm not used to
using it and the default 'Reply' option doesn't quote. :\ Not a good
excuse, I know.

[...]

Well you could do worse than use the gmane.comp.python.general newsgroup
if you want to use an NNTP newsreader. I recently left the ISP who had
provided me with news services for years, and I am very happy with the
gmane service (though heaven only knows why they chose to use a name
other than comp.lang.python for the group: perhaps they were only aware
of the list it gateways when they established the service).

Anyway, ther price is certainly right.

regards
Steve

 

[Robert Kern]

Well you could do worse than use the gmane.comp.python.general newsgroup
if you want to use an NNTP newsreader. I recently left the ISP who had
provided me with news services for years, and I am very happy with the
gmane service (though heaven only knows why they chose to use a name
other than comp.lang.python for the group: perhaps they were only aware
of the list it gateways when they established the service).

Their service is *only* about gatewaying mailing lists to their NNTP
server. I'm certain they were aware of comp.lang.python and how it is
gatewayed to python-list, but they aren't a general USENET node and
don't get USENET feeds from other nodes. gmane.comp.python.general
articles come only from python-list. I imagine that it would be impolite
to name any of their newsgroups using one of the controlled Big Eight
hierarchies.

But I agree that it is quite nice and is what I use to access
c.l.py/python-list so that I can use the same server both at home and at
work.

http://gmane.org

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Antoon Pardon]

Ouch! That didn't occur to me. How sloppy to just assume that
time periods can't end before they start.

I have no trouble that you assume a time period starts before
it ends.

But two pieces of code that only give the same result under
particular assumptions are not equivallent. For all I know
his code might work without this assumption and thus be
usefull in circumstances where yours is not.

Maybe someone uses a convention where time intervals that
stop before they start can have some meaning.

Equivallent code IMO always gives the same results, not
only under the particular constraints you are working with.

I'll shut up now. You win,
I'm obviously the idiot here, and Python's must be
redesigned from ground up. Pyrdon maybe?

If I ever design a language it'll be called: 'Queny'

 

[Steve Holden]

This is just too fatuous to ignore, sorry.

I have no trouble that you assume a time period starts before
it ends.

But two pieces of code that only give the same result under
particular assumptions are not equivallent. For all I know
his code might work without this assumption and thus be
usefull in circumstances where yours is not.

Maybe someone uses a convention where time intervals that
stop before they start can have some meaning.

Equivallent code IMO always gives the same results, not
only under the particular constraints you are working with.




If I ever design a language it'll be called: 'Queny'

....and you will regard it as perfect and be completely unable to
understand why nobody likes it.

Could we possibly reduce the number of arguments about ridiculous
postulates such as , and try to remember that most people on this list
are dealing with real life?

Magnus gave you a perfectly reasonable example of some code that could
be simplified. You say the two pieces of code aren't equivalent. While
you may be (strictly) correct, your assertion signally fails to add
enlightenment to the discussion.

I continue to look forward to the first post in which you actually
accept someone else's point of view without wriggling and squirming to
justify your increasingly tenuous attempts to justify every opinion
you've ever uttered on this group

regards
Steve

 

[Antoon Pardon]

This is just too fatuous to ignore, sorry.

You mean you can't think of circumstances where this
could be valid data.

...and you will regard it as perfect

I doubt that. I've looked at the problem of designing a language
and IMO it is a very complex matter, that is difficult to do
good, let alone perfect. At this moment I think very highly
about the designers of Python, because I think they have
done a very good job and Python is for the moment my language
of choice. Sure Python has its warts, but I can live with
them. I just don't like it if I get the impression that
people want to deny the warts.

and be completely unable to
understand why nobody likes it.

Could we possibly reduce the number of arguments about ridiculous
postulates such as , and try to remember that most people on this list
are dealing with real life?

So? Real life is full of thinss that could be better. If people just
want deal with that, fine. But arguing that there is nothing wrong
with how python treats conditional context is not dealing with
real life. AFAIAC writing articles in newsgroups isn't dealing
with real life, unless maybe if you have a question you need an
answer for. I deal plenty with real life myself, during working
hours that consists partly in writing python software, with all
the good and the few bad python brings.

Magnus gave you a perfectly reasonable example of some code that could
be simplified. You say the two pieces of code aren't equivalent. While
you may be (strictly) correct, your assertion signally fails to add
enlightenment to the discussion.

No, my assertion pointed out, that his code would only work under
specific constraints. Constraints that may have been very natural
in the context the program was written, but that doesn't mean
all programs work with such a constraint. He was also talking
about logical equivallence and his expression was not logical
equivallent with the one that was replaced.

I continue to look forward to the first post in which you actually
accept someone else's point of view without wriggling and squirming to
justify your increasingly tenuous attempts to justify every opinion
you've ever uttered on this group

I don't post me too's. There have been plenty of posts here, I have no
problem with. I read them, give a slight nod and go on. And if I have
an opinion I was to express here, I certainly will to justify it.