while c = f.read(1)

[James Kim]

http://www.catb.org/~esr/faqs/smart-questions.html

Is it a *smart* way or *necessary* way?

Plus, my question was not for the detail description but for the
intuitive guide leading the beginner's further study.

I understand that too many repeated talks make cyberian tired. However,
over and over discussions of basic concepts is also very important for
technology enhancements. Thus, Commands 'iter' and 'lambda' should be
discussed over and over about their necessity and convenience in the
news-group as long as they are the principle keywords distinguished from
the conventional languages like c/c++, pascal, etc.

-James

 

[Jeremy Jones]

Robert Kern wrote:



Is it a *smart* way or *necessary* way?

Of course it's not *necessary*. I mean, the world isn't going to come
to an end if it doesn't happen. There is no logical contingency making
it so. But, if everyone in the group adheres to the ESR "smart
questions" guide, what's the difference?

Plus, my question was not for the detail description but for the
intuitive guide leading the beginner's further study.

But, I'll try to answer your question the best I can. From a
quasi-sensory intuitive level, ``iter`` is red - kinda warm - and smells
a little like cinnamon, but not too strong. ``lambda`` on the other
hand is blue-green, sometimes grey, cooler, almost cold, has a damp feel
to it, and tastes like pork - not chicken, mind you - that's the ``for``
statement.

I understand that too many repeated talks make cyberian tired. However,
over and over discussions of basic concepts is also very important for
technology enhancements.

Here's the deal. If you have a general question about something, ask
it. But ask smartly. For example, "What is the benefit of using
``iter`` as opposed to something else? What are the alternatives to
using ``iter``?" Asking questions like "What are the meanings of
Commands 'iter' and 'lambda'" will not fly well here - and you may find
less so elsewhere. The reason is, it smells of laziness (I'm not saying
you *are* lazy - that's just the impression it leaves) and this group is
full of people who have reached for the docs, wrestled with them, and
have come away from it better informed programmers.

Thus, Commands 'iter' and 'lambda' should be
discussed over and over about their necessity and convenience

This is different from what you were asking. I quoted your exact words
above and it's different from what you're asking here. And I'm not so
sure I would put a *should* on your statement. I think usage
discussions of different functions, standard library modules, practices,
etc. *will* arise perpetually. But I don't think we *need* to
constantly bat around the necessity of X keyword or Y function or Z
module. Convenience - probably. Necessity - no.

in the
news-group as long as they are the principle keywords distinguished from
the conventional languages like c/c++, pascal, etc.

-James

So, if you have a question that's in line with Robert's advice, please
post it and it will have a much higher chance of getting answered. I
sincerely hope this helps.


Jeremy Jones

 

[Robert Kern]

Is it a *smart* way or *necessary* way?

It's the polite way. And probably the only way you're going to get your
questions actually answered.

Read the documentation. If you still don't understand something, come
back and ask a specific question about what you didn't understand. Then
we will respond to your politeness with our own polite answer.

Until then, your rudeness gets my rudeness in response.

Plus, my question was not for the detail description but for the
intuitive guide leading the beginner's further study.

I understand that too many repeated talks make cyberian tired. However,
over and over discussions of basic concepts is also very important for
technology enhancements. Thus, Commands 'iter' and 'lambda' should be
discussed over and over about their necessity and convenience in the
news-group as long as they are the principle keywords distinguished from
the conventional languages like c/c++, pascal, etc.

Then use Google Groups to see that they have been, in great detail, and
at a higher level than giving a newbie an "intuitive" rundown of the
features.

Now go read the documentation.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Paul Rubin]

It's the polite way. And probably the only way you're going to get
your questions actually answered.

I wonder if there's a way to killfile posts that contain that url.
They are cluttering up the newsgroup much more than repeated newbie
questions do.

Sheesh people, if you see a question that you don't feel like
answering, then don't answer.

 

[Grant Edwards]

What are the meanings of Commands 'iter' and 'lambda', respectively? I
do not want you to indicate merely the related help pages.

Rude much?

If somebody is kind enough to point out the documentation that
answers your question you ought to read it.

Just your ituitive and short explanations would be enough
since I'm really newbie to Python.

What makes you think my "intuitive and short" explanation is
goign to be better than what's aready written? Why should I
take the time?

 

[Robert Kern]

I wonder if there's a way to killfile posts that contain that url.
They are cluttering up the newsgroup much more than repeated newbie
questions do.

You can get a good start by killfiling me.

Sheesh people, if you see a question that you don't feel like
answering, then don't answer.

Same goes to you. You don't like the "smart-questions.html" response, so
why do you respond? Because you want to change people's behavior. Or
perhaps because it pleases you to express your displeasure about it
(regardless of any effect that expression might have on other people).

Coincidentally, those are exactly the reasons why I posted it in the
first place. I care not a whit about decluttering the newgroup, an
impossible task.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Bengt Richter]

What are the meanings of Commands 'iter' and 'lambda', respectively? I
do not want you to indicate merely the related help pages. Just your
ituitive and short explanations would be enough since I'm really newbie
to Python.

Would you please read the above, as if you were someone else?
The above sounds to me like you think you're typing commands
to an enhanced google, or worse, to an abused employee or servant.

To say "I do not want you to ..." sounds so arrogantly presumptious
that I'm sure if that "works" for you IRL, you must be living in a very
pampered and sheltered environment.

The downside of such environments is that if simple demanding "works"
and you get things and services from people as if they were robots,
you lose out on the thing that draws most of us, I think, to c.l.p
and similar social spaces -- which is a feeling of shared consciousness
and contact with other humans within a sphere common interests.

You're welcome to try again ;-)

(BTW, sorry if "James" doesn't mean English is your mother tongue, and I have
misinterpreted your manner of expression. If so, this may still be a useful
indication to you of how wording can determine response ;-)

Regards,
Bengt Richter

 

[James Sungjin Kim]

Robert Kern 쓴 글:

Now go read the documentation.

Thanks to your comments, I read the corresponding helps searched by
Google. (Sorry to say a specific search engine here, but I must say that
it is really convinient.)

Now I realized that Command 'lambda' is a similar to Command 'inline' in
C++. In addition, Command 'iter' is something new but not much new to c
engineers, since it is related to 'for loops', e.g.,

I would want to express my appreciation to Paul, Robert, Jeremy, Grant,
and Bengt, who provide good guidelines to get the answers in newsgroups.

-James

 

[Paul Rubin]

Now I realized that Command 'lambda' is a similar to Command 'inline'
in C++. In addition, Command 'iter' is something new but not much new
to c engineers, since it is related to 'for loops', e.g.,

Actually not related at all. Nothing like lambda or iter exist in
C++. They are a bit complicated to explain to newbies (unless you've
programmed in Lisp or in functional-programming languages), thus the
suggestion of looking in the docs. That's also the reason that
suggesting using lambda and iter that way was considered a bad answer
to "how do you write something like 'while c = f.read(1)'".

Anyway, quick explanation of lambda: it's a way of creating functions
without having to give then names. Example:

lambda x: x*x

is a function that computes the square of x. So you could say

n = (lambda x: x*x)(3)

which sets n = 9.

There is a faction on clpy which says lambda is bad and all functions
should have their own names. Others say they're useful for things
like callbacks, and requiring all functions to be named makes no more
sense than requiring all intermediate results in infix expressions to
be named.

You unfortunately happened to trip into a slightly contentious topic
on clpy and got flamed a bit as a result.

I won't explain iter here, since it's more complex than lambda.

 

[Reinhold Birkenfeld]

... AND it's about time that list is updated to include False explicitly
-- save nitpicking arguments about whether False is covered by
"numeric zero of all types"
Done.


How about
for c in f.read():
?
Note that this reads the whole file into memory (changing \r\n to \n on
Windows) ... performance-wise for large files you've spent some memory
but clawed back the rather large CPU time spent doing f.read(1) once per
character. The "more nicely" factor improves outasight, IMHO.

Mild curiosity: what are you doing processing one character at a time
that can't be done with a built-in function, a standard module, or a
3rd-party module?

Don't forget

for line in f:
for c in line:
# do stuff

Reinhold

 

[Paul Rubin]

Don't forget

for line in f:
for c in line:
# do stuff

As mentioned before, that's careless programming, since it can read
the whole file into memory if the file contains no newlines.

 

[John Machin]

John Machin wrote:




Done.

Thanks -- those of us who actually read manuals salute you.

w about

for c in f.read(): [snip]

Don't forget

for line in f:
for c in line:
# do stuff

OK. I'll pay that one.

 

[Steve Holden]

As mentioned before, that's careless programming, since it can read
the whole file into memory if the file contains no newlines.

I agree that there may be circumstances where this naiive approach is
sub-optimal. However, calling this careless programming is a bit over
the top. Reading files into memory is often the preferred approach
nowadays, and on a modern laptop (say) files several megabytes in size
will be processed perfectly happily.

Efficiency is good, but remember Kernighan and Plauger (elements of
Programming Style): First, make it work. Then (if it doesn't work fast
enough) make it work faster.

Most times the simple solution works for the problem under
consideration, and useless optimization work is avoided.

regards
Steve

 

[Antoon Pardon]

But '', {}, [] and () are not nothing. They are empty containers.

Oh come on, "empty" is all about nothing.

No it is not. There are situation where False or None should
be treated differently from an empty sequence.

Empty can mean, "nothing yet" which should be treated
differently from "nothomg more".

Sure, if your function's type is "None | int", then certainly
you must explicitly check for None.

The fact is that python doesn't know which type a function is.
So why does python guess that zero should be treated as false.

That is not the case with
fileobject read(), nor with many functions in Python that
reasonably and ideally return a value of a type that may
meaningfully test false. In this case, comparison (==) with
the false value ('') is silly.

No is is not. The comparison with the specific false value
makes it easier for the reader of the code to find out what
to expect. I also find the use of '' as false in this context
wrong. A read can be used on all kind of objects including
a network connection. Returning '' on a network read would
be IMO the most natural answer to say, the network connection
is still open but no data is available for the moment. '' here
would mean "nothing yet" while '' is now made into "nothing more"

 

[Greg McIntyre]

Okay, the 1st option seems more complicated and the 2nd option, while
simpler to my eye makes me worry about file descriptors, resource
management and memory running out.

My files are large, hence 1 character at a time, not f.read().

This is code from another employee and I'm just in the stages of going
through it and doing a basic clean-up before I get on to a proper
efficiency assessment, hence I don't want to change the way it works,
just make it as short and lucid as I can.

Your suggestion using the generator expression again seems more complex
than the original.

They're good suggestions though. Thank you! I'm just now catching up
with everybody's comments. Some of them seem quite entriguing.

 

[Greg McIntyre]

The 2nd option has real potential for me. Although the total amount of
code is greater, it factors out some complexity away from the actual
job, so that code is not obscured by unnecessary compexity. IMHO that's
great practice.

I like it! Thank you! The assurance that the code was clear was good
too - that is also what I need to hear (decisively dismissing the quest
for something shorter and punchier will make me happy as much as
succeeding in it).

I'll read all the other suggestions first though.

 

[Greg McIntyre]

Actually I'd make it a little less compact -- put the "break"
on its own line -- but in any case this is fine. It's a natural
and ordinary way to express this in Python.

...
| But I get a syntax error.
|
| while c = f.read(1):
| ^
| SyntaxError: invalid syntax
|
| And read() doesn't work that way anyway because it returns '' on EOF
| and '' != False. If I try:

This is the part I really wanted to respond to. Python managed
without a False for years (and of course without a True), and if
the introduction of this superfluous boolean type really has led
to much of this kind of confusion, then it was a bad idea for sure.

Sorry that was patently untrue - as far as the while loop is concerned
'' and False both cause the loop to exit so for all intents and
purposes, in that context, '' may as well equal False.

I think True and False constants are a good idea. And I like the way it
worked. The reason I got confused though was because I cannot do this:

while c = f.read(1):
# ...

The condition that we're looking at here, and this is often the
way to look at conditional expressions in Python, is basically
something vs. nothing. In this and most IO reads, the return
value will be something, until at end of file it's nothing.
Any type of nothing -- '', {}, [], 0, None - will test "false",
and everything else is "true". Of course True is true too, and
False is false, but as far as I know they're never really needed.

I'm all in favour of logic that looks like:

if '' or 0 or {} or None:
# never gets done
else:
# always gets done

My confusion stems from the syntax error not from Python's boolean
logic. Sorry, my statement at the end regarding '' != False was a bit
of a red herring. :\

I'm also in the habit of doing things like this:

def f(dict=None):
dict = dict or {}
# ...

You are no doubt wondering when I'm going to get to the part where
you can exploit this to save you those 3 lines of code. Sorry,
it won't help with that.

Ah but if it helps me to understand why Python forces me to do it then
although it won't save 3 lines it will stop me from complaining and
posting naff newbie questions.

| Is this related to Python's expression vs. statement syntactic
| separation? How can I be write this code more nicely?

Yes, exactly. Don't worry, it's nice as can be. If this is
the worst problem in your code, you're far better off than most
of us.

Okay well that is reassuring but a little disappointing.

Are there any plans (PEPs?) in the distant future to unify statements
and expressions in the Python syntax so I can generally do things like
this:

x = if aboolean:
<expression1>
else:
<expression2>

and

while c = f.read(1):
# ...

I couldn't find any general PEPs along these lines, only specific ones
(e.g. 308 re. an if-then-else expression). It does seem quirky given
Python's indentation syntax. However I notice I can already do some
things I wouldn't expect given a strict statement/expression
separation, such as:

x = y = z = 0

I suppose I have to learn these as special cases. (?)

 

[Greg McIntyre]

I've always accepted the None vs. 0 as a cavaet of the added
convenience but I think it's ultimately worth it.

Sorry, I didn't want to start a
"nothing values evaluate to false" argument.

I'll go read python-dev archives a bit and see if there's anything
useful for me to know.

Thanks

 

[Paul Rubin]

while c = f.read(1):
# ...

I couldn't find any general PEPs along these lines, only specific ones
(e.g. 308 re. an if-then-else expression).

I often end up doing something like this:

class foo:
def set(self, x):
self.x = x
return x
c = foo()

while c.set(f.read(1)):
# do stuff with c.x

In that file example, it's too much nuisance, but when you're
comparing some input against a series of regexps and you'd otherwise
need a multi-line construction for each one, this method comes in
quite handy.

 

[Greg McIntyre]

Not at all, IMHO. This is a simple forward-branching exit from a loop in
explicable circumstances (EOF). It is a common-enough idiom that doesn't
detract from readability & understandability. Spaghetti is like a GOTO
that jumps backwards into the middle of a loop for no discernable reason.

While that is true, I had high hopes that I could do this:

while c = f.read(1): # ...

And relative to that, it is more complex. And although I am nit-picking
to try to simplify this code, I wanted to understand why Python works
in this way (even if that's just "historical reasons"), and check to
see if there was not some shorter more modern Pythonic alternative.

I did actually like Robert Kern's suggestion which used an iterator and
a function to factor out the complexity of setting it up. I think that
is actually better code than the original.

It matches my philosophy in programming of pushing complexity *out* of
the code which does the actual job, even if it means writing a few
support functions/classes/whatever. I know they can be re-used and
refined and I know that it is the code that does the actual job that is
most likely to be rewritten in future revisions of the code with shifts
in requirements.

You have a bit of a misunderstanding here that needs correcting:

In "if <blah>" and "while <blah>", <blah> is NOT restricted to being in
(True, False). See section 5.10 of the Python Reference Manual:

I'm sorry! I realise that now and I'm sorry to have caused the traffic
I did. Thank you for pointing it out to me though - it's pretty
fundamental Python!

*Greg thumbtacks a note to his forehead*

How about
for c in f.read():
?
Note that this reads the whole file into memory (changing \r\n to \n on
Windows) ... performance-wise for large files you've spent some memory
but clawed back the rather large CPU time spent doing f.read(1) once per
character. The "more nicely" factor improves outasight, IMHO.

I would if only I had any kind of guarrantee on the file size but I
don't - this code is for reading a header out of a binary file which
uses delimiters and escape characters to mark out its fields. I didn't
design the format, but after cleaning up the code that deals with it, I
may *re*design it.

Mild curiosity: what are you doing processing one character at a time
that can't be done with a built-in function, a standard module, or a
3rd-party module?

Our company is designing a new file type. *sigh*. Confidentiality
prevents me from saying any more, too. If that bugs you because it's
not open source, sorry I need a job. Don't worry though, I'm developing
an open source remote GUI for the code management system we're using
called Aegis (http://aegis.sf.net). It's not sufficiently implemented
yet to warrant posting publically (I'd describe its current state as
"framework") but if anybody reading this is interested then give me a
yell and it'll have a public project page or something overnight.