opening a file

[Tim Greer]

Tim Greer wrote:
[...]

Even if you build the bridge the same way, one has fail safe's and
checking to allow the inspectors or even the people driving and
walking across the bridge some warnings about an existing problem
that may lead to a disaster, allowing workers to remedy the problem
or close down the
bridge until the problem is resolved. [...]

Unless, of course, you live in Minneapolis.

Oh, SNAP.

 

[Jürgen Exner]

I might be in a little different environment than most people. Much of
what I do involves querying a big institutional database over which I
have no control, and which is characterized by constant change. Two
quick examples: (1) Grades were identified by characters (A, B, C)
until one day my queries failed completed. I discovered that the
powers that be decided that henceforth grades would be identified by
digits (6, 7, 8), with no sort of notice to me or anyone else.

What did you do about that? Did you talk to those guys? Did you talk to
your boss about it? What did he do about it? If an agreed upon interface
is not respected and modified without discussion and even without notice
then something is seriously, I mean SERIOUSLY, wrong. If it happenens
more than once this is bad enough to raise hell at least two levels up
the hierachie.

And afterwards, what happened to prevent further incidentcs like this?
Did those guys update the spec for the database format? Did they at the
very least appologize? Did _YOU_ or your boss establish a better line of
communication with those guys?

(2)
This week, I discovered that two columns had been transposed, so that
the expected thousands of students were only 76. I couldn't just un-
transpose the columns, because of the 76 'right' students, so I had to
select and merge two columns. Obviously, the script that I wrote in
December had to be rewritten again this week to account for the
change.

So -theoretically speaking- you boss doesn't care how you create
whatever results you create. As far as he is concerned you could use
pencil and paper, right? Then you are not a programmer but a data
analyst or data processor who happens to be using some scripts for his
own pleasure.

My scripts frequently fail for this reason, so I can't rely on static
scripts that never change.

Excellent case in point. In a stable environment you can get away
without much error checking, because errors are infrequent.

It is in an ever changing environment where error checking becomes a
mandatory survival strategy and where you need to do every error
checking imaginable, so that your scripts will alert you about those
changes right away and pinpoint the cause as detailed as possible.

In your first example you were assuming, the grades would be letters.
Had you verified this assumptions by a test in your program then your
program would have told you right away and you wouldn't have to start an

Then I must not be a programmer, because most of what I do gets
printed with a physical printer on real paper. My point is (and I'm
being redundant here) is that in doing a job you minimize costs while
maximizing benefits.

Yeah, right. And then you are posting about "Perl crisis". Right.

Again, I'm not advocating not checking the return value of a call to
open, or not following all the other best practices. What I said was
that in the daily grind I frequently omit some niceties. I'm an adult
and am perfectly capable of dealing with it.

Do it right or suffer the consequences. Sorry, but your arguments are
very short sighted and your problems at your job are very much caused by
yourself: you keep patching the symptoms instead of addressing the
cause. Well, keep patching. It will keep you busy, but it will never
improve the situation.

And if you can't do it, then it's your bosses job to recognize your
problem and assign someone else to fix the reason or to do it himself.
If he doesn't then he is incompetent.

jue

 

[Charlton Wilbur]

JE> Yeah, right. And then you are posting about "Perl
JE> crisis". Right.

Ah, in doing a job you minimize *overall* costs while maximizing
*overall* benefits.

cartercc's fundamental problem is that he is apparently completely
unaware of long-term costs (or considers them his manager's problem, not
his problem), and is thus minimizing *short-term* costs.

Eventually, this bites you in the ass. If cartercc's posting history is
to be believed, it has bitten him in the ass several times, but he has
yet to learn from it.

Charlton

 

[cartercc]

What did you do about that? Did you talk to those guys? Did you talk to
your boss about it? What did he do about it? If an agreed upon interface
is not respected and modified without discussion and even without notice
then something is seriously, I mean SERIOUSLY, wrong. If it happenens
more than once this is bad enough to raise hell at least two levels up
the hierachie.

I can't talk to 'those guys.' I've tried many times but have gotten no
response. I seriously doubt they create any kind of documentation.
Some of the tables are horrendous, something like 500 columns across,
and with no rhyme or reason in the calculated fields. Example: in one
table there are three calculated fields named 'Start Term.' You would
think that all three fields would return the same value, except this
morning (for the new term) one returned about 2500, one returned 2488,
and the third returned less than 100. I honestly think that the
keepers of the database keep it locked up because they are afraid of
exposing it to public scrutiny.

And afterwards, what happened to prevent further incidentcs like this?
Did those guys update the spec for the database format? Did they at the
very least appologize? Did _YOU_ or your boss establish a better line of
communication with those guys?

Hell, no. Trying to have a conversation with the Lords of the Database
is as futile as trying to fly by flapping your arms.

So -theoretically speaking- you boss doesn't care how you create
whatever results you create. As far as he is concerned you could use
pencil and paper, right? Then you are not a programmer but a data
analyst or data processor who happens to be using some scripts for his
own pleasure.

Exactly right. He's a business type, and couldn't tell the difference
between an array and a scalar to save his live. He doesn't care what I
do as long as it's right, and he mostly leaves me alone. I can go
weeks without doing more than just saying hello. In some ways, it's a
very good work environment because I don't have anyone looking over my
shoulder.

Excellent case in point. In a stable environment you can get away
without much error checking, because errors are infrequent.

Go back to your previous point. I live in an ad hoc world, writing
mostly ad hoc scripts and rely more on technique than checklists. (It
wasn't always this way, when I started I was building web front ends
for databases and paid attention to exception and error checking --
you have to do this building apps for a web server.)

It is in an ever changing environment where error checking becomes a
mandatory survival strategy and where you need to do every error
checking imaginable, so that your scripts will alert you about those
changes right away and pinpoint the cause as detailed as possible.

Actually, there's a difference between querying the database and
processing the data. Once I acquire the data, I can do anything I want
with it, and it's all transparent. Yes, I am frequently surprised by
something that does not turn out as expected (like the transposition
of columns), but I can protect myself against personal error (by using
strict, using warnings, etc.). I can't protect myself against garbage
input, usually because I have no way of knowing that the initial data
import is garbage.

In your first example you were assuming, the grades would be letters.
Had you verified this assumptions by a test in your program then your
program would have told you right away and you wouldn't have to start an
expedition to <quote>discover</quote> the cause.

Yes, you are right. In a perfect world maybe I should have assumed
that grades wouldn't always be characters, but I would still have no
way to know the translation of letter grades to numbers. A is now 6, B
is 7, and Incomplete is 13. Who wouldda thought it?

Do it right or suffer the consequences. Sorry, but your arguments are
very short sighted and your problems at your job are very much caused by
yourself: you keep patching the symptoms instead of addressing the
cause. Well, keep patching. It will keep you busy, but it will never
improve the situation.

Hey, it's a job, and I'm somewhat fortunate to have a job. I know
several IT guys who have been laid off recently. The good points are
that I have a lot of free time, a very non-intrusive manager, the
total freedom to use any technology that I want, a steady pay check,
non-existent stress, and a collegial environment (among the IT guys,
all but me are network and sys admins.)

And if you can't do it, then it's your bosses job to recognize your
problem and assign someone else to fix the reason or to do it himself.
If he doesn't then he is incompetent.

Mostly, my boss cares about not getting complaints about the work that
I do -- he certainly has no desire to acquaint himself with what I do
on a technical level. I'm pretty successful keeping my customers
happy, so he almost never gets a complaint.

CC

 

[Tad J McClellan]

[ recap: I wrote:

You should always, yes *always*, check the return value from open()
]

I would personally never intentionally fail
to check a return on a call

Just to show that my "pendulum of inflexibility" can also swing
the other way, I *never* check the return value from print().

print() has a return value to indicate success, as do many functions in Perl.

But once you have a successfully opened write filehandle, about the only
thing that can go wrong with a print() is "filesystem full".

(assuming the filehandle is connected to a real file rather than a
socket or something.)

If the filesystem is full, I won't need my little Perl program to tell
me that something is wrong, because just about everything will fail to work.





Everything in Engineering is a trade-off.

Goldilocks is the archtype engineer.

How much error checking should you do?

Not too much, not too little, just right!


Therein lies our job security.

 

[Eric Pozharski]

[ recap: I wrote:

You should always, yes *always*, check the return value from open()
]

I would personally never intentionally fail
to check a return on a call

Just to show that my "pendulum of inflexibility" can also swing
the other way, I *never* check the return value from print().

print() has a return value to indicate success, as do many functions in Perl.

But once you have a successfully opened write filehandle, about the only
thing that can go wrong with a print() is "filesystem full".

(assuming the filehandle is connected to a real file rather than a
socket or something.)

If the filesystem is full, I won't need my little Perl program to tell
me that something is wrong, because just about everything will fail to work.

Some time ago I'd concluded that checking return of B<close> of
RO filehandle is useles since that syscall would fail only in case when
a whole system crashed. While reading that braindead thread I've came
to idea that I was somewhat wrong. Am I?

*CUT*

 

[Tim Greer]

Just to show that my "pendulum of inflexibility" can also swing
the other way, I *never* check the return value from print().

print() has a return value to indicate success, as do many functions
in Perl.

Okay, you got me. Good point. I didn't mean it as all should be. I
can't say I've ever check the return value of print either. I guess I
meant for any functions you should check.

 

[Andrew DeFaria]

Okay, you got me. Good point. I didn't mean it as all should be. I
can't say I've ever check the return value of print either. I guess I
meant for any functions you should check.

And, if you think about it, if print fails what are you gonna do? I mean
how are you gonna inform the outside world of the failure if you can't
talk to the outside world?!?

;-)

Now print to a file... That's another matter...

 

[Tad J McClellan]

[ recap: I wrote:

You should always, yes *always*, check the return value from open()
]

I would personally never intentionally fail
to check a return on a call

Just to show that my "pendulum of inflexibility" can also swing
the other way, I *never* check the return value from print().

print() has a return value to indicate success, as do many functions in Perl.

But once you have a successfully opened write filehandle, about the only
thing that can go wrong with a print() is "filesystem full".

(assuming the filehandle is connected to a real file rather than a
socket or something.)

If the filesystem is full, I won't need my little Perl program to tell
me that something is wrong, because just about everything will fail to work.

Some time ago I'd concluded that checking return of B<close> of
RO filehandle is useles since that syscall would fail only in case when
a whole system crashed. While reading that braindead thread I've came
to idea that I was somewhat wrong. Am I?

If it was a pipe open, then yes, you should have checked.

See the 3rd paragraph in:

perldoc -f close

 

[Peter J. Holzer]

Okay, you got me. Good point. I didn't mean it as all should be. I
can't say I've ever check the return value of print either. I guess I
meant for any functions you should check.

Errors in file handle accumulate, so you can often defer checking until
close:

open(my $fh, '>', $filename) or die ...;
while (...) {
my $a_record = compute_next_record();
print $fh $a_record;
}
close($fh) or die "cannot close $filename: $!";

will die with the message "cannot close foo: No space left on device" if
the filesystem gets full during the loop.

That's fine for programs which run only a short time. But if your
program writes 100 million records and the disk gets full after 1
million, you don't want to waste time computing another 99 million
records which are never going to be written anyway.

Similarily for filehandles which are never or infrequently closed, e.g.
logfiles of daemon processes. If you care about the data you write (or
about the rest of the system) you have to check the return value of
print.

You may not have to check each print, because - as mentioned above -
errors are "sticky". So if you have a loop like

while (...) {
print $fd $record_header;
print $fd $first_part;
some_sub_which_prints_more_parts();
print $fd $record_trailer or die "...";
}

it's probably sufficient to check the last print in the loop.
Or you can use $io->error:

use IO::Handle;
...
while (...) {
print $fd $record_header;
print $fd $first_part;
some_sub_which_prints_more_parts();
print $fd $record_trailer;
die "..." if $fd->error;
}

 

[Peter J. Holzer]

[ recap: I wrote:

You should always, yes *always*, check the return value from open()
]

I would personally never intentionally fail
to check a return on a call

Just to show that my "pendulum of inflexibility" can also swing
the other way, I *never* check the return value from print().

print() has a return value to indicate success, as do many functions in Perl.

But once you have a successfully opened write filehandle, about the only
thing that can go wrong with a print() is "filesystem full".

(assuming the filehandle is connected to a real file rather than a
socket or something.)

If the filesystem is full, I won't need my little Perl program to tell
me that something is wrong, because just about everything will fail to work.

You may still do something sensible, like removing the partial file(s)
you have just written. Especially if they are several Gigs long.
And a message "no space left on device" is much more user-friendly than
causing "everything to fail" in mysterious ways.

Besides, on a properly administrated multi-user system, one program
should not be able to cause "everything" to fail. There are such things
like partitions, quotas, etc. which can be used to limit the damage.

Some time ago I'd concluded that checking return of B<close> of
RO filehandle is useles since that syscall would fail only in case when
a whole system crashed. While reading that braindead thread I've came
to idea that I was somewhat wrong. Am I?

Yes. Errors on a file handle are sticky, so close will also fail if any
I/O operation on that handle has failed. So checking the return value of
close is often a handy short-cut for checking every single print,
printf, etc. But see my other posting on this matter.

hp

 

[Eric Pozharski]

[ recap: I wrote:

You should always, yes *always*, check the return value from open()
]


I would personally never intentionally fail
to check a return on a call


Just to show that my "pendulum of inflexibility" can also swing
the other way, I *never* check the return value from print().

print() has a return value to indicate success, as do many functions
in Perl.

But once you have a successfully opened write filehandle, about the only
thing that can go wrong with a print() is "filesystem full".

(assuming the filehandle is connected to a real file rather than a
socket or something.)

If the filesystem is full, I won't need my little Perl program to tell
me that something is wrong, because just about everything will fail to work.

Some time ago I'd concluded that checking return of B<close> of
RO filehandle is useles since that syscall would fail only in case when
a whole system crashed. While reading that braindead thread I've came
to idea that I was somewhat wrong. Am I?

If it was a pipe open, then yes, you should have checked.

I recall the thread about closing pipes. It was very noisy, thanks to
me.

See the 3rd paragraph in:

perldoc -f close

I was about reading regular files though.

 

[Tim Greer]

Andrew DeFaria wrote:

And, if you think about it, if print fails what are you gonna do? I
mean how are you gonna inform the outside world of the failure if you
can't talk to the outside world?!?

;-)

Now print to a file... That's another matter...

 

[Tim Greer]

You may not have to check each print, because - as mentioned above -
errors are "sticky". So if you have a loop like

while (...) {
print $fd $record_header;
print $fd $first_part;
some_sub_which_prints_more_parts();
print $fd $record_trailer or die "...";
}

it's probably sufficient to check the last print in the loop.
Or you can use $io->error:

use IO::Handle;
...
while (...) {
print $fd $record_header;
print $fd $first_part;
some_sub_which_prints_more_parts();
print $fd $record_trailer;
die "..."

I'm sure there are cases where if you really want or need to check if
print was successful, it could be used and come in handy. I don't
recall ever running into a situation where it would have really
necessary in that case -- but again, I could see, depending on what
you're doing, some things could benefit from ultra-paranoid checking.