this should work

[Rainer Weikusat]

This strikes me as premature optimization.

'Premature optimization' is a term which originally referred to the idea
of trying to write an 'optimal' machine-language programming 'from the
start', instead of concentrating on writing a working machine-language
program first and then - possibly - transform that into an
also-working but faster machine-language program and this in the
context of highly 'irregular' CISC-architecture as they were (still)
common in the 1970s where a lot of relatively 'bizarre' ways to
perform semantically identical operations in different ways actually
existed and it makes a hell lot of sense in this context.

It was never meant to be a universal justification for "But I don't
want to think about that!" and certainly also not for "But I don't
want to change my habits!"

Unless I know my program is
spending too much time doing my $tmp = 'ar!'; inside the for loop, I
would much prefer to keep $tmp properly scoped and initialized inside
the loop.

It is 'properly scoped' in the second example because the variable
stays (logically) alive for as long as it is going to be used and
'improperly scoped' in the first because (again, logically) it keeps
being destroyed and recreated (this is supposed to be a demonstration
that 'proper' means nothing on its own).

Unless you are having a serious out of memory problem,
'micro-optimizing' your code such that each object has the shortest
possible livetime can also be considered 'a premature optimization',
especially as it might not even help, as in this example.

 

[George Mpouras]

+1000 !

 

[Charles DeRykus]

....

It is 'properly scoped' in the second example because the variable
stays (logically) alive for as long as it is going to be used and
'improperly scoped' in the first because (again, logically) it keeps
being destroyed and recreated (this is supposed to be a demonstration
that 'proper' means nothing on its own).

But it (illogically) stays alive longer than is needed unless you insert
an additional, artificial scope as another poster showed:

{ my $tmp; for (...) { $tmp='ar!'} }.

[ And, of course, Ben's refaddr demo showed it really isn't "destroyed
and re-created". Digression: I seem to recall having seen a demo that
confirmed the older behavior once - it just printed \$tmp in lieu of
refaddr but the addr's differed each time and I could swear (almost) it
was newer than 5.6 ]

Unless you are having a serious out of memory problem,
'micro-optimizing' your code such that each object has the shortest
possible livetime can also be considered 'a premature optimization',
especially as it might not even help, as in this example.

But, clearly, the narrowest possible scope for variables does help...
with potential variable collisions, havoc, and confusion in that
dangerous "no-man's land outside your own trench".

IMO, the insane optimization, is running a timing benchmark and
obsessing about microseconds of speed over hundreds, or more often
thousands, of iterations.

 

[Rainer Weikusat]

But it (illogically) stays alive longer than is needed unless you
insert an additional, artificial scope as another poster showed:

The 'logically' was supposed to refer to the fact that perl employs
self-defense mechanisms against code of this type by NOT doing what
the author requested: It keeps the variable alive despite this seems
'ill/im' (arbitrary suffix attached) to some people: The computer has
been programmed to work around this obsession with the
'micro-optimized lifecycle management' (And this is a euphemism. I'd
wager a bet this this is practically just 'create a new variable
whenever you need one because remembering what variables were created
two lines ago would be SO cumbersome' [and 'plan in advance variables
will be needed' an imposition beyond any physically tolerable by
man]).

Do you think it was programmed to work around that because this is
such a great idea? I don't. Especially since the computer can only
work around the execution time penalty of this convention and not
against the mess in the source code ("which of the 1,375 $i I
encountered in the last 2000 lines of code is it this time?").

 

[Rainer Weikusat]

This is not about habits:

It is about habits grown out of the perfectly accidental fact that a
certain language allows variable declarations in inner scopes. Not all
of them do ...

anyone who's been here (or anywhere else
programming is taught) for a while will know that getting beginners to
understand the value of proper scoping is not easy. It's about
cleanliness:

.... and specifically, a certain language who was explicitly created to
teach beginners about 'proper scoping' and 'cleanly designed code',
namely, Pascal, doesn't and IMHO, with a good reason: When people
can't have multiple disjunct sets of variables used by unrelated parts
of the same 'aggregate subroutine', they do what they should be doing
instead, namely, structure their code.

 

[Ivan Shmakov]

[...]

It is about habits grown out of the perfectly accidental fact that a
certain language allows variable declarations in inner scopes. Not
all of them do ...
[...]

... and specifically, a certain language who was explicitly created
to teach beginners about 'proper scoping' and 'cleanly designed
code', namely, Pascal, doesn't

Somehow, my impression was that it /does/ allow for nested
variable scopes. However, I have only a passing experience with
the language.

and IMHO, with a good reason: When people can't have multiple
disjunct sets of variables used by unrelated parts of the same
'aggregate subroutine', they do what they should be doing instead,
namely, structure their code.

I disagree. I deem the use of nested scopes as crucial to code
structuring. Should the "roles" of the variables (whether
input, output, or local) become apparent later, it'd be trivial
to split the function, -- and that's likely to be done exactly
along the scope boundaries previously coded in.

 

[Peter J. Holzer]

Pascal was designed in the late 1960s, only about a decade after
FORTRAN and COBOL. I hope we have learned something in the more than
four decades since then.

Somehow, my impression was that it /does/ allow for nested
variable scopes.

Maybe ISO Pascal does now. But Pascal as originally designed by Wirth
didn't. All the variable declarations had to be at the beginning of the
program, procedure or function and the code had to be at the end (with
possibly more nested procedures or functions in between). Of course the
nested procedures and functions could have their own properly scoped
variables, so there were nested scopes, but only at the
procedure/function level, not at the block level. This had two negative
consequences: Variable declarations in the outer procedures were
separated widely (often hundreds or thousands of lines) from the place
where they were used and they were visible in all the inner procedures,
so it was easy to accidentally reuse a loop variable in an inner
procedure called from that loop. I think a large part of the motivation
for Knuth's WEB came from the rigid structure of Pascal which forced you
to put stuff which logically belonged together into different portions
of the program.

However, I have only a passing experience with the language.

Likewise. And all that experience happened in the 1980s, so it's almost
as dated as my experience with BASIC (the line-numbers-and-goto kind of
BASIC).

I disagree. I deem the use of nested scopes as crucial to code
structuring. Should the "roles" of the variables (whether
input, output, or local) become apparent later, it'd be trivial
to split the function, -- and that's likely to be done exactly
along the scope boundaries previously coded in.

Right. I split functions/methods quite frequently. Either if a function
becomes too long or if I notice that I need part of a function in a
different function. Nested scopes make this kind of refactoring a
no-brainer. And of course they make reading the code a lot simpler, too.

hp

 

[Dr.Ruud]

Are you really proposing that the programmer should be expected to
write:

{
my $tmpdir;
foreach my $dir (qw/commands_pre commands_post/) {
$tmpdir = "/tmp/$dir";
print "$tmpdir\n"
}
}

I would expect her to write it as:

print "/tmp/$_\n"
for qw/ commands_pre commands_post /;

 

[Rainer Weikusat]

Pascal was designed in the late 1960s, only about a decade after
FORTRAN and COBOL. I hope we have learned something in the more than
four decades since then.

ALGOL 60 was (as the name may suggest) designed at the beginning of
the 1960s and it included the concept of nested lexical scopes with
attached variable sets, cf

http://www.masswerk.at/algol60/report.htm

This means your statement would be more appropriately worded as: Four
decades after someone suggested a different approach, we are as
unconvinced as ever that anything was deficient with the tradional way
of handling this.

[Pascal program structure description]

This had two negative consequences: Variable declarations in the
outer procedures were separated widely (often hundreds or thousands
of lines) from the place where they were used

That's not a consequence of having to put variable definitions at the
beginning of a procedured but of including 'hundreds or thousands' of
lines of loosely related code in between and the simple solution to
this problem is "Don't do that".

and they were visible in all the inner procedures, so it was easy to
accidentally reuse a loop variable in an inner procedure called from
that loop.

Again, the way to solve the problem that 'nobody can still find his
way through this mess' is to avoid creating it in the first place:
Don't separate 'declaration and use' of a variable by 'thousands of
lines of code', move these 'thousands of lines' into subroutines which
don't have to share state with some parent routine and pass the data
they need to them as arguments.

[...]

Right. I split functions/methods quite frequently. Either if a function
becomes too long or if I notice that I need part of a function in a
different function. Nested scopes make this kind of refactoring a
no-brainer.

The point is that this 'kind of refactoring' shouldn't ever be
necessary because the '15,000 lines of code and counting' 'subroutine'
shouldn't have been created to begin with: That's another of these
'outlandish new concepts' people have been ignoring for four decades
now ('stepwise refinement') because 'the old ways' don't need no
fixing especially since ...

And of course they make reading the code a lot simpler, too.

.... abstraction is evil because it hides details. It is claimed that
some of them wouldn't be significant at some higher level but
actually, higher level already sounds highly suspicious ...

 

[Rainer Weikusat]

On Thu, 11 Jul 2013 22:04:45 +0100 in comp.lang.perl.misc, Ben


Sorry to hear that.

When adding an additional block to the last out_of_loop example I
posted, it doesn't become any slower than it was without the block.

 

[John Black]

Rainer has a point, buried somewhere in the bluster, in that if a sub
needs to be divided into two or more bare blocks in order to limit the
scope of variables those blocks should probably be diked out into
separate subs. I very rarely use bare blocks, and when I do it's usually
for their loop properties (so that next/last/redo will work) rather than
their scoping.

Ben, what is the meaning of a bare block? Trying to follow...

John Black

 

[Dr.Ruud]

However, it's very rare to see anyone use do {} for
scoping

Often used in "libary" code here, like:

my $sql_stmt = do {

local $" = q{,};

qq{
SELECT
$columns_csv
FROM
$table
WHERE
id IN (@ids) -- hundreds easily
AND
$and_where
};
};


We also have quite some occurences of
eval { ... } or do { my ...; ... };


The worst are the do-blocks of many hundreds of lines.
They tend to keep growing, and prove hard to get rid of.

 

[Dr.Ruud]

I was unclear. What I meant was, if you just want a simple variable-
scope-limiting block, the cheapest way to do that is with a do block (in
void context).

Using do to turn a block into an expression is what it was meant for, of
course.

Not much need for do() in void context.

Please, someone tell your cow-orkers about placeholders...

I had put the 'library' and the 'hundreds easily' there to prevent this
remark.

This kind of code is only feasible if you know the quality and security
of the parameters. The contents of @ids is, of course, strictly controlled.

With hundreds (or thousands) of placeholders, quite some CPU-time gets
wasted. That time is simply too expensive for us.
(And without type-ing, the values even get quoted.)


[...]

Um, just turn 'em into subs. (Of course, a sub of hundreds of lines
isn't much better, but then you can start breaking it up into sensible
pieces.)

If we would have the time to spend on them, we would. Now we just
complain about them. They hardly ever get smaller, unless they really
annoy the dev that is hunting for the bug.

 

[Tim McDaniel]

Please, someone tell your cow-orkers about placeholders...

The classic "Bobby Tables" strip:
http://xkcd.com/327/

 

[Dr.Ruud]

The classic "Bobby Tables" strip:
http://xkcd.com/327/

Yeah, also doesn't apply.

See how I left out what @ids is. Now, in stead of assuming anything else
again, assume that @ids can only contain numbers, and that each is
between 1 and some maximum.

 

[Xho Jingleheimerschmidt]

I was unclear. What I meant was, if you just want a simple variable-
scope-limiting block, the cheapest way to do that is with a do block (in
void context).

Using do to turn a block into an expression is what it was meant for, of
course.


Please, someone tell your cow-orkers about placeholders...

....which alas don't work well with IN-lists, or column names, or table
names.

Um, just turn 'em into subs. (Of course, a sub of hundreds of lines
isn't much better, but then you can start breaking it up into sensible
pieces.)

I'd much rather read one sub with hundreds of lines, than 49 individual
subs each of 10 lines, each only used once, in a linear fashion from one
meta-sub.

You should create subs when they encapsulate a nicely self-contained
re-usable nugget, not because the line count achieved a certain value.

Xho

 

[Xho Jingleheimerschmidt]

Fair enough. I'm surprised that's the case, but you've obviously tested
it with your database driver. I generally use Pg, and I'm reasonably
certain (I haven't benchmarked it) that the most efficient way to write
that statement would be with a single placeholder that gets bound to an
arrayref (which is then sent to the database as a Pg array), but
presumably that isn't the case in your situation.

Oooh, cool. I'll have to try that. This seems to the defy the docs of
DBI itself, so I'd never considered the possibility.

Any neat tips for making this annoying mess less of an annoying mess?

'UPDATE foo set (bar, baz, bxw, snaz1, snaz2, snaz3, snaz4,snaz5) =
($2,$3,$4,$5,$6,$7,$8,$7+$8) where pkey=$1'

Xho

 

[Dr.Ruud]

A statement like

do {
...
};

is equivalent to

{
...
}

except it's a little cheaper, and the loop operators don't work.

Another variant: if (1) { ... }

Fair enough. I'm surprised that's the case, but you've obviously tested
it with your database driver. I generally use Pg, and I'm reasonably
certain (I haven't benchmarked it) that the most efficient way to write
that statement would be with a single placeholder that gets bound to an
arrayref (which is then sent to the database as a Pg array), but
presumably that isn't the case in your situation.

Good to mention the driver difference. Some drivers handle the 'prepare'
in Perl (at least by default), and only send the final SQL to the db-server.

Ah, the luxury of a single placeholder bound to an arrayref, and a
server side prepare. .

 

[John Black]

Ben, what is the meaning of a bare block? Trying to follow...

A block like this:

{
my $x = ...;
}

with no other control structure attached. They can be used to limit the
scope of a variable to something smaller than a sub; in Perl they are
also one-iteration loops, meaning that the loop control operators (next,
last, redo) and the loop-naming syntax will work on them.

[Strictly speaking the 'barest' form of block in Perl is the do {}
block, which, like the block form of 'if', is just a scope without any
loop semantics. However, it's very rare to see anyone use do {} for
scoping, partly because it looks weird and partly because do is an
expression and so requires a trailing semicolon, like eval {}.]

Thanks!

John Black

 

[Ivan Shmakov]

[...]

I disagree. I deem the use of nested scopes as crucial to code
structuring. Should the "roles" of the variables (whether input,
output, or local) become apparent later, it'd be trivial to split
the function, -- and that's likely to be done exactly along the
scope boundaries previously coded in.

To quote my boss: Make it work now quickly and clean it up later .
Which means: Take some existing code which performs a more-or-less
related task, copy'n'paste it to some part of the countryside where
no trenches have been dug yet or the old ones worn out over time,
create a new nested scope lest all hell breaks lose because any
accidental interaction with the surrounings, any details about them
long lost in the land of ancient lore,

[...]

... Once, I will find the patience to wait for the food
engineers out there to design a sound nutritional solution.

Meanwhile, I'm forced to rely on the off-the-shelf products,
which are known to be full of undocumented features, deviate
from the specifications every now and then, and (while I'm yet
to see one myself) are reported to contain actual bugs...

[Cross-posting to and Just in case.]