Perl array bug?

[Wolf Behrenhoff]

Here's some isolated code that demonstrates my problem:

@Big1 = ();
$Big1[4] = 0;
$Big1[9] = 1;
$Big1[6] = 2;
my $len = 0; # This appears to be significant
foreach $key (@Big1) {
$len = length ($key);
print "key = $key, len=$len\n";
}
exit;

Operating system is 64-bit Windows 7.

Using ActivePerl version 5.12.1.1201 64-bit I get:

C:\ZIP>c:\perl512\bin\perl.exe test.pl
key = , len=0
key = , len=0
key = , len=0
key = , len=0
key = 0, len=1
key = , len=1 <==== ???
key = 2, len=1
key = , len=1 <==== ???
key = , len=1 <==== ???
key = 1, len=1

So, what say you guys? Bug or user error?

Clearly programmer error. Neither printing the value of nor computing
the length of undef is well defined.

This is Perl, not C. Undef is an empty string when converted to a
string, not some random garbage. It is zero when used in calculations.

It *is* well defined!

Read perlsyn!
------
The only things you need to declare in Perl are report formats and
subroutines (and sometimes not even subroutines). A variable holds the
undefined value ("undef") until it has been assigned a defined value,
which is anything other than "undef". When used as a number, "undef" is
treated as 0; when used as a string, it is treated as the empty string,
""; and when used as a reference that isn't being assigned to, it is
treated as an error.

 

[Peter J. Holzer]

foreach $key (@Big1) {
$len = length ($key);
if ($len) {
if ($key eq '' && $len == 4) {print "You're kidding me!\n"; }

Where we encounter a bizarre situation whereby an element
is undefined but has a positive length.

Your code doesn't test whether it is undefined. It tests whether it is
equal to the empty string. However, I don't see how the empty string
could have length 4, so there is something fishy going on. Use the
Dump() function from Devel:eek to look at the affected values.

hp

 

[Malcolm Hoar]

This is Perl, not C. Undef is an empty string when converted to a
string, not some random garbage. It is zero when used in calculations.

It *is* well defined!

Read perlsyn!
------
The only things you need to declare in Perl are report formats and
subroutines (and sometimes not even subroutines). A variable holds the
undefined value ("undef") until it has been assigned a defined value,
which is anything other than "undef". When used as a number, "undef" is
treated as 0; when used as a string, it is treated as the empty string,
""; and when used as a reference that isn't being assigned to, it is
treated as an error.
------

Ahhh, thank you. I had been searching for a more formal definition
and explanation of the undef value and had not yet found this.
I kept finding more and more references to the undef function
which wasn't what I needed.

This basically confirms and formalizes all of my instincts!

I am very curious to know if anyone can reproduce the problem
on a non-Windows system. If the issue is Windows specific, I
will certainly contact ActiveState directly.

It's been a while since I've formally reported a Perl bug.
What's the preferred channel these days? perlbug, or something
else?

--
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
| Malcolm Hoar "The more I practice, the luckier I get". |
| (e-mail address removed) Gary Player. |
| http://www.malch.com/ Shpx gur PQN. |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

[C.DeRykus]

No. This array name is reused a lot (which helps memory usage).
But it's explicitly cleared at the start of the problem module:

@Big1 = ();

Does anything change with the alternative:

undef @Big1;

This might be a quick fix or at least help
diagnose the scope of the bug...

Elements are set, modified, and used but never deleted/undef'ed.

Before the module returns, the array is completely cleared
once again.

is there some reason for clearing again...

 

[Malcolm Hoar]

Does anything change with the alternative:

undef @Big1;
No.


is there some reason for clearing again...

To reduce virtual memory consumption. This is one of
many huge arrays in the "real" application. I had to
opmtimize the heck out of memory usage.

--
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
| Malcolm Hoar "The more I practice, the luckier I get". |
| (e-mail address removed) Gary Player. |
| http://www.malch.com/ Shpx gur PQN. |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

[Malcolm Hoar]

It's going to be tough to reproduce without a testcase. If you can, it
would be good to start stripping things out of you app until you can't
reproduce the problem any more.

Yeah, I think you missed the top part of that post, with a
test case ;-)

perlbug is correct, but without a test case it's rather unlikely anyone
can do anything about it. (It is, of course, possible someone on p5p
will spot something I haven't.)

Here's a draft of what I plan to submit via perlbug including
the test case:

This is a bug report for perl from (e-mail address removed),

Operating System: 64-bit Windows 7.

Perl Version: ActivePerl version 5.12.1.1201 64-bit:

---
Flags:
category=core
severity=high
---

Code to reproduce this problem:

#====================================================

@Big1 = ();
$Big1[4] = 0;
$Big1[9] = 1;
$Big1[6] = 2;
my $len = 0; # This appears to be significant
foreach $key (@Big1) {
$len = length ($key);
print "key = $key, len=$len\n";
}
exit;

#=====================================================

Using ActivePerl version 5.12.1.1201 64-bit I get:

C:\ZIP>c:\perl512\bin\perl.exe test.pl
key = , len=0
key = , len=0
key = , len=0
key = , len=0
key = 0, len=1
key = , len=1 <==== ???
key = 2, len=1
key = , len=1 <==== ???
key = , len=1 <==== ???
key = 1, len=1

Using ActivePerl version 5.10.1.1007 64-bit I get:

C:\ZIP>C:\perl510\bin\perl.exe test.pl
key = , len=0
key = , len=0
key = , len=0
key = , len=0
key = 0, len=1
key = , len=0
key = 2, len=1
key = , len=0
key = , len=0
key = 1, len=1


Some other Perl programmers have taken the position that this
is a programmer error because the code attempts to establish
the length() of an undef value. That, they say, is garbage
analogous to division by zero.

I disagree and cite the following in support of my position
that this constitutes a bug:

1. Perl Version 5.10.1.1007 and every other major Perl version
going back 10 plus years have consistently returned zero
for the length of an undef value.

2. perlsyn clearly states:

"A variable holds the undefined value ("undef") until it has been
assigned a defined value, which is anything other than "undef".
When used as a number, "undef" is treated as 0; when used as a
string, it is treated as the empty string, ""; and when used as
a reference that isn't being assigned to, it is treated as an error."

Thus "undef" should be treated as the empty string. And the
length of the empty string is unambiguously zero. Hence the
behaviour of the 5.12.1.1201 is clearly a bug!


Summary of my perl5 (revision 5 version 12 subversion 1) configuration:
[snip]




--
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
| Malcolm Hoar "The more I practice, the luckier I get". |
| (e-mail address removed) Gary Player. |
| http://www.malch.com/ Shpx gur PQN. |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

[Dilbert]

It's going to be tough to reproduce without a testcase. If you can, it
would be good to start stripping things out of you app until you can't
reproduce the problem any more.

Code to reproduce this problem:

#====================================================

@Big1 = ();
$Big1[4] = 0;
$Big1[9] = 1;
$Big1[6] = 2;
my $len = 0; # This appears to be significant
foreach $key (@Big1) {
  $len = length ($key);
  print "key = $key, len=$len\n";}

exit;

#=====================================================

This sounds very much like a regression in Perl 5.12.

Here is an even simpler test case:

==============================
use Test::More tests => 4;

my $my_len;

$my_len = length('abc');
is($my_len, 3, 'length("abc") with my variable must be 3');

$my_len = length(undef);
is($my_len, 0, 'length(undef) with my variable must be 0');

our $our_len;

$our_len = length('abc');
is($our_len, 3, 'length("abc") with our variable must be 3');

$our_len = length(undef);
is($our_len, 0, 'length(undef) with our variable must be 0');
==============================

On Perl 5.10 (x64) I get:
==============================
1..4
ok 1 - length("abc") with my variable must be 3
ok 2 - length(undef) with my variable must be 0
ok 3 - length("abc") with our variable must be 3
ok 4 - length(undef) with our variable must be 0
==============================

On Perl 5.12 (x64), however, I get two test failures:
==============================
1..4
ok 1 - length("abc") with my variable must be 3
not ok 2 - length(undef) with my variable must be 0
# Failed test 'length(undef) with my variable must be 0'
# at C:\testbug512.pl line 9.
# got: '3'
# expected: '0'
ok 3 - length("abc") with our variable must be 3
not ok 4 - length(undef) with our variable must be 0
# Failed test 'length(undef) with our variable must be 0'
# at C:\testbug512.pl line 17.
# got: undef
# expected: '0'
# Looks like you failed 2 tests of 4.
==============================

 

[Malcolm Hoar]

Malcolm: since I've now got both a clean testcase and a clear
understanding of the issue (and probably soon a patch), should I make
the bug report?

I did submit one here:

http://rt.perl.org/rt3//Public/Bug/Display.html?id=77336

--
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
| Malcolm Hoar "The more I practice, the luckier I get". |
| (e-mail address removed) Gary Player. |
| http://www.malch.com/ Shpx gur PQN. |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

[Malcolm Hoar]

Yeah, I saw that. I've attached a patch to that bug which fixes the
issue.

If you felt so inclined you could probably build a 5.12.1 with that
patch.

Thanks. I'm not really setup for making builds at the moment.
I have pretty much finished testing a workaround for my app
so provided this gets fixed for the next major update, I'm
a very happy camper.

Thanks for your help and support with this Ben. I really
appreciate it.



--
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
| Malcolm Hoar "The more I practice, the luckier I get". |
| (e-mail address removed) Gary Player. |
| http://www.malch.com/ Shpx gur PQN. |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 

[Ilya Zakharevich]

To reduce virtual memory consumption.

I'm not sure this does what you expect it to do. What you want is
undef()ing; it *would* clear everything.

Yours,
Ilya

 

[Malcolm Hoar]

For the benefit of those who haven't been following the discussion on
p5p: this patch has now been applied to maint-5.12, which means it will
be in 5.12.2.

Yes, and it's much appreciated!

--
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
| Malcolm Hoar "The more I practice, the luckier I get". |
| (e-mail address removed) Gary Player. |
| http://www.malch.com/ Shpx gur PQN. |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~