time.mktime memory access violation bug

[Bengt Richter]

Python 2.3.2 (#49, Oct 2 2003, 20:02:00) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information. [crash with popup]


I got:

The instruction at "0x7802a7ff" referenced memory at "0x00000000". The memory
could not be "read".

I would have hoped for an informative ValueError exception.

This is on NT4. My local time zone is PST.

If I let the MSVC++6 debugger try to chase it, it says
Unhandled exception in python.exe (MSVCRT.DLL): 0xC0000005: Access Violation

and fwiw without source it's pointing to the rep movs in this context:

7802A7F1 call 7802A4BF
7802A7F6 pop ecx
7802A7F7 push 9
7802A7F9 mov esi,eax
7802A7FB mov eax,dword ptr [ebp+8]
7802A7FE pop ecx
7802A7FF rep movs dword ptr [edi],dword ptr [esi]
7802A801 pop edi
7802A802 pop esi
7802A803 pop ebp
7802A804 ret
7802A805 test ecx,ecx


and esi is zero, so I guess that did it.

Someone have a debug version to check this out?

Regards,
Bengt Richter

 

[Irmen de Jong]

Python 2.3.2 (#49, Oct 2 2003, 20:02:00) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.[crash with popup]

Curious. I get (windows XP):

Python 2.3.2 (#49, Oct 2 2003, 20:02:00) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):

On Linux I get the same results. Something fishy is going
on with your Python...

--Irmen de Jong

 

[Bengt Richter]

Python 2.3.2 (#49, Oct 2 2003, 20:02:00) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import time
time.mktime((1969, 12, 31, 17, 0, 0, 0, 0, 0)) 3600.0
time.mktime((1969, 12, 31, 17, 0, 0, 0, 0, 1)) 0.0
time.mktime((1969, 12, 31, 16, 0, 0, 0, 0, 1))

[crash with popup]

Curious. I get (windows XP):

Python 2.3.2 (#49, Oct 2 2003, 20:02:00) [MSC v.1200 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.Traceback (most recent call last):
Traceback (most recent call last):
Traceback (most recent call last):

On Linux I get the same results. Something fishy is going
on with your Python...

Well, you're in a different time zone. I think you have to compensate for
that in setting up the equivalent of zero time epoch. Then hit that with dst=1.
Or there could be something fishy ;-)

Regards,
Bengt Richter

 

[Peter Otten]

Python 2.3.2 (#49, Oct 2 2003, 20:02:00) [MSC v.1200 32 bit (Intel)] on
win32 Type "help", "copyright", "credits" or "license" for more
information.[crash with popup]

More data points:

import time
n = 1000000
for i in range(-n, n):
lt = time.localtime(i)
try:
time.mktime(lt)
except OverflowError:
print "Cannot cope with %r %r" % (i, lt)

Python 2.3.2 (#1, Oct 21 2003, 10:03:19)
[GCC 3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Now omitting localtime():
Traceback (most recent call last):

So, on Linux:

- times < 0 seem to be OK.
- but -1 is used (abused?) as an error value

Attempts to provoke an OverflowError with the above technique for the lower
limit failed due to the following behaviour of localtime():
(1901, 12, 13, 21, 45, 52, 4, 347, 0)

I. e. always the same for low values, so, "guessing" the lower limit:

But mktime() seems OK:
Traceback (most recent call last):

Conclusion: The fatal error is probably due to NT's mktime() implementation.
Times before (1901, ...) should raise an Overflow error. I don't like the
-1 special value, but I am not sure how the error value of the underlying C
library and one second before the Unix epoch could be disambiguated.

Peter

 

[Bengt Richter]

Python 2.3.2 (#49, Oct 2 2003, 20:02:00) [MSC v.1200 32 bit (Intel)] on
win32 Type "help", "copyright", "credits" or "license" for more
information.

import time
time.mktime((1969, 12, 31, 17, 0, 0, 0, 0, 0)) 3600.0
time.mktime((1969, 12, 31, 17, 0, 0, 0, 0, 1)) 0.0
time.mktime((1969, 12, 31, 16, 0, 0, 0, 0, 1))

[crash with popup]

More data points:

import time
n = 1000000
for i in range(-n, n):
lt = time.localtime(i)
try:
time.mktime(lt)
except OverflowError:
print "Cannot cope with %r %r" % (i, lt)

Python 2.3.2 (#1, Oct 21 2003, 10:03:19)
[GCC 3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Now omitting localtime():
Traceback (most recent call last):

So, on Linux:

- times < 0 seem to be OK.
- but -1 is used (abused?) as an error value

Attempts to provoke an OverflowError with the above technique for the lower
limit failed due to the following behaviour of localtime():
(1901, 12, 13, 21, 45, 52, 4, 347, 0)

I. e. always the same for low values, so, "guessing" the lower limit:

But mktime() seems OK:
Traceback (most recent call last):

Conclusion: The fatal error is probably due to NT's mktime() implementation.
Times before (1901, ...) should raise an Overflow error. I don't like the
-1 special value, but I am not sure how the error value of the underlying C
library and one second before the Unix epoch could be disambiguated.

What does the following sequence do on your machine? Your tests did not apparently
exercise the daylight savings time path involved in my crash. E.g.,

>>> import time
>>> time.localtime(0) (1969, 12, 31, 16, 0, 0, 2, 365, 0)
>>> time.mktime(time.localtime(0)) 0.0
>>> time.mktime(time.localtime(0)[:6]+(0,0,1))

(my NT4 crashes here)

BTW, I wonder if mktime does anything with those zeroes that I supplied at [6:8]
Hm, well,

>>> time.mktime(time.localtime(0)[:-1]+(1,))

crashes too, so probably that's not involved.

No time to pursue this in the code, sorry ...

Regards,
Bengt Richter

 

[Peter Otten]

What does the following sequence do on your machine? Your tests did not
apparently exercise the daylight savings time path involved in my crash.

My (tacit, sorry) assumption was that setting daylight savings to 1 would
subtract 3600s from the Unix epoch, making it a negative value and thus
provoke the crash. But that was wrong:
.... tpl = time.localtime(t)
.... return time.mktime(tpl[:-3] + (0, 0, 0)) - time.mktime(tpl[:-3] +
(0, 0, 1))
....0.0

With an arbitrary summer time:

E.g.,

import time
time.localtime(0) (1969, 12, 31, 16, 0, 0, 2, 365, 0)
time.mktime(time.localtime(0)) 0.0
time.mktime(time.localtime(0)[:6]+(0,0,1))

(my NT4 crashes here)

For completeness:

time.localtime(0) (1970, 1, 1, 1, 0, 0, 3, 1, 0)
time.mktime(time.localtime(0)) 0.0
time.mktime(time.localtime(0)[:6] + (0, 0, 1)) 0.0

From the docs I would expect that such a "smart" behaviour would require a
DST flag of -1.

Looking at the two zero times (UTC + x vs UTC - x hours) I wonder if an NT
machine on this side of the zero meridian would encounter the same
problems.

Peter

 

[Bengt Richter]

What does the following sequence do on your machine? Your tests did not
apparently exercise the daylight savings time path involved in my crash.

My (tacit, sorry) assumption was that setting daylight savings to 1 would
subtract 3600s from the Unix epoch, making it a negative value and thus
provoke the crash. But that was wrong:
... tpl = time.localtime(t)
... return time.mktime(tpl[:-3] + (0, 0, 0)) - time.mktime(tpl[:-3] +
(0, 0, 1))
...0.0

With an arbitrary summer time:

d(962924461) 3600.0

E.g.,

import time
time.localtime(0)

(1969, 12, 31, 16, 0, 0, 2, 365, 0)

time.mktime(time.localtime(0)) 0.0
time.mktime(time.localtime(0)[:6]+(0,0,1))

(my NT4 crashes here)

For completeness:

time.localtime(0) (1970, 1, 1, 1, 0, 0, 3, 1, 0)
time.mktime(time.localtime(0)) 0.0
time.mktime(time.localtime(0)[:6] + (0, 0, 1)) 0.0

From the docs I would expect that such a "smart" behaviour would require a
DST flag of -1. Agreed.


Looking at the two zero times (UTC + x vs UTC - x hours) I wonder if an NT
machine on this side of the zero meridian would encounter the same
problems.

I suspect so. I think it is in the mktime.c of the MS library:I suspect that "can't get NULL" is wrong in this snippet from D:\VC98\CRT\SRC\MKTIME.C
when passed the zero epoch, because the associated localtime doesn't handle negative epoch time.

====

/*
* Convert this second count back into a time block structure.
* If localtime returns NULL, return an error.
*/
if ( (tbtemp = localtime(&tmptm1)) == NULL )
goto err_mktime;

/*
* Now must compensate for DST. The ANSI rules are to use the
* passed-in tm_isdst flag if it is non-negative. Otherwise,
* compute if DST applies. Recall that tbtemp has the time without
* DST compensation, but has set tm_isdst correctly.
*/
if ( (tb->tm_isdst > 0) || ((tb->tm_isdst < 0) &&
(tbtemp->tm_isdst > 0)) ) {
tmptm1 += _dstbias;
tbtemp = localtime(&tmptm1); /* reconvert, can't get NULL */
}

====
I think it dies a little later on
*tb = *tbtemp;
.... stepping through disassembly of optimized code with no source except the separate file
to infer what is going on ;-/

This is from MSVC++6.0 BTW, which UIAM windows python 2.3.2 still was compiled with(?)

What is Python policy when a vendor lib module causes a bug?

Regards,
Bengt Richter