Bug in time, part 2

[Joshua J. Kugler]

Or could I entitle this, "A Wrinkle in time.py," with apologies to Madeleine
L'Engle. Either I've found a bug (or rather room for improvement) in
time.py, or I *really* need a time module that doesn't assume so much.

After the suggestions to try setting the dst flag to zero, I did some more
experimentation, and here is what I came up with:

import os
import time
os.environ['TZ'] = 'GMT'
time.tzset()
time.tzname ('GMT', 'GMT')
time.mktime((2007, 3, 11, 2, 30, 0, 6, 70, 0)) 1173580200.0
time.mktime((2007, 3, 11, 3, 30, 0, 6, 70, 0))

1173583800.0

OK, so far I'm seeing a one hour difference.
'2007-03-11 03:30:00'

Yay! it works! But, let's do this:
[0:8] + (0,)))
1173612600[0:8] + (0,)))
1173616200
OK, that looks good, a one hour difference, and I told it not to figure in
dst (the last 0). BUT:

time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(1173612600)[0:8] +

(0,))
'2007-03-11 03:30:00'

time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(1173616200)[0:8] +

(0,))
'2007-03-11 04:30:00'

So, apparently localtime assumes DST (since we're in DST right now) and
there is no way to tell it the unixtime given to it was calculated under
the assumption of no daylight savings. And would this particular test case
work if I told it to use DST? Yes, but then I'd have several hundred other
test cases failing. I think localtime needs to have a parameter to tell it
not to assume DST, even if DST would normally be active in that date range.

So, at any rate, my fix/workaround is to manually set time.py to use UTC to
convert to and from, and then deal with the times I have from there. All I
wanted to do was convert an 18 or 19 byte character string (ISO date/time)
to a four byte character string (key in a dict, can't use ints in a shelve
db), and I end up spending HOURS trying to figure out 1) why I was getting
duplicate keys, then 2) why mktime was generating duplicate unix times, and
finally 3) why localtime wasn't converting back correctly. Sigh. Some
days I hate DST.

Thanks to all for the pointers and help.

In other news, setting time.tzname = ('GMT', 'GMT') does nothing. You have
to set the environment variable, then call tzset. Is there a rational for
this? Seems odd.

j

 

[Paul Boddie]

Or could I entitle this, "A Wrinkle in time.py," with apologies to Madeleine
L'Engle. Either I've found a bug (or rather room for improvement) in
time.py, or I *really* need a time module that doesn't assume so much.

I'd be inclined to use the datetime module to avoid certain underlying
library assumptions. I originally tried to retrace your steps and make
comments about what is going on, but the UNIX time API is so bizarre
that it's better for me to suggest a few things instead.

First of all, you managed to get things working nicely in a GMT/UTC
environment. Sadly, the time module doesn't provide all the functions
you'd need to work with UTC without such hacks as calling tzset,
although I've been trying to refine some patches which do provide such
functions - see this link for more details:

https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1667546&group_id=5470

Although I suggested setting tm_isdst to zero previously, I'm more
convinced now that it's quite difficult to do the right thing with
such simple measures alone. Again, the time module doesn't provide
access to timezone offsets on individual time structures in its
current form, nor does it support the %z format option for strftime
and strptime, although the patches mentioned above do provide it; both
of these things provide the foundations to convert localtimes to UTC
times and to convert back only when needed, a bit like the way you're
supposed to work with Unicode and only produce specific textual
encodings once you're finished.

[...]

Anyway, here are some suggestions:

In your original test case (or problem), it looked like you were
seeing a DST "boundary" which then made the times appear further apart
than they actually were. Adjusting this test case for my timezone, I
can reproduce the issue but then add the %z format option (see the
patches) to demonstrate that with access to timezone offsets there's
nothing really wrong:
'2007-03-25 03:00:00 +0200'

Certainly, the localtime (with undecided timezone) of 2am on 25th
March gets converted to the DST timezone, but we can be more specific
and make informed guesses as to what happens:
'2007-03-25 03:00:00 CEST'

And trying with 1:59am gives us what we'd expect:
'2007-03-25 01:59:00 CET'

Now, we might like to avoid dealing with localtime altogether, and for
such work I've introduced a function called mktimetz which attempts to
use the timezone offset to calculate a time value which works with
localtime and gmtime mostly without surprises - something which mktime
doesn't guarantee. However, since strptime tends to sit on the fence
and not assert any knowledge about DST, and since mktimetz depends on
reliable timezone information, you do need to be careful about
supplying vague information in the first place:
'2007-03-25 05:00:00 CEST'

(I can only speculate as to what happens here, but it really isn't
worth too much consideration.)

Here, you really need mktime instead, because it will probably make
the right guesses about the presumed localtime given. However, if you
indicate the timezone, the result will be better:
'2007-03-25 01:00:00 GMT'

Here, an offset wouldn't be precise enough in the input (something
like +0100) because it wouldn't tell strptime about DST.

I could go on for a very long time about all this, and I've spent a
very long time messing around with the different time functions. My
personal opinion is that Python's time/datetime support needs a boost
if only in terms of useful additional library endorsements and better
documentation. Perhaps we just need to give the datetime module the
support it deserves and treat the time module as legacy code.

In other news, setting time.tzname = ('GMT', 'GMT') does nothing. You have
to set the environment variable, then call tzset. Is there a rational for
this? Seems odd.

It's the bizarre, even perverse way of the old UNIX time API. Various
enhancements have come along to diminish the need for such hacks, but
as you can imagine, they aren't fully standardised *and* universally
adopted.

Paul