Changing date backward messes up threaded sleep

[Earle Clubb]

Can anyone explain this behavior? If the system time is set backward,
the sleep method in the thread sleeps until the system clock has caught
back up to the original time. The main thread does not display this
behavior.

The following script demonstrates the issue (run as root or sudo):
--------------------
#!/usr/bin/ruby -w

t = Thread.new do
20.times do |i|
puts "TH:#{Time.now}"
sleep 1
end
end

20.times do |i|
puts "MN:#{Time.now}"
sleep 1
if i == 5
now=Time.now
now+=ARGV[0].to_f
puts "Changing time to: #{now}"
`date -s "#{now}"`
end
end

t.join
--------------------

Running the script with and arg of -4 causes the thread to sleep for 5
seconds after the time change, but the main thread keeps on ticking
along. An arg of +4 causes no problems.

Earle

 

[Robert Williams]

Bump...I also need this resolved.

This also occurs with select having a timeout...which I believe is
ruby's definition of sleep anyway.

 

[Gary Wright]

Bump...I also need this resolved.

This also occurs with select having a timeout...which I believe is
ruby's definition of sleep anyway.

What do you mean by 'also'? Your first posting was
about a timeout with select.

Ruby's sleep actually is a request to the thread scheduler to
stop the current thread until the system clock has reached
(now + delay). So if you move the clock backwards, it will take
that much longer to reach (now + delay). If you move the clock
forwards then the thread becomes runnable immediately.

This might also explain why your C version works since it is
probably calling the sleep/nanosleep system call rather than
communicating with some process based/green thread scheduler.

Gary Wright

 

[MenTaLguY]

Ruby's sleep actually is a request to the thread scheduler to
stop the current thread until the system clock has reached
(now + delay). So if you move the clock backwards, it will take
that much longer to reach (now + delay). If you move the clock
forwards then the thread becomes runnable immediately.

I think this is arguably a bug in the scheduler, but it is not
easy to remedy in a portable way.

-mental

 

[Rak Rok]

FYI was a thread about this back in March:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/241281

-rr-

 

[Gary Wright]

I think this is arguably a bug in the scheduler, but it is not
easy to remedy in a portable way.

I'd be curious to even see a non-portable way to do this. I just
think that the Unix/Posix semantics of time/timers basically
assumes monotonically increasing ticks.

If you want a process to detect decreasing time and/or skewed
ticks you are going to have to compare the clock to other
clocks (i.e. you are going to have to re-invent NTP).

I guess you could detect decreasing time easily enough but I don't
see how you can do it other than by polling and comparing timestamps.

To go back to the original posters situation, I don't think that
clock skew and/or discontinuities can be dealt with at the application
level. It has to be solved at a higher/operational level.

Gary Wright

 

[Gary Wright]

I think this is arguably a bug in the scheduler, but it is not
easy to remedy in a portable way.

I'd be curious to even see a non-portable way to do this. I just
think that the Unix/Posix semantics of time/timers basically
assumes monotonically increasing ticks.

If you want a process to detect decreasing time and/or skewed
ticks you are going to have to compare the clock to other
clocks (i.e. you are going to have to re-invent NTP).

I guess you could detect decreasing time easily enough but I don't
see how you can do it other than by polling and comparing timestamps.

To go back to the original posters situation, I don't think that
clock skew and/or discontinuities can be dealt with at the application
level. It has to be solved at a higher/operational level.

Gary Wright

 

[Gary Wright]

I'd be curious to even see a non-portable way to do this. I just
think that the Unix/Posix semantics of time/timers basically
assumes monotonically increasing ticks.

Maybe I spoke to quickly. I guess it depends on the types
of timers provided by the OS. You would have to use something
other than a real-time clock timer. I'm not sure what semantics
the various Posix timers and/or OS-specific timers have with
respect to the system clock changing.

Gary Wright

 

[Robert Williams]

My application is a server based scheduler...
Schedules are scheduled for a calendar year.
My admin user may test the schedule by changing the system time time and
time again.

Anyway, my work around was to have a c program send a IPC message to the
scheduler every second. At which point the scheduler would watch for
unusual time deviations and handle things accordingly...i.e. clean up a
sleeping thread.

In other scenarios, as mentioned in the ruby rails thread (which I
bumbed). Daylight savings, or a move, may require a system clock change
by the user.
It would be better to handle this case cleanly with out requireing the
user shut down all ruby daemons which may make use of select in
trheads....or require them to reboot.

my 2 cents.
Cheers!

 

[Gary Wright]

FYI was a thread about this back in March:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/241281

In that thread, Tomas Pospisek suggested switching timeofday()
to use a POSIX monotonic clock but his solution would have
screwed up Time.now since it would no longer be returning wall clock
time.

It seems like there there are at least two use cases that tend to be
obscured in this conversation:

delay by a time interval # i.e. a stopwatch, not a wall clock
delay until a time is reached # i.e. a wall clock, not a stopwatch

The problem is that many systems don't have an OS supported stopwatch,
only an OS supported wall clock.

In addition, MRI Ruby treats the wall clock as a stopwatch and so you
would have to refactor things if you had both types of clocks available.
You can't just switch everything to use the stopwatch.

I suspect there might be lots of hidden problems in application code
if you changed sleep(), for example, to behave like a stopwatch. What
if the application computes an interval based on the time of day and
then calls sleep(interval)? In that case, the application *wants*
a wall-clock model and not a stopwatch model.

Gary Wright

 

[Gary Wright]

Anyway, my work around was to have a c program send a IPC message
to the
scheduler every second. At which point the scheduler would watch for
unusual time deviations and handle things accordingly...i.e. clean
up a
sleeping thread.

Yikes! You have a scheduling system that spans an entire calendar
year that needs 1-second granularity? The fact that you even mention
a 'calendar' indicates to me that you want events to be synched to
wall-clock time anyway--so it sounds like a contradiction to want
to ignore wall-clock changes. Most likely though, I'm not aware
of the entire context you are working in.

FYI, in reading the Ruby code during this thread I think I noticed
that if there is only a single thread that sleep(x) actually uses
select() directly to wait for the elapsed time. I guess my point
is that if you really want a process to send you an IPC message
every second you can do that with Ruby also--just don't start any
other threads.

Gary Wright

 

[Robert Williams]

I don't want to waste a lot time describing my application. I have a
work around and it meets my customers' needs.

Back to the timeout issue. The interesting thing is that this problem
does not occur in the main thread. Earle's example at the top
demonstrates this. So the main thread clock acts like a stop watch and
the spawned thread acts like a wall clock waiting for the time. It
seems the should both behave the same way.

Cheers,
Robert

 

[Earle Clubb]

For the record, this is not an issue with ruby 1.9.1. The native
threads in 1.9.1 handle the negative time change just fine, whereas
1.8.6's green threads do not.

Earle