threading/Queue: join() and task_done() not working?(deadlock)

[Gal Aviel]

Hello All,

I'm seeing strange behavior where one thread waits on a queue using join() and
the later (or so I think, according to order of the printing in STDOUT) another
thread calls task_done() on the very same queue, however the first thread does
not wake up, it keeps blocking forever.

I'm using a queue of size 1, my print's:

# @115000 : DEBUG PY wait(): waiting for task_done
# on <Queue.Queue instance at 0x416c7a8c>
# @115000 : DEBUG PY callback_cb() - 2
# @115000 : DEBUG PY callback_cb() - signaling
# task_done on <Queue.Queue instance at 0x416c7a8c>
# @115000 : DEBUG PY callback_cb() - 3

As you can see, the queue object is the same on both threads.

Any help would be greatly appreciated, since I've spent days and days on this
already !!

Python 2.5.1, SUSE LINUX Enterprise Server 9.

Thanks in advance, Gal Aviel.

 

[Raymond Hettinger]

Hello All,

I'm seeing strange behavior where one thread waits on a queue using join() and
the later (or so I think, according to order of the printing in STDOUT) another
thread calls task_done() on the very same queue, however the first thread does
not wake up, it keeps blocking forever.

I'm using a queue of size 1, my print's:

Does the problem persist with a queue size of 2?


Raymond

 

[7stud]

Hello All,

I'm seeing strange behavior where one thread waits on a queue using join() and
the later (or so I think, according to order of the printing in STDOUT) another
thread calls task_done() on the very same queue,

What is task_done()? The python docs don't list any such function.

 

[Dennis Lee Bieber]

What is task_done()? The python docs don't list any such function.

I didn't even know Queue objects /had/ a join() method... The only
..join() I'm familiar with is the one that waits for a thread to
complete... <G>
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Gal Aviel]

What is task_done()? The python docs don't list any such function.

I'm using Python 2.5 and it's in the docs ...

 

[Gal Aviel]

I didn't even know Queue objects /had/ a join() method... The only
.join() I'm familiar with is the one that waits for a thread to
complete... <G>

Well it's probably new in Python 2.5 although I'm not sure.

 

[Gal Aviel]

Does the problem persist with a queue size of 2?

Raymond

Unfortunately yes.

 

[7stud]

I'm using Python 2.5 and it's in the docs ...

According to the docs, join() unblocks only after you call task_done()
for every item that was entered into the Queue. However, just because
the Queue size is 1 doesn't mean there aren't other items waiting to
be added to the Queue. With one item in the Queue, presumably the
task-count is 1. But, if you get() an item out of the Queue and
another item is waiting to be added to the Queue, then it seems likely
that the task_count will jump to 2 immediately after calling get().
Thereafter, a call to task_done() will decrement the task_count to 1,
which means join() will continue blocking.

Please recognize that it is very difficult to debug imaginary code.
If you want some help that is more relevant, then post a short example
that demonstrates your problem. Do not post some lengthy program with
cryptic variable names.

 

[Dennis Lee Bieber]

Well it's probably new in Python 2.5 although I'm not sure.

After checking the 2.5 documents online, it is...

Apparently some means of allowing the main process to /wait/ until
the daemon thread have nothing left to work on, instead of suddenly
killing them mid-effort when the main process exits because no other
means of synchronizing was done.
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/