thread.error: release unlocked lock

[John P. Speno]

So you have this problem, and you decide to use threads to solve it...

System is Python 2.3.3 on Solaris 9.

I'm using the classic Python thread model. Main thread puts stuff into
a Queue.Queue() and worker threads get stuff out of it and do their thing.

On occaision, I get an exception in the main thread when it tries to
put something into the Queue.

File "/usr/local/bin/poller.py", line 47, in fragnap_it
work_queue.put((foo, bar, baz))
File "/usr/local/lib/python2.3/Queue.py", line 106, in put
self.fsema.release()
thread.error: release unlocked lock

Seems like a bad thing, and I'm not sure what can be done. Should I catch
the thread.error and retry the put()? Or just give up and shutdown nicely?
(Or quickly finish the twisted version of this code?)

Any idea on the underlying cause of this exception?

....and now you have (at least) two problems.

Thanks.

 

[Tim Peters]

[John P.Speno]

So you have this problem, and you decide to use threads to solve it...

System is Python 2.3.3 on Solaris 9.

I'm using the classic Python thread model. Main thread puts stuff into
a Queue.Queue() and worker threads get stuff out of it and do their
thing.

A bounded queue or an unbounded queue? Blocking get/put or
non-blocking get/put? About how many items are in the queue? Since
nobody has reported your symptom before, some detail or other is going
to be important. How do the standard test_thread, test_threading, and
test_queue tests fare on this box? Was this Python configured to use
pthreads or native Solaris threads?

On occaision, I get an exception in the main thread when it tries to
put something into the Queue.

File "/usr/local/bin/poller.py", line 47, in fragnap_it
work_queue.put((foo, bar, baz))
File "/usr/local/lib/python2.3/Queue.py", line 106, in put
self.fsema.release()
thread.error: release unlocked lock

Seems like a bad thing, and I'm not sure what can be done. Should I
catch the thread.error and retry the put()? Or just give up and
shutdown nicely? (Or quickly finish the twisted version of this code?)

Any idea on the underlying cause of this exception?

Something has gone insane -- this should be impossible. fsema is a
mutex. put() always acquires fsema. Before it exits, put() releases
fsema again (unless the queue is bounded and put() has just filled
it). The error you're seeing is the mutex complaining about an
attempt to release it when it's not in the acquired state (which
should be impossible because put() acquires fsema before this point).

 

[John P. Speno]

[John P.Speno]

I'm using the classic Python thread model. Main thread puts stuff into
a Queue.Queue() and worker threads get stuff out of it and do their
thing.

A bounded queue or an unbounded queue? Blocking get/put or
non-blocking get/put? About how many items are in the queue? Since

Thanks for asking. Here's what I can tell you.

Unbounded queues with non-blocking get/put. At most there are 1500 items
in the queue. There are 20 threads getting from the queue, and one
putting.

I did have bounded queues (max 40 items) and blocking puts but got the same
insane error then too.

nobody has reported your symptom before, some detail or other is going
to be important. How do the standard test_thread, test_threading, and
test_queue tests fare on this box? Was this Python configured to use
pthreads or native Solaris threads?

% grep -i thread pyconfig.h | grep -v ^/
#define HAVE_PTHREAD_H 1
#define HAVE_PTHREAD_SIGMASK 1
#define HAVE_THREAD_H 1
#define PTHREAD_SYSTEM_SCHED_SUPPORTED 1
#define SIZEOF_PTHREAD_T 4
#define WITH_THREAD 1

I think that means it is using native Solaris threads, assuming my reading
of thread.c is good.

All of test_thread, test_threading, and test_queue all passed.

 

[Tim Peters]

[John P.Speno]
[Tim Peters]

A bounded queue or an unbounded queue? Blocking get/put or
non-blocking get/put? About how many items are in the queue?
Since nobody has reported your symptom before, some detail or
other is going to be important.

[John]
Thanks for asking. Here's what I can tell you.

Unbounded queues with non-blocking get/put.

That's odd -- non-blocking put doesn't really make sense with an
unbounded queue, esp. when only one thread does put(). It will work
anyway, it's just peculiar.

So if the queue is unbounded, the put() method is the only code
anywhere that acquires or releases fsema. That simplifies analysis,
but only in that it leads most directly to the conclusion that it's
impossible to get the error you're seeing. The candidates remaining
are things like compiler optimization bugs, buggy platform threads,
buggy C extension modules.

You should really file a bug report, on Python's SourceForge tracker
-- there's already too much detail to keep straight in
email/newsgroup.

At most there are 1500 items in the queue. There are 20 threads
getting from the queue, and one putting.

I did have bounded queues (max 40 items) and blocking puts but got
the same insane error then too.

Since nobody else has reported this problem, a successful conclusion
is going to require analyzing a small-as-possible failing test case.
I'm afraid that falls on you, unless someone else can reproduce your
symptom. Here's a start, modeling one plausible set of guesses for
exactly what all the above means:

"""
import threading
import time
from Queue import Queue, Empty
from random import random

class Worker(threading.Thread):
def run(self):
while True:
try:
k = q.get_nowait()
assert k == 1
except Empty:
pass
time.sleep(random()/10.0)

q = Queue()
ts = [Worker() for dummy in range(20)]
for t in ts:
t.start()

while True:
if q.qsize() > 1500:
print q.qsize()
time.sleep(3)
q.put_nowait(1)
time.sleep(0.001)
"""

I can apparently run that all day under 2.3.4 (or 2.4c1, which has an
entirely different Queue implementation) and not see anything odd.
Does it fail for you? If not, how does it differ from what your app
does? IOW, can you fiddle it so that it does fail?

% grep -i thread pyconfig.h | grep -v ^/
#define HAVE_PTHREAD_H 1
#define HAVE_PTHREAD_SIGMASK 1
#define HAVE_THREAD_H 1
#define PTHREAD_SYSTEM_SCHED_SUPPORTED 1
#define SIZEOF_PTHREAD_T 4
#define WITH_THREAD 1

I think that means it is using native Solaris threads, assuming my
reading of thread.c is good.

I don't really know about Solaris config (another reason this would be

All of test_thread, test_threading, and test_queue all passed.

That figures. They check for sanity, but don't really count as stress tests.

 

[John P. Speno]

[John]

Unbounded queues with non-blocking get/put.

[Tim]
That's odd -- non-blocking put doesn't really make sense with an
unbounded queue, esp. when only one thread does put(). It will work
anyway, it's just peculiar.

I'll bring this issue into a bug report but I wanted to clarify on
this one issue.

My worker threads used a bare queue.get() with no arguments which does
block. I mistakenly thought the default for the block argument was False
when, in fact, it defaults to True. Sorry about that. I'll modify your
example code accordingly and start testing this asap.

Take care.