Threaded server

[Giampaolo Rodola']

Hi,
I'm trying to run an asynchronous FTP server I wrote into a thread for
being able to run a test suite against it.
The code below is the threaded FTP server code I'm using:


--- snippet ---
class FTPd(threading.Thread):

def __init__(self):
self.active = False
threading.Thread.__init__(self)

def start(self, flag=None):
assert not self.active
self.flag = flag
threading.Thread.start(self)

def run(self):
assert not self.active
ftpd = ftpserver.FTPServer(address, ftp_handler)
if self.flag:
self.flag.set()
self.active = True
while self.active:
ftpd.server_forever(timeout=1, count=1)
ftpd.close()

def stop(self):
assert self.active
self.active = False

flag = threading.Event()
ftpd = FTPd()
ftpd.start(flag)
flag.wait() # wait for it to start
unittest.main() # run the test suite
ftpd.stop()
--- /snippet ---


Sometimes I get a strange error when all the tests have finished, the
server is stopped and Python is exiting:

----------------------------------------------------------------------
Ran 50 tests in 1.515s

OK
Exception exceptions.TypeError: "'NoneType' object is not callable" in
<bound me
thod FTPHandler.__del__ of <pyftpdlib.ftpserver.FTPHandler connected
127.0.0.1:2
249 at 0xa4b080>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in
<bound me
thod FTPServer.__del__ of <pyftpdlib.ftpserver.FTPServer listening
127.0.0.1:543
21 at 0x9e1a30>> ignored


I sincerely don't know why that happens but it's likely because I'm
not using threads properly.
My concern is that this could be caused by a sort of race condition
(e.g. Python tries to exit when ftpd.close call is not yet completed).

I tried to put a lock in the close() method for waiting the run()
method to be completed before returning but I didn't solve the
problem.
Another information, in case it could be useful, is that this seems to
happen with Python 2.3 only.
By using 2.4 and Python 2.5 I have no problems.

In such cases which is the right way for doing things?
Using setDaemon(True)?
Could someone point me in the right direction?
I've always used the asynchronous approach and dealing with threads is
a real pain for me.


Thanks in advance.

-- Giampaolo

 

[Nick Craig-Wood]

I'm trying to run an asynchronous FTP server I wrote into a thread for
being able to run a test suite against it.
The code below is the threaded FTP server code I'm using:

class FTPd(threading.Thread):

def __init__(self):
self.active = False
threading.Thread.__init__(self)

def start(self, flag=None):
assert not self.active
self.flag = flag
threading.Thread.start(self)

def run(self):
assert not self.active
ftpd = ftpserver.FTPServer(address, ftp_handler)
if self.flag:
self.flag.set()
self.active = True
while self.active:
ftpd.server_forever(timeout=1, count=1)
ftpd.close()

def stop(self):
assert self.active
self.active = False

flag = threading.Event()
ftpd = FTPd()
ftpd.start(flag)
flag.wait() # wait for it to start
unittest.main() # run the test suite
ftpd.stop()

Sometimes I get a strange error when all the tests have finished, the
server is stopped and Python is exiting:

Ran 50 tests in 1.515s

OK
Exception exceptions.TypeError: "'NoneType' object is not callable" in
<bound me
thod FTPHandler.__del__ of <pyftpdlib.ftpserver.FTPHandler connected
127.0.0.1:2
249 at 0xa4b080>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in
<bound me
thod FTPServer.__del__ of <pyftpdlib.ftpserver.FTPServer listening
127.0.0.1:543
21 at 0x9e1a30>> ignored


I sincerely don't know why that happens but it's likely because I'm
not using threads properly.
My concern is that this could be caused by a sort of race condition
(e.g. Python tries to exit when ftpd.close call is not yet
completed).

It looks like when python is shutting down, it has removed an object
the ftphandler code relies on.

I see you attempt to kill the ftp server with ftpd.stop(). That is
good, but you don't wait for the thread to finish (it might take up to
a second in ftpd.server_forever if I understand correctly).

I expect if you put a self.join() at the end of the stop() method the
problem will go away.

 

[Giampaolo Rodola']

It looks like when python is shutting down, it has removed an object
the ftphandler code relies on.

I see you attempt to kill the ftp server with ftpd.stop().  That is
good, but you don't wait for the thread to finish (it might take up to
a second in ftpd.server_forever if I understand correctly).

I expect if you put a self.join() at the end of the stop() method the
problem will go away.

Tried it but the problem remains.
The strange thing is that it sometimes happens, sometimes doesn't.

 

[Diez B. Roggisch]

Tried it but the problem remains.
The strange thing is that it sometimes happens, sometimes doesn't.

AFAIK you can safely ignore this error. It essentially stems from
non-determinism when shutting down threads.

Diez