asynchronous comunication, wxPython and threads.

[Zunbeltz Izaola]

Hi,

I have the following problem.

I'm developing a GUI program (wxPython). This program has to comunicate
(TCP) whit other program that controls a laboratory machine to do a
measurement. I have a dialog box, wiht two buttoms "Start measurement" and
"Stop". "Start" executes a function that do the measurement in the
following way.

1) My program send a socket.
2) The other program recives it an send confirmation.
3) My program waits for the confirmation and send the next when
confirmation received.

This comunication is done in a new thread not to frezee the GUI.
The problem is that when "Stop" is done (it kills the thread) some
confirmation sockets are mixed (are not receibed in the correct order
although i use tcp).

I have been told not to do comunication in a new thread; instead, I should
use asyncronus comunication.

My question:

What module should i use, asyncore, asynchat, twisted,(anohter)
How can i implement this asynchronous comunication and the ability to
stop this long run funciton (the measurement can take hours or days, so
i need a way to stop it)?
Can asynchronous comunication garantee that the confirmation socket will
arrive in the correct order (one after each sended socket)?

Thanks for your help

Zunbeltz Izaola

 

[Toby Dickenson]

This comunication is done in a new thread not to frezee the GUI.
The problem is that when "Stop" is done (it kills the thread) some
confirmation sockets are mixed (are not receibed in the correct order
although i use tcp).

I guess you are accessing the socket from both your GUI thread and
communications thread. Dont do that. An action in the GUI thread should
signal the communictions thread, then the communictions thread talks to the
socket.

I have been told not to do comunication in a new thread; instead, I should
use asyncronus comunication.

Using non-blocking sockets in the GUI thread may cause the opposite problem to
the one that led you to use threads in the first place: a blocking operation
in the GUI may freeze the communications. Maybe that isnt a problem for you.
If it is, I suggest sticking to two threads.

What module should i use, asyncore, asynchat, twisted,(anohter)

If you are talking to only one device, then using blocking sockets is a good
approach. However Ive never written an application like this that didnt need
to support a second (or third) machine sooner or later, and three
communictions threads is starting to get ugly. A framework like Twisted will
let you handle many machines in the one thread, but it still makes sense to
keep a second one for the GUI.

 

[Zunbeltz Izaola]

On Tuesday 21 June 2005 14:22, Zunbeltz Izaola wrote:


I guess you are accessing the socket from both your GUI thread and
communications thread. Dont do that. An action in the GUI thread should
signal the communictions thread, then the communictions thread talks to the
socket.

I see ..., Could be the problem that the socket is created in the GUI
thread? the function that end the theraded function (abort())
set want_abort = True
This make the Measurement() function to return. The Measurement()
funtion is called by startmeasurement() which is the threaded funciton.
After aborting i execute a function that FinalizeMeasuremnt() that
does comunication to some adjustament in the machine. Maybe i have to move
this portion to the threaded funtion.

Using non-blocking sockets in the GUI thread may cause the opposite problem to
the one that led you to use threads in the first place: a blocking operation
in the GUI may freeze the communications. Maybe that isnt a problem for you.
If it is, I suggest sticking to two threads.

Yes, i think i have to stick to two threads.

If you are talking to only one device, then using blocking sockets is a good
approach. However Ive never written an application like this that didnt need
to support a second (or third) machine sooner or later, and three
communictions threads is starting to get ugly. A framework like Twisted will
let you handle many machines in the one thread, but it still makes sense to
keep a second one for the GUI.

I didn't get this. What do you do when you have more tham one device; use thread or use
Twisted? Did you use it whit wxPython. I think thereis some problems of compatibility.


Thank you very much.

Zunbeltz

 

[Peter Hansen]

I'm developing a GUI program (wxPython). This program has to comunicate
(TCP) whit other program that controls a laboratory machine to do a
measurement. I have a dialog box, wiht two buttoms "Start measurement" and
"Stop". "Start" executes a function that do the measurement in the
following way.

1) My program send a socket.

Please clarify: what does this mean? "Sending a socket" is not a usual
way to describe TCP communications. Do you mean your program _opens_ a
socket (like a phone connection) and _sends_ some data, then waits for
data to be received from the other end? Or are you (as it almost
sounds) opening and closing sockets repeatedly for each part of the
conversation?

2) The other program recives it an send confirmation.
3) My program waits for the confirmation and send the next when
confirmation received.

This comunication is done in a new thread not to frezee the GUI.
The problem is that when "Stop" is done (it kills the thread) some
confirmation sockets are mixed (are not receibed in the correct order
although i use tcp).

I think you are using the term "socket" where you should be using
"packet". A socket is the virtual connection created by TCP. A packet
is either a single blob of data sent by the TCP code in the operating
system, or perhaps a single chunk of your own data.

If you are using a single TCP socket to send multiple packets, and you
are talking about those packets being sent out of order, it's very
unlikely and there must be another explanation. TCP _is_ reliable, and
you will not get data out of order unless you do something to screw
things up, for example by creating a race condition by doing
multithreaded code incorrectly.

I have been told not to do comunication in a new thread; instead, I should
use asyncronus comunication.

Some people advise that, but there's really nothing *wrong* with doing
this in a second thread, and in fact I do similar things all the time
with no ill effects. While async frameworks _can_ make this easier,
they could also make it harder (at least for a while) as you adjust your
brain to the new approach. Furthermore, at least in the case of
wxPython and Twisted (on Windows) there can be problems integrating the
two loops. I don't believe the latest Twisted claims to have fully
solved the problems involved yet, so you might still be required to have
a second thread for the TCP stuff.

What module should i use, asyncore, asynchat, twisted,(anohter)
How can i implement this asynchronous comunication and the ability to
stop this long run funciton (the measurement can take hours or days, so
i need a way to stop it)?

I use a non-blocking socket and select() calls in my thread, and
communicate with the GUI thread using appropriate Queue objects and
calls to PostEvent() (or CallAfter()) on the wx side of things. It's
pretty straightforward, so if you post a small piece of your application
which reproduces the problem it shouldn't be hard for someone here to
help you fix it.

Can asynchronous comunication garantee that the confirmation socket will
arrive in the correct order (one after each sended socket)?

No more so than using threads, unless your problem is caused by the
threads themselves (as I suggested above) in which case it might be
easier to just fix the problem.

-Peter

 

[Zunbeltz Izaola]

Please clarify: what does this mean? "Sending a socket" is not a usual
way to describe TCP communications. Do you mean your program _opens_ a
socket (like a phone connection) and _sends_ some data, then waits for
data to be received from the other end? Or are you (as it almost
sounds) opening and closing sockets repeatedly for each part of the
conversation?

Sorry for the lack of clarity. I opened the socket once (i don't know if
itit is important to open inside or outside the comunication thread). And
them send "packages" and wait for data.

I think you are using the term "socket" where you should be using
"packet". A socket is the virtual connection created by TCP. A packet
is either a single blob of data sent by the TCP code in the operating
system, or perhaps a single chunk of your own data.

Thanks for the vocabulary correction.

If you are using a single TCP socket to send multiple packets, and you
are talking about those packets being sent out of order, it's very
unlikely and there must be another explanation. TCP _is_ reliable, and
you will not get data out of order unless you do something to screw
things up, for example by creating a race condition by doing
multithreaded code incorrectly.

I think this is the case (see the post of Toby). I didn't try it out but I
think the problem is that i *do* comunication in both threads.

Some people advise that, but there's really nothing *wrong* with doing
this in a second thread, and in fact I do similar things all the time
with no ill effects. While async frameworks _can_ make this easier,
they could also make it harder (at least for a while) as you adjust your
brain to the new approach. Furthermore, at least in the case of
wxPython and Twisted (on Windows) there can be problems integrating the
two loops. I don't believe the latest Twisted claims to have fully
solved the problems involved yet, so you might still be required to have
a second thread for the TCP stuff.

Yes, i have read that there is problems yet.

I use a non-blocking socket and select() calls in my thread, and
communicate with the GUI thread using appropriate Queue objects and
calls to PostEvent() (or CallAfter()) on the wx side of things. It's
pretty straightforward, so if you post a small piece of your application
which reproduces the problem it shouldn't be hard for someone here to
help you fix it.

Thanks. First i would check if the problem is what Toby says.

No more so than using threads, unless your problem is caused by the
threads themselves (as I suggested above) in which case it might be
easier to just fix the problem.

-Peter

Thank you very much

Zunbeltz

 

[Peter Hansen]

I opened the socket once (i don't know if
itit is important to open inside or outside the comunication thread).

I don't believe it is important where it is opened, provided you don't
try to do things with it from two threads at a time (without adequate
protection).

I didn't try it out but I
think the problem is that i *do* comunication in both threads.

Almost certainly it is. It would be simplest to set up a worker thread
once, when the GUI thread begins, and simply send requests to it via a
Queue. It can create the socket, connect to the server, communicate,
close it down, and go back to waiting all in one place (so to speak...
of course this would be several methods all called from a top-level loop
in that thread). No chance of mis-steps.

-Peter

 

[Zunbeltz Izaola]

Almost certainly it is. It would be simplest to set up a worker thread
once, when the GUI thread begins, and simply send requests to it via a
Queue. It can create the socket, connect to the server, communicate,
close it down, and go back to waiting all in one place (so to speak...
of course this would be several methods all called from a top-level loop
in that thread). No chance of mis-steps.

I corrected the problem. A bit of the comunication was done in the
GUI thread.

Thanks again for your help.

Zunbeltz