Concurrency, I guess

[David S.]

I have a simple intranet web app whose job is to synchronize a couple of
databases. The web part is that you can change settings such as how often the
sync should happen, force the sync, or see the log. Since it runs periodically,
I want the process that does the db sync to run happily along but be aware of
when a setting is changed. So in my simple mind I envision 2 threads: start the
sync process, start the web server and hope that when the web server handles
some request and changes a setting, the sync process knows about it. The final
requirement is that the server can know if the sync process is running already
should someone try to manually start the sync.


Anyway, I have tried messing around with the threading module and referenced
some examples I have come across, but to no avail. Fundamentally I expected
that when I change a global variable in 1 thread, the new value would be picked
up in any other thread. This does not appear to be the case. (I understand that
there are lots of potential issues with this but only 1 thread ever shanges the
value anyhow.) I have not even tried blocking the sync process if it is already
going.

The candygram module (http://candygram.sourceforge.net/) might be the way for me
to go, but I would still like to understand this pretty basic question. So any
insight, advice, or references on the problem as described would be very
welcome. It seems it must be fairly common.

Peace, David S.

 

[dtecmeister]

Sometimes creating files and separate processes are all that's needed.
Write the settings into a config file with the web process and read
them from the file with the sync process. Then create a second file at
the start of the process and delete it once the process is complete.
Lastly run a procedure before the file creation to insure it doesn't
exist. If it does check the creation time. If it's older than
(however long the process should take plus a pad.) then killall on the
process, notify of the problem and procede on.

dtec

 

[Stormcoder]

Using candygram you can send the sync thread a message telling it to
reread the config file or you could send a message that changes the
setting directly. Candygram is much better than the standard threading
module which is designed after the Java threading library. Java
recently added another threading library because the old one was not
adequate.

The reason you can't simply share a global variable is that threads do
not share state. You have to go through a lot of trouble to share that
state. First you have to setup some form of interprocess communications
such as a memory mapped file, shared mem, pipe, socket etc. You then
have to implement locking and try to debug what you have done.
Debugging multi threaded programs is inherently difficult.
I would highly recommend using candygram or an asynchronous library,
since it sounds like you haven't done any multi-threaded programming
before. In candygram you can send threads messages which don't require
any synchronization or IPC setup.

 

[Grant Edwards]

Anyway, I have tried messing around with the threading module
and referenced some examples I have come across, but to no
avail. Fundamentally I expected that when I change a global
variable in 1 thread, the new value would be picked up in any
other thread.
Yup.

This does not appear to be the case.

Works fine for me:

------------------------------8<------------------------------
import threading,time

def t1():
global x
for i in range(20):
print "x =",x
time.sleep(0.2)

def t2():
global x
for i in range(20):
x += 1
time.sleep(0.2)

x = 7

threading.Thread(target=t1).start()
threading.Thread(target=t2).start()
------------------------------8<------------------------------

running the above program:

$ python te.py
x = 7
x = 8
x = 9
x = 10
x = 11
x = 12
x = 13
x = 14
x = 15
x = 16
x = 17
x = 18
x = 19
x = 20
x = 21
x = 22
x = 23
x = 24
x = 25
x = 26

 

[Grant Edwards]

Using candygram you can send the sync thread a message telling it to
reread the config file or you could send a message that changes the
setting directly. Candygram is much better than the standard threading
module which is designed after the Java threading library. Java
recently added another threading library because the old one was not
adequate.

The reason you can't simply share a global variable is that threads do
not share state.

Maybe that's true for "candygram", but it's certainly not true
for the the standard "treading" module.

You have to go through a lot of trouble to share that state.
First you have to setup some form of interprocess
communications such as a memory mapped file, shared mem, pipe,
socket etc. You then have to implement locking and try to
debug what you have done.

If that's what you have to do when using candygram, then it
sounds like a lot of work compared to the standard "threading"
module where threads all share a global address space.

Debugging multi threaded programs is inherently difficult. I
would highly recommend using candygram or an asynchronous
library, since it sounds like you haven't done any
multi-threaded programming before.

In candygram you can send threads messages which don't require
any synchronization or IPC setup.

Same with the "threading" module.