threads

B

Bart Nessux

Could someone explain the concept of threads and how I might use them in
Python? I was a math major, not a CS major (theory instead of practice).
Most of my programming knowledge has grown out of system administration
and shell scripting, not professional software development.

How could I make this script threaded, and why should I, how would it
benefit me? The script already takes up 100% CPU resources, what would
using threads gain for me:

#!/usr/bin/python #change this to your Python's path

def cpu_test():
x = 0
while x < 999999999:
x = x + 1
print x
cpu_test()
cpu_test()

If the above is not suitable for threading, please give me a short
example of something that is.

Thanks!!!
 
J

Jp Calderone

Could someone explain the concept of threads and how I might use them in
Python? I was a math major, not a CS major (theory instead of practice).
Most of my programming knowledge has grown out of system administration
and shell scripting, not professional software development.

How could I make this script threaded, and why should I, how would it
benefit me? The script already takes up 100% CPU resources, what would
using threads gain for me:

#!/usr/bin/python #change this to your Python's path

def cpu_test():
x = 0
while x < 999999999:
x = x + 1
print x
cpu_test()
cpu_test()

If the above is not suitable for threading, please give me a short
example of something that is.

On a single CPU machine, threads will never improve the performance of a
CPU-bound task. On multiple CPU machines, threads may improve performance
of CPU-bound tasks, but only in certain very special circumstances.

For more details about this, see http://python.org/doc/api/threads.html

Threads are useful in ways other than exploiting multiple CPUs, though.
They can be used to make blocking APIs appear to be asynchronous, and this
is the prime use of them in Python. For example, DB-API 2.0 specifies an
interface for working with databases, but every method it specifies blocks.
Threads can be used to create a new API wrapped around DB-API which
specifies all the same operations, but with an asynchronous interface,
allowing any regular DB-API 2.0 module to be used without blocking.

This can be useful for working around third-party limitations (it would be
pretty bad if one had to rewrite every DB-API module, just to get an
asynchronous interface). Aside from this, though, threads are only
marginally useful, and can leave to non-deterministic behavior if not used
carefully. Think twice before throwing them into your programs.

Jp
 
D

Diez B. Roggisch

Hi,
If the above is not suitable for threading, please give me a short
example of something that is.

Its not suitable for threading. The reason is that threading itself doesn't
give you magically more computational power. Threading simply allows for
severeal threads of execution - hence the name - to be performed
(quasi-)parallel. Its similar to multi-processing: you can start several
processes (programs, so to speak), and they appear to work simultanously.
This is achieved by interrupting each process after a certain amount of
time, put it to sleep and continue to execute another process. This is done
at so short time-slices that it appears to the human user that everything
works in parallel.

Threads and processes differ in what scope they run in. In a modern OS like
*NIXes and e.g XP/NT, all processes run in separate portions of memory, and
can't access other processes memory. This can only achieved explicitely by
inter process communication, which is a wide topic and I won't dive into it
right here.

OTH Threads are parallel threads of execution "inside" one process. They all
share the same memory and thus variables and data-structures. The
responsibilities are now reverse: You can access everything, so to prevent
thread A from altering a datum thread B is currently working on, you have
to somehow serialize/synchronize access to that datum. This is done using
semaphores, mutexes or locks - things you can read about in the api docs.

Now when do you _need_ threads then? This easier asked than answered, and
sometimes a question of choice or taste. But I think these are some
indicators:

- you want several parts of your program to be responsive to user input
while they perform some actual work. I bet you have expirienced locked GUIs
in programs that were doing some stuff, like decompression an archive or
the like - this could be avoided by a special gui-thread that allows you to
tinker with menus/buttons while the work is done in the background.

- if you actually have a mulitprocessor-machine, the quasi from
quasi-parallel vanishes. Then one processor can execute one thread, and
another one the other. Of course this scales only up to the number of
processsors you have.

- in languages that don't features like coroutines/generators, sometimes
applicatioin logic is much more natural when expressed in different
threads, where each thread has his own call-stack, state and so on. This is
e.g. the case in JAVA. However, there _is_ a cost for that, as the so
called context-switch between threads isnt't for free, and synchronizing
can eat up precious time.

- some people use threads to handle blocking io. This is similar to the
point above - if you have a net-based protocol, its much more natural to
implemnt when you don't have to care if and when its waiting for input and
then would pass back execution to a point where your program waits for all
your sockets to return with data. Then you'd have to dispatch to the
appropriate part of the protocol-implementation. A thread just would sit on
the socket and wait until it returns. Then the logic goes on uninterrupted.

Hope that shed some light on it for you,

Diez
 
B

Bart Nessux

Thank you both for the info on threads in Python it was very helpful to
me, I've written something that I think can benefit from threading...
here is the code, I copied this from a few books and a few Web examples
and modified it to suit my test:

class trivialthread(threading.Thread):
def url_open(self):
for x in xrange(999999):
f = urllib.urlopen("http://xxx.xxx.xxx")
f.read()
f.close()
print self, x

if __name__ == '__main__':
threads = []
for x in range(15):
thread = trivialthread()
threads.append(thread)
for thread in threads:
thread.start()
while threading.activeCount() > 0:
print str(threading.activeCount()), "threads running incl. main"
time.sleep(1)

However, when this runs, the output looks like this:

16 threads running incl. main
1 threads running incl. main
1 threads running incl. main
1 threads running incl. main
1 threads running incl. main
1 threads running incl. main
1 threads running incl. main
....
....

Why does the 16 appear initially and then be replaced by 1? Are there no
longer 16 threads running? If so, why not?

TIA!!!
 
A

Aahz

Thank you both for the info on threads in Python it was very helpful to
me, I've written something that I think can benefit from threading...
here is the code, I copied this from a few books and a few Web examples
and modified it to suit my test:

class trivialthread(threading.Thread):
def url_open(self):
for x in xrange(999999):
f = urllib.urlopen("http://xxx.xxx.xxx")
f.read()
f.close()
print self, x

Rename ``url_open()`` to ``run()``. You might want to read the
slideshow on my web site, paying particular attention to the examples.
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"The joy of coding Python should be in seeing short, concise, readable
classes that express a lot of action in a small amount of clear code --
not in reams of trivial code that bores the reader to death." --GvR
 
B

Bart Nessux

Aahz said:
Rename ``url_open()`` to ``run()``. You might want to read the
slideshow on my web site, paying particular attention to the examples.

Wow, that was a simple fix... thanks for the link. Threads are indeed
powerful. It's hard to believe what a difference they can make.
Question: is this equivalent to running the function in parallel? For
example, say I open 15 shells and run the program... is that doing the
same thing that threads do?
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
474,202
Messages
2,571,057
Members
47,665
Latest member
salkete

Latest Threads

Top