Question re threading and serial i/o

[Frank Millman]

Hi all

I am not sure if this question is about threading or serial i/o - it
has elements of both.

I want to capture the output from a hand-held scanner and write it to
a file. The scanner communicates with a base-station, which is
connected to the server via a serial port. The scanner is programmed
to input a code and quantity, and send the result directly to the
base-station in the form 'code' <tab> 'qty' <cr>.

Here is my first attempt - quick and dirty, but intended to prove the
concept.

#---------------------------------------------
import threading

x = 1

def main():
global x
print 'starting thread'
t = threading.Thread(target=scan)
t.start()
while x:
q = raw_input("Press 'q' to quit")
if q == 'q':
x = 0
print 'thread finished'

def scan():
p = file('/dev/ttyS0')
txt = []
while x:
ch = p.read(1)
if ord(ch) == 13:
print ''.join(txt)
txt = []
else:
txt.append(ch)
p.close()
print 'done'

if __name__ == '__main__':
main()
#---------------------------------------------

I use threading so that the user can control the program via the
keyboard at the same time as the program is reading the serial port.
It works ok, but I do not know how to stop it cleanly. When the user
presses 'q' to quit, it sets a variable to tell the thread to
terminate, but the thread is blocked at the read statement, so it does
not know it is supposed to terminate.

I can think of two theoretical solutions, but I do not know if either
of them are possible.

1. Turn the 'read' of the serial port into a 'non-blocking' read by
checking if there are any bytes waiting to be read before actually
issuing the read statement. I have done this in other languages, but I
cannot find anything in the Python documentation that explains how to
do this.

2. Have the main thread forcibly terminate the thread doing the
reading. I cannot find anything in the Python documentation that
explains how to do this.

Any advice will be much appreciated.

Thanks

Frank Millman

P.S. I have found a way of implementing point 1 above, that seems to
work. If you don't mind, I would still like to post this, to get some
feedback about whether this is a good approach.

#---------------------------------------------
import threading, os, time

x = 1

def main():
global x
print 'starting thread'
t = threading.Thread(target=scan)
t.start()
while x:
q = raw_input("Press 'q' to quit")
if q == 'q':
x = 0
time.sleep(0.1)
print 'thread finished'

def scan():
p = os.open('/dev/ttyS0',os.O_RDONLY|os.O_NONBLOCK)
txt = []
while x:
try:
ch = os.read(p,1)
if ord(ch) == 13:
print ''.join(txt)
txt = []
else:
txt.append(ch)
except OSError:
time.sleep(0.1)
os.close(p)
print 'done'

if __name__ == '__main__':
main()
#---------------------------------------------

Thanks

Frank

 

[Peter Hansen]

I am not sure if this question is about threading or serial i/o - it
has elements of both.

Definitely about both, and in various forms it's a common question.

def scan():
p = file('/dev/ttyS0')
txt = []
while x:
ch = p.read(1)

The last line above is the heart of the problem...

I use threading so that the user can control the program via the
keyboard at the same time as the program is reading the serial port.
It works ok, but I do not know how to stop it cleanly. When the user
presses 'q' to quit, it sets a variable to tell the thread to
terminate, but the thread is blocked at the read statement, so it does
not know it is supposed to terminate.

I can think of two theoretical solutions, but I do not know if either
of them are possible.

1. Turn the 'read' of the serial port into a 'non-blocking' read by
checking if there are any bytes waiting to be read before actually
issuing the read statement. I have done this in other languages, but I
cannot find anything in the Python documentation that explains how to
do this.

You need to use the select.select() function with a timeout, so that
you can wake up periodically to check a flag that is set by the other
thread, asking the i/o thread to terminate. select() will return
immediately if the file has data waiting to be read, so you don't
significantly decrease the latency of the response in this way.

2. Have the main thread forcibly terminate the thread doing the
reading. I cannot find anything in the Python documentation that
explains how to do this.

Not really possible, and especially not if the i/o thread is blocked
in an external call, as it is in your case.

Variations on the above are the typical way to proceed, and generally
work fairly well.

-Peter

 

[Frank Millman]

def scan():
p = file('/dev/ttyS0')
txt = []
while x:
ch = p.read(1)

The last line above is the heart of the problem...

You need to use the select.select() function with a timeout, so that
you can wake up periodically to check a flag that is set by the other
thread, asking the i/o thread to terminate. select() will return
immediately if the file has data waiting to be read, so you don't
significantly decrease the latency of the response in this way.

Thanks a lot for the reply, Peter.

I tried select(), but I cannot get it to work properly. If I show you
what I am doing, hopefully you can point out the error of my ways.

This is my Mk 2 version, that is a bit ugly, but seems to work -
def scan():
p = os.open('/dev/ttyS0',os.O_RDONLY|os.O_NONBLOCK)
while x:
try:
print ord(os.read(p,1))
except OSError:
time.sleep(0.1)
os.close(p)

This is my Mk 3 version, using select() -
def scan():
p = file('/dev/ttyS0')
while x:
ans = select.select([p],[],[],0.1)
if ans[0]:
print ord(p.read(1))
p.close()

The scanner sends a string consisting of 'code' <tab> 'qty' <cr>. If I
scan a code of '1' and a quantity of '1', I would expect the program
to display 49 9 49 13. The Mk 2 version does this correctly.

The Mk 3 version behaves differently. After the first scan, it
displays 49. After each subsequent scan, it displays 9 49 13 49.

If anyone can explain what I am doing wrong, I will be most grateful.
In the meantime I am sticking with Mk 2, as it is doing the job.

Platform is Python 2.2.2 on Redhat 9.

Thanks in advance

Frank

 

[Peter Hansen]

This is my Mk 3 version, using select() -
def scan():
p = file('/dev/ttyS0')
while x:
ans = select.select([p],[],[],0.1)
if ans[0]:
print ord(p.read(1))
p.close()

The scanner sends a string consisting of 'code' <tab> 'qty' <cr>. If I
scan a code of '1' and a quantity of '1', I would expect the program
to display 49 9 49 13. The Mk 2 version does this correctly.

The Mk 3 version behaves differently. After the first scan, it
displays 49. After each subsequent scan, it displays 9 49 13 49.

If anyone can explain what I am doing wrong, I will be most grateful.
In the meantime I am sticking with Mk 2, as it is doing the job.

Not entirely sure... but what is "x" in the while statement?

This might be the answer though: when select() returns, it says that
there is "some" data waiting to be read, not just one byte. You
are reading only one byte, however. Depending on what "x" is, this
might mean you are going back to read another single byte only after
some other condition comes true.

Another possibility is that you have not opened the file in non-blocking
mode. I don't know what the effect of that would be on the select()
statement, but if you did use non-blocking, you could change the read()
call to get a whole bunch of data at a time, instead of only one byte.
If you did p.read(1024), for example, on a non-blocking file, you should
get back anywhere from 1 to 1024 bytes after select indicates it is
readable. If nothing else, this will speed up your final result, in
either case. (You can't do that on a block read, of course, since
it would then block until all 1024 bytes were available, which might
never happen.)

-Peter

 

[Frank Millman]

This is my Mk 3 version, using select() -
def scan():
p = file('/dev/ttyS0')
while x:
ans = select.select([p],[],[],0.1)
if ans[0]:
print ord(p.read(1))
p.close()

The scanner sends a string consisting of 'code' <tab> 'qty' <cr>. If I
scan a code of '1' and a quantity of '1', I would expect the program
to display 49 9 49 13. The Mk 2 version does this correctly.

The Mk 3 version behaves differently. After the first scan, it
displays 49. After each subsequent scan, it displays 9 49 13 49.

If anyone can explain what I am doing wrong, I will be most grateful.
In the meantime I am sticking with Mk 2, as it is doing the job.

Not entirely sure... but what is "x" in the while statement?

"x" is a global variable with a value of 1. It stays 1 until the main
thread wants to terminate the program, in which case it sets it to 0
and the secondary thread stops. This works ok.

Another possibility is that you have not opened the file in non-blocking
mode. I don't know what the effect of that would be on the select()
statement, but if you did use non-blocking, you could change the read()
call to get a whole bunch of data at a time, instead of only one byte.
If you did p.read(1024), for example, on a non-blocking file, you should
get back anywhere from 1 to 1024 bytes after select indicates it is
readable. If nothing else, this will speed up your final result, in
either case. (You can't do that on a block read, of course, since
it would then block until all 1024 bytes were available, which might
never happen.)

-Peter

This was indeed the problem.

I changed
p = file('/dev/ttyS0')
p.read(1)
to
p = os.open('/dev/ttyS0',os.O_RDONLY|os.O_NONBLOCK)
os.read(p,1)
and it behaved correctly.

Out of interest, is there another way to open a serial port in
non-blocking mode?

Reading chunks of up to 1024 works as you predicted, and should be
faster - I will follow this up.

Thanks a lot, Peter, I really appreciate your valuable input.

Frank

 

[Peter Hansen]

I changed
p = file('/dev/ttyS0')
p.read(1)
to
p = os.open('/dev/ttyS0',os.O_RDONLY|os.O_NONBLOCK)
os.read(p,1)
and it behaved correctly.

Out of interest, is there another way to open a serial port in
non-blocking mode?

I think you're supposed to use module "fcntl" after opening with
the standard builtin open() method, rather than having to resort
to os.open(). Don't know the details.

-Peter