Need advise about an application

[azrael]

I am currently working on an application and I need a advise.

I am supposed to read data from a device connected to a serial port. I
am reading data using pySerial. The devise is sending signals with a
time between two signals of one second.

The application is supposed to collect the data and put it into a
sqlite DB. wxPython GUI.
I am worried about this asspect that I have two solutions how to do it
but as I never tried it I wanted to ask for advise which one should be
easier.

solution 1):
The GUI is set into a separate thread then the part of collecting and
saving data. Well one thread for getting signals and saving data and
the other for GUI and data presentation

solution 2):
create an event which is binded to pySerial read methon. Well when the
devise sends a signal the event function is triggered.

Or maybe combine the two?

 

[Ò»Ê×Ê«]

Do you have to show these data on GUI? If so, why not use the event
pattern?
As far as I know, it's the standard way wxPython works.

BTW : If it's not complicated, I think maybe some wysiwyg solution is
better.
I use python for almost every thing, except these related to GUI.

 

[azrael]

Do you have to show these data on GUI? If so, why not use the event
pattern?
As far as I know, it's the standard way wxPython works.

Yes, I have to show the signaled data, but also create some
statistical meassurement datasets as also some graphical
represenatations and graphs.
wyPython works this way and it supports custom made events. I thought
of something like that. But I am not sure how to combine the serial
port signal to the event to call the handler. Time to google for it
But I am still not sure if to use one thread for signal receiving and
DB saving, and anotherone for statistics and other stuff because it is
very important to save the signal which comes every second. if the
statistics generation can use the rest of the time, then it should be
done.

I have absolutly no expirience with multi thread programing. I know
about them only from theory I got to learn at college. I wanted to ask
for an advice from something from someone with lots expirience, but
also to learn new stuff by example.

BTW : If it's not complicated, I think maybe some wysiwyg solution is
better.
I use python for almost every thing, except these related to GUI.

How do you mean WYSIWYG. I know what you mean but Don't know egactly
what you mean.

 

[Stef Mientki]

I am currently working on an application and I need a advise.

I am supposed to read data from a device connected to a serial port. I
am reading data using pySerial. The devise is sending signals with a
time between two signals of one second.

The application is supposed to collect the data and put it into a
sqlite DB. wxPython GUI.
I am worried about this asspect that I have two solutions how to do it
but as I never tried it I wanted to ask for advise which one should be
easier.

solution 1):
The GUI is set into a separate thread then the part of collecting and
saving data. Well one thread for getting signals and saving data and
the other for GUI and data presentation

solution 2):
create an event which is binded to pySerial read methon. Well when the
devise sends a signal the event function is triggered.

Or maybe combine the two?

I think the easiest method is to use a timer (100 ms interval),
let the timer-completion check if new data from serial available, etc.
I use this method with far larger throughputs ( 10k bytes / sec)
and it works very good.

cheers,
Stef

 

[azrael]

I guess that this is not an option because of the case that the
calculation of the needed statistics takes not always the same time
nad I am afraid tht using sleep() would after a couple of time periods
skip a meassurement.

 

[Dennis Lee Bieber]

DB saving, and anotherone for statistics and other stuff because it is
very important to save the signal which comes every second. if the
statistics generation can use the rest of the time, then it should be
done.

In computer terms, 1 second is a very long time.

If the source of the data is that regular (give or take a quarter
second) I'd probably code a repeating timed event with a 0.25s repeat
interval.

<event triggers>
check serial port for data packet (been a long time, does
pySerial buffer stuff for one? how long are these packets, and are they
identified by a line end character or pure binary?).
if packet received:
process it


As long as the event process takes less than 0.25s when a packet is
received (the empty packet case should be practically instantaneous:
test/return) you should get back to other GUI events in time for screen
updates, and without running into the start of the next timed check.
(The simplistic route would be to just schedule the next check 0.25s
from when the previous one returned, but that leads to creeping time
quantums. More complex is to compute next = 0.25 -
time-spent-processing assuming the GUI library does not
incorporate a static timed repeat call.)

If going thread-based, the thread would contain no GUI related code
(it seems easier to make the GUI the main program, and shove non-UI work
into threads). The thread would basically loop

wait for packet [blocking I/O, not polling loop]
compute updates statistics
do database update

Now, how to get the statistics to the display? You either need some
GUI event (timed again) which reads the statistics values from
<somewhere> [global structure, from a queue with the thread PUT the
updates, etc.] and formats the display update. Or, again, some event
that can be triggered asynchronously (the timer is a synchronous update
-- you've said "every xxx update the status portion of the display") so
that the status portion is only updated when the data is changed.
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[azrael]

DB saving, and anotherone for statistics and other stuff because it is
very important to save the signal which comes every second. if the
statistics generation can use the rest of the time, then it should be
done.

        In computer terms, 1 second is a very long time.

        If the source of the data is that regular (give or take a quarter
second) I'd probably code a repeating timed event with a 0.25s repeat
interval.

        <event triggers>
                check serial port for data packet (been a long time, does
pySerial buffer stuff for one? how long are these packets, and are they
identified by a line end character or pure binary?).
                if packet received:
                        process it

        As long as the event process takes less than 0.25s when a packet is
received (the empty packet case should be practically instantaneous:
test/return) you should get back to other GUI events in time for screen
updates, and without running into the start of the next timed check.
(The simplistic route would be to just schedule the next check 0.25s
from when the previous one returned, but that leads to creeping time
quantums. More complex is to compute    next = 0.25 -
time-spent-processing           assuming the GUI library does not
incorporate a static timed repeat call.)

        If going thread-based, the thread would contain no GUI related code
(it seems easier to make the GUI the main program, and shove non-UI work
into threads). The thread would basically loop

        wait for packet [blocking I/O, not polling loop]
        compute updates statistics
        do database update

        Now, how to get the statistics to the display? You either need some
GUI event (timed again) which reads the statistics values from
<somewhere> [global structure, from a queue with the thread PUT the
updates, etc.] and formats the display update. Or, again, some event
that can be triggered asynchronously (the timer is a synchronous update
-- you've said "every xxx update the status portion of the display") so
that the status portion is only updated when the data is changed.

That's what I meant but was not sure. I guess that I will be
callculating the remaining time using decorators. this should give an
elegant way.
The part about the threads looks like the right thing I needed. Today
I will be seing the serial port device so to play a bit with it but
also to see what it supports. It would be great if the meassunring
signal from the device could have a function as a trigger. this way I
could form it as a custom trigger for wxPython. I guess I will just
have to have a look at it.

Could anyone give me some good literature about custom made events in
python. Either to send me the material to my email or to give me some
links

 

[Hendrik van Rooyen]

I guess that this is not an option because of the case that the
calculation of the needed statistics takes not always the same time
nad I am afraid tht using sleep() would after a couple of time periods
skip a meassurement.

If I understand correctly what you are attempting,
I would use three threads:

one to do the serial stuff - it is responsible for getting
a slug of data in, checking for errors, doing protocol
stuff like re tries, etc.

One to store the slug in the database, making sure it
is written.

One to display the results.

Glue them together with queues.

If ever you need more performance, make the
threads processes, and replace the queues by pipes.

- Hendrik