Is there an obvious way to do this in python?

[H J van Rooyen]

Hi,

I want to write a small system that is transaction based.

I want to split the GUI front end data entry away from the file handling and
record keeping.

Now it seems almost trivially easy using the sockets module to communicate
between machines on the same LAN, so that I want to do the record keeping on one
machine.

I want to keep the "server" machine as simple as possible - just doing record
keeping on a stimulus response basis - I would prefer it to do one thing at a
time to completion because this style of operation, though limited in
performance, keeps a lot of hassles out of life - a transaction has either
completed, or it has not - recovery scenarios are relatively easy...

Up to this point, I don't have a problem - my toy system can create a dummy
transaction, and I can echo it from the "server" machine, with more than one
"user" machine running - so I think it is feasible to have several tens of "data
entry terminal" systems running, served by one not very strong machine.

Now what I would really like to do is to differentiate between the 'User"
machines, so that some can do a full range of transactions, and others a limited
range.

And I would like to make this flexible, so that it becomes easy to introduce new
transactions, without having to run around updating the code in all the user
machines, with the concomitant version number hassles.

And I would like to do the whole thing in python - so my question is this - is
it possible to do the equivalent of dynamic linking? - i.e. if I keep a list of
what a user is allowed to do - can I somehow send him just the bits he needs to
do the job, without having to change the static code on his machine? - it seems
to me that the eval() thingy could possibly do this for me, by sending it data
that makes it do import statements followed by calls to whatever... - will this
work, or is there a better way?

Or has all this been done already? - and no I don't want a web server and php
and browsers and Java and html or xml... - I want to write something that works
simply and reliably - its just short message accounting type data...

- Hendrik

 

[Simon Forman]

Hi,

I want to write a small system that is transaction based.

I want to split the GUI front end data entry away from the file handling and
record keeping.

Now it seems almost trivially easy using the sockets module to communicate
between machines on the same LAN, so that I want to do the record keeping on one
machine.

I want to keep the "server" machine as simple as possible - just doing record
keeping on a stimulus response basis - I would prefer it to do one thing at a
time to completion because this style of operation, though limited in
performance, keeps a lot of hassles out of life - a transaction has either
completed, or it has not - recovery scenarios are relatively easy...

Up to this point, I don't have a problem - my toy system can create a dummy
transaction, and I can echo it from the "server" machine, with more than one
"user" machine running - so I think it is feasible to have several tens of "data
entry terminal" systems running, served by one not very strong machine.

Now what I would really like to do is to differentiate between the 'User"
machines, so that some can do a full range of transactions, and others a limited
range.

And I would like to make this flexible, so that it becomes easy to introduce new
transactions, without having to run around updating the code in all the user
machines, with the concomitant version number hassles.

And I would like to do the whole thing in python - so my question is this - is
it possible to do the equivalent of dynamic linking? - i.e. if I keep a list of
what a user is allowed to do - can I somehow send him just the bits he needs to
do the job, without having to change the static code on his machine? - it seems
to me that the eval() thingy could possibly do this for me, by sending it data
that makes it do import statements followed by calls to whatever... - will this
work, or is there a better way?

Or has all this been done already? - and no I don't want a web server and php
and browsers and Java and html or xml... - I want to write something that works
simply and reliably - its just short message accounting type data...

- Hendrik

Don't reinvent the wheel. Use a database...

You probably don't want to hear this, but what you just described is a
GUI client front-end with a database backend. The time it takes to
download, install, and learn to use, say, postgres will be similar to
the time you'd spend implementing what you've described above, but with
at least 10 to 100 times the payoff.


As for updating the client on the fly, one strategy would be to keep
the "dynamic" code in it's own module and have the clients reload()
that module when you upload a new version of it to the client machines.

Peace,
~Simon

 

[Yu-Xi Lim]

Don't reinvent the wheel. Use a database...

You probably don't want to hear this, but what you just described is a
GUI client front-end with a database backend. The time it takes to
download, install, and learn to use, say, postgres will be similar to
the time you'd spend implementing what you've described above, but with
at least 10 to 100 times the payoff.


As for updating the client on the fly, one strategy would be to keep
the "dynamic" code in it's own module and have the clients reload()
that module when you upload a new version of it to the client machines.

Yes, indeed, using a database with a GUI front end is the best way to
get "something that works simply and reliably." Writing everything in
Python isn't always the best solution. Python, however, is very good at
interfacing with most existing applications and thus makes a great "glue."

Using a proper DB system would give up transactions, multiple users, and
access control, which you have said you required, and probably more
features which you hadn't realized you needed but soon will when you
scale up beyond a toy system (optimized queries, backups, load
balancing, encrypted connections, etc).

As for updating the applications, why not just put them on the server
for each "user"/client to retrieve? There are of course several ways of
retrieving the centrally stored GUI program, and most likely you're
thinking Windows file sharing (which would require restarting the client
whenever updates are available). But don't rule out HTTP. Among the
benefits of web apps are the ability to update the application on the
fly and deploy it quickly. And no, web apps don't necessarily mean PHP,
Java or XML. You can easily use plain HTML and Python to create the GUI
and interface it with the database. AJAX may be th buzzword now, but it
isn't necessary for everything.

 

[Nick Vatamaniuc]

HJ,

As someone already posted, the backend sounds very much like a
database, so why not use a database: transactions, specific views for
different users, limited access and so on = database!
Give PostgresSQL a try...

As far as presenting a different GUI to users, you can also do it based
on the database. In other words have a common login screen and if the
usertype from the database is returned as 'restricted' draw one
interface, if it is returned as 'full' draw the full interface. Even if
the restricted user will get the full interface up it won' t be
functional because the database would restrict writes to certain
tables/columns.
Remote update of code is also possible, but you'll have to implement
some kind of update server to which you can periodically send Python
files, those files will be installed on the machine by the update
server. You can try playing with Twisted to handle the networking. Or
just write a simple script to send stuff over scp/ssh -- that's what I
would do (start the ssh server, install public keys and then just scp
stuff over to the machines assuming they are online most of the
time...).

The problem will be if something goes wrong in the updated file or with
the update server then the whole system will be down (an off-by-one
error in the GUI db client code and all of the sudden all your users
will be writing bad data to the database... all at the same time). So
you will need to do frequent backups of the database, but you probably
know this already...

Hope this helps,
Nick Vatamaniuc

 

[Bruno Desthuilliers]

H J van Rooyen a écrit :

Hi,

I want to write a small system that is transaction based.

I want to split the GUI front end data entry away from the file handling and
record keeping.

Now it seems almost trivially easy using the sockets module to communicate
between machines on the same LAN, so that I want to do the record keeping on one
machine.

I want to keep the "server" machine as simple as possible - just doing record
keeping on a stimulus response basis - I would prefer it to do one thing at a
time to completion because this style of operation, though limited in
performance, keeps a lot of hassles out of life - a transaction has either
completed, or it has not - recovery scenarios are relatively easy...

IOW, you want a SQL DBMS. May I recommand PostgreSQL ?

Up to this point, I don't have a problem - my toy system can create a dummy
transaction, and I can echo it from the "server" machine, with more than one
"user" machine running - so I think it is feasible to have several tens of "data
entry terminal" systems running, served by one not very strong machine.

Now what I would really like to do is to differentiate between the 'User"
machines, so that some can do a full range of transactions, and others a limited
range.

Any decent SQL DBMS is able to handle this. It's kind of builtin...

And I would like to make this flexible, so that it becomes easy to introduce new
transactions, without having to run around updating the code in all the user
machines, with the concomitant version number hassles.

Then you want a web front end.

And I would like to do the whole thing in python

You'll at least need bits of SQL (but SQLAlchemy may hide away most of
it) and HTML (but there are some python packages that knows how to build
HTML from declarative Python code).

- so my question is this - is
it possible to do the equivalent of dynamic linking? - i.e. if I keep a list of
what a user is allowed to do

In SQL : GRANT/REVOKE

- can I somehow send him just the bits he needs to
do the job, without having to change the static code on his machine?
HTTP/HTML.

- it seems
to me that the eval() thingy could possibly do this for me,

Err... I thought you wanted a reasonnably secure system, but I may have
misunderstood.

by sending it data
that makes it do import statements followed by calls to whatever... - will this
work, or is there a better way?

Or has all this been done already?

Yes, it's called a web frontend for a SQL DBMS. There's no shortage of
Python frameworks to do this kind of things.

- and no I don't want a web server

If you don't want Apache, there are Python-based application servers.
CherryPy comes to mind.

and php

Why PHP ?

and browsers and Java

Why Java ?

and html or xml... - I want to write something that works
simply and reliably

We do write SQL-based web apps all day long, and I can tell you they are
certainly more simple and reliable than whatever eval()-based home-made
solution we could imagine !-)

 

[H J van Rooyen]

| H J van Rooyen wrote:
| > Hi,
| >
| > I want to write a small system that is transaction based.
| >
| > I want to split the GUI front end data entry away from the file handling and
| > record keeping.
| >
| > Now it seems almost trivially easy using the sockets module to communicate
| > between machines on the same LAN, so that I want to do the record keeping on
one
| > machine.
| >
| > I want to keep the "server" machine as simple as possible - just doing
record
| > keeping on a stimulus response basis - I would prefer it to do one thing at
a
| > time to completion because this style of operation, though limited in
| > performance, keeps a lot of hassles out of life - a transaction has either
| > completed, or it has not - recovery scenarios are relatively easy...
| >
| > Up to this point, I don't have a problem - my toy system can create a dummy
| > transaction, and I can echo it from the "server" machine, with more than one
| > "user" machine running - so I think it is feasible to have several tens of
"data
| > entry terminal" systems running, served by one not very strong machine.
| >
| > Now what I would really like to do is to differentiate between the 'User"
| > machines, so that some can do a full range of transactions, and others a
limited
| > range.
| >
| > And I would like to make this flexible, so that it becomes easy to introduce
new
| > transactions, without having to run around updating the code in all the user
| > machines, with the concomitant version number hassles.
| >
| > And I would like to do the whole thing in python - so my question is this -
is
| > it possible to do the equivalent of dynamic linking? - i.e. if I keep a list
of
| > what a user is allowed to do - can I somehow send him just the bits he needs
to
| > do the job, without having to change the static code on his machine? - it
seems
| > to me that the eval() thingy could possibly do this for me, by sending it
data
| > that makes it do import statements followed by calls to whatever... - will
this
| > work, or is there a better way?
| >
| > Or has all this been done already? - and no I don't want a web server and
php
| > and browsers and Java and html or xml... - I want to write something that
works
| > simply and reliably - its just short message accounting type data...
| >
| > - Hendrik
|
| Don't reinvent the wheel. Use a database...

This believe it or not, is why I asked the question...

|
| You probably don't want to hear this, but what you just described is a
| GUI client front-end with a database backend. The time it takes to
| download, install, and learn to use, say, postgres will be similar to
| the time you'd spend implementing what you've described above, but with
| at least 10 to 100 times the payoff.
|

In a way you are right - having just read postgres vs mySQL wars in another
thread on this group - but on the other hand I am lazy and dont really want to
spend time writing ISAM or hashed random access file methods either - I did my
share of that in the late sixties and early seventies - and I have not yet done
any "research" into the capabilities of the various contenders in this arena...
*ducks*

| As for updating the client on the fly, one strategy would be to keep
| the "dynamic" code in it's own module and have the clients reload()
| that module when you upload a new version of it to the client machines.
|
| Peace,
| ~Simon

This will work - but it is unsatisfying to me - I would have kind of liked to
have the client machine not even have all the code available - having the "bits
that can change" in one module implies some kind of build for every client if
they are to be to different - what I would rather write is the kind of thing
that is implemented in banking transaction terminals - the terminal has only the
code for the transactions that it is authorised to have, and these different
modules along with their parameters can be downloaded into it and activated on
the fly, one at a time...

And how to do that efficiently in python is really the question that I would
like to have answered, if possible...

Thanks for the response, Simon.

- Hendrik

 

[H J van Rooyen]

| HJ,
|
| As someone already posted, the backend sounds very much like a
| database, so why not use a database: transactions, specific views for
| different users, limited access and so on = database!
| Give PostgresSQL a try...

*nods* - looks like I am going to have to do this....

| As far as presenting a different GUI to users, you can also do it based
| on the database. In other words have a common login screen and if the
| usertype from the database is returned as 'restricted' draw one
| interface, if it is returned as 'full' draw the full interface. Even if
| the restricted user will get the full interface up it won' t be
| functional because the database would restrict writes to certain
| tables/columns.

This is the guts of my question - if I dont know all the types now, how do I
make the front end so that I can easily update it as time reveals new
requirements - a la banking style terminals - see my reply to Simon please

| Remote update of code is also possible, but you'll have to implement
| some kind of update server to which you can periodically send Python
| files, those files will be installed on the machine by the update
| server. You can try playing with Twisted to handle the networking. Or
| just write a simple script to send stuff over scp/ssh -- that's what I
| would do (start the ssh server, install public keys and then just scp
| stuff over to the machines assuming they are online most of the
| time...).

This kind of addresses getting the stuff on to the server on the site for me - I
would like the front end to be more dynamic...

|
| The problem will be if something goes wrong in the updated file or with
| the update server then the whole system will be down (an off-by-one
| error in the GUI db client code and all of the sudden all your users
| will be writing bad data to the database... all at the same time). So
| you will need to do frequent backups of the database, but you probably
| know this already...
|
| Hope this helps,
| Nick Vatamaniuc

*grin* yes and it scares the s**t out of me - this is why I like the "do one
thing at a time to completion " approach - much easier to recover when things go
wrong...

- Hendrik

 

[H J van Rooyen]

| Simon Forman wrote:
| >> Or has all this been done already? - and no I don't want a web server and
php
| >> and browsers and Java and html or xml... - I want to write something that
works
| >> simply and reliably - its just short message accounting type data...
| >>
| >> - Hendrik
| >
| > Don't reinvent the wheel. Use a database...
| >
| > You probably don't want to hear this, but what you just described is a
| > GUI client front-end with a database backend. The time it takes to
| > download, install, and learn to use, say, postgres will be similar to
| > the time you'd spend implementing what you've described above, but with
| > at least 10 to 100 times the payoff.
| >
| >
| > As for updating the client on the fly, one strategy would be to keep
| > the "dynamic" code in it's own module and have the clients reload()
| > that module when you upload a new version of it to the client machines.
|
| Yes, indeed, using a database with a GUI front end is the best way to
| get "something that works simply and reliably." Writing everything in
| Python isn't always the best solution. Python, however, is very good at
| interfacing with most existing applications and thus makes a great "glue."
|
| Using a proper DB system would give up transactions, multiple users, and
| access control, which you have said you required, and probably more
| features which you hadn't realized you needed but soon will when you
| scale up beyond a toy system (optimized queries, backups, load
| balancing, encrypted connections, etc).

Can I not use the ssl module for encrypting the connections? - Please also
understand that the system is aimed at small to medium companies, in house -

From my perspective the only valid reason to use a database would be for the

ease of reporting - the files are not large - and the speed of a dict lookup in
python is hard to beat for normal transaction processing...

|
| As for updating the applications, why not just put them on the server
| for each "user"/client to retrieve? There are of course several ways of
| retrieving the centrally stored GUI program, and most likely you're
| thinking Windows file sharing (which would require restarting the client

NO! the last thing on my mind - want a dynamic process similar to banking
terminals - see my response to Simon please

| whenever updates are available). But don't rule out HTTP. Among the

I have been shying away from this - due to mental laziness - but obviously I
have to look at it - thanks

| benefits of web apps are the ability to update the application on the
| fly and deploy it quickly. And no, web apps don't necessarily mean PHP,
| Java or XML. You can easily use plain HTML and Python to create the GUI
| and interface it with the database. AJAX may be th buzzword now, but it
| isn't necessary for everything.

Thanks will *have* to look at html

 

[H J van Rooyen]

|H J van Rooyen a écrit :
|> Hi,
|>
|> I want to write a small system that is transaction based.
|>
|> I want to split the GUI front end data entry away from the file handling and
|> record keeping.
|>
|> Now it seems almost trivially easy using the sockets module to communicate
|> between machines on the same LAN, so that I want to do the record keeping on
one
|> machine.
|>
|> I want to keep the "server" machine as simple as possible - just doing record
|> keeping on a stimulus response basis - I would prefer it to do one thing at a
|> time to completion because this style of operation, though limited in
|> performance, keeps a lot of hassles out of life - a transaction has either
|> completed, or it has not - recovery scenarios are relatively easy...
|
|IOW, you want a SQL DBMS. May I recommand PostgreSQL ?
|

Looks like the way to go - after the argy bargy here in another thread seems
mySQL has no supporters left...

|> Up to this point, I don't have a problem - my toy system can create a dummy
|> transaction, and I can echo it from the "server" machine, with more than one
|> "user" machine running - so I think it is feasible to have several tens of
"data
|> entry terminal" systems running, served by one not very strong machine.
|>
|> Now what I would really like to do is to differentiate between the 'User"
|> machines, so that some can do a full range of transactions, and others a
limited
|> range.
|
|Any decent SQL DBMS is able to handle this. It's kind of builtin...

Yes - if you do the whole job on the server - the architecture I have mind is
more like a banking terminal scenario - please see my reply to Simon

|
|> And I would like to make this flexible, so that it becomes easy to introduce
new
|> transactions, without having to run around updating the code in all the user
|> machines, with the concomitant version number hassles.
|
|Then you want a web front end.

This seems to me to assume that the server does all the work

|
|> And I would like to do the whole thing in python
|
|You'll at least need bits of SQL (but SQLAlchemy may hide away most of
|it) and HTML (but there are some python packages that knows how to build
|HTML from declarative Python code).
|

that is good news - which packages?

|> - so my question is this - is
|> it possible to do the equivalent of dynamic linking? - i.e. if I keep a list
of
|> what a user is allowed to do
|
|In SQL : GRANT/REVOKE

again - centric thinking - I really would like to change the bits on the client,
as I explained to Simon

|
|> - can I somehow send him just the bits he needs to
|> do the job, without having to change the static code on his machine?
|
|HTTP/HTML.
|

everybody says this - I am being dragged, kicking and screaming...

|> - it seems
|> to me that the eval() thingy could possibly do this for me,
|
|Err... I thought you wanted a reasonnably secure system, but I may have
|misunderstood.
|

this is the guts of what I want - if eval is NFG then how do I implement such a
kind of "dynamic linking" of modules on the client?

|> by sending it data
|> that makes it do import statements followed by calls to whatever... - will
this
|> work, or is there a better way?
|>
|> Or has all this been done already?
|
|Yes, it's called a web frontend for a SQL DBMS. There's no shortage of
|Python frameworks to do this kind of things.
|

I kind of wanted to avoid the web based stuff - the application data is so small
and trivial, in a sense.

|> - and no I don't want a web server
|
|If you don't want Apache, there are Python-based application servers.
|CherryPy comes to mind.
|
|> and php
|
|Why PHP ?
|

seems popular

|> and browsers and Java
|
|Why Java ?
|

it addresses what I want - to dynamically and securely download bits of code and
execute them on the remote machine...

|> and html or xml... - I want to write something that works
|> simply and reliably
|
|We do write SQL-based web apps all day long, and I can tell you they are
|certainly more simple and reliable than whatever eval()-based home-made
|solution we could imagine !-)

Aah! but I would like you to stretch that imagination - to tell me how to do
some of what Java was designed to do, but better and easier, because this is
python we are talking about...

I saw something in another thread here - they were talking about weave - can I
use that if eval is nfg?

If my original post was unclear I am sorry - the point I want answered, if
possible, is how to make the client code effectively updateable on the fly -
because the answer to this will influence the whole design of the rest of the
system...

- Hendrik

 

[Yu-Xi Lim]

|
|> And I would like to make this flexible, so that it becomes easy to introduce
new
|> transactions, without having to run around updating the code in all the user
|> machines, with the concomitant version number hassles.
|
|Then you want a web front end.

This seems to me to assume that the server does all the work

Depends on what you mean by "all the work."

Operations such as filtering results are best done at the server unless
your have extremely high-bandwidth connections so that each client can
examine the entire data set and perform the operations themselves (with
minimal time of course, since your other clients may be waiting on that
transaction).

Transactions, too, will have to be supported by the server, or else you
may be left with partial transactions if a client gets disconnected
somehow as well as the need to implement complex locking systems yourself.

As for the other conditions, such as privilege and access control, I
think you'd find that centrally managed is the most manageable in the
long run.

You won't regret making it server-centric. The experts have already done
the optimizations and have ways of getting around the possible
bottleneck of having a single server perform most of the operations.

|Yes, it's called a web frontend for a SQL DBMS. There's no shortage of
|Python frameworks to do this kind of things.
|

I kind of wanted to avoid the web based stuff - the application data is so small
and trivial, in a sense.

You'd find that the python frameworks, especially those modeled after
Ruby on Rails, make creating trivial applications, such as the front-end
you describe, trivial. Go take a look at Django and TurboGears and the
others. Some have videos demonstrating stuff like how to make a
blog/wiki/other-database-app in 5 minutes. Most come with a built-in
webserver, though generally those aren't tested or guaranteed for
high-load environments.

All you need to add is a DB. Most recommend postgresql, and I'd
recommend that too, to provide the features you are looking for. Avoid
the lightweight DB systems such as Gadfly, sqlite, MS Jet/Access since
those don't have the necessary features.

 

[Bruno Desthuilliers]

|H J van Rooyen a écrit :
|> Hi,
|>
|> I want to write a small system that is transaction based.
|>
|> I want to split the GUI front end data entry away from the file handling and
|> record keeping.
|>
|> Now it seems almost trivially easy using the sockets module to communicate
|> between machines on the same LAN, so that I want to do the record keeping on
one
|> machine.
|>
|> I want to keep the "server" machine as simple as possible - just doing record
|> keeping on a stimulus response basis - I would prefer it to do one thing at a
|> time to completion because this style of operation, though limited in
|> performance, keeps a lot of hassles out of life - a transaction has either
|> completed, or it has not - recovery scenarios are relatively easy...
|
|IOW, you want a SQL DBMS. May I recommand PostgreSQL ?
|

Looks like the way to go - after the argy bargy here in another thread seems
mySQL has no supporters left...

Indeed, my choices would now be SQLite for simple things and PostgreSQL
for more serious stuff...

|> Up to this point, I don't have a problem - my toy system can create a dummy
|> transaction, and I can echo it from the "server" machine, with more than one
|> "user" machine running - so I think it is feasible to have several tens of
"data
|> entry terminal" systems running, served by one not very strong machine.
|>
|> Now what I would really like to do is to differentiate between the 'User"
|> machines, so that some can do a full range of transactions, and others a
limited
|> range.
|
|Any decent SQL DBMS is able to handle this. It's kind of builtin...

Yes - if you do the whole job on the server -

the architecture I have mind is
more like a banking terminal scenario - please see my reply to Simon

(...)
Done. And I still think that a CherryPy/PostgreSQL based solution is the
way to go.

|
|> And I would like to make this flexible, so that it becomes easy to introduce
new
|> transactions, without having to run around updating the code in all the user
|> machines, with the concomitant version number hassles.
|
|Then you want a web front end.

This seems to me to assume that the server does all the work

Not necessarily. Of course most computations are done on the server, but
what you describe here really feels like AJAX IMHO. Have a look at some
Turbogears AJAX addons like Catwalk and ModelDesigner...

|
|> And I would like to do the whole thing in python
|
|You'll at least need bits of SQL (but SQLAlchemy may hide away most of
|it) and HTML (but there are some python packages that knows how to build
|HTML from declarative Python code).
|

that is good news - which packages?
http://divmod.org/trac/wiki/DivmodNevow
http://divmod.org/trac/wiki/DivmodNevow/Athena
http://starship.python.net/crew/friedrich/HTMLgen/html/main.html
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/366000
http://dustman.net/andy/python/HyperText/
http://pyhtmloo.sourceforge.net/



|> - so my question is this - is
|> it possible to do the equivalent of dynamic linking? - i.e. if I keep a list
of
|> what a user is allowed to do
|
|In SQL : GRANT/REVOKE

again - centric thinking - I really would like to change the bits on the client,
as I explained to Simon

|
|> - can I somehow send him just the bits he needs to
|> do the job, without having to change the static code on his machine?
|
|HTTP/HTML.
|

everybody says this

Well, there must be a reason...

- I am being dragged, kicking and screaming...

|> - it seems
|> to me that the eval() thingy could possibly do this for me,
|
|Err... I thought you wanted a reasonnably secure system, but I may have
|misunderstood.
|

this is the guts of what I want - if eval is NFG then how do I implement such a
kind of "dynamic linking" of modules on the client?

You may want to look at stuff like Pyro (Python remote objects).

|> by sending it data
|> that makes it do import statements followed by calls to whatever... - will
this
|> work, or is there a better way?
|>
|> Or has all this been done already?
|
|Yes, it's called a web frontend for a SQL DBMS. There's no shortage of
|Python frameworks to do this kind of things.
|

I kind of wanted to avoid the web based stuff - the application data is so small
and trivial, in a sense.

So why even bother writing your own transaction manager, dynamic client
and protocol when the whole thing already exists - PostgreSQL,
Javascript-enabled browser and HTTP... FWIW, deploying a CherryPy
application is really trivial. I have not tested Divmod/nevow yet, but I
think you should have a look there too.

|> - and no I don't want a web server
|
|If you don't want Apache, there are Python-based application servers.
|CherryPy comes to mind.
|
|> and php
|
|Why PHP ?
|

seems popular

Yes, but why PHP ? Python is quite good for web apps.

|> and browsers and Java
|
|Why Java ?
|

it addresses what I want - to dynamically and securely download bits of code and
execute them on the remote machine...

If you really want security, you have to keep the critical parts on the
server.

|> and html or xml... - I want to write something that works
|> simply and reliably
|
|We do write SQL-based web apps all day long, and I can tell you they are
|certainly more simple and reliable than whatever eval()-based home-made
|solution we could imagine !-)

Aah! but I would like you to stretch that imagination - to tell me how to do
some of what Java was designed to do,

Initially, Java - it was named Oak by the time - was designed for
embedded programming. It happened to be a failure. Then it was renamed
to Java and "redesigned" (hum) for rich web client (applets). It still
failed there. It also failed for traditional cross-platform GUI
programming (too heavy and too autistic), and finally happened to make
it on the web server side, but even there it's a really heavy-weight
solution with no obvious practical advantage wrt/ more agile solutions
(PHP, Python, Ruby, Perl etc) IMHO.

but better and easier, because this is
python we are talking about...

indeed !-)

I saw something in another thread here - they were talking about weave
???

- can I
use that if eval is nfg?

Can't tell...

If my original post was unclear I am sorry - the point I want answered, if
possible, is how to make the client code effectively updateable on the fly -
because the answer to this will influence the whole design of the rest of the
system...

This is something I have been thinking about... IMHO what you want is
not to "update client code on the fly", but to make the client mostly a
kind of interpreter for what the server sends in. That is, the client
code itself doesn't contain any application logic, it gets it from the
server and execute it. This can certainly be done with Pyro.

Now while this may be an interesting project, I'm not really sure it's
worth the effort when we already have HTTP, HTML and AJAX...

 

[Nick Vatamaniuc]

HJ,

As far as GUI language/library goes:

Some people suggested HTML, but I think HTML is a very awkward way to
create a good looking dynamic GUI that is _both_ easy to use and fast
and easy to design (which is what you would want probably). Just to
have a nice user editable table for example, you would have to jump
through hoops (using Javascript, DOM, CSS etc), while you could do it
much easier with PyGTK and wxPython, especially if you use a gui
designer like Glade or wxGlade. Even Tkinter beats HTML as far as
building GUIs in Python goes. I believe this might change in the future
with the adaption of SVG but that will take a while... That said, if
your interface can "get by" with just buttons, text boxes, and text
areas HTML will be the best choice.


As far as "updating-on-the-fly" goes:

For the client to get the code on the fly you will have to implement
some sort of a downloader in the first place that when the user logs
in, it downloads the GUI code from the server and runs it. So if you
update the code the day before the next day they will get a different
interface, or if a new user/machine type is created you just have the
new code ready on the server and it will automatically be downloaded
and run, is that right?

Here is then how I see your use case:
1) You would define your business rules in your database, you will have
usernames, user types, access rights, data tables, columns types,
relations, views, etc...
2) Then each user type will have a specific interface GUI code kept on
the server (perhaps as a zipped binary GUI.zip in a database column
called ClientSpecificGUI).
3) The client starts your Application.py which is the same across all
clients. They will enter their username/password. The Application.py
then sends those to the server to log into the DB.
4) After successful login, the Application.py performs a SELECT query
to download the zipped GUI.py file.
5) GUI.py is unzipped and executed to start the GUI. The Application.py
code will pass the DB connection object to the GUI.py, so the GUI can
continue to talk with the database. GUI.py runs and does its magic, in
the meantime Application.py waits for GUI.py to finished and then both
exit.

Is that what you had in mind?

NOTE: This means that the client will need to have all the required
libraries at just the right versions. Imagine that your user decides to
upgrade to Python 3000 because it sounds cooler than plain old Python
2.4 , but then they won't realize that it will break your code and
they might not be able to run your application anymore. So you would
have to know at least roughly how much control over the whole client
machine you will have. Will they all be regular desktops that users
will use day to day and then once in a while launch your application
then close it, or will these all be dedicated terminals like an ATM?
The two are very different. You can assume complete control of all the
OS environment in a dedicated terminal but not in the case of a user
desktop.

Hope this helps,
Nick Vatamaniuc

 

[H J van Rooyen]

| H J van Rooyen wrote:
| > |
| > |> And I would like to make this flexible, so that it becomes easy to
introduce
| > new
| > |> transactions, without having to run around updating the code in all the
user
| > |> machines, with the concomitant version number hassles.
| > |
| > |Then you want a web front end.
| >
| > This seems to me to assume that the server does all the work
|
| Depends on what you mean by "all the work."

What I mean by this is that the server does stuff that I think belongs on the
client -
like getting involved in the nitty gritty of what the client should display -
I want the client to be smart enough to do the assembly of the elements of a
transaction
by itself, going back to the server for data only when its needed - remember
this is essentially an
accounting type data entry package - most of the stuff is typed in as text, when
processing documents from the outside, while the locally produced docs like
invoices would be generated by the system, to a large extent by making choices
amongst alternatives known to the server -

so I see the client interacting with the server quite a lot, eventually to be
able do things like auto completion of things like stock codes and descriptions,
customer details, etc. - but I don't want every keystroke flying over the LAN
and
being handled by the server...

In a sense I want to build an architecture that assembles a record from a mix of
user input and user choices out of alternatives (which unfortunately would have
to be kept on the server) and that then ships this to the server to process, and
to keep track of - such a record would be either accepted or rejected by the
server, and it would be the responsibility of the client to ensure that the
transaction is completed - like in a banking system, I envisage ways for the
client to 'poll' the server to get the state of the last transaction, to make
this possible.

|
| Operations such as filtering results are best done at the server unless
| your have extremely high-bandwidth connections so that each client can
| examine the entire data set and perform the operations themselves (with
| minimal time of course, since your other clients may be waiting on that
| transaction).
|
| Transactions, too, will have to be supported by the server, or else you
| may be left with partial transactions if a client gets disconnected
| somehow as well as the need to implement complex locking systems yourself.
|
| As for the other conditions, such as privilege and access control, I
| think you'd find that centrally managed is the most manageable in the
| long run.
|
| You won't regret making it server-centric. The experts have already done
| the optimisations and have ways of getting around the possible
| bottleneck of having a single server perform most of the operations.
|
| > |Yes, it's called a web frontend for a SQL DBMS. There's no shortage of
| > |Python frameworks to do this kind of things.
| > |
| >
| > I kind of wanted to avoid the web based stuff - the application data is so
small
| > and trivial, in a sense.
|
| You'd find that the python frameworks, especially those modeled after
| Ruby on Rails, make creating trivial applications, such as the front-end
| you describe, trivial. Go take a look at Django and TurboGears and the
| others. Some have videos demonstrating stuff like how to make a
| blog/wiki/other-database-app in 5 minutes. Most come with a built-in
| webserver, though generally those aren't tested or guaranteed for
| high-load environments.

I get the feeling I am drinking out of a fire hose... I will look.

|
| All you need to add is a DB. Most recommend postgresql, and I'd
| recommend that too, to provide the features you are looking for. Avoid
| the lightweight DB systems such as Gadfly, sqlite, MS Jet/Access since
| those don't have the necessary features.


postgres seems the way to go - if there is anything that is coming across clear,
it is this.

Have you any ideas about the "code change on the fly" requirement? or is it a
complete no no?

thanks - Hendrik

 

[H J van Rooyen]

|H J van Rooyen wrote:
|>
|>
8<-----------------------------------------------
|> |You'll at least need bits of SQL (but SQLAlchemy may hide away most of
|> |it) and HTML (but there are some python packages that knows how to build
|> |HTML from declarative Python code).
|> |
|>
|> that is good news - which packages?

http://divmod.org/trac/wiki/DivmodNevow
http://divmod.org/trac/wiki/DivmodNevow/Athena
http://starship.python.net/crew/friedrich/HTMLgen/html/main.html
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/366000
http://dustman.net/andy/python/HyperText/
http://pyhtmloo.sourceforge.net/

Thanks for the references I will try to check them all out

8<--------------------------------------------------
|> If my original post was unclear I am sorry - the point I want answered, if
|> possible, is how to make the client code effectively updateable on the fly -
|> because the answer to this will influence the whole design of the rest of the
|> system...
|
|This is something I have been thinking about... IMHO what you want is
|not to "update client code on the fly", but to make the client mostly a
|kind of interpreter for what the server sends in. That is, the client
|code itself doesn't contain any application logic, it gets it from the
|server and execute it. This can certainly be done with Pyro.
|
|Now while this may be an interesting project, I'm not really sure it's
|worth the effort when we already have HTTP, HTML and AJAX...

You may be right and it might not be worth the trouble - but what you mention
above is closer to the sort of thing I have in mind - it is essentially using
python to create a script language, and moving scripts around - but hey - python
is already a script language...

so if Pyro is for 'moving the scripts around' - Then that is what I must look at
very hard...

- thanks - Hendrik

 

[H J van Rooyen]

|HJ,
|
|As far as GUI language/library goes:
|
|Some people suggested HTML, but I think HTML is a very awkward way to
|create a good looking dynamic GUI that is _both_ easy to use and fast
|and easy to design (which is what you would want probably). Just to
|have a nice user editable table for example, you would have to jump
|through hoops (using Javascript, DOM, CSS etc), while you could do it
|much easier with PyGTK and wxPython, especially if you use a gui
|designer like Glade or wxGlade. Even Tkinter beats HTML as far as
|building GUIs in Python goes. I believe this might change in the future
|with the adaption of SVG but that will take a while... That said, if
|your interface can "get by" with just buttons, text boxes, and text
|areas HTML will be the best choice.
|

At the moment my toy system just uses Tkinter- Buttons, Labels, Entry boxes and
Listboxes (which I populate from a dict) - and it has logic in it to do only one
dummy transaction, which I just ship to the server machine where nothing
happens - it is just echoed back to the client which prints it on stdout - If I
stay with this it will be a most uninteresting GUI - but that is not the point
at issue now...

|
|As far as "updating-on-the-fly" goes:
|
|For the client to get the code on the fly you will have to implement
|some sort of a downloader in the first place that when the user logs
|in, it downloads the GUI code from the server and runs it. So if you
|update the code the day before the next day they will get a different
|interface, or if a new user/machine type is created you just have the
|new code ready on the server and it will automatically be downloaded
|and run, is that right?
|

This is broadly what I had in mind, yes - but sort of down to a transaction
level - this user does invoicing, this one enters cheques, this one does credit
notes, and their supervisor can do all three, and in a different department its
different because the jobs are different, but the invoicing GUI module is the
same for wherever its used...

|Here is then how I see your use case:
|1) You would define your business rules in your database, you will have
|usernames, user types, access rights, data tables, columns types,
|relations, views, etc...

Yes no matter how you do it, you need to keep a per user or per machine set of
what can be done. - in banking terms - a sort of Terminal Management System...

My thinking is simpler than this, because you are already thinking in "data base
speak" - now dont get me wrong - I realise that I will have to use a database -
but for me to start thinking in terms of views and stuff is kind of premature
when I dont even have a clear picture in my head of the kind of data the said
management system should keep, as I am not sure of what can, and cannot be done.

|2) Then each user type will have a specific interface GUI code kept on
|the server (perhaps as a zipped binary GUI.zip in a database column
|called ClientSpecificGUI).

I would like to split this down further - see above - so for each user there is
a sort of *pointer* to each of the kinds of transactions she can do, and these
are shipped separately and *linked* at the client side...

|3) The client starts your Application.py which is the same across all
|clients. They will enter their username/password. The Application.py
|then sends those to the server to log into the DB.

*nods*

|4) After successful login, the Application.py performs a SELECT query
|to download the zipped GUI.py file.
|5) GUI.py is unzipped and executed to start the GUI. The Application.py
|code will pass the DB connection object to the GUI.py, so the GUI can
|continue to talk with the database. GUI.py runs and does its magic, in
|the meantime Application.py waits for GUI.py to finished and then both
|exit.

|Is that what you had in mind?

Very close - I have not even thought this far - I did not think of building a
GUI for each user, I thought of building it for each transaction - kind of a
series of things like my toy - and then I got stuck on the "linking the separate
transaction guis into an app on the fly" bit, which is why I started the
thread - I really want to know if it is possible to do this sort of thing in
Python, and so far Bruno has come up with Pyro, while everybody else (including
Bruno) is beating me over the head with HTML

Now part of the reason I would like to go the transaction type route instead of
the "per user" route is robustness and maintainability, and the ability it
would give me to introduce new transaction types easily - as I see it if say an
invoice's GUI code is stable I would never have to touch it again even if I
combine it with anything else, as it would have been designed from the start to
combine with others of it's own ilk, under a kind of communications controller
that is standard...


|NOTE: This means that the client will need to have all the required
|libraries at just the right versions. Imagine that your user decides to
|upgrade to Python 3000 because it sounds cooler than plain old Python
|2.4 , but then they won't realize that it will break your code and
|they might not be able to run your application anymore. So you would
|have to know at least roughly how much control over the whole client
|machine you will have. Will they all be regular desktops that users
|will use day to day and then once in a while launch your application
|then close it, or will these all be dedicated terminals like an ATM?
|The two are very different. You can assume complete control of all the
|OS environment in a dedicated terminal but not in the case of a user
|desktop.

True - have not even considered this - this would, I imagine, vary from site to
site, and depend more on local IT policy - this is probably the strongest
argument to date against going this route, as these machines would not be as
tightly controlled as an ATM...

but then - If I want to use Python in the client at all, I would have to somehow
come to terms with this - its more the normal sort of version control that would
have to be done - after all if a user's machine is an XT running DOS - then its
going to have to be upgraded before it can be used as a terminal...

So this is more an argument against the use of Python on the client and I don't
like that...

|Hope this helps,
|Nick Vatamaniuc

8<----------------------------------------------------

Yes it does, Thanks. I feel we are getting closer to the point where I can
decide if what I am thinking of is practical or a pipe dream - it sounded so
simple - make a series of guis for a series of transactions, ship them to the
client, where there is a simple top level thingy that swallows them and ties
them all together and handles the comms to the server... I mean the language is
called Python...

- Hendrik

 

[Bruno Desthuilliers]

|> If my original post was unclear I am sorry - the point I want answered, if
|> possible, is how to make the client code effectively updateable on the fly -
|> because the answer to this will influence the whole design of the rest of the
|> system...
|
|This is something I have been thinking about... IMHO what you want is
|not to "update client code on the fly", but to make the client mostly a
|kind of interpreter for what the server sends in. That is, the client
|code itself doesn't contain any application logic, it gets it from the
|server and execute it. This can certainly be done with Pyro.
|
|Now while this may be an interesting project, I'm not really sure it's
|worth the effort when we already have HTTP, HTML and AJAX...

You may be right and it might not be worth the trouble - but what you mention
above is closer to the sort of thing I have in mind - it is essentially using
python to create a script language, and moving scripts around - but hey - python
is already a script language...

Yes, but it's not (alas) supported by browsers...

so if Pyro is for 'moving the scripts around' - Then that is what I must look at
very hard...

It's not for "moving the scripts around", it's for remote objects - kind
of like Java's RMI, but, well, much more pythonic !-). Now the point is
that Python being very powerful when it comes to dynamism and
introspection, it should be possible to have a common client that
basically just knows how to connect to the application server (using
pyro). Once connected, the client asks the server for a set of objects
(forms, menus etc) and the corresponding data. These objects then use
the same mechanism to interact with the server. It's basically similar
to the interaction between a browser and a web app - in that the client
is potentially able to run any application sent by the server -, but
with much more specialized client and app server and another protocol -
and no other language than Python.

 

[Dennis Lee Bieber]

Looks like the way to go - after the argy bargy here in another thread seems
mySQL has no supporters left...

There are a few... Or look at Firebird, MaxDB (SAP-DB)... I think
even Ingres is available for download under some conditions.

My recommendation would be to obtain as much
documentation/specifications/etc. for all of them, maybe spend a week or
so coding up "toy" programs (a mock-up of what your final needs would be
using, with maybe less than all fields defined) and see how each behaves
with the same logical data set.

MySQL's full-text search capability, while tied to the
non-transactional MyISAM table engine, may be ideal for a library to
permit keyword searching on book abstracts, say... But meaningless to a
payroll application.

--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Dennis Lee Bieber]

Can I not use the ssl module for encrypting the connections? - Please also
understand that the system is aimed at small to medium companies, in house -
ease of reporting - the files are not large - and the speed of a dict lookup in
python is hard to beat for normal transaction processing...

You might want to read the "Kode Vicious" column in a recent issue
of Queue (probably last months issue -- it's been in my carry-bag for a
few weeks).

For an "in house" effort, encrypting the LAN traffic is probably not
the most meaningful focus. Securing the data /storage/ is more important
-- why secure the LAN traffic if someone can walk off with a backup of
unsecured database. And who'd want to even spend time with a LAN sniffer
on unencrypted traffic if that same backup is available for filching.

NO! the last thing on my mind - want a dynamic process similar to banking
terminals - see my response to Simon please

? ATMs? Or internal clerk consoles?

Pretty much everything is already in the terminal software -- what
the operator has access to, and sees, is dependent upon the privileges
defined for their "account". No "dynamic" loading of code (for security,
I'd not even permit remote updates -- I'd require a floppy or CD from
inside the secure box to change operating software; as soon as you
permit updates to be pushed from outside you expose the risk of a
cracker pushing a customized code set).
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Dennis Lee Bieber]

What I mean by this is that the server does stuff that I think belongs on the
client -
like getting involved in the nitty gritty of what the client should display -
I want the client to be smart enough to do the assembly of the elements of a
transaction
by itself, going back to the server for data only when its needed - remember

One thing to consider: Where is the separation between the database
and the client. I believe most textbooks these days tend recommend:

[db-server] <-> [app-server] <-> [client]

(db-server and app-server can be the same hardware; the idea is that
clients do not have direct access to the database system, and hence the
back-end can be changed out without affecting any client... also,
clients don't need to handle database errors, etc.)

so I see the client interacting with the server quite a lot, eventually to be
able do things like auto completion of things like stock codes and descriptions,
customer details, etc. - but I don't want every keystroke flying over the LAN
and
being handled by the server...

And where did you see the client obtaining the "completion data" --
direct access to some other database tables or did you intend to
download /all/ possible data.

Typical web-based applications may have a minimal bit of data
validation running on the client (JavaScript ... things like making sure
/something/ has been entered into required fields, but not verifying
that it makes sense), and only when the user clicks on a submit is
everything sent to the application server, which then generates the
needed SQL from the data fields for submittal to the database server.

transaction is completed - like in a banking system, I envisage ways for the
client to 'poll' the server to get the state of the last transaction, to make
this possible.

Bad choice... Upon submittal to the server, there should be a
mandatory "good/bad" return code...
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Bruno Desthuilliers]

H J van Rooyen wrote:
(snip)

I would like to split this down further - see above - so for each user there is
a sort of *pointer* to each of the kinds of transactions she can do, and these
are shipped separately and *linked* at the client side...
(snip)

Now part of the reason I would like to go the transaction type route instead of
the "per user" route is robustness and maintainability,

Nothing prevents you from managing rights with a user -> allowed
transactions mapping...

BTW, note that "transaction" has a very definite meaning in DBMS jargon,
which is somewhat different from what you use this term for. This may
become a source of confusion...

|NOTE: This means that the client will need to have all the required
|libraries at just the right versions. Imagine that your user decides to
|upgrade to Python 3000 because it sounds cooler than plain old Python
|2.4 , but then they won't realize that it will break your code and
|they might not be able to run your application anymore. So you would
|have to know at least roughly how much control over the whole client
|machine you will have. Will they all be regular desktops that users
|will use day to day and then once in a while launch your application
|then close it, or will these all be dedicated terminals like an ATM?
|The two are very different. You can assume complete control of all the
|OS environment in a dedicated terminal but not in the case of a user
|desktop.

True - have not even considered this - this would, I imagine, vary from site to
site, and depend more on local IT policy - this is probably the strongest
argument to date against going this route, as these machines would not be as
tightly controlled as an ATM...

but then - If I want to use Python in the client at all, I would have to somehow
come to terms with this - its more the normal sort of version control that would
have to be done - after all if a user's machine is an XT running DOS - then its
going to have to be upgraded before it can be used as a terminal...

So this is more an argument against the use of Python on the client and I don't
like that...

Well, this is an argument against any kind of fat client whatever the
language, and one of the key reason for the growing demand for web
applications - reducing deployment problems to the minimum...