Invoking Python from Python

[John Henry]

Hi all,

I have a need to create a Python script on the fly from another Python
program and then execute the script so created. Do I need to invoke
Python through os.spawnl or is there a better way?

Thanks,

 

[Thomas Guettler]

Am Tue, 08 Nov 2005 08:10:25 -0800 schrieb John Henry:

Hi all,

I have a need to create a Python script on the fly from another Python
program and then execute the script so created. Do I need to invoke
Python through os.spawnl or is there a better way?

Hi,

creating source code with a script, is no good solution.

Once I had to maintain lisp code which stored its data in lisp code, too
(incl. conditions and loops). It was a nightmare.

Please explain what you want to do, and we will find a better solution.

HTH,
Thomas

 

[Cameron Laird]

.
.
.

creating source code with a script, is no good solution.

Once I had to maintain lisp code which stored its data in lisp code, too
(incl. conditions and loops). It was a nightmare.

Please explain what you want to do, and we will find a better solution.

.
.
.
Yes and no. There are times when it's justified. I ENTIRELY
agree, though, that many people who *think* that's what they
want to do simply don't understand how dynamic base Python is,
and therefore don't realize how much easier it can be to write
a single, unified application.

At this point, 'twould be appropriate to describe an instance
or two in which code generation is a good idea. While I have
some, they're tedious to make clear. Maybe I'll do so in a
follow-up ...

 

[Jeffrey Schwab]

Hi all,

I have a need to create a Python script on the fly from another Python
program and then execute the script so created. Do I need to invoke
Python through os.spawnl or is there a better way?

Could you import the generated script? This might be the way to go if,
e.g., you're generating a "template" configuration file that might
subsequently be edited by a human being.

 

[Martin Miller]

Hi all,

I have a need to create a Python script on the fly from another Python
program and then execute the script so created. Do I need to invoke
Python through os.spawnl or is there a better way?

When doing something similar to this I used the built-in 'execfile()'
function to execute the code created (and retrieve its results).

Once I had to maintain lisp code which stored its data in lisp code, too
(incl. conditions and loops). It was a nightmare.

This is exactly what my Python program does, but I've found it to be a
very powerful and useful technique while remaining relatively easy to
maintain (the generated code doesn't contain any conditionals or loops,
however). Another nice by-product is that the data stored this way is
portable to different platforms.

-Martin

 

[Mike Meyer]

Yes and no. There are times when it's justified. I ENTIRELY
agree, though, that many people who *think* that's what they
want to do simply don't understand how dynamic base Python is,
and therefore don't realize how much easier it can be to write
a single, unified application.

Yup. Python can do a lot of things directly that other languages might
solve with code that writes code. However, that's a *very* powerful
technic, and not everything it does can be done with lesser tools. On
the other hand, it's a *very* powerful technic, and abusing it can
easilyi create unmaintainable code.

At this point, 'twould be appropriate to describe an instance
or two in which code generation is a good idea. While I have
some, they're tedious to make clear. Maybe I'll do so in a
follow-up ...

Since Cameron didn't provide examples, let me grab a simple one. The
cheetah templating system works by creating Python programs from the
template. The programs, when run, output the "filled in" template. The
templates are generally more maintainable than the raw python - even
if you cleaned up all the things Cheetah does to make writing
templates easier. This model makes it possible for Cheetah templates
use inheritance - they can inherit from each other, from python
classes, and python classes can inherit from them.

<mike

 

[Cameron Laird]

.
.
.

Since Cameron didn't provide examples, let me grab a simple one. The
cheetah templating system works by creating Python programs from the
template. The programs, when run, output the "filled in" template. The
templates are generally more maintainable than the raw python - even
if you cleaned up all the things Cheetah does to make writing
templates easier. This model makes it possible for Cheetah templates
use inheritance - they can inherit from each other, from python
classes, and python classes can inherit from them.

.
.
.
Good example. Excellent one, even, for emphasizing the place
of inheritance in the design.

Functionalism, space-time, security, persistence, duality, ...
I have trouble talking in this area without starting to froth.
An unsatisfying treatment of some of these issues appears in
<URL: http://www.unixreview.com/documents/s=9884/ur0509m/ >.

I'll rein myself in and suggest an even easier introduction
to this subject: configuration files. RARELY is the correct
answer to create a new syntax, although many development
organizations give the impression that's their first choice.
".ini"-speak is a safe-enough choice. Most interesting,
though, is to interpret Python or some subset as a configu-
ration specification, so that one has immediately not just
HOME = "/some/folder"
STEP_LIMIT = 16
but
pool_size = cpu_count * 30
and even
if today == "Sunday":
total_process_maximum = 8
available in the configuration language. Neat, eh?

But if configuration is *that* powerful, then it can also
do great damage. How does one make Python interpretation safe?

That's a subject for another day.

 

[B Mahoney]

I also think something along the lines of execfile() may serve the
original poster. There was a thread last month about compile()
and exec() with a concise example from Fredrik Lundh.
Google "Changing an AST" in this group.

With dynamically generated code I prefer the separate compile()
step so that I can catch the compile exceptions separately.

 

[Mike Meyer]

I'll rein myself in and suggest an even easier introduction
to this subject: configuration files. RARELY is the correct
answer to create a new syntax, although many development
organizations give the impression that's their first choice.
".ini"-speak is a safe-enough choice. Most interesting,
though, is to interpret Python or some subset as a configu-
ration specification, so that one has immediately not just
HOME = "/some/folder"
STEP_LIMIT = 16
but
pool_size = cpu_count * 30
and even
if today == "Sunday":
total_process_maximum = 8
available in the configuration language. Neat, eh?

I once carried this a step further, and used methodless classes as a
configuration mechanism:

class PlainFoo:
# Values for a plain foo

class FancyFoo(PlainFoo):
# Overrides for just the things that are different

The program that used this created needed lots of objects, in a
variety of different flavers that were literally specified as "Just
like PlainFoo, but with ...". Doing it this way made configuring
things trivial.

At the time, I attached "configuration variables" to instances. If I
were going to do it today, I'd look into making the parent classes of
the class that implements Foo dynanmic.

plwm (an X11 window manager - sort of - built in top of python-xlib)
carries this another step further. You configure your window manager
by creating a subclass of the WindowManager (or other) class that
mixes in the features you want, and sets the attributes to control
specific features.

It's very flexible - but at this point, the "configuration file" is a
Python program, and not really suitable to use by non-programmers.

But if configuration is *that* powerful, then it can also
do great damage. How does one make Python interpretation safe?
That's a subject for another day.

We're all waiting for this, somewhat impatiently .

<mike

 

[Cameron Laird]

.
.
.

It's very flexible - but at this point, the "configuration file" is a
Python program, and not really suitable to use by non-programmers.

.
.
.
Or both. I have lived through a lot of experiences where
customers think a configuration file has a very simple
syntax--but in case of some puzzle/emergency/challenge/...,
I can tell 'em, "put 'complicated_script_that_I_can_recite_
on_the_telephone' right at the bottom, and tell me what
happens." That's saved weeks of thrashing around.

I should make that explicit: application developers, you
don't have to tell customers everything your programs do.
Your obligation is to make 'em meet requirements. If it
helps *you* that they do more, so be it.

 

[Scott David Daniels]

Cameron Laird wrote:
....

I should make that explicit: application developers, you
don't have to tell customers everything your programs do.
Your obligation is to make 'em meet requirements. If it
helps *you* that they do more, so be it.

I'd agree with the proviso that you at least inform your
customer if you are creating a security hole.