script files with python (instead of tcsh/bash)?

[Esmail]

Hello all,

I am wondering if anyone is using python to write script files?

Right now I have a bigg'ish bash/tcsh script that contain some grep/awk
command plus various files are processed and created, renamed and
moved to specific directories. I also write out some gnuplot scripts
that later get executed to generate .jpg images.

In any case, the scripts are starting to look pretty hairy and I was
wondering if it would make sense to re-write them in Python. I am not
sure how suitable it would be for this.

I've looked around the web w/o much luck for some examples but come
short. Any comments/suggestions?

Thanks,
Esmail

 

[Aahz]

I am wondering if anyone is using python to write script files?

These days, I always convert any even slightly complicated script to
Python.

I've looked around the web w/o much luck for some examples but come
short. Any comments/suggestions?

Not sure what you're looking for here -- many things you'd run an
external program for in scripting can be accomplished with Python library
calls, and for the rest, you can use the subprocess module (or os.system
if you have no acces to Python 2.4 or higher).
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"Programming language design is not a rational science. Most reasoning
about it is at best rationalization of gut feelings, and at worst plain
wrong." --GvR, python-ideas, 2009-3-1

 

[Esmail]

These days, I always convert any even slightly complicated script to
Python.

well .. that sounds encouraging ...

Not sure what you're looking for here -- many things you'd run an
external program for in scripting can be accomplished with Python library
calls, and for the rest, you can use the subprocess module (or os.system
if you have no acces to Python 2.4 or higher).

I have access to 2.5 or more recent. I guess I was looking for some
example scripts in Python and perhaps the equivalent in bash/tsch to
show some of the equivalences. I am being impatient, I guess I need to
dig into the language/library documentation a bit more on my own.

Esmail

 

[Aahz]

I have access to 2.5 or more recent. I guess I was looking for some
example scripts in Python and perhaps the equivalent in bash/tsch to
show some of the equivalences. I am being impatient, I guess I need to
dig into the language/library documentation a bit more on my own.

If you post a sample script you're trying to convert, you may get some
responses that show how different people would write it in Python.
--
Aahz ([email protected]) <*> http://www.pythoncraft.com/

"Programming language design is not a rational science. Most reasoning
about it is at best rationalization of gut feelings, and at worst plain
wrong." --GvR, python-ideas, 2009-3-1

 

[Peter Pearson]

I am wondering if anyone is using python to write script files?

If it can be done in a few simple lines of shell script,
fine: make it a shell script. But if it's more complex than
that, Python is clearer. Just my two cents.

 

[Esmail]

If you post a sample script you're trying to convert, you may get some
responses that show how different people would write it in Python.

That's a nice suggestion .. I may end up doing this after I do some
readings, just wanted to make sure this is not too outlandish of an
idea

Esmail

 

[Esmail]

If it can be done in a few simple lines of shell script,
fine: make it a shell script. But if it's more complex than
that, Python is clearer. Just my two cents.

Hi Peter,

Yes, I agree .. the complexity of the script(s) have evolved to the
stage that I was thinking moving to Python makes sense (if that's something
that Python is suitable for - which it appears it is

Cheers,
Esmail

 

[Ned Deily]

These days, I always convert any even slightly complicated script to
Python.

well .. that sounds encouraging ...>

Not sure what you're looking for here -- many things you'd run an
external program for in scripting can be accomplished with Python library
calls, and for the rest, you can use the subprocess module (or os.system
if you have no acces to Python 2.4 or higher).

I have access to 2.5 or more recent. I guess I was looking for some
example scripts in Python and perhaps the equivalent in bash/tsch to
show some of the equivalences. I am being impatient, I guess I need to
dig into the language/library documentation a bit more on my own.[/QUOTE]

Perhaps the recipe for Pyline might give you some ideas on how to write
python scripts that play well with other scripts.

<http://code.activestate.com/recipes/437932/>

 

[Esmail]

Perhaps the recipe for Pyline might give you some ideas on how to write
python scripts that play well with other scripts.

<http://code.activestate.com/recipes/437932/>

ah .. very nice .. thanks!

 

[Esmail]

Esmail wrote:

just a quick data point here -

so you might be better spending the time improving your bash skills than
doing what will be largely drudge work in a language you already know.

I'll have to think about it .. at this point I know both languages about
equally "well" .. (not really), but Python has more universal applicability,
so I thought this might be a good reason to hone my Python skills and come
up with something useful at the same time.

Thanks for the post, the more data points,the better.

Cheers,
Esmail

 

[Esmail]

Esmail wrote:

just a quick data point here -

so you might be better spending the time improving your bash skills than
doing what will be largely drudge work in a language you already know.

I'll have to think about it .. at this point I know both languages about
equally "well" .. (not really), but Python has more universal applicability,
so I thought this might be a good reason to hone my Python skills and come
up with something useful at the same time.

Thanks for the post, the more data points,the better.

Cheers,
Esmail

 

[Esmail]

Yes!

Almost any script that contains a loop I convert into python.


With python you get the advantages of a language with a very clear
philosophy which is exceptionally easy to read and write.

Add a few classes and unit tests to your scripts and they will be
better than you can ever achieve with bash (IMHO).

Hi Nick,

thanks for including the script, that really helps. Nice way of
finding files.

Two quick questions:

As a replacement for grep I would use the re module and its methods?

What about awk which I regularly use to extract fields based on position
but not column number, what should I be using in Python to do the same?

The other things I need to do consist of moving files, manipulating file
names and piping outputs of one command to the next, so I'm digging into
the documentation as much as I can.

So much to learn, so little time (but so much fun!)

Esmail

 

[MRAB]

Hi Nick,

thanks for including the script, that really helps. Nice way of
finding files.

Two quick questions:

As a replacement for grep I would use the re module and its methods?

What about awk which I regularly use to extract fields based on position
but not column number, what should I be using in Python to do the same?

Just use string slicing.

The other things I need to do consist of moving files, manipulating file
names and piping outputs of one command to the next, so I'm digging into
the documentation as much as I can.

The 'os' and 'shutil' modules.

 

[Gabriel Genellina]

Perhaps; but strings have methods too (`"abc" in line` is easier to read,
and faster, than the corresponding r.e.)

The 'os' and 'shutil' modules.

And for executing external commands with piping, I'd add the subprocess
module.

 

[Esmail]

Just use string slicing.

Would that be equivalent to splitting the string and then subscripting
on the result?

That seems to be the closest to awk .. wouldn't string slicing depend on
column numbers for the line (so to speak)i.e, specific know index values?

The 'os' and 'shutil' modules.

Excellent, thanks for the pointers. I was aware of the os module, but
didn't know about the shutil module. The more I dig into Python, the
more I like it and the more I'm impressed. (I do miss bock comments,
when you are trying out new things that is useful .. I know I can use
""" as a substitute though).

Thanks again,
Esmail

 

[Esmail]

Hi Gabriel,

Perhaps; but strings have methods too (`"abc" in line` is easier to
read, and faster, than the corresponding r.e.)

I'm a big fan of readability (which is one reason I am attracted
to Python) .. thanks for pointing out the alternative, good to have
options.

And for executing external commands with piping, I'd add the subprocess
module.

I will take a look at the subprocess module, thanks!

Esmail

 

[Esmail]

Hello again Nick,

thanks for the additional script example. I was able to put
something together where I read the whole file into a list
as a series of lines (via readlines()) and then loop through
the lines seeing if the target string was "in" the line .. seems
to have worked reasonably well.

I am sure over time I will pick up the more Python(ic?) ways of
doing things.

I presume you mean something like this

... | awk '{print $2}'

In python, assuming you've got the line in the "line" variable, then

In python an equivalent of the above would be

import fileinput
for line in fileinput.input():
print line.split()[1]

cool .. that is what I came up with too!

Note that the fileinput module is really useful for making shell
command replacements!

yes, I am going to have to process a number of files, so I'm also
looking at glob and os.walk(??)

Read up on the os module and the subprocess module. You'll find you
need to do much less piping with python as with shell because it has
almost everything you'll need built in.

Using built in functions is much quicker than fork()-ing an external
command too.

Thanks again for all the useful info and examples, this really has been
a great help.

Esmail

 

[Alan G Isaac]

I also write out some gnuplot scripts
that later get executed to generate .jpg images.

See Gnuplot.py

Alan Isaac

 

[Esmail]

See Gnuplot.py

Thanks Alan, I will!

Esmail

 

[R. David Murray]

Hello again Nick,

thanks for the additional script example. I was able to put
something together where I read the whole file into a list
as a series of lines (via readlines()) and then loop through
the lines seeing if the target string was "in" the line .. seems
to have worked reasonably well.

I am sure over time I will pick up the more Python(ic?) ways of
doing things.

Here's a more Pythonic way to do that:

with open('somefile') as f:
for line in f:
if 'somestring' in line:
#do something

In other words, you don't have to read the lines into a list first if all
you are going to do is iterate through them. (The 'with' clause closes
the file at block exit...which is overkill if this is all the program
is doing since the file will be closed at program termination anyway,
but is a good habit to get in to.)