Create directories and modify files with Python

[deltaquattro]

Hi,

I would like to automate some simple tasks I'm doing by hand. Given a text file
foobar.fo:

073 1.819
085 2.132
100 2.456
115 2.789

I need to create the directories 073, 085, 100, 115, and copy in each directory a modified version of the text file input.in:

..
..
..
foo = 1.5 ! edit this value
..
..
..
bar = 1.5 ! this one, too
..
..
..

Tthe modification consists in substituting the number in the above lines with the value associated to the directory in the file foobar.fo. Thus, the input.in file in the directory 100 will be:

..
..
..
foo = 2.456 ! edit this value
..
..
..
bar = 2.456 ! this one, too
..
..
..

At first, I tried to write a bash script to do this. However, when and if the script will work, I'll probably want to add more features to automate some other tasks. So I thought about using some other language, to have a more flexible and mantainable code. I've been told that both Python and perl are well suited for such tasks, but unfortunately I know neither of them. Can you show me how to write the script in Python? Thanks,

Best Regards

Sergio Rossi

 

[Irmen de Jong]

Hi, 0 I would like to automate some simple tasks I'm doing by hand. Given a text file
foobar.fo:
[...]

At first, I tried to write a bash script to do this. However, when and if the script
will work, I'll probably want to add more features to automate some other tasks. So I
thought about using some other language, to have a more flexible and mantainable
code. I've been told that both Python and perl are well suited for such tasks, but
unfortunately I know neither of them. Can you show me how to write the script in
Python? Thanks,

Err.. if you don't know Python, why do you think a Python script will be more flexible
and maintainable for you?

But if you really want to go this way (and hey, why not) then first you'll have to learn
some basic Python. A good resource for this might be: http://learnpythonthehardway.org/

Focus on file input and output, string manipulation, and look in the os module for stuff
to help scanning directories (such as os.walk).

Irmen

 

[deltaquattro]

Il giorno martedì 1 maggio 2012 01:57:12 UTC+2, Irmen de Jong ha scritto:

Hi, 0 I would like to automate some simple tasks I'm doing by hand. Given a text file
foobar.fo:
[...]

At first, I tried to write a bash script to do this. However, when and if the script
will work, I'll probably want to add more features to automate some other tasks. So I
thought about using some other language, to have a more flexible and mantainable
code. I've been told that both Python and perl are well suited for suchtasks, but
unfortunately I know neither of them. Can you show me how to write the script in
Python? Thanks,

Err.. if you don't know Python, why do you think a Python script will be more flexible
and maintainable for you?

Eheh good question. The point is that, when in the past I chose to use only languages that I know, I always ended up with one of these two solutions:

1) write a "simple" shell script, which three years from now will make me curse in Sumeric because I can't make heads or tails of it. Been there, donethat.

2) with time, the shell script may become larger and require some mathematics. For example, the second column of foobar.fo consists of mass flows, which today I get from another code, but that I could compute in the script. At this point, I go for a Fortran/C code, which takes me longer to write.

Now I'd like to try another road. The problem is that each time I stumble upon a good chance to learn Python, I never have enough time to learn the language from zero, and I end up with one of two solutions above. If you can provide me with a working solution or at least some code to start with, I should be able to solve my immediate problem, and then have time to learn some more (that's how I learned shell scripting). The (possibly wrong) underlying assumption is that the problem is simple enough for one of you Python experts to solve easily, without wasting too much of his/her time. Anyway, I'm willing to pay a reasonable fee for help, provided you have Paypal.

But if you really want to go this way (and hey, why not) then first you'll have to learn
some basic Python. A good resource for this might be: http://learnpythonthehardway.org/

Ehm...name's not exactly inspiring for a newbie who's short on time

Focus on file input and output, string manipulation, and look in the os module for stuff
to help scanning directories (such as os.walk).

Irmen

Thanks for the directions. By the way, can you see my post in Google Groups? I'm not able to, and I don't know why.

Sergio

 

[rurpy]

Hi,

I would like to automate some simple tasks I'm doing by hand. Given a text file
foobar.fo:

073 1.819
085 2.132
100 2.456
115 2.789

I need to create the directories 073, 085, 100, 115, and copy in each directory a modified version of the text file input.in:

.
.
.
foo = 1.5 ! edit this value
.
.
.
bar = 1.5 ! this one, too
.
.
.

Tthe modification consists in substituting the number in the above lines with the value associated to the directory in the file foobar.fo. Thus, theinput.in file in the directory 100 will be:

.
.
.
foo = 2.456 ! edit this value
.
.
.
bar = 2.456 ! this one, too
.
.
.

At first, I tried to write a bash script to do this. However, when and ifthe script will work, I'll probably want to add more features to automate some other tasks. So I thought about using some other language, to have a more flexible and mantainable code. I've been told that both Python and perlare well suited for such tasks, but unfortunately I know neither of them. Can you show me how to write the script in Python? Thanks,

Perhaps something like this will get you started? To
keep things simple (since this is illustrative code)
there is little parameterization and no error handling.
Apologies if Google screws up the formatting too bad.

--------------------------------------------------------
from __future__ import print_function #1
import os

def main():
listf = open ('foobar.fo')
for line in listf:
dirname, param = line.strip().split() #7
make_directory (dirname, param)

def make_directory (dirname, param):
os.mkdir (dirname) #11
tmplf = open ("input.in")
newf = open (dirname + '/' + 'input.in', 'w') #13
for line in tmplf:
if line.startswith ('foo = ') or line.startswith ('bar = '): #15
line = line.replace (' 1.5 ', ' '+param+' ') #16
print (line, file=newf, end='') #17

if __name__ == '__main__': main() #19
------------------------------------------------------------

#1: Not sure whether you're using Python 2 or 3. I ran
this on Python 2.7 and think it will run on Python 3 if
you remove this line.

#7:The strip() method removes the '\n' characters from
the end of the lines as well as any other extraneous
leading or trailing whitespace. The split() method
here breaks the line into two pieces on the whitespace
in the middle. See
http://docs.python.org/library/stdtypes.html#string-methods

#11: This will create subdirectory 'dirname' relative
to the current directory of course. See
http://docs.python.org/library/os.html#os.mkdir

#13: Usually, is is more portable to use os.path.join() to
concatenate path components but since you stated you are
on Linux (and "/" works on Windows too), creating the path
with "/" is easier to follow in this example. For open()
see
http://docs.python.org/library/functions.html#open

#15: Depending on your data, you might want to use the re
(regular expression) module here if the simple string
substitution is not sufficient.

#16: For simplicity I just blindly replaced the " 1.5 "
text in the string. Depending of your files, you might
want to parameterize this of do something more robust or
sophisticated.

#17: Since we did not strip the trailing '\n' of the lines
we read from "input.in", we use "end=''" to prevent
print from adding an additional '\n'. See
http://docs.python.org/library/functions.html#print

#19: This line is required to actually get your python
file to do anything.

Hope this gets you started. I think you will find doing
this kind of thing in Python is much easier in the long
run than with bash scripts.

A decent resource for learning the basics of Python is
the standard Python tutorial:
http://docs.python.org/tutorial/index.html

 

[Irmen de Jong]

Ehm...name's not exactly inspiring for a newbie who's short on time

It's just a name. That resource is generally regarded as one of the
better books to learn Python for total beginners.

If you can program already in another language, the standard Python
tutorial might be more efficient to start from:
http://docs.python.org/tutorial/

Thanks for the directions. By the way, can you see my post in Google
Groups? I'm not able to, and I don't know why.

I don't use Google groups. I much prefer a proper news agent to read my
usenet newsgroups.

Irmen

 

[deltaquattro]

I'm leaving the thread because I cannot read any posts, apart from Irmen's. Anyway, I would like to publicly thank all who contributed, in particular rurpy who solved my problem (and kindly sent me a personal email, so that I could see his/her post

Best Regards

Sergio Rossi

 

[Hans Mulder]

from __future__ import print_function #1

------------------------------------------------------------

#1: Not sure whether you're using Python 2 or 3. I ran
this on Python 2.7 and think it will run on Python 3 if
you remove this line.

You don't have to remove that line: Python3 will accept it.
It doesn't do anything in python3, since 'print' is a function
whether or not you include that line, but for backward
compatibility, you're still allowed to say it.

Incidentally, the same is true for all __future__ features.
For example, Python3 still accepts:

from __future__ import nested_scopes

, even though it's only really needed if you're using python
2.1, since from 2.2 onwards scopes have nested with or without
that command.

HTH,

-- HansM

 

[Robert Miles]

Il giorno martedì 1 maggio 2012 01:57:12 UTC+2, Irmen de Jong ha scritto: [snip]

Focus on file input and output, string manipulation, and look in the os module for stuff
to help scanning directories (such as os.walk).

Irmen

Thanks for the directions. By the way, can you see my post in Google Groups? I'm not able to, and I don't know why.

Sergio

They may have copied the Gmail idea that you never need to see anything
anything you posted yourself.

When I post anything using Google Groups, my posts usually show but
often very slowly - as in 5 minutes after I tell it to post.

Robert Miles

 

[Prasad, Ramit]

Thanks for the directions. By the way, can you see my post in Google Groups?

I'm not able to, and I don't know why.
They may have copied the Gmail idea that you never need to see anything
anything you posted yourself.

I can see all my posts in a Gmail thread/conversation but if there are no
replies then I would have to look in All Mail. But for "normal" email
conversations it does have my posts in there.

Ramit


Ramit Prasad | JPMorgan Chase Investment Bank | Currencies Technology
712 Main Street | Houston, TX 77002
work phone: 713 - 216 - 5423

--

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.