perl to python

[Olivier Scalbert]

Hello ,

What is the python way of doing this :
perl -pi -e 's/string1/string2/' file
?
Thanks
Olivier

 

[Jarek Zgoda]

What is the python way of doing this :
perl -pi -e 's/string1/string2/' file

Use sed.

 

[Olivier Scalbert]

Olivier Scalbert <[email protected]> pisze:




Use sed.

yes, but in python ?

 

[Jarek Zgoda]

yes, but in python ?

Are you paid for doing everything in Python? This problem is much easier
to sort out by other means.

But of course, it is possible. I'm pretty sure you will get such
solution here.

 

[Donald 'Paddy' McCarthy]

yes, but in python ?

I wonder what the motivation behind your question is?
Do you have Python and not Perl or sed available?
Is the request from part of a larger conversion task?
Do you just want to compare the Perl to the Python solution?

Pad.

 

[John Roth]

Hello ,

What is the python way of doing this :
perl -pi -e 's/string1/string2/' file
?
Thanks
Olivier

I'm not sure what the -pi and -e switches do, but the
rest is fairly simple, although not as simple as the perl
one-liner.

Just load the file into a string variable, and either
use the string .replace() method, or use a regx,
depending on which is appropriate. Then write
it back out.

from the python prompt (not the command prompt)
that's something like: (untested)

var = open("file", "r").read().replace("string1", "string2")
open("file", "w").write(var)

I think this is about as obfusticated as you can get -
you'll lose the file if you try for a one-liner.

John Roth

 

[Olivier Scalbert]

I'm not sure what the -pi and -e switches do, but the
rest is fairly simple, although not as simple as the perl
one-liner.

Just load the file into a string variable, and either
use the string .replace() method, or use a regx,
depending on which is appropriate. Then write
it back out.

from the python prompt (not the command prompt)
that's something like: (untested)

var = open("file", "r").read().replace("string1", "string2")
open("file", "w").write(var)

I think this is about as obfusticated as you can get -
you'll lose the file if you try for a one-liner.

John Roth

Thx John !

 

[Jason Mobarak]

I'm not sure what the -pi and -e switches do, but the
rest is fairly simple, although not as simple as the perl
one-liner.

Just load the file into a string variable, and either
use the string .replace() method, or use a regx,
depending on which is appropriate. Then write
it back out.

from the python prompt (not the command prompt)
that's something like: (untested)

var = open("file", "r").read().replace("string1", "string2")
open("file", "w").write(var)

I think this is about as obfusticated as you can get -
you'll lose the file if you try for a one-liner.

John Roth

More obfuscated:

python -c '(lambda fp: fp.write(fp.seek(0) or
"".join([L.replace("th","ht") for L in fp])))(file("foo","rw+"))'

 

[Michael Coleman]

yes, but in python ?

Jarek's answer is the correct one, for almost any real situation.

For the purposes of exposition, though, a pythonic equivalent would
be:

import fileinput

for l in fileinput.input():
print l.replace('string1', 'string2')

If you want regular expression substitution and not just constant
strings, use re.sub instead.

Mike

 

[Steven Rumbalski]

yes, but in python ?

print 'Use sed.'

 

[Ville Vainio]

Michael> Jarek's answer is the correct one, for almost any real
Michael> situation.

Not really. Using Python is more portable, and doesn't introduce a new
dependency. And if it's trivial in Python, why introduce yet another
dependency?

 

[Kirk Job-Sluder]

Hello ,

What is the python way of doing this :
perl -pi -e 's/string1/string2/' file
?

To expand on what others have said, python emphasizes readability over
compactness and obscure shortcuts. The perl "-pi" idiom wraps everything
around a nice ammount of code, and the "-e" idiom wraps some more code.

a script that sort of has some of the same functionality would go
something like this:

#############start bad code##################
#!/usr/local/bin/python
import getopt,sys,os,re

#get your command line options
#files will be in
optlist, args = getopt.getopt(sys.argv[1:],'e:')

#do the -p loop.
for filename in args:

#do the -i "in place" edit.
oldfilename = filename+'.bak'
os.rename(filename,oldfilename)

newfile = open(filename,'w')

#continue the -p loop
for line in open(oldfilename).readlines():
#execute all of the -e statements.
for command in optlist:
#warning bad mojo here
foo=(command[1] % line.rstrip("\n"))
exec(("line=%s" % foo))
#print line
#save to the new file
print line
newfile.write(line + "\n")
newfile.close()
os.unlink(oldfilename)


############end bad code##################

The above code runs, but is not very good because I'm not that familiar
with exec statements. Anyway I've tried to capture what "perl -pi -e"
actually does which is to execute an arbitrary command over every line of an
arbitrary list of files, editing them in place, with a temporary backup
copy.

Then you would call it with something like:
python badscript.py -e 're.sub("foo","bar","%s")' badtest.txt

However this is a place where an implicit loop works great.
You can just do:
perl -pi -e 's/foo/bar/' filelist

Or if you hate the perl/sed syntax, there is:
gawk '{gsub("foo", "bar", $0); print > FILENAME}' filelist

Both of these work because perl and awk have mechanisms to implicitly
loop over all the lines in a file. The python way tends to avoid
implicit loops except for a few cases.

 

[Josef Meile]

Olivier Scalbert wrote:




print 'Use sed.'

Yes, but you're assuming that the users are using Unix/linux. What's
about the windows users? Perhaps there is a sed for windows already, but
why to bother installing it?

 

[Daniel 'Dang' Griffith]

Yes, but you're assuming that the users are using Unix/linux. What's
about the windows users? Perhaps there is a sed for windows already, but
why to bother installing it?

There's definitely a sed available, possibly even in MingW (I have it
on my system, but am not sure if it arrived with MingW or something
else I installed). It's definitely available with cygwin. One reason
to install it is that it's smaller than perl or python; another is
that it probably performs the task faster, since it isn't a general
purpose state machine; another is that it's 25% shorter to type than
perl and 50% shorter to type than python.
--dang

 

[Duncan Booth]

The above code runs, but is not very good because I'm not that
familiar with exec statements. Anyway I've tried to capture what
"perl -pi -e" actually does which is to execute an arbitrary command
over every line of an arbitrary list of files, editing them in place,
with a temporary backup copy.

Your code might have been a bit shorter if you had used the existing
facility in Python for editing files in place. The code below is completely
untested, so I can all but guarantee it doesn't work, but you get the idea:

#!/usr/local/bin/python
import getopt,sys,os,re
import fileinput

#get your command line options
#files will be in
optlist, args = getopt.getopt(sys.argv[1:],'e:')

for line in fileinput.input(args, inplace=1):
#execute all of the -e statements.
for command in optlist:
#warning bad mojo here
foo=(command[1] % line.rstrip("\n"))
exec(("line=%s" % foo))
#save to the new file
print line

fileinput.close()

 

[Oliver Fromme]

> [on sed] One reason
> to install it is that it's smaller than perl or python; another is
> that it probably performs the task faster, since it isn't a general
> purpose state machine;

FWIW, sed _is_ a state machine, although not really "general
purpose". It is a programming language with variables, loops
and conditionals, and I believe it is turing-complete. Most
of the time it is abused to perform simple search-and-replace
tasks, though. ;-)

But seriously ... I agree that the OP should really use sed
instead of Python in this particular case, for the reasons
that you've outlined.

Best regards
Oliver

--
Oliver Fromme, secnetix GmbH & Co KG, Oettingenstr. 2, 80538 Munich
Any opinions expressed in this message may be personal to the author
and may not necessarily reflect the opinions of secnetix in any way.

"Python is an experiment in how much freedom programmers need.
Too much freedom and nobody can read another's code; too little
and expressiveness is endangered." -- Guido van Rossum

 

[Roy Smith]

FWIW, sed _is_ a state machine, although not really "general
purpose". It is a programming language with variables, loops
and conditionals, and I believe it is turing-complete. Most
of the time it is abused to perform simple search-and-replace
tasks, though. ;-)

I would disagree that the "simple search-and-replace" usage is abuse.
It's just using the tool to do what it's best at. Sure, there are some
more complex things you can do in sed, but the syntax is so baroque it
quickly becomes trying to bash a screw with a hammer.

In the old days, when the task became too complicated for sed, you
switched to awk. When things got even more complex, you pasted sed,
grep, awk, and shell together in various ways, and perl was invented to
cover all those functionalities in a single language.

In a sense, perl suffers from the same disease that C++ does; a desire
to maintain backwards compatability with its parents (thus the absurdly
eclectic syntax) while at the same time adding every new feature you
could imagine (and some that you can't).

Anyway, I think there's a lot of value in learning tools like grep and
sed, and using them when appropriate. The example that started this
thread is the canonical example of what sed does best. Sure, you can
make a general-purpose tool like Python do that job, but other than
proving that you can do it, I don't see any reason to bother.

 

[Kirk Job-Sluder]

There's definitely a sed available, possibly even in MingW (I have it
on my system, but am not sure if it arrived with MingW or something
else I installed). It's definitely available with cygwin. One reason
to install it is that it's smaller than perl or python; another is
that it probably performs the task faster, since it isn't a general
purpose state machine; another is that it's 25% shorter to type than
perl and 50% shorter to type than python.

There is also a windows-native ssed (super sed).

 

[Kirk Job-Sluder]

Your code might have been a bit shorter if you had used the existing
facility in Python for editing files in place. The code below is completely
untested, so I can all but guarantee it doesn't work, but you get the idea:

#!/usr/local/bin/python
import getopt,sys,os,re
import fileinput

Thanks! Learn something new every day. I would argue that length of
code is less an issue than the nasty exec statement.

 

[Ville Vainio]

Roy> Anyway, I think there's a lot of value in learning tools like
Roy> grep and sed, and using them when appropriate. The example

I tend to think pretty much the opposite. Most of the time you can do
things as easily with Python, with the added advantage of robust
exception handling (errors not passing silently) and not having to
learn the other things. You only need to know one regexp
syntax. Windows can also be quite unpredictable w/ customary Unix
tools. Cygwin has burned me a few times too many.

The things you usually do with the non-python tools are trivial, and
trivial things have the habit of being, well, trivial in Python too.

Roy> does best. Sure, you can make a general-purpose tool like
Roy> Python do that job, but other than proving that you can do
Roy> it, I don't see any reason to bother.

You can always implement modules to do the tasks you normally use sed
or awk for. I never saw much virtue in using the most specialized (or
crippled, if you wish) tool possible. Not even if it's "optimized" for
the thing. Actually, I tend to think that Python has to some extent
deprecated that part of the Unix tradition.

It's funny, but somehow I can't really think of cases that a
specialized language would do better (ignoring the performace, which
is rarely a concern in sysadmin tasks) than Python with some
modules. Specialized languages were great at time when the general
purpose languages sucked, but that's not the case anymore.

And yes, I'm aware that I'm exposing myself to some serious flammage
from "if it was good enough for my grandad, it's good enough for me"
*nix crowd. Emotional attachment to various cute little tools is
understandable, but sometimes it's good to take a fresh perspective
and just let go.