how to write a line in a text file

[nephish]

Hey there,
kinda newbie question here.
i know how to read the lines of a txt file.
i know how to write a txt file.

but how do i overwrite a line value with another value ?

i mean, how do go to, say, line 3 of a text file and replace
what is written on line 3 with something else?

thanks
<><

 

[Steve Holden]

Hey there,
kinda newbie question here.
i know how to read the lines of a txt file.
i know how to write a txt file.

but how do i overwrite a line value with another value ?

i mean, how do go to, say, line 3 of a text file and replace
what is written on line 3 with something else?

thanks
<><

You shouldn't try do "update in place" like that unless your file has
fixed-length lines or records. Altering the length of an existing file
isn't to be lightly undertaken, and even when possible will likely not
be undertaken.

The classic way to approach this problem is to read the file as input,
processing its contents and writing a new file, which eventually
replaces the original in a fairly complex dance of renaming and deletion.

In Python you can use a text file's readlines() method to build a list
of all the lines in a file. That makes it quite easy to change numbered
lines. Having modified the file's content in memory you can then create
a new file using the writelines() method of a new file. The trick is to
avoid losing both the old and the new files when a low-probability crash
occurs.

Since you are newbile (?) I would advise against paranoia - write your
code without worrying about error handling. You'll be pleased to know
that when you start to take a serious interest in error handling Python
has everything you'll need.

regards
Steve

 

[wittempj]

A recipe is
* open your file for reading: f = open('filename.txt', 'r')
* read all lines in a list: content = f.readlines()
* close the file: f.close()
* set the third element in the list to something else: content[2] =
'Blahdiblah'
* re-open the file for writing: f = open('filename.txt', 'w')
* write the list to the file: f.writelines(content)
* close the file: f.close()

 

[nephish]

Gee whiz, so easy.
thanks. Never thought about just changing it while it was read then
re-writing it. that will be just fine. these files are only 9 lines
long.
thanks again !

 

[Dan Sommers]

A recipe is
* open your file for reading: f = open('filename.txt', 'r')
* read all lines in a list: content = f.readlines()
* close the file: f.close()
* set the third element in the list to something else: content[2] =
'Blahdiblah'

Better make that ''Blahdiblah\n' (note the trailing newline) because:
- readlines leaves the newlines at the ends of the lines
- writelines assumes that each line (still) contains a newline

* re-open the file for writing: f = open('filename.txt', 'w')
* write the list to the file: f.writelines(content)
* close the file: f.close()

HTH,
Dan

 

[Steven D'Aprano]

In Python you can use a text file's readlines() method to build a list
of all the lines in a file. That makes it quite easy to change numbered
lines. Having modified the file's content in memory you can then create
a new file using the writelines() method of a new file. The trick is to
avoid losing both the old and the new files when a low-probability crash
occurs.

I'm usually opposed to creeping featuritis in programming languages ("it
would be really cool if Python had a built-in command to do my entire
application") but safe over-writing of files does cry out for a "batteries
included" approach:

- it is a common usage;
- it is tricky to get right;
- it is even trickier to get right in a platform independent way;
- and it is even trickier again to get right in a platform independent way
that doesn't introduce security risks.

The normal trick is to do something like:

read file A;
write temporary file B without clobbering any existing B;
rename file A to C;
rename B to A;
delete C when you know the writing and renaming has succeeded;
and do it all in such a way that it succeeds even if there is very little
available disk-space.

The platform-independence comes from the fact that different OSes expect
the temporary files to live in different places (eg /tmp/ under Linux).
Most operating systems consider it poor form to just write temporary files
any old place.

I'm told by those who claim to know what they're talking about that a
potential risk exists if an attacker can predict the temporary file name
and thus do nefarious things. The exact nature of those nefarious things
was not explained to me, but I do recall the occasional security advisory
for applications which use insufficiently-random temporary file names.

Does anyone have anything suitable for a "safe-overwrite" module?

 

[Wade]

I'm usually opposed to creeping featuritis in programming languages ("it
would be really cool if Python had a built-in command to do my entire
application") but safe over-writing of files does cry out for a "batteries
included" approach:

How about the fileinput module?

http://docs.python.org/lib/module-fileinput.html

"""
Optional in-place filtering: if the keyword argument inplace=1 is
passed to input() or to the FileInput constructor, the file is moved to
a backup file and standard output is directed to the input file (if a
file of the same name as the backup file already exists, it will be
replaced silently). This makes it possible to write a filter that
rewrites its input file in place. If the keyword argument
backup='.<some extension>' is also given, it specifies the extension
for the backup file, and the backup file remains around; by default,
the extension is '.bak' and it is deleted when the output file is
closed. In-place filtering is disabled when standard input is read.
"""

 

[Steven D'Aprano]

How about the fileinput module?

http://docs.python.org/lib/module-fileinput.html

Close, but not quite: the "inplace" argument is more
specific and less general than what I was talking about.

The fileinput solution only works for the specific idiom:

read file A in text mode
process text
write standard output back to file A

compared to the more general idiom:

get data from somewhere
process data
write data to existing file A

In the more general idiom, you don't care whether your
data is text or binary data, whether you got it from a
file, whether it was the same file you are writing to,
or anything else. All you care about is that the file
you write to happens to already exist.

Long ago, when dinosaurs roamed the Earth, (a.k.a.
"before OS X on the Macintosh") Apple suggested a bit
of Pascal code for safely updating a file:

http://developer.apple.com/documentation/mac/Files/Files-25.html#MARKER-9-163

Most of the code is Macintosh-specific, but the
principle is not: when over-writing a file, make sure
that the user can recover from any error up to and
including power failure without losing the data on disk.

(Presumably if the sun explodes and destroys the Earth,
your program is allowed to lose data.)

 

[Guest]

Tue, Jul 26, 2005 at 01:41:36PM +1000, Steven D'Aprano пишет:

Long ago, when dinosaurs roamed the Earth, (a.k.a.
"before OS X on the Macintosh") Apple suggested a bit
of Pascal code for safely updating a file:

http://developer.apple.com/documentation/mac/Files/Files-25.html#MARKER-9-163

That snippet doesn't write data to the existing file. It writes data into
the new tempfile and then renames it, as well as the FileInput object does.

Most of the code is Macintosh-specific, but the
principle is not: when over-writing a file, make sure
that the user can recover from any error up to and
including power failure without losing the data on disk.

Well, it's what (R)DBMS are for, but plain files are not.

 

[Steven D'Aprano]

Tue, Jul 26, 2005 at 01:41:36PM +1000, Steven D'Aprano пишет:



That snippet doesn't write data to the existing file. It writes data into
the new tempfile and then renames it, as well as the FileInput object does.

Obviously you can't *directly* overwrite a file and
expect good things to happen if the disk crashes
half-way through. Any sort of safe overwrite must use
an intermediate file somewhere. That goes without saying.

As for the FileInput object doing the job "as well as"
the old Mac file system, no it does not. I've already
described how the FileInput object is only usable in a
vanishingly small subset of all potential file
overwrites, namely when you are writing stdout to the
same text file you read from -- or at least one of the
multiple files you read from.

If that solution works for your needs, you should use
it. But if you are not writing to stdout, your data
isn't text, and you didn't get it from the same file
you are overwriting, the FileInput solution is no
solution at all.

Well, it's what (R)DBMS are for, but plain files are not.

This isn't 1970, users expect more from professional
programs than "keep your fingers crossed that nothing
bad will happen". That's why applications have multiple
levels of undo (and some of them even save the undo
history in the file) and change-tracking, and auto-save
and auto-backup. Even in the less-than-user-friendly
world of Linux many applications implement their own
version of data protection.

Now maybe you feel that it is overkill for your program
to bother about the tiny chance of data loss when
saving data. That's fine. For 99% of my scripts, I
don't even bother with try...except blocks when writing
to a file. I just open it and write to it and hope it
all works.

But for that other 1%, I take the time to check for
errors and allow the program to recover gracefully. And
that means, if you are about to save the user's work,
you better be sure that if the save fails for any
reason, you haven't blown away their precious data. If
you want to live dangerously with your own data, go for
it. But destroying the user's data is the biggest
programming sin of all.

 

[Andrew Dalke]

This isn't 1970, users expect more from professional
programs than "keep your fingers crossed that nothing
bad will happen". That's why applications have multiple
levels of undo (and some of them even save the undo
history in the file) and change-tracking, and auto-save
and auto-backup.

This isn't 1970. Why does your app code work directly with
files? Use a in-process database library (ZODB, SQLLite,
BerkeleyDB, etc.) to maintain your system state and let the
library handle transactions for you.

Andrew
(e-mail address removed)

 

[Steven D'Aprano]

This isn't 1970. Why does your app code work directly with
files? Use a in-process database library (ZODB, SQLLite,
BerkeleyDB, etc.) to maintain your system state and let the
library handle transactions for you.

And when users are happy to install a relational database system in order
for their editor to save their letter to grandma, I'm sure your scheme
will work well for them.

Until then, I'll continue storing my user's data in files. You may, of
course, choose to write buggy code that can lead to data loss, but I
prefer to take a modicum of care not to destroy their data.

 

[Peter Hansen]

And when users are happy to install a relational database system in order
for their editor to save their letter to grandma, I'm sure your scheme
will work well for them.

Given that ZODB and PySQLite are simply Python extension modules, which
get bundled by your builder tool and are therefore installed
transparently along with your app by your installer, this is a total
non-issue at least with those packages.

After all, it's not 1970 any more. ;-)

-Peter

 

[James Dennett]

Given that ZODB and PySQLite are simply Python extension modules, which
get bundled by your builder tool and are therefore installed
transparently along with your app by your installer, this is a total
non-issue at least with those packages.

After all, it's not 1970 any more. ;-)

Indeed; since 1970 we learned to prefer straightforward
file formats where possible, reserving use of databases
for structured data where the extra costs are justified.

Sometime maybe databases will get better to the point
that we don't need to distinguish so much between them
and filesystems, but we're not there yet. Managing raw
files, carefully, still has a place.

-- James

 

[Calvin Spealman]

Indeed; since 1970 we learned to prefer straightforward
file formats where possible, reserving use of databases
for structured data where the extra costs are justified.

Sometime maybe databases will get better to the point
that we don't need to distinguish so much between them
and filesystems, but we're not there yet. Managing raw
files, carefully, still has a place.

-- James

Filesystems are a horrible way to organize information, and even worse
at structuring it. The mentality that there are any benefits of
low-overhead the outweigh the benefits of minimal database layers,
such as ZODB, BSD DB, and SQLite, is a large part of the reason we are
still stuck with such a mess. Those "extra costs" are so minimal, that
you wouldn't even notice them, if you hadn't convinced yourself of
their presense before performing or researching any realy benchmarks.
A simple RDBMS can be much easier to work with than any flat file
format, will likely be far faster in processing the data, and has the
benefit of years of coding making the code that actually reads and
writes your data as fast and stable as possible.

 

[Jeff Schwab]

Filesystems are a horrible way to organize information, and even worse
at structuring it. The mentality that there are any benefits of
low-overhead the outweigh the benefits of minimal database layers,
such as ZODB, BSD DB, and SQLite, is a large part of the reason we are
still stuck with such a mess. Those "extra costs" are so minimal, that
you wouldn't even notice them, if you hadn't convinced yourself of
their presense before performing or researching any realy benchmarks.
A simple RDBMS can be much easier to work with than any flat file
format, will likely be far faster in processing the data, and has the
benefit of years of coding making the code that actually reads and
writes your data as fast and stable as possible.

You don't use files? You're telling me that instead of copying &
modifying my friend's bashrc, I should be querying a bunch of tables
with SQL? Bull. Ordinary files, preferably in line-separated or tagged
formats, are going to be a lot easier for most of us to work for a long
time to come. This is especially true now that we have lots of good
libraries for working with XML-based formats.

 

[James Dennett]

Filesystems are a horrible way to organize information, and even worse
at structuring it. The mentality that there are any benefits of
low-overhead the outweigh the benefits of minimal database layers,
such as ZODB, BSD DB, and SQLite, is a large part of the reason we are
still stuck with such a mess. Those "extra costs" are so minimal, that
you wouldn't even notice them, if you hadn't convinced yourself of
their presense before performing or researching any realy benchmarks.
A simple RDBMS can be much easier to work with than any flat file
format, will likely be far faster in processing the data, and has the
benefit of years of coding making the code that actually reads and
writes your data as fast and stable as possible.

Read again, and you'll note that I didn't say the costs were
performance related (though in some, just some, situations the
performance cost of using an RDBMS is a reason to avoid it,
and in other cases an RDBMS will be faster than naive file
access).

The reasons for using other formats are things such as better
interoperability, which extends to better future proofing,
easier installations (in some situations -- note again that
there are no hard and fast rules), and the fact that power
users can edit the files by hand (more easily than they can
tweak information in databases in many, but not all, cases).
It can also (again, in some cases) be easier to port filesystem
based storage between systems than to move database access code.

Oh, and please don't assume that I make my decisions without
measuring performance. If you store enough millions, or
billions, of items, then performance is worth measuring. I've
found situations where using various database engines made sense,
I've found situations where flat files made more sense, and I've
seen cases where either was used where the other might have been
a better choice.

-- James