shelve: writing out updates?!

[Paul Rubin]

Shelve uses dbm and pickle to make a persistent object store. The
"db" in "dbm" stands for "database" and while I didn't expect full
ACID capability, I'd have thought there'd be at least some minimum
gesture towards durability of updates. But say that s is a shelve
object. If I say

s[whatever] = value

there is no way apparent from the shelve docs to get the update
flushed out to the disk file until the shelve is actually closed. If
I'm using the shelve to store stuff in a long-running server, it could
be months before the shelve closes.

Is shelve really missing this capability?

Thanks.

 

[Robert Kern]

Shelve uses dbm and pickle to make a persistent object store. The
"db" in "dbm" stands for "database" and while I didn't expect full
ACID capability, I'd have thought there'd be at least some minimum
gesture towards durability of updates. But say that s is a shelve
object. If I say

s[whatever] = value

there is no way apparent from the shelve docs to get the update
flushed out to the disk file until the shelve is actually closed. If
I'm using the shelve to store stuff in a long-running server, it could
be months before the shelve closes.

Is shelve really missing this capability?

No. Call the .sync() method. Unfortunately, the shelve module is not
well-documented.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[phil hunt]

Shelve uses dbm and pickle to make a persistent object store. The
"db" in "dbm" stands for "database" and while I didn't expect full
ACID capability,

What is ACID?

I'd have thought there'd be at least some minimum
gesture towards durability of updates. But say that s is a shelve
object. If I say

s[whatever] = value

there is no way apparent from the shelve docs to get the update
flushed out to the disk file until the shelve is actually closed.

Wierd. I'd expect something like an s.flush() function.

 

[phil hunt]

Shelve uses dbm and pickle to make a persistent object store. The
"db" in "dbm" stands for "database" and while I didn't expect full
ACID capability, I'd have thought there'd be at least some minimum
gesture towards durability of updates. But say that s is a shelve
object. If I say

s[whatever] = value

there is no way apparent from the shelve docs to get the update
flushed out to the disk file until the shelve is actually closed. If
I'm using the shelve to store stuff in a long-running server, it could
be months before the shelve closes.

Is shelve really missing this capability?

No. Call the .sync() method. Unfortunately, the shelve module is not
well-documented.

Obviously it's good when stuff is well documented.

I wonder if the barrier to good documentation is set too high?
If i wanted to add some documentation here, I'd have to download the
current source for the latest Python documentation, download,
install and learn the code that processes the source documentation,
write my changes then send the results into the CVS. (I'm assuming
that's roughly correct -- I haven't actually done it).

The point is, that would be a major effort, too major to merely add
a few lines detailing the .sync method.

But, what if the Python documentation was on a Wiki? Then it would
be easy to update! Of course, we would have to guard against false
or malicious updates, but Wikipedia manage that OK. The
Documentation Wiki could then be used as a basis for the "official"
documentation that comes with each new release.

Does this idea make some sense? Or are there hidden pitfalls?

 

[Robert Kern]

Obviously it's good when stuff is well documented.

I wonder if the barrier to good documentation is set too high?
If i wanted to add some documentation here, I'd have to download the
current source for the latest Python documentation, download,
install and learn the code that processes the source documentation,
write my changes then send the results into the CVS. (I'm assuming
that's roughly correct -- I haven't actually done it).

No, write the content in a reasonable format (i.e. plain text), post it
to the bug tracker, assign it to Fred Drake (I think), and he'll put it
in the right format. If it's longer documentation, like for a full
module, then it's better to actually learn the LaTeX so it can be
dropped in as is.

The point is, that would be a major effort, too major to merely add
a few lines detailing the .sync method.

But, what if the Python documentation was on a Wiki? Then it would
be easy to update! Of course, we would have to guard against false
or malicious updates, but Wikipedia manage that OK.

That's a debatable assertion, but I don't think we'll have the same
problems of Wikipedia thanks to the apolitical nature of Python
documentation.

The
Documentation Wiki could then be used as a basis for the "official"
documentation that comes with each new release.

Does this idea make some sense? Or are there hidden pitfalls?

Yes! Someone actually has to do it! The same idea has come up time and
time again. It's still not here because no one has been able to commit
to the effort involved.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Robert Kern]

Obviously it's good when stuff is well documented.

I wonder if the barrier to good documentation is set too high?
If i wanted to add some documentation here, I'd have to download the
current source for the latest Python documentation, download,
install and learn the code that processes the source documentation,
write my changes then send the results into the CVS. (I'm assuming
that's roughly correct -- I haven't actually done it).

No, write the content in a reasonable format (i.e. plain text), post it
to the bug tracker, assign it to Fred Drake (I think), and he'll put it
in the right format. If it's longer documentation, like for a full
module, then it's better to actually learn the LaTeX so it can be
dropped in as is.

The point is, that would be a major effort, too major to merely add
a few lines detailing the .sync method.

But, what if the Python documentation was on a Wiki? Then it would
be easy to update! Of course, we would have to guard against false
or malicious updates, but Wikipedia manage that OK.

That's a debatable assertion, but I don't think we'll have the same
problems of Wikipedia thanks to the apolitical nature of Python
documentation.

The
Documentation Wiki could then be used as a basis for the "official"
documentation that comes with each new release.

Does this idea make some sense? Or are there hidden pitfalls?

Yes! Someone actually has to do it! The same idea has come up time and
time again. It's still not here because no one has been able to commit
to the effort involved.

--
Robert Kern
(e-mail address removed)

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter

 

[Paul Rubin]

Obviously it's good when stuff is well documented.

I wonder if the barrier to good documentation is set too high?
If i wanted to add some documentation here, ...

I entered a sourceforge bug asking that the doc be updated (thanks,
Robert!). I've done that several times before for similar small doc
bugs and they've always been taken care of reasonably quickly.
There's a "documentation" category in the bug reporting template for
this purpose.

 

[Paul Rubin]

What is ACID?

Basically it means full-blown a transactional database that handles
concurrent updates correctly. See:

http://en.wikipedia.org/wiki/ACID

Wierd. I'd expect something like an s.flush() function.

Thanks to Robert Kern for pointing out that the answer is s.sync().

 

[Michael Hoffman]

But, what if the Python documentation was on a Wiki? Then it would
be easy to update! Of course, we would have to guard against false
or malicious updates, but Wikipedia manage that OK. The
Documentation Wiki could then be used as a basis for the "official"
documentation that comes with each new release.

Does this idea make some sense? Or are there hidden pitfalls?

I thought the `Annotatable Python Docs`_ were a good idea but they seem
to have fallen into disuse.

... _Annotatable Python Docs: http://pydoc.amk.ca/frame.html

 

[Michael Hudson]

Yes! Someone actually has to do it!

I think someone is working on this as part of the summer of code
program (but am not sure about that...).

Cheers,
mwh