python-noob - which container is appropriate for later exportinginto mySql + matplotlib ?

[Cousin Stanley]

Yes, and I *really* wish they wouldn't.

It's my number 1 cause of major problems with Firefox.

Problems with software of any flavor,
especially software that is used regularly
and upon which we are somewhat dependent,
are always a source of frustration ....

My own personal use of firefox over the years
has been limited as I have not used it
for my primary browser and have not experienced
any problems with its bookmarks ....

I use opera as my primary browser
and would very much like to convert
the plain-vanilla bookmark.adr file
that opera uses into an sqlite data base
for diversity in bookmark searches
that would be independent of reglular
browser usage ....

$ grep FOLDER ~/.opera/bookmarks.adr | wc -l
631

$ grep URL ~/.opera/bookmarks.adr | wc -l
14944

http://kb.mozillazine.org/Bookmarks_history_and_toolbar_buttons_not_working_-_Firefox

Although there have been many reports entailing corruption
of the places.sqlite file, it isn't apparent to me
from the link above that sqlite itself is the culprit ....

Could the complexity/bugginess of the firefox code
possibly be the cause instead ?

"If Firefox works normally when you first open it
after starting up the computer but multiple symptoms arise
after you close and later reopen Firefox, it's likely
that a Firefox process from a previous session
did not close properly and the Places database
( "places.sqlite" file ) is locked."


If you check the headers of any of my posts here
you will find that I post with a python-based news client
named XPN that also uses sqlite for persistent storage,
one sqlite database for each different newsgroup ....

I've used xpn daily for many years and have never experienced
a corrupted sqlite database file ....

firefox + sqlite ----> buggy ? ....... :-(

python + sqlite ----> ok, hooray ....

Using a database for such lightweight data as bookmarks is, in my
opinion, gross overkill and adds to the complexity of Firefox.

More complexity leads to more bugs, e.g.:

https://bugzilla.mozilla.org/show_bug.cgi?id=465684#c11

https://bugzilla.mozilla.org/show_bug.cgi?id=431558

These pages show problems that are 4 and 5 years old
from 2008 & 2009 and are marked as Status: RESOLVED FIXED
at the top of the page ....

Are you still having firefox bookmark problems today ?

 

[Tim Chase]

DNS client lookups use published, well-understood
Internet-standard protocols, not at all like talking to a
third-party database, be it open-source or not.

That said, even though DNS is a publicly documented standard, I've
reached for DNS code in the Python stdlib on multiple occasions
(usually to try and snag the MX record for a customer, so smtplib can
send stuff to it), and get disappointed each time. I'd really love
if there was a simple DNS-lookup module available in the stdlib,
especially if it allowed overriding the server to ask. I mean...POP,
IMAP and SMTP are all publicly documented standards that Python makes
easily accessible. DNS would be a good addition.

-tkc

 

[Chris Angelico]

For the bindings to be useful, Python batteries-included distributions
(like python.org installers) would either need to also ship the various
DB client libraries for all supported platforms (including Windows),
which adds complexity and potentially intractable license issues, or
there would need to be reverse-engineered implementations of the client
libs or wire protocols, either option adding fragility and complex
testing issues. DNS client lookups use published, well-understood
Internet-standard protocols, not at all like talking to a third-party
database, be it open-source or not. Sqlite3 is certainly an anomaly in
that it is not-only open source but designed to be a lightweight,
compatible library that runs on just about everything, and with a
fanatical devotion to compatibility and documentation. These days just
about every major product or operating system platform ships with or
uses a copy of sqllite3 for something.

Understandable, but I'm actually referencing a discussion on either
python-dev or python-ideas where the statement was made that it didn't
make sense to include the client for something that the server for
wasn't included. I can't find the discussion thread off-hand, but
that, rather than the portability/complication issues, seemed to be
the primary line of argument.

I don't know about any others, but PostgreSQL's wire protocol isn't
all that difficult to work with, and since we're talking about
something where the far end is almost certainly going to consume some
time, it wouldn't hurt to implement it in pure Python. Based on
http://wiki.postgresql.org/wiki/Python it seems there are a few
modules that do just that (unchecked, but if they work on any platform
and don't require libpq, I strongly suspect they use Python's own
networking); if one of those is of sufficient code quality for the
stdlib, I think it would be an excellent addition. However, I am not a
core dev, therefore sqlite is the only one included.

ChrisA

 

[Roy Smith]

I'd really love if there was a simple DNS-lookup module available in
the stdlib, especially if it allowed overriding the server to ask.

pip install dnspython

 

[rusi]

I am trying to understand your points Chris. On the one hand you say:

No, no, a thousand times no! If I am doing financial transactions,
even if I'm alone on my machine, I will demand full ACID compliance.

On the other you describe a bookmark storage scheme (which it seems
you are recommending); to wit

Suppose bookmarks are stored like this:

r"""Some-Browser-Name web bookmarks file - edit with care
url:http://www.google.com/
title: Search engine
icon: whatever-format-you-want-to-use

url:http://www.duckduckgo.com/
title: Another search engine

url:http://www.python.org/

url:ftp://192.168.0.12/
title: My FTP Server
desc: Photos are in photos/, videos are in videos/
 Everything else is in other/
user: root
pass: secret
"""

The parsing of this file is pretty simple. Blank line marks end of
entry;…

So are you saying that if one switches from the non-ACID compliant
sqlite to your simple-text data-format, the new 'database' (note the
quote marks) will now become ACID compliant?

 

[Chris Angelico]

I am trying to understand your points Chris. On the one hand you say:



On the other you describe a bookmark storage scheme (which it seems
you are recommending); to wit
...
So are you saying that if one switches from the non-ACID compliant
sqlite to your simple-text data-format, the new 'database' (note the
quote marks) will now become ACID compliant?

Unlikely. It theoretically could be made ACID compliant (all it needs
is an OS-guaranteed atomic move/rename operation), but my point is
that some things don't _need_ full-on databases. Financial work *does*
(if I'm accepting money from people, I'd better make pretty sure I
know who's paid me and how much); bookmarks usually don't. Also,
bookmarks are the exclusive property of the person who creates them,
so it's helpful to store them in a way that can be edited; with money
movements, you often want some kind of indelibility guarantee, too
(you can't go back and edit a previous transaction, you have to put in
a correcting transaction). Different tasks demand different storage
schemes.

ChrisA