The Python Way for module configuration?

[kj]

[PYTHON NOOB ALERT]

I want to write a module that serves as a Python front-end to a
database. This database can be either in the form of tab-delimited
flat files, XML files, or a PostgreSQL server. The module is meant
to hide these database implementation details from its users.

But, minimally, the module needs to have some configuration details
to know where to get the data. There are many ways to provide this
configuration data to the module, but I would like to know what's
considered "best practice" for this type of problem in the Python
world.

Ideally, I'm looking for a system that is unobtrusive under normal
operations, but easy to override during testing and debugging.

I would appreciate your comments and suggestions.

TIA!

kynn

 

[MRAB]

[PYTHON NOOB ALERT]

I want to write a module that serves as a Python front-end to a
database. This database can be either in the form of tab-delimited
flat files, XML files, or a PostgreSQL server. The module is meant
to hide these database implementation details from its users.

But, minimally, the module needs to have some configuration details
to know where to get the data. There are many ways to provide this
configuration data to the module, but I would like to know what's
considered "best practice" for this type of problem in the Python
world.

Ideally, I'm looking for a system that is unobtrusive under normal
operations, but easy to override during testing and debugging.

I would appreciate your comments and suggestions.

There are already modules which provide access to databases.

 

[Aaron Sherman]

There are already modules which provide access to databases.

As you can see the "Python Way" is to be rude ;-)

Anyway, your answer is that there are some abstraction layers called
"ORMs". You can grab one of these and write a back end for it.
However, you might first want to look at SQLLite and see if it already
has what you want (e.g. a light-weight, zero-install database
interface).

 

[kj]

As you can see the "Python Way" is to be rude ;-)

Anyway, your answer is that there are some abstraction layers called
"ORMs". You can grab one of these and write a back end for it.
However, you might first want to look at SQLLite and see if it already
has what you want (e.g. a light-weight, zero-install database
interface).

Hi, thanks, but the database aspect of my question is tangential.
What I'm interested in is the general problem of providing
configuration parameters to a module.

TIA!

kynn

 

[Petr Messner]

Hi,

What I'm interested in is the general problem of providing
configuration parameters to a module.

Some database/ORM libraries are configured via simple strings in the
form "dialect://userassword@host/dbname[?key=value..]", for example
"mysql://me:topsecret@localhost/test?encoding=utf8". Is this what you
want to know?

Petr

 

[Dave Angel]

Hi, thanks, but the database aspect of my question is tangential.
What I'm interested in is the general problem of providing
configuration parameters to a module.

TIA!

kynn

Check out the ConfigParser module, a standard module. In Python 3.0,
it's renamed to configparser, but still part of the standard distribution.

This module interprets a file that's in roughly the INI format.

 

[Carl Banks]

Hi, thanks, but the database aspect of my question is tangential.
What I'm interested in is the general problem of providing
configuration parameters to a module.

The best way to do this totally depends on how complex these
parameters are.

Are they simple one-to-one key-value pairs? Then the frontend should
explose a dict interface. There are probably existing third-party
libraries that expose dict-like interfaces to all of your imagined
backends.
Check pypi.python.org, and also Python cookbook.

If the configuration is more complex, such as if it is hierarchical,
if it includes many-to-many relationships, if it is graph-like, then a
more sophisticated frontend will be needed. For these situations I
second the recommendation to consider an ORM (such as SqlAlchemy). I
am not aware of any ORMs that can use SQL, XML, and tab-delimited text
all as backends, though. I am not a big database guy though.

Also, I exhort to you consider whether you really need so many
different backends for a configuration file. Might your time be
better spent improving some other aspect of your application?


Carl Banks

 

[Steven D'Aprano]

(Even if you don't want to receive email, could you please give your
actual name in the ‘From’ field instead of just initials? It makes
conversation less confusing.)

Some people prefer to be known by their initials.

Some people's legal names *are* initials.

There's nothing confusing about addressing somebody by their initials.
The only complication in this case is knowing the correct way to
capitalise it -- should it be K.J. or Kj or something else?

kj <[email protected]> writes:

Apart from capitalisation, it works perfectly.

 

[Hendrik van Rooyen]

I want to write a module that serves as a Python front-end to a
database. This database can be either in the form of tab-delimited
flat files, XML files, or a PostgreSQL server. The module is meant
to hide these database implementation details from its users.

But, minimally, the module needs to have some configuration details
to know where to get the data. There are many ways to provide this
configuration data to the module, but I would like to know what's
considered "best practice" for this type of problem in the Python
world.

Ideally, I'm looking for a system that is unobtrusive under normal
operations, but easy to override during testing and debugging.

To pass in configuration to a module, you have basically two
choices:

- Use the command line and sys.argv
- Put the stuff in a file of some sort.

I would not recommend the first option, because it ultimately
ends up as an abomination such as we can see with gcc, where
it has spawned over elaborate systems of makefiles to tell
it how to do something - so we are back at a file, in any event.

So the questions become - how do you organise the file,
and how do you get hold of the stuff in it.

You have been pointed to configparser, and others.

You can also do a simple thing, as a module gets executed
upon import. If your parameters are simple, this may be the
easiest way of doing it, namely to just write the bunch of
them down, all in a configuration.py file, like this:

P1 = 7
P2 = 42
alloweds = [john,mary,sally]

Then where you want to use them, you do the following:

import configuration as cf

if cf.P1 > 3:
do something

if suspect in cf.alloweds:
let user in
else:
print "Go away you fiend from the nether regions"
sys.stdout.flush()
sys.exit()

if not cf.P2 == 42:
print "The end of the universe has arrived - hide!"
while True:
pass

Do not forget that you can put a dictionary
there to handle more complex stuff.

You can also get fancy and put whole classes
of data into such a file. In what I do, I have
never felt the need for that.

You get the drift (I hope)

- Hendrik

 

[Steven D'Aprano]

Some people prefer to spend their volunteer time conversing with people
who don't just use initials or pseudonyms.

And some people refuse to use words with the letter "E" in them.

Just out of curiosity, how are you supposed to recognise people who are
using pseudonyms? For all we know, "Steven D'Aprano" might be a pseudonym
-- you haven't seen my birth certificate.

I noticed that you ignored my comment about people's whose legal names
are initials.

I find it much less confusing to converse with someone if I can keep
straight who they are, and names work better than initials for that
purpose.

Really? Can you cope with two-letter names like Jo, Li, Ed, Cy, or even
Wm (a variant of William), or are you about to start insisting that
people choose three-letter names?

If you can cope with Al, Di or Mo, then what's the difference between
those names and two-letter initials?

 

[Aahz]

Some people prefer to spend their volunteer time conversing with people
who don't just use initials or pseudonyms.

Bully for you! Just don't bully people who don't follow your preferred
naming system.

 

[kj]

(Even if you don't want to receive email, could you please give your
actual name in the ‘From’ field instead of just initials? It makes
conversation less confusing.)

I don't know why, but for as long as I can remember everyone calls
me kj, even my mom. My name is Keaweikekahialiʻiokamoku
Jallalahwallalruwalpindi

kj

 

[Simon Forman]

I don't know why, but for as long as I can remember everyone calls
me kj, even my mom.  My name is Keaweikekahialiʻiokamoku
Jallalahwallalruwalpindi

kj

Now that's funny.

+1 QotW

~SF