Thoughts on language-level configuration support?

[David Stanek]

In message <36148830-22c0-4f19-ab23-


If you're advocating anything resembling php.ini, you deserve to die.

I am pretty sure that is uncalled for.

 

[jfager]

By magic: see later


There is actually.  The reason is that those different configuration
interfaces are actually accomplishing different things.  You're lumping
together both config files (wherever they may be and whatever format
they may be in) with command line parameters and GUI command interfaces
when the one is providing long-term defaults and the other short-term
overrides.

The level of detail that you can have in a config file is something that
you never want to see on a command line.  When you do, as with say "gcc",
the results are rather mind-numbing.  This distinction is something that
your reduction of configuration to a single black box fails to capture.

The fact that you 'can' expose something through a particular
interface doesn't mean you 'have to'. If command line args don't make
sense for a particular program, nobody is would be forced to use
them. That's the point - the interface the end user works with is
flexible, it's what they want to deal with.

Yes you do.  This spread-about inline style of configuration is in no
sense discoverable to the casual reader, who isn't going to know that
a particular config option even exists unless he or she chances upon
the line where it's registered.  

"Discoverable", as in built-in tools that let you have the following
conversation: "Program, tell me all the things I can configure about
you" - "Okay, here they all are". No digging through the source
required.

I'm not commenting on your code
snippet here -- or I'd be here all night, and you really wouldn't
like me at the end of it --

Dude, that was all of 15 minutes of coding (not counting the
diversions spent poking around the bytecode tools for the first
time).

but I am commenting on the usage style
it's obviously built to support.  That usage style is much less
visible than any of the common (or uncommon, for that matter) styles
of getting config input, and discourages documentation habits that
would otherwise mitigate it.



As I said, it's less visible because it's decentralised.  The moment
you do that, you lose all hope of centralising the documentation, and
the visibility of your options plummets.

The tools provided by the language do the centralization for you, that
was one of the main points.

What "discoverability"?  You still haven't told me anything to make
me believe that this configuration setup is discoverable at all without
external tools.

The whole point is that the tools for this would be provided by the
language system.

It remains magic when it requires the language to invent things on the
fly that the programmer has no means of discovering until the code is
running, by which time it's generally too late.  For example:

   spam = conf(1)

How does that translate into a command-line option?  -s?  --span?
Both?  -S, even?  --first-config-option-I-came-across?  If the first,
what happens when:

   span = conf(0)

shows up to initialise a class attribute some time later?  What
restrictions can be put on the value you get back?  What can the
help system say about this, or do we have to go back to doing all
that by hand?  Now translate all those questions into the very
different environment of a config file.  Repeat with a database,
and all it's quirks.  By the time your colossus has acquired
enough parameters to at least hint at the desirable answers to
these questions, you've effectively duplicated the interfaces to
all of the config mechanisms you're trying to replace and you've
still lost a whole heap of flexibility.

Yes, you're right, the code that actually injects the configuration
isn't trivial. I never intended to imply that it was. But it would
probably only have to be written once (people would only write their
own if they had a special need). The win is that the underlying code
doesn't have to change just because the end-user configuration format
did.

- Jason

 

[Aaron Brady]

I've written a short post on including support for configuration down
at the language level, including a small preliminary half-functional
example of what this might look like in Python, available athttp://jasonfager.com/?p=440.

The basic idea is that a language could offer syntactic support for
declaring configurable points in the program.  The language system
would then offer an api to allow the end user to discover a programs
configuration service, as well as a general api for providing
configuration values.

The included example implements the first bit and hints at the third,
defining a function that looks up what variable its output will be
assigned to and tries to find a corresponding value from a
configuration source.  It's very preliminary, but I hope it gives a
flavor of the general idea.

Any thoughts or feedback would be greatly appreciated.

Hi, joining late.

I think you are talking about modifying a running program's code and
data from an external process.

My first guess at the closest Python can come is:

import shelve
configs= shelve.open( 'configs.dat' )

#stuff
for some computation:
calc= Calc( )
calc.precision= configs[ 'precision' ]
calc.compute( )

All well and good if you can synchronize access to 'configs.dat', but
a subclass of shelve.DbfilenameShelf could accomplish that.

For background, "a shelf is a persistent, dictionary-like
object" (docs). 'configs.dat' would reside on disk, so any process
could access it.

 

[Kay Schluehr]

"Discoverable", as in built-in tools that let you have the following

conversation: "Program, tell me all the things I can configure about
you" - "Okay, here they all are". No digging through the source
required.

But this doesn't have any particular meaning. If I run a dir(obj)
command all attributes of obj will be returned and I can be sure these
are all. In C# I can reflect on attributes of an assembly which is a
well defined entity. "Program" is not an entity. It's kind of a user
interface name in StarTreck ( which prefers "computer" for this
purpose though ). This way we cannot design systems.

 

[jfager]

But this doesn't have any particular meaning. If I run a dir(obj)
command all attributes of obj will be returned and I can be sure these
are all. In C# I can reflect on attributes of an assembly which is a
well defined entity. "Program" is not an entity. It's kind of a user
interface name in StarTreck ( which prefers "computer" for this
purpose though ). This way we cannot design systems.

"Module and transitive graph of other modules (constructed through
'import' statements), tell me all the things I can configure about
you". Is that a little clearer?

 

[Aaron Brady]

"Module and transitive graph of other modules (constructed through
'import' statements), tell me all the things I can configure about
you".  Is that a little clearer?

Using shelve, which I suggested earlier, you have to rely on modules
to choose the options they want to make available. You could require
them to store configs. on a per-module basis in separate files, or
require them to use separate namespaces in the shelf.

It could also be possible to interface with the target process's
garbage collector, but you're limited to objects you can identify in
the list of tracked objects, and only the ones that are mutable at
that.

 

[CTO]

 I just mean that there should be a

clear and easy way to do it, that it should be considered a basic
service, and that if the best way to satisfy all the goals is to
integrate it directly into the language, that shouldn't be shied away
from.

Honestly, the programming language and the configuration languages
should never, ever, ever be inseparable. I don't see a reason for
it and I see some great reasons not to do it.

The example that I have on my blog post, I consider that 'syntax',
even though it's implemented as a function, mainly just because it
digs into the bytecode and modifies the normal way a function is
evaluated (the function's value is determined by where the output
would go).

I don't particularly see why this is needed. To my mind the strongest
argument you are making, by far, is the "discoverable" argument. If
you were able to build a system which permitted other components to
discover configuration options and give configurable levels of
permanence and visibility to them, that would be great. If you were
able to do it and make it easy for a programmer to interact with, you
would most definitely have a module I would use. But I daresay you're
going to have to build it (or at least mock it up) before you're going
to get a lot of folks jumping on the bandwagon, and its a *long* slog
before you hit the level of support and stability you're going to need
to convince folks to trust their precious config files to it.

 

[Steven D'Aprano]

The basic idea is that a language could offer syntactic support for
declaring configurable points in the program. The language system would
then offer an api to allow the end user to discover a programs
configuration service, as well as a general api for providing
configuration values.

Completely coincidentally, a colleague forwarded me this cartoon about
exposing program internals as the user interface:

http://www.ok-cancel.com/comic/4.html


I thought it was amusing, because I've seen programs just like that. Yes,
even "GUI applications", where the developer thought that creating a user
interface was exposing the values of internal variables to the user to
modify directly.

How does your proposal differ from that?

 

[jfager]

Honestly, the programming language and the configuration languages
should never, ever, ever be inseparable. I don't see a reason for
it and I see some great reasons not to do it.

I'm not saying the actual end-user configuration language would be
tied to the programming language. I'm starting to think a better way
to describe this is to play down configuration as a use case, and
instead focus on what the actual mechanism is: a way to define points
in your program that can have values injected into them at runtime by
some external entity, a uniform naming scheme for those points derived
from the code, and a way to expose those points to interested
consumers.

I don't particularly see why this is needed.

It was a stab at the 'uniform naming scheme' - the function fetches a
config value based on a key derived from the name of the variable its
output will be assigned to. It definitely needs more thought, but at
least I got to play around with the bytecode tools

To my mind the strongest
argument you are making, by far, is the "discoverable" argument. If
you were able to build a system which permitted other components to
discover configuration options and give configurable levels of
permanence and visibility to them, that would be great. If you were
able to do it and make it easy for a programmer to interact with, you
would most definitely have a module I would use. But I daresay you're
going to have to build it (or at least mock it up) before you're going
to get a lot of folks jumping on the bandwagon, and its a *long* slog
before you hit the level of support and stability you're going to need
to convince folks to trust their precious config files to it.

Agreed, and I do intend to do a proper mockup when I get the time; I
just wanted to get some initial reactions. Thanks for your feedback.