Metaclass with name overloading.

[Carlos Ribeiro]

On 27 Sep 2004 13:33:33 +0200, Jacek Generowicz
[...]

I was wondering whether it would be possible to achieve this by
forcing Python to use some dicitonary proxy (which accumulates
values, rather that keeping just the last value to be associated
with a key), instead of dict, when executing the class definiton?

[...]

No, you can't, and it's not just a parser issue. Python uses direct
C calls to the native dict type. It's hard coded,

I feared this would be the case.

p.s. In the particular case of the original poster, I'm wondering
what kind of application did he had in mind.

A standalone, lightweight SWIG-like tool, in this case. But in
general, I've had cause to wonder about declarative syntaxes in Python
every now and then.

I'm also exploring declarative alternatives for a lot of stuff in
Python. It started explicitly as an experiment, mainly because I could
not rationally explain why I did 'feel' that it was the right approach
for a class of applications: form definitions, reports, webpage
templates, etc. Now I think that I'm beginning to get a better
understanding that allows me to articulate better *why* should I
(ab)use Python for declarative programming, instead of using a data
driven approach with XML, or creating my own mini-declarative
language. In short, the argument goes like this:

Generic templating mechanisms start as simple variable substitution
engines, but as they start to be used, there's the need to add control
structures (if, for, etc); it's also needed to provide more ways for
the template to communicate with the main program, exchanging
variables and values. At this point, wouldn't be better to write all
templates in the main programming language of the system?

Thanks, to all who contributed ideas to the thread, particularly Alex,
Thomas and Lenard.

I learned a lot through this thread. As a matter of fact, I used your
problem as an exercise on decorators And I think that, while still
exploring and making some (dumb) mistakes, I'm beginning to feel
comfortable with the more esoteric introspection features of Python.

After looking Lenard example, I've come to think about other
alternatives. There are a some interesting things that can still be
done, some even more esoteric than all stuff that we've done so far. A
generic solution for this problem would greatly simplify my own
search, and I'll keep looking for it.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Alex Martelli]

I feared this would be the case.

It's not (not in 2.4 at least) -- the STORE_NAME is quite ready to find
a non-dict as the frame's f_locals. The problem is getting your object
to be used as the frame's f_locals in the first place -- hard but that
only affects a few spots in ceval.c.


Alex

 

[Alex Martelli]

Generic templating mechanisms start as simple variable substitution
engines, but as they start to be used, there's the need to add control
structures (if, for, etc); it's also needed to provide more ways for
the template to communicate with the main program, exchanging
variables and values. At this point, wouldn't be better to write all
templates in the main programming language of the system?

At this point, your templating is not declarative -- it's imperative.
Like everything in Python, btw -- not ONE 'declarative' in sight (except
the 'global' statement, which is part of what makes it a wart;-).

There IS a case for purely declarative stuff _embedding_ Python code,
like strakt.com's "blam" (purely informal name, as Strakt's marketing
may lot like it, we just can't keep saying "Business Logic Module
Language" forever;-) does for (basically) ERD + actions/triggers. The
embedding makes the whole non-declarative, of course. But the
declarative part can still be way prettier than it would be if it wasn't
a separate language, e.g. it could use such keywords as 'entity',
'relation', 'attribute' and the like...


Alex

 

[Carlos Ribeiro]

At this point, your templating is not declarative -- it's imperative.
Like everything in Python, btw -- not ONE 'declarative' in sight (except
the 'global' statement, which is part of what makes it a wart;-).

I knew I should have taken more time to write that paragraph The
way I'm writing my code "reads" more like declarative code than
imperative. One can surely argue with my lack of academic rigour. I
think that I'm writing "declarative" code because I'm using class
declarations to create complex, hierarchic data structures. I want to
state __what it is__, not state __how it should be done__ step by
step.

Your comment also made me realize a point that should be highlighted.
Normal templates [1] are clearly imperative, and that's part of my
problem with them. But complex object-oriented structures, although
including code (in the form of methods and descriptors) are much more
dynamic than a simple template. Better than this -- normal templates
are inherently sequential and imperative in the way they're written.
Object oriented structures are much more flexible in this respect.

[1] I stress the term "normal templates" because I'm focusing on
standard, run-of-the-mill templating systems.

There IS a case for purely declarative stuff _embedding_ Python code,
like strakt.com's "blam" (purely informal name, as Strakt's marketing
may lot like it, we just can't keep saying "Business Logic Module
Language" forever;-) does for (basically) ERD + actions/triggers. The
embedding makes the whole non-declarative, of course. But the
declarative part can still be way prettier than it would be if it wasn't
a separate language, e.g. it could use such keywords as 'entity',
'relation', 'attribute' and the like...

In the end, you've raised another interesting point -- on the whole,
my current approach is not purely declarative. It's rather a mix of
imperative and declarative, but with a mostly declarative
infrastructure holding things together.

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Carl Banks]

I would like to write a metaclass which would allow me to overload
names in the definition of its instances, like this

class Foo(object):

__metaclass__ = OverloadingClass

att = 1
att = 3

def meth(self):
pass

def meth(self, arg):
return arg [snip]
Is something like this at all possible in pure Python? or does in
require fiddling around in the guts of the parser?

Not exactly what you asked for, and a bit (litotes) ugly, but it does
allow convenient subgroups within a class. I used a similar trick
once when writing a little parser.


def alltuple(name,bases,clsdict):
return tuple(clsdict.values())


class Foo(object):

class att:
__metaclass__ = alltuple
_1 = 1
_2 = 3

class meth:
__metaclass__ = alltuple
def _1(self):
pass
def _2(self,arg):
return arg


Making it nice and pretty left as an exercise.

 

[Bengt Richter]

]

At this point, your templating is not declarative -- it's imperative.
Like everything in Python, btw -- not ONE 'declarative' in sight (except
the 'global' statement, which is part of what makes it a wart;-).

Hm ;-) Is a module source a declaration of (imperative) intent, passive until imported?
ISTM we are getting into shades of semantics. Interesting though ;-)
For a language that plays well both ways, I would try scheme or lisp, I think.

Regards,
Bengt Richter

 

[Carlos Ribeiro]

]

At this point, your templating is not declarative -- it's imperative.
Like everything in Python, btw -- not ONE 'declarative' in sight (except
the 'global' statement, which is part of what makes it a wart;-).

Hm ;-) Is a module source a declaration of (imperative) intent, passive until imported?
ISTM we are getting into shades of semantics. Interesting though ;-)
For a language that plays well both ways, I would try scheme or lisp, I think.

I was just about to reply to Alex, but managed to stop my fingers.
It's indeed a fine line, and I'm not enough of an academicist to
discuss it with all detail it deserves. We could go on weeks debating
it here (and I'm afraid we do). Broadly speaking, my take is as
follows:

Class definitions are executed (imperative), but are normally used to
store definitions (that's declarative, in a broad sense). I think
that's exactly what has attracted me to this kind of 'hack'. The
ability to write intelligent, complex, hierarchic data structures
seamlessly intermingled with code. Templating languages or XML fall
short in this respect. If you really want to *integrate* them both --
and I'm not talking about simply reading static resource files here --
either you have a cross beast that is data based but has some
imperative statements interspersed with a clumsy syntax, or you have
source code filled with unneeded clutter to manage the data
manipulation part, in a rather obstrusive way to the logic of the
system.

(XML based systems use complex parsers to work with the data. It's not
possible, in most cases, to include enough intelligence in the data
stream itself for it to instruct the parser to do something
"different" -- unless you care to define your own language to do it,
and that's clumsy, at best, given XML "great" readability).

--
Carlos Ribeiro
Consultoria em Projetos
blog: http://rascunhosrotos.blogspot.com
blog: http://pythonnotes.blogspot.com
mail: (e-mail address removed)
mail: (e-mail address removed)

 

[Alex Martelli]

]

At this point, your templating is not declarative -- it's imperative.
Like everything in Python, btw -- not ONE 'declarative' in sight (except
the 'global' statement, which is part of what makes it a wart;-).

Hm ;-) Is a module source a declaration of (imperative) intent, passive

until imported?

Every piece of code is 'passive unless executed', but that doesn't mean
every language is declarative.

ISTM we are getting into shades of semantics. Interesting though ;-)

Not all that much (to me), since redefining a word so that it applies to
every possible language is basically robbing that word of any meaning.

For a language that plays well both ways, I would try scheme or lisp, I think.

Hard to argue with this (or Dylan for syntax-sugar reasons, maybe). One
alternative might be to explore pure functional languages, which can be
seeing as remapping imperativeness into declarativeness.


Alex

 

[Jacek Generowicz]

[...]

For a language that plays well both ways, I would try scheme or
lisp, I think.

[...]

The ability to write intelligent, complex, hierarchic data
structures seamlessly intermingled with code.

Yes, this is one of the great advantages of Lisp ... and has been for
about four decades.

 

[Jacek Generowicz]

It's not (not in 2.4 at least)

For what definition of "hard coded" ?

-- the STORE_NAME is quite ready to find a non-dict as the frame's
f_locals. The problem is getting your object to be used as the
frame's f_locals in the first place -- hard but that only affects a
few spots in ceval.c.

So, from the perspective of trying to code it in pure Python, it _is_
hard coded, IIUC.

(Unfortunately, I cannot afford the luxury of playing with the Python
implementation itself; I must deliver code which works with a
bog-standard Python 2.3.4. I'd love to have the time to play with
ceval.c on my own account ... but that is another luxury I cannot
afford :-( )

 

[Jacek Generowicz]

def alltuple(name,bases,clsdict):
return tuple(clsdict.values())

__metaclass__ = alltuple

WBMSWA12FB !

It never occurred to me that a metaclass didn't have to be a _class_.

 

[Alex Martelli]

For what definition of "hard coded" ?

Is there more than one? As I already quoted on this very thread:

if (PyDict_CheckExact(x))
err = PyDict_SetItem(x, w, v);
else
err = PyObject_SetItem(x, w, v);

so, the "direct C call to the native dict type" only happens if x is
exactly of that type, otherwise the generic abstract call happens
instead and can deal with dispatching the functionality as needed.

Basically, the PyDict_SetItem is now there, and guarded with a
PyDict_CheckExact, only as an optimization: x will be of native dict
type overwhelmingly often.

So, from the perspective of trying to code it in pure Python, it _is_
hard coded, IIUC.

For some value of "it", sure, but NOT because of "direct C calls to the
native dict type" spread hither and yon (as used to be the case).
Rather, the issue is strictly with how a frame gets built and handled.

I never claimed nothing at all is hard-coded (even in 2.4), just that
the specific issue with "direct C calls" _isn't_ (in 2.4) for the case
of interest (STORE_NAME opcodes' execution).

(Unfortunately, I cannot afford the luxury of playing with the Python
implementation itself; I must deliver code which works with a
bog-standard Python 2.3.4. I'd love to have the time to play with
ceval.c on my own account ... but that is another luxury I cannot
afford :-( )

If you need to support 2.3.4 and can't even consider extensions, your
options are indeed severely limited -- I don't recall, but it's even
possible that, for THAT release, the "direct C calls" assertion is valid
(which is why I was careful to say "for 2.4 at least" every time). It
matters hugely (to most would-be extenders, who could surely afford to
use extensions for the purpose) whether it is or not, of course: making
some kind of special-purpose frame and getting it used appropriately
might be feasible, but if there are direct C calls hardwired all over
the place then no solution at all is feasible _within the 2.3.*
constraint_.


Alex

 

[Alex Martelli]

WBMSWA12FB !

It never occurred to me that a metaclass didn't have to be a _class_.

You should have seen Guido's face when he first saw me give a
presentation on "use and abuse of custom metaclasses" -- apparently,
judging from his horrified expression, it hadn't occurred to him,
either, and he didn't like it half a bit... since then I've been quite
careful against actually using this idea in production code.

Actually I believed I showed something more like:

class whatever:
def __metaclass__(clsname, clsbases, clsdict):
return <I don't remember what>
...etc etc...

i.e., an "anonymous metaclass", so to speak. But that's a minor aspect.
After all, something like:

class yetanother:
class __metaclass__(type):
...etc etc...

is just as so-to-speak "anonymous" yet IS nowadays quite an accepted
idiom...!


Alex

 

[David Mertz, Ph.D.]

I would like to write a metaclass which would allow me to overload
names in the definition of its instances, like this
class Foo(object):
__metaclass__ = OverloadingClass
def meth(self):
pass
def meth(self, arg):
return arg

It's not the literal syntax you're asking for, but using my technique
for multiple dispatch in Python achieves the effect you're looking
for. It has nothing to do with metaclass, but it lets you provide
multiple "signatures" for a call. Not just number of arguments, but
also their types (if you want: you can also generically specify a
descendent of object).

See http://www-106.ibm.com/developerworks/linux/library/l-pydisp.html
for more details.

Yours, David...