Generator metadata/attributes

[acooke.org]

Hi,

(I searched back and found some previous discussion on generator
attributes that seemed to be related to callable generators - this is
NOT that, as far as I can tell)

I want to associate some data with a generator. This is in a
decorator function, as it happens; the generator is being returned and
I want to add some info that is useful for debugging later. I have
tried modifying both .__doc__ and an arbitrary attribute, but the
first is read-only, and the second does not exist (ie you cant just
add one by writing to it; same error with setattr).

I realise I could create my own wrapper that implements __next__ (I am
using Python 3 and haven't checked the exact interface required, but I
guess it's something like that), and add the information that way, but
I am worried I am doing something too complicated. Is there really no
way to stick some arbitrary data onto a generator (from a function
that "yield"s)?

In case it's any help, the decorator is basically:

def mydecorator(f):
def decorate(self, *args):
generator = f(self, *args)
# none of these work
generator.__doc__ = 'From ' + str(self)
generator.description = 'From ' + str(self)
setattr(generator, 'description', 'From ' + str(self))
return generator
return decorate

and it's used like:

class ...
@mydeocrator
def foo(self, ...):
yield...

Thanks,
Andrew

 

[James Stroud]

Hi,

(I searched back and found some previous discussion on generator
attributes that seemed to be related to callable generators - this is
NOT that, as far as I can tell)

I want to associate some data with a generator. This is in a
decorator function, as it happens; the generator is being returned and
I want to add some info that is useful for debugging later. I have
tried modifying both .__doc__ and an arbitrary attribute, but the
first is read-only, and the second does not exist (ie you cant just
add one by writing to it; same error with setattr).

I realise I could create my own wrapper that implements __next__

That is what you are going to have to do. Immutable python builtins work
that way:

py> i = 4
py> i.bob = 2
------------------------------------------------------------
Traceback (most recent call last):
File "<ipython console>", line 1, in <module>
<type 'exceptions.AttributeError'>: 'int' object has no attribute 'bob'

py> i.__doc__ = 'I am an int'
------------------------------------------------------------
Traceback (most recent call last):
File "<ipython console>", line 1, in <module>
<type 'exceptions.AttributeError'>: 'int' object attribute '__doc__' is
read-only


James

 

[Jeff McNeil]

You'll see the same behavior if you attempt to add an attribute to an
instance of object as well.
Traceback (most recent call last):

You'll have to build your own iterator or wrap the generator object
and delegate calls to it. If you elect the latter, you should also
ensure that send and throw and gang work properly. Perhaps something
nice and thin using getattr within your wrapper class.

Jeff

 

[acooke.org]

Thanks folks. Will write my own class.... Andrew

PS So for the record, this works and isn't as ugly/verbose as I was
expecting:

class TaggedWrapper():

def __init__(self, generator, logMixin, stream):
self.__generator = generator
self.__tag = '%s@%s' % (logMixin.describe(), stream)
logMixin._debug('Created %s' % self)

def __next__(self):
return next(self.__generator)

def __str__(self):
return self.__tag


def tag_generator(f):
'''
Log the generator.
'''
def call(self, stream):
return TaggedWrapper(f(self, stream), self, stream)
return call

 

[Ant]

You could look at something like the following to turn the class
iteslf into a decorator (changed lines *-ed):

class TaggedWrapper():

*     def __init__(self, logMixin, stream):

        self.__tag = '%s@%s' % (logMixin.describe(), stream)
        logMixin._debug('Created %s' % self)

    def __next__(self):
        return next(self.__generator)

    def __str__(self):
        return self.__tag

def __call__(self, generator):
self.__generator = generator
return self

and then decorate your generator:

@TaggedWrapper(mixin, stream)
def myGen;
for a in range(1,100):
yield a

(Disclaimer: Completely untested...)

 

[Gerard Flanagan]

def mydecorator( f ):
def decorated(self, *args):
logging.debug( "Created %s", self.__class__.__name__ )
for i in f(self, *args):
yield i
return decorated

can optionally be written as:

def mydecorator( f ):
def decorated(self, *args):
logging.debug( "Created %s", self.__class__.__name__ )
return f(self, *args)
return decorated