A question on decorators

T

Tim Henderson

Hello

I am writing an application that has a mysql back end and I have this
idea to simplify my life when accessing the database. The idea is to
wrap the all the functions dealing with a particular row in a
particular in a particular table inside a class. So if you have a
table that looks like this:

id str1 str2 pickled_data1 pickled_data2
0 woeif aposf (bin) (bin)
1 ofime powe (bin) (bin)
....
n oiew opiwe (bin) (bin)

you can access this table like this

t = Table(id) #to load a pre-entered row
t2 = Table(id, str1, str2, data1, data2) #to create a new row

when you change a an attribute of the class like this...
t.str1 = 'new value'

it automatically updates the database backend.

I have what I just described working. However I want an easier way to
deal with my pickled_data. Right now I am pickling dictionaries and
list types. Now there is one problem with this, let me demonstrate

t.data.update({'new key':'new value'})
print t.data
{... 'new key':'new value' ...}

which makes it appear that the database has been updated as well, but
in fact it hasn't to update the database with this scheme you actually
have to do this.

t.data.update({'new key':'new value'})
t.data = t.data

this is not ideal so I subclassed the built in dict type like this:

class _my_dict(dict):

def __init__(self, row_index_name, row_index, column_name, a=None,
**kwargs):
self.row_index_name = row_index_name
self.row_index = row_index
self.column_name = column_name
self.write_access = True
if (a == None): dict.__init__(self, kwargs)
else: dict.__init__(self, a)

self.update_db()

def __delitem__(self, key):
if self.write_access:
dict.__delitem__(self, key)
self.update_db()

def __setitem__(self, key, value):
if self.write_access:
dict.__setitem__(self, key, value)
self.update_db()


def clear(self):
if self.write_access:
dict.clear(self)
self.update_db()

...
more methods which are simliar
...

def update_db(self):
if self.write_access:
con = get_dbConnection()
cur = con.cursor()

table = self.experiment.TABLE
row_index_name = self.row_index_name
row_index = self.row_index
column_name = self.column_name
column_value = MySQLdb.escape_string(pickle.dumps(self))

q1 = '''UPDATE %(table)s
SET %(column_name)s = '%(column_value)s'
WHERE %(row_index_name)s = '%(row_index)s' ''' % locals()

cur.execute(q1)
con.close()


Now while this works, it is a lot of work. What I want to be able to
do is something where I write one decorator function that
automatically updates the database for me. So let us pretend I have
this function.

let: dec_update_db() be my decorator which updates the dictionary.

to use this function it seems I would probably still have to subclass
dict like this:

class _my_dict2(dict):

@dec_update_db
def __init__(self, row_index_name, row_index, column_name, a=None,
**kwargs):
self.row_index_name = row_index_name
self.row_index = row_index
self.column_name = column_name
self.write_access = True
if (a == None): dict.__init__(self, kwargs)
else: dict.__init__(self, a)

@dec_update_db
def __delitem__(self, key):
dict.__delitem__(self, key)

@dec_update_db
def __setitem__(self, key, value):
dict.__setitem__(self, key, value)

@dec_update_db
def clear(self):
dict.clear(self)

... and so on ...

this is also not ideal. because I still have to apply the decorator to
every function which changes the dictionary.

What I really want is a way to have the decorator applied
automatically every time a method in dict or a sub class is called. I
feel like this must be possible. Has any one here done anything like
this before?

Thank you for reading my long post, I hope you understand what I am
asking especially since the code in it is not very good.

cheers
Tim Henderson
 
M

Mike Driscoll

Hello

I am writing an application that has a mysql back end and I have this
idea to simplify my life when accessing the database. The idea is to
wrap the all the functions dealing with a particular row in a
particular in a particular table inside a class. So if you have a
table that looks like this:

id str1 str2 pickled_data1 pickled_data2
0 woeif aposf (bin) (bin)
1 ofime powe (bin) (bin)
...
n oiew opiwe (bin) (bin)

you can access this table like this

t = Table(id) #to load a pre-entered row
t2 = Table(id, str1, str2, data1, data2) #to create a new row

when you change a an attribute of the class like this...
t.str1 = 'new value'

it automatically updates the database backend.

I have what I just described working. However I want an easier way to
deal with my pickled_data. Right now I am pickling dictionaries and
list types. Now there is one problem with this, let me demonstrate

t.data.update({'new key':'new value'})
print t.data
{... 'new key':'new value' ...}

which makes it appear that the database has been updated as well, but
in fact it hasn't to update the database with this scheme you actually
have to do this.

t.data.update({'new key':'new value'})
t.data = t.data

this is not ideal so I subclassed the built in dict type like this:

class _my_dict(dict):

def __init__(self, row_index_name, row_index, column_name, a=None,
**kwargs):
self.row_index_name = row_index_name
self.row_index = row_index
self.column_name = column_name
self.write_access = True
if (a == None): dict.__init__(self, kwargs)
else: dict.__init__(self, a)

self.update_db()

def __delitem__(self, key):
if self.write_access:
dict.__delitem__(self, key)
self.update_db()

def __setitem__(self, key, value):
if self.write_access:
dict.__setitem__(self, key, value)
self.update_db()

def clear(self):
if self.write_access:
dict.clear(self)
self.update_db()

...
more methods which are simliar
...

def update_db(self):
if self.write_access:
con = get_dbConnection()
cur = con.cursor()

table = self.experiment.TABLE
row_index_name = self.row_index_name
row_index = self.row_index
column_name = self.column_name
column_value = MySQLdb.escape_string(pickle.dumps(self))

q1 = '''UPDATE %(table)s
SET %(column_name)s = '%(column_value)s'
WHERE %(row_index_name)s = '%(row_index)s' ''' % locals()

cur.execute(q1)
con.close()

Now while this works, it is a lot of work. What I want to be able to
do is something where I write one decorator function that
automatically updates the database for me. So let us pretend I have
this function.

let: dec_update_db() be my decorator which updates the dictionary.

to use this function it seems I would probably still have to subclass
dict like this:

class _my_dict2(dict):

@dec_update_db
def __init__(self, row_index_name, row_index, column_name, a=None,
**kwargs):
self.row_index_name = row_index_name
self.row_index = row_index
self.column_name = column_name
self.write_access = True
if (a == None): dict.__init__(self, kwargs)
else: dict.__init__(self, a)

@dec_update_db
def __delitem__(self, key):
dict.__delitem__(self, key)

@dec_update_db
def __setitem__(self, key, value):
dict.__setitem__(self, key, value)

@dec_update_db
def clear(self):
dict.clear(self)

... and so on ...

this is also not ideal. because I still have to apply the decorator to
every function which changes the dictionary.

What I really want is a way to have the decorator applied
automatically every time a method in dict or a sub class is called. I
feel like this must be possible. Has any one here done anything like
this before?

Thank you for reading my long post, I hope you understand what I am
asking especially since the code in it is not very good.

cheers
Tim Henderson

Why aren't you using SQLAlchemy or SQLObject? I think they would work
better than this and give you a lot more flexibility.

Besides, you should use sqlite rather than pickle databases. It's
especially easy since sqlite is included with Python 2.5.

Mike
 
T

Tim Henderson

Mike Driscoll said:
Besides, you should use sqlite rather than pickle databases. It's
especially easy since sqlite is included with Python 2.5.

I am using mysql, and sqlite is not appropriate for my situation since
some of the databases and tables I access are being accessed by other
applications which are already written and live. I am not using the
SQLAlchemy or SQLObject, because I didn't want too (although in the
future I may consider using them). This question actually has nothing
to do with the database, I simply put the information on the database
in to give it some context.

cheers
Tim Henderson
 
D

Diez B. Roggisch

Tim said:
Hello

I am writing an application that has a mysql back end and I have this
idea to simplify my life when accessing the database. The idea is to
wrap the all the functions dealing with a particular row in a
particular in a particular table inside a class. So if you have a
table that looks like this:

id str1 str2 pickled_data1 pickled_data2
0 woeif aposf (bin) (bin)
1 ofime powe (bin) (bin)
...
n oiew opiwe (bin) (bin)

you can access this table like this

t = Table(id) #to load a pre-entered row
t2 = Table(id, str1, str2, data1, data2) #to create a new row

when you change a an attribute of the class like this...
t.str1 = 'new value'

it automatically updates the database backend.

I have what I just described working. However I want an easier way to
deal with my pickled_data. Right now I am pickling dictionaries and
list types. Now there is one problem with this, let me demonstrate

t.data.update({'new key':'new value'})
print t.data
{... 'new key':'new value' ...}

which makes it appear that the database has been updated as well, but
in fact it hasn't to update the database with this scheme you actually
have to do this.

t.data.update({'new key':'new value'})
t.data = t.data

this is not ideal so I subclassed the built in dict type like this:

class _my_dict(dict):

def __init__(self, row_index_name, row_index, column_name, a=None,
**kwargs):
self.row_index_name = row_index_name
self.row_index = row_index
self.column_name = column_name
self.write_access = True
if (a == None): dict.__init__(self, kwargs)
else: dict.__init__(self, a)

self.update_db()

def __delitem__(self, key):
if self.write_access:
dict.__delitem__(self, key)
self.update_db()

def __setitem__(self, key, value):
if self.write_access:
dict.__setitem__(self, key, value)
self.update_db()


def clear(self):
if self.write_access:
dict.clear(self)
self.update_db()

...
more methods which are simliar
...

def update_db(self):
if self.write_access:
con = get_dbConnection()
cur = con.cursor()

table = self.experiment.TABLE
row_index_name = self.row_index_name
row_index = self.row_index
column_name = self.column_name
column_value = MySQLdb.escape_string(pickle.dumps(self))

q1 = '''UPDATE %(table)s
SET %(column_name)s = '%(column_value)s'
WHERE %(row_index_name)s = '%(row_index)s' ''' % locals()

cur.execute(q1)
con.close()


Now while this works, it is a lot of work. What I want to be able to
do is something where I write one decorator function that
automatically updates the database for me. So let us pretend I have
this function.

let: dec_update_db() be my decorator which updates the dictionary.

to use this function it seems I would probably still have to subclass
dict like this:

class _my_dict2(dict):

@dec_update_db
def __init__(self, row_index_name, row_index, column_name, a=None,
**kwargs):
self.row_index_name = row_index_name
self.row_index = row_index
self.column_name = column_name
self.write_access = True
if (a == None): dict.__init__(self, kwargs)
else: dict.__init__(self, a)

@dec_update_db
def __delitem__(self, key):
dict.__delitem__(self, key)

@dec_update_db
def __setitem__(self, key, value):
dict.__setitem__(self, key, value)

@dec_update_db
def clear(self):
dict.clear(self)

... and so on ...

this is also not ideal. because I still have to apply the decorator to
every function which changes the dictionary.

What I really want is a way to have the decorator applied
automatically every time a method in dict or a sub class is called. I
feel like this must be possible. Has any one here done anything like
this before?

There are a few possibilities - one of them is using a metaclass to
apply the decorator to alle methods.

Diez
 
G

George Sakkis

I am using mysql, and sqlite is not appropriate for my situation since
some of the databases and tables I access are being accessed by other
applications which are already written and live. I am not using the
SQLAlchemy or SQLObject, because I didn't want too (although in the
future I may consider using them).

I'd strongly second looking into SQLAlchemy; from what you wrote, I
suspect sooner or later you'll end up creating (to paraphrase
Greenspun) an ad hoc, informally-specified, bug-ridden, slow
implementation of half of SQLAlchemy.

George
 
C

castironpi

Tim Henderson schrieb:







































There are a few possibilities - one of them is using a metaclass to
apply the decorator to alle methods.

Diez- Hide quoted text -

- Show quoted text -

I like his post. I thought that structures can be counter-primitive.
I don't know why lambda doesn't work. Where's the guy that thinks
everything in lambdas. I think that f*2=partial( partial( f ) ).
It's not: no 'of'.

I want to say that's on the internet. May I define multiplication of
operators? That's just order. Pick one. Do you play Civilization?
I am newsgroup.confuse over and out. Divide * conquer = cut *
run.decode().

I want the * to precede the dot too. Let's yack. I want to compile
Python. Did you see my new post? I like it. Do you have any time
you don't want? Time sale. Diez is still mad at me. I want
primitives to structure themselves so I can pick up a pen. I am
volunteer primitive structuring. Your structure sucks. assert
atmostphere has drags.

i don't think 'and' means the right thing. 'and' is composure. So
the above should be 'divide and conquer, normalize in the neighborhood
of, cut and run'. I want to run laps. Are generators good for it?
Can we punctuate differently Python? Try to explain to a 'stream' of
particles particle what it's doing. I'm particle, who's wave.
 
G

Gabriel Genellina

En Wed, 26 Mar 2008 18:01:31 -0300, George Sakkis
I'd strongly second looking into SQLAlchemy; from what you wrote, I
suspect sooner or later you'll end up creating (to paraphrase
Greenspun) an ad hoc, informally-specified, bug-ridden, slow
implementation of half of SQLAlchemy.

You beat me to say that, but using SQLObject instead.
Moreover, the OP is looking for the PersistentDict and PersistentList
classes that ZODB uses.
I'm not sure that triggering a database write for each and every modified
attribute is a good thing; I'd use a "dirty" flag instead (ZODB uses
_p_changed) and write all modifications at the end.
Since not all methods modify the object contents, using a metaclass to
flag all of them isn't a good idea either. I'd use a decorator; after all,
they're not so many methods; if you inherit from DictMixin you only have
to write __getitem__, __setitem__, __delitem__ and keys(), and of those,
only __setitem__ and __delitem__ modify the object.
 
C

castironpi

En Wed, 26 Mar 2008 18:01:31 -0300, George Sakkis  



You beat me to say that, but using SQLObject instead.
Moreover, the OP is looking for the PersistentDict and PersistentList  
classes that ZODB uses.
I'm not sure that triggering a database write for each and every modified  
attribute is a good thing; I'd use a "dirty" flag instead (ZODB uses  
_p_changed) and write all modifications at the end.
Since not all methods modify the object contents, using a metaclass to  
flag all of them isn't a good idea either. I'd use a decorator; after all,  
they're not so many methods; if you inherit from DictMixin you only have  
to write __getitem__, __setitem__, __delitem__ and keys(), and of those,  
only __setitem__ and __delitem__ modify the object.

That depends on technology: in volatile memory, persistence is a long-
term commitment. In static-state memory, it's a norm. Say you've got
a flash drive on the bus, you could store the objects themselves: the
Python module would have to allocate from the ROM, reobtain later.
Don't store the interpreter on the ROM, and trade-off feature sets for
expiration date. But standards expire?
 
A

alex23

I want the * to precede the dot too. Let's yack. I want to compile
Python. Did you see my new post? I like it. Do you have any time
you don't want? Time sale. Diez is still mad at me. I want
primitives to structure themselves so I can pick up a pen. I am
volunteer primitive structuring. Your structure sucks. assert
atmostphere has drags.

Could you possibly once, just once, put down the Robert Anton Wilson &
the meth pipe before you post?
 
B

Bruno Desthuilliers

Tim Henderson a écrit :
Hello

I am writing an application that has a mysql back end and I have this
idea to simplify my life when accessing the database. The idea is to
wrap the all the functions dealing with a particular row in a
particular in a particular table inside a class. So if you have a
table that looks like this:

id str1 str2 pickled_data1 pickled_data2
0 woeif aposf (bin) (bin)
1 ofime powe (bin) (bin)
...
n oiew opiwe (bin) (bin)

you can access this table like this

t = Table(id) #to load a pre-entered row
t2 = Table(id, str1, str2, data1, data2) #to create a new row

when you change a an attribute of the class like this...
t.str1 = 'new value'

it automatically updates the database backend.

Congratulation, you just reinvented ORM. Good news is that there are
already quite a few such packages in Python. May I recommand SQLAlchemy ?-)
I have what I just described working. However I want an easier way to
deal with my pickled_data. Right now I am pickling dictionaries and
list types. Now there is one problem with this,

Indeed, but not the one you mention. The problem is that storing
serialized dicts / lists / any other non atomic data in a blob is, well,
not exactly the best possible use of a *relational* database.

Do yourself a favour: learn what 'relational' means, and replace your
blobs with the appropriate tables in the database.

(snip code)
Now while this works, it is a lot of work.

This is why it's better to use an existing package whenever possible.
SQLAlchemy is here:
http://www.sqlalchemy.org/

(snip mode code)
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,990
Messages
2,570,211
Members
46,799
Latest member
Mercury_Dev

Latest Threads

Top