Python ORMs Supporting POPOs and Substituting Layers in Django

[Travis Parks]

Hello:

A new guy showed up at work a few weeks ago and has started talking
about replacing a 6 month old project, written in ASP.NET MVC, with an
open source solution that can handle massive scaling. I think his
primary concern is the "potential" need for massive web farms in the
future. In order to prevent high licensing costs, I think he wants to
move everything to open source technologies, such as the LAMP stack. I
also don't think he truly understands what ASP.NET MVC is and thinks
it is the older WebForms.

I have been researching open source MVC frameworks and came across
Django. It looks like an awesome tool, but I am willing to look at
others. I have experience in Python (and enough in PHP to want to
avoid it and absolutely none in Ruby) so I think it would be a good
language to develop in.

I was wondering if there were any ORMs for Python that used POPOs
(plain old Python objects). There is a lot of business logic in my
system, and so I want to keep my data objects simple and stupid. I
want the ORM to be responsible for detecting changes to objects after
I send them back to the data layer (rather than during business layer
execution). Additionally, being a stateless environment, tracking
objects' states isn't very useful anyway.

Honestly, I doubt this guy is going to get his wish. The people paying
for the application aren't going to be willing to throw 6 months of
work down the drain. Never the less, I want to have plenty of research
under my belt before being asked what my thoughts are. He was talking
about using the Zend Framework with PHP, but I want to avoid that if
possible. Django seems like one of the best MVC solutions in the
Python arena. I would be willing to replace Django's ORM solution with
something else, especially if it supported POPOs. I could even map all
of the non-POPOs to POPOs if I needed to, I guess.

Finally, I wanted to ask whether anyone has tried having Django call
out to Python 3 routines. I am okay using Python 2.7 in Django, if I
can have the controllers call business logic implemented in Python 3,
accepting POPOs from the data layer. Django would really just be a
coordinator: grab data from Django ORM, convert results into POPOs,
load up Python 3 module with business logic, passing POPOs, returning
POPOs and then converting those to view models. I'm sweating just
thinking about it. My guess is that there would be a severe penalty
for crossing process boundaries... but any insights would be
appreciated.

Thanks,
Travis Parks

 

[Marco Nawijn]

Hello:

A new guy showed up at work a few weeks ago and has started talking
about replacing a 6 month old project, written in ASP.NET MVC, with an
open source solution that can handle massive scaling. I think his
primary concern is the "potential" need for massive web farms in the
future. In order to prevent high licensing costs, I think he wants to
move everything to open source technologies, such as the LAMP stack. I
also don't think he truly understands what ASP.NET MVC is and thinks
it is the older WebForms.

I have been researching open source MVC frameworks and came across
Django. It looks like an awesome tool, but I am willing to look at
others. I have experience in Python (and enough in PHP to want to
avoid it and absolutely none in Ruby) so I think it would be a good
language to develop in.

I was wondering if there were any ORMs for Python that used POPOs
(plain old Python objects). There is a lot of business logic in my
system, and so I want to keep my data objects simple and stupid. I
want the ORM to be responsible for detecting changes to objects after
I send them back to the data layer (rather than during business layer
execution). Additionally, being a stateless environment, tracking
objects' states isn't very useful anyway.

Honestly, I doubt this guy is going to get his wish. The people paying
for the application aren't going to be willing to throw 6 months of
work down the drain. Never the less, I want to have plenty of research
under my belt before being asked what my thoughts are. He was talking
about using the Zend Framework with PHP, but I want to avoid that if
possible. Django seems like one of the best MVC solutions in the
Python arena. I would be willing to replace Django's ORM solution with
something else, especially if it supported POPOs. I could even map all
of the non-POPOs to POPOs if I needed to, I guess.

Finally, I wanted to ask whether anyone has tried having Django call
out to Python 3 routines. I am okay using Python 2.7 in Django, if I
can have the controllers call business logic implemented in Python 3,
accepting POPOs from the data layer. Django would really just be a
coordinator: grab data from Django ORM, convert results into POPOs,
load up Python 3 module with business logic, passing POPOs, returning
POPOs and then converting those to view models. I'm sweating just
thinking about it. My guess is that there would be a severe penalty
for crossing process boundaries... but any insights would be
appreciated.

Thanks,
Travis Parks

Hello Travis,

I am not an expert in the field, but I have used SQLAlchemy
(www.sqlalchemy.org)
for a while and was very happy with it. It should be able to scale up
to pretty
complex applications and large amounts of data.

Regards,

Marco

 

[Chris Angelico]

Finally, I wanted to ask whether anyone has tried having Django call
out to Python 3 routines. I am okay using Python 2.7 in Django, if I
can have the controllers call business logic implemented in Python 3,
accepting POPOs from the data layer.

What you're going to have is completely separate processes; either
invoking Python3 and having it terminate after one action, or keeping
it running concurrently and passing data between them.

Has anyone ever extended AND embedded Python at the same time? Written
a Python module that ... executes Python code? Might be a bit tricky,
but possibly isolating the Py2 and Py3 code into different C source
files would help conserve programmer sanity (at the possible cost of
performance). Would be a tricky thing to juggle, but if you can pull
it off, you'd be able to - effectively - have a module in Django
that's written in Python 3, and directly callable.

Propagating parameters and return values would be a matter of
unpacking them into C objects and repacking them on the other side.
Propagating exceptions would be.... fun?

ChrisA

 

[Travis Parks]

Hello:

A new guy showed up at work a few weeks ago and has started talking
about replacing a 6 month old project, written in ASP.NET MVC, with an
open source solution that can handle massive scaling. I think his
primary concern is the "potential" need for massive web farms in the
future. In order to prevent high licensing costs, I think he wants to
move everything to open source technologies, such as the LAMP stack. I
also don't think he truly understands what ASP.NET MVC is and thinks
it is the older WebForms.

I have been researching open source MVC frameworks and came across
Django. It looks like an awesome tool, but I am willing to look at
others. I have experience in Python (and enough in PHP to want to
avoid it and absolutely none in Ruby) so I think it would be a good
language to develop in.

I was wondering if there were any ORMs for Python that used POPOs
(plain old Python objects). There is a lot of business logic in my
system, and so I want to keep my data objects simple and stupid. I
want the ORM to be responsible for detecting changes to objects after
I send them back to the data layer (rather than during business layer
execution). Additionally, being a stateless environment, tracking
objects' states isn't very useful anyway.

Honestly, I doubt this guy is going to get his wish. The people paying
for the application aren't going to be willing to throw 6 months of
work down the drain. Never the less, I want to have plenty of research
under my belt before being asked what my thoughts are. He was talking
about using the Zend Framework with PHP, but I want to avoid that if
possible. Django seems like one of the best MVC solutions in the
Python arena. I would be willing to replace Django's ORM solution with
something else, especially if it supported POPOs. I could even map all
of the non-POPOs to POPOs if I needed to, I guess.

Finally, I wanted to ask whether anyone has tried having Django call
out to Python 3 routines. I am okay using Python 2.7 in Django, if I
can have the controllers call business logic implemented in Python 3,
accepting POPOs from the data layer. Django would really just be a
coordinator: grab data from Django ORM, convert results into POPOs,
load up Python 3 module with business logic, passing POPOs, returning
POPOs and then converting those to view models. I'm sweating just
thinking about it. My guess is that there would be a severe penalty
for crossing process boundaries... but any insights would be
appreciated.

Thanks,
Travis Parks

Which web frameworks have people here used and which have they found
to be: scalable, RAD compatible, performant, stable and/or providing
good community support? I am really trying to get as much feedback as
I can, to help form an unbiased opinion in case I need to make a
recommendation... or fight an all out battle.

 

[John Gordon]

Which web frameworks have people here used and which have they found
to be: scalable, RAD compatible, performant, stable and/or providing
good community support? I am really trying to get as much feedback as

I've used Django and it seems to be a very nice framework. However I've
only done one project so I haven't delved too deeply.

 

[John Gordon]

I've used Django and it seems to be a very nice framework. However I've
only done one project so I haven't delved too deeply.

You are probably looking for more detail than "It's a nice framework"

The database model in Django is powerful; it allows you to do queries in
native Python code without delving into backend SQL stuff.

I don't know how scalable/performant the database model is, as the one
project I worked on didn't deal with a ton of data. (But I'd be surprised
if it had poor performance.)

The URL dispatcher provides a very nice and logical way to associate a
given URL with a given method call.

Community support is excellent.

 

[Travis Parks]

You are probably looking for more detail than "It's a nice framework"

The database model in Django is powerful; it allows you to do queries in
native Python code without delving into backend SQL stuff.

I don't know how scalable/performant the database model is, as the one
project I worked on didn't deal with a ton of data.  (But I'd be surprised
if it had poor performance.)

The URL dispatcher provides a very nice and logical way to associate a
given URL with a given method call.

Community support is excellent.

I started the battle today. The "new guy" was trying to sell me on
CodeIgnitor. I haven't looked at it, but it is PHP, so I really want
to avoid it. The good thing is that all of his "friends" have been
telling him to get into Python. I have been trying to convince him
that PHP isn't cut out for background services and is mostly a front-
end language. Python is much more geared towards hardcore data
processing. Why write the system in two languages?

I have been spending a lot of time looking at the Pyramid project: the
next generation of the Pylons project. It looks powerful, but it seems
to be a lot more complex than Django.

 

[Lie Ryan]

I started the battle today. The "new guy" was trying to sell me on
CodeIgnitor. I haven't looked at it, but it is PHP, so I really want
to avoid it. The good thing is that all of his "friends" have been
telling him to get into Python. I have been trying to convince him
that PHP isn't cut out for background services and is mostly a front-
end language. Python is much more geared towards hardcore data
processing. Why write the system in two languages?

I have been spending a lot of time looking at the Pyramid project: the
next generation of the Pylons project. It looks powerful, but it seems
to be a lot more complex than Django.

CodeIgniter is a very fine framework, however it builds on top of a
shitty excuse of a language called PHP.

I've found that Django has a much better debugging tools; when a Django
page produces an exception, it would always produce a useful error page.
I haven't been able to do the same in CodeIgniter (nor in any PHP
framework I've used, I'm starting to think it's a language limitation);
often when you have errors, PHP would just silently return empty or
partial pages even with all the debugging flags on.

IMO, Python has a much nicer choice of built-in data structure for data
processing. Python has a much more mature object-orientation, e.g. I
prefer writing l.append(x) rather than array_push(l, x). I think these
qualities are what makes you think Python is much, much more suitable
for data processing than PHP; and I wholesomely agree.

Database abstraction-wise, Django's ORM wins hands down against
CodeIgniter's ActiveRecord. CodeIgniter's ActiveRecord is basically just
a thin wrapper that abstracts the perks of various database engine.
Django's ORM is a full blown ORM, it handles foreign key relationships
in OO way. The only disadvantage of Django's ORM is that since it's
written in Python, if you need to write a program working on the same
database that doesn't use Django nor Python, then you'll have a problem
since you'll have to duplicate the foreign key relationships.

With all the bashing of PHP, PHP do have a few advantages. PHP and
CodeIgniter is much easier to set up and running than Django; and the
ability to create a .php file and have it running without having to
write the routing file is sometimes a bliss. And PHP are often used as
their own templating language; in contrast with Django which uses a
separate templating language. Having a full blown language as your
templating language can be a double-edged sword, but it is useful
nevertheless for experimental work.

IMO, while it is easier to get up and running in PHP, in the long run
Python is much better in almost any other aspects.

 

[Chris Angelico]

IMO, Python has a much nicer choice of built-in data structure for data
processing. Python has a much more mature object-orientation, e.g. I prefer
writing l.append(x) rather than array_push(l, x). I think these qualities
are what makes you think Python is much, much more suitable for data
processing than PHP; and I wholesomely agree.

Two more examples where Python's lists are superior to PHP's arrays.
Array literal syntax feels like a function call, but list literals are
slim and easy to use inside expressions (try creating a nested array
as a function argument - you'll get a forest of parens). Also,
dereferencing an array only works on an array variable - if you have a
function that returns an array, you can't dereference it directly:

$foo = func()[1]; # doesn't work
$foo = func(); $foo=$foo[1]; # works

I much prefer the "everything's an object" notion. C's array literals
are just as weird (although in C, you can directly dereference a
literal character array - "ABCDEFG"[note_idx] will give you a note
name as a char)... much easier when a variable name is just an
expression, a function call is an expression, a literal is an
expression, and you can work with them all the same way.

ChrisA

 

[Dave Angel]

<SNIP>
I much prefer the "everything's an object" notion. C's array literals
are just as weird (although in C, you can directly dereference a
literal character array - "ABCDEFG"[note_idx] will give you a note
name as a char)

Hey, in C you can also do

note_idx["ABCDEFG"]

and get the selected character as if you had written what you showed.
Plenty of opportunity in C to write illegible code.

 

[Travis Parks]

CodeIgniter is a very fine framework, however it builds on top of a
shitty excuse of a language called PHP.

I've found that Django has a much better debugging tools; when a Django
page produces an exception, it would always produce a useful error page.
I haven't been able to do the same in CodeIgniter (nor in any PHP
framework I've used, I'm starting to think it's a language limitation);
often when you have errors, PHP would just silently return empty or
partial pages even with all the debugging flags on.

IMO, Python has a much nicer choice of built-in data structure for data
processing. Python has a much more mature object-orientation, e.g. I
prefer writing l.append(x) rather than array_push(l, x). I think these
qualities are what makes you think Python is much, much more suitable
for data processing than PHP; and I wholesomely agree.

Database abstraction-wise, Django's ORM wins hands down against
CodeIgniter's ActiveRecord. CodeIgniter's ActiveRecord is basically just
a thin wrapper that abstracts the perks of various database engine.
Django's ORM is a full blown ORM, it handles foreign key relationships
in OO way. The only disadvantage of Django's ORM is that since it's
written in Python, if you need to write a program working on the same
database that doesn't use Django nor Python, then you'll have a problem
since you'll have to duplicate the foreign key relationships.

With all the bashing of PHP, PHP do have a few advantages. PHP and
CodeIgniter is much easier to set up and running than Django; and the
ability to create a .php file and have it running without having to
write the routing file is sometimes a bliss. And PHP are often used as
their own templating language; in contrast with Django which uses a
separate templating language. Having a full blown language as your
templating language can be a double-edged sword, but it is useful
nevertheless for experimental work.

IMO, while it is easier to get up and running in PHP, in the long run
Python is much better in almost any other aspects.- Hide quoted text -

- Show quoted text -

The good thing is that I got the new guy to convert his thinking
towards Python. He did a little research of his own and realized what
he was missing.

He and I have been writing some tools in Python, accessing social
networking sites, and have been pleasantly surprised by Python's rich
support for web protocols. Even yesterday, I wrote some code using
xmlrpclib and it blew the equivalent C# code out of the water. urllib2
and urlparse make it really easy to work against RESTful services. It
seems like most of Google's APIs have a Python variant.

We are thinking we will go along with Pyramid, rather than Django. It
was a really hard decision to make. Django has a lot of community
support and is integrated with PyDev in eclipse. Nonetheless, we are
anticipating the need for massive through-put on our web servers, so
we want to make sure we don't have to change gears 2 years down the
road when we have 100,000+ users. Furthermore, it seems like Django
doesn't allow for customization - we might want to switch between ORMs
and the template engine.

It is going to be an interesting process of moving from an ASP.NET MVC
application to 100% Python. We'll see how well we can scaffold them
together.