Mod_python vs. application server like CherryPy?

[Vincent Delporte]

Hi

I'm still a newbie when it comes to web applications, so would like
some help in choosing a solution to write apps with Python: What's the
difference between using running it through mod_python vs. building an
application server using Python-based tools like CherryPy, Quixote,
Draco, etc.?

Thanks.

 

[fumanchu]

I'm still a newbie when it comes to web applications, so would like
some help in choosing a solution to write apps with Python: What's the
difference between using running it through mod_python vs. building an
application server using Python-based tools like CherryPy, Quixote,
Draco, etc.?

Well, let me start by saying that anything you can build with CherryPy,
you can build with mod_python. In a nutshell, mod_python gives you
access from Python to the Apache API, whereas CherryPy and friends give
you their own API.

I joined the CherryPy development team because I liked CherryPy's API
better, and at the time, needed to deploy my site on IIS, not Apache. I
continue to use the same site, written with CherryPy, but now using
Apache (and even mod_python!) to serve it. CherryPy allows me to focus
on the application layer and leave the server/deployment layer for
another day.

In other words, there's nothing about mod_python that leaves it out of
the "application server" category per se. The publisher handler, in
particular, is an example of an "application server" built on top of
mod_python's base API, and has some strong similarities to CherryPy's
traversal and invocation mechanisms. But IMO CherryPy has a cleaner API
for process control (engines and servers), application composition (via
the object tree and via WSGI), and plugins (like gzip, static content,
and header management).


Robert Brewer
System Architect
Amor Ministries
(e-mail address removed)

 

[Vincent Delporte]

In a nutshell, mod_python gives you
access from Python to the Apache API, whereas CherryPy and friends give
you their own API.

I didn't know Apache had an API of its own, or that it was even needed
when writing a web application in Python. What does it provide in
addition to Python/mod_python?

CherryPy allows me to focus
on the application layer and leave the server/deployment layer for
another day.

So you recommend using Apache as the front-end, and run an application
server like CherryPy in the background?

But IMO CherryPy has a cleaner API
for process control (engines and servers), application composition (via
the object tree and via WSGI), and plugins (like gzip, static content,
and header management).

Interesting. I'll see if I can find more information on writing an app
with Python in pure CGI, in FastCGI, in mod_python, and as an
application server with eg. CherryPy.

Thanks.

 

[Paul Boddie]

I didn't know Apache had an API of its own, or that it was even needed
when writing a web application in Python. What does it provide in
addition to Python/mod_python?

mod_python provides different layers of APIs, with the lowest levels
being those dealing with things like connections and requests - see the
apache module for details:

http://www.modpython.org/live/current/doc-html/module-apache.html

Other layers involve things like cookies, sessions and Python server
pages, and there are also things like the publisher handler which are
less to do with Apache itself and more to do with Python:

http://www.modpython.org/live/current/doc-html/hand-pub.html

Paul

 

[Graham Dumpleton]

I didn't know Apache had an API of its own, or that it was even needed
when writing a web application in Python. What does it provide in
addition to Python/mod_python?

Although Apache does have a quite considerable underlying API of its
own, mod_python doesn't actually provide direct access to much of it
from Python code. What mod_python does provide is still adequate for
getting basic stuff done, but Apache could perhaps be better harnessed
if all the Apache API were exposed and available.

Where the power of Apache comes into play is the ability to compose
together the functionality of different Apache modules to build up an
application. That is, you aren't just doing everything in Python code.
That said though, this doesn't mean you have to go off and write code
in another language such as C. This is because the Apache modules are
glued together through the Apache configuration files with some
features also being able to be enabled from Python code running under
mod_python.

In some respects you need to see the whole of Apache as being a
platform for building a web application. Unfortunately, most Python web
application developers don't see that and simply use Apache as a
convenient hopping off point for the main content handling phase of a
request. Even where people do write stuff which makes use of mod_python
as more than just a hopping off point, more often than not they still
work within just mod_python and don't bring in other parts of Apache to
any degree.

For example, consider an extreme case such as WSGI. Through a goal of
WSGI being portability it effectively ignores practically everything
that Apache has to offer. Thus although Apache offers support for
authentication and authorisation, a WSGI user would have to implement
this functionality themselves or use a third party WSGI component that
does it for them. Another example is Apache's support for enabling
compression of content returned to a client. The WSGI approach is again
to duplicate that functionality. Similarly with other Apache features
such as URL rewriting, proxying, caching etc etc.

Although WSGI is an extreme case because of the level it pitches at,
other systems such as CherryPy and Django aren't much different as they
effectively duplicate a lot of stuff that could be achieved using more
basic functionality of Apache as well. Once one starts to make use of
the underlying Apache functionality, you are binding yourself to Apache
though and your stuff isn't portable to other web servers. Also, your
job in part becomes more about integrating stuff to come up with a
solution, rather than just writing pure Python code, something that
many Python coders possibly wouldn't find appealing.

Graham

 

[Vincent Delporte]

Although WSGI is an extreme case because of the level it pitches at,
other systems such as CherryPy and Django aren't much different as they
effectively duplicate a lot of stuff that could be achieved using more
basic functionality of Apache as well.

Mmm... So how can I use those goodies from Apache? Just through their
configuration files, or do I have to somehow call them from Python?

Is the fact that Python developers tend to ignore resources in Apach
due to difficulties in making calls from Python, making the scripts
unpythonic?

 

[Graham Dumpleton]

Mmm... So how can I use those goodies from Apache? Just through their
configuration files, or do I have to somehow call them from Python?

Is the fact that Python developers tend to ignore resources in Apach
due to difficulties in making calls from Python, making the scripts
unpythonic?

It depends on what you are doing. For example, if you need to do URL
rewriting, you use the mod_rewrite module for Apache, not that it is
the most pleasant thing to use. If you need to proxy through some
subset of requests to a downstream web server, use mod_proxy. If you
need to compress content going back to clients, use mod_deflate. If you
need to do caching you use mod_cache.

How to configure each of these from the Apache configuration files, you
need to look at the Apache httpd documentation at httpd.apache.org.

Some, like mod_proxy and mod_deflate can be triggered from within
mod_python although finding the magic required isn't always straight
forward. How you setup responses can also control mod_cache.

If anything, the reason that Python developers tend to ignore a lot of
what Apache has to offer is that it means understanding Apache. The
Apache documentation isn't always the easiest thing to understand and
for some things it even requires looking at the Apache source code to
work out how to do things. The mod_python documentation at the moment
doesn't help either, as it doesn't provide much in the way of recipes
for doing things. The new mod_python wiki will hopefully address that
over time, but right now the mod_python mailing lists are the best it
gets.

In terms of whether it is 'unpythonic', what should be more important
is whether it gets the job done in a way that makes best use of what is
available. If you want something to be 'pythonic', then using Apache as
a framework probably isn't what you want, as as I said previously it
becomes more about integrating things rather than writing pure Python
code.

Getting perhaps back to the answer you were seeking right back at the
start, that is if you are new to web application and development and
Python, then you may well be better of just using a higher level
framework as they will make it easier and isolate you from any pains in
understanding Apache and how to use it properly.

Graham

 

[Vincent Delporte]

Getting perhaps back to the answer you were seeking right back at the
start, that is if you are new to web application and development and
Python, then you may well be better of just using a higher level
framework as they will make it easier and isolate you from any pains in
understanding Apache and how to use it properly.

Thanks a lot for the feedback. It's beginning to make sense

 

[fumanchu]

For example, consider an extreme case such as WSGI.
Through a goal of WSGI being portability it effectively
ignores practically everything that Apache has to offer.
Thus although Apache offers support for authentication
and authorisation, a WSGI user would have to implement
this functionality themselves or use a third party WSGI
component that does it for them. Another example is
Apache's support for enabling compression of content
returned to a client. The WSGI approach is again to
duplicate that functionality. Similarly with other Apache
features such as URL rewriting, proxying, caching etc etc.

Well, almost. I use Auth* directives for authentication (and the
Require directive for authorization) with my CherryPy apps. Many other
CP users use mod_rewrite and mod_proxy. So the WSGI user doesn't *have*
to implement that functionality themselves. Any sufficiently-messianic
framework will probably do so , but even the best admit there are
always alternatives.


Robert Brewer
System Architect
Amor Ministries
(e-mail address removed)

 

[Damjan]

For example, consider an extreme case such as WSGI. Through a goal of

WSGI being portability it effectively ignores practically everything
that Apache has to offer. Thus although Apache offers support for
authentication and authorisation, a WSGI user would have to implement
this functionality themselves or use a third party WSGI component that
does it for them.

OTOH
WSGI auth middleware already supports more auth methods than apache2 itself.

Another example is Apache's support for enabling
compression of content returned to a client. The WSGI approach is again
to duplicate that functionality.

the gzip middleware is really just an example... nobody would use that in
production.

Similarly with other Apache features
such as URL rewriting, proxying, caching etc etc.

Well, not everybody can use Apache ... and again there's already WSGI
middleware that's more flexible than the Apache modules for most of the
features you mention.

It's not that I think mod_python doesn't have uses.. I just think it's not
practical to make python web applications targeted solely to mod_python.

 

[Graham Dumpleton]

OTOH
WSGI auth middleware already supports more auth methods than apache2 itself.

A distinction needs to be made here. HTTP supports Basic and Digest
authentication mechanisms, both of which are in Apache by default. The
authentication mechanism though needs to be seen as distinct from the
auth provider, that is, who actually validates that a user is valid.
Apache 2.2 separates these two things with there being a pluggable auth
provider facility with backend implementations for such things as
passwd like files, dbm database, ldap and using mod_dbd various SQL
databases. Because it is pluggable it can be extended to support any
sort of auth provider implementation you want. It is even technically
possibly to write an auth provider which makes used of Python to
perform the check using some custom system, although mod_python itself
doesn't provide this yet, so you need to roll your own auth module to
do it.

Even as far as authentication mechanisms go, you aren't limited to just
those as the fact that you can provide a complete authentication and/or
authorisation handler means you can implement other as well. You might
for example build on top of SSL and use client certificates to control
access, or you could use HTTP forms based login and sessions. These
custom mechanisms could also use the auth provider plugin system so
that they aren't dependent on one particular user validation mechanism.

Now, when you say that WSGI already supports more auth methods than
Apache 2 itself, are you referring to how the login/authentication is
handled over HTTP, or how client validation is handled, ie., auth
providers.

I am not being critical here, asking more to build my knowledge of WSGI
and what capabilities it does have.

Well, not everybody can use Apache ... and again there's already WSGI
middleware that's more flexible than the Apache modules for most of the
features you mention.

It's not that I think mod_python doesn't have uses.. I just think it's not
practical to make python web applications targeted solely to mod_python.

For what you are doing that may not be practical, but in a lot of other
places it would be a more than reasonable way of going about it. To
clarify though, I am not talking about building something which is
completed implemented within the context of mod_python alone and not
even something that is written in Python exclusively. What I have been
talking about is using Apache as a whole as the platform.

Thus, taking authentication as an example, for basic forms of
authentication it is best to use the standard mod_auth and auth
provider facilities of Apache to do it. For more complicated mechanisms
such as using HTTP form, more customisation is usually required and
this might be implemented using Python under mod_python but could just
as well be implemented in mod_perl. A main thing to note though is that
if Apache authentication handlers are written properly, it can be used
to span not just mod_python but apply to static files served by Apache,
as well as handlers which serve up content using other languages
modules such as PHP, mod_perl.

This is where I said it can be more about integration rather than
writing pure Python code, because if you use Apache and mod_python
properly, you don't end having to write code within a monoculture
consisting of one implementation language as you are if you use WSGI.
Instead, you can pick and choose the modules and languages which make
the most sense as far as allowing you to implement something in the
easiest way possible given what you have and the environment you are
operating in

All I can say is that since this is a Python language list, that many
here will be here because they want to program specifically in Python
and nothing else. Others though may well see the bigger picture and the
realities of working with big systems, especially in a corporate
environment, where one doesn't have a choice but to deal with code
developed over time and in different languages with a result that most
of the time you are writing more glue and than actual application code.
Thus, in an ideal world you might be able to write in Python and
nothing else, but it isn't always an ideal world.

Graham