Dr. Dobb's Python-URL! - weekly Python news and links (Aug 12)

[Cameron Laird]

QOTW: "... So I started profiling the code and the slowdown was actually
taking place at places where I didn't expect it." -- Guyon Mor?e (and about
twenty-three thousand others)

"[A] suggestion from the world of 'agile development': stop making so many
decisions and start writing some actual code!" -- Peter Hansen


Scott David Daniels and others illustrate that the most common
answer for Python is, "It's (already) in there." In the case at
hand the subject is primitive symbolic arithmetic:
http://groups.google.com/group/comp.lang.python/index/browse_frm/thread/13ed519653969e55/

Andy Smith rails against "frameworks":
http://an9.org/devdev/why_frameworks_suck?sxip-homesite=&checked=1

"Porcupine is a [Python-based] Web application server that features
an embedded object database, the Porcupine Object Query Language,
XMLRPC support, and QuiX, an integrated JavaScript XUL motor."
http://www.innoscript.org

Getters and setters are (mostly) for other languages; Python has
properties (and descriptors!):
http://groups.google.com/group/comp.lang.python/browse_thread/thread/f903ab167c91e6ce/

Ramza Brown confusingly but helpfully repackages "the absolute
minimum libraries needed for Python to work with Win32":
http://groups.google.com/group/comp.lang.python/browse_thread/thread/d0d4e4303e985729/

Paolino offers a technique for *pruning* attributes in a subclass:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/1805446521e8f34e/

Hard though it may be for some of us to accept, a zillion threads
might not be all bad.
http://www.usenix.org/events/hotos03/tech/vonbehren.html

Python helps win awards:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/c6190563601c437f/

Python fits on small machines, sort-of:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/834f5b571be0b5f6/


========================================================================
Everything Python-related you want is probably one or two clicks away in
these pages:

Python.org's Python Language Website is the traditional
center of Pythonia
http://www.python.org
Notice especially the master FAQ
http://www.python.org/doc/FAQ.html

PythonWare complements the digest you're reading with the
marvelous daily python url
http://www.pythonware.com/daily
Mygale is a news-gathering webcrawler that specializes in (new)
World-Wide Web articles related to Python.
http://www.awaretek.com/nowak/mygale.html
While cosmetically similar, Mygale and the Daily Python-URL
are utterly different in their technologies and generally in
their results.

For far, FAR more Python reading than any one mind should
absorb, much of it quite interesting, several pages index
much of the universe of Pybloggers.
http://lowlife.jp/cgi-bin/moin.cgi/PythonProgrammersWeblog
http://www.planetpython.org/
http://mechanicalcat.net/pyblagg.html

comp.lang.python.announce announces new Python software. Be
sure to scan this newsgroup weekly.
http://groups.google.com/groups?oi=djq&as_ugroup=comp.lang.python.announce

Steve Bethard, Tim Lesher, and Tony Meyer continue the marvelous
tradition early borne by Andrew Kuchling, Michael Hudson and Brett
Cannon of intelligently summarizing action on the python-dev mailing
list once every other week.
http://www.python.org/dev/summary/

The Python Package Index catalogues packages.
http://www.python.org/pypi/

The somewhat older Vaults of Parnassus ambitiously collects references
to all sorts of Python resources.
http://www.vex.net/~x/parnassus/

Much of Python's real work takes place on Special-Interest Group
mailing lists
http://www.python.org/sigs/

Python Success Stories--from air-traffic control to on-line
match-making--can inspire you or decision-makers to whom you're
subject with a vision of what the language makes practical.
http://www.pythonology.com/success

The Python Software Foundation (PSF) has replaced the Python
Consortium as an independent nexus of activity. It has official
responsibility for Python's development and maintenance.
http://www.python.org/psf/
Among the ways you can support PSF is with a donation.
http://www.python.org/psf/donate.html

Kurt B. Kaiser publishes a weekly report on faults and patches.
http://www.google.com/groups?as_usubject=weekly python patch

Cetus collects Python hyperlinks.
http://www.cetus-links.org/oo_python.html

Python FAQTS
http://python.faqts.com/

The Cookbook is a collaborative effort to capture useful and
interesting recipes.
http://aspn.activestate.com/ASPN/Cookbook/Python

Among several Python-oriented RSS/RDF feeds available are
http://www.python.org/channews.rdf
http://bootleg-rss.g-blog.net/pythonware_com_daily.pcgi
http://python.de/backend.php
For more, see
http://www.syndic8.com/feedlist.php?ShowMatch=python&ShowStatus=all
The old Python "To-Do List" now lives principally in a
SourceForge reincarnation.
http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse
http://python.sourceforge.net/peps/pep-0042.html

The online Python Journal is posted at pythonjournal.cognizor.com.
(e-mail address removed) and (e-mail address removed)
welcome submission of material that helps people's understanding
of Python use, and offer Web presentation of your work.

del.icio.us presents an intriguing approach to reference commentary.
It already aggregates quite a bit of Python intelligence.
http://del.icio.us/tag/python

*Py: the Journal of the Python Language*
http://www.pyzine.com

Archive probing tricks of the trade:
http://groups.google.com/groups?oi=djq&as_ugroup=comp.lang.python&num=100
http://groups.google.com/groups?meta=site=groups&group=comp.lang.python.*

Previous - (U)se the (R)esource, (L)uke! - messages are listed here:
http://www.ddj.com/topics/pythonurl/ (requires subscription)
http://groups-beta.google.com/groups?q=python-url+group:comp.lang.python*&start=0&scoring=d&
http://purl.org/thecliff/python/url.html (dormant)
or
http://groups.google.com/groups?oi=djq&as_q=+Python-URL!&as_ugroup=comp.lang.python


Suggestions/corrections for next week's posting are always welcome.
E-mail to <[email protected]> should get through.

To receive a new issue of this posting in e-mail each Monday morning
(approximately), ask <[email protected]> to subscribe. Mention
"Python-URL!".


-- The Python-URL! Team--

Dr. Dobb's Journal (http://www.ddj.com) is pleased to participate in and
sponsor the "Python-URL!" project.

 

[Rocco Moretti]

Andy Smith rails against "frameworks":
http://an9.org/devdev/why_frameworks_suck?sxip-homesite=&checked=1

Slapdash Summary: Libraries good, frameworks bad - they are a
straightjackets and limit sharing.

Which lead me to the question - what's the difference between a library
and a framework?

 

[Simon Brunning]

Which lead me to the question - what's the difference between a library
and a framework?

If you call its code, it's a library. If it calls yours, it's a framework.

 

[Magnus Lycka]

If you call its code, it's a library. If it calls yours, it's a framework.

Erh, it's not quite that simple. I don't know of any frameworks where
you never call code that's part of the framework. I suppose there are
libraries that e.g. use callbacks without being frameworks.

Roughly, Simon is right though. Another way of saying it is that when
you don't use a framework, you write the main function (often a main
loop) of the application, you define the program's structure, and you
might use one or more libraries for certain functions in your code. A
framework, on the other hand, will provide the main function (e.g.
an event loop) and the basic structure of the application, and you fill
in the missing pieces with your code, to make the program do what you
desire. You might still use one or more libraries in your framework-
based app. You're not likely to use more than one framework at the
same time though. That would often be like trying to sit in two
different cars at the same time.

Besides that, people usually mean big thingies when they talk about
frameworks. For instance, Python's SocketServer and similar classes
share a lot of features with frameworks. These base classes implement
the basic logic flow of the applications you use them in. Just like in
a framework, you just fill in the missing pieces by implementing some
methods in your subclasses. Still, I never heard anyone call these
classes frameworks. Maybe they're to small and simple to qualify...
(GoF [ISBN:0201633612] calls this Design Pattern "Template Method",
but these server classes aren't just instances of that pattern,
you also use their main loops, so they *are* really tiny frameworks.)

Examples of popular Python frameworks are Zope and Twisted.

Frameworks cause some problems:
* It's non-trivial to use several frameworks at the same time, since
frameworks typically own event loops and mandate program structures.
It's not impossible though, e.g. Twisted can use a number of different
event loops (e.g. those of GTK+, Cocoa and Qt).

* You typically need to learn a lot about a framework before you can
use it well. It's much more easy to approach a library piece by piece.
This leads to a steep learning curve.

On the other hand, a well written framework should allow you to write
code which is useful outside that framework, and in other frameworks.
Using frameworks should not prevent you from writing generic and
reusable code. If it does (more than the average library), there is
something wrong with framework, or with the way you use it.

I think Andy is wrong there. He might not be wrong in practice when
it concerns certain frameworks. E.g. Zope Products are (as far as I
understand) only useful in Zope, but on the other hand, I guess that
most Zope Products would be best written as a generic library with
no Zopeisms in it, and a Zope wrapper outside that. (I know some
database adapters are written like that.)

I also think that it can be great fun to write Python code using a
framework. If a framework helps me focus on implementing the logic
I need to create, and can take care of low level junk, and also makes
my app more flexible, I'll enjoy that, even if I am somewhat restricted
in how I code. After all, if you write a big app, and don't use a
framework written by someone else, you will basically end up writing
your own framework...and that will probably restrain you as much as
a framework written by others.

In general, code is more reusable if we can avoid reliance on third
party code, whether it's libraries or frameworks, but when there are
such dependencies, framework dependencies typically cause bigger sharing
problems than library dependencies. Of course, it's better if we only
depend on something more abstract, that can be implemented in various
ways. Both Python's duck typing and concepts such as interfaces and
adapters can help us there.

It's interesting that Twisted has started to use Zope's interfaces.
We'll see if this will lead to python components that are useful in
both Zope, Twisted and in future apps and frameworks that use those
interfaces.

I'm just starting to use Twisted myself, even if I started to look at
it previously. I never really liked Zope, but I have some hopes for
Zope 3. I guess I might be much more enthusiatic, or much more
pessimistic about frameworks on a few months (or sooner)...

So far, Twisted looks fun. It has a lot of cool features, and I think
it will help me make the application I'm about to write just as flexible
as it needs to be, without too much headache.

 

[Rocco Moretti]

If you call its code, it's a library. If it calls yours, it's a framework.

Although that definition probably makes sense from a pure C perspective,
where practically everything you deal with is a primitive, I'm not sure
how much use the distinction is with a OO/duck typing language like Python.

Say you have a "library" of objects - you (as a user) subclass one and
change one of it's functions subtly (say to add logging). Now when the
library code runs, it'll call into your code. Voila! Framework.

Or say you have a library function which takes a file object as a
parameter. Instead of feeding it a Python file, you feed it your own
file-like object. Now it'll call your code whenever you do a
read/write/seek etc. In fact, since the parameter was probably
documented just as "takes a file", you're never quite sure which
functions in your objects will get called, short of reading the source
of the library - excuse me, it's a framework now.

In fact, since Python works with duck typing, and even basic operations
like addition and element access can be customized for any parameter,
there is no guarantee that users' code won't get called when they use
your "library."

So is the library/framework distinction useful in Python, especially
w/r/t Andy Smith's remarks?

 

[Terry Hancock]

If you call its code, it's a library. If it calls yours, it's a framework.

Such concision deserves applause. ;-)

 

[rafi]

If you call its code, it's a library. If it calls yours, it's a framework.

Trying to add my 2 cents (As I do not agree with the above sentence):

You build applications "on top of / using" libraries while you build
applications "in the context of" a framework.

A framework is more like a piece of swiss cheese were you fill the
holes. The framework provides an overall structure with particular rules
for its use, while the library provide basic blocks that you may
assemble in your own way (even if some constaints exist).

 

[Simon Brunning]

Such concision deserves applause. ;-)

Thank you. ;-)

As others have pointed out, this is a *drastic* simplification,
perhaps an oversimplification. You will inevitably need to call a
framework's code in addition to it calling yours, and a callback
argument or two won't turn a library into a framework. But I think it
captures the essence of the difference.

 

[Rocco Moretti]

Thank you. ;-)

As others have pointed out, this is a *drastic* simplification,
perhaps an oversimplification. You will inevitably need to call a
framework's code in addition to it calling yours, and a callback
argument or two won't turn a library into a framework. But I think it
captures the essence of the difference.

The point that passed me by the first time, and which Magnus et al.
helped me realize, is that it's referring not to an instantaneous,
mechanical view of calling, but to a more general, philosophical view of
calling.

With a library, the user's code is "in charge" of the program structure,
and calls the library to fill in the details and help out. With a
framework, the framework is "in charge", and the user code is filling in
with a supporting role. With this in mind, it's easy to see why Andy
Smith feels frameworks are restricting - after all, it's the framework,
not the user, who is "in charge" of program structure.

But I'm not sure if library vs. framework a fair comparison - the two
are doing different things. With a framework, you're not really writing
your own program, you're customizing someone else's. Sort of a vastly
more flexible version of command line options. Saying you can't reuse
code written for a framework is kind of like saying that it's difficult
to use an Apache config file with the Gimp.

 

[Terry Hancock]

But I'm not sure if library vs. framework a fair comparison - the two
are doing different things. With a framework, you're not really writing
your own program, you're customizing someone else's. Sort of a vastly
more flexible version of command line options. Saying you can't reuse
code written for a framework is kind of like saying that it's difficult
to use an Apache config file with the Gimp.

Yeah, well that's precisely the point. They *aren't* the same thing,
and which one do you want to use?

Zope recently started going through some massive changes to make
it more like a toolkit (which is the term I use instead of library here).
Even if there must be a framework, a thin framework with good tools
tends to be better than a complex framework, even if they can, in
principle do the same job.

This is really just another version of the Unix "small tools that do
one thing well" philosophy. The operating system is the "framework"
that ordinary programs run in -- and you must spend a fair amount
of time learning to use the operating system, which is why it involves
so much buy-in, and people are so religious about which one they
use. That's a general property of frameworks.

Utility programs are themselves a toolkit. You don't have to know
all of them, you are free to use some and not others, and you may
use tools from many different authors together.

OTOH, a massive Microsoft-style application is a framework. Instead
of scripting a bunch of LaTeX modules and scripts together to make
a newsletter, you do it all in Word, and maybe write Visual Basic
code to be interpreted by that framework. I'll probably get flamed
just for mentioning this system, even though I'm not advocating it. ;-)

Similarly, Zope has suffered because there is such a large learning
curve, it's so complicated, most people don't really understand what
it does -- essentially, *because it is a framework*. That's why, I think,
the developers have been moving towards a more toolkit-oriented
design with Zope 3. The "component architecture" is an attempt
to do that, by component-izing all of the important functionality
so that it can be used separately.

All of this is also linked heavily with the open-source bazaar-style
development model, too. A toolkit, with tools coming from many
different sources is more like the natural output of the bazaar. A
framework is a cathedral, so it needs a cathedral engineering
approach in order to make it work.

Where a framework shines is when you don't really want to program
it much at all -- you just need a tweak here and there beyond what
it already does. Gimp plugins are a great example of that.

Cheers,
Terry

 

[Peter Decker]

Where a framework shines is when you don't really want to program
it much at all -- you just need a tweak here and there beyond what
it already does. Gimp plugins are a great example of that.

I'd put it slightly differently. Where a framework really shines is
when you need to do the same thing over an over again; instead of
recoding that multiple times, the framework codes it once, and you
don't have to worry about it.

I've written several apps that need to update a database, and each one
had to do the same things: connect, grab data, create controls to
display/edit that data, validate any changes and then stuff the edited
data back into the database. I started playing with the Dabo framework
a few months ago, and it does all of that for me. I just set the
connection info, and then set a couple of properties on controls, and
the rest just works.

I'm sure others who don't like using frameworks will chime in that
they've done similar things; that they've created reusable classes
that make writing such appsmuch easier than having to rewrite the same
code time and time again. In that case, I'd contend that you've simply
created your own framework as you've gone along.

 

[Magnus Lycka]

Zope recently started going through some massive changes to make
it more like a toolkit (which is the term I use instead of library here).
Even if there must be a framework, a thin framework with good tools
tends to be better than a complex framework, even if they can, in
principle do the same job.

I think the Python community would benefit a lot if more of the
Zope code becomes usable independently from Zope, and I hope it
can influence not only the core Zope components, but also how
people who code for Zope write their code.

I've used ZODB successfully without Zope, and now I've started to
use Twisted, and thus Zope Interfaces. Once upon a time I used bobo
a little for things where Zope would be overkill. Zope contains a
lot of code written by very smart Python developers, and I'm sure
there are more gems to pick.

Concerning 3rd party software, there does for instance seem to be a
number of Zope workflow engines, but I don't know of any capable open
source workflow engines in the non-Zope Python world. Things like that
would be useful.

None of this contradicts that frameworks can be useful and productive
tools, but I think that just as with GUI tool kits, it's good practice
to write your main business logic in a generic and toolkit independent
way, and to use a thin layer of toolkit or framework etc to make things
work in just that context.

Just avoiding existing frameworks isn't the solution--you'll just end up
inventing your own framework instead.

 

[Paolino]

Slapdash Summary: Libraries good, frameworks bad - they are a
straightjackets and limit sharing.

Which lead me to the question - what's the difference between a library
and a framework?

A library is a set of base behaviours you have to learn separetly and
deeply to inherit a mix of them and build an aspect of your program.
A framework is a dumb runnable program in which you must insert your
plugins as specifications to drive it towards a useful instance of it.

A framework misses libraries of methods to become useful.


A library is a set of loosely coupled classes useful for machines to
attack a problem .
A framework is a set of highly coupled interfaces useful for humans to
describe a task.

Ciao Paolino

 

[Mike Meyer]

I've written several apps that need to update a database, and each one
had to do the same things: connect, grab data, create controls to
display/edit that data, validate any changes and then stuff the edited
data back into the database. I started playing with the Dabo framework
a few months ago, and it does all of that for me. I just set the
connection info, and then set a couple of properties on controls, and
the rest just works.

I'm sure others who don't like using frameworks will chime in that
they've done similar things; that they've created reusable classes
that make writing such appsmuch easier than having to rewrite the same
code time and time again. In that case, I'd contend that you've simply
created your own framework as you've gone along.

Well, they may have created a library class that does the job for
them. Figuring out which is which seemed to be the point of this
thread.

<mike

 

[Peter Decker]

Well, they may have created a library class that does the job for
them. Figuring out which is which seemed to be the point of this
thread.

I guess my summary of the thread was that a library is built to do one
thing, while a framework is built to tie many libraries together to do
a series of connected things. If you end up using libraries in the
same way all of the time, that's the beginning of a framework.

Neither is better, IMO. They exist at completely different levels in
completely different domains.

 

[Kay Schluehr]

If you call its code, it's a library. If it calls yours, it's a framework.

Pretty!

I don't think it is an oversimplification. The crucial difference is
that between interface and implementation. A framework forces the
programmers code to conform certain interfaces and contracts. It does
not restrict implementation in any way ( which might be empty ). The
implementation part uses library functions that do not tell much about
the implementation and nothing about interfaces. Libaries give you any
freedom but sometimes ( and only sometimes ) people also like to
structure their code

What are frameworks really good for - a very true success story.

A colleague of mine used to spread all kinds of flags ( state- and
property markers ) over the code that were used to glue everything
together until I raised my tyranny of frameworks. It was a hard
struggle for an OO warrier and took me almost a year or so to become
the undebated projects architect/dictator who trashed all kind of
misguided "freedom" ( i.e. bad code ): "What the **** is this?" "I
worked almost a week on it!" "It has to be reworked." "No, you don't do
it!" I did it. Who claims that social life is easy? What is nice about
this kind of cruelness is not only my colleague became finally happy
and hopefully learned at least a little bit about programming but also
our customers were gratefull about stable code, thight release
schedules and just-in-time requirement dispatch. Now we have some
bread-and-butter maintenance contract and true freedom to experiment
with other more interesting things besides this. But the struggle just
starts again with the new project

Kay

 

[Peter Maas]

If you call its code, it's a library. If it calls yours, it's a framework.

IOW Apache with modpython is a framework for web apps because it
calls your Python handlers. According to Andy Smith the Apache/
modpython combo sucks because it takes away the freedom to call a
HTTP library and write your own HTTP server

 

[Mike Meyer]

I guess my summary of the thread was that a library is built to do one
thing, while a framework is built to tie many libraries together to do
a series of connected things. If you end up using libraries in the
same way all of the time, that's the beginning of a framework.

Having read the thread, it seems that most of the suggested
definitions don't agree with yours.

Neither is better, IMO. They exist at completely different levels in
completely different domains.

I think everyone would agree with that. They're obviously different
things, and you can generally tell which is which when you're looking
at one, but defining them? That's hard.

I like the concise "Your code calls a library. A framework calls your
code", but I'm not sure it's right.

<mike

 

[Cameron Laird]

.
.
.

What are frameworks really good for - a very true success story.

A colleague of mine used to spread all kinds of flags ( state- and
property markers ) over the code that were used to glue everything
together until I raised my tyranny of frameworks. It was a hard
struggle for an OO warrier and took me almost a year or so to become
the undebated projects architect/dictator who trashed all kind of
misguided "freedom" ( i.e. bad code ): "What the **** is this?" "I
worked almost a week on it!" "It has to be reworked." "No, you don't do
it!" I did it. Who claims that social life is easy? What is nice about
this kind of cruelness is not only my colleague became finally happy
and hopefully learned at least a little bit about programming but also
our customers were gratefull about stable code, thight release
schedules and just-in-time requirement dispatch. Now we have some
bread-and-butter maintenance contract and true freedom to experiment
with other more interesting things besides this. But the struggle just
starts again with the new project

.
.
.
Kay, please say that over again (I recognize you've heard that
from me before). Are you saying that your colleague misapplied
your chosen framework by programming too much OUTside the frame-
work, and the global flags were a symptom of that? So is your
conclusion that framework use takes non-trivial education?