Idea: Webshare

[Michael Neumann]

Hi,

Ruby has a lot of cool web frameworks. For example:

* Borges
* CGIkit
* Iowa
* Radical
* Rails
* SWS

to mention only a few ;-)

Some of them share a significat (?) number of code, like:

* Cache (LRU,...)
* Session, unique session key generation
* Request, Response classes
* WEBrick servlets
* Dispatcher
* Html-generation stuff, templates (e.g. HTTP Error templates)

How about a common basic framework, that includes all this stuff? Of
course this might not work for all frameworks, but at least Iowa, SWS,
CGIkit (and Borges?) should have some common pieces of code.

Webshare... sounds like Webware ;-)

I think, if it's done right, every involved project would benefit from
it (of course each project is free to use this stuff or specialize).

Who's interested? Thoughts?

Regards,

Michael

 

[Kirk Haines]

On Fri, 20 Aug 2004 04:38:22 +0900, Michael Neumann wrote

Some of them share a significat (?) number of code, like:

* Cache (LRU,...)
* Session, unique session key generation
* Request, Response classes
* WEBrick servlets
* Dispatcher
* Html-generation stuff, templates (e.g. HTTP Error templates)

How about a common basic framework, that includes all this stuff? Of
course this might not work for all frameworks, but at least Iowa,
SWS, CGIkit (and Borges?) should have some common pieces of code.

Forgive me if I already sent this. I don't think I did, but it's been
another long day.

I would be interested in at least exploring this somewhat. I'd definitely
be interested in sharing & improving my caching class(es) and my
request/response classes.


Kirk Haines

 

[David Heinemeier Hansson]

How about a common basic framework, that includes all this stuff? Of

course this might not work for all frameworks, but at least Iowa, SWS,
CGIkit (and Borges?) should have some common pieces of code.

I what we need is just an upgrade of CGI.rb. There was a post about
this a long time back at ruby-talk:21315. Rails would definitely like
to use something like that.
--
David Heinemeier Hansson,
http://www.rubyonrails.org/ -- Web-application framework for Ruby
http://www.instiki.org/ -- A No-Step-Three Wiki in Ruby
http://www.basecamphq.com/ -- Web-based Project Management
http://www.loudthinking.com/ -- Broadcasting Brain
http://www.nextangle.com/ -- Development & Consulting Services

 

[Aredridel]

I what we need is just an upgrade of CGI.rb. There was a post about
this a long time back at ruby-talk:21315. Rails would definitely like
to use something like that.

Heartily agreed.

I'd love to see several other modules as well, but some of the core CGI
function keeps getting written over and over and over.

Time to make it make sense.

 

[Patrick May]

Hello,

I what we need is just an upgrade of CGI.rb. There was a post about
this a long time back at ruby-talk:21315. Rails would definitely like
to use something like that.

Narf is an upgrade of CGI.rb. I rebuilt CGI.rb, using Wakou's tests
and adding tests for everything else. I felt that the maintainer of an
IO libraries should support testing itself, so Narf expanded a bit.

For example, Tom Clarke added a templating library to support testing.
The idea was that writing webapps is easier if you write asserts on the
variables passed to the template, instead of writing asserts on the
HTML.

The good thing about Narf is that not too many folks use it. So it can
still be a bit of a playground for the moment. I'm open to suggestions.

To be honest, I think the api of Narf is pretty good. This stuff isn't
all that complicated. I think the main thing is supporting multiple
backends, so that's what I'm going to focus on.

Cheers,

Patrick

P.S. <rant> About the only cgi design suggestion I see on ruby-talk
that I outright disagree with is Request / Response.

I experimented with this in Narf, and I think it is a mistake. You end
up with two objects, with alot of shared implementation code, that
always go together, split apart for no practical reason. I don't mind
having Request / Response separated in the documentation, but splitting
it in the code just adds typing. When will you ever want a request
without the response? Martin Fowler has a name for this refactoring,
but I don't have his book.

The Java Servlet API is not one to imitate. It's a design which solved
the problem that Java was too slow to run as a CGI. Given that ruby
doesn't have that limitation, I'd rather design the API that makes
building web apps easy and fun.

Why bother with the API that made it possible to run web apps on a
bloated os / programming language / platform / all around lemon? </rant>

 

[Kirk Haines]

On Sun, 22 Aug 2004 06:19:08 +0900, Patrick May wrote

P.S. <rant> About the only cgi design suggestion I see on ruby-talk
that I outright disagree with is Request / Response.

I experimented with this in Narf, and I think it is a mistake. You
end up with two objects, with alot of shared implementation code,
that always go together, split apart for no practical reason. I
don't mind having Request / Response separated in the documentation,
but splitting it in the code just adds typing. When will you ever
want a request without the response? Martin Fowler has a name for
this refactoring, but I don't have his book.

You know, this is one thing I definitely agree with. In Iowa, I have an
Iowa::Request that encapsulates the request. However, it actually also
encapsulates the response, and I've found it to be a very natural thing that
the same object that brings the request into the app also carries the
response back out of it.

The Iowa::Request is modeled very closely on the mod_ruby Apache::Request,
mostly because it seems like a good, effective, familiar model, and doing so
helped me avoid reimplementing the wheel.


Kirk Haines

 

[Robert Klemme]

P.S. <rant> About the only cgi design suggestion I see on ruby-talk
that I outright disagree with is Request / Response.

I experimented with this in Narf, and I think it is a mistake. You end
up with two objects, with alot of shared implementation code, that
always go together, split apart for no practical reason. I don't mind
having Request / Response separated in the documentation, but splitting
it in the code just adds typing. When will you ever want a request
without the response? Martin Fowler has a name for this refactoring,
but I don't have his book.

The Java Servlet API is not one to imitate. It's a design which solved
the problem that Java was too slow to run as a CGI. Given that ruby
doesn't have that limitation, I'd rather design the API that makes
building web apps easy and fun.

Why bother with the API that made it possible to run web apps on a
bloated os / programming language / platform / all around lemon? </rant>

There are in fact good reasons in favor of the request / response split.
First of all request and response are in fact two different things, so from
that point of view it's not a bad idea to separate them. The interface of a
combined request response instance is likely to get too bloated. There are
use cases where you only need the response (say, a page that displays the
current time) and there might be others where you don't need the response
(can't think of any at the moment though, some kind of upload maybe ).
Another reason why I liked the separation (in Java) is that you can easily
wrap only one of them (common with filters in Java, but similar things can
be done with Ruby, too); examples are filtering of whitespace from the
response and adding default values for request parameters. If request and
response are encapsulated in a single instance this becomes much more
difficult to do - and the fun goes away.

That are my 0.02 on the matter...

Kind regards

robert

 

[Andreas Schwarz]

P.S. <rant> About the only cgi design suggestion I see on ruby-talk that
I outright disagree with is Request / Response.

I experimented with this in Narf, and I think it is a mistake. You end
up with two objects, with alot of shared implementation code

I strongly disagree. Request and response are two completely different
things; that their implementations _might_ have shared code (though I
can't see where) doesn't mean that they have shared data. You receive a
request object that contains the request method, URL, form parameters,
cookies and headers, and you create a response object that consists of
the body, cookies and headers and send it back to the server; the data
of both objects has nothing in common.

Example:

if request.params['password'] = 'xyz'
response.header['Content-Type'] = 'text/plain'
response.cookies << { 'session_id' => 'abcde' }
response.body = 'logged in'
end

I really can't see how a combined request/response object would make any
sense here.

 

[Austin Ziegler]

P.S. <rant> About the only cgi design suggestion I see on ruby-talk
that I outright disagree with is Request / Response.

I experimented with this in Narf, and I think it is a mistake. You end
up with two objects, with alot of shared implementation code, that
always go together, split apart for no practical reason. I don't mind
having Request / Response separated in the documentation, but splitting
it in the code just adds typing. When will you ever want a request
without the response? Martin Fowler has a name for this refactoring,
but I don't have his book.

As someone who thinks that a request/response is a good thing, I'll
first agree with Andreas that while they may share implementations,
they don't share data (which suggests a Request class, a Response
class, and a "Message" class). Separating request and response, to me,
is as simple as the splitting of STDIN, STDOUT, and STDERR. In theory,
I could choose to request from one computer but respond in a different
location -- why should I be restricted to responding to the person who
made the request? (Granted, I can't see why I would do such a thing,
but who knows whether such a feature would be useful?)

-austin

 

[Zach Dennis]

I am using the eval function excessively in one of my programs. Is there
any real performance drains from using it? Thanks,

Zach

 

[Patrick May]

Hello,

I strongly disagree. Request and response are two completely different
things; that their implementations _might_ have shared code (though I
can't see where) doesn't mean that they have shared data.

The common resource is the connection to the client.

Cheers,

Patrick

 

[Patrick May]

Hello,

There are in fact good reasons in favor of the request / response
split.
First of all request and response are in fact two different things, so
from
that point of view it's not a bad idea to separate them. The
interface of a
combined request response instance is likely to get too bloated.

I don't really think that request / response are all that different.
They share the connection to the client.

An object with many methods is one kind of bloat. Managing two objects
that share the same resource and lifecycle causes another kind of
bloat. The latter bloat is the kind that leads to bugs in my code.

For example, the library should delete the Tempfiles containing file
uploads (request state) when the final output is written and the
connection is closed (response state). A single object with a single
lifecycle makes this sort of thing easier to support.

There are
use cases where you only need the response (say, a page that displays
the
current time) and there might be others where you don't need the
response
(can't think of any at the moment though, some kind of upload maybe
).

I don't think the object will mind if you don't call all of its
methods. Just ask aString

Another reason why I liked the separation (in Java) is that you can
easily
wrap only one of them (common with filters in Java, but similar things
can
be done with Ruby, too); examples are filtering of whitespace from the
response and adding default values for request parameters. If request
and
response are encapsulated in a single instance this becomes much more
difficult to do - and the fun goes away.

You should be able to do this in Narf:

Web["param"] ||= "a value"

Note that the single object I recommend that you use is the Web module.
That way, you don't have to pass anything around! The output filter
isn't something I've thought about, but I think it is a good idea.

I don't want to be entirely negative. These are use cases I want to
support. I tried designing a Request / Response style interface. I
found that the separation made things more difficult for me. I
apologize for the earlier <rant />.

Cheers,

Patrick

 

[Andreas Schwarz]

Hello,




The common resource is the connection to the client.

Not necessarily; for example one could save response objects in a cache.

Can you give an example of what advantage a combination of request and
response would bring in practice?

 

[T. Onoma]

I don't really think that request / response are all that different.

They share the connection to the client.

Perhaps have both?

module ClientBehavior; # ...; end
module RequestBehavior; # ...; end
module ResponseBehavior; # ...; end

class Client; include ClientBehavior
# ...
end

class Request; include ClientBehavior, RequestBehavior
# ...
end

class Response; include ClientBehavior, ResponseBehavior
# ...
end

class RoundTrip; include RequestBehavior, ResponseBehavior
# ...
end

Use those modules!

 

[Patrick May]

Hello,

You know, this is one thing I definitely agree with. In Iowa, I have
an
Iowa::Request that encapsulates the request. However, it actually also
encapsulates the response, and I've found it to be a very natural
thing that
the same object that brings the request into the app also carries the
response back out of it.

The Iowa::Request is modeled very closely on the mod_ruby
Apache::Request,
mostly because it seems like a good, effective, familiar model, and
doing so
helped me avoid reimplementing the wheel.

I was checking out Iowa. It looks like you've done alot of work to
make Iowa run across multiple servers. It looked pretty nice

I am working on a model for supporting multiple web servers that is
similar to DBI (in my head, at least).

The CGI object would provide the useful methods that make building web
apps easier. Each server would have a "CGD" that actually provided the
necessary common functionality. Different environments may also have
different ways of embedding the application, just like you have
iowa_fcgi.rb, iowa_webrick.rb, and mod_iowa.rb.

I'm planning on writing a cgi script and a set of webunit tests, so
that I can start writing CGDs for different servers and test that
things are working.

Since you've had experience with solving this problem, I'm curious
about your opinion. Does this sound like a productive way of attacking
this problem?

Cheers,

Patrick

 

[Kirk Haines]

On Tue, 24 Aug 2004 04:33:12 +0900, Patrick May wrote

The CGI object would provide the useful methods that make building
web apps easier. Each server would have a "CGD" that actually
provided the necessary common functionality. Different environments
may also have different ways of embedding the application, just like
you have iowa_fcgi.rb, iowa_webrick.rb, and mod_iowa.rb.

I took what is essentially a lazy approach. I created a marshallable,
incomplete copy of Apache::Request, basically, because it was familiar to me
already. I then needed a way to populate it with all of the necessary
data. I partitioned the world into two models: mod_ruby and not-mod-ruby-
so-assume-CGI. The mod_ruby world makes it very easy to put all of the
necessary data from the Apache::Request into my copy of Apache::Request.

For the non-mod-ruby world, though, I simply assume that the standard CGI
environment variables will be available, and I use those to fill out the
request object.

The nice thing about that model is that it then makes it trivial to make it
work with any server that provides a CGI-like interface. All it really has
to do is to create a request object, letting the object worry about the
details of how to populate itself with the necessary data, and then send the
object to the application, and it works for most things.

My understanding is that IIS does not work like this, though, so I would
need to do some work to get it to do the right thing with IIS.

Overall, I like the model, though I am certain that it can be improved
upon. So much code to write, and only two hands to do it with.... Anyway,
I would think that for what you are talking about, you would probably do
something somewhat similar in concept.

Let me throw you a wrinkle to think about.

File uploads. If you adopt a model where you are passing a request object
to the webapp, and the webapp may not even be on the same machine as the
webserver, what is the best way to handle uploaded files?

You can't just create a tempfile like the current CGI does. Do you transfer
the data with the request, but then write it to tempfiles on the other
side? Does Drb provide a way that one could write the uploaded files to
tempfiles on the server, but that one could access the tempfiles via the
request object in the webapp? Could be cool. Might be a bad idea.


Kirk Haines

 

[Patrick May]

Can you give an example of what advantage a combination of request and
response would bring in practice?

Every web app I've written depends on a combination of request and
response. I think that combining the two simplifies makes for cleaner,
simpler code. Splitting up request / response you have to pass the
objects around:

def delegate( request, response )
case request["param"]
when "submit"
do_submit( request, response )
else
show_form( request, response )
end
end

request, response = cgi.create
delegate( request, response )

It seemed simpler to use a single object:

def delegate( cgi )
case cgi["param"]
when "submit"
do_submit( cgi )
else
show_form( cgi )
end
end

delegate( CGI.new )

Now, I think the best API is to let a global manage a singleton:

def delegate
case Web["param"]
when "submit"
do_submit
else
show_form
end
end

delegate

Passing around all those objects felt like useless typing to me. And
given that it made it easier to implement the library, that's how I've
arrived at my opinion. Splitting request / response up just didn't
seem to be justified, while merging them created visibly simpler code
in my apps and the library.

(I think caching is a side issue. None of these designs supports
caching by default because they all have a connection to the client in
there. Any of these designs can be made to support caching with some
effort.)

Cheers,

Patrick

p.s. another thread sprung up out of this conversation; see [ruby-talk
110096]

 

[T. Onoma]

Time to make it make sense.

Thought's this was intersting. On Matz's blog there was this snippet:

require 'cgi'
require 'fcgi.so'
class CGI
class Fast < CGI
CONTINUATION = []
def Fast::new(*args)
at_exit do
if CONTINUATION[0]
CONTINUATION[0].call
end
end
callcc do |c|
CONTINUATION[0] = c
end
fcgi = FCGI::accept
unless fcgi CONTINUATION[0] = nil
exit
end
$defout = fcgi.out super(*args)
end
end
end

Does that make sense? Here's what matz says about it:

<quote>
About how to make correction of the existing CGI program minimum and make it
FastCGI correspondence

Before, such a trick has been written.

[ above code here ]

I change "require'cgi'" into "require'fcgi'" and think that remarkable CGI
will operate as it is only by rewriting "CGI.new" to "CGI::Fast.new."
</quote>

T.

 

[Patrick May]

Hello,

Time to make it make sense.

Thought's this was intersting. On Matz's blog there was this snippet:

require 'cgi'
require 'fcgi.so'
class CGI
class Fast < CGI
CONTINUATION = []
def Fast::new(*args)
at_exit do
if CONTINUATION[0]
CONTINUATION[0].call
end
end
callcc do |c|
CONTINUATION[0] = c
end
fcgi = FCGI::accept
unless fcgi CONTINUATION[0] = nil
exit
end
$defout = fcgi.out super(*args)
end
end
end

Does that make sense? Here's what matz says about it:

<quote>
About how to make correction of the existing CGI program minimum and
make it
FastCGI correspondence

Before, such a trick has been written.

[ above code here ]

I change "require'cgi'" into "require'fcgi'" and think that remarkable
CGI
will operate as it is only by rewriting "CGI.new" to "CGI::Fast.new."
</quote>

Thanks, that certainly is a cute trick!

~ Patrick

 

[Aredridel]

There are in fact good reasons in favor of the request / response split.

First of all request and response are in fact two different things, so from
that point of view it's not a bad idea to separate them. The interface of a
combined request response instance is likely to get too bloated. There are
use cases where you only need the response (say, a page that displays the
current time) and there might be others where you don't need the response
(can't think of any at the moment though, some kind of upload maybe ).

Well, all HTTP requests have responses, though some are pretty spartan
(302, for example)

There's basically two models at work here: the servlet-ish model, with
separation, and the apache request-cycle model, without.

I don't see them as contradictory, particularly. They're two views of
the same thing, eventually.

Apache has "handlers", each of which can snoop on the request at that
moment and tweak it as neccesary for later handlers to get. Its stacking
makes having one, big, durable object that persists the whole cycle
valuable: one can look back on the whole cycle for information on what
to do.

The request-response split also makes sense: they're very different in
some ways. A request has a method, a path, some headers and a body. The
response has a status code, headers, and sometimes a body. It seems a
perfect place to put some inheritance, since there's some shared
implementation (headers and body), and some specific things to each.

Another reason why I liked the separation (in Java) is that you can easily
wrap only one of them (common with filters in Java, but similar things can
be done with Ruby, too); examples are filtering of whitespace from the
response and adding default values for request parameters. If request and
response are encapsulated in a single instance this becomes much more
difficult to do - and the fun goes away.

Not terribly, actually: in Java, you wrap; in Apache-style, you just
modify in place. (The analogy with gsub() and gsub!() is apparent to me,
at least)

I like the split, since I think all possibilities are available to both.
In one, one gets a req and a resp passed to the method; in the other,
you get req. In one, you have req.body and resp.body, the other is
req.request_body and req.response_body. I think the first is much
cleaner.

What really needs to happen is documentation of the request and response
cycle, and if filters need to happen, document how to add them, and what
hooks are available.

I think the real question is "what use cases are there for hooks,
handlers and filters?". I see several:

* Filters (ala aspect programming), both header-filters for security,
and body content filters for content adjustment (XHTML2 -> XHTML1.1 for
example)
* Additional file-type handlers (potentially for .erb, for .cgi, for
fast cgi, possibly.)[1]
* Implement additional HTTP methods (PUT, DELETE, PROPFIND like DAV)

And then there's the question of how to implement that with the
mountpoint-oriented structure of something like webrick.[2]

Ari


[1] Don't let me start on "file extension" based dispatch. I don't like
it at all.

[2] Personally, I like WEBrick a lot. Nice simple structure.