Desktop <-> Web

[Trans]

I want to interface a desktop application to a backend web
application. What's the easiest way to go about this --what's the best
way to communicate between the two?

Thanks,
T.

 

[Brian Candler]

SOAP and XML-RPC are both good options, as is a REST-style interface.
Net::HTTP on the client, whatever on the server.

There are APIs on the client side which are higher level than Net::HTTP.

* xmlrpc is bundled with Ruby
* soap4r (shudder) is bundled with Ruby
* ActiveResource is a REST client

Which you use typically depends on what the backend application
provides. Note that all three use XML, and so are inefficient for large
binary transfers, as you have to base64 encode them.

If you are writing both front and backend, you could consider other
protocols (e.g. DRb). That's harder to scale though, whereas it's easy
to set up a HTTP proxy for load-balancing.

The above are all synchronous protocols. If asynchronous semantics suit
you better (e.g. submit a request, poll for the response later) then you
could talk to a queuing system, e.g.
* Memcache protocol - Starling
* AMQP protocol - various servers inc RabbitMQ
* Stomp protocol over HTTP - various servers inc Apache ActiveMQ
* ... lots of others

Another option is to use XMPP (Jabber) as the transport. There is xmpp4r
as endpoint, and a selection of XMPP servers to route your messages
through. xmlrpc can run over XMPP. But this is likely to be overkill for
a single-server scenario.

Finally, consider seriously the possibility of using a plain web browser
as the client, using HTML and AJAX over HTTP. It's maybe a bit harder to
build an application with a desktop "feel" but it is doable (see Google
Docs), and you don't need to distribute any code to the clients.

Rails has particularly good support for this, and can generate most of
the necessary Javascript for you. However if you're not already a Rails
user, there's a steep learning curve to get there.

 

[James Gray]

* soap4r (shudder) is bundled with Ruby

Ruby 1.8. I believe it's gone in Ruby 1.9.

James Edward Gray II

 

[Martin DeMello]

I want to interface a desktop application to a backend web
application. What's the easiest way to go about this --what's the best
way to communicate between the two?

Are you writing the webapp too?

martin

 

[Trans]

Are you writing the webapp too?

yes.

 

[Martin DeMello]

yes.

I'd suggest returning data from the webapp as json, then, and reading
it as yaml from the ruby end. I did that in a recent project at work
and it was pretty painless. The benefit is if you want to build a rich
browser-based frontend later, client side javascript can use the same
json objects.

martin

 

[Trans]

I'd suggest returning data from the webapp as json, then, and reading
it as yaml from the ruby end. I did that in a recent project at work
and it was pretty painless. The benefit is if you want to build a rich
browser-based frontend later, client side javascript can use the same
json objects.

I thought about this as well, but how did you go about transporting
the json?

At the moment I've taken Aria's suggestion of using XML-RPC, that's
working fine via the XMLRPC::WEBrickServlet. But ultimately I want to
use Rack instead. An example of a simple Rack-based JSON servlet would
be awesome. Anyone have one?

T.

 

[Trans]

There are APIs on the client side which are higher level than Net::HTTP.

* xmlrpc is bundled with Ruby
* soap4r (shudder) is bundled with Ruby
* ActiveResource is a REST client

Which you use typically depends on what the backend application
provides. Note that all three use XML, and so are inefficient for large
binary transfers, as you have to base64 encode them.

If you are writing both front and backend, you could consider other
protocols (e.g. DRb). That's harder to scale though, whereas it's easy
to set up a HTTP proxy for load-balancing.

The above are all synchronous protocols. If asynchronous semantics suit
you better (e.g. submit a request, poll for the response later) then you
could talk to a queuing system, e.g.
* Memcache protocol - Starling
* AMQP protocol - various servers inc RabbitMQ
* Stomp protocol over HTTP - various servers inc Apache ActiveMQ
* ... lots of others

Another option is to use XMPP (Jabber) as the transport. There is xmpp4r
as endpoint, and a selection of XMPP servers to route your messages
through. xmlrpc can run over XMPP. But this is likely to be overkill for
a single-server scenario.

Finally, consider seriously the possibility of using a plain web browser
as the client, using HTML and AJAX over HTTP. It's maybe a bit harder to
build an application with a desktop "feel" but it is doable (see Google
Docs), and you don't need to distribute any code to the clients.

Rails has particularly good support for this, and can generate most of
the necessary Javascript for you. However if you're not already a Rails
user, there's a steep learning curve to get there.

Thanks Brian. Lots of useful info hear. I actually did create web-
browser client at first, but the problem is that I needed to save
files to the local file system, and that was just too cumbersome with
a web-client --it required that I had a local server running too,
which added additional headaches. I considered something like Google
Gears, Adobe Air, etc. but that ended real quick when I discovered
none of them supported 64bit Linux, which I run.

(Aside Rant: What the hell is going on with 64 bit support btw? Why
the hell are we stuck in 32-bit world? I mean how long has has it
bloody been already since 64 CPUs hit the mass-market? It's just
intolerable! Honestly, I can't say enough bad things about this state
of affairs. It really has to be the pinnacle example of how the whole
industry has gone to hell in a hand basket.)

So I'm using Shoes on the front-end. And for the most part that's
working pretty good so far.

T.

 

[Brian Candler]

So I'm using Shoes on the front-end. And for the most part that's
working pretty good so far.

Makes sense.

Some years ago I wrote a system which used DRb over HTTP: I had to
modify the DRb code slightly to do this (adding support for fastcgi
server processes). That was nice as it let me use native Ruby objects,
but if hashes containing just text are OK for you (and make sure they're
only ASCII or UTF-8 text), xmlrpc should be fine.

I did come across YAML RPC once; there's some code floating around on
the net for that, but it's possibly stale now.

I don't know if there's any sort of standard for "JSON RPC". I suspect
it just means REST with JSON response objects. If you want to try that,
and to keep as close to Rack as possible, then I suggest layering
something very lightweight like Sinatra on top of it. Of course, Rails
has render :json. Remember too that json supports *only* Unicode
strings, so anything binary needs base64 encoding.

Don't forget that your backend app will probably need some sort of
authentication framework, possibly just HTTP Basic Auth for simplicity,
to stop random clients making requests.

Even if you don't use a browser as a front-end, you might be able to
steal some ideas from those who do. e.g.
http://beebole.com/blog/pure/generate-html-from-a-json-without-any-template-but-html-and-javascript/

Anyway, that's just a few random thoughts.

Regards,

Brian.

 

[Brian Candler]

ultimately I want to
use Rack instead. An example of a simple Rack-based JSON servlet would
be awesome. Anyone have one?

How about this with Sinatra, which sits on top of Rack:

--- myapp.rb ---
require 'rubygems'
require 'sinatra'
require 'json'
require 'widget'

get '/widgets.json' do
Widget.all.to_json
end

get '/widgets/:id.json' do
Widget.find(Integer(params[:id])).to_json
end

post '/widgets.json' do
item = JSON.parse(request.body.read)
raise "Bad object" unless item.is_a? Widget
Widget.add(item).to_s
end

--- widget.rb ---
# Rubbish model, not thread-safe!
class Widget
attr_accessor :id, :name, rice
def initialize(id, name, price)
@id, @name, @price = id, name, price
end
def to_json(*a)
{
'json_class' => self.class.name,
'data' => [ @id, @name, @price ],
}.to_json(*a)
end
def self.json_create(o)
new *o['data']
end
def self.all
@all ||= []
end
def self.add(item)
@seq ||= 0
@seq += 1
item.id = @seq
all << item
return @seq
end
def self.create(*args)
add(new(*args))
end
def self.find(id)
all.find { |item| item.id == id }
end
end
Widget.create(nil, "flurble", 12.3)
Widget.create(nil, "boing", 4.56)

 

[Brian Candler]

P.S. I'm not sure what is the "right" way to serialize Ruby objects in
JSON.

I took the to_json and json_create code from the ruby json library
documentatation, which gives

{"json_class": "name", "data": ["values", ...]}

Maybe it would be more natural to serialize a Ruby object as
{"attr_name": "attr_value", ...}, since { ... } itself denotes an
"object" according to the JSON specification.

However I've also seen examples on the web using a hybrid:

{"json_class": "name", data: {"attr_name": "attr_value", ... } }

 

[James Gray]

P.S. I'm not sure what is the "right" way to serialize Ruby objects in
JSON.

I think it's probably better to think of JSON as simple primitives.
Thus JSON's "object" would map to Ruby's Hash. From there, I just
create an output method that translates my objects to primitives and
an input method that restores them.

Most libraries do seem to try to dump everything though, so maybe this
is just flawed thinking on my part.

James Edward Gray II

 

[Brian Candler]

I think it's probably better to think of JSON as simple primitives.
Thus JSON's "object" would map to Ruby's Hash. From there, I just
create an output method that translates my objects to primitives and
an input method that restores them.

For a POST it seems to make sense to think of JSON as a "params" hash,
like a form post or an XML-wrapped hash.

Following that logic, PUT ought to be the same, and hence GET. And the
'class' is implicit in the REST URL: /widgets/ or whatever.

OK, I'm sold, for web use anyway. Maybe the documentation in the JSON
library is a bit misleading then.

 

[Brian Candler]

That simplifies things a bit. Here it is for completeness:

--- myapp.rb ---
require 'rubygems'
require 'sinatra'
require 'json'
require 'widget'

get '/widgets.json' do
Widget.all.to_json
end

get '/widgets/:id.json' do
Widget.find(Integer(params[:id])).to_json
end

post '/widgets.json' do
item = JSON.parse(request.body.read)
Widget.create(item).to_s
end

--- widget.rb ---
# Rubbish model, not thread-safe!
class Widget
attr_accessor :id, :name, rice
def initialize(params)
@id = params["id"]
@name = params["name"]
@price = params["price"]
end
def to_json(*a)
instance_variables.inject({}) { |h,v|
h[v.to_s[1..-1]] = instance_variable_get(v)
h
}.to_json(*a)
end

@all = []
@seq = 0
def self.all
@all
end
def self.add(item)
item.id = (@seq += 1)
all << item
return @seq
end
def self.create(params)
add(new(params))
end
def self.find(id)
all.find { |item| item.id == id }
end
end
Widget.create("name" => "flurble", "price" => 12.3)
Widget.create("name" => "boing", "price" => 4.56)

If you require 'json/add/rails' then you don't need to define your own
to_json method, but it also allows people to create arbitrary Ruby
objects on your machine (which makes me uncomfortable)

 

[Trans]

That simplifies things a bit. Here it is for completeness:

--- myapp.rb ---
require 'rubygems'
require 'sinatra'
require 'json'
require 'widget'

get '/widgets.json' do
=A0 Widget.all.to_json
end

get '/widgets/:id.json' do
=A0 Widget.find(Integer(params[:id])).to_json
end

post '/widgets.json' do
=A0 item =3D JSON.parse(request.body.read)
=A0 Widget.create(item).to_s
end

--- widget.rb ---
# Rubbish model, not thread-safe!
class Widget
=A0 attr_accessor :id, :name, rice
=A0 def initialize(params)
=A0 =A0 @id =A0 =A0=3D params["id"]
=A0 =A0 @name =A0=3D params["name"]
=A0 =A0 @price =3D params["price"]
=A0 end
=A0 def to_json(*a)
=A0 =A0 instance_variables.inject({}) { |h,v|
=A0 =A0 =A0 h[v.to_s[1..-1]] =3D instance_variable_get(v)
=A0 =A0 =A0 h
=A0 =A0 }.to_json(*a)
=A0 end

=A0 @all =3D []
=A0 @seq =3D 0
=A0 def self.all
=A0 =A0 @all
=A0 end
=A0 def self.add(item)
=A0 =A0 item.id =3D (@seq +=3D 1)
=A0 =A0 all << item
=A0 =A0 return @seq
=A0 end
=A0 def self.create(params)
=A0 =A0 add(new(params))
=A0 end
=A0 def self.find(id)
=A0 =A0 all.find { |item| item.id =3D=3D id }
=A0 end
end
Widget.create("name" =3D> "flurble", "price" =3D> 12.3)
Widget.create("name" =3D> "boing", "price" =3D> 4.56)

If you require 'json/add/rails' then you don't need to define your own
to_json method, but it also allows people to create arbitrary Ruby
objects on your machine (which makes me uncomfortable)

Thanks. I'll give it a go.

T.

 

[Brian Candler]

I also learned from the Sinatra list that there is a Rack middleware
module you can load which which will perform the parsing of JSON POST
data for you, so it just ends up in params. Tidy.

http://groups.google.com/group/sinatrarb/msg/6579132bee5fc0b2

http://github.com/rack/rack-contrib...rack/contrib/post_body_content_type_parser.rb

 

[hemant]

I also learned from the Sinatra list that there is a Rack middleware
module you can load which which will perform the parsing of JSON POST
data for you, so it just ends up in params. Tidy.

http://groups.google.com/group/sinatrarb/msg/6579132bee5fc0b2

http://github.com/rack/rack-contrib...rack/contrib/post_body_content_type_parser.rb

If sole purpose is to ferry back JSON back forth, one may have a look at:

http://halcyon.rubyforge.org/

Also, why bother converting JSON to YAML on ruby side? Marshalling
Ruby objects to JSON is equally easier. If you are using AR or
something it already has support for that, otherwise JSON gem already
comes with support for marshalling built-in types which can be easily
extended to custom classes with to_json method.

 

[Matt Todd]

[Note: parts of this message were removed to make it a legal post.]

There's not much in Halcyon you'll find beyond what you've already learned
here. The interesting bit, I guess, would be the PostBodyContentTypeParser
middleware. Apologies for the name, haha. I wrote the middleware to
compliment Halcyon since everything coming out of it was JSON I figured it
should at least support it coming in.
Halcyon behaves very similar to what's going on in your application...
essentially it breaks down to a simplistic pattern: always have a resource
(a hash or an array of hashes, for example) to respond with as JSON objects.
A lot of the time it was just calling #to_json on an object as the body of
the response. Aside from that, everything is as you'd expect.

I recommend JSON wholeheartedly.

Matt

http://github.com/rack/rack-contrib...rack/contrib/post_body_content_type_parser.rb

If sole purpose is to ferry back JSON back forth, one may have a look at:

http://halcyon.rubyforge.org/

Also, why bother converting JSON to YAML on ruby side? Marshalling
Ruby objects to JSON is equally easier. If you are using AR or
something it already has support for that, otherwise JSON gem already
comes with support for marshalling built-in types which can be easily
extended to custom classes with to_json method.

--
Matt Todd
Highgroove Studios
www.highgroove.com
cell: 404-314-2612
blog: maraby.org

Scout - Web Monitoring and Reporting Software
www.scoutapp.com