client -server interaction over XML supporting multiple protocols

[rdh]

Hi all,

I am in process of developing a Server in C++ supporting multiple
protocols. The server will be exposing various functionalities, and the
clients can communicate over any of the protocols may be TCP, IPX, SAP,
NETBEUI to access the server to access the functionalities exposed. The
server doesnot know in advance which client is using what protocol.
ALSO, ALL THE INTERACTION WILL BE MADE OVER XML.

example my server has functionality X()

and i have n number of client.
client1 is communicating over protocol TCP to access X()
client2 is communicating over protocol IPX to access X()

...
and so on.


We had already developed ( prepared a rough code sketch), a server that
is able to handle multiple clients at the same time over TCP /IP. But
now, we need to enhance the same so that it can intearct with clients
irrespective of the protocol being used.

I am struck with following issue:

1) How to make the client-server flow so that the server knows from
which protocol the client has communicated so that the server can send
the reply over the same protocol and this whole process is
multithreaded.

What all steps are needed to take care of while building the system
most efficient.

In summary
need to make server in such a way that it may accept a request from
connection oriented protocol as well as connection-less protocol at the
same time. so need to develop a mechanism for the same the work for all
conditions.

Please suggest. Also please feel free to send your comments/suggestions
to make this system more efficient.

All suggestions are welcome.

Thanks,
rdh

 

[David]

Hi all,

I am in process of developing a Server in C++ supporting multiple
protocols. The server will be exposing various functionalities, and the
clients can communicate over any of the protocols may be TCP, IPX, SAP,
NETBEUI to access the server to access the functionalities exposed. The
server doesnot know in advance which client is using what protocol.
ALSO, ALL THE INTERACTION WILL BE MADE OVER XML.

example my server has functionality X()

and i have n number of client.
client1 is communicating over protocol TCP to access X()
client2 is communicating over protocol IPX to access X()

..
and so on.


We had already developed ( prepared a rough code sketch), a server that
is able to handle multiple clients at the same time over TCP /IP. But
now, we need to enhance the same so that it can intearct with clients
irrespective of the protocol being used.

I am struck with following issue:

1) How to make the client-server flow so that the server knows from
which protocol the client has communicated so that the server can send
the reply over the same protocol and this whole process is
multithreaded.

What all steps are needed to take care of while building the system
most efficient.

In summary
need to make server in such a way that it may accept a request from
connection oriented protocol as well as connection-less protocol at the
same time. so need to develop a mechanism for the same the work for all
conditions.

Please suggest. Also please feel free to send your comments/suggestions
to make this system more efficient.

All suggestions are welcome.

Thanks,
rdh

Hello rdh,

Look up some of the ancient examples of server code under Unix/linux
for TCP/IP. The typically most complicated example of how to write
the server side covers multi-service, multi-protocol servers.

You simply need to associate the proper protocol with each of your
connections. This may already be part of the connection information
if all of your protocols come in on a standard interface. If you
use multiple threads or interfaces to handle all of the protocols
you just need to provide a common interface that can be used by the
rest of your request-response engine.

You will need to decide if your target operating system, server
model, and request-response engine is best implemented as a single
process or multiple processes. 'process' here could mean just
about anything based on the various operating system models that
are available. For instance, the overhead of your XML decoder-
encoder or request-response engine for each conversation may
overload some operating system models. At times it is better to
keep some services separate. This all depends on the complexity
of your function X().

It is likely that the stream oriented (TCP) and packet oriented
(UDP) protocols will provide a unified view to another layer
of your server that actually performs the XML parse or encode.
The complexity of putting this all together also largely depends
on your experience and the tools that you choose for the project.
If you doubt your ability to put it all together the first time,
implement the final optimized strategy in steps. That also allows
you to change your design if problems are found in the design.

You may be able to find much of what you need already available
in other packages or combinations of packages. I have my own
client-server models that I can simply drop just the conversation
engine (request-response engine above) in and select what kind of
server or client model I want to implement. I'd offer them but
they are not likely to meet your needs as they target a certain
class of linux and other embedded systems.

In review, for (1) if your communication model is Unix/linux
based or Microsoft based, check out the definition of communications
sockets. You are likely to find that the definition of a socket
can be either UDP, TCP, IPX, and so on. If I recall correctly these
are called the Interface Families. You just need a plug-in or support
for all of your desired protocols in your operating system and the
socket communications layer will hide most of the details for you.
You will still need to address the stream vs. packet mode yourself.

The optimized solution (2) for the entire multi-protocol server,
XML, and function X() are best derived based on experience. The
size of the XML and X() functionality will determine the demands
on your operating system. C++ can easily handle multi-protocol
servers and the other components you are likely to need. However,
some of the underlying complexity of the XML engine or X()
functionality may degrade performance as the number of conversations
rise. This generally isn't a problem for simple communications
and X() functionality. If X() was a time consuming or large
overhead task, your performance may be better with a slightly
different model. You will need to play around with the design
if you cannot meet your target goals. You likely have some
idea of the complexity of the XML and X() functionality already.
Also consider the number of simultanious clients served, client-to-
client interactions if needed, and the desired response time.

David

 

[rdh]

Hi David,

Thanks for the reply. was really very informative. I am in process of
analyzing various models to develop the server. My basic requirement is
that :

1) Server can be easily ported to UNIX/LINUX OSes, initiallly to be
developed for Windows platform.
2) Implementation will be done in C++
3) The interaction / communication will be done via XML
4) Server should accept the connection over any of the protocols
connection oriented or connection less.
5) The data to be transeferred between client-server is very light and
doesnot involve complex heavy data.

So m in process of evaluating various servers. Can you please suggest a
server model that suits my requirement.

B.W how about RPC mechanism ? Can it be implemented in Windows also ???

Awaiting for your reply,
Thanks,
rdh

 

[David]

Hello rdh,

Hi David,

Thanks for the reply. was really very informative. I am in process of
analyzing various models to develop the server. My basic requirement is
that :

1) Server can be easily ported to UNIX/LINUX OSes, initiallly to be
developed for Windows platform.

When portability or cross-compatibilty are part of the initial
goals, I think it is best to design with those concepts in mind.
I generally presume these techniques even though many projects
seem to grow out of toolsets that already exist from a given
system. Each methodology has its costs and benefits that we
must balance.

The *ix sockets are the most standard across many platforms.
The Windows WinInet at just the socket layer is also very
campatible with the *ix interfaces. Initialization and
termination have mild differences.

Threading and process models will be different on most
platforms. Design with that in mind. You may find certain
thread classes useful. You can also build your own.
When you have the knoweldge it is usually best to presume
your custom abstraction can grow. When you have no idea
about the abstractions common libraries can be helpful
but often never provide an optimal solution on any plaftorm.
This is certainly true of GUI abstractions.

Try to keep the code that may vary between the OSes
separate from your common code. For instance, a good design
usually keeps the verification logic in one place and the
GUI component in another. The GUI components will be
expected to be different for each platform. When the
verification logic is pure C++ and common code libraries
there will be less need for OS-abstracted versions.

This list could go on forever. It might be helpful to
keep a light implementation on *ix just to verify things
as you build them under Windows.

Beware of the MFC and Windows libraries. They are not
portable and *ix variants may have significant use
characteristics that make portable code hard to maintain.

2) Implementation will be done in C++

C++ is available for many platforms. There may be slight
differences in what level of STL is available or other
common libraries. These can usually be worked around without
much difficulty.

3) The interaction / communication will be done via XML

That is up to you. You will need to decide how packetized
data will be handled. Will XML conversations ever span a
packet? If so, how will you write that abstraction? If not,
will the XML size constraints significantly hinder the request
or response size?

Even with XML, what will the conversation look like?
HTTP-like conversations are generally simple request-response
with a stateless conversation model. Will you have that
model or something a bit different?

4) Server should accept the connection over any of the protocols
connection oriented or connection less.

Your initialization of each protocol may be a bit different on each
OS. The base server can be standardized fairly easily. The threading
andsynchronization models are what might dictate separate implementations.

5) The data to be transeferred between client-server is very light and
doesnot involve complex heavy data.

I presume you also will not have heavy or complex processing to
service requests. Scalability will also be affected by how many
converations and protocols you plan to have active at the same time.

So m in process of evaluating various servers. Can you please suggest a
server model that suits my requirement.

I'd suggest a simple multi-threaded, multi-protocol server as the
base. These will probably need to be different for each platform
due to the threading and synchronization models required by each OS.

XML can be rather nasty to properly implement unless you are using
a rather restricted definition (which is okay). A library common to
your presumed target OSes may be helpful. Xerces(sp?) and other
libraries come to mind.

Your core request-response processor will likely be portable C++
with use of the XML library calls that you standardize on.

The STL or common C/C++ libraries may provide most of the
other tools that you will need.

Database concepts are available on many platforms if they
are needed. Other concepts that you masy need are likely to
be found on various platforms.

Gluing all of these layers together on all target platforms
should not be too difficult a task.

If you start on just Windows and plan to provide another OS
later, it may be wise to periodically check for Windows
specific concepts and adjust your developments as needed.

Keeping the install, path/file concepts, and any UIs separate
for each platform may be helpful.

B.W how about RPC mechanism ? Can it be implemented in Windows also ???

Windows has some RPC functionality built in. Care must be taken
when depending on IE for your XML or RPC processing. A different
tool must be found under *ix. Depending on each interface chosen
there may be significant rewrite on each platform. I've seen several
XML tools that approach the general API differently on the same
platform. Even under Windows, there are perhaps a dozen ways of
processing XML or RPC transactions.

In general, anything *ix can do Windows can do. Each platform
has certain APIs and abtractions geared toward solving problems
in a different way. What takes Windows and Visual Basic a few
lines, may take volumes when done elesewhere. There is a reason
for the bloat underneath Windows (and linux). Each has some
very powerful tools included. Choosing the common abstractions
that you need will probably server you best.

If you can avoid XML, there may be more optimal ways to
handle your conversation needs. I usually presume people
know when they require XML and accept its inherant processing
requirements. I've seen more than a few teams base products
on XML and later wonder why certain functions take a huge
amount of time or produce seemingly endless amounts of
data. Just remember that it may be an option to question
certain design constrants or decisions if you have control
of all the parts.

Awaiting for your reply,
Thanks,
rdh

Sorry for the long diatribe. I should have answered
tomorrow when I'd had time to get some much needed
rest. I'll probably be off-line tomorrow -- or is that
later today?

David

 

[rdh]

Hi David,
Thanks for the reply. That was very informative!!

The idea for making the model multithreaded-multiprocol seems quite
good according to the situation. But at the moment, I am exploring
other models as well, more keen in RPC mechanism.

How about using XML-RPC ?(XML-RPC is a simple, portable way to make
remote procedure calls over HTTP.) I feel this mechanism suits my
requierments. What do you suggest ?

Thanks and Regards,
rdh

 

[David]

Hello rdh,

Hi David,
Thanks for the reply. That was very informative!!

The idea for making the model multithreaded-multiprocol seems quite
good according to the situation. But at the moment, I am exploring
other models as well, more keen in RPC mechanism.

Exploring your options is a good tactic.

How about using XML-RPC ?(XML-RPC is a simple, portable way to make
remote procedure calls over HTTP.) I feel this mechanism suits my
requierments. What do you suggest ?

That may be appropriate. Perhaps we should take this discussion
to email. I'll send an email to your posted address. If that isn't
valid send me an email at [huey dot dll at gte dot net].

If you are exploring several conversation models as well as
existing technologies, don't forget the most simple and obvious
methods, such as text and binary messages. They only fail when
your needs exceed your initial design and are extremely easy
to document and implement. Take for example the basic design
of SMTP for email. I can't think of a particular example for
a binary mode that involved simple message protocols. The
IP headers might be a good example. If you are unfamiliar with
these look up the term RFC (Request For Comments) and check
out a few of the more popular legacy protocols. XML takes
a bit more to process but it does provide additional language
support and tools are starting to emerge for middleware.

Perhaps if we talk about your actual transactions I
could recomend a few protocols to evaluate.

Thanks and Regards,
rdh

David

 

[siddurampure]

u have to use tcp/ip,

for connection.

u can send me u r server code aswell as client code.

i may modified it.

 

[rdh]

Hi sidduramp,

XML-RPC mechanism seems quite feasible option , so I am in process of
evaluating/analyzing more into it.

rohit