Boost

[Öö Tiib]

Other serialization libraries are also focused on C++.
Corba takes language neutral approach. Is it thriving?
I tend to think it is stagnant and dying.

If someone asks for C++ solution for serialization then he
gets lot of answers like protocol buffers, jsoncpp,
codesynthesis xsd, tinyxml, boost serialize, and so
on. Like you see all are capable to serialize into language
neutral formats. Other languages have same or
different tools for serialization since what matters is
the format.

My approach is to focus first on C++. After dust
settles, I hope to provide support for one or two other
languages. C++ is most important language for services,
so good place to start.

Lot of network services are not written in C++. I trust majority.

Don't hold your breath if you have requests for some
libraries. The maintenance can be very thin.

I speak of experience. Boost tends to have lower amount of
defects than other C++ libraries. If there is a defect then it is
usually fixed soon and also workaround is found. Anyway if
it looks too complex for me to fix it myself then I avoid it.

 

[woodbrian77]

If someone asks for C++ solution for serialization then he
gets lot of answers like protocol buffers, jsoncpp,
codesynthesis xsd, tinyxml, boost serialize, and so

The C++ Middleware Writer has support for serializing
some of Boost. G-d willing we'll add support for other
types in the future including more of Boost. It will
depend on the needs of users.

on. Like you see all are capable to serialize into language
neutral formats. Other languages have same or
different tools for serialization since what matters is
the format.

I'm talking about binary serialization here.

Lot of network services are not written in C++.

Do you disagree with my claim that C++ is most important
language for services?

I speak of experience. Boost tends to have lower amount of
defects than other C++ libraries. If there is a defect then it is
usually fixed soon and also workaround is found.

Some of Boost is as you describe. Other parts don't
get much love.


Brian
Ebenezer Enterprises
http://webEbenezer.net

 

[Ian Collins]

Do you disagree with my claim that C++ is most important
language for services?

While C++ is an important language for services, it has to inter-operate
with servers and clients written in other languages. This is why most
serialisation formats in common use today (SOAP, XML-RPC, JSON) are
textual or binary extensions of them (BSON).

 

[Öö Tiib]

I'm talking about binary serialization here.

The binary formats are less used than text formats when traffic
is small enough. There are plenty of binary formats and most
have C++ or C serialization/deserialization available. Protocol
Buffers have language neutral binary format from above.

Boost Serialization binary format may have differences between
versions of library and it is C++ only. However it may be useful
for usage within single application.

Do you disagree with my claim that C++ is most important
language for services?

No. Claiming what is more or less important programming language
misses the point about importance of formats. I was saying that
there are lot of services that are not written in C++. These are
important too and the programs need to interoperate so need to
use common format.

Some of Boost is as you describe. Other parts don't
get much love.

Most of the libraries in Boost seem to be made with love.

 

[woodbrian77]

While C++ is an important language for services, it has to inter-operate
with servers and clients written in other languages.

If you had said --

it sometimes has to inter-operate

I'd agree. I know of a few companies that write C++
programs that communicate with each other. That's
not exactly unheard of.

And as I said I'm open to working on adding support
for another language, but I don't think that language
will be Java. Python or C# is more likely.

This is why most
serialisation formats in common use today (SOAP, XML-RPC, JSON) are
textual or binary extensions of them (BSON).

I think scientific apps and games often use binary serialization.

I believe the quality of my software is good and believe
that thinking about the software/service through another
project's eyes will help me to improve the software service.


Brian
Ebenezer Enterprises
http://webEbenezer.net

 

[Ian Collins]

If you had said --

it sometimes has to inter-operate

I'd agree. I know of a few companies that write C++
programs that communicate with each other. That's
not exactly unheard of.

Probably the most common type of "service" these days is the web
service. Web services almost exclusively use SOAP. If you are going to
write the back ends for web pages in C++, you will probably have to use
JSON to communicate with the client JavaScript.

And as I said I'm open to working on adding support
for another language, but I don't think that language
will be Java. Python or C# is more likely.


I think scientific apps and games often use binary serialization.

I believe the quality of my software is good and believe
that thinking about the software/service through another
project's eyes will help me to improve the software service.

By using a proprietary binary format, you are restricting yourself to a
narrow range of applications. The easiest way to support other
languages is to use a well known on wire format.

 

[woodbrian77]

(e-mail address removed) wrote:


Probably the most common type of "service" these days is the web
service. Web services almost exclusively use SOAP. If you are going to
write the back ends for web pages in C++, you will probably have to use
JSON to communicate with the client JavaScript.

By using a proprietary binary format, you are restricting yourself to a
narrow range of applications. The easiest way to support other
languages is to use a well known on wire format.

Maybe. Currently I'm doing byte-level marshalling, but am
thinking about switching to bit-level marshalling. In that
case a boolean would be marshalled as one bit rather than
taking a whole byte.

I'm aware of HDF, netCDF and ASN 1. Do any of them
support bit-level marshalling? Do any of the standards
have support for polymorphism?

I marshall string lengths as variable length integers. Do
any of the standards require fixed length integers for string
lengths? I don't think I want that so am asking to find out
what standards to avoid.

Brian
Ebenezer Enterprises - In G-d we trust.
http://webEbenezer.net

 

[Ian Collins]

Maybe. Currently I'm doing byte-level marshalling, but am
thinking about switching to bit-level marshalling. In that
case a boolean would be marshalled as one bit rather than
taking a whole byte.

Um, why? Surely the extra overhead would outweigh transmission time
savings?

I'm aware of HDF, netCDF and ASN 1. Do any of them
support bit-level marshalling? Do any of the standards
have support for polymorphism?

ASN.1 is awfully complex and out of favour these says. I'm not familiar
with the others.

It looks like you set out to serialise the object model rather than
simply serialising data. Isn't that tying things too tightly to C++?

I marshall string lengths as variable length integers. Do
any of the standards require fixed length integers for string
lengths? I don't think I want that so am asking to find out
what standards to avoid.

Most interchange standards currently in vogue value simplicity over
minimising the payload size.

 

[Jorgen Grahn]

Maybe. Currently I'm doing byte-level marshalling, but am
thinking about switching to bit-level marshalling. In that
case a boolean would be marshalled as one bit rather than
taking a whole byte.

I'm aware of HDF, netCDF and ASN 1. Do any of them
support bit-level marshalling?

ASN.1 with the PER (packed encoding rules) does. But PER is also the
one you need expert help to generate a parser for ... I used it
2000--2001 with some telecom protocol (RANAP?) and it wasn't much fun
....

Personally I suspect a (possibly gzipped) text stream is good enough
almost all the time.

/Jorgen

 

[woodbrian77]

ASN.1 with the PER (packed encoding rules) does. But PER is also the
one you need expert help to generate a parser for ... I used it
2000--2001 with some telecom protocol (RANAP?) and it wasn't much fu
...

Thanks and thanks to Ian for his comment on ASN.1. I think I'll
steer clear of that.

Personally I suspect a (possibly gzipped) text stream is good enough
almost all the time.

I think those working on scientific apps would disagree.
The serialization of their data is faster with binary and
there's probably no need for an additional compression step.

Brian
Ebenezer Enterprises
http://webEbenezer.net

 

[Jorgen Grahn]

.

I think those working on scientific apps would disagree.
The serialization of their data is faster with binary and
there's probably no need for an additional compression step.

Quite possible for some kinds of scientific apps -- I think my "almost
all the time" gets me off that particular hook!

/Jorgen

 

[woodbrian77]

Probably the most common type of "service" these days is the web
service. Web services almost exclusively use SOAP. If you are going to
write the back ends for web pages in C++, you will probably have to use
JSON to communicate with the client JavaScript.

I've been reading about JSON some. The following is
maybe due to some misunderstanding, but it seems
JSON can only deal with user defined objects.

I have a message template (not a C++ template) that
looks like this:

@out :cmw::marshalling_integer, const char*)

(A function is generated based on that.)

IIuc, with JSON, I'd have to create a class that has
those two types in it and then marshal an instance of
that type. I like not having to do that as there's
no need currently for such a class. Those two pieces
of data are from the command line and I just forward
them to the marshalling function without having to
create an object that's only use would be marshalling
it's data members. I will admit it wouldn't take long
to construct such an object, but there's a wrinkle.
If the class were:

class front_tier_type
{
cmw::marshalling_integer account_number;
char* path;
};

That class won't work for me in my middle tier where
it doesn't hurt to store the data from the path variable
in a std::string. I have to store it somewhere in the
middle tier so I choose to put it in a std::string.

Constructing a front_tier_type would be fast, but if
I changed it to have a std:string instead of a char*
it gets more expensive. I don't want to have to pay
for that in the front tier. So iiuc, JSON based
approach would have one class on sending side and
another on the receiving side. The other approach I
described is also different on the sending and receiving
ends, but is more efficient.

If there were more than two pieces of data, constructing
an object becomes more of a waste.

The C++ route seems to offer more flexibility than
the JSON approach.

By using a proprietary binary format,

The format isn't a secret.

you are restricting yourself to a
narrow range of applications. The easiest way to support other
languages is to use a well known on wire format.

I may go back to this angle.

Brian
Ebenezer Enterprises
http://webEbenezer.net

 

[Ian Collins]

I've been reading about JSON some. The following is
maybe due to some misunderstanding, but it seems
JSON can only deal with user defined objects.

It depends on how you use JSON and your chosen library. I would write
something like:

json::Object blob;
blob["number"] = accountNumber;
blob["name"] = accountName;

std::cout << blob;

Constructing a front_tier_type would be fast, but if
I changed it to have a std:string instead of a char*
it gets more expensive. I don't want to have to pay
for that in the front tier. So iiuc, JSON based
approach would have one class on sending side and
another on the receiving side. The other approach I
described is also different on the sending and receiving
ends, but is more efficient.

If there were more than two pieces of data, constructing
an object becomes more of a waste.

The C++ route seems to offer more flexibility than
the JSON approach.

In cases where speed of serialisation isn't critical (most cases I'd
suggest), the flexibility and interoperability of text on the wire wins out.

I often use all or part of the received JSON object directly in the
sever code, so it never gets converted to something else. I guess this
habit comes form having written a lot of JavaScript.

The format isn't a secret.

But you's have to convince others to adopt it.

 

[woodbrian77]

I've been reading about JSON some. The following is
maybe due to some misunderstanding, but it seems
JSON can only deal with user defined objects.

It depends on how you use JSON and your chosen library. I would write
something like:

json::Object blob;
blob["number"] = accountNumber;
blob["name"] = accountName;

std::cout << blob;

That kind of helps, but the indices ("number" and "name")
are a problem. I don't have a means of getting that
information from a user like I do if they provide a header
file with data member names.

Let me change the messsage template a little

@out (int, const char*)

The generated function for that will have a1 and a2
for the names of the arguments. So

json::Object blob;
blob["a1"] = a1;
blob["a2"] = a2;

That doesn't work for me on the receiving side where
I have a user defined type.

In cases where speed of serialisation isn't critical (most cases I'd
suggest), the flexibility and interoperability of text on the wire wins out.

Well where speed is critical, there's a good chance they'll
be using C++ so that much is promising from my perspective.

I often use all or part of the received JSON object directly in the
sever code, so it never gets converted to something else. I guess this
habit comes form having written a lot of JavaScript.

I recall your saying something like that previously. I'd like
to know how accessing a JSON object compares to accessing a C++
object. Is it a lot slower? I recall Juha talking about
something that may have been similar with Objective C.


Brian
Ebenezer Enterprises
http://webEbenezer.net

 

[Ian Collins]

I recall your saying something like that previously. I'd like
to know how accessing a JSON object compares to accessing a C++
object. Is it a lot slower? I recall Juha talking about
something that may have been similar with Objective C.

Given a JSON object is a C++ object, the access will depend on the
complexity of the object. My library is designed for programmer
convenience first, efficiency a close second. I wanted to support
arbitrary nesting, such as

json::Object blob;

blob["one"]["two"]["three"]["four"] << 1 << 2 << "hello";

which creates this object:

{"one":{"two":{"three":{"four":[1,2,"hello"]}}}}

This requires a degree of indirection than a more basic implementation
would.

If I want to use an internal C++ object, I code generate (usually from
an Open Office document) to/from JSON methods for the object's data.

 

[woodbrian77]

Personally I suspect a (possibly gzipped) text stream is good enough
almost all the time.

What about streaming video?

 

[Dombo]

Op 26-Jan-14 23:46, (e-mail address removed) schreef:

If you had said --

it sometimes has to inter-operate

I'd agree. I know of a few companies that write C++
programs that communicate with each other. That's
not exactly unheard of.

And as I said I'm open to working on adding support
for another language, but I don't think that language
will be Java. Python or C# is more likely.

When communicating with a web server chances are it is written in Java.
Python has its own serialization facilities as does .NET, as does Java.
Focusing on one or a very few languages puts your solution at a
disadvantage compared to the more established alternatives. If you want
people to convince to use your solution over the well known, proven,
well supported and industry standard solutions (SOAP, JSON, BSON...etc),
you have to provide some compelling reason, i.e. a unique selling point,
which as of yet I haven't seen in spite of your frequent postings
promoting your library. I think you need to do a bit more market
research before investing in reinventing the wheel.

 

[woodbrian77]

Op 26-Jan-14 23:46, (e-mail address removed) schreef:



When communicating with a web server chances are it is written in Java.
Python has its own serialization facilities as does .NET, as does Java.
Focusing on one or a very few languages puts your solution at a
disadvantage compared to the more established alternatives. If you want
people to convince to use your solution over the well known, proven,
well supported and industry standard solutions (SOAP, JSON, BSON...etc),
you have to provide some compelling reason, i.e. a unique selling point,
which as of yet I haven't seen in spite of your frequent postings
promoting your library. I think you need to do a bit more market
research before investing in reinventing the wheel.

I'm pioneering on line code generation. These people
http://springfuse.com

are also working on on line code generation, but
they use Java. Most of what they say in terms of
marketing their service is also true of the C++
Middleware Writer.

10 years ago there was some doubt about the
path, but I don't think there's any doubt today.
On line code generation is here to stay.

Brian
Ebenezer Enterprises - "Imitation is the sincerest
form of flattery."
http://webEbenezer.net

 

[Dombo]

Op 01-Feb-14 17:16, (e-mail address removed) schreef:

I'm pioneering on line code generation. These people
http://springfuse.com are also working on on line code
generation, but they use Java.

I'd hardly call that pioneering.

Most of what they say in terms of
marketing their service is also true of the C++
Middleware Writer.

Springfuse makes rather generic claims, the kind of marketing speak one
expects for just about any software development tool. And above all it
still not answers the questions how your product differentiates itself
from competing, well established, solutions.

10 years ago there was some doubt about the
path, but I don't think there's any doubt today.
On line code generation is here to stay.

Ask yourself what is the benefit of online code generation for the
client. If it is the only option for code generation it would be a
considered to be a serious disadvantage by many because:
1. It undesirable to make your build process dependant on some external
service not under your own control;
2. But more importantly...what if the company that provides the online
service goes out of business or just decides to pull the plug?

 

[Jorgen Grahn]

What about streaming video?

No. But it seems to me that's a different problem compared to the
ones discussed so far.

/Jorgen