conversion between string and numerical value

[Aman JIANG]

hi

i need a fast way to do lots of conversion that between
string and numerical value(integer, float, double...), and
boost::lexical_cast is useless, because it runs for a long time,
(about 60 times slower than corresponding C functions)
it's too expensive for my program.

is there any way([template] library?) to do this fast and safely,
please ?
(old C-style functions is unbecoming)

 

[Victor Bazarov]

i need a fast way to do lots of conversion that between
string and numerical value(integer, float, double...), and
boost::lexical_cast is useless, because it runs for a long time,
(about 60 times slower than corresponding C functions)
it's too expensive for my program.

So, use the C functions.

is there any way([template] library?) to do this fast and safely,
please ?

I don't know of any, but should be relatively easy for you to roll
your own, no?

(old C-style functions is unbecoming)

Why?

V

 

[Michal Nazarewicz]

hi

i need a fast way to do lots of conversion that between
string and numerical value(integer, float, double...), and
boost::lexical_cast is useless, because it runs for a long time,
(about 60 times slower than corresponding C functions)
it's too expensive for my program.

is there any way([template] library?) to do this fast and safely,
please ?
(old C-style functions is unbecoming)

How about:

#v+
std::string toString(long value) {
char buffer[100];
sprintf(buffer, "%ld", value);
return std::string(buffer);
}

std::string toString(unsigned long value) {
char buffer[100];
sprintf(buffer, "%lu", value);
return std::string(buffer);
}

long atol(std::string str) { return atol(str.c_str()); }
double atof(std::string str) { return atof(str.c_str()); }
#v-

It's C++ all right.

 

[Aman JIANG]

Why?

because its style, it doesn't support wchar_t, and template and so
on...

 

[Aman JIANG]

How about:

#v+
std::string toString(long value) {
char buffer[100];
sprintf(buffer, "%ld", value);
return std::string(buffer);

}

std::string toString(unsigned long value) {
char buffer[100];
sprintf(buffer, "%lu", value);
return std::string(buffer);

}

long atol(std::string str) { return atol(str.c_str()); }
double atof(std::string str) { return atof(str.c_str()); }
#v-

It's C++ all right.

thank you

char buffer[100];

that "100" is a magic number.
and this cannot support wchar_t, too

 

[Michal Nazarewicz]

How about:

#v+
std::string toString(long value) {
char buffer[100];
sprintf(buffer, "%ld", value);
return std::string(buffer);
}

std::string toString(unsigned long value) {
char buffer[100];
sprintf(buffer, "%lu", value);
return std::string(buffer);
}

One more:

std::string toString(double value) {
char buffer[100];
sprintf(buffer, "%f", value);
return std::string(buffer);
}

long atol(std::string str) { return atol(str.c_str()); }
double atof(std::string str) { return atof(str.c_str()); }
#v-

It's C++ all right.

thank you

char buffer[100];

that "100" is a magic number.

Yep, its a number long enough to hold a string representation of
a (unsigned) long value.

and this cannot support wchar_t, too

Uhm? You can rewrite it to support it. Just replace char by wchar_t and
sprintf with swprintf but I'm not sure if you really need it since the
value is converted into std::string anyways.

 

[Michael DOUBEZ]

Michal Nazarewicz a écrit :

How about:

#v+
std::string toString(long value) {
char buffer[100];
sprintf(buffer, "%ld", value);
return std::string(buffer);
}
[snip]
It's C++ all right. thank you
char buffer[100];

that "100" is a magic number.

Yep, its a number long enough to hold a string representation of
a (unsigned) long value.

You can use numeric_limits<>::digits10 to fit the size to you type.

// add one for \0 and for additional digit
// add one more for sign with signed
char buffer [std::numeric_limits<unsigned long>::digits10 + 2];

In fact, with traits, it should be possible to write a template.

Michael

 

[James Kanze]

How about:
#v+
std::string toString(long value) {
char buffer[100];
sprintf(buffer, "%ld", value);
return std::string(buffer);
}
std::string toString(unsigned long value) {
char buffer[100];
sprintf(buffer, "%lu", value);
return std::string(buffer);
}

One more:

std::string toString(double value) {
char buffer[100];
sprintf(buffer, "%f", value);
return std::string(buffer);
}

long atol(std::string str) { return atol(str.c_str()); }
double atof(std::string str) { return atof(str.c_str()); }
#v-
It's C++ all right.

thank you

char buffer[100];

that "100" is a magic number.

Yep, its a number long enough to hold a string representation of
a (unsigned) long value.

On some machines. There's no guarantee (although it's likely to
be sufficient in the near future).

It's not necessarily large enough to hold a "%f" conversion of a
float, and it's not large enough to hold a "%f" conversion of a
double on the machines I usually work on.

I might add that there is no guarantee that sprintf is faster
than boost::lexical_cast, although it's likely *if*
boost::lexical_cast doesn't have partial specializations for
std::string (since in the general case, boost::lexical_cast must
do two conversions, and not just one).

More generally, you might try:

std::string
toString( double value )
{
std:stringstream s ;
s << value ;
return s.str() ;
}

or if that's too slow:

std::string
toString( double value )
{
char buffer[ 100 ] ;
std:strstream s ;
s << value << std::ends ;
assert( s ) ;
return buffer ;
}

(This will at least cause an assertion failure, rather than
undefined behavior, if the converted string requires more than
99 characters. You can, of course, replace the assertion with
any other error handling you wish.)

If all else fails, you can try snprintf, which is not standard
C++, but is standard C99, and probably available with your
implementation (and allows for error checking as well).

Also, you probably want to use the "%g" format (the default for
iostream), unless you know that the input is in a very limited
range.

 

[James Kanze]

Michal Nazarewicz a écrit :

How about:
#v+
std::string toString(long value) {
char buffer[100];
sprintf(buffer, "%ld", value);
return std::string(buffer);
}
[snip]
It's C++ all right.
thank you
char buffer[100];
that "100" is a magic number.

Yep, its a number long enough to hold a string representation of
a (unsigned) long value.

You can use numeric_limits<>::digits10 to fit the size to you type.

The original poster also had a version for double.

// add one for \0 and for additional digit
// add one more for sign with signed
char buffer [std::numeric_limits<unsigned long>::digits10 + 2];

In fact, with traits, it should be possible to write a template.

Possible, but rather tricky. Don't forget that the format
specifier would have to depend on the type. You'd end up having
to explicitly specialize the traits for each type, with as much
work as if you'd just written a separate function for each type.

Just another reason why nobody in their right mind would want to
use the printf family of functions.

 

[Michal Nazarewicz]

Possible, but rather tricky. Don't forget that the format
specifier would have to depend on the type. You'd end up having
to explicitly specialize the traits for each type, with as much
work as if you'd just written a separate function for each type.

Just another reason why nobody in their right mind would want to
use the printf family of functions.

There are plenty of people (including myself) who prefer C stdio to C++
streams.

 

[anon]

std::string
toString( double value )
{
char buffer[ 100 ] ;
std:strstream s ;
s << value << std::ends ;
assert( s ) ;
return buffer ;
}

Looks like something is missing here.
1) Where is buffer used?
2) what is std::ends?

 

[BobR]

std::string
toString( double value )
{
char buffer[ 100 ] ;
std:strstream s ;
s << value << std::ends ;
assert( s ) ;
return buffer ;
}

Looks like something is missing here.
1) Where is buffer used?

In the 'return'! <G>

std:strstream s( buffer, 100 ) ; // ??

2) what is std::ends?

Manipulator: ends
Write `\0' (the string terminator character).

 

[Aman JIANG]

On some machines. There's no guarantee (although it's likely to
be sufficient in the near future).

It's not necessarily large enough to hold a "%f" conversion of a
float, and it's not large enough to hold a "%f" conversion of a
double on the machines I usually work on.

I might add that there is no guarantee that sprintf is faster
than boost::lexical_cast, although it's likely *if*
boost::lexical_cast doesn't have partial specializations for
std::string (since in the general case, boost::lexical_cast must
do two conversions, and not just one).

More generally, you might try:

std::string
toString( double value )
{
std:stringstream s ;
s << value ;
return s.str() ;
}

or if that's too slow:

std::string
toString( double value )
{
char buffer[ 100 ] ;
std:strstream s ;
s << value << std::ends ;
assert( s ) ;
return buffer ;
}

(This will at least cause an assertion failure, rather than
undefined behavior, if the converted string requires more than
99 characters. You can, of course, replace the assertion with
any other error handling you wish.)

If all else fails, you can try snprintf, which is not standard
C++, but is standard C99, and probably available with your
implementation (and allows for error checking as well).

Also, you probably want to use the "%g" format (the default for
iostream), unless you know that the input is in a very limited
range.

Thank you

I think use 'stringstream' to do this work is still expensive,
although it must be faster than boost::lexical_cast. We know that
'strstream' was "deprecated" in C++98.

Is there another way to do this, no stream, and no old C-style
functions, please ?
(sorry my english is not well)

 

[James Kanze]

std::string
toString( double value )
{
char buffer[ 100 ] ;
std:strstream s ;
s << value << std::ends ;
assert( s ) ;
return buffer ;
}

Looks like something is missing here.
1) Where is buffer used?

In the 'return'! <G>

std:strstream s( buffer, 100 ) ; // ??

Yep. That's what I meant. I've gotten so used to
std:stringstream that I forgot. Even though that was the
whole point of the thing: use an on stack buffer, rather than a
dynamically allocated one.

 

[James Kanze]

On 9 20 , 8 29 , James Kanze <[email protected]> wrote:

[...]

I think use 'stringstream' to do this work is still expensive,

In what way? It might use dynamic allocation, but then, if you
use std::string, you use dynamic allocation. How much depends;
if your implementation of std::string uses the short string
optimization, and the strings you generate fit into the short
string, it's possible that there is no dynamic allocation
anywhere.

In general, the relative performance will depend on the
implementation. I've used implementations where
std:stringstream was about the same speed as sprintf, maybe
even faster for some things, and I've used implementations where
it was almost an order of magnitude slower.

although it must be faster than boost::lexical_cast. We know
that 'strstream' was "deprecated" in C++98.

So. It's still there, and will be for a long time.

Is there another way to do this, no stream, and no old C-style
functions, please ?

You can write the conversion routines yourself. Since you don't
need all of the formatting options, you should be able to come
up with something considerably faster than any of the standard
functions. For integral types, it's actually not very
difficult. For floating point types, on the other hand...
correctly converting floating point is far from trivial.

 

[Kai-Uwe Bux]

How about:
#v+
std::string toString(long value) {
char buffer[100];
sprintf(buffer, "%ld", value);
return std::string(buffer);
}
std::string toString(unsigned long value) {
char buffer[100];
sprintf(buffer, "%lu", value);
return std::string(buffer);
}

One more:

std::string toString(double value) {
char buffer[100];
sprintf(buffer, "%f", value);
return std::string(buffer);
}

long atol(std::string str) { return atol(str.c_str()); }
double atof(std::string str) { return atof(str.c_str()); }
#v-
It's C++ all right.
thank you
char buffer[100];
that "100" is a magic number.

Yep, its a number long enough to hold a string representation of
a (unsigned) long value.

On some machines. There's no guarantee (although it's likely to
be sufficient in the near future).

It's not necessarily large enough to hold a "%f" conversion of a
float, and it's not large enough to hold a "%f" conversion of a
double on the machines I usually work on.

I might add that there is no guarantee that sprintf is faster
than boost::lexical_cast, although it's likely *if*
boost::lexical_cast doesn't have partial specializations for
std::string (since in the general case, boost::lexical_cast must
do two conversions, and not just one).

More generally, you might try:

std::string
toString( double value )
{
std:stringstream s ;
s << value ;
return s.str() ;
}

or if that's too slow:

std::string
toString( double value )
{
char buffer[ 100 ] ;
std:strstream s ;
s << value << std::ends ;
assert( s ) ;
return buffer ;
}

I do not understand this version. Where is the magic that modifies the
variable buffer? and how are buffer and the ostrstream s related? It looks
as though you return an uninitialized array of char.

(This will at least cause an assertion failure, rather than
undefined behavior, if the converted string requires more than
99 characters. You can, of course, replace the assertion with
any other error handling you wish.)

If all else fails, you can try snprintf, which is not standard
C++, but is standard C99, and probably available with your
implementation (and allows for error checking as well).

Also, you probably want to use the "%g" format (the default for
iostream), unless you know that the input is in a very limited
range.

Best

Kai-Uwe Bux

 

[Aman JIANG]

[...]

I think use 'stringstream' to do this work is still expensive,

In what way? It might use dynamic allocation, but then, if you
use std::string, you use dynamic allocation. How much depends;
if your implementation of std::string uses the short string
optimization, and the strings you generate fit into the short
string, it's possible that there is no dynamic allocation
anywhere.

'dynamic allocation' was not very slow, but re-copy was.
if there is no re-copy, string will be cheap.

In general, the relative performance will depend on the
implementation. I've used implementations where
std:stringstream was about the same speed as sprintf, maybe
even faster for some things, and I've used implementations where
it was almost an order of magnitude slower.

i have interest at the implementation what you mentioned.
what's the implementation's name, please ?

So. It's still there, and will be for a long time.


You can write the conversion routines yourself. Since you don't
need all of the formatting options, you should be able to come
up with something considerably faster than any of the standard
functions. For integral types, it's actually not very
difficult. For floating point types, on the other hand...
correctly converting floating point is far from trivial.

I have no idea to do this. As you said, integral is easy, but
floating-point is a hot potato.

 

[James Kanze]

On 9 20 , 8 29 , James Kanze <[email protected]> wrote:

[...]

I think use 'stringstream' to do this work is still expensive,

In what way? It might use dynamic allocation, but then, if
you use std::string, you use dynamic allocation. How much
depends; if your implementation of std::string uses the
short string optimization, and the strings you generate fit
into the short string, it's possible that there is no
dynamic allocation anywhere.

'dynamic allocation' was not very slow, but re-copy was.
if there is no re-copy, string will be cheap.

You'll get the re-copy just about anyway on output. Maybe
something like:

std::string
toString( double value )
{
std::string result( ???, '\0' ) ;
std:strstream s( const_cast< char* >( &result[ 0 ] ) ,
result.size() - 1 ) ;
s << value ;
result.resize( strlen( result.c_str() ) ) ;
return result ;
}

Not very pretty, though. (Technically, it's also undefined
behavior according to the current version of the standard. It
does work with all existing implementations, however, and will
be guaranteed in the next release of the standard.)

i have interest at the implementation what you mentioned.
what's the implementation's name, please ?

I'm not 100% sure now, but I think it was an older version of
the library included with g++. The one that was incredibly slow
was the Rogue Wave implementation that comes with Sun CC; file
IO was slow enough to render the library unusable in certain
applications (although we'd been using it before, with the
pre-standard version of the library that came with Sun CC 4.2,
without problems).

[...]

I have no idea to do this. As you said, integral is easy, but
floating-point is a hot potato.

The problem with floating-point is the usual problem with
floating point. Writing a conversion routine is easy. Writing
one which is correct for all input values, is much, much harder.

There are texts available on the Web which explain it (by Guy
Steele, I think), but quite frankly, I would only go that route
as a last resort, if all else failed.

Note that if you don't have a problem with your code ending up
under the GPL, you could probably use the core of whatever is
present in the GNU libc or the g++ library, adopting it to
whatever you need. Note that this use does NOT come under the
additional liberties granted by the LGPL or by the g++ library
itself, so your code would (at least formally) be
"contaminated".

 

[Kai-Uwe Bux]

hi

i need a fast way to do lots of conversion that between
string and numerical value(integer, float, double...), and
boost::lexical_cast is useless, because it runs for a long time,
(about 60 times slower than corresponding C functions)
it's too expensive for my program.

is there any way([template] library?) to do this fast and safely,
please ?
(old C-style functions is unbecoming)

I think it would be very hard to beat the sprintf family. Those functions
tend to be highly optimized and crucial parts are coded in assembler. I
would just wrap it. Note: according to the current standard, writing into
the string directly is undefined behavior. However, it is proposed for the
next standard an very likely to work with current std::string
implementations.


std::string to_string ( double d ) {
// TRICKY: [multithreading]
/*
The use of this static counter might
cause problems in multithreaded code.
*/
static unsigned long length_provided = 1;
std::string result;
result.resize( length_provided );
// TRICKY: [we assume that std::string is contiguous]
/*
This is proposed for the next version of the standard.
Check your library.
*/
int length_needed =
snprintf( &result[0], length_provided, "%f", d );
assert( length_needed >= 0 );
while ( length_needed > length_provided ) {
length_provided = length_needed;
result.resize( length_provided );
length_needed =
snprintf( &result[0], length_provided, "%f", d );
assert( length_needed >= 0 );
}
return ( result );
}


Best

Kai-Uwe Bux