wstring & wifstream

[toton]

Hi,
I have my program using wstring everywhere instead of string.
Similarly I need to process some file, which contains unicode or ascii
character. I need to stream them. Thus I use wifstream etc. However the
open member function of is not templated, and use const char* as
filename. I have my filename as wstring, where c_str() returns const
wchar_t* type. Thus how to convert a wstring to string or const char*
and pass to open of wifstream?

 

[Gernot Frisch]

Hi,
I have my program using wstring everywhere instead of string.
Similarly I need to process some file, which contains unicode or
ascii
character. I need to stream them. Thus I use wifstream etc. However
the
open member function of is not templated, and use const char* as
filename. I have my filename as wstring, where c_str() returns const
wchar_t* type. Thus how to convert a wstring to string or const
char*
and pass to open of wifstream?

I know of functions:
mbtowc and wctomb. I don't know if they are std, nor do I know of a
C++ pendant.

 

[=?iso-8859-1?q?Kirit_S=E6lensminde?=]

Hi,
I have my program using wstring everywhere instead of string.
Similarly I need to process some file, which contains unicode or ascii
character. I need to stream them. Thus I use wifstream etc. However the
open member function of is not templated, and use const char* as
filename. I have my filename as wstring, where c_str() returns const
wchar_t* type. Thus how to convert a wstring to string or const char*
and pass to open of wifstream?

You need to know a bit about the locale settings, but frankly they
could be almost anything. If you're lucky you may find some compiler
specific details on what form the const char * should be. It is a shame
that this part of the standard is still living in the older C universe.

You don't say which platform you're using but the wide character
streams are badly broken on Microsoft's compiler. In practice you
cannot write Unicode characters to them.

There is a part of the Boost library that deals with these issus, but
I've only taken a quick look at it and haven't actually tried them yet.


Kirit

 

[toton]

You need to know a bit about the locale settings, but frankly they
could be almost anything. If you're lucky you may find some compiler
specific details on what form the const char * should be. It is a shame
that this part of the standard is still living in the older C universe.

You don't say which platform you're using but the wide character
streams are badly broken on Microsoft's compiler. In practice you
cannot write Unicode characters to them.

Most of the cases I don't need to read wide charater from stream.
However I am passing the file name for reading which can be wchat type
(i.e only the file name, not the file itself). So file.open() is
creating problem. I pass the file name from some GUI (wxWidgets) which
has a string type of unicode build, a thin wrapper over STL string.n
Thus it's c_str() returns a const wchar_t*. Unfortunately I am using
Windows & visual studio 2003. I had expected a template open function
for ifstream or it's base! .even if ifstream can open the file with the
filename as wide char is fine form me (My file is still ascii, though
file format specifies nothing about encoding). The GUI library
(wxWidgets) gives a function to convert wide char to char type using
current codepage , but I am not sure still will it be able to take care
of languages other than english.

There is a part of the Boost library that deals with these issus, but
I've only taken a quick look at it and haven't actually tried them yet.

can you give a reference to the section is boost?

thanks

 

[P.J. Plauger]

You need to know a bit about the locale settings, but frankly they
could be almost anything. If you're lucky you may find some compiler
specific details on what form the const char * should be. It is a shame
that this part of the standard is still living in the older C universe.

You don't say which platform you're using but the wide character
streams are badly broken on Microsoft's compiler. In practice you
cannot write Unicode characters to them.

In practice they're not broken and you can write Unicode characters.
As with any other Standard C++ library, you need an appropriate
codecvt facet for the code conversion you favor. See our add-on
library, which includes a broad assortment.

There is a part of the Boost library that deals with these issus, but
I've only taken a quick look at it and haven't actually tried them yet.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

 

[=?iso-8859-1?q?Kirit_S=E6lensminde?=]

In practice they're not broken and you can write Unicode characters.
As with any other Standard C++ library, you need an appropriate
codecvt facet for the code conversion you favor. See our add-on
library, which includes a broad assortment.

I presume that I'm just going to show my ignorance of the iostreams
library, but I don't mind that if I can learn how to do this correctly.

Because the OP's question is about wide character filenames rather than
writing wide characters to streams I'm going to start a different
thread as I don't want to hijack this one (any further).


K

 

[P.J. Plauger]

In practice they're not broken and you can write Unicode characters.
As with any other Standard C++ library, you need an appropriate
codecvt facet for the code conversion you favor. See our add-on
library, which includes a broad assortment.

I presume that I'm just going to show my ignorance of the iostreams
library, but I don't mind that if I can learn how to do this correctly.

Because the OP's question is about wide character filenames rather than
writing wide characters to streams I'm going to start a different
thread as I don't want to hijack this one (any further).

Right, it was, and I forgot to address that point too. The latest
VC++ library, like our add-on library, lets you open files by
name specified as a wchar_t * string.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

 

[Jens Theisen]

In practice they're not broken and you can write Unicode characters.
As with any other Standard C++ library, you need an appropriate
codecvt facet for the code conversion you favor. See our add-on
library, which includes a broad assortment.

This is probably a bit OT, but I'm wondering why you would have to set
this up manually. I don't know what the language requirements are
(there are probably none, as usual), but on unix platforms

std::cout.imbue( std::locale("") );

usually inherit the locale from the environment, _except_ for the
character encoding as it appears.

However,

setlocale(LC_ALL, "");

does inherit even the code conversion from the environment on my
system.

Test case:

#include <iostream>
#include <cstdio>

using namespace std;

int main(int argc, char** argv)
{
cout.imbue( locale("") );

// 0xc3 0xa4 is UTF8 for 0xe4, which is ä

cout << " cout: " << char(0xc3) << char(0xa4) << endl; //prints ä
wcout << L"wcout: " << wchar_t(0xe4) << endl; //doesn't

setlocale(LC_ALL, "");

char const* narrow = "\xc3\xa4";
wchar_t const* wide = L"\xe4";

printf("printf narrow: %s\n", narrow); //prints ä
printf("printf wide: %ls\n", wide); //does likewise
}

I presume this is just my broken platform?

Regards,

Jens

 

[P.J. Plauger]

This is probably a bit OT, but I'm wondering why you would have to set
this up manually. I don't know what the language requirements are
(there are probably none, as usual), but on unix platforms

std::cout.imbue( std::locale("") );

usually inherit the locale from the environment, _except_ for the
character encoding as it appears.

However,

setlocale(LC_ALL, "");

does inherit even the code conversion from the environment on my
system.

Test case:

#include <iostream>
#include <cstdio>

using namespace std;

int main(int argc, char** argv)
{
cout.imbue( locale("") );

// 0xc3 0xa4 is UTF8 for 0xe4, which is ä

cout << " cout: " << char(0xc3) << char(0xa4) << endl; //prints ä
wcout << L"wcout: " << wchar_t(0xe4) << endl; //doesn't

setlocale(LC_ALL, "");

char const* narrow = "\xc3\xa4";
wchar_t const* wide = L"\xe4";

printf("printf narrow: %s\n", narrow); //prints ä
printf("printf wide: %ls\n", wide); //does likewise
}

I presume this is just my broken platform?

Nothing broken about it that I can see. The C++ Standard says regrettably
little about how all this stuff should work.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

 

[=?iso-8859-1?q?Kirit_S=E6lensminde?=]

In practice they're not broken and you can write Unicode characters.
As with any other Standard C++ library, you need an appropriate
codecvt facet for the code conversion you favor. See our add-on
library, which includes a broad assortment.

This is probably a bit OT, but I'm wondering why you would have to set
this up manually. I don't know what the language requirements are
(there are probably none, as usual), but on unix platforms

std::cout.imbue( std::locale("") );

usually inherit the locale from the environment, _except_ for the
character encoding as it appears.

However,

setlocale(LC_ALL, "");

does inherit even the code conversion from the environment on my
system.

Test case:
[snip]

I presume this is just my broken platform?

Nothing broken about it that I can see. The C++ Standard says regrettably
little about how all this stuff should work.

I've put my questions in another thread as promised at:

http://groups.google.com/group/comp...253708befb5/9e651dc92162b5fb#9e651dc92162b5fb

I'm not sure what the standard says about how wide character streams
should behave and questions about that are also in my post.

If the streams aren't broken then I'm sure you can tell me what I'm
missing to get them to work.


K