Converting Word Control characters to an ASCII format

[Murali]

Hi,

I support a web application the customers use to update information
regarding various products.
A lot of times, they copy and paste from a Word Document into the
textbox in the provided form.

The cgi beind, reads in this data and stores it in an ASCCI text file.
When the product
information is updated, and this new information is displayed, all the
Word special
characters are displayed as ? (including the apostrophes, hyphens and
other stuff).

I tried dos2unix program to convert this, but it's not working. Is
there a Perl regular
expression out there which can do this job? So I clean up the text
before storing it in a
text file?

Thanks,

-Murali

 

[Gregory Toomey]

Hi,

I support a web application the customers use to update information
regarding various products.
A lot of times, they copy and paste from a Word Document into the
textbox in the provided form.

The cgi beind, reads in this data and stores it in an ASCCI text file.
When the product
information is updated, and this new information is displayed, all the
Word special
characters are displayed as ? (including the apostrophes, hyphens and
other stuff).

I tried dos2unix program to convert this, but it's not working. Is
there a Perl regular
expression out there which can do this job? So I clean up the text
before storing it in a
text file?

Thanks,

-Murali

First use a unix program like od (octal dump) to get a hex dump of the
characters. This gives you an idea of what you're dealing with.

Hex use tr within Perl to remove what you dont want. Or just remove
non-acceptable ascii characters (ie characters other than
a-z,A-Z,0-9,!@#$%^&*() etc).

gtoomey

 

[A. Sinan Unur]

I support a web application the customers use to update information
regarding various products. A lot of times, they copy and paste from a
Word Document into the textbox in the provided form.

The cgi beind, reads in this data and stores it in an ASCCI text file.
When the product information is updated, and this new information is
displayed, all the Word special characters are displayed as ?
(including the apostrophes, hyphens and other stuff).

http://brent.epicserve.com/blog.php?id=20

might help.

I also found

http://www.indwes.edu/Faculty/bcupp/things/Characters/www/windows-
chars.html

informative.

On the other hand, please note that this is not particularly on-topic here.
The problem descriptions and solutions would be the same regardless of the
language you use in the CGI script.

Sinan

 

[Murali]

I found out that http://www.fourmilab.ch/webtools/demoroniser/
offers an excellent Perl-based solution. (Thanks for all who replied.)

-Murali