Encoding in ruby/tk internals

[Nikolay Ponomarenko]

Hello ruby-talk peoples,

It seems that something is wrong in ruby/tk libraries:

I'm trying to work with cp1251 encoding. Everything works fine, until
i try to use "percent substitutions" - params values are passed in
unicode, ignoring Tk.encoding value.

In pure Tcl all works normal -

encoding system "cp1251"
-command [list tblCmd $table(array) %i %C %s]

%s value is passing to tblCmd in cp1251
but when i try to do something like that in ruby

Tk.encoding = 'CP1251'
@entry = TkEntry.new
@entry.configure('validatecommand',[proc {|s| v(s)},'%s'])

%s is passing to v in utf-8

Such behavior is bug, feature or i'm doing something wrong?
It there any way to retrive "percent substitutions" values in correct
encoding?

 

[Hidetoshi NAGAI]

Hi,

From: Nikolay Ponomarenko <[email protected]>
Subject: Encoding in ruby/tk internals
Date: Wed, 19 Nov 2003 17:39:02 +0900

In pure Tcl all works normal -

encoding system "cp1251"
-command [list tblCmd $table(array) %i %C %s]

%s value is passing to tblCmd in cp1251
but when i try to do something like that in ruby

Tk.encoding = 'CP1251'
@entry = TkEntry.new
@entry.configure('validatecommand',[proc {|s| v(s)},'%s'])

%s is passing to v in utf-8

After calling Tk.encoding_system = 'cp1251',
do you get the same result?

 

[Nikolay Ponomarenko]

Hello Hidetoshi,

In pure Tcl all works normal -
encoding system "cp1251"
-command [list tblCmd $table(array) %i %C %s]

%s value is passing to tblCmd in cp1251
but when i try to do something like that in ruby

Tk.encoding = 'CP1251'
@entry = TkEntry.new
@entry.configure('validatecommand',[proc {|s| v(s)},'%s'])

%s is passing to v in utf-8

HN> After calling Tk.encoding_system = 'cp1251',
HN> do you get the same result?

Yes.
something like

Tk.encoding_system = 'cp1251' # or Tk.encoding = 'cp1251'
$btn = TkButton.new($root) {
text "and here text in cp1251"
}
works right

Problem is only with "%" values

 

[Hidetoshi NAGAI]

Hi,

From: Nikolay Ponomarenko <[email protected]>
Subject: Re: Encoding in ruby/tk internals
Date: Thu, 20 Nov 2003 15:58:37 +0900

HN> After calling Tk.encoding_system = 'cp1251',
HN> do you get the same result?

Yes.
something like

Tk.encoding_system = 'cp1251' # or Tk.encoding = 'cp1251'
$btn = TkButton.new($root) {
text "and here text in cp1251"
}
works right

Problem is only with "%" values

Well, could you try the following patch?

Index: tkentry.rb
===================================================================
RCS file: /src/ruby/ext/tk/lib/tkentry.rb,v
retrieving revision 1.19
diff -u -r1.19 tkentry.rb
--- tkentry.rb 3 Aug 2003 22:07:46 -0000 1.19
+++ tkentry.rb 20 Nov 2003 08:41:54 -0000
@@ -27,9 +27,10 @@

class ValidateArgs
VARG_KEY = 'disvPSVW'
- VARG_TYPE = 'nxsssssw'
+ VARG_TYPE = 'nxeseesw'

def self.scan_args(arg_str, arg_val)
+ enc = Tk.encoding
arg_cnv = []
arg_str.strip.split(/\s+/).each_with_index{|kwd,idx|
if kwd =~ /^%(.)$/
@@ -39,6 +40,12 @@
arg_cnv << TkComm::number(arg_val[idx])
when ?s
arg_cnv << TkComm::string(arg_val[idx])
+ when ?e
+ if enc
+ arg_cnv << Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
+ else
+ arg_cnv << TkComm::string(arg_val[idx])
+ end
when ?w
arg_cnv << TkComm::window(arg_val[idx])
when ?x

 

[Nikolay Ponomarenko]

Hello Hidetoshi,

Thursday, November 20, 2003, 10:44:10 AM, you wrote:

HN> Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)

Seems that something wrong here.

I try to replace it with this for testing and debug purpose

puts Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
puts Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))
arg_cnv << Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))

works right

 

[Hidetoshi NAGAI]

Hi,

From: Nikolay Ponomarenko <[email protected]>
Subject: Re: Encoding in ruby/tk internals
Date: Thu, 20 Nov 2003 18:25:58 +0900

HN> Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
Seems that something wrong here.

I try to replace it with this for testing and debug purpose

puts Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
puts Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))
arg_cnv << Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))

works right

Hmmm...
Could you try to replace
'Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)'
to 'Tk.fromUTF8(TkComm::string(arg_val[idx]), enc)'?

# Is the value of enc is 'cp1251'?

 

[Nikolay Ponomarenko]

Hello, Hidetoshi!
You wrote on Fri, 21 Nov 2003 02:04:10 +0900:

??>> I try to replace it with this for testing and debug purpose
??>>
??>> puts Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)
??>> puts Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))
??>> arg_cnv << Iconv.iconv('CP1251','UTF-8',TkComm::string(arg_val[idx]))
??>>
??>> works right

HN> Hmmm...
HN> Could you try to replace
HN> 'Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)'
HN> to 'Tk.fromUTF8(TkComm::string(arg_val[idx]), enc)'?

Works right. Thanks a lot. Will this fix became official?

HN> # Is the value of enc is 'cp1251'?

Yes, of course.

By the way, why such functions as
Tk.encoding_convertto
Tk.fromUTF8

are not described anywhere?

With best regards,
Nikolay Ponomarenko.

 

[Hidetoshi NAGAI]

Hi,

From: "Nikolay Ponomarenko" <[email protected]>
Subject: Re: Encoding in ruby/tk internals
Date: Fri, 21 Nov 2003 16:17:14 +0900

HN> Hmmm...
HN> Could you try to replace
HN> 'Tk.encoding_convertto(TkComm::string(arg_val[idx]), enc)'
HN> to 'Tk.fromUTF8(TkComm::string(arg_val[idx]), enc)'?
Works right. Thanks a lot. Will this fix became official?

Thank you for your help to debug the problem.
I'll commit the patch.
It will be included in 1.8.1-preview3.