Finding "\" in a string

[Eric Armstrong]

I'm going crazy, right? Surely it is possible
to find a backslash (\) in a string, and
replace it with two backslashes. That's simple,
right? Just gsub("\\", '\\'). That's all there
is to it.

Except that...

#!/usr/bin/env ruby

testpath = "\\foo\\bar"
p testpath # => "\\foo\\bar" --representation
puts testpath # => \foo\bar --what you see

result = testpath.gsub("\\", '\\')
p result # => "\\foo\\bar"
(represenation unchanged. Should be "\\\\foo\\\\bar")

puts result # => \foo\bar
(unchanged. should be \\foo\\bar)

I've tried lots of gsub variations, like
/\\/, '\\', "\\\\", and anything else I could
think of that came even close to being sensible,
but I've yet to find anything that works.

My workaround will be to add a sed script to
filter the input. But surely that isn't necessary.
Is it?

 

[Arnaud Bergeron]

I'm going crazy, right? Surely it is possible
to find a backslash (\) in a string, and
replace it with two backslashes. That's simple,
right? Just gsub("\\", '\\'). That's all there
is to it.

irb> puts '\\'
\
=> nil

You are replacing single slashes with sigle slashes.

 

[Eric Armstrong]

irb> puts '\\'
\
=> nil

You are replacing single slashes with single slashes.

You would think so, wouldn't you? But note that
I've used single quotes for the replacement.
So it /should/ equate to two backslashes.

So I tried four of them in the replacement string.
I did so yesterday, in fact, and repeated the
experiment today:

testpath = "\\foo\\bar"
result = testpath.gsub("\\", '\\\\')
p result # => "\\foo\\bar" (s/b "\\\\foo\\\\bar"
puts result # => \foo\bar (s/b \\foo\\bar)

Notice that the results are unchanged. So I tried a
simple character: testpath.gsub("\\", 'r')

That worked as expected:

"\\foo\\bar"
\foo\bar
"rfoorbar"
rfoorbar

So it would appear that something strange is happening
in the evaluation of the gsub replacement string.

 

[Paul Battley]

I'm going crazy, right? Surely it is possible
to find a backslash (\) in a string, and
replace it with two backslashes. That's simple,
right? Just gsub("\\", '\\'). That's all there
is to it.

You're not going crazy. You just need eight of 'em. Yes, really!

testpath = "\\foo\\bar"
result = testpath.gsub("\\", "\\\\\\\\")
p result # => "\\\\foo\\\\bar"

Paul.

 

[Paul Battley]

So it would appear that something strange is happening
in the evaluation of the gsub replacement string.

That's right. The replacement string understands back-references of
the form \1, \2 etc, so there's another level of escaping needed.

"\\" <- one backslash
"\\\\" <- two backslashes
"\\\\\\\\" <- four backslashes, or two backslashes in replacement string

Paul.

 

[Eric Armstrong]

That's right. The replacement string understands back-references of
the form \1, \2 etc, so there's another level of escaping needed.

"\\" <- one backslash
"\\\\" <- two backslashes
"\\\\\\\\" <- four backslashes, or two backslashes in replacement string

Very helpful. At least now I have a good idea that it
can be made to work. Now let's see if I can dial in my
understanding with respect to replacement strings...

The fact that \1, \2 are recognized /shouldn't/ matter,
because the tokenizer really ought to ignore any \x
sequence where the x isn't a known special character.
But it's rapidly becoming apparent that the tokenizer
takes any \x and translates it to /something/, so we
need \\ to get one \ in the string.

Now then, we need two backslashes, so four of them
/should/ have worked inside single quotes. (And if
if takes 4 to make 1, why wasn't I effectively
removing the existing quotes when I only supplied two.)

I've seen a similar situation in Java, where it took
2 slashes to get one past the language parser, and
it also took two to get one past the regular expression
parser, so in that case it took four slashes to get
one into the regular expression. That's the most
complex case I've seen before this, but note that it
was the _regular expression_ that needed four to get one.
It wasn't the replacement string. The replacement string
shouldn't be going through the reg exp parser, so it
shouldn't require that many.

It seems that it does, though. Huh?

 

[Paul Battley]

Very helpful. At least now I have a good idea that it
can be made to work. Now let's see if I can dial in my
understanding with respect to replacement strings...

The fact that \1, \2 are recognized /shouldn't/ matter,
because the tokenizer really ought to ignore any \x
sequence where the x isn't a known special character.
But it's rapidly becoming apparent that the tokenizer
takes any \x and translates it to /something/, so we
need \\ to get one \ in the string.

In fact, the tokenizer *does* ignore non-special sequences:

"\\foo".gsub("\\", "\\x") # => "\\xfoo"

And if there's nothing else left in the string, that works, too:

"\\foo".gsub("\\", "\\") # => "\\foo"

However, in order to be able to produce a literal \1, it also needs to
understand \\. Therefore, this gives the same effect:

"\\foo".gsub("\\", "\\\\") # => "\\foo"

But I missed a trick earler. We only actually need six backslashes to
emit two: the end of the string will terminate the sequence as in the
second example:

"\\foo".gsub("\\", "\\\\\\") # => "\\\\foo"

Is it getting clearer?

Paul.

 

[Logan Capaldo]

In fact, the tokenizer *does* ignore non-special sequences:

"\\foo".gsub("\\", "\\x") # => "\\xfoo"

And if there's nothing else left in the string, that works, too:

"\\foo".gsub("\\", "\\") # => "\\foo"

However, in order to be able to produce a literal \1, it also needs to
understand \\. Therefore, this gives the same effect:

"\\foo".gsub("\\", "\\\\") # => "\\foo"

But I missed a trick earler. We only actually need six backslashes to
emit two: the end of the string will terminate the sequence as in the
second example:

"\\foo".gsub("\\", "\\\\\\") # => "\\\\foo"

Is it getting clearer?

Paul.

This is why when I need to gsub backslashes I use a block:

gsub(%r{\}) { '\\' }

 

[Paul Battley]

This is why when I need to gsub backslashes I use a block:

gsub(%r{\}) { '\\' }

That's much easier, but significantly slower:

s = "\\foo" * 1000000

t0 = Time.now
s.gsub("\\", "\\\\\\")
p(Time.now - t0) # 1.394492

t0 = Time.now
s.gsub("\\"){ "\\\\" }
p(Time.now - t0) # 2.863728

If you are dealing with very long strings, it's probably worth the
effort of ciphering the backslashes.

Paul.

 

[Eric Armstrong]

Omigawd. I'm more confused than ever! But
I have some great patterns to follow until
the day of my enlightenment arrives!

Thanks guys. I look forward to the day this
all makes sense...
:_)

 

[Logan Capaldo]

That's much easier, but significantly slower:

s = "\\foo" * 1000000

t0 = Time.now
s.gsub("\\", "\\\\\\")
p(Time.now - t0) # 1.394492

t0 = Time.now
s.gsub("\\"){ "\\\\" }
p(Time.now - t0) # 2.863728

If you are dealing with very long strings, it's probably worth the
effort of ciphering the backslashes.

Paul.

I'm sure it is. I'll learn how to decipher backslashes in gsub when I
come across those strings <g>

 

[Eric Armstrong]

when I need to gsub backslashes I use a block:

gsub(%r{\}) { '\\' }

That's odd. I would have expected that to work,
since the value of the block is the value of
the string.

But %r{\} produces:
"unterminated string meets end of file"

And { '\\' } inserts a single backslash.

Paul's version worked:
gsub("\\"){ "\\\\" }

As did this:
gsub(%r{\\}) { '\\\\' }

Thanks for all the choices! I'm picking the
version with the simplest syntax I'm likely
to recall:
gsub("\\") { '\\\\' }

It may be slower, but it's readable.

Thanks again, guys. There is no way on earth
I would found the solution. Heck, even now
that I see it, I don't really understand it.
(I'm hoping that Paul's in-depth explanation
will make more sense after I've read it 5 or
6 times.)
:_)

 

[William James]

I'm going crazy, right? Surely it is possible
to find a backslash (\) in a string, and
replace it with two backslashes. That's simple,
right? Just gsub("\\", '\\'). That's all there
is to it.

Except that...

#!/usr/bin/env ruby

testpath = "\\foo\\bar"
p testpath # => "\\foo\\bar" --representation
puts testpath # => \foo\bar --what you see

result = testpath.gsub("\\", '\\')
p result # => "\\foo\\bar"
(represenation unchanged. Should be "\\\\foo\\\\bar")

puts result # => \foo\bar
(unchanged. should be \\foo\\bar)

I've tried lots of gsub variations, like
/\\/, '\\', "\\\\", and anything else I could
think of that came even close to being sensible,
but I've yet to find anything that works.

My workaround will be to add a sed script to
filter the input. But surely that isn't necessary.
Is it?

\\foo\\bar

 

[Logan Capaldo]

That's odd. I would have expected that to work,
since the value of the block is the value of
the string.

But %r{\} produces:
"unterminated string meets end of file"

And { '\\' } inserts a single backslash.

Paul's version worked:
gsub("\\"){ "\\\\" }

As did this:
gsub(%r{\\}) { '\\\\' }

Thanks for all the choices! I'm picking the
version with the simplest syntax I'm likely
to recall:
gsub("\\") { '\\\\' }

It may be slower, but it's readable.

Thanks again, guys. There is no way on earth
I would found the solution. Heck, even now
that I see it, I don't really understand it.
(I'm hoping that Paul's in-depth explanation
will make more sense after I've read it 5 or
6 times.)
:_)

Sorry, that wasn't meant to be a _real_ example. It was just the idea
of throwing the block out there so you didn't have to deal with the
very domain specific logic of gsub's second argument.

 

[Paul Battley]

\\foo\\bar

Nice solution.

 

[Eric Armstrong]

Sorry, that wasn't meant to be a _real_ example. It was just the idea of
throwing the block out there so you didn't have to deal with the very
domain specific logic of gsub's second argument.

No problem. If I had sufficient understanding, I would
simply have filled in the blanks. Going through the
exercise gave me a few more data points, so I have a
better handle on what works--although I'm still mystified
as to why.

At any rate, I appreciate the block trick. It's something
I'm beginning to make more use of.

 

[Eric Armstrong]

Nice solution.

I'll say! Wild! What makes /that/ work???

 

[William James]

I'll say! Wild! What makes /that/ work???

\& is simply what was matched by the regex.

It's similar in awk:

BEGIN {
s = " foo "
sub(/[^ ]+/, "&&&", s)
print s
}

===> foofoofoo

 

[Eric Armstrong]

\& is simply what was matched by the regex.

Too cool. I love it.
Thanks.

Another instructive thread.
What a great list.
:_)