[Q] specify start postion of Regexp matching

[makoto kuwata]

Hi, all.

Is it possible to specify start position of Regexp matching?

str = "foo bar baz"
m = /ba/.match(str)
p m.begin(0) #=> 4
m = /ba/.match(str, 5) # is it possible?
p m.begin(0) #=> 8 (if possible)

If it is possible, some kind of parser or scanner can be
implemented easily.
# StringScanner is a litte too big, I think.

 

[Eric I.]

Hi, all.

Is it possible to specify start position of Regexp matching?

str = "foo bar baz"
m = /ba/.match(str)
p m.begin(0) #=> 4
m = /ba/.match(str, 5) # is it possible?
p m.begin(0) #=> 8 (if possible)

If it is possible, some kind of parser or scanner can be
implemented easily.
# StringScanner is a litte too big, I think.

You could try something like this:

m = /^.{5,}(ba)/.match(str)
p m.begin(1)

In the regular expression, you're saying start at the beginning and
skip at least 5 characters. But then we have to use parens to "note"
the part you're interested in, and then we have to pass 1 rather than
0 to begin, so it reports the location of the first noted match (0
would report where the entire Regexp matched, and that would be the
beginning of the line).

An alternative would be to slice the first n characters off the front
of the string and then do the match.

Eric

====

Interested in hands-on, on-site Ruby training? See http://LearnRuby.com
for information about a well-reviewed class.

 

[Robert Klemme]

You could try something like this:

m = /^.{5,}(ba)/.match(str)
p m.begin(1)

In the regular expression, you're saying start at the beginning and
skip at least 5 characters. But then we have to use parens to "note"
the part you're interested in, and then we have to pass 1 rather than
0 to begin, so it reports the location of the first noted match (0
would report where the entire Regexp matched, and that would be the
beginning of the line).

An alternative would be to slice the first n characters off the front
of the string and then do the match.

Another alternative is to use String#scan - we would have to know what
the OP really wants to parse though to decide whether it's a feasible
solution.

Kind regards

robert

 

[Axel Etzold]

-------- Original-Nachricht --------

Datum: Mon, 26 Nov 2007 00:20:25 +0900
Von: makoto kuwata <[email protected]>
An: (e-mail address removed)
Betreff: [Q] specify start postion of Regexp matching

Hi, all.

Is it possible to specify start position of Regexp matching?

str = "foo bar baz"
m = /ba/.match(str)
p m.begin(0) #=> 4
m = /ba/.match(str, 5) # is it possible?
p m.begin(0) #=> 8 (if possible)

If it is possible, some kind of parser or scanner can be
implemented easily.
# StringScanner is a litte too big, I think.

Dear Makoto,

what about :

class Regexp
def match_index_offset(string,start_pos)
temp=string[start_pos..-1]
ref=self.match(temp)
return temp.index(ref[0])+start_pos
end
end

str = "foo bar baz"
m = /ba/.match_index_offset(str,5)
p m

Best regards,

Axel

 

[makoto kuwata]

Thank you, all.

You could try something like this:
m = /^.{5,}(ba)/.match(str)
p m.begin(1)

In my program, start position is variable such as
def f(n)
m = /^.{n,}(ba)/.match(str)
...
end
In this case, /^.{n,}(ba)/ is created for each time.
It is not effective.

Another alternative is to use String#scan -

String#scan is useful only when regexp pattern is fixed.
input.scan(/FIXED-REGEXP/) do ... end
Using String#scan, it is not able to change regexp pattern
in the loop.

temp=string[start_pos..-1]
ref=self.match(temp)
return temp.index(ref[0])+start_pos

In this solution, temp substring is created every time.
If input string is long, it is not efficient.


Thanks to all your advices.
I'm going to propose to support start position in Regexp#match().

 

[Robert Klemme]

String#scan is useful only when regexp pattern is fixed.
input.scan(/FIXED-REGEXP/) do ... end
Using String#scan, it is not able to change regexp pattern
in the loop.

But in various situations it is possible to use a unified regexp for
scanning or a regexp that comprises all other patterns.

temp=string[start_pos..-1]
ref=self.match(temp)
return temp.index(ref[0])+start_pos

In this solution, temp substring is created every time.
If input string is long, it is not efficient.

This is not true. Creating a substring is fairly cheap because the
character buffer is not copied (copy on write).

I'm going to propose to support start position in Regexp#match().

For the time being it's faster to use one of the other alternatives.
Also, with the new regexp engine in 1.9 your feature might be present
already.

Kind regards

robert

 

[makoto kuwata]

This is not true. Creating a substring is fairly cheap because the
character buffer is not copied (copy on write).

You are right. If input string is not modified, creating substring
doesn't copy anything.
Creating substring may be the solution I wanted.

For the time being it's faster to use one of the other alternatives.
Also, with the new regexp engine in 1.9 your feature might be present
already.

I found that Regexp#match() can take optional 2nd argument which
specifies matching start position in Ruby1.9. Good news.

Thank you, Robert.

 

[7stud --]

You have to create a sub-string with
the 1.8 version, but according to Robert Klemme (above) it's just
creating a pointer into the original string if you're not changing the
substring or original string.

I'm having a hard time confirming that:

str = "hello"
sub_str = str[1, 2]

puts str.object_id
-->76750

puts sub_str.object_id
-->76740

puts sub_str.class
-->String

 

[MonkeeSage]

I'm having a hard time confirming that:

I'm not sure how to confirm it, other than just looking at the source,
and since I'm very poor at C programming, it probably wouldn't help
for me to try that. I'm sure Robert can demonstrate. But I will say
that I'm not suprised that they have different object_id, because they
are different objects. The copy on write is just a back-end
optimization where you pretend that two objects that point to the same
data are unique copies in the front-end, but you don't actually move
any data in the back-end until you have to (i,e., when one of the
objects is changed).

Regards,
Jordan

 

[MonkeeSage]

I'm not sure how to confirm it, other than just looking at the source,
and since I'm very poor at C programming, it probably wouldn't help
for me to try that.

Well, I did anyhow...

http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8/ruby.h
http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8/string.c

And I think the functions of interest are str_new3 and str_new4
(called from rb_str_substr). Specifically, the assignment of
RSTRING(str2)->aux.shared. But like I said, I'm not great with C, so I
could be mistaken.

Regards,
Jordan

 

[MonkeeSage]

Here's a test to show that my reading of the source, and Robert's
assertion, is correct (there is probably a better way to do this...):

#!/usr/bin/env ruby

# disable GC to get fair reading of actual allocation cost
GC.disable

def free_megs
(`free -o`.split("\n")[1].split(' ')[3].to_i/1024).to_s
end

puts "Free megabytes " + free_megs
# make a one megabyte string
s1 = "a" * 1048576
s100 = "" # placeholder to be filled in below
# make 100 substrings of it
0.upto(101) { |i| eval("s#{i}=s1[0..-1]") }

puts s100.length.to_s
puts "Free megabytes " + free_megs

Output:

Free megabytes 588
1048576
Free megabytes 587

Only one meg is used, which is the length of the original string. So,
by inductive inference, the substrings are only pointers back to the
original string rather than copies of the data.

Regards,
Jordan

 

[George]

You have to create a sub-string with
the 1.8 version, but according to Robert Klemme (above) it's just
creating a pointer into the original string if you're not changing the
substring or original string.

I'm having a hard time confirming that:

str = "hello"
sub_str = str[1, 2]

puts str.object_id
-->76750

puts sub_str.object_id
-->76740

puts sub_str.class
-->String

A new ruby object is created, but the string buffer that it points to
is only copied on write.