Possible to insert variables into regular expressions?

[Chris Lasher]

Hello,
I would like to create a set of very similar regular expression. In
my initial thought, I'd hoped to create a regular expression with a
variable inside of it that I could simply pass a string into by
defining this variable elsewhere in my module/function/class where I
compile the regular expression, thus making for an easy substitution.
However, still after JFGI'ing, I still cannot find the syntax to place
a variable inside a regular expression in Python. Is such a thing
possible, and if so what is the syntax to do it? If this is not
possible, what would be the Python way of swapping a small part of the
insides of a regexp without having to copy and paste the code to create
as many variations as necessary?

To make this more concrete, I think an example is in order. Let's say
I have a document like this

The Beatles
Rubber Soul
Track Listings
1. Drive My Car
2. Norwegian Wood (This Bird Has Flown)
3. You Won't See Me
4. Nowhere Man
5. Think for Yourself
6. Word
7. Michelle
8. What Goes On
9. Girl
10. I'm Looking Through You
11. In My Life
12. Wait
13. If I Needed Someone
14. Run for Your Life
Original Release Date: December 3, 1965
Number of Discs: 1
Label: Capitol
....

and I want to look for tracks that contain 'oo'. Then, maybe later, I
want to look for 'ee', or maybe 'ou'. All the regular expressions would
start by searching for a digit at the beginning of the line, and they
would allow for characters up until the desired two-letter vowel
combination, and then search for the two-letter vowel combination. How
can I provide for a way to "drop-in" the two-letter combination I want?
Thanks very much for your help and assistance in advance.

Chris

 

[Steven Bethard]

I would like to create a set of very similar regular expression. In
my initial thought, I'd hoped to create a regular expression with a
variable inside of it that I could simply pass a string into by
defining this variable elsewhere in my module/function/class where I
compile the regular expression, thus making for an easy substitution.

Can you use the %-formatting operations? This is what I normally do
with a situation like this. It means you have to compile a new regular
expression for each different value you substitute into the expression,
but that's to be expected anyway when you're trying to check several
different regular expressions...
.... 1. Drive My Car
.... 2. Norwegian Wood (This Bird Has Flown)
.... 3. You Won't See Me
.... 4. Nowhere Man
.... 5. Think for Yourself
.... 6. Word
.... 7. Michelle
.... 8. What Goes On
.... 9. Girl
.... 10. I'm Looking Through You
.... 11. In My Life
.... 12. Wait
.... 13. If I Needed Someone
.... 14. Run for Your Life"""

>>> expr = r'(\w*%s\w*)'
>>> re.compile(expr % r'oo').findall(s) ['Wood', 'Looking']
>>> re.compile(expr % r'ou').findall(s)

['You', 'Yourself', 'Through', 'You', 'Your']

Steve

 

[Chris Lasher]

Thanks for the reply, Steve! That ought to work quite nicely! For some
reason, I hadn't thought of using %-formatting. I probably should have,
but I'm still learning Python and re-learning programming in general.
This helps a lot, so thanks again.

Chris

 

[Craig Ringer]

Thanks for the reply, Steve! That ought to work quite nicely! For some
reason, I hadn't thought of using %-formatting. I probably should have,
but I'm still learning Python and re-learning programming in general.
This helps a lot, so thanks again.

Just be careful, when doing this, that your inserted text is also a
regular expression part. You don't want to do this:

re.compile(r'%s made (\d{1,4}.\d{2})' % "J. Smith"

because you'll match "JB Smith made 24.21" etc as well. You could also
end up inserting ?s , *s etc, resulting in some rather frustrating bugs.

 

[Chris Lasher]

Robert, Craig, thanks for your replies. I'll only be dealing with
alphanumerics for this project, I think, but it's good to be aware of
this possible problem for later projects. The re.escape(...) is a handy
method to know about. Thanks very much.

 

[Terry Hancock]

Thanks for the reply, Steve! That ought to work quite nicely! For some
reason, I hadn't thought of using %-formatting. I probably should have,
but I'm still learning Python and re-learning programming in general.
This helps a lot, so thanks again.

Remember, in Python, a regex (well, regex source) is just a string, so
you can do any string manipulation you want to with it, and python has
a lot of tools for doing that.

One cool way to make regexes more readable, that i've seen proposed,
is to break them into components like the following (please pardon my
lack of regex skill, these may not actually work , but you get the idea):

normalword = r'[A-Za-z][a-z]*'
digit = r'\d'
capletter = r'[A-Z]'

codeblock = normalword + 4*digit + '-' + 3*digit + 2*capletter + '\s+' + normalword

or somesuch (the 'r' BTW stands for 'raw', not 'regex', and while it's the most
obvious use, there's nothing particularly special about r strings, except for
not trying to process escape characters, thus avoiding the "pile of toothpicks"
problem).

And hey, you could probably use a regex to modify a regex, if you were
really twisted. ;-)

Sorry. I really shouldn't have said that. Somebody's going to do it now.

Cheers,
Terry

 

[Steven Bethard]

And hey, you could probably use a regex to modify a regex, if you were
really twisted. ;-)

Sorry. I really shouldn't have said that. Somebody's going to do it now.

Sure, but only 'cause you asked so nicely. =)
.... letter_matcher=re.compile(r'\[A-(?:Za-)?z\]')):
.... return letter_matcher.sub(r'[^\W_\d]', expr)
........ def item_str(matcher):
.... return ' '.join(matcher.findall(text))
.... print 'reg: ', item_str(re.compile(expr))
.... print 'intl:', item_str(re.compile(internationalize(expr),
.... re.UNICODE))
....

>>> compare(r'\d+\s+([A-z]+)', '1 viola. 2 voilà')

reg: viola voil
intl: viola voilà

>>> compare(r'\d+\s+([A-Za-z]+)', '1 viola. 2 voilà')

reg: viola voil
intl: viola voilà

This code converts [A-z] style regexps to a regexp that is suitable for
use with other encodings. Note that without the conversion, characters
like 'à' are not found.

Steve