Use Regular Expressions to extract URL's

[Jimbo]

Hello

I am using regular expressions to grab URL's from a string(of HTML
code). I am getting on very well & I seem to be grabbing the full URL
but
I also get a '"' character at the end of it. Do you know how I can get
rid of the '"' char at the end of my URL

Example of problem:

I get this when I extract a url from a string
http://google.com"

I want to get this
http://google.com

My regular expression:

def find_urls(string):
    """ Extract all URL's from a string & return as a list """

    url_list = re.findall(r'(?:http://|www.).*?["]',string)
    return url_list

 

[Steven D'Aprano]

Hello

I am using regular expressions to grab URL's from a string(of HTML
code). I am getting on very well & I seem to be grabbing the full URL
but
I also get a '"' character at the end of it. Do you know how I can get
rid of the '"' char at the end of my URL

Live dangerously and just drop the last character from string s no matter
what it is:

s = s[:-1]


Or be a little more cautious and test first:

if s.endswith('"'):
s = s[:-1]


Or fix the problem at the source. Using regexes to parse HTML is always
problematic. You should consider using a proper HTML parser. Otherwise,
try this regex:

r'"(http://(?:www)?\..*?)"'

 

[Novocastrian_Nomad]

Or perhaps more generically:

import re
string = 'scatter "http://wwww.yahoo.com quotes and text anywhere www.google.com" "www.bing.com" or not'
print re.findall(r'(?:http://|www.)[^"\s]+',string)

['http://wwww.yahoo.com', 'www.google.com', 'www.bing.com']

 

[Walter Overby]

A John Gruber post from November seems relevant. I have not tried his
regex in any language.

http://daringfireball.net/2009/11/liberal_regex_for_matching_urls

Regards,

Walter.