Discussion Locked

Need URL regex

0 helpful
3 replies
Closed to new repliesThis discussion is closed. It stays readable, but it takes no new replies.
I need a posix compliant regular expresion string for extracting URL's from html text and (if possible) from other garbage files. HTTP:, FTP: variants are necessary; other variants would be OK. Just don't have the time to figure this one out. Your help is greatly and graciously appreciated.

3 replies

by steved

There's a (long) answer at http://www.foad.org/~abigail/Perl/url3.regex and a complete treatment of how it was produced at http://www.foad.org/~abigail/Perl/url2.html.

I also found a nifty perl implementation that goes like this, for what it's worth:

From: Tom Christiansen
Newsgroups: comp.lang.perl.misc,comp.lang.misc,comp.lang.tcl,comp.lang.python

#!/usr/bin/perl
# urlify -- tchrist@perl.com
require 5.002; # well, or 5.000 if you see below

$urls = '(' . join ('|',qw{
http
telnet
gopher
file
wais
ftp
} )
. ')';

$ltrs = '\w';
$gunk = '/#~:.?+=&%@!\-';
$punc = '.:?\-';
$any = "${ltrs}${gunk}${punc}";

while (<>) {
## use this if early-ish perl5 (pre 5.002)
## s{\b(${urls}:[$any]+?)(?=[$punc]*[^$any]|\Z)}{$1}goi;
## otherwise use this -- it just has 5.002ish commentss{
\b # start at word boundary
( # begin $1 {
$urls : # need resource and a colon
[$any] +? # followed by on or more # of any valid character, but
# be conservative and take only
# what you need to....
) # end $1 } (?= # look-ahead non-consumptive assertion
[$punc]* # either 0 or more puntuation
[^$any] # followed by a non-url char
| # or else
$ # then end of the string
)
}{$1}igox;
print;
}

This discussion is closed to new replies.