When will we be able to search for all the web by "regular expressions"?

Hi all!

As you (should) know, the most powerful method for searching is that by
"extended regular expressions". If you don't know what is it, you can find
a short explaination with examples here:
http://en.wikipedia.org/wiki/Regular_expression.

Much web search engines already support the boolean search, some with the
"near" operator too, but my dream would be to be able to use all the (other)
potentialities that the "extended regular expressions" notation offer.

I believe that the main reason for which it's still not possible are the
high computational costs.

Do you believe that we will have an implementation of the "extended regular
expressions" in future popular web search engines? if so, when? 2010? 2020?

V.

"Vime" <vime (AT) rcate (DOT) com> wrote

I believe that the main reason for which it's still not possible are the
high computational costs.

Incorrect. The main reason its not possible is because no company is going
to waste development resources putting in something that maybe a dozen
people would use once every other month.

If you honestly think that there is a real demand for this out there then I
would say you are not only lacking common sense but are entirely devoid of
any business sense.

On Sat, 16 Oct 2004 02:20:50 +0200, Vime wrote:

Much web search engines already support the boolean search, some with the
"near" operator too, but my dream would be to be able to use all the
(other) potentialities that the "extended regular expressions" notation
offer.

Yahoo! and Google both offer wildcard, replacement, and some level
of "mini" regex, when combined with their other dozen search modifiers and
parameters.

I believe that the main reason for which it's still not possible are the
high computational costs.

Incorrect. The main reason its not possible is because no company is
going
to waste development resources putting in something that maybe a dozen
people would use once every other month.

If you honestly think that there is a real demand for this out there then
I
would say you are not only lacking common sense but are entirely devoid of
any business sense.

The development cost would be about 100 dollars, since the technology is
already existent even in open source tools.
Of course I meant high computational costs comparated to revenues. Anyway I
am available to pay something for searching the web by regexps.

Vime wrote:

I believe that the main reason for which it's still not possible are
the high computational costs.

Incorrect. The main reason its not possible is because no company is
going
to waste development resources putting in something that maybe a
dozen people would use once every other month.

If you honestly think that there is a real demand for this out there
then I
would say you are not only lacking common sense but are entirely
devoid of any business sense.

The development cost would be about 100 dollars, since the technology
is already existent even in open source tools.
Of course I meant high computational costs comparated to revenues.
Anyway I am available to pay something for searching the web by
regexps.

If you understand a bit about how regexps work, you will soon discover that
it takes way to much time to search a few million pages, let alone the
google database.

--
John -> http://johnbokma.com/ MexIT: http://johnbokma.com/mexit/
Perl & Google/WWW: http://johnbokma.com/perl/
Experienced programmer and SEO available: PR7 http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html

Ron Burk wrote:

If you understand a bit about how regexps work, you will soon
discover
that
it takes way to much time to search a few million pages, let alone
the google database.

Really. Last time I surveyed the literature on inverted index
text search schemes, I believe there was at least one paper
demonstrating an efficient index structure for implementing
regular expression searches (I use the term in the precise,
mathematical sense of computer science; I'm not referring
to whatever big pile of widgets Perl might be calling "regexp"
on any given day :-).

So what part of perlres you don't consider a regexp?

Why you would imagine it
too time-consuming, I cannot guess.

back tracking?

I can think up many bad regexps that will consume a lot of time.

When you're allowed to
pre-compile any form of data structure you like, it's hard
to think of any style of search that cannot be made quite
efficient.

You have a link to the above paper?

--
John -> http://johnbokma.com/ MexIT: http://johnbokma.com/mexit/
Perl & Google/WWW: http://johnbokma.com/perl/
Experienced programmer and SEO available: PR7 http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html

Vime wrote:

As you (should) know, the most powerful method for searching is that
by "extended regular expressions". If you don't know what is it, you
can find a short explaination with examples here:
http://en.wikipedia.org/wiki/Regular_expression.

I tend to disagree. It may be the most powerful to a programmer reading
the manual for Regular Expressions (and having the necessary gray
matter to memorize it), but it's pretty weak to anyone who doesn't.
Read: regular users.

Much web search engines already support the boolean search, some with
the "near" operator too, but my dream would be to be able to use all
the (other) potentialities that the "extended regular expressions"
notation offer.

I believe that the main reason for which it's still not possible are
the high computational costs.

No. If it would be of high commercial value, Google and others would
implement it. Not to say it doesn't come with any costs -- it certainly
would. But nobody's willing to pay them because of the little use for
most visitors.

Do you believe that we will have an implementation of the "extended
regular expressions" in future popular web search engines? if so,
when? 2010? 2020?

Maybe via the Google API, in 2010 -- I certainly wouldn't mind.
Though I expect more of a "GQL" (a Google Query Language) than what you
have in mind.

--
Google Blogoscoped
http://blog.outer-court.com

On Sat, 16 Oct 2004 02:20:50 +0200, Vime wrote:

Do you believe that we will have an implementation of the "extended regular
expressions" in future popular web search engines? if so, when? 2010? 2020?

One called "Exalead" supporting regular expression searches was
recently introduced:
<http://www.researchbuzz.org/archives/002094.shtml>
(I recommend subscribing to the newsletter while you're there).

In addition to the regular expression search, I notice ExaLead
supports the "NEAR" operator that I used to use heavily with
AltaVista.

--
Mark R. Blain, markrblain (AT) earthlink (DOT) invalid
(change .INVALID to .NET to reply)

In article <ckppj1$tfm$1 (AT) lacerta (DOT) tiscalinet.it>, Vime
<vime (AT) rcate (DOT) com> writes

Hi all!

As you (should) know, the most powerful method for searching is that by
"extended regular expressions". If you don't know what is it, you can find
a short explaination with examples here:
http://en.wikipedia.org/wiki/Regular_expression.

Much web search engines already support the boolean search, some with the
"near" operator too, but my dream would be to be able to use all the (other)
potentialities that the "extended regular expressions" notation offer.

I believe that the main reason for which it's still not possible are the
high computational costs.

Do you believe that we will have an implementation of the "extended regular
expressions" in future popular web search engines? if so, when? 2010? 2020?

A major advance in hardware technology is needed. The primary
database a web SE has is a list of words with pointers to web pages.
You can't use this technique with regular expressions since the
total set of expressions in indefinitely large.
--
Philip Baker

Do you believe that we will have an implementation of the "extended
regular
expressions" in future popular web search engines? if so, when? 2010?
2020?

One called "Exalead" supporting regular expression searches was
recently introduced:
http://www.researchbuzz.org/archives/002094.shtml
(I recommend subscribing to the newsletter while you're there).

In addition to the regular expression search, I notice ExaLead
supports the "NEAR" operator that I used to use heavily with
AltaVista.

It's wonderful!!! Thanks!!!
I noticed that the regular expression method doesn't work in Exalead with
multiple words. I'm going to ask the developers how regex are supported.