187870 - allow show_bug.cgi for robots

Status: RESOLVED DUPLICATE of bug 223988

(bugzilla.mozilla.org :: General, defect)

Description

[Adam Hauner]

Maybe you should allow access to /show_bug.cgi for WebSpiders. I suppose, that
showing bugs isn't heavy load for resources and spiders can index pages linked
from third party servers (as in articles).

It's also provide low-cost fulltext search on all bugs page via Google.

Comment 1

[Matthias Versen [:Matti]]

i don't like this and suggest a WONTFIX

a) bugzilla is always slow if a spiders starts indexing bugs
(we had this x times that they killed bugzilla)
b) Spammer could easy get many mail adresses via google.

Comment 2

[Adam Hauner]

> a) bugzilla is always slow if a spiders starts indexing bugs
> (we had this x times that they killed bugzilla)

It was for indexing bugs or buglists? If it was for bugs, maybe this bug should
depend on bug 87406 and on installation for b.m.o.

BTW it should be allowed only several spiders... 

> b) Spammer could easy get many mail adresses via google.

Spammers don't respect robots.txt - so they haven't actually problem to scan
pages of BZ for e-mails.

For my information - how is possible to get mail addresses from/via Google? It's
archive?

Comment 3

[Bradley Baetz (:bbaetz)]

Regardless of mod_perl, spidering close to 200000 bugs will be slow. Last week
there was a spider which ignored robots.txt and slowed things down for ages.

WONTFIX

Comment 4

[Adam Hauner]

Okay.

Comment 5

[Dave Miller [:justdave]]

*** Bug 127452 has been marked as a duplicate of this bug. ***