Last Comment Bug 603506 - pipeline host blacklist
: pipeline host blacklist
Status: RESOLVED WONTFIX
:
Product: Core
Classification: Components
Component: Networking: HTTP (show other bugs)
: Trunk
: x86_64 Linux
: -- enhancement with 2 votes (vote)
: ---
Assigned To: Patrick McManus [:mcmanus]
:
Mentors:
Depends on: 603505
Blocks: 603503 603512 pipelining-review
  Show dependency treegraph
 
Reported: 2010-10-11 17:02 PDT by Patrick McManus [:mcmanus]
Modified: 2012-03-29 02:05 PDT (History)
8 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Dynamic blacklist of hostnames wrt pipelining v1 (13.37 KB, patch)
2010-10-21 12:19 PDT, Patrick McManus [:mcmanus]
no flags Details | Diff | Review
Dynamic blacklist of hostnames wrt pipelining v2 (13.43 KB, patch)
2010-11-03 10:37 PDT, Patrick McManus [:mcmanus]
no flags Details | Diff | Review
Dynamic blacklist of hostnames wrt pipelining v3 (14.20 KB, patch)
2010-12-03 15:27 PST, Patrick McManus [:mcmanus]
no flags Details | Diff | Review
Dynamic blacklist of hostnames wrt pipelining v4 (14.41 KB, patch)
2011-02-18 18:38 PST, Patrick McManus [:mcmanus]
no flags Details | Diff | Review
hostname blacklist 5 (14.39 KB, patch)
2011-06-26 22:10 PDT, Patrick McManus [:mcmanus]
honzab.moz: feedback-
Details | Diff | Review

Description Patrick McManus [:mcmanus] 2010-10-11 17:02:27 PDT
in addition to the static blacklist of servers, the dynamic blacklisting based on the pipelining feedback module, and the pipeline pretest - we can also add a downloadable list of known-broken-with-pipelining host names.

opera does something very similar.

it makes sense that this list is retrieved as a side effect of a pre-test (603505)
Comment 1 Patrick McManus [:mcmanus] 2010-10-21 12:19:57 PDT
Created attachment 485102 [details] [diff] [review]
Dynamic blacklist of hostnames wrt pipelining v1

as mentioned in the summary, the lists of hosts is piggybacked onto the pipeline-sanity check (it comes back as the first response body).

how often to update that list is an open question, but one I hope to discuss in 603505.
Comment 2 Patrick McManus [:mcmanus] 2010-11-03 10:37:01 PDT
Created attachment 487945 [details] [diff] [review]
Dynamic blacklist of hostnames wrt pipelining v2

minor update for an interface that should have taken nscstring instead of nscautostring
Comment 3 Patrick McManus [:mcmanus] 2010-12-03 15:27:44 PST
Created attachment 495143 [details] [diff] [review]
Dynamic blacklist of hostnames wrt pipelining v3

update bitrot, confrom better to style guide, updates based on experience (i.e. bugs and tweaks), etc..
Comment 4 Patrick McManus [:mcmanus] 2011-02-18 18:38:05 PST
Created attachment 513673 [details] [diff] [review]
Dynamic blacklist of hostnames wrt pipelining v4
Comment 5 Patrick McManus [:mcmanus] 2011-06-26 22:10:26 PDT
Created attachment 542084 [details] [diff] [review]
hostname blacklist 5

larch bitrot
Comment 6 Honza Bambas (:mayhemer) 2011-07-20 15:16:03 PDT
As I mentioned earlier in an email thread, we might consider basing this on a web service, something similar to update check or safe browsing updates, but this mostly depends on estimation how long the list might be.  If significantly long, then using pretest to fetch it will probably not be the most optimal way, it could get blocked and influenced with a larger response.

LevelDB is a perfect candidate to work with here to persist the list.  SQLite is overkill.


This needs to think a bit:
- on what data we base the black list?  how do we collect it?  just on reported bugs?
- should we allow running Firefox instances around the globe report us the list of hosts that actively fail to pipeline with some other data like topology etc? (probably a privacy issue..)
- how long the list we believe might be?  the publishing protocol should be designed based on that
- how often are we going to update it?
- could this be based on bloom filter, transferred compressed on the wire?
Comment 7 Honza Bambas (:mayhemer) 2011-11-03 13:10:36 PDT
Comment on attachment 542084 [details] [diff] [review]
hostname blacklist 5

Dropping the review flag.  Feedback to this attachment in comment 6.
Comment 8 Patrick McManus [:mcmanus] 2011-12-13 07:40:29 PST
I'm going to mark this as wontfix until we have a demonstrated need for it.

The only model I have for it is opera which has a very short list that appears frankly out of date to me. There isn't a lot of value there as opposed to dynamic problem detection.

I also have concerns about privacy and adding dependencies to a production level services implementation.

Note You need to log in before you can comment on or make changes to this bug.