Closed Bug 552992 Opened 12 years ago Closed 3 years ago

Create an addon prototyping Herdict-Firefox integration

Categories

(Mozilla Labs :: General, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED INACTIVE

People

(Reporter: avarma, Assigned: avarma)

References

()

Details

This is documented here:

  https://wiki.mozilla.org/Firefox/Projects/Herdict

A prototype is currently being built here, in the packages/herdict directory:

  http://hg.mozilla.org/users/avarma_mozilla.com/atul-packages

A good initial release of the prototype would probably be an addon that simply replaces one of the network error pages, perhaps the "Server not found" one. We could then refine that and expand the prototype to cover more network error pages.
Assignee: nobody → avarma
Status: NEW → ASSIGNED
I managed to get a very rough proof-of-concept done yesterday at this HG revision:

  http://hg.mozilla.org/users/avarma_mozilla.com/atul-packages/file/423e12b697b7/packages/herdict

It's currently available here:

  https://secure.toolness.com/xpi/herdict.xpi

It doesn't yet look anything like Boriss' mocks, but does display a "Click here to ask Herdict for assistance" link at the bottom of all network error pages; if it's clicked, it issues a request to the Herdict server and displays some basic information.

While this is really rudimentary right now, I think that this actually gets most of the technical hurdles out of the way; the rest is mostly straightforward web development. I'll start work on that once we get Jetpack SDK 0.3 out the door early this week.

I've tested this with Firefox 3.6 on OS X, but I think it actually ought to work with 3.5 as well.
Boriss and I just discovered bug 482874, "Provide a friendlier/more useful alternative when the user encounters a 404 error page", which has implications for this. It also blocks some other bugs that are relevant, too, such as bug 479922, "Network error pages should give me tools instead of asking me questions" (which this bug could be considered a blocker for, in some sense).
Daniel Veditz, Boriss and I discussed the privacy and scalability issues of Herdict-FF integration on a phone call and Veditz emailed us the following thoughts:

I know you would like this to be an opt-out rather than opt-in feature.
We could _maybe_ do that if the client is only reporting when it cannot
connect, because then we have a big error page to show the results where
we can explain what's going on and let people opt-out.

Failure-only reporting might still be semi-useful, at least for popular
sites.
   "Twitter.com is reported unreachable globally"
   "twitter.com is unreachable from China"
   "there are 24 reports that Twitter.com is unreachable"

Or better yet, just a global graphic (like the Firefox download map)
with red dots representing IP's (or groups of IPs) reporting it's
unreachable.

For less popular sites do two dots mean the site is really down (because
only two people have tried to reach it recently)? Or it is two
coincidental local screw-ups?

You could have shaded dots representing some amount of aging of the data
(careful of color-only representations, but shading should be OK;
different shapes would be better with a legend on the side). Maybe last
hour, last two hours, last four hours, last day. Dynamic length
groupings depending on the popularity of the site or number of reports?
Probably confusing, better to be consistent.

We would get much much better data if every user sent us reachability
stats for every domain -- users wouldn't have to guess how popular the
site was, they'd see 20% couldn't reach it and 80% could. But that will
freak people out and might melt our servers so it would definitely have
to be opt-in -- and then we'll get less data overall. You could still
have the opt-in be on the error page -- eventually everyone will get one
and then you could offer a button to check on reported site status with
a check-box to participate in the data collection. Like the
crash-reporting dialog we should remember the checkbox state so if the
user unchecks it the first time we're not trying to trick them into
joining up next time they click the "check site report" button.

Even though the data would be better if people reported on every site, I
think the default-on error-only report would let us do cooler things.
The error page could just have the map right there showing the state
instead of making the user click a button to get it. Although if users
did opt-in to ongoing data collection we could still do that too for at
least those users. My gut feel is that there will be less general
freak-out for the error-only reporting because there's an immediate
connection between the data sent and the user-benefit.

Way cool idea, thanks for pinging me about it.
-Dan
Boriss, Laura and I just had a phone meeting to discuss the status of this project. I updated the "current status" section of the wiki page with the meeting notes:

https://wiki.mozilla.org/index.php?title=Firefox/Projects/Herdict&oldid=218750#Current_Status
Bumping this bug--we ought to write up a postmortem on this experiment, at the very least.
I just spoke with the Herdict team, IE and Google earlier today. It looks like Herdict just got a $1.5M grant from Omidyar to take the project to the next level:

  http://www.omidyar.com/about_us/news/2010/09/08/omidyar-network-advances-internet-freedom-with-15m-grant-herdict

This means they'll be formally creating a non-profit to steward the crowdsourced data, among other things, and they're interested in working with browser vendors to figure out what's needed to provide useful information on the network error pages. We discussed the notion of making the Herdict REST API richer--it's not currently very rich, which is part of why this bug languished--and/or doing more logic on Herdict's server-side to make this possible, among other things.
Closing all inactive bugs in Mozilla labs :: *
Please reopen if this are still relevant.
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → INACTIVE
You need to log in before you can comment on or make changes to this bug.