Closed Bug 498999 Opened 15 years ago Closed 15 years ago

Improved search engine

Categories

(addons.mozilla.org Graveyard :: Public Pages, defect, P1)

defect

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: osunick, Assigned: davedash)

References

Details

(Whiteboard: sphinx percona)

Basically phase 3 of spec here:

http://docs.google.com/Doc?id=dds6vwb4_11wbmfwfdj

Any other ideas for improving search results would be welcome
Assigning to dash after talking to wil- and DD is the search master
Priority: -- → P1
Assignee: dd-bugzilla → dd
To wil's question in 503113: After talking with Laura, members of the internet, and my own gut - I'm going to start down this road using Sphinx.

Over the next week, this bug will be split up into multiple dependencies.

My general approach is going to be as follows:

* Research Sphinx and integrate it with mysql to build our search indexes - verify this via some commandline tools - that this is searchable.  Get some sense of timing on the index building too.
* If the index build takes a while (I imagine it will), we'll make a delta-indexer as well - and run that as often as possible to simulate real-time indexing.
* Cron-ify and package this up nicely so it's easy to install in development and production environments.
* Write an API for search this'll be part of the regular AMO API.
* Use this API to perform search on AMO.
* Collect current production metrics: mysql QPS, lag, cpu load, anything useful that we think might get better.
* Collect new statistics when we launch and blog about how much better our lives are.

I'm adding ozten and laura to the CC since they probably have input to give me.  Or rather Laura has input and Ozten might be using sphinx for something else.

Also - Nick's GDoc needs to be split into separate bugs as well as Sphinx won't do everything and a lot of those are UX.
Status: NEW → ASSIGNED
No longer blocks: 502946
Blocks: 503502
Blocks: 496976
Moving to 5.0.9 - the only thing that could block me is Fennec work - which is up in the air, but I expect to eat a week out of me.

Per 503354 we *DO NOT* need to do delta indexing.

Here's what happens next:
* Write a new Search API in Cake - versioned as 1.3 as not to disrupt Firefox
* Use API in Frontend to get results
* Package up and write crontab's for indexer as well as fabric tasks for indexing
* Collect production metrics, mysql QPS, search qps, lag, cpu load, anything useful
* Load test sphinx SE based on existing logs
Target Milestone: 5.0.8 → 5.0.9
Blocks: 487500
Blocks: 506273
Target Milestone: 5.0.9 → 5.1
Whiteboard: sphinx
No longer blocks: 441708
Target Milestone: 5.1 → 5.2
Depends on: 517575
Depends on: 517572
Depends on: 517589
Depends on: 517954
Depends on: 518646
No longer depends on: 517555
Whiteboard: sphinx → sphinx percona
Target Milestone: 5.2 → 5.3
No longer blocks: 519531
Status: ASSIGNED → RESOLVED
Closed: 15 years ago
Resolution: --- → FIXED
Calling this fixed -- not all bugs have to be fixed (nor were they), before we can call this good.
Status: RESOLVED → VERIFIED
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.