Improved search engine

VERIFIED FIXED in 5.3

Status

P1
normal
VERIFIED FIXED
10 years ago
3 years ago

People

(Reporter: osunick, Assigned: davedash)

Tracking

unspecified
Dependency tree / graph

Details

(Whiteboard: sphinx percona)

(Reporter)

Description

10 years ago
Basically phase 3 of spec here:

http://docs.google.com/Doc?id=dds6vwb4_11wbmfwfdj

Any other ideas for improving search results would be welcome
(Reporter)

Comment 1

10 years ago
Assigning to dash after talking to wil- and DD is the search master
Priority: -- → P1
Assignee: dd-bugzilla → dd
Duplicate of this bug: 503113
To wil's question in 503113: After talking with Laura, members of the internet, and my own gut - I'm going to start down this road using Sphinx.

Over the next week, this bug will be split up into multiple dependencies.

My general approach is going to be as follows:

* Research Sphinx and integrate it with mysql to build our search indexes - verify this via some commandline tools - that this is searchable.  Get some sense of timing on the index building too.
* If the index build takes a while (I imagine it will), we'll make a delta-indexer as well - and run that as often as possible to simulate real-time indexing.
* Cron-ify and package this up nicely so it's easy to install in development and production environments.
* Write an API for search this'll be part of the regular AMO API.
* Use this API to perform search on AMO.
* Collect current production metrics: mysql QPS, lag, cpu load, anything useful that we think might get better.
* Collect new statistics when we launch and blog about how much better our lives are.

I'm adding ozten and laura to the CC since they probably have input to give me.  Or rather Laura has input and Ozten might be using sphinx for something else.

Also - Nick's GDoc needs to be split into separate bugs as well as Sphinx won't do everything and a lot of those are UX.
Status: NEW → ASSIGNED

Updated

10 years ago
No longer blocks: 502946
Blocks: 503502
Blocks: 496976
Moving to 5.0.9 - the only thing that could block me is Fennec work - which is up in the air, but I expect to eat a week out of me.

Per 503354 we *DO NOT* need to do delta indexing.

Here's what happens next:
* Write a new Search API in Cake - versioned as 1.3 as not to disrupt Firefox
* Use API in Frontend to get results
* Package up and write crontab's for indexer as well as fabric tasks for indexing
* Collect production metrics, mysql QPS, search qps, lag, cpu load, anything useful
* Load test sphinx SE based on existing logs
Target Milestone: 5.0.8 → 5.0.9
Blocks: 487500
Blocks: 506273
Target Milestone: 5.0.9 → 5.1
Whiteboard: sphinx
Blocks: 511543
No longer blocks: 441708
Target Milestone: 5.1 → 5.2
Depends on: 517547
Depends on: 517555

Updated

10 years ago
Depends on: 517575
Depends on: 517587

Updated

10 years ago
Depends on: 517572

Updated

10 years ago
Depends on: 517589

Updated

10 years ago
Depends on: 517954
Depends on: 518618

Updated

10 years ago
Depends on: 518646
No longer depends on: 517555
Whiteboard: sphinx → sphinx percona
Target Milestone: 5.2 → 5.3
No longer blocks: 519531
Status: ASSIGNED → RESOLVED
Last Resolved: 10 years ago
Resolution: --- → FIXED
Calling this fixed -- not all bugs have to be fixed (nor were they), before we can call this good.
Status: RESOLVED → VERIFIED
Product: addons.mozilla.org → addons.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.