If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

Framework that indexes into ES

RESOLVED FIXED

Status

Webtools
DXR
RESOLVED FIXED
3 years ago
2 years ago

People

(Reporter: erik, Assigned: erik)

Tracking

Trunk
x86
Mac OS X

Firefox Tracking Flags

(Not tracked)

Details

(Assignee)

Description

3 years ago
Write the framework code that extracts information from the plugins and sticks it into ES. This also includes porting the static-HTML-building bits to the new plugin architecture, unless it turns out to be easier to transition immediately to request-time rendering.
(Assignee)

Comment 1

3 years ago
Questions that will need to be ironed out:

Figure out how to build indices independently:
    Figure out where to put the registry of indices:
        config file?
        another index?
        Do we use ES aliases?
    Who chooses the index name?
        Does it get passed into dxr-index?
(Assignee)

Updated

3 years ago
Assignee: nobody → erik
(Assignee)

Comment 2

3 years ago
Answers:

It gets complicated keeping on-disk folders and ES indices synched when there are format versions and parallel building in play. Let's just stick with serial builds until we have request-time rendering, at which time we can do the full-on https://wiki.mozilla.org/DXR_Parallel_Tree_Indexing.

We'll build in series, lay down a config.py file that can keep holding all the tree metadata, and treat format versions just as we do now, with numbered symlinks pointing to instance folders. The main change will be the addition of an ES alias name to the value side of the TREES OrderedDict.

There will be a little race period between the FS move and the alias change, but the effects will be negligible: a duration of <= 1s (ES refresh time) and no interesting effects (the search results might be older than the rendered markup for 1s).

dxr-build.py should make up the index names based on the alias names in the config file. This fits with my policy of relieving the build system of requirements. The build script is not in the DXR source tree and is fairly Mozilla-specific. Thus, anything we require of it is a burden on each community deployer.

Comment 3

3 years ago
Commit pushed to es at https://github.com/mozilla/dxr

https://github.com/mozilla/dxr/commit/9dc89dac25130a99bf2cc1ba6d5303ee725a440f
Port the build pipeline and test harness to ES. Fix bug 1030309. Close #308.

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → FIXED

Comment 4

2 years ago
Commit pushed to master at https://github.com/mozilla/dxr

https://github.com/mozilla/dxr/commit/9dc89dac25130a99bf2cc1ba6d5303ee725a440f
Port the build pipeline and test harness to ES. Fix bug 1030309. Close #308.
You need to log in before you can comment on or make changes to this bug.