1281443 - [tracker] Index remaining MXR trees

Status: RESOLVED FIXED

(Webtools Graveyard :: DXR, defect)

Description

[Kendall Libby [:fubar] (he/him)]

Tracker bug for indexing the remaining MXR trees here in DXR. Repos will be indexed in the following order unless otherwise specified. Please file blocking bugs against this if you wish to prioritize the indexing of specific repos.

Initial indexing of some trees (eg comm-*) may be text-only on the first pass to match MXR; we'll note this as we work through the list and circle back to add proper build indices.

addons
servo
l10n-central
l10n-mozilla-aurora
l10n-mozilla-beta
l10n-mozilla-esr*
l10n-mozilla-release
mozilla-b2g*
comm-aurora
comm-beta
comm-esr*
comm-release
mozilla-esr*
mozilla2.0
mozilla1.9.1
mozilla1.9.2
comm-2.0
mozilla-org
l10n-gaia*
blink
webkit
chromium
projects-central
webtools-central
bugzilla*
bugzilla-org
camino
comm-1.9.1
comm-1.9.2
fuel
incubator-central
jss
l10n-mozilla1.9.1
l10n-mozilla1.9.2
l10n-mozilla2.0
labs-central
mobile-browser
mozilla1.0
mozillasvn
mozillasvn-all
mozillausers-central
mozmill-tests
python-nss
services-central
tamarin-central

Comment 1

[Kendall Libby [:fubar] (he/him)]

static analysis build of servo is blocked on https://github.com/servo/servo/issues/11823, but a text-only index is done. it's on the latest format version, so won't show up until the rest of the (existing) trees and web nodes catch up, alas. working on that now...

Comment 2

[Kent James (:rkent)]

I've asked on the tb-planning list if anyone cares about comm-1.9.1 or comm-1.9.2 and nobody mentioned a need. I'm not even sure where these reside in the repositories these days. So if you are interested in cutting back the work, you could consider not indexing these in dxr. It I could figure out where their repositories are, the standard repository view should be sufficient.

With the time saved, it would be great to have an index of the pre-mercurial version of the mozilla repository, see https://github.com/ehsan/mozilla-cvs-history

Comment 3

[Tanvi Vyas[:tanvi]]

What is the eta on this?

Comment 4

[Kendall Libby [:fubar] (he/him)]

(In reply to Tanvi Vyas[:tanvi] from comment #3)
> What is the eta on this?

For all of them? None. Is there a specific tree you're waiting on? 

I've been working on addons, which has indexing issues, and l10n, which has required redoing how repo collections are handled; that plus ongoing break/fix has taken all of my time.

Comment 5

[Tanvi Vyas[:tanvi]]

(In reply to Kendall Libby [:fubar] from comment #4)
> (In reply to Tanvi Vyas[:tanvi] from comment #3)
> > What is the eta on this?
> 
> For all of them? None. Is there a specific tree you're waiting on? 
> 
> I've been working on addons, which has indexing issues, and l10n, which has
> required redoing how repo collections are handled; that plus ongoing
> break/fix has taken all of my time.

HI Kendall,

We are waiting on the addons repo.  For bug 1270680, we need to check the addons usuage for a specific function call.  This will help us determine whether we can safely use a systemPrincipal or not.  Thanks!

Comment 6

[Kendall Libby [:fubar] (he/him)]

(In reply to Tanvi Vyas[:tanvi] from comment #5)
>
> We are waiting on the addons repo.  For bug 1270680, we need to check the
> addons usuage for a specific function call.  This will help us determine
> whether we can safely use a systemPrincipal or not.  Thanks!

Ok, we're tracking that over in bug 1280948; having some issues getting it index, but it's the top priority.

Comment 7

[Kendall Libby [:fubar] (he/him)]

(In reply to Kendall Libby [:fubar] from comment #0)
> 
> addons

bug 1280948; nearly done

> servo

has some resource issues on the indexing run; needs investigating.

> l10n-central
> l10n-mozilla-aurora
> l10n-mozilla-beta
> l10n-mozilla-esr*
> l10n-mozilla-release

bug 1281218; done

> mozilla-b2g*

text-only indexes added; will circle back to get compiled indexes added

> comm-aurora
> comm-beta
> comm-esr*
> comm-release

done. older ESR builds are text-only for the time being; esr45 is compiled.

> mozilla-esr*

as above for the comm-esr* jobs.

> mozilla2.0
> mozilla1.9.1
> mozilla1.9.2

done.

> comm-2.0
> comm-1.9.1
> comm-1.9.2
> l10n-gaia*
> l10n-mozilla1.9.1
> l10n-mozilla1.9.2
> l10n-mozilla2.0
> tamarin-central

these repos have all been retired are no longer on hg.mozilla.org. :pike indicated that the l10n repos were no longer required. will give the others the CVS treatment as a lower priority 


remaining:

> mozilla-org
> blink
> webkit
> chromium
> projects-central
> webtools-central
> bugzilla*
> bugzilla-org
> camino
> fuel
> incubator-central
> jss
> labs-central
> mobile-browser
> mozilla1.0
> mozillasvn
> mozillasvn-all
> mozillausers-central
> mozmill-tests
> python-nss
> services-central

some of these are CVS repos that I missed in the first pass.

Comment 8

[Kendall Libby [:fubar] (he/him)]

added, if not yet indexed:
> projects-central
> webtools-central
> services-central
> incubator-central
> mozmill-tests
> python-nss
> labs-central
> mobile-browser
> mozilla-org
> camino
> fuel
> jss
> mozillasvn
> mozillasvn-all

trees to skip:
> mozilla1.0
> mozillausers-central
> bugzilla*
> bugzilla-org

remaining:
> blink
> webkit
> chromium
> comm-2.0
> comm-1.9.1
> comm-1.9.2
> tamarin-central


for posterity, or in case we need to resurrect the svn trees:
mkdir /data/dxr_data/mozillasvn
cd $!
svn co https://svn.mozilla.org/addons/trunk addons
svn co http://svn.mozilla.org/projects/webrunner/trunk projects/webrunner
mkdir /data/dxr_data/mozillasvn-all
cd $!
svn co http://svn.mozilla.org

Comment 9

[Kendall Libby [:fubar] (he/him)]

old repos copied over and added (not yet indexed):
> comm-2.0
> comm-1.9.1
> comm-1.9.2
> tamarin-central


remaining:
> blink
> webkit
> chromium

Comment 10

[Kendall Libby [:fubar] (he/him)]

> blink
> webkit
> chromium

chromium is the only one required for mxr parity, but since there's a bug for all three, will track it there: bug 1029748

also adding bug 1024438 for gaia. it's indexing as text-only at the moment but we should build it if at all possible

Comment 11

[Kendall Libby [:fubar] (he/him)]

Closing; the unindexed, or problematic, trees all have individual bugs open and folks can stay tuned there.