Closed Bug 822440 Opened 12 years ago Closed 11 years ago

Mirror l10n repos to git

Categories

(Release Engineering :: General, defect)

All
Gonk (Firefox OS)
defect
Not set
normal

Tracking

(blocking-basecamp:-, b2g18 fixed)

RESOLVED FIXED
B2G C4 (2jan on)
blocking-basecamp -
Tracking Status
b2g18 --- fixed

People

(Reporter: mwu, Assigned: Callek)

References

Details

Attachments

(6 files, 6 obsolete files)

949 bytes, patch
hwine
: review+
Details | Diff | Splinter Review
1.32 KB, text/plain
nthomas
: review+
hwine
: checked-in+
Details
2.54 KB, patch
mozilla
: review+
hwine
: checked-in+
Details | Diff | Splinter Review
13.91 KB, patch
catlee
: review+
mozilla
: checked-in+
Details | Diff | Splinter Review
799 bytes, patch
catlee
: review+
mozilla
: checked-in+
Details | Diff | Splinter Review
4.24 KB, patch
catlee
: review+
mozilla
: checked-in+
Details | Diff | Splinter Review
The l10n-central repos are necessary for localizations of certain gecko strings like network error messages. Git repos are necessary for downstream partners to receive anything we make. Therefore, we should mirror l10n-central to git.mozilla.org .
blocking-basecamp: --- → ?
I strongly disagree.

Also, downstream partners will receive pointers to our hg clones for those things developed in git, AFAIK.
It's worth noting that the current Beta (18) l10n is in releases/l10n/mozilla-beta, not l10n-central. Doesn't matter if we're not mirroring though.
Michael - thanks for raising this issue - obviously, it's something we need deliver somehow.

Are you saying the l10n repos (with changesets) will be added to the b2g manifests? 

What's your understanding of how the partners will receive the per locale status information they need?

Axel - are you expecting partners to interact with the l10n dashboards to find the change sets they should use per language? What's your understanding of how the partners will receive the pointers?
When talking to akeybl, I heard that we're passing over tagged hg repos.
s/l10n-central/l10n/ to reflect that we want localizations, not necessarily specific repos
Summary: Mirror l10n-central repos to git → Mirror l10n repos to git
AFAIK partners don't get anything if it doesn't come in a git repo and added to the manifest.
(looping in dietrich; so he and akeybl can track coordinating how to proceed here with axel, stas)

Unclear from reading bug thus far, so being explicit to verify: 
1) for gaia: we have l10n repos which are currently under hg.m.o./gaia-l10n/*. I assume these repos should also be part of the handover to partners? I note that the location of these is still being finalised, per bug#768373 comment #24, and (as far as I know), these l10n repos are not currently in any manifests.

2) for gecko: since we switched to the rapid release, most locales are moving focus from l10n-central and now instead use  http://hg.mozilla.org/releases/l10n/mozilla-aurora and  http://hg.mozilla.org/releases/l10n/mozilla-beta for localization work, and http://hg.mozilla.org/releases/l10n/mozilla-release for releases. Therefore, for b2g releases, I would assume you want us to use http://hg.mozilla.org/releases/l10n/mozilla-release instead of http://hg.mozilla.org/l10n-central. I see the subject change from mentioning l10n-central, which is appropriate, but want to explicitly spell this out to verify.
Component: Release Engineering → Release Engineering: Developer Tools
QA Contact: hwine
(In reply to John O'Duinn [:joduinn] from comment #7)
> Therefore,
> for b2g releases, I would assume you want us to use
> http://hg.mozilla.org/releases/l10n/mozilla-release instead of
> http://hg.mozilla.org/l10n-central.

B2G is shipping off of Gecko 18. For now, that's mozilla-beta.
+'ing and assigning to Alex - Alex please minus if appropriate.
Assignee: nobody → akeybl
blocking-basecamp: ? → +
Target Milestone: --- → B2G C3 (12dec-1jan)
This all hinges on https://bugzilla.mozilla.org/show_bug.cgi?id=822440#c6, so emailing about that.
After Dietrich re-raised the "how to build multilocale" topic in yesterday's meeting, I started thinking about that and came to the following conclusions:


* the current multilocale build instructions involve a lot of manual cloning of hg repos for gaia locales, gecko locales, and compare-locales
* following these steps may work, but this lacks the automation and reproducibility that our current manifest-based repo pulling has
* aiui, our current build manifest requires git shas

The above, combined, tell me that we need a git mirror of:

* http://hg.mozilla.org/build/compare-locales/
* gecko l10n:
** at the very least the locales listed in b2g/locales/all-locales:
*** http://hg.mozilla.org/releases/l10n/mozilla-beta/pt-BR/
*** http://hg.mozilla.org/releases/l10n/mozilla-beta/es-ES/
** possibly up to all of releases/l10n/mozilla-beta and l10n-central/
* gaia l10n:
** dev build locales, at the very least:
*** http://hg.mozilla.org/gaia-l10n/ar/
*** http://hg.mozilla.org/gaia-l10n/en-US/
*** http://hg.mozilla.org/gaia-l10n/es/
*** http://hg.mozilla.org/gaia-l10n/fr/
*** http://hg.mozilla.org/gaia-l10n/pt-BR/
*** http://hg.mozilla.org/gaia-l10n/zh-TW/
** more likely all of gaia-l10n/


Then we need b2g_build.py to add the git manifests to the build manifest (releng).  This would tell our partners how to replicate the multilocale portion of the build in question.

Then we probably need to have build.sh call compare-locales with the appropriate env (PATH and PYTHONPATH for compare-locales) (eng) so this doesn't have to be done manually.
(In reply to Aki Sasaki [:aki] from comment #11)
> After Dietrich re-raised the "how to build multilocale" topic in yesterday's
> meeting, I started thinking about that and came to the following conclusions:
> 
> 
> * the current multilocale build instructions involve a lot of manual cloning
> of hg repos for gaia locales, gecko locales, and compare-locales
> * following these steps may work, but this lacks the automation and
> reproducibility that our current manifest-based repo pulling has
> * aiui, our current build manifest requires git shas
> 
> The above, combined, tell me that we need a git mirror of:

Are you suggesting that in order to fulfill the requirement of documenting  reproducible builds of our 1.0, we'll need to mirror l10n repos to git regardless? If so, I'll pass this bug back to you for resolution. If not, we'll continue the handoff requirement email thread (just CC'd you).
Hey y'all, what's the next step here?
So far, we haven't confirmed what we're actually shipping, git or hg repos.

I'd like to note that there are a bunch of Brazil-vendor-specific settings in gaia right now, like currency and bookmarks. That's not a good basis for unmodified repos going in to the builds that our downstreams would make. That'd be to the extent that you can just kick off one script and it'll get all the right versions etc without any manual intervention.
Bob suggested that we should only ship on hg. John, how did your conversation with Bob play out?
Assignee: akeybl → joduinn
Also adding a needinfo for Kevin, who I reached out to about comment 6 again. We need to make a final decision on how to ship l10n here.
Flags: needinfo?(khu)
Target Milestone: B2G C3 (12dec-1jan) → B2G C4 (2jan on)
We discussed this during triage yesterday and decided it didn't need to happen before the v1 cutoff so I'm getting it off that list.  Please don't think this means it's any less important and if we're wrong, please re-nom.
blocking-b2g: --- → tef+
blocking-basecamp: + → ---
Heard back from Michael Vines: "We can't deal with mercurial, git only please." Let's move forward here.
Flags: needinfo?(khu)
Assignee: joduinn → hwine
(In reply to Alex Keybl [:akeybl] from comment #18)
> Heard back from Michael Vines: "We can't deal with mercurial, git only
> please." Let's move forward here.


per mtg w/hwine, aki, we think we can achieve this with a modification to comment#11. 


1) Generate a new "otoro b2g18" nightly build, which contains the specific 3 locales to deliver to TEF. (en-US, es, pt-BR). This requires us to create one new 3-locale languages file, a text file which lives alongside our existing 6-locale languages file (for dev builds) and our existing 50-locale languages file (for localizers).
2) Create repos on git.m.o for l10n repos for gecko and l10n repos for gaia. Replicate gecko+gaia l10n repos for (en-US, es, pt-BR) locales to git.m.o. Note this is currently different repos, per gaia/gecko component and per locale, and we feel is safer to keep separate. 
3) Modify mapfile logic and mapfile app to handle these additional gaia and gecko l10n repos.
4) Change nightly build process to:
* for most nightly builds, add l10n SHAs (from hg) to manifest
* for nightly "otoro b2g18" builds, generate maps (from git.m.o) repos (similar to what we already do for gecko mapfiles).
5) This means handoff of source will include manifests, with SHAs for l10n repos on git.m.o, to TEF.

This approach gets us meeting new requirement quickly, with minimal risk to existing infrastructure, and still gives us flexibility to 
a) add more locales as needed for v1.0 (TEF)
b) add more locales as needed for v1.1 and beyond (other partners).

If we've missed anything, please chime in. Meanwhile, we're filing dependent bugs to start on this plan of record.
(In reply to John O'Duinn [:joduinn] from comment #19)
> 1) Generate a new "otoro b2g18" nightly build, which contains the specific 3
> locales to deliver to TEF. (en-US, es, pt-BR). This requires us to create
> one new 3-locale languages file, a text file which lives alongside our
> existing 6-locale languages file (for dev builds) and our existing 50-locale
> languages file (for localizers).

Ideally we would just change otoro builds to point to this new languages file rather than create a new type of build.  (A single 3-locale otoro b2g18 build, rather than a 3-locale otoro b2g18 build + a 6-locale otoro b2g18 build).
(In reply to Aki Sasaki [:aki] from comment #20)
> (In reply to John O'Duinn [:joduinn] from comment #19)
> > 1) Generate a new "otoro b2g18" nightly build, which contains the specific 3
> > locales to deliver to TEF. (en-US, es, pt-BR). This requires us to create
> > one new 3-locale languages file, a text file which lives alongside our
> > existing 6-locale languages file (for dev builds) and our existing 50-locale
> > languages file (for localizers).
> 
> Ideally we would just change otoro builds to point to this new languages
> file rather than create a new type of build.  (A single 3-locale otoro b2g18
> build, rather than a 3-locale otoro b2g18 build + a 6-locale otoro b2g18
> build).

aki: wfm.

:dietrich, :akeybl: This is the fastest + safest path in this short timefram, so is our recommendation. Anyone who wants to use one of the non-shipping locales can grab the 50-locale builds, which we continue to generate also. If you've any objections, or we're missing something, please chime in, but otherwise we're continuing with this plan asap.
blocking-b2g: tef+ → ---
Target Milestone: B2G C4 (2jan on) → B2G C3 (12dec-1jan)
per akeybl on Friday morning, this needs to be completed in production before the 1/15 deliverable, so nominating for bb+.

(Note: Actual localization strings are expected after this date, and are not bb+, but that is separate issue. This bug is about setting up newly-asked-for-infrastructure which is required for handoff.)
blocking-basecamp: --- → ?
Target Milestone: B2G C3 (12dec-1jan) → B2G C4 (2jan on)
This is not a blocker for the 15th in the sense that if this doesn't happen, we'll still release. If this needs to be tracked for the 15th for some reason, that's fine, we just need to do that using some other mechanisms, but this is not a basecamp-blocker in the platform/gaia sense of holding the release.
blocking-basecamp: ? → -
(In reply to Johnny Stenback (:jst, jst@mozilla.com) from comment #23)
> This is not a blocker for the 15th in the sense that if this doesn't happen,
> we'll still release. If this needs to be tracked for the 15th for some
> reason, that's fine, we just need to do that using some other mechanisms,
> but this is not a basecamp-blocker in the platform/gaia sense of holding the
> release.

jst: hmmm. thats different to what akeybl told us Friday. There's a lot going on in the closing stages, so important to keep everyone on same page and avoid churn. 

akeybl: is this, or is this not a blocker for 15th? If not, how do we track this?
This is not a blocker. If releng fails to close this bug, we will land the 2 localizations we need directly on trunk and have our partners pull from there as a post 1/15 update. I would approve a patch for this bug, if it gets done in time. We will not further track this bug. Just get it done.
Created https://github.com/mozilla-b2g/gaia/pull/7345 for the new gaia languages-basecamp.json file. (1)

Next, I can look at adding the hg l10n shas to the build manifest.  I think that's as far as I can go without the l10n git repos + mapper, and without the languages-basecamp.json file landing.
blocking-basecamp: - → ---
I wish bugzilla wouldn't let me reset a +'ed/-'ed flag if I don't have perms to set the flag.
Thanks to Callek for helping out.

The first ones to move across are:
Gaia
===
Refers to hg.m.o/gaia-l10n/

Gaia dev languages file (6)
http://hg.mozilla.org/integration/gaia-nightly/raw-file/a9dd7add95e8/shared/resources/languages-dev.json

Gecko
====
Refers to hg.m.o/releases/l10n/mozilla-*/ & hg.m.o/l10n-central (i.e. all 4 repos per locale will be converted and pushed to one repo on git.m.o with a corresponding branch name)

http://hg.mozilla.org/releases/mozilla-b2g18/file/ea80b6e8293b/b2g/locales/all-locales

After those set, additional gaia ones are:
Gaia localizer languages file (50)
http://hg.mozilla.org/integration/gaia-nightly/file/a9dd7add95e8/shared/resources/languages-all.json
Assignee: hwine → bugspam.Callek
Please wait on aki's languages file for basecamp.

Migrating the -dev locales is a non-goal, some of those aren't even maintained. That's just gonna increase the amount of trouble to go through if someone misinterprets those mirrors.
Attached file [repo sync] gaia dev locales (obsolete) —
Attachment #699040 - Flags: review?(hwine)
includes new file this time
Attachment #699040 - Attachment is obsolete: true
Attachment #699040 - Flags: review?(hwine)
Attachment #699041 - Flags: review?(hwine)
Attachment #699113 - Flags: review?(kaze)
Staś: our guidelines require files to be named with underscores. I understand it sucks but could you change this file to `languages_basecamp.json` — and rename the other languages-*.json to language_*.json please?
We've had that discussion before, renaming the existing files will break the existing build automation.
Apart from that, I don’t understand how this is related to the bug title.

1/ we really need the git repository to be updated with the strings from the mercurial repo once in a while, it’d make the developers life *much* easier to test l10n-specific bugs;

2/ if for some reason the l10n team refuses to update the git repository, please open another bug.
(In reply to Axel Hecht [:Pike] from comment #34)
> We've had that discussion before, renaming the existing files will break the
> existing build automation.

BTW, these files are not used by Gaia and never will be, so why are they in the shared/resources/ directory at all?

It looks like their only purpose is to help the build magic, so they should be in the locales/ directory — along with an updated README.md file, if possible.
(In reply to Fabien Cazenave [:kaze] from comment #36)
> (In reply to Axel Hecht [:Pike] from comment #34)
> > We've had that discussion before, renaming the existing files will break the
> > existing build automation.
> 
> BTW, these files are not used by Gaia and never will be, so why are they in
> the shared/resources/ directory at all?
> 
> It looks like their only purpose is to help the build magic, so they should
> be in the locales/ directory — along with an updated README.md file, if
> possible.

I agree, filed bug 827778 for this.
Comment on attachment 699041 [details] [diff] [review]
[repo sync] gaia dev locales

lgtm
Attachment #699041 - Flags: review?(hwine) → review+
In bug 827778, :bhearsum suggests to copy these `shared/resources/languages-*.json' files to `locales/languages_*.json' first, adapt the build system, and remove them from shared/resources afterwards. So could we land this new JSON file in locales right now, along with the other languages-*.json ones?
Depends on: 827914
(In reply to Fabien Cazenave [:kaze] from comment #39)
> In bug 827778, :bhearsum suggests to copy these
> `shared/resources/languages-*.json' files to `locales/languages_*.json'
> first, adapt the build system, and remove them from shared/resources
> afterwards. So could we land this new JSON file in locales right now, along
> with the other languages-*.json ones?

https://bugzilla.mozilla.org/show_bug.cgi?id=827778#c4
https://github.com/mozilla-b2g/gaia/pull/7399
Just to document it, somewhere. I named these dir's (on the conversion machine) as follows. This does *not* reflect end-result of git.m.o, nor imply the plan has changed from what was above:

drwxr-x--- 3 vcs2vcs vcs2vcs 4096 Jan  9 04:41 ./releases-l10n-aurora-es-ES-gecko
drwxr-x--- 3 vcs2vcs vcs2vcs 4096 Jan  9 04:43 ./releases-l10n-aurora-pt-BR-gecko
drwxr-x--- 3 vcs2vcs vcs2vcs 4096 Jan  9 04:42 ./releases-l10n-beta-es-ES-gecko
drwxr-x--- 3 vcs2vcs vcs2vcs 4096 Jan  9 04:44 ./releases-l10n-beta-pt-BR-gecko
drwxr-x--- 3 vcs2vcs vcs2vcs 4096 Jan  9 04:39 ./releases-l10n-central-es-ES-gecko
drwxr-x--- 3 vcs2vcs vcs2vcs 4096 Jan  9 04:40 ./releases-l10n-central-pt-BR-gecko
drwxr-x--- 3 vcs2vcs vcs2vcs 4096 Jan  9 04:43 ./releases-l10n-release-es-ES-gecko
drwxr-x--- 3 vcs2vcs vcs2vcs 4096 Jan  9 04:44 ./releases-l10n-release-pt-BR-gecko

I'm running the --> git conversion now, but this will not *yet* push any repo to git.m.o
We may or may not need this patch, depending on how mapper works.  This would give us the location for the git repo.  The existence of these keys can also indicate that we want to add the git sha's to the b2g18 otoro nightly build manifest.
(In reply to Aki Sasaki [:aki] from comment #42)
> We may or may not need this patch, depending on how mapper works.

Let's make it work so you don't need these. aiui, this is simply to do with:
 a) what data we put in the mapfile (i.e. combining how many repos into one file)
 b) what we name the access URL (symlink I believe)  

> This
> would give us the location for the git repo.  The existence of these keys
> can also indicate that we want to add the git sha's to the b2g18 otoro
> nightly build manifest.

If we do need the keys, I would suggest using the "git usable" url, not the URL to the website, e.g.: http://git.mozilla.org/releases/l10n/%(locale)s/gaia.git
(In reply to Hal Wine [:hwine] from comment #43)
> Let's make it work so you don't need these. aiui, this is simply to do with:
>  a) what data we put in the mapfile (i.e. combining how many repos into one
> file)
>  b) what we name the access URL (symlink I believe)  

I think that may require extra work on the mozharness side to consume that data, though I'm still not quite sure what I'm getting back from the app.

I'll need some sort of indicator in this file, so if we don't use these I'll have a bool or something.

> If we do need the keys, I would suggest using the "git usable" url, not the
> URL to the website, e.g.:
> http://git.mozilla.org/releases/l10n/%(locale)s/gaia.git

Ok.
Updated paths.
Attachment #700065 - Attachment is obsolete: true
Attached patch (mozharness) hg l10n manifest (obsolete) — Splinter Review
I've got placeholders for git sha lines, but am waiting to hook that up til I have more to work with in mapper.

Needs testing.
Still needs testing, but now I can test everything.
Attachment #700122 - Attachment is obsolete: true
currently deployed, manually checked. Run every 5 minutes via crontab.
Attachment #700194 - Flags: review?
changes to master script to update directory of mapfiles when we want to collect and publish them.

running in production, but needs review.
Attachment #700196 - Flags: review?
triage says this should be blocking tef+ release.   marking flags.
blocking-b2g: --- → tef+
blocking-basecamp: --- → -
(In reply to Tony Chung [:tchung] from comment #51)
> triage says this should be blocking tef+ release.   marking flags.

we're tracking this work (remove unshipping locales) into bug 818306.  so clearing the blocking-b2g: tef+ flag.
blocking-b2g: tef+ → ---
ok, the l10n beta gecko repos for es-ES and pt-BR are being mirrored now, and are part of our monitoring.

http://hg.mozilla.org/users/hwine_mozilla.com/repo-sync-configs/rev/3d6552ea29d9

The code we have in place for the other branches of the l10n repos causes slight havok with beta atm, so is our tier_2 plan and needs to be figured out, but this should unblock aki and jan15 work aiui.
This will make b2g18 otoro nightlies include git shas in their nightly manifests.
Attachment #700118 - Attachment is obsolete: true
Attachment #700159 - Attachment is obsolete: true
Attachment #700745 - Flags: review?(catlee)
Comment on attachment 700742 [details] [diff] [review]
(mozharness) add hg + git l10n shas to build manifest

Review of attachment 700742 [details] [diff] [review]:
-----------------------------------------------------------------

::: scripts/b2g_build.py
@@ +435,5 @@
>          vcs = self.config['compare_locales_vcs']
>          abs_rev = self.vcs_checkout(repo=repo, dest=dest, revision=rev, vcs=vcs)
>          self.set_buildbot_property('compare_locales_revision', abs_rev, write_to_file=True)
>  
> +    def query_translate_hg_to_git(self, gecko_config_key=None):

nit - rename to query_do_translate_XXX or something? the function name makes it sound like it's actually doing the translation.

@@ +478,5 @@
> +                if gecko_l10n_git_root:
> +                    url = manifest_config['translate_base_url']
> +                    git_repo = gecko_l10n_git_root % {'locale': locale}
> +                    l10n_git_sha = self.query_translated_revision(url, 'l10n', revision)
> +                    locale_manifest.append('  <project name="%s" path="gecko-l10n/%s" remote="mozillaorg" revision="%s"/>' % (git_repo.replace(git_base_url, ''), locale, l10n_git_sha))

would be nice to refactor those two nearly-identical if blocks. doesn't block landing though.
Attachment #700742 - Flags: review?(catlee) → review+
Comment on attachment 700742 [details] [diff] [review]
(mozharness) add hg + git l10n shas to build manifest

Landed, with nits addressed.
http://hg.mozilla.org/build/mozharness/rev/77726c546f0e
Attachment #700742 - Flags: checked-in+
Until https://bugzilla.mozilla.org/attachment.cgi?id=700745 is reviewed, approved, and landed on mozilla-b2g18, we'll just get hg l10n sha's in the build manifests.

Afterwards we'll get git shas in otoro b2g18 nightlies only.
Attachment #700745 - Flags: review?(catlee) → review+
Comment on attachment 700745 [details] [diff] [review]
(mozilla-b2g18) config.json update for otoro

[Approval Request Comment]
Bug caused by (feature/regressing bug #): 

New functionality (git shas for l10n repos in build manifest) - bug 822440.

User impact if declined: 

No git shas for l10n repos in build manifest.

Testing completed: 

Was able to create git shas for l10n repos in staging (off a user repo mozilla-b2g18).

Risk to taking this patch (and alternatives if risky):

This will trigger additional build-time logic for the git shas.  I'm not too worried about it due to testing, but if there's a bug in that logic there could be build bustage.

String or UUID changes made by this patch:

None.
Attachment #700745 - Flags: approval-mozilla-b2g18?
Comment on attachment 700745 [details] [diff] [review]
(mozilla-b2g18) config.json update for otoro

Approving. Although this isn't b-b+, we do want this for the partner handoff.
Attachment #700745 - Flags: approval-mozilla-b2g18? → approval-mozilla-b2g18+
Comment on attachment 699113 [details]
Pointer to Github pull request: https://github.com/mozilla-b2g/gaia/pull/7345/files

I think this was replaced by the pull request in bug 827778.
Attachment #699113 - Attachment is obsolete: true
Attachment #699113 - Flags: review?(kaze)
I busted this when I addressed your nits the last review.
Also, for debugging purposes, I'm now catting the config.json into the log.
Attachment #701918 - Flags: review?(catlee)
Attachment #701918 - Flags: review?(catlee) → review+
Latest b2g18 otoro nightly has git shas.  Phew!
I think I'm done with my part; leaving this bug open for Hal+Callek.
Attachment #700196 - Flags: review? → review?(aki)
Attachment #700194 - Flags: review? → review?(nthomas)
Attachment #700196 - Flags: review?(aki) → review+
Comment on attachment 700194 [details]
new script to produce uber l10n mapfile

>if $update_needed; then
>    cp -f $master_mapfile $master_mapfile.old

I prefer a -p on copies to preserve the mtime of the original, since that can make debugging a little simpler. Otherwise seems fine.
Attachment #700194 - Flags: review?(nthomas) → review+
Comment on attachment 700194 [details]
new script to produce uber l10n mapfile

checked in with changes per review
 http://hg.mozilla.org/users/hwine_mozilla.com/repo-sync-tools/rev/3a862f42f82e

and deployed on in production
Attachment #700194 - Flags: checked-in+
Comment on attachment 700196 [details] [diff] [review]
changes to support updating centralized mapfiles as needed

checked in with suggested changes from irc:
 http://hg.mozilla.org/users/hwine_mozilla.com/repo-sync-tools/rev/3a862f42f82e

and deployed on in production
Attachment #700196 - Flags: checked-in+
we're done
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
Component: Tools → General
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: