Closed Bug 556236 Opened 14 years ago Closed 10 years ago

Migrate l10n from components in "Mozilla Localizations" product to category

Categories

(bugzilla.mozilla.org :: Administration, task)

All
macOS
task
Not set
normal

Tracking

()

RESOLVED INVALID

People

(Reporter: Pike, Assigned: dkl)

References

()

Details

Attachments

(1 file)

I don't think we have all the data and details, but let's have a bug to find those out.

We want to move our l10n bugs from being in a component per locale in the "Mozilla Localizations" product to product per locale in a new category. Details in http://groups.google.com/group/mozilla.governance/msg/011a024c7fb39360.

How do we do that migration? What do we need?

What I can do is:
- gather localized product and component descriptions
- hack up an extension to help with the data gathering and bugzilla grind work (*)

I think we want to tweak the enter_bug setup, not sure how, nor when.

I'd love to see folks that know better than I to file bugs on the tasks they see and mark dependencies between them, so that we can pick up things one by one.

Would it make sense to migrate the existing localizations in chunks?

I'd also would like to start using the new scheme for new locales directly, so getting into a shape where it works for 3-5 locales quickly would be great.

(*) The token logic in the bugzilla scripts doesn't really like plain POST requests, which makes an extension with html and form support easier than a command line script, IMHO.
Well, one approach is to schedule this for Whistler 2010 :). If you're in more of a hurry, it can be done sooner....
I for one won't be in whistler.
Oh, you wouldn't be part of the workers. Presumably it'd be reed/gerv/justdave/myself as with last time.

It's just a suggested timeline (well the end-point really). As long as your plans are settled by then, it'd be done then/there.
Actually, whistler is still a quarter of a year out, we should be done with this way sooner.
Sorry to pile on here, but getting this done sooner would be might nice.
Hrm, I'd suggest a google docs spreadsheet (that anyone can edit).
you could use https://spreadsheets.google.com/ccc?key=0AoAvrVnmoSSNdHN3R25CUC1YcWlPcTNJN3FMdFByM2c&hl=en as a template.

roughly we need:

product, component, description, qacontact

i'm unclear as to what you want to do for enter_bug.
do you expect to provide localized ui or something?

fwiw, it's possible to view a single classification, as in:
https://bugzilla.mozilla.org/enter_bug.cgi?classification=Components
... although we don't seem to expose that much.

it's certainly possible to create a couple at a time.

wrt moving bugs from the existing components, by default, i'd move all bugs to a general component. if you would rather provide mappings, you can include extra tabs (one tab per product) with rows for each component. alternatively you can just ask me not to move them and promise to move them later.
ok, i won't be at whistler either.

Do you want us to just create the classification, products, components and qa contacts and leave the bug moving to the individual localizations?

If that's the case, then there's really no reason not to create the components immediately.

It's also possible for us to mark products as requiring "entry" for a localization-triage group and give you (sethb, pike) grant for that group. This would enable your localization teams to move bugs to the components and then we could open the individual products (by removing the "entry" requirement) as a given component is migrated to its product. In this model, it'd probably be best if there was 1 bug per new product (you or I could file it).

The bug's life would be:
1. <filed with product and components descriptions>
2. product + components are created (product is set with entry group)
3. people indicate that they're watching the various components (this is asynchronous with 4)
4. bugs are moved from the old component to the new product
5. someone indicates that they've completed moving bugs from the old component (and which component should get any bugs they couldn't move) and are ready for 'new' bugs to be entered into the new product.
6. reed/marcia move any stubborn bugs
7. the product's entry requirement is removed
8. the old component is deleted
9. the bug is closed
Depends on: 564721
I'm picking up some automation tests again on https://landfill.bugzilla.org/l10ntest/, could I get that updated to a version that's closer to what we're running on bugzilla.mozilla.org? In particular, I'm missing https://landfill.bugzilla.org/l10ntest/config.cgi?ctype=json&flags=0. Both the json format, and that I don't get the classifications for the other formats.
Blocks: 523126
Update, I got a local bugzilla install now, and applied bug 523325, which is what's actually giving the json output.
How close are we on this?
Assignee: mozillamarcia.knous → nobody
Component: Bugzilla: Keywords & Components → Administration
Product: mozilla.org → bugzilla.mozilla.org
This bug doesn't seem to have anything to do with bcp47, removing that dep.
No longer blocks: bcp47
QA Contact: timeless → administation
(In reply to comment #11)
> This bug doesn't seem to have anything to do with bcp47, removing that dep.

My hope for BCP 47 implementation extends across the entirety of Mozilla, not just Gecko. That includes the l10n aspects, which this bug would serve to improve. Ergo, this blocks the BCP 47 tracking bug.

But that discussion is somewhat out of the scope of this bug, and I'm more interested in getting this bug fixed than I am about arguing about whether it fits into the BCP 47 plan.

(In reply to comment #10)
> How close are we on this?

This question is still pending after 10 months.
I'm reading over the old discussion on the newsgroups, and one concern that had come up multiple times was what to do with the QA contacts and how to have people track the various components. (Not that the current L10n QA contacts are very useful, but that point is moot.)

It should be noted that BMO now has a separate "Component Watching" extension that bypasses the QA contacts and allows you to directly watch any number of components. It allow allows you to easily watch all the components within a product. And it has a nice UI, so you no longer have to go tracking down QA contact addresses in disparate locations.

To access the Component Watching extension, just visit your preferences and click on the appropriate tab.

So, I don't know if that had anything to do with what was holding up this bug, but at least it's one less thing to worry about.
Ok taking this on. Seems like we want to do this in two phases:

Phase 1. 

Create the new structure as outlined in http://groups.google.com/group/mozilla.governance/msg/011a024c7fb39360 in the current BMO and allow people to start using it for new issues.

Classification: Mozilla Localizations
Product Name Format: i10n: <Locale Name>
Components for each Locale: General, Firefox, Thunderbird, etc.

The assignee would be nobody@mozilla.org and the qa contacts would follow the standard qa contact addressing convention currently used. 

We can work on implementing optgroups for the product drop down on show_bug.cgi which should clean up the list a little and make it less confusing to the user. I would still want to leave the i10n: prefix as product names are not unique to classifications and so no two product names can be the same.

Phase 2.

The Bugzilla developers use some of our current migration scripts to start moving over the current localization issues from their previous location to the proper new one. We can do a couple initially to see how it works out and then do the rest at one time once it looks good.

I do not think the initial cc values need to be populated due to the usefulness of the new Component Watching extension currently being used by BMO. If a user wants to watch all localization bugs they can simply watch the products of their choice.

Are we good to go ahead then with the plan laid out in the forum?

We can of course test all of this on our internal test instances of BMO before doing this live so everyone can see how it will look beforehand.

dkl
Assignee: nobody → dkl
Status: NEW → ASSIGNED
Thanks for picking this up. A few comments, it's been a while.

The prefix should the l10n, 'ell-ten-enn', no 'i', and include the locale code as well as name.

I'd prefer to not create bug components for products that aren't available in that locale, mostly because that's making things more confusing in particular for new locales.

I'd like to have specific bugzilla aliases for each locale/component combination. The combo aliases serve a purpose beyond the QA alias, as you can CC them on bugs in core Firefox/Thunderbird etc, when input from particular locales would be valuable.

We should use localized component descriptions in addition to the generic ones, there are existing ones at https://localize.mozilla.org/projects/bugzilla_components/. (You didn't wait for those for csb and sw, we'll need to update those when we get them.)

There should not be "General", but "Other".

I'm not sure how much component watching changes my thinking, we didn't have that yet when we discussed that. Technically, I'm leaning towards defaultCC, no idea how to set that up in a scalable fashion.

I started working on an add-on to help with filling out the forms and all, but that predates both you and the recent bugzilla updates, so I don't know how much value that still brings. I could tar that up, though.
fwiw, imo, defaultcc isn't a great idea. if a bug changes components, it's likely to grow the combination of defaultcc's. also, when someone stops caring about a component, they're still cc'd to all bugs created while they were a defaultcc.

a better approach is periodic whines for 'unassigned' bugs, you could control this yourself - and you wouldn't even need to use bugzilla's whine feature (altough you could), just send people periodic status update checks with prebuilt queries so they can learn to manage bugzilla.
(In reply to comment #16)
> fwiw, imo, defaultcc isn't a great idea. if a bug changes components, it's
> likely to grow the combination of defaultcc's. also, when someone stops
> caring about a component, they're still cc'd to all bugs created while they
> were a defaultcc.

In the scenarios where I can see bugs moving out of the Localizations product, keeping the local team on the bug sounds like a feature rather than a bug. Also, if the localizer stops caring, nobody's going to notice if they just drop watching the component behind the scenes. Not there's a lot of action in practice on bugs in forgotten locales.

This is really about the default on how to hook the initial contributors onto their newly created bugzilla components, I think. Ask them to watch QA aliases, ask them to watch components, or add them as default CC. Both "ask" here could also be things we do admin-wise? Anyway, for l10n-drivers, seeing people on CC is a much better measure.

For new volunteers coming in to existing l10n teams, I don't mind which tool they use to keep up with bugzilla, really.

We're not talking about a whole ton of bugs either, like, if someone wants to un-CC themselves from the l10n bugs, that's not a 1k bug mail effort.

> a better approach is periodic whines for 'unassigned' bugs, you could
> control this yourself - and you wouldn't even need to use bugzilla's whine
> feature (altough you could), just send people periodic status update checks
> with prebuilt queries so they can learn to manage bugzilla.

whining is of no use for us, we need participation to be driven from the outside.
Depends on: 539894
I have created a Google Docs spreadsheet that we can use to populate the information for each of the new components to be created. Please add to that going forward and when we are satisfied with the list, I can use it to create a script to create the new components.

https://spreadsheets.google.com/spreadsheet/ccc?key=0AnGvkTdJLZN_dFpFOVNtazVGUFBibWtjZWh1eW5WNXc&hl=en_US

(In reply to comment #15)
> Thanks for picking this up. A few comments, it's been a while.
> 
> The prefix should the l10n, 'ell-ten-enn', no 'i', and include the locale
> code as well as name.

Thanks. That was a typo on my part. 

> I'd prefer to not create bug components for products that aren't available
> in that locale, mostly because that's making things more confusing in
> particular for new locales.

Understood. I will let others determine which components will be created for which product.
 
> I'd like to have specific bugzilla aliases for each locale/component
> combination. The combo aliases serve a purpose beyond the QA alias, as you
> can CC them on bugs in core Firefox/Thunderbird etc, when input from
> particular locales would be valuable.

Currently the naming scheme as I understand it would be like:

other@af-localization.bugs
firefox@af-localization.bugs
thunderbird@af-localization.bugs
...
other@ar-localization.bugs
...

Then for initial cc members we can create other broader aliases such as:

af@localization.bugs
thunderbird@localization.bugs
...

> We should use localized component descriptions in addition to the generic
> ones, there are existing ones at
> https://localize.mozilla.org/projects/bugzilla_components/. (You didn't wait
> for those for csb and sw, we'll need to update those when we get them.)

Once all of the descriptions have been localized, we can add the text to the above spreadsheet before we create the new components.

> There should not be "General", but "Other".

Noted

> I started working on an add-on to help with filling out the forms and all,
> but that predates both you and the recent bugzilla updates, so I don't know
> how much value that still brings. I could tar that up, though.

I would be interested in seeing that. Although we should probably just fix the XMLRPC API to allow adding/editing of components and then we can use a simple script to parse the spreadsheet.

dkl
(In reply to comment #18)
> > I'd like to have specific bugzilla aliases for each locale/component
> > combination. The combo aliases serve a purpose beyond the QA alias, as you
> > can CC them on bugs in core Firefox/Thunderbird etc, when input from
> > particular locales would be valuable.
> 
> Currently the naming scheme as I understand it would be like:
> 
> other@af-localization.bugs
> firefox@af-localization.bugs
> thunderbird@af-localization.bugs
> ...
> other@ar-localization.bugs
> ...
> 
> Then for initial cc members we can create other broader aliases such as:
> 
> af@localization.bugs
> thunderbird@localization.bugs
> ...
> 

Not to be nitpicky, but those should probably "subdomains" rather than hyphenated "domains":

other@af.localization.bugs
firefox@af.localization.bugs
thunderbird@af.localization.bugs

Because there will also (at some point, if not now) be, for example:

firefox@en-GB.localization.bugs
thunderbird@sr-Latn.localization.bugs

etc.
(In reply to comment #19)
> Not to be nitpicky, but those should probably "subdomains" rather than
> hyphenated "domains":
> 
> other@af.localization.bugs
> firefox@af.localization.bugs
> thunderbird@af.localization.bugs

Thanks. Will fix.

dkl
I realize that Dave joined after the wider discussion, so there's a bit of loss, but we should pick things up where we left them in the thread in .governance.

Sorry, but we're not going to use a google spreadsheet, but the infrastructure that our l10n teams are used to, which is the existing project on localize.mozilla.org.

As for the aliases, http://groups.google.com/group/mozilla.governance/msg/011a024c7fb39360 mentions (gotta go to nntp for that):

Each of those components has a QA contact like "firefox-af@localization.bugs" and a default-CC of "af@localization.bugs, firefox-all@localization.bugs", default assignee is nobody@mozilla.org.
(In reply to comment #21)
> I realize that Dave joined after the wider discussion, so there's a bit of
> loss, but we should pick things up where we left them in the thread in
> .governance.

I will try to get up to speed quickly and was admittedly alot to sift back through after the fact. Some of it got lost in translation when I was working up a plan to move forward.

> Sorry, but we're not going to use a google spreadsheet, but the
> infrastructure that our l10n teams are used to, which is the existing
> project on localize.mozilla.org.

Thats fine. I originally created the spreadsheet as I could use it to feed to a script to create the things we needed. I didn't see in the Localization tool an easy way to determine exactly which components to create for each locale (except for Other of course). Is there a way we can use your tool and get that information in there as well?

> As for the aliases,
> http://groups.google.com/group/mozilla.governance/msg/011a024c7fb39360
> mentions (gotta go to nntp for that):
> 
> Each of those components has a QA contact like
> "firefox-af@localization.bugs" and a default-CC of "af@localization.bugs,
> firefox-all@localization.bugs", default assignee is nobody@mozilla.org.

Works for me. 

Thanks
dkl
(In reply to comment #21)
> I realize that Dave joined after the wider discussion, so there's a bit of
> loss, but we should pick things up where we left them in the thread in
> .governance.

Given the length of time that has elapsed since that discussion, perhaps it's time to take a look at the issue anew, from the point of view of the current state of Bugzilla and the implementation plans of BCP 47 (though you believe that it doesn't apply here).

> Sorry, but we're not going to use a google spreadsheet, but the
> infrastructure that our l10n teams are used to, which is the existing
> project on localize.mozilla.org.

I'm afraid I don't see the connection between using a Google spreadsheet and localize.mozilla.org. What do the l10n teams have to do with the spreadsheet?

> As for the aliases,
> http://groups.google.com/group/mozilla.governance/msg/011a024c7fb39360
> mentions (gotta go to nntp for that):
> 
> Each of those components has a QA contact like
> "firefox-af@localization.bugs" and a default-CC of "af@localization.bugs,
> firefox-all@localization.bugs", default assignee is nobody@mozilla.org.

That's not a very good idea: 'all' is the code for the Allar language.

What's wrong with the combo of what David and I suggested?
After discussing some more with other Bugzilla developer, I do not see why the Component Watching feature of BMO would not be a better choice than maintaining all of the additional default CC addresses you are proposing. I understand the comp watch feature was not around when this was first being discussed, but is there any other reason why using it would not be a better choice? As Bugzilla admins we are responsible for creating and maintaining all of these new accounts when a user can simply select a whole product, or any combination of components to receive notifications for. With the default CCs being suggested the users would still need to add the right ones to their watch list so no different in the end.

dkl
(In reply to comment #23)
> I'm afraid I don't see the connection between using a Google spreadsheet and
> localize.mozilla.org. What do the l10n teams have to do with the spreadsheet?

As I mentioned before, I suggested using the spreadsheet as it gives me an easy way to export that data to a text file that I can use to mass create the new products/components/users using scripts for Bugzilla.

Is there a way to get something similar out of localize.mozilla.org that I can use to start working on the necessary scripts to get this going?

Dkl
(In reply to comment #25)
> (In reply to comment #23)
> > I'm afraid I don't see the connection between using a Google spreadsheet and
> > localize.mozilla.org. What do the l10n teams have to do with the spreadsheet?
> 
> As I mentioned before, I suggested using the spreadsheet as it gives me an
> easy way to export that data to a text file that I can use to mass create
> the new products/components/users using scripts for Bugzilla.
> 
> Is there a way to get something similar out of localize.mozilla.org that I
> can use to start working on the necessary scripts to get this going?
> 
> Dkl

Oh, no, I understand why you want to use the spreadsheet. What I don't understand is why Axel doesn't want to.
David, as much as I value your productivity, the key point here is to make things scalable and productive for those working per locale.

That is, for you and bugzilla admins, we can hack on tools or workarounds or cumbersome things, but for the people in the field, things need to be smooth.

That means, we'll use the smoothest tool we have to get localized component description parts, and then tool our way to get those out of that tool for the bugzilla admins.

Also, we need to serve those people that want to 

- follow/contribute to a particular project in a particular language
- f/c to a particular project in all languages
- f/c to all projects in a particular language

The first of the three is easy, as it's static. The latter two need an entry point that allows people to get what they want if the list of projects or languages change, without needing to jump through awkward loops or miss things just because nobody told them there was a new product/component around.

I'm happy for suggestions here, but we shouldn't just push that challenge away.

I'll attach a tarball of that addon I hacked on, which includes gettext parsers in js and other things. I can't really speak for it's functionality at this point, don't have the cycles to see what worked and didn't at the time, and how that blends with what bugzilla offers today.
This is what I have on disk from my interactions with landfill. It did something pretty close to what we wanted when the discussion was live. Most of it was interacting with forms and iframes or something, to get the tokens for CSRF, I guess. It implements two different schemes, and probably neither is exactly what we'd want today.
(In reply to comment #27)
> Also, we need to serve those people that want to 
> 
> - follow/contribute to a particular project in a particular language
> - f/c to a particular project in all languages
> - f/c to all projects in a particular language

Assuming this structure:

Mozilla In Your Language
- en-GB
-- Firefox
-- Thunderbird
-- etc.
- etc.

Component following allows you to follow (for example) only en-GB > Firefox, or all of en-GB > *.

That directly takes care of points 1 and 3 above.

So the only concern (which also, IMO, is the least likely) is point 2: someone who wants to do l10n in all languages for a singular particular project. For that person, there is no reason they can't track * > Firefox that currently exist, as well as the component where new l10n registrations are made, if they're that worried about missing the addition of a new locale.

What am I missing?
We'll have "Firefox", "mozilla.com", "Other" to begin with for a new localization. Now someone's coming up for thunderbird, and another person's coming up for mobile, and we add the two.

Everyone that used to follow all components needs to start from scratch.

Also, mozilla starts a new project, say, a new something on iOS. Everyone watching any localization that participates needs to edit their settings to get what they want.

Also, using an existing example, I can't see a setting to watch "* > Build Config", so neither scheme is covered. Following a particular component for all locales is in particular interesting to the website folks, which follow a slightly different process to track things as we do for product.
(In reply to comment #30)
> We'll have "Firefox", "mozilla.com", "Other" to begin with for a new
> localization. Now someone's coming up for thunderbird, and another person's
> coming up for mobile, and we add the two.
> 
> Everyone that used to follow all components needs to start from scratch.

It is my understanding that the Bugzilla folks will be migrating everyone from QA following to Component Watching, so nobody needs to start from scratch.

> Also, mozilla starts a new project, say, a new something on iOS. Everyone
> watching any localization that participates needs to edit their settings to
> get what they want.

Anyone watching en-GB > * will automatically start getting news about the new en-GB > iOS component, without having to edit anything.

> Also, using an existing example, I can't see a setting to watch "* > Build
> Config", so neither scheme is covered. Following a particular component for
> all locales is in particular interesting to the website folks, which follow
> a slightly different process to track things as we do for product.

That would be the rare case, as I mentioned before. Is that something that is truly locale-specific, anyway? (This is an actual question—I don't know.)
(In reply to comment #31)
> (In reply to comment #30)
> > Also, using an existing example, I can't see a setting to watch "* > Build
> > Config", so neither scheme is covered. Following a particular component for
> > all locales is in particular interesting to the website folks, which follow
> > a slightly different process to track things as we do for product.
> 
> That would be the rare case, as I mentioned before. Is that something that
> is truly locale-specific, anyway? (This is an actual question—I don't know.)

I think that watching "components across products" is something that could be seen quite frequently, actually. Build Config is one area, where there's a interest group across Firefox, Thunderbird, Toolkit.

In the localization case, we're having people that coordinate localization tasks purely bug-based, concretely, our websites. If someone comes in and wants a fix or new page to be created for a flock of localizations, the coordinators for our website want to see the bugs being created, resolved, and check that against the landings they see. There may also be "bug hungry" individuals or volunteers interested in triage to do the same for products they have expertise in, which might cover Firefox, but not Calendar, to pick two that are quite distinct.
(In reply to comment #27)
> That means, we'll use the smoothest tool we have to get localized component
> description parts, and then tool our way to get those out of that tool for
> the bugzilla admins.

Ok. I noticed that a lot is still left to be translated. So should we wait to do this when the list is fully complete? In the meantime we can work on either 1) getting the addon up to speed with the latest Bugzilla code or 2) work with the localize.mozilla.org developer to find a way to export the data in a way that can be used with Bugzilla import scripts. I have more experience with doing it the number 2 way in the past but could go with 1 if it won't take too much work.

(In reply to comment #29)
> So the only concern (which also, IMO, is the least likely) is point 2:
> someone who wants to do l10n in all languages for a singular particular
> project. For that person, there is no reason they can't track * > Firefox
> that currently exist, as well as the component where new l10n registrations
> are made, if they're that worried about missing the addition of a new locale.

Yes currently the Component Watch feature doesn't have a shortcut for saying * > Firefox so the user would have to go into the UI and manually select all language products and Firefox component for each. If a new language is added they would then need to go back and add that new language product to the list as well.

On the other side, if we used the default CC method, we would just need to add the appropriate aliases to the default CC of any new language but since it would be a one time task, people would start getting notified automatically.

dkl
(In reply to comment #31)
> It is my understanding that the Bugzilla folks will be migrating everyone
> from QA following to Component Watching, so nobody needs to start from
> scratch.

There is no plan currently to do this. The Component Watching feature was just an enhancement that people have been asking for. We do not plan to force people to migrate away from using the qa contact aliases to purely component watching unless creating the qa contact aliases becomes too much of an admin burden. Currently it is not.

> Anyone watching en-GB > * will automatically start getting news about the
> new en-GB > iOS component, without having to edit anything.

That is true. If some one is watching en-GB:__Any__ then any new component they would get notifications for. Unfortunately the same is not true on the product level.

dkl
(In reply to comment #32)
> I think that watching "components across products" is something that could
> be seen quite frequently, actually. Build Config is one area, where there's
> a interest group across Firefox, Thunderbird, Toolkit.

Technically it is possible now, just not automatic. They would need to setup their watch as so:

Firefox:Build Config
Thunderbird:Build Config
Toolkit:Build Config
...

We could look into doing __ANY__:Component as an option but then you stand the risk of getting notified on bugs you may not want if another product is using a component of similar name.

dkl
(In reply to comment #34)
> (In reply to comment #31)
> > It is my understanding that the Bugzilla folks will be migrating everyone
> > from QA following to Component Watching, so nobody needs to start from
> > scratch.
> 
> There is no plan currently to do this. The Component Watching feature was
> just an enhancement that people have been asking for. We do not plan to
> force people to migrate away from using the qa contact aliases to purely
> component watching unless creating the qa contact aliases becomes too much
> of an admin burden. Currently it is not.

I thought somebody in IRC told me that it was. Must have been a misunderstanding.
Closing as request is no longer needed with changes that have happened since the filing.
Status: ASSIGNED → RESOLVED
Closed: 10 years ago
Resolution: --- → INVALID
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: