590902 - cannot add new locales through the tuxedo API because of HTTP 400 error

Status: RESOLVED FIXED

(Webtools :: Bouncer, defect)

Description

[[e-mail] (:bhearsum)]

Starting with Firefox 3.6.9 we've been unable to add Bouncer entries through the Tuxedo API using tuxedo-add.py because of the following error:
Adding product: Firefox-3.6.9
Locales: af, ar, as, ast, be, bg, bn-BD, bn-IN, ca, cs, cy, da, de, el, en-GB, en-US, eo, es-AR, es-CL, es-ES, es-MX, et, eu, fa, fi, fr, fy-NL, ga-IE, gd, gl, gu-IN, he, hi-IN, hr, hu, id, is, it, ja, ja-JP-mac, ka, kk, kn, ko, ku, lt, lv, mk, ml, mr, nb-NO, nl, nn-NO, oc, or, pa-IN, pl, pt-BR, pt-PT, rm, ro, ru, si, sk, sl, sq, sr, sv-SE, ta, ta-LK, te, th, tr, uk, vi, zh-CN, zh-TW
FAILED: Tuxedo API error. URL: https://bounceradmin.mozilla.com/api/product_add/
POST data: product=Firefox-3.6.9&languages=af&languages=ar&languages=as&languages=ast&languages=be&languages=bg&languages=bn-BD&languages=bn-IN&languages=ca&languages=cs&languages=cy&languages=da&languages=de&languages=el&languages=en-GB&languages=en-US&languages=eo&languages=es-AR&languages=es-CL&languages=es-ES&languages=es-MX&languages=et&languages=eu&languages=fa&languages=fi&languages=fr&languages=fy-NL&languages=ga-IE&languages=gd&languages=gl&languages=gu-IN&languages=he&languages=hi-IN&languages=hr&languages=hu&languages=id&languages=is&languages=it&languages=ja&languages=ja-JP-mac&languages=ka&languages=kk&languages=kn&languages=ko&languages=ku&languages=lt&languages=lv&languages=mk&languages=ml&languages=mr&languages=nb-NO&languages=nl&languages=nn-NO&languages=oc&languages=or&languages=pa-IN&languages=pl&languages=pt-BR&languages=pt-PT&languages=rm&languages=ro&languages=ru&languages=si&languages=sk&languages=sl&languages=sq&languages=sr&languages=sv-SE&languages=ta&languages=ta-LK&languages=te&languages=th&languages=tr&languages=uk&languages=vi&languages=zh-CN&languages=zh-TW
Traceback (most recent call last):
  File "tuxedo-add.py", line 143, in tuxedoRequest
    return urllib2.urlopen(request).read()
  File "/tools/python-2.5.1/lib/python2.5/urllib2.py", line 121, in urlopen
    return _opener.open(url, data)
  File "/tools/python-2.5.1/lib/python2.5/urllib2.py", line 380, in open
    response = meth(req, response)
  File "/tools/python-2.5.1/lib/python2.5/urllib2.py", line 491, in http_response
    'http', request, response, code, msg, hdrs)
  File "/tools/python-2.5.1/lib/python2.5/urllib2.py", line 418, in error
    return self._call_chain(*args)
  File "/tools/python-2.5.1/lib/python2.5/urllib2.py", line 353, in _call_chain
    result = func(*args)
  File "/tools/python-2.5.1/lib/python2.5/urllib2.py", line 499, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 400: BAD REQUEST


It was originally suspected that the two new languages add ('ast' and 'gd') were the issue, but the locales that Bouncer knows about were updated in bug 590496 and the issue is still not resolved.

The client scripts that we use have not changed between the last successful addition and this one.

Comment 1

[Fred Wenzel [:wenzel]]

Please consult with the product details list to see our currently recognized locales:

http://viewvc.svn.mozilla.org/vc/libs/product-details/localeDetails.class.php?view=markup

"ast" is not one of them, however, "ast-ES" is. So either we need to build our clients with "ast-ES" or change our websites to use "ast" instead of "ast-ES". Please consult with L10n on what to do here (I was not the one to add that locale to the list, so I don't know where else it is used).

"gd" isn't in the list in the first place, so no Mozilla website has the opportunity to recognize it :(

I did not check the other locales now, btw.

Comment 2

[[e-mail] (:bhearsum)]

I thought we added those the other day in bug 518652? All of the locales listed in this file are valid, separate from one another, and should be in Bouncer:
http://hg.mozilla.org/releases/mozilla-1.9.2/file/FIREFOX_3_6_9_RELEASE/browser/locales/shipped-locales

Note, 'ast' and 'ast-ES' are separate locales AFAIK.

Comment 3

[Axel Hecht [:Pike]]

ast-ES is OOOOOLD, and not worthy. Use 'ast'.

Comment 4

[Fred Wenzel [:wenzel]]

Hm, who maintain the language-details library except "emergency" changes by webdev whenever bugs like this prop up? You guys should add the languages when in the process of getting a new Firefox localization up to speed -- much like the product details library is updated when a new *version* is prepared.

That being said, I can change ast-ES to ast in the library, and I can also add "gd" if anyone gets me an English as well as a native-language name for that locale.

Comment 5

[[e-mail] (:bhearsum)]

(In reply to comment #4)
> Hm, who maintain the language-details library except "emergency" changes by
> webdev whenever bugs like this prop up?

Axel?

Comment 6

[Axel Hecht [:Pike]]

We shouldn't block on problems like this, please file bugs to make whatever fails here fail gracefully?

As a bustage fix, gd is Scottish Gaelic, native name is "Gàidhlig".

If you can give me the correct way to file a bug on this, I can add it to the https://wiki.mozilla.org/L10n:Bugogram

Comment 7

[[e-mail] (:bhearsum)]

(In reply to comment #6)
> We shouldn't block on problems like this, please file bugs to make whatever
> fails here fail gracefully?

That still doesn't answer the question of "who maintain the language-details library".

We've already got a bug on file to make Bouncer languages addition easier. I'm not sure there's really not much we can do on the automation side.

Comment 8

[Axel Hecht [:Pike]]

(In reply to comment #7)
> That still doesn't answer the question of "who maintain the language-details
> library".

I wouldn't know. Maybe pascalc does?

Comment 9

[Pascal Chevrel:pascalc]

Since I am its main consumer now, I do maintain product-details indeed, what changes do you need?

Comment 10

[[e-mail] (:bhearsum)]

(In reply to comment #9)
> Since I am its main consumer now, I do maintain product-details indeed, what
> changes do you need?

RelEng recently automated adding entries to Bouncer through the new Tuxedo API. During 3.6.9 (which added 2 new locales) we discovered that this automation fails when Bouncer is unaware of any locales we try to send.

So, the acute issue is that we are unable to add these entries for 3.6.9 because 'ast' and 'gd' are missing from Bouncer.

The chronic issue is figuring out what we do when we add locales in the future.

Comment 11

[Pascal Chevrel:pascalc]

ok, r73326 and r73327 adds gd and updates the locale codes for ast-ES (-> ast) and br-FR (-> br).

I didn't know you were also a consumer of this library but that's probably because we haven't added locales for a while. The best thing to do is to file a bug and assign it to me when you need localeDetails to be updated, I'll try to be proactive on updating this file and update it before we actually need that on mozilla.com.

Comment 12

[[e-mail] (:bhearsum)]

Thanks a ton, Pascal! Do we need to wait for the next push to production for this to be active?

Out of curiosity, who normally files bugs to get this updated?

Comment 13

[Pascal Chevrel:pascalc]

Usually Pike and Beltzner file the bugs to get our product-details library updated with new locales for a release, I would suggest asking them to remind me that localeDetails is to be checked as well in the bug, it doesn't hurt :)

You don't need to wait for a push to production for this one since this library doesn't work with a production tag, websites using the library usually include a specific svn revision of the library which corresponds to a specific point in our release timeline. For example mozilla.com currently includes r72843 while the latest change is 73327 now.

I guess that you just include the latest version in your application so it should work for you as soon as your application picks up the changes.

Comment 14

[[e-mail] (:bhearsum)]

Ahh...I think we're on different pages. The RelEng side of things is a client application to the service running at https://bounceradmin.mozilla.com, which Frederic Wenzel recently worked on. Based on his attempt to fix this last week (bug 590496), it seems like we do need to wait on the next production push.

Comment 15

[Fred Wenzel [:wenzel]]

Thanks for picking this up Pascal. I wouldn't point the responsibility for that library that far away from me, but while I am probably one of the main devs to consume locale data on the webdev side, I am not in the loop on our client locales, so I am the wrong person to maintain it content-wise.

Ben: I'll update the locales list again and file a new IT bug to get that updated. The long-term fix to avoid this bug-filing frenzy is bug 590493.

Comment 16

[[e-mail] (:bhearsum)]

Managed to add 3.6.9 entries after the latest push. Thanks for all your help with this matter Pascal & Frederic.