610708 - cannot add new locales through the tuxedo API because of HTTP 400 error

Status: RESOLVED FIXED

(Webtools :: Bouncer, defect)

Description

[Armen [:armenzg]]

The same as bug 590902.

For beta7 we tried to use tuxedo-add.py but we got:
> HTTPError: HTTP Error 400: BAD REQUEST
because new locales are being added.

The new locales have not been used in any of the previous releases (betas or stable ones).

I believe this is the list of new locales that have to be added (from diffing shipped-locales on 1.9.2 and the one used for beta7):
+ak
+br
+en-ZA
+hy-AM
+lg
+nso
+son

Could you please add them?
Or is this just moving a tag up? (I see the locales in http://viewvc.svn.mozilla.org/vc/libs/product-details/localeDetails.class.php?view=markup)

I am also not sure against which source code I should have compared if these locales are added or not.

Comment 1

[Armen [:armenzg]]

So it seems that we can't ship beta7 without this.
I thought I could add the locales manually trough bounceradmin.

Please let me know if I understand this incorrectly.

Comment 2

[Fred Wenzel [:wenzel]]

I am looking at this now. We can have IT push a stop-gap, by updating the product details that Bouncer uses, and then I should fix the problem permanently by making it auto-update its data.

Comment 3

[Fred Wenzel [:wenzel]]

Armen: Oremj updated the data on tuxedo.staging. Please run your Python script against staging and report any errors that might occur. If everything goes smoothly, please tell us as well, and we'll have this update pushed to production.

Comment 4

[Armen [:armenzg]]

wenzel sorry to take long.
Could you please push to production as-is? I would like to test it there as I am not set-up to test it on staging (lacking the proper tuxedo username/password).

Comment 5

[Fred Wenzel [:wenzel]]

The new data should be deployed on production, please test if it does what you want.

I'll keep this bug open for now, in order to implement a long-term solution that won't break every time new locales are added.

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Unfortunately we're still hitting an error:

Adding product: Firefox-4.0b7
Locales: af, ak, ar, be, bg, bn-BD, br, ca, cs, da, de, el, en-GB, en-US, en-ZA, eo, es-AR, es-ES, et, eu, fi, fr, fy-NL, ga-IE, gd, he, hu, hy-AM, id, is, it, ja, ja-JP-mac, ko, ku, lg, lt, lv, mk, nb-NO, nl, nn-NO, nso, pa-IN, pl, pt-BR, pt-PT, rm, ro, ru, sk, son, sq, sv-SE, tr, uk, zh-CN, zh-TW
FAILED: Tuxedo API error. URL: https://bounceradmin.mozilla.com/api/product_add/
POST data: product=Firefox-4.0b7&languages=af&languages=ak&languages=ar&languages=be&languages=bg&languages=bn-BD&languages=br&languages=ca&languages=cs&languages=da&languages=de&languages=el&languages=en-GB&languages=en-US&languages=en-ZA&languages=eo&languages=es-AR&languages=es-ES&languages=et&languages=eu&languages=fi&languages=fr&languages=fy-NL&languages=ga-IE&languages=gd&languages=he&languages=hu&languages=hy-AM&languages=id&languages=is&languages=it&languages=ja&languages=ja-JP-mac&languages=ko&languages=ku&languages=lg&languages=lt&languages=lv&languages=mk&languages=nb-NO&languages=nl&languages=nn-NO&languages=nso&languages=pa-IN&languages=pl&languages=pt-BR&languages=pt-PT&languages=rm&languages=ro&languages=ru&languages=sk&languages=son&languages=sq&languages=sv-SE&languages=tr&languages=uk&languages=zh-CN&languages=zh-TW

Looking at the language list in 'Add product' ui I don't see lg or son there. so perhaps we didn't pull in http://viewvc.svn.mozilla.org/vc?view=revision&revision=77144 ? I only checked the delta between 4.0b6 and b7, which is adding af, ak, ar, bg, bn-BD br, en-GB, en-ZA, eu, fy-NL, gd, hy-AM, lg, mk, nso, rm, son, sq.

Comment 7

[Fred Wenzel [:wenzel]]

Updating the product details data by hand fixed this. The long-term fix is in bug 590493.

Comment 8

[Armen [:armenzg]]

This has now worked.

The change that wenzel pushed [1] was good but it required an apache restart to pick up the change.
[1] https://github.com/jbalogh/mozilla-product-details/commit/d25b6e983ea9ad145be07cda2dbd3f56c2723a7e

As nthomas mentions, we can look at the api/docs/product_add page we can see the known loaded locales. Once the apache server was restarted the last two locales were refreshed.

wenzel has filed bug 610893 to deal with the issue that when new locales are added to the PHP library we have to generate a new json file.

nthomas do we have to product_add anything for the directory change for mac?

python tuxedo-add.py --config firefox-tuxedo.ini --product firefox --version 4.0b7 --tuxedo-server-url https://bounceradmin.mozilla.com/api/ --shipped-locales shipped-locales --bouncer-product-name Firefox --brand-name Firefox --add-mars --old-version 4.0b6 --milestone 2.0b7 --platform linux --platform linux64 --platform macosx64 --platform win32 --credentials-file BuildSlaves.py
 in dir /builds/slave/bouncer_submitter/tools/release (timeout 1200 secs)
 watching logfiles {}
 argv: ['python', 'tuxedo-add.py', '--config', 'firefox-tuxedo.ini', '--product', 'firefox', '--version', '4.0b7', '--tuxedo-server-url', 'https://bounceradmin.mozilla.com/api/', '--shipped-locales', 'shipped-locales', '--bouncer-product-name', 'Firefox', '--brand-name', 'Firefox', '--add-mars', '--old-version', '4.0b6', '--milestone', '2.0b7', '--platform', 'linux', '--platform', 'linux64', '--platform', 'macosx64', '--platform', 'win32', '--credentials-file', 'BuildSlaves.py']
 environment:
  CC=/tools/gcc/bin/gcc
  CVS_RSH=ssh
  CXX=/tools/gcc/bin/g++
  G_BROKEN_FILENAMES=1
  HISTSIZE=1000
  HOME=/home/cltbld
  HOSTNAME=mv-moz2-linux-ix-slave03.build.mozilla.org
  INPUTRC=/etc/inputrc
  JAVA_HOME=/builds/jdk
  LANG=en_US.UTF-8
  LESSOPEN=|/usr/bin/lesspipe.sh %s
  LOGNAME=cltbld
  LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
  MAIL=/var/spool/mail/cltbld
  PATH=/opt/local/bin:/tools/python/bin:/tools/buildbot/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/home/cltbld/bin
  PWD=/builds/slave/bouncer_submitter/tools/release
  PYTHONPATH=../lib/python:
  SHELL=/bin/bash
  SHLVL=1
  SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
  TBOX_CLIENT_CVS_DIR=/builds/tinderbox/mozilla/tools
  TERM=linux
  USER=cltbld
  _=/tools/buildbot/bin/buildbot
 closing stdin
 using PTY: True
Adding product: Firefox-4.0b7
Locales: af, ak, ar, be, bg, bn-BD, br, ca, cs, da, de, el, en-GB, en-US, en-ZA, eo, es-AR, es-ES, et, eu, fi, fr, fy-NL, ga-IE, gd, he, hu, hy-AM, id, is, it, ja, ja-JP-mac, ko, ku, lg, lt, lv, mk, nb-NO, nl, nn-NO, nso, pa-IN, pl, pt-BR, pt-PT, rm, ro, ru, sk, son, sq, sv-SE, tr, uk, zh-CN, zh-TW
Server response:
<?xml version="1.0" encoding="utf-8"?><products><product id="910" name="Firefox-4.0b7"><language locale="af"/><language locale="ak"/><language locale="ar"/><language locale="be"/><language locale="bg"/><language locale="bn-BD"/><language locale="br"/><language locale="ca"/><language locale="cs"/><language locale="da"/><language locale="de"/><language locale="el"/><language locale="en-GB"/><language locale="en-US"/><language locale="en-ZA"/><language locale="eo"/><language locale="es-AR"/><language locale="es-ES"/><language locale="et"/><language locale="eu"/><language locale="fi"/><language locale="fr"/><language locale="fy-NL"/><language locale="ga-IE"/><language locale="gd"/><language locale="he"/><language locale="hu"/><language locale="hy-AM"/><language locale="id"/><language locale="is"/><language locale="it"/><language locale="ja"/><language locale="ja-JP-mac"/><language locale="ko"/><language locale="ku"/><language locale="lg"/><language locale="lt"/><language locale="lv"/><language locale="mk"/><language locale="nb-NO"/><language locale="nl"/><language locale="nn-NO"/><language locale="nso"/><language locale="pa-IN"/><language locale="pl"/><language locale="pt-BR"/><language locale="pt-PT"/><language locale="rm"/><language locale="ro"/><language locale="ru"/><language locale="sk"/><language locale="son"/><language locale="sq"/><language locale="sv-SE"/><language locale="tr"/><language locale="uk"/><language locale="zh-CN"/><language locale="zh-TW"/></product></products>
Adding product: Firefox-4.0b7-Complete
Locales: af, ak, ar, be, bg, bn-BD, br, ca, cs, da, de, el, en-GB, en-US, en-ZA, eo, es-AR, es-ES, et, eu, fi, fr, fy-NL, ga-IE, gd, he, hu, hy-AM, id, is, it, ja, ja-JP-mac, ko, ku, lg, lt, lv, mk, nb-NO, nl, nn-NO, nso, pa-IN, pl, pt-BR, pt-PT, rm, ro, ru, sk, son, sq, sv-SE, tr, uk, zh-CN, zh-TW
Server response:
<?xml version="1.0" encoding="utf-8"?><products><product id="911" name="Firefox-4.0b7-Complete"><language locale="af"/><language locale="ak"/><language locale="ar"/><language locale="be"/><language locale="bg"/><language locale="bn-BD"/><language locale="br"/><language locale="ca"/><language locale="cs"/><language locale="da"/><language locale="de"/><language locale="el"/><language locale="en-GB"/><language locale="en-US"/><language locale="en-ZA"/><language locale="eo"/><language locale="es-AR"/><language locale="es-ES"/><language locale="et"/><language locale="eu"/><language locale="fi"/><language locale="fr"/><language locale="fy-NL"/><language locale="ga-IE"/><language locale="gd"/><language locale="he"/><language locale="hu"/><language locale="hy-AM"/><language locale="id"/><language locale="is"/><language locale="it"/><language locale="ja"/><language locale="ja-JP-mac"/><language locale="ko"/><language locale="ku"/><language locale="lg"/><language locale="lt"/><language locale="lv"/><language locale="mk"/><language locale="nb-NO"/><language locale="nl"/><language locale="nn-NO"/><language locale="nso"/><language locale="pa-IN"/><language locale="pl"/><language locale="pt-BR"/><language locale="pt-PT"/><language locale="rm"/><language locale="ro"/><language locale="ru"/><language locale="sk"/><language locale="son"/><language locale="sq"/><language locale="sv-SE"/><language locale="tr"/><language locale="uk"/><language locale="zh-CN"/><language locale="zh-TW"/></product></products>
Adding product: Firefox-4.0b7-Partial-4.0b6
Locales: af, ak, ar, be, bg, bn-BD, br, ca, cs, da, de, el, en-GB, en-US, en-ZA, eo, es-AR, es-ES, et, eu, fi, fr, fy-NL, ga-IE, gd, he, hu, hy-AM, id, is, it, ja, ja-JP-mac, ko, ku, lg, lt, lv, mk, nb-NO, nl, nn-NO, nso, pa-IN, pl, pt-BR, pt-PT, rm, ro, ru, sk, son, sq, sv-SE, tr, uk, zh-CN, zh-TW
Server response:
<?xml version="1.0" encoding="utf-8"?><products><product id="912" name="Firefox-4.0b7-Partial-4.0b6"><language locale="af"/><language locale="ak"/><language locale="ar"/><language locale="be"/><language locale="bg"/><language locale="bn-BD"/><language locale="br"/><language locale="ca"/><language locale="cs"/><language locale="da"/><language locale="de"/><language locale="el"/><language locale="en-GB"/><language locale="en-US"/><language locale="en-ZA"/><language locale="eo"/><language locale="es-AR"/><language locale="es-ES"/><language locale="et"/><language locale="eu"/><language locale="fi"/><language locale="fr"/><language locale="fy-NL"/><language locale="ga-IE"/><language locale="gd"/><language locale="he"/><language locale="hu"/><language locale="hy-AM"/><language locale="id"/><language locale="is"/><language locale="it"/><language locale="ja"/><language locale="ja-JP-mac"/><language locale="ko"/><language locale="ku"/><language locale="lg"/><language locale="lt"/><language locale="lv"/><language locale="mk"/><language locale="nb-NO"/><language locale="nl"/><language locale="nn-NO"/><language locale="nso"/><language locale="pa-IN"/><language locale="pl"/><language locale="pt-BR"/><language locale="pt-PT"/><language locale="rm"/><language locale="ro"/><language locale="ru"/><language locale="sk"/><language locale="son"/><language locale="sq"/><language locale="sv-SE"/><language locale="tr"/><language locale="uk"/><language locale="zh-CN"/><language locale="zh-TW"/></product></products>
Adding location for Firefox-4.0b7, linux: /firefox/releases/4.0b7/linux-i686/:lang/firefox-4.0b7.tar.bz2
Adding location for Firefox-4.0b7-Complete, linux: /firefox/releases/4.0b7/update/linux-i686/:lang/firefox-4.0b7.complete.mar
Adding location for Firefox-4.0b7-Partial-4.0b6, linux: /firefox/releases/4.0b7/update/linux-i686/:lang/firefox-4.0b6-4.0b7.partial.mar
Adding location for Firefox-4.0b7, linux64: /firefox/releases/4.0b7/linux-x86_64/:lang/firefox-4.0b7.tar.bz2
Adding location for Firefox-4.0b7-Complete, linux64: /firefox/releases/4.0b7/update/linux-x86_64/:lang/firefox-4.0b7.complete.mar
Adding location for Firefox-4.0b7-Partial-4.0b6, linux64: /firefox/releases/4.0b7/update/linux-x86_64/:lang/firefox-4.0b6-4.0b7.partial.mar
Adding location for Firefox-4.0b7, osx64: /firefox/releases/4.0b7/mac64/:lang/Firefox%204.0%20Beta%207.dmg
Adding location for Firefox-4.0b7-Complete, osx64: /firefox/releases/4.0b7/update/mac64/:lang/firefox-4.0b7.complete.mar
Adding location for Firefox-4.0b7-Partial-4.0b6, osx64: /firefox/releases/4.0b7/update/mac64/:lang/firefox-4.0b6-4.0b7.partial.mar
Adding location for Firefox-4.0b7, win: /firefox/releases/4.0b7/win32/:lang/Firefox%20Setup%204.0%20Beta%207.exe
Adding location for Firefox-4.0b7-Complete, win: /firefox/releases/4.0b7/update/win32/:lang/firefox-4.0b7.complete.mar
Adding location for Firefox-4.0b7-Partial-4.0b6, win: /firefox/releases/4.0b7/update/win32/:lang/firefox-4.0b6-4.0b7.partial.mar
program finished with exit code 0
elapsedTime=4.535444