800432 - Email Axel & release-mgmt about l10n issues with builds on mozilla-aurora

Status: RESOLVED WONTFIX

(Release Engineering :: General, defect)

Description

[Alex Keybl [:akeybl]]

Axel points out that if he was able to see emails about re-pack failures for l10n on mozilla-aurora, we could have avoided this delay. Would it be possible to add him to those failure emails? This is followup from bug 800329.

Comment 1

[Alex Keybl [:akeybl]]

The other point made was whether anybody in RelEng saw bug 800329 in build emailss while 17 was on mozilla-aurora, or whether anybody is currently checking those emails.

Comment 2

[Chris AtLee [:catlee]]

We have no such emails.

RelEng has gone to pretty big lengths to keep l10n data going to Tinderbox - http://tinderbox.mozilla.org/showbuilds.cgi?tree=Mozilla-l10n. I see aurora l10n jobs on there, and busted win32 from a few days ago: http://tinderbox.mozilla.org/showlog.cgi?log=Mozilla-l10n/1349820563.1349820832.1940.gz&fulltext=1.

l10n jobs are the only things we have left going to tinderbox. Are you saying that nobody is monitoring it for bustage?

Comment 3

[Axel Hecht [:Pike]]

I was saying that nobody is monitoring the build outcome in the meeting with relman that we just had.

Alex thinks that we need to email, and web reporting would not be enough.

Also, looking at my proof-of-concept at http://l10n.mozilla-community.org/~axel/nightlies/, I suspect that we're seeing a bunch of infrastructure-related failures and no-reports these days.

My point was, aurora fails, so someone could have taken action earlier. There's no "someone" at this point, and I believe it must not be a single person.

Comment 4

[Chris AtLee [:catlee]]

We'd like to have this for next merge day in November, but if not, then for the following merge day in January.

Comment 5

[Rail Aliiev [:rail]]

Attachment: report_l10n

I think we can exploit l10n_report.py and cron to send http://builddata.pub.build.mozilla.org/buildjson/l10n_reports/latest.txt filtered by a branch.

something like

python l10n_report.py  -b mozilla-aurora ./builds-xxxxxxx.js.gz | mail -E -s "[mozilla-aurora l10n] build failures for `date`" [e-mail]

Comment 6

[Chris AtLee [:catlee]]

Comment on attachment 804640 [details] [diff] [review]
report_l10n

Review of attachment 804640 [details] [diff] [review]:
-----------------------------------------------------------------

::: buildapi/scripts/l10n_report.py
@@ -49,5 @@
>              output.write("    %s\n" % vals['log_url'])
> -            if 'packageUrl' not in vals:
> -                output.write("    no package url\n")
> -            else:
> -                output.write("    %s\n" % vals['packageUrl'])

why did you remove this block?

Comment 7

[Rail Aliiev [:rail]]

IIRC it's not set for l10n repacks or at least for the failed ones.

Comment 8

[Rail Aliiev [:rail]]

We unified reporting of en-US builds and l10n repacks. They both report to treeherder.