959747 - mozilla-aurora android nightly l10n repacks need fixed branch, revision to show up on tbpl

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[Aki Sasaki (not active)]

I'm not entirely sure where the issue is... is it not getting put in builds-4hr.json ?  Ignored?

This makes it harder to debug and fix things like bug 923950. Even if the jobs are hidden by default, being able to see the status and retrigger is helpful.

Comment 1

[Ed Morley [:emorley]]

The jobs are in builds-4hr.js.gz, eg:

    {
      "builder_id": 184954,
      "buildnumber": 3,
      "endtime": 1389674279,
      "id": 32955346,
      "master_id": 100,
      "properties": {
        "basedir": "/builds/slave/rel-m-beta-and_rpk_1-000000000",
        "branch": "release-mozilla-beta",
        "build_number": 1,
        "builddir": "release-mozilla-beta-android_repack_1",
        "buildername": "release-mozilla-beta-android_repack_1/10",
        "buildid": "20140113182705",
        "buildnumber": 3,
        "builduid": "7f9f30aeb181499ebcdc4b624cbc62f7",
        "log_url": "http://stage.mozilla.org/pub/mozilla.org/mobile/candidates/27.0b6-candidates/build1/logs/release-mozilla-beta-android_repack_1-bm65-build1-build3.txt.gz",
        "master": "http://buildbot-master65.srv.releng.usw2.mozilla.com:8001/",
        "platform": "android",
        "product": "fennec",
        "products": "fennec",
        "project": "",
        "release_config": "mozilla/release-fennec-mozilla-beta.py",
        "repository": "",
        "request_ids": [
          34765474
        ],
        "request_times": {
          "34765474": 1389673049
        },
        "revision": null,
        "scheduler": "release-mozilla-beta-android_repack",
        "script_repo_revision": "672a39a80c5f",
        "slavebuilddir": "rel-m-beta-and_rpk_1-000000000",
        "slavename": "bld-linux64-ec2-400",
        "toolsdir": "/builds/slave/rel-m-beta-and_rpk_1-000000000/scripts",
        "version": "27.0b6"
      },
      "reason": "Triggerable(release-mozilla-beta-android_repack)",
      "request_ids": [
        34765474
      ],
      "requesttime": 1389673049,
      "result": 2,
      "slave_id": 3723,
      "starttime": 1389673050
    },

However, the data import script skips over any jobs where the revision is missing or null:
https://hg.mozilla.org/webtools/tbpl/file/52200b01cdde/dataimport/import-buildbot-data.py#l206

The revision needs to be specified & ideally the branch needs to be 'mozilla-{aurora,beta,...}' rather than 'release-mozilla-{aurora,beta,...}', so the jobs will hang off of the existing aurora/beta/... pushlogs. I guess we could always munge the release-* psuedo branches as part of the data import - but I would prefer changing upstream, unless there is a specific reason why we can't? :-)

Whilst the bigger l10n story (bug 756594) needs treeherder, this seems like a good quick win for now :-)

Comment 2

[Aki Sasaki (not active)]

Attachment: mobile_l10n_revision

Set mobile_l10n.py revision property

Comment 3

[Aki Sasaki (not active)]

Attachment: mobile_l10n_branch

Set the mobile_l10n branch to |name| in getBranchObjects()

Comment 4

[Aki Sasaki (not active)]

Comment on attachment 8360033 [details] [diff] [review]
mobile_l10n_revision

https://hg.mozilla.org/build/mozharness/rev/745b72b8ed45

Comment 5

[Aki Sasaki (not active)]

Comment on attachment 8360036 [details] [diff] [review]
mobile_l10n_branch

https://hg.mozilla.org/build/buildbotcustom/rev/202f7d35b9df

Comment 6

[Nick Thomas [:nthomas] (UTC+12)]

Uh, I think Ed got a release build by accident. How about this for an aurora nightly build, where the revision is set but branch is wrong:

    {
      "builder_id": 133132,
      "buildnumber": 19,
      "endtime": 1389694307,
      "id": 32965640,
      "master_id": 86,
      "properties": {
        "basedir": "/builds/slave/m-aurora-and-l10n_3-0000000000",
        "branch": "releases/mozilla-aurora",
        "builddir": "mozilla-aurora-android-l10n_3-l10n_3",
        "buildername": "Android 2.2 mozilla-aurora l10n nightly 3/5",
        "buildid": "20140114004002",
        "buildnumber": 19,
        "builduid": "decd2ceffbd54522b4e90b68cc11c97c",
        "hu_failure": "hu failed in make installers-hu!  Can't create a snippet for hu without an upload url.",
        "id_failure": "id failed in make installers-id!  Can't create a snippet for id without an upload url.",
        "it_failure": "it failed in make installers-it!  Can't create a snippet for it without an upload url.",
        "ja-JP-mac_failure": "ja-JP-mac failed in make installers-ja-JP-mac!  Can't create a snippet for ja-JP-mac without an upload url.",
        "ja_failure": "ja failed in make installers-ja!  Can't create a snippet for ja without an upload url.",
        "ko_failure": "ko failed in make installers-ko!  Can't create a snippet for ko without an upload url.",
        "locales": "{\"ja-JP-mac\": \"Failed\", \"nl\": \"Failed\", \"ko\": \"Failed\", \"it\": \"Failed\", \"id\": \"Failed\", \"lt\": \"Failed\", \"nb-NO\": \"Failed\", \"hu\": \"Failed\", \"ja\": \"Failed\"}",
        "log_url": "http://ftp.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/releases/mozilla-aurora-l10n/mozilla-aurora-android-l10n_3-unknown-bm63-build1-build19.txt.gz",
        "lt_failure": "lt failed in make installers-lt!  Can't create a snippet for lt without an upload url.",
        "master": "http://buildbot-master63.srv.releng.use1.mozilla.com:8001/",
        "nb-NO_failure": "nb-NO failed in make installers-nb-NO!  Can't create a snippet for nb-NO without an upload url.",
        "nl_failure": "nl failed in make installers-nl!  Can't create a snippet for nl without an upload url.",
        "platform": "android",
        "product": "mobile",
        "project": "",
        "repository": "",
        "request_ids": [
          34775544
        ],
        "request_times": {
          "34775544": 1389693396
        },
        "revision": "29c5b8def408",
        "scheduler": "mozilla-aurora-android-l10n",
        "script_repo_revision": "a3ca4936ba02",
        "slavebuilddir": "m-aurora-and-l10n_3-0000000000",
        "slavename": "bld-linux64-ec2-056",
        "stage_platform": "android",
        "toolsdir": "/builds/slave/m-aurora-and-l10n_3-0000000000/scripts"
      },
      "reason": "Triggerable(mozilla-aurora-android-l10n)",
      "request_ids": [
        34775544
      ],
      "requesttime": 1389693396,
      "result": 2,
      "slave_id": 2486,
      "starttime": 1389693456
    },

From builds-2014-01-14.js.gz.

Comment 7

[Aki Sasaki (not active)]

The revision might get overwritten with my mh patch, but the buildbotcustom patch should fix the branch either way.  I was wondering how repo_path got turned into releases-mozilla-* ... this explains it, thanks!

Comment 8

[Ed Morley [:emorley]]

(In reply to Nick Thomas [:nthomas] from comment #6)
> Uh, I think Ed got a release build by accident. How about this for an aurora
> nightly build, where the revision is set but branch is wrong:

Ah sorry yeah I was looking at bug 923950 mentioned in comment 0 to try and find a job name to search for and used the log in bug 923950 comment 28, which is for a release build. 

Since the revision is set, these jobs will be being imported - but just not displayed on the correct tree in the TBPL UI, so we should be all good after the patch lands:
https://tbpl.mozilla.org/php/getRevisionBuilds.php?branch=releases/mozilla-aurora&rev=5bb1837de7c0

However, any reason why we can't set the revision for release builds so that they display in TBPL too? :-)

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

All the builder names are different, and mostly not applicable to on-change builds, so how would they be displayed on TBPL ? Now if treeherder had a view for the release automation that would be cool for us+RelMan to check status.

Comment 10

[[e-mail] (:bhearsum)]

in production

Comment 11

[Ed Morley [:emorley]]

(In reply to Ben Hearsum [:bhearsum] from comment #10)
> in production

l10n nightlies now showing up :-)
https://tbpl.mozilla.org/?tree=Mozilla-Aurora&showall=1&jobname=l10n

(In reply to Nick Thomas [:nthomas] from comment #9)
> All the builder names are different, and mostly not applicable to on-change
> builds, so how would they be displayed on TBPL ? Now if treeherder had a
> view for the release automation that would be cool for us+RelMan to check
> status.

I think it would still be useful to have the release jobs display on the revision (they are spun from that revision after all, and any respins on the same rev will just appear as duplicate runs, same as currently happens for self-serve retriggers on normal builds); any new job names can be added to the regexp.

Comment 12

[Aki Sasaki (not active)]

Yay!