1107747 - Create build for device branch 2.1s

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Comment 1

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: bug-1107747-part-1.patch

Hal, Could you review this patch? Thanks!

Comment 2

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: bug-1107747-part-2.patch

Hal, Could you review this patch? Thanks!

Comment 3

[Hal Wine [:hwine] (use NI)]

Comment on attachment 8532286 [details] [diff] [review]
bug-1107747-part-1.patch

Review of attachment 8532286 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 4

[Hal Wine [:hwine] (use NI)]

Comment on attachment 8532288 [details] [diff] [review]
bug-1107747-part-2.patch

Review of attachment 8532288 [details] [diff] [review]:
-----------------------------------------------------------------

lgtm

Comment 5

[Kai-Zhen Li [:seinlin][:kli]]

Running test-masters.sh is Passed.

Comment 6

[Jordan Lund (:jlund)]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/a15f53cafbd0

Comment 7

[Jordan Lund (:jlund)]

In production: https://hg.mozilla.org/build/buildbotcustom/rev/c46bfda22efe

Comment 8

[Hal Wine [:hwine] (use NI)]

Comment on attachment 8532288 [details] [diff] [review]
bug-1107747-part-2.patch

http://hg.mozilla.org/build/buildbot-configs/rev/a15f53cafbd0

Comment 9

[Hal Wine [:hwine] (use NI)]

Comment on attachment 8532288 [details] [diff] [review]
bug-1107747-part-2.patch

backed out due to bustage found by sheriffs. 's' missing on a number of device builds, disabling them on 2.1 instead

backout commit:
 http://hg.mozilla.org/build/buildbot-configs/rev/2cc554bc3a33

Comment 10

[Hal Wine [:hwine] (use NI)]

Please fix the config, and re-attach for review. My apologies for missing this in PDX.

Comment 11

[Chris Cooper [:coop] (he/him)]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/2cc554bc3a33

Comment 12

[Kai-Zhen Li [:seinlin][:kli]]

Hal, Thanks! We'll re-attach for review and land it after it is fixed.

Comment 13

[Peipei Cheng (needinfo if you need my action)]

Kai-Zhen, bug 1115297 is already fixed. Could you please help with this one?

Comment 14

[Kai-Zhen Li [:seinlin][:kli]]

Peipei, OK, I will do that. But the target device for v2.1s got some changes, so I need to finish bug 1120745. Keep NI for todo.

Comment 15

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: bug-1107747-buildbot-configs.patch

Hwine, Now branch 2.1s is ready. Could you review this patch? Thanks!

Comment 16

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Add short name for dolphin.

Hwine, We need to add short name for dolphin, otherwise dolphin-512 will get error in test-masters.sh because of len > 30. Could you review this patch? Thanks!

Comment 17

[Kai-Zhen Li [:seinlin][:kli]]

Comment on attachment 8532286 [details] [diff] [review]
bug-1107747-part-1.patch

This is already landed.

Comment 18

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Add dolphin-512 for buildbot.

Hwine, Could you review this patch? Thanks!

Comment 19

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Add in-tree files for dolphin-512 on master.

Nick, We need to add a new device config. Could you review this patch? Thanks!

Comment 20

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: [v2.1s] Add in-tree files for dolphin-512.

Comment 21

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: bug-1107747-mozharness.patch

Nick, Could you review this patch? Thanks!

Comment 22

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8548911 [details] [diff] [review]
bug-1107747-buildbot-configs.patch

Driveby comment ....

>diff --git a/mozilla/b2g_config.py b/mozilla/b2g_config.py
>+# dolphin-512 is for selected branches only
>+for branch in BRANCHES:
>+    if branch not in ('mozilla-b2g34_v2_1s'):

This needs to have a trailing comma in the tuple, ie 
    if branch not in ('mozilla-b2g34_v2_1s',):

Comment 23

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8548924 [details] [diff] [review]
Add in-tree files for dolphin-512 on master.

Seems fine. I see you've not copied b2g/config/dolphin/releng-dolphin.tt over, could you comment on why that is ? (for anyone who comes an looks at this bug later).

Comment 24

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8548924 [details] [diff] [review]
Add in-tree files for dolphin-512 on master.

Actually, I don't think you need this patch, because the buildbot patch isn't adding dolphin-512 builders on master (aka mozilla-central), just 2.1s.

Comment 25

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8548945 [details] [diff] [review]
bug-1107747-mozharness.patch

Review of attachment 8548945 [details] [diff] [review]:
-----------------------------------------------------------------

Could you clarify where dolphin-512 builds are going to run ? If it's only 2.1s then changes to master.py, v2.1.py and v2.2.py are unnecessary.

::: configs/b2g_bumper/v2.1s.py
@@ +7,5 @@
> +        "gittool.py": ["/usr/local/bin/gittool.py"],
> +    },
> +    'gecko_pull_url': 'https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/',
> +    'gecko_push_url': 'ssh://hg.mozilla.org/releases/mozilla-b2g34_v2_1/',
> +    'gecko_local_dir': 'mozilla-b2g34_v2_1',

Looks like you've done a copy of v2.1.py, and these three need updating.

@@ +11,5 @@
> +    'gecko_local_dir': 'mozilla-b2g34_v2_1',
> +    'git_ref_cache': '/builds/b2g_bumper/git_ref_cache.json',
> +
> +    'manifests_repo': 'https://git.mozilla.org/b2g/b2g-manifest.git',
> +    'manifests_revision': 'origin/v2.1',

And this.

@@ +19,5 @@
> +    "ssh_user": "ffxbld",
> +
> +    'hgtool_base_bundle_urls': ['https://ftp-ssl.mozilla.org/pub/mozilla.org/firefox/bundles'],
> +
> +    'gaia_repo_url': 'https://hg.mozilla.org/integration/gaia-2_1',

And this, and I'll stop looking now but please check for others.

Comment 26

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: bug-1107747-mozharness.patch

Hi Nick,

Yes, dolphin-512 will run on v2.1s only. I updated the patch, could you review it again when you have time? 

Thanks!

Comment 27

[Kai-Zhen Li [:seinlin][:kli]]

(In reply to Nick Thomas [:nthomas] from comment #23)
> Comment on attachment 8548924 [details] [diff] [review]
> Add in-tree files for dolphin-512 on master.
> 
> Seems fine. I see you've not copied b2g/config/dolphin/releng-dolphin.tt
> over, could you comment on why that is ? (for anyone who comes an looks at
> this bug later).

To build dolphin and dolphin-512, we don't need the system backup tar file, because we can build all necessary binary from source.

Comment 28

[Hal Wine [:hwine] (use NI)]

Comment on attachment 8548914 [details] [diff] [review]
Add short name for dolphin.

Glad the tests caught this! :)

Comment 29

[Hal Wine [:hwine] (use NI)]

Comment on attachment 8548917 [details] [diff] [review]
Add dolphin-512 for buildbot.

Review of attachment 8548917 [details] [diff] [review]:
-----------------------------------------------------------------

I'm not familiar with the issues in adding a new device, :rail does.

Comment 30

[Hal Wine [:hwine] (use NI)]

Comment on attachment 8548911 [details] [diff] [review]
bug-1107747-buildbot-configs.patch

new branch looks right, deferring to :rail for new device changes.

Comment 31

[Rail Aliiev [:rail]]

Comment on attachment 8548911 [details] [diff] [review]
bug-1107747-buildbot-configs.patch

Review of attachment 8548911 [details] [diff] [review]:
-----------------------------------------------------------------

In overall it looks good, but i requires another patch to make it work. I get the following errors trying to run the tests:

ValueError: Cannot shorten "b2g_mozilla-b2g34_v2_1_dolphin-512_eng_dep" to maximum length (30). Got to: b2g_m-b34_21_dolphin-512_eng_dep

You need to add a rule to shorten "dolphin" in http://hg.mozilla.org/build/buildbotcustom/file/2b02c9c12a9c/common.py#l48

Comment 32

[Kai-Zhen Li [:seinlin][:kli]]

(In reply to Rail Aliiev [:rail] from comment #31)
> Comment on attachment 8548911 [details] [diff] [review]
> bug-1107747-buildbot-configs.patch
> 
> Review of attachment 8548911 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> In overall it looks good, but i requires another patch to make it work. I
> get the following errors trying to run the tests:
> 
> ValueError: Cannot shorten "b2g_mozilla-b2g34_v2_1_dolphin-512_eng_dep" to
> maximum length (30). Got to: b2g_m-b34_21_dolphin-512_eng_dep
> 
> You need to add a rule to shorten "dolphin" in
> http://hg.mozilla.org/build/buildbotcustom/file/2b02c9c12a9c/common.py#l48

Rail, Thanks! I added a patch in comment 16 to fix it error. I think it should work.

Comment 33

[Rail Aliiev [:rail]]

Comment on attachment 8548911 [details] [diff] [review]
bug-1107747-buildbot-configs.patch

Oh, I missed that patch. With both applied it worked fine.

Thanks for your contribution and making our lives easier. :)

Comment 34

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Kai-Zhen Li [:seinlin] from comment #27)
> To build dolphin and dolphin-512, we don't need the system backup tar file,
> because we can build all necessary binary from source.

OK. I was confused by http://hg.mozilla.org/mozilla-central/file/default/b2g/config/dolphin/releng-dolphin.tt but that was only recently added.

Comment 35

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8549303 [details] [diff] [review]
bug-1107747-mozharness.patch

>diff --git a/configs/b2g_bumper/v2.1s.py b/configs/b2g_bumper/v2.1s.py
>+    'gaia_repo_url': 'https://hg.mozilla.org/integration/gaia-2_1s',

This doesn't exist yet, please ask hwine/pmoore to set it up (if you haven't already). Otherwise looks fine.

Comment 36

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: Add in-tree files for dolphin-512

Correct DEVICE name for dolphin-512, carry r+.

Comment 37

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: [v2.1s] Add in-tree files for dolphin-512

Comment 38

[Kai-Zhen Li [:seinlin][:kli]]

https://hg.mozilla.org/build/braindump/rev/120ee7210327
https://hg.mozilla.org/build/buildbotcustom/rev/34e410cf73ca
https://hg.mozilla.org/build/buildbot-configs/rev/2232ba781e1e
https://hg.mozilla.org/build/mozharness/rev/fd342d8b4916
https://hg.mozilla.org/integration/b2g-inbound/rev/bf0e7704b8a4
https://hg.mozilla.org/releases/mozilla-b2g34_v2_1s/rev/4b542cf7b5eb

Comment 39

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/bf0e7704b8a4

Comment 40

[Jordan Lund (:jlund)]

buildbotcustom patch is in production

Comment 41

[Jordan Lund (:jlund)]

buildbot-config patch(es) are live in production

Comment 42

[Jordan Lund (:jlund)]

a mozharness patch has from this bug is in production

Comment 43

[Kai-Zhen Li [:seinlin][:kli]]

Attachment: bug-1107747-quick-fix.patch

Nick, Could you review this patch for me? Thanks!

Comment 44

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 8555000 [details] [diff] [review]
bug-1107747-quick-fix.patch

Confirmed this drops these two builders:
< b2g_mozilla-b2g34_v2_1_dolphin-512_periodic ScriptFactory
< b2g_mozilla-b2g34_v2_1_dolphin-512_eng_periodic ScriptFactory

leaving 4 on 2.1 (opt+eng, periodic+nightly).

Comment 45

[Kai-Zhen Li [:seinlin][:kli]]

https://hg.mozilla.org/build/buildbot-configs/rev/d9b62528567c

Comment 46

[Jordan Lund (:jlund)]

In production: https://hg.mozilla.org/build/buildbot-configs/rev/d9b62528567c

Comment 47

[Pete Moore [:pmoore][:pete]]

Attachment: bug1107747_mozharness_fix-b2g-bumper-config_v1.patch

This v2.1s config was invalid, and causing mozharness to fail.

We were also not alerted (see bug 1135030) - note this has not been working since it landed several days ago.

This error was only discovered as a side effect of investigating bug 1136148, which was unrelated (an hg process hanging).

Comment 48

[Pete Moore [:pmoore][:pete]]

Comment on attachment 8568562 [details] [diff] [review]
bug1107747_mozharness_fix-b2g-bumper-config_v1.patch

Landed on default, jlund will merge later today during a reconfig.

https://hg.mozilla.org/build/mozharness/rev/21f00dd7bda6

Comment 49

[Chris Cooper [:coop] (he/him)]

In production: https://hg.mozilla.org/build/mozharness/rev/21f00dd7bda6