753501 - Add empty tooltool manifests to some platforms

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P2)

Description

[Rail Aliiev [:rail]]

... so devs can easier test new toolkits.
Platforms: linux{,64}{,-debug}, macosx64{,-debug}, android{,-xul}

Comment 1

[Rail Aliiev [:rail]]

Attachment: empty manifests

Comment 2

[Rail Aliiev [:rail]]

Attachment: configs

I moved tooltool_url_list to localcofig's GLOBAL_VARS, so it's not platform specific anymore (no need to copy paste it around) and you can easily redefine it for staging/pp.

Comment 3

[Rail Aliiev [:rail]]

Attachment: buildbotcustom

Inline comments incoming

Comment 4

[Rail Aliiev [:rail]]

Comment on attachment 622719 [details] [diff] [review]
buildbotcustom

Review of attachment 622719 [details] [diff] [review]:
-----------------------------------------------------------------

::: process/factory.py
@@ +768,5 @@
>                   mozharnessMultiOptions=None,
>                   mock_packages=[],
>                   tooltool_manifest_src=None,
>                   tooltool_bootstrap="setup.sh",
> +                 tooltool_url_list=None,

Using None is safer

@@ +1202,5 @@
>          ))
>          if self.tooltool_manifest_src:
>              self.addStep(RetryingShellCommand(
>                  name='fetch_tooltool_resources',
> +                command=[self.tooltool_script, '--url', self.tooltool_url_list[0],

"python /tools/tooltool.py" ignores the shebang "/tools/tooltool.py" doesn't.

@@ +1209,5 @@
>                  name='tooltool_bootstrap',
> +                command=['bash', '-c',
> +                         'if [ -e "%s" ]; then bash "%s"; fi' % \
> +                         (self.tooltool_bootstrap, self.tooltool_bootstrap)]
> +            ))

I added this check to allow empty manifests.

The patches can't be landed before we deploy tooltool.py to builders.

Comment 5

[Rail Aliiev [:rail]]

Attachment: tooltool

Some enhancements and fixes. 
Still need to see how pretty is the output of Manifest.__eq__() and add download progress (optional) + some testing.

Comment 6

[Rail Aliiev [:rail]]

Comment on attachment 622718 [details] [diff] [review]
configs

Probably I should define tooltool_manifest_src per branch and disable it by default in PLATFORM_VARS (except b2g).

Comment 7

[John Ford [:jhford] CET/CEST Berlin Time]

Comment on attachment 622737 [details] [diff] [review]
tooltool

Please open a pull request on https://github.com/jhford/tooltool for this.  The copy checked in here is a snapshot of that repository's tooltool.  Also, there are tests that should be updated if behaviour is changed.

Comment 8

[Rail Aliiev [:rail]]

(In reply to John Ford [:jhford] from comment #7)
> Comment on attachment 622737 [details] [diff] [review]
> tooltool
> 
> Please open a pull request on https://github.com/jhford/tooltool for this. 
> The copy checked in here is a snapshot of that repository's tooltool.  Also,
> there are tests that should be updated if behaviour is changed.

Woooo! Much better! Will do.

Comment 9

[Rail Aliiev [:rail]]

Attachment: configs

Sorry for rush with the first patch, this one is more conservative. Let's start using this on try only.

Comment 10

[Rail Aliiev [:rail]]

Attachment: puppet-manifests

Almost copy/paste from /build/puppet

Comment 11

[John Ford [:jhford] CET/CEST Berlin Time]

Comment on attachment 623121 [details] [diff] [review]
puppet-manifests

my assumption is that this is using a tooltool.py that is mostly identical to the HEAD of github.com/jhford/tooltool.  If that's correct, r+

Comment 12

[Rail Aliiev [:rail]]

Comment on attachment 622717 [details] [diff] [review]
empty manifests

And the last piece! The hardest one. :)

Comment 13

[Rail Aliiev [:rail]]

One todo left: lion has python 2.7 installed, probably the patch for puppet-manifests will be resubmitted. :/

Comment 14

[Rail Aliiev [:rail]]

Comment on attachment 623121 [details] [diff] [review]
puppet-manifests

http://hg.mozilla.org/build/puppet-manifests/rev/073c226cc489

Comment 15

[Rail Aliiev [:rail]]

Attachment: puppet-manifests

* Use "wheel" group, which is common for linux/mac

Comment 16

[Rail Aliiev [:rail]]

Comment on attachment 624129 [details] [diff] [review]
puppet-manifests

http://hg.mozilla.org/build/puppet-manifests/rev/bbea6dfcacfd

Comment 17

[Rail Aliiev [:rail]]

Comment on attachment 622717 [details] [diff] [review]
empty manifests

Need ted's rubber stamp here. Empty manifests passing staging tests.

Comment 18

[(not currently active) Ted Mielczarek]

Comment on attachment 622717 [details] [diff] [review]
empty manifests

I have no idea what a tooltool manifest is, but if this makes your life easier then go ahead.

Comment 19

[Rail Aliiev [:rail]]

Comment on attachment 622717 [details] [diff] [review]
empty manifests

https://hg.mozilla.org/integration/mozilla-inbound/rev/8ffae46336ed

Comment 20

[Rail Aliiev [:rail]]

Comment on attachment 622717 [details] [diff] [review]
empty manifests

It was backed out by khuey's "big hammer" (multiple backouts due to the tree's red status)
https://hg.mozilla.org/integration/mozilla-inbound/rev/dad0e473e803

Comment 21

[Rail Aliiev [:rail]]

Comment on attachment 622717 [details] [diff] [review]
empty manifests

http://hg.mozilla.org/integration/mozilla-inbound/rev/ff731bcba745

Comment 22

[Rail Aliiev [:rail]]

Comment on attachment 622719 [details] [diff] [review]
buildbotcustom

http://hg.mozilla.org/build/buildbotcustom/rev/dcc7c67d2612

Comment 23

[Rail Aliiev [:rail]]

Comment on attachment 622827 [details] [diff] [review]
configs

http://hg.mozilla.org/build/buildbot-configs/rev/9a3c8b17ea66

Comment 24

[Ryan VanderMeulen [:RyanVM]]

(In reply to Rail Aliiev [:rail] from comment #21)
> Comment on attachment 622717 [details] [diff] [review]
> empty manifests
> 
> http://hg.mozilla.org/integration/mozilla-inbound/rev/ff731bcba745

https://hg.mozilla.org/mozilla-central/rev/ff731bcba745

Comment 25

[Mike Taylor [:bear]]

in production 2012-05-17 0750 PDT

Comment 26

[Phil Ringnalda (:philor)]

Comment on attachment 622827 [details] [diff] [review]
configs

>+BRANCHES['try']['platforms']['android-xul']['tooltool_manifest_src'] = 'mobile/android-xul/config/tooltool-manifests/android

It's a good thing that nobody cares about their Android results on try, much less their Android XUL results on try, because the manifest is in mobile/xul/ rather than mobile/android-xul/, so every single build since this landed has burned.

Comment 27

[Phil Ringnalda (:philor)]

Attachment: try xul config

Comment 28

[Rail Aliiev [:rail]]

Comment on attachment 625855 [details] [diff] [review]
try xul config

Bah! My bad...

Comment 29

[Phil Ringnalda (:philor)]

Comment on attachment 625855 [details] [diff] [review]
try xul config

http://hg.mozilla.org/build/buildbot-configs/rev/d30c4e6286c2

Comment 30

[Phil Ringnalda (:philor)]

Caught a reconfig yesterday, and I see green XUL builds.