1190773 - Revamp/update how SeaMonkey uses tooltool.

Status: RESOLVED FIXED

(SeaMonkey :: Release Engineering, defect)

Description

[Edmund Wong (:ewong)]

Bug 1190256 stems from us not having the same tooltool configuration as
Moco-proper.

The inclusion of gtk3/setup.sh at http://hg.mozilla.org/comm-central/file/eace0c64cb12/build/unix/mozconfig.gtk#l25 so what's choking us.

Moco-proper's tooltool gtk3.tar.xz package has the setup.sh.  Ours
don't.  It's pointless to somehow 'fudge' it by copying our setup.sh to
gtk3/.  Somehow during the build, it deletes the setup.sh.

So, since we *need* to update our tooltool setup, might as well do it
now, especially since bug 1184951 is looming in the forefront as well.

Plan of attack:

1) Get the updated tooltool.py file and push them to the slaves.
2) Fix our buildbotcustom tooltool code.
3) *preliminary scan* implies that we won't need to do much in terms
   of modifying our mozconfigs.  

#1 is simple.

#2 is a little more complicated in that we have a bunch of manually
applied patches(without a proper review, call 'em hacks) (which 'fixes'
our previously busted releng (mock.. I'm looking at you).

Comment 1

[Edmund Wong (:ewong)]

(In reply to Edmund Wong (:ewong) from comment #0)
> Bug 1190256 stems from us not having the same tooltool configuration as
> Moco-proper.
> 
> The inclusion of gtk3/setup.sh at
> http://hg.mozilla.org/comm-central/file/eace0c64cb12/build/unix/mozconfig.
> gtk#l25 so what's choking us.
> 
> Moco-proper's tooltool gtk3.tar.xz package has the setup.sh.  Ours
> don't.  It's pointless to somehow 'fudge' it by copying our setup.sh to
> gtk3/.  Somehow during the build, it deletes the setup.sh.
> 
> So, since we *need* to update our tooltool setup, might as well do it
> now, especially since bug 1184951 is looming in the forefront as well.
> 
> Plan of attack:
> 
> 1) Get the updated tooltool.py file and push them to the slaves.
> 2) Fix our buildbotcustom tooltool code.
> 3) *preliminary scan* implies that we won't need to do much in terms
>    of modifying our mozconfigs.  
> 

While the mozconfigs 'might' not need changing, then tooltool-manifests
do.

Comment 2

[Edmund Wong (:ewong)]

Attachment: buildbotcustom changes (v1)

Comment 3

[Edmund Wong (:ewong)]

Attachment: buildbot-configs changes (v1)

Comment 4

[Edmund Wong (:ewong)]

Attachment: c-c manifest changes (v1)

Comment 5

[Edmund Wong (:ewong)]

Attachment: c-a manifest changes (v1)

Comment 6

[Edmund Wong (:ewong)]

Attachment: c-b manifest changes (v1)

Comment 7

[Edmund Wong (:ewong)]

Attachment: c-r manifest changes (v1)

Comment 8

[Edmund Wong (:ewong)]

Those are the WIP patches.

It's my belief that c-c is currently the only tree that requires the
changes.  The others can wait for 1) train movement, or 2) uplift.
[train movement would render the c-a -> c-r patches irrelevant.)

Comment 9

[Edmund Wong (:ewong)]

Addendum: I pro-actively added the Win64 for our future Win64 builds.

Comment 10

[Edmund Wong (:ewong)]

Attachment: c-c manifests changes (v2)

Fixed some nits and removed sccache.* mentions.

Comment 11

[Edmund Wong (:ewong)]

Attachment: c-c manifests changes (v3)

Forgot to remove the extra ,

Comment 12

[Edmund Wong (:ewong)]

Attachment: buildbotcustom code changes (v2) (unbitrot)

Comment 13

[Edmund Wong (:ewong)]

Attachment: buildbotcustom patch (v3)

Comment 14

[Edmund Wong (:ewong)]

Attachment: tooltool.py

This is the tooltool.py file that is required to be put in all our
slaves.

Comment 15

[Edmund Wong (:ewong)]

Attachment: [c-a] tooltool manifest changes

Comment 16

[Edmund Wong (:ewong)]

Attachment: [c-a] tooltool manifest changes (v3)

Comment 17

[Edmund Wong (:ewong)]

Attachment: [c-b] tooltool manifest changes

Comment 18

[Edmund Wong (:ewong)]

Attachment: [c-r] tooltool manifest changes.

Comment 19

[Edmund Wong (:ewong)]

Attachment: [c-a] tooltool manifest changes (v4)

Comment 20

[Edmund Wong (:ewong)]

Attachment: [c-c] tooltool manifest changes (v4)

Comment 21

[Edmund Wong (:ewong)]

Attachment: [c-b] tooltool manifest changes (v4)

Comment 22

[Edmund Wong (:ewong)]

Attachment: [c-r] tooltool manifest changes. (v4)

Comment 23

[Edmund Wong (:ewong)]

Attachment: [puppet] support tooltool tokens

Comment 24

[Edmund Wong (:ewong)]

Attachment: [configs] change configs to use new tooltool

Comment 25

[Edmund Wong (:ewong)]

With regards to the tooltool.token puppet and config changes, it's the
start of using the public api (which I believe, the tooltool.token
at this point, is optional).

Comment 26

[Edmund Wong (:ewong)]

Attachment: [custom] custom code changes (v2)

Comment 27

[Edmund Wong (:ewong)]

Attachment: [puppet] support tooltool tokens (v2)

Comment 28

[Edmund Wong (:ewong)]

Attachment: [custom] custom code changes (v3)

Comment 29

[Edmund Wong (:ewong)]

Attachment: [custom] custom code changes (v4) [checked-in]

Comment 30

[Edmund Wong (:ewong)]

Attachment: [configs] Update configs to support new tooltool.py (v3) [checked-in]

Comment 31

[Edmund Wong (:ewong)]

Attachment: [puppet] support tooltool tokens (v3) [checked-in]

Comment 32

[Edmund Wong (:ewong)]

Comment on attachment 8736159 [details] [diff] [review]
[puppet] support tooltool tokens (v3) [checked-in]

Pushed to puppet: (default)
https://hg.mozilla.org/SeaMonkey/puppet/rev/e28e041616db

Merged to production:
 https://hg.mozilla.org/SeaMonkey/puppet/rev/9efd7daf6f24

Comment 33

[Philip Chee]

Comment on attachment 8735714 [details] [diff] [review]
[c-c] tooltool manifest changes (v4)

> # HG changeset patch
> # User Edmund Wong <[e-mail]>
> # Parent  709e8de02a1540e270af41bbb608b321e13f6a06
> Bug 1190773 - comm-central tooltool manifest changes

> +++ b/suite/config/tooltool-manifests/linux32/clang.manifest

> -"clang_version": "r170890"
> +"clang_version": "r247539"
Bug 1254451 - Better document toolchain versions in tooltool manifests.
"version": "clang 3.8.0/r247539"
Also please fix linux64/clang.manifest etc

> +"size": 105219872,
> +"digest": "aa8de2fa535d0667e079019c475c631ea008f1bb5228505510867255b4d9c30663e2c97e579220a575a5887aa3bcf250021b50f76b90c2fa8c65a7aa19270066",
OK.

>  "algorithm": "sha512",
> +"filename": "clang.tar.bz2",
Bug 1182727 - Part 17: Fix some incorrect file names in tooltool mnifests
"filename": "clang.tar.xz"
Also please fix /macosx64/releng.manifest etc

> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/linux32/releng.manifest
>  "size": 80458572,
>  "digest": "e5101f9dee1e462f6cbd3897ea57eede41d23981825c7b20d91d23ab461875d54d3dfc24999aa58a31e8b01f49fb3140e05ffe5af2957ef1d1afb89fd0dfe1ad",
This only gets you gcc 4.7.3
Bug 1175546 - Update GCC to 4.8.5
"version": "gcc 4.8.5",
"size": 81065660,
"digest": "db26f498ab56a3b5c65d7cda290cbb74174af9f2d021ca9c158f53b0382924ccf5ed9638d41eef449434aa9383a9113994d9729d9dd910321d1f35f9411eae38",
Also please fix up linux64/releng.manifest

>  "algorithm": "sha512",
> +"filename": "gcc.tar.xz",
> +"unpack": true
OK

>  "size": 11179576,
>  "digest": "91567ce8e2bb8ab0ebc60c31e90731d88a1ea889fb71bcf55c735746a60fa7610b7e040ea3d8f727b6f692ae3ee703d6f3b30cdbd76fdf5617f77d9c38aa20ed",
Do we need Bug 1242641?
"size": 11189216,
"digest": "18bc52b0599b1308b667e282abb45f47597bfc98a5140cfcab8da71dacf89dd76d0dee22a04ce26fe7ad1f04e2d6596991f9e5b01fd2aaaab5542965f596b0e6",

>  "algorithm": "sha512",
> -"filename": "gtk3.tar.xz"
> +"filename": "gtk3.tar.xz",
"setup": "setup.sh", (Bug 124264) Also please fix up linux64/releng.manifest.

> +"unpack": true
OK.

>  "size": 167175,
>  "digest": "0b71a936edf5bd70cf274aaa5d7abc8f77fe8e7b5593a208f805cc9436fac646b9c4f0b43c2b10de63ff3da671497d35536077ecbc72dba7f8159a38b580f831",
>  "algorithm": "sha512",
> -"filename": "sccache.tar.bz2"
> +"filename": "sccache.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/linux64/clang.manifest
> +"clang_version": "r247539"
"version": "clang 3.8.0/r247539",

> +"size": 105219872,
> +"digest": "aa8de2fa535d0667e079019c475c631ea008f1bb5228505510867255b4d9c30663e2c97e579220a575a5887aa3bcf250021b50f76b90c2fa8c65a7aa19270066",
>  "algorithm": "sha512",
> +"filename": "clang.tar.bz2",
"filename": "clang.tar.xz"

> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/linux64/releng.manifest

>  "size": 80458572,
>  "digest": "e5101f9dee1e462f6cbd3897ea57eede41d23981825c7b20d91d23ab461875d54d3dfc24999aa58a31e8b01f49fb3140e05ffe5af2957ef1d1afb89fd0dfe1ad",
>  "algorithm": "sha512",
> +"filename": "gcc.tar.xz",
> +"unpack": true
Bug 1175546 - Update GCC to 4.8.5
"version": "gcc 4.8.5",
"size": 81065660,
"digest": "db26f498ab56a3b5c65d7cda290cbb74174af9f2d021ca9c158f53b0382924ccf5ed9638d41eef449434aa9383a9113994d9729d9dd910321d1f35f9411eae38",

>  "size": 12057960,
>  "digest": "6105d6432943141cffb40020dc5ba3a793650bdeb3af9bd5e56d3796c5f03df9962a73e521646cd71fbfb5e266c1e74716ad722fb6055589dfb7d35175bca89e",
Bug 1242641
"size": 12072532,
"digest": "3915f8ec396c56a8a92e6f9695b70f09ce9d1582359d1258e37e3fd43a143bc974410e4cfc27f500e095f34a8956206e0ebf799b7287f0f38def0d5e34ed71c9",

>  "algorithm": "sha512",
> -"filename": "gtk3.tar.xz"
> +"filename": "gtk3.tar.xz",
Bug 1242641
"setup": "setup.sh",

> +"unpack": true
OK.

>  "size": 167175,
>  "digest": "0b71a936edf5bd70cf274aaa5d7abc8f77fe8e7b5593a208f805cc9436fac646b9c4f0b43c2b10de63ff3da671497d35536077ecbc72dba7f8159a38b580f831",
>  "algorithm": "sha512",
> -"filename": "sccache.tar.bz2"
> +"filename": "sccache.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/macosx64/releng.manifest
> @@ -1,17 +1,12 @@
>  "clang_version": "r247539"
"version": "clang 3.8.0/r247539",

> +"size": 121393888,
> +"digest": "6ae4e651e545538e6de326a7fb8b44b6e6d0b3acdb6a969ecb2b6f63b9995bbad2111cabf044ba575464f17f1f948d78ec92ad3a6922a7bfdf3ad6b6b2cad050",
Where does this size/digest come from? I think this should be:
"size": 121389802,
"digest": "2be6b42cfa1e92de4b49a57123f54043fec2d3cf8385276516dc6aaed99c88768ac4aebd7ce2e007ab074163523da29223436a4d1aef82f0f750f08f1b14cd71",

>  "algorithm": "sha512",
> +"filename": "clang.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/win32/l10n.manifest

>  "size": 266240,
>  "digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
>  "algorithm": "sha512",
>  "filename": "mozmake.exe"
OK.

> +++ b/suite/config/tooltool-manifests/win32/releng.manifest

>  "size": 266240,
>  "digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
>  "algorithm": "sha512",
>  "filename": "mozmake.exe"
OK.

Do we need entries for sccache, rust, and VS 2015u1?

> +++ b/suite/config/tooltool-manifests/win64/l10n.manifest

> +"size": 266240,
> +"digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
> +"algorithm": "sha512",
> +"filename": "mozmake.exe"
OK.

> +++ b/suite/config/tooltool-manifests/win64/releng.manifest

> +"size": 266240,
> +"digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
> +"algorithm": "sha512",
> +"filename": "mozmake.exe"
OK.

Do we need entries for sccache, rust, and VS 2015u1?

Comment 34

[Philip Chee]

(In reply to Edmund Wong (:ewong) from comment #22)
> Created attachment 8735717 [details] [diff] [review]
> [c-r] tooltool manifest changes. (v4)
This appears to be identical to the comm-beta patch. Is this intentional?

Comment 35

[Philip Chee]

Comment on attachment 8735711 [details] [diff] [review]
[c-a] tooltool manifest changes (v4)

> # HG changeset patch
> # User Edmund Wong <[e-mail]>
> # Parent  aa7262925765e09922407bb43f0af40569f72ef1
> Bug 1190773 - comm-aurora manifest changes.

> +++ b/suite/config/tooltool-manifests/linux32/clang.manifest

> -"clang_version": "r170890"
> +"clang_version": "r247539"
"version": "clang 3.8.0/r247539"

> +"size": 105219872,
> +"digest": "aa8de2fa535d0667e079019c475c631ea008f1bb5228505510867255b4d9c30663e2c97e579220a575a5887aa3bcf250021b50f76b90c2fa8c65a7aa19270066",
>  "algorithm": "sha512",
>  "filename": "clang.tar.bz2"
"filename": "clang.tar.xz",

> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/linux32/releng.manifest
>  "size": 80458572,
>  "digest": "e5101f9dee1e462f6cbd3897ea57eede41d23981825c7b20d91d23ab461875d54d3dfc24999aa58a31e8b01f49fb3140e05ffe5af2957ef1d1afb89fd0dfe1ad",
>  "algorithm": "sha512",
> +"filename": "gcc.tar.xz",
> +"unpack": true
OK.

>  "size": 11179576,
>  "digest": "91567ce8e2bb8ab0ebc60c31e90731d88a1ea889fb71bcf55c735746a60fa7610b7e040ea3d8f727b6f692ae3ee703d6f3b30cdbd76fdf5617f77d9c38aa20ed",
"size": 11189216,
"digest": "18bc52b0599b1308b667e282abb45f47597bfc98a5140cfcab8da71dacf89dd76d0dee22a04ce26fe7ad1f04e2d6596991f9e5b01fd2aaaab5542965f596b0e6",

>  "algorithm": "sha512",
> +"filename": "gtk3.tar.xz",
"setup": "setup.sh",

> +"unpack": true,
OK.

>  "size": 167175,
>  "digest": "0b71a936edf5bd70cf274aaa5d7abc8f77fe8e7b5593a208f805cc9436fac646b9c4f0b43c2b10de63ff3da671497d35536077ecbc72dba7f8159a38b580f831",
>  "algorithm": "sha512",
> +"filename": "sccache.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/linux64/clang.manifest

> +"clang_version": "r247539"
"version": "clang 3.8.0/r247539"

> +"size": 105219872,
> +"digest": "aa8de2fa535d0667e079019c475c631ea008f1bb5228505510867255b4d9c30663e2c97e579220a575a5887aa3bcf250021b50f76b90c2fa8c65a7aa19270066",
"size": 106877168,
"digest": "1c50c6348eaf429ed59bb603cff63bcc1f870f59216dd3c234db5b1156cfd351d5ee7b820ec31be4d2661eb4213b2e0030e2ba2782b42905d1ec19c7f8bd322a",

>  "algorithm": "sha512",

> +"filename": "clang.tar.bz2",
"filename": "clang.tar.xz",

> +"unpack": true

> +++ b/suite/config/tooltool-manifests/linux64/releng.manifest

>  "size": 80458572,
>  "digest": "e5101f9dee1e462f6cbd3897ea57eede41d23981825c7b20d91d23ab461875d54d3dfc24999aa58a31e8b01f49fb3140e05ffe5af2957ef1d1afb89fd0dfe1ad",
>  "algorithm": "sha512",
> +"filename": "gcc.tar.xz",
> +"unpack": true
OK.

>  "size": 12057960,
>  "digest": "6105d6432943141cffb40020dc5ba3a793650bdeb3af9bd5e56d3796c5f03df9962a73e521646cd71fbfb5e266c1e74716ad722fb6055589dfb7d35175bca89e",
"size": 12072532,
"digest": "3915f8ec396c56a8a92e6f9695b70f09ce9d1582359d1258e37e3fd43a143bc974410e4cfc27f500e095f34a8956206e0ebf799b7287f0f38def0d5e34ed71c9",

>  "algorithm": "sha512",
> +"filename": "gtk3.tar.xz",
"setup": "setup.sh",

> +"unpack": true
OK.

>  "size": 167175,
>  "digest": "0b71a936edf5bd70cf274aaa5d7abc8f77fe8e7b5593a208f805cc9436fac646b9c4f0b43c2b10de63ff3da671497d35536077ecbc72dba7f8159a38b580f831",
>  "algorithm": "sha512",
> +"filename": "sccache.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/macosx64/clang.manifest

> +"clang_version": "r247539"
"version": "clang 3.8.0/r247539"

> +"size": 121393888,
> +"digest": "6ae4e651e545538e6de326a7fb8b44b6e6d0b3acdb6a969ecb2b6f63b9995bbad2111cabf044ba575464f17f1f948d78ec92ad3a6922a7bfdf3ad6b6b2cad050",
Where do these numbers come from?
"size": 121389802,
"digest": "2be6b42cfa1e92de4b49a57123f54043fec2d3cf8385276516dc6aaed99c88768ac4aebd7ce2e007ab074163523da29223436a4d1aef82f0f750f08f1b14cd71",

> +"algorithm": "sha512",
> +"filename": "clang.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/macosx64/releng.manifest

>  "clang_version": "r247539"
"version": "clang 3.8.0/r247539"

>  "size": 121389802,
>  "digest": "2be6b42cfa1e92de4b49a57123f54043fec2d3cf8385276516dc6aaed99c88768ac4aebd7ce2e007ab074163523da29223436a4d1aef82f0f750f08f1b14cd71",
>  "algorithm": "sha512",
> +"filename": "clang.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/win32/l10n.manifest
>  "size": 266240,
>  "digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
>  "algorithm": "sha512",
>  "filename": "mozmake.exe"
OK.

> +++ b/suite/config/tooltool-manifests/win32/releng.manifest

>  "size": 266240,
>  "digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
>  "algorithm": "sha512",
>  "filename": "mozmake.exe"
OK.

> +++ b/suite/config/tooltool-manifests/win64/l10n.manifest

> +"size": 266240,
> +"digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
> +"algorithm": "sha512",
> +"filename": "mozmake.exe"
OK.

> +++ b/suite/config/tooltool-manifests/win64/releng.manifest

> +"size": 266240,
> +"digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
> +"algorithm": "sha512",
> +"filename": "mozmake.exe"
OK.

Comment 36

[Philip Chee]

>  https://hg.mozilla.org/SeaMonkey/puppet/rev/9efd7daf6f24
What is the difference between
https://hg.mozilla.org/SeaMonkey/puppet/manifests/seamonkey-config.pp
and
https://hg.mozilla.org/build/puppet/manifests/seamonkey-config.pp

Comment 37

[Edmund Wong (:ewong)]

(In reply to Philip Chee from comment #36)
> >  https://hg.mozilla.org/SeaMonkey/puppet/rev/9efd7daf6f24
> What is the difference between
> https://hg.mozilla.org/SeaMonkey/puppet/manifests/seamonkey-config.pp
> and
> https://hg.mozilla.org/build/puppet/manifests/seamonkey-config.pp

You don't need to review the puppet changes.  

the difference is that we no longer use build/puppet and we
strictly use SeaMonkey/puppet now.

Comment 38

[Edmund Wong (:ewong)]

(In reply to Philip Chee from comment #35)
> Comment on attachment 8735711 [details] [diff] [review]
> [c-a] tooltool manifest changes (v4)
> 
> > # HG changeset patch
> > # User Edmund Wong <[e-mail]>
> > # Parent  aa7262925765e09922407bb43f0af40569f72ef1
> > Bug 1190773 - comm-aurora manifest changes.
> 
> > +++ b/suite/config/tooltool-manifests/linux32/clang.manifest
> 
> > -"clang_version": "r170890"
> > +"clang_version": "r247539"
> "version": "clang 3.8.0/r247539"
> 
> > +"size": 105219872,
> > +"digest": "aa8de2fa535d0667e079019c475c631ea008f1bb5228505510867255b4d9c30663e2c97e579220a575a5887aa3bcf250021b50f76b90c2fa8c65a7aa19270066",
> >  "algorithm": "sha512",
> >  "filename": "clang.tar.bz2"
> "filename": "clang.tar.xz",
> 

the digest describes the bz2 file.  the xz requires
the digest and size to be changed as well. 

We might as well start using .xz. :)

I'll get an updated patch done.

Comment 40

[Edmund Wong (:ewong)]

incidentally, those #s come from the tooltool repository and they represent
the digest and the size of the filename, which is stored in the tooltool
repository as a sha512 hash.

Comment 41

[Edmund Wong (:ewong)]

We're supposed to move to VS2015u1 (or was it it u2?); but
our Win32 machines can't handle that (couldn't even have vs2013 installed on
them); but there'd be too many changes to include and the VS2015u1/2 change
needs its own bug since it requires tooltool manifest changes and
mozconfig changes.

Comment 42

[Edmund Wong (:ewong)]

Attachment: [c-c] tooltool manifest changes (v5) [checked-in]

Comment 43

[Philip Chee]

Comment on attachment 8737452 [details] [diff] [review]
[c-c] tooltool manifest changes (v5) [checked-in]

> +++ b/suite/config/tooltool-manifests/win32/releng.manifest
> -{
> -"size": 165226,
> -"digest": "79280f7595bc9e1613e05f8b2f0db3798ac739b96191e0f133e8ccd8ad149fedc84a1046e59863574189db28363a01712ae7b368ad1714e30ff88e7ebd5dad23",
> -"algorithm": "sha512",
> -"filename": "sccache.tar.bz2"
We don't need sccache here?

Comment 44

[Edmund Wong (:ewong)]

(In reply to Philip Chee from comment #43)
> Comment on attachment 8737452 [details] [diff] [review]
> [c-c] tooltool manifest changes (v5)
> 
> > +++ b/suite/config/tooltool-manifests/win32/releng.manifest
> > -{
> > -"size": 165226,
> > -"digest": "79280f7595bc9e1613e05f8b2f0db3798ac739b96191e0f133e8ccd8ad149fedc84a1046e59863574189db28363a01712ae7b368ad1714e30ff88e7ebd5dad23",
> > -"algorithm": "sha512",
> > -"filename": "sccache.tar.bz2"
> We don't need sccache here?

we don't use sccache on win32.

Comment 45

[Edmund Wong (:ewong)]

Comment on attachment 8736141 [details] [diff] [review]
[custom] custom code changes (v4) [checked-in]

Pushed to buildbotcustom:
 https://hg.mozilla.org/build/buildbotcustom/rev/9df5790d4725

Comment 46

[Edmund Wong (:ewong)]

Comment on attachment 8736142 [details] [diff] [review]
[configs] Update configs to support new tooltool.py (v3) [checked-in]

Pushed to buildbot-configs:
https://hg.mozilla.org/build/buildbot-configs/rev/6cfd37e02e84

Comment 47

[Edmund Wong (:ewong)]

Comment on attachment 8737452 [details] [diff] [review]
[c-c] tooltool manifest changes (v5) [checked-in]

Pushed to comm-central:
https://hg.mozilla.org/comm-central/rev/62a82225206d

Comment 48

[Edmund Wong (:ewong)]

Attachment: [puppet] update tooltool.py in packages.

Comment 49

[Edmund Wong (:ewong)]

Attachment: [puppet] update tooltool.py in packages. (v2) [checked-in]

Comment 50

[Edmund Wong (:ewong)]

Comment on attachment 8737684 [details] [diff] [review]
[puppet] update tooltool.py in packages. (v2) [checked-in]

Pushed to puppet:
https://hg.mozilla.org/SeaMonkey/puppet/rev/cdc5476c2064

Comment 51

[Edmund Wong (:ewong)]

Attachment: [custom] fix for custom code. change ShellCommand to MockCommand. (v1) [checked-in]

Comment 52

[Edmund Wong (:ewong)]

Comment on attachment 8737691 [details] [diff] [review]
[custom] fix for custom code. change ShellCommand to MockCommand. (v1) [checked-in]

Pushed to buildbotcustom:
https://hg.mozilla.org/build/buildbotcustom/rev/2418962d2be8

Comment 53

[Edmund Wong (:ewong)]

Attachment: [custom] fix for custom code. change ShellCommand to MockCommand. (v2) [checked-in]

Comment 54

[Edmund Wong (:ewong)]

Comment on attachment 8737692 [details] [diff] [review]
[custom] fix for custom code. change ShellCommand to MockCommand. (v2) [checked-in]

Pushed to buildbotcustom:

(backout of previous patch):
https://hg.mozilla.org/build/buildbotcustom/rev/4b80bebf2018

Pushed:
https://hg.mozilla.org/build/buildbotcustom/rev/09fe2c1233bb

Comment 55

[Edmund Wong (:ewong)]

Attachment: [custom] 2nd fix. Missed the ShellCommand->MockCommand change.

Comment 56

[Edmund Wong (:ewong)]

Attachment: [custom] fix for custom code. change ShellCommand to MockCommand. (v3) [checked-in]

Comment 57

[Edmund Wong (:ewong)]

Comment on attachment 8737696 [details] [diff] [review]
[custom] fix for custom code. change ShellCommand to MockCommand. (v3) [checked-in]

Pushed to buildbotcustom:

(backed out previous patch)
https://hg.mozilla.org/build/buildbotcustom/rev/937f5dfa6f68

(pushed)
https://hg.mozilla.org/build/buildbotcustom/rev/55aeadc24f9b

Comment 58

[Edmund Wong (:ewong)]

Attachment: [c-a] tooltool manifest changes (v5)

Comment 59

[Philip Chee]

Comment on attachment 8738004 [details] [diff] [review]
[c-a] tooltool manifest changes (v5)

Please fix clang.tar.bz2/clang.tar.xz before check-in
r=me for comm-aurora. a=me for CLOSED TREE

> +"version": "clang 3.8.0/r247539",
> +"size": 106877168,
> +"digest": "1c50c6348eaf429ed59bb603cff63bcc1f870f59216dd3c234db5b1156cfd351d5ee7b820ec31be4d2661eb4213b2e0030e2ba2782b42905d1ec19c7f8bd322a",
>  "algorithm": "sha512",
>  "filename": "clang.tar.bz2"
> +"unpack": true
This size/digest appears to be for "clang.tar.xz"

> +++ b/suite/config/tooltool-manifests/linux32/releng.manifest

>  "size": 80458572,
>  "digest": "e5101f9dee1e462f6cbd3897ea57eede41d23981825c7b20d91d23ab461875d54d3dfc24999aa58a31e8b01f49fb3140e05ffe5af2957ef1d1afb89fd0dfe1ad",
>  "algorithm": "sha512",
> -"filename": "gcc.tar.xz"
> +"filename": "gcc.tar.xz",
> +"unpack": true
OK.

> +"size": 11189216,
> +"digest": "18bc52b0599b1308b667e282abb45f47597bfc98a5140cfcab8da71dacf89dd76d0dee22a04ce26fe7ad1f04e2d6596991f9e5b01fd2aaaab5542965f596b0e6",
>  "algorithm": "sha512",
> +"filename": "gtk3.tar.xz",
> +"setup": "setup.sh",
> +"unpack": true,
OK.

>  "size": 167175,
>  "digest": "0b71a936edf5bd70cf274aaa5d7abc8f77fe8e7b5593a208f805cc9436fac646b9c4f0b43c2b10de63ff3da671497d35536077ecbc72dba7f8159a38b580f831",
>  "algorithm": "sha512",
> +"filename": "sccache.tar.bz2",
> +"unpack": true
OK

> +++ b/suite/config/tooltool-manifests/linux64/clang.manifest

> +"version": "clang 3.8.0/r247539",
> +"size": 106877168,
> +"digest": "1c50c6348eaf429ed59bb603cff63bcc1f870f59216dd3c234db5b1156cfd351d5ee7b820ec31be4d2661eb4213b2e0030e2ba2782b42905d1ec19c7f8bd322a",
>  "algorithm": "sha512",
> +"filename": "clang.tar.xz",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/linux64/releng.manifest
> +"version": "gcc 4.8.5",
>  "size": 80458572,
>  "digest": "e5101f9dee1e462f6cbd3897ea57eede41d23981825c7b20d91d23ab461875d54d3dfc24999aa58a31e8b01f49fb3140e05ffe5af2957ef1d1afb89fd0dfe1ad",
>  "algorithm": "sha512",
> +"filename": "gcc.tar.xz",
> +"unpack": true
OK.

>  "size": 12057960,
>  "digest": "6105d6432943141cffb40020dc5ba3a793650bdeb3af9bd5e56d3796c5f03df9962a73e521646cd71fbfb5e266c1e74716ad722fb6055589dfb7d35175bca89e",
>  "algorithm": "sha512",
> +"filename": "gtk3.tar.xz",
> +"setup": "setup.sh",
> +"unpack": true

Do we need to update gtk3 to this?
Bug 1242641 - Update gtk3 package in tooltool to specify the fontconfig path in the setup.sh
+"size": 12072532,
+"digest": "3915f8ec396c56a8a92e6f9695b70f09ce9d1582359d1258e37e3fd43a143bc974410e4cfc27f500e095f34a8956206e0ebf799b7287f0f38def0d5e34ed71c9",


>  "size": 167175,
>  "digest": "0b71a936edf5bd70cf274aaa5d7abc8f77fe8e7b5593a208f805cc9436fac646b9c4f0b43c2b10de63ff3da671497d35536077ecbc72dba7f8159a38b580f831",
>  "algorithm": "sha512",
> -"filename": "sccache.tar.bz2"
> +"filename": "sccache.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/macosx64/clang.manifest

> +"version": "clang 3.8.0/r247539",
> +"size": 121393888,
> +"digest": "6ae4e651e545538e6de326a7fb8b44b6e6d0b3acdb6a969ecb2b6f63b9995bbad2111cabf044ba575464f17f1f948d78ec92ad3a6922a7bfdf3ad6b6b2cad050",
> +"algorithm": "sha512",
> +"filename": "clang.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/macosx64/releng.manifest

> +"version": "clang 3.8.0/r247539",
>  "size": 121389802,
>  "digest": "2be6b42cfa1e92de4b49a57123f54043fec2d3cf8385276516dc6aaed99c88768ac4aebd7ce2e007ab074163523da29223436a4d1aef82f0f750f08f1b14cd71",
>  "algorithm": "sha512",
> -"filename": "clang.tar.bz2"
> +"filename": "clang.tar.bz2",
> +"unpack": true
OK.

> +++ b/suite/config/tooltool-manifests/win32/l10n.manifest
>  "size": 266240,
>  "digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
>  "algorithm": "sha512",
>  "filename": "mozmake.exe"
OK.

> +++ b/suite/config/tooltool-manifests/win32/releng.manifest

>  "size": 266240,
>  "digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
>  "algorithm": "sha512",
>  "filename": "mozmake.exe"
OK.

> +++ b/suite/config/tooltool-manifests/win64/l10n.manifest

> +"size": 266240,
> +"digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
> +"algorithm": "sha512",
> +"filename": "mozmake.exe"
OK.

> +++ b/suite/config/tooltool-manifests/win64/releng.manifest

> +"size": 266240,
> +"digest": "bb345b0e700ffab4d09436981f14b5de84da55a3f18a7f09ebc4364a4488acdeab8d46f447b12ac70f2da1444a68b8ce8b8675f0dae2ccf845e966d1df0f0869",
> +"algorithm": "sha512",
> +"filename": "mozmake.exe"
OK.

Comment 60

[Edmund Wong (:ewong)]

(In reply to Philip Chee from comment #59)
> Comment on attachment 8738004 [details] [diff] [review]
> [c-a] tooltool manifest changes (v5)
> 
> Please fix clang.tar.bz2/clang.tar.xz before check-in
> r=me for comm-aurora. a=me for CLOSED TREE
> 
> > +"version": "clang 3.8.0/r247539",
> > +"size": 106877168,
> > +"digest": "1c50c6348eaf429ed59bb603cff63bcc1f870f59216dd3c234db5b1156cfd351d5ee7b820ec31be4d2661eb4213b2e0030e2ba2782b42905d1ec19c7f8bd322a",
> >  "algorithm": "sha512",
> >  "filename": "clang.tar.bz2"
> > +"unpack": true
> This size/digest appears to be for "clang.tar.xz"
> 

My bad.  fixed.


> >  "size": 12057960,
> >  "digest": "6105d6432943141cffb40020dc5ba3a793650bdeb3af9bd5e56d3796c5f03df9962a73e521646cd71fbfb5e266c1e74716ad722fb6055589dfb7d35175bca89e",
> >  "algorithm": "sha512",
> > +"filename": "gtk3.tar.xz",
> > +"setup": "setup.sh",
> > +"unpack": true
> 
> Do we need to update gtk3 to this?
> Bug 1242641 - Update gtk3 package in tooltool to specify the fontconfig path
> in the setup.sh
> +"size": 12072532,
> +"digest":
> "3915f8ec396c56a8a92e6f9695b70f09ce9d1582359d1258e37e3fd43a143bc974410e4cfc27
> f500e095f34a8956206e0ebf799b7287f0f38def0d5e34ed71c9",
> 

Fixed.

Comment 61

[Edmund Wong (:ewong)]

Attachment: [c-a] tooltool manifest changes (v6) [checked-in]

Comment 62

[Edmund Wong (:ewong)]

Comment on attachment 8739285 [details] [diff] [review]
[c-a] tooltool manifest changes (v6) [checked-in]

[Approval Request Comment]
Regression caused by (bug #): 
User impact if declined: 
Testing completed (on m-c, etc.): 
Risk to taking this patch (and alternatives if risky):
String changes made by this patch:

Comment 63

[Edmund Wong (:ewong)]

(In reply to Edmund Wong (:ewong) from comment #62)
> Comment on attachment 8739285 [details] [diff] [review]
> [c-a] tooltool manifest changes (v6)
> 
> [Approval Request Comment]
> Regression caused by (bug #): 
> User impact if declined: cannot build in automation once the old tooltool repo is gone.
> Testing completed (on m-c, etc.): 
> Risk to taking this patch (and alternatives if risky):
> String changes made by this patch: none

Comment 64

[Edmund Wong (:ewong)]

https://hg.mozilla.org/releases/comm-aurora/rev/c21a3b196cf4f771fc3083d52409a3099fb2a48e
Bug 1190773 - tooltool manifest changes to support new tooltool repo. r+a=RattyAway

Comment 65

[Edmund Wong (:ewong)]

Comment on attachment 8739285 [details] [diff] [review]
[c-a] tooltool manifest changes (v6) [checked-in]

Pushed to comm-aurora:
https://hg.mozilla.org/releases/comm-aurora/rev/c21a3b196cf4

Comment 66

[Edmund Wong (:ewong)]

Attachment: [c-a] tooltool manifest fix (v1) [checked-in]

bustagefix

Comment 67

[Edmund Wong (:ewong)]

https://hg.mozilla.org/releases/comm-aurora/rev/54445fccee69ad5133897ecc49684a2e3758be5a
Bug 1190773 - Fix tooltool linux32 manifest. r+a=bustagefix

Comment 68

[Edmund Wong (:ewong)]

Comment on attachment 8744771 [details] [diff] [review]
[c-a] tooltool manifest fix (v1) [checked-in]

Pushed to comm-aurora:
https://hg.mozilla.org/releases/comm-aurora/rev/54445fccee69

Comment 69

[Edmund Wong (:ewong)]

Attachment: [c-r] tooltool manifest changes(v5)

Comment 70

[Edmund Wong (:ewong)]

(In reply to Philip Chee from comment #34)
> (In reply to Edmund Wong (:ewong) from comment #22)
> > Created attachment 8735717 [details] [diff] [review]
> > [c-r] tooltool manifest changes. (v4)
> This appears to be identical to the comm-beta patch. Is this intentional?

No.  Mistake on my part. Just fixed that.

Comment 71

[Edmund Wong (:ewong)]

Comment on attachment 8735715 [details] [diff] [review]
[c-b] tooltool manifest changes (v4)

With the recent merge from c-a -> c-b, this patch is no longer needed.
Just need c-r to be reviewed.

Comment 72

[Philip Chee]

Comment on attachment 8744813 [details] [diff] [review]
[c-r] tooltool manifest changes(v5)

This doesn't apply to comm-release tip. It appears to come from a 2.40 build?

Comment 73

[Edmund Wong (:ewong)]

(In reply to Philip Chee from comment #72)
> Comment on attachment 8744813 [details] [diff] [review]
> [c-r] tooltool manifest changes(v5)
> 
> This doesn't apply to comm-release tip. It appears to come from a 2.40 build?

Ah, right.  ;/ Sorry.

Comment 74

[Edmund Wong (:ewong)]

Attachment: [c-r] tooltool manifest changes (v6)

Comment 75

[Edmund Wong (:ewong)]

Attachment: [c-r] tooltool manifest changes (v6)

the VS changes will be done in Bug 1092468. (mainly because we're not
ready to use VS2015u2 yet.)

Comment 76

[Philip Chee]

Comment on attachment 8746901 [details] [diff] [review]
[c-r] tooltool manifest changes (v6)

r=me with the missing newlines fixed and sccache either restored or explained why we don't need this.

> suite/config/tooltool-manifests/linux32/releng.manifest

> +"version": "gcc 4.8.5",
> +"size": 81065660,
> +"digest": "db26f498ab56a3b5c65d7cda290cbb74174af9f2d021ca9c158f53b0382924ccf5ed9638d41eef449434aa9383a9113994d9729d9dd910321d1f35f9411eae38",
>  "algorithm": "sha512",
> -"filename": "setup.sh"
> +"filename": "gcc.tar.xz",
> +"unpack": "true"
I don't know if this makes a difference but in other parts of this file and in other manifests true is not "quoted"

> suite/config/tooltool-manifests/linux64/clang.manifest
Ah we don't use sccache here I remember.
> suite/config/tooltool-manifests/macosx64/releng.manifest
But we do here? What are the reasons we do or don't use sccache?

> suite/config/tooltool-manifests/win32/l10n.manifest
> -]
> +]
> \ No newline at end of file
Please add back all the missing NL at the ends of all the files you touched.

> suite/config/tooltool-manifests/win32/releng.manifest
> -"size": 165226,
> -"digest": "79280f7595bc9e1613e05f8b2f0db3798ac739b96191e0f133e8ccd8ad149fedc84a1046e59863574189db28363a01712ae7b368ad1714e30ff88e7ebd5dad23",
> -"algorithm": "sha512",
> -"filename": "sccache.tar.bz2"
What reason are we removing this? Thunderbird and Firefox both still have this.

> suite/config/tooltool-manifests/win64/l10n.manifest
> suite/config/tooltool-manifests/win64/releng.manifest
Missing NewLines?
Missing sccache?

Comment 77

[Edmund Wong (:ewong)]

Attachment: [c-r] tooltool manifests (v7)

Comment 78

[Edmund Wong (:ewong)]

Attachment: [c-r] tooltool manifests (v7)

Comment 79

[Philip Chee]

Comment on attachment 8747586 [details] [diff] [review]
[c-r] tooltool manifests (v7)

I don't know what happened but your linux32/releng.manifest and linux64/releng.manifest look totally broken.

Comment 80

[Edmund Wong (:ewong)]

Attachment: [c-r] tooltool manifest changes (v7c) (for real) r+Ratty a+Ratty [checked-in]

Comment 81

[Philip Chee]

Comment on attachment 8748649 [details] [diff] [review]
[c-r] tooltool manifest changes (v7c) (for real) r+Ratty a+Ratty [checked-in]

r=Ratty a=Ratty for comm-release (with minor fixes

Comment 82

[Philip Chee]

> [c-r] tooltool manifest changes (v7c) (for real) r+Ratty a+Ratty
> r=Ratty a=Ratty for comm-release (with minor fixes)
http://hg.mozilla.org/releases/comm-release/rev/07a4595692ad

Comment 83

[Edmund Wong (:ewong)]

Apart from the post-land-review patches, should I consider this bug fixed?

Puppet, build-configs, buildbotcustom and the tooltool manifest changes
have all been checked in.

Am I missing anything?