1543026 - enable sccache on windows gcp builders

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: OpenCloudConfig, task)

Description

[Rob Thijssen [:grenade (EET/UTC+0300)]]

thanks to ted's documenting sccache support for gcp in bug 1529951, comment 11, it might be simple to enable sccache on windows gcp builders.

let's give it a shot.

Comment 1

[Rob Thijssen [:grenade (EET/UTC+0300)]]

the windows gcp builders now have all the pieces required to use sccache

• buckets exist ccgc/storage/browser
• iam level 1, 2 & 3 roles exist and are applied to the workers when provisioned
  next step is to attempt some try pushes where we make use of ted's step 1 instructions in bug 1529951, comment 11.

Comment 2

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=f1acdf6
https://treeherder.mozilla.org/#/jobs?repo=try&tier=1%2C2%2C3&revision=b3fe62c

Comment 3

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Attachment: enable sccache on gcp

this change comprises the in-tree changes required to make use of sccache in gcp.
specifically:

• a gcp metadata lookup for availability-zone is added to mozconfig, enabling a build to determine its regional gcp sccache bucket
• the sccache cargo build command is modified to include the gcs feature when the environment contains gcs configuration

note that further changes are required on infra to support sccache use. the required changes already exist and are enabled for gcp windows infra, including:

• a json credential file on the build instance filesystem, containing credentials valid for the appropriate scm level bucket for the gcp region
• an SCCACHE_GCS_KEY_PATH env variable containing the path to the json credential file
• an SCCACHE_GCS_RW_MODE env variable containg the text READ_WRITE
• sccache buckets must exist for each region and scm levels 1 & 3
• credentials for scm level 1 buckets must not be valid for scm level 3 buckets

on gcp systems which do not contain credential files and the above mentioned env variables (eg gecko-[1-3]-b-linux), sccache should fail gracefully without breaking builds.

Comment 4

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c56116718a33499f0b98c3d3e09555b95c5f1c9f

Comment 5

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0ce6de8a8523170ecf438e0dbc6fcf6506bad59f

Comment 6

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=e8f96d8bfcb18b67c58154f358daf8dbcf62a3e9

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=6a088c18c5018e6e444112a1df4c0edf78e99c11

Comment 8

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=324c4eefc78c9b571b5bbf83d438c0f926a904c1

Comment 9

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=cb7b1b61250c9ba6f9e78dbf767acd60d9a9d2c9

Comment 10

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=a90acfefbf6dfdcf242891ca258dfeb3a304c46b

Comment 11

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=c0f3d3d63d24a7c89b82519cddea15962659c168

Comment 12

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=8cec9e76a8ceee44046eaf945660e36917370232

Comment 13

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=effbc8e24ca93658a1a6af7c4c34472831e6f9c4

Comment 14

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=b6084df1423881d6952aa444ea91b276cee1167c

Comment 15

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=9687c2b76ba86f83e97900eca9f945abaac564f3

Comment 16

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f693109749369f84b15db68d91d65a082c42e722

Comment 17

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=17617939ee005aef5d212aedceb450e83014cf28

Comment 18

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=dc485bf076faee03b193d6c5a0bff0393e79c5ba

Comment 19

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=edc42e19e39d56a3e85eb067c4ddf4ac73cc9bab

Comment 20

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=3e9e7c301523db66d86c6f6d37806a727e8b97cb

Comment 21

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=37d9d5f3e4451e153634bba909e313895d65180c

Comment 22

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=2d2eb48bae05cee1133b44924b0328a22a723590

Comment 23

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=85063bde04d9bf57943c272f43a4f5b8fa1ea7cf

Comment 24

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=38dd5c07bc420131e2d76f730d2702b0c3a0e8da

Comment 25

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=4a6e7710244208cd6fbab850b03da80d320324de

Comment 26

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=f3cbcd0485fd7ce3ec873803e7d716b8a2047eb7

Comment 27

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=d3a18c1d409203027a3e53c8667e46d7255250d5

Comment 28

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=796907523f03069e1b6be4cbb575bf332788dd75

Comment 29

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=0cb7085b7d9dd155c6d7a23a1667f96d6f922b05

Comment 30

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=495945c372af514e6c871790ae72ca42e747d25a

Comment 31

[Rob Thijssen [:grenade (EET/UTC+0300)]]

Comment on attachment 9062135 [details]
enable sccache on gcp

r+ in phabricator

Comment 32

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/5b08dd3eeec9
enable sccache on gcp r=wcosta

Comment 33

[Raul Gurzau (:RaulG)]

https://hg.mozilla.org/mozilla-central/rev/5b08dd3eeec9