1563785 - Enable sparse checkout for Windows workers in hgrc

Status: RESOLVED FIXED

(Infrastructure & Operations :: RelOps: OpenCloudConfig, task)

Description

[Michael Shal [:mshal]]

Similar to bug 1517831 that did this for mac, I'd like to try using a sparse hg profile in Windows to help bug 1561474.

Comment 1

[Michael Shal [:mshal]]

Attachment: GitHub Pull Request

Is there a way to test this before deploying?

Comment 2

[Rob Thijssen [:grenade (EET/UTC+0300)]]

yes, deploying to beta worker types now.
when deployment is complete, you can use a try push that includes a patch which redirects tasks to beta workers like this: https://hg.mozilla.org/try/rev/cd19190, in conjunction with code that makes use of sparse checkout (if more than having the extension in config is required).

Comment 3

[Rob Thijssen [:grenade (EET/UTC+0300)]]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=510c8f84da45189dcd5503e5dcad8f01cf32ca9c

Comment 4

[Rob Thijssen [:grenade (EET/UTC+0300)]]

not sure what happened in my try push above.
the mercurial.ini config file contains the change, however the build logs show:

14:33:03     INFO -  checking for hg... c:/PROGRA~1/Mercurial/hg.exe
14:33:03     INFO -  checking for Mercurial version... 4.7.1
14:33:03     INFO -  checking for sparse checkout... no

so, it would be interesting to work out what the sparse checkout check is doing. eg: is it looking to see if the mercurial config has the extension enabled, or is it checking something else?

Comment 5

[Michael Shal [:mshal]]

Looks like it's just checking if the current checkout was sparse, not whether hg itself supports sparse checkouts: https://searchfox.org/mozilla-central/rev/f372470e10c8cb0691681603a1d6324dee5b3b8a/python/mozversioncontrol/mozversioncontrol/__init__.py#295

I'll give my patch a test on try using your beta-worker patch and report back.

Comment 6

[Michael Shal [:mshal]]

Looks like it's working as far as I can tell: https://taskcluster-artifacts.net/JMfPQKDxRty9JrMEeNsTgQ/0/public/logs/live_backing.log

Apparently the "warning: sparse profile 'build/sparse-profiles/profile-generate' not found in rev 000000000000 - ignoring it" message is spurious and can be ignored. What's it take to go from beta to release workers?

Comment 7

[Rob Thijssen [:grenade (EET/UTC+0300)]]

i'ts just an ami deployment triggered by commit syntax in the occ repo.
since having the extension enabled makes no difference to builds if they aren't attempting sparse checkout, i have gone ahead with deployment to the windows try builders (gecko-1-b-win2012).
https://github.com/mozilla-releng/OpenCloudConfig/commit/878fc40
https://tools.taskcluster.net/groups/CIxrjOLnSqSf4SRE9h6cwQ/tasks/dch_0PMzRIGT71iNssKQ_w/details
^ when this task finishes, you shouldn't need the beta builder patch/hack to test on try anymore
we can probably do the others on utc monday morning.

Comment 8

[Rob Thijssen [:grenade (EET/UTC+0300)]]

deployment to gecko-2-b-win2012 and gecko-3-b-win2012 in progress...
https://tools.taskcluster.net/groups/WYaXv0NiTR2YdrzzvcqNHQ

Comment 9

[Michael Shal [:mshal]]

Is there anything else we need to do here or can this be marked fixed?

Comment 10

[Rob Thijssen [:grenade (EET/UTC+0300)]]

all deployments are complete. it should be possible to use sparse on any windows worker.

Comment 11

[Michael Shal [:mshal]]

Great, thanks!