Closed
Bug 1417497
Opened 6 years ago
Closed 6 years ago
Sign the Windows code coverage build
Categories
(Testing :: Code Coverage, enhancement)
Testing
Code Coverage
Tracking
(firefox60 fixed)
RESOLVED
FIXED
mozilla60
Tracking | Status | |
---|---|---|
firefox60 | --- | fixed |
People
(Reporter: marco, Assigned: marco)
References
Details
Attachments
(2 files, 1 obsolete file)
4.01 KB,
patch
|
mozilla
:
review+
catlee
:
checked-in+
|
Details | Diff | Splinter Review |
3.89 KB,
patch
|
mozilla
:
review+
|
Details | Diff | Splinter Review |
This is needed for some xpcshell tests. The build definition is being added in bug 1417436, but signing is not done yet (https://bugzilla.mozilla.org/page.cgi?id=splinter.html&bug=1417436&attachment=8928507). Aki, can you help?
Flags: needinfo?(aki)
Comment 1•6 years ago
|
||
I think you need to add the appropriate labels (e.g. win64-ccov/debug ?) to this tuple: https://hg.mozilla.org/mozilla-central/file/tip/taskcluster/taskgraph/loader/build_signing.py#l11
Flags: needinfo?(aki)
Assignee | ||
Comment 2•6 years ago
|
||
This patch enables signing for the Windows coverage build. The build signing task fails though, as the server doesn't accept the xul.dll file as it's too big: > 2017-11-16 12:14:48,583 - signingscript.sign - INFO - sign_file(): signing /builds/scriptworker/work/zipelqhj91g/firefox/xul.dll with sha2signcode... > 2017-11-16 12:14:48,583 - signingscript.sign - INFO - Certificate types: ['project:releng:signing:cert:dep-signing'] > 2017-11-16 12:14:48,583 - signingscript.utils - INFO - Running "/builds/scriptworker/bin/signtool -v -n /builds/scriptworker/work/nonce -t /builds/scriptworker/work/token -c /builds/scriptworker/lib/python3.5/site-packages/signingscript/data/host.cert -H signing4.srv.releng.scl3.mozilla.com:9110 -H signing5.srv.releng.scl3.mozilla.com:9110 -H signing6.srv.releng.scl3.mozilla.com:9110 -f sha2signcode -o /builds/scriptworker/work/zipelqhj91g/firefox/xul.dll /builds/scriptworker/work/zipelqhj91g/firefox/xul.dll" > 2017-11-16 12:14:48,586 - signingscript.utils - INFO - COMMAND OUTPUT: > 2017-11-16 12:14:48,837 - signingscript.utils - INFO - 2017-11-16 12:14:48,836 - in / > 2017-11-16 12:14:48,837 - signingscript.utils - INFO - 2017-11-16 12:14:48,837 - doing sha2signcode signing > 2017-11-16 12:14:48,837 - signingscript.utils - INFO - 2017-11-16 12:14:48,837 - possible hosts are ['https://signing4.srv.releng.scl3.mozilla.com:9110', 'https://signing6.srv.releng.scl3.mozilla.com:9110', 'https://signing5.srv.releng.scl3.mozilla.com:9110'] > 2017-11-16 12:14:48,837 - signingscript.utils - INFO - 2017-11-16 12:14:48,837 - /builds/scriptworker/work/zipelqhj91g/firefox/xul.dll > 2017-11-16 12:14:48,838 - signingscript.utils - INFO - 2017-11-16 12:14:48,837 - checking /builds/scriptworker/work/zipelqhj91g/firefox/xul.dll for signature... > 2017-11-16 12:14:49,711 - signingscript.utils - INFO - 2017-11-16 12:14:49,711 - 42972962ad4f1818bc268d91a2c7945564154680: processing /builds/scriptworker/work/zipelqhj91g/firefox/xul.dll on https://signing4.srv.releng.scl3.mozilla.com:9110 > 2017-11-16 12:14:49,711 - signingscript.utils - INFO - 2017-11-16 12:14:49,711 - 42972962ad4f1818bc268d91a2c7945564154680: GET https://signing4.srv.releng.scl3.mozilla.com:9110/sign/sha2signcode/42972962ad4f1818bc268d91a2c7945564154680 > 2017-11-16 12:14:49,714 - signingscript.utils - INFO - 2017-11-16 12:14:49,714 - Starting new HTTPS connection (1): signing4.srv.releng.scl3.mozilla.com > 2017-11-16 12:14:50,082 - signingscript.utils - INFO - 2017-11-16 12:14:50,082 - https://signing4.srv.releng.scl3.mozilla.com:9110 "GET /sign/sha2signcode/42972962ad4f1818bc268d91a2c7945564154680 HTTP/1.1" 404 None > 2017-11-16 12:14:50,193 - signingscript.utils - INFO - 2017-11-16 12:14:50,193 - 42972962ad4f1818bc268d91a2c7945564154680: uploading for signing > 2017-11-16 12:14:52,417 - signingscript.utils - INFO - 2017-11-16 12:14:52,417 - Starting new HTTPS connection (1): signing4.srv.releng.scl3.mozilla.com > 2017-11-16 12:16:11,383 - signingscript.utils - INFO - 2017-11-16 12:16:11,382 - https://signing4.srv.releng.scl3.mozilla.com:9110 "POST /sign/sha2signcode HTTP/1.1" 400 0 > 2017-11-16 12:16:11,389 - signingscript.utils - INFO - 2017-11-16 12:16:11,384 - 42972962ad4f1818bc268d91a2c7945564154680: error uploading file for signing: 400 Client Error: File too large for url: https://signing4.srv.releng.scl3.mozilla.com:9110/sign/sha2signcode > 2017-11-16 12:16:11,389 - signingscript.utils - INFO - Traceback (most recent call last): > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - File "/builds/scriptworker/lib/python3.5/site-packages/signtool/signing/client.py", line 86, in remote_signfile > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - r.raise_for_status() > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - File "/builds/scriptworker/lib/python3.5/site-packages/requests/models.py", line 937, in raise_for_status > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - raise HTTPError(http_error_msg, response=self) > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://signing4.srv.releng.scl3.mozilla.com:9110/sign/sha2signcode/42972962ad4f1818bc268d91a2c7945564154680 > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - During handling of the above exception, another exception occurred: > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - Traceback (most recent call last): > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - File "/builds/scriptworker/lib/python3.5/site-packages/signtool/signing/client.py", line 137, in remote_signfile > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - r.raise_for_status() > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - File "/builds/scriptworker/lib/python3.5/site-packages/requests/models.py", line 937, in raise_for_status > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - raise HTTPError(http_error_msg, response=self) > 2017-11-16 12:16:11,390 - signingscript.utils - INFO - requests.exceptions.HTTPError: 400 Client Error: File too large for url: https://signing4.srv.releng.scl3.mozilla.com:9110/sign/sha2signcode Here's the try build: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f036c0c2e7fdc0fb58de296649a845bd4ef0ce71&selectedJob=145294162.
Flags: needinfo?(aki)
Comment 3•6 years ago
|
||
We have a max_filesize_sha2signcode = 157286400, or ~150 mb. We have a ccov xul.dll that's 480mb. We've adjusted the max filesize before, but I'm not sure we've ever increased it more than 3x the current limit. Asking around about this limit. In the meantime, it may be worth asking if it's possible to set a flag in xpcshell to not require the signature.
Flags: needinfo?(aki)
Assignee | ||
Comment 4•6 years ago
|
||
(In reply to Aki Sasaki [:aki] from comment #3) > We have a max_filesize_sha2signcode = 157286400, or ~150 mb. > We have a ccov xul.dll that's 480mb. > > We've adjusted the max filesize before, but I'm not sure we've ever > increased it more than 3x the current limit. Asking around about this limit. > > In the meantime, it may be worth asking if it's possible to set a flag in > xpcshell to not require the signature. I think we can, but we would need to disable a few tests, which means not collecting coverage for them. We are trying to get a coverage build that is as close as possible to a normal debug build, as otherwise users of the coverage data will be confused.
Comment 5•6 years ago
|
||
Ben, Chris, what's the main reason we're setting max_filesize ? Security? Making sure we don't run out of disk?
Flags: needinfo?(catlee)
Flags: needinfo?(bhearsum)
Assignee | ||
Comment 6•6 years ago
|
||
Compressed as ZIP, it is ~150 MB.
Comment 7•6 years ago
|
||
(In reply to Aki Sasaki [:aki] (back nov27) from comment #5) > Ben, Chris, what's the main reason we're setting max_filesize ? Security? > Making sure we don't run out of disk? I don't have a clear memory on this, but my suspicion is security.
Flags: needinfo?(bhearsum)
Assignee | ||
Comment 8•6 years ago
|
||
Can we lift this limit? Or upload compressed files?
Assignee | ||
Comment 9•6 years ago
|
||
I spoke with catlee on IRC. He has concerns increasing the file limit would slow things down. The Windows coverage build only runs on mozilla-central pushes (and on try only when explicitly requested), so the overhead might be negligible compared to the other builds which are done on every commit on inbound, try, autoland. Catlee, does this resolve your concern? Otherwise, can somebody look into uploading compressed files or help me do it?
Assignee | ||
Comment 10•6 years ago
|
||
Aki, Ben, Chris, can you help me figure out the next step here?
Flags: needinfo?(bhearsum)
Flags: needinfo?(aki)
Comment 11•6 years ago
|
||
(In reply to Marco Castelluccio [:marco] from comment #9) > I spoke with catlee on IRC. He has concerns increasing the file limit would > slow things down. > > The Windows coverage build only runs on mozilla-central pushes (and on try > only when explicitly requested), so the overhead might be negligible > compared to the other builds which are done on every commit on inbound, try, > autoland. > > Catlee, does this resolve your concern? Otherwise, can somebody look into > uploading compressed files or help me do it? The limit is for all windows files on all branches, so raising the limit would allow for these files to grow significantly without error. Someone could potentially allow for signing locally with a dev/testing cert.. the downstream tests would have to accept that testing cert. If we have to keep the filesize limit on the signing server for security reasons, that may be our next most likely solution.
Flags: needinfo?(aki)
Updated•6 years ago
|
Flags: needinfo?(bhearsum)
Comment 12•6 years ago
|
||
If we can figure out a way to restrict this size increase to the dep signing servers, that would be best.
Flags: needinfo?(catlee)
Comment 13•6 years ago
|
||
Attachment #8947200 -
Flags: review?(aki)
Updated•6 years ago
|
Attachment #8947200 -
Flags: review?(aki) → review+
Updated•6 years ago
|
Attachment #8947200 -
Flags: checked-in+
Assignee | ||
Comment 14•6 years ago
|
||
Thanks :catlee!
Attachment #8928981 -
Attachment is obsolete: true
Attachment #8947306 -
Flags: review?(aki)
Updated•6 years ago
|
Attachment #8947306 -
Flags: review?(aki) → review+
Comment 15•6 years ago
|
||
Pushed by mcastelluccio@mozilla.com: https://hg.mozilla.org/integration/mozilla-inbound/rev/5b713577bd83 Enable signing for the Windows code coverage build. r=aki
Comment 16•6 years ago
|
||
bugherder |
https://hg.mozilla.org/mozilla-central/rev/5b713577bd83
Status: NEW → RESOLVED
Closed: 6 years ago
status-firefox60:
--- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla60
Updated•5 years ago
|
Assignee: nobody → mcastelluccio
You need to log in
before you can comment on or make changes to this bug.
Description
•