1399320 - Update rich icons to be httpdisabled:// to prevent network connections

Status: RESOLVED FIXED

(Testing :: Talos, enhancement)

Description

[Ed Lee :Mardak]

Bug 1352459 is adding fetching of rich icons including <link rel="apple-touch-icon"/>. Various links for icons already point to local files or httpdisabled:// urls but not all, e.g.,

en.wikipedia.org/en.wikipedia.org/wiki/Rorschach_test.html:<link rel="apple-touch-icon" href="http://en.wikipedia.org/apple-touch-icon.png" />

Comment 1

[Ed Lee :Mardak]

Attachment: replace with httpdisabled:

Comment 2

[Ed Lee :Mardak]

How can I test this locally?

./mach talos-test seems to run into

FATAL -   File "mozilla-central/testing/mozharness/mozharness/mozilla/testing/talos.py", line 271, in query_pagesets_name
FATAL -     if self.query_talos_json_config() and self.suite is not None:
FATAL - AttributeError: 'Talos' object has no attribute 'suite'

Comment 3

[Joel Maher ( :jmaher ) (UTC -0800)]

Comment on attachment 8907399 [details] [diff] [review]
replace with httpdisabled:

Review of attachment 8907399 [details] [diff] [review]:
-----------------------------------------------------------------

we will need to package this up as a new tp5n.zip and get it uploaded to tooltool.  I have been meaning to add tp5n.zip to https://github.com/rwood-moz/talos-pagesets for easier access, you could add it there, then edit mozharness to not use tooltool but fetch it from github instead:
http://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/talos.py#398

and an example in the same file of downloading from github:
http://searchfox.org/mozilla-central/source/testing/mozharness/mozharness/mozilla/testing/talos.py#460

with those changes in a patch, you could test on try :)

Comment 4

[Ed Lee :Mardak]

I pushed this to try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=b3eb95bca1be5891dd40ec773c38b1e41b4a0081

Here's the talos.py changes for my testing:
https://hg.mozilla.org/try/rev/b3eb95bca1be5891dd40ec773c38b1e41b4a0081

It points to a commit that has the patch from this bug applied on top of the tp5n PR:
https://github.com/rwood-moz/talos-pagesets/pull/2

Assuming enough rich icons are disabled, what's the process for getting the new tp5n.zip uploaded to tooltool? The releng documentation seems to point to non-existent people.mozilla

Comment 5

[Joel Maher ( :jmaher ) (UTC -0800)]

all that is needed is to file a bug to get tp5n.zip uploaded to tooltool (see prior art in bug 1121049).  I will check on this in a few hours and try to help out where I can to make this happen.

Comment 6

[Ed Lee :Mardak]

jmaher: Hurray! Finally got the test to even "build" by deleting enough special character files:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ab8413c2d4ab011b0c081407ddd8f33332a91415&selectedJob=130738587

I suppose somebody else will need to apply attachment 8907399 [details] [diff] [review] to some more official copy and get it to tooltool.

For anybody's future reference:

https://hg.mozilla.org/try/rev/ab8413c2d4ab011b0c081407ddd8f33332a91415
```diff
-                manifest_file = os.path.join(self.talos_path, 'tp5n-pageset.manifest')
+                tp5n_zip_url = 'https://github.com/Mardak/talos-pagesets/blob/rm-badfiles/tp5n.zip?raw=true&tp5n.zip=/tp5n.zip'
+                self.download_unpack(tp5n_zip_url, src_talos_pageset)
```

Make sure the url ends with the desired file extension. Otherwise download_unpack thinks the extension is "zip?raw=true". See https://treeherder.mozilla.org/logviewer.html#?job_id=130697054&repo=try&lineNumber=569

Also, download_unpack eventually calls python's zipfile.py and blows up on special character filenames, e.g., those with "?" or "@". See https://treeherder.mozilla.org/logviewer.html#?job_id=130723371&repo=try&lineNumber=573

Comment 7

[Joel Maher ( :jmaher ) (UTC -0800)]

:Mardak, thanks for doing this- I think there were a few edge cases that you ended up fixing.  Lets take your changes and build a tp5n.zip, then get it on tooltool, I will get a bug on file for that!

Comment 8

[Ed Lee :Mardak]

Attachment: Bug 1399320 - Update rich icons to be httpdisabled:// to prevent network connections.

Review commit: https://reviewboard.mozilla.org/r/181080/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/181080/

Comment 9

[Joel Maher ( :jmaher ) (UTC -0800)]

Comment on attachment 8909607 [details]
Bug 1399320 - Update rich icons to be httpdisabled:// to prevent network connections.

https://reviewboard.mozilla.org/r/181080/#review186344

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/6c1c2d830119
Update rich icons to be httpdisabled:// to prevent network connections. r=jmaher

Comment 11

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/6c1c2d830119