[FastBlock] write an automated test for the platform "fastblock" feature from bug 1474280

RESOLVED FIXED in Firefox 63

Status

()

enhancement
P1
normal
RESOLVED FIXED
Last year
11 months ago

People

(Reporter: mayhemer, Assigned: xeonchen)

Tracking

(Blocks 1 bug)

Trunk
mozilla64
Points:
---
Dependency tree / graph
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(firefox63 fixed, firefox64 fixed)

Details

(Whiteboard: [necko-triaged])

Attachments

(3 attachments)

As this feature is hard to test in live conditions a controlled mochitest seems more appropriate.

I think what we want is to make sure that all following types of requests are left to go when opened before the preferred FastBlock delay and blocked (immediately cancelled) when opened after the FastBlock delay:

- iframe document loads
- delayed (dynamically) added iframe document loads
- a sub-resource (<img> or XHR) referred from a delayed added iframe
- a redirected request (a simple <img> request or XHR from the top-level document that returns a delayed redirect response)

(the two delayed iframe tests make sure we are really using the top-level navigation timestamp and not the iframe's load start timestamp)
This test should try the following conditions:

- browser.contentblocking.enabled = true; browser.fastblock.enabled = true (fastblock ON)
- browser.contentblocking.enabled = false; browser.fastblock.enabled = true (fastblock OFF)
- browser.contentblocking.enabled = true; browser.fastblock.enabled = false (fastblock OFF)
Depends on: 1480448
Also, this test should include a check that shows that FastBlock honors user overrides (bug 1481250).
See Also: → 1481250
Basic parts of FastBlock tests, no iframe/xhr tests in this patch.
Comment on attachment 9005141 [details]
Bug 1477046 - Part 1: Add basic FastBlock automation tests; r=mayhemer

Honza Bambas (:mayhemer) has approved the revision.
Attachment #9005141 - Flags: review+
Comment on attachment 9005646 [details]
Bug 1477046 - Part 2: Add FastBlock iframe tests; r=mayhemer

Honza Bambas (:mayhemer) has approved the revision.
Attachment #9005646 - Flags: review+
Pushed by xeonchen@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/757bbd2e3176
Part 1: Add basic FastBlock automation tests; r=mayhemer
Pushed by xeonchen@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/dcbe562ce87a
Part 2: Add FastBlock iframe tests; r=mayhemer
Backed out for failing eslint at builds/worker/checkouts/gecko/toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html

Push with failure: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=757bbd2e31766ef88dabab6b89e5516ec223383f

Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=199063556&repo=autoland&lineNumber=282

Backout: https://hg.mozilla.org/integration/autoland/rev/0c938920ecdf88b9fe1f578ce720e45d1a3c8c9d
Flags: needinfo?(xeonchen)
Other failures appeared such as https://treeherder.mozilla.org/logviewer.html#?job_id=199066639&repo=autoland&lineNumber=3840

[task 2018-09-13T12:38:05.645Z] 12:38:05     INFO - TEST-START | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html
[task 2018-09-13T12:38:06.816Z] 12:38:06     INFO - TEST-INFO | started process screentopng
[task 2018-09-13T12:38:07.367Z] 12:38:07     INFO - TEST-INFO | screentopng: exit 0
[task 2018-09-13T12:38:07.368Z] 12:38:07     INFO - Buffered messages logged at 12:38:06
[task 2018-09-13T12:38:07.370Z] 12:38:07     INFO - TEST-PASS | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | resources are loaded 
[task 2018-09-13T12:38:07.371Z] 12:38:07     INFO - TEST-PASS | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | is not a tracker 
[task 2018-09-13T12:38:07.372Z] 12:38:07     INFO - TEST-PASS | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | is a fast tracker 
[task 2018-09-13T12:38:07.373Z] 12:38:07     INFO - TEST-PASS | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | is a blocked tracker 
[task 2018-09-13T12:38:07.376Z] 12:38:07     INFO - TEST-PASS | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | fast iframe loaded 
[task 2018-09-13T12:38:07.378Z] 12:38:07     INFO - TEST-PASS | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | is not a tracker 
[task 2018-09-13T12:38:07.380Z] 12:38:07     INFO - TEST-PASS | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | is a fast tracker 
[task 2018-09-13T12:38:07.381Z] 12:38:07     INFO - TEST-PASS | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | is a blocked tracker 
[task 2018-09-13T12:38:07.382Z] 12:38:07     INFO - Buffered messages finished
[task 2018-09-13T12:38:07.387Z] 12:38:07     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | 5 trackers found - got 6, expected 5
[task 2018-09-13T12:38:07.389Z] 12:38:07     INFO - SimpleTest.is@chrome://mochikit/content/tests/SimpleTest/SimpleTest.js:312:5
[task 2018-09-13T12:38:07.390Z] 12:38:07     INFO - testFastBlock@chrome://mochitests/content/chrome/toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html:347:3
[task 2018-09-13T12:38:07.397Z] 12:38:07     INFO - async*runTest/</<@chrome://mochitests/content/chrome/toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html:240:13
[task 2018-09-13T12:38:07.398Z] 12:38:07     INFO - async*testOnWindow/</<@chrome://mochitests/content/chrome/toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html:96:7
[task 2018-09-13T12:38:07.399Z] 12:38:07     INFO - process@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:928:23
[task 2018-09-13T12:38:07.400Z] 12:38:07     INFO - walkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:812:29
[task 2018-09-13T12:38:07.401Z] 12:38:07     INFO - Promise*scheduleWalkerLoop@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:745:11
[task 2018-09-13T12:38:07.402Z] 12:38:07     INFO - schedulePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:776:7
[task 2018-09-13T12:38:07.408Z] 12:38:07     INFO - completePromise@resource://gre/modules/Promise.jsm -> resource://gre/modules/Promise-backend.js:713:7
[task 2018-09-13T12:38:07.409Z] 12:38:07     INFO - receiveMessage@resource://testing-common/ContentTask.jsm:112:9
[task 2018-09-13T12:38:07.411Z] 12:38:07     INFO - MessageListener.receiveMessage*@resource://testing-common/ContentTask.jsm:127:1
[task 2018-09-13T12:38:07.412Z] 12:38:07     INFO - @resource://testing-common/BrowserTestUtils.jsm:26:1
[task 2018-09-13T12:38:07.413Z] 12:38:07     INFO - @chrome://mochitests/content/chrome/toolkit/components/url-classifier/tests/mochitest/test_advisory_link.html:21:1
[task 2018-09-13T12:38:07.414Z] 12:38:07     INFO - Not taking screenshot here: see the one that was previously logged
[task 2018-09-13T12:38:07.415Z] 12:38:07     INFO - TEST-UNEXPECTED-FAIL | toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.html | 2 tracker blocked - got 3, expected 2
Pushed by xeonchen@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/d965284aca14
Part 1: Add basic FastBlock automation tests; r=mayhemer
(In reply to Andreea Pavel [:apavel] from comment #14)
> Backed out for failing
> toolkit/components/url-classifier/tests/mochitest/test_fastblock_bug1477046.
> htm
> 
> Push that started the failures: 
> https://treeherder.mozilla.org/#/
> jobs?repo=autoland&revision=d965284aca1416b104471807b2cb48b85de0bf59
> 
> Failure log: 
> https://treeherder.mozilla.org/logviewer.
> html#?job_id=199118810&repo=autoland&lineNumber=3846
> 
> Backout:
> https://hg.mozilla.org/integration/autoland/rev/
> 02e3f12f4eb64104eb12796ddf6401e4a3854e6a

It's odd because I can't reproduce this on try.
I'm going to move some code from part 2 to part 1 and see if this fixes the intermittent.
Flags: needinfo?(xeonchen)
Pushed by xeonchen@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/9fbf60a45883
Part 1: Add basic FastBlock automation tests; r=mayhemer
Pushed by xeonchen@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/98ad06740e81
Part 2: Add FastBlock iframe tests; r=mayhemer
Comment on attachment 9008901 [details]
Bug 1477046 - Part 3: Add FastBlock redirect tests; r=mayhemer

Honza Bambas (:mayhemer) has approved the revision.
Attachment #9008901 - Flags: review+
Pushed by xeonchen@gmail.com:
https://hg.mozilla.org/integration/autoland/rev/5986c5a57f68
Part 3: Add FastBlock redirect tests; r=mayhemer
https://hg.mozilla.org/mozilla-central/rev/5986c5a57f68
Status: NEW → RESOLVED
Closed: 11 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla64
Any reason we shouldn't uplift this to Beta? It can go test-only without an approval request, just making sure it's worth doing so first.
Flags: needinfo?(xeonchen)
(In reply to Ryan VanderMeulen [:RyanVM] from comment #24)
> Any reason we shouldn't uplift this to Beta? It can go test-only without an
> approval request, just making sure it's worth doing so first.

It's worthy to uplift to beta, could you guide me how to do this?
Flags: needinfo?(xeonchen) → needinfo?(ryanvm)
Flags: needinfo?(ryanvm)
Whiteboard: [necko-triaged] → [necko-triaged][checkin-needed-beta]
Flags: in-testsuite+
Whiteboard: [necko-triaged][checkin-needed-beta] → [necko-triaged]
Depends on: 1495075, 1491260
Depends on: 1495110
You need to log in before you can comment on or make changes to this bug.