Investigating tp6m-6 raptor test for raptor-tp6m-bbc-geckoview

RESOLVED FIXED in Firefox 68

Status

enhancement
P1
normal
RESOLVED FIXED
5 months ago
3 months ago

People

(Reporter: alexandrui, Assigned: alexandrui)

Tracking

(Blocks 1 bug)

Version 3
mozilla68
Points:
---
Dependency tree / graph

Firefox Tracking Flags

(firefox68 fixed)

Details

Attachments

(3 attachments)

Assignee: nobody → alexandru.ionescu
Summary: Investigating tp6-m raptor test for raptor-tp6m-bbc-geckoview → Investigating tp6m-7 raptor test for raptor-tp6m-bbc-geckoview
Summary: Investigating tp6m-7 raptor test for raptor-tp6m-bbc-geckoview → Investigating tp6m-6 raptor test for raptor-tp6m-bbc-geckoview

I tried running this on Pixel2 with different parameters changed (timeout, startup delay, TTFI, run_at) but still couldn't get rid of the TIMEOUT fail.
Every time the test fails there is a js error:

Error: permission denied to access object
onload https://static.bbc.co.uk/bbcdotcom/...

I used the --debug-mode to see if window.performance.timing.* is available when this error is received, and it is! (screenshot attached)
I searched for permission denied to access object and this pointed me to something that might have to do with nss certificates?
https://searchfox.org/mozilla-central/search?q=Permission+denied+to+access+object&case=false&regexp=false&path=

Flags: needinfo?(rwood)
Flags: needinfo?(dave.hunt)

Do you see the same JavaScript error when the test passes? :acreskey have you seen this before when running tests?

Flags: needinfo?(dave.hunt) → needinfo?(acreskey)

davehunt, I don't see this error when the tests passes. If you want the log of any test in the spreadsheet (https://docs.google.com/spreadsheets/d/1NcKPjWAl7roU04o8j7tZBCM2-_-hRl2vlPWi2P_Lbos/edit#gid=0) I saved all of them.

davehunt, I haven't seen this error before.

Possibly window.performance is undefined when we store this in perfData?
https://searchfox.org/mozilla-central/source/testing/raptor/webext/raptor/measure.js#6

We could test this by instead of storing window.performance in perfData we could verify that it's defined and use it directly in measure.js.

Flags: needinfo?(acreskey)
Flags: needinfo?(rwood)

:acreskey, I replaced perfData with a function perfData returning window.performance but I am getting the same result with a startup delay of 120s and page timeout of 90s.

Hmm...

Alexandru, do you know if it's window or window.performance that's not defined at this point?

As a test we may have to hold off on the measuring until the undefined object becomes defined.

In Bug 1533059 I'm proposing that the raptor web extension isn't injecting itself at a stable time. But note that we haven't found a solution that solves all pages there either.

I am analyzing this thing, I am trying to find out at which point becomes window or window.performance defined. Or rather to find a dynamic logic that will wait for it to become defined no matter what point in time it is.
I will look into Bug 1533059, thanks.

Bad news. When the test fails, for some reason the execution never gets into the overloading of window.onload and apparently it's not blocked by the undefined state of window.performance.
Ignore the undefined, put it by mistake before the assignment.

(In reply to Alexandru Ionescu from comment #10)

Created attachment 9051934 [details]
Screenshot from 2019-03-19 10-52-45.png

Bad news. When the test fails, for some reason the execution never gets into the overloading of window.onload and apparently it's not blocked by the undefined state of window.performance.
Ignore the undefined, put it by mistake before the assignment.

Is this a race condition between Raptor and the website for defining window.onload? If the website gets there first, Raptor preserves the onload and defines it's own, but if Raptor get there first the website would be replacing Raptor's. Can you experiment with :acreskey's patch in https://hg.mozilla.org/try/rev/880f845a9c3a1a7d4adb664e06b3bcbfb35e9dba to see if using the load event listener instead of the onload helps?

Depends on: 1536758
Priority: -- → P1
Pushed by dhunt@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/11979156a93e
remove disabled attribute for raptor-tp6m-bbc-geckoview r=davehunt
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla68
You need to log in before you can comment on or make changes to this bug.