on pageload timeout, it would be nice to know if we failed at the beginning, middle, or end
Categories
(Testing :: Raptor, defect, P3)
Tracking
(Not tracked)
People
(Reporter: jmaher, Unassigned)
Details
if we fail to get metrics and timeout it could be because tooling or networks are failing us and we cannot get anything to load or work (i.e. facebook.com was down on april 14th).
That error message looks the same as one where we fail on cycle 23/25 and miss collecting a metric.
I don't know how to solve this, but doing this would lead to more actionable failure classification.
|
||
Updated•6 years ago
|
|
||
Comment 1•6 years ago
|
||
We recently added the pending metrics to the timeout message, which gives some kind of indication on the progress of the page load. Would adding the current/total page or browser cycle be useful here? This is something that can be determined from the console output, but maybe exposing it in the failure message would be useful?
Current:
TEST-UNEXPECTED-FAIL: test 'raptor-tp6m-microsoft-support-geckoview' timed out loading test page: https://support.microsoft.com/en-us pending metrics: fcp
Proposal:
TEST-UNEXPECTED-FAIL: test 'raptor-tp6m-microsoft-support-geckoview' timed out loading test page: https://support.microsoft.com/en-us; browser-cycle: 1/1; page-cycle: 1/25; pending metrics: fcp
Perhaps we could also make it clear when all metrics are pending instead of listing them, as I would expect cycle 1/25 and pending: all to be a different cause than cycle 23/25 and pending: fcp.
|
Reporter | |
Comment 2•6 years ago
|
||
if we don't have a cycle or it is 1/25, then it is browser startup
if we have cycles 25/25, then it is shutdown
I think anything else between 1 and 25 is in the middle and should maybe have the same message "timeout during pageload".
My only concern with page-cycle is that it will create different bugs for the sheriffs and if they combine them, then we will have no easy way to distinguish between start, middle, end.
|
|
||
Comment 3•6 years ago
|
||
+1 (re: would cause too many different bugs for sheriffs)
|
||
Comment 4•6 years ago
|
||
I think this information is outputted now in the logs.
Can we mark this as Worksforme?
|
|
||
Comment 5•6 years ago
|
||
We have the pending measurements but I don't believe we have the browser-cycle and page-cycle in the actual timeout message. However yep that can easily be found in the logs. IMO I think it's best not to add those to the actual failure message because we would have so many different classifications/bugs. I find we already have too many when we decided to add the pending metrics to the error message (in hindsight I think that info should have been logged yes, but not added to the actual final failure message) as that caused alot of new bugs/confusion. So yep long story short I think we should close this, thanks! We could always re-visit in the future if we find the need to further classify failures.
Description
•