Closed Bug 1596080 Opened 3 years ago Closed 3 years ago

Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_title.py TestTitle.<subtest> | AssertionError: 'foo' != u''

Categories

(Testing :: Marionette, defect, P5)

Version 3
defect

Tracking

(firefox72 fixed)

RESOLVED FIXED
mozilla72
Tracking Status
firefox72 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: Gijs)

References

(Regression)

Details

(Keywords: intermittent-failure)

Filed by: nbeleuzu [at] mozilla.com
Parsed log: https://treeherder.mozilla.org/logviewer.html#?job_id=275986262&repo=autoland
Full log: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Ciob45FhTIipQ2SDHaWZVQ/runs/0/artifacts/public/logs/live_backing.log


[task 2019-11-13T11:13:20.085Z] 11:13:20 INFO - TEST-START | testing/marionette/harness/marionette_harness/tests/unit/test_title.py TestTitle.test_title_from_top
[task 2019-11-13T11:13:20.086Z] 11:13:20 INFO - 1573643600073 Marionette DEBUG Closed connection 138
[task 2019-11-13T11:13:20.088Z] 11:13:20 INFO - 1573643600074 Marionette DEBUG Accepted connection 139 from 127.0.0.1:45990
[task 2019-11-13T11:13:20.090Z] 11:13:20 INFO - 1573643600075 Marionette DEBUG 139 -> [0,1,"WebDriver:NewSession",{"strictFileInteractability":true}]
[task 2019-11-13T11:13:20.091Z] 11:13:20 INFO - 1573643600077 Marionette TRACE [152] Frame script loaded
[task 2019-11-13T11:13:20.103Z] 11:13:20 INFO - 1573643600079 Marionette TRACE [152] Frame script registered
[task 2019-11-13T11:13:20.103Z] 11:13:20 INFO - 1573643600087 Marionette DEBUG 139 <- [1,1,null,{"sessionId":"a4b9d209-ce56-4e03-9961-db8012ceb255","capabilities":{"browserName":"firefox","browserVersion":"72.0a ... p/tmpHqlYXK.mozrunner","moz:shutdownTimeout":180000,"moz:useNonSpecCompliantPointerOrigin":false,"moz:webdriverClick":true}}]
[task 2019-11-13T11:13:20.103Z] 11:13:20 INFO - 1573643600091 Marionette DEBUG 139 -> [0,2,"WebDriver:SetTimeouts",{"script":30000}]
[task 2019-11-13T11:13:20.103Z] 11:13:20 INFO - 1573643600091 Marionette DEBUG 139 <- [1,2,null,{"value":null}]
[task 2019-11-13T11:13:20.103Z] 11:13:20 INFO - 1573643600093 Marionette DEBUG 139 -> [0,3,"WebDriver:SetTimeouts",{"pageLoad":300000}]
[task 2019-11-13T11:13:20.103Z] 11:13:20 INFO - 1573643600094 Marionette DEBUG 139 <- [1,3,null,{"value":null}]
[task 2019-11-13T11:13:20.103Z] 11:13:20 INFO - 1573643600094 Marionette DEBUG 139 -> [0,4,"WebDriver:SetTimeouts",{"implicit":0}]
[task 2019-11-13T11:13:20.103Z] 11:13:20 INFO - 1573643600095 Marionette DEBUG 139 <- [1,4,null,{"value":null}]
[task 2019-11-13T11:13:20.103Z] 11:13:20 INFO - 1573643600096 Marionette DEBUG 139 -> [0,5,"WebDriver:Navigate",{"url":"data:text/html;charset=utf-8,%3Ctitle%3Efoo%3C/title%3E"}]
[task 2019-11-13T11:13:20.105Z] 11:13:20 INFO - 1573643600101 Marionette TRACE [152] Received DOM event beforeunload for data:text/html;charset=utf-8,%3Ctitle%3Efoo%3C/title%3E%0A%3Ciframe%20id%3Dbar%20src%3D%22data%3Atext/html%3Bcharset%3Dutf-8% ... Dbaz%2520src%253D%2522data%253Atext/html%253Bcharset%253Dutf-8%252C%25253Ctitle%25253Ebaz%25253C/title%25253E%2522%253E%22%3E
[task 2019-11-13T11:13:20.161Z] 11:13:20 INFO - 1573643600152 Marionette TRACE [152] Received DOM event pagehide for data:text/html;charset=utf-8,%3Ctitle%3Efoo%3C/title%3E%0A%3Ciframe%20id%3Dbar%20src%3D%22data%3Atext/html%3Bcharset%3Dutf-8% ... Dbaz%2520src%253D%2522data%253Atext/html%253Bcharset%253Dutf-8%252C%25253Ctitle%25253Ebaz%25253C/title%25253E%2522%253E%22%3E
[task 2019-11-13T11:13:20.168Z] 11:13:20 INFO - 1573643600164 Marionette TRACE [152] Received DOM event DOMContentLoaded for data:text/html;charset=utf-8,%3Ctitle%3Efoo%3C/title%3E
[task 2019-11-13T11:13:20.193Z] 11:13:20 INFO - 1573643600189 Marionette TRACE [152] Received DOM event pageshow for data:text/html;charset=utf-8,%3Ctitle%3Efoo%3C/title%3E
[task 2019-11-13T11:13:20.201Z] 11:13:20 INFO - 1573643600197 Marionette DEBUG 139 <- [1,5,null,{"value":null}]
[task 2019-11-13T11:13:20.209Z] 11:13:20 INFO - 1573643600203 Marionette DEBUG 139 -> [0,6,"WebDriver:GetTitle",{}]
[task 2019-11-13T11:13:20.209Z] 11:13:20 INFO - 1573643600204 Marionette DEBUG 139 <- [1,6,null,{"value":""}]
[task 2019-11-13T11:13:20.209Z] 11:13:20 INFO - 1573643600205 Marionette DEBUG 139 -> [0,7,"Marionette:GetContext",{}]
[task 2019-11-13T11:13:20.209Z] 11:13:20 INFO - 1573643600205 Marionette DEBUG 139 <- [1,7,null,{"value":"content"}]
[task 2019-11-13T11:13:20.210Z] 11:13:20 INFO - 1573643600206 Marionette DEBUG 139 -> [0,8,"Marionette:SetContext",{"value":"chrome"}]
[task 2019-11-13T11:13:20.213Z] 11:13:20 INFO - 1573643600210 Marionette DEBUG 139 <- [1,8,null,{"value":null}]
[task 2019-11-13T11:13:20.228Z] 11:13:20 INFO - 1573643600221 Marionette DEBUG 139 -> [0,9,"WebDriver:TakeScreenshot",{"full":true,"hash":false,"id":null,"scroll":true}]
[task 2019-11-13T11:13:20.453Z] 11:13:20 INFO - 1573643600449 Marionette DEBUG 139 <- [1,9,null,{"value":"iVBORw0KGgoAAAANSUhEUgAABdwAAARMCAYAAACDJFGTAAAgAElEQVR4nOzd649U95kncF6ONNK+3D9gVorEKF45L+LNrCPF0c7MypGlx ... PhDgAAAAAAA+EOAAAAAAAD4Q4AAAAA+3rs5AAAACdJREFUAAPhDgAAAAAAA+EOAAAAAAAD4Q4AAAAAAAPhDgAAAAAAgwAIMdpmGtDTiQAAAABJRU5ErkJggg=="}]
[task 2019-11-13T11:13:20.460Z] 11:13:20 INFO - 1573643600456 Marionette DEBUG 139 -> [0,10,"Marionette:SetContext",{"value":"content"}]
[task 2019-11-13T11:13:20.461Z] 11:13:20 INFO - 1573643600456 Marionette DEBUG 139 <- [1,10,null,{"value":null}]
[task 2019-11-13T11:13:20.461Z] 11:13:20 INFO - 1573643600457 Marionette DEBUG 139 -> [0,11,"Marionette:GetContext",{}]
[task 2019-11-13T11:13:20.462Z] 11:13:20 INFO - 1573643600458 Marionette DEBUG 139 <- [1,11,null,{"value":"content"}]
[task 2019-11-13T11:13:20.463Z] 11:13:20 INFO - 1573643600459 Marionette DEBUG 139 -> [0,12,"Marionette:SetContext",{"value":"content"}]
[task 2019-11-13T11:13:20.464Z] 11:13:20 INFO - 1573643600459 Marionette DEBUG 139 <- [1,12,null,{"value":null}]
[task 2019-11-13T11:13:20.465Z] 11:13:20 INFO - 1573643600462 Marionette DEBUG 139 -> [0,13,"WebDriver:GetPageSource",{}]
[task 2019-11-13T11:13:20.472Z] 11:13:20 INFO - 1573643600470 Marionette DEBUG 139 <- [1,13,null,{"value":"<html><head><title>foo</title></head><body></body></html>"}]
[task 2019-11-13T11:13:20.472Z] 11:13:20 INFO - 1573643600471 Marionette DEBUG 139 -> [0,14,"Marionette:SetContext",{"value":"content"}]
[task 2019-11-13T11:13:20.476Z] 11:13:20 INFO - 1573643600472 Marionette DEBUG 139 <- [1,14,null,{"value":null}]
[task 2019-11-13T11:13:20.496Z] 11:13:20 INFO - TEST-UNEXPECTED-FAIL | testing/marionette/harness/marionette_harness/tests/unit/test_title.py TestTitle.test_title_from_top | AssertionError: 'foo' != u''
[task 2019-11-13T11:13:20.496Z] 11:13:20 INFO - Traceback (most recent call last):
[task 2019-11-13T11:13:20.496Z] 11:13:20 INFO - File "/builds/worker/workspace/build/venv/lib/python2.7/site-packages/marionette_harness/marionette_test/testcases.py", line 159, in run
[task 2019-11-13T11:13:20.496Z] 11:13:20 INFO - testMethod()
[task 2019-11-13T11:13:20.496Z] 11:13:20 INFO - File "/builds/worker/workspace/build/tests/marionette/tests/testing/marionette/harness/marionette_harness/tests/unit/test_title.py", line 24, in test_title_from_top

Probably fixed by bug 1595925 but hard to say given the really low frequency. Let's at least track all of these in 1 bug.

Summary: Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_title.py TestTitle.test_title_from_top | AssertionError: 'foo' != u'' → Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_title.py TestTitle.test_title** | AssertionError: 'foo' != u''
See Also: 1596003
Summary: Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_title.py TestTitle.test_title** | AssertionError: 'foo' != u'' → Intermittent testing/marionette/harness/marionette_harness/tests/unit/test_title.py TestTitle.<subtest> | AssertionError: 'foo' != u''

Gijs, you simply go to the annotations tab, and click the x behind the starred bug number. Then you can star again. I just did it for both of the referenced failures.

Thanks for duping the others. I will check later this week if the failures have stopped.

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #6)

Gijs, you simply go to the annotations tab, and click the x behind the starred bug number. Then you can star again. I just did it for both of the referenced failures.

I don't understand - this bug still doesn't come up as a suggestion for those intermittents. Are you just manually filling in the bug number? That's not going to help for future failures...

I asked Aryx on IRC to have a look at it given that you may have forgotten to set needinfo to him.

Yes, if it doesn't get suggested, one has to have the idea that there might be a general bug for it and search for it in a document about not suggested bugs and if there are no matches on bugzilla.

(In reply to Sebastian Hengst [:aryx] (needinfo on intermittent or backout) from comment #9)

Yes, if it doesn't get suggested, one has to have the idea that there might be a general bug for it and search for it in a document about not suggested bugs and if there are no matches on bugzilla.

So why isn't this bug coming up as a suggestion?

Flags: needinfo?(aryx.bugmail)

Treeherder uses a MySQL full text search of bug summaries against failure lines and using <subtest> as placeholder might move the match score below the threshold.

Flags: needinfo?(aryx.bugmail)

For other bugs we used <random>. Would that be better?

Assignee: nobody → gijskruitbosch+bugs
Status: NEW → RESOLVED
Closed: 3 years ago
Depends on: 1595925
Regressed by: 1594457
Resolution: --- → FIXED
Target Milestone: --- → mozilla72

(In reply to Henrik Skupin (:whimboo) [⌚️UTC+2] from comment #12)

For other bugs we used <random>. Would that be better?
It's one character less for the placeholder but will also depend on the length of the test name.

Has Regression Range: --- → yes
You need to log in before you can comment on or make changes to this bug.