Intermittent browser/base/content/test/static/browser_parsable_css.js | Unused whitelist item. sourceName: /responsive-ua\.css$/i errorMessage: /Unknown pseudo-class.*moz-dropdown-list/i -

REOPENED
Assigned to

Status

()

Core
General
REOPENED
7 months ago
4 months ago

People

(Reporter: Treeherder Bug Filer, Assigned: RyanVM)

Tracking

({intermittent-failure, leave-open})

unspecified
mozilla55
intermittent-failure, leave-open
Points:
---

Firefox Tracking Flags

(firefox55 fixed)

Details

(Whiteboard: [stockwell disabled][ele:1a])

Attachments

(1 attachment)

(Reporter)

Description

7 months ago
treeherder
Filed by: wkocher [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=90960559&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/UqZEwag9QrqQ1LcY8Yj6Sw/runs/0/artifacts/public/logs/live_backing.log

This has happened two out of the tree times the test has run on linux asan since bug 1352523 landed. At a 60+ minutes, backfilling and retriggering to see how often it's failing is going to take a while.

Comment 1

7 months ago
12:44:43 <RyanVM> KWierso: isn't that test already disabled on debug?
12:45:03 <RyanVM> smaug: FWIW, I'd vote for skipping on ASAN too since I'm pretty sure we already did on debug for similar perf reasons
12:45:13 <RyanVM> and the incredibly low value it has running on both opt & debug
12:46:29 <%KWierso> RyanVM: https://dxr.mozilla.org/mozilla-central/source/browser/base/content/test/static/browser.ini#13
12:46:51 <RyanVM> aha, it was parsable_script that got the asan skipping treatment already
12:47:13 <RyanVM> I feel like we should just make those asan/debug skips global at the top of the manifest
(Assignee)

Comment 2

7 months ago
Gijs, would you object to skipping this directory on ASAN/Debug? I'll write the patch :)
Flags: needinfo?(gijskruitbosch+bugs)
(Assignee)

Comment 3

7 months ago
Looks like bug 1349307 has some previous discussion about skipping globally.
I'd be happy to skipping this (and the other tests in this dir) on debug/asan.

That said, I'm still mildly curious why the vsync changes are triggering this failure... if we can improve the test to be more deterministic, we should. Shouldn't stop us from skipping this on debug/asan to relieve the immediate pain for sheriffs though.
Flags: needinfo?(gijskruitbosch+bugs)
(Assignee)

Comment 5

7 months ago
While we're on the topic, what about skipping on non-e10s too? Though I suppose we may still have front-end code that exercises different code paths, so maybe not.
(Assignee)

Comment 6

7 months ago
Created attachment 8857690 [details] [diff] [review]
patch

I skipped it on Linux32 as well since 2/4 tests were already skipped due to similar-looking issues (and it was previously decided that Linux64 was good enough).

Looks green on Try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a7e94b6cc939c470819a3bcaac7ec3103268b933&group_state=expanded
Assignee: nobody → ryanvm
Status: NEW → ASSIGNED
Attachment #8857690 - Flags: review?(gijskruitbosch+bugs)
Comment on attachment 8857690 [details] [diff] [review]
patch

Review of attachment 8857690 [details] [diff] [review]:
-----------------------------------------------------------------

::: browser/base/content/test/static/browser.ini
@@ +1,5 @@
>  [DEFAULT]
> +# These tests can be prone to intermittent failures on slower systems.
> +# Since the specific flavor doesn't matter from a correctness standpoint,
> +# just skip the tests on ASAN and debug builds. Also known to be flaky on
> +# Linux32, so skip them there as well.

Can we keep the two bug number references from the existing comments? (1172468 and 1349307)
Attachment #8857690 - Flags: review?(gijskruitbosch+bugs) → review+
Whiteboard: [stockwell disabled]

Comment 8

7 months ago
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/cf7f6cdfaaf2
Don't run the browser/base/content/test/static directory on ASAN, debug, or Linux32. r=Gijs

Comment 9

7 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/cf7f6cdfaaf2
Status: ASSIGNED → RESOLVED
Last Resolved: 7 months ago
status-firefox55: --- → fixed
Resolution: --- → FIXED
Target Milestone: --- → mozilla55

Comment 10

7 months ago
31 failures in 205 pushes (0.151 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* autoland: 16
* mozilla-inbound: 9
* mozilla-central: 4
* try: 2

Platform breakdown:
* linux64: 31

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1355984&startday=2017-04-13&endday=2017-04-13&tree=all
I bet it would be easier to keep starring the very frequent failures in this if the bug was open instead of closed.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
(Assignee)

Comment 12

7 months ago
You're saying this is still running on ASAN? WTF...
(Assignee)

Comment 13

7 months ago
Oh, no wonder why. Need to update hdevtools/client/framework/test/browser.ini as well I bet.
(Assignee)

Comment 14

7 months ago
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f16ba7c507e0b2b0b581c79dbff372785b62e2e7

If this works, it'd have the added benefit of not requiring people to remember to update multiple manifests every time a new test is added or skipped.

Comment 15

7 months ago
15 failures in 155 pushes (0.097 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-inbound: 7
* autoland: 4
* mozilla-central: 3
* oak: 1

Platform breakdown:
* linux64: 13
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1355984&startday=2017-04-14&endday=2017-04-14&tree=all

Comment 16

7 months ago
85 failures in 894 pushes (0.095 failures/push) were associated with this bug in the last 7 days. 

This is the #14 most frequent failure this week. 

** This failure happened more than 75 times this week! Resolving this bug is a very high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 1 week, the affected test(s) may be disabled. **  

Repository breakdown:
* mozilla-inbound: 39
* autoland: 29
* mozilla-central: 13
* try: 3
* oak: 1

Platform breakdown:
* linux64: 83
* osx-10-10: 2

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1355984&startday=2017-04-10&endday=2017-04-16&tree=all

Comment 17

7 months ago
16 failures in 100 pushes (0.16 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* mozilla-inbound: 6
* autoland: 6
* graphics: 2
* try: 1
* mozilla-central: 1

Platform breakdown:
* linux64: 16

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1355984&startday=2017-04-17&endday=2017-04-17&tree=all
(In reply to Ryan VanderMeulen [:RyanVM] from comment #14)
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=f16ba7c507e0b2b0b581c79dbff372785b62e2e7
> 
> If this works, it'd have the added benefit of not requiring people to
> remember to update multiple manifests every time a new test is added or
> skipped.


[task 2017-04-14T04:29:54.514388Z] 04:29:54     INFO -  /home/worker/workspace/build/src/obj-firefox/_virtualenv/bin/python -m mozbuild.action.process_install_manifest --no-remove _tests _build_manifests/install/_test_files
[task 2017-04-14T04:29:54.514477Z] 04:29:54     INFO -  Traceback (most recent call last):
[task 2017-04-14T04:29:54.514612Z] 04:29:54     INFO -    File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
[task 2017-04-14T04:29:54.514727Z] 04:29:54     INFO -      "__main__", fname, loader, pkg_name)
[task 2017-04-14T04:29:54.514926Z] 04:29:54     INFO -    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
[task 2017-04-14T04:29:54.515113Z] 04:29:54     INFO -      exec code in run_globals
[task 2017-04-14T04:29:54.515316Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozbuild/action/process_install_manifest.py", line 119, in <module>
[task 2017-04-14T04:29:54.515440Z] 04:29:54     INFO -      main(sys.argv[1:])
[task 2017-04-14T04:29:54.515621Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozbuild/action/process_install_manifest.py", line 106, in main
[task 2017-04-14T04:29:54.515788Z] 04:29:54     INFO -      defines=args.defines)
[task 2017-04-14T04:29:54.515956Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozbuild/action/process_install_manifest.py", line 69, in process_manifest
[task 2017-04-14T04:29:54.516099Z] 04:29:54     INFO -      remove_empty_directories=remove_empty_directories)
[task 2017-04-14T04:29:54.516261Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozpack/copier.py", line 399, in copy
[task 2017-04-14T04:29:54.516408Z] 04:29:54     INFO -      copy_results.append((destfile, f.copy(destfile, skip_if_older)))
[task 2017-04-14T04:29:54.516539Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozpack/files.py", line 316, in copy
[task 2017-04-14T04:29:54.516747Z] 04:29:54     INFO -      raise ErrorMessage('Symlink target path does not exist: %s' % self.path)
[task 2017-04-14T04:29:54.516931Z] 04:29:54     INFO -  mozpack.errors.ErrorMessage: Symlink target path does not exist: /home/worker/workspace/build/src/devtools/client/framework/test/bug1262648_string_with_newlines.dtd

Is that a fundamental flaw with the approach you're trying out or just an oversight that can be fixed? Not entirely sure where this is coming from.

Should we just add the skip-if to the devtools manifest for now to shut up this regular intermittent?
Flags: needinfo?(ryanvm)
(Assignee)

Comment 19

7 months ago
Yeah, I think we should just add the skips to the devtools manifest for now. I still think my approach is the better long term one, but I fear there's harness bugs blocking it that I don't want this to wait for.
Flags: needinfo?(ryanvm)
Keywords: leave-open

Comment 20

7 months ago
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/d2b15fba0e2d
Skip browser static analysis check tests on ASAN when running in the devtools suite as well. r=Gijs

Comment 21

7 months ago
15 failures in 159 pushes (0.094 failures/push) were associated with this bug yesterday.   

Repository breakdown:
* graphics: 6
* autoland: 5
* mozilla-inbound: 3
* mozilla-central: 1

Platform breakdown:
* linux64: 15

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1355984&startday=2017-04-18&endday=2017-04-18&tree=all

Comment 22

7 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/d2b15fba0e2d

Comment 23

7 months ago
44 failures in 817 pushes (0.054 failures/push) were associated with this bug in the last 7 days. 

This is the #17 most frequent failure this week.  

** This failure happened more than 30 times this week! Resolving this bug is a high priority. **

** Try to resolve this bug as soon as possible. If unresolved for 2 weeks, the affected test(s) may be disabled. ** 

Repository breakdown:
* autoland: 18
* mozilla-inbound: 11
* graphics: 9
* try: 4
* mozilla-central: 2

Platform breakdown:
* linux64: 43
* linux32: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1355984&startday=2017-04-17&endday=2017-04-23&tree=all
Whiteboard: [stockwell disabled] → [stockwell disabled][ele:1a]

Comment 24

5 months ago
1 failures in 718 pushes (0.001 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* try: 1

Platform breakdown:
* linux64: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1355984&startday=2017-06-26&endday=2017-07-02&tree=all

Comment 25

4 months ago
2 failures in 656 pushes (0.003 failures/push) were associated with this bug in the last 7 days.   

Repository breakdown:
* mozilla-inbound: 1
* mozilla-beta: 1

Platform breakdown:
* osx-10-10: 1
* linux64-devedition: 1

For more details, see:
https://brasstacks.mozilla.com/orangefactor/?display=Bug&bugid=1355984&startday=2017-07-03&endday=2017-07-09&tree=all
You need to log in before you can comment on or make changes to this bug.