Closed Bug 1355984 Opened 3 years ago Closed 11 months ago

Intermittent browser/base/content/test/static/browser_parsable_css.js | Unused whitelist item. sourceName: /responsive-ua\.css$/i errorMessage: /Unknown pseudo-class.*moz-dropdown-list/i -

Categories

(Core :: General, defect)

defect
Not set

Tracking

()

RESOLVED WONTFIX
mozilla55
Tracking Status
firefox55 --- fixed

People

(Reporter: intermittent-bug-filer, Assigned: RyanVM)

Details

(Keywords: intermittent-failure, Whiteboard: [stockwell disabled][ele:1a])

Attachments

(1 file)

Filed by: wkocher [at] mozilla.com

https://treeherder.mozilla.org/logviewer.html#?job_id=90960559&repo=mozilla-inbound

https://queue.taskcluster.net/v1/task/UqZEwag9QrqQ1LcY8Yj6Sw/runs/0/artifacts/public/logs/live_backing.log

This has happened two out of the tree times the test has run on linux asan since bug 1352523 landed. At a 60+ minutes, backfilling and retriggering to see how often it's failing is going to take a while.
12:44:43 <RyanVM> KWierso: isn't that test already disabled on debug?
12:45:03 <RyanVM> smaug: FWIW, I'd vote for skipping on ASAN too since I'm pretty sure we already did on debug for similar perf reasons
12:45:13 <RyanVM> and the incredibly low value it has running on both opt & debug
12:46:29 <%KWierso> RyanVM: https://dxr.mozilla.org/mozilla-central/source/browser/base/content/test/static/browser.ini#13
12:46:51 <RyanVM> aha, it was parsable_script that got the asan skipping treatment already
12:47:13 <RyanVM> I feel like we should just make those asan/debug skips global at the top of the manifest
Gijs, would you object to skipping this directory on ASAN/Debug? I'll write the patch :)
Flags: needinfo?(gijskruitbosch+bugs)
Looks like bug 1349307 has some previous discussion about skipping globally.
I'd be happy to skipping this (and the other tests in this dir) on debug/asan.

That said, I'm still mildly curious why the vsync changes are triggering this failure... if we can improve the test to be more deterministic, we should. Shouldn't stop us from skipping this on debug/asan to relieve the immediate pain for sheriffs though.
Flags: needinfo?(gijskruitbosch+bugs)
While we're on the topic, what about skipping on non-e10s too? Though I suppose we may still have front-end code that exercises different code paths, so maybe not.
Attached patch patchSplinter Review
I skipped it on Linux32 as well since 2/4 tests were already skipped due to similar-looking issues (and it was previously decided that Linux64 was good enough).

Looks green on Try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a7e94b6cc939c470819a3bcaac7ec3103268b933&group_state=expanded
Assignee: nobody → ryanvm
Status: NEW → ASSIGNED
Attachment #8857690 - Flags: review?(gijskruitbosch+bugs)
Comment on attachment 8857690 [details] [diff] [review]
patch

Review of attachment 8857690 [details] [diff] [review]:
-----------------------------------------------------------------

::: browser/base/content/test/static/browser.ini
@@ +1,5 @@
>  [DEFAULT]
> +# These tests can be prone to intermittent failures on slower systems.
> +# Since the specific flavor doesn't matter from a correctness standpoint,
> +# just skip the tests on ASAN and debug builds. Also known to be flaky on
> +# Linux32, so skip them there as well.

Can we keep the two bug number references from the existing comments? (1172468 and 1349307)
Attachment #8857690 - Flags: review?(gijskruitbosch+bugs) → review+
Whiteboard: [stockwell disabled]
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/cf7f6cdfaaf2
Don't run the browser/base/content/test/static directory on ASAN, debug, or Linux32. r=Gijs
https://hg.mozilla.org/mozilla-central/rev/cf7f6cdfaaf2
Status: ASSIGNED → RESOLVED
Closed: 3 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla55
I bet it would be easier to keep starring the very frequent failures in this if the bug was open instead of closed.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
You're saying this is still running on ASAN? WTF...
Oh, no wonder why. Need to update hdevtools/client/framework/test/browser.ini as well I bet.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=f16ba7c507e0b2b0b581c79dbff372785b62e2e7

If this works, it'd have the added benefit of not requiring people to remember to update multiple manifests every time a new test is added or skipped.
(In reply to Ryan VanderMeulen [:RyanVM] from comment #14)
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=f16ba7c507e0b2b0b581c79dbff372785b62e2e7
> 
> If this works, it'd have the added benefit of not requiring people to
> remember to update multiple manifests every time a new test is added or
> skipped.


[task 2017-04-14T04:29:54.514388Z] 04:29:54     INFO -  /home/worker/workspace/build/src/obj-firefox/_virtualenv/bin/python -m mozbuild.action.process_install_manifest --no-remove _tests _build_manifests/install/_test_files
[task 2017-04-14T04:29:54.514477Z] 04:29:54     INFO -  Traceback (most recent call last):
[task 2017-04-14T04:29:54.514612Z] 04:29:54     INFO -    File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
[task 2017-04-14T04:29:54.514727Z] 04:29:54     INFO -      "__main__", fname, loader, pkg_name)
[task 2017-04-14T04:29:54.514926Z] 04:29:54     INFO -    File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
[task 2017-04-14T04:29:54.515113Z] 04:29:54     INFO -      exec code in run_globals
[task 2017-04-14T04:29:54.515316Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozbuild/action/process_install_manifest.py", line 119, in <module>
[task 2017-04-14T04:29:54.515440Z] 04:29:54     INFO -      main(sys.argv[1:])
[task 2017-04-14T04:29:54.515621Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozbuild/action/process_install_manifest.py", line 106, in main
[task 2017-04-14T04:29:54.515788Z] 04:29:54     INFO -      defines=args.defines)
[task 2017-04-14T04:29:54.515956Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozbuild/action/process_install_manifest.py", line 69, in process_manifest
[task 2017-04-14T04:29:54.516099Z] 04:29:54     INFO -      remove_empty_directories=remove_empty_directories)
[task 2017-04-14T04:29:54.516261Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozpack/copier.py", line 399, in copy
[task 2017-04-14T04:29:54.516408Z] 04:29:54     INFO -      copy_results.append((destfile, f.copy(destfile, skip_if_older)))
[task 2017-04-14T04:29:54.516539Z] 04:29:54     INFO -    File "/home/worker/workspace/build/src/python/mozbuild/mozpack/files.py", line 316, in copy
[task 2017-04-14T04:29:54.516747Z] 04:29:54     INFO -      raise ErrorMessage('Symlink target path does not exist: %s' % self.path)
[task 2017-04-14T04:29:54.516931Z] 04:29:54     INFO -  mozpack.errors.ErrorMessage: Symlink target path does not exist: /home/worker/workspace/build/src/devtools/client/framework/test/bug1262648_string_with_newlines.dtd

Is that a fundamental flaw with the approach you're trying out or just an oversight that can be fixed? Not entirely sure where this is coming from.

Should we just add the skip-if to the devtools manifest for now to shut up this regular intermittent?
Flags: needinfo?(ryanvm)
Yeah, I think we should just add the skips to the devtools manifest for now. I still think my approach is the better long term one, but I fear there's harness bugs blocking it that I don't want this to wait for.
Flags: needinfo?(ryanvm)
Keywords: leave-open
Pushed by ryanvm@gmail.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/d2b15fba0e2d
Skip browser static analysis check tests on ASAN when running in the devtools suite as well. r=Gijs
Whiteboard: [stockwell disabled] → [stockwell disabled][ele:1a]

Bug 1467572 removed responsive-ua.css. Maybe this can be re-enabled and closed?

Flags: needinfo?(ryanvm)
Flags: needinfo?(gijskruitbosch+bugs)

Note https://bugzilla.mozilla.org/show_bug.cgi?id=1519846#c0 .

In terms of re-enabling this, it's a bit messy because all the tests in this dir are disabled on asan because they're so slow. It'd need some attempts on try (w/ retriggers) to see if this is really OK to re-enable on ASAN. TBH, given how slow they are and that we do have test coverage on opt, I'm not super bothered, I could also live with just wontfixing this and keeping all of that stuff disabled on asan.

Flags: needinfo?(gijskruitbosch+bugs)

Wontfix sounds fine to me. I don't see myself getting back to this bug any time soon.

Status: REOPENED → RESOLVED
Closed: 3 years ago11 months ago
Flags: needinfo?(ryanvm)
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.