2045117 - Fix Nova-enabled test failures from UrlbarInput's Nova changes

Status: RESOLVED WORKSFORME

(Firefox :: Address Bar, task, P1)

Description

[Stephen Thompson [:sthompson]]

:mossop ran a try run with browser.nova.enabled to see whether there were any test issues hiding from us since Nova is off by default.

I think 56 of the 81 test failures have errors like uncaught exception - TypeError: can't access property "p1", this.parentNode.getBoxQuads(...)[0] is undefined with UrlbarInput in the top of the stack: https://tests.firefox.dev/try.html?rev=813998049ff1835af3a60def7853df3d8be83924 These code paths look like they are only called with Nova enabled after bug 2019152 was fixed.

We'd like to get these Nova-specific test failures addressed before the planned 155 Nightly pref flip to make Nova enabled by default. We don't want to have to burden the sheriffs with permafails that we know about.

Comment 1

[Stephen Thompson [:sthompson]]

I'm glad this was already on your radar. Thanks!

Comment 2

[Drew Willcoxon :adw]

Thanks for filing this, it's definitely good to know that tests are failing, and it's good to have a meta-ish bug for tracking failures.

Comment 3

[Drew Willcoxon :adw]

Two developments:

• We landed several patches last week that seem to have fixed the error in comment 0, probably bug 2041255, bug 2044297, bug 2044849, or some combination
• Bug 2030183 has landed, so we're now running all urlbar tests with and without Nova enabled, and there aren't any failures

Before closing this bug I'd like to go through the non-urlbar tests in the link in comment 0 and verify they're fixed too.

Comment 4

[Drew Willcoxon :adw]

New try push same as mossop's but with a fresh tree, including the aforementioned fixes:

https://treeherder.mozilla.org/jobs?repo=try&revision=ce0871a82976d6e9d97d5ae7fdb8bfc70c36d000

https://tests.firefox.dev/try.html?rev=ce0871a82976d6e9d97d5ae7fdb8bfc70c36d000

Comment 5

[Drew Willcoxon :adw]

Based on that try run, it looks like there are a few categories of failures:

• Performance tests that now have unexpected reflows or UI differences. A couple of these are specifically for the urlbar, but of course they're also measuring the entire browser chrome, which has substantially changed. I don't know whose responsibility it is to fix these but they'll need major updates.
  • Examples: https://searchfox.org/firefox-main/source/browser/base/content/test/performance/browser_urlbar_keyed_search.js and https://searchfox.org/firefox-main/source/browser/base/content/test/performance/browser_startup_flicker.js in this run
• Non-urlbar tests that check for colors and styles in various parts of the browser
• Intermittents

There don't appear to be any failures that are obvious defects or problems.

Given the above and the fact that we've fixed the error in comment 0, I'll go ahead and close this. Now that we're running all urlbar tests with and without Nova enabled, we should be able to catch new problems as we create them.

If it turns out I've missed some existing failures or we introduce new ones that aren't caught by the urlbar test suite, let's file new bugs as we find them.