1514164 - Identify tests failing with the Quantum Bar

Status: RESOLVED FIXED

(Firefox :: Address Bar, enhancement, P1)

Description

[Marco Bonardo [:mak]]

We must get a full list of all the tests that will fail when we switch on the quantumbar pref, those tests should be moved into a legacy folder in the urlbar component so that we can switch them on/off based on the quantum bar pref, and then ported to the new codebase.

Comment 1

[Mark Banner (:standard8)]

Note: in bug 1521043, I'm going to move all the existing urlbar tests that we know of (without digging), into the legacy directory.

We can then work through them to work out which needs porting etc.

Comment 2

[Mark Banner (:standard8)]

Changed my mind, lets just do this.

Comment 3

[Mark Banner (:standard8)]

Attachment: Bug 1514164 - Change urlbar tests to use the test path rather than hard-coding it. r?mak

Comment 4

[Mark Banner (:standard8)]

Attachment: Bug 1514164 - Move urlbar tests to the new urlbar directory and split out tests that already pass with QuantumBar. r?mak

This moves the tests to browser/components/urlbar and sets them up for running both against QuantumBar as well as the legacy address bar.

Depends on D16995

Comment 5

[Mark Banner (:standard8)]

Try build: https://treeherder.mozilla.org/#/jobs?repo=try&revision=cee5c6247e57759dd08e06616113433ede3447a1

For context:

• .ini files now let you set prefs for the test to run (not sure if its is just per dir or per test but we're using it in the dir scope)
• the plan is to have tests/browser which runs quantumbar=true and tests/legacy with quantumbar=false
• Since you can also reference tests in other directories, tests/legacy references all the legacy tests, plus the ones in tests/browser which are compatible with the old urlbar.
• The minor possible downside is if you run one specific test it’ll run it twice (once with the pref true, once false), but that could also been seen as an advantage.

Currently the files are where they are most compatible with - tests/browser if they run with quantumbar, legacy otherwise. We can move them across as we go.

Additionally, at the moment, tests/browser fails with quantumbar=true. I think that's because we haven't quite fixed something we should have, and I'm looking into it. I might do that toggle in a third changeset or a follow-up bug, depending on when it is ready, reviews etc.

Comment 6

[Marco Bonardo [:mak]]

IS this commented out?

BROWSER_CHROME_MANIFESTS += [
'tests/browser/browser.ini',
# 'tests/legacy/browser.ini',
]

Comment 7

[Mark Banner (:standard8)]

Bah, yes. Shouldn't have been, I was testing something locally and left it in by mistake.

New try push: https://treeherder.mozilla.org/#/jobs?repo=try&revision=3af25c5560b57fc07060fc5aaf82161858fe72e9

Comment 8

[Mark Banner (:standard8)]

Attachment: Bug 1514164 - Enable running some of the urlbar tests against QuantumBar. r?mak

Although these tests are all able to pass individually, some still fail when run together. Bug 1521490 will address those.

Depends on D16996

Comment 9

[Mark Banner (:standard8)]

Try push of just the first two changesets:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=50de25ff65ce6c795032df8411e341a96fd27b14

Try push of all three:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=75bd42fec055ee39281e30d35c494290ba7397eb

Comment 10

[Mark Banner (:standard8)]

Last try push that included disabling a few tests due to shutdown leaks: https://treeherder.mozilla.org/#/jobs?repo=try&revision=7159404735d7c289ca86d8158100abae280b0b24

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/ba9f9c0ce256
Change urlbar tests to use the test path rather than hard-coding it. r=mak
https://hg.mozilla.org/integration/autoland/rev/306a66424aad
Move urlbar tests to the new urlbar directory and split out tests that already pass with QuantumBar. r=mak
https://hg.mozilla.org/integration/autoland/rev/3ee7a4b57a9a
Enable running some of the urlbar tests against QuantumBar. r=mak

Comment 12

[Cosmin Sabou [:CosminS]]

https://hg.mozilla.org/mozilla-central/rev/ba9f9c0ce256
https://hg.mozilla.org/mozilla-central/rev/306a66424aad
https://hg.mozilla.org/mozilla-central/rev/3ee7a4b57a9a