1814884 - Add generic marionette test that checks startup with a clean profile and verifies no error console messages are created

Status: RESOLVED FIXED

(Firefox :: General, task, P3)

Description

[:Gijs (he/him)]

We keep having issues where people accidentally add console spam. This is annoying when debugging issues with users (and trying to find the salient bits), and often reflects errors in implementation/testing if they were not otherwise caught before landing in nightly.

To remedy this, I think it'd help if we added a marionette test that starts the browser with a clean new profile, and with an existing profile, and waits for startup to be complete (including all the idle tasks etc.) and then checks that there are no (pending) console messages.

We'll initially need an allowlist of things that already exist, but hopefully this would clamp down on patches that introduce new errors/message spam.

Comment 1

[:Gijs (he/him)]

Recent examples: bug 1813320, bug 1814688, bug 1743495, etc.

Comment 2

[BugBot [:suhaib / :marco/ :calixte]]

The Bugbug bot thinks this bug should belong to the 'DevTools::Console' component, and is moving the bug to that component. Please correct in case you think the bot is wrong.

Comment 3

[:Gijs (he/him)]

Attachment: Bug 1814884 - add marionette test to ensure we catch newly introduced startup console spew, r?#devtools-reviewers,bgrins!

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/fc859e1a124c
add marionette test to ensure we catch newly introduced startup console spew, r=webdriver-reviewers,devtools-reviewers,bgrins,nchevobbe,whimboo

Comment 5

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/fc859e1a124c

Comment 6

[BugBot [:suhaib / :marco/ :calixte]]

Since nightly and release are affected, beta will likely be affected too.
For more information, please visit auto_nag documentation.