1406042 - Copy relevant old console test to the new frontend

Status: RESOLVED FIXED

(DevTools :: Console, enhancement, P1)

Description

[Nicolas Chevobbe [:nchevobbe]]

In this bug, all the bug tagged as relevant in https://docs.google.com/spreadsheets/d/11RBIwROFsdPzEXkt94Bzk-y49KxRu08dwC6hvA-MOGA/edit#gid=0, or that have the "Browser Console" extra should be copied other the new frontend test directory.
- The tests that contains the bug number in their file should be renamed
- The tests that are prefixed with browser_console without actually testing the browser console should be renamed browser_webconsole_xx
- The tests that are about the console input should be renamed browser_jsterm_xx
- The support files should also be copied (and renamed if matches some of the elements described above)

Then, in the browser.ini file, all the new test should have a `skip-if=true`, followed by the bug number in which should happen the migration, if there is one in the dashboard.

Beware of also copying any extra info (subsuite, skip on specific platforms, …) from the old browser-ini

Comment 1

[Nicolas Chevobbe [:nchevobbe]]

Attachment: Bug 1406042 - Copy relevant old console test to the new frontend;

This patch copies the tests from the old frontend that are relevant
in the context of the new frontend. They are all added in the
browser.ini and all are skipped. The number of the bug to do the test
migration is indicated as a comment.
Some tests were renamed to match what they actually do, or to match
which component they test (console, webconsole, jsterm), or simply to remove
a bug number in the filename. In the latter case, a reference to the bug
was added inside the test if it wasn't already the case.

All the support files from the old frontend were copied over, because it would
have been time consuming to test if each one of them was used in the tests. A bug
will be filed to do a support files cleanup when all the tests are migrated.

Review commit: https://reviewboard.mozilla.org/r/187144/diff/#index_header
See other reviews: https://reviewboard.mozilla.org/r/187144/

Comment 2

[Nicolas Chevobbe [:nchevobbe]]

Comment on attachment 8915887 [details]
Bug 1406042 - Copy relevant old console test to the new frontend;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187144/diff/1-2/

Comment 3

[Jan Honza Odvarko [:Honza] (always need-info? me)]

Comment on attachment 8915887 [details]
Bug 1406042 - Copy relevant old console test to the new frontend;

https://reviewboard.mozilla.org/r/187144/#review192202

Looks reasonable to me.

R+ assuming try is green.

This is great amount of work, thanks!

Honza

Comment 4

[Nicolas Chevobbe [:nchevobbe]]

I'm seeing ESLint failures, I think eslint was disabled on the old tests. Let me eslint ignore the new tests, and we'll enable them one by one when doing the migration.

Comment 5

[Nicolas Chevobbe [:nchevobbe]]

Comment on attachment 8915887 [details]
Bug 1406042 - Copy relevant old console test to the new frontend;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187144/diff/2-3/

Comment 6

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/8149514f0473
Copy relevant old console test to the new frontend; r=Honza

Comment 7

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backed out for failing devtools/client/webconsole/new-console-output/test/mochitest/browser_webconsole_property_provider.js:

https://hg.mozilla.org/integration/autoland/rev/e27e466cec06d42e12328206bc5026bd68d5f7fa

That got catched by the early running test-verify which runs modified tests. It failed in the first run, so either it will also fail in devtools runs or depends on the status from a test which usually runs before it.

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&revision=8149514f04738b632122746c6b03d8ec420197f9&filter-resultStatus=testfailed&filter-resultStatus=busted&filter-resultStatus=exception&filter-resultStatus=retry&filter-resultStatus=usercancel&filter-resultStatus=runnable&filter-resultStatus=success&filter-searchStr=test-verify
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=135517779&repo=autoland

[task 2017-10-07T14:26:42.079Z] 14:26:42     INFO - *** Start BrowserChrome Test Results ***
[task 2017-10-07T14:26:42.120Z] 14:26:42     INFO - checking window state
[task 2017-10-07T14:26:42.182Z] 14:26:42     INFO - TEST-START | devtools/client/webconsole/new-console-output/test/mochitest/browser_webconsole_property_provider.js
[task 2017-10-07T14:26:42.359Z] 14:26:42     INFO - TEST-INFO | started process screentopng
[task 2017-10-07T14:26:43.327Z] 14:26:43     INFO - TEST-INFO | screentopng: exit 0
[task 2017-10-07T14:26:43.328Z] 14:26:43     INFO - TEST-UNEXPECTED-FAIL | devtools/client/webconsole/new-console-output/test/mochitest/browser_webconsole_property_provider.js | Exception thrown - at chrome://mochitests/content/browser/devtools/client/webconsole/new-console-output/test/mochitest/browser_webconsole_property_provider.js:14 - ReferenceError: loadTab is not defined

Comment 8

[Nicolas Chevobbe [:nchevobbe]]

Comment on attachment 8915887 [details]
Bug 1406042 - Copy relevant old console test to the new frontend;

Review request updated; see interdiff: https://reviewboard.mozilla.org/r/187144/diff/3-4/

Comment 9

[Nicolas Chevobbe [:nchevobbe]]

This is a test migration and I forgot to mark that one as always skipped.
Doing it now, and it will be enabled in a specific bug.

Comment 10

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a5584789d1d0
Copy relevant old console test to the new frontend; r=Honza

Comment 11

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/a5584789d1d0