970669 - gaia-ui-test "filenames" need to go on a diet

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::UI Tests, defect, P4)

Description

[Phil Ringnalda (:philor)]

tbpl believes that in the case of "TEST-UNEXPECTED-FAIL | thing here | failure type here" the "thing here" part is a filename, perhaps a path to a file, so it splits on / or \ and takes the last hunk from that split and returns all the bugs with the intermittent-failure keyword and that filename in the summary as suggestions for starring.

Typical "filenames" for gaia-ui-tests are "test_clock_set_alarm.py test_clock_set_alarm.TestClockSetAlarm.test_clock_set_alarm" and "test_settings_change_keyboard_language.py test_settings_change_keyboard_language.TestChangeKeyboardLanguage.test_change_keyboard_language_settings." That's... verbose. Are the final three sometimes different from the first one? Are they ever all four different? I filed a bug on one of them yesterday, and I didn't actually have any room left in the bug summary for how it failed after the "filename".

Comment 1

[Zac C (:zac)]

Hey Philor, I agree. we already discussed a few months ago naming rules to simplify these, although we never took action on it.

I'll dig it out and see what we can do.

Comment 2

[Florin Strugariu [:Bebe]]

From a long forgotten thread: 
https://mail.mozilla.org/private/gaia-ui-automation/2013-August/000386.html

Zac:

We could also come up with some rules for this to make it systematic to follow:

Filename:
test_<appname>_<verb/action>_<whatistested>.py

Class:
Test<Appname><Subsectionofapp>

Test method:
test_<verb/action>_<whatistested>


Bebe:

But if we will have multiple tests in the same area we would have:

test_settings_change_keyboard_language.py - TestSettingsKeyboard.test_change_keyboard_language
test_settings_change_keyboard_style.py - TestSettingsKeyboard.test_change_keyboard_style
test_settings_change_keyboard_something_else.py - TestSettingsKeyboard.test_change_keyboard_something_else

Dave:

The small suggestion I would make is to remove the folder name from the test file name, as it's implied by the parent folder. For example settings/test_settings_* would be settings/test_*

Comment 3

[Zac C (:zac)]

Thanks Bebe! 

There's one thing blocking this which is if there are two test methods with the same name but different classes then MarionetteTestRunner will report it twice. I need to file it as a bug with a test case.

Comment 4

[Joel Maher ( :jmaher ) (UTC -8)]

is this bug still active, I am under the impression firefox os is not an active project anymore.