1139215 - test_launch_l10n.py:test_launch_by_localised_name consistently fails on-device

Status: RESOLVED WONTFIX

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

Example:

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.b2g-inbound.tinderbox.ui.unit/2895/testReport/%28root%29/test_launch_l10n_py%20TestLaunchL10n_test_launch_by_localised_name/test_launch_l10n_py_TestLaunchL10n_test_launch_by_localised_name/

Error Message

Expected PASS, got FAIL

Stacktrace

Traceback (most recent call last):
  File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.tinderbox.ui.unit/.env/local/lib/python2.7/site-packages/marionette_client-0.9-py2.7.egg/marionette/marionette_test.py", line 290, in run
    testMethod()
  File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.tinderbox.ui.unit/tests/python/gaia-ui-tests/gaiatest/tests/unit/test_launch_l10n.py", line 40, in test_launch_by_localised_name
    self.apps.launch(app)
  File "/var/jenkins/1/workspace/flame-kk-319.b2g-inbound.tinderbox.ui.unit/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 67, in launch
    assert result, "Failed to launch app with name '%s'" % name

AssertionError: Failed to launch app with name '\u015e\u1e17\u1e17\u0167\u0167\u012b\u012b\u019e\u0260\u015f'

I can reproduce the issue "manually" by walking through the test's Marionette calls in REPL, and it might very well be a product bug. 

Settings will launch with the English name, but not the Unicode name.

Comment 1

[Staś Małolepszy :stas]

(In reply to Geo Mealer [:geo] from comment #0)
> AssertionError: Failed to launch app with name
> '\u015e\u1e17\u1e17\u0167\u0167\u012b\u012b\u019e\u0260\u015f'

The Unicode name is Şḗḗŧŧīīƞɠş and this might be related to pseudolocales, although I'm not sure it is.

Marionette's launch method uses https://github.com/mozilla-b2g/gaia/blob/f0fc93d9b3db93c4fd0b6062366e3b1ec4349d25/tests/atoms/gaia_apps.js#L126 to find the name in the manifest and launch the app.

Pseudolocales can be built on buildtime (by being listed in LOCALES_FILE) in which case our multilocale logic will put localized app names into the manifest, or they can be produced dunamically on runtime, in which case the ManifestHelper takes care of localizing app names:

https://github.com/mozilla-b2g/gaia/blob/827289ee7f74292a17bfa73a89e43d5a2bcc4e86/shared/js/manifest_helper.js#L17

I'd expect this test to fail with runtime pseudlocales (e.g. Runtime Accented) b/c the localized name shouldn't be present in the manifest which is used to launch the app.  However, I also see it consistently fail on a device with pre-built pseudolocales (e.g. Packages Accented) in which case the proper localized name is present in the manifest (I checked).  It's not clear to me why the locateWithName method fails.

Comment 2

[Geo Mealer [:geo] -- This account is inactive after 2015-07-07]

I haven't looked at this for quite awhile, and really should have unassigned myself. I have disabled the test in bug 1165026, at least.

What I'm hearing from you re: comment 1 is that there might be an issue, but the specific case this test touches shouldn't work anyway. If that's so, we at least need to update the test if we're going to keep it.

I am, however, suspicious of the device-side atom code it hits. There's a lot going on in between the test and the actual launch, and (as you hinted in the other bug) I'm we might be testing the assumptions our shim makes way more than anything real. 

Any ideas as to a non-automated scenario (or at least simpler snippet of code) we can try to isolate away the framework behavior? Doesn't have to be repeatable, I just want any hint as to whether the test is identifying a real problem or not.

Comment 3

[BMO Automation]

Firefox OS is not being worked on