1192967 - Create Imagecompare RTL test for Settings:Account Management

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::UI Tests, defect)

Description

[No-Jun Park [:njpark]]

Create Imagecompare script that covers the Account Management section of Settings app

Comment 1

[GitHub Autolander Bot]

Attachment: [gaia] npark-mozilla:1192967 > mozilla-b2g:master

Comment 2

[Martijn Wargers (dead)]

Not adding it to the manifest.ini file?

Comment 3

[Martijn Wargers (dead)]

Never mind, it is already added.

Comment 4

[No-Jun Park [:njpark]]

ah, but i need to uncomment it

Comment 5

[Martijn Wargers (dead)]

Comment on attachment 8645992 [details] [review]
[gaia] npark-mozilla:1192967 > mozilla-b2g:master

I ran the test and I got a difference between 1 test:
http://people.mozilla.org/~mwargers/imgcomparescreenshots/mismatches4/
But that seems to be a real bug in the app.

Also got this error, but that's bug 1173819:
TEST-START | test_settings_AC_RTL.py TestSettingsRTLAC.test_settings_app_invalid_password
TEST-UNEXPECTED-ERROR | test_settings_AC_RTL.py TestSettingsRTLAC.test_settings_app_invalid_password | JavascriptException: JavascriptException: TypeError: window.wrappedJSObject.Service.query(...) is undefined
stacktrace:
	execute_script @gaia_test.py, line 113
	inline javascript, line 370
	src: "      app = window.wrappedJSObject.Service"



Traceback (most recent call last):
  File "/Users/mwargers/.virtualenvs/test3/lib/python2.7/site-packages/marionette_client-0.16-py2.7.egg/marionette/marionette_test.py", line 277, in run
    self.setUp()
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/tests/graphics/RTL/test_settings_AC_RTL.py", line 12, in setUp
    GaiaImageCompareTestCase.setUp(self)
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/gaia_graphics_test.py", line 46, in setUp
    GaiaTestCase.setUp(self)
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 901, in setUp
    self.cleanup_gaia(full_reset=False)
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 945, in cleanup_gaia
    self.device.turn_screen_off()
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 724, in turn_screen_off
    apps.switch_to_displayed_app()
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 121, in switch_to_displayed_app
    self.marionette.switch_to_frame(self.displayed_app.frame)
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/gaia_test.py", line 113, in displayed_app
    result = self.marionette.execute_script('return GaiaApps.getDisplayedApp();')
  File "/Users/mwargers/.virtualenvs/test3/lib/python2.7/site-packages/marionette_driver-0.9-py2.7.egg/marionette_driver/marionette.py", line 1509, in execute_script
    filename=os.path.basename(frame[0]))
  File "/Users/mwargers/.virtualenvs/test3/lib/python2.7/site-packages/marionette_driver-0.9-py2.7.egg/marionette_driver/decorators.py", line 36, in _
    return func(*args, **kwargs)
  File "/Users/mwargers/.virtualenvs/test3/lib/python2.7/site-packages/marionette_driver-0.9-py2.7.egg/marionette_driver/marionette.py", line 715, in _send_message
    self._handle_error(response)
  File "/Users/mwargers/.virtualenvs/test3/lib/python2.7/site-packages/marionette_driver-0.9-py2.7.egg/marionette_driver/marionette.py", line 751, in _handle_error
    raise errors.lookup(status)(message, stacktrace=stacktrace)
TEST-INFO took 54228ms
TEST-START | test_settings_AC_RTL.py TestSettingsRTLAC.test_settings_app_unverified_acct
TEST-PASS | test_settings_AC_RTL.py TestSettingsRTLAC.test_settings_app_unverified_acct | took 104478ms
TEST-START | test_settings_AC_RTL.py TestSettingsRTLAC.test_settings_app_valid_login
TEST-UNEXPECTED-ERROR | test_settings_AC_RTL.py TestSettingsRTLAC.test_settings_app_valid_login | AssertionError: WARNING: 13190 pixels mismatched between screenshots/test_settings_app_valid_login_findmydevice-password_flame_en-US_2+2015-08-11-19-45-29.png and reference_images/test_settings_app_valid_login_findmydevice-password_flame_en-US_2.png
This screenshot and diff file will be moved to screenshots/mismatches


Traceback (most recent call last):
  File "/Users/mwargers/.virtualenvs/test3/lib/python2.7/site-packages/marionette_client-0.16-py2.7.egg/marionette/marionette_test.py", line 326, in run
    self.tearDown()
  File "/Users/mwargers/B2G/gaia_clean/tests/python/gaia-ui-tests/gaiatest/gaia_graphics_test.py", line 51, in tearDown
    self.assertTrue(self.test_passed, msg=self.failcomment)
TEST-INFO took 119056ms

SUMMARY
-------
passed: 1
failed: 2
todo: 0

FAILED TESTS

Comment 6

[No-Jun Park [:njpark]]

(In reply to Martijn Wargers [:mwargers] (QA) from comment #5)
> Comment on attachment 8645992 [details] [review]
> [gaia] npark-mozilla:1192967 > mozilla-b2g:master
> 
> I ran the test and I got a difference between 1 test:
> http://people.mozilla.org/~mwargers/imgcomparescreenshots/mismatches4/
> But that seems to be a real bug in the app.


That's odd, I didn't get this error yesterday when I ran it, I should make a note of this. thanks!

Comment 7

[No-Jun Park [:njpark]]

Merged: https://github.com/mozilla-b2g/gaia/commit/a3240939ebacdb8bc9fe1847a31b2b1b13a86472