Open Bug 1214362 Opened 9 years ago Updated 1 year ago

Enable DOM push Mochitests on Android

Categories

(Core :: DOM: Push Subscriptions, task, P2)

task

Tracking

()

Tracking Status
firefox44 --- affected

People

(Reporter: nalexander, Assigned: saschanaz)

References

Details

Attachments

(3 files, 12 obsolete files)

2.39 KB, patch
kmoir
: review+
kmoir
: checkin+
Details | Diff | Splinter Review
2.35 KB, patch
kmoir
: review+
Details | Diff | Splinter Review
1.12 KB, patch
Details | Diff | Splinter Review
Right now, we're not running M(p) on Android Fennec. I don't know why that is, but we're going to want them pretty much now. This ticket tracks enabling that test suite, which might be as simple as updating https://dxr.mozilla.org/mozilla-central/source/dom/push/test/mochitest.ini and addressing fallout. N.b.: this is only for Fennec, i.e., not for b2gdroid! (Eventually we can find out how to support b2gdroid, but this isn't blocked on that.)
Component: General → DOM: Push Notifications
Product: Firefox for Android → Core
edwong: I will push a try build, but I may not be able to drive this too far. Can you add this to your "to wrangle" list?
Flags: needinfo?(edwong)
Depends on: 1214366
Worth noting that the tests themselves uses PlacesUtils: https://dxr.mozilla.org/mozilla-central/source/dom/push/test/xpcshell/head.js#13.
kmoir: can you arrange to run mochitest-push on all Android platforms? There's some Node server needed; I know nothing more. Thanks!
Flags: needinfo?(kmoir)
I don't see a new test job running on the try runs. Is this just a change in tree or is a new test job needed to be added to the list of build jobs for Android?
Flags: needinfo?(kmoir)
(In reply to Kim Moir [:kmoir] from comment #6) > I don't see a new test job running on the try runs. Is this just a change > in tree or is a new test job needed to be added to the list of build jobs > for Android? jlund tells me it's a new test job: 17:05 nalexander: jlund: can you redirect me to somebody who would know how to turn on M(p) for Fennec? That's "MochiTest push" 17:05 nalexander: jlund: I can't figure out where this stuff is defined. 17:06 nalexander: Hmm, maybe: "mochitest-push": ["--subsuite=push"], 17:06 jlund: nalexander: where is it currently run? 17:07 nalexander: jlund: linux, windows, mac AFAICT. 17:07 nalexander: jlund: it's in these mozharness/unittest files. 17:07 nalexander: jlund: maybe that just needs to go into the Android config. 17:07 nalexander: jlund: like https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/android/androidarm.py#79 17:08 nalexander: jlund: perhaps with node stuff in the mozharness as well. 17:08 jlund: nalexander: off a glance, we need to 1) schedule it in a similar manner as https://dxr.mozilla.org/build-central/source/buildbot-configs/mozilla-tests/config.py#2379 17:08 nalexander: jlund: if I push a try build with changes to mozharness, it'll work. 17:08 jlund: and 2) as you say add the mozharness config 17:08 nalexander: jlund: oh, ffs. 17:08 jlund: nalexander: I don't think it will work as this is still scheduled through buildbots afaik. 17:08 nalexander: jlund: alright, who do I flag to deal with this? 17:09 nalexander reads commit messages 17:09 jlund: nalexander: kmoir-afk knows the most about android tests (https://hg.mozilla.org/build/buildbot-configs/annotate/917902a0847052e350c591c2e235ed3e20aad983/mozilla-tests/mobile_config.py)
Flags: needinfo?(kmoir)
I'll take a look at this but I have a couple other bugs ahead of this in the my queue
Assignee: nobody → kmoir
Flags: needinfo?(kmoir)
vladC: This bug is a request to add mochitest-push on android. I would only work on enabling it on Android 2.3 and Android 4.3 since Android 4.0 are riding themselves off the trains (being deprecated). So I the first step I would take suggest would be write patches to update https://dxr.mozilla.org/mozilla-central/source/testing/mozharness/configs/android/androidarm_4_3.py to include mochitest-push. You could use mochitest-gl as an example but instead of --subsuite=webgl you would have --subsuite=push Not sure how many chunks this test requires, I would assume one to start or ask on the bug. You will also have to add this new test suite to http://hg.mozilla.org/build/buildbot-configs/file/ad46981f4fc2/mozilla-tests/mobile_config.py and have it ride the trains You could enable these tests on cedar to test if you'd like. I find running tests on a branch much easier than setting them up in staging.
Attached you can find the patch that add mochitest-push in android 4.3
Attachment #8682583 - Flags: review?(kmoir)
(In reply to Vlad Ciobancai [:vladC] from comment #10) > Created attachment 8682583 [details] [diff] [review] > bug1214362_androidarm_4_3.py.patch > > Attached you can find the patch that add mochitest-push in android 4.3 Wow, thanks for jumping on this so fast. If you can provide as much explanation of what you're doing with buildbot and cedar as possible, I'd appreciate it -- me and vivek want to run more jobs like this in automation.
:nalexander I started working on that but I don't think I will finish today. I'm on EET time zone
Comment on attachment 8682583 [details] [diff] [review] bug1214362_androidarm_4_3.py.patch You also have to add "mochitest-push to "test_suite_definitions": {... like the other tests
Attachment #8682583 - Flags: review?(kmoir) → review-
Updated the patch
Attachment #8682583 - Attachment is obsolete: true
Comment on attachment 8683055 [details] [diff] [review] bug1214362_androidarm_4_3.py.patch looks good but we won't land until you have tested on cedar with the associated mobile buildbot configs
Attachment #8683055 - Flags: review?(kmoir) → review+
Attached the patch to add mochitest-push in mobile_config.py
Attachment #8683104 - Flags: review?(kmoir)
Attached file difference_bug1214362.txt (obsolete) —
Attached the difference
I updated the pathch in order to remove the tab lines
Attachment #8683104 - Attachment is obsolete: true
Attachment #8683104 - Flags: review?(kmoir)
Attachment #8683107 - Flags: review?(kmoir)
Updated the patch by removing the tab lines
Attachment #8683055 - Attachment is obsolete: true
Attachment #8683109 - Flags: review?(kmoir)
Comment on attachment 8683109 [details] [diff] [review] bug1214362_androidarm_4_3.py.patch Good progress so far :-) I thought we were going test run this test on cedar instead of just all branches? In this case, you'll have to add this test to cedar only by specifying it separately and then adding it to the suites that run on cedar. Also, what platform will you test it on? Probably 2.3 adn 4.3 would be best as Android 4.0 is being deprecated. ANDROID_4_3_MOZHARNESS_DICT and ANDROID_2_3_MOZHARNESS_DICT are the dictionaries in this case, not ANDROID_UNITTEST_DICT. A builder list to verify this would be great!
Attachment #8683109 - Flags: review?(kmoir) → review-
Attached patch bug1214362_androidarm.py.patch (obsolete) — Splinter Review
Added mochitest-push in android/androidarm.py and also in android/androidarm_4_3.py
Attachment #8683109 - Attachment is obsolete: true
Attachment #8683679 - Flags: review?(kmoir)
Attachment #8683679 - Flags: review?(kmoir) → review+
Comment on attachment 8683107 [details] [diff] [review] bug1214362_mobile_config.py.patch I commented on this last night on comment #20
Attachment #8683107 - Flags: review?(kmoir) → review-
Updated the mobile_config file
Attachment #8683107 - Attachment is obsolete: true
Attachment #8684222 - Flags: review?(kmoir)
Comment on attachment 8684222 [details] [diff] [review] bug1214362_mobile_config.py.patch Do we want this test to be enabled on all branches or for it to ride the trains? Usually when we enable a new test we enable it to ride the trains because the associated mozharness changes need to be uplifted for the tests to run. Could you ask nalexander on the bug? If this is the case, we will need to enable this test on a based on the version of gecko similar to what you did for enabling 10.10.5 on trunk
:nalexander I would like to ask you, the new test needs to be enabled for all branches or just for it to ride the trains?
Flags: needinfo?(nalexander)
(In reply to Vlad Ciobancai [:vladC] from comment #25) > :nalexander I would like to ask you, the new test needs to be enabled for > all branches or just for it to ride the trains? I interpret this to mean, what branches do I want to enable M(p) on Android on? I'm happy with the integration branches (fx-team, mozilla-inbound) and Nightly (mozilla-central). It can ride the trains after that. It will be orange for a while on Nightly, anyway: we'll be disabling a bunch of tests on day one, since they assume Desktop...
Flags: needinfo?(nalexander)
VladC: Here is a bit of code to enable on enable the tests on android 4.3 on cedar. This will allow you to test that your mozharness scripts work etc before we enable on all branches. You can modify this to run on other android platforms too. MOCHITEST_PUSH = [ ('mochitest-push-1', { 'use_mozharness': True, 'script_path': 'scripts/android_emulator_unittest.py', 'extra_args': [ '--cfg', 'android/androidarm_4_3.py', '--test-suite', 'mochitest-push-1', ], 'blob_upload': True, 'timeout': 2400, 'script_maxtime': 14400, }, ), ] BRANCHES['cedar']['platforms']['android-api-11']['ubuntu64_vm_armv7_mobile']['opt_unittest_suites'] += MOCHITEST_PUSH BRANCHES['cedar']['platforms']['android-api-11']['ubuntu64_vm_armv7_mobile']['debug_unittest_suites'] += MOCHITEST_PUSH
Updated the patch with the suggestion received from :kmoir
Attachment #8684222 - Attachment is obsolete: true
Attachment #8684222 - Flags: review?(kmoir)
Attachment #8685481 - Flags: review?(kmoir)
Comment on attachment 8685481 [details] [diff] [review] bug1214362_mobile_config.py.patch You don't need to define ANDROID_2_3_MOCHITEST_PUSH and ANDROID_4_3_MOZHARNESS_DICT you can just define it once and use it for both android 2.3 and android 4.3 In the part where you define the tests to be added to cedar, you need to have an android-api-9 platform defined, for debug and opt, not just android-api-11. This must also specify the correct associated slave platform. Please attach a builder diff with the next patch
actually, the two definitions for ANDROID_2_3_MOCHITEST_PUSH and ANDROID_4_3_MOZHARNESS_DICT and fine. And Android 2.3's platform is android, not android-api-9 as discussed in our quick video call
Updated the patch, I run a difference on dev2-master but the results are null. :kmoir can you please take a look and try to give us some hints?
Attachment #8685481 - Attachment is obsolete: true
Attachment #8685481 - Flags: review?(kmoir)
Attachment #8686135 - Flags: review?(kmoir)
Looking at your master,you need to redefine the tests here ubuntu64_vm_mobile' : deepcopy(ANDROID_2_3_MOCHITEST_PUSH_UNITTEST_DICT), We don't want them to run on all branches PLATFORM_UNITTEST_VARS = { 'android': { 'product_name': 'fennec', 'app_name': 'browser', 'brand_name': 'Minefield', 'is_remote': True, 'enable_opt_unittests': True, 'enable_debug_unittests': True, 'remote_extras': ANDROID_UNITTEST_REMOTE_EXTRAS, 'panda_android': deepcopy(ANDROID_MOZHARNESS_PANDA_UNITTEST_DICT), 'ubuntu64_vm_mobile' : deepcopy(ANDROID_2_3_MOCHITEST_PUSH_UNITTEST_DICT), }, If your builders are not being added, it may be an ordering problem, and you need to add the push tests on cedar lower down in the file. You want to add the tests like this not replace them entirely BRANCHES['cedar']['platforms']['android-api-11']['ubuntu64_vm_armv7_mobile']['opt_unittest_suites'] += MOCHITEST_4_3_PUSH to just add them on cedar, I have down this on my test master http://dev-master2.bb.releng.use1.mozilla.com:8036/builders Also, as a general rule, we don't ask for review until we have a working solution. You usually just ask for feedback if you are stuck so use that flag instead :-)
I didn't get a chance to poke at the master yesterday. Any progress to report?
(In reply to Chris Cooper [:coop] from comment #33) > I didn't get a chance to poke at the master yesterday. Any progress to > report? Not really so far, we are still trying to find a solution to enable android platform only for cedar in order to work.
Attachment #8686135 - Attachment is obsolete: true
Attachment #8686135 - Flags: review?(kmoir)
Attachment #8687865 - Flags: review?(kmoir)
Attached file difference_bug1214362.txt (obsolete) —
Attachment #8683106 - Attachment is obsolete: true
Comment on attachment 8687865 [details] [diff] [review] bug1214362_mobile_config.py.patch Why did you remove the panda_android platform here? PLATFORMS['android']['slave_platforms'] = \ - ['panda_android', 'ubuntu64_vm_mobile', 'ubuntu64_vm_
(In reply to Kim Moir [:kmoir] from comment #37) > Comment on attachment 8687865 [details] [diff] [review] > bug1214362_mobile_config.py.patch > > Why did you remove the panda_android platform here? > PLATFORMS['android']['slave_platforms'] = \ > - ['panda_android', 'ubuntu64_vm_mobile', 'ubuntu64_vm_ That's because, at the moment, android is disabled. Since we need to enable mochitest-push jobs for Android 2.3 too, we had to re-enable the platform and remove all the other jobs beside mochitest-push and keep them only on "cedar". So, keeping "panda_android" there would generate extra jobs. Of course, we can remove them afterwards, but that is the most easy way to do that.
Folks, I just realized I gave incomplete information. I need this for Android API 11+, i.e., not for Android API 9, which is 2.3. (Android API 9 does not compile with Google Play Services, which is necessary for Google Cloud Messaging, which we use for Push.) Does that make things simpler?
Flags: needinfo?(kmoir)
yes, Alin, you can remove the Android 2.3 part of the patches
Flags: needinfo?(kmoir)
Attached patch bug1214362.patchSplinter Review
Attached the patch to enable mochitest-push tests on cedar. Builder diff: (push_test)[alin.selagea@dev-master2.bb.releng.use1.mozilla.com push_test]$ diff -pU 1 old new --- old 2015-11-20 00:29:34.367511797 -0800 +++ new 2015-11-20 00:31:27.012425673 -0800 @@ -1382,2 +1382,3 @@ Android 4.3 armv7 API 11+ cedar opt test Android 4.3 armv7 API 11+ cedar opt test cppunit ScriptFactory +Android 4.3 armv7 API 11+ cedar opt test mochitest-push-1 ScriptFactory Android 4.3 armv7 API 11+ cedar debug test cppunit ScriptFactory @@ -1522,2 +1523,3 @@ Android 4.3 armv7 API 11+ cedar debug te Android 4.3 armv7 API 11+ cedar debug test crashtest-4 ScriptFactory +Android 4.3 armv7 API 11+ cedar debug test mochitest-push-1 ScriptFactory Android armv7 API 9 mozilla-beta opt test mochitest-1 ScriptFactory
Attachment #8687865 - Attachment is obsolete: true
Attachment #8687868 - Attachment is obsolete: true
Attachment #8687865 - Flags: review?(kmoir)
Attachment #8689948 - Flags: review?(kmoir)
Comment on attachment 8689948 [details] [diff] [review] bug1214362.patch Looks good. You also need to modify the previous patch since we don't need to modify testing/mozharness/configs/android/androidarm.py
Attachment #8689948 - Flags: review?(kmoir) → review+
Attached the updated patch. Thanks, Kim!
Attachment #8683679 - Attachment is obsolete: true
Attachment #8690101 - Flags: review?(kmoir)
Attachment #8690101 - Flags: review?(kmoir) → review+
Attachment #8689948 - Flags: checkin+
After the reconfig finishes at 1pm ET you can clone cedar and push you mozharness patch there and see if the tests pass. If the tests pass you can write a patch to enable this test on trunk branches, not just cedar. This will require that it the mozharness patch be uplifted to trunk as well.
Actually the reconfig should kick off at noon ET
The results of the push to cedar done by Kim: https://treeherder.mozilla.org/#/jobs?repo=cedar&revision=6de20da10b02 Both mochitest-push jobs failed, the error found in the logs is the following: https://treeherder.mozilla.org/logviewer.html#?job_id=169538&repo=cedar
I think we need to add an alias for mochitest-push I added it and pushed it to cedar https://treeherder.mozilla.org/#/jobs?repo=cedar&revision=499623acdcd7
:kmoir, :aselagea: thanks for your work! I think what's happening is that all the relevant tests are being skipped. Can I push to TH myself to address this, or should I post a patch for you to push?
Flags: needinfo?(kmoir)
Flags: needinfo?(edwong)
Flags: needinfo?(alin.selagea)
the builders are just enabled on cedar for now so if could could push the change there and see if the tests run that would be great
Flags: needinfo?(kmoir)
Flags: needinfo?(alin.selagea)
:nalexander are you planning on landing the patch or cedar so we can enable this test on other branches? Or if you give us a patch to enable the tests, we can land it and push to cedar to run the tests
Flags: needinfo?(nalexander)
(In reply to Nick Alexander :nalexander from comment #51) > https://hg.mozilla.org/projects/cedar/rev/ > 03e770c55c12514acec5fcc92a355445fe52b934 > Bug 1214362 - Enable DOM push mochi and xpcshell tests on Fennec. r?dragana > > https://hg.mozilla.org/projects/cedar/rev/ > 3150c768471cf5800888c009a0f3ed8d752e7038 > Bug 1214362 - Enable DOM push API in Fennec Nightly. r?mfinkle Here goes nothing. Let's see what we see!
Flags: needinfo?(nalexander)
(In reply to Nick Alexander :nalexander from comment #51) > https://hg.mozilla.org/projects/cedar/rev/ > 03e770c55c12514acec5fcc92a355445fe52b934 > Bug 1214362 - Enable DOM push mochi and xpcshell tests on Fennec. r?dragana > > https://hg.mozilla.org/projects/cedar/rev/ > 3150c768471cf5800888c009a0f3ed8d752e7038 > Bug 1214362 - Enable DOM push API in Fennec Nightly. r?mfinkle OK, this failed hard 'cuz dom.push.serverURL is not set, so we try to make a non-local connection, resulting in the entire suite getting hard-killed. See https://treeherder.mozilla.org/#/jobs?repo=cedar&selectedJob=171629 for logs. dragana, kitcambridge: It's not clear to me how this works on Desktop. Can you explain how the M(p) harness gets set up, or re-direct to somebody who knows? Thanks!
Flags: needinfo?(kcambridge)
Flags: needinfo?(dd.mozilla)
(In reply to Nick Alexander :nalexander from comment #53) > (In reply to Nick Alexander :nalexander from comment #51) > > https://hg.mozilla.org/projects/cedar/rev/ > > 03e770c55c12514acec5fcc92a355445fe52b934 > > Bug 1214362 - Enable DOM push mochi and xpcshell tests on Fennec. r?dragana > > > > https://hg.mozilla.org/projects/cedar/rev/ > > 3150c768471cf5800888c009a0f3ed8d752e7038 > > Bug 1214362 - Enable DOM push API in Fennec Nightly. r?mfinkle > > OK, this failed hard 'cuz dom.push.serverURL is not set, so we try to make a > non-local connection, resulting in the entire suite getting hard-killed. > See https://treeherder.mozilla.org/#/jobs?repo=cedar&selectedJob=171629 for > logs. > > dragana, kitcambridge: It's not clear to me how this works on Desktop. Can > you explain how the M(p) harness gets set up, or re-direct to somebody who > knows? Thanks! Push test have subsuite = push which allows non-local connections. http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtests.py#2249 (sets MOZ_DISABLE_NONLOCAL_CONNECTIONS to "0") I am not sure how that works for android.
Flags: needinfo?(dd.mozilla)
I have not debugged this fully, but I suspect that it might help to change: http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtests.py#2253 del self.browserEnv['MOZ_DISABLE_NONLOCAL_CONNECTIONS'] to: self.browserEnv['MOZ_DISABLE_NONLOCAL_CONNECTIONS'] = '0'
(In reply to Geoff Brown [:gbrown] from comment #55) > I have not debugged this fully, but I suspect that it might help to change: > > http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtests. > py#2253 > > del self.browserEnv['MOZ_DISABLE_NONLOCAL_CONNECTIONS'] > > to: > > self.browserEnv['MOZ_DISABLE_NONLOCAL_CONNECTIONS'] = '0' https://treeherder.mozilla.org/#/jobs?repo=cedar&revision=ba09f0129780 http://archive.mozilla.org/pub/mobile/tinderbox-builds/cedar-android-api-11/1449773273/cedar_ubuntu64_vm_armv7_mobile_test-mochitest-push-1-bm121-tests1-linux64-build0.txt.gz 12:05:52 INFO - 0 INFO SimpleTest START 12:05:52 INFO - 1 INFO TEST-START | dom/push/test/test_data.html 12:06:28 INFO - 2 INFO TEST-UNEXPECTED-FAIL | dom/push/test/test_data.html | Some test failed with error AbortError: Error retrieving push subscription 12:06:28 INFO - runTest/<@dom/push/test/test_data.html:172:7 12:06:28 INFO - 3 INFO TEST-OK | dom/push/test/test_data.html | took 31781ms 12:06:28 INFO - 4 INFO TEST-START | dom/push/test/test_has_permissions.html 12:06:28 INFO - 5 INFO TEST-OK | dom/push/test/test_has_permissions.html | took 6150ms 12:06:40 INFO - 6 INFO TEST-START | dom/push/test/test_multiple_register.html 12:06:40 INFO - 7 INFO TEST-OK | dom/push/test/test_multiple_register.html | took 6434ms 12:06:40 INFO - 8 INFO TEST-START | dom/push/test/test_multiple_register_different_scope.html 12:06:52 INFO - 9 INFO TEST-OK | dom/push/test/test_multiple_register_different_scope.html | took 8494ms 12:06:52 INFO - 10 INFO TEST-START | dom/push/test/test_multiple_register_during_service_activation.html 12:07:04 INFO - 11 INFO TEST-OK | dom/push/test/test_multiple_register_during_service_activation.html | took 8599ms 12:07:04 INFO - 12 INFO TEST-START | dom/push/test/test_permissions.html 12:07:16 INFO - 13 INFO TEST-OK | dom/push/test/test_permissions.html | took 6231ms 12:07:16 INFO - 14 INFO TEST-START | dom/push/test/test_register.html 12:07:27 INFO - 15 INFO TEST-OK | dom/push/test/test_register.html | took 11679ms 12:07:27 INFO - 16 INFO TEST-START | dom/push/test/test_serviceworker_lifetime.html 12:08:18 INFO - 17 INFO TEST-OK | dom/push/test/test_serviceworker_lifetime.html | took 48924ms 12:08:18 INFO - 18 INFO TEST-START | dom/push/test/test_unregister.html 12:08:18 INFO - 19 INFO TEST-OK | dom/push/test/test_unregister.html | took 5682ms I don't have any insight into the new error, but I think the non-local connection issue is fixed.
Depends on: 1243856
Assignee: kmoir → alin.selagea
Depends on: 1245596
We're going to change the mochitests to avoid making external calls, either via a test backend, or mock server (bug 1244816).
Flags: needinfo?(kcambridge)
See Also: → 1311849
Assignee: aselagea → nobody
Priority: -- → P5
Summary: Enable DOM push Mochitests on Android Fennec → Enable DOM push Mochitests on Android
Severity: normal → S4
Assignee: nobody → krosylight

It fails hard there, huh.

Type: defect → task
Severity: S4 → S3
Priority: P5 → P2

Logcat says:

  • [JavaScript Error: "TypeError: pushService.replaceServiceBackend is not a function" {file: "http://mochi.test:8888/tests/dom/push/test/mockpushserviceparent.js" line: 75}]
  • [JavaScript Error: "Error restoring service: TypeError: can't access property "uninit", pushService.service is undefined" {file: "http://mochi.test:8888/tests/dom/push/test/mockpushserviceparent.js" line: 195}].

Turns out we are using a different PushService module for GeckoView. We use PushComponents.sys.mjs for desktop and GeckoViewPush.sys.mjs for GeckoView. Somehow those modules have diffferent members and thus the mocked module fails to use the latter.

(It's fun, didn't know live_backing does not cover every log on GeckoView)

Depends on: 1847217
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: