Closed Bug 1188337 Opened 10 years ago Closed 10 years ago

Enable Mulet TaskCluster Gu

Categories

(Firefox OS Graveyard :: Runtime, defect)

Product:
Firefox OS Graveyard
Component:
Runtime
Platform:
ARM
Gonk (Firefox OS)
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(firefox43 fixed)

Status:
RESOLVED FIXED
Milestone:
FxOS-S4 (07Aug)
Tracking Flags:
Tracking Status
firefox43 --- fixed

People

(Reporter: gerard-majax, Assigned: gerard-majax)

References

Details

(Whiteboard: [systemsfe])

Attachments

(2 files, 12 obsolete files)

Mulet Gaia Unit tests with chunking
3.98 KB, patch
gerard-majax
: review+
Details | Diff | Splinter Review
Enable Mulet TaskCluster Gu and prepare for Gu-oop
9.42 KB, patch
gerard-majax
: review+
Details | Diff | Splinter Review
Let's enable running Gu with Mulet on TaskCluster
Aus, it looks like we are properly making use of Mulet to run this, but I'm not 100% certain and reading the logs to understand this is not very easy. Can you give it a look? Thanks!
Flags: needinfo?(aus)
Attachment #8639836 - Attachment is obsolete: true
Looks like this one has some issues and isn't calling the right binary (and that's why it's failing to find it in the first place). See this log snippet: 17:34:22 INFO - Calling ['/home/worker/build/venv/bin/python', '-u', '/home/worker/gaia/tests/python/gaia-unit-tests/gaia_unit_test/main.py', '--binary', '/home/worker/build/application/firefox/b2g-bin', '--profile', '/home/worker/gaia/profile-debug', '--symbols-path', 'https://queue.taskcluster.net/v1/task/N3hOrVWqQVSHHTYImrY11w/artifacts/public/build/target.linux-x86_64.crashreporter-symbols.zip'] with output_timeout 1760 I'm guessing that should be calling something else. The '--binary' and following argument value are the ones that look wrong to me.
Flags: needinfo?(aus)
Yes, and yet I do change them in the python code.
Hacked in build-mozharness and changed JSON to point to it: https://treeherder.mozilla.org/#/jobs?repo=try&revision=4294364ddaa7
(In reply to Alexandre LISSY :gerard-majax from comment #7) > Hacked in build-mozharness and changed JSON to point to it: > https://treeherder.mozilla.org/#/jobs?repo=try&revision=4294364ddaa7 I'm seeing some errors on Gij but that are related to npm stuff. Invalid 'ui' etc., remind me of some mess ... Gareth, can you have a look?
Flags: needinfo?(gaye)
(In reply to Alexandre LISSY :gerard-majax from comment #8) > (In reply to Alexandre LISSY :gerard-majax from comment #7) > > Hacked in build-mozharness and changed JSON to point to it: > > https://treeherder.mozilla.org/#/jobs?repo=try&revision=4294364ddaa7 > > I'm seeing some errors on Gij but that are related to npm stuff. Invalid > 'ui' etc., remind me of some mess ... Gareth, can you have a look? Commented on the wrong bug ...
Flags: needinfo?(gaye)
(In reply to Alexandre LISSY :gerard-majax from comment #7) > Hacked in build-mozharness and changed JSON to point to it: > https://treeherder.mozilla.org/#/jobs?repo=try&revision=4294364ddaa7 As expected, Gu now fails on B2G Desktop because it makes use of "b2g/firefox" binary :)
Still depends on mozharness being intree but it looks good.
Attachment #8639918 - Attachment is obsolete: true
Attachment #8640433 - Flags: review?(garndt)
(In reply to Alexandre LISSY :gerard-majax from comment #12) > Created attachment 8640433 [details] [diff] [review] > Enable Mulet TaskCluster Gu/Gu-oop > > Still depends on mozharness being intree but it looks good. That is working and exposing some failures: 11:30:21 INFO - TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should emit a 'busy', then 'ready' event - this.mozCamera.release(...) is undefined 11:30:21 INFO - TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should call `navigator.mozCameras.getCamera()` with currently selected camera - this.mozCamera.release(...) is undefined 11:30:21 INFO - TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should call get camera with the passed config - this.mozCamera.release(...) is undefined 11:30:21 INFO - TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should flag a `this.configured` - this.mozCamera.release(...) is undefined 11:30:21 INFO - TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should call .setupNewCamera - this.mozCamera.release(...) is undefined 11:30:21 INFO - TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should emit a 'configured' - this.mozCamera.release(...) is undefined 11:32:03 INFO - TEST-UNEXPECTED-FAIL | settings/test/unit/modules/state_model_test.js | StateModel set target state the final state should be the last state if success - timeout of 10000ms exceeded 11:33:50 INFO - TEST-UNEXPECTED-FAIL | settings/test/unit/panels/operator_settings/models/auto_selection_model_test.js | AutoSelectionModel set target state mobile connection is busy the final state should be the last targetState - timeout of 10000ms exceeded 11:36:32 INFO - TEST-UNEXPECTED-FAIL | system/test/unit/lockscreen/lockscreen_notifications_test.js | system/LockScreenNotifications Remove top mask when actionable notification is at the visual top ofthe container viewport top-actionable class is correctly added in onNotificationHighlighted test on 0-th notification - with notification index 0 container should not have the top-actionable class: expected true to equal false 11:36:32 INFO - TEST-UNEXPECTED-FAIL | system/test/unit/lockscreen/lockscreen_notifications_test.js | system/LockScreenNotifications Remove top mask when actionable notification is at the visual top ofthe container viewport top-actionable class is correctly added in onNotificationHighlighted test on 1-th notification - with notification index 1 container should have the top-actionable class: expected false to equal true 11:36:32 INFO - TEST-UNEXPECTED-FAIL | system/test/unit/lockscreen/lockscreen_notifications_test.js | system/LockScreenNotifications Remove top mask when actionable notification is at the visual top ofthe container viewport top-actionable class is correctly removed with tryAddTopMaskByNotification test on 0-th notification - with notification index 0 container should have the top-actionable class: expected false to equal true 11:36:32 INFO - TEST-UNEXPECTED-FAIL | system/test/unit/lockscreen/lockscreen_notifications_test.js | system/LockScreenNotifications Remove top mask when actionable notification is at the visual top ofthe container viewport top-actionable class is correctly removed with tryAddTopMaskByNotification test on 1-th notification - with notification index 1 container should not have the top-actionable class: expected true to equal false 11:43:40 INFO - TEST-UNEXPECTED-FAIL | system/test/unit/app_chrome_test.js | system/AppChrome Theme-Color "after each" hook - timeout of 10000ms exceeded 11:44:33 INFO - TEST-UNEXPECTED-FAIL | music/test/unit/ui/views/player_view_test.js | Player View Test update seekbar on visibilitychange - expected false to be truthy 11:51:31 INFO - TEST-UNEXPECTED-FAIL | sharedtest/test/unit/tag_visibility_monitor_test.js | tag_visibility_monitor basics - timeout of 10000ms exceeded 11:51:41 INFO - TEST-UNEXPECTED-FAIL | sharedtest/test/unit/tag_visibility_monitor_test.js | tag_visibility_monitor addRemoveContainer - timeout of 10000ms exceeded 11:51:43 INFO - TEST-UNEXPECTED-FAIL | sharedtest/test/unit/stop_recording_event_test.js | StopRecordingEvent test runner has does not have document.hidden set - StopRecordingEvent only works when !document.hidden: expected true to be false 11:51:43 INFO - TEST-UNEXPECTED-FAIL | sharedtest/test/unit/stop_recording_event_test.js | StopRecordingEvent setting private.broadcast.stop_recording triggers event - expected false to be truthy 11:52:38 INFO - TEST-UNEXPECTED-FAIL | sharedtest/test/unit/font_size_utils_test.js | shared/js/text_utils.js FontSizeUtils.centerTextToScreen Should truncate a barely overflowing header title - timeout of 10000ms exceeded 11:52:48 INFO - TEST-UNEXPECTED-FAIL | sharedtest/test/unit/font_size_utils_test.js | shared/js/text_utils.js FontSizeUtils.centerTextToScreen Should truncate a very long header title - timeout of 10000ms exceeded 11:52:58 INFO - TEST-UNEXPECTED-FAIL | sharedtest/test/unit/font_size_utils_test.js | shared/js/text_utils.js Lazy-Loading DOM MutationObserver Non-header lazy load should not cause reformat - timeout of 10000ms exceeded 11:53:46 INFO - TEST-UNEXPECTED-FAIL | bookmark/test/unit/activity_handler_test.js | activity_handler.js > Adding and updating bookmarks > The new bookmark has been added correctly - timeout of 10000ms exceeded 11:53:56 INFO - TEST-UNEXPECTED-FAIL | bookmark/test/unit/activity_handler_test.js | activity_handler.js > Adding and updating bookmarks > The bookmark already exists and it is updated - timeout of 10000ms exceeded 11:54:06 INFO - TEST-UNEXPECTED-FAIL | bookmark/test/unit/activity_handler_test.js | activity_handler.js > Adding and updating bookmarks > The status is displayed when a bookmark has been added correctly - timeout of 10000ms exceeded 11:57:06 INFO - TEST-UNEXPECTED-FAIL | communications/contacts/test/unit/import/import_test.js | Import Friends Test Suite Import UI. items created. not already present - timeout of 10000ms exceeded 11:57:16 INFO - TEST-UNEXPECTED-FAIL | communications/contacts/test/unit/import/import_test.js | Import Friends Test Suite Import UI with some contacts already imported - timeout of 10000ms exceeded 11:57:26 INFO - TEST-UNEXPECTED-FAIL | communications/contacts/test/unit/import/import_test.js | Import Friends Test Suite Import UI with all contacts already imported - timeout of 10000ms exceeded 11:58:14 INFO - TEST-UNEXPECTED-FAIL | communications/contacts/test/unit/contacts_test.js | Contacts Visibility changes > in settings view, should refresh for new timestamp - expected updateTimestamps to have been called at least once but was never called 12:05:17 INFO - TEST-UNEXPECTED-FAIL | callscreen/test/unit/calls_handler_test.js | calls handler > Public methods > CallsHandler.switchToDefaultOut when visible should connect bluetooth SCO - expected connectSco to be called once but was called 0 times
So Gu can get green but it's badly intermittent with several failures.
Comment on attachment 8640433 [details] [diff] [review] Enable Mulet TaskCluster Gu/Gu-oop Review of attachment 8640433 [details] [diff] [review]: ----------------------------------------------------------------- r- because I think that relying on mozharness.json will be broken very soon if the patch is worked on in bug 1185643 which then could break whatever custom changes are in your mozharness fork. ::: testing/mozharness/mozharness.json @@ +1,2 @@ > { > + "repo": "https://github.com/lissyx/build-mozharness.git", Are there fixes done here that are needed to get these working? Also, :dustin is working on removing looking at the intree mozharness.json file so this would break the tests. Also, I'm not sure what the rules are pinning to user repos in tree like this. ::: testing/taskcluster/tasks/tests/mulet_gaia_unit_oop.yml @@ +35,5 @@ > + treeherderEnv: > + - production > + - staging > + treeherder: > + symbol: 'Gu-oop' Do we not need groupsymbol here? I can't remember what "groupSymbol: "?"" does.
Attachment #8640433 - Flags: review?(garndt) → review-
Comment on attachment 8640433 [details] [diff] [review] Enable Mulet TaskCluster Gu/Gu-oop Review of attachment 8640433 [details] [diff] [review]: ----------------------------------------------------------------- r- because I think that relying on mozharness.json will be broken very soon if the patch is worked on in bug 1185643 which then could break whatever custom changes are in your mozharness fork. ::: testing/mozharness/mozharness.json @@ +1,2 @@ > { > + "repo": "https://github.com/lissyx/build-mozharness.git", Are there fixes done here that are needed to get these working? Also, :dustin is working on removing looking at the intree mozharness.json file so this would break the tests. Also, I'm not sure what the rules are pinning to user repos in tree like this. ::: testing/taskcluster/tasks/tests/mulet_gaia_unit_oop.yml @@ +35,5 @@ > + treeherderEnv: > + - production > + - staging > + treeherder: > + symbol: 'Gu-oop' Do we not need groupsymbol here? I can't remember what "groupSymbol: "?"" does.
(In reply to Greg Arndt [:garndt] from comment #15) > Comment on attachment 8640433 [details] [diff] [review] > Enable Mulet TaskCluster Gu/Gu-oop > > Review of attachment 8640433 [details] [diff] [review]: > ----------------------------------------------------------------- > > r- because I think that relying on mozharness.json will be broken very soon > if the patch is worked on in bug 1185643 which then could break whatever > custom changes are in your mozharness fork. > > ::: testing/mozharness/mozharness.json > @@ +1,2 @@ > > { > > + "repo": "https://github.com/lissyx/build-mozharness.git", > > Are there fixes done here that are needed to get these working? Also, > :dustin is working on removing looking at the intree mozharness.json file so > this would break the tests. Also, I'm not sure what the rules are pinning > to user repos in tree like this. That's just a temporary workaround until bug 1188330 is fixed. > > ::: testing/taskcluster/tasks/tests/mulet_gaia_unit_oop.yml > @@ +35,5 @@ > > + treeherderEnv: > > + - production > > + - staging > > + treeherder: > > + symbol: 'Gu-oop' > > Do we not need groupsymbol here? I can't remember what "groupSymbol: "?"" > does. I don't know.
Whiteboard: [systemsfe]
Target Milestone: --- → FxOS-S4 (07Aug)
Depends on: 1185643
Now using in-tree mozharness. https://treeherder.mozilla.org/#/jobs?repo=try&revision=c00d1c1ef70d
Attachment #8640433 - Attachment is obsolete: true
So in-tree mozharness is not working, new try: https://treeherder.mozilla.org/#/jobs?repo=try&revision=20f3acdd42a3
With Gu tests chunked, we are starting to see some green behavior https://treeherder.mozilla.org/#/jobs?repo=try&revision=914c0c8fcc7d
Attachment #8658068 - Attachment is obsolete: true
Switching over to 20 chunks (failure of #30 is because it is empty) https://treeherder.mozilla.org/#/jobs?repo=try&revision=d486abcbe78e We are not running tv_apps unit tests. And we don't consider disabled tests.
Attachment #8660417 - Attachment is obsolete: true
Now with disables.json considered, and tv_apps included. All over 20 chunks: https://treeherder.mozilla.org/#/jobs?repo=try&revision=08f7ab92155c
Attachment #8660418 - Attachment is obsolete: true
Attachment #8660437 - Flags: review?(garndt)
Just triggering what we really want: https://treeherder.mozilla.org/#/jobs?repo=try&revision=62bf61e7d865
Retriggering seems not to work :(
Comment on attachment 8660437 [details] [diff] [review] Enable Mulet TaskCluster Gu/Gu-oop Review of attachment 8660437 [details] [diff] [review]: ----------------------------------------------------------------- The taskcluster pieces look good to me. I skimmed the mozharness stuff and on the surface looks ok but I'm not completely familiar with mozharness or our chunking strategies so you might want another set of eyes on that part.
Attachment #8660437 - Flags: review?(garndt) → review+
Comment on attachment 8660437 [details] [diff] [review] Enable Mulet TaskCluster Gu/Gu-oop Johnathan or Aus, can you review the mozharness parts ?
Attachment #8660437 - Flags: review?(jgriffin)
Attachment #8660437 - Flags: review?(aus)
Comment on attachment 8660437 [details] [diff] [review] Enable Mulet TaskCluster Gu/Gu-oop Review of attachment 8660437 [details] [diff] [review]: ----------------------------------------------------------------- Changes look good to me. Ship it! ::: testing/mozharness/mozharness.json @@ +4,1 @@ > } You'll want to update this shortly after landing the mozharness changes into the repository. :)
Attachment #8660437 - Flags: review?(aus) → review+
Removed mozharness tag changes
Attachment #8660437 - Attachment is obsolete: true
Attachment #8660437 - Flags: review?(jgriffin)
Attachment #8660847 - Flags: review+
Copy of the gaia_unit.py script from Gecko to mozharness tree
Attachment #8660848 - Flags: review+
Aus, thanks for handling the proper landing of this while I'm sleeping :)
Flags: needinfo?(aus)
El Doctor, unfortunately I didn't have enough bandwidth to get to this on Monday but I'll have time Tuesday. :) Cheers! (Leaving ni? on as a reminder)
Flags: needinfo?(aus)
Flags: needinfo?(aus)
After a false start, this has now been landed. Hooray!
Flags: needinfo?(aus)
I don't understand, Gu-oop is not supposed to be triggered ?
Flags: needinfo?(lissyx+mozillians)
Removed reporting to staging and production for Gu-oop
Attachment #8660847 - Attachment is obsolete: true
Attachment #8661447 - Flags: review+
Wes, I have removed the treeherderEnv part as suggested, so I hope it will be good enough. Also added the mercurial changeset to pick proper mozharness.
Flags: needinfo?(wkocher)
Keywords: checkin-needed
Looks like the proper fix would rather to: - remove gaia-unit-oop from base and try jobs - ensure gaia-unit is configured in base and try jobs to work with mulet
Attachment #8661447 - Attachment is obsolete: true
Attachment #8661470 - Flags: review+
(In reply to Wes Kocher (:KWierso) from comment #48) > Relanded in https://hg.mozilla.org/integration/b2g-inbound/rev/8cf7b6d58a22 No Gaia Unit on Mulet. I think I understood the mistake I did, confere comment 49. We may want to backout 8cf7b6d58a22 and land attachment 8661470 [details] [diff] [review]
Flags: needinfo?(wkocher)
Flags: needinfo?(aus)
I landed https://hg.mozilla.org/integration/b2g-inbound/rev/33ab2e156c30 instead of doing a backout and landing the new patch because I'm almost positive this is all that's needed to get non-oop mulet Gu tests running. If this works like I really really hope it'll work, Mulet Gu tests will run and submit to both Treeherder production and Treeherder staging ( https://treeherder.allizom.org/#/jobs?repo=b2g-inbound ), and Mulet Gu-oop tests will run and submit to only Treeherder staging. If this doesn't work, I'd say we just back out both 33ab2e156c30 and 8cf7b6d58a22, and then try again tomorrow.
Flags: needinfo?(wkocher)
Looking at production[1] and staging[2], everything is looking like I expect it to look on Treeherder. Once Gu-oop is working correctly and is no longer permafailing, all that would be needed to start submitting those results to Treeherder production is to add a line that says "- production" above line 38 in [3]. [1] https://treeherder.mozilla.org/#/jobs?repo=b2g-inbound&revision=33ab2e156c30&filter-searchStr=mulet [2] https://treeherder.allizom.org/#/jobs?repo=b2g-inbound&revision=33ab2e156c30&filter-searchStr=mulet [3] https://hg.mozilla.org/integration/b2g-inbound/file/8cf7b6d58a22/testing/taskcluster/tasks/tests/mulet_gaia_unit_oop.yml
(In reply to Wes Kocher (:KWierso) from comment #53) > Looking at production[1] and staging[2], everything is looking like I expect > it to look on Treeherder. > > Once Gu-oop is working correctly and is no longer permafailing, all that > would be needed to start submitting those results to Treeherder production > is to add a line that says "- production" above line 38 in [3]. > > > > [1] > https://treeherder.mozilla.org/#/jobs?repo=b2g- > inbound&revision=33ab2e156c30&filter-searchStr=mulet > [2] > https://treeherder.allizom.org/#/jobs?repo=b2g- > inbound&revision=33ab2e156c30&filter-searchStr=mulet > [3] > https://hg.mozilla.org/integration/b2g-inbound/file/8cf7b6d58a22/testing/ > taskcluster/tasks/tests/mulet_gaia_unit_oop.yml Looks like it's all good from my end as well. Thanks Wes!
Flags: needinfo?(aus)
Blocks: 1205196
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: