Enable Mulet TaskCluster Gu

RESOLVED FIXED in Firefox 43

Status

defect
RESOLVED FIXED
4 years ago
4 years ago

People

(Reporter: gerard-majax, Assigned: gerard-majax)

Tracking

unspecified
FxOS-S4 (07Aug)
ARM
Gonk (Firefox OS)
Dependency tree / graph

Firefox Tracking Flags

(firefox43 fixed)

Details

(Whiteboard: [systemsfe])

Attachments

(2 attachments, 12 obsolete attachments)

3.98 KB, patch
gerard-majax
: review+
Details | Diff | Splinter Review
9.42 KB, patch
gerard-majax
: review+
Details | Diff | Splinter Review
Let's enable running Gu with Mulet on TaskCluster
Assignee

Comment 3

4 years ago
Aus, it looks like we are properly making use of Mulet to run this, but I'm not 100% certain and reading the logs to understand this is not very easy. Can you give it a look? Thanks!
Flags: needinfo?(aus)
Assignee

Comment 4

4 years ago
Attachment #8639836 - Attachment is obsolete: true
Looks like this one has some issues and isn't calling the right binary (and that's why it's failing to find it in the first place).

See this log snippet:

17:34:22     INFO - Calling ['/home/worker/build/venv/bin/python', '-u', '/home/worker/gaia/tests/python/gaia-unit-tests/gaia_unit_test/main.py', '--binary', '/home/worker/build/application/firefox/b2g-bin', '--profile', '/home/worker/gaia/profile-debug', '--symbols-path', 'https://queue.taskcluster.net/v1/task/N3hOrVWqQVSHHTYImrY11w/artifacts/public/build/target.linux-x86_64.crashreporter-symbols.zip'] with output_timeout 1760

I'm guessing that should be calling something else. The '--binary' and following argument value are the ones that look wrong to me.
Flags: needinfo?(aus)
Assignee

Comment 6

4 years ago
Yes, and yet I do change them in the python code.
Assignee

Comment 7

4 years ago
Hacked in build-mozharness and changed JSON to point to it:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=4294364ddaa7
Assignee

Comment 8

4 years ago
(In reply to Alexandre LISSY :gerard-majax from comment #7)
> Hacked in build-mozharness and changed JSON to point to it:
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=4294364ddaa7

I'm seeing some errors on Gij but that are related to npm stuff. Invalid 'ui' etc., remind me of some mess ... Gareth, can you have a look?
Flags: needinfo?(gaye)
Assignee

Comment 9

4 years ago
(In reply to Alexandre LISSY :gerard-majax from comment #8)
> (In reply to Alexandre LISSY :gerard-majax from comment #7)
> > Hacked in build-mozharness and changed JSON to point to it:
> > https://treeherder.mozilla.org/#/jobs?repo=try&revision=4294364ddaa7
> 
> I'm seeing some errors on Gij but that are related to npm stuff. Invalid
> 'ui' etc., remind me of some mess ... Gareth, can you have a look?

Commented on the wrong bug ...
Flags: needinfo?(gaye)
Assignee

Comment 10

4 years ago
(In reply to Alexandre LISSY :gerard-majax from comment #7)
> Hacked in build-mozharness and changed JSON to point to it:
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=4294364ddaa7

As expected, Gu now fails on B2G Desktop because it makes use of "b2g/firefox" binary :)
Assignee

Comment 12

4 years ago
Still depends on mozharness being intree but it looks good.
Attachment #8639918 - Attachment is obsolete: true
Attachment #8640433 - Flags: review?(garndt)
Assignee

Comment 13

4 years ago
(In reply to Alexandre LISSY :gerard-majax from comment #12)
> Created attachment 8640433 [details] [diff] [review]
> Enable Mulet TaskCluster Gu/Gu-oop
> 
> Still depends on mozharness being intree but it looks good.

That is working and exposing some failures:

11:30:21     INFO -  TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should emit a 'busy', then 'ready' event - this.mozCamera.release(...) is undefined
11:30:21     INFO -  TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should call `navigator.mozCameras.getCamera()` with currently selected camera - this.mozCamera.release(...) is undefined
11:30:21     INFO -  TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should call get camera with the passed config - this.mozCamera.release(...) is undefined
11:30:21     INFO -  TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should flag a `this.configured` - this.mozCamera.release(...) is undefined
11:30:21     INFO -  TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should call .setupNewCamera - this.mozCamera.release(...) is undefined
11:30:21     INFO -  TEST-UNEXPECTED-FAIL | camera/test/unit/lib/camera/camera_test.js | lib/camera/camera Camera#requestCamera() Should emit a 'configured' - this.mozCamera.release(...) is undefined
11:32:03     INFO -  TEST-UNEXPECTED-FAIL | settings/test/unit/modules/state_model_test.js | StateModel set target state the final state should be the last state if success - timeout of 10000ms exceeded
11:33:50     INFO -  TEST-UNEXPECTED-FAIL | settings/test/unit/panels/operator_settings/models/auto_selection_model_test.js | AutoSelectionModel set target state mobile connection is busy the final state should be the last targetState - timeout of 10000ms exceeded
11:36:32     INFO -  TEST-UNEXPECTED-FAIL | system/test/unit/lockscreen/lockscreen_notifications_test.js | system/LockScreenNotifications Remove top mask when actionable notification is at the visual top ofthe container viewport top-actionable class is correctly added in onNotificationHighlighted test on 0-th notification - with notification index 0 container should not have the top-actionable class: expected true to equal false
11:36:32     INFO -  TEST-UNEXPECTED-FAIL | system/test/unit/lockscreen/lockscreen_notifications_test.js | system/LockScreenNotifications Remove top mask when actionable notification is at the visual top ofthe container viewport top-actionable class is correctly added in onNotificationHighlighted test on 1-th notification - with notification index 1 container should have the top-actionable class: expected false to equal true
11:36:32     INFO -  TEST-UNEXPECTED-FAIL | system/test/unit/lockscreen/lockscreen_notifications_test.js | system/LockScreenNotifications Remove top mask when actionable notification is at the visual top ofthe container viewport top-actionable class is correctly removed with tryAddTopMaskByNotification test on 0-th notification - with notification index 0 container should have the top-actionable class: expected false to equal true
11:36:32     INFO -  TEST-UNEXPECTED-FAIL | system/test/unit/lockscreen/lockscreen_notifications_test.js | system/LockScreenNotifications Remove top mask when actionable notification is at the visual top ofthe container viewport top-actionable class is correctly removed with tryAddTopMaskByNotification test on 1-th notification - with notification index 1 container should not have the top-actionable class: expected true to equal false
11:43:40     INFO -  TEST-UNEXPECTED-FAIL | system/test/unit/app_chrome_test.js | system/AppChrome Theme-Color "after each" hook - timeout of 10000ms exceeded
11:44:33     INFO -  TEST-UNEXPECTED-FAIL | music/test/unit/ui/views/player_view_test.js | Player View Test update seekbar on visibilitychange - expected false to be truthy
11:51:31     INFO -  TEST-UNEXPECTED-FAIL | sharedtest/test/unit/tag_visibility_monitor_test.js | tag_visibility_monitor basics - timeout of 10000ms exceeded
11:51:41     INFO -  TEST-UNEXPECTED-FAIL | sharedtest/test/unit/tag_visibility_monitor_test.js | tag_visibility_monitor addRemoveContainer - timeout of 10000ms exceeded
11:51:43     INFO -  TEST-UNEXPECTED-FAIL | sharedtest/test/unit/stop_recording_event_test.js | StopRecordingEvent test runner has does not have document.hidden set - StopRecordingEvent only works when !document.hidden: expected true to be false
11:51:43     INFO -  TEST-UNEXPECTED-FAIL | sharedtest/test/unit/stop_recording_event_test.js | StopRecordingEvent setting private.broadcast.stop_recording triggers event - expected false to be truthy
11:52:38     INFO -  TEST-UNEXPECTED-FAIL | sharedtest/test/unit/font_size_utils_test.js | shared/js/text_utils.js FontSizeUtils.centerTextToScreen Should truncate a barely overflowing header title - timeout of 10000ms exceeded
11:52:48     INFO -  TEST-UNEXPECTED-FAIL | sharedtest/test/unit/font_size_utils_test.js | shared/js/text_utils.js FontSizeUtils.centerTextToScreen Should truncate a very long header title - timeout of 10000ms exceeded
11:52:58     INFO -  TEST-UNEXPECTED-FAIL | sharedtest/test/unit/font_size_utils_test.js | shared/js/text_utils.js Lazy-Loading DOM MutationObserver Non-header lazy load should not cause reformat - timeout of 10000ms exceeded
11:53:46     INFO -  TEST-UNEXPECTED-FAIL | bookmark/test/unit/activity_handler_test.js | activity_handler.js > Adding and updating bookmarks > The new bookmark has been added correctly  - timeout of 10000ms exceeded
11:53:56     INFO -  TEST-UNEXPECTED-FAIL | bookmark/test/unit/activity_handler_test.js | activity_handler.js > Adding and updating bookmarks > The bookmark already exists and it is updated  - timeout of 10000ms exceeded
11:54:06     INFO -  TEST-UNEXPECTED-FAIL | bookmark/test/unit/activity_handler_test.js | activity_handler.js > Adding and updating bookmarks > The status is displayed when a bookmark has been added correctly  - timeout of 10000ms exceeded
11:57:06     INFO -  TEST-UNEXPECTED-FAIL | communications/contacts/test/unit/import/import_test.js | Import Friends Test Suite Import UI. items created. not already present - timeout of 10000ms exceeded
11:57:16     INFO -  TEST-UNEXPECTED-FAIL | communications/contacts/test/unit/import/import_test.js | Import Friends Test Suite Import UI with some contacts already imported - timeout of 10000ms exceeded
11:57:26     INFO -  TEST-UNEXPECTED-FAIL | communications/contacts/test/unit/import/import_test.js | Import Friends Test Suite Import UI with all contacts already imported - timeout of 10000ms exceeded
11:58:14     INFO -  TEST-UNEXPECTED-FAIL | communications/contacts/test/unit/contacts_test.js | Contacts Visibility changes > in settings view, should refresh for new timestamp - expected updateTimestamps to have been called at least once but was never called
12:05:17     INFO -  TEST-UNEXPECTED-FAIL | callscreen/test/unit/calls_handler_test.js | calls handler > Public methods > CallsHandler.switchToDefaultOut when visible should connect bluetooth SCO - expected connectSco to be called once but was called 0 times
Assignee

Comment 14

4 years ago
So Gu can get green but it's badly intermittent with several failures.
Comment on attachment 8640433 [details] [diff] [review]
Enable Mulet TaskCluster Gu/Gu-oop

Review of attachment 8640433 [details] [diff] [review]:
-----------------------------------------------------------------

r- because I think that relying on mozharness.json will be broken very soon if the patch is worked on in bug 1185643 which then could break whatever custom changes are in your mozharness fork.

::: testing/mozharness/mozharness.json
@@ +1,2 @@
>  {
> +    "repo": "https://github.com/lissyx/build-mozharness.git",

Are there fixes done here that are needed to get these working?  Also, :dustin is working on removing looking at the intree mozharness.json file so this would break the tests.  Also, I'm not sure what the rules are pinning to user repos in tree like this.

::: testing/taskcluster/tasks/tests/mulet_gaia_unit_oop.yml
@@ +35,5 @@
> +    treeherderEnv:
> +      - production
> +      - staging
> +    treeherder:
> +      symbol: 'Gu-oop'

Do we not need groupsymbol here?  I can't remember what "groupSymbol: "?"" does.
Attachment #8640433 - Flags: review?(garndt) → review-
Comment on attachment 8640433 [details] [diff] [review]
Enable Mulet TaskCluster Gu/Gu-oop

Review of attachment 8640433 [details] [diff] [review]:
-----------------------------------------------------------------

r- because I think that relying on mozharness.json will be broken very soon if the patch is worked on in bug 1185643 which then could break whatever custom changes are in your mozharness fork.

::: testing/mozharness/mozharness.json
@@ +1,2 @@
>  {
> +    "repo": "https://github.com/lissyx/build-mozharness.git",

Are there fixes done here that are needed to get these working?  Also, :dustin is working on removing looking at the intree mozharness.json file so this would break the tests.  Also, I'm not sure what the rules are pinning to user repos in tree like this.

::: testing/taskcluster/tasks/tests/mulet_gaia_unit_oop.yml
@@ +35,5 @@
> +    treeherderEnv:
> +      - production
> +      - staging
> +    treeherder:
> +      symbol: 'Gu-oop'

Do we not need groupsymbol here?  I can't remember what "groupSymbol: "?"" does.
Assignee

Comment 17

4 years ago
(In reply to Greg Arndt [:garndt] from comment #15)
> Comment on attachment 8640433 [details] [diff] [review]
> Enable Mulet TaskCluster Gu/Gu-oop
> 
> Review of attachment 8640433 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> r- because I think that relying on mozharness.json will be broken very soon
> if the patch is worked on in bug 1185643 which then could break whatever
> custom changes are in your mozharness fork.
> 
> ::: testing/mozharness/mozharness.json
> @@ +1,2 @@
> >  {
> > +    "repo": "https://github.com/lissyx/build-mozharness.git",
> 
> Are there fixes done here that are needed to get these working?  Also,
> :dustin is working on removing looking at the intree mozharness.json file so
> this would break the tests.  Also, I'm not sure what the rules are pinning
> to user repos in tree like this.

That's just a temporary workaround until bug 1188330 is fixed.

> 
> ::: testing/taskcluster/tasks/tests/mulet_gaia_unit_oop.yml
> @@ +35,5 @@
> > +    treeherderEnv:
> > +      - production
> > +      - staging
> > +    treeherder:
> > +      symbol: 'Gu-oop'
> 
> Do we not need groupsymbol here?  I can't remember what "groupSymbol: "?""
> does.

I don't know.
Whiteboard: [systemsfe]
Target Milestone: --- → FxOS-S4 (07Aug)
Assignee

Updated

4 years ago
Depends on: 1185643
Assignee

Comment 18

4 years ago
Now using in-tree mozharness.
https://treeherder.mozilla.org/#/jobs?repo=try&revision=c00d1c1ef70d
Attachment #8640433 - Attachment is obsolete: true
Assignee

Comment 20

4 years ago
So in-tree mozharness is not working, new try:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=20f3acdd42a3
Assignee

Comment 25

4 years ago
With Gu tests chunked, we are starting to see some green behavior
https://treeherder.mozilla.org/#/jobs?repo=try&revision=914c0c8fcc7d
Attachment #8658068 - Attachment is obsolete: true
Assignee

Comment 26

4 years ago
Switching over to 20 chunks (failure of #30 is because it is empty)
https://treeherder.mozilla.org/#/jobs?repo=try&revision=d486abcbe78e

We are not running tv_apps unit tests. And we don't consider disabled tests.
Attachment #8660417 - Attachment is obsolete: true
Assignee

Comment 27

4 years ago
Now with disables.json considered, and tv_apps included. All over 20 chunks:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=08f7ab92155c
Attachment #8660418 - Attachment is obsolete: true
Attachment #8660437 - Flags: review?(garndt)
Assignee

Comment 29

4 years ago
Retriggering seems not to work :(
Comment on attachment 8660437 [details] [diff] [review]
Enable Mulet TaskCluster Gu/Gu-oop

Review of attachment 8660437 [details] [diff] [review]:
-----------------------------------------------------------------

The taskcluster pieces look good to me.  I skimmed the mozharness stuff and on the surface looks ok but I'm not completely familiar with mozharness or our chunking strategies so you might want another set of eyes on that part.
Attachment #8660437 - Flags: review?(garndt) → review+
Assignee

Comment 33

4 years ago
Comment on attachment 8660437 [details] [diff] [review]
Enable Mulet TaskCluster Gu/Gu-oop

Johnathan or Aus, can you review the mozharness parts ?
Attachment #8660437 - Flags: review?(jgriffin)
Attachment #8660437 - Flags: review?(aus)
Comment on attachment 8660437 [details] [diff] [review]
Enable Mulet TaskCluster Gu/Gu-oop

Review of attachment 8660437 [details] [diff] [review]:
-----------------------------------------------------------------

Changes look good to me. Ship it!

::: testing/mozharness/mozharness.json
@@ +4,1 @@
>  }

You'll want to update this shortly after landing the mozharness changes into the repository. :)
Attachment #8660437 - Flags: review?(aus) → review+
Assignee

Comment 36

4 years ago
Removed mozharness tag changes
Attachment #8660437 - Attachment is obsolete: true
Attachment #8660437 - Flags: review?(jgriffin)
Attachment #8660847 - Flags: review+
Assignee

Comment 37

4 years ago
Copy of the gaia_unit.py script from Gecko to mozharness tree
Attachment #8660848 - Flags: review+
Assignee

Comment 38

4 years ago
Aus, thanks for handling the proper landing of this while I'm sleeping :)
Flags: needinfo?(aus)
El Doctor, unfortunately I didn't have enough bandwidth to get to this on Monday but I'll have time Tuesday. :) Cheers! (Leaving ni? on as a reminder)
Flags: needinfo?(aus)
Assignee

Updated

4 years ago
Flags: needinfo?(aus)
After a false start, this has now been landed. Hooray!
Flags: needinfo?(aus)
Gu-oop is failing across the board after this landed: https://treeherder.mozilla.org/logviewer.html#?job_id=2776771&repo=b2g-inbound

Backed out in https://hg.mozilla.org/integration/b2g-inbound/rev/b40d85977564
Flags: needinfo?(lissyx+mozillians)
Assignee

Comment 45

4 years ago
I don't understand, Gu-oop is not supposed to be triggered ?
Flags: needinfo?(lissyx+mozillians)
Assignee

Comment 46

4 years ago
Removed reporting to staging and production for Gu-oop
Attachment #8660847 - Attachment is obsolete: true
Attachment #8661447 - Flags: review+
Assignee

Comment 47

4 years ago
Wes, I have removed the treeherderEnv part as suggested, so I hope it will be good enough. Also added the mercurial changeset to pick proper mozharness.
Flags: needinfo?(wkocher)
Keywords: checkin-needed
Assignee

Comment 49

4 years ago
Looks like the proper fix would rather to:
 - remove gaia-unit-oop from base and try jobs
 - ensure gaia-unit is configured in base and try jobs to work with mulet
Attachment #8661447 - Attachment is obsolete: true
Attachment #8661470 - Flags: review+
Assignee

Comment 50

4 years ago
(In reply to Wes Kocher (:KWierso) from comment #48)
> Relanded in https://hg.mozilla.org/integration/b2g-inbound/rev/8cf7b6d58a22

No Gaia Unit on Mulet. I think I understood the mistake I did, confere comment 49. We may want to backout 8cf7b6d58a22 and land attachment 8661470 [details] [diff] [review]
Flags: needinfo?(wkocher)
Flags: needinfo?(aus)
I landed https://hg.mozilla.org/integration/b2g-inbound/rev/33ab2e156c30 instead of doing a backout and landing the new patch because I'm almost positive this is all that's needed to get non-oop mulet Gu tests running.


If this works like I really really hope it'll work, Mulet Gu tests will run and submit to both Treeherder production and Treeherder staging ( https://treeherder.allizom.org/#/jobs?repo=b2g-inbound ), and Mulet Gu-oop tests will run and submit to only Treeherder staging.


If this doesn't work, I'd say we just back out both 33ab2e156c30 and 8cf7b6d58a22, and then try again tomorrow.
Flags: needinfo?(wkocher)
Looking at production[1] and staging[2], everything is looking like I expect it to look on Treeherder.

Once Gu-oop is working correctly and is no longer permafailing, all that would be needed to start submitting those results to Treeherder production is to add a line that says "- production" above line 38 in [3].



[1] https://treeherder.mozilla.org/#/jobs?repo=b2g-inbound&revision=33ab2e156c30&filter-searchStr=mulet 
[2] https://treeherder.allizom.org/#/jobs?repo=b2g-inbound&revision=33ab2e156c30&filter-searchStr=mulet
[3] https://hg.mozilla.org/integration/b2g-inbound/file/8cf7b6d58a22/testing/taskcluster/tasks/tests/mulet_gaia_unit_oop.yml
(In reply to Wes Kocher (:KWierso) from comment #53)
> Looking at production[1] and staging[2], everything is looking like I expect
> it to look on Treeherder.
> 
> Once Gu-oop is working correctly and is no longer permafailing, all that
> would be needed to start submitting those results to Treeherder production
> is to add a line that says "- production" above line 38 in [3].
> 
> 
> 
> [1]
> https://treeherder.mozilla.org/#/jobs?repo=b2g-
> inbound&revision=33ab2e156c30&filter-searchStr=mulet 
> [2]
> https://treeherder.allizom.org/#/jobs?repo=b2g-
> inbound&revision=33ab2e156c30&filter-searchStr=mulet
> [3]
> https://hg.mozilla.org/integration/b2g-inbound/file/8cf7b6d58a22/testing/
> taskcluster/tasks/tests/mulet_gaia_unit_oop.yml

Looks like it's all good from my end as well. Thanks Wes!
Flags: needinfo?(aus)
Assignee

Updated

4 years ago
Blocks: 1205196
You need to log in before you can comment on or make changes to this bug.