Closed Bug 988118 Opened 7 years ago Closed 7 years ago

Gaia unit test failures on TBPL due to the camera merge

Categories

(Firefox OS Graveyard :: Gaia::Camera, defect)

ARM
Gonk (Firefox OS)
defect
Not set
blocker

Tracking

(b2g-v1.4 fixed, b2g-v2.0 fixed)

RESOLVED FIXED
1.4 S5 (11apr)
Tracking Status
b2g-v1.4 --- fixed
b2g-v2.0 --- fixed

People

(Reporter: RyanVM, Assigned: justindarc)

References

Details

Attachments

(1 file, 3 obsolete files)

https://tbpl.mozilla.org/php/getParsedLog.php?id=36696317&tree=B2g-Inbound

b2g_ubuntu64_vm b2g-inbound opt test gaia-unit on 2014-03-25 17:32:08 PDT for push ba5dabcb650f
slave: tst-linux64-spot-966

17:44:30     INFO -  gaia-unit-tests TEST-UNEXPECTED-FAIL | camera/test/unit/lib/sounds_test.js | Sounds Sounds#createAudio() Should set the mozAudioChannel type | expected false to be truthy
17:44:30     INFO -  gaia-unit-tests INFO       | stack trace:
17:44:30     INFO -      Error: expected false to be truthy
17:44:30     INFO -          at chaiAssert (http://camera.gaiamobile.org:8080/common/test/helper.js:33:1)
17:44:30     INFO -          at get (http://camera.gaiamobile.org:8080/common/vendor/chai/chai.js:352:1)
17:44:30     INFO -          at ok (http://camera.gaiamobile.org:8080/common/vendor/chai/chai.js:1225:5)
17:44:30     INFO -          at (anonymous) (http://camera.gaiamobile.org:8080/test/unit/lib/sounds_test.js:176:7)
17:44:30     INFO -          at wrapper (http://camera.gaiamobile.org:8080/common/test/mocha_generators.js:62:13)
17:44:30     INFO -          at run (http://camera.gaiamobile.org:8080/common/vendor/mocha/mocha.js:3709:7)
17:44:30     INFO -          at runTest (http://camera.gaiamobile.org:8080/common/vendor/mocha/mocha.js:4081:5)
17:44:30     INFO -          at (anonymous) (http://camera.gaiamobile.org:8080/common/vendor/mocha/mocha.js:4127:7)
17:44:30     INFO -          at next (http://camera.gaiamobile.org:8080/common/vendor/mocha/mocha.js:4007:7)
17:44:30     INFO -          at (anonymous) (http://camera.gaiamobile.org:8080/common/vendor/mocha/mocha.js:4016:7)
17:44:30     INFO -          at next (http://camera.gaiamobile.org:8080/common/vendor/mocha/mocha.js:3964:16)
17:44:30     INFO -          at (anonymous) (http://camera.gaiamobile.org:8080/common/vendor/mocha/mocha.js:3984:5)
17:44:30     INFO -          at (anonymous) (http://camera.gaiamobile.org:8080/common/vendor/mocha/mocha.js:4932:28)
17:44:30     INFO -  gaia-unit-tests TEST-END   | camera/test/unit/lib/sounds_test.js | Sounds#createAudio()

Since nobody's around to look into these failures, b2g-inbound and Gaia are both closed until this is resolved.
Maybe bug 986276 is the cause? That seems to be the last patch that modified that unit test.
Reverted: https://github.com/mozilla-b2g/gaia/commit/9b093b55fd98d8a86251397a196fd9591a35a88e

The problem is that this is work on a feature branch, and there is no bug tracking the work to merge the work into master. Because of this I'm leaving this bug as open ni? on Diego.

I might recommend landing in chunks of 3-4 commits over a week or two - just a suggestion, that's what they did on the recent bubble tea branch.
Flags: needinfo?(dmarcos)
Looks like we had disabled the `sounds_test.js` suite back in Bug 958200 for this exact same reason:

https://bugzilla.mozilla.org/show_bug.cgi?id=958200#c21
If you do want to try re-landing this - definitely use gaia-try first, and ensure you have a green build before merging.

https://wiki.mozilla.org/ReleaseEngineering/TryServer#Using_a_custom_Gaia

I know it sucks that tbpl runs tests differently than travis. This is causing a huge problem recently and I'll follow up this week as to what we can do to fix this.
(In reply to Justin D'Arcangelo [:justindarc] from comment #3)
> Looks like we had disabled the `sounds_test.js` suite back in Bug 958200 for
> this exact same reason:
> 
> https://bugzilla.mozilla.org/show_bug.cgi?id=958200#c21

It is true that the patch of bug 986276 re-enabled the `sounds_test.js`.

I don't know the environment of TBPL. But according to the code[1][2], it should work. It just checks *mozAudioChannelType*.

To fix it, we may just re-disable it.

[1] https://github.com/dmarcos/gaia/blob/711a2bd41d756287a8731b83abea06b9e554311e/apps/camera/test/unit/lib/sounds_test.js#L175-L176
[2] https://github.com/dmarcos/gaia/blob/711a2bd41d756287a8731b83abea06b9e554311e/apps/camera/js/lib/sounds.js#L151-L155
Attached file pull-request (master) (obsolete) —
This PR reverts the revert and fixes the TBPL blacklist to skip the `sounds_test.js` suite.

kgrandon: Setting R? so you can look at testing this on gaia-try before merging. Thanks!
Assignee: nobody → jdarcangelo
Status: NEW → ASSIGNED
Attachment #8396909 - Flags: review?(kgrandon)
Thanks Kevin for helping. Where do I have to look to see the results of the unit tests?
Flags: needinfo?(dmarcos) → needinfo?(kgrandon)
I'm really sorry for this. I think this indicates that something either on Travis (we have to match a TBPL environment) or TBPL (make easier to run stuff there) has to change. In the case of camera we feel we're fighting the environment to get stuff shipped. We made a big effort to add more than 200 hundred unit tests and we spent almost a day fighting with Travis to get green due to again differences between our local environments and travis: Travis env is case sensitive and one of the dependencies for one of the tests was capitalized and we were exceeding the node maxBuffer for one of the build tests (bug 988010). If we don't make shipping easier and more transparent two things will happen:

1. Some people will stop caring about doing any contribution of substance since it's so difficult to ship anything. A quasi heroic attitude is needed to get anything done and it's pretty exhausting to play hero all the time.
2. Some obstreperous people will find ways to game the system to follow the path of least resistance to do what they have to do. For example, I'm tempted to disable all unit tests on the repo and only run them on my local environment where I find them useful.
Attachment #8396909 - Attachment is obsolete: true
Attachment #8397026 - Attachment is obsolete: true
Attachment #8396909 - Flags: review?(kgrandon)
Attachment #8397026 - Flags: review?(kgrandon)
Attachment #8397027 - Flags: review?(kgrandon)
Diego - I agree with the current pain points, and it is something we are looking to solve in the future by unifying the two environments with great integration into Github. The project is called Task Cluster, but it is still a ways away. In the meantime I think we can adjust our processes a bit to help ease the burden. Landing smaller chunks to master more frequently would probably help quite a bit for now. Relief should be coming in the next few months :)
Flags: needinfo?(kgrandon)
TBPL looks pretty happy, but travis failed. I am pretty sure it's due to an intermittent failure in another app so I restarted the job. Let's see what it comes back with.
Attachment #8397027 - Flags: review?(kgrandon)
Attached file Pull Request
Attachment #8397027 - Attachment is obsolete: true
Attachment #8397260 - Flags: review+
Looks like this landed, https://github.com/mozilla-b2g/gaia/commit/e328318d5e27a3253061fb1679ab9abc82e60fcc.

Let's keep a close eye on it.
Status: ASSIGNED → RESOLVED
Closed: 7 years ago
Resolution: --- → FIXED
Bulk edit for camera bugs.

If earlier comments do not show how this bug landed to master, it probably landed as part of https://github.com/mozilla-b2g/gaia/pull/17599 which merged the camera-new-features branch into master.

This bug was uplifted from master to v1.4 as part of https://github.com/mozilla-b2g/gaia/commit/a8190d08e61316a86bba572ba8d894d081a20530
Target Milestone: --- → 1.4 S5 (11apr)
Depends on: 991344
Depends on: 991953
You need to log in before you can comment on or make changes to this bug.