1021707 - Intermittent Travis Unit test failure: [sharedtest-test/unit/crop_resize_rotate_test.js] cropResizeRotate tests 250x250 Resize image to: 125x125: Error timeout

Status: RESOLVED WORKSFORME

(Firefox OS Graveyard :: Gaia, defect)

Description

[Julien Wajsberg [:julienw]]

1) [sharedtest-test/unit/crop_resize_rotate_test.js] cropResizeRotate tests 250x250 Resize image to: 125x125:

Error: timeout of 10000ms exceeded

at (anonymous) (http://sharedtest.gaiamobile.org:8080/common/vendor/mocha/mocha.js:3680:9)

Comment 1

[Julien Wajsberg [:julienw]]

Heu Russ, looks like you own this file, would you know what happens?
Is it just something that's meant to happen sometimes if the VM is too slow?

Comment 2

[Russ Nicoletti [:russn]]

Hi Julien, typically the resize tests run in less than 50ms (locally). It seems reasonable to expect that when the VMs are very slow they will be affected, like many tests would be. But taking 10 seconds seems extreme. 

NI djf for his opinion.

Comment 3

[David Flanagan [:djf]]

Julien,

Its my test case, written for 1.3T, but it is probably being run a lot more now that it has landed on master.

I've looked again at the code and don't see anything that is wrong with it.

Timeouts are only for individual tests, right? We don't have timeouts for entire suites, do we? It is a pretty massive suite of tests, and can take > 10s to run everything locally, IIRC. But none of the individual tests should take long. Note that this test file runs a suite of tests four times on four different size source images.  So this test that timed out on a 250x250 image passed twice before on other size images.

The test are creating, manipulating and comparing images, so they probably use more memory than typical tests, and maybe that stresses the Travis VM too much? We could reduce the number of tests.  We could increase the timeout.  

Maybe we're running into some kind of graphics bug on the host machine? We could modify the verifyImage() function in the test so that it uses { willReadFrequently: true } in the call to canvas.getContext().  I don't know if that does anything on desktop environment, but on the phone it takes 2d graphics off of the accelerated path and into pure software. In this case, it might actually speed the tests up since we are using the canvas just to read the image data.

Julien: have you seen this multiple times? If we try the change above, will we even be able to tell if it has any effect?

Russ: will you add willReadFrequently to line 440 in the test file?  If the tests still work with that addition, you have my r+ to land that change.  (Especially if it makes the tests run faster when you run them locally)

Comment 4

[Russ Nicoletti [:russn]]

Created PR (https://github.com/mozilla-b2g/gaia/pull/20169) for adding willReadFrequently in crop_resize_rotate_test.js verifyImage function. Will wait to hear from Julien regarding the frequency of the travis failures before landing.

Comment 5

[Julien Wajsberg [:julienw]]

I really doubt "willReadFrequently" changes anything on Firefox/Linux.

The frequency is very low too.

What IMO happens is that there are lots of asynchronous operations happening in one test and this is completely legit and there is nothing much we can do. Maybe increasing mocha's timeout for this test file only to make it happen yet less often?

To answer the question, the 10s timeout is for one test only, not for the whole suite or even file.

Comment 6

[Julien Wajsberg [:julienw]]

The main issue is that there are lots of tests in this file, and each tests is doing lots of asynchronous operations. So it's perfectly expected that in the constrained Travis environment, once in a while, a timeout happens. Let's increase the timeout for the file.

Comment 7

[Ed Morley [:emorley]]

Inactive; closing (see bug 1180138).