1236723 - [gatt] GijTV chunks not working

Status: RESOLVED FIXED

(Testing Graveyard :: JSMarionette, defect)

Description

[Aus Lacroix [:aus]]

From comment that triggered this new bug --

@Aus,
Now GijTV has 10 chunks.
However, we found that unlike that the phone Gij tests would be split into different chunks to run.
The GijTV tests would be run in every chunk again.

For example, in this test https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=b1e472387992d4b67981350248c77891d5081c48&selectedJob=3237992

We can see the Gij1 and Gij2 ran different tests.
Gij1: https://public-artifacts.taskcluster.net/QKb2MjmITqaIoShVfbQO4Q/0/public/logs/live_backing.log
Gij2: https://public-artifacts.taskcluster.net/eBbCHFeoSsSh57zob2xdGg/0/public/logs/live_backing.log

But we can see the GijTV1 to GijTV10 all ran GijTV's app_modal_dialog_test.js
GijTV1: https://public-artifacts.taskcluster.net/Esb5zYf_Sc-beQ9o0Hjtlw/0/public/logs/live_backing.log
GijTV10: https://public-artifacts.taskcluster.net/D1URV0fJQHe_nvmxuMJ0qQ/0/public/logs/live_backing.log

This makes we have to wait for test which is ran again and again
And sometimes tests are smooth and sometimes encounter unexpected failure like, 'before each' or 'after each' error.

Is there any setting GijTV missng so GijTV behaves different from Gij ?
Thanks.

---

Turns out the PART and NBPARTS environment variables are not being set so the chunks are effectively running the same tests over and over again.

Comment 1

[GitHub Autolander Bot]

Attachment: [gaia] nullaus:bug1236723 > mozilla-b2g:master

Comment 2

[Aus Lacroix [:aus]]

The pull request shows the tests being split into pieces properly but we don't seem to be running the correct tests (well, any tests at all for that matter) with the chunks working.

I'm a little confused but I would imagine it's a fairly straight forward problem in the chunking logic or in gaia-marionette.

Will update this bug later with more information.

Comment 3

[Fischer [:Fischer]]

Attachment: GijTV9_live_backing.log

Comment 4

[Fischer [:Fischer]]

Attachment: GijTV10_live_backing.log

Comment 5

[Fischer [:Fischer]]

The rea(In reply to Aus Lacroix [:aus] from comment #2)
> The pull request shows the tests being split into pieces properly but we
> don't seem to be running the correct tests (well, any tests at all for that
> matter) with the chunks working.

@Aus,
The reason that no TV tests run is because now the TV tests are temporarily skipped to prevent from slowing down other tests due to this over-run test issue.

I tried your patch. It works.
Now the GijTV tests are split into chunks.

From the attached GijTV9_live_backing.log and GijTV10_live_backing.log, the GijTV tests are no longer being tested in every chunk.

Here are the test results I tried:
https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=a6683e0e659a5ae8f17efe496a50d823a558f59f
https://treeherder.mozilla.org/#/jobs?repo=gaia&revision=cb7ec34db9a98bca036a5e4154c45f9936ba595e

Thanks

Comment 6

[Aus Lacroix [:aus]]

(In reply to [:Fischer]Fischer from comment #5)
> The rea(In reply to Aus Lacroix [:aus] from comment #2)
> > The pull request shows the tests being split into pieces properly but we
> > don't seem to be running the correct tests (well, any tests at all for that
> > matter) with the chunks working.
> 
> @Aus,
> The reason that no TV tests run is because now the TV tests are temporarily
> skipped to prevent from slowing down other tests due to this over-run test
> issue.

Ah, gotcha, that would certainly explain it. :) I'll go ahead and merge this pull request then and you should be able to re-enable the TV tests. 
> 
> I tried your patch. It works.
> Now the GijTV tests are split into chunks.
> 
> From the attached GijTV9_live_backing.log and GijTV10_live_backing.log, the
> GijTV tests are no longer being tested in every chunk.
> 
> Here are the test results I tried:
> https://treeherder.mozilla.org/#/
> jobs?repo=gaia&revision=a6683e0e659a5ae8f17efe496a50d823a558f59f
> https://treeherder.mozilla.org/#/
> jobs?repo=gaia&revision=cb7ec34db9a98bca036a5e4154c45f9936ba595e
> 
> Thanks

Oh, that would explain it :) I hadn't checked into that yet imagining it was probably a small problem. :)

Comment 7

[Aus Lacroix [:aus]]

Commit (master): https://github.com/mozilla-b2g/gaia/commit/57e5568deffd0c2fdeae99f9b93258ed538e8bec

Fixed!