913174 - Represent Android x86 emulator test jobs with "S#" rather than "U"

Status: RESOLVED FIXED

(Tree Management Graveyard :: TBPL, defect)

Description

[Armen [:armenzg]]

They should show up under here:
https://tbpl.mozilla.org/?tree=Ash&rev=0d4ae6057ef5&jobname=Android%20x86%20Emulator%20ash%20opt%20test%20androidx86-set&showall=1

Comment 1

[Armen [:armenzg]]

I think if I follow the steps on bug 820882 I should be able to get you a patch.

Comment 2

[Armen [:armenzg]]

Attachment: androidx86.tbpl.diff

I have no idea what I'm doing.

Comment 3

[Ryan VanderMeulen [:RyanVM]]

Does this work the way you would expect when loading Ash in TBPL locally (by opening index.html in the browser)?

Comment 4

[Armen [:armenzg]]

FTR, I filed this bug because I had run a sendchange incorrectly and obviously tbpl would not show them.
I can see them now with the letter "U".

I don't know how to change the letter from U to something else.

I was thinking of naming them like S1, S2, S3, S4 & S5 and group them if possible.

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #3)
> Does this work the way you would expect when loading Ash in TBPL locally (by
> opening index.html in the browser)?

oooohh I didn't know.

It seems that I don't need the patch actually.

Comment 5

[Ed Morley [:emorley]]

Comment on attachment 800339 [details] [diff] [review]
androidx86.tbpl.diff

We'll ideally want the test jobs on the same row as the builds, whereas this patch will put them on another row (the existing /android x86/ regex catches these).

TBPL (presuming all works correctly!) will display unknown job types/platforms generically - eg "Other" for platform, and "M", "T", "U", or "?" for job types depending on which of the generic regexps catch it. As such, we should never need a TBPL patch to even be able to see the job - they're just for aesthetics :-)

What are these job types? Mochitest?

Comment 6

[Armen [:armenzg]]

Sorry! I forgot to remove the review request after I determined in comment 4 that it was not needed.

We run 4 test suites per job so we do not necessary have to have all suites within to be of the same category.

We have 5 sets of tests [1]
- set1: m-1 to m-4
- set2: m-4 to m-8
- set3: r-1 to r3 + crashtest
- set4: m-gl, xpcshell & robocop{1,2}
- set5: jsreftest-[1-3]

These sets are probably going to change a bit since if all four run successfully they could generate to much output for us to handle (more than 50MB). Breaking them up is the easy way but we have other more time consuming options (We don't want to bump the maxLogSize due to its perf hit on the masters).

For now, I think the best naming would be "S#" (# being a decimal) but I'm open to other options if this is not wanted.

[1] http://hg.mozilla.org/build/buildbot-configs/file/default/mozilla-tests/mobile_config.py#l671

Comment 7

[Ryan VanderMeulen [:RyanVM]]

(In reply to Ed Morley [:edmorley UTC+1] from comment #5)
> What are these job types? Mochitest?

These are the unholy concatenation of multiple different jobs in one "result".

Comment 8

[Ryan VanderMeulen [:RyanVM]]

"S" is already in use for static analysis builds, so I would prefer to avoid duplicating it for something that's entirely unrelated.

Comment 9

[Ryan VanderMeulen [:RyanVM]]

Taking this, even if I don't like it...

Comment 10

[Ryan VanderMeulen [:RyanVM]]

Attachment: Add support for Android x86 test sets

Still not a huge fan, but it gets the job done. I'm all ears to any suggested changes. I also went ahead and added the conditional 4.2 to the x86 regex to accommodate the changes in bug 823900.

Comment 11

[Ed Morley [:emorley]]

Comment on attachment 801992 [details] [diff] [review]
Add support for Android x86 test sets

Yuck, really not a fan of grouping Android test suites together, but seeing as that's the path we're heading down :-/

Comment 12

[Ryan VanderMeulen [:RyanVM]]

LAAAAAME

https://hg.mozilla.org/webtools/tbpl/rev/e1f18663a8d8

Comment 13

[Ed Morley [:emorley]]

In production