Android tests not being run on try; logs and builds in separate dirs

RESOLVED FIXED

Status

defect
P2
major
RESOLVED FIXED
8 years ago
6 years ago

People

(Reporter: coop, Assigned: aki)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [try][unittest][mobile])

Attachments

(6 attachments, 1 obsolete attachment)

jfkthame reported in IRC this morning that he had not received any unittests for his Android build on Try despite specifying the following syntax:

 try: -p android-r7 -u all

http://hg.mozilla.org/try/rev/290b0456b5f1

I was unable to trigger follow-up tests for him with try_sendchange.py. After doing some quick digging, I noticed that Android builds are ending up in sub dirs that look like this:

/pub/mozilla.org/firefox/try-builds/jkew@mozilla.com-290b0456b5f1/try-mob-andrd-r7-bld/

...but the build logs end up here:

/pub/mozilla.org/firefox/try-builds/jkew@mozilla.com-290b0456b5f1/try-android-r7/

AFAICT, the tools are looking for builds+tests in the try-android-r7/ dir and not finding them.

However, it may not be so simple. cpearce ended up with Android tests for another non-Android-specific push shortly before this:

http://tbpl.mozilla.org/?tree=Try&rev=b96a33ce33c2

Maybe constraining the platform causes the tools look for builds in the wrong dir?
A little digging and it looks like http://hg.mozilla.org/build/buildbotcustom/rev/119176006f3e broke this, will look into a fix.
Assignee

Updated

8 years ago
Assignee: lsblakk → aki
Assignee

Updated

8 years ago
Duplicate of this bug: 656763
Assignee

Updated

8 years ago
Priority: P3 → P2
Assignee

Updated

8 years ago
Duplicate of this bug: 636704
Assignee

Comment 4

8 years ago
So "Android tests not being run on try" is incorrect.
This is part of bug 653736, which causes buildbot to email "sendchange" about the mobile try builds rather than the submitter.

This patch should fix that portion; I still need to investigate upload directories and trychooser.
Attachment #532302 - Flags: review?(lsblakk)
Assignee

Comment 5

8 years ago
Comment on attachment 532302 [details] [diff] [review]
set user to %(who)s on mobile try sendchanges

There are more buildbotcustom changes for mobile try uploads, and possible try syntax. Removing r? flag for now.
Attachment #532302 - Flags: review?(lsblakk)
Assignee

Comment 6

8 years ago
This adds android-r7 to try_sendchange.py, and adds .eabi-arm.apk as a valid build extension (not just .apk because that has 2 matches).

This is dependent on uploads going to try-android-r7/ instead of try-mob-andrd-r7-bld (patch incoming).

This will need to s,android-r7,android, when John Ford lands his MercurialBuildFactory patch.

To test, I softlinked the contents of try-mob-andrd-r7-bld/. into try-android-r7/. in http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/eakhgari@mozilla.com-d8078fc9279e/ and tested using --dry-run :

(bb08)deathduck:~/src/android-try/build-tools/buildfarm/maintenance [14:04:22]
1299$ python try_sendchange.py eakhgari@mozilla.com-d8078fc9279e --dry-run -p android-r7 -t all
Scanning ftp...

Found test pkg for android-r7
Found build for android-r7

What will be sent:

buildbot sendchange --master production-master01.build.mozilla.org:9009 --branch try-android-r7-talos --revision d8078fc9279e --comment "try: --t all" --user eakhgari@mozilla.com http://stage.mozilla.org/pub/mozilla.org/firefox/try-builds/eakhgari@mozilla.com-d8078fc9279e/try-android-r7/fennec-6.0a1.en-US.eabi-arm.apk
Attachment #532346 - Flags: review?(lsblakk)
Assignee

Comment 7

8 years ago
In http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/eakhgari@mozilla.com-d8078fc9279e/try-android-r7/try-mob-android-r7-build-build256.txt.gz (chosen at random), I noticed that during make upload,

POST_UPLOAD_CMD=post_upload.py --tinderbox-builds-dir try-android-r7 -p mobile -i 20110512124038 --revision d8078fc9279e --who eakhgari@mozilla.com --builddir try-mob-andrd-r7-bld --release-to-try-builds

So the --tinderbox-builds-dir is exactly what we want, and the --builddir is wrong.

Setting |uploadArgs['builddir'] = tinderboxBuildsDir| inside |if self.enable_try:| is pretty easy after that.
Attachment #532302 - Attachment is obsolete: true
Attachment #532348 - Flags: review?(lsblakk)
Comment on attachment 532348 [details] [diff] [review]
fix mobile try upload directory and set user to %(who)s

right on!
Attachment #532348 - Flags: review?(lsblakk) → review+
Attachment #532346 - Flags: review?(lsblakk) → review+
Assignee

Comment 9

8 years ago
Comment on attachment 532348 [details] [diff] [review]
fix mobile try upload directory and set user to %(who)s

http://hg.mozilla.org/build/buildbotcustom/rev/6812cce361f7
Attachment #532348 - Flags: checked-in+
Assignee

Comment 10

8 years ago
Comment on attachment 532346 [details] [diff] [review]
try sendchange fix, for now

http://hg.mozilla.org/build/tools/rev/a678252151f1
Attachment #532346 - Flags: checked-in+
Assignee

Updated

8 years ago
Duplicate of this bug: 653736

Updated

8 years ago
Duplicate of this bug: 657110

Comment 13

8 years ago
Also see bug 657357 for a similar issue if not the same (the developer indicated "reftest" rather than "reftest-1" and "reftest-2").

I will try to see if I can do a reconfig and pick aki's change up.

Comment 14

8 years ago
Reconfigured masters this morning with this change.
Assignee

Comment 15

8 years ago
pm02-try was missing the reconfig -- did so today, and pushed another android/maemo build with all tests to try to test.

Any builds that started before this reconfig may still have the wrong upload/sendchange behavior.
Not sure if this is supposed to be fixed yet, but it still seems broken: http://tbpl.mozilla.org/?tree=Try&rev=396b9ecc8e04
Assignee

Comment 17

8 years ago
Yeah, I noticed that yesterday while trying to verify my fix.
The upload looks fixed; the sendchanges look right; the tests seem like they're not triggering.

I need to trace this.

Updated

8 years ago
Duplicate of this bug: 657357
Assignee

Comment 19

8 years ago
Try is building and running android tests when the comment is try: -a. When specifying android separately, it doesn't work.

I'm going to guess this is at least partially due to bug 592078.
Assignee

Comment 20

8 years ago
Ok, as I guessed, the discrepancy of naming the platform 'android-r7' in the mozilla/config.py dictionary and 'android' in the mozilla-tests/config.py dictionary caused problems in trychooser.

I tested a patch in staging where I renamed mozilla-tests/config.py's 'android' to 'android-r7' and that triggered tests.

Pretty soon, bug 557260 is going to rename the dictionary key in mozilla/config.py to 'linux-android' while making user-facing strings say 'android'.

I can push a temporary fix in the meantime that changes the 'android' strings in mozilla-tests/config.py to 'android-r7' and then change them to 'linux-android' when bug 557260 lands, or I can wait until bug 557260 lands and just change it once.

Are people needing this functionality sooner, or ok to wait til the end of the week?
Assignee

Updated

8 years ago
Attachment #534642 - Flags: review?(jhford)
Assignee

Updated

8 years ago
Attachment #534643 - Flags: review?(jhford)
Assignee

Updated

8 years ago
Attachment #534637 - Flags: review?(lsblakk)
Assignee

Updated

8 years ago
Attachment #534638 - Flags: review?(lsblakk)
Attachment #534637 - Flags: review?(lsblakk) → review+
Comment on attachment 534638 [details] [diff] [review]
trychooser: accept android-r7, linux-android, or android as valid platforms

ugly, but if it works...it will have to do until there's one config file to rule them all :)
Attachment #534638 - Flags: review?(lsblakk) → review+
Comment on attachment 534643 [details] [diff] [review]
trychooser: accept android-r7, linux-android, or android as valid platforms (linux-android version)

Review of attachment 534643 [details] [diff] [review]:
-----------------------------------------------------------------

looks good to me!
Attachment #534643 - Flags: review?(jhford) → review+
Comment on attachment 534642 [details] [diff] [review]
mozilla-tests android -> linux-android

Review of attachment 534642 [details] [diff] [review]:
-----------------------------------------------------------------

as implemented, i don't think mozilla-tests knows about stage_platform or stage_product (correct me if i am wrong), so what is going to happen is that logs are going to upload to linux-android in /firefox/ instead of android in /mobile/ (i think). This is not a regression, just a change in behaviour.
In order to get the logs to upload to android, we could set a stage_platform and stage_product factory variable.  Those variables could be used to set builder properties and in turn used in the log upload command generation function, similar to the patch in 557260.

Marking as r+ because I don't think this is regressing behaviour and will normalize mozilla/ and mozilla-tests/ platform naming!

Thanks for writing the patch.
Attachment #534642 - Flags: review?(jhford) → review+
Assignee

Comment 28

8 years ago
Android tests should now be triggering properly.
I still need to deal with test log upload locations; I'll finish that off in bug 637838.
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.