Closed Bug 655046 Opened 10 years ago Closed 10 years ago

Android tests not being run on try; logs and builds in separate dirs

Categories

(Release Engineering :: General, defect, P2)

All
Android

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: coop, Assigned: aki)

References

Details

(Whiteboard: [try][unittest][mobile])

Attachments

(6 files, 1 obsolete file)

jfkthame reported in IRC this morning that he had not received any unittests for his Android build on Try despite specifying the following syntax:

 try: -p android-r7 -u all

http://hg.mozilla.org/try/rev/290b0456b5f1

I was unable to trigger follow-up tests for him with try_sendchange.py. After doing some quick digging, I noticed that Android builds are ending up in sub dirs that look like this:

/pub/mozilla.org/firefox/try-builds/jkew@mozilla.com-290b0456b5f1/try-mob-andrd-r7-bld/

...but the build logs end up here:

/pub/mozilla.org/firefox/try-builds/jkew@mozilla.com-290b0456b5f1/try-android-r7/

AFAICT, the tools are looking for builds+tests in the try-android-r7/ dir and not finding them.

However, it may not be so simple. cpearce ended up with Android tests for another non-Android-specific push shortly before this:

http://tbpl.mozilla.org/?tree=Try&rev=b96a33ce33c2

Maybe constraining the platform causes the tools look for builds in the wrong dir?
A little digging and it looks like http://hg.mozilla.org/build/buildbotcustom/rev/119176006f3e broke this, will look into a fix.
Assignee: lsblakk → aki
Duplicate of this bug: 656763
Priority: P3 → P2
Duplicate of this bug: 636704
So "Android tests not being run on try" is incorrect.
This is part of bug 653736, which causes buildbot to email "sendchange" about the mobile try builds rather than the submitter.

This patch should fix that portion; I still need to investigate upload directories and trychooser.
Attachment #532302 - Flags: review?(lsblakk)
Comment on attachment 532302 [details] [diff] [review]
set user to %(who)s on mobile try sendchanges

There are more buildbotcustom changes for mobile try uploads, and possible try syntax. Removing r? flag for now.
Attachment #532302 - Flags: review?(lsblakk)
This adds android-r7 to try_sendchange.py, and adds .eabi-arm.apk as a valid build extension (not just .apk because that has 2 matches).

This is dependent on uploads going to try-android-r7/ instead of try-mob-andrd-r7-bld (patch incoming).

This will need to s,android-r7,android, when John Ford lands his MercurialBuildFactory patch.

To test, I softlinked the contents of try-mob-andrd-r7-bld/. into try-android-r7/. in http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/eakhgari@mozilla.com-d8078fc9279e/ and tested using --dry-run :

(bb08)deathduck:~/src/android-try/build-tools/buildfarm/maintenance [14:04:22]
1299$ python try_sendchange.py eakhgari@mozilla.com-d8078fc9279e --dry-run -p android-r7 -t all
Scanning ftp...

Found test pkg for android-r7
Found build for android-r7

What will be sent:

buildbot sendchange --master production-master01.build.mozilla.org:9009 --branch try-android-r7-talos --revision d8078fc9279e --comment "try: --t all" --user eakhgari@mozilla.com http://stage.mozilla.org/pub/mozilla.org/firefox/try-builds/eakhgari@mozilla.com-d8078fc9279e/try-android-r7/fennec-6.0a1.en-US.eabi-arm.apk
Attachment #532346 - Flags: review?(lsblakk)
In http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/eakhgari@mozilla.com-d8078fc9279e/try-android-r7/try-mob-android-r7-build-build256.txt.gz (chosen at random), I noticed that during make upload,

POST_UPLOAD_CMD=post_upload.py --tinderbox-builds-dir try-android-r7 -p mobile -i 20110512124038 --revision d8078fc9279e --who eakhgari@mozilla.com --builddir try-mob-andrd-r7-bld --release-to-try-builds

So the --tinderbox-builds-dir is exactly what we want, and the --builddir is wrong.

Setting |uploadArgs['builddir'] = tinderboxBuildsDir| inside |if self.enable_try:| is pretty easy after that.
Attachment #532302 - Attachment is obsolete: true
Attachment #532348 - Flags: review?(lsblakk)
Comment on attachment 532348 [details] [diff] [review]
fix mobile try upload directory and set user to %(who)s

right on!
Attachment #532348 - Flags: review?(lsblakk) → review+
Attachment #532346 - Flags: review?(lsblakk) → review+
Comment on attachment 532348 [details] [diff] [review]
fix mobile try upload directory and set user to %(who)s

http://hg.mozilla.org/build/buildbotcustom/rev/6812cce361f7
Attachment #532348 - Flags: checked-in+
Duplicate of this bug: 653736
Duplicate of this bug: 657110
Also see bug 657357 for a similar issue if not the same (the developer indicated "reftest" rather than "reftest-1" and "reftest-2").

I will try to see if I can do a reconfig and pick aki's change up.
Reconfigured masters this morning with this change.
pm02-try was missing the reconfig -- did so today, and pushed another android/maemo build with all tests to try to test.

Any builds that started before this reconfig may still have the wrong upload/sendchange behavior.
Not sure if this is supposed to be fixed yet, but it still seems broken: http://tbpl.mozilla.org/?tree=Try&rev=396b9ecc8e04
Yeah, I noticed that yesterday while trying to verify my fix.
The upload looks fixed; the sendchanges look right; the tests seem like they're not triggering.

I need to trace this.
Duplicate of this bug: 657357
Try is building and running android tests when the comment is try: -a. When specifying android separately, it doesn't work.

I'm going to guess this is at least partially due to bug 592078.
Ok, as I guessed, the discrepancy of naming the platform 'android-r7' in the mozilla/config.py dictionary and 'android' in the mozilla-tests/config.py dictionary caused problems in trychooser.

I tested a patch in staging where I renamed mozilla-tests/config.py's 'android' to 'android-r7' and that triggered tests.

Pretty soon, bug 557260 is going to rename the dictionary key in mozilla/config.py to 'linux-android' while making user-facing strings say 'android'.

I can push a temporary fix in the meantime that changes the 'android' strings in mozilla-tests/config.py to 'android-r7' and then change them to 'linux-android' when bug 557260 lands, or I can wait until bug 557260 lands and just change it once.

Are people needing this functionality sooner, or ok to wait til the end of the week?
Attachment #534642 - Flags: review?(jhford)
Attachment #534643 - Flags: review?(jhford)
Attachment #534637 - Flags: review?(lsblakk)
Attachment #534638 - Flags: review?(lsblakk)
Attachment #534637 - Flags: review?(lsblakk) → review+
Comment on attachment 534638 [details] [diff] [review]
trychooser: accept android-r7, linux-android, or android as valid platforms

ugly, but if it works...it will have to do until there's one config file to rule them all :)
Attachment #534638 - Flags: review?(lsblakk) → review+
Comment on attachment 534643 [details] [diff] [review]
trychooser: accept android-r7, linux-android, or android as valid platforms (linux-android version)

Review of attachment 534643 [details] [diff] [review]:
-----------------------------------------------------------------

looks good to me!
Attachment #534643 - Flags: review?(jhford) → review+
Comment on attachment 534642 [details] [diff] [review]
mozilla-tests android -> linux-android

Review of attachment 534642 [details] [diff] [review]:
-----------------------------------------------------------------

as implemented, i don't think mozilla-tests knows about stage_platform or stage_product (correct me if i am wrong), so what is going to happen is that logs are going to upload to linux-android in /firefox/ instead of android in /mobile/ (i think). This is not a regression, just a change in behaviour.
In order to get the logs to upload to android, we could set a stage_platform and stage_product factory variable.  Those variables could be used to set builder properties and in turn used in the log upload command generation function, similar to the patch in 557260.

Marking as r+ because I don't think this is regressing behaviour and will normalize mozilla/ and mozilla-tests/ platform naming!

Thanks for writing the patch.
Attachment #534642 - Flags: review?(jhford) → review+
Android tests should now be triggering properly.
I still need to deal with test log upload locations; I'll finish that off in bug 637838.
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Product: mozilla.org → Release Engineering
You need to log in before you can comment on or make changes to this bug.