Closed Bug 1624649 Opened 5 months ago Closed 3 months ago

Switch android emulator to be pulled via toolchain instead of tooltool

Categories

(Firefox Build System :: Task Configuration, enhancement, P1)

enhancement

Tracking

(firefox78 fixed)

RESOLVED FIXED
mozilla78
Tracking Status
firefox78 --- fixed

People

(Reporter: tomprince, Assigned: gbrown)

References

(Regressed 1 open bug)

Details

Attachments

(2 files)

Currently, we build android emulator packages manually and upload them to tooltool. It would be better to build them in-tree as private artifacts via toolchain jobs.

The current code for packing sdks is here and here. There are instructions here on mana (private) about how the current tooltool package is built. Looking at the instructions and existing script, it may be possible to use the existing packages; with just some adjustment to expected paths.

There will also need to be adjustments to remove the existing files from the tooltool manifests, adding the appropriate fetch stanza to tasks that need it, and dealing with the differences between where tooltool and fetch-content extract downloads.

Blocks: 1578460
Priority: -- → P1
Assignee: nobody → wcosta
Status: NEW → ASSIGNED

Currently, we build android emulator packages manually and upload to
tooltool.

This patch switches it to be pulled from the toolchain built artifacts.

This also allows android tests to run in the staging environment.

Pushed by wcosta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/0592b572c58d
Move android emu to pull from toolchain instead tooltool r=nalexander

Backed out changeset 0592b572c58d (bug 1624649) for many failures complaining about minidump_stackwalk path

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=295810461&resultStatus=testfailed%2Cbusted%2Cexception&fromchange=0592b572c58de39307828c80c076ff135da649aa&tochange=1cddc2938e75fa6ed39192891c386e9ff7d58904

Backout link: https://hg.mozilla.org/integration/autoland/rev/1cddc2938e75fa6ed39192891c386e9ff7d58904

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=295810461&repo=autoland&lineNumber=978

[task 2020-04-01T18:39:23.917Z] 18:39:23     INFO - Running pre test command restart windows explorer with 'powershell -command "&{&Stop-Process -ProcessName explorer}"'
[task 2020-04-01T18:39:23.917Z] 18:39:23     INFO - Running command: ('powershell', '-command', '"&{&Stop-Process -ProcessName explorer}"') in Z:\task_1585763962\build
[task 2020-04-01T18:39:23.917Z] 18:39:23     INFO - Copy/paste: powershell -command "\"&{&Stop-Process -ProcessName explorer}\""
[task 2020-04-01T18:39:24.659Z] 18:39:24     INFO -  &{&Stop-Process -ProcessName explorer}
[task 2020-04-01T18:39:24.683Z] 18:39:24     INFO - Return code: 0
[task 2020-04-01T18:39:24.712Z] 18:39:24     INFO - Unable to complete system-info.log: <type 'exceptions.TypeError'>
[task 2020-04-01T18:39:24.712Z] 18:39:24     INFO - #### Running mochitest suites
[task 2020-04-01T18:39:24.712Z] 18:39:24     INFO - Structured output parser in use for mochitest.
[task 2020-04-01T18:39:24.733Z] 18:39:24    ERROR - minidump_stackwalk path was not fetched?
[task 2020-04-01T18:39:24.733Z] 18:39:24  WARNING - # TBPL WARNING #
[task 2020-04-01T18:39:24.733Z] 18:39:24  WARNING - setting return code to 1
Flags: needinfo?(wcosta)

(In reply to Bogdan Tara[:bogdan_tara | bogdant] from comment #3)

Backed out changeset 0592b572c58d (bug 1624649) for many failures complaining about minidump_stackwalk path

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=autoland&selectedJob=295810461&resultStatus=testfailed%2Cbusted%2Cexception&fromchange=0592b572c58de39307828c80c076ff135da649aa&tochange=1cddc2938e75fa6ed39192891c386e9ff7d58904

Backout link: https://hg.mozilla.org/integration/autoland/rev/1cddc2938e75fa6ed39192891c386e9ff7d58904

Failure log: https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=295810461&repo=autoland&lineNumber=978

[task 2020-04-01T18:39:23.917Z] 18:39:23     INFO - Running pre test command restart windows explorer with 'powershell -command "&{&Stop-Process -ProcessName explorer}"'
[task 2020-04-01T18:39:23.917Z] 18:39:23     INFO - Running command: ('powershell', '-command', '"&{&Stop-Process -ProcessName explorer}"') in Z:\task_1585763962\build
[task 2020-04-01T18:39:23.917Z] 18:39:23     INFO - Copy/paste: powershell -command "\"&{&Stop-Process -ProcessName explorer}\""
[task 2020-04-01T18:39:24.659Z] 18:39:24     INFO -  &{&Stop-Process -ProcessName explorer}
[task 2020-04-01T18:39:24.683Z] 18:39:24     INFO - Return code: 0
[task 2020-04-01T18:39:24.712Z] 18:39:24     INFO - Unable to complete system-info.log: <type 'exceptions.TypeError'>
[task 2020-04-01T18:39:24.712Z] 18:39:24     INFO - #### Running mochitest suites
[task 2020-04-01T18:39:24.712Z] 18:39:24     INFO - Structured output parser in use for mochitest.
[task 2020-04-01T18:39:24.733Z] 18:39:24    ERROR - minidump_stackwalk path was not fetched?
[task 2020-04-01T18:39:24.733Z] 18:39:24  WARNING - # TBPL WARNING #
[task 2020-04-01T18:39:24.733Z] 18:39:24  WARNING - setting return code to 1

Ok, that is unexpected, I understood the patch should only affect android-x86[_64]. Let me investigate this

Flags: needinfo?(wcosta)
Attachment #9136414 - Attachment description: Bug 1624649: Move android emu to pull from toolchain instead tooltool r=glandium → Bug 1624649: Move android emu to pull from toolchain instead tooltool r=nalexander

:nalexander, I pushed another version of the patch, would you mind reviewing it?

Flags: needinfo?(nalexander)

(In reply to Wander Lairson Costa from comment #5)

:nalexander, I pushed another version of the patch, would you mind reviewing it?

If it's green for you, it's good for me.

Flags: needinfo?(nalexander)
Pushed by wcosta@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/1ad3c93e1e07
Move android emu to pull from toolchain instead tooltool r=nalexander
Assignee: wcosta → nobody
Status: ASSIGNED → NEW

This seems really close. I'll try to push it over the line...

Assignee: nobody → gbrown

(In reply to Geoff Brown [:gbrown] from comment #10)

This seems really close. I'll try to push it over the line...

The latest version I have is here. Theoretically it should work, but I don't know why even after fixing wrench.py it keeps looking for adb in the tooltool path.

This is very similar to wcosta's earlier patch, with a few adjustments to overcome
special cases:

  • remove the wrench override of adb path; handle in config instead
  • fix taskcluster config syntax in a couple of places
  • add android-sdk-linux fetch for test-verify
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/56b9dd8bd253
Move android-em to pull emulator from toolchain instead of tooltool; r=wcosta,kats
Regressions: 1635905
Status: NEW → RESOLVED
Closed: 3 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla78
Regressions: 1636514
You need to log in before you can comment on or make changes to this bug.