Closed Bug 1457012 Opened Last year Closed 9 months ago

Update hostutils for Android tests

Categories

(Firefox for Android :: Testing, enhancement, P5)

enhancement

Tracking

()

RESOLVED DUPLICATE of bug 1514075
Tracking Status
firefox61 --- affected

People

(Reporter: kmag, Assigned: gbrown)

References

Details

Attachments

(1 file)

+++ This bug was initially created as a clone of Bug #1433279 +++

+++ This bug was initially created as a clone of Bug #1415242 +++

This needs to be done again after the first parts of bug 1456035 merge.
gbrown: given the rapid refactoring of core parts of XPCOM, maybe it's finally time to make the hostutils a toolchain task (easy for Linux and Windows) and figure out what the story can be for signing with the signing server on macOS (harder).

Alternatively, could we make it so that the things that expect to unpack hostutils can just unpack a Firefox binary directly so that we don't have to repack these bits at all?
Flags: needinfo?(gbrown)
I'd really like to get the rest of bug 1456035 landed in this cycle. I don't want to ship the current half-landed state to release users if I can possibly help it.

Can I get an ETA on this?

Thanks!
(In reply to Nick Alexander :nalexander from comment #1)
> gbrown: given the rapid refactoring of core parts of XPCOM, maybe it's
> finally time to make the hostutils a toolchain task (easy for Linux and
> Windows) and figure out what the story can be for signing with the signing
> server on macOS (harder).

I would welcome that and would be happy to help if I can -- but I'm not familiar with toolchain tasks. I think it would be fine to have a linux-only automated solution: that's all we need for success on treeherder. We can fallback to manual updates for macOS to keep mac hosted local tests happy.

> Alternatively, could we make it so that the things that expect to unpack
> hostutils can just unpack a Firefox binary directly so that we don't have to
> repack these bits at all?

We need more than a Firefox installer: some required components are only packaged in the host common.tests.zip. I think a hostutils-consuming task (say, a mochitest run) could download a host (linux/macOS) Firefox install (target.tar.bz2) and a target.common.tests.zip, extract them, and then use that instead of a tooltool-based host-utils archive. I suppose we could eliminate tooltool this way, instead relying on configured links to a host build, but I don't see much advantage to that.
Flags: needinfo?(gbrown)
(In reply to Kris Maglione [:kmag] (long backlog; ping on IRC if you're blocked) from comment #2)
> Can I get an ETA on this?

I should be able to put together a linux host-utils update tomorrow.

(I imagine a toolchain solution will take longer - Nick?)
There's one more small complication: We need elfhack in host-utils now (at least for treeherder), and elfhack is in neither the firefox installer nor common.tests.zip. It should probably be added to common.tests.zip.
Assignee: nobody → gbrown
Keywords: leave-open
:kmag -- This isn't exhaustive, but so far I'm not seeing Android test failures when I push your final patch from bug 
1456035:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=5c4d14c7215b42f844f2295d374cd2b2e131e74e

Are you sure you need a host-utils update?
Flags: needinfo?(kmaglione+bmo)
(In reply to Geoff Brown [:gbrown] from comment #6)
> :kmag -- This isn't exhaustive, but so far I'm not seeing Android test
> failures when I push your final patch from bug 
> 1456035:
> 
> https://treeherder.mozilla.org/#/
> jobs?repo=try&revision=5c4d14c7215b42f844f2295d374cd2b2e131e74e
> 
> Are you sure you need a host-utils update?

I'm pretty sure, yes. The last patch shouldn't be a problem, since none of those files are needed by the mochitest server. The second-to-last patch will be a problem, since it changes files that are used by the mochitest server to user ChromeUtils.generateQI. Removing the stub part 3.1 patch would also break XPCOMUtils.generateQI with the current version of hostutils.
Flags: needinfo?(kmaglione+bmo)
Got it. Sorry, I didn't look at your patches closely enough.
:bc - Will you update autophone please?
Flags: needinfo?(bob)
done. thanks.
Flags: needinfo?(bob)
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/mozilla-inbound/rev/29afe33cb61d
Update Android host-utils for Linux; r=me,a=test-only
Keywords: leave-open
See Also: → 1294256
:nalexander - Bug 1294256 seems like a good place to move forward on the toolchain task idea. You may be our best bet for driving that. :)

Can you also try to deploy a new host-utils for Mac?
Flags: needinfo?(nalexander)
:kmag - You should be able to go ahead with bug 1456035 now.
Flags: needinfo?(kmaglione+bmo)
(In reply to Geoff Brown [:gbrown] from comment #16)
> :kmag - You should be able to go ahead with bug 1456035 now.

Great. Thanks!
Flags: needinfo?(kmaglione+bmo)
Re-triaging per https://bugzilla.mozilla.org/show_bug.cgi?id=1473195

Needinfo :susheel if you think this bug should be re-triaged.
Priority: P1 → P5
(In reply to Geoff Brown [:gbrown] from comment #14)
> :nalexander - Bug 1294256 seems like a good place to move forward on the
> toolchain task idea. You may be our best bet for driving that. :)
> 
> Can you also try to deploy a new host-utils for Mac?

I haven't managed to get to this in 4 months, and I'm pretty sure I no longer have a valid Xcode signing configuration, so this will require another person if and when it's really required.
Flags: needinfo?(nalexander)
:egao - Can you take this bug?

The initial need here is to deploy a new version of the Android host-utils for linux and osx. For linux, the attached patch should be instructive -- get the parts in place, replace version numbers and paths, and all's well. For osx, it's similar, but requires some additional signing.

In the longer term, we'd like to automate this procedure using a toolchain task. This part is less well defined and will require some investigation.

I'm happy to provide background, answer questions, etc.
Flags: needinfo?(egao)
Depends on: 1471987
Flags: needinfo?(egao)
It looks like elfhack will no longer be required in host-utils - see bug 1499915. :)
Marking as duplicate of 1514075.
Status: NEW → RESOLVED
Closed: 9 months ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1514075
You need to log in before you can comment on or make changes to this bug.