Closed Bug 1623134 Opened 1 year ago Closed 11 months ago

Update android host utils

Categories

(Testing :: General, task, P2)

task

Tracking

(firefox76 wontfix, firefox77 fixed)

RESOLVED FIXED
mozilla76
Tracking Status
firefox76 --- wontfix
firefox77 --- fixed

People

(Reporter: gbrown, Assigned: aerickson)

References

Details

(Whiteboard: dev-prod-2020)

Attachments

(3 files)

+++ This bug was initially created as a clone of Bug #1585422 +++

Please update host-utils again.

See the discussion in bug 1622701: We want to make sure that this archive contains the latest fix_stacks.py, from the bin directory of the Linux common.tests.

Flags: needinfo?(aerickson)
Assignee: nobody → aerickson
Status: NEW → ASSIGNED
Flags: needinfo?(aerickson)
Whiteboard: dev-prod-2020

Aw nuts, this backout means that D66918 in bug 1622701 will need to be backed out too.

Status: ASSIGNED → RESOLVED
Closed: 1 year ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 76
Status: RESOLVED → REOPENED
Resolution: FIXED → ---

Something has changed in hostutils that's caused the test to fail. We'll need to land a test fix with this change.

Flags: needinfo?(aerickson)

aerickson: Do you know what the required test fix will be? Do you have an idea how long it might take? Thanks.

Flags: needinfo?(aerickson)

test_fetch_cors_sw_reroute.html uses .sjs; we have had trouble in the past with host-utils updates and .sjs tests, as in bug 1514075 -- not sure if that's helpful.

:snorp - Any idea why test_fetch_cors_sw_reroute.html might fail on geckoview when we update host-utils? Alternately, any objection to skipping the test on geckoview? (It is currently skipped on Linux and Windows 10.)

Flags: needinfo?(aerickson) → needinfo?(snorp)

:evilpie - I notice you made some changes to fetch tests in bug 1605854 and had a little trouble with test_fetch_cors_sw_reroute.html; do you have any idea what might be going wrong here? (That test fails on Android/geckoview when we update host-utils, which updates xpcshell and the test server support files for Android. See logs in comment 5.)

Flags: needinfo?(evilpies)

I really just did enough to keep the test running without toSource/uneva;, you should find someone who actually understands ServiceWorkers.

Flags: needinfo?(evilpies)

To prevent this from dragging out, I suggest we simply disable that test on Android.

:gbrown and :snorp, thoughts on disabling? I did a quick look at the tests and I'm not sure how to disable them.

Flags: needinfo?(gbrown)

Yeah, I guess disabling is fine with me.

Flags: needinfo?(snorp)

This broke when hostutils was updated and we don't know why.

Flags: needinfo?(gbrown)

Right now running ./mach reftest <some-reftest> with an Android (emulator) build fails with a Python error about not finding the fix_stacks module. Does anybody know if this bug is supposed to address that, or should I file something new?

I filed bug 1626945 for it anyway, since it doesn't seem like the patches on this bug would fix what I'm seeing.

Pushed by jwillcox@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/9cf0341dd964
Disable test_fetch_cors_sw_reroute.html on Android r=asuth

I'm rerunning the tests on my change now that :snorp's change to disable the tests has landed.

Component: Testing → General
Product: Firefox for Android → Testing
Target Milestone: Firefox 76 → ---
Status: REOPENED → RESOLVED
Closed: 1 year ago11 months ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla76

Still need to actually update host utils of course.

Status: RESOLVED → REOPENED
Resolution: FIXED → ---

gbrown: how can I see what is within the host-utils tarball?

I ask because in bug 1626272 I had to make a change to fix_stacks.py, which necessitated a corresponding change in utils.py. I'm hoping that utils.py is also in the host-utils tarball, otherwise we'll end up using an old version of fix_stacks.py with a new version of utils.py and they won't be happy together. (Using the old version for both will be fine, because the change win bug 1626272 was to fix a problem on Windows.)

Flags: needinfo?(gbrown)

:njn, you can use tooltool to download the package and expand the archive.

arc patch D67236
python testing/mozharness/external_tools/tooltool.py fetch -m testing/config/tooltool-manifests/linux64/hostutils.manifest

I built the linux hostutils from this build/commit: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=5100681103a8c27f67a1acc7f657422b6b79e23b

Thanks for the info, aerickson.

Alas, utils.py is not in that tarball, so AFAICT I am faced with the prospect of an up-to-date, in-tree utils.py running in tandem with an old, host-util'd fix_stacks.py. That's going to be a big problem for bug 1626272, because it's difficult to fix that bug without being able to change fix_stacks.py and utils.py in tandem, sigh.

Hmm, I see a path forward. The fix for bug 1626272 can land, because it only affects opt builds, while the change in bug 1622701 only affects debug builds. And Android testing only runs on debug builds on CI. So, we can:

  • Land bug 1626272, which changes fix_stacks.py and utils.py, but only for opt builds, which means it doesn't affect Android tests on CI.
  • Update host-utils again, to incorporate the changes to fix_stacks.py in bug 1626272.
  • Land bug 1622701, which affects debug builds, and thus Android tests on try.

Goodness. If Android tests ran on opt builds as well as debug builds on CI, bug 1626272 would be impossible to fix without temporarily having some kind of fallback code to handle things if an old fix_stack.py is present.

This whole "Android tests use old versions of stuff" is a real time waster :(

(In reply to Nicholas Nethercote [:njn] from comment #27)

Goodness. If Android tests ran on opt builds as well as debug builds on CI
But they do! Both Android tests run against both opt and debug in CI:
https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&searchStr=test-android

Usually the host-utils update is straight-forward and the host-utils patch is short -- easy enough to land in the same push as whatever requires it.

Flags: needinfo?(gbrown)

But they do! Both Android tests run against both opt and debug in CI:

You are right. I keep forgetting that stack-fixing doesn't occur via utils.py on opt builds.

Anyway, the path forward in comment 27 will still work.

Depends on: 1626272

aerickson: I just auto-landed the last change in bug 1626272, called "Make fix-stacks initialization eager". Assuming it sticks, once it merges to mozilla-central could you update the Linux hostutils update patch to include its revision? The Mac hostutils doesn't need updating, but if you like to keep Linux and Mac hostutils on the same revision then you could re-update it too.

Thank you, and sorry for the complications!

Flags: needinfo?(aerickson)

:njn, no worries. :) Will do.

Flags: needinfo?(aerickson)

aerickson: I just auto-landed the last change in bug 1626272, called "Make fix-stacks initialization eager". Assuming it sticks

It didn't, so I'm taking a different approach in that bug. Which means that the Linux hostutils update here can now land. Apologies for all the back and forth.

OK, landing. No worries. :)

Status: REOPENED → RESOLVED
Closed: 11 months ago11 months ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.