Closed Bug 1049202 Opened 6 years ago Closed 6 years ago

automationutils fakes mozinfo

Categories

(Testing :: Mochitest, defect)

defect
Not set

Tracking

(Not tracked)

RESOLVED FIXED
mozilla34

People

(Reporter: gps, Assigned: ted)

References

Details

Attachments

(1 file)

Bug 1038943 added code to automationutils.py to check mozinfo.info['toolkit']. This resulted in a KeyError due to 'toolkit' not being present. https://tbpl.mozilla.org/php/getParsedLog.php?id=45277842&tree=B2g-Inbound&full=1

I downloaded the tests zip archive (https://ftp-ssl.mozilla.org/pub/mozilla.org/mobile/tinderbox-builds/b2g-inbound-android/1407273498/fennec-34.0a1.en-US.android-arm.tests.zip) and examined the mozinfo.json file:

$ python -mjson.tool mochitest/mozinfo.json
{
    "appname": "fennec",
    "asan": false,
    "bin_suffix": "",
    "bits": 32,
    "buildapp": "mobile/android",
    "crashreporter": true,
    "datareporting": true,
    "debug": false,
    "healthreport": true,
    "mozconfig": "/builds/slave/b2g-in-and-0000000000000000000/build/.mozconfig",
    "os": "android",
    "processor": "arm",
    "tests_enabled": true,
    "toolkit": "android",
    "topsrcdir": "/builds/slave/b2g-in-and-0000000000000000000/build",
    "wave": true,
    "webm": true
}

Since mozinfo.json doesn't populate info['toolkit'] by default, I suspect the problem is that find_and_update_from_json() isn't being called from automation and/or that the mozinfo.json from the tests zip is not being loaded.

I'm pretty sure this is a regression - and a serious one at that, since automation is no longer respecting our build configs :/
I'm not sure if this ever worked. We attempt to import mozbuild in the "load mozinfo.json" code path and this will never work in automation since mozbuild is not included as part of tests.zip.

https://hg.mozilla.org/mozilla-central/file/71497ed2e0db/testing/mozbase/mozinfo/mozinfo/mozinfo.py#l152
That code path is just for running locally. The test harnesses should have logic to find the mozinfo.json in the tests.zip. For example, mochitests do: http://mxr.mozilla.org/mozilla-central/source/testing/mochitest/runtests.py#417
I don't think the problem here is that automation isn't respecting our build configs. If you look at the list of skipped tests just before the traceback, they are correctly skipped because 'toolkit == "android"'. So at the very least manifestparser is receiving the correct mozinfo.info. I'm not sure why automationutils.py isn't.
Yeah, bummer. Wonder if that's still true?
Summary: mozinfo.json isn't being loaded in automation → automationutils fakes mozinfo
I have no idea why mozinfo can't be imported, or if it is still the case. However if we want to continue using this hack, setting toolkit to 'android' would "fix" the immediate problem and let us backout khuey's bustage fix.

(Note: I'm not advocating we do that)
So, last I knew that was a workaround for non-mozharness Android test jobs. I believe that since we've killed tegra tests on trunk we don't have any more of those, and we could probably remove that hack. I'll test that theory.
Try push with that junk removed:
https://tbpl.mozilla.org/?tree=Try&rev=c5dd8d66bcb1
Assignee: nobody → ted
We only take that path if the import fails so that means the import is failing, no?
Yeah, but it's possible we aren't bothering to copy mozbase or setup sys.path because it previously wasn't working.

We need Joel or Geoff to chime in here.
Flags: needinfo?(jmaher)
Flags: needinfo?(gbrown)
Yeah, it's definitely still broken:
https://tbpl.mozilla.org/?tree=Try&rev=c5dd8d66bcb1

"ImportError: No module named mozinfo"ImportError: No module named mozinfo" on the Android 2.3 emulator tests. They're running under mozharness, but it looks like they're not installing the mozbase modules.
From quickly reviewing the mozharness scripts, it looks like PandaTest uses MozbaseMixin:
https://hg.mozilla.org/build/mozharness/file/03a994d2ec21/scripts/android_panda.py

where AndroidEmulatorTest does not:
https://hg.mozilla.org/build/mozharness/file/03a994d2ec21/scripts/android_emulator_unittest.py

I'll file another bug to fix that.
Depends on: 1049739
so it looks as though the panda tests work but only the 2.3 emulator tests fail.  Once that is fixed we could remove this!
Flags: needinfo?(jmaher)
I think we are on the right track here. I just put a patch up for review in bug 1049739.
Flags: needinfo?(gbrown)
Might as well get this reviewed then. This can't land until gbrown's mozharness patch makes it into production.
Attachment #8470057 - Flags: review?(jmaher)
Comment on attachment 8470057 [details] [diff] [review]
remove automationutils mozinfo workaround for non-mozharness Android tests

Review of attachment 8470057 [details] [diff] [review]:
-----------------------------------------------------------------

can you verify this on try server?  Are there concerns with b2g here?
Attachment #8470057 - Flags: review?(jmaher) → review+
I pushed it to try this morning after I saw gbrown's patch go into production:
https://tbpl.mozilla.org/?tree=Try&rev=d3d3c2229522

Pretty green!
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #6)
> When this is fixed
> https://hg.mozilla.org/integration/b2g-inbound/rev/d419d642fb4e should be
> reverted.

I forgot to do this, feel free to do that anytime.
https://hg.mozilla.org/mozilla-central/rev/ce924184d6b1
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla34
You need to log in before you can comment on or make changes to this bug.