921180 - B2G desktop mochitest runner strips '-b2g' from the appname

Status: RESOLVED FIXED

(Testing :: Mochitest, defect)

Description

[Andrew Halberstadt [:ahal]]

Error: http://pastebin.mozilla.org/3152269

The "TypeError: container is null" happens when the test gaia apps aren't installed. In the past this has happened from a build configuration that excludes them (such as setting PRODUCTION=1).

In this case I don't know what is causing them to not be available. All I know is that I can get them to run with a local b2g desktop build, but not with a build downloaded from tinderbox (note: this is using the same gaia profile for both).

For the record my local mozconfig: http://pastebin.mozilla.org/3152271
Production mozconfig (I think): http://mxr.mozilla.org/mozilla-central/source/b2g/config/mozconfigs/linux64_gecko/nightly

Aki or catlee, do either of you see anything in the production mozconfig that would cause the gaia test apps to be unavailable?

Comment 1

[Aki Sasaki (not active)]

It might be part of MOZILLA_OFFICIAL, but b2g devs probably know more.

Comment 2

[Andrew Halberstadt [:ahal]]

Some extra data points..

- If I ctrl-f for 'test-container' in the b2g desktop build log, there *are no* results.
- If I ctrl-f for 'test-container' in the b2g emulator build log, there *are* results.
- For some reason that is incomprehensible to me, the gaia profile that is packaged alongside the b2g desktop build *does* contain the test-container app (which explains why the profile works with my local build). The downloaded gecko build just seems to ignore it.

Comment 3

[Andrew Halberstadt [:ahal]]

(In reply to Aki Sasaki [:aki] from comment #1)
> It might be part of MOZILLA_OFFICIAL, but b2g devs probably know more.

Yeah that might be it, the ics_armv7a_gecko mozconfig doesn't have that set. If we get the mozharness patch landed and the jobs scheduled on Cedar, I'll be able to more easily try playing around with the mozconfig to see what happens.

Comment 4

[Andrew Halberstadt [:ahal]]

I tried disabling MOZ_OFFICIAL=1 on Cedar, we'll see how that turns out.

https://tbpl.mozilla.org/?tree=Cedar&rev=34def15b58b9

Comment 5

[Andrew Halberstadt [:ahal]]

That didn't seem to fix anything. Either MOZILLA_OFFICIAL=1 has no affect, or they aren't using those mozconfigs. I'll try to play around with the builds locally a bit more. Jonathan, do you see anything in these mozconfigs [1] that might cause the "container is null" error (i.e test apps not being packaged)?

[1] http://mxr.mozilla.org/mozilla-central/source/b2g/config/mozconfigs/linux64_gecko/nightly

Comment 6

[Andrew Halberstadt [:ahal]]

I noticed from the mozconfig.common import, --enable-release is set. New cedar run to test disabling it:
https://tbpl.mozilla.org/?tree=Cedar&rev=f104dd20fbb1

Comment 7

[Jonathan Griffin (:jgriffin)]

This seems to affect any build that's made with GAIADIR in the mozconfig, and is not related to the specific gaia profile used; I can reproduce it locally with a GAIDIR build regardless of how I create my gaia profile.  I'm not sure why this occurs, however.

Comment 8

[Andrew Halberstadt [:ahal]]

With hg blame, looks like the use of a built-in gaia profile is from bug 776783. I imagine we either want to make that built-in profile use test apps, or probably better, just have a separate mozconfig for non-nightly builds that doesn't include the GAIADIR variable.

The motivation seems to be that QA wanted an easier set up process, but the drawback is that you are forced into using a specific profile that is built with the application. I think most b2g devs will want to have the ability to specify their own profiles into downloaded builds. Tony, would having nightly builds use the pre-built profile and normal builds not be an acceptable compromise?

Comment 9

[Tony Chung [:tchung]]

(In reply to Andrew Halberstadt [:ahal] from comment #8)
> With hg blame, looks like the use of a built-in gaia profile is from bug
> 776783. I imagine we either want to make that built-in profile use test
> apps, or probably better, just have a separate mozconfig for non-nightly
> builds that doesn't include the GAIADIR variable.
> 
> The motivation seems to be that QA wanted an easier set up process, but the
> drawback is that you are forced into using a specific profile that is built
> with the application. I think most b2g devs will want to have the ability to
> specify their own profiles into downloaded builds. Tony, would having
> nightly builds use the pre-built profile and normal builds not be an
> acceptable compromise?

I'm not clear from reading this bug if the Device builds from releng will be affected.   I'm most interested if i can still flash builds from pvtbuilds on device, and still see a profile with production apps.   The engineering builds from pvtbuilds also need to continue to be flashed and load engineering apps in the list.

Does your proposal break the above?

Comment 10

[Andrew Halberstadt [:ahal]]

(In reply to Tony Chung [:tchung] from comment #9)
> I'm not clear from reading this bug if the Device builds from releng will be
> affected.   I'm most interested if i can still flash builds from pvtbuilds
> on device, and still see a profile with production apps.   The engineering
> builds from pvtbuilds also need to continue to be flashed and load
> engineering apps in the list.
> 
> Does your proposal break the above?

No, my proposal is to stop bundling the profile for non-nightly b2g_desktop builds (a.k.a the ones here: http://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-inbound-linux64_gecko). B2G device builds would be unaffected.

I just wanted to check with you first because you filed bug 776783 (over a year ago) and I wasn't sure how this special built-in profile is getting used for b2g_desktop these days (if at all).

Comment 11

[Andrew Halberstadt [:ahal]]

Wait this, might just be the whole "b2g-bin" vs "b2g" problem.. Just noticed we are using "b2g". I re-triggered the job from the latest ash run: https://tbpl.mozilla.org/?tree=Ash&rev=c49e41aa05bb

Comment 12

[Andrew Halberstadt [:ahal]]

Still broken, ignore comment 11 :)

Comment 13

[Jonathan Griffin (:jgriffin)]

Yeah, I've reproduced this using local builds, and running the command-line myself with b2g-bin. :(

Comment 14

[Jonathan Griffin (:jgriffin)]

This is due to a bug in mozrunner, I think.  Even when passing --app /path/to/b2g-bin, what mozrunner actually ends up launching is /path/to/b2g.  Debugging...

Comment 15

[Jonathan Griffin (:jgriffin)]

No, it's a bug in runtestsb2g.py.  Patch incoming.

Comment 16

[Jonathan Griffin (:jgriffin)]

Attachment: Use b2g-bin if it exists,

Comment 17

[Andrew Halberstadt [:ahal]]

Comment on attachment 815457 [details] [diff] [review]
Use b2g-bin if it exists,

Review of attachment 815457 [details] [diff] [review]:
-----------------------------------------------------------------

Oh wow. Good catch!

I guess this makes the patch in bug 925375 redundant.

::: testing/mochitest/runtestsb2g.py
@@ +345,3 @@
>      mochitest = B2GDesktopMochitest(marionette, options.profile_data_dir)
>  
>      # b2g desktop builds don't always have a b2g-bin file

nit: update comment since we are now adding '-bin' instead of removing it

Comment 18

[Tony Chung [:tchung]]

(In reply to Andrew Halberstadt [:ahal] from comment #10)
> (In reply to Tony Chung [:tchung] from comment #9)
> > I'm not clear from reading this bug if the Device builds from releng will be
> > affected.   I'm most interested if i can still flash builds from pvtbuilds
> > on device, and still see a profile with production apps.   The engineering
> > builds from pvtbuilds also need to continue to be flashed and load
> > engineering apps in the list.
> > 
> > Does your proposal break the above?
> 
> No, my proposal is to stop bundling the profile for non-nightly b2g_desktop
> builds (a.k.a the ones here:
> http://ftp.mozilla.org/pub/mozilla.org/b2g/tinderbox-builds/mozilla-inbound-
> linux64_gecko). B2G device builds would be unaffected.
> 
> I just wanted to check with you first because you filed bug 776783 (over a
> year ago) and I wasn't sure how this special built-in profile is getting
> used for b2g_desktop these days (if at all).

ah okay. i havent been working with desktop builds in the last year so this doesnt apply to me anymore.  thanks for calling that out!

Comment 19

[Andrew Halberstadt [:ahal]]

I pushed jgriffin's patch to ash and I see orange! Morphing bug.

Comment 20

[Jonathan Griffin (:jgriffin)]

https://hg.mozilla.org/integration/mozilla-inbound/rev/de4adb31aeb6

Comment 21

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/de4adb31aeb6

Comment 22

[Andrew Halberstadt [:ahal]]

I uplifted this to mozilla-b2g26 as it was causing bustage for the newly enabled b2g desktop mochitests:
https://hg.mozilla.org/releases/mozilla-b2g26_v1_2/rev/a91df191b189