1011016 - Marketplace-dev is not installed on Firefox OS v2.0 releng engineering builds

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia::Build, defect)

Description

[Zac C (:zac)]

Marketplace dev is not installed on engineering builds of Firefox OS v2.0.

Only Marketplace stage and prod are installed.. We need dev to do testing against.

This is blocking QA smoketests.

(NB, dev is installed on v1.3 builds so something changed in the build config)

Gaia      2f89c43e798ccba631025bedc47a1fb24e830cf2
Gecko     https://hg.mozilla.org/mozilla-central/rev/4b6d63b05a0a
BuildID   20140512160204
Version   32.0a1
ro.build.version.incremental=324
ro.build.date=Thu Dec 19 14:04:55 CST 2013

Comment 1

[[:AndreiH]]

Attachment: v1.3 Marketplace dev.png

Screenshot from v1.3 build. You can see that all 3 Marketplace icons are there.
On v1.4 and v2.0 Marketplace dev is missing.

Comment 2

[Zac C (:zac)]

Krupa, can you speak to the right people to make this happen?

Comment 3

[krupa raj[:krupa]]

Adding clouserw

A little history: Currently, the engineering builds have the hosted dev app which comes with limited functionality. This will be a good time to ship the engineering builds with packaged dev app (with certs)

Talking about dev certs, Bug 864820 is still open. 

What we need to add is the packaged dev app from https://marketplace-dev.allizom.org/app/marketplace installed by default on 2.0 Engineering builds.

Clouser can add more details if needed.

Comment 4

[Zac C (:zac)]

Jsmith, this is blocking automation of a QA smoketest, do you know who can work on this?

Comment 5

[Jason Smith [:jsmith]]

I don't know. I know what needs to be done here, but I don't have time to look into right now.

Comment 6

[Jason Smith [:jsmith]]

Why can't you use marketplace stage for testing?

Comment 7

[Wil Clouser [:clouserw]]

Can you link to where these are in the gaia tree?  I see these two on master:

https://github.com/mozilla-b2g/gaia/tree/master/dev_apps/marketplace-dev.allizom.org
https://github.com/mozilla-b2g/gaia/tree/master/dev_apps/marketplace.allizom.org

but I'm not sure where 2.0 is.

Comment 8

[Bob Silverberg [:bsilverberg]]

Just adding a note that we want to move _all_ marketplace testing on Firefox OS to v1.4, but the lack of the Marketplace Dev packaged app on the engineering build is blocking us from doing that too, so let's please try to get this fixed asap. As noted above, the Marketplace Dev app is currently installed in the v1.3 build, but is missing from v1.4 and above. Perhaps it's just an issue of comparing the two build configs and then having the appropriate person implement the fix? Feel free to contact me directly if you need anything from me.

Comment 9

[Wil Clouser [:clouserw]]

My question in comment 7 is unanswered, but on the 2.0 branch there exists a marketplace-dev app also:

https://github.com/mozilla-b2g/gaia/tree/v2.0/dev_apps/marketplace-dev.allizom.org

I don't know why that wouldn't show up in the build.  Fabrice will have better input than me.

Comment 10

[Bob Silverberg [:bsilverberg]]

Thanks Will. I don't know the answer to your question in comment 7, although it looks like you provided the link above. Fabrice, please note that we also need this on v1.4, not only on v2.0.

Comment 11

[[:fabrice] Fabrice Desré]

The cause on 2.0 is that the marketplace-dev is missing from the list at https://github.com/mozilla-b2g/gaia/blob/v2.0/build/config/phone/apps-engineering.list

On 1.4 the list is in https://github.com/mozilla-b2g/gaia/blob/v1.4/build/config/apps-engineering.list

Comment 12

[Bob Silverberg [:bsilverberg]]

Thanks Fabrice.

The list at https://github.com/mozilla-b2g/gaia/blob/v1.4/build/config/apps-engineering.list includes `test_external_apps/*`, and there is a folder under `test_external_apps` called `marketplace-dev.allizom.org` which seems to be the marketplace-dev app, so I would expect it to be installed.

Comparing the marketplace-dev.allizom.org folders between v1.3 (which does have Marketplace-dev installed) and v1.4 (which does not) shows that the metadata.json file in the latter is missing a line that is in the former. Specifically, https://github.com/mozilla-b2g/gaia/blob/v1.3/test_external_apps/marketplace-dev.allizom.org/metadata.json#L5 is missing from the file on v1.4.

Could that be the reason that the app is not being installed on the v1.4 build?

Comment 13

[Wil Clouser [:clouserw]]

That seems reasonable, but I don't know.  Fabrice is the gaia expert.

Comment 14

[[:fabrice] Fabrice Desré]

the origin was removed from metadata.json in bug 929602. So we use the origin from the manifest, but marketplace-dev has the origin set to "origin": "app://marketplace.firefox.com" (see manifest.webapp in https://github.com/mozilla-b2g/gaia/blob/v1.4/test_external_apps/marketplace-dev.allizom.org/application.zip), and that fails to install alongside the non-dev marketplace.

Comment 15

[John Ford [:jhford] CET/CEST Berlin Time]

The push for this change is failing in a reproducible way.  I can reproduce it on my local machine.

Specifically:

  1) Build Integration tests make without rule & variable:
     Uncaught AssertionError: expected -1 to equal 0
      at /Users/jhford/b2g/gaia/build/test/integration/helper.js:62:12
      at Array.forEach (native)
      at Object.checkWebappsScheme (/Users/jhford/b2g/gaia/build/test/integration/helper.js:51:24)
      at /Users/jhford/b2g/gaia/build/test/integration/build.test.js:369:14
      at ChildProcess.exithandler (child_process.js:645:7)
      at ChildProcess.emit (events.js:98:17)
      at maybeClose (child_process.js:755:16)
      at Process.ChildProcess._handle.onexit (child_process.js:822:5)

  2) Build Integration tests make with DEBUG=1:
     Uncaught AssertionError: expected 2 to equal 1
      at complete (/Users/jhford/b2g/gaia/build/test/integration/build.test.js:670:18)
      at /Users/jhford/b2g/gaia/node_modules/dive/dive.js:72:19
      at Object.oncomplete (fs.js:107:15)


I've run the build for this commit with "make" and "make DEBUG=1" and I have an exit code of 0 both times.  Do these tests check for more than just exit code?

The assertion for the first error is: 

  var scheme =
    webapp.origin.indexOf('mochi.test') !== -1 ||
    webapp.origin.indexOf('marketplace.allizom.org') !== -1 ||
    webapp.origin.indexOf('inapp-pay-test.paas.allizom.org') !== -1 ?
    'http' : 'app';
  assert.equal(webapp.origin.indexOf(scheme), 0);

The assertion for the second error is:

  // only expect one zip file for marketplace.
  assert.equal(zipCount, 1);

So it looks like we have two zip files being created?  I wonder if this package is being included in a place where it shouldn't be?

Yuren, do you have any ideas what might be going wrong here?

Comment 16

[Andy McKay]

>     webapp.origin.indexOf('marketplace.allizom.org') !== -1 ||

I'm not sure of the rest, but on dev that assertion should be something like marketplace-dev.allizom.org, that domain points to stage, not dev.

Comment 17

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

the reason is we use origin field in application.zip file to create directory in profile/webapps, in this case application.zip in marketplace-dev use "app://marketplace.firefox.com" in origin field, so marketplace-dev will be copy to profile/webapps/marketplace.firefox.com.

Comment 18

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

https://github.com/mozilla-b2g/gaia/blob/master/build/webapp-manifests.js#L45-L47

Comment 19

[Wil Clouser [:clouserw]]

That makes sense and I see that issue in the code in b2g right now.  Bug 1038350 will fix this problem so I'd suggest we fix that bug and get this one for free.

Comment 20

[Wil Clouser [:clouserw]]

(In reply to Wil Clouser [:clouserw] from comment #19)
> That makes sense and I see that issue in the code in b2g right now.  Bug
> 1038350 will fix this problem so I'd suggest we fix that bug and get this
> one for free.

https://github.com/mozilla-b2g/gaia/pull/21722 is the PR from bug 1038350 that will fix this bug but we need help with why the tests fail.

Comment 21

[Andy McKay]

We got new builds on bug 1038350, any idea what those errors are?

Comment 22

[Yuren [:yurenju] (AFK ~ Oct. 12) (volunteer mode)]

replied there.

Comment 23

[Zac C (:zac)]

I just flashed with a b2g32_2.0 Flame build from pvt using https://github.com/Mozilla-TWQA/B2G-flash-tool/blob/master/flash_pvt.py flash tool and I could not find Marketplace dev on the homescreen.

Here is my Flame's configuration:
Gaia      295c7f50707372e5af6d8e83148d2d970076dfd6
Gecko     https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/879c5208084f
BuildID   20140815000200
Version   32.0
ro.build.version.incremental=110
ro.build.date=Fri Jun 27 15:57:58 CST 2014
B1TC00011230

Comment 24

[Wil Clouser [:clouserw]]

The origin issues in comment 17 were fixed when Fabrice landed the patches in bug 1038350.  If I follow the rest of the discussion above, I think this (untested) patch will fix this bug: https://github.com/mozilla-b2g/gaia/pull/22944

Comment 25

[James Lal [:lightsofapollo]]

I landed this https://github.com/mozilla-b2g/gaia/commit/ddec117b2d6ac8ea50d7fd833a9cf0605d5b358b if shit is broken after it its my fault and lets back it out (should be fine)

Comment 26

[Ryan VanderMeulen [:RyanVM]]

Reverted for Gaia build test perma-fail.
v2.0: https://github.com/mozilla-b2g/gaia/commit/57b4f805719cca636b3b94224a4925d06025887a

https://tbpl.mozilla.org/php/getParsedLog.php?id=47826763&tree=Mozilla-B2g32-v2.0

Comment 27

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Ryan VanderMeulen [:RyanVM UTC-4] from comment #26)
> Reverted for Gaia build test perma-fail.
> v2.0:
> https://github.com/mozilla-b2g/gaia/commit/
> 57b4f805719cca636b3b94224a4925d06025887a
> 
> https://tbpl.mozilla.org/php/getParsedLog.php?id=47826763&tree=Mozilla-B2g32-
> v2.0

Reopening -- James, mind taking another look?  Thanks!

Comment 28

[Stephen Donner [:stephend] Not actively reading bugmail]

Dylan/James -- can we please get some <3 on this?  Thanks!

Comment 29

[James Lal [:lightsofapollo]]

Attachment: https://github.com/mozilla-b2g/gaia/pull/24539

Will uplift after we land on master this is not part of the production build but does effect eng builds we generate

Comment 30

[Ricky Chien [:rickychien] (inactive)]

Comment on attachment 8497284 [details] [review]
https://github.com/mozilla-b2g/gaia/pull/24539

 It looks good and works fine on my machine. Please set r=@RickyChien.

Comment 31

[James Lal [:lightsofapollo]]

Comment on attachment 8497284 [details] [review]
https://github.com/mozilla-b2g/gaia/pull/24539

[Approval Request Comment]
[Bug caused by] (feature/regressing bug #):
[User impact] if declined:
[Testing completed]:
[Risk to taking this patch] (and alternatives if risky):
[String changes made]:

NOTE: Please see https://wiki.mozilla.org/Release_Management/B2G_Landing to better understand the B2G approval process and landings.

Not part of the production build just needs uplift (this does effect eng builds however!)

[Approval Request Comment]
[Bug caused by] (feature/regressing bug #):
[User impact] if declined:
[Testing completed]:
[Risk to taking this patch] (and alternatives if risky):
[String changes made]:

Comment 32

[Kevin Grandon :kgrandon]

Just making a note that I saw a `bug-1011016` branch was created on upstream against this bug. I assume it was accidental, and since this has landed I've gone ahead and removed it.

Comment 33

[bhavana bajaj [:bajaj]]

Comment on attachment 8497284 [details] [review]
https://github.com/mozilla-b2g/gaia/pull/24539

low-risk, NPOTB approving on 2.0 as well

Comment 34

[Martijn Wargers (dead)]

Iiutc, this should be fixed on trunk now, right?

Comment 35

[Bob Silverberg [:bsilverberg]]

When is this expected to land? This morning's v2.0 engineering build does not have Marketplace Dev installed. This is blocking re-enabling Marketplace tests on device as they are now going to be running on v2.0.

Comment 36

[Bob Silverberg [:bsilverberg]]

kgrandon: Is this supposed to have landed, or is it still waiting for someone/something?

Comment 37

[Kevin Grandon :kgrandon]

Hmm, it looks like this was landed, but never uplifted due to it not being marked as fixed. We can resolve this!

Looks like it was landed to master in: https://github.com/mozilla-b2g/gaia/commit/b9374ad2c4c407847b756c1f6dcb36461962183f

Comment 38

[Martijn Wargers (dead)]

Ok, I can see a "Dev" marketplace on trunk now on the Flame, thanks.

Comment 39

[Ryan VanderMeulen [:RyanVM]]

v2.1: https://github.com/mozilla-b2g/gaia/commit/9e79e1f4fd177bc2901b17644fc2174d5d0febe3
v2.0: https://github.com/mozilla-b2g/gaia/commit/31a49c7932c7085961760a6bef9ed381ea38d7e3

Comment 40

[Kai-Zhen Li [:seinlin][:kli]]

https://github.com/mozilla-b2g/gaia/commit/31a49c7932c7085961760a6bef9ed381ea38d7e3

Comment 41

[Coler]

Verify passed, this issue can't be repro on Woodduck 2.0;Flame2.0.

Attached: Verify_Woodduck_DEV.mp4
Reproducing rate: 0/5

Woodduck build:
Gaia-Rev        3a98f1287fa7b604891220ba5d86982ae8f9971e
Gecko-Rev       03d3ab62d5b07b915434f2d1d68495ad5915ecd2
Build-ID        20141120103003
Version         32.0

Flame2.0 build:
Gaia-Rev        1ede2666f1e6c1b3fd3b282011caf0cbc59544b0
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/54f1b0ee07a6
Build-ID        20141120000206
Version         32.0

Flame 2.1 build:
Gaia-Rev        f8d3bf44029e0afc0124600a4bb34dba8fc1ad21
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/f70a67a7f846
Build-ID        20141120001207
Version         34.0

FLame2.2 build:
Gaia-Rev        1abe09b4925547699dfdb2d358aed019137c3aa6
Gecko-Rev       https://hg.mozilla.org/mozilla-central/rev/6ce1b906c690
Build-ID        20141120040205
Version         36.0a1

Note:
On Flame 2.1&2.2,there are three Marketplace with same name,can't confirm whether containing DEV or not, please check it.

Comment 42

[Coler]

Attachment: Verify_Woodduck_DEV.mp4

Comment 43

[Hubert Lu[:hlu] <[e-mail]>]

(In reply to Coler from comment #41)
> Verify passed, this issue can't be repro on Woodduck 2.0;Flame2.0.
> 
> Attached: Verify_Woodduck_DEV.mp4
> Reproducing rate: 0/5
> 
> Woodduck build:
> Gaia-Rev        3a98f1287fa7b604891220ba5d86982ae8f9971e
> Gecko-Rev       03d3ab62d5b07b915434f2d1d68495ad5915ecd2
> Build-ID        20141120103003
> Version         32.0
> 
> Flame2.0 build:
> Gaia-Rev        1ede2666f1e6c1b3fd3b282011caf0cbc59544b0
> Gecko-Rev      
> https://hg.mozilla.org/releases/mozilla-b2g32_v2_0/rev/54f1b0ee07a6
> Build-ID        20141120000206
> Version         32.0
> 
> Flame 2.1 build:
> Gaia-Rev        f8d3bf44029e0afc0124600a4bb34dba8fc1ad21
> Gecko-Rev      
> https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/f70a67a7f846
> Build-ID        20141120001207
> Version         34.0
> 
> FLame2.2 build:
> Gaia-Rev        1abe09b4925547699dfdb2d358aed019137c3aa6
> Gecko-Rev       https://hg.mozilla.org/mozilla-central/rev/6ce1b906c690
> Build-ID        20141120040205
> Version         36.0a1
> 
> Note:
> On Flame 2.1&2.2,there are three Marketplace with same name,can't confirm
> whether containing DEV or not, please check it.

Hi Coler,
    You could check it via Firefox app manager. After I check it, Marketplace dev is installed on latest build.

Comment 44

[Coler]

Hi Hubert,
    OK,Thank you very much!