1083528 - Ship engineer builds with right dev/stage builds

Status: VERIFIED FIXED

(Firefox OS Graveyard :: Gaia, defect)

Description

[krupa raj[:krupa]]

Engineering builds ship with marketplace dev/stage apps installed by default.

However, these apps are really really old and updating them doesn't work. These apps cannot be deleted which means we are pretty much being forced to use user builds. This also blocks our automation efforts using marionette.

We need to uplift the right dev/stage versions to all gaia eng versions (2.2 to 1.1)

verified on:
Platform version: 36.0a1
Build Identifier: 2014105040201
Git commit info: 2014-10-14 16:11:51 5f1f0960

what needs to happen?
We need to have eng builds shipping with the latest builds from
dev: https://marketplace.firefox.com/app/dev
stage: https://marketplace.firefox.com/app/marketplace-6

This needs to be uplifted to older versions of gaia since the marketplace team tests on 1.1, 1.3, 1.4, 2.0 and 2.1

I'm marking this a blocker since it is blocking our QA.

Comment 1

[Wil Clouser [:clouserw]]

Why are your dev and stage links on production?  The dev and stage apps are automatically updated on the dev and stage sites...

https://marketplace-dev.allizom.org/app/marketplace
https://marketplace.allizom.org/app/marketplace

Comment 2

[krupa raj[:krupa]]

(In reply to Wil Clouser [:clouserw] from comment #1)
> Why are your dev and stage links on production?  The dev and stage apps are
> automatically updated on the dev and stage sites...
> 
> https://marketplace-dev.allizom.org/app/marketplace
> https://marketplace.allizom.org/app/marketplace

https://bugzilla.mozilla.org/show_bug.cgi?id=1059546 means that on gaia versions older than 2.2 will require the pref change. That is why I linked to the prod versions. Being iframed, it is not a big deal.

Comment 3

[krupa raj[:krupa]]

When this patch lands we need a way to uplift it to older versions of gaia eng builds. Any ideas on how to update the dev_apps cleanly on older versions of gaia? Thanks!

Comment 4

[Wil Clouser [:clouserw]]

Final URLs:

https://marketplace.firefox.com/app/mkt-dev/
https://marketplace.firefox.com/app/mkt-stage/

And here is a pull request putting them in Gaia's master:

https://github.com/mozilla-b2g/gaia/pull/25231

I'd like to see if the tests pass, but I think it also needs review from someone who either knows manifests or can actually build gaia and test it.  I changed the origins and installOrigins and I'm not sure if that's how they should look in the metadata file.

Comment 5

[Wil Clouser [:clouserw]]

Oh, the PR also makes the packages removable

Comment 6

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

I pinged Fabrice in IRC about getting a patch uplifted to ancient branches like 1.3 and 1.4; he suggested simply landing the patch & requesting uplift.

So, I'll try to apply this patch on master, as a starting point, and we'll figure out who the right reviewer should be.

For future reference, Wil, the process for attaching a github gaia pull request to bugzilla is documented at http://globau.wordpress.com/2013/10/21/github-pull-requests-and-bugzilla/

Comment 7

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Attachment: Github PR 25231 by :clouserw

Comment 8

[Jared Hirsch [:jhirsch] (he/him) (Needinfo please)]

Flashing onto device, I see:

> System JS : ERROR resource://gre/modules/commonjs/toolkit/loader.js -> file:///Users/jhirsch/codez/github/mozilla-b2g-gaia/build/webapp-manifests.js:194 - Error: We got 2 manifest errors while building:
> * External webapp `marketplace-dev.allizom.org` can not have origin in metadata because is packaged
> * External webapp `marketplace.allizom.org` can not have origin in metadata because is packaged

I also see a ton of broken tests due to the same error: https://treeherder.mozilla.org/ui/#/jobs?repo=gaia-try&revision=1e13b06793f9

Maybe tweak the manifest a bit and try flashing on device locally, then resubmit when it seems to be working.

Comment 9

[Wil Clouser [:clouserw]]

Comment on attachment 8506308 [details] [review]
Github PR 25231 by :clouserw

Looking for r?fabrice.  Latest patch is green across the board except it has two timeouts on Gij: https://treeherder.mozilla.org/ui/#/jobs?repo=gaia-try&revision=d9f3c2dd0933

Not sure if there is more I can do?

Comment 10

[Caitlin Galimidi [:c8o]]

Tony

Who in Gaia needs to look at this so we can land? We're looking to eliminate this 40-60 minutes of downtime every day for testers and devs in our ecosystem. I know some folks can flash and configure new bugs super fast (30 min), but many of our contributing devs and testers do not have the same prowess and skill. 

This will greatly help them.
Who can help us land / move forward?
Lots of testing right now in Firefox Accounts, Marketplace and Apps with our Tier 1 partners.

Comment 11

[Tony Chung [:tchung]]

I'm assuming you're asking who will approve uplifting?  if so:
- trunk (2.2 now), doesnt require any approval.  just land on m-c
- (2.0 and 2.1), request blocking 2.0? and set flag: status-b2g-v2.1:=? and status-b2g-v2.0=?.   ask for approval
- (1.4 and behind), i'm not sure.  you'll have to loop in all the different TAMs on eahch release because they manage all the partner expectations since it's their branch.  But given these are already production releases, why do you need even want these stage and dev apps to go into production?

Comment 12

[Caitlin Galimidi [:c8o]]

krupa - 
can you address Tony's second question?
my assumption is that because 70% of our users are still on 1.1 we are still actively testing there.

Comment 13

[krupa raj[:krupa]]

Tony,

Since marketplace is an iframed app right now, whatever changes we push every week goes to all of our existing users. This means that it is paramount that we don't break functionality on older versions of gaia given that most of our real world users are on 1.1 or 1.3. This will still be true when we move to a fully packaged app since marketplace can update itself.

Having 1.1, 1.3 and 1.4 engineer builds with dev, stage installed would be awesome. However, I'd at least like to get them shipping on 2.0+ builds asap.

Comment 14

[Wil Clouser [:clouserw]]

The patch above is for master.  I'm asking fabrice for approval because the tests are flaky (Gij failed 66% of the time above).  He's familiar with the situation so he'll know what to do if it lands and breaks tests again even though I don't think it will.

For older versions we'll need to make new patches for each version, which we can do, but I wanted to get this landed on master first.

Comment 15

[[:fabrice] Fabrice Desré]

https://github.com/mozilla-b2g/gaia/commit/3536e6038212f9c5dc08a763ed6555494f24e38b

Comment 16

[Bob Silverberg [:bsilverberg]]

I hate to be the bearer of bad news, but I just flashed to the latest m-c engineering build on my Flame and I am still having the issue where, when I start Marketplace-dev it just remains on the splash screen (with the shopping bag and loading dots) forever. The same thing happens with Marketplace-stage, ftr, but Marketplace-prod works fine.

Comment 17

[Wil Clouser [:clouserw]]

Is there more for me to do in this bug, Krupa?

Comment 18

[krupa raj[:krupa]]

(In reply to Wil Clouser [:clouserw] from comment #17)
> Is there more for me to do in this bug, Krupa?

Clouser, we need to get your patch uplifted to older versions of gaia. I will check why the patch isn't working on central tomorrow.

Comment 19

[Wil Clouser [:clouserw]]

Ok, happy to do that tomorrow.  Would you let me know which versions you want?  Thanks.  Or ... let's figure out the problem on central first and then I'll make patches.

Comment 21

[Wil Clouser [:clouserw]]

Krupa: let me know if there is more for me to do here

Comment 22

[krupa raj[:krupa]]

(In reply to Wil Clouser [:clouserw] from comment #21)
> Krupa: let me know if there is more for me to do here


Clouser, we still need to get your patch uplifted to older versions of gaia.

Comment 23

[krupa raj[:krupa]]

To clarify, we want the package currently on master to be also on 2.1, 2.0, and 1.4. Thanks!

Comment 24

[Wil Clouser [:clouserw]]

Attachment: Add to 1.4

Comment 25

[Wil Clouser [:clouserw]]

Attachment: Add to 2.0

Comment 26

[Wil Clouser [:clouserw]]

Attachment: Add to 2.1

Comment 27

[Wil Clouser [:clouserw]]

2.0 had one failed test but subsequent re-tests show it passing so that patch seems good.  The other two have more failures than I've seen.  Judging from the lack of output, maybe the testing is disabled and they simply are failing to start?  We are talking about older versions so that would be reasonable.

Comment 28

[Wil Clouser [:clouserw]]

https://github.com/mozilla-b2g/gaia/commit/22c80a708329321a2fdeed4ece019498c0cec90d
https://github.com/mozilla-b2g/gaia/commit/124c2f85f1b956e9e8429dab5121de702a3bc197
https://github.com/mozilla-b2g/gaia/commit/3283875f151f9982bf9d40e10d5f5bc0e52bde76

thanks

Comment 29

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

Setting the tracking flags so this gets out of the needs-uplift list. Please correct this if I misread the status here.

Comment 30

[krupa raj[:krupa]]

2.0 now has the latest marketplace builds. Thanks everyone!

Comment 31

[Kai-Chih Hu [:khu]]

Looks like 2.2 includes this as well.