910779 - Remove mozmarket.js from zamboni

Status: RESOLVED FIXED

(Marketplace Graveyard :: Payments/Refunds, defect, P2)

Description

[Andy McKay]

Mozmarket.js hasn't been updated in ages, even though receipt verifier has (and split out into two files). We should update mozmarket.js and ensure it copes with the changes in bug 902946.

Comment 1

[Andy McKay]

Based on @kumars comments in IRC, we don't really maintain mozmarket anymore. So thinking this should be removed.

There is MDN documentation recommending using this for testing, so we should check where this is used before removing it:

https://developer.mozilla.org/en-US/docs/Tools/Firefox_OS_Simulator/Simulator_Walkthrough

Comment 2

[David Larlet [:davidbgk]]

The simulator documentation only uses the receiptverifier lib https://github.com/mozilla/receiptverifier
which uses the mozmarket "namespace" but doesn't require the lib in itself as far as I understand it(?)

Comment 3

[David Larlet [:davidbgk]]

PRs: https://github.com/mozilla/zamboni/pull/1113 & https://github.com/mozilla/zamboni-lib/pull/7

Comment 4

[Andy McKay]

Looks like we've got quite a few apps using that. 

Removing that would cause those apps to break. So we've either got to get deprecation plan and work with dev evangelism on that. Or we bite the bullet and keep updating mozmarket.js so that its up to date.

Comment 5

[Andy McKay]

http://jason.pastebin.mozilla.org/3020193

Is a list of referers over the last few days. I think we need to deprecate this before removing.

Comment 6

[David Larlet [:davidbgk]]

To be discussed before removal considering https://bugzilla.mozilla.org/show_bug.cgi?id=956358

Comment 7

[Andy McKay]

Jason, could you do a grep in the access logs for referers to mozmarket.js please over the last 30 days or so? We'd like to see how many apps are using it.

Comment 8

[Jason Thomas [:jason]]

These are the referrers I see in the logs for the last 30 days:

"http://bits.wikimedia.org/WikipediaMobileFirefoxOS/"
"https://bits.wikimedia.org/WikipediaMobileFirefoxOS/"
"https://bits.wikimedia.org/WikipediaMobileFirefoxOS/index.html"

Comment 9

[Adam Baso]

https://gerrit.wikimedia.org/r/#/c/107755/ has been submitted for review in response to this BZ bug and the email thread http://lists.wikimedia.org/pipermail/mobile-l/2014-January/006417.html.

Comment 10

[Adam Baso]

https://gerrit.wikimedia.org/r/#/c/107755/ has been merged and is slated for deployment, probably in the next week or two.

Comment 11

[Andy McKay]

Thanks for doing that.

Comment 12

[Kumar McMillan [:kumar]]

Hi Adam. If I read the diff right, your patch just removes mozmarket.js but does not replace it. If you weren't using it then that's fine but if you were using it, you can replace it with this: https://github.com/mozilla/receiptverifier

Comment 13

[Adam Baso]

(In reply to Kumar McMillan [:kumar] from comment #12)
> Hi Adam. If I read the diff right, your patch just removes mozmarket.js but
> does not replace it. If you weren't using it then that's fine but if you
> were using it, you can replace it with this:
> https://github.com/mozilla/receiptverifier

Kumar, thanks for the note. We do not plan to process payments or restrict installation, so the plan is to not replace the old stuff with the receiptverifier.

Comment 14

[Adam Baso]

The new version has been loaded to the server, and pending a Varnish cache flush, the changes should be reflected in the app. It appears that the updated JavaScript file, https://bits.wikimedia.org/WikipediaMobileFirefoxOS/js/app.js, has already been flushed and the correct version is showing.

(A cache flush is pending for the assets overall to ensure things like https://bits.wikimedia.org/WikipediaMobileFirefoxOS/css/wikiapp.css reflect the latest version across all cache servers - see https://bits.wikimedia.org/WikipediaMobileFirefoxOS/css/wikiapp.css?nocache to see the diff for the .spinner class - the ?nocache version being the updated one.)

Comment 15

[Andy McKay]

Jason, could you do a quick access log grep on mozmarket.js please and tell us if anyone has been using it over the last 30 days please?

Comment 16

[Jason Thomas [:jason]]

[[e-mail]3 logs]# zgrep 'mozmarket.js' web*.mktweb.services.phx1.mozilla.com/marketplace.firefox.com/* | awk '{print $11}' | sort | uniq
"-"
"http://bits.wikimedia.org/WikipediaMobileFirefoxOS/"
"http://ming-inside.github.io/truck-loader-4/"
"https://bits.wikimedia.org/WikipediaMobileFirefoxOS/"
"https://bits.wikimedia.org/WikipediaMobileFirefoxOS/index.html"

Comment 17

[Jason Thomas [:jason]]

I see http://bits.wikimedia.org/WikipediaMobileFirefoxOS/ in yesterday's logs. http://ming-inside.github.io/truck-loader-4/ last seen on 2014-03-25.

Comment 18

[Adam Baso]

Is it possible this is coming from an older (non-upgraded) version of the Wikipedia app? Also, comparatively speaking, how many hits to the affected endpoint are coming in with that Referer relative to a couple months ago?

If it's related to stale installations, I'm not sure how to deal with that?

Comment 19

[Jason Thomas [:jason]]

I think it is possible this is due to stale installations. 

Here are the last 3 months of requests made to /mozmarket.js. Looks like there is a slow decline.

Top 10 URLs By Hits Mar 01, 2014 - May 28, 2014 GMT
URL 	EDGE HITS 	EDGE VOLUME(MB)
marketplace-static.addons.mozilla.net/mozmarket.js 	36252 	1200.33


Top 10 URLs Trends By All Hits Mar 01, 2014 - May 28, 2014 GMT
URL 	Total 	03/01 	03/02 	03/03 	03/04 	03/05 	03/06 	03/07 	03/08 	03/09 
03/10 	03/11 	03/12 	03/13 	03/14 	03/15 	03/16 	03/17 	03/18 	03/19 	03/20 
03/21 	03/22 	03/23 	03/24 	03/25 	03/26 	03/27 	03/28 	03/29 	03/30 	03/31 
04/01 	04/02 	04/03 	04/04 	04/05 	04/06 	04/07 	04/08 	04/09 	04/10 	04/11 
04/12 	04/13 	04/14 	04/15 	04/16 	04/17 	04/18 	04/19 	04/20 	04/21 	04/22 
04/23 	04/24 	04/25 	04/26 	04/27 	04/28 	04/29 	04/30 	05/01 	05/02 	05/03 
05/04 	05/05 	05/06 	05/07 	05/08 	05/09 	05/10 	05/11 	05/12 	05/13 	05/14 
05/15 	05/16 	05/17 	05/18 	05/19 	05/20 	05/21 	05/22 	05/23 	05/24 	05/25 
05/26 	05/27 	05/28
marketplace-static.addons.mozilla.net/mozmarket.js 	36252 	539 	538 	490 
484 	557 	541 	514 	571 	536 	518 	485 	478 	491 	449 	553 	502 	501 	507 
488 	456 	487 	473 	496 	532 	452 	456 	413 	430 	422 	444 	405 	396 	436 
426 	452 	404 	400 	423 	424 	395 	366 	394 	374 	355 	390 	377 	412 	376 
376 	366 	388 	389 	444 	391 	367 	370 	363 	400 	379 	393 	366 	401 	336 
348 	367 	295 	341 	362 	355 	343 	341 	307 	319 	320 	287 	380 	327 	323 
313 	336 	331 	323 	312 	367 	320 	332 	357 	326 	183

 Here are the user agent for devices that requested /mozmarjet.js on 2014-05-28.

      1 Mozilla/5.0 (Android; Mobile; rv:27.0) Gecko/27.0 Firefox/27.0
      1 Mozilla/5.0 (Mobile; ALCATELOneTouch4012A; rv:18.1) Gecko/18.1 Firefox/18.1
      1 Mozilla/5.0 (Mobile; ALCATEL ONE TOUCH 4012A; rv:18.1) Gecko/18.1 Firefox/18.1
      1 Mozilla/5.0 (Mobile; ALCATELOneTouch4012X/SVN 01012P; rv:18.1) Gecko/18.1 Firefox/18.1
     10 Mozilla/5.0 (Mobile; LG-D300; rv:18.1) Gecko/18.1 Firefox/18.1
     31 Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0
     57 Mozilla/5.0 (Mobile; ZTEOPEN; rv:18.1) Gecko/18.1 Firefox/18.1

Comment 21

[Andy McKay]

I don't want to increase your context switching, but this is causing over 23k 500's to be sent to sentry. Would be nice to get this fixed for this Fridays tag.

Comment 22

[Jared Lockhart [:jlockhart]]

Merged in https://github.com/mozilla/zamboni/pull/2592