1430147 - Img File has Millions of GA Server Calls

Status: RESOLVED FIXED

(www.mozilla.org :: Analytics, defect)

Description

[pgerman]

On November 21st, GA started logging page views for /media/img/firefox/templat/page-image-quantum.4b108ed0b8d8.png

What do we know about this traffic?
- Between Nov 22, and Nov 29, GA was logging between 250,000 and 650,000 per day
- Between Nov 30 and Dec 22, GA was logging roughly 3-5 Million pageviews per day
- Between Dec 22 to present, GA is logging roughly 2.5-3 Million pageviews per day
- Traffic is >95% direct
- Traffic is from a wide variety of countries
- Traffic is from 99% users in firefox

I'm having a difficult time reproducing the pageviews that i'm seeing in the reporting UI to https://www.mozilla.org/media/img/firefox/template/page-image-quantum.4b108ed0b8d8.png

Any thoughts on what might be happening?

Comment 1

[Alex Gibson [:agibson]]

Attachment: new-tab-page-image-404.png

So, the most interesting bit of this is that 99% of users are in Firefox. Given that it's also much a hugh volume of traffic, this led me to suspect something that every Firefox user sees a lot: the new-tab page.

Testing on one of my profiles, I can see that the new-tab page renders our 404 page instead of a page-image (see attached), because I'm assuming the data for the new tab image was saved to my machine locally before the page image URL changed.

Comment 2

[Alex Gibson [:agibson]]

Peter, is there anyone we could speak to from product who could help validate this theory?

Comment 3

[Mike Kaply [:mkaply]]

The reason this is happening is because on first run, you get this page:

https://www.mozilla.org/en-US/firefox/57.0.4/firstrun/

In this page, the og data looks like this:

    <meta property="og:type" content="website">
    <meta property="og:site_name" content="Mozilla">
    <meta property="og:locale" content="en_US">
    <meta property="og:url" content="https://www.mozilla.org/en-US/firefox/">
    <meta property="og:image" content="https://www.mozilla.org/media/img/firefox/template/page-image.4b108ed0b8d8.png">

So the image https://www.mozilla.org/media/img/firefox/template/page-image.4b108ed0b8d8.png is being used as a thumbnail.

This pages gets added to highlights for new users. And then that image is asked for again when the new tab is opened (depending on caching).

Comment 4

[Alex Gibson [:agibson]]

> This pages gets added to highlights for new users. And then that image is asked for again when the new tab is opened (depending on caching).

If this hashed filename is added to highlights, how long is it cached for? Also, how frequently are requests made when users open the new-tab page again? Is the URL ever updated at any point?

Comment 5

[Mike Kaply [:mkaply]]

Christian on my team should be able to answer that. He worked on some of this.

Comment 6

[Mike Kaply [:mkaply]]

Also side note, we should figure out what page has https://www.mozilla.org/media/img/firefox/template/page-image-quantum.4b108ed0b8d8.png specifically as the og:image. I'm seeing https://www.mozilla.org/media/img/firefox/template/page-image.4b108ed0b8d8.png. (Although they are the same image)

Comment 7

[Liz Henry (:lizzard) (relman/hg->git project)]

Tracking for 57 and other channels. Are beta and nightly also affected?

Ritu, FYI.

Comment 8

[Liz Henry (:lizzard) (relman/hg->git project)]

Seems like something that may affect 58 on release, n-i to Gerry as well.

Comment 9

[Alex Gibson [:agibson]]

(In reply to Mike Kaply [:mkaply] from comment #6)
> Also side note, we should figure out what page has
> https://www.mozilla.org/media/img/firefox/template/page-image-quantum.
> 4b108ed0b8d8.png specifically as the og:image. I'm seeing
> https://www.mozilla.org/media/img/firefox/template/page-image.4b108ed0b8d8.
> png. (Although they are the same image)

The first image no longer exists, we renamed it with an updated hash to the second image (something that could easily happen again in the future). This issue seems to have arisen because the new-tab page is rendering an image of our 404 page as the thumbnail instead, which is how we noticed it in GA.

Comment 10

[pgerman]

For the time being, I'm going to build a suppression rule for /media/img/ in GTM so that we stop these requests from being triggered by the tag manager.

Comment 11

[Alex Gibson [:agibson]]

We're also going to see if we can restore the original URL via a redirector to the new page image. This will plug the leak for now, but we'll likely face the same issue next time we update an image (unless activity stream finds a fix to help prevent this)

Comment 12

[GitHub Bugzilla PR Linker]

Attachment: Link to GitHub pull-request: https://github.com/mozilla/bedrock/pull/5390

Comment 13

[[github robot]]

Commit pushed to master at https://github.com/mozilla/bedrock

https://github.com/mozilla/bedrock/commit/e26de23f672d0a4cdd89899ee581bb5795fc4e83
Bug 1430147: Restore page-image-quantum.png

Add a symlink to page-image.png, and since it's the same
file as page-image-quantum.png, the hashed name will be
the same as it was originally, thus continuing to serve
the image at the old URL.

Comment 14

[Christian Sadilek [:csadilek]]

Clearing my n-i. This is best discussed with the Activity Stream team.

Comment 15

[Mike Kaply [:mkaply]]

So is there anything product wise to do with this at this team? Has the Activity Stream team responded?

Comment 16

[Tim Spurway [:tspurway]]

In 57, Activity Stream created a Highlight for the mozilla.org onboarding page. Every user would wind up with a highlight of the onboarding experience.

This was changed in 58 so that particular page no longer creates a highlight.  This should be easy to verify by creating a new profile and seeing if a highlight tile is created in 58 or 59.

Also, your GA queries should *not* show Firefox version 58 for the user agent (if you can query browser version in GA).

Comment 17

[Tim Spurway [:tspurway]]

Here's where this bug was fixed for 58:

https://bugzilla.mozilla.org/show_bug.cgi?id=1409661

Comment 18

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

Thanks Tim. Based on Tim's reply 58/59 is unaffected.

Hi Andrei, can we get your team to test whether this is a concern for 58 or not?

Comment 19

[Iulia Cristescu, QA [:JuliaC] (away, please needinfo? [e-mail])]

(In reply to Ritu Kothari (:ritu) from comment #18)
> Thanks Tim. Based on Tim's reply 58/59 is unaffected.
> 
> Hi Andrei, can we get your team to test whether this is a concern for 58 or
> not?

Hello, Ritu! I can confirm that in 58 and 59 Firefox doesn't create a highlight for the firstrun page anymore (as already mentioned in comment 16). 
I also tried to figure out what is the root problem and what implications has this fix (bug 1409661) for it. Asked Tim's opinion and it seems that, per AS team, 58 and 59 are OK as far as newtab page is concerned for this issue. 
In rest, all I managed to gather is that this may be a more complex issue and it should be determined if removing /firstrun from being included in highlights really fixes the underlying bug.

Comment 20

[Andrei Oprea [:andreio]]

Similar to bug 1424987 this has been fixed by bug 1413550 (registering an expiration filter for highlight card images) not bug 1409661.