Closed Bug 1350032 Opened 3 years ago Closed 3 years ago

Google seeing duplicate requests to custom URIs that should only be requested once

Categories

(Core :: Networking, defect)

defect
Not set
normal

Tracking

()

RESOLVED DUPLICATE of bug 1349921

People

(Reporter: jduell.mcbugs, Unassigned)

References

Details

[not sure this belongs in DOM--but it's my guess.  Also, I'm not sure this needs to be Moco-confidential, but I haven't gotten permission from the reporter yet to make the report public]

We've gotten a report from Google about Firefox 51/52 making duplicate HTTP requests for certain ad-related URIs. May be a dupe of bug 1350005?

--
We've been seeing a large number of repeated/replayed GET requests coming from Firefox 52 that we don't see from previous releases. In all the cases we've seen so far, the requests are ones we would originally send from the src of a script or an img tag. These requests are generated on the fly during ad serving, and so are purposely unique and intended to fire only once.

We manged to somewhat blindly recreate this issue in a local Firefox 52, and were able to detect the replays happening live (although it required the use of Charles proxy 

<https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjF-9jDtNfSAhUHjlQKHe_ZDu8QFggaMAA&url=https%3A%2F%2Fwww.charlesproxy.com%2F&usg=AFQjCNEsxHYGeAoXTF7VGYSn0riAkfs9xw&sig2=Wb0m7QqQeu1n1J_Ld9OOXw&bvm=bv.149397726,d.eWE> 

 -- the requests were not logged in the Network DevTools).

The issue was triggered while browsing one of the pages that we've seen this type of unexpected extra traffic from (an eBay auction page <http://www.ebay.com/itm/SIT-N-CYCLE-RED-LOW-RESISTANCE-EXERCISE-BIKE-BY-SMOOTH-FITNESS-LOCAL-PICK-UP-/252801228952?hash=item3adc20a098:g:TF8AAOSwol5Yvezl>, the exact one doesn't appear to matter). Based on referrer headers, we believe the original GETs that are replayed are loaded inside the iframes at the very bottom of these pages. Most often, they're in the form of three 300x250 iframes loaded from http://ir.ebaystatic.com/ cr/v/c1/x-frame-3.html, although sometimes it's a single banner ad (.../x-frame-4.html). These frames contain only a small block of code to overwrite their own DOM via document.write, using the window.name property to provide the frame contents.

Following some amount of navigating across different auction pages (all generally have the same template, with these same iframes showing on most pages), some requests seem to get "stuck". Meaning that navigating or reloading any auction page causes certain stale requests to refire, verbatim. All URL parameters and headers are identical, with the exception of accept, which is sent as "text/html,application/xhtml+x ml,application/xml;q=0.9,*/*;q=0.8", even for requests which would have originally used "*/*" when loaded from the src of a script tag. The same is not true of "stuck" requests originating from img tags; they still repeat accept as "image/gif". 

We've seen this happen as many as three times for the same request, every time one of these pages is loaded. It only happens when navigating to an auction page. You can navigate away to another page or site, and no duplicate requests fire. Once you navigate back to an auction page, the original "stuck" requests can fire again. We've attached a Charles session where you can see this (the relevant requests are under https://ad.doubleclick.net => ddm => adj => N149404.149937VARICKMM). In this instance, there are three identical requests all fired within seconds of each other after an affected page is loaded. During each subsequent reload, these same three requests fired again and again (unfortunately not captured in this particular session log). The ads these requests refer to were not present during any of said page loads, and none of these requests were logged in Firefox's Network tab. But we did find them in our ad serving logs -- about 30 of this same request just from a few minutes spent navigating auctions.

Clearing Firefox state from the "Clear All History" menu stopped this behavior from occurring. We are not able to reliably reproduce the effect, although we have managed to trigger it multiple times.

Are you aware of any changes in Firefox 52 that might be relevant to this investigation? 

[from later emails] 

> For reference, the rate of duplicate requests is about 2-2.5% for Firefox
> 51. This matches the rate we see from Firefox 52 on Sept 19th, but we also
> have very little data for that date.
>
> On Sept 20th, when we have about 5x more requests from FF52 compared to the
> 19th, the duplicate rate jumps to 8.5%. It hovers between 6.5-9.5% from
> then until the beginning of November. On November 2nd, the rate hits 11.3%,
> and is pretty consistently over 10% after that, hitting 14% on November
> 14th. The rate for the past two days was around 13%.

This makes it sound like the regression may have happened earlier. We only updated the version number on nightly on September 19th, so it's possible the regression on nightly would have happened during the Firefox 51 (or earlier) timeframe.

Might the duplicate request numbers have jumped at some point for Firefox 51 during the period it was on our nightly channel (August 1
- September 19)?

-DBaron

[ another email ]

Ok yes, I actually see a 8% rate from Firefox 51 on September 19th and actually
20% back on September 16th, but unfortunately that's is as far back as these
logs go.
(Bug now public and Google contact--Jon Guarino--cc'd)
Group: mozilla-employee-confidential
Per comments in bug 1349921 it's possible that this is a regression from bug 1304387.
Blocks: 1304387
Component: DOM → Networking
(In reply to Liz Henry (:lizzard) (needinfo? me) from comment #3)
> Looks like a duplicate of bug 1349921.

This seems quite likely. Nick, can you confirm that it's caused by the same issue?
Flags: needinfo?(hurley)
Sorry for the lag here, yes, I believe this is a dupe of 1349921. If we see further issues with nightly (where an actual fix landed, instead of a pref-off), we should open a new bug, so I don't lose track of them.
Status: NEW → RESOLVED
Closed: 3 years ago
Flags: needinfo?(hurley)
Resolution: --- → DUPLICATE
Duplicate of bug: 1349921
You need to log in before you can comment on or make changes to this bug.