1311807 - Some images and media players delayed for minutes

Status: RESOLVED FIXED

(Core :: DOM: Security, defect, P2)

Description

[laugeo]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0
Build ID: 20161020030211

Steps to reproduce:

On https://www.franceinter.fr/ page, click headphone icone to start audio player
This opens audio player in the bottom , 


Actual results:

 audio playing won't start before several minutes


Expected results:

audio playing starts within 1 or 2 seconds

Comment 1

[laugeo]

mozregression gives this result:

016-10-20T21:32:32: INFO : application_version: 52.0a1
2016-10-20T21:32:32: INFO : platform_buildid: 20160927125019
2016-10-20T21:32:32: INFO : platform_changeset: 689f6fee184628d3dd968e3b6518cc811cfb3e43
2016-10-20T21:32:32: INFO : platform_repository: https://hg.mozilla.org/integration/mozilla-inbound
2016-10-20T21:32:32: INFO : platform_version: 52.0a1
2016-10-20T21:32:33: INFO : ATTENTION: default value of option force_s3tc_enable overridden by environment.
2016-10-20T21:32:33: INFO : ATTENTION: default value of option force_s3tc_enable overridden by environment.
2016-10-20T21:32:34: INFO : ATTENTION: default value of option force_s3tc_enable overridden by environment.
2016-10-20T21:33:05: INFO : Narrowed inbound regression window from [9dfac0d1, e1b0aaa2] (4 revisions) to [689f6fee, e1b0aaa2] (2 revisions) (~1 steps left)
2016-10-20T21:33:05: DEBUG : Starting merge handling...
2016-10-20T21:33:05: DEBUG : Using url: https://hg.mozilla.org/integration/mozilla-inbound/json-pushes?changeset=e1b0aaa20f1098b08537b2e13f9e97ee9759d4c7&full=1
2016-10-20T21:33:06: DEBUG : Found commit message:
Bug 1302414 - Theme preview is not working. r=sebastian

2016-10-20T21:33:06: INFO : The bisection is done.
2016-10-20T21:33:06: INFO : Stopped

Comment 2

[laugeo]

ok with chrome

Comment 3

[laugeo]

ok with FF nightly before 2016/09/27 as shown by mozregression

Comment 4

[Alice0775 White]

[Tracking Requested - why for this release]:  performance regression on the certain site

I can reproduce on Windows10 home 64bit Nightly52.01 32bit.


Regression window:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=689f6fee184628d3dd968e3b6518cc811cfb3e43&tochange=e1b0aaa20f1098b08537b2e13f9e97ee9759d4c7

Suspect: 380eebfd9d89	Kate McKinley — Bug 1246540 - HSTS Priming Proof of Concept. r=ckerschb, r=mayhemer, r=jld, r=smaug, r=dkeeler, r=jmaher, p=ally


Setting security.mixed_content.send_hsts_priming=false seems to fix the issue.

Comment 5

[laugeo]

Setting security.mixed_content.send_hsts_priming=false  is ok for me too

Comment 6

[Kate McKinley [:kmckinley, :Kate]]

The audio player loads the mp3 content over HTTP, which HSTS priming attempts to upgrade to HTTPS. When it makes the HSTS priming request to https://direct.franceinter.fr/live/franceinter-midfi.mp3, the HTTPS connection times out. Then it requests HTTP, which results in a 302 redirect to http://aifae8cah8.lb.vip.cdn.dvmr.fr/franceinter-midfi.mp3. Since HSTS priming has not seen this new domain, a new priming request is sent, which also times out. Then the stream starts to play.

Bug 1310955 fixed an issue where the cached result was not being properly read, and a new request was sent each time. In the current macOS Nightly, the first time I load the page it takes about a minute to load the audio, but subsequent loads are nearly instantaneous.

Timeouts like this can be fixed by the site owner by one (or more) of the following:

* Migrate the content to HTTPS and use HSTS
* Change the firewall to not drop packets to unused ports, but instead send RST (nothing listening)
* Have an HTTPS server listening which returns 404

Comment 7

[Alice0775 White]

Edge and Chrome Dev55.0.2883.11 on Win10 are working properly(fast).

Comment 8

[Alice0775 White]

(In reply to Kate McKinley [:kmckinley] from comment #6)

> but subsequent loads are nearly instantaneous.

I can still reproduce the slow connection the subsequent load.

Steps
1. Open https://www.franceinter.fr/ and Click headphone icon
   --- Toooooooo slow
2. Open https://www.franceinter.fr/ again and Click headphone icon
   --- Still Toooooooo slow

Comment 9

[Kate McKinley [:kmckinley, :Kate]]

This fix landed on Nightly on the 20th, so it will depend on whether a new build is up. With a macOS build I downloaded just prior to my comment, the behavior was correct.

Comment 10

[Alice0775 White]

(In reply to Kate McKinley [:kmckinley] from comment #9)
> This fix landed on Nightly on the 20th, so it will depend on whether a new
> build is up. With a macOS build I downloaded just prior to my comment, the
> behavior was correct.

Confirmed on Windows10 build(m-c f0f1aaf051d6798e1e73d1feee07ca847333167a). 
The first loading is still extremely slow.
However, the subsequent load is normal.

Comment 11

[Alice0775 White]

Chrome and Edge is fast, even if it is first loaded.

Comment 12

[laugeo]

It is ok for me (including 1st load)  since this morning with build 20161021030210  
( "security.mixed_content.send_hsts_priming" back  to true of course)

Comment 13

[Alice0775 White]

(In reply to laugeo from comment #12)
> It is ok for me (including 1st load)  since this morning with build
> 20161021030210  
> ( "security.mixed_content.send_hsts_priming" back  to true of course)

The first load means using new profile or profile that expired the HSTS cache.

Comment 14

[laugeo]

Ok, trying with new profile,  problem remain .

Comment 15

[Milan Sreckovic [:milan] (needinfo for best results)]

Kate, does it make sense to back this out of 51 while we're sorting things out?  Bug 1246540 comment 145 is where the request for the uplift was made, but it looks like we're running into some problems.

And by "backing out", I mean flip the preference, if that's enough to stop this regression in 51.

Comment 17

[Loic]

Another example with this video embedded in the page:
http://www.geenstijl.nl/mt/archieven/2016/10/geweldsmonopolie.html

Comment 18

[Kate McKinley [:kmckinley, :Kate]]

It is fine to turn security.mixed_content.send_hsts_priming off in Release. Keeping it on in Nightly/Aurora will help us to identify these types of issues and how widespread they are.

As noted in comment 6, this is ultimately something site owners can and should fix.

Comment 19

[Marcia Knous [:marcia]]

Tracking 52+ so we can keep an eye on the scope of how many sites might be affected.

Comment 20

[Gerry Chang [:gchang]]

Track 51+ as perf regression.

Comment 21

[Gerry Chang [:gchang]]

Hi :kmckinley,
per comment #18, does that mean the issue will be disappeared when 51 enters beta?

Comment 22

[Kate McKinley [:kmckinley, :Kate]]

We will still send the priming request in beta and release. Since the priming request timing out is the root cause, setting security.mixed_content.send_hsts_priming is set to false in all.js for beta/release will make this will go away.

Bug 1313595 will reduce the timeout to default 3s, and the timeout will be adjustable.

Comment 23

[Milan Sreckovic [:milan] (needinfo for best results)]

Patch up for review in bug 1313595

Comment 24

[Milan Sreckovic [:milan] (needinfo for best results)]

Because things landed around the train changes, can we be explicit about where bug 1313595 landed - it looks  it's currently at 52+53, and it should go to 51 as well.
At which point, we should be able to close this bug, or wontfix it for 51.

Comment 25

[Kate McKinley [:kmckinley, :Kate]]

If possible, I would like to get bug 1313595 uplifted, but if not, we can turn it off via pref. I want to make sure no new problems are cropping up, and there are a few tests that have started to orange, I need to make sure they are not broken. Sometimes the priming request shows up in a test as an unexpected request, even though it has no impact on the result.

Comment 27

[Milan Sreckovic [:milan] (needinfo for best results)]

This feature is disabled in release, which means that we won't have the problem as 51 hits release, and the problem (see bug 1313595) is fixed in 52.

Comment 28

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

patch from bug #1313595 was backed out,
what's more it still adds delay (like 3s),
so it's not perfect fix, but workaround

Comment 29

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

[Tracking Requested - why for this release]: Regression

Comment 30

[Julien Cristau [:jcristau]]

Kate, as we're getting close to merge day is the plan still to fix this for 52 (via bug 1313595?), or are we going to pref hsts priming off?  It seems like it would be better to do something about this bug before we go to beta.  Thanks.

Comment 31

[Kate McKinley [:kmckinley, :Kate]]

I am hopeful that 1313595 can be landed since the rework is completed, but until it is, I think hsts priming should be pref'd off.

Comment 32

[Ryan VanderMeulen [:RyanVM]]

I talked to Kate about this and HSTS priming isn't going to be enabled on 52 past Aurora, so this will become a non-issue for that release.

Comment 33

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

[Tracking Requested - why for this release]: Regression

Unfortunately this is still enabled in release and it's still the issue, same as bug #1334074.

Comment 34

[Julien Cristau [:jcristau]]

Confirmed the issue with the steps from comment 0, in 52.0b1.  It seems we still need to pref off security.mixed_content.send_hsts_priming?

Comment 35

[Kate McKinley [:kmckinley, :Kate]]

Bug 1335134 has been created to disable it in the codebase, and bug 1335224 has been created to create a go-faster addon to disable the pref.

Comment 36

[Julien Cristau [:jcristau]]

Disabled on release (via system add-on) and beta (as of 52.0b3).

Comment 37

[Nathan Froyd [:froydnj]]

Priming is disabled on release, marking 54 as such.

Comment 38

[Liz Henry (:lizzard) (relman/hg->git project)]

Are we still planning to ship with this disabled in 55 and 56?

Comment 39

[Kate McKinley [:kmckinley, :Kate]]

This will still be disabled on Release in 55 and 56. It will be enabled in Beta 56 for data gathering. However, the underlying problem causing the delay has been fixed, so we should be able to close this.

Comment 40

[laugeo]

The problem was  fixed ,  but re-appeared some days ago  . 
Try the audio player on https://www.franceinter.fr/

Comment 41

[Liz Henry (:lizzard) (relman/hg->git project)]

laugeo, what version of Firefox are you using, where you see the issue?

Comment 42

[laugeo]

Problem fixed using today  Nightly ( August 4 )

Comment 43

[Liz Henry (:lizzard) (relman/hg->git project)]

Kate, want to close this now?

Comment 44

[Kate McKinley [:kmckinley, :Kate]]

I think we can close this.

Comment 45

[Liz Henry (:lizzard) (relman/hg->git project)]

Noting this was disabled in 56 in bug 1385035 (after EARLY_BETA_OR_EARLIER).