Open Bug 1520173 Opened 5 years ago Updated 2 years ago

Two separate entries are listed in History for the same website using HSTS

Categories

(Toolkit :: Places, defect, P3)

defect

Tracking

()

Tracking Status
firefox-esr68 --- wontfix
firefox-esr78 --- wontfix
firefox64 --- wontfix
firefox65 --- wontfix
firefox66 --- wontfix
firefox67 --- wontfix
firefox77 --- wontfix
firefox78 --- wontfix
firefox79 --- fix-optional
firefox80 --- fix-optional

People

(Reporter: cbaica, Unassigned)

References

Details

(Keywords: regression)

Attachments

(1 file)

Affected versions

  • Fx65.0b11
  • Fx66.0a1
  • Fx64.0RC

Affected platforms

  • Ubuntu 16.04 x32
  • Windows 10 x64
  • macOS 10.13

Steps to reproduce

  1. Launch Firefox with a fresh profile.
  2. Type 'youtube.com'and hit enter.
  3. After the page loads open the menu bar and click History.

Expected result

  • Only the start page and one youtube entry is displayed in the section.

Actual result

  • There are 2 entries for youtube in the history section.

Regression range
I will come back with a regression range ASAP.

Additional notes

  • This does not happen for all websites. When user goes to 'facebook.com' there is only 1 entry in the history section for Facebook.
  • Separate entries are listed when navigating to '9gag.com' and 'www.9gag.com' even though it is the same website.

(In reply to Cristian Baica [:cbaica], Release Desktop QA from comment #1)

Using moz-regression tool I ended up with the following pushlog url:

It is unlikely to be this - that's a blocklist update which shouldn't affect anything in play here.

I have also noticed that. For some reason, it pushed me off the mozilla central build repo and it reached autoland.
I ran the regression again and the pushlog for the first bad build is:

From that list, Bug 1486445 stands out for me as a possible cause for the issue, but I might be wrong.

Has Regression Range: --- → yes
Keywords: regression

I'm not sure the regression range makes sense considered that change only seems to affect service workers and fetch...

Maybe a few pages made changes server side for security reasons? (hsts or such).
We need a network log and eventually we'll have to debug in History.cpp to understand why it's not being detected as a redirect.

Component: Bookmarks & History → Places
Priority: -- → P3
Product: Firefox → Toolkit
Status: NEW → RESOLVED
Closed: 5 years ago
Resolution: --- → DUPLICATE

While the causes may be related, are we sure this is a duplicate? This bug is about Global History, not Session History.

Flags: needinfo?(jh+bugzilla)

You're right, it's not, sorry for the confusion.

Status: RESOLVED → REOPENED
Flags: needinfo?(jh+bugzilla)
Resolution: DUPLICATE → ---
Status: REOPENED → NEW

So, "youtube.com" is being typed in which I think ends up just being "http://youtube.com" which is then subject to an HSTS (HTTP Strict Transport Security) upgrade (which we preload). If I type in "youtube.com" into my nightly and make sure to prevent the awesomebar from auto-completing, I also end up with 2 entries in my history. One for "http://youtube.com/" and one for "https://www.youtube.com". I would suggest using the full "Show All History" UI to make sure we can see the full URIs at play.

I think this is unrelated, but once the site is loaded, we do expect it to perform ServiceWorker installation. The SW's install event invokes skipWaiting() and the activate event invokes clients.claim() which means the page will become controlled. The fetch event doesn't currently look to be handled, which means that we'll take our no-fetch optimization path, but after pursuing a very specialized HSTS upgrade path for ServiceWorkers. It's possible that this ends up looking different from an HSTS upgrade for a URL that's not covered by a ServiceWorker registration. But on a totally fresh profile, we shouldn't see the ServiceWorker code paths involved at all.

To elaborate on that, what I mean is that on first load of having typed "youtube.com", we expect the following chain of events

  • docshell is navigated to "http://youtube.com"
  • HttpChannelParent sees "http://youtube.com" request and does an HSTS upgrade, which triggers a redirect which goes down to the child.
  • HttpChannelChild request now made for "https://youtube.com" and that all goes to the parent
  • The "https://youtube.com/" page is fully loaded and begins ServiceWorker registration
  • ServiceWorker installs, activates, and claims the page, making it controlled.

On a future visit to the page, we expect:

  • docshell is navigated to "http://youtube.com" (although it's likely the user uses the awesomebar and goes directory to "https://youtube.com")
  • HttpChannelChild performs the specialized HSTS upgrade path in the child, where it notices that the page should be intercepted after a redirect.
  • Redirect happens.
  • The redirected channel is immediately interecepted by HttpChannelChild.
  • The ServiceWorker logic realizes there's no "fetch" handler and so resets the interception which now creates the HttpChannelParent.

Marco, could you please find someone to look into this for our 67 soft freeze (Mar 11)?

Flags: needinfo?(mak77)

I'll try to do some debugging tomorrow, but it doesn't look like a critical problem off-hand, it's also possible it's just a server side change related to hsts.

I just tried on Firefox 60 ESR, and it behaves the same, that makes me think this is not a regression, we just always acted like this.
http://youtube.com/ redirects (through HSTS) to https://youtube.com that redirects (through the channel) to https://www.youtube.com/.
History doesn't do deduplication because it misses information to tell if http://youtube.com/ and https://www.youtube.com/ are the same page (there's nothing enforcing that in specs).

I think the docshell doesn't notify properly history of the hsts redirect, indeed I see a VisitURI call with aFlags == 1, that means only VISIT_FLAGS::TOP_LEVEL is set, while I'd expect it to also have REDIRECT_SOURCE and REDIRECT_SOURCE_PERMANENT (or eventually a new flag, if necessary, but doesn't look like).
That info should be filled by nsDocShell::AddURIVisit.

Afaict, this has always been a miss, and considered more sites are starting using HSTS, we're likely to see this more often.
Because it's a longstanding problem, it doesn't apparently have an higher priority than the ongoing projects, it would surely be nice to fix it in the future, but it doesn't seem necessary to move resources to hit the 67 soft freeze.

Flags: needinfo?(mak77)
Keywords: regression
Summary: Two separate entries are listed in History for the same website → Two separate entries are listed in History for the same website using HSTS

The issue still occurs in Fx79.0b1. Updating flags.

Severity: normal → S3
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: