1178720 - Weather.com appears to use old cache data for it's forecasts

Status: RESOLVED DUPLICATE of bug 1198747

(Core :: Networking: Cache, defect)

Description

[Gary [:streetwolf52]]

User Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:42.0) Gecko/20100101 Firefox/42.0
Build ID: 20150629193017

Steps to reproduce:

1. Make sure you are using a disk cache.

2. Go to http://www.weather.com/weather/hourbyhour/l/07746:4:US


Actual results:

Keep going to the site until you notice that the forecast times are not current.  This might take 1/2 hour or more to see.  Forecasts update every 15 minutes. Don't clear the cache as this seems to 'fix' the problem for the moment. Forcing a reload from the site does not work.

Problem does not appear if in Private Mode or if running without a disk cache.


Expected results:

Forecast should update approximately every 15 minutes.  IE11 appears to work OK.

Comment 1

[Gary [:streetwolf52]]

Attachment: Screenshot taken at 6:10 am EDT.

Screenshot taken at 6:10 am EDT.

Comment 2

[Gary [:streetwolf52]]

Just wondering if my problem could be related to the Network Predictor?  I found this in my disk cache:

Key: predictor-origin:http://www.weather.com/

Cache entry information
key: http://www.weather.com/
fetch count: 7
last fetched: 2015-06-30 15:47:41
last modified: 2015-06-30 15:47:04
expires: No expiration time
Data size: 0 B
Security: This document does not have any security info associated with it.
predictor::seen: 1
predictor::http://injections.adguard.com/: 1,6,1435693616,0
predictor::http://tags.crwdcntrl.net/: 1,3,1435693616,0
predictor::http://www.weather.com/: 1,6,1435693616,0
predictor::http://c.amazon-adsystem.com/: 1,3,1435693616,0
predictor::http://s.w-x.co/: 1,3,1435693610,0
predictor::http://ajax.googleapis.com/: 1,3,1435693616,0
predictor::http://rtax.criteo.com/: 1,3,1435693616,0
predictor::http://ad.crwdcntrl.net/: 1,3,1435693616,0
predictor::http://aax.amazon-adsystem.com/: 1,3,1435693616,0
predictor::http://dsx.weather.com/: 1,6,1435693616,0
predictor::http://triggers.wfxtriggers.com/: 1,3,1435693616,0
predictor::http://fonts.googleapis.com/: 1,3,1435693616,0
predictor::http://cdn.taboola.com/: 1,3,1435693616,0
predictor::http://cdn.gigya.com/: 1,3,1435693616,0
predictor::http://fonts.gstatic.com/: 1,2,1435693606,0
predictor::http://gima.weather.com/: 1,3,1435693616,0
predictor::http://dev.virtualearth.net/: 1,1,1435693606,0
predictor::http://g3.imwx.com/: 1,2,1435693606,0
predictor::http://g1.imwx.com/: 1,2,1435693606,0
predictor::http://g2.imwx.com/: 1,2,1435693606,0
predictor::http://g0.imwx.com/: 1,2,1435693606,0
predictor::http://tags.tiqcdn.com/: 1,4,1435693616,0
predictor::http://www.googletagservices.com/: 1,4,1435693616,0
predictor::http://odc.weather.com/: 1,2,1435693616,0
predictor::http://h.nexac.com/: 1,2,1435693616,0
predictor::http://b.scorecardresearch.com/: 1,4,1435693616,0
predictor::http://secure-us.imrworldwide.com/: 1,2,1435693616,0
predictor::http://c2.taboola.com/: 1,2,1435693616,0
predictor::http://partner.googleadservices.com/: 1,2,1435693616,0
predictor::resource-count: 1
predictor::http://gscounters.us1.gigya.com/: 1,1,1435693616,0

Comment 3

[Patrick McManus [:mcmanus]]

(In reply to Gary [:streetwolf] from comment #2)
> Just wondering if my problem could be related to the Network Predictor?  

not likely.

the predictor tracks relationships between the origins you visit in order to warm up connections before you know you need them.. but it doesn't actually transfer any URLs (into the cache, or otherwise).

Comment 4

[Gary [:streetwolf52]]

Has anyone tried to verify this bug report?  The site updates perfectly using IE11.  I've tried other versions of Fx even portable version 38 beta 5 I believe and they alll failed to update the site.  I even tried PaleMoon and CyberFox with the same results.

If I clear the cache and then visit the site it displays properly.  When the next site update takes affect, which is every quarter hour on the hour plus a minute or two, it usually updates fine.  After that I will get forecasts that are from previous times.  If I don't go to the site for many hours and then do I might get a forecast from the last time I visited the site.

I tried spoofing the User Agent string to no avail.  I used IE11's UA as that works.

I know someone else who tried to confirm this.  He stated that going to the site didn't always update things but refreshing the page did.  I've found that doing a Ctrl-F5 many times in a row sometimes get's things current but eventually things will revert back to a previous time.

I tried running in Windows safe mode and had the same problems.  As IE11 works fine I would think that rules out the site as being the problem.

Comment 5

[Gary [:streetwolf52]]

Looked at the source for one of the forecast pages and I notice that it starts with 11 blank lines as does IE11.  Could it be that Fx isn't handling this situation correctly? Just a guess.
  
                      << 11 blank line >>
       <!DOCTYPE html>
  <!--[if IEMobile 7]><html class="ie iem7" lang="en" dir="ltr"><![endif]-->
  <!--[if lte IE 8]><html class="ie lt-ie9" lang="en" dir="ltr"><![endif]-->
  <!--[if IE 9]><html class="ie ie-9 lt-ie10" lang="en" dir="ltr"><![endif]-->
  <!--[if (gte IE 10)|(gt IEMobile 7)]><html class="ie ie-10" lang="en" dir="ltr" prefix="fb: http://ogp.me/ns/fb# og: http://ogp.me/ns# article: http://ogp.me/ns/article# book: http://ogp.me/ns/book# profile: http://ogp.me/ns/profile#"><![endif]-->
  <!--[if !IE]><!--><html lang="en" dir="ltr" prefix="fb: http://ogp.me/ns/fb# og: http://ogp.me/ns# article: http://ogp.me/ns/article# book: http://ogp.me/ns/book# profile: http://ogp.me/ns/profile#"><!--<![endif]-->
<head>

Comment 6

[Gary [:streetwolf52]]

Attachment: screenshot taken ~ 8:20 am EDT

Notice the time and forecast are from yesterday evening.

Comment 7

[Gary [:streetwolf52]]

I should have mentioned that I use ad blockers. AdGuard for Windows (Windows program) and uBlock origin.  Running with AG seems to be the root cause of the problem.  I then tried uBlock Origin and that worked fine.

I did notice a new option in uBlock to turn off link prefetching.  It was set to disable it by default.  This got me thinking so I enabled link prefetching in uBlock and sure enough I started pulling in old pages, probably from the cache I suspect.  Disabled link prefetching and all was well again.

I'll test with AG which I am assuming has the same problem.  While at first glance it might seem to be a bug in the ad blockers maybe it's how link prefetching is implemented on Fx.  I think link prefetching is turned on in IE11 yet it doesn't have the problems that Fx does.

Comment 8

[Gary [:streetwolf52]]

Disabling link prefetch didn't fix the problem when running AG.

Comment 9

[Honza Bambas (:mayhemer)]

(In reply to Gary [:streetwolf] from comment #8)
> Disabling link prefetch didn't fix the problem when running AG.

But disabling AG does fix the problem?

Comment 10

[Honza Bambas (:mayhemer)]

also a test with a clean profile would be good to do.  and maybe a clean profile + copy the content of cache2/ folder under the local part of the profile (I can provide more info where to find it if needed)

Comment 11

[Gary [:streetwolf52]]

It looks like the problem lies with AdGuard.  The developer of AG is making some changes he hopes will fix the problem.  Unless you have other reports about weather.com like this one I think we can close this.

Comment 12

[Honza Bambas (:mayhemer)]

Closing based on comment 11.

Comment 13

[Gary [:streetwolf52]]

I've determined that this problem has nothing to do with ad blockers as I have the problem on a new profile and I no longer use AdGuard.

The only way that weather.com has up to date forecasts is if I set my Fx cache to 0MB.  Changing the different prefs for the cache has no affect.  For some reason weather.com is pulling in old forecast data presumably from the cache.

Comment 14

[Gary [:streetwolf52]]

Something changed today because I've been getting current times and dates on forecasts.  Either something landed on inbound or weather.com changed something on their end as I emailed them about the problem.

Will wait a couple of days to see how it pans out.

Comment 15

[Gary [:streetwolf52]]

AN email I received from TWC.....


Greetings -


My job is to ensure we deliver a world-class technical experience for our fans all over the world. Over the past week, we experienced problems delivering on that promise for some of our Firefox users. I know you were impacted and I'm sorry for that.

The problem was related to how Firefox 39 and 40 is handling properly expiring old (cached) data. This problem, in some cases, is showing users running Firefox 39 and 40 old weather data and weather forecasts. We believe this is a bug in the most recent versions of Firefox and are working with Mozilla.org to ensure proper resolution.

In the mean time, we have deployed over 5 different changes on weather.com to work around this issue, and I believe this problem is now fully resolved. I apologize for the inconvenience this has caused.

Please know that I am a huge weather fan myself and take this very seriously. If you continue to see issues, please send an email to me at [e-mail] so I can make sure we deliver on our commitment to provide the world's best weather forecast.


Thank you,

Bryson Koehler

CTO, The Weather Channel

Comment 16

[Patrick McManus [:mcmanus]]

Would one of our cache experts contact the email in comment 15 to figure out who they are working with at mozilla? My wife has seen this too.

Comment 17

[Patrick McManus [:mcmanus]]

see bug 1198747 and bug 1087320 ..

Comment 18

[Honza Bambas (:mayhemer)]

(In reply to Patrick McManus [:mcmanus] from comment #16)
> Would one of our cache experts contact the email in comment 15 to figure out
> who they are working with at mozilla? My wife has seen this too.

Done.

Comment 19

[Honza Bambas (:mayhemer)]

(In reply to Patrick McManus [:mcmanus] from comment #17)
> see bug 1198747 

related (probably a duplicate)

> and bug 1087320 ..

unrelated

Comment 20

[Honza Bambas (:mayhemer)]

(In reply to Honza Bambas (:mayhemer) from comment #19)
> (In reply to Patrick McManus [:mcmanus] from comment #17)
> > and bug 1087320 ..
> 
> unrelated

Hmm.. and maybe yes.  If the flags are set wrong on sub-channels (LOAD_FROM_CACHE) we use the expired entries that are left because of not being evicted by lower layers in the meantime.

Comment 21

[Honza Bambas (:mayhemer)]

To explain: this started to manifest after we have landed bug 1136897 on version 39.  Before that expired entries were evicted by lowest I/O levels of the HTTP cache.  Now we leave them.  But HTTP channels should under normal circumstances ignore them (not reuse them) unless LOAD_FROM_CACHE flag is set.  Which may happen because of bug 1087320.