358148 - microsummaries update with browser.microsummaries.updateInterval after loading failed

Status: RESOLVED WONTFIX

(Firefox Graveyard :: Microsummaries, defect)

Description

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1) Gecko/20061010 Firefox/2.0
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.8.1) Gecko/20061010 Firefox/2.0

The update element is not correctly supported as described in http://developer.mozilla.org/en/docs/Microsummary_XML_grammar_reference

Reproducible: Sometimes

Steps to Reproduce:
1.Create a microsummary with the update element, for example include <update interval="5"></update>
2.Make a Live Title Bookmark with this microsummary.
3.Check the update interval (open the bookmark after a while, see the site has changed and wait the time specified in the update interval).

Actual Results:  
The live title is update sometimes.

Expected Results:  
Should update at the forced time.

Because microsummaries don't allow relative install-URLs, I'm not able to append an example.
The Microsummary offen updates at the start of firefox, but not always. browser.microsummary.updateInterval is set to 1000 days.

Comment 1

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Problem occurs in Firefox 2.0 and newest Minefield (Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20061026 Minefield/3.0a1)

Comment 2

[Jackie Liu]

Can we have a testcase?

Comment 3

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

So, here the steps to reproduce the problem (Branch and Trunk):

1. Create a new profile.
2. set showInConsole to true
3. Go to http://www.spreadfirefox.com
4. Bookmark it.
5. Go to http://www.erweiterungen.org/team/archaeopteryx/bugzilla/358148/ and install the microsummary.
6. Change the title of the bookmark from static to dynamic, it won't update (update interval is set to five minutes).

This I only tested with Trunk:
7. After restarting Firefox, it works.
(Now I tested if adding a bookmark would break the update, it doesn't, I restarted again.)
8. Now add a bookmark with opening the bookmarks tree, open a directory and add  a normal bookmark there.
9. I didn't received an update for the live title after this.
10. The error console contains:
"Error: [Exception... "Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIWebNavigation.sessionHistory]"  nsresult: "0x80004005 (NS_ERROR_FAILURE)"  location: "JS frame :: chrome://global/content/bindings/browser.xml ::  :: line 573"  data: no]
Source File: chrome://global/content/bindings/browser.xml
Line: 579"

Comment 4

[Myk Melez [:myk] [@mykmelez]]

I'm not able to reproduce this problem on a 2.0 branch nightly.  After installing the microsummary, setting it as the live title of the bookmark, and waiting 8 minutes, I found that the live title had updated to the current value for Firefox downloads on the Spread Firefox home page.

Is it possible that the behavior you were noticing is the result of bug 366877?  That could cause this behavior if you had www.spreadfirefox.com open in a tab while you were waiting to see if the live title would get updated.

Comment 5

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Ok, bug 366877 broke the reproduce, but I still this error. Last monday I updated a microsummary by hand, it seemed as the page had been updated before (but I couldn't verify because the microsummary checked only once a day). Now I have a microsummary which is updated every five minute and is broken for several hours. Changes which were made since it is broken: Network device has changed and add-ons have been updated, but Firefox not restarted yet. Only this in the error console sounds interesting:

Error: 'Component does not have requested interface' when calling method: [nsIInterfaceRequestor::getInterface] = NS_NOINTERFACE

(but only 4x)

Comment 6

[Myk Melez [:myk] [@mykmelez]]

(In reply to comment #5)
> Now I have a microsummary which is updated every five minute and is broken for
> several hours. Changes which were made since it is broken: Network device has
> changed and add-ons have been updated, but Firefox not restarted yet. Only 
> this in the error console sounds interesting:

When the microsummary is broken for several hours, does updating it manually work?

> Error: 'Component does not have requested interface' when calling method:
> [nsIInterfaceRequestor::getInterface] = NS_NOINTERFACE

That message is actually harmless and should not be the cause of this problem (see bug 347310).

What appears on the error console if you set the browser.microsummary.log preference to true?

Comment 7

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

(In reply to comment #6)
> When the microsummary is broken for several hours, does updating it manually
> work?
Yes.

> What appears on the error console if you set the browser.microsummary.log
> preference to true?
> 
Nothing. Wireshark shows that there haven't been requests in the time during which at least two requests should have be done.

Error: this.docShell has no properties
Source file: chrome://global/content/bindings/browser.xml

is the only error which sounds useful.

Comment 8

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Attachment: correct microsummaries log

this was shown for the manually forced updated for microsummaries which had been working correctly

Comment 9

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Attachment: not correct microsummary log

this was shown for the manually forced updated for the microsummary which had not been working correctly

Comment 10

[Myk Melez [:myk] [@mykmelez]]

> Error: this.docShell has no properties
> Source file: chrome://global/content/bindings/browser.xml
> 
> is the only error which sounds useful.

Hmm, the microsummary service does stuff with docshells, so this error could be related.  In particular, it loads pages being summarized using hidden iframes in the browser window.  Then, once the page has been summarized, it destroys the iframe.

This all happens very quickly, but perhaps some other code (core or extension) tries to do something with the page after it gets loaded that interferes with the microsummary service, and conversely the microsummary service destroys the iframe before the other code tries to access its docshell, triggering the error.

But that's just speculation.  As for the two log files, nothing seems amiss in the latter file for the microsummary that was not working correctly.  The sequence of messages in both files are the same and are the correct messages.

I do note that the page whose microsummary isn't getting updated (http://141.30.204.2/Homepage/index.php?main=userlounge&sub=traffic) seems password-protected.

Is it possible that the microsummary is failing to get updated when you aren't logged in because the XSLT stylesheet doesn't work when the page displays a "log in" form instead of whatever content it is supposed to display when you are logged in?

Comment 11

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Found the reason why it isn't updated: MICSUM_EXPIRATION (from the bookmark file) is far in the future (for all microsummaries except two which i updated this week manually):
For the five minute update generator, it's 1256050991609 (October 2009). I'll now watch the file for changes.

(The page is only shown for a special ip range without login form and I am in this range.)

Comment 12

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Haven't changed any settings, hibernated the system and let it wake up five hours later (firefox session was open), now MICSUM_EXPIRATION 1256247400890. this.docShell errors are not enough in the console for every time the microsummary should be updated.

Comment 13

[Myk Melez [:myk] [@mykmelez]]

(In reply to comment #11)
> Found the reason why it isn't updated: MICSUM_EXPIRATION (from the bookmark
> file) is far in the future (for all microsummaries except two which i updated
> this week manually):

Are all these microsummaries generated by generators, and do all of the generators have update intervals specified via an <update> tag?  What is the value of MICSUM_EXPIRATION for the two that you updated manually?


(In reply to comment #12)
> Haven't changed any settings, hibernated the system and let it wake up five
> hours later (firefox session was open), now MICSUM_EXPIRATION 1256247400890.

Is that the value that was previously 1256050991609, and, if so, did you manually update that microsummary?

Note that the bookmark file doesn't necessarily get updated every time the value changes.  Changes to bookmarks get written to the file upon shutdown, and I think they also get written every couple of minutes, but if you make a change (f.e. by manually updating a microsummary) and then immediately check the file, the file might not reflect the change.

Comment 14

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Maybe I found a reason: After a network crash (the router had no connection to the internet), the microsummary stopped working:

...
*** Microsummaries: microsummary.update called for page:
  http://<ip>/Homepage/index.php?main=userlounge&sub=traffic
with generator:
  urn:source:file:///E:/SEB.NET/Hobbies/Programmierung/Mozilla/Microsummaries/bu
da-traffic-v5.xml
*** Microsummaries: page content not yet loaded; downloading it
*** Microsummaries: http://<ip>/Homepage/index.php?main=userlounge&sub=t
raffic loading
*** Microsummaries: http://<ip>/Homepage/index.php?main=userlounge&sub=t
raffic load succeeded; invoking callback
*** Microsummaries: microsummary.update called for page:
  http://<ip>/Homepage/index.php?main=userlounge&sub=traffic
with generator:
  urn:source:file:///E:/SEB.NET/Hobbies/Programmierung/Mozilla/Microsummaries/bu
da-traffic-v5.xml
*** Microsummaries: generator (and page, if needed) both loaded; generating micr
osummary
*** Microsummaries: processing template [object Element] against document [objec
t XPCNativeWrapper [object HTMLDocument]]
*** Microsummaries: template processing result: 637.83 MB / 2.74 GB
*** Microsummaries: updated microsummary for page http://<ip>/Homepage/i
ndex.php?main=userlounge&sub=traffic to 637.83 MB / 2.74 GB
*** Microsummaries: generated microsummary: 637.83 MB / 2.74 GB
*** Microsummaries: microsummary.update called for page:
  http://<ip>/Homepage/index.php?main=userlounge&sub=traffic
with generator:
  urn:source:file:///E:/SEB.NET/Hobbies/Programmierung/Mozilla/Microsummaries/bu
da-traffic-v5.xml
*** Microsummaries: page content not yet loaded; downloading it
*** Microsummaries: http://<ip>/Homepage/index.php?main=userlounge&sub=t
raffic loading
*** Microsummaries: http://<ip>/Homepage/index.php?main=userlounge&sub=t
raffic load failed
*** Microsummaries: microsummary.update called for page:
  http://rcswww.urz.tu-dresden.de/~filmclub/mitte.htm
with generator:
  urn:source:file:///E:/SEB.NET/Hobbies/Programmierung/Mozilla/Microsummaries/en
glish-film-club.xml
*** Microsummaries: page content not yet loaded; downloading it
*** Microsummaries: http://rcswww.urz.tu-dresden.de/~filmclub/mitte.htm loading
*** Microsummaries: http://rcswww.urz.tu-dresden.de/~filmclub/mitte.htm load fai
led

After losing the connection, it should continue to try the download, shouldn't it? Will not be able to test this before the weekend because I have no access to the router here.

Comment 15

[Myk Melez [:myk] [@mykmelez]]

If the load fails, the microsummary service should continue to try to update microsummary, yes, but it applies the standard interval as specified in the browser.microsummary.updateInterval preference (or using the hard-coded value of 30 minutes if that preference is not defined).

It works something like this:

1. The service notices the microsummary has expired.
2. The service resets the microsummary's expiration time to the current time + the standard update interval as specified by the browser.microsummary.updateInterval preference (in minutes; the default is 30).
3. The service starts a microsummary update.
4a. If the update succeeds, and it results in a non-standard update interval, the service sets the microsummary's expiration time to the current time + the non-standard interval.
4b. If the update fails, the service doesn't change the microsummary's expiration time, which remains as set in step 2.

Hmm, since you're seeing this after a network failure, I wonder if perhaps the value of browser.microsummary.updateInterval is the culprit.  Has that value perhaps been expressed in milliseconds rather than minutes?

For example, if the value of that preference was 1500000, which is 25 minutes in milliseconds, then the microsummary service, which expects the value to be in minutes, would add 1500000*60*1000 milliseconds to the current time to derive the expiration time, which at the moment results in a time in 2009.

javascript:alert(new Date(Date.now()+(25*60*1000)*60*1000))

Comment 16

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

(In reply to comment #15)
Thank you, that is the problem. browser.microsummary.updateInterval was set to 1000 days (1440000 minutes) because I own around 100 live titles or so and loading them every 30 minutes while I am browsing would not be a good solution. Furthermore, many of them can only be accessed after I have logged in.

So at the moment, I have to edit all provided and installed microsummary generators in a editor for having a generator-specific update interval which allows me to prevent some live titles from updating and fix the issue with browser.microsummary.updateInterval so that it can be used with the default value 30 (minutes).

Because the problem now has been found, the microsummary generator should continue trying to update with the interval written into the generator after a load failed and not with browser.microsummary.updateInterval (or there should be at least a possibility to do this if you implemented this feature because access to the microsummary is not granted because it updates too fast).

Comment 17

[Myk Melez [:myk] [@mykmelez]]

Hmm, so I guess it would make sense to use the previous update interval rather than the default update interval if a load fails.  As I recall, our current way of storing the expiration time for a given live title rather than the last updated time + the update interval was also considered problematic in another bug.

Comment 18

[Marco Bonardo [:mak]]

- BUGSPAM -
Wontfixing all Microsummaries bugs, since the feature has been removed from the core product and previous versions won't get further fixes for it.
If interested in supporting Microsummaries in your add-on, you're free to use our old microsummaries code and to search all previously open bugs by looking for [microsummaries-feature-removal] in the status whiteboard field.