338527 - Load of remote calendar fails if 'Automatic proxy configuration URL' is set

Status: RESOLVED FIXED

(Calendar :: Provider: CalDAV, defect)

Description

[Stefan Sitter [:ssitter]]

Load of remote calendar fails during Sunbird startup if 'Automatic proxy configuration URL' setting is used.

Steps to Reproduce:
1. Go to 'Tools -> Options -> General -> Connection Settings...'

2. Select 'Automatic proxy configuration URL', enter the correct
   proxy conf url and press Reload, close dialog

3. Subscribe to remote calendar, 
   e.g. http://icalx.com/public/icalshare/US32Holidays.ics
   --> OK, events are displayed in calendar view

4. Quit and restart Sunbird
   --> Error, no events are displayed in calendar view

5. Select 'File -> Reload Remote Calendars' command
   --> OK, events are displayed in calendar view

Expected Results:
Remote calendar is loaded and displayed after Sunbird startup.

Additional Information:
After step 4 the following JavaScript error is displayed:

Error: [Exception... "Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIHttpChannel.responseStatus]"  nsresult: "0x80040111 (NS_ERROR_NOT_AVAILABLE)"  location: "JS frame :: file:///D:/sunbird/components/calICSCalendar.js :: anonymous :: line 917"  data: no]

Loading remote calendar on startup works if the proxy is set using 'Manual proxy configuration' setting.

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20060518 Mozilla Sunbird/0.3a2+

Comment 1

[Stefan Sitter [:ssitter]]

This is a regression:

WORKS in Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/2006030507 Mozilla Sunbird/0.3a1+

(2006030607 does not start)

FAILS in Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/2006030707 Mozilla Sunbird/0.3a1+

Checkins to directory mozilla/calendar during that time:
http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=CalendarClient&branch=HEAD&branchtype=match&dir=mozilla%2Fcalendar&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2006-03-05+06%3A00&maxdate=2006-03-07+07%3A00&cvsroot=%2Fcvsroot

Checkins to directory mozilla during that time:
http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=CalendarClient&branch=HEAD&branchtype=match&dir=mozilla&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2006-03-05+06%3A00&maxdate=2006-03-07+07%3A00&cvsroot=%2Fcvsroot

Problem has been reported with Lightning 0.1+ and Thunderbird 1.5.0.x in german forum too --> happens on branch and trunk.

Comment 2

[Stefan Sitter [:ssitter]]

I verified the regression range with Lightning:

Works in Thunderbird 1.5.0.5 (20060709) + Lightning 0.1+ (2006030508)
Fails in Thunderbird 1.5.0.5 (20060709) + Lightning 0.1+ (2006030608)

Therefore I think this is caused by one of the checkins to calICSCalendar.js during that day: probably Bug 326116, Bug 315511 or Bug 329373.

http://bonsai.mozilla.org/cvsquery.cgi?treeid=default&module=CalendarClient&branch=HEAD&branchtype=match&dir=mozilla%2Fcalendar&file=&filetype=match&who=&whotype=match&sortby=Date&hours=2&date=explicit&mindate=2006-03-05+08%3A00&maxdate=2006-03-06+08%3A00&cvsroot=%2Fcvsroot

Comment 3

[Stefan Sitter [:ssitter]]

Request for blocking0.3 flag because users might miss important events from a remote calendar.

Comment 4

[Stefan Sitter [:ssitter]]

lilmatt asked for retest: issue still exist with Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20060908 Calendar/0.3a2+

Error: [Exception... "Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIHttpChannel.responseStatus]"  nsresult: "0x80040111 (NS_ERROR_NOT_AVAILABLE)"  location: "JS frame :: file:///D:/sunbird/components/calICSCalendar.js :: anonymous :: line 936"  data: no]

Comment 5

[Stefan Sitter [:ssitter]]

Workaround:
Execute 'Reload Remote Calendars' after startup or 
Use proxy setting 'Manual proxy configuration'.

Remove from blocking list per irc discussion but add to Known Issues section of Release Notes.

Comment 7

[Jörg Raftopoulos]

(In reply to comment #5)
> Workaround:
> Execute 'Reload Remote Calendars' after startup or 
> Use proxy setting 'Manual proxy configuration'.
> 
> Remove from blocking list per irc discussion but add to Known Issues section of
> Release Notes.
> 

same issue with CalDAV calendars. 'Manual proxy configuration' works but 'Reload Remote Calendars' is not available for CalDav.

Comment 8

[Andrew DeFaria]

Same issue with WebDAV calendars too. This problem I see in all of 0.31, 0.5 and the new 0.7 pre. With 0.5 and 0.7 even reload calendars does not work. All of 0.31, 0.5 and 0.7 work fine without a proxy.

WebDAV: Version? How do you tell?
Apache: 2.0.53
Fedora Core 3.

Comment 9

[Stefan Sitter [:ssitter]]

I can confirm that this feature is now completely broken. The 'Reload' workaround from Comment #0 no longer works using Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7pre) Gecko/20070913 Calendar/0.7pre.

Error: [Exception... "Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIHttpChannel.responseStatus]"  nsresult: "0x80040111 (NS_ERROR_NOT_AVAILABLE)"  location: "JS frame :: file:///D:/sunbird/components/calICSCalendar.js :: anonymous :: line 885"  data: no]

Comment 12

[Duncan]

I tried to trace the error with the JavaScript plug-in debugger: Venkman.  Even though I am not a mozilla programmer I have managed to find and repair some bugs in other plug-ins with this before.

Unfortunately I did not get very far, perhaps somebody can make a suggestion where I can look further?

Here is how far I got.

I fired up the Venkman debugger in Sunbird and got the usual error:

Exception ``[Exception... "Component returned failure code: 0x80040111 (NS_ERROR_NOT_AVAILABLE) [nsIHttpChannel.responseStatus]" nsresult: "0x80040111 (NS_ERROR_NOT_AVAILABLE)" location: "JS frame :: file:///C:/Programme/Mozilla%20Sunbird/components/calICSCalendar.js :: anonymous :: line 885" data: no]'' thrown from function anonymous() in <file:/C:/Programme/Mozilla%20Sunbird/components/calICSCalendar.js> line 885.
Stopped for thrown exception.
#0: function anonymous() in <file:/C:/Programme/Mozilla%20Sunbird/components/calICSCalendar.js> line 885
883:
884: // 304: Not Modified
885: // Can use the old data, so tell the caller that it can skip parsing.
886: if (httpchannel.responseStatus == 304)
887: return false;

In the "onAfterGet" function call the "httpchannel" variable is only partially filled at this point, the problem here is that the httpchannel.responseStatus is empty, but the code is trying to evaluate the contents of the variable.

httpchannel {
httpchannel.allowPipelining == true
httpchannel.contentCharset == EMPTY-NO-PROPERTIES
httpchannel.contentLength == EMPTY-NO-PROPERTIES
httpchannel.contentType == EMPTY-NO-PROPERTIES
httpchannel.loadFlags == 768
httpchannel.loadGroup == null
httpchannel.name == "https://www.family.local/calendar/mycalendar.ics"
httpchannel.notificationCallbacks == const-XPcomponent
httpchannel.originalURI == const-XPcomponent
httpchannel.owner == null
httpchannel.redirectionLimit == 20
httpchannel.referrer == null
httpchannel.requestMethod == "GET"
httpchannel.responseStatus == EMPTY-NO-PROPERTIES
httpchannel.status == 2152398851
}
 
Here is the function, the error occurs at line 886:

 -   864 function httpHooks() {
     865     this.mChannel = null;
     866 }
     867 
     868 httpHooks.prototype = {
 -   869     onBeforeGet: function(aChannel) {
 -   870         this.mChannel = aChannel;
 -   871         if (this.mEtag) {
     872             var httpchannel = aChannel.QueryInterface(Components.interfaces.nsIHttpChannel);
     873             // Somehow the webdav header 'If' doesn't work on apache when
 -   874             // passing in a Not, so use the http version here.
     875             httpchannel.setRequestHeader("If-None-Match", this.mEtag, false);
     876         }
 -   877 
     878         return true;
     879     },
     880     
 -   881     onAfterGet: function() {
     882         var httpchannel = this.mChannel.QueryInterface(Components.interfaces.nsIHttpChannel);
     883 
     884         // 304: Not Modified
 -   885         // Can use the old data, so tell the caller that it can skip parsing.
 -   886         if (httpchannel.responseStatus == 304)
     887             return false;
     888 
     889         // 404: Not Found
     890         // This is a new calendar. Shouldn't try to parse it. But it also
 -   891         // isn't a failure, so don't throw.
 -   892         if (httpchannel.responseStatus == 404)
     893             return false;
 -   894 
 -   895         try {
 -   896             this.mEtag = httpchannel.getResponseHeader("ETag");
     897         } catch(e) {
 -   898             // No etag header. Now what?
     899             this.mEtag = null;
 -   900         }
 -   901         this.mChannel = null;
     902         return true;
     903     },
    
The call stack shows that this is called from the function "onStreamComplete", line 226 (see below).

I did not see any functions above this in the debugger stack, so I could not trace any further at this point.  However this function looks like a post HTTP-GET evaluation function. Interestingly enough the result is not empty, so the onAfterGet function gets called.

     214     // nsIStreamLoaderObserver impl
     215     // Listener for download. Parse the downloaded file
     216 
     217     onStreamComplete: function(loader, ctxt, status, resultLength, result)
     218     {
 -   219         // No need to do anything if there was no result
 -   220         if (!resultLength) {
 -   221             this.unlock();
     222             return;
     223         }
     224         
 -   225         // Allow the hook to get needed data (like an etag) of the channel
 -   226         var cont = this.mHooks.onAfterGet();
 -   227         if (!cont) {
 -   228             this.unlock();
     229             return;
     230         }
     231 
     232         // This conversion is needed, because the stream only knows about
     233         // byte arrays, not about strings or encodings. The array of bytes
 -   234         // need to be interpreted as utf8 and put into a javascript string.
 -   235         var unicodeConverter = Components.classes["@mozilla.org/intl/scriptableunicodeconverter"]
     236                                          .createInstance(Components.interfaces.nsIScriptableUnicodeConverter);
 -   237         // ics files are always utf8
 -   238         unicodeConverter.charset = "UTF-8";
 -   239         var str;
 -   240         try {
 -   241             str = unicodeConverter.convertFromByteArray(result, result.length);
 -   242         } catch(e) {
 -   243             this.mObserver.onError(calIErrors.CAL_UTF8_DECODING_FAILED, e.toString());
 -   244             this.unlock();
     245             return;
     246         }
     247 
 -   248         // Create a new calendar, to get rid of all the old events
 -   249         this.mMemoryCalendar = Components.classes["@mozilla.org/calendar/calendar;1?type=memory"]
 -   250                                          .createInstance(Components.interfaces.calICalendar);
 -   251         this.mMemoryCalendar.uri = this.mUri;
     252         this.mMemoryCalendar.wrappedJSObject.calendarToReturn = this;
 -   253 
     254         this.mObserver.onStartBatch();
     255 
     256         // Wrap parsing in a try block. Will ignore errors. That's a good thing
     257         // for non-existing or empty files, but not good for invalid files.
 -   258         // That's why we put them in readOnly mode
 -   259         try {
 -   260             var parser = Components.classes["@mozilla.org/calendar/ics-parser;1"].
 -   261                                     createInstance(Components.interfaces.calIIcsParser);
 -   262             parser.parseString(str);
     263             var items = parser.getItems({});
 -   264             
 -   265             for each (var item in items) {
     266                 this.mMemoryCalendar.adoptItem(item, null);
 -   267             }
 -   268             this.unmappedComponents = parser.getComponents({});
 -   269             this.unmappedProperties = parser.getProperties({});
 -   270         } catch(e) {
 -   271             LOG("Parsing the file failed:"+e);
     272             this.mObserver.onError(e.result, e.toString());
 -   273         }
 -   274         this.mObserver.onEndBatch();
     275         this.mObserver.onLoad(this);
     276         
     277         // Now that all items have been stuffed into the memory calendar
     278         // we should add ourselves as observer. It is important that this
     279         // happens *after* the calls to adoptItem in the above loop to prevent
 -   280         // the views from being notified.
     281         this.mMemoryCalendar.addObserver(this.mObserver);
 -   282         
     283         this.unlock();
     284     },
     285 

Comment 13

[Duncan]

By the way you may want to (vote) for this bug, see Votes above.

Also it would probably be good to have the "clean-report" and "helpwanted" key words set.

Comment 14

[Joey Officer]

I'm not a programmer, but I have a thought/suggestion.  How difficult would it be to implement a 'check-box' style of over-ride for using the configured proxy config, versus not using a proxy at all?

In theory ignoring the network settings completely.

Just a thought.
joey

Comment 15

[Simon Paquet [:sipaq]]

Not going to happen for 0.8.

Comment 17

[Kai Stukenbrock]

I am not sure I agree that bug 428260 is a duplicate of this bug, as it contains a more specific scenario. That is why I filed a separate bug.

To me it appears that the issue reported in bug 338527 is now half-resolved; at least if I only had http calendars, I assume automatic proxy configuration would be working for me. So it is not fully broken. As reported in bug 428260 half of my calendars load with automatic proxy configuration.

Just to retain the information posted in bug 428260 I am posting it here again:

My setup:
a) a number of calendars, some on Google calendar, with Google Calendar
Provider 0.4 (http), some remote ical calendars (https, hosted on GMX Media
Center)
b) automatic proxy configuration
c) authenticated proxy

My problem:
Under the above setup, after proxy authentication, the Google Calendar
calendars (http) get loaded and displayed, while the ics calendars (https) do
not get loaded, not even after reloading calendars or reloading proxy setting.
This problem occurs under both Sunbird and Lightning 0.8

Fix for now:
When I chose manual proxy configuration, the above works; being sometimes in
and sometimes outside our corporate network, this is inconvenient, however, as
I constantly have to switch options between manual proxy configuration and
direct connection to the internet.

Comment 20

[Philipp Kewisch [:Fallen][📱📆]]

Attachment: Fix interface requestor and prefs dialog

This bug fixes the issue. The interface requestor didn't return nsIChannelEventSink, so the (internal) redirect wasn't processed, but returned directly.

Also fixes a bug in the connection prefs dialog that causes an exception on branch, and fixes up the gdata provider to better handle getInterfaces.

Comment 21

[Philipp Kewisch [:Fallen][📱📆]]

Attachment: WCAP changes

Note these patches don't contain the needed bits for caldav. While wcap seems pretty easy, if a redirection happens on an upload channel, then the upload bits need to be set again. This might require some restructuring in the caldav provider.

Since I'm pretty short on time, maybe someone else (Daniel? Bruno?) could look into caldav. The important part is to implement onChannelRedirect and set up everything needed on the new channel. Maybe it works without setting the upload data, but I remember it didn't work with gdata and an external (i.e http server) redirect. The redirect thats happening here is internal though.

Also, using the Components.returnCode method, we don't need to add all sorts of interfaces we aren't implementing.

Comment 22

[Daniel Boelzle [:dbo]]

Comment on attachment 322454 [details] [diff] [review]
Fix interface requestor and prefs dialog

Great Philipp, this is one has bugged us too long already!

>     calGoogleRequest: {
>         getInterfaces: function cI_cGR_getInterfaces (aCount) {
>             var ifaces = [
>                 Components.interfaces.nsISupports,
>                 Components.interfaces.calIGoogleRequest,
>                 Components.interfaces.calIOperation,
>                 Components.interfaces.nsIStreamLoaderObserver,
>-                Components.interfaces.nsIDocShellTreeItem,
>                 Components.interfaces.nsIInterfaceRequestor,
>                 Components.interfaces.nsIChannelEventSink,
>-                Components.interfaces.nsIProgressEventSink,
>-                Components.interfaces.nsIHttpEventSink,
>                 Components.interfaces.nsIStreamListener,
>                 Components.interfaces.nsIClassInfo
>             ];

Oops, leftover? I took those over into wcap, thinking they relate to base interfaces...

>         try {
>             return this.QueryInterface(aIID);
>         } catch (e) {
>-            WARN("nsIInterfaceRequestor requesting invalid interface: " + Components.interfacesByID[aIID]);
>-            throw e;
>+            Components.returnCode = e;
>         }
>+        return null;
>     },

Ok, so we step back and use returnCode again. It would be great to know when returnCode needs to be used and when throwing is the correct way.
<http://developer.mozilla.org/en/docs/Components.returnCode>

looks good, r=dbo

Comment 23

[Daniel Boelzle [:dbo]]

Comment on attachment 322488 [details] [diff] [review]
WCAP changes

>     getInterface: function calWcapNetworkRequest_getInterface(aIID) {
>         // Support Auth Prompt Interfaces
>         if (aIID.equals(Components.interfaces.nsIAuthPrompt) ||
>             (Components.interfaces.nsIAuthPrompt2 &&
>              aIID.equals(Components.interfaces.nsIAuthPrompt2))) {
>             return new calAuthPrompt();
>         } else if (aIID.equals(Components.interfaces.nsIAuthPromptProvider) ||
>                    aIID.equals(Components.interfaces.nsIPrompt)) {
>-            return getWindowWatcher().getNewPrompter(null);
>+            return Components.classes["@mozilla.org/embedcomp/window-watcher;1"]
>+                             .getService(Components.interfaces.nsIWindowWatcher)
>+                             .getNewPrompter(null);
>         }

Why did you change the above? wcap has getWindowWatcher() in calWcapUtils.js.


r=dbo

Comment 24

[Daniel Boelzle [:dbo]]

(In reply to comment #21)
> Note these patches don't contain the needed bits for caldav. While wcap seems
> pretty easy, if a redirection happens on an upload channel, then the upload
> bits need to be set again. This might require some restructuring in the caldav
> provider.
> 
> Since I'm pretty short on time, maybe someone else (Daniel? Bruno?) could look
> into caldav. The important part is to implement onChannelRedirect and set up
> everything needed on the new channel. Maybe it works without setting the upload
> data, but I remember it didn't work with gdata and an external (i.e http
> server) redirect. The redirect thats happening here is internal though.

I've granted Bruno's "get rid of webdav component patch" (bug 16239) which hasn't landed yet, and is supposed to be bitrotten already. AFAIR it adds a general prepare function for http channels, so it makes sense to adopt the code when debitrotting it.

Comment 25

[Philipp Kewisch [:Fallen][📱📆]]

(In reply to comment #22)
> Ok, so we step back and use returnCode again. It would be great to know when
> returnCode needs to be used and when throwing is the correct way.
> <http://developer.mozilla.org/en/docs/Components.returnCode>
I think getInterface() is special. If you throw there, then the error message is displayed in the error console, and I'm not sure if anything else goes ballistic. It may even be enough to not throw or set returnCode at all, and just return null if there is no interface to satisfy the request.

> Oops, leftover? I took those over into wcap, thinking they relate to base
> interfaces...
I thought I'd have to specify each and every interface that is requested through getInterface(), since error messages were shown for the ones that were missing (see reason above). The way getInterface() returns now, those interfaces don't need to be implemented at all:

If something like nsIDocShellTreeItem is requested in getInterface(), then the QI will fail, null will be returned, and the calling channel will not try to execute the methods on the prototype. If something implemented like nsIChannelEventSink is requested, QI will return |this| and the channel will correctly call the methods of those interfaces.


(In reply to comment #23)
> (From update of attachment 322488 [details] [diff] [review])
> >-            return getWindowWatcher().getNewPrompter(null);
> >+            return Components.classes["@mozilla.org/embedcomp/window-> 
> Why did you change the above? wcap has getWindowWatcher() in calWcapUtils.js.
Sorry, not needed. I just copied over the function from gdata or ics since it looked quite the same, but didn't notice getWindowWatcher().



> looks good, r=dbo
Feel free to check in for me, I might not get to it before Wednesday.

Comment 26

[Philipp Kewisch [:Fallen][📱📆]]

Both patches:

Checked in on HEAD and MOZILLA_1_8_BRANCH


Keeping open until caldav is fixed.

Comment 27

[Bruno Browning]

(In reply to comment #24)

> I've granted Bruno's "get rid of webdav component patch" (bug 16239) which
> hasn't landed yet, and is supposed to be bitrotten already. AFAIR it adds a
> general prepare function for http channels, so it makes sense to adopt the code
> when debitrotting it.
> 

Yes, I'll incorporate this into the debitrot of 416239, now in progress.

Comment 28

[Philipp Kewisch [:Fallen][📱📆]]

Closing this bug to get it out of my queue, I added a comment to bug 416239 that that bug cannot be closed without fixing the caldav part of this bug.