Open Bug 941241 Opened 11 years ago Updated 2 years ago

Displaying a better error message when severs send application/vnd.wap.xhtml+xml content type to Firefox OS and Firefox Android

Categories

(Core :: DOM: Core & HTML, defect)

Product:
Core
Component:
DOM: Core & HTML
Platform:
x86
macOS
Type:
defect

Tracking

()

People

(Reporter: karlcow, Unassigned)

References

Details

(Keywords: testcase-wanted) 

For mobile products:

When a user is accessing a Web site sending WAP content-type such as

    Content-Type: application/vnd.wap.xhtml+xml; charset=iso-8859-9

the user experience is sub-par.

1. On Firefox Android, if the user has also Opera Mobile. The user will have the choice to view the content with Opera Mobile or HTML classic viewer and will be able to access the content.

2. On Firefox OS, the browser tries to access the Web site and after a while just stop, with no error messages at all. Not giving any information if things worked or not.

Suggestions:

1. Reparse on the fly the content as HTML so the user can see the content.
2. Send a message in the console.log for Web developers to know that automatic reparsing has happened. 
3. (if none of 1 and 2) Give a more informative message to the user with maybe a way to report the site.


See https://bugzilla.mozilla.org/show_bug.cgi?id=841432#c6 for an example
See Also: → 841432
See Also: → 732957
See Also: → 840896
Is the request here to treat "application/vnd.wap.xhtml+xml" as "text/html"? Or is it to have better error handling when we are served "application/vnd.wap.xhtml+xml"?

Is "application/vnd.wap.xhtml+xml" content generally remotely html-compatible at all?
Blocks: 979333
Jonas, 

First it seems, there is a difference of behavior for the browser in between these two content types:

* Content-Type: application/vnd.wap.xhtml+xml
  generates a blank page
* Content-Type: text/vnd.wap.wml
  displays the content, I guess interpreted as HTML.

(I have to create test cases to be sure.)

We could in terms of user experience do:

*    reparse as HTML anything WAP. (need tests with real wap files)
* OR create a content transformation on the client side to convert it as HTML with a CSS.
* OR have a better error handling (aka a message).


I would prefer we display something to the user than nothing, but if it's too complicated (or insecure) I would fallback on an error message well designed. Currently there's none.

Most of the WAP page which are sent as "application/vnd.wap.xhtml+xml" are obviously not XHTML. See for example Bug 732957.


→ http GET http://adventureworld.net.au/ "User-Agent: Mozilla/5.0 (Android; Mobile; rv:20.0) Gecko/20.0 Firefox/20.0"



HTTP/1.1 200 OK
Cache-Control: private
Content-Length: 6681
Content-Type: application/vnd.wap.xhtml+xml; charset=utf-8
Date: Wed, 05 Mar 2014 00:55:19 GMT
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
X-UA-Compatible: IE=Edge,chrome=1
X-Umbraco-Version: 4.7

<!doctype html>
<!-- Conditional comment for mobile ie7 http://blogs.msdn.com/b/iemobile/ -->
<!--[if IEMobile 7 ]>    <html class="no-js iem7"> <![endif]-->
<!--[if (gt IEMobile 7)|!(IEMobile)]><!--> <html class="no-js"> <!--<![endif]-->

<head>
  <meta charset="utf-8">

  <title>Adventure World Mobile</title>


ooops meta charset not closed. 

Reparsing as HTML for these cases would be fine. IMHO. Maybe I should rephrase the title of that bug.
Summary: Reparsing WAP content type as HTML for Firefox OS and Firefox Android → Reparsing application/vnd.wap.xhtml+xml content type as HTML for Firefox OS and Firefox Android
Keywords: testcase-wanted
I don't think it's useful to debate what we currently do and why. Especially since there are various bugs in play and those bugs affect different platforms differently.

Let's instead focus on what we want to do. Anything is possible.


It's a good point that there's likely security issues involved in simply reinterpreting application/vnd.wap.xhtml+xml and/or text/vnd.wap.wml as HTML. In particular it could lead to XSS attacks since websites might not realize that such content can cause script to run with website privileges. Same applies if we were to translate the content into HTML.

We could still make such pages be parsed/translated into HTML, but disable any and all script on the resulting page.


So, looking specifically at the options you mentioned:

> *    reparse as HTML anything WAP. (need tests with real wap files)

I still don't understand if this ever would produce useful results? I.e. would the user ever see anything understandable. I'm guessing basic text would still render, though I'd like that verified (XUL parsed as HTML doesn't produce even basic text most of the time for example).

But would anything beyond basic text work? Would basic links work? Would images work? I have never seen WAP content, so apologies if this is a silly question.

I'm worried about displaying something to the user that it's almost always going to be useless to the user. Firefox(OS) isn't a data recovery tool of content from other formats.

> * OR create a content transformation on the client side to convert it as
> HTML with a CSS.

I'm somewhat reluctant to do this as it essentially means adding WAP support to Gecko. Is getting WAP content really common enough that we should make it a first-class citizen next to HTML?

> * OR have a better error handling (aka a message).

I agree that this is the minimal thing we should do. But I'd like to understand the above options better first.
No longer blocks: 979333
See Also: → 979333
Trying to have a better idea of the issue at the same time we are discussing it.

About Wireless Markup Language format WML used for WAP
It's difficult to find a good source on the OMA Web site http://technical.openmobilealliance.org/
There's a very short intro but which is not covering the full language.
http://www.sitepoint.com/markup-language-introduction/

WAP is the name of the protocol. WML is the name of the markup.
WML 1.0 is an XML syntax with specific element names such as card, anchor, go which I'm pretty sure are not working in Firefox Rendering engine.
Last versions of WML were simply XHTML.

I put a few controlled tests

* Broken XHTML (application/xhtml+xml)
  http://www.la-grange.net/2014/03/05/tests-wap/simple
  - Desktop: yellow error message
  - Firefox OS: yellow error message (not readable, need to be improved. too small.)
  - Firefox Android: yellow error message (not readable, need to be improved. too small.)

* WML (text/vnd.wap.wml)
  http://www.la-grange.net/2014/03/05/tests-wap/text-wap
  - Desktop: Modal window for downloading the file
  - Firefox OS: Blank page
  - Firefox Android: Download started and nothing

* XHTML (application/vnd.wap.xhtml+xml)
  http://www.la-grange.net/2014/03/05/tests-wap/xhtml-wap
  - Desktop: Modal window for downloading the file
  - Firefox OS: Blank page
  - Firefox Android: Download started and nothing


A last test for the purpose of seeing the effects.

* WML (text/html)
  http://www.la-grange.net/2014/03/05/tests-wap/html-wap
  - Desktop: Displays the HTML, but the specific WML elements are not interpreted
  - Firefox OS: same as Desktop
  - Firefox Android: same as Desktop


I guess a better error message would be the "best" solution with maybe a call to action on contacting the site owner.
(And also fix the error message for XML)
> I guess a better error message would be the "best" solution with maybe a
> call to action on contacting the site owner.

So sounds like "application/vnd.wap.xhtml+xml" and "text/vnd.wap.wml" should always be treated as errors when loaded as a page to be rendered. Sounds good to me. Care to propose a text?

> (And also fix the error message for XML)

Could you file a separate bug for fixing the XML error page. In general that should look less like geocities and more like our other error pages. Hopefully someone in the DOM team can help here.
Proposal for a text on Mobile.
1st draft.

   The <DOMAIN> is sending WAP content incompatible 
   with the capabilities of this modern device. 
   Contact the Web site owner. 

Asking to the Web Compat group too for review and opinions.
I don't think most of our users will know what WAP is or care. I'm also not sure that there is any point in suggesting that the user contact the Web site owner as they have no way of easily obtaining the contact information.

I suggest that we write an error message that makes it clear that the issue is with the site and not with Firefox OS. Here's an attempt:

"This site does not serve Web content. It is designed specifically for older devices and is incompatible with this device."
Lawrence,

Hmm Good point about WAP. Though it does serve Web Content (HTTP).

   "This site does serve Web content incompatible with this device."
OR "This site does not serve Web content compatible with this device."


The message has to be readable. There is a viewport issue on the current messages. I guess the HTML is not really adjusted to be responsive. :)
Jonas,

(In reply to Jonas Sicking (:sicking) from comment #5)
> > (And also fix the error message for XML)
> 
> Could you file a separate bug for fixing the XML error page. In general that
> should look less like geocities and more like our other error pages.
> Hopefully someone in the DOM team can help here.

done Bug 982979.
I think there might be a need for Design/UX.
I think we can safely call WML as "not web content". Web content really is HTML. Lots of things are served over HTTP that is not consumable by browsers.
heh. I don't want to enter in a semantic debate over what is the Web ;)

 What about:

   "This site does serve content incompatible with this device."
OR "This site does not serve content compatible with this device."
My concern is that it makes it sound like it's the fault of the device. The fact that they are serving WML to a modern browser is IMO crazy.
Hmm...

"This site serves content designed for feature phones and other devices incapable of displaying rich content".

"This site serves content that cannot be displayed by modern web browsers. It was designed for older devices."

Unsure if "feature phones" means anything to people outside of the mobile world.
I think the points that we want to get across, which I tried to do in comment #7 are:
1. The content isn't Web content. (We can debate this all day but I think making a statement like this is easiest for our users to understand.)
2. The content is designed for older devices and that is why it can't be displayed on this device. (It's not our fault and not a limitation of our device. We're simply not being served content that any modern device can display.)

Is my suggestion in comment #7 too long?
See Also: → 934857
See Also: → 956391
See Also: → 957444
See Also: → 957453
See Also: → 957457
See Also: → 957459
See Also: → 957464
At one point, one can need a particular bit of information which can be served on a misconfigured, badly managed web server. In some cases, the content creator could be deceased and obviously can't make the changes. In lot of cases, like the case of mine, you can't communicate with the right people to make a simple text change. The "user" is being the victim here and nobody else. 

Would it be too much to ask if Firefox displayed the simple rich text, provide simple basic functionality (click on link, it navigates to it) and log to console?
Lawrence,

I guess we could move forward with:
   "This site does not serve Web content. It is designed specifically 
    for older devices and is incompatible with this device."

It should there be a message in the console too more technical.

How do we move forward to have the change being made to Gecko?


ilgaz,

Agreed, that the user is victim, but the complexity (and its associated security issues) has to be taken into account too. Btw, have you been on a site which such messages, which one? Could you fill a bug. It would help us. Thanks.
https://bugzilla.mozilla.org/form.mobile.compat
Jonas - Can you provide guidance on who would be next to review and consider implementing the change suggested in this bug?
Flags: needinfo?(jonas)
I think this is a docshell change, which means that Olli is a good person to talk to.
Flags: needinfo?(jonas)
Olli - Can you please comment on the proposal in this bug? If we are to proceed, are you the right person to make this change?
Flags: needinfo?(bugs)
So it is not clear to me whether this bug is about
parsing wap/wml as HTML, or just about showing an error message or both?
Flags: needinfo?(bugs)
(In reply to Olli Pettay [:smaug] from comment #20)
> So it is not clear to me whether this bug is about
> parsing wap/wml as HTML, or just about showing an error message or both?

Sorry about that. The bug is about displaying a better error message for the scenario described in comment 0. The proposed error message is in comment 16.
...and prevent other apps to be opened, like Opera on Android?

(Trying to figure out if this all is just about changing the UI.)
See Also: → 1036987
Olli, 

The bug started at Reparsing. But after discussions, we figured out that it was better to just have a better user interaction. I rephrased the title.

What would be the next steps?
The current experience is still underwhelming.
Flags: needinfo?(bugs)
Summary: Reparsing application/vnd.wap.xhtml+xml content type as HTML for Firefox OS and Firefox Android → Displaying a better error message when severs send application/vnd.wap.xhtml+xml content type to Firefox OS and Firefox Android
Sounds like some UI changes only then.

Find someone to implement the better UI? I don't think this needs any docshell changes for example -
well, depends on what kind of UI we want.
Perhaps Margaret knows who from Mobile team could take a look?
Flags: needinfo?(bugs) → needinfo?(margaret.leibovic)
(In reply to Olli Pettay [:smaug] from comment #24)
> Sounds like some UI changes only then.
> 
> Find someone to implement the better UI? I don't think this needs any
> docshell changes for example -
> well, depends on what kind of UI we want.
> Perhaps Margaret knows who from Mobile team could take a look?

Is the issue here that we want a new flavor of network error page? Fennec overrides the default netError.xhtml that's included in docshell:
http://mxr.mozilla.org/mozilla-central/source/mobile/android/chrome/content/netError.xhtml

But I think we still may need some docshell change to add this new type of error.

wesj wrote this code, so he would know more.
Flags: needinfo?(margaret.leibovic) → needinfo?(wjohnston)
Yeah, I think we need docshell changes as well. Our current desktop behavior (using test cases in comment 4) is to show a dialog that offers to save or send the data to an app (i.e. we send this through the NSExternalAppHelperService). That's basically what Android does as well. I'm guessing B2G just doesn't support this type of redirecting yet?

To show an error, we'd need it to instead push to an error page, which requires some docshell changes to show error pages if the content-type is one of the "wap" ones listed above (or maybe we'd have a whitelist?) Then on the error page itself we can show/do whatever (including buttons to save or launch it in an external app if we wanted...)

Alternatively, I guess Android could catch these loads in HelperAppDialog.js or ContentDispatchChooser.js and try to redirect from there. B2G would need its own separate fix then?
Flags: needinfo?(wjohnston) → needinfo?(bugs)
A few data points from some WAP telemetry we've recorded:

Nightly 38[1]
B2G
non-WAP response 1139440 99.980345%
WAP response 	 224 	 0.019655%

Fennec
non-WAP response 23759046 99.999276%
WAP response 	 172 	  0.000724%

Nightly 39[2]
B2G
non-WAP response 235755 	99.988125%
WAP response 	 28 	0.011875%

Fennec
non-WAP response 87585112 99.999589%
WAP response 	 360 	  0.000411%

The one thing to note is that our Chinese Fennec users don't have these telemetry patches yet, nor do they have a Nightly channel so that market is entirely excluded from the results. Also the Nightly B2G community must be very small--probably just hackers and Mozilla employees.

Comparing B2G to Fennec, it seems like we should do something for B2G, and maybe do something for Fennec to be consistent (0.000411% is exceedingly rare).

Talking with blassey about this, it might be interesting to re-request a page with a spoofed UA string when we are served WAP content (as Chrome Mobile, for example)--collecting Telemetry along the way. My hunch is that we're being served WAP because our UA string is not known to these servers, but otherwise serve HTML content to known UAs.

[1] http://miketaylr.github.io/compat-telemetry-dashboards/wap-38-nightly.html
[2] http://miketaylr.github.io/compat-telemetry-dashboards/wap-39-nightly.html
Flags: needinfo?(bugs)
Severity: normal → S3

The severity field for this bug is relatively low, S3. However, the bug has 13 See Also bugs.
:jstutte, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

Flags: needinfo?(jstutte)

Moving to Dom Core for further triage.

Component: General → DOM: Core & HTML
Flags: needinfo?(jstutte)
You need to log in before you can comment on or make changes to this bug.