1158510 - Language metadata for saved webpage titles

Status: RESOLVED INVALID

(Firefox for iOS :: General, defect)

Description

[Boris Dušek]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/600.5.17 (KHTML, like Gecko) Version/8.0.5 Safari/600.5.17

Steps to reproduce:

There are many situations where the UI displays a webpage title to refer to a webpage: Top Sites, Reading List, Cloud Tabs, etc.

As the webpage title is a user-visible content string originating in webpage, then for proper accessibility it should have accessibilityLanguage set to proper value. WebKit on iOS does it for each element in the webpage and this enables VoiceOver to switch languages (voices) on the fly when reading a multilingual webpage (or a monolingual webpage in different language than the user uses, i.e. if an English user opens a webpage on French wikipedia, then VoiceOver reads the webpage in French voice with proper French pronunciation. For even more fun, try the wikipedia frontpage, which contains links to different language wikipedias in those respective languages and iOS VoiceOver reads these all properly in their respective language/voice).

In a similar way, we would preferably make it so that e.g. in Top Sites, VoiceOver reads the webpage cell in the webpage language, in Cloud tabs, VoiceOver reads the webpage title in webpage language and the address in the user's current language, etc.

This would require learning the webpage title language by injecting a JavaScript to inspect the various lang attributes in the HTML. Then saving the learned webpage title language along with the webpage title, in a separate field/column. We would then use this language value to set accessibilityLanguage where appropriate to take effect on the webpage title in the UI.

This presents one challenge as I thought about it - some of these saved websites will be synced to some "Firefox cloud" (forgive me that I don't know its precise name :-). I.e. Cloud Tabs. For this feature to work here, there would need to be support for language attribute (field, column) also in the cloud backend, and also on the other Firefox products (not necessarily to use the language value, but at least extract it from the webpage and save it).

Other place where this could be used similarly is with Firefox for Windows when used with JAWS, which supports language switching (at least I saw JAWS do that with some browser and in Word, not 100% sure it can do it with Firefox, whether it requires MSAA/IAccessible2/or UIA).

Comment 1

[Boris Dušek]

The sequence I am thinking about for determining the title language is:
1. Does the <title> element have lang attribute? If yes, use it (and stop).
2. Does the URL response have Content-Language attribute? If yes, use it (and stop).
3. Does the webpage have meta http-equiv for Content-Language? If yes, use it (and stop).
4. If we made it here, set the language to nil (i.e. iOS default).

For 2., as far as I looked, it seems there is no way with WKWebView to get our fingers on the HTTP response (e.g. NSURLResponse), so keep fingers crossed that iOS 9 adds it. I have no idea whether most webpages today that return Content-Language HTTP header also provide the same in the html meta http-equiv tag.

Comment 2

[Boris Dušek]

Some concept JS code that I think worked for 1. and 3. (I wrote it ~2 months ago...):

var _firefox_Utils = {

    getTitleLang: function() {
        var lang = null;
        var titleElement = document.getElementsByTagName("title")[0];
        // 1. check which nearest ascendant (including title element itself)
        // has lang attribute set
        while (titleElement && lang == null) {
            lang = titleElement.getAttribute("lang");
            titleElement = titleElement.parentNode;
        }
        // 2. if none, then use the HTTP response Content-Language header.
        // As we do not have access to the HTTP response, try at least the
        // http-equiv="content-language" meta element, if present
        if (lang == null) {
            var metas = document.getElementsByTagName("meta");
            for (i = 0; i < metas.length; i++) {
                var meta = metas[i];
                if (meta.httpEquiv == "content-language") {
                    lang = meta.content;
                    break;
                }
            }
        }
        return lang;
    }
}

Comment 3

[Richard Newman [:rnewman]]

The challenge you note in Comment 0 is very real: synced tabs, downloaded history, synced bookmarks, etc. -- all of which are shown in lists in the UI -- won't have language annotations.

So I think this is legitimate, but in the foreseeable future this could only be for local data.

For v1/v1.5 this might include the tabs panel and other 'active' UI elements, if we don't do that already.

In the future we'd be able to extend storage to support language annotations for strings (so local history and bookmarks will work), and then in the far future sync these around.

Thanks for the excellent report!