846179 - Implement a minimal HTML API for linking to resources

Status: RESOLVED FIXED

(L20n :: HTML Bindings, defect, P1)

Description

[Staś Małolepszy :stas]

In bug 838176 and in https://groups.google.com/forum/?fromgroups=#!topic/mozilla.dev.webapi/ZMl7MBe4x1Y we're discussing our ideal solution for linking to localization resources in L20n.

In the mean time, I suggest we implement the simplest solution possible which is compatible with what Gaia does.

This will allow us to have "something" while we work towards a better approach.

The solution that is currently implemented on master (URL templates + manifests) feels incomplete and I wouldn't like for it to spread (e.g. as we do presentations or blog posts or write docs on L20n) until we flesh the details out.

Here's my minimal proposal:


Resource files
--------------

<link rel="localization" type="application/l10n" href="../locales/pl/browser.l20n" lang="pl">

Gaia uses re="resource" but I'd argue that pretty much everything in HTML (and <link>s in particular) could be described as such.

I'm not sure about type.  For starters, it's not required on link elements and can be inferred from rel.  Then, I'm not sure if it should say "application/l10n" (for localization in general), "application/l20n" (for the specific type of localization being used), or maybe "application/l10n+l20n" (please no).  FWIW, Gaia uses "application/l10n".

Lastly, I have doubts about "lang" vs. "hreflang".  Hreflang describes the language the linked resource is in, which is true for this case, but at the same, doesn't have to be.  We need something which describes the language the resource is *for*.  As in, imagine using a resource in German for Polish:

  <link rel="localization" type="application/l10n" href="../locales/de/browser.l20n" lang="pl">

Hence I suggest "lang".

Every locale would need to have its own <link>.


Inlined resources
-----------------

For performance reasons, we'd allow inlining localization resources to save on file IO.

<script type="application/l10n">
  <foo "Foo">
</script>

We should use the same type as in <link> above here.


Context data
------------

<script type="application/l10n-data+json">
{
    "user": "Joe"
}
</script>


Manifest files
--------------

No manifest files.  KISS.

Comment 1

[Staś Małolepszy :stas]

(In reply to Staś Małolepszy :stas from comment #0)

> No manifest files.  KISS.

For parity with Gaia's l10n.js, let's introduce simple manifests that act similar to Gaia's INI files.

I suggest using the following JSON structure:

{
  "locales": {
    "en-US": "../locales/en-US/browser.l20n",
    "pl": "../locales/pl/browser.l20n"
  }
}

JSON is easier to work with than INI because of a larger set of tools that support it.

In HTML, we'd link to the manifest as follows:

    <link rel="localization manifest" href="../locales/browser.json">

The per-resource syntax is still be supported:

    <link rel="localization" href="../locales/pl/browser.l20n" lang="pl">

(and I think we can drop type for now, as it's not required).

In the HTML bindings code, we'd use the following queries to get these elements:

    link[rel~="localization"][rel~="manifest]
    link[rel="localization"][lang]

Comment 2

[Staś Małolepszy :stas]

G, what do you think?

Comment 3

[Zibi Braniecki [:zbraniecki][:gandalf]]

I like it. I started hacking on it and it seems that there are two conceptual changes that we need to incorporate for that are related to removal of schemes:

1) locale fallback must know which resources to load
2) resources will be bind to locales

The issue in this model is that we need to be ready for resources that are not bind to any locale, like "single locale" dummy model:

ctx = L20n.getContext();
ctx.addResource('./locale.res');

or in HTML:

<link rel="localization" href="./locale.res">

It's easy to get this working but it requires a described heuristics for what to do when locale-specific ones will mix with undefined locales like:

<link rel="localization" href="./locale.res">
<link rel="localization" href="./locale2.res" lang="fr">

or:

ctx.addResource('./locale.res');
ctx.addResource('./locale2.res', "fr");

It also requires change to ctx.addResource to let us specify the language of the resource which means we're changing API again and we should think about what we will do for resource WebAPI so that we do not have to change the API again...

Comment 4

[Staś Małolepszy :stas]

Gandalf, these are great point.  Based on your feedback, I'd like to suggest a counter-proposal to my own comment 0 (and comment 1).

After a careful reading of http://tools.ietf.org/html/rfc5988 I came to a conclusion that we shouldn't overload the meaning of <link>.  Its primary use is to link to other web resources, like copyright notices, help pages, navigation etc.  I.e. other resources that the client may have interest in going to.  Its use for stylesheets seems to be more of a historical legacy and exception, than a rule.

More over, the RFC states the following which also made me reconsider the rel="localization manifest" proposal from comment 1.

   Relation types SHOULD NOT infer any additional semantics based upon
   the presence or absence of another link relation type, or its own
   cardinality of occurrence.  An exception to this is the combination
   of the "alternate" and "stylesheet" registered relation types, which
   has special meaning in HTML4 for historical reasons.


Here is a revised proposal for the minimal HTML API for L20n which I'd like us to implement in 1.0.


Manifest files
--------------

Manifest files are the recommended way of linking to localization files.

    <link rel="localization" href="../locales/browser.json">

The manifest files looks like this:

    {
      "locales": {
        "en-US": "../locales/en-US/browser.l20n",
        "pl": "../locales/pl/browser.l20n"
      } 
    }


Resource files
--------------

    <script type="application/l20n" src="../locales/pl/browser.l20n"></script>

This method of linking to a resource is intended for development, debugging, server-side optimizations, build-time optimizations and as a way to quickly get you started with L20n before you move on to the manifest approach.

I propose that per-resource script elements be single-locale only, with no way of defining which locale or which context they belong to.

If such element is present, L20n will ignore the manifest from the <link> (if it was found).  L20n will then create a new context with the resources specified in all <script type="application/l20n"> it can find.  No language negotiation nor locale fallback happens in this scenario.

Note that I'm using application/l20n, not /l10n, for type here.  This comes from http://www.w3.org/TR/html5/scripting-1.html#the-script-element:

    The type attribute gives the language of the script or format of the 
    data. If the attribute is present, its value must be a valid MIME 
    type. The charset parameter must not be specified. The default, which 
    is used if the attribute is absent, is "text/javascript".


Inlined resources
-----------------

For performance reasons, we'd allow inlining localization resources to save on file IO.

    <script type="application/l20n">
      <foo "Foo">
    </script>

If the <script> element has "src" attribute, the content of the script element is ignored.


Context data
------------

<script type="application/l10n-data+json">
{
    "user": "Joe"
}
</script>

Comment 5

[Staś Małolepszy :stas]

One last piece of functionality for feature parity with Gaia is the language negotiation.

Gaia doesn't really do much in this regard:  

 1) it uses navigator.language and observes mozSettings's language.current to know which locale to load.
 2) it always falls back on en-US because of how locales.ini files are written:

      @import url(browser.en-US.properties)
      
      [pl]
      @import url(browser.pl.properties)
    
    It always imports en-US and then overrides it with the other locale taken from the value of navigator.language.

Should we aim for a more sophisticated approach using the ICU, or maybe introduce a lang-nego shim specially for Gaia in 1.0, which mimics the above behavior?

Comment 6

[Zibi Braniecki [:zbraniecki][:gandalf]]

I like it, but I'm worried about the depth of changes in the API that it imposes.

I'd like to explicitly plan the Context API to match the changes here - in particular, the plan for fallbacks and addResource/injectResource/freeze.

I have a running POC for this proposal that alters how those two concepts work, and I believe it requires more explicit decisions before we go there.

Comment 7

[Zibi Braniecki [:zbraniecki][:gandalf]]

I updated the branch in URL with the latest POC. I believe Stas and I differ in how we want to translate this bindings into JS API.

Comment 8

[Zibi Braniecki [:zbraniecki][:gandalf]]

landed

Comment 9

[Staś Małolepszy :stas]

Gandalf, Pike and I worked on this both during in-person meetings, on the phone and via email.  We used etherpads to keep share ideas.  I found the following list of etherpads relevant to this bug.

https://l20n.etherpad.mozilla.org/html-api
https://l20n.etherpad.mozilla.org/html-bindings
https://l20n.etherpad.mozilla.org/html-types
https://l20n.etherpad.mozilla.org/rec
https://l20n.etherpad.mozilla.org/webapi-langpacks
https://l20n.etherpad.mozilla.org/addResource

Gandalf, can you distill the contents of these etherpads and put the gist of it in a subpage of https://developer.mozilla.org/en-US/docs/L20n?

I tried to summarize the basic usage in the README.  See bug 805234 and https://github.com/l20n/l20n.js/blob/master/README.md.  I hope this will be a good starting point.  Thanks.

Comment 10

[Staś Małolepszy :stas]

I created https://github.com/l20n/l20n.js/blob/master/bindings/README.md, but a full-blown MDN page would still be nice.