747166 - kumascript: Optimize wiki.page

Status: RESOLVED FIXED

(developer.mozilla.org Graveyard :: Wiki pages, defect, P2)

Description

[Les Orchard [:lorchard]]

In building DekiScript API shims, I found wiki.page. This allows one page to include the content of another page, with macros evaluated and class="noinclude" blocks omitted.

As it happens, some pages use this liberally and that generates a lot of HTTP requests. en/XUL/textbox, for one, generates a few hundred requests to build that page from dozens of others.

Part of this can be addressed with bug 747151, by minimizing how many requests kumascript needs to make to kuma for auto-required libs and other templates.

bug 747160 might help too, if wiki.page can use memcache to cache some of its own results.

Comment 1

[Les Orchard [:lorchard]]

And here's a peek at the code from http://mdn.lmorchard.com/en-US/docs/en-US/Template:DekiScript:Wiki


    // Retrieve the content of a document for inclusion
    // TODO: NEED CACHING HERE, BADLY!
    page: function (path) {
        var p = kuma.url.parse(env.url, true);
        var base = p.protocol + '//' + p.host + '/en-US/docs/';           
        var doc_url = base + path + '?raw=1&macros=1&include=1';
        var opts = { method: 'GET', url: doc_url };
        var result = '';
 
        var f = new Future();
        request(opts, function (err, resp, body) {
            if (resp && 200 == resp.statusCode) { result = body; }
            f.return();
        });
        f.wait();
         
        return result;       
    },

Comment 2

[Les Orchard [:lorchard]]

Maybe part of this optimization is to set really long expiration times on cached results on the KumaScript side. Then, invalidate cache entries from the Kuma script side when pages are saved. This would assume a shared naming convention in the cache on both sides.

Comment 3

[Les Orchard [:lorchard]]

Progress: https://github.com/mozilla/kumascript/pull/11

Comment 4

[Les Orchard [:lorchard]]

Merged: https://github.com/mozilla/kumascript/commit/736845b167278dbcf3c452bb2a320a255b5767e0

Comment 5

[James Socol [:jsocol, :james]]

Retroactively setting assignee to Les.