765642 - kumascript: Implement a way to fetch XML from the web and extract content via xpath

Status: RESOLVED WONTFIX

(developer.mozilla.org Graveyard :: General, defect)

Description

[Les Orchard [:lorchard]]

This template uses web.xml to fetch XML from the web, and xml.text to extract content from the XML via xpath:

https://developer-new.mozilla.org/en-US/docs/Template:webkitbug

KumaScript doesn't support this, yet. Make it so, somehow.

Comment 1

[Les Orchard [:lorchard]]

Maybe libxmljs will help here?

https://github.com/polotek/libxmljs

Comment 2

[David Bruant]

Or jsdom: https://github.com/tmpvar/jsdom

The XPath queries are looking for "span" elements. It looks very HTML-ish to me...
And indeed, On https://bugs.webkit.org/show_bug.cgi?id=22049 , I've tried

    var x = document.getElementById('static_bug_status');
    console.log(x.textContent);

and it logged:
    
    "RESOLVED FIXED" (useless spaces removed).

I think you can go for jsdom and use DOM queries. Apparently jQuery can work well with jsdom, so it might help in writing the queries.

As far as "// Remove \n and redundant spaces" is concerned, a simple

    s = s.trim().replace(/\s+/g, ' ');

will do the trick in JS.


For that matter, I think Bugzilla as a product has a REST API, so if Webkit has activated it, it may be worth working with that rather than downloading the whole page (which may contain long-and-useless-for-what-we-need conversations), parsing it and extracting the useful info.
Maybe an idea for later if there are other priorities.

Comment 3

[David Bruant]

There is an actual XML version: https://bugs.webkit.org/show_bug.cgi?ctype=xml&id=22049 ... It could be better than the HTML version, but the XML version contains all attachments encode in base64 -_-# So what was ~40k in HTML is ~110k in the XML version.

According to http://lpsolit.wordpress.com/bugzilla-usage-worldwide/ Webkit has a very old version of Bugzilla (with potential security breaches gnark gnark...) and according to https://wiki.mozilla.org/Bugzilla:REST_API it would be quite some work for them to get the REST API, so I don't think they have it.

Comment 4

[David Bruant]

I've just discovered that Bugzilla has an XML-RPC interface since version 3... (Webkit is 3.2.3): https://wiki.mozilla.org/Bugzilla:WebService
But, for 3.2, I don't think its powerful enough.
Anyway, XML-RPC...

Comment 5

[[github robot]]

Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/a7fd42aada9fe4312474e56ef2a15308d62a7fbc
bug 765642: Upgrade KumaScript to use FeedParser, expose zlib

https://github.com/mozilla/kuma/commit/4c8b49a5b972d5ce6b171da8c01f24fb52464591
Merge pull request #1694 from lmorchard/765642-feeds-and-xml

bug 765642: Upgrade KumaScript to use FeedParser

Comment 6

[Les Orchard [:lorchard]]

So, now we have an RSS/Atom feed parser in kumascript, which uses an XML parser under the hood. But, that parser is not yet accessible for general use.

What would help to close out this bug entirely are some use cases. That would help me figure out the best way to expose the XML parser to kumascript macro code.

In particular, are there some XML-based web APIs that we'd really like to access? I think the original use case was Bugzilla, but they seem to have a JSON API available now too. Do we still need XML there?

Comment 7

[Justin Crawford [:hoosteeno] [:jcrawford]]

> What would help to close out this bug entirely are some use cases. That
> would help me figure out the best way to expose the XML parser to kumascript
> macro code.
> 
> In particular, are there some XML-based web APIs that we'd really like to
> access? I think the original use case was Bugzilla, but they seem to have a
> JSON API available now too. Do we still need XML there?

Les asked this question a year ago, and since then nobody has offered any use cases. Since this sounds like a significant investment to build and maintain, and since no use cases have emerged, and since XML, I suggest WONTFIXING.