Last Comment Bug 765642 - kumascript: Implement a way to fetch XML from the web and extract content via xpath
: kumascript: Implement a way to fetch XML from the web and extract content via...
Status: RESOLVED WONTFIX
:
Product: Mozilla Developer Network
Classification: Other
Component: General (show other bugs)
: unspecified
: All All
: -- normal (vote)
: ---
Assigned To: Nobody; OK to take it and work on it
:
Mentors:
Depends on:
Blocks: 769503
  Show dependency treegraph
 
Reported: 2012-06-17 20:24 PDT by Les Orchard [:lorchard]
Modified: 2016-06-12 10:38 PDT (History)
6 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments

Description Les Orchard [:lorchard] 2012-06-17 20:24:04 PDT
This template uses web.xml to fetch XML from the web, and xml.text to extract content from the XML via xpath:

https://developer-new.mozilla.org/en-US/docs/Template:webkitbug

KumaScript doesn't support this, yet. Make it so, somehow.
Comment 1 Les Orchard [:lorchard] 2012-06-17 20:48:32 PDT
Maybe libxmljs will help here?

https://github.com/polotek/libxmljs
Comment 2 David Bruant 2012-06-18 10:59:56 PDT
Or jsdom: https://github.com/tmpvar/jsdom

The XPath queries are looking for "span" elements. It looks very HTML-ish to me...
And indeed, On https://bugs.webkit.org/show_bug.cgi?id=22049 , I've tried

    var x = document.getElementById('static_bug_status');
    console.log(x.textContent);

and it logged:
    
    "RESOLVED FIXED" (useless spaces removed).

I think you can go for jsdom and use DOM queries. Apparently jQuery can work well with jsdom, so it might help in writing the queries.

As far as "// Remove \n and redundant spaces" is concerned, a simple

    s = s.trim().replace(/\s+/g, ' ');

will do the trick in JS.


For that matter, I think Bugzilla as a product has a REST API, so if Webkit has activated it, it may be worth working with that rather than downloading the whole page (which may contain long-and-useless-for-what-we-need conversations), parsing it and extracting the useful info.
Maybe an idea for later if there are other priorities.
Comment 3 David Bruant 2012-06-18 11:17:49 PDT
There is an actual XML version: https://bugs.webkit.org/show_bug.cgi?ctype=xml&id=22049 ... It could be better than the HTML version, but the XML version contains all attachments encode in base64 -_-# So what was ~40k in HTML is ~110k in the XML version.

According to http://lpsolit.wordpress.com/bugzilla-usage-worldwide/ Webkit has a very old version of Bugzilla (with potential security breaches gnark gnark...) and according to https://wiki.mozilla.org/Bugzilla:REST_API it would be quite some work for them to get the REST API, so I don't think they have it.
Comment 4 David Bruant 2012-06-18 11:34:23 PDT
I've just discovered that Bugzilla has an XML-RPC interface since version 3... (Webkit is 3.2.3): https://wiki.mozilla.org/Bugzilla:WebService
But, for 3.2, I don't think its powerful enough.
Anyway, XML-RPC...
Comment 5 [github robot] 2013-12-02 08:18:18 PST
Commits pushed to master at https://github.com/mozilla/kuma

https://github.com/mozilla/kuma/commit/a7fd42aada9fe4312474e56ef2a15308d62a7fbc
bug 765642: Upgrade KumaScript to use FeedParser, expose zlib

https://github.com/mozilla/kuma/commit/4c8b49a5b972d5ce6b171da8c01f24fb52464591
Merge pull request #1694 from lmorchard/765642-feeds-and-xml

bug 765642: Upgrade KumaScript to use FeedParser
Comment 6 Les Orchard [:lorchard] 2013-12-02 09:09:00 PST
So, now we have an RSS/Atom feed parser in kumascript, which uses an XML parser under the hood. But, that parser is not yet accessible for general use.

What would help to close out this bug entirely are some use cases. That would help me figure out the best way to expose the XML parser to kumascript macro code.

In particular, are there some XML-based web APIs that we'd really like to access? I think the original use case was Bugzilla, but they seem to have a JSON API available now too. Do we still need XML there?
Comment 7 Justin Crawford [:hoosteeno] [:jcrawford] 2014-12-15 15:50:17 PST
> What would help to close out this bug entirely are some use cases. That
> would help me figure out the best way to expose the XML parser to kumascript
> macro code.
> 
> In particular, are there some XML-based web APIs that we'd really like to
> access? I think the original use case was Bugzilla, but they seem to have a
> JSON API available now too. Do we still need XML there?

Les asked this question a year ago, and since then nobody has offered any use cases. Since this sounds like a significant investment to build and maintain, and since no use cases have emerged, and since XML, I suggest WONTFIXING.

Note You need to log in before you can comment on or make changes to this bug.