Closed Bug 839306 Opened 11 years ago Closed 11 years ago

Content Curation API

Categories

(developer.mozilla.org Graveyard :: API, defect)

Product:
developer.mozilla.org Graveyard
Component:
API
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: openjck, Unassigned)

References

Details

(Whiteboard: [specification][type:feature]) 

What problems would this solve?
===============================
A content curation API would allow other groups within Mozilla (and elsewhere) to re-style MDN documentation. The DevHub team would especially benefit from this. They are writing documentation that could live on MDN, but have chosen not to because they would like additional control over things like branding and content organization.

Who would use this?
===================
This feature would be used by groups that wish to host documentation on MDN but also control the style of that documentation to a greater degree.

What would users see?
=====================
Remember that the "users" here are developers. They would not really "see" anything. They would have /access/ to an API that allows them to change the style of MDN documentation.

What would users do? What would happen as a result?
===================================================
Developers would use the API to specify details such as background color, font style, logo and more. The MDN documentation they target with these API calls will change accordingly.

The proposed API has been outlined in good detail in the following spreadsheet:
https://docs.google.com/spreadsheet/ccc?key=0ApY0XuSv-jEbdE9HRHQ4cG1ZMElsb0dJdTJNZjYxZVE#gid=0

One open question is whether the content these groups target should look different to all readers (including those just stumbling across the documentation on MDN) or whether the documentation should only look different to readers under certain circumstances (for example, when the reader is explicitly linked to the re-styled documentation).

Is there anything else we should know?
======================================
Some technical research into building the JSON API was completed in bug 815134. That said, we should decide on the design (exactly how this feature will be used) before we do much more tech research.
(In reply to John Karahalis [:openjck] from comment #0)
> One open question is whether the content these groups target should look
> different to all readers (including those just stumbling across the
> documentation on MDN) or whether the documentation should only look
> different to readers under certain circumstances (for example, when the
> reader is explicitly linked to the re-styled documentation).

As far as the open question above, only users who are viewing a page with a curation parameter present in the URL should see any change to the style or content arrangement. All URL links visited without a curation option(s) specified should resolve and display in the default MDN style.

> Is there anything else we should know?
> ======================================

In bug 815134 mentioned in Comment 1, it suggests the parameter that specifies the location of the JSON curation descriptor  file be keyed as "guide". I feel this is too general, and should be something unlikely to be confused with another general noun or already loaded term. I would suggest "curation" or "customstyle", something more specific to the action performed.

Other than that, I agree with the other comments David Walsh made in bug 815134.

Let me know if you need anything further!
Something to keep in mind as a building block for this - we have an API in the form of URL parameters for manipulating the output of MDN document URLs.

    https://developer.mozilla.org/en-US/docs/Project:The_Kuma_API

So, say you wanted the "Minimum requirements" section of Apps/For_Web_Developers, stripped of site chrome but with kumascript macros executed. You'd use this URL:

    https://developer.mozilla.org/en-US/docs/Apps/For_Web_developers?raw&macros&section=Minimum_requirements

If you had an inventory of pages & sections, it wouldn't be too hard to build a tool that queries the raw&macros URLs for those and remixes the content into a new presentation template.
That said, looking at the google spreadsheet, I don't really understand how this turns into an MDN feature. Could probably use some more spec work & fleshing out.
(In reply to Les Orchard [:lorchard] from comment #5)
> That said, looking at the google spreadsheet, I don't really understand how
> this turns into an MDN feature. Could probably use some more spec work &
> fleshing out.

And by this, I mean... are we talking about building a rigid HTML template that looks exactly like that marketplace page, only with those 7 "slots" for customization along with a list of pages/sections? Is there more implied here that I'm missing?
(In reply to Les Orchard [:lorchard] from comment #5)
> That said, looking at the google spreadsheet, I don't really understand how
> this turns into an MDN feature. Could probably use some more spec work &
> fleshing out.
I would formulate it differently:
Is what's already possible to do with the Kuma API enough? Because it looks so from the description (write the doc on MDN, but have it hosted elsewhere) that what's available would be enough.

If not, what else would be needed from these groups?
I think that with the edit API, it'd be even possible for another website to push content updates to the "main MDN" so that people would read and write from a different domain than developer.mozilla.org
Is this even still a priority? How likely are we to actually move DevHub content to MDN? From the last couple of devhub/deveco meetings I've been thru, it seems we're teasing out better-defined boundaries between MDN and Marketplace DevHub content. I.e., similar to the Addons Developer Hub <-> MDN relationships? (https://addons.mozilla.org/developers/)
(In reply to Les Orchard [:lorchard] from comment #6)
> And by this, I mean... are we talking about building a rigid HTML template
> that looks exactly like that marketplace page, only with those 7 "slots" for
> customization along with a list of pages/sections? Is there more implied
> here that I'm missing?

The idea is not to build the template to match the Marketplace page, the two screens are shown just to illustrate the content types that the API would allow users to express. This is not about a template, it is about a user being able to specify any number of a collection of parameters that adjust styles and add various content (re)arrangements to the output of the MDN pages they include in their content list.

Think of each of the API's now and future allowances as distinct options that allow for one thing - they should operate independently from each other (wherever possible).

If I want my content tree to have a sidebar that shows it, cool, I add an option asking for a sidebar. If I want a breadcrumb trail that resolves in relation to my content tree's root, cool, I add that parameter.

People should be able to use this API to mashup existing MDN content in new and interesting ways that make sense for their desired output format - this could be a tutorial, a handbook, or any other format - while not being strictly bound to rigid definitions of what we believe those look like.

Does this clarify the issue?
(In reply to David Bruant from comment #7)
> I would formulate it differently:
> Is what's already possible to do with the Kuma API enough? Because it looks
> so from the description (write the doc on MDN, but have it hosted elsewhere)
> that what's available would be enough.

Just kind of thinking out loud here to understand the problem we're solving:

If you have a site (say devhub) and you want MDN content to appear in a spot in some page template, go ahead and HTTP GET the appropriate Kuma URL and dump in the response. Cache and sanitize as appropriate (though we already use Bleach), but the facility is there to do content transclusion.

But, what I remember was an issue with that is MDN / Kuma is edit first, moderate later. And I recall there being some trepidation over exposing high-profile devhub docs to vandalism.

With regard to that, we've also got bug 768498 to set up editing access control lists per-page or per-section.
(In reply to Daniel Buchner [:dbuc] from comment #10)

> Does this clarify the issue?

Not really, no :)

> The idea is not to build the template to match the Marketplace page, the two
> screens are shown just to illustrate the content types that the API would
> allow users to express. This is not about a template, it is about a user
> being able to specify any number of a collection of parameters that adjust
> styles and add various content (re)arrangements to the output of the MDN
> pages they include in their content list.

So... in my mind, that's asking for a template: To what do the adjustable styles apply, and how are the various content rearrangements assembled as MDN output? That would be a template.

> Think of each of the API's now and future allowances as distinct options
> that allow for one thing - they should operate independently from each other
> (wherever possible).

I don't think I understand what you're saying here.

> If I want my content tree to have a sidebar that shows it, cool, I add an
> option asking for a sidebar. If I want a breadcrumb trail that resolves in
> relation to my content tree's root, cool, I add that parameter.

I guess what I'm saying is... what does the HTML for that sidebar & breadcrumb trail look like? That's what I mean by a template - we'd build a template with our (MDN's) idea of what a sidebar & a breadcrumb trail looks like, and you could switch it on & off. But that feels rigid to me.

> People should be able to use this API to mashup existing MDN content in new
> and interesting ways that make sense for their desired output format - this
> could be a tutorial, a handbook, or any other format - while not being
> strictly bound to rigid definitions of what we believe those look like.

Well, so that's what I'm saying about the query param API we have now. You can extract the HTML from pages & sections and plug it into what ever kind of template or structure you want - HTML, ePub, PDF, or whatever

But, you'd supply the HTML template or ebook processing chain, we supply the query param API for getting at raw MDN content. That feels the most flexible to me.
Are there other sites that do something like this? If no-one else is doing something like this now, has someone else tried in the past? As :jsocol put it, this just seems very "un-webby" to me?
(In reply to Les Orchard [:lorchard] from comment #12)
> Well, so that's what I'm saying about the query param API we have now. You
> can extract the HTML from pages & sections and plug it into what ever kind
> of template or structure you want - HTML, ePub, PDF, or whatever
> 
> But, you'd supply the HTML template or ebook processing chain, we supply the
> query param API for getting at raw MDN content. That feels the most flexible
> to me.

Sure, you can do that - all that's required is understanding how to fetch URLs with XHR or backend code, parse data on the client, write a web page to display it, and host it somewhere...burgeoning barriers to entry Batman!

The goal is to enable all manner of folks to easily curate our content for presentation in interesting ways we never thought of in a visual style that is complimentary to their own, where they prefer.

The idea that parameters that visually change the representation of a page are "un-webby", while parameters that retrieve a unique representation/curation of that same page's data are "webby", amounts to sandline-drawing on the arbitrary basis of output format and style.

Folks, consider this a more pluggable version of CSS Zen Garden with some light templating features - CSS Zen Gardenis one of the longest-standing, user-generated mashup sites in this same vein: http://www.csszengarden.com/
(In reply to Daniel Buchner [:dbuc] from comment #14)
> (In reply to Les Orchard [:lorchard] from comment #12)
> > Well, so that's what I'm saying about the query param API we have now. You
> > can extract the HTML from pages & sections and plug it into what ever kind
> > of template or structure you want - HTML, ePub, PDF, or whatever
> > 
> > But, you'd supply the HTML template or ebook processing chain, we supply the
> > query param API for getting at raw MDN content. That feels the most flexible
> > to me.
> 
> Sure, you can do that - all that's required is understanding how to fetch
> URLs with XHR or backend code, parse data on the client, write a web page to
> display it, and host it somewhere...burgeoning barriers to entry Batman!

Not trying to be snarky - but these all sound like basic web mashup skills. Maybe we need to step back and define the audience / users for this feature?

> The goal is to enable all manner of folks to easily curate our content for
> presentation in interesting ways we never thought of in a visual style that
> is complimentary to their own, where they prefer.

That's a fine goal. Now how, exactly, do we accomplish it? I'm looking for more specifics, so we can build something.

> The idea that parameters that visually change the representation of a page
> are "un-webby", while parameters that retrieve a unique
> representation/curation of that same page's data are "webby", amounts to
> sandline-drawing on the arbitrary basis of output format and style.

I don't understand what you're saying here.

> Folks, consider this a more pluggable version of CSS Zen Garden with some
> light templating features - CSS Zen Gardenis one of the longest-standing,
> user-generated mashup sites in this same vein: http://www.csszengarden.com/

Yes, and CSS Zen Garden comes with structured HTML to which CSS is applied. That's the clever part: "You may modify the style sheet in any way you wish, but not the HTML." And so, it is shown that you can do a lot with CSS.

But, they start with a fixed HTML document as the scaffolding from which to hang the CSS. So... what's our scaffolding look like? And what's our CSS-equivalent look like? And is a styled-scaffolding the right approach?
(In reply to Les Orchard [:lorchard] from comment #15)
> > The idea that parameters that visually change the representation of a page
> > are "un-webby", while parameters that retrieve a unique
> > representation/curation of that same page's data are "webby", amounts to
> > sandline-drawing on the arbitrary basis of output format and style.
> 
> I don't understand what you're saying here.

I'm saying that if you have a REST API that serves a curated version of a page's data via parameters, what there is an arbitrary line drawn between that and having the same page display that data differently based on parameters - this is self-defined hand-wavery. 

> > Folks, consider this a more pluggable version of CSS Zen Garden with some
> > light templating features - CSS Zen Gardenis one of the longest-standing,
> > user-generated mashup sites in this same vein: http://www.csszengarden.com/
> 
> Yes, and CSS Zen Garden comes with structured HTML to which CSS is applied.
> That's the clever part: "You may modify the style sheet in any way you wish,
> but not the HTML." And so, it is shown that you can do a lot with CSS.
> 
> But, they start with a fixed HTML document as the scaffolding from which to
> hang the CSS. So... what's our scaffolding look like? And what's our
> CSS-equivalent look like? And is a styled-scaffolding the right approach?

That was the choice of CSS Zen Garden - if they had told people they can rearrange the nav structure and the CSS, does the sky begin to fall? Is there some unwritten rule I missed on the web that says allowing user styles on sites/wikis is "webby", but allowing user to remix content via parameters is "un-webby"? Who made these fundamental rules? --> again, this is all subjective, arbitrary, conjecture about what is "webby"...sigh
If we do reach an impasse that results in discarding this feature, can we, for accuracy and comedic value, mark the bug: "RESOLVED" > "UN-WEBBY"?
(In reply to Daniel Buchner [:dbuc] from comment #14)
>
> The idea that parameters that visually change the representation of a page
> are "un-webby", while parameters that retrieve a unique
> representation/curation of that same page's data are "webby", amounts to
> sandline-drawing on the arbitrary basis of output format and style.

Oh, on second reading, I think you're responding to an argument I'm not making: I'm not trying to be a stickler for REST or of-the-web or anything like that. I'm just trying to pin down what we're building, for whom, and why, and whether it's the best fit.

My notion of using the current content API is that it gives the most flexibility for a web developer. That works & can be used right now. That's like a lunch buffet.

It sounds like maybe you want something more like a Tumblr / SquareSpace layout that's themeable? That's kind of like a prix fixe menu that we define. In that case, I'm looking for things like... what's the structure of the layout, what options can be turned on and off, what can users style (just the header & background?), etc.
(In reply to Daniel Buchner [:dbuc] from comment #16)

> I'm saying that if you have a REST API that serves a curated version of a
> page's data via parameters, what there is an arbitrary line drawn between
> that and having the same page display that data differently based on
> parameters - this is self-defined hand-wavery. 

I'm not trying to hand-wave - just the opposite. If we build the assembly-and-output mechanism (I've been calling that a template), I want to pin down the parameters and define the range of display variability.

> That was the choice of CSS Zen Garden - if they had told people they can
> rearrange the nav structure and the CSS, does the sky begin to fall?
> Is there some unwritten rule I missed on the web that says allowing user styles
> on sites/wikis is "webby", but allowing user to remix content via parameters
> is "un-webby"? Who made these fundamental rules? --> again, this is all
> subjective, arbitrary, conjecture about what is "webby"...sigh

I don't know who said "webby" / "un-webby", but it wasn't me. I just want clarity on precisely what these parameters are, what effects they have, and how things are remixed in their use. That way, we know exactly what to build.
(In reply to Luke Crouch [:groovecoder] from comment #13)
> Are there other sites that do something like this? If no-one else is doing
> something like this now, has someone else tried in the past? As :jsocol put
> it, this just seems very "un-webby" to me?

Oh wait, I just page-searched and found where the "webby" thing started. I see now.

Basically, I guess what I'm concerned about is that whatever we build in terms of a parameterized-remixer won't be flexible enough for all comers. But, whatever, maybe it'll make a good trade off with ease-of-use. Bottom line, I just want to nail it down so we're all on the same page.

Meanwhile, the content API exists, and more advanced webdevs can do ahead and use it if they find whatever we build is insufficient.
Our use-cases for this are:

- Product pages that need unique visual style and collect together various resources from all over MDN

- Blogs like David Walsh's may want to gather together a bunch of content to form a handbook that goes with a particular blog post topic - now they can present it in a cohesive, visually appealing way that presents the user with a common content zone.

- Developers who want to link to MDN documentation within their own non-standard documentation can now do so and have it look more in step with the presentation and style of their content.

What we need in terms of APIs is the following:

- Something to gather various links and resources in a way that scopes presentation of the UI in the page (other than MDN's global nav) to just that content tree.

- The tree should be represented via a persistent sidebar that contains the tree as specified in the curation JSON file

- The user may want breadcrumbs, if this option is enabled, the breadcrumbs should be scoped to the content tree they provide

- The styles and layout specified in the curation should disappear if the user navigates away via MDN logo, global UI, or search page. If the user performs one of these 'escape actions', MDN should cease presenting content in their curated style

Side-Bonus-Yahtzee-Benefit:

I'll buy everyone who participated in this bug a beer if this doesn't increase traffic to MDN ;)
(In reply to Daniel Buchner [:dbuc] from comment #21)
> Our use-cases for this are:
> 
> - Product pages that need unique visual style and collect together various
> resources from all over MDN
> 
> - Blogs like David Walsh's may want to gather together a bunch of content to
> form a handbook that goes with a particular blog post topic - now they can
> present it in a cohesive, visually appealing way that presents the user with
> a common content zone.
> 
> - Developers who want to link to MDN documentation within their own
> non-standard documentation can now do so and have it look more in step with
> the presentation and style of their content.

Okay, so this sounds like a good blue-sky list. But, my impression from recent conversations is that our (MDN's) first priority is to build something that makes it easier to integrate MDN documentation with DevHub & other Mozilla projects. So, we might want to narrow the focus, at least for the first iteration, to something achievable sooner-than-later.

> What we need in terms of APIs is the following:
> 
> - Something to gather various links and resources in a way that scopes
> presentation of the UI in the page (other than MDN's global nav) to just
> that content tree.

Various links & resources... from just MDN? or web-wide? And what's a resource - MDN page content? anything from the web?

> - The tree should be represented via a persistent sidebar that contains the
> tree as specified in the curation JSON file

So, our template includes a sidebar consisting of an outline tree of documents. The documents (from MDN?) can be declared in a JSON structure, maybe like what :davidwalsh sketched in bug 815134

> - The user may want breadcrumbs, if this option is enabled, the breadcrumbs
> should be scoped to the content tree they provide

So, our template includes a breadcrumb section, backed by a parameter to show/hide. Links in the breadcrumb follow current position within the doc tree declared for the sidebar.

> - The styles and layout specified in the curation should disappear if the
> user navigates away via MDN logo, global UI, or search page. If the user
> performs one of these 'escape actions', MDN should cease presenting content
> in their curated style

So, let's specify what these style and layout choices are. This is where my understanding is really vague. So far, it looks to me like we need to build a template with these parts:

* Overall - customizable with font face & background image parameters
* Header - with title, logo (size / aspect constraints?), & background image parameters
* Breadcrumbs - with parameter to hide/show
* Sidebar - populated by tree declared in JSON
* Content area - populated by doc content based on current nav position in sidebar tree

Is this it? Am I missing anything? Let's be exhaustive & explicit. This is what I was asking in comment 6.
(In reply to Les Orchard [:lorchard] from comment #22)

> Is this it? Am I missing anything? Let's be exhaustive & explicit. This is
> what I was asking in comment 6.

(And I ask if this is it, because it doesn't seem like much. That rings my hidden assumptions / scope-creep-incoming alarm bells.)
Oh, and just to pre-cache my next questions in case the answer to "is this it?" in comment 22 is "yes":

* Other than the parameterized parts, what will the design of these pages look like?

* If I declare nothing else but a tree of documents, what are the defaults for all the other parameters? (eg. logo, font, background images, etc)

* Are we at all concerned about security / privacy implications of including third-party images?

* Is it possible that user-specified CSS will be a requirement? If so, are we concerned about security / privacy implications involved in third-party CSS?
(In reply to Daniel Buchner [:dbuc] from comment #21)
 
> - The styles and layout specified in the curation should disappear if the
> user navigates away via MDN logo, global UI, or search page. If the user
> performs one of these 'escape actions', MDN should cease presenting content
> in their curated style

Further thought, on the "escape actions" notion:

We'd need to rewrite hyperlinks in the assembled content, such that any "intra-collection" links point to URLs that stay within the curated sub-site.

Otherwise, a click on any link other than within the sidebar would jump our to the original, non-customized doc. This will be interesting, because MDN docs are sometimes inconsistent in the use of relative vs absolute URLs.

And from there, I'm wondering: What do the URLs for one of these curated sub-sites look like? 

They'll need to somehow identify the JSON that defines the sub-site, as well as the current position within the collection. We could do something like this, but it makes me sad:

    https://developer.mozilla.org/en-US/curated/?json={URL-to-some-JSON}&section={some-pointer-to-within-the-collection}

Or, even this, but also sad-making:

    https://developer.mozilla.org/en-US/docs/Apps/For_Web_developers?section=Minimum_requirements&curation={URL-to-some-JSON}

We can also make this a heavy-client/thin-server pushState() driven web app and hide some of the ugliness under the hood there. But, that also has its own complications & isn't a silver bullet.
Just wanted to make a point about process. We have a "Design" phase that all changes go through before development begins. The goal of this phase is to reach a decision on how the change should look and behave, and get relevant stakeholders to sign off on it.

It sounds like sign-offs will be especially important here. I would like to see Daniel, Les and Luke all sign off on an agreement before we get started with development. After development is complete, we can pass the result by these same people for verification. This is a very important feature, so I want to be sure we get it right and ensure all stakeholders are pleased.
Yeah, all I'm after with my comments here is to nail things down with respect to design - both technical and visual - before we commit to building anything. And, I don't feel like the initial info I've seen for this feature is there yet.

Once we've all agreed we're on the same page for the design, then we can talk more precisely about feasibility & suitability & estimation & etc.
I could be wrong, but it looks like we never came to a consensus on the Kuma API. At the risk of beating a dead horse, is it enough to solve this problem? If not, why not?
My "un-webby" vs. "webby" line is not necessarily between technical design like ?raw vs. ?curation=<JSON-URL> ... it's between MDN and other websites. We can't be the first ones to have this idea, so why don't other websites allow everyone to curate and theme the content on the site?

CSS Zen Garden is a (maybe the only) example, but visual theme customization is the entire and only purpose of that site. And even their workflow is full of barriers to entry: "Download the sample html file and css file to work on a copy locally. Once you have completed your masterpiece (and please, don’t submit half-finished work) upload your .css file to a web server under your control. Send us a link to the file and if we choose to use it, we will spider the associated images. Final submissions will be placed on our server."

I'm pushing back on this feature to protect our domain reputation with web developers.

When I visit a link, and especially a link to web developer documentation, I decide whether or not to click based on the domain in the status bar. github.com? stackoverflow.com? readthedocs.org? developer.mozilla.org? No problem. w3schools.com? No way.

The largest factors in our reputation with web developers are quality content and then a consistent experience - *within the domain itself*. e.g., on every page of readthedocs.org I use ToC & Previous/Next Topic links in the left sidebar; on every github page I scroll up and click the repos breadcrumbs to get to its home page. If RTD or github had a feature like this, someone on StackO could link to those sites with their favorite content curation/theme and I might end up on a readthedocs page without a navigation sidebar, or a github page without breadcrumbs. I would be so pissed.

Parting shots:

1. readthedocs.org is the best model of a documentation platform site. It has 2 theme options - sphinx default, or their custom color variation of it.

2. Would we allow this feature on DevHub too? E.g., I could link from MDN to https://marketplace.firefox.com/developers/docs/firefox_os?curation={'sidebar':'right', 'breadcrumbs': false, 'logo': 'mdn-dino.png'}
Seems like we stalled out here on pinning down the requirements / design. 

For what it's worth, we closed bug 720068 last week and I built this quick demo using the Kuma API + CORS:

http://dl.dropbox.com/u/2798055/mdn-mash/index.html
(In reply to Les Orchard [:lorchard] from comment #30)
> Seems like we stalled out here on pinning down the requirements / design. 
> 
> For what it's worth, we closed bug 720068 last week and I built this quick
> demo using the Kuma API + CORS:
> 
> http://dl.dropbox.com/u/2798055/mdn-mash/index.html

I'm going to bow out about now. We had spent time discussing this feature at lenght with many of the stakeholders, including Ali and several other MDN devs. Obviously there other ways to display content with a site's own selection of docs and flavor of style. I could "easily" spin up a Node.js app and scrape to my heart's content or I could "more easily" do a JS-driven, dev coded CORS mash-up of content via your example. The thing is, what you see as easy and the answer, is this: 

// index.js
(function () {

    var MDN_BASE = 'https://developer.mozilla.org';

    // Trap link clicks in sidebar or content for within-app nav
    $('#sidebar ul').delegate('a', 'click', trap_link);
    $('#content').delegate('a', 'click', trap_link);

    // Any links starting with "/" probably point at MDN, so fix them.
    function absolutify_link (url) {
        if ('/' == url.substr(0,1)) {
            url = MDN_BASE + url;
        }
    }

    function trap_link (ev) {
        return load_content($(ev.target).attr('href'));
    }

    function load_content (url) {
        if ('/' == url.substr(0,1)) {
            url = MDN_BASE + url;
        }
        if (url.indexOf(MDN_BASE) !== 0) {
            return true;
        }
        $('#content').addClass('loading');
        $.ajax({
            url: url + '?raw&macros', 
            complete: function (xhr, stat) {
                var loaded_ct = $('<div>' + xhr.responseText + '</div>');
                loaded_ct.find('a').each(function (idx) {
                    var a = $(this);
                    a.attr('target', '_blank');
                    var href = ''+a.attr('href');
                    if ('/' == href.substr(0, 1)) {
                        href = MDN_BASE + href;
                        a.attr('href', href);
                    }
                });
                $('#content').empty().append(loaded_ct);
                $('#content').removeClass('loading');
            }
        });
        return false;
    }
    
    // When pinboard.in bookmarks are loaded, use them to populate the sidebar.
    window.bookmarksLoaded = function (bookmarks) {
        
        var list = $('#sidebar ul');
        _.each(bookmarks, function (item) {
            // Create list item with text and link based on the bookmark.
            var li = $('<li><a href=""></a></li>');
            li.find('a').text(item.d).attr('href', item.u);

            // Try fetching the first para of the summary at the bookmark URL.
            $.ajax({
                url: item.u + '?raw&macros&section=Summary', 
                complete: function (xhr, stat) {
                    var ct = $('<div>'+xhr.responseText+'</div>');
                    ct.find("p:first").appendTo(li);
                }
            });

            // Add the item to the list.
            list.append(li);
        });

        // All sidebar links should open in a new window.
        $('#sidebar a').each(function () {
            $(this).attr('target', '_blank');
        });

    };

}());

Yup, that doesn't look special case, difficult to scale (for people who want to manage several curation views), or simple to iterate on from the tinkerer's perspective.

You say this in your previous comment: "If RTD or github had a feature like this, someone on StackO could link to those sites with their favorite content curation/theme and I might end up on a readthedocs page without a navigation sidebar, or a github page without breadcrumbs. I would be so pissed." Wow, doesn't sound like anything a link at the top saying: "view MDN default" couldn't cure, next?

Another goal of this API, was for people in less technical roles internally and externally to be able to create a handbook/hit sheet with little effort - think of the cases where we have Firefox releases that need a curated sheet of what changed, new web APIs introduced, etc. With this API, a PR person could easily collect those and curate the page themselves. You think the same person could use your "easy" JS-based, CORS-fueled code example? I think not.

Anywho, my job wasn't to fight for a feature in a bug, it was to spec one and get agreement from a reasonable set of stakeholders then move to implementation if that agreement was attained. After we came to a reasonable agreement, it moved to this phase - clearly it is not realizing the intended outcome.

I simply don't have the time to outlast this filibuster and present the case more clearly than I already have --> Not doing this will make scraping and non-linked use of MDN content more prevalent as time goes on, it will prevent non-technical folks from curating helpful groups of content, and will make it a managerial chore to style and maintain a one-off  grouping of MDN content via other Mozilla properties.

I'm fine with closing this out WONTFIX, because that's obviously where we're at.
(In reply to Daniel Buchner [:dbuc] from comment #31)

> I'm going to bow out about now. We had spent time discussing this feature at
> lenght with many of the stakeholders, including Ali and several other MDN
> devs.

Yes, and now we're trying to nail down the requirements. Please help. I'm asking very specific questions and trying to understand this feature, not arguing philosophy or trying to filibuster.

See comment 22, comment 23, comment 24, and comment 25. I asked questions that I think we need answered to get closer to an implementation, which is all I've been doing since comment 6 on this bug.

> ... The thing is, what you see as easy and the answer, is this:

I'm trying to understand what you have in mind, so I made a demo. I'm a webdev, so yes, that medium was easy for me. What we build for this bug doesn't have to be that demo. It could even be that what we build is a UI for generating pages like that. Or, it could be something totally different.

I'm sorry I said that demo was simple for me, since it seems to have clouded the intent of just getting to clarity on this feature.

> Yup, that doesn't look special case, difficult to scale (for people who want
> to manage several curation views), or simple to iterate on from the
> tinkerer's perspective.

Then, please answer my questions so we can nail down the requirements for this. What things would make this easier to scale or simple to iterate. Please be specific.

> With this API, a PR
> person could easily collect those and curate the page themselves. You think
> the same person could use your "easy" JS-based, CORS-fueled code example? I
> think not.

Again, I made a demo. It was an exploration. I'm trying to get clarity.

> Anywho, my job wasn't to fight for a feature in a bug, it was to spec one
> and get agreement from a reasonable set of stakeholders then move to
> implementation if that agreement was attained.

This is what I'm trying to get - a spec. Please help & answer questions.

> After we came to a reasonable agreement, it moved to this phase - clearly
> it is not realizing the intended outcome.

But, this is exactly what I'm trying to do. You keep saying things that this should do, and I'm asking how. What do we build? What are the parameters and what are the results? Let's nail this down.

> I simply don't have the time to outlast this filibuster and present the case
> more clearly than I already have

I'm not asking you to make a case. I'm asking you to answer questions, so we know what we're building.

> I'm fine with closing this out WONTFIX, because that's obviously where we're
> at.

No, that's not fine. We started this thing, as far as I understand, by asking what MDN needs to do to better serve DevHub. That need still exists. Maybe we need to open a different bug for that?
This is still a needed feature; folks need to relax and focus on coming up with the final requirements for it so we can actually build something.
Thank you for being honest, Daniel.

Even if we agree to mark this as WONTFIX, we need to understand what that means. We absolutely want to work with you. We absolutely want to support your efforts. We absolutely want to help you publish amazing documentation. We absolutely want to give you the opportunity to use our existing tools while still retaining control over branding and other important elements.

We just may not agree on this as the solution to these problems. And given the discussion so far, it appears we do not. But that's no excuse to stop trying. This is at worst just a crashed glider that will one day become an airplane.
(In reply to Daniel Buchner [:dbuc] from comment #31)

> You say this in your previous comment: "If RTD or github had a feature like
> this, someone on StackO could link to those sites with their favorite
> content curation/theme and I might end up on a readthedocs page without a
> navigation sidebar, or a github page without breadcrumbs. I would be so
> pissed." Wow, doesn't sound like anything a link at the top saying: "view
> MDN default" couldn't cure, next?

Also, I think you're confusing Luke & I for each other, because *I* didn't say that. I feel like you're having an argument with a version of me who's not in the room.  Seriously, I'm just trying to get some answers here.
Les: I think I have a fairly good understanding of what Daniel has in mind. If you have any questions about requirements, please bug me directly.

Let's keep this civil. We need to remember that we all have the same goals in mind here. It's just how we get there that might take some discussion.
I'm really not sure what the questions left are, the needs are as follows:

1. A 0-barrier API/mechanism that quickly enables people with little to no technical skill to curate content from MDN in a collection that serves their purpose

2. An ability for cross-product display of curated MDN content that can be styled via simple inputs to match an origin domain's stylistic needs

To this end I generated a strawman for various parameters and a rough mock of what they mapped to. David Walsh took my product description and expounded upon it technically to more specifically describe a draft API.

I'm really bewildered here, you say you want to get down to making something here, but most of what has occurred in this thread has been an assault on the premise for why we even need the requested API, and a feature-by-feature refutation of each of the items that were previously specified.

You say that: "Yes, and now we're trying to nail down the requirements" - "What do we build?" - "What are the parameters and what are the results?" --> they've already been nailed down in my product description and David Walsh's API bug: https://docs.google.com/spreadsheet/ccc?key=0ApY0XuSv-jEbdE9HRHQ4cG1ZMElsb0dJdTJNZjYxZVE#gid=0 <-- Please make something that largely resembles this, I'll trust you on the details (I'm being told I'm supposed to do that)

You want to omit one, sure, that's fine, modify something a bit, have at it - but those are the requirements. I'm not really supposed to go beyond product descriptions, mocks, and strawman API examples that show the ballpark concept of the features needed. If you have an issue with a specific feature, or one of the parameters, please list them and I will give a final answer from my perspective. From there, I will trust that we will be delivered something that, for the most part, looks like what we spec'd and the other MDN devs drafted - is this an unreasonable course?
Adding David Walsh to the party. Sounds relevant.
(In reply to John Karahalis [:openjck] from comment #36)
> Les: I think I have a fairly good understanding of what Daniel has in mind.
> If you have any questions about requirements, please bug me directly.

I do have questions, already asked here - see comment 22, comment 23, comment 24, and comment 25.

(In reply to Daniel Buchner [:dbuc] from comment #37)
> I'm really not sure what the questions left are, the needs are as follows:

I pointed to my questions - see comment 22, comment 23, comment 24, and comment 25.

I've seen the spreadsheet mock, I've seen David Walsh's JSON sketch (and even cited it), and I still have the above questions.

> If you have an issue with a
> specific feature, or one of the parameters, please list them and I will give
> a final answer from my perspective.

I did list them - see comment 22, comment 23, comment 24, and comment 25.
I am going to ask that we shift gears here. We need to do three things.

1. Take a step back. We are all getting a little worked up and need to take a break
   from this discussion. I suggest that we let this go for now and continue the
   discussion in a face-to-face meeting after we (and our ideas) have settled down.
2. Decide: Do we want to build the feature described in comment 0? We have discussed
   it enough already. We do not need any more arguments about whether the idea is
   perfect. We need a decision on whether it is what we want to build.
3. If we decide not to build this, we need to decide what we will build instead to
   solve these problems.

Look for a meeting request momentarily.
We should table this for now because it's not a top priority.  So yes, meet about it but schedule it for 2 weeks from now after MWC.  Invite me to the meeting.

I want to talk about this in relation to:
https://www.dropbox.com/s/itscbk8a9r7kb4q/docs.png

We should also consider the why and who -- because I don't think we have clarity on that across the board.
I'm glad to see there there is already a discussion going about this. I met with Tony Santos and Bram on Monday to discuss what seems to be a parallel effort to improve developer resources. It also is great timing and a huge benefit to both of our teams that we are working on this at the same time. I showed them the phased approach to the MDN redesign and they had been thinking of a very similar approach to DevHub. Even though MDN and DevHub may have different marketing strategies, we have very similar goals. 



Regarding the following comment from John:
"One open question is whether the content these groups target should look different to all readers (including those just stumbling across the documentation on MDN) or whether the documentation should only look different to readers under certain circumstances (for example, when the reader is explicitly linked to the re-styled documentation)."

I agree with Daniel's response to this. The way I envision it (let me know if I'm on track with this) is that all documentation would live and be editable on MDN. The question is, do we send users from DevHub to MDN to view docs, or do we allow MDN content to be displayed (and not editable) on DevHub. 

All documentation and template styles on MDN should have an expected style, readability, and taxonomy for the users. If we need to indicate that a doc is Apps specific, I don't see an issue with that, but the look and feel of all docs should be consistent.



How can we work together? 
I think we can work together to make sure that our efforts are complementing eachother and that we don't have the kind of overlap that will confuse users and cause our developer community to live in 2 places while not be aware of one another. It's a benefit for both of us if our audience knows the purpose of MDN vs. DevHub. Let's get this defined internally asap. 

I am also looking forward to working on a docs style that is unique to Mozilla. We can learn from the work on SUMO in making a sub-Sandstone style that is better for readability. 

Looking forward to working together on this and talking in person to clear up some of the questions in this thread and step back to define our goals at a little higher level before diving into decisions about the API.
It sounds like these ideas have been incorporated into and superseded by bug 861991. Please reopen if we still intend to implement the feature described in comment 0 in addition to bug 861991.
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → WONTFIX
Product: developer.mozilla.org → developer.mozilla.org Graveyard
You need to log in before you can comment on or make changes to this bug.