488429 - Platform-specific SHOWFOR directives (for menu locations/names) show up in search-results' summaries

Status: VERIFIED FIXED

(support.mozilla.org :: General, defect)

Description

[Stephen Donner [:stephend] Not actively reading bugmail]

Summary: Platform-specific SHOWFOR directives (for menu locations/names) show up in search-results' summaries

STR:

1. Load https://support-stage.mozilla.org/tiki-newsearch.php?locale=en&q=options+window&where=d&sa=&filter_lang=1&l=en&en_too=1&lastmodif=0&type=0&author= while logged in
2. Look at the first result

Actual Result:

"Options window

This document explains all of the optionspreferences available in the OptionsPreferences window of Mozilla Firefox. To access the optionspreferences window, At the top of the Firefox windowOn the me..."

Expected Results:

"optionspreferences" should not be together, since those are probably SHOWFOR platform-specific references (Tools -> Options on Windows, Edit -> Preferences on Mac/Linux).

Is it possible to use SHOWFOR detection in search-results summaries?  Or something similar?

Comment 1

[Paul Craciunoiu [:paulc]]

It is theoretically possible to implement SHOWFOR detection in search, but it probably needs a bit of rewriting. We'd also need to discuss how it integrates graphically to not take too much room, etc. Laura and/or David should probably decide what we should go for.

Comment 2

[E. Cooper [:ecooper]]

SHOWFOR is being parsed, but HTML tags are stripped from the text (for sphinx and for the description text).

Something would have to happen pre-parsing during indexing to 'fix' this.

Comment 3

[Laura Thomson :laura]

The temporary fix is to show Firefox 3 / Windows in the summaries.

Cww will provide the number of affected articles.

Comment 4

[Laura Thomson :laura]

I think this bug has the same root cause as bug 489441.

Comment 5

[E. Cooper [:ecooper]]

Okay, after banging my head against my keyboard for most of the night, this is where we're at with this.

As mentioned in comment 2, html tags are stripped from the articles after they're parsed for wiki text and transformed into html. Without html tags, showfor is just a bunch of words strung together.

The above happens around line 109 of /scripts/sphinx/indexer.php.

I looked into maybe interrupting the parsing process, but that was a bit ridiculous. I tried rolling out my own set of regexs to pre-parse the article data and remove unneed showfors, but that was error prone. The cleanest way so far is to insert some DOM handling code after it's parsed but before the tags are stripped.

Using DOM, we can strip out 'noWin', 'mac', 'linux' spans/divs very easily, eliminating half of the problem. The other half, the showfor content blocks, are another matter. They don't have any sort of identifying characteristics, really once it's parsed into html. They have an id of 'wikiplugin_showforSOMENUMBER" and a class of 'showfor_contents'.

So, suggestions? Does the problem and it's cause make sense? Or am I being crazy?

Comment 6

[E. Cooper [:ecooper]]

Attachment: v1

This patch also fixes bug 489441.

Here's how it works: During indexing, every wiki page is parsed for wiki text and becomes an html document. After this happens, using xpath and dom, three different sets of xpaths are queried and the matching elements are modified.

The first set removes elements with the class of 'showfor_label' and 'showfor_footer' since they are usless for searching.

The second set goes and adds a space between content blocks so they aren't concatenated when tags are stripped, addressing the problem with bug 489411.

The third set is for the description text, and removes any OS showfor content block that isn't 'win'. This *does not* remove all showfor browser content blocks as no discernable means exist to identify them (see comment 5). This, however, shouldn't be a problem because I have yet to see one within a description.

So, that's how it works. DOM and xpath seem to be enabled on staging and prod, so that shouldn't be a problem. <https://support-stage.mozilla.org/tiki-phpinfo.php>

Locally, this appears to take care of the problem. If anyone can think of a better route, please let me know.

Comment 7

[E. Cooper [:ecooper]]

r24713/r24714

Comment 8

[E. Cooper [:ecooper]]

Attachment: v2

This patch is an alternative to the first one. No DOM or XPath to handler showfor content blocks, but just some regexs.

This only works for OS showfor content blocks as there's only so much you can do with a regex without writing a full-blown html parser. Fortunately, the OS showfor content blocks cause the most trouble and this patch should address this bug.

Comment 9

[Laura Thomson :laura]

Good stuff, commit at will.

Comment 10

[E. Cooper [:ecooper]]

r24788/r24789

Comment 11

[Stephen Donner [:stephend] Not actively reading bugmail]

Eric -- reindexing has been run; is this now considered fixed?  I'd like to start testing...

Comment 12

[Stephen Donner [:stephend] Not actively reading bugmail]

Verified FIXED on https://support-stage.mozilla.org/tiki-newsearch.php?locale=en&q=options+window&where=d&sa=&filter_lang=1&l=en&en_too=1&lastmodif=0&type=0&author; I'll keep testing too :-)

(Tested on both Windows and Mac.)