Closed Bug 501880 Opened 10 years ago Closed 10 years ago

[ForumUX] Advanced forum and KB search with more options

Categories

(support.mozilla.org :: Search, defect, P1, blocker)

defect

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: paulc, Assigned: paulc)

References

()

Details

(Whiteboard: sumo_only)

Attachments

(8 files, 5 obsolete files)

As per https://wiki.mozilla.org/Support/ForumUX/PRD contributors should be able to search the forums using a more advanced search options.

Contributors: this would be the place where you start mentioning options you'd like to see.
Implementation-wise, my proposal is to add these options on the tiki-newsearch.php page, for contributors (=logged-in users), and, if needed, modify the newsearch indexer to account for new fields.
Priority: -- → P1
Summary: Allow more search options for contributors (not just solved threads) → [ForumUX] Allow more search options for contributors (not just solved threads)
Target Milestone: --- → 1.4
Cheng, what would be some needed search options? If you could prioritize them that would be great.
* Post status/type (solved, sticky, etc)
* Last post date (show posts later than / before / between dates)
* Parent post date
* # of replies
* author (already done; is it in the UI?)

Others?
The most important ones are:

Post status: Combo box: solved, proposed, requesting more info, open (status = n or last reply is a user), no replies. (most important)

Posted by user:

Sort results by: (dropdown): relevance, last post date, original post date, # of replies

Posted after: (original post date)

The rest aren't that important IMO.

We should also have
I've been working on this today. I will implement the functionality with a first-draft UI, and then we can adjust the UI (either for this milestone or a future one).

I will focus on Cheng's requirements for forum searches. It would be good to also think about KB searches (although those already have implemented the author, and category, for example).

I will file a patch late this week or early next that rewrites a good deal of the current newsearch page to make it easier to maintain, etc.

I'd rather make this bug about p10n applied to the newsearch page and shoot it for 1.3 instead.
(In reply to comment #4)
Also, the rewrite aspect deals with the implementation -- the page will look the same (aside from the search options UI).
So, I have newsearch mostly rewritten (and I must say, it looks much better!) but I would rather wait on the UI input from Neil until I post anything.
Blocks: 502009
Fields needed:     
*  Forum search
                + Search terms
                + thread status (pick one or many)
                + post author
                + tags
                + created/last posted after date (or in the last XXX)
                + sort by (relevance, date of last post, date of original post) 
* Advanced KB search
                + Search terms
                + language (dropdown)
                + category (Firefox 3, 3.5, Help, troubleshooting) should be able to select multiple
                + tags 
    * We can merge forum and kb search on one page but they have different options so we need to be very clear on what they're searching for.
Just so there's something to look at and comment on in advance of Thursday's meeting, here's what I've got so far. Let me know if you think this is moving in the right direction:

Rejigged "normal" search box with updated text, new advanced search link
http://snaps.beatnikpad.com/sumo-normalsearch-20090810-222618.png

Advanced search page (set to "forum" options):
http://snaps.beatnikpad.com/sumo-advsearch-20090810-222920.png

I still need to do the KB search options and could use the final list of options for each of these (such as what thread states there are, language options, etc.) so I know what to design for.
Thread statuses are:
* Thread has no replies
* Thread is waiting on more info from user (requesting more info)
* Thread contains a proposed solution
* Thread is solved
* The user has replied in this thread

We'd possibly add more but those are the main usecases

language options: we have 30+ languages on SUMO so you should be able to search in just one language or multiple.


As far as comments, will the text in the right box change as you focus on different fields?
> As far as comments, will the text in the right box change as you focus on
different fields?

It could, if what you wanted was contextual help -- my original thinking was it'd be a "how do I actually *use* this to find something?". I find most advanced search pages just dump the fields in front of the user without much help, and seeing how some people might not know how to really take advantage of these options, a bit of in-page help might be useful.
Duplicate of this bug: 510194
Summary: [ForumUX] Allow more search options for contributors (not just solved threads) → [ForumUX] Advanced forum and KB search with more options
Assignee: paul.craciunoiu → neilio
Here's the advanced search mockup with all of the forum advanced fields in place:

http://snaps.beatnikpad.com/advancedsearch-20090821-151525.png

The Knowledge base is an identical design except with those fields swapped in.
Blocks: 505425
I'd like to see this finalized, but I've got a head start on the implementation already.
Attached image Search design notes
Here are the design notes for the search design - as discussed, I'd like us to use box-shadow and border-radius for the drop shadow and rounded corners. The button uses the same bg graphic and design as the "ask a question" buttons.
I think that's everything from Neil.  Reassigning to Paul for implementation.
Assignee: neilio → paul.craciunoiu
Blocks: 513703
Depends on: 514479
I'm adding all bugs that will be fixed by this as blocked by it.
No longer depends on: 514479
Blocks: 514501
The patch is ready, I'm waiting for the merge to see if there are any conflicts. We should do the merge first so I can update this patch accordingly.
Blocks: 514820
No longer blocks: 502009
No longer blocks: 514820
Attached patch giant patch, v1 (obsolete) — Splinter Review
Sorry. This is really a huge patch. Listing details here:

-- List of files it adds:
webroot/tiki-newsearch_advanced.php
webroot/tiki-newsearch.php
webroot/images/search
webroot/lib/search/sphinxlib.php
webroot/lib/cache/memcachelib.php
webroot/styles/jui/smoothness/jquery-ui-1.7.1.custom.css (moved to webroot/styles/jquery-ui-1.7.1.custom.css)
webroot/styles/mozcommon.css
webroot/styles/mozsadvanced.css
webroot/templates/tiki-admin_newsearch.tpl
webroot/templates/tiki-newsearch_advanced.tpl
webroot/templates/tiki-newsearch.tpl
scripts/sphinx/search.conf.php.dist
scripts/sphinx/indexer.php
scripts/sphinx/indexer-forums.php
scripts/minify/minify.conf.php.dist
webroot/js/mozsadvanced.js
-- Also add images in:
webroot/images/search (attached in next comment)

After applying the patch:
* Reindex Sphinx, both KB and forums
* Run these:
cp scripts/minify/minify.conf.php.dist scripts/minify/minify.conf.php
cp scripts/sphinx/search.conf.php.dist scripts/sphinx/search.conf.php
* Run the attached SQL (to add the permission)
* Copy the images over from the next attachment

Explanation of changes:
I'm sure I'll miss stuff. Here we go...
* Fixes all the search bugs dependent on this bug
* Rewrites sphinxlib, tiki-newsearch almost entirely
* Adds required XML tags to indexer and indexer-forums
* Removes "session" cache from admin panel, as it is not yet implemented
* Updates search.conf.php.dist with lots of constants used in search. Also crucial to make search productizable
* Adds a useful memcache function to build a cache key from the $_GET array
* Updates the paths in jquery-ui-1.7.1.custom.css, the images wouldn't be showing up through minify (relative paths become incorrect)
* Updates minify.conf.php.dist with the new css files
Attachment #400137 - Flags: review?(laura)
Attachment #400137 - Flags: review?(jsocol)
Attached patch permission SQLSplinter Review
Attachment #400141 - Flags: review?(laura)
I forgot: the final step in the indexing consist of running /usr/local/sphinx/bin/indexer
This is the sphinx.conf that goes with the updated indexer.php and indexer-forums.php
Attachment #400143 - Flags: review?(laura)
Attachment #400143 - Flags: review?(jsocol)
If permissions are not set for advanced search, go to tiki-admingroups.php to set them.
Attachment #400137 - Flags: review?(jsocol)
Attachment #400143 - Flags: review?(jsocol)
Attachment #400141 - Flags: review?(laura) → review+
Comment on attachment 400143 [details] [diff] [review]
updated sphinx.conf

Would have preferred to see it as a patch but this is ok for me
Attachment #400143 - Flags: review?(laura) → review+
Comment on attachment 400137 [details] [diff] [review]
giant patch, v1

Took a long time to go through all this! Looks really good but I'm sure I haven't tested it out enough.  QA will need to do a lot on this one.
Attachment #400137 - Flags: review?(laura) → review+
r51742 / r51743
Status: NEW → RESOLVED
Closed: 10 years ago
Resolution: --- → FIXED
Reopening; https://support-stage.mozilla.org/tiki-admin_newsearch.php is blank right now -- James has a fix.
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attached patch extra patchSplinter Review
patch to the giant patch, fixes the blank admin page
r51801 / r51802
Status: REOPENED → RESOLVED
Closed: 10 years ago10 years ago
Resolution: --- → FIXED
Backed out in r51894
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Attached patch indexer patch (obsolete) — Splinter Review
I was testing locally and some stuff lurked in. The "LIMIT 2500" in particular would be a problem on production.
Attachment #401642 - Flags: review?(laura)
Attachment #401642 - Flags: review?(james)
I was setting up a fresh SUMO, and noticed another issue with a dump from Aug 13. If no search results show up, it can be because of the weights. Advanced search removes these fields:
* polls (pollyes, pollcoeff)
* answered
These fields need to be removed from the weights at tiki-admin_newsearch.php
This is backed out, moving to 1.4.1
Target Milestone: 1.4 → 1.4.1
Further backed out in r52135
Blocks: 438360
Target Milestone: 1.4.1 → 1.5
Comment on attachment 401642 [details] [diff] [review]
indexer patch

Just going through my r? list
Attachment #401642 - Flags: review?(laura)
No longer blocks: 512894
Attachment #401642 - Flags: review?(james)
Search branch now has basic search frontend code committed. Patch attached.
r55392 (branches/search)
Attachment #400137 - Attachment is obsolete: true
Attachment #401642 - Attachment is obsolete: true
I should add that the (basic) search will now be at /search.php, and the patch above updates all templates accordingly. We are left to update the sidebar modules.

James will take care of updating the sphinx.conf file and setting up search branch to work with sphinx so we can test.
r55847 (search branch)
This removes calls to Google translate and shows result counts separately for current locale versus default (English). Patch attached for reference.
Component: Forum → Search
Attached patch advanced search, v1 (obsolete) — Splinter Review
This patch adds advanced search to the same file.

While doing this, I also noticed that searching for "firefox" suggests "Firefox" on my machine. Maybe we should ignore case?
Attachment #412340 - Flags: review?(james)
A few notes on the patch:
* It does not yet add search for *any* poster in a thread -- just the original author
* I think we also mentioned adding some other filters?

I only made the original stuff (that was in the mockups) work. So James, you can either make me add the other filters in this same patch, or we can land this and then add more :)
Attachment #412340 - Flags: review?(james) → review-
Comment on attachment 412340 [details] [diff] [review]
advanced search, v1

The form looks great, and seems to be working fine, but...

- Including the form on the results page pushes the results below the fold. A better behavior is to add an "Edit your search" link below the search/refine form (bug 527859) on advanced search. (IE: refining your search should persist the options you selected, clicking "Edit your search" should take you back to the advanced form with your choices pre-made.)

- The language on the KB search defaults to Arabic if no locale is set (eg: /search.php?advanced=1) English is preferable as a fallback.

- Searching the forums still results in "0 results in and ### results in English". I don't know if that's in scope of this bug or not, but thought I'd mention it.

- 1/3 of this seems unnecessary:
+    $smarty->assign('title', tra("Advanced Search"));
+    $smarty->assign('page', tra("Advanced Search"));
+    $smarty->assign('current_lang', $search_parameters['lang']);
+    $smarty->assign('current_locale', $search_parameters['locale']);
+    $smarty->assign('title', tra("Advanced Search"));
+    $smarty->assign('page', tra("Advanced Search"));
All good points. I will do the simple search form Friday and finish this up after.
This is an absolute must before we can push search. Please prioritize this.
Severity: normal → blocker
Depends on: 527859
I'm almost done with this, fixed everything except I'm waiting to integrate it with the form in bug 527859 (e.g. I also have a ready link for "refine your search" that toggles the advanced form to make it visible). I'd rather wait for bug 527859 to file the patch, but let me know if you want me to file it anyway.
I will get a patch to bug 527859 first thing in the morning, so you can hold off til then.
Attached patch v2 (obsolete) — Splinter Review
This should do it.
Attachment #412340 - Attachment is obsolete: true
Attachment #413493 - Flags: review?(james)
Comment on attachment 413493 [details] [diff] [review]
v2

Woah, not sure what happened here. After applying the patch, going to search.php with no parameters tells me there are 1300 results in English (and lists them) and adding ?advanced=1 doesn't seem to do anything.
Attachment #413493 - Flags: review?(james) → review-
Attached patch v3, no search on empty query (obsolete) — Splinter Review
Attachment #413493 - Attachment is obsolete: true
Attachment #413513 - Flags: review?(james)
Attached patch v4Splinter Review
This should have all the fixes as we discussed today. Outline of the behavior:
* No search if query is empty, and replaced with slightly larger search form.
** Exception: in advanced search, if filters are set (all filters count except article language and forum results sorting.
* Added "advanced search" link for logged-in users, which takes you to the advanced form.
* After searching by using the advanced form, you can refine your search by clicking "refine search" and you are again taken to the advanced form, with the values pre-filled.

From a dev/QA standpoint, these URL parameters determine behavior:
* refine=1, advanced form shows up, for refining results; no search results shown
* advanced=1, use advanced search for the given parameters
* if both are present, refine takes predecence (i.e. advanced=1 makes no difference)
Attachment #413513 - Attachment is obsolete: true
Attachment #413719 - Flags: review?(james)
Attachment #413513 - Flags: review?(james)
Blocks: 530139
Blocks: 530143
Comment on attachment 413719 [details] [diff] [review]
v4

Woo, ok!
Attachment #413719 - Flags: review?(james) → review+
r56619
Finally!
Status: REOPENED → RESOLVED
Closed: 10 years ago10 years ago
Flags: in-testsuite?
Flags: in-litmus?
Resolution: --- → FIXED
Blocks: 530227
Whiteboard: sumo_only
Verified FIXED; we've verified all associated bugs, as well as done ad-hoc testing.  Will continue to file new issues as we discover them.
Status: RESOLVED → VERIFIED
You need to log in before you can comment on or make changes to this bug.