308674 - Monitor and implement the OpenSearch 1.1 standard

Status: RESOLVED WORKSFORME

(Firefox :: Search, enhancement)

Description

[Michael Ward]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.7.10) Gecko/20050717 Firefox/1.0.6
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-GB; rv:1.7.10) Gecko/20050717 Firefox/1.0.6

Microsoft Internet Explorer 7 will support the OpenSearch 1.1 specification for
hooking into its integrated web search function (
http://opensearch.a9.com/spec/opensearchdescription/1.1/ ).

Although the specification is still draft, I think it would be prudent for
Mozilla to monitor the progress of the specification and to implement it in
Firefox when it becomes final.

Here is the overview of the specification from the specification document:

"OpenSearch™ Description files are used by search engines to describe what kinds
of content they index, how they can be queried, and how they can return search
results. OpenSearch Description documents can be used to integrate third-party
searches into search clients using the OpenSearch Query Syntax. These documents
are published as simple XML files over the web. Search results are usually
enhanced with the OpenSearch Response extensions."

Reproducible: Always

Steps to Reproduce:
n/a
Actual Results:  
n/a

Expected Results:  
n/a

Comment 1

[Peter van der Woude [:Peter6]]

see http://www.mozdex.com/developer/
and http://linux.mikeasoft.com/mozdex.php

Comment 2

[O. Atsushi (Torisugari)]

http://opensearch.a9.com/spec/opensearchquerysyntax/1.1/#parameter-syntax

>   {startIndex} – The offset of the first search result, starting with one.
>   Restrictions: A positive integer.
>   Default: "1"


Google begin with 0
http://www.google.com/search?q=foo&start=0
http://www.google.com/search?q=foo&start=10
http://www.google.com/search?q=foo&start=20

Yahoo begin with 1
http://search.yahoo.com/search?p=foo&b=1
http://search.yahoo.com/search?p=foo&b=11
http://search.yahoo.com/search?p=foo&b=21

So, "A positive integer" is ok for Yahoo, but it does not work for Google.
I'm afraid it's not a service-provider neutral format...

Comment 3

[Michael Fagan]

The intention is that search provides will be able to provide either a 1-based startIndex or 1-based startPage. If they are currently 0-based, it's such a minor change/addition for search engines to implement.

Failing that, a second possiblity is to create an extension for a new search parameter (eg 'ext:startIndexZero'), and support that.

The goal of OpenSearch *is* to be search-provider neutral.

(In reply to comment #2)
> http://opensearch.a9.com/spec/opensearchquerysyntax/1.1/#parameter-syntax
> 
> >   {startIndex} – The offset of the first search result, starting with one.
> >   Restrictions: A positive integer.
> >   Default: "1"
> 
> 
> Google begin with 0
> http://www.google.com/search?q=foo&start=0
> http://www.google.com/search?q=foo&start=10
> http://www.google.com/search?q=foo&start=20
> 
> Yahoo begin with 1
> http://search.yahoo.com/search?p=foo&b=1
> http://search.yahoo.com/search?p=foo&b=11
> http://search.yahoo.com/search?p=foo&b=21
> 
> So, "A positive integer" is ok for Yahoo, but it does not work for Google.
> I'm afraid it's not a service-provider neutral format...

Comment 4

[O. Atsushi (Torisugari)]

(In reply to comment #3)
> The intention is that search provides will be able to provide either a 1-based
> startIndex or 1-based startPage. If they are currently 0-based, it's such a
> minor change/addition for search engines to implement.

Are you serious? If the client *adjust* it to 0, then yahoo does not work.
If the client *adjust* it to 1, google doesn't work. I'm talking about
that structual defect, not only about google nor yahoo.
 
> Failing that, a second possiblity is to create an extension for a new search
> parameter (eg 'ext:startIndexZero'), and support that.

lol. An extension is not a patch.

> The goal of OpenSearch *is* to be search-provider neutral.

I apologize if you got offended at my words. I really appriciate the fact
that it's a xml-base format.

Comment 5

[Michael Fagan]

I wasn't offended, merely trying to clarify.

Anyhow I didn't mean for the client to change, I meant for the search engines to change to accommodate the format. Today some use 1-based, some use 0-based. Nevertheless, I think having two start settings is plenty - four would be overkill.

Do inputnext and inputprev not have similar problems with the current plugin model?

I'm not sure what you meant by 'an extension is not a patch'

(In reply to comment #4)
> Are you serious? If the client *adjust* it to 0, then yahoo does not work.
> If the client *adjust* it to 1, google doesn't work. I'm talking about
> that structual defect, not only about google nor yahoo.

> lol. An extension is not a patch.

> I apologize if you got offended at my words. I really appriciate the fact
> that it's a xml-base format.

Comment 6

[O. Atsushi (Torisugari)]

(In reply to comment #5)

> Anyhow I didn't mean for the client to change, I meant for the search
> engines to change to accommodate the format.

That is, "Hey guys, I created a new format!! Follow me."

I'd like to call it "non-neutral". I wonder why you are so true to
OpenSearch. Is there any reason we can't request them to modify the
draft spec? I guess OpenSearch would have never been discussed in
such a *open* way (it's a proprietary project/format, though I don't
mean to say the goal is nothing but to get money).

My point is fairly simple. Just support "default" attribute.
e.g.
<SearchIndex default="0" factor="10" />

> Do inputnext and inputprev not have similar problems with
> the current plugin model?

It has exactly the same problems, of course.

> I'm not sure what you meant by 'an extension is not a patch'

In theory, an extension will fix a security hole and maybe lung cancer
in the end. Don't rely on such a workaround till the day before branch
freeze. That must be painful.

Comment 7

[Wesley Johnston (:wesj)]

My complete lack of knowledge on how this will be used is going to come out here. 

Someone can write and ask them to change the spec though. I'd be surprised if somewhere there isn't watching this bug. What's wrong with just allowing 0 as a value for StartIndex though? Here's their feedback page if someone really wants to write: http://a9.com/-/feedback/A9

Does it even matter? Honestly, when is a Google user going to be doing a search and want to see entries starting at Index 0? A non-entered value starts you at the first result anyway, right? So why does anyone care? The search bar has no feature to allow you to specify the start index anyway that I know of. That's where this would be used as far as I can tell.

IE7 is adding support for this through a window.external.AddSearchProvider(“URL”) call. I think it's likely that a ton of search sites will support it, all with little "Add me to the SearchBar!" links on them that will not work in FF. I don't care if the whole thing is implemented, but it would be nice if enough of it was available that FF didn't look like a goon.

Comment 8

[Michael Fagan]

Replying to the previous two comments here...

Yes, A9.com is absolutely open to suggestions. The reason there isn't a mailing list yet, or anything like that, is probably because as a new format, the interest is only just approaching enough people for that. OS 1.1 is creative commons licensed, btw.

Regardless, anything said here is being listened to at A9 (where I work right now), but e-mailing via the feedback link works too.

One thing to think about is that with the 1.1 format, the goal was not to make too many differences with 1.0 and confuse people. However if there is enough need/want for a change, it will happen. Draft 2 of the specification will be out shortly, although not with any significant changes to the OS Description.

In terms of semantics, <SearchIndex default="0" factor="10" /> won't work with the current model, as the actual search parameters are set within a string, not in XML (for GET, that is). "Factor" is handled by a search parameter called "count".

Anothing thing to think about, is that OpenSearch is a set of *three* specifications, *two* of which together, are roughly equivalent to a Mycroft plugin. The third is OpenSearch Response, which works rather better than using an <interpret> section to scrape HTML. The OpenSearch Description document can specify multiple types of responses actually, such as HTML, RSS, whatever.

One change that the OpenSearch specs will mention shortly is *autodiscovery* (just like RSS, Atom, and others have now), which IMHO is great.

Comment 9

[Michael Fagan]

Oh, one thing I missed. wesley-johnston mentions that people are only searching the 1st page of results anyways. That's true today, but the Mozilla project used to have (and now is an extension) a way to paginate. So it is an issue to some extent. Personally I like it and have the extension installed http://searchsidebar.mozdev.org/

Comment 10

[Axel Hecht [:Pike]]

Hrm. There is a mailing list now, http://opensearch.org/mailman/listinfo/discuss
and according to DeWitt, 1.1 is approaching quickly.

Seems like 1.1 won't be ready to deal with 0 based search engines, does that
kick OpenSearch for FX2?

Comment 11

[Michael Fagan]

> Seems like 1.1 won't be ready to deal with 0 based search engines, does that
> kick OpenSearch for FX2?

I really don't think so. When engines realize they can become part of the search box in IE and Firefox, added to A9.com and other aggregators, etc., most will just implement the appropriate start index themselves. Bear in mind that any engine could simple not make use of startIndex/startPage, and everything would still work, except for pagination.

Comment 12

[DeWitt Clinton]

Hi -- sorry, just catching up on this thread.  I apologize for not jumping in sooner.

First, OpenSearch is definitely "open" (CC-licensed, extensible, public discussions, etc.).  If anyone has ideas on how to make it more open, please let me/us/the whole world know.  The point really is to foster search syndication, community, and interoperability.

Second, regarding the whole start-index question, it wasn't so much an issue of  being engine neutral, it was more a matter of trying to balance the simplicity of the specification.  In this particular case, it appeared simpler to ask the search engine, which would already have some awareness of being "OpenSearch-ed", do the simple arithmetic of adjusting 1 to 0 if need be.  Rather than the alternative of asking the client to keep track of that adjustment on a per-engine/per-query basis.  Perhaps this bias was wrong, but we argued back and forth both ways on it, and ultimately decided that this (minor) burden best belonged on the engine, considering it is trivial compared to everything else the search engine does.

Again, please ping me personally, or our mailing list (Michael added the link above), if you have any questions or ideas about how to improve the format.

For the record, I'd love to see OpenSearch in Firefox -- it would be amazing to have a common format adopted between the search engines and the browsers.

Comment 13

[:Gavin Sharp [email: [e-mail]]]

The OpenSearch support in Firefox 2 is far from perfect, but I'm not sure that such a general bug about "monitoring and implementing" OpenSearch has any use, so I'm going to resolve this WORKSFORME. Specific issues with Firefox's OpenSearch support should be filed separately, if they're not already filed.