Allow "application/json" as well as "application/x-suggestions+json" as MIME types for Suggestions in OpenSearch

RESOLVED FIXED in Firefox 63

Status

()

defect
P2
normal
RESOLVED FIXED
a year ago
7 months ago

People

(Reporter: feluchs, Assigned: standard8)

Tracking

({dev-doc-complete})

58 Branch
Firefox 63
Points:
---

Firefox Tracking Flags

(firefox63 fixed)

Details

(Whiteboard: [fxsearch])

Attachments

(1 attachment)

(Reporter)

Description

a year ago
User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

Steps to reproduce:

add following line to opensearch document; referenced in root html head section
<Url type="application/json" rel="suggestions" ...></Url>


Actual results:

firefox doese not allow such search engine; and if it doese it doese not use suggestion service; firefox expects "application/x-suggestions+json"


Expected results:

As  can be seen here: http://www.opensearch.org/Specifications/OpenSearch/1.1#Url_rel_values
under the "The "Url" element" headline. Multiple Examples for implementation "suggestion" are given. Look at "Example of a Url element describing a request for search suggestions over JSON:".
while reference is showing type="application/json" ... firefox doese not except it. Firefox doese except => application/x-suggestions+json. In my opinion, this is counterproductive. Not only is this not an accepted IANA standard, it is also not compliant with the opensearch standard.

Also: Could not verify that installed opensearch search engines are updated when source xml on server is changed. Server logs dont show any request after initial fetch!

Updated

a year ago
Component: Untriaged → Search
(Assignee)

Comment 1

a year ago
Having read through the specification, and the suggestions extension (http://www.opensearch.org/Specifications/OpenSearch/Extensions/Suggestions), I believe the main specification's example is wrong.

The suggestions specification explicitly references the response type:

```
The following type is used to indicate that the response will contain search suggestions:

    application/x-suggestions+json 
```

The main specification gives the example of 'application/json' as you say, but I believe that is wrong and definitely conflicts with the definition in the suggestions spec.

I'm trying to report this back to opensearch, but their website won't allow me to log in, so I'm trying via the email list.
Status: UNCONFIRMED → RESOLVED
Last Resolved: a year ago
Resolution: --- → INVALID
(Reporter)

Comment 2

a year ago
(In reply to Mark Banner (:standard8) from comment #1)
> Having read through the specification, and the suggestions extension
> (http://www.opensearch.org/Specifications/OpenSearch/Extensions/Suggestions),
> I believe the main specification's example is wrong.
> 
> The suggestions specification explicitly references the response type:
> 
> ```
> The following type is used to indicate that the response will contain search
> suggestions:
> 
>     application/x-suggestions+json 
> ```
> 
> The main specification gives the example of 'application/json' as you say,
> but I believe that is wrong and definitely conflicts with the definition in
> the suggestions spec.
> 
> I'm trying to report this back to opensearch, but their website won't allow
> me to log in, so I'm trying via the email list.

Hi Mark,

were you successful in the attempt to report this bug to the 'opensearch' website? Did you get any new information whether the 'type' could/should be set to 'application/json'?

feluchs
(Assignee)

Comment 3

a year ago
There was a discussion here, although action seems to have stalled a bit...

https://groups.google.com/d/msg/opensearch/24v1tPcrVe8/5nyYJ6rECQAJ
(Assignee)

Comment 4

a year ago
They've now moved the site: https://groups.google.com/d/msg/opensearch/oUnQz1X3Ugk/MjMuyg3DCAAJ

I've submitted a PR for the correction:

https://github.com/dewitt/opensearch/pull/1

Note: I probably won't update this bug any more. Please follow along there.
(Reporter)

Comment 5

a year ago
great. thanks for all your help.
(Assignee)

Comment 6

10 months ago
Following along from the PR, we've had a few discussions and we're proposing to support both. There's an intent to ship here:

https://groups.google.com/d/msg/mozilla.dev.platform/mlNx35Xjw0U/AOVtEv41BgAJ
Assignee: nobody → standard8
Status: RESOLVED → REOPENED
Ever confirmed: true
Priority: -- → P2
Resolution: INVALID → ---
Summary: incorrect implementation of opensearch → Allow "application/json" as well as "application/x-suggestions+json" as MIME types for Suggestions in OpenSearch
Whiteboard: [fxsearch]
(Assignee)

Updated

10 months ago
Status: REOPENED → ASSIGNED
Keywords: dev-doc-needed
(Assignee)

Comment 7

10 months ago
Eric, what needs documenting here? As far as I know we don't have anything referencing or describing OpenSearch on developer.mozilla.org. We will try and get the OpenSearch specs updated when we do this though...
Flags: needinfo?(eshepherd)
We do, in fact, but the default search options on MDN leave it out because none of its tags are included in the set of tags that show up in search results by default. I have updated MDN's settings (to create a new "Web Standards" tag that will show up in search by default) and have added that tag to our OpenSearch page at https://developer.mozilla.org/en-US/docs/Web/OpenSearch. This change needs to propagate through our CDN but should start showing up soon.
Flags: needinfo?(eshepherd)
(Assignee)

Comment 9

10 months ago
(In reply to Eric Shepherd [:sheppy] from comment #8)
> We do, in fact, but the default search options on MDN leave it out because
> none of its tags are included in the set of tags that show up in search
> results by default. I have updated MDN's settings (to create a new "Web
> Standards" tag that will show up in search by default) and have added that
> tag to our OpenSearch page at
> https://developer.mozilla.org/en-US/docs/Web/OpenSearch. This change needs
> to propagate through our CDN but should start showing up soon.

Great, thank you!
(Reporter)

Comment 10

10 months ago
Hi Mark and Eric,
I think it's good that you take care of this.

However, I believe that a greater acceptance range of MIME types is not a good solution, even though the motives are clear to me. This approach leads in my opinion only to softening of existing standards. I would argue that we should drop 'application/x-suggestions+json' completely in favor of 'application/json'. Introducing 'application/x-suggestions+json' was already a mistake.

[+] Format/Schema used to structure server provided suggestions is valid JSON (https://www.json.org/json-de.html)
[+] IETF RFC6648 is deprecating MIME types with prefix 'x-' (https://tools.ietf.org/html/rfc6648)
[+] 'application/x-suggestions+json' would imply that code is execute on the server (https://pagerangers.com/seo-handbuch/onpage/allgemein/was-ist-ein-mime-typ)
[+] Clearer format announcement
[-] Maybe missing compatible server implementations

Alternatively, a motion could be submitted for 'application/suggestions+json', as this is apparently not yet occupied (https://www.iana.org/assignments/media-types/media-types.xhtml). Personally, I do not consider this solution desirable.

[+] MIME type with fixed defined schema for search suggestions
[+] Recognizability of the content without a context
[-] A further MIME standard without real gain over 'application/json'
[-] Additional creation effort and maintenance needed
[-] Compatibility issues with browsers already expecting 'application/json' today
(Assignee)

Comment 11

10 months ago
(In reply to feluchs from comment #10)
> However, I believe that a greater acceptance range of MIME types is not a
> good solution, even though the motives are clear to me. This approach leads
> in my opinion only to softening of existing standards. I would argue that we
> should drop 'application/x-suggestions+json' completely in favor of
> 'application/json'. Introducing 'application/x-suggestions+json' was already
> a mistake.

From a site compatibility viewpoint, we need to support both on the Firefox side. If we switched to application/json, we'd break compatibility with many sites straight away.

For the specification itself, we can propose that application/json becomes the new standard with application/x-suggestions+json being marked as deprecated.
(Assignee)

Comment 13

8 months ago
feluchs: I think the patch is working correctly, though I've a little more testing to do.

Could you check one of the builds and see if it works for you? They are based on the latest Firefox nightly, so you might want to set up a new profile to test with.

Linux (64bit): https://queue.taskcluster.net/v1/task/SmLLJgyqS7KPAC3m02_lKQ/runs/0/artifacts/public/build/target.tar.bz2
Mac: https://queue.taskcluster.net/v1/task/DIyLlDigT0eupXzDHpJaXA/runs/0/artifacts/public/build/target.dmg
Windows: https://queue.taskcluster.net/v1/task/H4j5dOECSSGpDA1RWNWYFg/runs/0/artifacts/public/build/install/sea/target.installer.exe
Flags: needinfo?(feluchs)
(Reporter)

Comment 14

8 months ago
Mark: Unfortunately, I currently have no test environment and pinched under a big pile of work. I try to get one in the next few days and can then run a few tests. Really great that you have stuck through and have now fixed this bug. Thank you very much!
Flags: needinfo?(feluchs)
(Assignee)

Comment 15

8 months ago
Ok, I just set up a local test with `<Url type='application/json' rel='suggestions' ...` in the xml, and it seemed to work fine.

I also tested returning the suggestions as a text/html content type (i.e. something not application/json), and it still works ok (we don't seem to care what type, as long as the file is json itself - but that's probably OK, as the type is specified in the url definition).
Comment on attachment 9003190 [details]
Bug 1425827 - Allow "application/json" as well as "application/x-suggestions+json" as MIME types for Suggestions in OpenSearch

Mike de Boer [:mikedeboer] has approved the revision.
Attachment #9003190 - Flags: review+

Comment 17

8 months ago
Pushed by mbanner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a6f19768aa91
Allow "application/json" as well as "application/x-suggestions+json" as MIME types for Suggestions in OpenSearch r=mikedeboer

Comment 18

8 months ago
bugherder
https://hg.mozilla.org/mozilla-central/rev/a6f19768aa91
Status: ASSIGNED → RESOLVED
Last Resolved: a year ago8 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 63
I've documented this by:

mentioning that this new MIME type is now accepted as a search URL type, and that it is an alias of application/x-suggestions+json; see https://developer.mozilla.org/en-US/docs/Web/OpenSearch

Adding a note to the Fx63 rel notes: https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/63#Other

Does that sound OK?
You need to log in before you can comment on or make changes to this bug.