814633 - Firefox 17 breaks a Greasemonkey script that relies on E4X (New : Firefox 21 delete totally its support)

Status: RESOLVED WONTFIX

(Core :: JavaScript Engine, defect)

Description

[decembre56]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0
Build ID: 20121024073032

Steps to reproduce:

This greasemonkey script is break after update to firefox 17.0 :
<a href="http://userscripts.org/scripts/show/5016">Flickr Functional Suite</a> 


Actual results:

I update to 17.0 and lost the use of this script.
Before i used firefox 16.0.2 and all worked fine...

To reproduce :

1 - Install the script :
<a href="http://userscripts.org/scripts/show/5016">Flickr Functional Suite</a> 

2 - Open an Flickr photo page (no needs to be connected) , by example :
<a href="http://www.flickr.com/photos/bruno-decembre/6755194447/in/photostream">inauguration de la Bibliotheque Elsa Triolet (Bobigny - 1986)Bib - Tomi Ungerer</a>
3 - Click on the user name , here mine "decembre"
3 - If the script work , you can show a popup with Color Flags choices for this user.
     notes , there is a bug for this popup :
it's open on the very bottom of the page... so you need to scroll at the bottom :-(
  I correct that with an userstyle :
<a href="http://userstyles.org/styles/69350/flickr-widescreen-script-flickr-functional-suite">Flickr WideScreen - SCRIPT/Flickr Functional Suite</a> , wich open it near the username.



Expected results:

Can you find a way to use this script without downgrade my Firefox version?

Comment 1

[Loic]

(In reply to decembre56 from comment #0)
> 3 - Click on the user name , here mine "decembre"

Are you sure it's clicking on the username? Because if I click, Flickr sends me to the user's gallery page and I cant' stay on the current page.

In addition, could you attach a screenshot of the popup displayed by the userscript.

Comment 2

[Loic]

Okay, the regression range I found:

m-c
good=2012-08-26
bad=2012-08-27
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=b3cce81fef1a&tochange=8af6a22827ec

I saw an error in the error console too, but I'm not sure if it's related to this issue anyway:

Error: missing name after . operator
Source File: file:///C:/Users/<user>/AppData/Roaming/Mozilla/Firefox/Profiles/fc0brci7.test7/gm_scripts/Flickr_Functional_Suite/5016.user.js
Line: 558

Comment 3

[Matthias Versen [:Matti]]

The first bad revision is:
changeset:   103481:6564c3183a04
user:        Jason Orendorff <[e-mail]>
date:        Sun Aug 26 09:04:33 2012 -0500
summary:     Bug 778851 - Disable E4X in content by default.

confirmed by toggling javascript.options.xml.content to true in Firefox17
(can you confirm that Loic ?)

Comment 4

[Matthias Versen [:Matti]]

Reporter:
This will very likely end up wontfix because the E4X feature in the JavaScript Engine will be removed. I can't tell you who if using E4x, the website or the script but maybe some of the JS QA/developers can.

Comment 5

[Loic]

(In reply to Matthias Versen (Matti) from comment #3)
> The first bad revision is:
> changeset:   103481:6564c3183a04
> user:        Jason Orendorff <[e-mail]>
> date:        Sun Aug 26 09:04:33 2012 -0500
> summary:     Bug 778851 - Disable E4X in content by default.
> 
> confirmed by toggling javascript.options.xml.content to true in Firefox17
> (can you confirm that Loic ?)

Yep, confirmed. And as you said, it's likely WONTFIX.

Comment 6

[Boris Zbarsky [:bzbarsky]]

Line 558 of the user script is:

  var pid  = comments.@photo_id;

which definitely looks like an e4x-ism to me.  There are a few more lines like that.  Just search for the function called "ricCB" in http://userscripts.org/scripts/review/5016

Comment 7

[decembre56]

"Disable E4X in content by default" :
 - Why disable that now, it's an security reason ? 
 - How to re-enable it , maybe in about-config ?
 - Can i change something in the script to convert Ext4 in something else which work proprely?

Yes, i can remember something about Ext4 things when the author of this useful script respond to our requests (but seems not the case for now):

He said that Ext4 is used to store sets variable for each User color Flags.


Sorry i am not a developer like you can see.

Comment 8

[Loic]

E4X is deprecated, read the doc: https://developer.mozilla.org/en-US/docs/E4X
You need to modify the userscript or find someone who is able to do it.

Comment 9

[Matthias Versen [:Matti]]

>How to re-enable it , maybe in about-config ?
You can enable it again (the option is in comment#3) in Firefox17 but it will get removed with bug 788293 in future releases

Comment 10

[decembre56]

Sorry it's EX4 and not Ext4 , like i write before...

Thanks Matthias .
But where /how toggling javascript options ?


Some links i found :
 - ECMAScript for XML (wikipedia) :
http://en.wikipedia.org/wiki/ECMAScript_for_XML
 - JS Trickshot: E4X(...a very nice JS embedded XML processing language... ):
http://www.ilker.de/javascript-trickshot-e4x.html
 - Can we remove E4X support? (Mozilla - JavaScript Engine) : 
href="http://mozilla.6506.n7.nabble.com/Can-we-remove-E4X-support-td170661.html

A bout Flickr Functional suite :
http://userscripts.org/scripts/show/5016
I use this script since 2006, and it store many useful informations for me - colors flags , used like a quick tag...

There are some error in it that i cant find/know how to fix myself like :
- On pool page , if you click on the user name of a shot, "Flickr sends me to the user's gallery page and I cant' stay on the current page" like Loic said.
but it work on Photo Page (the "photo gallery" of Loic)
- Some user name with special characters (@ >...) are not coloured .
- Popup which open at the bottom of the page .
But if you follow my "Steps to reproduce" that ok .

 Strange , in wikipedia they said EX4 is not supported by Chrome , but this script work always with it and i can't imagine Firefox stop support E4X on its side.
Or if not , i hope firefox give to us a way to choose if we want enable it , or give a tool to convert EX4 to another thing easily.

Comment 11

[decembre56]

 :-(
Notice: E4X is deprecated in SpiderMonkey :
https://groups.google.com/forum/#!topic/mozilla.dev.tech.js-engine/yYQyMCcMf-0/discussion
"The SpiderMonkey implementation of E4X is deprecated and will be removed.

The rationale: E4X has a standard, but is not part of the de facto open web: for a long time, it's implemented by only one browser, Firefox, and is not used for open web content. We are trying to advance SpiderMonkey and Firefox rapidly under intense competition, and keeping E4X slows down our development and increases the security attack surface"

Comment 12

[Alex Keybl [:akeybl]]

This sounds like expected behavior (to some degree) - no need to track for upcoming releases.

Comment 13

[Gervase Markham [:gerv]]

E4X is going away. Sorry :-(

Gerv

Comment 14

[Nick]

I recognize the futility of pursuing this matter further, but I feel obligated to point out the massive number of GreaseMonkey scripts that have completely stopped functioning due to the removal of E4X.  For a long time E4X has been a suggested way of storing static content (such as CSS) in GreaseMonekey scripts.  Up until the 20th of December, 2012 (less than TEN days ago), it was listed in the GM Wiki (http://wiki.greasespot.net/index.php?title=Multi_Line_Strings&action=history) as a way to store multi-line strings.

Was any discussion or thought given to the damage this removal will have on the userscripts community and users?  So many scripts have authors that are MIA leaving tens of thousands of users helpless to continue using their scripts.

Arguably the most compelling feature that keeps people on Firefox is the addon ecosystem.  Destroying the work and tools of so many addon/script authors and users is another step down the road to Firefox's obsolescence.

I think this issue should be revisited, not ignored in the name of "progress".

Comment 15

[Jeff Walden [:Waldo]]

Discussion and thought were given to the damage to addons, to websites, to user scripts, the whole lot.  And the end conclusions were that:

That E4X is a single-browser feature that no other browser will implement (and I, as an implementor, have actively dissuaded others from attempting to get it in other browsers -- or more precisely, from getting an E4X-using test added to Acid3, which might have forced them to implement it, but which in the meantime would have given us a better Acid3 score and them a worse one).

That E4X is perhaps the single biggest existing source of security bugs in the JavaScript engine.

That the E4X specification is bug-ridden (we've stumbled over literally dozens and dozens of them, some of them which became security vulnerabilities for us) and unmaintained, and the spec algorithms were not written in such a way as to provide provable correctness, so fixing the issues is very much non-trivial.

That the E4X spec is based on ES3, but ES3 is a dead end, and ES5/ES6 have moved in a totally different, irreconcilably incompatible direction (for internal meta-object operations, to be precise -- things hard to observe using JavaScript features, but it's possible).

That the code implementing E4X is an area of the JavaScript engine that no JS hackers fix bugs in because it's not what the web cares about (which is mostly performance improvements on public websites, apps, and benchmarks).

That if we didn't have our E4X implementation, we could move faster at modernizing the JS engine, and at implementing new ES6 features in it, and at improving performance in it, and at making it better in ways that benefit everyone, not the E4X-using minority.  (For example, E4X is actively standing in the way of implementing modern, memory-efficient garbage collection, and its removal will substantially reduce the work preparing for that.)

And that none of this is going to change.

So in the end we decided to cut our losses.  Not lightly, oh no, not lightly at all.  This has been discussed off and on for *years* -- E4X has been known to be that bad, in all these ways, for that long.  And we've been rolling out the removal of E4X -- to webpages, to chrome scripts, and completely -- for many, many release cycles, precisely because we're aware this is a substantial change.  But sometimes, we have to break things, when the benefits to doing so substantially outweigh the losses.  And they really do, here.

This answer probably wasn't entirely satisfactory, but hopefully it shows where we were coming from when we made this change, and why we made it.

(Regarding userscripts specifically: aren't userscripts that use E4X incompatible with the Greasemonkey implementation for Chrome?)

Comment 16

[Nick]

(In reply to Jeff Walden [:Waldo] (remove +bmo to email) from comment #15) 
> This answer probably wasn't entirely satisfactory, but hopefully it shows
> where we were coming from when we made this change, and why we made it.

As satisfactory of an answer as possible without having been "Dang, you're right. Lemme click Undo." ;)

I appreciate your detailed answer and the time you took to reply.  As someone with a bunch of GM scripts (many on UserScripts, some not) it's just frustrating to me to suddenly have a bunch of stuff stop working.  I've preferred using E4X solely for storing multiline strings and now I've got to convert to something else (really looking forward to support for ES6 template strings...).
 
> (Regarding userscripts specifically: aren't userscripts that use E4X
> incompatible with the Greasemonkey implementation for Chrome?)

They are, though it was far from the only exclusive Firefox GM feature.

Comment 17

[decembre56]

For lower versions than 21, i used Before i use this "about:config" thing (toggling javascript.options.xml.content to true):
1 - Open about:config
2 - find:
javascript.options.xml.content
- Set its the value to :
"true"
(Solution only usable for Firefox smaller than 21.00)
Now with Firefox 21 , the E4X support is totally deleted...
No solution to renable it by a tweaks in about config, like before?

Comment 18

[Loic]

The best way is to update the script code. E4X is EOL now.

Comment 19

[decembre56]

Yes thanks, i understand;
But you know, i am not programmer, coder, etc...

I am a just an user which need to have only one userscript working (not updated by its author).

I can try to update the script, effectively i try , but i fail...
Too difficult for me.
I don't know how to recognize where i need to make some change :
The only thing i can see  (with the help of Boris Zbarsky ):
<i>these "E4X-ismes":  var pid  = comments.@photo_id;</i>
but how to fix that... ? I have no idea.

So, now that you have cut all possibility for us to choice to enable or not E4X, 
can you give to us a (comprehensive) way to convert these scripts ?.
Yet i have not found something useful for me.

After promoted it, Have you now a dedicated place to follow / help after breaking the E4X support ?

Yes, i know that here is a bug Report place and not a forum to ask help, but the situation is special, no?

Comment 20

[decembre56]

Always no dedicated place to follow / help after breaking the E4X support ?

Comment 21

[Loic]

Certainly not here, on BMO. Try http://forums.mozillazine.org/ or another forum about coding.

Comment 22

[decembre56]

thanks...