1676008 - XML-XSL-SVG-based applications stopped working

Status: RESOLVED INVALID

(Firefox :: Untriaged, defect)

Description

[Aleksey F.]

Attachment: ms2prot_for_testing.zip is the ZIP archive of an ms2pot report prepared for reproducing and testing.

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0

Steps to reproduce:

• Preamble:
  In 2012-2013 I developed XML-XSL-SVG-based application for reporting the results of mass-spectrometry analysis, namely ms2prot (i.e. converting mass spectra to protein identifications). This application uses a very simple JS-dispatcher that processes XML-requests. There were two main purposes: reports should be easily shared just by copying them to web folders and be viewable locally without the need for a web server. Thus, they worked, i.e. rendered, by opening from local file storage as an ordinary file and from a web site via a link to the start page of an ms2prot report.

And there were several other requirements: the reports should be readable by human and external software and be independent from JavaScript and its libraries. The rationale of this requirement is as follows:

1. XML and SVG may be rendered much faster than extremely complex and buggy JS code implementing the same features.
2. To stop being in the hell of discontinuous versioning instead of respecting standards and use stable environment and requirements.

The critical disadvantage of JS is that JS code disallows reading its internal state, which is required for processing reports by external means. The most important and usable feature is importing SVG from a report into MS Visio to prepare complex function graphs (chromatograms) and embed them into slides, posters and manuscripts.

As the same origin policy was accepted as intentionally wrong security mean just to complicate the architecture of web applications and to make impossible a lot of advantages of shared library code, I used the following file structure for ms2prot report:

<ms2prot report>
data
{directory of each report page}*
*.svg
tr_grp_rec.xml
assay.dtd
assay.xml
rpt.xml
sample_types.xml
tr_grp_recs.xml
methods
*.js
*.xsl
styles
style.css
ms2prot.htm

The same represented directory structure has a sample ms2prot report attached (ms2prot_for_testing.zip).

In a web server, the directory "methods", "styles" and common files in "data" are mapped to a specific version thus allowing updating the shared code just in one directory instead of in all reports. For the local version of ms2prot reports, the mentioned directories reside in the report directory itself. As you see, in both the cases the same origin policy is fullfilled, and the common directory is the root directory of a report.

• Previous state:
  At least in the version 47, ms2prot reports function as desired without any errors. However, they worked only on Mozilla Firefox as only this browser had simultaneously the correct implementation of the same origin policy for XML and SVG as well as XML and SVG themselves. Neither Chromium/Safari nor MS Internet Explorer/Edge processed XML, SVG and the same origin policy as defined by the corresponding standards and concepts. Only Gecko did.

The title page for the above-mentioned work published in a scientific journal (that is why updating code is disallowed and impossible):
http://msr.ibmc.msk.ru/jpr2012/
The links to a few ms2prot reports for reproducing the bug are in the first column of the table in it.

Actual results:

ms2prot does no longer function: the browser even won't opening ms2prot reports, say nothing to rendering them as desired. When clicking to the ms2prot link, the browser creates new tab/window and then close it immediately. When opening developer tools and loading ms2prot.htm directly, the browser does not report any errors after putting XML request via the data/xml protocol. So, I wasn't able to debug ms2prot to clarify the bug.

Expected results:

Respecting the standards and the concept's definitions disregarding any versions, browsers, individual interpretations and beliefs. Any security means must have lower priority than the purpose for which they were invented, exist and applied: without reaching the purpose, any security means are nonsense as they are applied to noting. Otherwise, anyone who forced breaking security means became the intruder.

ms2prot reports should function as previously in the version 47. The same origin policy should determine the local filesystem directories as well as web directories and process them as defined by this concept in XML, XSL and SVG taking into account that the same origin can be any up level directory as defined by the policy.

XML and SVG should function at least with the features supported by the version 47. Again, respect standards and the efforts made to develop them so that users do not longer depend on individual interpretations of the standards and concepts. That is why mathematically correct definitions should be forced, applied and used as they are computable and hence verifiable, i.e. objective.

Most things can be and should be implemented without the use of JavaScript since immutable directives are processed by the layout engine much faster than any JS code, which disallows reading its internal state thus making impossible exporting the view state, e.g. polygons in SVG to external software.

This case rationales:

1. why things should work long time after without the need for any updates,
2. why the respect of standards is crucial and more important than discontinuous updates and critical dependencies on security "innovations", versions, browsers, engines, libraries and so on.
   Just stop breaking things (that work earlier) to force any ideas. It is much better to complete the implementation of standards by removing all gaps, intermediate assumptions and temporary simplifications.

Comment 1

[Aleksey F.]

The reformatted internal structure of an ms2prot report:

<ms2prot report>\
    data\
        <directory of each report page>*\
            *.svg
            tr_grp_rec.xml
        assay.dtd
        assay.xml
        rpt.xml
        sample_types.xml
        tr_grp_recs.xml
    methods\
        sortable.js
        *.xsl
    styles\
    style.css
    ms2prot.htm

Thus the origin of each subdirectory or file inside the ms2prot report is set by ms2prot.htm loaded first and is the same for all the contained elements. ms2prot.htm contains the simple JS dispatcher that processes internal requests and translates them into xml-requests then puts them into window.location' indata:text/xml;base64` format and protocol at the very end. As I said earlier, this structure I had to use because of the same origin policy to implement the requirements regarding modularity, deployment and functioning as a local application just by opening it as an ordinary file without the need to put the report into the document directory of a web server.

Comment 2

[Robert Longson [:longsonr]]

I'm afraid we're really not going to look at a huge archive. We'd need something much much smaller.

Most things that require more than one file don't work from local storage now for good reason so if that's the issue i.e. your application works from a web server then we'll just WONTFIX this.

If you'd care to determine a regression range using mozregression then we might be able to give you more insight into what's happened and whether it could be fixed or not.

Comment 3

[Aleksey F.]

Dear Robert,

Why do you say that the archive is huge? I prepared it from a real huge ms2prot report intentionally leaving just one page and other required files for reproducing either working state (on v47) or fail state (on v77). The version 77 won't run the ms2prot report neither via http nor from the local file system. However, the version 47 did both things.

The fail occurs when opening ms2prot.htm (via http or locally) just while executing the last JS line window.location = .... As the browser does not report any errors, I have no details what happens. The same happens when opening any ms2prot report from the above-mention page at http://msr.ibmc.msk.ru/jpr2012/. So, there are at least two options for reproducing the fail in one step without the need for long debugging or investigating files in the archive.

Comment 4

[Robert Longson [:longsonr]]

917.51 KB, application/x-zip-compressed. We'd be looking for a testcase that a few hundred bytes really.

Feel free to use mozregression then.

Comment 5

[Aleksey F.]

ms2prot.htm has 54 lines of code. The exact number of the line where the fail occurs is 50. Could you please open this file in the test environment provided by other files from the archive? The size of the archive says nothing on the real complexity because SVGs comprise most its size. Moreover, while the xml request is not processed by the browser, those files are not even used. And again, if you just follow the link http://msr.ibmc.msk.ru/jpr2012/QC0 then ms2prot.htm is opened.

Comment 6

[Robert Longson [:longsonr]]

I think I'll wait for you to run mozregression. If you do and there's a regression range then I'd be more interested in looking at why and what we could do.

Comment 7

[Aleksey F.]

What result from mozregression do you expect? I had a look at this tool, however it said me nothing on why it and what it does. My assumption is that it check different builds for something to determine in which build or version something occurred. Googling regression range didn't shed any light on what regression range is. The only association on a regression range I have is regression analysis in statistics as I implemented a few methods.

Could you please describe mozregression in a few words and what do you expect from applying it to this bug?
Should I deploy the build environment for the browser and download the browser's source code and all the prerequisites to use mozregression?

Comment 8

[Robert Longson [:longsonr]]

Developers like me make changes to firefox all the time. Each day there may be a hundred changes to Firefox. If we can narrow down when your application stopped working to a few of those changes in a specific day we might be able to spot what happened and why it happened.

A regression range contains two changesets a starting one (it breaks here or afterwards) and an ending one (its broken here or before)

The site contains pretty clear details of what it is and what it does. You don't need to download any browser source code, nor do you need to set up a build environment for the browser. There really are no prerequisites, other than to install it and run it. The GUI will take you through everything else.

Comment 9

[Aleksey F.]

I searched changeset on http://msr.ibmc.msk.ru/jpr2012/QC0/:
If the start page of the report is loaded, then it works. Otherwise -- doesn't.

At the very end, mozregression displayed the message Unable to find enough data to bisect.
Last good changeset: d1c70c20e7b52f7295411343e4dc5db8ee7c92b9
Last basd changeset: 2306e153fba9ca55726ffcce889eaca7a479c29f

Comment 10

[Aleksey F.]

Thank you so much for the explanation. mozregression is a very good and useful tool!

Comment 11

[Robert Longson [:longsonr]]

Well that give us this lot...

https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=d1c70c20e7b52f7295411343e4dc5db8ee7c92b9&tochange=2306e153fba9ca55726ffcce889eaca7a479c29f

From that, bug 1324406 looks like the winning entry. Looks like that aligns our same origin policy with other browsers and the spec as you're seeing. I'm afraid since this was a deliberate change, that's where we're now stuck.

Comment 12

[Aleksey F.]

Robert, thank you for finding the "bug fix" that broke all the application like mine. My point is that an architecture bug fix should not produce new architecture bugs or restrict the variety of applications or practically important use cases. I mean that the same origin policy is the architecture bug and the mean that serves just for the benefits of some very famous corporations. And I am sorry that Mozilla follow that corporations being their competitor.

Comment 13

[Robert Longson [:longsonr]]

The point of standards is so that you can write something and have a reasonable chance of it working in all browsers. If one browser works differently then something needs to change. In this case it was Firefox. If you want the standard to change you should argue that with the standards body and get commitment for all browsers to change.

Comment 14

[Aleksey F.]

I fully agree with you, especially regarding standards. That is why I rely on standards instead of developing something highly dependent from a platform/browser. Now I see what happened in 2017: four web-leading companies broke W3C and left it. After that it is evident why XML/XSL/SVG applications became under pressure or even broken: it was intentionally done seeing that mass web developers didn't use widely high level of abstractions, specifications and descriptions like XML and other W3C' standards promoted -- PHP, JSON and procedural code, i.e. primitiveness, won.

Comment 15

[Aleksey F.]

Why do you explain the purpose of standards? There is one clue to understand my point as well as others': the agreement you called "HTML standard" is not a standard because it is just the agreement between four corporations against other community and companies. And the second point is that there is this agreement and a lot of its interpretations by the developers from those companies. In the case of the same origin policy, we have the wrong interpretation on what to do if origins are considered different by the definition. The interpretations are not the standard. And we faced with the interpretations rather than standard.

In the second sentence Two actors in the web platform that share an origin are assumed to trust each other and to have the same authority we see the different case rather than in the bug under discussion. First of all, in the case of applications like mine, there is just one actor: it's the same application, the same document. Thus the same origin policy can not be applied in accordance with the above-cited principle. And hence one application is assumed to trust to the internal self-requests it constructs and puts to window.location in a short XML document so that the browser renders it as a result of the internal navigation request. That is why such XML documents should be considered trusted inside such applications as well as all other documents at the level and under the directory structure of the root html document containing JS code making XML requests. Second, HTTPS is not needed at all in such applications as they consist of only static content and thus there are no any security risks or a need to differentiate schemes (protocols).

Otherwise you should consider any documents, especially XSD and JS documents that are used by a rendered document (i.e. actor) via references, as having different authorities and thus isolated from each other to varying degrees as said in the HTML agreement standard of four. In case of SVG, it is impossible, otherwise it could break SVG document behavior. Hence, the current interpretation of the same origin policy makes impossible web API and code sharing between different libraries, developers and hosts because of different authorities and more than one actor. However this does not happen for evident reasons, which means double standards in case of single front-end applications trying to use its own documents via data: protocol. By design and purpose, other applications should be able to use the same requests via data: to achieve their own purposes and ways of representing data. However, because of intentionally and wrongly forced the current interpretation of the same origin policy against data:, it became impossible.