Open Bug 40873 Opened 26 years ago Updated 19 days ago

Save as rfc 2557 MHTML; complete webpage in one file

Categories

(Core :: DOM: Serializers, enhancement, P3)

Product:
Core
Component:
DOM: Serializers
Type:
enhancement

Tracking

()

People

(Reporter: sidr, Assigned: jkt)

References

(Depends on 2 open bugs, Blocks 1 open bug,
URL
)

Details

(4 keywords)

Attachments

(9 files, 4 obsolete files)

This page as complete MHTML
74.18 KB, text/plain
Details
complex frame document as MHTML archive
64.57 KB, text/plain
Details
bug-40873.patch
40.43 KB, patch
Details | Diff | Splinter Review
Buggy backup of import and export
161.85 KB, text/plain
Details
out-interceptor-approach-parent-cid.patch
215.39 KB, patch
Details | Diff | Splinter Review
WIP: Bug 40873 - Add MHTML read/loading support
48 bytes, text/x-phabricator-request
Details | Review
about:support of the build
341.81 KB, image/png
Details
Screenshot showing build opening Chromium patches and vice versa
2.35 MB, image/png
Details
WIP: Bug 40873 - Add MHTML export/writing support
48 bytes, text/x-phabricator-request
Details | Review
As an option to IE-style saving of complete webpages, including stylesheets, images, objects, and applets, in a subdirectory (see bug 11632), it might be useful to save all of those in one file with the webpage, as an MHTML file. Quoting from the RFC: "While initially designed to support e-mail transfer of complete multi-resource HTML multimedia documents, these conventions can also be employed to resources retrieved by other transfer protocols such as HTTP and FTP to retrieve a complete multi-resource HTML multimedia document in a single transfer or for storage and archiving of complete HTML-documents." <URL:http://www.faqs.org/rfcs/rfc2557.html>, "MIME Encapsulation of Aggregate Documents, such as HTML (MHTML)" This enhancement would only make sense if bug 18764, "RFE: Full rfc2557 MHTML multipart/related support in BROWSER", were also implemented, so that pages saved as MHTML could also be viewed again! Also, implementing this feature would imply adding .mhtml to the list in filepicker.properties (see bug 39951, ".shtml, .xhtml should be "HTML Files" in file pickers".
Depends on: 18764
Editor would want to do something similar here as well, and it may well fit in with our (as yet nonexistent) publishing plans.
Cool idea. I like it. But let's wait on it for awhile and ship this puppy first.
Target Milestone: --- → Future
Composer already has this as a "future" bug: 36419. It's technically a regression relative to 4.x, and is part of publishing work. Linking these bugs.
is this a dup of bug 11632 ?
>is this a dup of bug 11632 ? Kind of, but I actually this this bug is more useful, as it proposes a good general solution. Set one up as blocking/depending on the other? While we're on this, we should also mark a dependency between these bugs and: http://bugzilla.mozilla.org/show_bug.cgi?id=41480 I'd do it but I've been told off for getting "blocks" and "depends on" the wrong way around before. I'm not sure I understand the difference well enough...
It's good to have both bug 11632, "[RFE] Save Page With Images, Stylesheets, Objects, Applets" and bug 41480, "[RFE] Support drag and drop of complex selections from browser", mentioned here, but strictly there are no dependency relationships nor any duplicate relationships among these three bugs. This bug is a feature request for a particular way of saving complete web pages in one file, which has its advantages. In both of the other bugs, there is no clear consensus that using MHTML is the best way to solve the problems they were created for. Depending on those decisions, this RFE could become either a blocker or a WONTFIX, but as all three are marked "Future", such decisions will be waiting until after shipping.
Perhaps 11632 should be marked dup of this one? They seem to suggest the same improvements, but this bug (40873) has an explicit suggestion about how to approach it. Questions about a feature like this "comes up" a lot, people mention it on irc etc. and are referred to these bugs. I suspect there's a handful of "undiscovered" dups floating around as well. Thing is.. when they are scattered around like this it's hard to get a real overview of how "hot" the topic is, but i believe it would soon be a "mostfreq" if more collected. There is 11632 with 3 dups, one of the dups (23027) had a dup, there is bug 25756 "[RFE] Enhanced `file-> save as'" With this one, there are in other words at least 7 bugs till now, about what is really the same feature request. Perhaps at least a collection bug would be an idea?
Since Don has left, Vishy is taking his bugs in bulk, pending reassignment. thanks, Vishy
Assignee: don → vishy
nav triage team: This would be way cool, but we won't get to this for beta1, marking nsbeta1-, adding helpwanted
Keywords: helpwanted, nsbeta1-
Depends on: 82118
I created a tracking (meta) bug 82118 to track these kinds of bugs and to unify the efforts. Maybe a few duplicates will also become aparent this way - then we can assign the keyword MostFreq.
Removing dependancy to bug 82118 as it should be the other way round (bug 82118 depends on this bug).
No longer depends on: 82118
Blocks: 82118
I'll try to implement this together with bug 18764. This is easy to do for normal web pages, but rather hard when frames are involved, because we have to rewrite the SRC attribute of <frame> and <iframe> if the user has browsed within the frameset. Rewriting must be done at the parser level, because writing a sequentialized DOM would break scripts that change the DOM. Rewriting will suffer from the same bugs as view-source does, but I'm about to fix many of them too (some of them are not only cosmetic). BTW, the target milestone is for the worst case I thought about when I began to work on bug 18764; this may be ready within the 0.9.4 timeframe, but I don't promise anything.
Assignee: vishy → clarence
Keywords: helpwanted, nsbeta1-
Target Milestone: Future → mozilla1.1
Status: NEW → ASSIGNED
*** Bug 95090 has been marked as a duplicate of this bug. ***
Depends on: 98550
spam: over to File Handling. i have not changed the assigned developer [or the other fields for that matter], so if anyone realizes that a bug should have a more appropriate owner, go ahead and change it. :)
Component: XP Apps → File Handling
since 0.9.7 is possible to save a page with its images, however the images are saved in a folder. instead of write lot of code to implement mhtml you can simply zip the html page and images in a single .zip archive, like kde konqueror does (it uses a tar gzip with the filename extension changed).
saving the files as a zip is not the same, it is not an acceptable solution to this bug. Clarence was working on this bug awhile ago, any update?
*** Bug 121793 has been marked as a duplicate of this bug. ***
Bug 121793 is reopened, please, discuss it, I think, this is very interesting suggestion comparing with MHTML. http://bugzilla.mozilla.org/show_bug.cgi?id=121793
IE has this option already. It uses mht extension. Maybe they are using MHTML standard? Can anyone find out? Interoperability would useful, meaning if Mozilla and IE shared the same format, hopefully a standard based. Maybe Opera would get on the bandwagon too.
yes, a common file format based on mhtml is highly desirable (at least to me :-)
removing myself from the cc list
With reference to the comment made by "r_roman", IE does use the MHTML format as can be seen on this page: http://support.microsoft.com/support/kb/articles/q221/7/87.asp or as you can find out for yourself by either opening the file in a text editor or in Mozilla. MHTML is indeed a very useful feature, at least for people like me who would like to have the convenience of a single complete document which can be rendered clearly, as opposed to the unfriendly manner in which PDFs are rendered, at least by Acrobat Reader. Therefore, it is highly desirable for Mozilla to support MHTML - this would not result in using an MS-proprietary file format and will actually help interoperability. Actually I would have really liked a feature similar to what Konquerer has (as has been pointed out) - a complete, self- contained Web Archive, compressed, and containing related HTML pages (e.g. a tutorial broken into chapters) and resources like images, stylesheets, etc. Composer should be able to produce such archives and Navigator should be able to render them properly. In the interim, we should definitely have proper support for MHTML and get rid of PDFs as much as possible! Ranjit.
> get rid of PDFs as much as possible! Not gonna happen until Mozilla gets support for CSS paged media (bug 72321 and bug 110154). One of the primary reasons I've seen in practice for using PDF is not that it's all one filesystem object but rather that it preserves page breaks, especially for forms that the user prints to paper, fills out, and submits by mail or in person. PDF has also been popping up as a replacement for .ppt slides because more users have the Acrobat viewer than the PowerPoint viewer. Mozilla + SVG + MHTML (or the XML format discussed in bug 121793) equals .pdf and .ppt killer.
*** Bug 131544 has been marked as a duplicate of this bug. ***
*** Bug 141650 has been marked as a duplicate of this bug. ***
this is scheduled for mozilla 1.1 alpha. is there any work going on ? thanks.
Its not clear from reading the above whether MHTML supports multi-page saves but it seems to me that this should be the target for this implementation. Specifically, while there may be a lot of single-page information pages out there, anymore (what with all the ad stuff), many sites break their topic content into many pages (ref. CNET or Tom's Hardware, for example.) What is really needed is the ability for the user to enable some type of URL recorder such that, once enabled, it will grab "clicked" links until the user signifies the recording setup session over. At which time, Mozilla should then traverse the saved URL list and save all pages in a self-contained file. Also, some means should be implemented for saving the root URL withing the saved content so that the user can easily return to the page at some later point to see if the content has been updated.
Yes, yes! Those are excellent suggestions. While I have enjoyed IE's ability to save a page along with images in one file, I didn't like the fact that sometimes I had to end up with multiple files. I usually zip them up to have just one file. Using a good file manager, like Servant Salamander (http://www.altap.cz/salam_en/index.html), makes working with archive files very easy. You can "go into" the archive as if it were a folder.
r_rom, thinking about it a bit more, Mozilla could exploit its group-bookmark design when saving multi-page content, i.e., when the saved pages are re-openned, each one should be openned in its own tab. In this way, saving multi-page content would appear more like saving a book having the tabs representing the chapters. The tabs, of course, should having the respective page's title in the tab and expanded for mouse-over hints.
Another alternative: It opens in one window -- one that contains two frames. One of the frames contains a tree with links to sections/pages. Click a link, and the other frame loads its content.
The problem I see with using the "frame" paradigm is that many sites already use them. Thus, creating a framelist for a "framed" site would be problematic and, perhaps, a bit busy looking...
That is true. But do you really want to load all pages at once, even if you want to read one of them. Mozilla is already slow as it is, so I wouldn't want to load more than necessary. And imaging what the UI would like with you suddenly got 20 tabs. I think some other way of selecting pages should be found. Perhaps a quick pop up window (on a shortcut press) that would display the list of pages?
Re comment 30: It would be better to utilize the Sidebar for the tree frame.
My thought is that openning all the pages in tabs is a convenient way to show that one, its a multipage document and two, which pages were actually saved (versus, all the other potential links on the pages that weren't selected.) The loading of the pages shouldn't slow it down too much since they will all be on the local disk already. Although, maybe there could be a way to save the group-bookmark as an embedded link during the save. Then, when the saved-pages are loaded, it would open the "root" page and the user could then decide to click the group-bookmark link to open the other pages as tabs??
"Then, when the saved-pages are loaded, it would open the "root" page and the user could then decide to click the group-bookmark link to open the other pages as tabs??" As tabs or in place of the main page. I was talking about slowness of overall Mozilla's UI and memory consumption. If you load 20 (it could be many more) pages at once, much resources will be waisted. And it will be difficult to select tabs because they would become small if number of pages is large. You wouldn't be able to read their titles.
My latter suggestion (using a pseudo group-bookmark) would solve this, no? In effect, it would load the "root" page and then allow the user to click the embedded bookmark to expand the multi-pages into tabbed pages, if they desire. Of course, if they choose not to, then they could still move through the saved pages using the regular links on it. As to the scrunched tabs, the mouse-over hint capability would allow the to see the full titles, no? Actually, now that I think about it, perhaps a better design would be to allow one to save a tabbed browser window as a single file. That way, one would be able to create tabs of disparate, yet related, articles and then save them as a file for later offline perusal/reference?
Yes, the pseudo group-bookmark mechanism would solve that. Your last suggestion is not bad. Now I have a concern. If the multiple page saving is not supported by the standard, then only Mozilla will be able to open these files, and that is not good at all. In that case, I would probably stick with using compressed files.
Hmmm, needing them to be viewed in other non-Mozilla browsers could be problematic? Take the case where one opens several pages in a tabbed window. Since the user can open and close tabbed pages at will, its highly likely that the page containing the link to a tabbed page has been closed, thus, removing the ability for one to "get" to it without a tabbed interface. I suppose that the saving algorithm could save a TOC page containing all the links that were saved, thus, allowing non-tabbed browsers the ability to get to the other pages but that seems kinda kludgey to me? Perhaps it could be implemented in such a way that only non-tab browsers are presented with this TOC??? Anyway, following through with my previous thoughts, it seems that "keying-off" the fact that a window has tabs as the flag for multi-page saves may make the implementation of this feature easier (rather than designing some type of recorder on/off feature?) This would certainly be a positive extension to the tabbed design, I think...
Here's a thought. Mozilla saves the tabbed pages in a single file but includes a "flag" in the file for later interrogation. Now, for non-Mozilla browsers, the file will open with a TOC page listing the offline links contained in the saved file. However, for Mozilla, it could detect this flag and then query whether the user wish to open the TOC page or the root page with the others in tabs? That way, it should be able to work with most browsers, no?
Sounds good to me... if other browser wouldn't get confused by all this additional info (flags, etc.). I haven't studied the standard, so I don't know if it's flexible enough.
Pat, I think Mozilla should find another way of doing this, since MHTML is about a single document. Should this be another bug or RFE?
Bamm, yeah, an RFE is 'prolly the more appropriate vehicle. Now, if I only understood how one goes about doing that?
Just make sure Mozilla supports MHTML too. Or this will be another layer tag.
Pat and r_rom, this bug is about *saving* *one* html file with images and embedded objects in one MHTML file (RFC 2557). It is not about displaying them or creating proprietary file formats. Please file one ore more bugs with summaries of your ideas concerning saving and displaying multiple html files. Note: MHTML is IMHO not built to support frames. But in theory one could create an "envelop" of type multipart/related with the frame and the related parts would be again of type multipart/related containg the frames. Dunno if any mail client or browser supports this.
Martin, done. It's bug 157422
If i wanted to be able to save as Mulitpart Hypertext Gzipped (.mhgz? which does not exist AFAIK) should i file a seperate bug report / feature request?
You would save it as MHTML, then type 'gzip foo.mhtml', just like for an HTML file or anything else. Please people, the next comment should contain a patch or some damned technical discussion on how to go about creating one. No comment from the assignee in over a year. Removing past-due target.
Target Milestone: mozilla1.1alpha → ---
so when this will be (finally) implemented? please set a milestone, at least... thanks.
mporta@mail.vu: Feel free to assign it to you and attach a patch.
*** Bug 162549 has been marked as a duplicate of this bug. ***
QA Contact: sairuh → petersen
Works fine on Microsoft IE for archiving, except some javascript specials. Would be a useful feature for Mozilla too.
I'm curious, is there a reason why you can't reuse the Mail/Newsgroup code for saving email files? From looking at a .eml file and a .mht file, they look nearly identical, the only difference being the Return-Path and other email-only headers. Both save the document, along with all the images/etc stored in base64, and this could result in a quick, simple solution. As well, you could reuse the Mail/Newsgroup code for viewing HTML emails for loading an MHTML file.
Summary: RFE: Save as rfc 2557 MHTML; complete webpage in one file → Save as rfc 2557 MHTML; complete webpage in one file
Similar to Bug 18764, comment 32, MHTML support could make Composer a true BiDi word processor for platforms that have no built-in BiDi support (e.g. BeOS, and to a great extent, even OS X). Prog.
Please, please implement the file saving as a single file, preferably as a standard compliant html archive ASAP. This feature alone keeps me in Internet Explorer as a default browser. Could someone give a status report on this? It has been three years since the first comment here and it would be useful to know whether this capability is likely to be implemented or a decision has been taken to not implement it.
There is no such thing as "standard compliant html archive". I see three ways to safe a HTML page with all related content in one file: Safe all data in a format designed for mails: MIME type miltipart/related. This is what this bug is about. The RFC does not define any file format (or Windows file extension). Though if you use .mhtml IE will display it correctly. Mind that such a file may have mail headers in it! Another one is to store the images etc within the file using the data: protocol. The resulting file would be standard conformeing. There is a bug filed for this method. The last way is to store the html file and all its part in a zip/jar archive. Not compatible but should be much easier to implement than all the other methods.
there is already support for saving a page with images, althought it isn't a single file. a simple solution is to write a "converter" from the format already supported by mozilla to the .mht format used by internet explorer (and viceversa).
What is the current status on this? I'm going to remove my vote for it as I keep getting more and more people just adding their email address to CC:
The last comment by the bug assignee (Andreas M. "Clarence" Schneider) was written almost two years ago, so I think it is safe to assume that this bug is not going to be fixed in the foreseeable future. Prog.
All respect to Andreas for his previous effort, but this bug should be reassigned to somebody else, please.
thank you for volunteering, please indicate when you expect to have this fixed. if you're interested in fixing other bugs feel free to comment in them, i'll gladly give you them too. i see you have a wide interest, that's great, i'd suggest you limit your contributions for the time being.
Assignee: c → thebeastwitheyesthatstared
Status: ASSIGNED → NEW
hello timeless did you see me writing, that I was going to hack on this? Whatever it is you are up to, this is not the way to go about it mister. I imagine and appreciate, that you would rather see a patch attached probably, but my comment really is about expressing my point of view. I do not as a user or a potential developer appreciate such behavoir or insinuendo. I help with what I can as of now, I don't need your help to tell me what I can or what I am doing. Don't forget timeless that not everybody are alike.
bugzilla is a forum for developers. we discuss patches and analyze bugs. we don't whine about a lack of implementation nor is it proper to ask if something is fixed or when it will be fixed. http://bugzilla.mozilla.org/page.cgi?id=etiquette.html again, if you're interested in contributing to any of the other bugs you've touched please continue to act as you have, i have a long list and you can certainly make it longer.
Blocks: majorbugs
I tried out Mozilla a couple of times. But the most important reason for me to stay with IntExp is the lack of saving web pages into a single file. I wonder wether there is no progress on this topic for years. It should be easy to implement this feature in comparison to other projected items on the todo list. Thomas
*** Bug 209613 has been marked as a duplicate of this bug. ***
Is anybody interested in jointly funding a developer to implement this? Would anybody be able to make time available to implement it if payment were available?
yes. i would happily contribute $10 to such a fund. it would have to be modelled after the amiga project, <URL:http://www.discreetfx.com/AmiZilla.html> and somebody would have to manage it (not me). a note - i will not be the only contributor; my $10 offer stands only if the total contributions exceed $50. additionally, i would require mhtml read/write support and ie compatibility. this offer expires 12/2005. void where prohibited by law (fnord). if this becomes a popular discussion topic, we should open another bug for it as it does not pertain to actual development.
Okay, it looks like there's enough interest to place a bounty on this bug, but Bugzilla's not the place to do it. I've created a wiki to organise this effort at http://www.dezyne.net/codebounty. If this bug is worth something to you, please hit the site and add to the total. I haven't added the details/amounts of those people that emailed me, on the theory that it's not right to go putting other people's stuff up on the web without asking.
*** Bug 214382 has been marked as a duplicate of this bug. ***
In short: I did not ask to have this bug assigned to me, maybe you thought/think I did, but I didn't. It's certainly not going to get me to contribute more, when this is the end result of trying to start a discussion. In short: Counter-productive.
Assignee: thebeastwitheyesthatstared → file-handling
*** Bug 224709 has been marked as a duplicate of this bug. ***
Whiteboard: parity-ie
Please look also at 18764 RFE: Full rfc2557 MHTML multipart/related support in BROWSER it seems it is the same feature request... I would like to suggest to set up such a feature for the firebird 1.0 release, too, because it is very important to "hardcore" net navigators and "pagesavers" and Opera does NOT support it ;-)
> it seems it is the same feature request... no - this bug is about saving, that bug is about displaying.
*** Bug 231697 has been marked as a duplicate of this bug. ***
Flags: blocking1.7a?
Flags: blocking1.7a?
Getting this "bug" fixed ought to the highest priority of the development team in my view. I know that I am not alone in demanding this capability of a browser. The lack of this capability in Mozilla is the primary reason why Internet Explorer has not been displaced as the default browser in my computer. Without this capability Mozilla (and any of the Mozilla derivatives will remain "nice to have", but imature and incomplete browsers lacking the capability of Internet Explorer. The bug fix should also save the URL (as Internet Explorer does) so that a link to the page is preserved, if the page still exists, to check for updates to the saved item. Just look at Internet Explorer and give Mozilla its capability and power! The lack of "votes" for this bug is merely indicative of a lack of knowledge of the bugzilla system by the vast majority of Mozilla users, myself included, until urged to vote for the bug. The system of prioritization of bug fixes based upon such voting is a serious management deficiency. The team needs to prioritize the efforts based upon some rational system other than this so that the efforts of the development group are not misguided by the sheer weight of the bureaucracy. There seems little explanation for why this was not accomplished a very long time ago.
So you think that this _enhancement_ is a higher priority than the 23243 real bugs? There is rarely a need for a "me too" response to any bug.
> The system of prioritization of bug fixes based upon such voting There is no such system, actually. Developers work on what they want to work on, when all is said and done. > There seems little explanation for why this was not accomplished a very long > time ago. Quite simply, because no one has written the code. I'm glad to see you volunteering.
> > There seems little explanation for why this was not accomplished a very long > > time ago. > > Quite simply, because no one has written the code. I'm glad to see you > volunteering. There isn't much to write. The code already exists in the mail client. RFC2557 files are the same format as a multi-part email message.
Have you ever looked at the mail client code? It's such intertangled spaghetti that extracting a chunk of it out like this would be a pretty major undertaking... The MIME code is _particularly_ nasty.
(In reply to comment #79) > intertangled spaghetti... code is ...nasty. I remember reading that that was the (or one of several?) reason the code of old Netscape couldn't be modified to improve the software. Has this coding style gotten into Mozilla's development too?
Bad code happens. It happens more in some parts of Mozilla than others.
I believe the MIME handling code was basically copied verbatim from mozclassic to mozilla... that's what it looks like, certainly.
I belive *.eml have and *.mht has same format. And thunderbird can save *.eml file, and also can view *.mht and *.eml files You can try steps at following comment http://bugzilla.mozilla.org/show_bug.cgi?id=18764#c38 So why not take the code from messanger component. May be mscott or somebody from Tb team can help on this
Saving a single web page as a single file should be implemented both in two different ways: 1. In the generic MHTML format. Advantage: Compatibility. 2. In a compressed format which contains the web-page elements (html, graphics, css, etc.) in a single compressed container file. The proprietary file type extension (e.g. ccf) should be associated to Mozilla, so when opening this file type in the file system brings up Mozilla which internally opens the archive and displays the original web page. Advantage: Keeping the original files in their original state for research and further usage.
<i>In a compressed format which contains the web-page elements (html, graphics, css, etc.) in a single compressed container file.</i> Excuse me for sticking my neck out, but wouldn't it be possible to, say, put the compressed content inside of a mhtml file with the mimetype of gzipped, implying that the browser should uncompress it as it would a compressed stream coming from a compressing webserver? Or something? With the generic nature of the mhtml file it seems like this could be done in a relatively non-proprietary way.
Forgive my ignorance, but I'm wondering WHY this is seen as such an unbelievably important feature? Where is it the case that it is absolutely required to save an HTML page as a single file versus saving it as a page and an accompanying folder? As a part-time web developer, if anything, I find the other method (file + folder) more useful since it better emulates the layout on the server. In addition I have NEVER used IE's feature of saving as a single ".mht" file, and I would be willing to bet that nearly 95+ percent of IE users never have either. I just don't see the inherent (evidently overwhelming) benefit.
Forgive my ignorance, but I'm wondering WHY this is seen as such an unbelievably important feature? Where is it the case that it is absolutely required to save an HTML page as a single file versus saving it as a page and an accompanying folder? As a part-time web developer, if anything, I find the other method (file + folder) more useful since it better emulates the layout on the server. In addition I have NEVER used IE's feature of saving as a single ".mht" file, and I would be willing to bet that nearly 95+ percent of IE users never have either. I just don't see the inherent (evidently overwhelming) benefit.
(In reply to comment #86) > In > addition I have NEVER used IE's feature of saving as a single ".mht" file *I* ( a professional user) use it VERY often! > and I > would be willing to bet that nearly 95+ percent of IE users never have either. *I* bet that more than 80% of IE users use this feature and find it useful.
Comments 86-88: This is not the place for this discussion. Sorry for the bugspam everyone.
(In reply to comment #87) > ...I'm wondering WHY this is seen as such an unbelievably > important feature? Where is it the case that it is absolutely required to save > an HTML page as a single file versus saving it as a page and an accompanying > folder? > I just don't see the inherent (evidently overwhelming) benefit. The reason that saving the web page as a single file, at least for me and others with whom I have communicated about this issue, is that if you place the file in a folder or system of folders with more than one page saved per folder it quickly becomes very cluttered trying to look for a particular subject matter. If you should decide to move it to a different folder there is the matter of trying to get the correct associated folder moved with it. It is an unnecessary (because it can be done as a single file) and, in my opinion, undesirable complication to have multiple files for a single web page. The utility of saving the URL (which none of the Mozilla project OS X browsers do to my knowledge) is that you can click on the page and see if there is anything new or if the page even still exists. It also serves as useful reminder of the web site where there may also be related information. I use the page saving capability of IE on a daily basis to save information for later reference in a "Filing Cabinet" which I use as a repository for these files. Mine is currently 12 GB in size. That may not be much by some standards, but it is information which is readily accessible which I find to be sufficiently desirable that if I were forced to choose only one browser it would be the one with this capability. To date I have only encountered IE and iCab which have this capability. This is only one person's opinion, but I have encountered enough people who either use this capability or have admitted that they miss the capability with whatever browser they are using that I believe the number of potential users warrants the effort. Apart from that, it makes the browser a more powerful tool. There are some programs which attempt to save data in a searchable format. Desireable though that may be I have found IE's ability to do this easily to be highly desireable. Robert Parenton, I am sorry if you believe this is not the place for such comments, but I felt it was appropriate to make a direct explain of why I feel the matter is important. thebeastwitheyesthatstared, Thanks for taking on this project! I hope that the development team will assist you. I have been disappointed that the development team has not tried to get this taken care of for so long as I feel it to be a basic capability of a browser and now here we are going into 1.7 RC1
(In reply to comment #90 ) Alternative may be Bug#: 64286 see my example, goto http://snipurl.com/56pg and look your browser "Address bar"
*** Bug 241661 has been marked as a duplicate of this bug. ***
*** Bug 242280 has been marked as a duplicate of this bug. ***
I never knew that so many of them including me where requesting the feature. I hope by the time FireFox version 1 rolls out we can have this feature. thanks anyway for letting me know this was a duplicate bug.
Flags: blocking1.8a?
I can give yet another example on how this feature is useful. There's a newspaper (Público <http://www.publico.pt/>) that has its articles online only for a few days. Hence if I want to mention one of them to someone, just sending a link isn't good enough, as later neither me nor the other person would have access to the text on which our discussion was based on. If the article consists only of unformatted text, I simply copy it from the browser and paste it on the e-mail message. The recipient has access to the article as he/she opens the e-mail message. If it's formatted text, I select the text only version of the article, then save it as “Web Page, HTML only” and attach the file to an e-mail message. The recipient has access to the article by double-clicking the attachment icon displayed in the e-mail message. If there are images, then I just use Internet Explorer to save it as “Web Archive, single file (*.mht)” and attach the file to an e-mail message. The recipient has access to the article by double-clicking the attachment icon displayed in the e-mail message. If I were to use Mozilla, I would have to save it as “Web Page, complete,” then create a zip file containing the HTML file plus the associated folder and attach this compressed file to an e-mail message. The recipient has to open the e-mail message, save the attachment, uncompress it and finally open it with a browser in order to access the article. After viewing the article, he/she would have to additionally delete the compressed file plus the uncompressed files. Now you see how easier things would be if the MHTML was supported in Mozilla.
I think that's enough "I want this too" posts. If you have a patch, submit it.
(In reply to comment #96) > I think that's enough "I want this too" posts. If you have a patch, submit it. That's the nature of voting.
Flags: blocking1.8a? → blocking1.8a-
*** Bug 244343 has been marked as a duplicate of this bug. ***
Well, I just entered a new "bug" with the same issue because I did not find this one when searching before entering the new one. It looks like MANY people would like to see this feature implemented. There is (IMO) one single point that really makes the discussion about it and alternative ways of implementation questionable: interoperability with Internet Explorer. Face it, this is really important. And in this special case, the format is not even MS proprietary but even open! So I think there is not much to discuss about this. It simply is a feature that belongs to a modern browser. It is for me the sole reason why I still keep using IE. And because already four years are over now since the RFE has been initially entered, it does not look like it will happen soon ... I am offering a funding of US$ 250 if someone commits to implement it within a time frame of half a year from now on. Please, others: don't whine and discuss around this issue, but also offer funding in significant amounts so that we can get this cow off the ice.
(In reply to comment #99) [...] > I am offering a funding of US$ 250 if someone commits to implement it within a > time frame of half a year from now on. Please, others: don't whine and discuss > around this issue, but also offer funding in significant amounts so that we can > get this cow off the ice. I'll add $100 if a this is implemented by November 22,2004 (6 months from now :-).
If you all are serious about funding this project might I suggest you start a project on sourcesupport.org, much like the one that was started for the browser spellchecker: http://sourcesupport.org/?project=43. Note, the site seems broken at this every moment, but getting this project started and funded will add credibility to your claims. - rmjb
Please add your pledges to http://www.dezyne.net/codebounty instead of adding them as Bugzilla comments.
MAF (http://maf.mozdev.org/) is an extension which might be useful.
Here's another reason that adding MHTML support to Mozilla would be a Good Thing: MHTML is the format that OnFolio (www.onfolio.org) uses for its reports. As OnFolio becomes more widely used, the inability to access its reports when using Mozilla will become more significant as a disadvantage. (Of course, OnFolio also doesn't integrate with Mozilla when capturing information from the Web, but that's a different problem and one that is up to OnFolio to address.)
Here's another reason that adding MHTML support to Mozilla would be a Good Thing: MHTML is the format that OnFolio (www.onfolio.com) uses for its reports. As OnFolio becomes more widely used, the inability to access its reports when using Mozilla will become more significant as a disadvantage. (Of course, OnFolio also doesn't integrate with Mozilla when capturing information from the Web, but that's a different problem and one that is up to OnFolio to address.)
Just for info and inspiration: Convert any URL to a MHTML archive using native .NET code (CodeProject) http://www.codeproject.com/vb/net/MhtBuilder.asp
Adding some flames: Microsoft loves MHT for its (appearant) DRM capabilities ("security", "trade secrets"). US gov complains, MS defenseless because nobody else supports the format. <http://seattlepi.nwsource.com/business/apbiz_story.asp?category=1310&slug=Microsoft%20Antitrust> <http://www.heise.de/newsticker/meldung/52001>
Did you already notice that MAF can load and save MHTML ? http://maf.mozdev.org/ My IE-Saved files were displayed correctly and saved Files could open in IE later. So, the only thing to do is to integrate that basic function into the core of Mozilla. Up to now it needs some external programs and scripts. Also a decision about a standard format would be nice. I don't trust the MAFF format now, because who knows how long this will be supported. MTHML has a disadvantage because it's uncompressed. So a .mht.zip or .mhtz would be nice. If this would dissapear, at least with unzipping the docs could convert easily into the RFC-standardized MTHML.
Flags: blocking1.8a1-
*** Bug 276552 has been marked as a duplicate of this bug. ***
My Bug 276552 was marked as a duplicate of this bug. I don't see it that way, but I think as a user, not as a developer. It is related. I'll accept Matti's decision to group them together. Bug 276552 was entered to document the fact that the installer or the "check that Firefox is the default browser" operation didn't claim the MHT filetype. When I look at the file in file explorer, and doubleclick on the MHT file, IE will open with it. When I drag the file onto an open Firefox window, Firefox seems to render it correctly. I can tell XP that MHT belongs to Firefox, so this is a trivial issue, but it affects the look'n'feel of the product to IE users who are moving to Firefox. Separately ... I realize it's much later than the flurry of discussions last April, but from the viewpoint of an end user, the saved web page should be a "document". One document. One item that will open with the page the way it was originally. An HTML and a directory of junk to support it looks like mega-clutter to an end-user such as myself. The airline confirmed my reservation and I wanna save that fact. I hit a couple of keys and save a "document" proving that I was confirmed. I don't want a directory of (@#$#) in such a case. Such is the viewpoint of a long-time IE user just arriving in Firefox.
(In reply to comment #110) > My Bug 276552 was marked as a duplicate of this bug. I don't see it that way, > but I think as a user, not as a developer. It is related. I'll accept Matti's > decision to group them together. > > Bug 276552 was entered to document the fact that the installer or the "check > that Firefox is the default browser" operation didn't claim the MHT filetype. > When I look at the file in file explorer, and doubleclick on the MHT file, IE > will open with it. When I drag the file onto an open Firefox window, Firefox > seems to render it correctly. > > I can tell XP that MHT belongs to Firefox, so this is a trivial issue, but it > affects the look'n'feel of the product to IE users who are moving to Firefox. > > Separately ... > I realize it's much later than the flurry of discussions last April, but from > the viewpoint of an end user, the saved web page should be a "document". One > document. One item that will open with the page the way it was originally. An > HTML and a directory of junk to support it looks like mega-clutter to an > end-user such as myself. The airline confirmed my reservation and I wanna save > that fact. I hit a couple of keys and save a "document" proving that I was > confirmed. I don't want a directory of (@#$#) in such a case. Such is the > viewpoint of a long-time IE user just arriving in Firefox. The single file or document is what this bug is all about. IE for the Mac does this (and has for a very long time). Exactly as you describe, this capability is useful for keeping records of online purchases or simply saving web pages (which I do quite frequently). IE does this better than any existing browser for the Mac, bar none. iCab does it, but not as well. Just why such basic capability has been so willfully neglected by the developers remains a mystery. In view of the recent downward trend in both Mozilla and Firefox releases I am not sure that it matters much. I barely use either any more. The team appears to be spending more time proclaiming how great Moz/FF are rather than making it so. Even the "bounty" placed on this seems to have attracted absolutely no attention whatsoever. I am in hopes that some of the commercial browsers will do better, and soon.
Apologies for the bug spam (as this is slightly off topic) but I may have found a work-around that people may find useful in the interim. http://forums.mozillazine.org/viewtopic.php?p=1237227#1237227 Thought I'd post it here as there does seem to be such a strong demand for this functionality (if earlier comments are anything to go by).
*** Bug 285569 has been marked as a duplicate of this bug. ***
*** Bug 290938 has been marked as a duplicate of this bug. ***
No longer blocks: majorbugs
Flags: blocking-aviary2.0?
Flags: blocking-aviary2.0?
Maybe it would be easier to use the "data:" url scheme [1]. Firefox already supports it and the end result of having whole page in one file would be meet. [1] ftp://ftp.isi.edu/in-notes/rfc2397.txt
(In reply to comment #12) > This is easy to do for normal web pages, but rather hard when frames are > involved What about doing it for simple pages only, at first? When one needs to save a page with several frames, he can open the frame in a new window/tab, and save it as a complete page. In the meantime, I hope pages with frames will no longer exist in a few years. Paolo
(In reply to comment #116) > (In reply to comment #12) > > This is easy to do for normal web pages, but rather hard when frames are > > involved > > What about doing it for simple pages only, at first? When one needs to save a > page with several frames, he can open the frame in a new window/tab, and save it > as a complete page. > In the meantime, I hope pages with frames will no longer exist in a few years. > > Paolo If anything, it appears that web pages with frames are on the increase. The inability to save a page with frames (unless it was a temporary situation) would not remedy the underlying problem. I remain puzzled tha IE (for OS X) has had this capability for years and the Mozilla Foundation's leadership remains so clueless.
rbriscoe@satx.rr.com: i'd like to thank you for volunteering to fix this bug. when can we expect to see your first patch?
Assignee: file-handling → rbriscoe
(In reply to comment #118) > rbriscoe@satx.rr.com: i'd like to thank you for volunteering to fix this bug. > when can we expect to see your first patch? Josh Soref: Your "contributions" thus far to this bug have done nothing to help it in any positive direction. This is the second time you have reassigned it to an unwilling/unknowing bugzilla user. You have done so much for Mozilla; you should be above this.
Assignee: rbriscoe → file-handling
(In reply to comment #118) > rbriscoe@satx.rr.com: i'd like to thank you for volunteering to fix this bug. > when can we expect to see your first patch? Dear Whoeverthehellyouare: I'm not a coder and neither are most of the people who are *allegely* the target audience of the Mozilla Foundation's efforts. Your snide remarks have no bearing, and no merit, on the lack of guidance by the Foundation's managers. If you are one of them, so be it. If not, then it does not apply to you. If, as stated many times, the objective of the foundation is to produce a better browser than IE I would have to conclude that it has only partially met those objectives at this time. If you are a coder, then why don't you take a look at the bounty that was posted earlier? One of the problems with the management of Mozilla is the disorganization and lack of a cohesive plan...roadmaps notwithstanding and people, like you, who lack the ability to think and reason, despite their technical skills. If you have nothing better than your remarks to contribute to the discussion, may I respectfully suggest that you put your time to better use.
@Richard Briscoe: The use of HTML-FRAMEs decreases, there are statistics (at least for German web-sites) showing this. In XHTML 1.1 it is dropped entirely (though there will be XFRAMES as replacement). Nevertheless I think saving frames in MHTML files should be supported. I think this is not the place for this discussion and especially not the place for insults. A better place for the discussion would be forums.mozillazine.org. Harassing developers won't help your concern.
In reply to comment #115 Zbynek, see bug 121793
*** Bug 304372 has been marked as a duplicate of this bug. ***
nice-to-have, if someone comes up with a branch-safe patch (probably unlikely) we'd consider it
Flags: blocking-aviary2?
Flags: blocking-aviary2-
Flags: blocking-firefox2-
I also think this should be added to firefox core functionality. Now there exists an extension called MAF (mozilla archive format) that is quite good, has been in development for quite some time, and the developer has been evolving it. It can save .mht and read them too, but it still fails every now and then.
Why not saving to pdf to save complete webpages? Would'nt it be more usefull and easier to implement? The bug related is bug [url=https://bugzilla.mozilla.org/show_bug.cgi?id=162659]162659[/url) and is marked as wontfix maybe it should be reconsider.
* Can't extract original HTML, CSS, JavaScript, ... * Probably very VERY complicated to get even basic dynamic behaviour right (if at all); in MHTML, at least stuff that only works inside the page should continue to work. * Can't display in Browser without plugin. Also, it's possible to save as PDF already: Print, and use a PDF printer. Somebody write a nice extension around this, to provide a separate button and menu command? (In reply to comment #126) > Why not saving to pdf to save complete webpages?
(In reply to comment #126) > Why not saving to pdf to save complete webpages? Would'nt it be more usefull PDF is print-page orientated, not screen orientated as HTML, the original page look gets lost, images get resized. And to view a saved page, I want to use Firefox and not Acrobat reader (not even the reader plugin).
Firefox can use xml to save page to single file (use another extension). Define a dtd to the xml. And when firefox open the xml, then show the origin face. And I think it will be a standard for another reader to support the file. The mht is not a standard.
MHT *is* a standard - did you read the bug description? It is RFC 2557.
Save as one .mht file (and open .mht files) is very useful to avoid clutter; for now I still need my Internet Explorer. - But what is this stuff about MAF? Note that IE also sometimes messes up. - And what about saveing as .xml?! I now tried to save THIS page as .xml and it seemed to work (it's NOT yet in the save as menu!) but then I got an error message when trying to open it, both with IE and Firefox: "XML Parsing Error: mismatched tag. Expected: </link>." ...
*** Bug 342615 has been marked as a duplicate of this bug. ***
MAFF is a zipped collection of completely saved webpages. MHT is what Thunderbird calls EML. Firefox's complete save function misses some style stuff. Maybe that affects MAF's MHT output compared to IE's. Save Complete (https://addons.mozilla.org/firefox/2925/) should fix that.
As for the tag error in XML: <link> tags should be <link /> tags there.
I'm using the MAFF regularly http://maf.mozdev.org/ So, many of my saved docs are now in the *.maff format. The most important thing is to have kind of a standard. It would be a deasaster if this type of file would not be supported any more in future. As a core functionality, this should be part of the Firefox program. As an extension there is always some trouble with upgrading. Sometimes I had an infinite-loop of spawning empty windows; at the moment it's working again. It seemed impossible to repair/workaround the Javascript Plugin. The Javascript integration in the file handlers was very chaotic. I would suggest a clean implementation in the Firefox core code. Just because file operations need standards and document types are the most important data structures. Web pages dissapear quickly, so archiving of interesting content is very important.
The previous commenter mentions the ODF format. Ought not Firefox support ODF too? It would finally give users the benefit of a single-file archival format, and permit some added interoperability with office software.
(In reply to comment #136) > The previous commenter mentions the ODF format. Ought not Firefox support ODF > too? Open a different bug for that, please.
No need for a new one, ODF support is bug 114376.
Bug 114376 is for display of ODF files. OTOH this bug is concerned with a way to save web pages into a single file. Since ODF is standardized and spreading, I think it would make an optimal target format to satisfy the functionality requested here. IMO either MHT or ODF would be very nice to have.
Please don't spam this bug about ODF. Yes, we want to be able to view and/or save ODF files, but this bug is about MHT (and possibly MAFF) - there already exist documents in this format, especially MHT mostly saved by IE users, and Firefox needs to be able to view these.
(In reply to comment #135) > I'm using the MAFF regularly > http://maf.mozdev.org/ Soon to be obsolete once 2.0 comes out. "MAF 0.7.0 is currently under development and will be compatible with Firefox 1.5 only."
> Soon to be obsolete once 2.0 comes out. "MAF 0.7.0 is currently under > development and will be compatible with Firefox 1.5 only." > No I have my problem. MAF doesn't work with 2.0, when I force to the new version it has side effects with other plugins. Once I tried to fix MAFF, this is really a piece of spaghetti code hacking. It's not the fault of the programmer. The data format is just such a basic function, that has to be implemented in the core of Firefox. It's risky to save documents as MAF, because you are never sure, if you are able to read it in later versions. Microsoft has it's elegant way. Save as MHTML, it's standarized and it's supported for years. Why don't concentrate on the most elementar things. Other features of Firefox are less important and more luxuary.
AFAICT MAF is much safer than MHTML because you can always just unzip it and you get all files. For me it's almost the same difference as zipped OpenOffice XML files versus binary image of memory in MS Office. (Not speaking that it's smaller of course.) But yes, Firefox should be probably able to read MHTML just to ease transition of people from IE. And as for saving - 20 years old technology invented for emails in times, when computers understood ASCII only... No, we can do better :) Internal structure of MAF file is pretty good (it remembers scroll position for example). But what we definitely need is toolkit to be able to save really ALL files in page (yes, I'm speaking about references in CSS).
BTW: I lost a potential convert from IE to Firefox today because he saves recipes from e.g., http://www.verybestbaking.com/recipes/detail.aspx?ID=18476 as MHTML files to his hard drive. Firefox can't do MHTML, and Firefox offers a silly default filename ("detail.aspx.htm"). So I lost a "customer". :-(
(In reply to comment #144) You could show him how to install a free pdf printer-like driver like PrimoPdf and convert the page to pdf. Much more usable than the .mht crap which is often useless when IE fails to convert all remote images to local ones. Generally, the above tip is a very good workaround for this bug, except for pages that don't print properly or for users who don't have the slightest idea about installing printer drivers.
(In reply to comment #145) > You could show him how to install a free pdf printer-like driver I did, but (A) it's additional work and not what he wanted, and (B) MHTML preserves link and flexible layout and PDF doesn't. Also the fact that Firefox still offers some useless filename (at that site) instead of the "title" killed it for him. > users who don't have the slightest idea about installing printer drivers. When people blame the user for something the software should enable them to do, I stop listening.
(In reply to comment #145) > (In reply to comment #144) > You could show him how to install a free pdf printer-like driver like PrimoPdf This is like some graphical artists saying in the early nineties to build websites using PDF only. A browser is about HTML and one expects that a browser can save a HTML page and reload it at a later time. What would the users say if Openoffice couldn't save and reload a document?
This Firefox extension solves this bug: http://www.unmht.org/unmht/en_index.html
> You could show him how to install a free pdf printer-like driver like PrimoPdf > and convert the page to pdf. Much more usable than the .mht crap Why do you call a standard format RFC 2557 crap and then recommend a proprietary format PDF as its alternative? We cheer when Firefox implements standards that IE doesn't, but when IE implements a standard that Firefox doesn't, we call it crap for the sole reason that IE popularized it? IMHO, embracing standards and hating IE are two different things, not to be confused.
If you are on a Mac, you might want to look at how iCab 4.1.1 saves a page. Perhaps it could offer some ideas for either an add-on or change to the browser itself. It not only saves the contents of the page, but preserve the URL so that you can check a page to see if there have been changes (if the page still exits on the web). <http://www.icab.de/>
(In reply to comment #146) > (In reply to comment #145) > Also the fact that Firefox still offers some useless filename (at that site) > instead of the "title" killed it for him. FWIW there's also this https://addons.mozilla.org/en-US/firefox/addon/834
The latter is bug 254139 (File | Save Page As should default to <title>, not filename)
Comment #151 from Justin Kerk: > FWIW there's also this https://addons.mozilla.org/en-US/firefox/addon/834 This extension uses IE to save MHT file == works only on Windows platform. The unmht extension I wrote about: http://www.unmht.org/unmht/en_index.html is multiplatform. Works very well on my Linux. That is why I think it fully resolves this issue.
Personally, since I have IE-Tab installed, I simply reload the page in IE-mode and then use the File>Save_Page_As menu from there to have the trident engine save it as an *.mht file. fwiw
(In reply to comment #153) > Comment #151 from Justin Kerk: > > FWIW there's also this https://addons.mozilla.org/en-US/firefox/addon/834 > > This extension uses IE to save MHT file == works only on Windows platform. Uh, no? It has nothing to do with MHT, just the default Save As filename.
As there is no assignee on this bug, I allowed myself to change the component from 'file handling' to 'serializers' as that seemed to better fit the bug. I apologize if I am mistaken.
Assignee: file-handling → nobody
Component: File Handling → Serializers
Keywords: conversion
QA Contact: chrispetersen → dom-to-text
I found this Microsoft patent: http://www.patentstorm.us/patents/6886132.html I guess it rules out MHTML support in other browsers?
Oh well, I guess I misunderstood the patent. There is a clarification here: http://www.informationweek.com/news/global-cio/showArticle.jhtml?articleID=162100345
(In reply to comment #153) > The unmht extension I wrote about: > http://www.unmht.org/unmht/en_index.html > is multiplatform. Works very well on my Linux. > That is why I think it fully resolves this issue. I have tried UnMHT (using Win XP). Works very nicely. I am withdrawing my vote for this bug because I agree that the issue is resolved.
Can't we incorporate UnMHT in Firefox itself?
Worth noting that Google Chrome's had support for this (albeit hidden behind about:flags) since version 25, released in February 2013.
Basic information about .mht file should also be automatically added within each saved .mht file. So when user would open previously saved .mht file, there would be a "notification bar" on top of that opened offline webpage with: - saved file name info (for example: page.mht) - saving time and date - original webpage link (URL source), from which the .mht file was saved This functionality is already offered by the UnMHT add-on (among the others). But the new signed version doesn't work properly anymore. Nevertheless, this should be integrated in the browser by default, because it is probably the most useful way of saving the whole webpage in one file.
IMO this should be a mainstream document format, instead of being hidden behind an add-on in ff or a flag in chrome, and this bug is quite important. Like for example to make HTML so pervasive it replaces Word docs. Currently HTML is the most used format in the world, but for the vast majority of the world population it is read-only. HTML is impractical as a document format in that it requires at least to throw a bunch of files in a directory, while doc or odt are single files. This is a real issue for most people. MHTML solves this issue and moreover, it has the potential to store not one but several documents (pages) in a single archive file, brilliantly allowing hyperlinks among them, including deep links to # parts. Additionally, (M)HTML documents can be made responsive, which turned out to be !important nowadays, saving users the hideous per-line horizontal scrolling of so many docs and pdfs. To complete the picture, it would be excellent to have a WYSIWYG editor available, like for example w3c's Amaya http://www.w3.org/Amaya/ (not mantained since 2012). This particular editor's paradigm departs from the usual word-like UIs in that it fosters the structure over the form; I think this is the way for the future (like ff overcoming IE). Amaya could be reimplemented in JS with ease, enabling ff as an editor (like Netscape). With some JS magic this would allow normal people to use HTML for text docs, but also slides and why not tables with some custom calculations. Normal people (non-IT) should be able to attach public web pages either as links and local pages as MHTML attachments, download them and being able to open, edit and resend said pages as single files. The line that divides could storage and web hosting should blur.
On what grounds was this marked as WONTFIX?
(In reply to ajf from comment #166) On the grounds that they don't really care to explain themselves to us. Like many other decisions and lack-of-decisions over the past several years. Sorry for sounding bitter but - you really should not be so surprised.
On the ground of Redmond?
Blocks: 1261339
Mass bug change to replace various 'parity' whiteboard flags with the new canonical keywords. (See bug 1443764 comment 13.)
Keywords: parity-ie
Whiteboard: parity-ie

Just highlighting that this issue isn't dead. It is as important as ever to be able to load and save .mhtml files.

We need a better alternative to sharing documents than PDF files. MHTML files are going to be more accessible, and more convenient for people to share content.

See this appraoch to PDFs from the UK - https://gds.blog.gov.uk/2018/07/16/why-gov-uk-content-should-be-published-in-html-and-not-pdf/

Now all Chrome based browsers can load/save MHTML files to make it easy to share a snapshot of a site in time. Unfortunately FF users still need to install an extension..

(In reply to Mike Gifford from comment #172)

Now all Chrome based browsers can load/save MHTML files to make it easy to share a snapshot of a site in time. Unfortunately FF users still need to install an extension..

Can't someone just pull that code from the extension, and build it right into Firefox itself? Isn't this how a lot of programming happened here?

(In reply to Worcester12345 from comment #173)

(In reply to Mike Gifford from comment #172)

Now all Chrome based browsers can load/save MHTML files to make it easy to share a snapshot of a site in time. Unfortunately FF users still need to install an extension..

Can't someone just pull that code from the extension, and build it right into Firefox itself? Isn't this how a lot of programming happened here?

Or just pull the code from Chromium?
I'm sure such a feature would be welcomed by many users. :)

Mike Gifford <mike@openconcept.ca> wrote :

Now all Chrome based browsers can load/save MHTML files to make it easy to share a snapshot of a site in time. Unfortunately FF users still need to install an extension.

What? An extension? Did I missed something? I was under impression that since the notorious ver. 57 (web)extensions are no longer capable of adding a ability to view a format, that Mozillaʼs browser itself does not support; was I wrong?

Flags: needinfo?(mike)

For the purpose of saving complete webpage in one file, much more convenient file format would be HTML instead of MHTML, since HTML is widely-supported file type.

One of web extensions that already make this possible is called "Save Page WE". It has been actively maintained. Users can save a complete webpage or selected items. Further, users can save one or more selected webpages (i.e. selected browser tabs). Information bar at top of each saved HTML file is supported as well and can be enabled among settings. Web extension is available for Firefox - link and Chrome - link.

In this case it is questionable whether the development of native MHTML support makes sense nowadays – except for the purpose of opening old (archived) MTHML files. After all, this feature has been pending for more than 20 years.

I like Leon's pragmatic approach.
I think on the users scenarios, like dharing a page with somebody else.
One can send the link, but not always. For example when the page is local.
Also, the link is not useful when trying to share a form with some fields already loaded.
Sendint the HTML is a chore, it usually involves a number of files. Not useful for "normal" people. Let alone all the alarms raised by an email client receiving all those JS files.

People are sending each other MSWord or PDF files. The MSWord files don't have JS but VBA and fostered a terrible wave of destructive viruses (you opened the file and the Windows PC was nuked) because VBA had access to the OS bowels.
People are also sharing PDF files, which brings us back to the 1999 HTML pages that had fixed format and a gazillion spacer.gif refs.

For now, and 20 years later, people can't share web pages, or sets of web pages (what Leon mentions as a tabs set).
Ideally those complete web page(s) would travel encapsulated in a compressed file, optionally encrypted.
this would top the MSWord or PDF formats by being responsive and by including some of the link targets.

The issue is how to protext normal people from scams.
Like external links to bad places, or JS attacks.
I don't know, but I think that nowadays it should be easier than 20 years ago, when the VBA viroses were averriden.

Leon <leon.slo@outlook.com> wrote:

For the purpose of saving complete webpage in one file, much more convenient file format would be HTML instead of MHTML since HTML is widely-supported file type.

HTML is not a format for saving a complete page to a single file.

One of web extensions that already make this possible is called "Save Page WE". It has been actively maintained.

And another one (arguably, more actively maintained) is ‘Single File’.

In other words, by ‘format’ you mean specific hacks, employed by those extensions.

Widely-supported, you say? The width of their support among webbrowsers is a round sum of 0. Zero browsers support saving to that format. Please correct me, if I am mistaken.

In this case it is questionable whether the development of native MHTML support makes sense nowadays – except for the purpose of opening old (archived) MTHML files. After all, this feature has been pending for more than 20 years.

Please note, that if you count third-party addons, than prior Mozilla decided to trash all the labour of XUL-extension devs, they had managed to develop at least couple of addons, that implemented MHTML: one was ‘MAFF’, another had some self-descriptive name; so the issue was not that outstanding until year 2017.

In any case, I strongly doubt, that the lack of adoption of interchange format in a browser with usage share, that all recent years have been positively heading towards a statistical error, is a good measure to make judgements about its future.

The truth is that MHTML is simply the only saving format, supported by the most widely used browser on this planet: Google Chrome for Android.

(In reply to Dmitry Alexandrov from comment #178)

HTML is not a format for saving a complete page to a single file.

Not by original design, but it can be modified for this purpose with the help of extensions, as you mentioned.

Widely-supported, you say? The width of their support among webbrowsers is a round sum of 0. Zero browsers support saving to that format. Please correct me, if I am mistaken.

You are right, but I wasn't talking from the perspective of saving. My point was that you don't need an extension to be able to open single-file HTML. You only need such extension for saving a webpage into one file. In terms of universal ability for opening it, single-file HTML is recognized by every major web browser: tested with single-file HTML (saved by Save Page WE) on Firefox and Edge for Windows, as well as Chrome and Brave for Android.

Redirect a needinfo that is pending on an inactive user to the triage owner.
:hsinyi, since the bug has recent activity, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(mike) → needinfo?(htsai)

HTML is the most used language in the world.
Sadly, we all can read it (rendered) but comparatively few people can write and communicate it.
It should be mainstream-easy to write a page, or a linked pages cluster, using a tool similar to MSWord or the various WYSIWYG online editors, and then to send said page(s) wuth ease.
As the result of this lack of tools, people are sending each other PDF documents, an infamous printing format that generally requires to display a portrait A3-sized page in a landscape screen, and can't reflow its text(!).

IMO this MHTML format is the seed, a first step, to take over the documents communications world.

Flags: needinfo?(htsai)
See Also: → 1784695
Severity: normal → S3

(In reply to Leon from comment #179)

(In reply to Dmitry Alexandrov from comment #178)

HTML is not a format for saving a complete page to a single file.

Not by original design, but it can be modified for this purpose with the help of extensions, as you mentioned.

The problem is that it is difficult for the average person to differentiate between single-page HTML and HTML pages saved as separate files. If they want to send an HTML file to a friend, they must either bundle up a bunch of files (which is beyond the technical expertise of many people), point them to a website, or just give up.

MHTML is a great format for sending web pages. Beside the obvious single-page format, IT IS SUPPORTED BY ALL MAJOR BROWSERS EXCEPT FIREFOX. (I don't count Safari, which also doesn't support it, because Macs don't count as computers.)

Google Chrome, Microsoft Edge, and Opera all open MHTML files natively, without extensions. WHY IS FIREFOX SO FAR BEHIND???

Widely-supported, you say? The width of their support among webbrowsers is a round sum of 0. Zero browsers support saving to that format. Please correct me, if I am mistaken.

You are right, but I wasn't talking from the perspective of saving. My point was that you don't need an extension to be able to open single-file HTML. You only need such extension for saving a webpage into one file. In terms of universal ability for opening it, single-file HTML is recognized by every major web browser: tested with single-file HTML (saved by Save Page WE) on Firefox and Edge for Windows, as well as Chrome and Brave for Android.

So...your argument is that people who are non-technical should have to learn how to install an extension in Firefox to do what EVERY OTHER MAJOR BROWSER CAN DO NATIVELY?!?

It seems much more reasonable to add support for saving HTML pages to a single-file directly into Firefox, in all major Web archive file formats, including MHTML/MHT, MAFF (old Firefox format), WARC (Internet Archive's Web ARChive format for entire sites), HTMLD (HTML Directory), and even Webarchive (in Safari browsers).

The default should be MHTML/MHT:

(1) It is widely supported.
(2) It doesn't end with HTML/HTM, so it won't be confused with those
(3) It is compatible with email messaging (being identical or almost identical to the EML format)
(4) It is a handy way to save web pages into a single-file and send them to others.

The only one that is better is the MAFF format, because it allows for compression. (I don't know if MHTML/MHT does or not, but it should...)

I was extremely angry when Mozilla removed the ability to save and read MAFF and MHTML/MHT formats. I've saved hundreds of pages in those formats and really don't want to change browsers or convert every single file to the new and less useful single-page HMTL format.

If security is a concern, then turn off this feature by default, but let users select it in Settings. Browsers should never become so secure that their usefulness disappears. Security must be coupled with ability for software to have value.

ADDING MHTML/MHT SUPPORT SHOULD BE A PRIORITY FOR FIREFOX!

(In reply to Leon from comment #176)

For the purpose of saving complete webpage in one file, much more convenient file format would be HTML instead of MHTML, since HTML is widely-supported file type.

The problem with your argument is that complete page HTML is not distinguished from HTML which is scattered into dozens and dozens of images and folders and files. That makes it confusing for the average user who just wants to send a webpage to someone else. Links are inadequate and single-file HTML works, but to repeat the obvious, it gets confused with non-single-file HTML.

So the single-file HTML is NOT useful in this case. It's the opposite of useful.

One of web extensions that already make this possible is called "Save Page WE". It has been actively maintained. Users can save a complete webpage or selected items. Further, users can save one or more selected webpages (i.e. selected browser tabs). Information bar at top of each saved HTML file is supported as well and can be enabled among settings. Web extension is available for Firefox - link and Chrome - link.

Save Page WE is a fantastic extension! However, it still doesn't address the main problem... Single files to send to other people should be named with something OTHER than HTML/HTM. Otherwise, normal users will not send those files correctly, losing images and other information from a web page.

In this case it is questionable whether the development of native MHTML support makes sense nowadays – except for the purpose of opening old (archived) MTHML files. After all, this feature has been pending for more than 20 years.

MHTML/MHT makes fantastic sense today!

At this point, we can save web pages as "Web Page, complete" or "Web Page, HTML only" natively in Firefox and the HTML single-file format from the "Save Page WE" or "Singlefile" extensions.

"Web Page, complete" saves dozens and dozens of objects...for this page it saved 162 separate files and/or folders.

"Web Page, HTML only" doesn't save anything but the basic HTML, losing all the images and other important features on a web page.

The single-file HTML format from the extensions is great...but it saves with the HTML extension. So that means that people who want to send a file or open the file can get easily confused. It also doesn't allow for compression, which would be tremendously useful.

Thus it is proved that MHTML is very useful for many, many people, since normal users who could use this feature represents the vast majority of browser users.

Plus, I really like it too.

See Also: → 1332920

(In reply to Biju from comment #83)

And thunderbird can save *.eml file, and also can view *.mht and *.eml files

This comment is 21 years old and still valid, as far as I understand.
So TB has this functions out of box but FF can't view mht and needs third-party addon for saving.
And there is no more common option to save a page to single file (pdf is not capable of complex layout).

I recently rediscovered the MHTML single-file format while looking for a better way to save AI conversations, and it turns out to be incredibly useful for my needs. I often have interesting discussions with AI, so much so that I’ve resorted to saving some of them in .docx files. However, the formatting was far from ideal.

Upon discovering MHTML, I realized it perfectly suits my use case: it’s fast and convenient (Ctrl+S, name the file, done) compared to opening a word processor and manually copying and pasting content. More importantly, the final result closely preserves the original appearance of the live webpage, making it much more readable.

Given how long this format has existed, I was genuinely surprised that it hasn’t been implemented yet. I strongly support its adoption in Firefox.

@vlakoff there is an extension that can be used for this:

https://www.getsinglefile.com/

This is the format we all should be using, instead of the ancient PDF that forces us to read portrait pages in landscape screens without a chance to reflow the text.
Not only for @vlakoff's use case, but for everything except the most boring legal docs, those we all skip.
I think that we could start by binding the MHTML format to the PC's default browser, so those files could be opened with a double click.
And perhaps having a chrome less browser choice; in this case less might we'll be more.

Check comment https://bugzilla.mozilla.org/show_bug.cgi?id=40873#c165
about the same as above, 10 years ago

Attached patch bug-40873.patch — — Splinter Review

Hey Folks (Emilio, Smaug and Baku)!

I'm interested if Mozilla would accept a patch here if someone worked on it.

  • I've got a working patch that I hacked together to exporting .mhtml files, it adds an export file type in the save menu.
  • I think it's worthwhile just landing export if we're happy with it and I'd be happy to work in it.
  • The patch I've worked on needs some more validation, I've tested only a couple of pages but wanted to validate if it'd be accepted first. I'm also not expecting this patch to be close to what should be landed either, happy to discuss further direction.

On my travels, I spotted that there's a bunch of unanswered issues:

I think they're ok, to leave as is. This isn't a regular web standard that will impact a significant number of users. I think issues could be resolved iteratively.

The follow up work here would be:

  • Supporting mhtml viewing, because of prior Chromium security issues in viewing I think a different origin attribute might be worth considering to reduce storage exfiltration.
  • Adding the extension API to download mhtml snapshots from the background script.

Thanks!

Assignee: nobody → jonathan
Flags: needinfo?(smaug)
Flags: needinfo?(emilio)
Flags: needinfo?(amarchesini)

(In reply to Jonathan Kingston [:jkt] he/him from comment #191)

Thanks for looking into this Jonathan!

I'm interested if Mozilla would accept a patch here if someone worked on it.

I don't have any problem with it in principle, but I don't have the whole context here. Looking a bit at the backscroll here I don't see any obvious objection, so seems to me worth experimenting with / landing behind a pref at least. We probably want to check with product before enabling by default but that can be done alongside the implementation, and as long as the implementation isn't super-invasive I don't expect others to object...

  • Handling of @ import doesn't always work in Chromium
  • Shadow DOM changes seems a little hacky.

Yeah serialization of shadow dom is always fun. But it seems we can use declarative shadow dom nowadays for that if needed?

I think they're ok, to leave as is. This isn't a regular web standard that will impact a significant number of users. I think issues could be resolved iteratively.

I agree they're probably not blockers for an initial implementation.

The follow up work here would be:

  • Supporting mhtml viewing, because of prior Chromium security issues in viewing I think a different origin attribute might be worth considering to reduce storage exfiltration.

Should this really be a follow-up? It seems to me it might be worth doing it the other way around (first implement viewing, then exporting)? Exporting something we can't see isn't quite useless, but... In any case landing some initial version of the export code seems fine since you already got started on it, but I would expect turning it on by default to be blocked on us being able to display the file...

Regarding the origin stuff, should we just use an opaque origin (like file:// or data: uris) instead? Not sure we need a new origin attribute, you conceptually want each individual mhtml archive to be "isolated" in a way, right? An opaque origin gives you that.

Flags: needinfo?(emilio)

Should this really be a follow-up? It seems to me it might be worth doing it the other way around

Oh I agree on the framing of blocking release for sure. Certainly a pref to start with too. I agree reading would make more sense to start with, I'm most interested in exporting crawls from extension context if I'm honest (kinda was sucky to write an extension that's Chrome only and this is a blocker). Essentially I wanted it as an archival format, I'm not really reading it at all (in fact I'm converting it back to html for my use-case too).

It seems like Chrome uses file extension mapping to mthml parsing after mime checking it additionally preventing all scripts and only allowing in file:// loads.

Not sure we need a new origin attribute, you conceptually want each individual mhtml archive to be "isolated" in a way, right?

Yeah fair, I'll check their prior security issues some more too. I think restricting script loads or it's ability to load network requests at all is closer to what model that's suitable.

Yeah serialization of shadow dom is always fun. But it seems we can use declarative shadow dom nowadays for that if needed?

Good point yeah! I think we can iterate on this rather than expect it all to be supported in one patch. (which it seems like you're happy with too).
I think there's enough test cases we can switch to WPT to get a working solution.

In terms of alternatives here there's WARC/WACZ that wget and http archive use, it's much heavier on implementation (it's sort of similar to MHTML+HAR in fidelity) the implementation would be radically different and similar to HAR would need to be a dev tool only opt-in due to performance and complexity.

Anyway, thanks for your attention here. I'll continue with implementing a prototype that does import and export.

Flags: needinfo?(smaug)
Flags: needinfo?(amarchesini)

I think there should be at least some ideas how to implement viewing before adding exporting, since I don't think we should add only the latter.
Then adding exporting behind a pref would be fine.
dmcintosh has been looking into this too recently.

Flags: needinfo?(dmcintosh)

Broadly the viewing support is:

  1. MHTMLStartup.sys.mjs
    Registers .mhtml/.mht → "multipart/related" via category manager

  2. components.conf
    Registers MHTMLDocumentLoader as Gecko-Content-Viewer for "multipart/related"

  3. When Firefox loads an MHTML file:
    docshell → content type = "multipart/related" → MHTMLDocumentLoader.createInstance()

  4. MHTMLDocumentLoader:

    • Changes content type to "text/html"
    • Gets the standard HTML document loader factory
    • Wraps it with MHTMLStreamListener

  5. MHTMLStreamListener:

    • Buffers all MHTML data in onDataAvailable()
    • In onStopRequest(): parses MHTML, converts to HTML with embedded data: URLs
    • Feeds converted HTML to the underlying HTML viewer

I did have a DocShell implementation but this seems much cleaner and is working.

Subresource loading (including iframes) and principal handling are perhaps the trickiest things to figure out.

I was kind of assuming that they would be treated virtually like data uris...

But data urls aren't virtual, they are there in the source code, in src attributes or so. mhtml is different, no? It defines some resources in the .mhtml.

But data urls aren't virtual, they are there in the source code, in src attributes or so. mhtml is different, no? It defines some resources in the .mhtml.

They support cid:, relative and absolute paths. The latter doesn't seem super common but I think it's handled by Chrome. cid: are references to the embedded mime chunks.

cid: I think can just be a new protocol handler that knows where to get it's data from.

The absolute/relative HTTP paths, I was thinking would probably be best handled with a custom principal or load info that routes it to the document also.

This what I get for not CC'ing myself on this! :P Thanks smaug for the needinfo, hadn't realized things were happening here. I'll comment quickly before I can take more of a look later.

I've been working on opening MHTML files a bit over the last few weeks as an exploratory/learning thing, only the opening part though. I've uploaded what I've made so far to GitHub; it seems like my approach is pretty similar to what you've done. I haven't read much of your patch yet, I'll try to take a look later today.

Going off of your comment, it looks like we both take the same approach of intercepting the content (I used nsIStreamConverter instead of Gecko-Content-Viewer, not sure how they differ?), then translating URLs in the document into data URIs. I don't like this approach (that's why I reached out to smaug), but it does kind of work. I feel like there's something with synthesizing responses like service workers do that might work better, or maybe your idea with custom principals or loadinfo might work too, I'm not familiar with any of this. (I usually work on frontend-y things and the installer :) )

My parser is also a lot longer (and a bit ugly internally), but I do like how it can issue warnings for some issues with the file—Chrome's 'malformed multipart archive' annoyed me. Might be worth mixing the two patches and tests.

Re shipping this, I've been talking to product a little bit about it—from what I can tell the thinking is that shipping MHTML for the sake of shipping MHTML might not fly, but using it to satisfy some usecases might work. I'll keep talking about that internally, especially if these patches start coming together. I'll note that the opening half is technically bug 18764, which was wontfix'd due to not being prioritized, but internally I haven't really seen much opposition to the concept.

Flags: needinfo?(dmcintosh)

Going off of your comment, it looks like we both take the same approach of intercepting the content (I used nsIStreamConverter instead of Gecko-Content-Viewer, not sure how they differ?),

Yeah I think the DocShell approach I had was much more mature in that sense, it intercepted the requests properly and assigned an opaque origin to the loads. I gave up because it had trouble loading the files and felt a little heavyweight but I think the current approach is far too simplistic.

Might be worth mixing the two patches and tests.

Yeah not protective, I think having ref rendering tests between mthml and html would be nice.

from what I can tell the thinking is that shipping MHTML for the sake of shipping MHTML might not fly,

I mean it's only going to be visible as a single export option in save. My expectation is it could be argued as an extension parity case. I think if the implementation isn't too cumbersome then that's enough justification in my mind.

I feel like there's something with synthesizing responses like service workers do that might work better, or maybe your idea with custom principals or loadinfo might work too, I'm not familiar with any of this. (I usually work on frontend-y things and the installer :) )

I think I can continue to explore these and we can compare and contrast easily.

but using it to satisfy some usecases might work.

My use case is refine.page a currently Chrome only extension that there's kinda no alternatives, Single File is the only close alternative, there's loads of reasons not to use it imo.
Additionally I wanted this only a few months back for RAG'ing page content, the snapshots of pages via an extension is a good stable format to use for that.

Attached patch out-interceptor-approach.patch (obsolete) — — Splinter Review

Yeah modelling it on loadinfo seems the cleanest for a more indepth approach that isn't doing parsing.
I was thinking the ID could be stored on the principal but they're not really designed for storage and more load info. I can then use that to store an index into a global registry that keeps track of mhtml loads.

I feel like there's something with synthesizing responses like service workers do that might work better, or maybe your idea with custom principals or loadinfo might work too

The current approach I'm using seems in this direction. It's a bit like a blob handler for the document itself which then intercepts the document requests like a worker does.

Attached patch out-interceptor-approach.patch (obsolete) — — Splinter Review
Attachment #9535840 - Attachment is obsolete: true
Attached patch out-interceptor-approach.patch (obsolete) — — Splinter Review

I need to do a bunch more validation (namely I didn't check iframes, I think we might need to propagate the loadinfo flags downwards into the child frame), testing and reading but I'm pretty happy with this approach now.

I explained this new approach more above. It'd be good to validate with product regarding what would block landing this type of work. We could split out the export, but we'd lose some ability to do comparison tests of HTML->MHTML and then reftesting those two files.

I think the only real missing part now is adding a pref to wrap it.

Attachment #9535852 - Attachment is obsolete: true
Flags: needinfo?(dmcintosh)

Wow, thanks!

I've taken a quick look at the opening half of the patch, and it looks pretty neat. Main question: what commit is it based on? :) I tried applying it, and it didn't seem to apply to 43c97a06 or 33bba5cf with git apply, but I'm also not used to applying patches off Bugzilla so maybe I'm doing it wrong. I'll look at it more once I can apply it properly.

I tried implementing the opening part of your approach based on your patch, and I'm a little unsure how it works—I think? nsDocShell::ShouldIntercept et al are only called in the parent process, but it looks like the actual MHTMLArchive storing the contents is only in the child process, which'd mean nothing gets intercepted...? My attempt doesn't seem to actually intercept requests, but my approach is quite different so it could work out in yours.

I'm also a little unsure whether nsDocShell's implementation of nsINetworkInterceptionController is the right place to put this, since e.g. FetchDriver also implements that interface, but I figure that's a 'figure that out once it works' problem.

Lastly, I'm thinking that we might want to avoid naming the classes 'MHTML' unless they literally deal with MHTML syntax, since it's possible other uses are found for the code and/or other formats are supported. If you have suggestions for a better name, let me know; I've been going with OfflineArchive, but I don't know if that's much of an improvement. Certainly can wait until a better name arrives. (SavedArchive? RestrictedViewOfTheWeb?)

Re product, I'll be speaking to them tomorrow so I can discuss it then.

Flags: needinfo?(dmcintosh) → needinfo?(jonathan)

Sorry haven't had chance to revalidate the patches some more.

what commit is it based on

b82cded8c5b732c2ea15b7871d14e13b5fadeffd it should be based on. I can make you a git commit instead, might be me as it's been a while since using hg so I'm using git as a the flow we used to use, forgive my external contributor noise if it's me.

nsDocShell::ShouldIntercept et al are only called in the parent process, but it looks like the actual MHTMLArchive storing the contents is only in the child process, which'd mean nothing gets intercepted...

I think you're right, sorry. The test case I had was all embedded data/file URLs. Let's get a test case written to validate sub resource loaded in some complex cases.
My intent here was to have this handled in the parent process and then IPC message though to child process.

I'm also a little unsure whether nsDocShell's implementation of nsINetworkInterceptionController is the right place to put this, since e.g. FetchDriver also implements that interface, but I figure that's a 'figure that out once it works' problem.

Yeah let's get this working in this format, then shuffle things about.

Lastly, I'm thinking that we might want to avoid naming the classes 'MHTML' unless they literally deal with MHTML syntax,

Yeah I was thinking this too. Apple uses WebArchive, there's some room for overloading that or riffing from it perhaps. Chrome's extension API here is chrome.pageCapture.saveAsMHTML

Flags: needinfo?(jonathan)

(In reply to Jonathan Kingston [:jkt] he/him from comment #202)

My use case is refine.page a currently Chrome only extension that there's kinda no alternatives, Single File is the only close alternative, there's loads of reasons not to use it imo.
Additionally I wanted this only a few months back for RAG'ing page content, the snapshots of pages via an extension is a good stable format to use for that.

I'm very happy to see a native implementation which is better.
For now, I use SingleFile to save a lot of pages, it works very well.
But what are the reasons not to use it?

But what are the reasons not to use it?

  • License if you're bundling it.
  • It's not quite as stable as using the browser to do this work. I had a bunch of edge cases in a directory, I couldn't find fixes for.

(In reply to Jonathan Kingston [:jkt] he/him from comment #209)

But what are the reasons not to use it?

  • License if you're bundling it.
  • It's not quite as stable as using the browser to do this work. I had a bunch of edge cases in a directory, I couldn't find fixes for.

Thanks for the answers, I feared you wanted to say that they track my visited pages and collect to much data.
I really hope your code gets merged and it's much faster than SinglePage.

Backup of moving the patch to the parent process. This seems to be working much better now.

Attachment #9535855 - Attachment is obsolete: true

This fixes rendering of CID, I've not seen any disparity with Chrome now.

Attachment #9536774 - Attachment is obsolete: true

Duncan does it make sense to sync on this, I think we're close to being landable. What does product say?

Additionally in:

Flags: needinfo?(dmcintosh)

Thanks for working on this and submitting the Phabricator revision!

I feel like the serialization improvements (the CSS url extraction and your shadowdom/canvas work) might be good, but they might be better done in parallel under a different bug so they don't block MHTML and vice versa, since I suspect the patches don't overlap much.

I tried building/testing your patch. Saving seems to work at least at a first glance; images on www.wikipedia.org didn't seem to get saved but that's a relatively minor issue given that the rest of the page saved well. The main issue is that I can't seem to open MHTML files? The tests in toolkit/components/windowcreator/test can't seem to either, in both cases it just opens the 'what do you want to do with this file' dialog. How are you running Firefox/do you have other patches? I was also thinking that nsDocShell's nsINetworkInterceptController only runs for docshells in the parent, which isn't most of them, so it shouldn't intercept much... do you have e10s off for some reason?

I've been working on my own branch for viewing specifically, you can see the full diff on GitHub if you're interested. The resources are made available by a property on the document, which is exposed by native IPC to the parent process for interception, where the intercept controller is attached to the ParentChannelListener. It still needs work, but the initial feedback that I've got seems positive, and it worked well for a demo I did on Tuesday. I'm hoping to refactor MHTMLParser a bit shortly, add documentation and iframe support, etc.

My current inclination is to keep your saving code (since the patch already works and I suspect it's the more important part to you) and use my opening code (since I can get it to work, I think the design is a bit better, and it's more important to me :) ), but I'm open to something else. I'll probably submit a more complete stack of patches in a few days to bug 18764, and I'll leave some comments on your patch for a few things I noticed.

Again, thanks for working on this—even though my code is pretty different, there's no chance I'd have gotten there without your patches.

Re product—they had incomplete information on the security properties, so if everything's good there then there might not be much problem. User needs were (a) to deal with pushback internally if that appears, and (b) because MHTML will realistically be a power-user-y feature, which is OK but there might be interesting things that can be done with the underlying platform feature, in my patch the nsIOfflineArchive stuff and in yours the MHTMLArchive stuff. My overall understanding is that a solid implementation that passes some sort of security scrutiny should be ok to ship.

(I'll also note that the 'hack cycle' during which I was working on this is over, but hopefully I can still find time to push this forward.)

Flags: needinfo?(dmcintosh)

I feel like the serialization improvements (the CSS url extraction and your shadowdom/canvas work) might be good, but they might be better done in parallel under a different bug so they don't block MHTML and vice versa, since I suspect the patches don't overlap much.

Yeah lower priority for sure. Especially the canvas thing that's a whole discussion. Would be great to get your input.

opens the 'what do you want to do with this file' dialog.

I think that issue sounds like the the MIME type mapping or document loader. Let me validate the moz-phab has everything on my local. I've just updated it to latest central, clobbered and still works fine.

in the parent, which isn't most of them, so it shouldn't intercept much... do you have e10s off for some reason?

Nah it's on. I've been able to import and export the files from Chromium and back.

Broadly my read approach is:

  • MHTMLStartup registers .mhtml/.mht extensions → multipart/related MIME type via category manager.
  • MHTMLDocumentLoader is registered as a Gecko-Content-Viewers for multipart/related.
  • When Firefox opens an MHTML file, the document loader factory intercepts it, parses the MHTML, and feeds the HTML directly to the standard HTML document loader.

I just rebuilt and was able to see Wikipedia images into Chromium (and Chromium ones in my build).

Are you doing debug builds, I'm not. I'm running on MacOS. Attached some screenshots.

but I'm open to something else

Not super fussed, as you say export was more interesting to me. I can follow up with the extension API once landed. Maybe ask someone else the best direction to take. I suspect there's tests from both of us that are worth taking. Additionally we might want to move them to wpt tests too.

Re product... might be interesting things that can be done with the underlying platform feature

Great! Yeah, the most interesting thing I see is resolving the export via extensions which as I say I've wrote two recently.

(In reply to Jonathan Kingston [:jkt] he/him from comment #218)

in the parent, which isn't most of them, so it shouldn't intercept much... do you have e10s off for some reason?

Nah it's on. I've been able to import and export the files from Chromium and back.

I just built your patch off of Phabricator (onto the main branch, 28398d11 which is a bit after your patch), and I still can't open MHTML files. I still think there's some multiprocess thing going on—I noticed in your tests, e.g. browser_mhtml_load.js:

    let doc = tab.linkedBrowser.contentDocument;
    ok(doc, "Document should be loaded");

but contentDocument only works if the browser is in-process, which it shouldn't be for a file/http page. (I ran into this problem in my tests and had to do a bunch of SpecialPowers fun.) If you open the browser toolbox and type gBrowser.selectedBrowser.contentDocument, what do you get? I'd expect null except on about: pages.

I just rebuilt and was able to see Wikipedia images into Chromium (and Chromium ones in my build).

Looks like images do work on Wikipedia articles, but not ones around the UI. I meant https://www.wikipedia.org/ proper, and I see in your screenshot that the hamburger menu is missing from the Firefox-saved one. A glance at the inspector suggests they use url() for some reason, so that part of your patch might not be working right? Probably not an MHTML problem though, and doesn't change that the rest of the page saves fine :).

Are you doing debug builds, I'm not. I'm running on MacOS. Attached some screenshots.

I'm on Windows with non-debug builds, I wouldn't expect the debug builds to break it though?

I also meant to leave some comments on your Phabricator revision, I'll go do that now.

but contentDocument only works if the browser is in-process, which it shouldn't be for a file/http page. (I ran into this problem in my tests and had to do a bunch of SpecialPowers fun.) If you open the browser toolbox and type gBrowser.selectedBrowser.contentDocument, what do you get? I'd expect null except on about: pages.

Null yeah. I'll investigate what the test is doing. I think that's why they're marked as skipped currently, they predate the last rewrite.

Looks like images do work on Wikipedia articles, but not ones around the UI. I meant https://www.wikipedia.org/ proper, and I see in your screenshot that the hamburger menu is missing from the Firefox-saved one. A glance at the inspector suggests they use url() for some reason, so that part of your patch might not be working right?

I think this is the parser parts that in phab you asked if could be split out. I'm wondering if there's a better approach overall there. Removing that code certainly breaks more things.

I'm on Windows with non-debug builds, I wouldn't expect the debug builds to break it though?

I wouldn't no. Just checking an assert wasn't giving the different behaviour.

Thanks for the continued reviews. I'll upload the commit split I just made but there's some issues I need to investigate more as mentioned.

Attachment #9537795 - Attachment description: WIP: Bug 40873 - Add MHTML export and read support → WIP: Bug 40873 - Add MHTML read/loading support

Adds the ability to save web pages as MHTML (single-file archive).

Components added:

  • nsMHTMLPersist: Core MHTML generation with MIME multipart encoding
  • nsWebBrowserPersist changes: Integration with persist framework
  • MHTMLArchive.sys.mjs: JS-level archive creation
  • contentAreaUtils.js: Save dialog MHTML option
  • PERSIST_FLAGS_SAVE_AS_MHTML flag for enabling MHTML output

Includes roundtrip tests to verify reading and writing work together.

Depends on D279105

Currently working on stripping out any parsing logic at all, other than the mime parsing. I think all of that is just hacks working around a problem that should be a little simpler.

I've pushed it to gh, it'll need refactoring back into the split commits: https://github.com/mozilla-firefox/firefox/compare/main...jonathanKingston:firefox:jkt/40873-remove-parse?expand=1

To avoid any stripping or changing of the parser I've added a media query to hide things like <noscript>

Once finished I'll upload it again to moz-phab.

I pushed the updates to moz-phab.
I've have removed any parser parts fully, I've also used an export change to drop <noscript> elements from being exported to fix exporting to Chrome.

Flags: needinfo?(dmcintosh)

On the saving side, I've left some comments on your Phabricator revision. I'm not sure why you seem to remap all of the resources to their CIDs, a few more tests of the saving might be helpful, and there's some cleanups and things to move around, but once those are addressed if you think it's ready for review you can probably just request review. (I'm guessing you have an idea of who good reviewers would be, and I'll add others if needed.)

I might suggest changing the revisions so they don't depend on each other, since in any case the viewing side will likely need more work. You mention having roundtrip tests, which is a good idea, but maybe they should go into a follow-up revision so saving doesn't have to wait for viewing?

On the viewing front, I talked to :asuth and he suggested against using nsINetworkInterceptController since he sees that as a service-worker-specific thing that outside code shouldn't use, despite the comment in that IDL suggesting it's more general. I've ported my viewing code over to using redirects to the CID protocol, which is now working!, but I haven't cleared that by Necko developers yet. I also was figuring that I think it could be intercepted in the IOService, in which case we could match Chromium's behaviour on non-HTTP URLs and avoid cid: needing to be valid outside of that context, but I haven't explored that one yet; I plan to ask some Necko developers on Monday as to what the best approach would be, and I'll update this bug when I find out.

(In reply to Jonathan Kingston [:jkt] he/him from comment #223)

I've have removed any parser parts fully, I've also used an export change to drop <noscript> elements from being exported to fix exporting to Chrome.

I like the new <noscript> handling much better than before, thanks for working on it! I'd like to clarify some of your patches' behaviour around the tags, though: my understanding as for why to remove them when saving is that we want the page to display as it was rendered, which means not displaying noscript if they wouldn't have been displayed at the time. In this case, I agree with removing them, although we maybe should only remove them if scripting is enabled for the document.

On the viewing side, though, I don't agree with the CSS/parser change that hides them if they're already in the file. If Chromium already removes them, which it seems to, then in my eyes most MHTML files already don't have these tags, and if the tags are there then—since scripting is disabled, and it is just rendering the HTML page—it should render them as usual. This also would match Chromium's behaviour, where noscript tags are rendered if present. Is there a specific reason to hide them?

In any case, thanks for your patience and continuing to work on this!

Flags: needinfo?(dmcintosh)

I've gotten word that modifying NS_NewChannel/nsIOService can work, and I've updated my GitHub branch to match, although it currently doesn't do stylesheet load progress correctly for some reason. (FYI I'll be out next week.)

Sorry have been meaning to split the patches into separate bugs, then move away from nsINetworkInterceptController. I probably will have time tomorrow.

Has this bug split happened? Not sure where to look for this.

Flags: needinfo?(jonathan)
Depends on: 2030897
Depends on: 2030898

I've created both: Add MHTML (RFC 2557) export support to "Save Page As" and Add MHTML (RFC 2557) read support for .mhtml / .mht files to separate out from here. I'll rebase and move my patches there.

We likely will continue to want follow up work here for example there's:

Flags: needinfo?(jonathan)
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: