The default bug view has changed. See this FAQ.

Implement local filesystem read/write access.

NEW
Unassigned

Status

()

Toolkit
WebExtensions: General
P2
normal
a year ago
9 hours ago

People

(Reporter: billjm05, Unassigned, NeedInfo)

Tracking

(Blocks: 3 bugs)

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: [design-decision-approved] triaged)

(Reporter)

Description

a year ago
User Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36

Steps to reproduce:

N/A


Actual results:

N/A


Expected results:

N/A
Status: UNCONFIRMED → NEW
Ever confirmed: true
See Also: → bug 1215028

Updated

11 months ago
Whiteboard: [design-decision-needed] triaged

Comment 1

9 months ago
I'm confused. How will this differ from the Filesystem API implementation of bug 1265767?

Comment 2

9 months ago
It's undecided at this point.

Comment 3

8 months ago
This seems like a good idea with multiple use cases and there's been a few suggestions how we could do it. The first stage would probably be something like:
* implement the Blink Filesystem APIs (as noted in bug 1265767)
* make it a permission that a developer has to request
* limit the reading and writing to an add-on specific sandbox in the developers profile

Once we've got that done, we could move on to the next stage which would be allowing an add-on to read and write from any directory that the user would approve. Because it would require user approval we'd need some UX and security review to ensure that we can limit malicious activity.
Whiteboard: [design-decision-needed] triaged → [design-decision-approved] triaged

Updated

6 months ago
Blocks: 1219940

Comment 4

6 months ago
We'd at the very least need required permission support for this. Its probably an optional permission so add as depending on that one.
Component: WebExtensions: Untriaged → WebExtensions: General
Depends on: 1197420
Priority: -- → P2
Blocks: 1310316

Comment 5

5 months ago
What permit is required? when it will be implemented for the Nightly?

Updated

4 months ago
Duplicate of this bug: 1320518

Comment 7

4 months ago
Any news or updates on this CRITICAL issue of WebExtensions?

A lot of useful addons CAN'T be made without ability to read/write local real filesystem.
Like: SessionManager, Mozilla Archive Format, DownloadThemAll, Places Maintains and etc.

The feature maybe limited to some folder (e.g. add-on's folder), or with user's manual approval or user's folder choice or something else if needed, but dropping the whole thing is obviously not a right decision, we don't want another quite dull and limited half-a-browser like Chrome, do we?

So, the question is - When should developers expect filesystem support for the-only-recommended WevExtensions?
I mean, it's one of the most desired feature (according to votes!) for WebExtensions, that always was in Firefox and is required for many tasks. Is this feature decided to be added soon?

Comment 8

4 months ago
Hi,
I have just started contributing to Mozilla and would like to work on this bug. It would be really great if I was given pointers as to how to start working on it.
Thank you :)

Comment 9

4 months ago
Suhan thanks for wanting to help out. We are happy to have contributors for bugs, but if you are just starting contributing to Mozilla then you might want to start with some easier bugs. Check out this link: https://wiki.mozilla.org/Add-ons/Contribute#Improve_add-ons_in_Firefox, which links to the onboarding and bugs that are mentored and recommended for your first bug.

Updated

4 months ago
Blocks: 1118289

Comment 10

3 months ago
See also bug 1323414.

Updated

3 months ago
See Also: → bug 1323414

Updated

3 months ago
webextensions: --- → +

Updated

2 months ago
Duplicate of this bug: 1332927

Comment 12

2 months ago
Dropping down to ? for Firefox 57.

At this time we think it makes sense to:

* implement a streaming API for downloads, this is basically the proposal on bug 1323414 and would allow add-ons that stream a large amount of data (for example Video Download Helper) to be ported over to WebExtensions. More bugs will be filed for this.

* bug 1331618 allows extensions to store up to 20gb per file into indexeddb and use that as a virtual file system and should meet most storage systems. We might need to add in a library to make this easier to use, or re-use one that already exists.

Most use cases we've found so far would fit into one of the two use cases above. So for the moment I'm dropping this down until we can find more use cases.
webextensions: + → ?
(In reply to Andy McKay [:andym] from comment #12)
> Most use cases we've found so far would fit into one of the two use cases
> above. So for the moment I'm dropping this down until we can find more use
> cases.

For my add-on withExeEditor *1, it is essential that this bug is FIXED by Firefox 57.
withExEditor creates a temporary file in the OS's temporary folder for source view, selection view, and text editing, and passes that file path to the text editor.
And when text editing, after user edits text, add-on gets that temporary file and update it in the web page.

Apart from this bug remaining, almost all the functionality has already been migrated to WebExtensions (not released yet).

*1 https://addons.mozilla.org/en-US/firefox/addon/withexeditor/

Comment 14

2 months ago
I presume withExeEditor uses native messaging to communicate with a local editor?  If so, the native application could do the filesystem reading and writing...
(In reply to Andrew Swan [:aswan] from comment #14)
> I presume withExeEditor uses native messaging to communicate with a local
> editor?  If so, the native application could do the filesystem reading and
> writing...

Pass whole content instead of file path to the host?

Comment 16

2 months ago
(In reply to Kazumasa Hasegawa (Kazz) from comment #15)
> Pass whole content instead of file path to the host?

Yes.  Are you worried about the size?  Its already data that's been moved across the network and is moving to a file, an extra inter-process copy doesn't seem like a problem...
(In reply to Andrew Swan [:aswan] from comment #16)
> Yes.  Are you worried about the size?  Its already data that's been moved
> across the network and is moving to a file, an extra inter-process copy
> doesn't seem like a problem...

Thanks. I'll think about that.
It seems to need total redesign again...

Comment 18

2 months ago
With the proposed approach, how would one go about implementing addons in the vein of ASF (Automatic Save Folder) which need to circumvent the target folder restrictions described in Chrome's download API docs? 

(Or would an addon developer just have to ask non-Windows users to set the default download location to the filesystem root and warn Windows users that it's constrained to a single drive?)

Comment 19

2 months ago
We would like to create a Firefox extension for Kiwix, based on https://github.com/kiwix/kiwix-html5/
It's an offline wikipedia reader, that reads its content inside big local files (and they can be very big : http://download.kiwix.org/zim/wikipedia/?C=S;O=D ). These files are usually copied from another media (or downloaded separately).
It already works by simply opening the local HTML file. But the user needs to manually select the file(s) each time (through an "input type=file").

A browser extension with such filesystem API would allow us to :
- remember the selected file(s), and re-open them automatically
- "package" the application in a more user-friendly way than a local directory where the users needs to open an HTML file

The streaming API would be useless for us.
A local sandboxed storage might technically work, but does not seem adequate to our use-case : the files will probably be too big for that, and the user will (I suppose?) be forced to download them from within the extension itself (instead of copying them offline).

Comment 20

2 months ago
Direct local file system access is useful in may cases:

1. Save addon data in a Dropbox / Google Drive / Box folder so that they can be synced across the web automatically.

2. Specify an "input" and an "output" directory and perform a batch convert.

3. Flush HTML + CSS + JS files in a specifically folder dynamically so that they can be opened from the  browser.

These effects cannot be achieved by FileSystem API, indexDB API, streaming API, or downloads API.

Comment 21

2 months ago
>Andy McKay: So for the moment I'm dropping this down until we can find more use cases.
What does "dropping this down" mean exactly (literally)?

This is this critical issue, WebExtensions can not be offered as the only recommended replacement if they are still so limited.

As for use cases, here one more: Addon wants to store some files with folders hierarchy in folder/pass, that was set/chosen by user.
It could be result of some parsing of page, some script's downloads or math evaluations, doesn't matter.

The previous extensions API allowed this, and the current - doesn't. It should be fixed, it's critical for such addons, you take what worked and replace with something less functional, it can't end well.

Local file system access IS MANDATORY, users and developers said that far more than once. It can be limited by permissions, by user manual approval as offered in commends, but it MUST be, otherwise Firefox will become another dull chromish-browser.
Local file system access IS MANDATORY. Thank you for your time.

Comment 22

2 months ago
> Local file system access IS MANDATORY. Thank you for your time.

I have to agree. At its core, there are two gaping holes in the new extension API which have me scrambling to arrange a Firefox exit strategy:

1. The ability for extensions like Automatic Save Folder, downThemAll!, Greasemonkey, savetexttofile, ScrapBook, and TiddlyFox to integrate with the filesystem without going through the aforementioned crippled APIs

2. The ability for Classic Theme Restorer and Stylish to revert enough of Australis to at *least* match Chrome's level of UI native-ness.

Greasemonkey and savetexttofile should be simple fixes with some kind of external helper, so I'm not too worried about them. (I can't be the only userscript developer who iterates on scripts by opening the installed script and using `<Ctrl+S> <Tab> <F5>` to apply changes and my experiments say that it should be possible to force no-dialog saving so an inotify-based helper can pick up the output of a savetexttofile successor and append it to the actual destination file.)

Automatic Save Folder and downThemAll will be more painful, but I can probably replace them by writing a WebExtensions shim which essentially hijacks all Ctrl+S functionality and pipes the resulting stuff to JDownloader or KGet. (Possibly requiring an XGrabKey+XTest helper to remap my muscle memory if it doesn't have permission to universally hijack Ctrl+S)

ScrapBook may be OK if the raw on-disk format of Firefox's IndexedDB is documented enough for me to do I/O on it from outside the way I can with ~/incoming/ffox_scrapbook/

It's TiddlyFox and Classic Theme Restorer that are the big problems.

TiddlyFox is already a life-support patch for the XPConnect-based save code in TiddlyWiki Classic and, while I really don't want to have to invent some kind of HTTP daemon whose only purpose is to open up a hole in Firefox to restore the ability for an HTML+JS quine to move foo.html to foo.html.backup and then write new foo.html and foo.xml (RSS) files, if I must then I must.

As for Classic Theme Restorer and the XUL style functionality in Stylish, well, let's just say that Chrome is closer to the browser I want than Australis would be without CTR. (The only use I have for "lightweight" theming is to recolor Chrome so it matches my Qt/GTK+ theme.)

Without Classic Theme Restorer, my exit strategy is looking like it'll be "migrate as much as possible to Chrome-compatible WebExtensions, migrate the rest to external stuff like the Privoxy HTTP proxy server, install those extensions in Chrome, and then rely on Chrome's UI being closer to what I want while I wait for something like QWebEngine to implement WebExtensions support so it's feasible for me to maintain my own browser frontend."

Comment 23

2 months ago
...and I just realized that I went off on a tangent there. My apologies.

My original intent in bringing CTR into the mix was to use it as part of an argument as to the degree of importance filesystem-accessing extensions play in my workflow, but I can't remember exactly how.

Comment 24

2 months ago
After a quick review of Automatic Save Folder, it seems it does not need full filesystem access, it just needs an API to downloads to decide where the file should go. The API could be passed the originating domain or the file name and decide where it gets written. There's some nuance there like, it shouldn't allow overwriting files and there might be permissions we need to prompt the user about there, but seems doable.

Similarly for saving to a drop box folder, again the ability to run downloads to a specific location, or generate them using downloads.download API gets them into your DropBox folder. Then you can perform any processing of files on the native file system using nativeMessaging.

DownThemAll is not planning on moving to WebExtensions as far as I know (although I'm not the author of that add-on so please don't take that as gospel). 

SaveTextToFile can use the downloads API to write a file to a location of the users choosing.

Comment 25

2 months ago
(In reply to Andy McKay [:andym] from comment #24)
> After a quick review of Automatic Save Folder, it seems it does not need
> full filesystem access, it just needs an API to downloads to decide where
> the file should go. The API could be passed the originating domain or the
> file name and decide where it gets written. There's some nuance there like,
> it shouldn't allow overwriting files and there might be permissions we need
> to prompt the user about there, but seems doable.

That'd definitely be doable for ASF. Heck, a proper WebExtensions API for determining destinations like that would actually be *better* for my use because it would turn ASF into little more than a visual programming frontend for an API I could easily and comfortably script directly.

(There are various situations where ASF falls short currently because it doesn't support matching against selectors in the DOM and I'd like to implement things like "On 'Save Image As...' on this list of *booru-style image boards, scrape the tags sidebar and map recognized combinations of tags to download folders based on priority ordering.")

> 
> Similarly for saving to a drop box folder, again the ability to run
> downloads to a specific location, or generate them using downloads.download
> API gets them into your DropBox folder. Then you can perform any processing
> of files on the native file system using nativeMessaging.

Without overwrite, I have no reason to save into the Dropbox folder. ~/Dropbox isn't inside ~/incoming and, as a Linux user, I see the reliance on native messaging as a MAJOR step backward since I'll probably have to maintain forks of various extensions myself to get Linux support.

...so, again, I'd probably just resort to some hack like setting my default downloads folder to to ~/.downloads_dispatch/ and then using `mount --bind` to outwit any "don't follow symlinks" mechanism.

As for TiddlyFox, Without the ability to either rename or overwrite arbitrary files, it'd be almost useless as anything more than a life-support harness for TiddlyWiki Classic. (TW5 already falls back to opening a save dialog every time you click "Save") ...so it would most definitely have to rely on a nativeMessaging helper without an API for that.

(I suppose I could contribute one to them if it comes to that, given that Rust makes it easy to cross-compile to a fully statically-linked musl-libc target. Still, speculatively bundling dozens of different helpers for platforms like my arm-unknown-linux-gnueabi with X11 (my OpenPandora) is inferior to Just Working™ on any platform as soon as someone cross-compiles Firefox itself... but, then, for my OpenPandora, it'd probably just be easier to keep an old Firefox build around.)

> 
> DownThemAll is not planning on moving to WebExtensions as far as I know
> (although I'm not the author of that add-on so please don't take that as
> gospel). 

Yeah. DTA was sort of loud about that.

Given that Giorgio Maone is more receptive, I'm hoping that FlashGot with either JDownloader or KGet will prove to be a suitable replacement. (Using the "legacy" JDownloader API which doesn't require a My.JDownloader account to add URLs is a pre-requisite for JDownloader being considered suitable.)

I've already got a Chromium->JDownloader extension using the "legacy" API, so that's covered if I do decide that Firefox's demographic targeting has left me behind.

> 
> SaveTextToFile can use the downloads API to write a file to a location of
> the users choosing.

The reason I mentioned SaveTextToFile is that I use it in Append mode only and I don't know of any way to append to files using the downloads API.

(In fact, I use it in append mode so habitually that I forgot to mention that because I forgot it supports anything else.)

Give me an "append to file" API and I wouldn't mind spending the 15 minutes or less to replace SaveTextToFile with a webextension that hard-codes my preferred settings and displays a context menu entry that captures the selected text.

Updated

2 months ago
Duplicate of this bug: 1338780

Comment 27

2 months ago
In bug 1338780 I gave two scenarios of usage, where necessary overwrite the files and full access to the file system at least in a special directory.

It should not be implemented with restrictions writed above. Need implementation, writeAtomic relevant in the current context with full access to the extension directory

Comment 28

a month ago
I don't get what problem!
 - First of all we should define protocol for this (local://|fs://);
 - Second there must be unified paths on all platforms, so not this "C:\\...", "/home/...", but this "local://shared/...", "local://secure/..."
 - Accordingly to the second point there must be logical to physical paths mapping, which user can setup in about://settings
 - In manifest file (of webext) there should be field where developer define all files which he intent to use, for example: "fs_perms": {"read":["local://etc/1.txt"],"write":["local://etc/1.txt"],"exec":["local://etc/", "python", "npm"]}. Any attempts to use not declared file should throw exception, and as option send the claim to AMO.
 - In AMO should be insistent block which in plain language tells what fs permissions this webext is require. And when xpi is downloaded an user push the install button, the should be another warning? for example popup with "Omg! This extension attempt to override your /etc folder" (in this stage we already know physical paths which ext will use.
 - Developers should can make self folders where they will store files easy accessible to user. For example "local://books/", "local://pdfs" etc. For this in fs_perms there should be additional field "create". When user install the extension there should be info alert which confirm creation of listed folders in desktop dir of user.
 - There should be "local://tmp/" which points to the system/user tmp dir.
 - There should be limits settings in about://settings page. If developer attempt to overcommit limits then he must specify in AMO webext home page that user should increase self limits otherwise his webext will crash when rich limits.
 - If webext require to read system files, then developer should do this through native-messaging to some system utils.
 - There should be opportunity to encrypt files in "local://secure/" folder. For example now it will be unencrypted, but we will explain in api docs that files in this folder can be encrypted in the future and developer should put to this folder only that files which should be readable only from his webext and none of others exts/programs/utils. Or he can provide sha/rsa/... keys and files in this folder will be encrypted transparently.
 - Each webext has own "local://secure/" folder. So should be dynamic mapping.
 - Should be opportunity to use other protocols. For example "ftp", "sftp", "webdav", "http", "custom"

Comment 29

a month ago
> I don't get what problem!

In my opinion, you did not read the comments above. For example, #27.
Extensions need to write and to write in files with unknown in advance names.

Comment 30

a month ago
Yes I read this thread. But from security view override any file in system is ugly. As alternative we can give opportunity to make personal folder for each extension where it can create any what it want names of files and read and write to. For example folders created with "create" property of fs_perms field has this omitted perms {"read": ["local://folder/*"], "write": ["local://folder/*"]}. Or we can accept specify folder as subject to write permissions ({"write":["local://etc/"]}) then in this folder extension can do create rename delete files operations.

Comment 31

a month ago
Again, that doesn't help with cases like these:

1. The whole purpose of the TiddlyFox extension is to allow TiddlyWiki HTML files to rename and overwrite themselves

2. SaveTextToFile's append mode is predicated on allowing the user to define an arbitrary file path which will be appended to by selecting some text and then choosing a context menu entry.

Comment 32

a month ago
First of all TiddlyWiki not designed to be webextension. If you want, you can use localStorage for this purpose (not append themself but store changes to the some of storage).
Secondary if you want to make extension based on TiddlyWiki, you can change sources that: In extension package html contain iframe which sourced to the created by extension folder, then in this extension you define command on "Ctrl+s" or define page action or browser action which read the iframe content and saves it to the folder. And voila!

Comment 33

a month ago
Ok, I was having a hard time making sense of your writing before, but now it's gotten so bad that I'm very worried I've misunderstood you somewhere.

That said, here's my best attempt...

> First of all TiddlyWiki not designed to be webextension. If you want, you can use localStorage for this purpose (not append themself but store changes to the some of storage).

Completely unacceptable. I can't turn my localStorage into a URL like http://ssokolow.com/story_ideas.html (a hobby TiddlyWiki of mine) simply by running `rsync`, nor can I drop it onto a thumbdrive for anyone to read with whatever browser they have on hand.

> Secondary if you want to make extension based on TiddlyWiki, you can change sources that: In extension package html contain iframe which sourced to the created by extension folder, then in this extension you define command on "Ctrl+s" or define page action or browser action which read the iframe content and saves it to the folder. And voila!

I'm not entirely sure what you're trying to say here, but I'm pretty sure that I'd have to stick to versions of Firefox that allow me to disable extension signing enforcement in order to accomplish it.

All in all, I get the impression that you're saying something equivalent to "No, we refuse to support Encrypted Media Extensions and you're not allowed to switch to Chrome for your NetFlix either." It's ridiculous and an impractical demand to make.

(And, sure enough, if WebExtensions-only Firefox won't meet my needs, I'll grab the last pre-WebExtensions-only ESR, disable updates, and use that. "Defining the problem away" is no way to build a tool that people actually need to use to get work done.)

Comment 34

a month ago
Why you are want that TiddlyWiki will be webextension? Just save it where you want and open it when you want by double click on *.html. What problem? It was designed for such use. And no one will take away that opportunity.

Comment 35

a month ago
When I click "Save", TiddlyWiki Classic is supposed to save changes. It has no fallback mechanism for presenting a download dialog like TiddlyWiki 5.

It used to save via XULConnect. When that was taken away, they wrote TiddlyFox, a XUL extension which un-breaks the save feature and allows TiddlyWiki 5 to do it without the Save As dialog.

Mozilla wants to kill off XUL extensions and replace them with WebExtensions.

The only way TiddlyFox could allow saving via WebExtensions, currently, is to present a download dialog. 

It's unacceptable for me to have to meddle with a download dialog every time I click Save or to store my TiddlyWikis in some special Firefox-specific place, rather than in the project folders where they currently reside alongside related things.

Comment 36

a month ago
Cool it guys, we all want Firefox to be good, flexible, powerful and secure.

Let's try to agree on statement that there CAN BE extensions that NEED to write to local filesystem. The must-have requirement that should be fixed in memory, otherwise this issue will exist for years!

Maybe it should be limited to some fixed folder, maybe with some permission-ask, maybe even with manual allowance via about:config, maybe this extension should be hidden by default on addons.mozilla.org, but let's fix that such extensions CAN and MIGHT be, agree?

If you agree, than the API to write to local filesystem in some folder is mandatory.

PS. The issue was created more than a YEAR ago, any hope that someone will take responsibility for this task, mozilla-guys?

Comment 37

a month ago
PS2: And I agree with Stephan, that saving dialog is not acceptable for some tasks, because the extension might save the results in batch of files, and what the user should do, click 150 times? So the future solution should not be limited to per-file-manual-user-clicking thing like saving dialog.

Comment 38

a month ago
You will can map for example "local://tiddlyFox/" logical folder (where tiddlywiki future webext will use) to the your "C://Projects/Wiki/somethingelse" physical folder by two ways:
1. In about:config make record "webext.folders_mapping.tiddlyFox" = "C://Projects/Wiki/somethingelse" (its not real key, it's for example);
2. Make the system symlink from path where Firefox will want to store files appurtenant to the "local://tiddlyFox" to the needed destination.

You will do it only once and none of confirmation dialogs will be showed when this extension will be write to this folder (save your session). Except only one confirmation dialog (for creating new folder and report what fs actions this ext will do in your pc) when you install this extension.

Comment 39

a month ago
(In reply to jamy023 from comment #38)
> You will can map for example "local://tiddlyFox/" logical folder (where
> tiddlywiki future webext will use) to the your
> "C://Projects/Wiki/somethingelse" physical folder by two ways:
> 1. In about:config make record "webext.folders_mapping.tiddlyFox" =
> "C://Projects/Wiki/somethingelse" (its not real key, it's for example);
> 2. Make the system symlink from path where Firefox will want to store files
> appurtenant to the "local://tiddlyFox" to the needed destination.
> 
> You will do it only once and none of confirmation dialogs will be showed
> when this extension will be write to this folder (save your session). Except
> only one confirmation dialog (for creating new folder and report what fs
> actions this ext will do in your pc) when you install this extension.

Not very helpful, because:

1. A lot of users (especially Windows users) have no idea what a symlink is.

2. I'd just end up using `mount --bind` to point `local://tiddlyFox` at `/` in a way that any kind of "protect you from yourself" symlink checks wouldn't notice.

2a. The only common root for /media/ssokolow/16GB_Flash, /home/ssokolow/src/ssokolow.com/story_ideas.html, /home/ssokolow/Documents/Critical/tiddlyme.html, and /home/ssokolow/Documents/writing/Non-Fiction/ is the root of the filesystem.

2b. I don't feel like babysitting a bunch of symlinks as I create, move, and rename TiddlyWikis.

3. Running TiddlyWiki off a thumbdrive using a PortableApps copy of Firefox is not uncommon, and that means you can't rely on the drive letter to remain constant. (TiddlyFox just pops up an "Allow saving?" dialog the first time you open up a TiddlyWiki on an unrecognized path)

Comment 40

a month ago
> 1. A lot of users (especially Windows users) have no idea what a symlink is.

This users actually have symlink opportunity. This users simply will use original folder which Firefox will made for this extension. Developers of extension can teach this users how map folder to the needed destination through about:config in his AMO webext home page. Or even how to make symlink in Windows.
 
> 2. I'd just end up using `mount --bind` to point `local://tiddlyFox` at `/`
> in a way that any kind of "protect you from yourself" symlink checks
> wouldn't notice.

If you run Firefox as root then maybe true. But usually the system's acl will reject any attempts to override system files. Then extension will not work. You will not get any profit of this trick :)

> 2a. The only common root for /media/ssokolow/16GB_Flash,
> /home/ssokolow/src/ssokolow.com/story_ideas.html,
> /home/ssokolow/Documents/Critical/tiddlyme.html, and
> /home/ssokolow/Documents/writing/Non-Fiction/ is the root of the filesystem.

I don't understand what this mean actually. Maybe you can use that:
ls $HOME/tiddlyFox/
  ssokolow.com -> /home/ssokolow/src/ssokolow.com/
  Critical -> /home/ssokolow/Documents/Critical/
  writing -> /home/ssokolow/Documents/writing/

> 2b. I don't feel like babysitting a bunch of symlinks as I create, move, and
> rename TiddlyWikis.

Otherwise you will hosting malware software (If webext's will have full filesystem privileges). Remember, webext's will run from your user (ssokolow), and theoretically have choice to look at any file in your home folder. Do you want that other (not tiddlyWiki) extension for example some advertise blocker or so, as payment for itself, collect info about you. Your conversations, bookmarks, web navigation history, passwords, keys from your ssh logins to the ssokolow.com host, bank moneykeepers keys, photos, etc is right paying for that freedom which you beg from us? 
 
> 3. Running TiddlyWiki off a thumbdrive using a PortableApps copy of Firefox
> is not uncommon, and that means you can't rely on the drive letter to remain
> constant. (TiddlyFox just pops up an "Allow saving?" dialog the first time
> you open up a TiddlyWiki on an unrecognized path)

And what problem? Just don't use manual mapping settings an lookup extension's data in most ease place -> Desktop or Home dir. Any desktop os propagate homedir path. And you can (if you want) tell Firefox to use different home dir, for example on thumbdrive. Developers of portable distribution of Firefox can do it for you.

Comment 41

a month ago
> I don't get what problem!
> - First of all we should define protocol for this (local://|fs://);


FireFox must implement very simple functions. The first requirement of security is simplicity.
You offered so intricate scheme that they themselves do not understand that it does not implement the use case in comments #27 .
Or I did not understand. That is, your scheme is so complicated that some of us do not understand what is happening.


From intricate schemes started the main security problems in architecture.

Therefore, it is easier and safer make implementation the standard and simple operation. Which now supports FireFox.
If you want to restrict addons, the user could be asked to give permission for an extension.


The file operation should be implemented through a specially designed file API. Security operations should be implemented through a specially designed security features.

Comment 42

a month ago
(In reply to fdsc from comment #41)
> > I don't get what problem!
> > - First of all we should define protocol for this (local://|fs://);
> 
> 
> FireFox must implement very simple functions. The first requirement of
> security is simplicity.

Who saying that it's not simple? You? This is very simple and secure scheme.

> You offered so intricate scheme that they themselves do not understand that
> it does not implement the use case in comments #27 .
> Or I did not understand. That is, your scheme is so complicated that some of
> us do not understand what is happening.

The scheme like my is used for a long time. Remember chrome protocol. And even now in webextensions for access to extensions self files used similar scheme (web-extension://) the difference is it using some hash key for folder name, but mine scheme using human readable folders name. If you don't understand mine scheme how you will go forward? Maybe I can explain how my scheme is working and how it rich a safety? Give me a full usecase and I give you description how you can develop this with my scheme of FS API.

> From intricate schemes started the main security problems in architecture.

Again: my scheme is not intricate. If it not clear for you it's does not mean that it not clear for others.

> Therefore, it is easier and safer make implementation the standard and
> simple operation. Which now supports FireFox.

As I explained before, Firefox using similar operations for a long time. And even more difficult.

> If you want to restrict addons, the user could be asked to give permission
> for an extension.

Many users don't understand what happened and scared if some dialog windows appearing unexpected. And then they stopped touch the computer and call the support. Or simply accept all and doesn't worry.

> The file operation should be implemented through a specially designed file
> API. Security operations should be implemented through a specially designed
> security features.

Examples please. What security features?

Comment 43

a month ago
I think you need to add a bug 1215059 in blocks
Flags: needinfo?(billjm05)

Comment 44

a month ago
> This is very simple and secure scheme.

Once again, if your scheme is simple, it does not satisfy the use case from Bug 1338780

Comment 45

a month ago
(In reply to fdsc from comment #44)
> > This is very simple and secure scheme.
> 
> Once again, if your scheme is simple, it does not satisfy the use case from
> Bug 1338780

> 1. The user should be able to provide detailed logs if he gets an error and developer cannot be repeated.

local://shared/logs/ for logs

> 2. A developer wants get full user setting the additions. It is easiest to implement if it's possible to export the settings to the file.

local://secure/settings.json

>USE CASE 2
It is possible to transfer add-on settings between different versions of the add-on or between different user profiles (to set up similar profiles).

local://shared/tiddlyFox/settings.json

I think if developer a real developer he can explain to the user how to find ".mozila/firefox/webext_shared_data" folder in his home folder if he searching files from local://shared/ logical folder. If user needs files from local://secure/ folder then he should find folder of his firefox profile and here he can find "webext_secure_data" folder where places folders with human readable names equals to extension names. And it's intricate for you?

And this scheme of File Api eliminates of differences in different platforms as well as originality of technology allow developer to not worry about verry old versions of firefox, maybe even not worry about of absents of about:debugging

Comment 46

a month ago
> local://shared/logs/ for logs

And in order to destroy the entire world you want local://destroyTheWorld

Do you think or just write the names?

If you are not a developer or you have no qualification, let's not clutter this bug with useless arguments?


> I think if developer a real developer he can explain to the user how to find

I think if the developer is a real developer, there is no need to explain to the user how to do the actions that user do not want to do.

Comment 47

a month ago
> And this scheme of File Api eliminates of differences in different platforms as well as originality of technology allow developer to not worry about verry old versions of firefox, maybe even not worry about of absents of about:debugging

It is not the schemes, but that addition is not able to write and read files. This scheme is just useless, as it does not implement file operations.

Comment 48

a month ago
(In reply to fdsc from comment #46)
> > local://shared/logs/ for logs
> 
> And in order to destroy the entire world you want local://destroyTheWorld

If developer of extension is declare in manifest that he want to create destroyTheWorld folder then he can write to or read form. The user when will install this extension will see that this extension wants ot create that folder on his desktop.
 
> Do you think or just write the names?
> 
> If you are not a developer or you have no qualification, let's not clutter
> this bug with useless arguments?

I am a developer. And you my dear friend, come up the unreal usecases.

> > I think if developer a real developer he can explain to the user how to find
> 
> I think if the developer is a real developer, there is no need to explain to
> the user how to do the actions that user do not want to do.

This your idea that user must send logs and settings to the developer if extension is throw exceptions.

(In reply to fdsc from comment #47)
> > And this scheme of File Api eliminates of differences in different platforms as well as originality of technology allow developer to not worry about very old versions of firefox, maybe even not worry about of absents of about:debugging
> 
> It is not the schemes, but that addition is not able to write and read
> files. This scheme is just useless, as it does not implement file operations.

It is concept. Which we can use. If you my dear friend don't get yet, then I explain you that new webextension is much more different from old addons and as proof I can give you a quote: 

http://webextensions-experiments.readthedocs.io/en/latest/new.html

There are some basic principles around WebExtensions APIs as they currently stand. Any API that wants to get merged into Firefox would need to meet these principles.

    Security: the API should not expose an unacceptable risk to the end user.
    Privacy: similar to above.
    Performance: APIs should be async by default and focus on not causing jank, hangs or any sort of bad performance in Firefox.
    Multi-process: all APIs need to be multi-process aware.
    Useful: APIs that land in Firefox should be useful since all APIs have a maintenance burden.
    High level: WebExtensions provide a known public API layer on top of Mozilla code, allowing the underlying code to change quicker and easier.
    Alternatives: Do alternatives exist for example, WebAPIs?

Compared to legacy Firefox add-ons there is a significant sandbox around WebExtension APIs that is very restrictive. Access to priviledged APIs within Firefox, preferences, arbitrary File System access, socket access and so on are restricted (at the time of writing this documentation). APIs that break out of this sandbox will need very clear justification for why.

Comment 49

a month ago
> This your idea that user must send logs and settings to the developer if extension is throw exceptions.

How this log will be recorded by the extension?
Write the example of code that write to log


> need to meet these principles.

First of all, these APIs should function. If someone thinks that some feature is dangerous, the user must choose if you want it or not. The user and not the developer who chooses.

The most secure API, that which does not exist.
I now write the browser without any extensions. Or to do without a browser. After all, it will be safer.

Comment 50

a month ago
(In reply to fdsc from comment #49)
> > This your idea that user must send logs and settings to the developer if extension is throw exceptions.
> 
> How this log will be recorded by the extension?
> Write the example of code that write to log

1) function someOuterFunction(){
  //some code here
  try {
    //some code that may not work
    dangerousMethod();
  } catch (error) {
    fs.appendFile('local://shared/logs/tiddlyFox/erros.log', error.message);
  }
  //some code here
}

2) function someOuterFunction(){
  //some code here

  //some code that may not work
  dangerousMethod().then(success, function(error){
    fs.appendFile('local://shared/logs/tiddlyFox/erros.log', error.message);
  });
  //some code here
}

3) function someOuterFunction(){
  //some code here

  //some code that may not work
  var result = dangerousMethod();

  if(result == undefined) {
    fs.appendFile('local://shared/logs/tiddlyFox/erros.log', runtime.lastError);
  }
  //some code here
}

4) runtime.onUnhandledException = function(error) {
     fs.appendFile('local://shared/logs/tiddlyFox/erros.log', error.message);
}

More questions?

> > need to meet these principles.
> 
> First of all, these APIs should function.

First rule -> don't harm
Second rule -> help him if can don't break the first rule

> If someone thinks that some feature is dangerous, the user must choose if you want it or not. The user
> and not the developer who chooses.

Who knows is it danger or not? Nobody! It can work fine and malicious code can sleep long time, meanwhile this extension will acquire big group of users. And when user will see confirm dialog, he look at extension home page and find: "over thousand downloads, five stars, no abuse reports" - ok! this is good extension! Confirm!

> The most secure API, that which does not exist.

Do you propose that we are choose:
  or make most secure API (that which does not exist)
  or make most unsecure API (that you beg from us)
?

We are choosing middle. 

> I now write the browser without any extensions. Or to do without a browser.
> After all, it will be safer.

You can still use legacy bootstrapped add-ons if you don't like webextensions

Comment 51

a month ago
> More questions?

It dawned on me. You suggest to make the file API with the limited special scheme "local". But the API is generally the same.
And the user can install the appropriate directories and local directories on your computer.

But do this in FireFox and so it was convenient. That is someone everyone needs to realize, so much so that the user chooses his need. And that it could, if user necessary, to remove all restrictions, as if the extension works with the whole filesystem.

Otherwise it can be very uncomfortable for the user.


> You can still use legacy bootstrapped add-ons if you don't like webextensions

As I understand it, the SDK functionality will be disabled in November.


> It can work fine and malicious code can sleep long time

The user have risks when installing application on your computer. For example, installation a licensed game with a safe driver or even a network card driver.
With them, the situation is much worse.


> Who knows is it danger or not? Nobody!
1. AMO gallery editors for gallery extensions.
2. User. User takes risks when installing any software on his computer. How he decides is his business. We only need to give him information about what he may really need it. The decision is the one who manages risk. Manages the user.
The developers of FireFox don't have to solve everything for everyone. People don't like it. People want to live in self mind. No Mozilla mind.


Working with the file system can be limited by means of the operating system if the user is afraid. I do for certain applications.

Comment 52

a month ago
>> And that it could, if user necessary, to remove all restrictions, as if the extension works with the whole filesystem.

Usage scenario: any save files: for example, simple encryption and file conversion.


That is, the user must still have the ability to set "local" scheme for this extension as if it was on the entire file system. I am afraid, however, it will be difficult to properly implement. But it will allow the user to find a balance between security and convenience.

Comment 53

a month ago
(In reply to fdsc from comment #51)
> But do this in FireFox and so it was convenient. That is someone everyone
> needs to realize, so much so that the user chooses his need. And that it
> could, if user necessary, to remove all restrictions, as if the extension
> works with the whole filesystem.

> Otherwise it can be very uncomfortable for the user.

If user so need to use this danger opportunity, then he can:
1. stick with "Nightly" or "Firefox developer edition" and install webexperiment (which has privileged API access) and install the webextension which will requests to webexperiment for privileged actions.
2. Some portion of time developers can use embedded webextensions. This is hybrid of legacy bootstrapped addon with new webextension in one single addon. This type of privileged add-ons we will support as long as possible so the developers have time to port their addons to new API (and migrate data of course). Two main difference is that you can't use XUL and can't use "chrome" module API.
 
> > You can still use legacy bootstrapped add-ons if you don't like webextensions
> 
> As I understand it, the SDK functionality will be disabled in November.

I am described the situation above.
 
> > It can work fine and malicious code can sleep long time
> 
> The user have risks when installing application on your computer.

On my computer? Or maybe on self computer?

> For example, installation a licensed game with a safe driver or even a network
> card driver.
> With them, the situation is much worse.

It's not the Mozilla problems. They only want to deliver better product, like Chrome/Opera developers.

> > Who knows is it danger or not? Nobody!
> 1. AMO gallery editors for gallery extensions.

No resources for that. You will be volunteer? Even that, who knows, maybe you friend of extension developer and purposely miss the danger extension.

> 2. User. User takes risks when installing any software on his computer. How
> he decides is his business. We only need to give him information about what
> he may really need it. The decision is the one who manages risk. Manages the
> user.

Many users don't read just confirm. And then "Omg! This browser so slow! It eat all of my RAM! It hang my system....". Again if some restriction is redundant then we away it. But at this moment you don't give me real justification that full filesystem access is very important for the main part of extensions. Example about tiddlyWiki is very custom, but I am resolve it for you. Example about logs and settings too. Do you have more usecases where full fs access very need?

> The developers of FireFox don't have to solve everything for everyone.
> People don't like it. People want to live in self mind. No Mozilla mind.

I think it's you trying to solve "everything for everyone". Who says that this concept don't cover all usecases? Or you trying to solve just self usecase and never mind about security of others?
 
> Working with the file system can be limited by means of the operating system
> if the user is afraid. I do for certain applications.

No! The reality is so that one system almost nevermind on ACL's (if you specially don't turn on this, what don't do 90% of users of this system), and other type of modern system use same user profile for all applications that user run (any app can see data produced by other app, and even modify). Little better situation with this in modern mobile systems, but we are talking about desktop systems now (if interesting, then I can give the cue that they use similar concept which I propose there).

Comment 54

a month ago
(In reply to fdsc from comment #52)
> >> And that it could, if user necessary, to remove all restrictions, as if the extension works with the whole filesystem.
> 
> Usage scenario: any save files: for example, simple encryption and file
> conversion.

Ok! I will describe an this usecase. Developer make extension which have page with form on it. In this form there is <input type="file" html component. And declare in manifest file of his extension that will be write to "local://tmp/megaencryptapp.out". Then when user open extension he must simply push on input=file button, then system's choose file dialog appears. When user choose and confirm, then content of selected file will appear for js scripts of extension through DOM File API (https://developer.mozilla.org/en/docs/Web/API/File). Then extension do the job on it. When it done, it write result to the "local://tmp/megaencryptapp.out" and open next page where will be written "Success! You can save the result right now by clicking on this <a href="local://tmp/megaencryptapp.out">link</a>". When user click on this link the standard open/save dialog appear and user can save result where would he like. And voila!

> That is, the user must still have the ability to set "local" scheme for this
> extension as if it was on the entire file system. I am afraid, however, it
> will be difficult to properly implement. But it will allow the user to find
> a balance between security and convenience.

Completely don't get whole paragraph.
Hello fdsc and jamy,

Thank you for your information, I think we have enough information to properly evaluate this proposal at the next meeting.

There is no reasons for you to continue arguing the points anymore, so I encourage you to end this discussion, as it doesn't seem very productive anymore.  Thank you.

Comment 56

14 days ago
I can just echo fdsc, many plugin depend on file access, TiddlyWikibeing one of them. We need some sort of solution not to loose many useful plugins.

Updated

3 days ago
Blocks: 1215059

Comment 57

a day ago
The easiest way to do this would probably be using the chrome.fileSystem app api. Firefox already added support for those filesystem interfaces for web folder upload. It may be possible to convince Chrome to enable that api for extensions.

https://developer.chrome.com/apps/fileSystem

Updated

9 hours ago
webextensions: ? → ---
You need to log in before you can comment on or make changes to this bug.