1642951 - After file upload on local network site firefox locks the file for a minute or so

Status: RESOLVED FIXED

(Core :: Networking, defect, P2)

Description

[László]

Hi,

I have a problem with Firefox v77. I uploaded a jpeg file to our server in a local network site, (so http://192.168.... not to http://somesite.com) and the file upload is a success, but after that I cant delete the file on my computer for a minute or so. It seems Firefox locks the file for a while.

Its only happening on Firefox v77. I uninstalled Firefox v77, installed v76 and works flawlessly. I tried it with a new profile on v77 with no add-ons installed, the bug remains. I tried it on another computer, same results. v76 works, v77 not.

I checked the upload on http://somesites, and it works with v77 too. So the problem seems to be on local network file uploads.

Please look into it.

Comment 1

[László]

After v77.0.1, the upload problem sometimes occurs at http://somesites, not just local sites.

Comment 2

[Stephen Gilbert]

I can now confirm this. I am uploading zip files to an external web-site (not a local network site). Previous to the v77 update it worked. Now the file is locked and I have to kill Firefox.

Comment 3

[Marco Bonardo [:mak]]

Could you please help us figuring out when this broke? You can use a tool called mozregression, from https://mozilla.github.io/mozregression/ that will download Firefox builds and let you tell whether they are good (no bug) or bad (has bug). In the end it provides you a link to a series of changes in our code that we can look into to identify the bug causing the problem.

Comment 4

[László]

Marco, thanks for getting back to me. I know when this is broke: After installing v77.

The last good version is this: https://download-installer.cdn.mozilla.net/pub/firefox/releases/76.0.1/win64/en-US/Firefox%20Setup%2076.0.1.exe

After that, the newer versions are bad:
https://download-installer.cdn.mozilla.net/pub/firefox/releases/77.0/win64/en-US/Firefox%20Setup%2077.0.exe
https://download-installer.cdn.mozilla.net/pub/firefox/releases/77.0.1/win64/en-US/Firefox%20Setup%2077.0.1.exe

I tried 77, and 77.0.1, still the same. I never used beta, or any other versions, I allways used stable versions.

I tried the regression app, but I have no idea what to do with it, sorry.

Comment 5

[Marco Bonardo [:mak]]

The problem is that 77 contains literally hundreds of bug fixes, while mozregression allows to to identify the exact day (so a small chunk of fixes) when it broke. The video tutorial on the page should help.
I'd like to be able to do it, but for that I should be able to reproduce the problem...

Comment 6

[László]

Attachment: the logfile

Okay, thanks for the tip, I could use the mozregression with the video.

Comment 7

[László]

Okay, thanks for the tip, I could use the mozregression with the video.

Looks like the wrong builds are:
2020-04-27
2020-04-27 15:52:48.475000
2020-04-27 11:19:31.980000
2020-04-27 07:21:41.893000
2020-04-27 07:07:05.662000
2020-04-27 07:04:13.119000

The good ones are:
build_date: 2020-04-20
build_date: 2020-04-24
build_date: 2020-04-26
build_date: 2020-04-27 11:15:09.185000
build_date: 2020-04-27 06:36:32.221000

I have the log attached.

Comment 8

[Marco Bonardo [:mak]]

Somehow it is pointing at the backout of bug 1626506... Lina, any idea?

Comment 9

[Dimitar Angelov]

I can confirm that bug is happens in Firefox v77.0.1 64-bit and also Firefox Developer Edition 78.0b9 64-bit in Windows 10 64-bit.
No matter on local file upload or network share file. The uploaded file is keep locked.

Comment 10

[:Gijs (he/him)]

Valentin/Matt, the backout in comment 8 seems unlikely given it relanded and doesn't relate to file uploads. Looking at a wider window from the log, https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=c9955025&tochange=12999095 , I noticed bug 1624318. Is it possible that could have something to do with the issue the reporter is seeing?

Comment 11

[Andrea Marchesini [:baku]]

Andrew do you have time to work on this issue?

Comment 12

[Daniel Bodea [:danibodea]]

I cannot confirm this issue since I don't have detailed enought steps to reproduce, but it was confirmed in comment 2.
Furthermore, a possible regression window and/or regressor were provided, so I will remove the "regressionwindow-wanted" tag and set the tracking flags according to it.

If the regressionwindow provided is incorrect or improbable, please put the "regressionwindow-wanted" tag bag and NI me. Some more detailed steps to reproduce would be necessary for me to attempt to provide a regressor.

Thank you for your contribution!

Comment 13

[Gian-Carlo Pascutto [:gcp]]

Comment 10 already explains the regressor is very unlikely to be correct. Knowing sure what the regressing bug is would help.

Comment 14

[Negritas Sergiu, Desktop QA]

Hello László !
Could you please provide more detailed steps on how to reproduce the issue?
Thank you!

Comment 15

[László]

(In reply to Negritas Sergiu from comment #14)

Hello László !
Could you please provide more detailed steps on how to reproduce the issue?
Thank you!

What do you need?

OS is windows 8.1 with the latest updates.
Firefox is v78, updated today with clean profile, so no addons.

PHP code for file uploading:
<?php

if (isset($_POST['submit'])) {
$src = $_FILES['upload']['tmp_name'];
$dest = 'pictures/picture.jpg';
move_uploaded_file($src,$dest);

echo "<META HTTP-EQUIV=Refresh CONTENT="0; URL=upload.php">";
exit;
}

echo '
<form method="post" enctype="multipart/form-data">
<input name="upload" type="file"></input>
<input type="submit" height="60" name="submit" value="upload">
</form>

<br>
<iframe src="pictures/" width="500" height="500" hspace="10" frameborder="0" scrolling="no"></iframe>';

?>

Comment 16

[László]

I made a video for easy understanding: https://youtu.be/sUrAiV8DMho

Comment 17

[:Gijs (he/him)]

(In reply to László from comment #15)

(In reply to Negritas Sergiu from comment #14)

Hello László !
Could you please provide more detailed steps on how to reproduce the issue?
Thank you!

What do you need?

OS is windows 8.1 with the latest updates.
Firefox is v78, updated today with clean profile, so no addons.

PHP code for file uploading:
<?php

if (isset($_POST['submit'])) {
$src = $_FILES['upload']['tmp_name'];
$dest = 'pictures/picture.jpg';
move_uploaded_file($src,$dest);

echo "<META HTTP-EQUIV=Refresh CONTENT="0; URL=upload.php">";
exit;
}

Does the issue persist if you remove the meta redirect?

Comment 18

[László]

(In reply to :Gijs (he/him) from comment #17)

(In reply to László from comment #15)

(In reply to Negritas Sergiu from comment #14)

Hello László !
Could you please provide more detailed steps on how to reproduce the issue?
Thank you!

What do you need?

OS is windows 8.1 with the latest updates.
Firefox is v78, updated today with clean profile, so no addons.

PHP code for file uploading:
<?php

if (isset($_POST['submit'])) {
$src = $_FILES['upload']['tmp_name'];
$dest = 'pictures/picture.jpg';
move_uploaded_file($src,$dest);

echo "<META HTTP-EQUIV=Refresh CONTENT="0; URL=upload.php">";
exit;
}

Does the issue persist if you remove the meta redirect?

Yes.

Comment 19

[Daniel Bodea [:danibodea]]

Thank you for the help, Laszlo! Unfortunately, we still don't know how to use this code, as it is a bit above our technical skill level.

@Gijs or @Laszlo: Can you help us include that code in a test page so we can attempt to provide a regression window?

Comment 20

[László]

Daniel, so what do you want me to do? I'm gladly help if I can.

Comment 21

[Negritas Sergiu, Desktop QA]

Hello Laszlo !
If you could provide a html file or a test page where the code provided by you can be used in order to find a regression window.

Thank you for your contribution!

Comment 22

[László]

The entire code is here:

<?php

if (isset($_POST['submit'])) {
$src = $_FILES['upload']['tmp_name'];
$dest = 'pictures/picture.jpg';
move_uploaded_file($src,$dest);

echo "<META HTTP-EQUIV=Refresh CONTENT="0; URL=upload.php">";
exit;
}

echo '
<form method="post" enctype="multipart/form-data">
<input name="upload" type="file"></input>
<input type="submit" height="60" name="submit" value="upload">
</form>

<br>
<iframe src="pictures/" width="500" height="500" hspace="10" frameborder="0" scrolling="no"></iframe>';

?>

Just save this code to "upload.php", make a folder on the server called pictures and make it writable (777). I cant give you an url, because its a local server. I tried the mozregression on it as Marco Bonardo asked.

Comment 23

[Dimitar Angelov]

I think that is not needed to focus on server backend script.
To reproduce the problem you can do following sequence on form with file upload element.

1. Open URL with form with upload element. For example: https://ps.uci.edu/~franklin/doc/file_upload.html
2. Select local file (there is no matter which will be). For example c:\temp\somefile.txt
3. Submit form to start file upload.
4. Try to delete local uploaded file c:\temp\somefile.txt. You will get error "Can't delete file" (Exact text of message will differ).
5. Close all Firefox windows.
6. Try to delete local uploaded file c:\temp\somefile.txt. OK

I hope that this scenario will be helpful.

Comment 24

[:Gijs (he/him)]

(In reply to Dimitar Angelov from comment #23)

I think that is not needed to focus on server backend script.
To reproduce the problem you can do following sequence on form with file upload element.

1. Open URL with form with upload element. For example: https://ps.uci.edu/~franklin/doc/file_upload.html
2. Select local file (there is no matter which will be). For example c:\temp\somefile.txt
3. Submit form to start file upload.

This doesn't work, I just get a 404 immediately (the form posts to https://www.oac.uci.edu/indiv/franklin/cgi-bin ). The file also can still be deleted.

I strongly suspect there is some other factor involved here, whether that's 3rd party software (antivirus?) or prefs or network latency or file size or... something... because so far I haven't been able to reproduce.

Comment 25

[László]

Gijs but if its not firefox fault how can the earlier version does not have this problem? How does this not affect firefox before v77 at all?

I tried the upload script with waterfox, what released yesterday. Works flawlessly. I know it is not firefox, but the base code is the same, right?

Comment 26

[:Gijs (he/him)]

(In reply to László from comment #25)

Gijs but if its not firefox fault how can the earlier version does not have this problem? How does this not affect firefox before v77 at all?

I'm not saying it's not Firefox's fault, I'm saying there's some other factor involved that trips this bug and we need to find out what that is, so that we can fix Firefox to behave correctly even when that other factor is present. If we don't know what this other factor is, we (the developers) cannot reproduce the bug and cannot fix it.

Comment 27

[László]

(In reply to László from comment #25)

I'm not saying it's not Firefox's fault, I'm saying there's some other factor involved that trips this bug and we need to find out what that is, so that we can fix Firefox to behave correctly even when that other factor is present. If we don't know what this other factor is, we (the developers) cannot reproduce the bug and cannot fix it.

Maybe you are trying on windows 10 (or other), while I am on windows 8.1?

Comment 28

[Dimitar Angelov]

Attachment: resource-monitorpng

(In reply to :Gijs (he/him) from comment #24)

(In reply to Dimitar Angelov from comment #23)

I think that is not needed to focus on server backend script.
To reproduce the problem you can do following sequence on form with file upload element.

1. Open URL with form with upload element. For example: https://ps.uci.edu/~franklin/doc/file_upload.html
2. Select local file (there is no matter which will be). For example c:\temp\somefile.txt
3. Submit form to start file upload.

This doesn't work, I just get a 404 immediately (the form posts to https://www.oac.uci.edu/indiv/franklin/cgi-bin ). The file also can still be deleted.

I strongly suspect there is some other factor involved here, whether that's 3rd party software (antivirus?) or prefs or network latency or file size or... something... because so far I haven't been able to reproduce.

Sorry that I'm not exact.
I will explain another cleaner case.

1. Start Windows Resource Monitor and in "Associated handles" section
   in search field type name of the file which you have to upload TEST-FILE_1.txt
2. Start Firefox and upload text file TEST-FILE_1.txt here
   http://www.csm-testcenter.org/test?do=show&subdo=common&test=file_upload
   using File upload via POST (HTTP)
3. Go to reasource monitor and click refresh search. After search complete will see
   in associates handles that firefox.exe is keep handle to that file TEST-FILE_1.txt.
   The handle is not free while you close firefox.

I'm attaching sample picture and hope to be useful.

Comment 29

[:Gijs (he/him)]

(In reply to Dimitar Angelov from comment #28)

Sorry that I'm not exact.

No worries!

I will explain another cleaner case.

1. Start Windows Resource Monitor and in "Associated handles" section
   in search field type name of the file which you have to upload TEST-FILE_1.txt
2. Start Firefox and upload text file TEST-FILE_1.txt here
   http://www.csm-testcenter.org/test?do=show&subdo=common&test=file_upload
   using File upload via POST (HTTP)
3. Go to reasource monitor and click refresh search. After search complete will see
   in associates handles that firefox.exe is keep handle to that file TEST-FILE_1.txt.
   The handle is not free while you close firefox.

Thanks, this is very helpful! I can reproduce this, but the handle does get freed if I close the tab that uploads. Perhaps that's a clue. Based on these steps, I found the following regression window:

https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=1e06eab47c7a65d6ff0a8671a8f678c8c53ba40c&tochange=612772266861a8e4864ae15e609a284d9ffda0e8

but there aren't Windows builds for some of these, so I can't narrow it down further. Still, bug 1624318 seems like the obvious choice given the others are test-only or backouts, or to do with webrender.

I'm attaching sample picture and hope to be useful.

I don't suppose you have any idea why on my (Windows 10) machine, even when the handle is kept open, I can delete the file fine in windows explorer? Does this depend on it being a network share, or the filesystem (NTFS versus FAT or whatever) or the OS version, or is Windows explorer significantly different from Total Commander?

Comment 30

[Dimitar Angelov]

(In reply to :Gijs (he/him) from comment #29)

I don't suppose you have any idea why on my (Windows 10) machine, even when the handle is kept open, I can delete the file fine in windows explorer? Does this depend on it being a network share, or the filesystem (NTFS versus FAT or whatever) or the OS version, or is Windows explorer significantly different from Total Commander?

I did some little more experiments.

1. If I open uploaded text file with notepad I can write in it after upload, but if I open with PSPad editor I can't write anymore after upload.
   I suppose that this depends of mode in which is opened file handle from firefox and another application.
2. If I have opened more that one tab to upload page file handle is not freed until close all tabs to this site.
   The file handle is freed little bit time after close, may be result of some resource releasing process.

Additional info for my OS environment: Windows 10 Pro 64-bit, Version 1909, OS Build 18363.900.
For tests I have disabled antivirus protection and all firefox addons.

Comment 31

[:Gijs (he/him)]

Through complete coincidence in reviewing some unrelated code, I was enlightened to find that MS in its infinite wisdom has created some specific flags to allow other files to access files you open with CreateFile APIs per operation. That is, you have to specify separate flags to allow other programs to read, write or delete the file in question. See the FILE_SHARE_* flags on https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea . So I guess that's it, and somewhere is failing to call PR_Close on the nspr file desc which was opened without the FILE_SHARE_DELETE flag...

So the next question is... why aren't we closing that file once read? Do we treat file uploads as blobs (presumably because we create an html File instance for the file input or something) which we have to keep alive while they could be accessed by the webpage - even though they can't be accessed directly after upload, AFAICT?

Comment 32

[Matt Woodrow (:mattwoodrow)]

(In reply to :Gijs (he/him) from comment #31)

Through complete coincidence in reviewing some unrelated code, I was enlightened to find that MS in its infinite wisdom has created some specific flags to allow other files to access files you open with CreateFile APIs per operation. That is, you have to specify separate flags to allow other programs to read, write or delete the file in question. See the FILE_SHARE_* flags on https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilea . So I guess that's it, and somewhere is failing to call PR_Close on the nspr file desc which was opened without the FILE_SHARE_DELETE flag...

So the next question is... why aren't we closing that file once read? Do we treat file uploads as blobs (presumably because we create an html File instance for the file input or something) which we have to keep alive while they could be accessed by the webpage - even though they can't be accessed directly after upload, AFAICT?

I think we're keeping a reference to the Blob alive in the content process, so that if you attempt to reload the page we can resubmit the form.

I'm not sure why this keeps it actually locked though.

Do you have any ideas baku?

Comment 33

[Jens Stutte [:jstutte]]

Hi Tom, can you take a look here, please, if you can help with your Windows file expertise?

Comment 34

[Toshihito Kikuchi [:toshi]]

I tested the upload scenario with Firefox 80.0.1 x64 on Win8.1 and Win10. On both Windows versions, I saw the exact same behavior as described in comment 29: A handle to the uploaded file was kept alive in the browser process, but I could edit, rename, and delete it. Upon looking under debugger, the remaining handle was opened with FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE in the following callstack. So everything seems expected.

00 000000a5`90ffce28 00007fff`bdeb6207 xul!nsLocalFile::OpenNSPRFileDescShareDelete
01 000000a5`90ffce30 00007fff`be6b5d76 xul!nsFileStreamBase::DoOpen+0x147
02 (Inline Function) --------`-------- xul!nsFileStreamBase::DoPendingOpen+0x2c
03 000000a5`90ffcee0 00007fff`be6b2105 xul!nsFileStreamBase::Seek+0x36
04 000000a5`90ffcf20 00007fff`be5d1d66 xul!nsBufferedStream::Seek+0xe5
05 000000a5`90ffcfa0 00007fff`be6c5a35 xul!nsMultiplexInputStream::Seek+0x1d6
06 000000a5`90ffd040 00007fff`be954773 xul!nsMIMEInputStream::Seek+0x75
07 000000a5`90ffd0a0 00007fff`bea21e7c xul!mozilla::net::HttpBaseChannel::CloneReplacementChannelConfig+0x7a3
08 000000a5`90ffd460 00007fff`bea21ac5 xul!mozilla::net::DocumentLoadListener::SerializeRedirectData+0x31c
09 000000a5`90ffda70 00007fff`bea32a72 xul!mozilla::net::DocumentChannelParent::RedirectToRealChannel+0x105
0a (Inline Function) --------`-------- xul!mozilla::net::DocumentChannelParent::Init::<unnamed-tag>::operator()+0x1c

Comment 35

[Tom Tung [:tt, :ttung]]

Files and Blobs can be read via their nsIInputStream and the inputStream can be implemented differently (and thus be created differently) depends on the purpose/usages of File/Blob.

If the implementation of the File/Blob is FileBlobImpl, the stream should be created with nsIFileInputStream::SHARE_DELETE. And that matches the observation in comment 34.

I need to look closer to implementation.

Comment 36

[Tom Tung [:tt, :ttung]]

I tested the STR in comment 28 with FF82.0a1 on Win8.1 (VM) and I saw the filehandle still existed and I couldn't end the process through right-clicking after I closed firefox. (I was able to end the process through right-clicking before I closed firefox)

However, if I then refresh the search for associates handles, the filehandle disappears.

I will check if the InputStream is created with passing the FILE_SHARE_DELETE flag.

Comment 37

[Tom Tung [:tt, :ttung]]

The inputStream for reading the file is created by FileBlobImpl on the parent process and the content side creates StreamBlobImpl.
If it's FileBlobImpl, then it's created with FILE_SHARE_DELETE flag.

I tested the STR in comment 28 with FF78.0 and I was able to delete the file after firefox was closed.

Comment 38

[Toshihito Kikuchi [:toshi]]

László, thank you for your patience. Unfortunately we have not yet reproduced the problem or got to the root cause.

Could you capture Firefox's File I/O with Process Monitor when you upload a file and share it with us? It can record the process's file access in API level, so it might be helpful to understand how an uploaded file is locked and becomes undeletable.

Comment 39

[László]

(In reply to Toshihito Kikuchi [:toshi] from comment #38)

László, thank you for your patience. Unfortunately we have not yet reproduced the problem or got to the root cause.

Could you capture Firefox's File I/O with Process Monitor when you upload a file and share it with us? It can record the process's file access in API level, so it might be helpful to understand how an uploaded file is locked and becomes undeletable.

Sure. I attached the process monitor cat.jpg upload log.

Comment 40

[László]

Attachment: Logfile for cat.jpg upload.

Comment 41

[László]

The funny thing that I could delete the log file right away after upload, but still can't delete cat.jpg.

Comment 42

[Toshihito Kikuchi [:toshi]]

Can you share a .PML file so that we can see callstacks of each event?

Comment 43

[László]

(In reply to Toshihito Kikuchi [:toshi] from comment #42)

Can you share a .PML file so that we can see callstacks of each event?

Sorry but no. I cant verify PML file contents what it is sending to you.

Comment 44

[Toshihito Kikuchi [:toshi]]

(In reply to László from comment #43)

Sorry but no. I cant verify PML file contents what it is sending to you.

I see. As an alternative I was checking Profiler, but I think that's not so helpful for this case.

Looking at the CSV closely, I noticed something different from my local result.

First, I think the operations recorded in the CSV could be split into three parts:

1. QueryDirectory@18:13:15.0201251 - CloseFile@18:13:15.1441686
2. CreateFile@18:13:15.5632049 - CloseFile@18:13:15.5967348
3. CreateFile@18:13:16.8952399 - ReadFile@18:13:16.9034269

The 3rd part should be the actual uploading part. This is the exact same pattern as my local result. The last handle in this part is kept alive, but it's not a problem here because it's opened in the shared mode as we already discussed.

The 2nd part should be the selecting part on the common dialogbox. I saw the same pattern here, too, and there is no handle leak here.

A suspicious part is the first part. First of all, I don't see these operations on my end. Especially, calling CreateFileMapping for a jpeg image looks strange to me. If a file mapping object was created for a local file and somehow a handle to the mapping object was kept alive, I believe the file should be locked like this case. I'm suspecting this part was done by a shell extension when the image file was displayed in the dialogbox.

Comment 41 could be a different hint. László, was there any difference between the log file and cat.jpg, especially the file location? For example, if a file is located at a special folder managed by DropBox or OneDrive, a shell extension is involved.

Another experiment that might narrow down the issue is to check when the file is locked. Can you please check after you select a file in the dialog and click the ok button, whether a file is locked BEFORE submitting it to the server via POST (i.e. before clicking the submit button)?

Comment 45

[László]

(In reply to Toshihito Kikuchi [:toshi] from comment #44)

I checked. Before upload the cat.jpg file is not held hostage by firefox.exe, only after the upload.

I tried uploading from various locations. Tried from desktop, folder from anywhere in pc, same results. No dropbox, or onedrive is installed.

Log from nirsoft openedfilesview if that helps:

==================================================
Filename          : cat.jpg
Full Path         : C:\Users\PC\Desktop\cat.jpg
Handle            : 0x2130
Created Time      : 2020-09-12 15:39:18
Modified Time     : 2020-09-12 15:39:04
Attributes        : A
File Size         : 157.955
Read Access       : *
Write Access      : 
Delete Access     : 
Shared Read       : *
Shared Write      : *
Shared Delete     : *
Granted Access    : 0x00120089
File Position     : 8.192
Process ID        : 3220
Process Name      : firefox.exe
Process Path      : C:\Program Files\Mozilla Firefox\firefox.exe
Process User      : PC\PC
Elevated Process  : No
Extension         : jpg
% Position        : 5.19%
==================================================

The only "solution" is windows 10. Under windows 10 the problem does not exists.

Comment 46

[László]

Sorry for the big fonts, I don't know why it became like this.

Comment 47

[:Gijs (he/him)]

(In reply to László from comment #46)

Sorry for the big fonts, I don't know why it became like this.

No worries, it's markdown - I fixed the formatting for you.

Passing needinfo back to :toshi for comment #45.

Comment 48

[László]

(In reply to :Gijs (he/him) from comment #47)

(In reply to László from comment #46)

Sorry for the big fonts, I don't know why it became like this.

No worries, it's markdown - I fixed the formatting for you.

Thanks!

Comment 49

[Toshihito Kikuchi [:toshi]]

(In reply to László from comment #45)

(In reply to Toshihito Kikuchi [:toshi] from comment #44)

I checked. Before upload the cat.jpg file is not held hostage by firefox.exe, only after the upload.

I tried uploading from various locations. Tried from desktop, folder from anywhere in pc, same results. No dropbox, or onedrive is installed.

Log from nirsoft openedfilesview if that helps:

Okay, thank you for verifying those scenarios. The result on OpenedFilesView shows that the handle was opened with the shared mode correctly. Keeping that handle should not block the file. I tried on Win8.1, but it behaved the same as Win10.

I'm still suspecting CreateFileMapping. I confirmed even with a file was opened in the shared mode, if a mapping object was created based on that handle, the file was locked. So it can explain the situation. Another reason is because I don't see that mapping operation on my end, I'm afraid that I'm doing steps a bit differently.

To understand who does CreateFileMapping for cat.jpg, Process Monitor is still helpful. You don't have to share a .PML file, but can you double-click each CreateFileMapping operation and save the content in the Stack tab to share with us? The stack info will tell us who triggered it and how. If any third-party module is involved, it will be shown there.

Comment 50

[László]

Attachment: 111.png

Okay, I tried to do a screenshot, hope that helps.

Comment 51

[László]

(In reply to Toshihito Kikuchi [:toshi] from comment #49)

Okay, I tried to do a screenshot, hope that helps.

Comment 52

[Toshihito Kikuchi [:toshi]]

Thank you, László. It seems this file mapping was done by Explorer's preview feature, but I confirmed the same operation, but it closes a handle correctly. I think I went to a wrong path too far. Sorry about that.

Now I needed to come back to the original doubt about the remaining handle.

Even after upload, we still keep a file handle opened. It is opened in shared mode, but I realized file deletion behaves a bit differently between Win10 and Win8.1. On Win10, when a file which has been opened in shared mode is deleted, it's not totally deleted but internally moved to a special NTFS directory $Extend\$Deleted. On Win8.1, on the other hand, deletion succeeds as Win10, but the deleted file still stays in the same location. And what is worse is after the first deletion, subsequent operations fail with access denied. When the last handle is closed, the file is finally deleted. Maybe this is the issue here.

As Gijs found out, the remaining handle was introduced by bug 1624318. More specifically, this line opens a handle and the browser process keeps it. Although the browser process owns it, it's closed when the tab process is terminated.

Matt, I think we need your help now. In comment 32, you said we keep a reference to the Blob in the content process, but when I debug this scenario, the handle is owned by the browser process. Upon reading the code, I think (not 100% sure) the stream is serialized and stored in the browser process as RemoteLazyInputStreamStorage. The problem is the stream holds a handle opened. Do you think it is possible to store the stream without opening a handle, or any other idea to avoid this issue?

Comment 53

[Matt Woodrow (:mattwoodrow)]

We send a reference to the file back to the content process (using RemoteLazyInputStreamStorage) so that reloading a page with a form submission can find the file to submit again.

As part of doing that, we seek the back to the start of the file, so that it's ready for reading again. Unfortunately this actually opens the file, which we had previously closed once we'd finish submitting it the first time.

We already support having a lazily-opened file, it's just that seeking to 0 fails to respect this.

Comment 54

[Matt Woodrow (:mattwoodrow)]

Attachment: Bug 1642951 - Don't open the underlying file unnecessarily when seeking nsFileInputStream to the start. r?baku

Comment 55

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/cfae2cf1e250
Don't open the underlying file unnecessarily when seeking nsFileInputStream to the start. r=baku,necko-reviewers,valentin

Comment 56

[Dorel Luca [:dluca]]

https://hg.mozilla.org/mozilla-central/rev/cfae2cf1e250

Comment 57

[BugBot [:suhaib / :marco/ :calixte]]

Since the status are different for nightly and release, what's the status for beta?
For more information, please visit auto_nag documentation.

Comment 58

[Toshihito Kikuchi [:toshi]]

(In reply to Matt Woodrow (:mattwoodrow) from comment #53)

We send a reference to the file back to the content process (using RemoteLazyInputStreamStorage) so that reloading a page with a form submission can find the file to submit again.

As part of doing that, we seek the back to the start of the file, so that it's ready for reading again. Unfortunately this actually opens the file, which we had previously closed once we'd finish submitting it the first time.

We already support having a lazily-opened file, it's just that seeking to 0 fails to respect this.

Thank you for landing a patch! Unfortunately changing nsFileInputStream::SeekInternal is not enough. We still open a handle at the last line of nsBufferedStream::Seek where we end up opening a handle. Below is the callstack. Can we suspend this Fill(), too?

00 000000e2`1a9fcb48 00007ffb`aebb16c4 xul!nsLocalFile::OpenNSPRFileDescShareDelete
01 000000e2`1a9fcb50 00007ffb`aebb1351 xul!nsFileStreamBase::DoOpen+0x114
02 (Inline Function) --------`-------- xul!nsFileStreamBase::DoPendingOpen+0x39
03 000000e2`1a9fcc10 00007ffb`aebb1eb6 xul!nsFileStreamBase::Read+0x41
04 000000e2`1a9fcc60 00007ffb`aebad936 xul!nsFileInputStream::Read+0x16
05 000000e2`1a9fcca0 00007ffb`aebac9d6 xul!nsBufferedInputStream::Fill+0x86
06 000000e2`1a9fccf0 00007ffb`aeaac671 xul!nsBufferedStream::Seek+0x176
07 000000e2`1a9fcd70 00007ffb`aebc2f1a xul!nsMultiplexInputStream::Seek+0x281
08 000000e2`1a9fce10 00007ffb`aee86379 xul!nsMIMEInputStream::Seek+0x8a
09 000000e2`1a9fce70 00007ffb`aef653a1 xul!mozilla::net::HttpBaseChannel::CloneReplacementChannelConfig+0x8e9

Comment 59

[Matt Woodrow (:mattwoodrow)]

(In reply to Toshihito Kikuchi [:toshi] from comment #58)

(In reply to Matt Woodrow (:mattwoodrow) from comment #53)

We send a reference to the file back to the content process (using RemoteLazyInputStreamStorage) so that reloading a page with a form submission can find the file to submit again.

As part of doing that, we seek the back to the start of the file, so that it's ready for reading again. Unfortunately this actually opens the file, which we had previously closed once we'd finish submitting it the first time.

We already support having a lazily-opened file, it's just that seeking to 0 fails to respect this.

Thank you for landing a patch! Unfortunately changing nsFileInputStream::SeekInternal is not enough. We still open a handle at the last line of nsBufferedStream::Seek where we end up opening a handle. Below is the callstack. Can we suspend this Fill(), too?

00 000000e2`1a9fcb48 00007ffb`aebb16c4 xul!nsLocalFile::OpenNSPRFileDescShareDelete
01 000000e2`1a9fcb50 00007ffb`aebb1351 xul!nsFileStreamBase::DoOpen+0x114
02 (Inline Function) --------`-------- xul!nsFileStreamBase::DoPendingOpen+0x39
03 000000e2`1a9fcc10 00007ffb`aebb1eb6 xul!nsFileStreamBase::Read+0x41
04 000000e2`1a9fcc60 00007ffb`aebad936 xul!nsFileInputStream::Read+0x16
05 000000e2`1a9fcca0 00007ffb`aebac9d6 xul!nsBufferedInputStream::Fill+0x86
06 000000e2`1a9fccf0 00007ffb`aeaac671 xul!nsBufferedStream::Seek+0x176
07 000000e2`1a9fcd70 00007ffb`aebc2f1a xul!nsMultiplexInputStream::Seek+0x281
08 000000e2`1a9fce10 00007ffb`aee86379 xul!nsMIMEInputStream::Seek+0x8a
09 000000e2`1a9fce70 00007ffb`aef653a1 xul!mozilla::net::HttpBaseChannel::CloneReplacementChannelConfig+0x8e9

How did you reproduce this stack happening?

Using the steps in comment 28, this appeared to be fixed already for me.

Comment 60

[Toshihito Kikuchi [:toshi]]

(In reply to Matt Woodrow (:mattwoodrow) from comment #59)

How did you reproduce this stack happening?

Using the steps in comment 28, this appeared to be fixed already for me.

I used my local PHP page on the latest Nightly (20200924093415), but I confirmed http://www.csm-testcenter.org/test?do=show&subdo=common&test=file_upload also 100% reproduced the same. Uploading a jpg image or a simple plaintext was enough.

Comment 61

[Matt Woodrow (:mattwoodrow)]

Ah, my test file was 0 bytes, which took a different path.

Comment 62

[Matt Woodrow (:mattwoodrow)]

Attachment: Bug 1642951 - Don't fill the nsBufferedStream buffer when seeking to 0, to avoid opening lazy inner streams. r?baku

Comment 63

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a70128c7f6b1
Don't fill the nsBufferedStream buffer when seeking to 0, to avoid opening lazy inner streams. r=baku,necko-reviewers

Comment 64

[Cristina Coroiu [:ccoroiu]]

https://hg.mozilla.org/mozilla-central/rev/a70128c7f6b1

Comment 65

[László]

Thanks for the whole team fixing this bug!