Closed Bug 496825 Opened 11 years ago Closed 8 years ago

Norton Antivirus software triggering NS_ERROR_FILE_ACCESS_DENIED exceptions when using

Categories

(Core :: Networking: File, defect, critical)

x86
Windows XP
defect
Not set
critical

Tracking

()

RESOLVED WORKSFORME

People

(Reporter: morac, Unassigned)

References

(Blocks 1 open bug)

Details

(Keywords: dataloss, regression)

Attachments

(1 file)

Lately I've been noticing that trying to install an add-on results in one of the following exceptions that vast majority of the time, unless I disable the "Antivirus Auto-Protect" functionality in Norton Internet Security 2009:

Error: Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsISafeOutputStream.finish]
Source file: file:///C:/Program%20Files/Mozilla%20Firefox%203.5/components/nsExtensionManager.js
Line: 530

Error: Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIRDFRemoteDataSource.Flush]
Source file: file:///C:/Program%20Files/Mozilla%20Firefox%203.5/components/nsExtensionManager.js
Line: 8858

With the help of mzz on irc.mozilla.org, I determined that the fix to bug 477934 is triggering this problem.  Prior to this fix being checked in this wasn't a problem.

Currently trying to install an add-on fails about 80% of the time in a profile that has 40 add-ons installed (even when said add-ons are disabled).  I have another profile with only 15 add-ons in it and while it also throws the same exceptions, add-ons install "successfully" far more often.


This not only affects nsExtensionManager.js, but also can cause exceptions to occur when extensions try to save the prefs.js file when the browser starts up.  I now regularly see one or more of the following errors when Shiretoko or Minefield starts up (the addons are Tab Mix Plus, NoScript and Session Manager respectively):

Error: Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIPrefService.savePrefFile]
Source file: chrome://tabmixplus/content/session/session.js
Line: 471

Error: Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIPrefService.savePrefFile]
Source file: file:///C:/Documents%20and%20Settings/Michael%20Kraft/Application%20Data/Mozilla/Firefox/Profiles/7pqoqhhx.Test%20Profile/extensions/%7B73a6fe31-595d-460b-a920-fcc0f8843232%7D/components/noscriptService.js
Line: 1023

Error: Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIPrefService.savePrefFile]
Source file: file:///C:/Documents%20and%20Settings/Michael%20Kraft/Application%20Data/Mozilla/Firefox/Profiles/dfrpobgv.Default%20User/extensions/%7B1280606b-2510-4fe0-97ef-9b5a22eafe30%7D/components/SessionManagerHelperComponent.js
Line: 99


See also bug 430579 which describes only part of the problem (addon, but not preferences).

The work around is to exclude the profile directory from the antivirus auto-protection.
Note the specific operation causing the problem as reported by procmon is as follows:

7694	4:44:03.1159779 PM	firefox.exe	1144	SetRenameInformationFile	C:\Documents and Settings\Michael Kraft\Application Data\Mozilla\Firefox\Profiles\7pqoqhhx.Test Profile\extensions-1.ini	ACCESS DENIED	ReplaceIfExists: True, FileName: C:\Documents and Settings\Michael Kraft\Application Data\Mozilla\Firefox\Profiles\7pqoqhhx.Test Profile\extensions.ini


It can happen with any of the extensions.* files.  I've also occasionally seen it with the pref.js file, but not anywhere nearly as often as with the extension.* files.

54	4:57:02.7100975 PM	firefox.exe	2208	SetRenameInformationFile	C:\Documents and Settings\Michael Kraft\Application Data\Mozilla\Firefox\Profiles\7pqoqhhx.Test Profile\prefs-1.js	ACCESS DENIED	ReplaceIfExists: True, FileName: C:\Documents and Settings\Michael Kraft\Application Data\Mozilla\Firefox\Profiles\7pqoqhhx.Test Profile\prefs.js


Note I saw the above error when exiting Firefox which results in the prefs.js file not be saved on exit.
Just saw the following as well:

Error: Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsISafeOutputStream.finish]
Source file: file:///C:/Program%20Files/Mozilla%20Firefox%203.5/components/nsSessionStore.js
Line: 2790


Excluding my profile from being "auto-protected" by Norton will stop these errors from occurring so that's a work around.
I've tried to reproduce this with non-safe stream files and simply can't.  I
even wrote an add-on that does a moveTo over and over and over and I couldn't
reproduce this, so the problem only occurs with "safeStream" files.

I'm not sure why exactly.  The only files I've seen get the "ACCESS DENIED"
error are:

extensions.rdf, extensions.cache, extensions.ini, prefs.js and sessionstore.js

I'm guessing localstore.rdf might also be susceptible because it does the same
thing as the other files.

I don't know why any of the other files (like the *.sqlite) don't seem to have
this problem.  I'm also not sure why doing a Flush() before a MoveToNative()
call triggers the problem.

The only thing I can think of is that it's not the rename to file that's
locked, but the rename from file.  In other words it's not that extensions.rdf
is locked, but extensions-1.rdf is locked.  This would make sense because a
Flush() would trigger the A/V program to scan the file which would prevent the
file from being renamed.

If that's the case, then simply waiting a second or two after the Flush()
should work around the problem.  I'm not sure that's practical though.
I have an idea on how to possibly fix this which I'll hopefully get around to trying.  Assuming my presumption above is correct, the problem can be fixed by doing a copyTo and then remove.

So basically something like this:

rv = mTempFile->MoveToNative(nsnull, targetFilename);
if (NS_FAILED(rv)) {
  rv = mTempFile->MoveToNative(nsnull, targetFilename);
  mTempFile->Remove(PR_FALSE);
}


timeless mentioned to me in irc that this might not be a good idea since it can can hurt time stats, but I think that's better than having files completely fail to be written.  I'll provide more info when I try the above (and put together a test case).
In hindsight I'm thinking the above won't work.  The reason being is that everything works if I exclude the original files.  If the file being locked was the temporary file, this wouldn't have worked.  So the issue must be with the original file being locked.  I'm not sure why Norton would be locking the destination file on a move.

I'm convinced there is something else at play here since I created an extension which opens a nsSafeFileOutputStream, writes to it and then "Finishes" it.  I've had it repeat these steps 100 times in a row and never saw an exception.  So there must be something else happening with the browser safe files that's causing Norton to lock them.
Just to clarify when I say "exclude" above, I mean add the files to the list of Norton's files to exclude from A/V auto-protection.  For example if I exclude "extensions.rdf", then that file will never cause an ACCESS_DENIED error.
I just realized something.  What file is the "Flush()" that was added by bug 477934 acting on?

Since it is written as "Flush()" and not "nsFileOutputStream::Flush()", I wonder if the original file is being flushed instead of the temporary file.  If that is the case, then that would make more sense since doing a write on the original right before the temporary file was moved on top of it could result in an ACCESS_DENIED if Norton traps the flush.

I feel like I'm having a conversation with myself.  Hopefully someone else will comment.
Anyone have any contacts at Norton ?
I started a thread over at Norton's feedback boards.  Symantec people are known to read the boards, but officially they don't support Firefox 3.5 until it's released.

http://community.norton.com/norton/board/message?board.id=nis_feedback&thread.id=55461

Hopefully someone has better contacts.
I tried to reproduce this problem on a different machine with McAfee installed, but could not.  Since the hardware on that machine is different (the drive is faster), I don't know how valid a test that is.

Further testing showed that if I prevent Norton from scanning either the original files or the temporary (*-#.*) ones everything works fine.   My guess as to what's happening here is that when the Flush() is done, Norton detects this and tries to scan it (though I see no indication of such in Procmon), this effectively causes Windows to not allow the file to be moved or renamed.  The other possibility is that the actual move is causing it to be scanned.  The flush could trigger this since any changes to a file will cause Norton to scan that file the next time it is accessed.

This has the potential of data loss for any component using a safe-file-output-stream. http://mxr.mozilla.org/mozilla-central/search?string=@mozilla.org/network/safe-file-output-stream

As for why I see this frequently when installing add-ons, it is because doing so results in nsSafeFileOutputStream::Finish() being called a fairly large number of times during the installation process.  Each time the nsSafeFileOutputStream::Close() is executed there is a small chance the move will fail.

I tried putting together a build that tried to do another moveToNative if the first one failed, but that didn't help as I don't think it gave enough time for the file to be unlocked.  I can also try doing a remove of the target file, a copyToNative and then a remove of the temp file, but that has the potential of losing all data on a crash.

At this point the only solutions are to get Norton to change something on their end (Tech Evang) or change the code to detect a failed move and try again a short time later instead of deleting the temporary file (losing all changes) as it does currently.
Dataloss is critical. Adjusting the severity of this bug. Is it worth blocking1.9.1?
Severity: major → critical
Flags: blocking1.9.1?
Keywords: dataloss
FWIW this is really just bug 431065 and has been around since Firefox 3.0 though it is possible that some of the recent changes make it more likely for us to hit it.
Technically this bug, bug 431065 and bug 430579 are all the same problem. 

In my case I never saw the problem as documented in either bug 431065 or bug 430579 in Firefox 3.0 or later until the fix for bug 477934 dropped.  Now with the constant flushing of the safe output stream files I see it fairly frequently.

What's strange though is that I wrote an addon based off the StartupCache_write() function in nsExtensionManager.js, which does a hundred nsSafeFileOutputStream::Init() and nsSafeFileOutputStream::Finish() calls in a row and never once triggered an ACCESS DENIED error.  Yet I see the error very frequently when installing add-ons, less frequently when saving preferences and rarely in other "safe-file" cases.  I don't see why that's any different than what the current code does.
Summary: Antivirus software triggering NS_ERROR_FILE_ACCESS_DENIED exceptions when trying to install add-on and saving preferences → Norton Antivirus software triggering NS_ERROR_FILE_ACCESS_DENIED exceptions when using
Just talked to Sam and it sounds like that this is something we deal with for every release. The fix is for Symantec to add the firefox binary to some database.

Sam, can we somehow confirm that that's what's going on here, and if so either mark this a non-blocker or move it to some component where we can treat this as a non-code blocker.
I've emailed Symantec (Jonas is CCed) to see if this is fixed when we send them our final binaries or not.
I'm wondering if this is something that can be fixed on our end.  

The reason I say that is I put together a test program based on the code listed at http://msdn.microsoft.com/en-us/library/aa363875%28VS.85%29.aspx and what's in the nsFileStreams.cpp and nsLocalFileWin.cpp files.  I'm attaching it.

The only things I changed was to have the temporary file created in the current folder, for the program to flush the temp file before close and for the temp file to be moved onto the original file.  So it basically behaves exactly like our "safe file" except that it uses the GetTempFileName Windows call instead of manually creating one.  If this kind of operation fails in Firefox with Norton enabled, I would think it should also fail with this program.

I wrote a batch file to run the executable continuously and let it run for a few minutes and not once received any kind of error, no matter how large or small the input file I gave it.  I even went so far as to open the original file in notepad++ and modify and save it while the batch file was still running, which worked fine.  I also tried switching share mode from 0 to FILE_SHARE_READ|FILE_SHARE_WRITE which also worked.

My question is why does this work fine with Norton, but Firefox does not?
(In reply to comment #14)
> Just talked to Sam and it sounds like that this is something we deal with for
> every release. The fix is for Symantec to add the firefox binary to some
> database.

That's somewhat lame as this means that everyone doing any Mozilla-based application needs to contact them and get their binaries included there, right? Is there any way we can deal with this in our code to make it more error-proof against this case?
Shawn: can you look at comment 16 and let us know if we should be making a change here?

I filed blocking bug 497360 to ensure we get ourselves added for Firefox 3.5
Flags: blocking1.9.2?
Flags: blocking1.9.1?
Flags: blocking1.9.1-
(In reply to comment #18)
> Shawn: can you look at comment 16 and let us know if we should be making a
> change here?
I don't know anything about our windows file system API's/uses, so jimm or rob_strong would probably be better people to take a look here.  If they are too busy, I can jump in and try though.
Bug 477934 was initially only for Linux (bug 477934 comment #3 has a link to a specific Linux issue) and was later changed to All / All in order to make the writing to disk from nsSafeFileOutputStream more reliable across all platforms per bug 318156. Note that there are several bugs duped to Bug 477934 for Windows so it isn't a simple thing to just ifdef the Flush so it doesn't happen on Windows.

Looking at nsSafeFileOutputStream / nsFileStreams.cpp and nsLocalFileWin.cpp nothing looks like it would cause this though as have been seen elsewhere multiple writes to a file has caused havoc with AV software in the past. It also appears that this may in part be caused by Norton aggressively inspecting files in case the file is related to a rootkit (which they should be doing IMO). It was noted in bug 430579 that Avira AV made some changes to the way they handle move operations which seems to have improved things and perhaps there is something similar that Norton can do as well though I am not suggesting this would be a complete fix.
(In reply to comment #15)
> I've emailed Symantec (Jonas is CCed) to see if this is fixed when we send them
> our final binaries or not.

It's not (per Symantec), but they've passed the email on to the appropriate team and we should hear back from them soon.
(In reply to comment #17)
> That's somewhat lame as this means that everyone doing any Mozilla-based
> application needs to contact them and get their binaries included there, right?

That's mostly correct. We currently do this for Firefox and Thunderbird, but they also manually update when they see if another app has been released. I'm guessing this is done for SeaMonkey by Symantec or you wouldn't have any Windows users who used Norton at all. The consequence of them not having final binaries and/or doing the update themselves is Norton blocking internet access for the application. Users can add exceptions manually, however.

> Is there any way we can deal with this in our code to make it more error-proof
> against this case?

If by "this case" you mean this bug, then I don't know. If by "this case" you mean what I just described above in this comment, then the answer is "no, there's nothing we can do".
I just want to clarify in case there is any confusion.  The problem I documented has nothing to do with Norton's software firewall.  It has to do with it's Norton anti-virus scanning.

So far the only work around I've found that works is to manually add an exception to prevent the profile directory from being automatically scanned (technically only the "safe-files" need to be excluded, but excluding the whole folder is easier).
(In reply to comment #22)
> (In reply to comment #17)
> > Is there any way we can deal with this in our code to make it more error-proof
> > against this case?
> 
> If by "this case" you mean this bug, then I don't know. If by "this case" you
> mean what I just described above in this comment, then the answer is "no,
> there's nothing we can do".

I mean this bug, yes. If we get an "access denied" and that causes dataloss, I wonder how long they block access while scanning, i.e. if it might make sense to spin/wait for (up to) a short, tolerable time and retry so the safe file write we want to perform can happen and actually be safe.
Depends on: 431065
For the record, I just encountered a somewhat similar problem with BitDefender Antivirus 2009. Any attempt to use the Firefox search bar would fail with the following message logged to the error console:

Error: uncaught exception: [Exception... "Component returned failure code: 0x80520015 (NS_ERROR_FILE_ACCESS_DENIED) [nsIFormHistory2.addEntry]"  nsresult: "0x80520015 (NS_ERROR_FILE_ACCESS_DENIED)"  location: "JS frame :: chrome://browser/content/search/search.xml :: handleSearchCommand :: line 440"  data: no]

Some digging with Process Explorer revealed that BitDefender's vsserv.exe was holding exclusive access to:

C:\Documents and Settings\Aapo Laitinen\Application Data\Mozilla\Firefox\Profiles\yqmzwcdq.Firefox 3\formhistory.sqlite-journal

Manually killing and restarting BitDefender solved the problem for now. I have previously seen BitDefender randomly hamper other applications too, so my guess would be that this is a bug in BitDefender.
Flags: blocking1.9.2? → blocking1.9.2-
FWIW, I've been running Norton 360 and Firefox nightlies for years and only recently noticed this error appearing in the Error Console;
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2a1pre) Gecko/20090728 Minefield/3.6a1pre (.NET CLR 3.5.30729) ID:20090728045737

I'm not installing add-ons, just surfing.  I found that if I bring up form auto-fill, and delete an entry, then submit, then the error appears in the Error Console about 1/3 the time.   The one occurrence I traced in ProcMon was much the same as comment #1:

85169	4:24:53.9095653 PM	firefox.exe	3144	SetRenameInformationFile	C:\Documents and Settings\S Page\Application Data\Mozilla\Firefox\Profiles\<MYPROFILE>\sessionstore-1.js	ACCESS DENIED	ReplaceIfExists: True, FileName: C:\Documents and Settings\S Page\Application Data\Mozilla\Firefox\Profiles\<MYPROFILE>\sessionstore.js	464

I have a 2KB session.bak and 50KB session.js in my profile, browser.sessionstore.resume_from_crash is true and browser.sessionstore.resume_session_once is false.

BTW, to turn off the scanning in Norton 360 v3: Settings > Antivirus > Scans and Risks > Scan Exclusions > Auto-Protect Exclusions > Add [path\to\profile].
Is this still a problem ?
wfm
Status: NEW → RESOLVED
Closed: 8 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.